# How to Interpret Model Performance with Cost Functions

In this 10-part video series we discuss the concept of cost functions, which are directly related to the performance of data mining and predictive models.

### Part 1: An Introduction to Understanding Cost Functions

This video is the first of 10 in a series that will help you, the analyst, understand the importance of cost functions, how they are used, and how the relate to model performance. In this 12-minute video we will introduce:

- The supervised learning problem: what is it and how is it applied in machine learning?
- How cost functions are used to solve the supervised learning problem
- Evaluating the ‘fit’ of the current response surface on the data available
- What is special about predicting the response in a regression problem?

### Part 2: Least Squares Deviation Cost for a Regression Problem

In this 11-minute video we will cover:

- What is the Least Squares Deviation Cost function?
- What are the advantages of LS?
- An introduction to the underpinnings of the LSD’s statistical properties on the formula level
- What are the disadvantages of LS?

### Part 3: Least Absolute Deviation and Huber-M Cost for a Regression Problem

In this 14-minute video we conclude our analysis of cost functions that can be applied to a regression problem. The outline for this video is as follows:

- What is the Least Absolute Deviation Cost function?
- How is LAD different from LS (part2)?
- Understanding of how LAD handles outliers
- What are LAD’s negative attributes?
- What is Huber-M Cost, and how does it compare to LS and LAD?
- Conclusion to cost functions used for a regression problem

### Part 4: Introducing the Binary Classification Problem

In this 10-minute video we will quickly introduce the binary classification problem. This will set the stage for a later discussion of the various cost functions that can be applied to this type of problem.

We discuss:

We discuss:

- Why binary classification is commonly used among data analysts
- What are the fundamentals of the binary classification problem
- How to construct a simple response surface using linear regression
- How to use decision rules to make predictions

### Part 5: Evaluating Prediction Success with Precision and Recall

In this 13-minute video we will begin our analysis of cost functions for binary classification. We will hone in on the following topics:

- Recap: working with a binary classification problem
- Evaluating the ‘positive’ group of predictions
- Define: precision, recall (sensitivity), and specificity

### Part 6: Measuring Performance with the ROC Curve

In this 19-minute video we will review:

- The prediction success table
- Sensitivity vs. Specificity
- What is the ROC Curve, and how is it used to evaluate model performance?
- Advantages of evaluating based on ROC
- How to utilize the Area Under Curve (AUC)

### Part 7: Assessing Model Performance with Gains and Lift

In this 20-minute video we will explore how to evaluate the performance of a binary classification model using gains and lift charts. In this data mining tutorial we use a direct marketing example to explain the concept of gains and lift; which is essentially a plot that represents sensitivity vs. support in the example marketing campaign.

### Part 8: Direct Interpretation of Response with Logistic Function

In this 19-minute video we will cover:

- The mathematical structure behind the algorithm
- Direct interpretation of likelihood as a cost function
- Two ways to write logistic cost function

### Part 9: Multinomial Classification- Expected Cost

In this 23-minute video we will discuss:

- What is multinomial classification?
- Assigning prior probabilities to your model
- Interpreting the expected cost
- Define: base-line, relative cost, and unit costs, and
- Alternative approaches to finding the expected cost

### Part 10: Multinomial Classification- Log Likelihood

This video concludes our series on cost functions. We will:

- Build a probability response model that predicts probabilities
- Evaluate the performance of your multinomial classification model with Log-Likelihood
- Applying Log Likelihood to ensemble modeling scenarios, and discover an
- Alternative method: Margin-based cost function

[J#220:1602]

Tags: Videos, Webinars, Tutorials, Salford-Systems