  ## For beginners and expert users

• ### Webinars & Tutorials: Tips & Tricks and industry specific insights •  ## For beginners and expert users

• ### Webinars & Tutorials: Tips & Tricks and industry specific insights  # How to Interpret Model Performance with Cost Functions

In this 10-part video series we discuss the concept of cost functions, which are directly related to the performance of data mining and predictive models.

### Part 1: An Introduction to Understanding Cost Functions

This video is the first of 10 in a series that will help you, the analyst, understand the importance of cost functions, how they are used, and how the relate to model performance. In this 12-minute video we will introduce:
1. The supervised learning problem: what is it and how is it applied in machine learning?
2. How cost functions are used to solve the supervised learning problem
3. Evaluating the ‘fit’ of the current response surface on the data available
4. What is special about predicting the response in a regression problem?

### Part 2: Least Squares Deviation Cost for a Regression Problem

In this 11-minute video we will cover:
1. What is the Least Squares Deviation Cost function?
2. What are the advantages of LS?
3. An introduction to the underpinnings of the LSD’s statistical properties on the formula level
4. What are the disadvantages of LS?

### Part 3: Least Absolute Deviation and Huber-M Cost for a Regression Problem

In this 14-minute video we conclude our analysis of cost functions that can be applied to a regression problem. The outline for this video is as follows:
1. What is the Least Absolute Deviation Cost function?
2. How is LAD different from LS (part2)?
3. Understanding of how LAD handles outliers
4. What are LAD’s negative attributes?
5. What is Huber-M Cost, and how does it compare to LS and LAD?
6. Conclusion to cost functions used for a regression problem

### Part 4: Introducing the Binary Classification Problem

In this 10-minute video we will quickly introduce the binary classification problem. This will set the stage for a later discussion of the various cost functions that can be applied to this type of problem.
We discuss:
1. Why binary classification is commonly used among data analysts
2. What are the fundamentals of the binary classification problem
3. How to construct a simple response surface using linear regression
4. How to use decision rules to make predictions

### Part 5: Evaluating Prediction Success with Precision and Recall

In this 13-minute video we will begin our analysis of cost functions for binary classification. We will hone in on the following topics:
1. Recap: working with a binary classification problem
2. Evaluating the ‘positive’ group of predictions
3. Define: precision, recall (sensitivity), and specificity

### Part 6: Measuring Performance with the ROC Curve

In this 19-minute video we will review:
1. The prediction success table
2. Sensitivity vs. Specificity
3. What is the ROC Curve, and how is it used to evaluate model performance?
4. Advantages of evaluating based on ROC
5. How to utilize the Area Under Curve (AUC)

### Part 7: Assessing Model Performance with Gains and Lift

In this 20-minute video we will explore how to evaluate the performance of a binary classification model using gains and lift charts. In this data mining tutorial we use a direct marketing example to explain the concept of gains and lift; which is essentially a plot that represents sensitivity vs. support in the example marketing campaign.

### Part 8: Direct Interpretation of Response with Logistic Function

In this 19-minute video we will cover:
1. The mathematical structure behind the algorithm
2. Direct interpretation of likelihood as a cost function
3. Two ways to write logistic cost function

### Part 9: Multinomial Classification- Expected Cost

In this 23-minute video we will discuss:
1. What is multinomial classification?
2. Assigning prior probabilities to your model
3. Interpreting the expected cost
4. Define: base-line, relative cost, and unit costs, and
5. Alternative approaches to finding the expected cost

### Part 10: Multinomial Classification- Log Likelihood

This video concludes our series on cost functions. We will:
1. Build a probability response model that predicts probabilities
2. Evaluate the performance of your multinomial classification model with Log-Likelihood
3. Applying Log Likelihood to ensemble modeling scenarios, and discover an
4. Alternative method: Margin-based cost function

[J#220:1602]

# Get In Touch With Us

Request online support

Ph: 619-543-8880
9685 Via Excelencia, Suite 208, San Diego, CA 92126