Download Now Instant Evaluation
Get Price Quote

How to Interpret Model Performance with Cost Functions

In this 10-part video series we discuss the concept of cost functions, which are directly related to the performance of data mining and predictive models.

Part 1: An Introduction to Understanding Cost Functions

 
This video is the first of 10 in a series that will help you, the analyst, understand the importance of cost functions, how they are used, and how the relate to model performance. In this 12-minute video we will introduce:
  1. The supervised learning problem: what is it and how is it applied in machine learning?
  2. How cost functions are used to solve the supervised learning problem
  3. Evaluating the ‘fit’ of the current response surface on the data available
  4. What is special about predicting the response in a regression problem?
 

Part 2: Least Squares Deviation Cost for a Regression Problem

 
In this 11-minute video we will cover:
  1. What is the Least Squares Deviation Cost function?
  2. What are the advantages of LS?
  3. An introduction to the underpinnings of the LSD’s statistical properties on the formula level
  4. What are the disadvantages of LS?
 

Part 3: Least Absolute Deviation and Huber-M Cost for a Regression Problem

 
In this 14-minute video we conclude our analysis of cost functions that can be applied to a regression problem. The outline for this video is as follows:
  1. What is the Least Absolute Deviation Cost function?
  2. How is LAD different from LS (part2)?
  3. Understanding of how LAD handles outliers
  4. What are LAD’s negative attributes?
  5. What is Huber-M Cost, and how does it compare to LS and LAD?
  6. Conclusion to cost functions used for a regression problem
 

Part 4: Introducing the Binary Classification Problem

 
In this 10-minute video we will quickly introduce the binary classification problem. This will set the stage for a later discussion of the various cost functions that can be applied to this type of problem.
We discuss:
  1. Why binary classification is commonly used among data analysts
  2. What are the fundamentals of the binary classification problem
  3. How to construct a simple response surface using linear regression
  4. How to use decision rules to make predictions
 

Part 5: Evaluating Prediction Success with Precision and Recall

 
In this 13-minute video we will begin our analysis of cost functions for binary classification. We will hone in on the following topics:
  1. Recap: working with a binary classification problem
  2. Evaluating the ‘positive’ group of predictions
  3. Define: precision, recall (sensitivity), and specificity
 

Part 6: Measuring Performance with the ROC Curve

 

In this 19-minute video we will review:
  1. The prediction success table
  2. Sensitivity vs. Specificity
  3. What is the ROC Curve, and how is it used to evaluate model performance?
  4. Advantages of evaluating based on ROC
  5. How to utilize the Area Under Curve (AUC)
 

Part 7: Assessing Model Performance with Gains and Lift

 
In this 20-minute video we will explore how to evaluate the performance of a binary classification model using gains and lift charts. In this data mining tutorial we use a direct marketing example to explain the concept of gains and lift; which is essentially a plot that represents sensitivity vs. support in the example marketing campaign.
 

Part 8: Direct Interpretation of Response with Logistic Function

 
In this 19-minute video we will cover:
  1. The mathematical structure behind the algorithm
  2. Direct interpretation of likelihood as a cost function
  3. Two ways to write logistic cost function
 

Part 9: Multinomial Classification- Expected Cost

 
In this 23-minute video we will discuss:
  1. What is multinomial classification?
  2. Assigning prior probabilities to your model
  3. Interpreting the expected cost
  4. Define: base-line, relative cost, and unit costs, and
  5. Alternative approaches to finding the expected cost
 

Part 10: Multinomial Classification- Log Likelihood

 
This video concludes our series on cost functions. We will:
  1. Build a probability response model that predicts probabilities
  2. Evaluate the performance of your multinomial classification model with Log-Likelihood
  3. Applying Log Likelihood to ensemble modeling scenarios, and discover an
  4. Alternative method: Margin-based cost function
 

[J#220:1602]

Tags: Videos, Webinars, Tutorials, Salford-Systems

Get In Touch With Us

Contact Us

9685 Via Excelencia, Suite 208, San Diego, CA 92126
Ph: 619-543-8880
Fax: 619-543-8888
info (at) salford-systems (dot) com