What is the key difference in how supervised and unsupervised learning models are evaluated?
Supervised models are evaluated using metrics that compare predictions to true labels, while unsupervised models are evaluated based on the task, as there is no ground truth.
Which classification metric is defined as the ratio of correctly predicted instances to the total number of instances?
Accuracy.
Why is Accuracy not a suitable metric for imbalanced datasets?
It can be misleadingly high if the model simply predicts the majority class for all instances.
What is the formula for Accuracy using TP, TN, FP, and FN?
Accuracy = $\frac{TP+TN}{TP+TN+FP+FN}$
Term: Precision
Definition: The ratio of true positive predictions to the total number of positive predictions made by the model.
In what type of scenario is maximizing Precision particularly important?
In cases where the cost of a false positive is high.
What is the formula for Precision?
Precision = $\frac{TP}{TP+FP}$
Term: Recall (Sensitivity)
Definition: The ratio of true positive predictions to the total number of actual positive instances in the dataset.
In what type of scenario is maximizing Recall particularly important?
In scenarios where the cost of a false negative is high.
What is the formula for Recall?
Recall = $\frac{TP}{TP+FN}$
The _____ is the harmonic mean of precision and recall.
F1 Score.
What is the main purpose of using the F1 Score?
It provides a single metric that balances the trade-off between precision and recall.
What is the formula for the F1 Score?
F1 = $2 \times \frac{Precision \times Recall}{Precision+Recall}$
How does Macro F1 calculate the final score for a multi-class problem?
It calculates the F1 score for each class independently and then takes the unweighted average, treating all classes equally.
How does Micro F1 calculate the final score for a multi-class problem?
It calculates the F1 score globally by considering the total number of true positives, false negatives, and false positives across all classes.
Which variant of the F1 score gives more weight to the performance on larger classes?
Micro F1.
What does AUC-ROC stand for?
Area Under the Receiver Operating Characteristic curve.
The ROC curve plots the true positive rate (sensitivity) against which other rate?
The false positive rate (1 - specificity).
What does the AUC-ROC metric quantify about a classification model?
The model’s overall discriminative power across all possible classification thresholds.
An AUC-ROC score of 0.5 suggests the model’s performance is equivalent to ____.
random guessing
What is a confusion matrix?
A table showing the counts of true positives, false positives, true negatives, and false negatives for a classification model.
Which regression metric measures the average of the squared differences between predicted and actual values?
Mean Squared Error (MSE).
What is the primary drawback of using MSE as an evaluation metric?
It is sensitive to outliers due to the squaring of errors.
What is the formula for Mean Squared Error (MSE)?
MSE = $\frac{1}{m}\sum_{i=1}^{m}(y_{true}^{(i)} - y_{pred}^{(i)})^2$
