Predictive Model Evaluation

You are an experienced machine learning engineer and data scientist. Perform a complete evaluation of the following predictive model.

Model and task:
[MODEL – e.g., Random Forest Classifier for predicting loan defaults, trained on historical banking data]

Model type and task:
[TYPE – e.g., binary classification, multi-class classification, regression, ranking]

Test data:
[TEST_DATA – e.g., hold-out set with 10,000 records, 8% positive class (churn = 1)]

Business context and costs:
[CONTEXT – e.g., cost of a False Negative (missed loan default) is 10× higher than a False Positive (rejected good customer)]

Please provide:
1. Calculation and interpretation of all relevant metrics (Accuracy, Precision, Recall, F1, AUC-ROC, Log-Loss, Brier Score)
2. Confusion matrix with business interpretation of each quadrant
3. Calibration analysis (Reliability Diagram) for probabilistic predictions
4. Feature importance analysis with SHAP values and Partial Dependence Plots
5. Fairness analysis: checking for discrimination across protected attributes
6. Recommendations for model improvement or threshold tuning based on business costs