Model Evaluation Toolkit

What it does

Evaluates a model with a train/test split and cross-validation, using metrics that match the task.

Use it to compare models on equal terms.

Data you need

• Target
• Numeric features

Supported data: Numeric, Categorical. Level: Intermediate.

Formula and method

F1 = 2PR/(P+R) · RMSE = √[Σ(y−ŷ)²/n]

The data are split once using a fixed seed and stratified for classification. Cross-validation runs inside the training set, and final metrics are computed only on the held-out test rows, next to a no-skill baseline option.

How to read the result

Compare with the baseline model on the same split.

Worked example

Illustrative run on Fisher iris measurements. These figures are the actual output of this model on that example, and they say nothing about your own data.

• Test accuracy = 0.9211 against a majority-class baseline of 0.3421; F1 = 0.923.
• The model is better than always predicting the most common class on this test set.

Limitations

• One split gives one estimate

Numerical validation

Validated 16 of 16 automated checks passed, including numerical results, invalid input, missing values, small samples, constant variables, charts and the interpretation text.

Computed with python 3.13.2, numpy 2.2.5, scipy 1.14.1, pandas 2.3.3, statsmodels 0.14.4, scikit-learn 1.7.0, in your browser. A passing check confirms the listed comparison only. It does not make any result suitable for your data, and it is not a claim of equivalence with other statistical software.

Questions

Which metric should I use?

For balanced classes, accuracy is reasonable. For imbalanced classes, use precision, recall, F1 and AUC. For regression, use RMSE and MAE.