Evaluation Metrics

The types of evaluation metrics used in measuring the accuracy of your predictive query will vary depending on your target type (i.e., what kind of prediction problem you're solving).

---

Binary Classification

If your predictive query is determining one of two values (e.g., true/false, yes/no), Kumo provides the following evaluation metrics:

• Accuracy
• Area Under The Receiver Operating Characteristic Curve (AUROC)
• Area Under The Precision-Recall Curve (AUPRC)
• Confusion Matrix
• Gain Chart

Multiclass Classification

If your predictive query is distinguishing between three or more values (e.g., the LASTvalue from a high cardinality categorical column in the target table), Kumo provides the following evaluation metrics:

• Accuracy

Multilabel Classification

If your predictive query returns one or more values from a list of categorical value candidates (e.g., the LIST_DISTINCT of values from a high cardinality categorical column in the target table), Kumo provides the following macro/micro/per-label versions of the following evaluation metrics:

• AUPRC
• AUROC
• AP

If your predictive query uses LIST_DISTINCT over a foreign key column, it will trigger a multilabel classification task.

Learn More:

• Classification

---

Link Prediction

If your pQuery returns a list of distinct values associated with your entity (e.g. LIST_DISTINCT), Kumo provides top-k retrieval metrics (for K = 1, 10, and 100):

• F1@K
• MAP@K
• Precision@K
• Recall@K

Your pQuery is deemed a link prediction task if it uses LIST_DISTINCT over a foreign key column; otherwise it will trigger a multilabel classification task.

Learn More:

• Link Prediction

---

Regression

If your pQuery returns a numeric value (e.g. the SUM/MAX/etc. of values from a numerical column in the target table), Kumo provides the following metrics:

• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)
• Root Mean Square Error (RMSE)
• Mean Absolute Percentage Error (MAPE)
• Symmetric Mean Absolute Percentage Error (SMAPE)
• Distribution of Predictions Histogram

Learn More:

• Regression