How do I perform backtesting on a holdout dataset?

You can perform backtesting on a holdout dataset by either using Kumo's model planner to define the holdout time split (i.e., having Kumo train on data up to a specific date, and then holdout the data after this date), or—if you don't want to use Kumo's built-in time split feature—physically separate your training data from your holdout data before ingesting it into Kumo.

In the latter case, you would train your model using Kumo on your training dataset, replace the graph with the training and holdout dataset, and then generate your batch predictions.

Learn More:

• Model Planner and AutoML
• split
• timeframe_step
• train_end_offset
• train_start_offest