---
url: https://jer-k.github.io/effect-prophet/benchmarks/cross-validation.md
---

# Cross-validation and choosing settings benchmarks

**25 of 25 cases match Python Prophet.**

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check, on every trend type.

Each case runs the same input through Effect Prophet and Python Prophet 1.4.0, then compares every forecast row, the trend, each component and the fitted noise. A case **matches** when all of them agree within its tolerances, and only matching cases are timed.

Each case's dot shows its result:  matches,  matches with internal differences flagged,  intentional difference,  doesn't match. Hover over a dot for details.

How to read the tables:

* **Effect output** and **Python output** are each library's final forecast value. Open a case to see every row side by side.
* **Largest difference** is the biggest gap between the two forecasts, as a share of the input data's range.
* **Effect time** and **Python time** are the median time for the **timed step**, usually fitting and predicting. Open a case for every step, including first-run and save/load times.
* A timed step marked **different work** calls public operations that produce different outputs in the two libraries, so its times aren't a speed comparison. The case page explains the difference.

::: details About these runs

Each trend's cases come from that trend's own run.

**Linear growth**

* Run `2026-11-07T174133-142Z-3a5a6cec` on 2026-11-07, commit `3a5a6ce`
* Both libraries ran in `linux/arm64` containers on Apple M4 Pro
* effect-prophet 0.0.0
* effect 4.0.6
* prophet 1.4.0
* [Full benchmark report](https://github.com/jer-k/effect-prophet/blob/main/benchmark/results/retained/linear-growth/2026-11-07T174133-142Z-3a5a6cec/report.md)

**Flat growth**

* Run `2026-11-07T181422-912Z-3a5a6cec` on 2026-11-07, commit `3a5a6ce`
* Both libraries ran in `linux/arm64` containers on Apple M4 Pro
* effect-prophet 0.0.0
* effect 4.0.6
* prophet 1.4.0
* [Full benchmark report](https://github.com/jer-k/effect-prophet/blob/main/benchmark/results/retained/flat-growth/2026-11-07T181422-912Z-3a5a6cec/report.md)

**Logistic growth**

* Run `2026-11-07T184211-397Z-3a5a6cec` on 2026-11-07, commit `3a5a6ce`
* Both libraries ran in `linux/arm64` containers on Apple M4 Pro
* effect-prophet 0.0.0
* effect 4.0.6
* prophet 1.4.0
* [Full benchmark report](https://github.com/jer-k/effect-prophet/blob/main/benchmark/results/retained/logistic-growth/2026-11-07T184211-397Z-3a5a6cec/report.md)

Timings come from one machine and describe this workload only. They are not a general speed ranking. Where Python is faster, it is often because Python runs Stan's optimizer as compiled C++, while Effect Prophet runs the same algorithm as WebAssembly with narrower vector instructions. The gap is widest for Newton's method on short histories with many parameters.

:::

## Linear growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check. [All linear growth benchmarks →](/benchmarks/linear)

| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
| --- | --- | ---: | ---: | ---: | --- | ---: | ---: |
| [Cross-validation with a forecast regressor](/benchmarks/linear/evaluation-linear-regressor-model-point) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/map-regressors-medium.json) | 106.8125 | 106.8124 | 0.0025% | Cross-validate (different work) | 51.6 ms | 185 ms |
| [Cross-validation with a forecast regressor and ranges](/benchmarks/linear/evaluation-linear-regressor-model-intervals) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/map-regressors-medium.json) | 106.8125 | 106.8124 | 0.0025% | Cross-validate with ranges (different work) | 51.9 ms | 255 ms |
| [Cross-validation and metrics](/benchmarks/linear/evaluation-linear-point) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/linear-offset-scaling.json) | 108.5823 | 108.5823 | < 0.0001% | Cross-validate (different work) | 10.1 ms | 72.0 ms |
| [Cross-validation with ranges](/benchmarks/linear/evaluation-linear-intervals) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/linear-offset-scaling.json) | 108.5823 | 108.5823 | < 0.0001% | Cross-validate with ranges (different work) | 10.5 ms | 78.2 ms |
| [Choosing settings and a final holdout](/benchmarks/linear/evaluation-linear-search) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/linear-offset-scaling.json) | 108.5823 | 108.5823 | < 0.0001% | Search settings (different work) | 20.2 ms | 151 ms |
| [Search with a failing candidate](/benchmarks/linear/evaluation-linear-failure-search) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/linear-offset-scaling.json) | 108.5823 | 108.5823 | < 0.0001% | Search settings (different work) | 21.4 ms | 153 ms |
| [Choosing settings by a custom metric](/benchmarks/linear/evaluation-linear-custom-search) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/linear-offset-scaling.json) | 108.5823 | 108.5823 | < 0.0001% | Search settings (different work) | 20.4 ms | 143 ms |
| [Twelve folds, three years, sequential and parallel](/benchmarks/linear/evaluation-linear-long-history-parallel) | [1095 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v3/linear-long-history.json) | 420.8894 | 420.8323 | 0.052% | Cross-validate (different work) | 953 ms | 2.61 s |
| [Cross-validation with repeated timestamps](/benchmarks/linear/evaluation-training-duplicates) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/training-duplicates.json) | 15.5221 | 15.5221 | < 0.0001% | Cross-validate (different work) | 6.85 ms | 53.4 ms |

## Flat growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on flat models. [All flat growth benchmarks →](/benchmarks/flat)

| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
| --- | --- | ---: | ---: | ---: | --- | ---: | ---: |
| [Cross-validation with a forecast regressor](/benchmarks/flat/evaluation-flat-regressor-model-point) | [730 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v4/flat-correlated-regressors.json) | 45.0435 | 45.0356 | 0.094% | Cross-validate (different work) | 60.3 ms | 227 ms |
| [Cross-validation with a forecast regressor and ranges](/benchmarks/flat/evaluation-flat-regressor-model-intervals) | [730 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v4/flat-correlated-regressors.json) | 45.0435 | 45.0356 | 0.094% | Cross-validate with ranges (different work) | 61.8 ms | 291 ms |
| [Cross-validation and metrics](/benchmarks/flat/evaluation-flat-mixed-point) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components.json) | 24.0746 | 24.0746 | < 0.0001% | Cross-validate (different work) | 4.50 ms | 46.2 ms |
| [Cross-validation with ranges](/benchmarks/flat/evaluation-flat-mixed-intervals) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components.json) | 24.0746 | 24.0746 | < 0.0001% | Cross-validate with ranges (different work) | 5.57 ms | 61.3 ms |
| [Choosing settings and a final holdout](/benchmarks/flat/evaluation-flat-mixed-search) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components.json) | 24.0746 | 24.0746 | < 0.0001% | Search settings (different work) | 12.3 ms | 146 ms |
| [Cross-validation, eight months](/benchmarks/flat/evaluation-flat-large-point) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components-large.json) | 34.9930 | 34.9930 | < 0.0001% | Cross-validate (different work) | 6.27 ms | 39.0 ms |
| [Cross-validation with ranges, eight months](/benchmarks/flat/evaluation-flat-large-intervals) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components-large.json) | 34.9930 | 34.9930 | < 0.0001% | Cross-validate with ranges (different work) | 8.36 ms | 49.1 ms |
| [Twelve folds, sequential and parallel](/benchmarks/flat/evaluation-flat-large-parallel) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components-large.json) | 35.0008 | 35.0008 | < 0.0001% | Cross-validate (different work) | 27.6 ms | 184 ms |
| [Monthly, custom and scaled metrics](/benchmarks/flat/evaluation-flat-large-months-metrics) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/flat-mixed-components-large.json) | 49.8561 | 49.8561 | < 0.0001% | — | — | — |

## Logistic growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on logistic models. [All logistic growth benchmarks →](/benchmarks/logistic)

| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
| --- | --- | ---: | ---: | ---: | --- | ---: | ---: |
| [Cross-validation and metrics](/benchmarks/logistic/evaluation-logistic-point) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor.json) | 59.1345 | 59.1345 | < 0.0001% | Cross-validate (different work) | 17.1 ms | 59.5 ms |
| [Cross-validation with ranges](/benchmarks/logistic/evaluation-logistic-intervals) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor.json) | 59.1345 | 59.1345 | < 0.0001% | Cross-validate with ranges (different work) | 18.1 ms | 64.1 ms |
| [Choosing settings and a final holdout](/benchmarks/logistic/evaluation-logistic-search) | [96 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor.json) | 59.1345 | 59.1345 | < 0.0001% | Search settings (different work) | 35.5 ms | 115 ms |
| [Cross-validation, eight months](/benchmarks/logistic/evaluation-logistic-large-point) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor-large.json) | 88.5367 | 88.5214 | 0.034% | Cross-validate (different work) | 14.1 ms | 41.5 ms |
| [Cross-validation with ranges, eight months](/benchmarks/logistic/evaluation-logistic-large-intervals) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor-large.json) | 88.5367 | 88.5214 | 0.034% | Cross-validate with ranges (different work) | 15.1 ms | 56.4 ms |
| [Twelve folds and a search, sequential and parallel](/benchmarks/logistic/evaluation-logistic-large-parallel) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor-large.json) | 90.7830 | 90.7874 | 0.079% | Cross-validate (different work) | 61.3 ms | 170 ms |
| [Monthly, custom and scaled metrics](/benchmarks/logistic/evaluation-logistic-large-months-metrics) | [256 rows](https://github.com/jer-k/effect-prophet/blob/main/benchmark/inputs/v1/logistic-implicit-floor-large.json) | 90.9241 | 90.9228 | 0.20% | — | — | — |