Cross-validation and choosing settings benchmarks

15 of 15 cases match Python Prophet.

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check, on every trend type.

Each case runs the same input through Effect Prophet and Python Prophet 1.4.0, then compares every forecast row, the trend, each component and the fitted noise. A case matches when all of them agree within its tolerances, and only matching cases are timed.

Each case's dot shows its result: matches, matches with internal differences flagged, intentional difference, doesn't match. Hover over a dot for details.

How to read the tables:

• Effect output and Python output are each library's final forecast value. Open a case to see every row side by side.
• Largest difference is the biggest gap between the two forecasts, as a share of the input data's range.
• Effect time and Python time are the median time for the timed step, usually fitting and predicting. Open a case for every step, including first-run and save/load times.

About these runs

Each trend's cases come from that trend's own run.

Linear growth

• Run 2026-10-20T170110-379Z-27b7ec98 on 2026-10-20, commit 27b7ec9
• Both libraries ran in linux/arm64 containers on Apple M4 Pro
• effect-prophet 0.0.0
• effect 4.0.0
• prophet 1.4.0
• Full benchmark report

Flat growth

• Run 2026-10-20T171807-628Z-27b7ec98 on 2026-10-20, commit 27b7ec9
• Both libraries ran in linux/arm64 containers on Apple M4 Pro
• effect-prophet 0.0.0
• effect 4.0.0
• prophet 1.4.0
• Full benchmark report

Logistic growth

• Run 2026-10-20T174321-690Z-27b7ec98 on 2026-10-20, commit 27b7ec9
• Both libraries ran in linux/arm64 containers on Apple M4 Pro
• effect-prophet 0.0.0
• effect 4.0.0
• prophet 1.4.0
• Full benchmark report

Timings come from one machine and describe this workload only. They are not a general speed ranking. Where Python is faster, it is often because Python runs Stan's optimizer as compiled C++, while Effect Prophet runs the same algorithm as WebAssembly with narrower vector instructions. The gap is widest for Newton's method on short histories with many parameters.

Linear growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check. All linear growth benchmarks →

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Cross-validation and metrics	96 rows	108.5823	108.5823	< 0.0001%	Cross-validate	9.52 ms	75.7 ms
Cross-validation with ranges	96 rows	108.5823	108.5823	< 0.0001%	Cross-validate with ranges	9.74 ms	76.9 ms
Choosing settings and a final holdout	96 rows	108.5823	108.5823	< 0.0001%	Search settings	20.5 ms	164 ms
Search with a failing candidate	96 rows	108.5823	108.5823	< 0.0001%	Search settings	20.5 ms	156 ms
Cross-validation with repeated timestamps	96 rows	15.5221	15.5221	< 0.0001%	Cross-validate	5.76 ms	60.4 ms

Flat growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on flat models. All flat growth benchmarks →

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Cross-validation and metrics	96 rows	24.0746	24.0746	< 0.0001%	Cross-validate	6.50 ms	46.1 ms
Cross-validation with ranges	96 rows	24.0746	24.0746	< 0.0001%	Cross-validate with ranges	7.10 ms	60.1 ms
Choosing settings and a final holdout	96 rows	24.0746	24.0746	< 0.0001%	Search settings	13.1 ms	150 ms
Cross-validation, eight months	256 rows	34.9930	34.9930	< 0.0001%	Cross-validate	6.39 ms	39.8 ms
Cross-validation with ranges, eight months	256 rows	34.9930	34.9930	< 0.0001%	Cross-validate with ranges	8.02 ms	46.5 ms

Logistic growth

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on logistic models. All logistic growth benchmarks →

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Cross-validation and metrics	96 rows	59.1345	59.1345	< 0.0001%	Cross-validate	16.6 ms	60.2 ms
Cross-validation with ranges	96 rows	59.1345	59.1345	< 0.0001%	Cross-validate with ranges	18.5 ms	64.8 ms
Choosing settings and a final holdout	96 rows	59.1345	59.1345	< 0.0001%	Search settings	36.8 ms	123 ms
Cross-validation, eight months	256 rows	88.5367	88.5214	0.034%	Cross-validate	14.2 ms	42.2 ms
Cross-validation with ranges, eight months	256 rows	88.5367	88.5214	0.034%	Cross-validate with ranges	16.1 ms	56.8 ms