Flat growth benchmarks

63 of 63 cases match Python Prophet.

Each case runs the same input through Effect Prophet and Python Prophet 1.4.0, then compares every forecast row, the trend, each component and the fitted noise. A case matches when all of them agree within its tolerances, and only matching cases are timed.

Each case's dot shows its result: matches, matches with internal differences flagged, intentional difference, doesn't match. Hover over a dot for details.

How to read the tables:

• Effect output and Python output are each library's final forecast value. Open a case to see every row side by side.
• Largest difference is the biggest gap between the two forecasts, as a share of the input data's range.
• Effect time and Python time are the median time for the timed step, usually fitting and predicting. Open a case for every step, including first-run and save/load times.

About this run

• Run 2026-10-20T171807-628Z-27b7ec98 on 2026-10-20, commit 27b7ec9
• Both libraries ran in linux/arm64 containers on Apple M4 Pro
• effect-prophet 0.0.0
• effect 4.0.0
• prophet 1.4.0
• Full benchmark report

Timings come from one machine and describe this workload only. They are not a general speed ranking. Where Python is faster, it is often because Python runs Stan's optimizer as compiled C++, while Effect Prophet runs the same algorithm as WebAssembly with narrower vector instructions. The gap is widest for Newton's method on short histories with many parameters.

Country holiday calendars

Pinned country calendars, matching explicit-event controls, dynamic future years and portable saved models. Known-gap opt-in is partial provider output, not a completeness guarantee.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
US holidays, additive	96 rows	21.4447	21.4446	0.0048%	Fit and predict	14.2 ms	53.4 ms
Explicit US control, additive	96 rows	21.4447	21.4446	0.0048%	Fit and predict	13.1 ms	51.3 ms
US holidays, multiplicative	96 rows	21.4459	21.4459	< 0.0001%	Fit and predict	3.50 ms	22.2 ms
Explicit US control, multiplicative	96 rows	21.4459	21.4459	< 0.0001%	Fit and predict	2.69 ms	17.9 ms
Observed spillovers and unseen future names	96 rows	23.0350	23.0350	< 0.0001%	Fit and predict	21.9 ms	72.7 ms
Deliberately empty country	96 rows	20.0473	20.0473	< 0.0001%	Fit and predict	1.68 ms	12.6 ms
No-calendar empty control	96 rows	20.0473	20.0473	< 0.0001%	Fit and predict	1.43 ms	10.9 ms
Singapore estimated-name transition	96 rows	20.0151	20.0151	< 0.0001%	Fit and predict	19.9 ms	70.3 ms
India with explicit known-gap opt-in	96 rows	20.0510	20.0510	< 0.0001%	Fit and predict	4.03 ms	20.5 ms
Country, custom events and mixed features	96 rows	20.7451	20.7450	0.00062%	Fit and predict	17.5 ms	62.1 ms
SE calendar, 730 training rows	730 rows	20.0026	20.0026	< 0.0001%	Fit	7.69 ms	23.2 ms
SE calendar, 2190 training rows	2190 rows	20.0033	20.0033	< 0.0001%	Fit	16.3 ms	44.9 ms
CN calendar, 730 training rows	730 rows	20.0048	20.0048	< 0.0001%	Fit	7.47 ms	27.2 ms
CN calendar, 2190 training rows	2190 rows	20.0044	20.0044	0.0082%	Fit	27.6 ms	85.9 ms
Sweden with a smaller prediction batch	2190 rows	20.0033	20.0033	< 0.0001%	Fit	16.6 ms	44.8 ms
No-calendar control, 730 training rows / 512 prediction rows	730 rows	20.0121	20.0121	< 0.0001%	Fit	3.27 ms	9.05 ms
Explicit SE events, 730 training rows / 512 prediction rows	730 rows	20.0026	20.0026	< 0.0001%	Fit	6.72 ms	20.1 ms
Explicit CN events, 730 training rows / 512 prediction rows	730 rows	20.0048	20.0048	< 0.0001%	Fit	5.94 ms	20.6 ms
No-calendar control, 2,190 training rows / 512 prediction rows	2190 rows	20.0121	20.0121	< 0.0001%	Fit	7.13 ms	16.7 ms
Explicit SE events, 2,190 training rows / 512 prediction rows	2190 rows	20.0033	20.0033	< 0.0001%	Fit	13.1 ms	40.6 ms
Explicit CN events, 2,190 training rows / 512 prediction rows	2190 rows	20.0044	20.0044	0.0082%	Fit	22.9 ms	84.9 ms
No-calendar control, 2,190 training rows / 32 prediction rows	2190 rows	20.0121	20.0121	< 0.0001%	Fit	7.72 ms	18.4 ms
Explicit SE events, 2,190 training rows / 32 prediction rows	2190 rows	20.0033	20.0033	< 0.0001%	Fit	11.8 ms	41.8 ms

Default settings

fit(history, { growth: "flat" }) against Prophet(growth="flat").fit(df) with nothing else set. Each library decides on its own which seasonal patterns to use.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Defaults, steady level	96 rows	35.0025	35.0025	< 0.0001%	Fit and predict	1.86 ms	11.4 ms
Defaults, weekly pattern	96 rows	35.0025	35.0025	< 0.0001%	Fit and predict	104 ms	395 ms
Defaults, negative values	96 rows	-34.9975	-34.9975	< 0.0001%	Fit and predict	1.82 ms	12.1 ms

Steady levels

A single constant level with no seasonal patterns: noisy, perfectly constant, below zero, and every row on the same date. Each runs with both target scalings.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Noisy level, absmax scaling	96 rows	35.0019	35.0019	< 0.0001%	Fit	1.78 ms	7.40 ms
Noisy level, minmax scaling	96 rows	35.0015	35.0015	< 0.0001%	Fit	3.89 ms	20.0 ms
Perfectly constant, absmax scaling	96 rows	35.0000	35.0000	0	Fit	1.16 ms	3.10 ms
Perfectly constant, minmax scaling	96 rows	35.0000	35.0000	0	Fit	1.16 ms	3.03 ms
Negative level, absmax scaling	96 rows	-34.9989	-34.9989	< 0.0001%	Fit	1.91 ms	7.63 ms
Negative level, minmax scaling	96 rows	-34.9985	-34.9985	< 0.0001%	Fit	3.88 ms	19.8 ms
Every row on the same date, different values	3 rows	4.3286	4.3289	0.0059%	Fit	888 µs	5.74 ms
Every row on the same date, same value	3 rows	2.0000	2.0000	0	Fit	530 µs	2.86 ms

Seasonality, events and regressors

Weekly patterns, events and regressors on a flat trend: with a switchable multiplicative weekly pattern at full length and cut to the history lengths cross-validation uses, with every effect additive, and with regressors that move almost or exactly in step with the level or the yearly pattern.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Weekly pattern, absmax scaling	96 rows	35.0039	35.0039	< 0.0001%	Fit	155 ms	603 ms
Weekly pattern, minmax scaling	96 rows	35.0039	35.0039	< 0.0001%	Fit	4.23 ms	20.4 ms
Mixed effects, minmax scaling	96 rows	35.0283	35.0283	< 0.0001%	Fit and predict	1.41 ms	12.4 ms
Mixed effects, absmax scaling	96 rows	35.0127	35.0127	< 0.0001%	Fit	1.58 ms	7.92 ms
Mixed effects, eight months, minmax scaling	256 rows	19.5305	19.5305	< 0.0001%	Fit and predict	2.23 ms	13.7 ms
Additive effects, minmax scaling	96 rows	35.0127	35.0127	< 0.0001%	Fit and predict	4.67 ms	26.9 ms
Additive effects, eight months, minmax scaling	256 rows	28.9196	28.9196	< 0.0001%	Fit and predict	2.12 ms	13.3 ms
Mixed effects, both libraries on default settings	96 rows	35.0012	35.0127	0.073%	Fit and predict	3.17 ms	48.4 ms
Regressors that track other effects	730 rows	56.9753	56.9753	< 0.0001%	Fit and predict	5.42 ms	29.6 ms
Mixed effects, eight months, absmax scaling	256 rows	28.9196	28.9196	< 0.0001%	Fit	3.82 ms	8.99 ms
First 63 days, absmax scaling	63 rows	24.0826	24.0826	< 0.0001%	Fit	2.24 ms	7.62 ms
First 63 days, minmax scaling	63 rows	7.1798	7.1798	< 0.0001%	Fit	2.21 ms	7.61 ms
First 70 days, absmax scaling	70 rows	24.0795	24.0795	< 0.0001%	Fit	2.10 ms	7.44 ms
First 70 days, minmax scaling	70 rows	7.1718	7.1718	< 0.0001%	Fit	2.04 ms	7.85 ms
First 77 days, absmax scaling	77 rows	24.0746	24.0746	< 0.0001%	Fit	2.52 ms	7.51 ms
First 77 days, minmax scaling	77 rows	7.1594	7.1594	< 0.0001%	Fit	2.16 ms	8.16 ms
First 225 days, absmax scaling	225 rows	34.9960	34.9960	< 0.0001%	Fit	3.48 ms	8.67 ms
First 225 days, minmax scaling	225 rows	34.9891	34.9891	< 0.0001%	Fit	3.17 ms	8.79 ms
First 232 days, absmax scaling	232 rows	34.9930	34.9930	< 0.0001%	Fit	2.98 ms	8.86 ms
First 232 days, minmax scaling	232 rows	34.9807	34.9807	< 0.0001%	Fit	3.20 ms	8.89 ms

Uncertainty ranges

Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Country calendar uncertainty replay	96 rows	21.4447	21.4446	0.0048%	Simulate uncertainty	318 µs	14.8 ms
Defaults, ranges with a weekly pattern	96 rows	35.0025	35.0025	< 0.0001%	Simulate uncertainty	4.48 ms	125 ms
Ranges with mixed effects	96 rows	35.0283	35.0283	< 0.0001%	Simulate uncertainty	1.10 ms	73.6 ms
Samples, eight months	256 rows	19.5305	19.5305	< 0.0001%	Simulate uncertainty	1.21 ms	22.3 ms

Cross-validation and choosing settings

Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on flat models.

Case	Input	Effect output	Python output	Largest difference	Timed step	Effect time	Python time
Cross-validation and metrics	96 rows	24.0746	24.0746	< 0.0001%	Cross-validate	6.50 ms	46.1 ms
Cross-validation with ranges	96 rows	24.0746	24.0746	< 0.0001%	Cross-validate with ranges	7.10 ms	60.1 ms
Choosing settings and a final holdout	96 rows	24.0746	24.0746	< 0.0001%	Search settings	13.1 ms	150 ms
Cross-validation, eight months	256 rows	34.9930	34.9930	< 0.0001%	Cross-validate	6.39 ms	39.8 ms
Cross-validation with ranges, eight months	256 rows	34.9930	34.9930	< 0.0001%	Cross-validate with ranges	8.02 ms	46.5 ms