Flat growth benchmarks
72 of 72 cases match Python Prophet.
Each case runs the same input through Effect Prophet and Python Prophet 1.4.0, then compares every forecast row, the trend, each component and the fitted noise. A case matches when all of them agree within its tolerances, and only matching cases are timed.
Each case's dot shows its result: matches, matches with internal differences flagged, intentional difference, doesn't match. Hover over a dot for details.
How to read the tables:
- Effect output and Python output are each library's final forecast value. Open a case to see every row side by side.
- Largest difference is the biggest gap between the two forecasts, as a share of the input data's range.
- Effect time and Python time are the median time for the timed step, usually fitting and predicting. Open a case for every step, including first-run and save/load times.
- A timed step marked different work calls public operations that produce different outputs in the two libraries, so its times aren't a speed comparison. The case page explains the difference.
About this run
- Run
2026-11-07T181422-912Z-3a5a6cecon 2026-11-07, commit3a5a6ce - Both libraries ran in
linux/arm64containers on Apple M4 Pro - effect-prophet 0.0.0
- effect 4.0.6
- prophet 1.4.0
- Full benchmark report
Timings come from one machine and describe this workload only. They are not a general speed ranking. Where Python is faster, it is often because Python runs Stan's optimizer as compiled C++, while Effect Prophet runs the same algorithm as WebAssembly with narrower vector instructions. The gap is widest for Newton's method on short histories with many parameters.
Country holiday calendars
Pinned country calendars, matching explicit-event controls, dynamic future years and portable saved models. Known-gap opt-in is partial provider output, not a completeness guarantee.
Default settings
fit(history, { growth: "flat" }) against Prophet(growth="flat").fit(df) with nothing else set. Each library decides on its own which seasonal patterns to use. Warm-start cases fit the first rows, then refit the whole history from the parameters of that fit, as when new data arrives.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Defaults, steady level | 96 rows | 35.0025 | 35.0025 | < 0.0001% | Fit and predict | 1.48 ms | 12.2 ms |
| Defaults, weekly pattern | 96 rows | 35.0025 | 35.0025 | < 0.0001% | Fit and predict | 105 ms | 398 ms |
| Refit after a week of new data | 96 rows | 35.0025 | 35.0025 | 0.00019% | Refit from an earlier fit | 6.38 ms | 8.52 ms |
| Defaults, negative values | 96 rows | -34.9975 | -34.9975 | < 0.0001% | Fit and predict | 1.58 ms | 12.1 ms |
Steady levels
A single constant level with no seasonal patterns: noisy, perfectly constant, below zero, and every row on the same date. Each runs with both target scalings.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Noisy level, absmax scaling | 96 rows | 35.0019 | 35.0019 | < 0.0001% | Fit | 1.92 ms | 7.48 ms |
| Noisy level, minmax scaling | 96 rows | 35.0015 | 35.0015 | < 0.0001% | Fit | 3.85 ms | 19.8 ms |
| Perfectly constant, absmax scaling | 96 rows | 35.0000 | 35.0000 | 0 | Fit | 1.23 ms | 3.10 ms |
| Perfectly constant, minmax scaling | 96 rows | 35.0000 | 35.0000 | 0 | Fit | 1.20 ms | 3.08 ms |
| Negative level, absmax scaling | 96 rows | -34.9989 | -34.9989 | < 0.0001% | Fit | 1.79 ms | 7.23 ms |
| Negative level, minmax scaling | 96 rows | -34.9985 | -34.9985 | < 0.0001% | Fit | 3.90 ms | 20.1 ms |
| Every row on the same date, different values | 3 rows | 4.3286 | 4.3289 | 0.0059% | Fit | 937 µs | 5.39 ms |
| Every row on the same date, same value | 3 rows | 2.0000 | 2.0000 | 0 | Fit | 496 µs | 2.87 ms |
Seasonality, events and regressors
Weekly patterns, events and regressors on a flat trend: with a switchable multiplicative weekly pattern at full length and cut to the history lengths cross-validation uses, with every effect additive, and with regressors that move almost or exactly in step with the level or the yearly pattern.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Weekly pattern, absmax scaling | 96 rows | 35.0039 | 35.0039 | < 0.0001% | Fit | 159 ms | 630 ms |
| Weekly pattern, minmax scaling | 96 rows | 35.0039 | 35.0039 | < 0.0001% | Fit | 4.35 ms | 20.7 ms |
| Mixed effects, minmax scaling | 96 rows | 35.0283 | 35.0283 | < 0.0001% | Fit and predict | 1.38 ms | 12.3 ms |
| Mixed effects, absmax scaling | 96 rows | 35.0127 | 35.0127 | < 0.0001% | Fit | 2.00 ms | 8.16 ms |
| Mixed effects, eight months, minmax scaling | 256 rows | 19.5305 | 19.5305 | < 0.0001% | Fit and predict | 2.20 ms | 14.3 ms |
| Additive effects, minmax scaling | 96 rows | 35.0127 | 35.0127 | < 0.0001% | Fit and predict | 4.82 ms | 27.5 ms |
| Additive effects, eight months, minmax scaling | 256 rows | 28.9196 | 28.9196 | < 0.0001% | Fit and predict | 2.15 ms | 13.8 ms |
| Mixed effects, both libraries on default settings | 96 rows | 35.0012 | 35.0127 | 0.073% | Fit and predict | 3.11 ms | 48.8 ms |
| Regressors that track other effects | 730 rows | 56.9753 | 56.9753 | < 0.0001% | Fit and predict | 6.06 ms | 29.2 ms |
| A regressor forecast by its own model | 730 rows | 24.0248 | 24.0247 | 0.0020% | Fit and predict | 24.5 ms | 127 ms |
| Mixed effects, eight months, absmax scaling | 256 rows | 28.9196 | 28.9196 | < 0.0001% | Fit | 2.57 ms | 9.07 ms |
| First 63 days, absmax scaling | 63 rows | 24.0826 | 24.0826 | < 0.0001% | Fit | 1.83 ms | 7.36 ms |
| First 63 days, minmax scaling | 63 rows | 7.1798 | 7.1798 | < 0.0001% | Fit | 1.77 ms | 7.74 ms |
| First 70 days, absmax scaling | 70 rows | 24.0795 | 24.0795 | < 0.0001% | Fit | 2.15 ms | 8.05 ms |
| First 70 days, minmax scaling | 70 rows | 7.1718 | 7.1718 | < 0.0001% | Fit | 1.72 ms | 7.54 ms |
| First 77 days, absmax scaling | 77 rows | 24.0746 | 24.0746 | < 0.0001% | Fit | 2.01 ms | 7.52 ms |
| First 77 days, minmax scaling | 77 rows | 7.1594 | 7.1594 | < 0.0001% | Fit | 2.24 ms | 7.68 ms |
| First 225 days, absmax scaling | 225 rows | 34.9960 | 34.9960 | < 0.0001% | Fit | 2.24 ms | 9.04 ms |
| First 225 days, minmax scaling | 225 rows | 34.9891 | 34.9891 | < 0.0001% | Fit | 2.23 ms | 8.94 ms |
| First 232 days, absmax scaling | 232 rows | 34.9930 | 34.9930 | < 0.0001% | Fit | 2.60 ms | 8.74 ms |
| First 232 days, minmax scaling | 232 rows | 34.9807 | 34.9807 | < 0.0001% | Fit | 2.80 ms | 8.87 ms |
Uncertainty ranges
Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws. Anomaly cases also check that both libraries flag the outliers in a history.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Country calendar uncertainty replay | 96 rows | 21.4447 | 21.4446 | 0.0048% | Simulate uncertainty (different work) | 341 µs | 14.7 ms |
| Ranges with a forecast regressor | 730 rows | 24.0248 | 24.0247 | 0.0020% | Simulate uncertainty (different work) | 18.1 ms | 277 ms |
| Anomalies in a history | 180 rows | 34.0579 | 34.0579 | < 0.0001% | Fit and find anomalies | 10.4 ms | 133 ms |
| Defaults, ranges with a weekly pattern | 96 rows | 35.0025 | 35.0025 | < 0.0001% | Simulate uncertainty (different work) | 4.83 ms | 129 ms |
| Ranges with mixed effects | 96 rows | 35.0283 | 35.0283 | < 0.0001% | Simulate uncertainty (different work) | 985 µs | 72.1 ms |
| Samples, eight months | 256 rows | 19.5305 | 19.5305 | < 0.0001% | Simulate uncertainty (different work) | 860 µs | 20.2 ms |
Full Bayesian fitting
fit(history, { mcmc: { samples, seed } }) against Prophet(mcmc_samples=...) and fit(df, seed=...): both run Stan's NUTS sampler with the same settings. Their chains can't match draw for draw, so each forecast, component and coefficient, a posterior mean, must agree within 0.15 of the narrowest 80% range, and each range bound within the variation expected from that many posterior draws. Divergences and R-hat are reported, and flagged when Effect's chains mix clearly worse than Python's.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Mixed features, two chains | 96 rows | 35.0285 | 35.0305 | 0.022% | Simulate uncertainty (different work) | 2.02 ms | 130 ms |
Cross-validation and choosing settings
Rolling-origin cross-validation, metrics, baselines, settings search and a final holdout check on flat models.
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Cross-validation with a forecast regressor | 730 rows | 45.0435 | 45.0356 | 0.094% | Cross-validate (different work) | 60.3 ms | 227 ms |
| Cross-validation with a forecast regressor and ranges | 730 rows | 45.0435 | 45.0356 | 0.094% | Cross-validate with ranges (different work) | 61.8 ms | 291 ms |
| Cross-validation and metrics | 96 rows | 24.0746 | 24.0746 | < 0.0001% | Cross-validate (different work) | 4.50 ms | 46.2 ms |
| Cross-validation with ranges | 96 rows | 24.0746 | 24.0746 | < 0.0001% | Cross-validate with ranges (different work) | 5.57 ms | 61.3 ms |
| Choosing settings and a final holdout | 96 rows | 24.0746 | 24.0746 | < 0.0001% | Search settings (different work) | 12.3 ms | 146 ms |
| Cross-validation, eight months | 256 rows | 34.9930 | 34.9930 | < 0.0001% | Cross-validate (different work) | 6.27 ms | 39.0 ms |
| Cross-validation with ranges, eight months | 256 rows | 34.9930 | 34.9930 | < 0.0001% | Cross-validate with ranges (different work) | 8.36 ms | 49.1 ms |
| Twelve folds, sequential and parallel | 256 rows | 35.0008 | 35.0008 | < 0.0001% | Cross-validate (different work) | 27.6 ms | 184 ms |
| Monthly, custom and scaled metrics | 256 rows | 49.8561 | 49.8561 | < 0.0001% | — | — | — |