Uncertainty ranges benchmarks
24 of 24 cases match Python Prophet.
Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws, on every trend type. Anomaly cases also check that both libraries flag the outliers in a history.
Each case runs the same input through Effect Prophet and Python Prophet 1.4.0, then compares every forecast row, the trend, each component and the fitted noise. A case matches when all of them agree within its tolerances, and only matching cases are timed.
Each case's dot shows its result: matches, matches with internal differences flagged, intentional difference, doesn't match. Hover over a dot for details.
How to read the tables:
- Effect output and Python output are each library's final forecast value. Open a case to see every row side by side.
- Largest difference is the biggest gap between the two forecasts, as a share of the input data's range.
- Effect time and Python time are the median time for the timed step, usually fitting and predicting. Open a case for every step, including first-run and save/load times.
- A timed step marked different work calls public operations that produce different outputs in the two libraries, so its times aren't a speed comparison. The case page explains the difference.
- Flagged cases match, but the two libraries stopped their fitting search at slightly different points. The case page shows the internal numbers.
About these runs
Each trend's cases come from that trend's own run.
Linear growth
- Run
2026-11-07T174133-142Z-3a5a6cecon 2026-11-07, commit3a5a6ce - Both libraries ran in
linux/arm64containers on Apple M4 Pro - effect-prophet 0.0.0
- effect 4.0.6
- prophet 1.4.0
- Full benchmark report
Flat growth
- Run
2026-11-07T181422-912Z-3a5a6cecon 2026-11-07, commit3a5a6ce - Both libraries ran in
linux/arm64containers on Apple M4 Pro - effect-prophet 0.0.0
- effect 4.0.6
- prophet 1.4.0
- Full benchmark report
Logistic growth
- Run
2026-11-07T184211-397Z-3a5a6cecon 2026-11-07, commit3a5a6ce - Both libraries ran in
linux/arm64containers on Apple M4 Pro - effect-prophet 0.0.0
- effect 4.0.6
- prophet 1.4.0
- Full benchmark report
Timings come from one machine and describe this workload only. They are not a general speed ranking. Where Python is faster, it is often because Python runs Stan's optimizer as compiled C++, while Effect Prophet runs the same algorithm as WebAssembly with narrower vector instructions. The gap is widest for Newton's method on short histories with many parameters.
Linear growth
Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws. Anomaly cases also check that both libraries flag the outliers in a history. All linear growth benchmarks →
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Ranges with a forecast regressor | 256 rows | 139.2608 | 139.2648 | 0.0040% | Simulate uncertainty (different work) | 9.28 ms | 246 ms |
| Defaults, three years of ranges | 1095 rows | 611.2687 | 611.3156 | 0.011% | Simulate uncertainty (different work) | 69.7 ms | 375 ms |
| Defaults, ranges on irregular dates | 192 rows | 108.5108 | 108.4320 | 0.15% | Simulate uncertainty (different work) | 11.4 ms | 158 ms |
| Samples with repeated timestamps | 96 rows | 17.9039 | 17.9039 | 0.0013% | Simulate uncertainty (different work) | 471 µs | 16.8 ms |
| Ranges over past dates | 96 rows | 110.0950 | 110.0950 | < 0.0001% | Simulate uncertainty (different work) | 331 µs | 19.8 ms |
| Samples for one future date | 96 rows | 110.0819 | 110.0819 | < 0.0001% | Simulate uncertainty (different work) | 204 µs | 16.8 ms |
| Ranges with mixed effects | 96 rows | 45.1353 | 45.1353 | < 0.0001% | Simulate uncertainty (different work) | 487 µs | 26.4 ms |
| Samples with mixed effects | 96 rows | 45.1353 | 45.1353 | < 0.0001% | Simulate uncertainty (different work) | 798 µs | 68.6 ms |
| Anomalies in a history | 365 rows | 149.0161 | 148.9947 | 0.11% | Fit and find anomalies | 25.2 ms | 199 ms |
Flat growth
Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws. Anomaly cases also check that both libraries flag the outliers in a history. All flat growth benchmarks →
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Country calendar uncertainty replay | 96 rows | 21.4447 | 21.4446 | 0.0048% | Simulate uncertainty (different work) | 341 µs | 14.7 ms |
| Ranges with a forecast regressor | 730 rows | 24.0248 | 24.0247 | 0.0020% | Simulate uncertainty (different work) | 18.1 ms | 277 ms |
| Anomalies in a history | 180 rows | 34.0579 | 34.0579 | < 0.0001% | Fit and find anomalies | 10.4 ms | 133 ms |
| Defaults, ranges with a weekly pattern | 96 rows | 35.0025 | 35.0025 | < 0.0001% | Simulate uncertainty (different work) | 4.83 ms | 129 ms |
| Ranges with mixed effects | 96 rows | 35.0283 | 35.0283 | < 0.0001% | Simulate uncertainty (different work) | 985 µs | 72.1 ms |
| Samples, eight months | 256 rows | 19.5305 | 19.5305 | < 0.0001% | Simulate uncertainty (different work) | 860 µs | 20.2 ms |
Logistic growth
Both libraries simulate possible futures with random draws, so individual simulated values never match. Each range must still match Python's within the variation expected from that many draws. Anomaly cases also check that both libraries flag the outliers in a history. All logistic growth benchmarks →
| Case | Input | Effect output | Python output | Largest difference | Timed step | Effect time | Python time |
|---|---|---|---|---|---|---|---|
| Anomalies in a history | 365 rows | 157.6301 | 157.6272 | 0.073% | Fit and find anomalies | 71.6 ms | 298 ms |
| Defaults, ranges with changing growth | 365 rows | 193.9557 | 193.9523 | 0.0086% | Simulate uncertainty (different work) | 23.3 ms | 293 ms |
| Defaults, ranges with changing growth, second seed | 365 rows | 193.9557 | 193.9523 | 0.0086% | Simulate uncertainty (different work) | 23.5 ms | 298 ms |
| Samples, weekly pattern | 96 rows | 71.9060 | 71.9060 | < 0.0001% | Simulate uncertainty (different work) | 1.22 ms | 80.4 ms |
| Ranges, eight months | 256 rows | 98.0804 | 98.0804 | < 0.0001% | Simulate uncertainty (different work) | 1.55 ms | 25.9 ms |
| Ranges with an explicit floor | 96 rows | 72.2190 | 72.2190 | < 0.0001% | Simulate uncertainty (different work) | 1.01 ms | 92.3 ms |
| Samples with an explicit floor | 96 rows | 72.2190 | 72.2190 | < 0.0001% | Simulate uncertainty (different work) | 733 µs | 24.1 ms |
| Ranges with mixed effects | 96 rows | 72.2270 | 72.2270 | < 0.0001% | Simulate uncertainty (different work) | 760 µs | 30.3 ms |
| Samples with mixed effects | 96 rows | 72.2270 | 72.2270 | < 0.0001% | Simulate uncertainty (different work) | 759 µs | 25.1 ms |