---
url: https://jer-k.github.io/effect-prophet/python/accuracy.md
description: >-
  How closely Effect Prophet matches Python Prophet 1.4.0, how that's checked,
  and why results aren't always identical.
---

# How close are the results?

Effect Prophet is checked against Python Prophet 1.4.0, feature by feature. Here's where each
trend type stands:

| Trend                                                  | Status                                     | Evidence                                           |
| ------------------------------------------------------ | ------------------------------------------ | -------------------------------------------------- |
| [Linear](../examples/trend)                            |  | [Linear growth benchmarks](/benchmarks/linear)     |
| [Flat](../examples/flat-trend)                         |  | [Flat growth benchmarks](/benchmarks/flat)         |
| [Growth with a ceiling](../examples/saturating-growth) |  | [Logistic growth benchmarks](/benchmarks/logistic) |

All three trend types are ready to use. That covers changepoints, seasonality, events,
regressors and the models that forecast them, multiplicative effects, uncertainty ranges, full
Bayesian fitting, cross-validation and choosing settings. The benchmarks for
[linear](/benchmarks/linear), [flat](/benchmarks/flat) and [logistic](/benchmarks/logistic) growth
show every case: its input, both libraries' forecasts side by side, and how long each took. The
feature pages gather the [seasonality, events and regressors](/benchmarks/features),
[uncertainty](/benchmarks/uncertainty), [Bayesian](/benchmarks/bayesian) and
[cross-validation](/benchmarks/cross-validation) cases from all three trends.

## How we test

1. Both libraries get the **same input file**: the same history rows and the same future dates.
2. They run with the **same settings**, either both on their defaults or both with the same options.
3. Each runs in its own container on the same machine. Python Prophet is version 1.4.0,
   unmodified.
4. We compare **every forecast row**, the trend, each seasonal pattern, event and regressor, and
   the fitted noise level.
5. A case matches when every number agrees within a small tolerance. Only matching cases are timed.

Separately, every pull request checks the library against stored Python Prophet results for each
feature, and runs property-based tests that throw generated input, valid and invalid, at fitting
and forecasting to check that every failure is a typed error rather than a crash.

Every week, a differential comparison goes further than the fixed benchmark cases. It generates
hundreds of new inputs, mixing trend types, features, cadences, gaps, outliers and very large or
very small values, then fits each one with both libraries and compares the results. Any case where
Effect Prophet fits worse than Python is recorded with a seed that reproduces it.

## Why the numbers aren't exactly zero

Fitting a model means searching for the best parameters, and both libraries stop searching once
they're close enough. Tiny rounding differences between computers can shift exactly where that
search stops, so forecasts often agree to many decimal places rather than all of them. Python
Prophet itself gives slightly different answers on Intel and ARM computers for the same reason.

On some histories, often ones with outliers, the default L-BFGS optimizer stops well before the
best fit in both libraries, and at slightly different points, so their forecasts differ by more.
Fitting with Newton brings both to the same answer.
[Did the optimizer finish?](../examples/residuals#did-the-optimizer-finish) shows how to check.

## Known exceptions

A few inputs are handled differently on purpose. For example, if every row has the same
timestamp, there's no way to draw a trend through them, so we return an error where Python
Prophet returns `NaN` forecasts. Each exception is listed in the
[benchmarks](/benchmarks/linear#intentional-differences) and in
[What's different](./differences).