Country calendar uncertainty replay

match ← All flat growth benchmarks

32 seeded scalar draws on the US/custom-event model, with interval reduction, portable JSON and fresh-process replay checks. Individual Python and Effect draws need not match.

Input

benchmark/inputs/v5/country-basic-96.json: 96 history rows and 4 forecast rows.

Output

Largest difference: 0.00015 (0.0048% of the input's range).

Every forecast row from both libraries.

Date	Effect Prophet	Python Prophet	Difference
2024-12-25	20.0010	20.0010	2.2e-6
2025-01-01	23.0356	23.0354	0.00015
2025-01-02	20.0010	20.0010	2.2e-6
2025-01-05	21.4447	21.4446	7.0e-5

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Convert input rows	13 µs	457 µs	35× faster
Save model to JSON	87 µs	713 µs	8.2× faster
Load model from JSON	1.12 ms	4.00 ms	3.6× faster
Simulate uncertainty	341 µs	14.7 ms	Not compared: different work
First simulation in a new process	191 ms	1.65 s	Not compared: different work
Load and simulate in a new process	165 ms	1.61 s	Not compared: different work

Some steps aren't compared, because the two libraries' public calls produce different outputs. Their times show what each call costs, not which library is faster.

• Simulate uncertainty, First simulation in a new process, Load and simulate in a new process: Python's predict(vectorized=False) also builds the point forecast and every component, while Effect Prophet's predictUncertainty returns only the ranges.

Settings

The case declaration both libraries ran.

{
  "kind": "map",
  "comparison": {
    "kind": "equivalent-objective",
    "evidenceId": "country-calendar-public-v1"
  },
  "configuration": {
    "growth": "flat",
    "countryHolidays": {
      "country": "US",
      "coverage": "reject-known-gaps",
      "priorScale": 10
    },
    "scaling": "absmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "auto",
      "count": 25,
      "range": 0.8
    },
    "changepointPriorScale": 0.05,
    "seasonalities": [],
    "events": [
      {
        "name": "launch",
        "date": "2024-01-01",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2024-01-05",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2025-01-01",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2025-01-05",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      }
    ],
    "regressors": []
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "uncertainty": {
    "seed": 2024,
    "samples": 32,
    "intervalWidth": 0.8,
    "output": "intervals",
    "algorithm": "scalar-continuous-time",
    "evidence": "docs/validation/uncertainty.md#evidence-classes",
    "samplerMemoryLimitBytes": 67108864
  }
}