Country calendar uncertainty replay

match ← All flat growth benchmarks

32 seeded scalar draws on the US/custom-event model, with interval reduction, portable JSON and fresh-process replay checks. Individual Python and Effect draws need not match.

Input

benchmark/inputs/v5/country-basic-96.json: 96 history rows and 4 forecast rows.

Output

Largest difference: 0.00015 (0.0048% of the input's range).

Every forecast row from both libraries.

Date	Effect Prophet	Python Prophet	Difference
2024-12-25	20.0010	20.0010	2.2e-6
2025-01-01	23.0356	23.0354	0.00015
2025-01-02	20.0010	20.0010	2.2e-6
2025-01-05	21.4447	21.4446	7.0e-5

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Convert input rows	14 µs	444 µs	32× faster
Save model to JSON	115 µs	687 µs	6.0× faster
Load model from JSON	1.24 ms	3.96 ms	3.2× faster
Simulate uncertainty	318 µs	14.8 ms	46× faster
First simulation in a new process	197 ms	1.66 s	8.4× faster
Load and simulate in a new process	160 ms	1.62 s	10× faster

Settings

The case declaration both libraries ran.

{
  "kind": "map",
  "comparison": {
    "kind": "equivalent-objective",
    "evidenceId": "country-calendar-public-v1"
  },
  "configuration": {
    "growth": "flat",
    "countryHolidays": {
      "country": "US",
      "coverage": "reject-known-gaps",
      "priorScale": 10
    },
    "scaling": "absmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "auto",
      "count": 25,
      "range": 0.8
    },
    "changepointPriorScale": 0.05,
    "seasonalities": [],
    "events": [
      {
        "name": "launch",
        "date": "2024-01-01",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2024-01-05",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2025-01-01",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "launch",
        "date": "2025-01-05",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      }
    ],
    "regressors": []
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "uncertainty": {
    "seed": 2024,
    "samples": 32,
    "intervalWidth": 0.8,
    "output": "intervals",
    "algorithm": "scalar-continuous-time",
    "evidence": "docs/validation/uncertainty.md#evidence-classes",
    "samplerMemoryLimitBytes": 67108864
  }
}