Cross-validation with ranges

match ← All linear growth benchmarks

The same folds with 64 simulated futures per fold for interval coverage.

Input

benchmark/inputs/v1/linear-offset-scaling.json: 96 history rows and 24 forecast rows.

Output

Largest difference: 2.9e-11 (< 0.0001% of the input's range).

Each row is one forecast made during cross-validation, next to the value that actually happened.

Date	Actual	Effect Prophet	Python Prophet	Difference
2020-03-11	107.2568	107.1688	107.1688	2.9e-11
2020-03-12	107.4776	107.4962	107.4962	2.9e-11
2020-03-13	107.6213	107.7018	107.7018	2.8e-11
2020-03-14	107.7396	107.6554	107.6554	2.8e-11
2020-03-15	107.6534	107.5295	107.5295	2.7e-11
2020-03-16	107.5228	107.5532	107.5532	2.8e-11
2020-03-17	107.7277	107.7317	107.7317	2.8e-11
2020-03-18	108.1448	108.0262	108.0262	1.8e-11
2020-03-19	108.3767	108.3442	108.3442	1.8e-11
2020-03-20	108.4456	108.5444	108.5444	1.8e-11
2020-03-21	108.5155	108.5134	108.5134	1.7e-11
2020-03-22	108.4600	108.3912	108.3912	1.7e-11
2020-03-23	108.3204	108.4014	108.4014	1.8e-11
2020-03-24	108.4700	108.5823	108.5823	1.8e-11

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Cross-validate with ranges	9.74 ms	76.9 ms	7.9× faster

Settings

The case declaration both libraries ran.

{
  "kind": "evaluation",
  "comparison": {
    "kind": "different-public-work",
    "evidenceId": "rolling-origin-evaluation-v1"
  },
  "configuration": {
    "growth": "linear",
    "scaling": "absmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "explicit",
      "timestamps": [
        "2020-02-10T00:00:00.000Z"
      ]
    },
    "changepointPriorScale": 0.2,
    "seasonalities": [
      {
        "name": "weekly-custom",
        "periodDays": 7,
        "fourierOrder": 2,
        "priorScale": 10
      }
    ],
    "events": [],
    "regressors": []
  },
  "effectOptimizer": {
    "algorithm": "auto",
    "maxIterations": 10000,
    "fallback": "newton",
    "lbfgs": {
      "historySize": 5,
      "initAlpha": 0.001,
      "tolObj": 1e-12,
      "tolRelObj": 10000,
      "tolGrad": 1e-8,
      "tolRelGrad": 10000000,
      "tolParam": 1e-8
    }
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "plan": {
    "horizonMs": 604800000,
    "cutoffs": [
      "2020-03-10T00:00:00.000Z",
      "2020-03-17T00:00:00.000Z"
    ]
  },
  "metricAggregation": "overall",
  "mapeZeroActual": "exclude",
  "interval": {
    "seed": 19,
    "samples": 64,
    "intervalWidth": 0.8
  }
}