Cross-validation, eight months

match ← All logistic growth benchmarks

Two rolling cutoffs on the 256-day history, with rolling-window metrics.

Input

benchmark/inputs/v1/logistic-implicit-floor-large.json: 256 history rows and 64 forecast rows.

Output

Largest difference: 0.028 (0.034% of the input's range).

Each row is one forecast made during cross-validation, next to the value that actually happened.

Date	Actual	Effect Prophet	Python Prophet	Difference
2020-08-13	87.3514	87.3331	87.3137	0.019
2020-08-14	87.6488	87.5376	87.5413	0.0037
2020-08-15	87.4249	87.4024	87.4110	0.0086
2020-08-16	87.0921	87.1415	87.1515	0.010
2020-08-17	87.0732	87.0283	87.0564	0.028
2020-08-18	87.2845	87.2244	87.2453	0.021
2020-08-19	87.6215	87.6939	87.6790	0.015
2020-08-20	88.1043	88.1796	88.1595	0.020
2020-08-21	88.4210	88.3871	88.3894	0.0024
2020-08-22	88.2403	88.2486	88.2558	0.0072
2020-08-23	87.8794	87.9853	87.9940	0.0086
2020-08-24	87.8558	87.8750	87.9012	0.026
2020-08-25	88.1455	88.0707	88.0902	0.019
2020-08-26	88.5220	88.5367	88.5214	0.015

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Convert input rows	45 µs	479 µs	11× faster
Plan cross-validation folds	2.04 ms	721 µs	2.8× slower
Cross-validate	14.2 ms	42.2 ms	3.0× faster
Compute metrics	238 µs	5.40 ms	23× faster
Score baselines	2.34 ms	1.35 ms	1.7× slower
First cross-validation in a new process	185 ms	1.44 s	7.8× faster

Settings

The case declaration both libraries ran.

{
  "kind": "evaluation",
  "comparison": {
    "kind": "different-public-work",
    "evidenceId": "rolling-origin-evaluation-v1"
  },
  "configuration": {
    "growth": "logistic",
    "scaling": "minmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "explicit",
      "timestamps": [
        "2020-02-10T00:00:00.000Z"
      ]
    },
    "changepointPriorScale": 0.2,
    "seasonalities": [
      {
        "name": "weekly-custom",
        "periodDays": 7,
        "fourierOrder": 2,
        "priorScale": 10
      }
    ],
    "events": [],
    "regressors": []
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "plan": {
    "horizonMs": 604800000,
    "cutoffs": [
      "2020-08-12T00:00:00.000Z",
      "2020-08-19T00:00:00.000Z"
    ]
  },
  "metricAggregation": "rolling",
  "mapeZeroActual": "exclude"
}