A regressor forecast by its own model

match ← All linear growth benchmarks

256 daily rows and four regressors, one forecast by its own default model. That model is fit by optimizing in both libraries, as mcmc_samples doesn't reach it in Python, and each of the 1,200 futures takes the next of its 1,000 draws in turn. Python doesn't save regressor models, so there are no save and load timings.

Input

benchmark/inputs/v1/map-regressors-medium.json: 256 history rows and 64 forecast rows.

Output

Largest difference: 0.018 (0.019% of the input's range).

Every forecast row from both libraries.

Date	Effect Prophet	Python Prophet	Difference
2020-09-13	107.1988	107.1947	0.0041
2020-09-14	107.6487	107.6433	0.0054
2020-09-15	108.0970	108.0904	0.0066
2020-09-16	108.5453	108.5374	0.0079
2020-09-17	107.9487	107.9411	0.0076
2020-09-18	110.1974	110.1895	0.0079
2020-09-19	110.6471	110.6380	0.0091
2020-09-20	111.0968	111.0865	0.010
2020-09-21	110.1079	110.0977	0.010
2020-09-22	110.5647	110.5532	0.011
2020-09-23	111.0258	111.0132	0.013
2020-09-24	111.4912	111.4774	0.014
2020-09-25	112.3190	112.3064	0.013
2020-09-26	112.7944	112.7807	0.014
2020-09-27	115.0738	115.0600	0.014
2020-09-28	115.5605	115.5458	0.015
2020-09-29	116.4115	116.3980	0.013
2020-09-30	113.7098	113.6937	0.016
2020-10-01	114.2174	114.2005	0.017
2020-10-02	114.7323	114.7145	0.018
2020-10-03	115.6121	115.5958	0.016
2020-10-04	116.1397	116.1226	0.017
2020-10-05	116.6760	116.6583	0.018
2020-10-06	119.0198	119.0024	0.017
2020-10-07	119.9291	119.9134	0.016
2020-10-08	120.4876	120.4713	0.016
2020-10-09	119.2526	119.2349	0.018
2020-10-10	119.8242	119.8060	0.018
2020-10-11	120.7578	120.7413	0.016
2020-10-12	121.3407	121.3239	0.017
2020-10-13	120.5265	120.5086	0.018
2020-10-14	121.1202	121.1020	0.018
2020-10-15	123.8759	123.8606	0.015
2020-10-16	124.4774	124.4618	0.016
2020-10-17	125.0819	125.0660	0.016
2020-10-18	123.8868	123.8697	0.017
2020-10-19	124.8528	124.8377	0.015
2020-10-20	125.4622	125.4469	0.015
2020-10-21	126.0729	126.0573	0.016
2020-10-22	126.6832	126.6675	0.016
2020-10-23	127.6504	127.6367	0.014
2020-10-24	130.0585	130.0455	0.013
2020-10-25	130.6623	130.6491	0.013
2020-10-26	129.8624	129.8482	0.014
2020-10-27	129.0191	129.0058	0.013
2020-10-28	129.6148	129.6013	0.014
2020-10-29	130.2060	130.1921	0.014
2020-10-30	130.7921	130.7778	0.014
2020-10-31	131.7303	131.7178	0.012
2020-11-01	132.3030	132.2900	0.013
2020-11-02	134.6709	134.6584	0.012
2020-11-03	135.2320	135.2190	0.013
2020-11-04	136.1446	136.1333	0.011
2020-11-05	134.8922	134.8793	0.013
2020-11-06	135.4327	135.4191	0.014
2020-11-07	135.9657	135.9514	0.014
2020-11-08	135.4447	135.4312	0.014
2020-11-09	135.9625	135.9482	0.014
2020-11-10	136.4730	136.4578	0.015
2020-11-11	138.7772	138.7621	0.015
2020-11-12	139.6320	139.6182	0.014
2020-11-13	140.1224	140.1077	0.015
2020-11-14	138.8063	138.7895	0.017
2020-11-15	139.2825	139.2647	0.018

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Fit	4.67 s	4.13 s	1.1× slower
Predict	1.10 ms	17.6 ms	Not compared: different work
Simulate uncertainty	16.1 ms	306 ms	Not compared: different work

Some steps aren't compared, because the two libraries' public calls produce different outputs. Their times show what each call costs, not which library is faster.

• Predict: Python's predict on a posterior also computes every component's range over the draws, while Effect Prophet's predict returns point forecasts and leaves ranges to predictWithIntervals.
• Simulate uncertainty: Python's predict(vectorized=False) also builds the point forecast and every component, while Effect Prophet's predictUncertainty returns only the ranges.

Settings

The case declaration both libraries ran.

{
  "kind": "map",
  "comparison": {
    "kind": "equivalent-posterior",
    "evidenceId": "posterior-nuts-v1"
  },
  "configuration": {
    "growth": "linear",
    "scaling": "absmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "auto",
      "count": 25,
      "range": 0.8
    },
    "changepointPriorScale": 0.05,
    "seasonalities": [],
    "events": [],
    "regressors": [
      {
        "name": "binary-auto",
        "priorScale": 10,
        "standardization": "auto"
      },
      {
        "name": "numeric-auto",
        "priorScale": 10,
        "standardization": "auto",
        "predictor": true
      },
      {
        "name": "numeric-always",
        "priorScale": 7,
        "standardization": "always"
      },
      {
        "name": "numeric-never",
        "priorScale": 5,
        "standardization": "never"
      }
    ]
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "uncertainty": {
    "seed": 19,
    "samples": 1000,
    "intervalWidth": 0.8,
    "output": "intervals",
    "algorithm": "scalar-continuous-time",
    "evidence": "docs/validation/uncertainty.md#evidence-classes",
    "samplerMemoryLimitBytes": 67108864
  },
  "mcmc": {
    "samples": 300,
    "seed": 19,
    "pointWidthFraction": 0.15
  }
}