Ranges with a forecast regressor

match flagged ← All linear growth benchmarks

80% ranges from 1,000 simulated paths on every past and future date, each future path using one of the regressor model's own simulated values.

Input

benchmark/inputs/v1/map-regressors-medium.json: 256 history rows and 64 forecast rows.

Output

Largest difference: 0.0039 (0.0040% of the input's range).

Every forecast row from both libraries.

Date	Effect Prophet	Python Prophet	Difference
2020-09-13	107.1941	107.1951	0.00099
2020-09-14	107.6427	107.6439	0.0012
2020-09-15	108.0898	108.0913	0.0014
2020-09-16	108.5369	108.5386	0.0017
2020-09-17	107.9404	107.9420	0.0016
2020-09-18	110.1882	110.1900	0.0018
2020-09-19	110.6368	110.6388	0.0020
2020-09-20	111.0853	111.0875	0.0022
2020-09-21	110.0970	110.0991	0.0021
2020-09-22	110.5526	110.5549	0.0023
2020-09-23	111.0126	111.0151	0.0025
2020-09-24	111.4768	111.4796	0.0028
2020-09-25	112.3056	112.3082	0.0026
2020-09-26	112.7799	112.7827	0.0028
2020-09-27	115.0587	115.0616	0.0029
2020-09-28	115.5445	115.5476	0.0031
2020-09-29	116.3966	116.3995	0.0029
2020-09-30	113.6928	113.6960	0.0032
2020-10-01	114.1996	114.2030	0.0034
2020-10-02	114.7136	114.7171	0.0035
2020-10-03	115.5947	115.5980	0.0033
2020-10-04	116.1216	116.1250	0.0034
2020-10-05	116.6573	116.6608	0.0035
2020-10-06	119.0008	119.0044	0.0036
2020-10-07	119.9115	119.9148	0.0034
2020-10-08	120.4694	120.4728	0.0035
2020-10-09	119.2335	119.2371	0.0036
2020-10-10	119.8046	119.8083	0.0037
2020-10-11	120.7398	120.7432	0.0034
2020-10-12	121.3223	121.3258	0.0035
2020-10-13	120.5070	120.5106	0.0036
2020-10-14	121.1003	121.1040	0.0037
2020-10-15	123.8581	123.8614	0.0033
2020-10-16	124.4592	124.4627	0.0034
2020-10-17	125.0634	125.0669	0.0035
2020-10-18	123.8677	123.8713	0.0035
2020-10-19	124.8355	124.8387	0.0032
2020-10-20	125.4447	125.4479	0.0033
2020-10-21	126.0550	126.0583	0.0033
2020-10-22	126.6651	126.6685	0.0034
2020-10-23	127.6341	127.6371	0.0030
2020-10-24	130.0422	130.0453	0.0031
2020-10-25	130.6458	130.6489	0.0031
2020-10-26	129.8449	129.8481	0.0032
2020-10-27	129.0029	129.0058	0.0029
2020-10-28	129.5983	129.6013	0.0030
2020-10-29	130.1891	130.1921	0.0031
2020-10-30	130.7747	130.7779	0.0032
2020-10-31	131.7146	131.7174	0.0029
2020-11-01	132.2868	132.2897	0.0029
2020-11-02	134.6545	134.6575	0.0030
2020-11-03	135.2151	135.2182	0.0031
2020-11-04	136.1292	136.1321	0.0028
2020-11-05	134.8758	134.8788	0.0030
2020-11-06	135.4156	135.4187	0.0031
2020-11-07	135.9478	135.9511	0.0033
2020-11-08	135.4275	135.4306	0.0031
2020-11-09	135.9445	135.9477	0.0032
2020-11-10	136.4541	136.4575	0.0034
2020-11-11	138.7578	138.7613	0.0035
2020-11-12	139.6137	139.6171	0.0033
2020-11-13	140.1032	140.1067	0.0035
2020-11-14	138.7857	138.7894	0.0037
2020-11-15	139.2608	139.2648	0.0039

Internal fit differences

The forecasts match, so this case passes. Both libraries search for the best-fitting parameters and stop when they are close enough. On this input they stopped at slightly different points, so these internal numbers differ by more than the threshold where we take a closer look.

Check	Effect Prophet	Python Prophet	Threshold
Fitting objective (lower is a better fit)	-1758.3378	-1758.3912	0.01
Stationarity residual (lower is closer to the best fit)	16848.4352	11056.4239	0.01

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Convert input rows	89 µs	680 µs	7.7× faster
Simulate uncertainty	9.28 ms	246 ms	Not compared: different work
First simulation in a new process	230 ms	1.82 s	Not compared: different work

Some steps aren't compared, because the two libraries' public calls produce different outputs. Their times show what each call costs, not which library is faster.

• Simulate uncertainty, First simulation in a new process: Python's predict(vectorized=False) also builds the point forecast and every component, while Effect Prophet's predictUncertainty returns only the ranges.

Settings

The case declaration both libraries ran.

{
  "kind": "map",
  "comparison": {
    "kind": "equivalent-objective",
    "evidenceId": "linear-output-first-v4"
  },
  "configuration": {
    "growth": "linear",
    "scaling": "absmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "auto",
      "count": 25,
      "range": 0.8
    },
    "changepointPriorScale": 0.05,
    "seasonalities": [],
    "events": [],
    "regressors": [
      {
        "name": "binary-auto",
        "priorScale": 10,
        "standardization": "auto"
      },
      {
        "name": "numeric-auto",
        "priorScale": 10,
        "standardization": "auto",
        "predictor": true
      },
      {
        "name": "numeric-always",
        "priorScale": 7,
        "standardization": "always"
      },
      {
        "name": "numeric-never",
        "priorScale": 5,
        "standardization": "never"
      }
    ]
  },
  "effectOptimizer": {
    "algorithm": "auto",
    "maxIterations": 10000,
    "fallback": "newton",
    "lbfgs": {
      "historySize": 5,
      "initAlpha": 0.001,
      "tolObj": 1e-12,
      "tolRelObj": 10000,
      "tolGrad": 1e-8,
      "tolRelGrad": 10000000,
      "tolParam": 1e-8
    }
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "uncertainty": {
    "seed": 19,
    "samples": 1000,
    "intervalWidth": 0.8,
    "output": "intervals",
    "algorithm": "scalar-continuous-time",
    "evidence": "docs/validation/uncertainty.md#evidence-classes",
    "samplerMemoryLimitBytes": 67108864
  }
}