Samples with mixed effects

match ← All logistic growth benchmarks

128 raw simulated values per date for the same mixed model.

Input

benchmark/inputs/v1/logistic-uncertainty-mixed.json: 96 history rows and 24 forecast rows.

Output

Largest difference: 1.8e-11 (< 0.0001% of the input's range).

Every forecast row from both libraries.

Date	Effect Prophet	Python Prophet	Difference
2020-04-06	65.8253	65.8253	1.8e-11
2020-04-07	66.2469	66.2469	1.7e-11
2020-04-08	67.2932	67.2932	1.7e-11
2020-04-09	67.5695	67.5695	1.6e-11
2020-04-10	67.5171	67.5171	1.6e-11
2020-04-11	68.0061	68.0061	1.6e-11
2020-04-12	67.8998	67.8998	1.5e-11
2020-04-13	67.9356	67.9356	1.5e-11
2020-04-14	68.6942	68.6942	1.4e-11
2020-04-15	68.9301	68.9301	1.4e-11
2020-04-16	70.9005	70.9005	1.5e-11
2020-04-17	69.8892	69.8892	1.3e-11
2020-04-18	69.9044	69.9044	1.3e-11
2020-04-19	69.7593	69.7593	1.3e-11
2020-04-20	70.5952	70.5952	1.2e-11
2020-04-21	70.0978	70.0978	1.1e-11
2020-04-22	70.6932	70.6932	1.1e-11
2020-04-23	71.2651	71.2651	1.1e-11
2020-04-24	71.5724	71.5724	1.1e-11
2020-04-25	71.3613	71.3613	1.0e-11
2020-04-26	71.7778	71.7778	1.0e-11
2020-04-27	71.3481	71.3481	9.6e-12
2020-04-28	71.6595	71.6595	9.2e-12
2020-04-29	72.2270	72.2270	8.9e-12

Timing

Median of all measured repetitions.

Step	Effect Prophet	Python Prophet	Effect Prophet is
Convert input rows	58 µs	517 µs	8.9× faster
Save model to JSON	25 µs	667 µs	26× faster
Load model from JSON	179 µs	3.74 ms	21× faster
Simulate uncertainty	759 µs	25.1 ms	Not compared: different work
First simulation in a new process	194 ms	1.48 s	Not compared: different work
Load and simulate in a new process	163 ms	1.45 s	Not compared: different work

Some steps aren't compared, because the two libraries' public calls produce different outputs. Their times show what each call costs, not which library is faster.

• Simulate uncertainty, First simulation in a new process, Load and simulate in a new process: Python's predictive_samples(vectorized=False) and Effect Prophet's predictUncertainty are different public calls: Python's timing includes resetting NumPy's random generator and returns NumPy matrices, and the two don't draw the same random numbers.

Settings

The case declaration both libraries ran.

{
  "kind": "map",
  "comparison": {
    "kind": "equivalent-objective",
    "evidenceId": "logistic-map-v1"
  },
  "configuration": {
    "growth": "logistic",
    "scaling": "minmax",
    "seasonalityMode": "additive",
    "holidaysMode": "additive",
    "changepoints": {
      "mode": "explicit",
      "timestamps": [
        "2020-02-10T00:00:00.000Z"
      ]
    },
    "changepointPriorScale": 0.2,
    "seasonalities": [
      {
        "name": "weekly-custom",
        "periodDays": 7,
        "fourierOrder": 2,
        "priorScale": 10,
        "conditionName": "active"
      }
    ],
    "events": [
      {
        "name": "campaign",
        "date": "2020-01-22",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      },
      {
        "name": "campaign",
        "date": "2020-04-16",
        "lowerWindowDays": 0,
        "upperWindowDays": 0,
        "priorScale": 10
      }
    ],
    "regressors": [
      {
        "name": "promotion",
        "priorScale": 10,
        "standardization": "never",
        "mode": "multiplicative"
      }
    ]
  },
  "pythonOptimizer": {
    "algorithm": "Auto",
    "maxIterations": 10000,
    "newtonFallback": true,
    "sigFigs": 12
  },
  "uncertainty": {
    "seed": 19,
    "samples": 128,
    "intervalWidth": 0.8,
    "output": "samples",
    "algorithm": "scalar-continuous-time",
    "evidence": "docs/validation/uncertainty.md#evidence-classes",
    "samplerMemoryLimitBytes": 67108864
  }
}