Checking the fit

How well does a model follow your data? residuals predicts each of your measurements with a fitted model and returns how far off it was. A good fit leaves residuals that bounce around zero; a pattern in them shows something the model missed.

import { Effect } from "effect";
import { fit, prophetFittingBackendLayer, residuals } from "effect-prophet";

const date = (day: number) => new Date(Date.UTC(2025, 0, 1 + day)).toISOString();

// Sales grow steadily, then jump on day 60 and stay higher.
const history = Array.from({ length: 90 }, (_, day) => ({
  timestamp: date(day),
  value:
    100 +
    day * 0.5 +
    10 * Math.sin((2 * Math.PI * day) / 7) +
    4 * Math.sin(day * 12.9898) +
    (day >= 60 ? 15 : 0),
}));

const rows = await Effect.runPromise(
  Effect.gen(function* () {
    const model = yield* fit(history, {
      map: { changepoints: { mode: "explicit", timestamps: [] } },
    });

    return yield* residuals(model, history);
  }).pipe(Effect.provide(prophetFittingBackendLayer)),
);

// One point per day, ready for any charting library.
const points = rows.map((row) => ({
  date: new Date(row.timestamp).toISOString().slice(0, 10),
  residual: Number(row.residual.toFixed(1)),
}));

console.log(points.slice(0, 3));

// A straight trend can't follow the jump, so the residuals drift instead of centering on zero.
for (const start of [0, 30, 60]) {
  const block = rows.slice(start, start + 30);
  const mean = block.reduce((total, row) => total + row.residual, 0) / block.length;

  console.log(`days ${start}-${start + 29}: mean residual ${mean.toFixed(1)}`);
}

[
  { date: '2025-01-01', residual: 4.2 },
  { date: '2025-01-02', residual: 5.8 },
  { date: '2025-01-03', residual: 7.1 }
]
days 0-29: mean residual 1.2
days 30-59: mean residual -5.0
days 60-89: mean residual 3.7

This model has no changepoints, so its straight trend can't follow the jump on day 60. The residuals show it: the model predicts too high in the weeks before the jump, so they turn negative, and too low after it. Fitted with Prophet's automatic changepoints instead, the trend bends toward the jump and the drift shrinks to under 2 either way.

What you get back

One row per measurement, sorted by date. Measurements with the same timestamp keep the order you passed them in.

Field	Meaning
actual	The value you measured
predicted	The model's forecast for that moment, exactly as predict returns it
residual	actual - predicted: positive when the model predicted too low

Each measurement is predicted from its own row, so it needs whatever the model needs to forecast it: a capacity for growth with a ceiling, regressor values, and conditions for conditional seasonalities. Rows are never matched by timestamp, so two measurements on the same date with different regressor values each get their own prediction.

residuals never refits the model, and doesn't need a fitting backend. If a residual is too large to represent, which only happens with values near the limits of a JavaScript number, it fails with a PredictionError whose reason is non-finite-forecast.

Measurements the model hasn't seen

Residuals on the history a model was fitted on are always a little flattering: the model has already adjusted to those values. To see how it does on new data, hold some back:

import { Effect } from "effect";
import { fit, prophetFittingBackendLayer, residuals } from "effect-prophet";

const date = (day: number) => new Date(Date.UTC(2025, 0, 1 + day)).toISOString();

const sales = Array.from({ length: 104 }, (_, day) => ({
  timestamp: date(day),
  value: 100 + day * 0.5 + 10 * Math.sin((2 * Math.PI * day) / 7) + 4 * Math.sin(day * 12.9898),
}));

// Hold back the last two weeks; the model never sees them.
const training = sales.slice(0, 90);

const heldOut = sales.slice(90);

const meanAbsolute = (rows: ReadonlyArray<{ readonly residual: number }>) =>
  rows.reduce((total, row) => total + Math.abs(row.residual), 0) / rows.length;

const [inSample, outOfSample] = await Effect.runPromise(
  Effect.gen(function* () {
    const model = yield* fit(training);

    return [yield* residuals(model, training), yield* residuals(model, heldOut)] as const;
  }).pipe(Effect.provide(prophetFittingBackendLayer)),
);

console.log(`training days: mean absolute residual ${meanAbsolute(inSample).toFixed(2)}`);

console.log(`held-out days: mean absolute residual ${meanAbsolute(outOfSample).toFixed(2)}`);

training days: mean absolute residual 2.47
held-out days: mean absolute residual 2.64

residuals can't tell which measurements the model was trained on; that's up to you. For a proper accuracy estimate over several periods, use cross-validation.

Did the optimizer finish?

For histories of 100 rows or more, fit uses the L-BFGS optimizer, as Python Prophet does. On some histories, often ones with outliers or growth that changes sharply, it stops before it reaches the best fit. Python Prophet 1.4.0 stops short on the same histories, by about as much, but not at exactly the same point, so the two libraries agree less closely there.

To check a fit, take one Newton step from where it stopped:

import { Effect } from "effect";
import {
  fit,
  isMcmcModel,
  prophetFittingBackendLayer,
  warmStartParameters,
  type FittedProphet,
} from "effect-prophet";

const date = (day: number) => new Date(Date.UTC(2025, 0, 1 + day)).toISOString();

// A year of daily sales: growth that slows mid-year, a weekly pattern and four one-off spikes.
const sales = Array.from({ length: 365 }, (_, day) => ({
  timestamp: date(day),
  value:
    100 +
    (day < 180 ? day * 0.5 : 90 - (day - 180) * 0.2) +
    10 * Math.sin((2 * Math.PI * day) / 7) +
    3 * Math.sin(day * 12.9898) +
    ([50, 120, 230, 300].includes(day) ? 40 : 0),
}));

const report = await Effect.runPromise(
  Effect.gen(function* () {
    const model = yield* fit(sales);
    const warmStart = warmStartParameters(model);

    // One Newton step from where the fit stopped, on the same history.
    const step = yield* fit(sales, {
      map: { warmStart, optimizer: { algorithm: "newton", maxIterations: 1 } },
    });

    // Finish the fit with Newton, starting from the same place.
    const finished = yield* fit(sales, { map: { warmStart, optimizer: { algorithm: "newton" } } });

    return { model, step, finished };
  }).pipe(Effect.provide(prophetFittingBackendLayer)),
);

const { model, step, finished } = report;

// Models fitted with `mcmc` report a sampler instead of an optimizer.
const objective = (fitted: FittedProphet) =>
  isMcmcModel(fitted) ? Number.NaN : fitted.fitSummary.objective;

console.log(`stopped by:      ${model.fitSummary.termination}`);

console.log(`one step gains:  ${(objective(model) - objective(step)).toFixed(3)}`);

console.log(`Newton gains:    ${(objective(model) - objective(finished)).toFixed(3)}`);

stopped by:      relative-gradient
one step gains:  0.418
Newton gains:    0.713

• fitSummary.objective measures how badly the model fits its history and priors: lower is better. Compare it only between models fitted to the same history with the same settings.
• A gain above 0.01 means L-BFGS stopped early. In our tests, every fit that stopped more than 1 short of the best gained more than 0.02 from one step. Fits that finished usually gained less than 0.0001, though a few that stopped within 0.1 of the best gained up to 0.03. Finishing the fit with Newton from the same warmStart, as above, came within 0.2 of the best fit every time.
• fitSummary.termination can't tell you this. Fits that stop early end with the same reasons, usually relative-gradient, as fits that finish. Tighter L-BFGS tolerances in map.optimizer.lbfgs rarely help: the optimizer is creeping along, not giving up too soon.
• Stopping early rarely matters much for fits from scratch. In our tests it left the objective up to 22 short, which moved fitted values by at most a tenth of the history's standard deviation, and usually by less than 4%. A warm start after many new rows can do far worse; see Updating a model with new data.
• The check costs one Newton step: 1 to 50 milliseconds on our histories of up to six years of daily data. Newton itself is much slower than L-BFGS: finishing a fit, or fitting from scratch with Newton, took 2 to more than 100 times as long as the L-BFGS fit. Check first, and finish only the fits that need it. Newton is also subject to a memory limit on very large models.
• fit keeps L-BFGS and its tolerances by default so that results match Python Prophet's.