Neural and global forecasting

Every model so far shares an unstated constraint: one series, one model. ETS and ARIMA learn a store’s demand from that store’s history alone — fine with ten years of data, hopeless with ten weeks. But a retailer has 50,000 stores × products, and their histories rhyme: promotions spike the same way, seasonality repeats across items, launches follow launch-shaped curves. The deep-learning era’s real contribution to forecasting is not “neural beats ARIMA on a series” (it usually doesn’t). It is the global model: one network fitted across all series, so that statistical strength pools and a two-week-old product borrows the seasonal shape of a thousand mature cousins. This lesson maps that shift and prices it honestly.

Why global works: the bias–variance ledger

A local model per series has low bias (each series gets bespoke parameters) and brutal variance on short histories. A global model asserts shared structure — one function $f_\theta(\text{history}, \text{covariates})$ for every series — trading bias (series that genuinely differ get averaged tendencies) for a massive variance reduction (parameters fitted on millions of observations instead of dozens). The bias is then bought back with capacity and covariates: series-ID embeddings (learned per-series vectors, the LLM track’s token embeddings doing a forecasting job), price/promo/ holiday features, category metadata. The ledger explains the empirical regularities: many related series + short histories ⟹ global wins big; few, long, idiosyncratic series ⟹ local classical methods remain champions.

Two anatomies worth knowing

DeepAR (Amazon, 2017) — the probabilistic RNN template. An LSTM (built in the Deep Learning track) consumes each series’ recent history plus covariates and emits, at every step, the parameters of a distribution (Gaussian, or negative binomial for counts): $p(y_t \mid \text{past}) = \mathcal N(\mu_\theta(h_t), \sigma_\theta(h_t))$. Training maximizes likelihood — the MLE lesson, verbatim — and forecasting samples trajectories autoregressively, so you get calibrated quantiles (the P10/P50/P90 that inventory decisions actually need), not just a line. Its descendants swap the RNN for a transformer (TFT adds attention-based covariate selection and interpretable gates) but keep the probabilistic head.

N-BEATS (2019) — the pure MLP rebuttal. No recurrence, no attention: stacked blocks each look at a fixed lookback window, emit a backcast (what of the input this block explains) and a forecast, and pass the backcast-subtracted residual to the next block — boosting’s residual-fitting loop (Supervised ML module) reborn as an architecture. With basis-restricted blocks (polynomial trend, Fourier seasonality) it even yields decomposition- style interpretability: the components of lesson 1, learned end to end. Its significance was empirical: on M4, an ensemble of these plain MLPs beat the winning hybrid — architecture sophistication is not where forecasting accuracy comes from.

What the M-competitions actually showed

The field’s honesty ritual: blind forecasting tournaments since 1982. The results practitioners should carry:

• M3 (2000): simple statistical methods (damped-trend ETS!) beat every early ML entrant. This result held for nearly two decades and is why seasoned forecasters roll their eyes at model-of-the-week claims.
• M4 (2018): 100k heterogeneous series. Winner: Smyl’s hybrid — ES handling seasonality per-series with a global RNN learning shared residual structure. Pure ML entries mostly underperformed statistical baselines; the lesson was pooling + hybridization, not replacement.
• M5 (2020): Walmart’s hierarchical retail data — the global-model home turf. Winners: LightGBM with heavy feature engineering (lag features, rolling stats, calendar covariates), ahead of most deep entries. Gradient boosting on tabularized series remains the strongest practical baseline for large retail-style panels, a fact marketing rarely mentions.
• Across all of them: ensembling helps, uncertainty matters (M5’s separate quantile track), and the gap between a tuned classical baseline and the winner is far smaller than the gap between a careless pipeline and a careful one.

The decision rule

Choose by regime, not fashion: few long series → ETS/ARIMA (+ their state-space uncertainty); many related series, rich covariates → global LightGBM on engineered features first (cheap, strong, debuggable), deep global models (DeepAR-style if you need coherent sampled trajectories; N-BEATS/transformer variants for raw accuracy) when the boosted baseline is demonstrably saturated; any regime → keep a seasonal-naive and an ETS in the evaluation as tripwires. If the fancy model cannot beat damped ETS backtested properly (next lesson), the fancy model is a liability with a maintenance bill. And intermittent/count demand (mostly zeros) is its own subfield — Croston’s method and negative-binomial heads, not Gaussians.

One more caveat the papers bury: global models couple series at inference time — a data-quality disaster in one region can shift forecasts everywhere, and retraining cadence becomes a governance question, not just an MLOps one.

# the tabular-global recipe that wins M5-style problems, sketched
import pandas as pd, lightgbm as lgb

# df: columns [series_id, date, y, price, promo, ...]
def features(df):
    g = df.groupby("series_id")["y"]
    for lag in [1, 7, 28]:
        df[f"lag_{lag}"] = g.shift(lag)
    for w in [7, 28]:
        df[f"rmean_{w}"] = g.shift(1).rolling(w).mean().reset_index(0, drop=True)
    df["dow"] = df["date"].dt.dayofweek
    return df.dropna()

# one model, ALL series — series_id as a categorical feature = learned embedding
# model = lgb.LGBMRegressor(objective="tweedie")   # tweedie: retail's zero-heavy friend
# model.fit(X[feats + ["series_id"]], X["y"], categorical_feature=["series_id"])

Exercises