Decomposition and stationarity

Forecasting is the discipline machine learning keeps trying to absorb and keeps getting humbled by. The reason is structural: a time series is one realization of a process — you cannot i.i.d.-resample your way to more Tuesdays — so everything the Probability module built on independence needs rebuilding on dependence. This track does that rebuild. It starts where every competent analysis starts: not with a model, but with a decomposition that makes the series tell you what it is made of.

The classical decomposition

The working hypothesis for most business and physical series:

or $y_t = T_t \cdot S_t \cdot R_t$ (multiplicative — seasonal swings that scale with the level; take logs and it becomes additive, the standard trick). The classical estimation, still worth knowing because every fancier method refines it: estimate $T_t$ by a centered moving average over one full seasonal period (a 2×12-MA for monthly data — averaging over exactly one cycle cancels the seasonality); detrend; estimate $S_t$ as the average of each season’s detrended values; what remains is $R_t$. Modern practice uses STL (loess-based, robust, allows slowly-changing seasonality), but the logic is identical.

The readout’s strength statistics ($F_T$, $F_S$ — variance ratios due to Hyndman) turn the visual into numbers you can threshold: they are how automated pipelines decide whether a series is “seasonal enough” to model seasonally.

The ACF: dependence made visible

The autocorrelation function $\rho_k = \operatorname{Corr}(y_t, y_{t-k})$ is the field’s stethoscope. Its shapes are diagnostic vocabulary: slow near-linear decay ⟹ trend (each value drags its neighbors); spikes at the seasonal lag and multiples ⟹ seasonality; everything inside the $\pm 2/\sqrt{n}$ band ⟹ white noise, nothing left to model. The band comes straight from the CLT lesson: under whiteness, sample autocorrelations are approximately $\mathcal N(0, 1/n)$, so bars outside it are evidence of real structure. Two ACF habits pay for the whole track: always look at the ACF of your residuals (a model is done when they are white), and never trust an ACF computed on a trending series to tell you about anything but the trend.

Stationarity: the license every model needs

A process is (weakly) stationary when its mean, variance and autocovariances do not depend on $t$ — the statistical rules of the game are time-invariant. Why it matters is almost philosophical: forecasting from one realization is only possible if patterns learned from the past apply to the future, and stationarity is that assumption stated precisely. Trends break it (mean moves), seasonality breaks it (mean cycles), variance growth breaks it (heteroscedasticity).

The repair kit, in the order to try it:

• Log (or Box–Cox) transform — stabilizes growing variance and turns multiplicative structure additive.
• First difference $y'_t = y_t - y_{t-1}$ — removes a (stochastic) trend. A random walk differences to white noise; that a series needs differencing is called a unit root.
• Seasonal difference $y_t - y_{t-m}$ — removes stable seasonality.
• Formal check: the ADF test (null: unit root — small p ⟹ stationary) and KPSS (null reversed); the honest workflow uses both plus the ACF’s visual verdict, because near-unit-root cases genuinely straddle the line.

Over-differencing is real: difference a stationary series and you inject an MA(1) with $\rho_1 \approx -0.5$, making forecasts worse. The next two lessons build the models this preparation feeds — exponential smoothing (which handles trend and seasonality inside the model) and ARIMA (where the “I” is literally the differencing you just learned).

import numpy as np, pandas as pd
from statsmodels.tsa.seasonal import STL
from statsmodels.tsa.stattools import adfuller, acf

rng = np.random.default_rng(0)
t = np.arange(120)
y = pd.Series(10 + 0.08*t + 2*np.sin(2*np.pi*t/12) + 0.8*rng.standard_normal(120))

res = STL(y, period=12).fit()                 # trend/seasonal/resid components
print(acf(res.resid.dropna(), nlags=13)[[1, 12]])   # both should be ≈ 0

print(adfuller(y)[1])                          # p large: trending, not stationary
print(adfuller(y.diff().dropna())[1])          # p tiny: differencing bought it

Exercises