Time Series Forecasting Using Foundation Models

Time Series Forecasting Using Foundation Models demystifies the rapidly evolving world of large time models by guiding readers from first principles through real-world deployment. Author Marco Peixeiro, an active developer of TimeGPT at Nixtla, begins by unpacking the transformer architecture from a forecasting lens and then walks readers through building their own tiny foundation model with N-BEATS to viscerally experience concepts like pretraining, transfer learning, and fine-tuning. From there, the book systematically covers every major open and proprietary foundation forecasting model—TimeGPT, Lag-Llama, Chronos, Moirai, and TimesFM—explaining each model's architecture, pretraining corpus, hyperparameters, and optimal use cases before applying it to a consistent weekly store-sales benchmark. The book then ventures into LLM territory, showing how Flan-T5 and Llama-3.2 can be prompted for forecasting and how Time-LLM reprograms LLMs through patch reprogramming and Prompt-as-Prefix. A capstone project ties everything together by racing all methods—including classical SARIMA—against real blog-traffic data and comparing both accuracy and inference latency, leaving readers with a rigorous, reusable model-selection protocol they can apply to any forecasting challenge.

What it argues

A causal framework describing how design levers (model architecture choices, pretraining corpus properties, fine-tuning decisions, input configuration, and prompt engineering) combine with contextual conditions (dataset characteristics, hardware resources) to shape intermediate psychological and behavioral practitioner states, which in turn drive forecasting accuracy and operational outcomes.

Key ideas it contributes

• Pretraining Corpus Diversity — The breadth and heterogeneity of time series data used to pretrain a foundation model, encompassing the number of distinct application domains, the range of temporal sampling frequencies, the total volume of data (tokens or data points), and the variety of observable temporal patterns such as diverse trend shapes, seasonal periods, and noise structures.
• Pretraining Horizon Range — The minimum and maximum number of future time steps the model was trained to forecast during pretraining, establishing the practical forecast horizon within which the model's zero-shot and fine-tuned performance is reliable and beyond which accuracy typically degrades.
• Model Architecture Type — The structural design of the model's computational backbone, specifically whether it uses a full encoder-decoder transformer, an encoder-only transformer, a decoder-only transformer, or augmented variants with components such as mixture-of-experts layers or patch-based tokenization. Architecture type determines inference modality, whether output is generated autoregressively or in a single shot, and the model's suitability for probabilistic versus deterministic prediction.
• Model Parameter Count — The total number of trainable parameters in a foundation model, serving as a proxy for model capacity—the model's theoretical ability to represent complex patterns. Larger models can capture more nuanced temporal structures but require more memory, compute, and storage.
• Patching Strategy — The approach used to group consecutive time-series data points into multi-step tokens (patches) before embedding and processing by the transformer backbone, including the patch length, overlap policy, and whether patch length varies based on data frequency. Patching reduces the number of tokens fed to the attention mechanism, lowering computational cost and enabling the model to capture local temporal semantics across grouped time steps.
• Output Distribution Type — The probabilistic mechanism through which the model generates its predictions, ranging from a single deterministic point forecast to a full probability distribution over future values. Richer distributions (mixtures of multiple component distributions) enable more flexible, potentially asymmetric prediction intervals that better represent real-world forecast uncertainty.
• Fine-Tuning Steps / Epochs — The number of gradient update steps (or complete passes over the training data for epoch-based models) applied to adapt a pretrained foundation model to a specific target dataset during fine-tuning, balancing the tradeoff between specialization and overfitting.
• Fine-Tuning Depth — The proportion of the model's parameters that are updated during fine-tuning, ranging from updating only the final output layers (shallow) to updating all layers (deep). Greater depth allows more comprehensive adaptation to the target dataset but risks overfitting and requires more computation per step.