Activity25

Theoretical Background: ARIMA(\(p,d,q\))(\(P,D,Q\))[\(s\)] Model Interpretations

1. Autoregressive (AR) Model

An AR(1) model predicts the current value from its immediately preceding value. In its simplest form, it is written as

\[ y_t = c + \phi_1\,y_{t-1} + \varepsilon_t, \]

where
- \(y_t\) is the observed value at time \(t\),
- \(c\) is a constant (or drift),
- \(\phi_1\) is the AR coefficient, and
- \(\varepsilon_t\) is the error (white noise) at time \(t\).

2. Moving Average (MA) Component

An MA(1) model expresses the current value as a function of the current and one past error term:

\[ y_t = c + \varepsilon_t + \theta_1\,\varepsilon_{t-1}, \]

where \(\theta_1\) is the MA coefficient. This shows that the noise from the previous period also influences the current observation.

3. Combine Them into an ARMA Model

When both AR and MA parts are included, we have an ARMA(1,1) model:

\[ y_t = c + \phi_1\,y_{t-1} + \varepsilon_t + \theta_1\,\varepsilon_{t-1}. \]

This model now explains the series using both its past values and past errors.

5. Add Seasonality with Seasonal Differencing

Seasonal patterns can be removed by seasonal differencing. For example, if your data shows a weekly pattern (period \(s=7\)), the seasonal difference is

\[ \Delta_7 y_t = y_t - y_{t-7}. \]

If seasonality is also modeled through seasonal AR or MA terms, you can add these effects after differencing.

6. Build a Multiplicative Seasonal ARIMA Model

A full seasonal ARIMA model is usually denoted as ARIMA(\(p,d,q\))(\(P,D,Q\))[\(s\)], which combines:

  • Nonseasonal components: AR(\(p\)), differencing \(d\), and MA(\(q\)).
  • Seasonal components: Seasonal AR(\(P\)), seasonal differencing \(D\), and seasonal MA(\(Q\)) with period \(s\).

A simplified way to “build up” the full model in one long equation is as follows:

Step A: Apply nonseasonal differencing (order \(d\)) and seasonal differencing (order \(D\) at period \(s\)). Denote the fully differenced series as

\[ z_t = \Delta^d\,\Delta_s^D y_t. \]

Step B: Model \(z_t\) with nonseasonal AR and MA parts:

\[ z_t = c + \phi_1\,z_{t-1} + \cdots + \phi_p\,z_{t-p} + \varepsilon_t + \theta_1\,\varepsilon_{t-1} + \cdots + \theta_q\,\varepsilon_{t-q}. \]

Step C: Add the seasonal components. For a seasonal AR(\(P\)) and MA(\(Q\)) (applied to the same differenced series), include:

\[ -\,\Phi_1\,z_{t-s} - \cdots - \Phi_P\,z_{t-Ps} \]

on the AR side and

\[ -\,\Theta_1\,\varepsilon_{t-s} - \cdots - \Theta_Q\,\varepsilon_{t-Qs} \]

on the MA side.

Step D: Combine the pieces into one sequential equation:

\[ \begin{aligned} z_t &= c \\ &\quad + \underbrace{\phi_1\,z_{t-1} + \cdots + \phi_p\,z_{t-p}}_{\text{Nonseasonal AR terms}} \\ &\quad - \underbrace{\Phi_1\,z_{t-s} + \cdots + \Phi_P\,z_{t-Ps}}_{\text{Seasonal AR terms}} \\ &\quad + \varepsilon_t \\ &\quad + \underbrace{\theta_1\,\varepsilon_{t-1} + \cdots + \theta_q\,\varepsilon_{t-q}}_{\text{Nonseasonal MA terms}} \\ &\quad - \underbrace{\Theta_1\,\varepsilon_{t-s} + \cdots + \Theta_Q\,\varepsilon_{t-Qs}}_{\text{Seasonal MA terms}}. \end{aligned} \]

Here, \(z_t\) represents the series after both nonseasonal and seasonal differencing. This equation is a conceptual “blueprint” that shows how each modeling component is added sequentially.

Example Configurations

ARIMA(1,1,1) with Drift (Nonseasonal)

In this configuration, we first remove trends by differencing the series, then model the differenced series with one AR term and one MA term.

  1. Nonseasonal Differencing: \[ \begin{aligned} \Delta y_t = y_t - y_{t-1}. \end{aligned} \]

  2. Modeling the Differenced Series: \[ \begin{aligned} \Delta y_t &= c + \phi_1\,\Delta y_{t-1} + \varepsilon_t + \theta_1\,\varepsilon_{t-1}. \end{aligned} \]

Key Points: - The first difference \(\Delta y_t\) helps stabilize the mean. - \(\phi_1\) captures the autoregressive effect on the differenced data. - \(\theta_1\) models the influence of the previous error.

ARIMA(1,0,1)(0,1,1)[4] with Drift (Seasonal)

This model introduces seasonal differencing with period 4 and includes nonseasonal AR(1) and MA(1) terms along with a seasonal MA(1) component.

  1. Seasonal Differencing: \[ \begin{aligned} \Delta_4 y_t = y_t - y_{t-4}. \end{aligned} \]

  2. Sequentially Building the Model: \[ \begin{aligned} y_t - y_{t-4} &= c + \phi_1\,\Bigl[(y_{t-1} - y_{t-5})\Bigr] + \varepsilon_t + \theta_1\,\varepsilon_{t-1} - \Theta_1\,\varepsilon_{t-4}. \end{aligned} \]

Key Points:

  • Seasonal differencing (\(y_t - y_{t-4}\)) removes periodic effects.
  • \(\phi_1\) is the nonseasonal AR term applied to the lagged (but also seasonally adjusted) data.
  • \(\theta_1\) is the nonseasonal MA term.
  • \(\Theta_1\) is the seasonal MA term that adjusts the error from one season ago.
  • \(c\) is the drift.

Practice Configurations

Practice Problem 1

Problem: Write the sequential equation for a simple AR(1) model with drift.

Answer:

\[ \begin{aligned} y_t &= c + \phi_1\,y_{t-1} + \varepsilon_t. \end{aligned} \]

Practice Problem 2

Problem: Write the sequential equation for an ARIMA(0,1,1) model with drift—that is, first-difference the series and then apply an MA(1) model.

Answer:

  1. Differencing: \[ \begin{aligned} \Delta y_t = y_t - y_{t-1}. \end{aligned} \]

  2. Model Equation: \[ \begin{aligned} y_t - y_{t-1} &= c + \varepsilon_t + \theta_1\,\varepsilon_{t-1}. \end{aligned} \]

Practice Problem 3

Problem: Write the sequential equation for a seasonal ARIMA(0,1,1)(1,0,0)[12] model with drift.
(Hint: This model applies nonseasonal differencing and includes a seasonal AR(1) component with period 12 along with a nonseasonal MA(1) term.)

Answer:

  1. Nonseasonal Differencing: \[ \begin{aligned} \Delta y_t = y_t - y_{t-1}. \end{aligned} \]

  2. Incorporate Seasonal AR(1): \[ \begin{aligned} (y_t - y_{t-1}) - \Phi_1\,(y_{t-12} - y_{t-13}) &= c + \varepsilon_t + \theta_1\,\varepsilon_{t-1}. \end{aligned} \]

Key Points:

  • \(\Phi_1\) is the seasonal AR coefficient with period 12.
  • \(\theta_1\) is the nonseasonal MA coefficient.
  • The left-hand side shows the nonseasonally differenced data adjusted by the seasonal AR component.