Noise and stochastic model

Noise and stochastic model#

The code on this page can be used interactively: click –> Live Code in the top right corner, then wait until the message Python interaction ready! appears.

In the previous section you have learned about the different components that can be present in a time series. Removing all these component, i.e. the functional model, we will be left with the residual term \(\epsilon(t)\). In this section we will take a closer look at the difference between signal and noise, and introduce other types of noise than the traditional white noise.

Additional concepts#

In Signal Processing the data is just considered to be the signal of interest, whereas here we assume the data is “contaminated” with noise, i.e.

\[Y = \text{signal} + \text{noise} \]

Time series analysis means understanding patterns and, hence, extracting the signal of interest from the noisy data.

Signal and noise#

How can we describe both signal and noise?

  • Signal - the meaningful information that we want to detect: deterministic characteristics by means of mathematical expressions to capture for example trend, seasonality and offsets.

  • Noise - random and undesired fluctuation that interferes with the signal: stochastic process are needed to describe this. Parts of the time-correlated noise needs to be accounted for in prediction, see later AR process.

The example in Fig. 5 shows that the signal can be described by \(\cos(2\pi t f) + \sin(2\pi t f)\). The stochastic model (assuming independent normally distributed observations) would be a scaled identity matrix with variance equal to 1 (middle panel) and 9 (bottom panel), respectively. The signal of interest has been entirely hidden in the background noise in the bottom panel. Techniques from signal processing can be used to detect the frequency.

../_images/signal_noise.png

Fig. 5 Example of a time series (top graph) affected by noise with different strengths (middle and bottom figures). Note the different scales on the vertical axes.#

Signal to noise ratio#

In signal processing the signal to noise ratio is commonly used to report on the amount of noise present in the model. If we analyze the model \(Y = signal + noise\), then Y is a random variable with \(E[Y] = E[signal] = \mu\), and its variance \(D(Y) = D(noise) = \sigma^2\). Using this the signal to noise ratio is often defined as:

\[ SNR = \frac{\mu}{\sigma}\]

The signal to noise ratio is a measure of how much the signal stands out from the noise. The higher the signal to noise ratio, the more the signal stands out from the noise. Better equipment or more data can increase the signal to noise ratio.

Different types of noise#

In the ideal case, when the signal is removed, you are left with white noise. A white noise stochastic model has the following properties:

\[\begin{split} \mathbb{E}(Y) = \mathbb{E} \left[\begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_m \end{array}\right] = \left[\begin{array}{c} 0 \\ 0 \\ \vdots \\ 0 \end{array}\right] \end{split}\]

and

\[\begin{split} \mathbb{D}(Y) = \Sigma_{Y} = \sigma^2 \left[\begin{array}{ccc} 1 & 0 & \ldots{} & 0 \\ 0 & 1 & \ldots{} & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots{} & 1 \end{array}\right] \end{split}\]

Most notable, all observations are uncorrelated (off-diagonal elements of the covariance matrix are equal to 0). When we compute the PSD, the resulting density will be flat over the entire range of frequencies. In other words, a white noise process has equal energy over all frequencies, just like white light. We will show this in the interactive plot at the bottom of this page.

Colored noise#

In time series it is not guarantied that the individual observations are uncorrelated. At the bottom of this page you will find an interactive plot. You can select four different types of noise: white, pink, red and blue. The noise processes are plotted in combination with the PSD. The PSD #TODO(add ref to psd)is a measure of the power of the signal at different frequencies. The white noise process has a flat PSD, while the other noise processes have a different shape. The pink noise process has a PSD that decreases with frequency, the red noise process has a PSD that decreases quadratically with frequency, and the blue noise process has a PSD that increases with frequency.

## create a white noise signal and plot it
import numpy as np  
import matplotlib.pyplot as plt
import ipywidgets as widgets

# create a white noise signal
np.random.seed(0)
N = 1000
x = np.random.randn(N)

# Function to generate pink noise
def pink_noise(N):
    uneven = N % 2
    X = np.random.randn(N//2+1+uneven) + 1j * np.random.randn(N//2+1+uneven)
    S = np.sqrt(np.arange(len(X)) + 1.)  # +1 to avoid divide by zero
    y = (np.fft.irfft(X/S)).real
    if uneven:
        y = y[:-1]
    return y

# Function to generate red (brown) noise
def red_noise(N):
    return np.cumsum(np.random.randn(N))

# Function to generate blue noise
def blue_noise(N):
    uneven = N % 2
    X = np.random.randn(N//2+1+uneven) + 1j * np.random.randn(N//2+1+uneven)
    S = np.sqrt(np.arange(len(X)))  # no +1 here
    y = (np.fft.irfft(X*S)).real
    if uneven:
        y = y[:-1]
    return y
# BEGIN: white_noise function
def white_noise(N):
    return np.random.randn(N)


# Generate different noise signals
pink = pink_noise(N)
red = red_noise(N)
blue = blue_noise(N)
x = white_noise(N)

noise_options = ['Pink Noise', 'Red Noise', 'Blue Noise', 'White Noise']

# Create a dropdown menu for noise types
dropdown = widgets.Dropdown(
    options=noise_options,
    value='White Noise',
    description='Noise Type:',
)

# Function to update the plot based on selected noise type
def update_plot_dropdown(noise_type):

    plt.figure(figsize=(12, 4))
    plt.subplot(2, 1, 1)
    
    if noise_type == 'Pink Noise':
        plt.plot(pink, label='Pink Noise')
    elif noise_type == 'Red Noise':
        plt.plot(red, label='Red Noise')
    elif noise_type == 'Blue Noise':
        plt.plot(blue, label='Blue Noise')
    elif noise_type == 'White Noise':
        plt.plot(x, label='White Noise')
    
    plt.title(f'{noise_type} Signal')
    plt.xlabel('Time Index')
    plt.ylabel('Amplitude')
    plt.legend()
    plt.grid()
    
    plt.subplot(2, 1, 2)
    if noise_type == 'Pink Noise':
        plt.psd(pink, NFFT=2048, Fs=1, color='r', label='Pink Noise')
    elif noise_type == 'Red Noise':
        plt.psd(red, NFFT=2048, Fs=1, color='r', label='Red Noise')
    elif noise_type == 'Blue Noise':
        plt.psd(blue, NFFT=2048, Fs=1, color='r', label='Blue Noise')
    elif noise_type == 'White Noise':
        plt.psd(x, NFFT=2048, Fs=1, color='r', label='White Noise')

    # plt.yscale('log')
    plt.show()

widgets.interactive(update_plot_dropdown, noise_type=dropdown)

Note

If you are interested, you can read more about the different types of noise in the Wikipedia article. In here you can also listen to the different types of noise, which might give you a better understanding of the differences.

Contents