*Esa Vilkama, President and Founder at Process Data Insights, LLC*

**Introduction**

In signal processing, time–frequency analysis comprises those techniques that study a signal in both the time and frequency domains simultaneously, using various time–frequency representations. The motivation for this analysis is that variables and their transform representation are often tightly connected, and they can be understood better by studying them jointly, as a two-dimensional object, rather than separately.

Frequency domain data analysis, like the Fast Fourier Transform (FTT), is a powerful tool in many applications. It can reveal periodic variation in time series data indicating e.g. mechanical problems in a manufacturing process. One drawback is that it does not work well for a time series in which the frequencies vary over time. The frequency spectrum does not show the time instant when the frequency changed. For that we need time-frequency analysis methods and representations. This blog covers some basic aspects in time-frequency analysis.

tftb (Time-frequency toolbox) is a Python module for time-frequency analysis and visualization built with SciPy and matplotlib.

Time-Frequency Toolbox for use with MATLAB.

**Spectrogram**

A spectrogram is a visual representation of the time-dependent spectral estimate of frequencies and strength (magnitude) of a nonstationary signal. Spectrograms are commonly used to display frequencies of sound waves produced by human speech, musical instruments, sonar, machinery, animals, etc., as recorded by microphones or other types of receivers. The vibration measured by seismometers can help distinguish and characterize different types of earthquakes. Spectrograms can be used to display various signals from a manufacturing process.

A spectrogram can be generated e.g. by an optical spectrometer, a bank of band-pass filters, Fourier transform, STFT or wavelet transform (in which case it is also known as **a scaleogram or scalogram**)**. **A spectrogram is usually depicted as a **heat map**, i.e., as an image with the intensity shown by varying the color or brightness, as shown in this example (the spectrogram of the of the spoken words “nineteenth century”).

Frequencies are shown increasing up the vertical axis, and time on the horizontal axis. Color indicates the signal strength.

In the example below is shown a signal (a signal duration of 20s sampled at 400Hz) with three frequency time periods (frequency changes at 7.5sec and 12.5sec). We can see how the FFT spectrum does not give any indication when these frequencies were present in the signal, whereas the spectrogram shows the timing for the frequencies.

One of the most basic forms of time–frequency analysis is the short-time Fourier transform (STFT). The STFT divides a longer time signal into shorter segments of equal length (which may overlap) and then computes the Fourier transform (FT) separately for each shorter segment. The STFTs can be used as a way of quantifying the change of a nonstationary signal’s frequency and phase content over time. It provides the time-localized frequency information, whereas the FT provides the frequency information averaged over the entire signal time interval. Below are simplified diagrams showing differences between the time-series, the FT and STFT.

Below is a signal, a sine wave whose frequency is slowly modulated around 3kHz, corrupted by white noise of exponentially decreasing magnitude sampled at 10 kHz, and its FFT spectrum and STFT.

The **Gabor transform** is a special case of the short-time Fourier transform. It is used to determine the sinusoidal frequency and phase content of local sections of a signal as it changes over time. The function to be transformed is first multiplied by a Gaussian function, which can be regarded as a window function, and the resulting function is then transformed with a Fourier transform to derive the time-frequency analysis.

Python libraries: scipy.signal.stft, stft for numpy

**Wavelet Transforms**

One of the drawbacks of the STFT is that it has a fixed resolution based on the selected time-segment width that determines whether there is good frequency resolution (frequency components close together can be separated) or good time resolution (the time at which frequencies change). A wide window gives better frequency resolution but poor time resolution. A narrower window gives good time resolution but poor frequency resolution as seen below.

A better approach for analyzing signals with a dynamical frequency spectrum is the Wavelet Transform (WT) that has a high resolution in both the frequency- and the time-domain. It shows which frequencies are present in a signal, and also at which time these frequencies have occurred.

The WT uses a series of functions called wavelets, each with a different scale. A wavelet is a wave-like oscillation with an amplitude that begins at zero, increases, and then decreases back to zero. It can typically be visualized as a “brief oscillation”. It is localized in time to obtain time-information in addition to frequency information whereas the FT uses a series of sine waves with different frequencies to analyze a signal. The sine wave is not localized in time and is infinitely long. Below are shown a sine wave, and examples of a (ricker) wavelet at different scales. There are many different types of wavelets.

In the picture below we can see how frequency resolution is decreasing for increasing frequencies (the more compressed wavelet) while the temporal resolution increases. This shows that the WT is good in time resolution of high frequencies, while for slowly varying functions, the frequency resolution is good. Based on the uncertainty principle of signal processing, the higher the resolution is in time, the lower the resolution in frequency has to be.

**Continuous Wavelet Transform (CWT) and Discrete Wavelet Transform (DWT)**

Wavelet transforms are classified into discrete wavelet transforms (DWTs) and continuous wavelet transforms (CWTs). The CWT and DWT differ in how they discretize the scale and transition (position) factor. The CWT typically uses exponential scales with a base smaller than 2, for example 21/12. DWT uses discrete values for the scale and translation factor. The scale factor increases in powers of two, so a = 1, 2, 4, .. and the translation factor increases integer values ( b = 1, 2, 3 .. ). These differences result in advantages and disadvantages for the two classes of wavelet transforms, and also determine use cases where one wavelet transform is likely to provide better results.

The finer sampling of scales in the CWT typically results in a higher-resolution signal analysis that is beneficial in localizing transients in signals. This enables many important manufacturing applications, e.g. automated detection of abrupt subtle disturbances and anomalies in process variables. The DWT is suitable e.g. for image compression and signal transmission. The DWT coefficients can be used to synthesize an exact reproduction of the signal to within numerical precision.

An example of CWT:

Python libraries: PyWavelets, scipy.signal.cwt