Visualizing Sounds with the Librosa Python Library

Introduction

Audio is all around us – in the music we listen to, the videos we watch, the podcasts and audiobooks we play, even in the background noise of our everyday lives. While we instinctively process audio signals with our ears, computers require audio to be represented and analyzed digitally. This is where librosa comes in.

Librosa is a popular Python library for music and audio analysis. It provides a wide range of tools for loading, processing, analyzing, and visualizing audio data. With librosa, you can easily extract meaningful information and insights from audio files using signal processing and machine learning techniques.

One of the most powerful aspects of librosa is its ability to generate visual representations of audio data. Visualizing sound allows us to gain a deeper understanding of its structure and characteristics that may not be perceivable by ear alone. By plotting an audio signal in different ways, we can identify patterns, anomalies, similarities and more.

In this blog post, we‘ll take a deep dive into visualizing sounds with librosa. We‘ll cover everything you need to get started, from installing the library and loading audio files, to generating basic plots, extracting audio features, applying filters and effects, and exploring real-world use cases and applications. Whether you‘re an audio processing newbie or a seasoned data scientist, by the end of this post, you‘ll have the knowledge and code samples to start creating your own audio visualizations with Python and librosa. Let‘s jump in!

Getting Started

Before we start visualizing audio, we need to make sure we have librosa installed and we can load an audio file to work with. Librosa can be installed using pip:

pip install librosa

You‘ll also need to install some dependencies like NumPy, SciPy, matplotlib, and soundfile. Once installed, we can load an audio file using librosa‘s load() function:

import librosa

audio_file = "song.wav"
y, sr = librosa.load(audio_file)

This loads the audio time series data as a NumPy array y and the sampling rate sr. We can actually play back this audio data directly in a Jupyter notebook using Audio():

import IPython.display as ipd

ipd.Audio(y, rate=sr)

Basic Visualizations

Now that we have an audio signal loaded, let‘s plot it! The most basic plot is the waveform, which shows the amplitude of the signal over time:

import matplotlib.pyplot as plt

plt.figure(figsize=(14, 5))
librosa.display.waveshow(y, sr=sr)
plt.title(‘Audio Waveform‘)
plt.show()

We can also generate a spectrogram, which is a visual representation of the frequency spectrum of the signal as it varies over time. Librosa‘s stft() function computes the short-time Fourier transform, which we then plot using specshow():

spectrogram = librosa.stft(y) 
S_db = librosa.amplitude_to_db(abs(spectrogram))

plt.figure(figsize=(14, 5))
librosa.display.specshow(S_db, sr=sr, x_axis=‘time‘, y_axis=‘hz‘)
plt.colorbar()
plt.title(‘Spectrogram‘)
plt.show()

For better visualization, we can plot the spectrogram on a log-scaled frequency axis using a Mel-scaled spectrogram:

mel_spectrogram = librosa.feature.melspectrogram(y=y, sr=sr)  

plt.figure(figsize=(14, 5))
librosa.display.specshow(librosa.power_to_db(mel_spectrogram, ref=np.max), 
                         y_axis=‘mel‘, x_axis=‘time‘)
plt.colorbar(format=‘%+2.0f dB‘)
plt.title(‘Mel-scaled Spectrogram‘)
plt.show()

Feature Extraction

Beyond plotting the raw audio signal, the real power of librosa lies in its ability to extract meaningful features and characteristics from audio data. Let‘s look at a few key features:

The mel-frequency cepstral coefficients (MFCCs) are a compact representation of the frequency spectrum. They are widely used in speech recognition and music information retrieval:

mfccs = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=40)

plt.figure(figsize=(14, 5))
librosa.display.specshow(mfccs, x_axis=‘time‘)
plt.colorbar(format=‘%+2.0f dB‘)
plt.title(‘MFCC‘)
plt.show()

Chromagrams represent the tonal content of music in a condensed form. They project the entire spectrum onto 12 bins representing the 12 distinct semitones (or chroma) of the musical octave:

chroma = librosa.feature.chroma_stft(y=y, sr=sr) 

plt.figure(figsize=(14, 5)) 
librosa.display.specshow(chroma, y_axis=‘chroma‘, x_axis=‘time‘)
plt.colorbar()
plt.title(‘Chromagram‘)
plt.show()

Other useful features that can be extracted and visualized with librosa include:

• Spectral centroid
• Spectral bandwidth
• Spectral contrast
• Tonnetz
• Zero-crossing rate
• Tempo and beat information

Visualizing Features

When visualizing extracted audio features, there are a few techniques we can use to make the plots more interpretable and insightful.

Choosing perceptually-relevant colormaps, like those based on color temperature or diverging schemes, can make differences in feature values stand out. Librosa has many colormap options:

librosa.display.specshow(mfccs, x_axis=‘time‘, cmap=‘magma‘)

Properly scaling the axes, color ranges and plot dimensions helps maximize readability. Matplotlib‘s imshow() accepts extent and axis scaling arguments:

extent = [0, duration, 0, sr/2]
plt.imshow(spectrogram, cmap=‘viridis‘, aspect=‘auto‘, extent=extent) 
plt.yscale(‘log‘)

We can combine multiple feature visualizations into a single plot to compare and contrast them. Here we display the waveform, log-scaled spectrogram and MFCCs together:

fig, axs = plt.subplots(3, 1, figsize=(10, 10), sharex=True)
librosa.display.waveshow(y, sr=sr, ax=axs[0])
axs[0].set(title=‘Audio Waveform‘)
axs[0].label_outer()

D = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=np.max)
img = librosa.display.specshow(D, y_axis=‘linear‘, x_axis=‘time‘, sr=sr, ax=axs[1])
axs[1].set(title=‘Linear-frequency Spectrogram‘)
axs[1].label_outer()

librosa.display.specshow(mfccs, x_axis=‘time‘, ax=axs[2])
axs[2].set(title=‘MFCC‘)

fig.colorbar(img, ax=axs, format="%+2.f dB")

Interactive plot toolkits like Bokeh and Plotly allow zooming, panning, and hover information for exploring visualizations in more depth. Librosa‘s display module has functions for Bokeh figure conversion:

import librosa.display as disp

mel = librosa.feature.melspectrogram(y=y)
fig = disp.specshow(mel, y_axis=‘mel‘, x_axis=‘s‘, sr=sr)
disp.bokeh_figure(fig, width=800, height=400)

Audio Effects & Filters

In addition to feature extraction and visualization, librosa provides ways to manipulate audio with effects, filtering, and other digital signal processing techniques.

We can apply effects like reverberation, pitch shifting, time stretching, and harmonic-percussive source separation to alter the sound:

# Apply reverb
y_reverb = librosa.effects.reverb(y)

# Pitch shift  
y_pitch = librosa.effects.pitch_shift(y, sr, n_steps=4)

# Time stretch
y_stretch = librosa.effects.time_stretch(y, 1.5) 

Custom filters can be designed and applied to emphasize or attenuate certain frequencies. Here‘s an example of a lowpass filter:

import scipy

# Create lowpass filter
b, a = scipy.signal.butter(4, 100, ‘low‘, fs=sr) 

# Apply to audio
y_filt = scipy.signal.filtfilt(b, a, y)

Use Cases & Applications

Audio visualization techniques with librosa have a wide range of applications across different domains:

• Music Information Retrieval (MIR): Classify songs by genre, mood, instrumentation, etc. based on extracted features. Identify remixes, covers and samples. Detect key, chords, tempo, beats and structure.

• Speech Recognition: Visualize phonemes, formants and prosody for speech analysis. Enhance speech clarity and intelligibility. Identify speakers and sentiment from vocal characteristics.

• Environmental Sound Analysis: Monitor and detect acoustic events like urban noise, wildlife calls, machinery sounds. Measure sound intensity levels over time and frequency.

• Audio Editing & Production: Inspect and manipulate audio attributes visually. Apply effects and filters. Identify and remove noise, hum, clipping and other unwanted artifacts.

• Data Sonification: Represent data and patterns through sound. Create audio displays and alerts. Enhance accessibility of visual information.

Tips & Best Practices

To make the most of audio visualization with librosa, keep these tips and best practices in mind:

• Always listen to the audio while visually analyzing it. Don‘t rely on plots alone to draw conclusions.
• Choose appropriate plot types, colormaps and scaling to effectively convey relevant information. Experiment with different parameters.
• Start with shorter clips and segments before visualizing long files to speed up iteration. Use resampling and duration checks if needed.
• Be aware of inherent tradeoffs in time vs. frequency resolution for spectrograms and other representations.
• Apply pre-processing like DC removal, amplitude normalization and filtering when appropriate. But be careful not to destroy useful information.
• Pay attention to axes labels, legends and titles. Clearly communicate what is being visualized and how to interpret it.
• Use interactive and multi-panel plots to add depth and reveal additional insights.
• Consider your target audience and application context when designing visualizations. Adapt information density and complexity accordingly.
• Refer to librosa‘s documentation and examples for usage details and ideas. Follow the latest updates and join the community.

Conclusion

We‘ve covered a lot of ground in this deep dive on visualizing sounds with librosa. From simple waveforms to multi-panel displays of advanced features, the range of visual representations that can be created from audio data is truly amazing.

With an understanding of the core functions and techniques in librosa, you‘re well-equipped to start exploring audio visualization for your own projects and applications. The key is to experiment, iterate, and always let your eyes and ears guide you.

While we focused on a single library here, audio visualization is a vast and vibrant field with many other great tools and resources available. I encourage you to explore additional Python libraries like pyAudioAnalysis, pydub, and FMP, as well as awesome open-source audio software like Sonic Visualizer.

No matter what your goals are – whether it‘s analyzing your favorite songs, building an automatic transcription tool, or detecting hidden patterns in urban soundscapes – audio visualization with librosa is a powerful way to gain insights and create compelling experiences with sound. So dive in, have fun, and happy visualizing!