# How to Build a Gradio App to Detect Sexist Audio Content

- Canonical: https://33rdsquare.com/gradio-app-for-detecting-whether-the-audio-content-is-sexist/
- Published: 2024-09-03
- Author: Jordan Brown
- Categories: [Artificial Intelligence & Machine Learning & ChatGPT](https://33rdsquare.com/category/tech/ai/)

---

Sexism refers to prejudice, stereotyping, or discrimination based on a person‘s sex or gender. It is a pervasive issue in society that can manifest in many ways, including through language. Sexist language reinforces harmful gender roles and power imbalances. It is often used to demean, objectify, or exclude people, especially women and girls.

Sexist language can be overt, like using gendered slurs, or more subtle, like making assumptions about someone‘s abilities based on their gender. It occurs in all areas of life, from casual conversations to professional settings to public spaces. Even if unintentional, sexist language causes real harm by perpetuating gender-based prejudice and inequity.

With the rise of virtual assistants, smart speakers, and audio-based social media, there is a growing need to be able to automatically detect sexist content in audio format. Building AI systems to flag potentially sexist audio can help content moderators quickly identify and review problematic material at scale. It can also be used to give real-time feedback to users to prevent them from unwittingly sharing sexist content.

In this tutorial, we‘ll walk through how to create an app that can transcribe audio in Spanish and classify whether the audio contains sexist language. We‘ll use Hugging Face‘s Gradio library to build the app interface and host it on Hugging Face Spaces. Let‘s get started!

## Overview of Key Components

To build our sexist audio detection app, we‘ll need four key components:

1. An automatic speech recognition (ASR) model to transcribe Spanish audio into text
2. A sexism detection model to classify the transcribed text as sexist or not
3. Gradio to create the user interface for the app
4. Hugging Face Spaces to host and share the app

For the ASR component, we‘ll use a pre-trained wav2vec2 model fine-tuned on Spanish audio data. Wav2vec2 is a state-of-the-art speech recognition model developed by Facebook AI. It learns powerful audio representations in a self-supervised way and can be fine-tuned for specific languages with limited labeled data.

There are several open source wav2vec2 models available that have been fine-tuned for Spanish ASR. For this app, we‘ll use the "jonatasgrosman/wav2vec2-xls-r-1b-spanish" model hosted on the Hugging Face model hub. This model achieves a word error rate of just 7.13% on the Spanish Common Voice test set.

To handle the sexism detection, we‘ll use a RoBERTuito model that has been fine-tuned for sexism detection on Spanish social media data. RoBERTuito is a Spanish language model based on the popular RoBERTa architecture. The specific model we‘ll use is "hackathon-pln-es/twitter_sexismo-finetuned-robertuito-exist2021", which has been fine-tuned on the EXIST corpus of sexist tweets.

With these two models, our app pipeline will work as follows:

1. The user records or uploads Spanish audio
2. The wav2vec2 ASR model transcribes the audio into text
3. The text is passed to the RoBERTuito sexism classifier
4. The model‘s predicted label (sexist or not sexist) is returned

To build the user-facing part of our app, we‘ll use Gradio. Gradio is an open source Python library that makes it very easy to create web interfaces for machine learning models. With just a few lines of code, we can create an interactive UI that allows users to record audio samples and view the model outputs.

Finally, we‘ll deploy our finished Gradio app on Hugging Face Spaces. Spaces is a free hosting platform for machine learning apps. It provides an easy way to share Gradio apps with the world without needing to set up our own web hosting.

Now that we understand the key pieces, let‘s walk through the step-by-step process of putting it all together.

## Step 1: Set Up a Hugging Face Account and Repository

The first thing we need to do is create a Hugging Face account if you don‘t already have one. Go to [https://huggingface.co/](https://huggingface.co/) and click "Sign Up" in the top right corner. Once you‘re logged in, head over to the Spaces page: [https://huggingface.co/spaces](https://huggingface.co/spaces)

Click on "Create New Space" and select Gradio as the SDK. Give your Space a name and then hit "Create Space". This will create a new repository where we‘ll store all the code for our app.

## Step 2: Install Dependencies

Next up, we need to install all the required Python libraries that our app will use. Within the Space repo, create a new file called `requirements.txt` and add the following lines:

```
gradio
transformers
torch
librosa
```

This will install Gradio, the HuggingFace Transformers library for our ASR and sexism models, PyTorch, and Librosa for audio processing. Whenever your Space is run, these dependencies will automatically be installed in the virtual environment.

## Step 3: Write the App Code

Now we‘re ready to code our app! In the Space repo, create a file called `app.py`. This is where we‘ll load our models, define the prediction pipeline, and set up the Gradio interface.

First, let‘s import all the libraries we‘ll need:

```
import gradio as gr
import librosa
import torch
from transformers import AutoModelForCTC, AutoTokenizer, AutoModelForSequenceClassification, pipeline
```

We‘ll load our pre-trained models using HuggingFace‘s `pipeline` function. For speech recognition, we‘ll use the "jonatasgrosman/wav2vec2-xls-r-1b-spanish" model and tokenizer:

```
asr_model_id = "jonatasgrosman/wav2vec2-xls-r-1b-spanish"
asr_model = AutoModelForCTC.from_pretrained(asr_model_id)
asr_tokenizer = AutoTokenizer.from_pretrained(asr_model_id)

asr_pipeline = pipeline("automatic-speech-recognition", model=asr_model, tokenizer=asr_tokenizer)
```

And for sexism detection, we‘ll load the "hackathon-pln-es/twitter_sexismo-finetuned-robertuito-exist2021" model:

```
sexism_model_id = "hackathon-pln-es/twitter_sexismo-finetuned-robertuito-exist2021"
sexism_model = AutoModelForSequenceClassification.from_pretrained(sexism_model_id)
sexism_tokenizer = AutoTokenizer.from_pretrained(sexism_model_id)

sexism_pipeline = pipeline("text-classification", model=sexism_model, tokenizer=sexism_tokenizer)
```

Next, we need to define a function that will take in an audio file, transcribe it, and classify the transcription. We‘ll call this function `transcribe_and_classify`:

```
def transcribe_and_classify(audio_file):
  # Load and resample audio to 16kHz mono
  speech, sample_rate = librosa.load(audio_file, sr=16000, mono=True)

  # Transcribe audio to text with ASR model
  transcription = asr_pipeline(speech)["text"]

  # Classify text as sexist or not with sexism model
  sexism_result = sexism_pipeline(transcription)

  return transcription, sexism_result[0]["label"]
```

This function does the following:

1. Loads in the audio file and resamples it to 16kHz mono, as required by the wav2vec2 model
2. Passes the audio to the ASR pipeline to get the Spanish transcription
3. Passes the transcribed text to the sexism detection pipeline
4. Returns the transcription and predicted label ("LABEL_0" for not sexist, "LABEL_1" for sexist)

Finally, we‘ll create our Gradio interface and launch the app:

```
# Define interface components
audio_input = gr.Audio(source="microphone", type="filepath")
text_output = gr.Textbox(label="Transcripción")
label_output = gr.Textbox(label="Clasificación")

# Define interface layout
interface = gr.Interface(
    fn=transcribe_and_classify,
    inputs=audio_input,
    outputs=[text_output, label_output],
    title="Detección de Sexismo en Español",
    description="Grabe o suba un clip de audio en español para obtener la transcripción y averiguar si contiene lenguaje sexista.",
)

interface.launch()
```

This code sets up our Gradio interface with an audio input component that allows recording or uploading files, and two text output components for displaying the transcription and sexism label.

We pass our `transcribe_and_classify` function to `gr.Interface`, specifying the input and output components. The `title` and `description` parameters let us give the app a header and subheading describing what it does.

Finally, we call `interface.launch()` to start the app. That‘s it! The complete code in `app.py` should look like this:

```
import gradio as gr
import librosa
import torch
from transformers import AutoModelForCTC, AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load ASR model and tokenizer
asr_model_id = "jonatasgrosman/wav2vec2-xls-r-1b-spanish"
asr_model = AutoModelForCTC.from_pretrained(asr_model_id)
asr_tokenizer = AutoTokenizer.from_pretrained(asr_model_id)
asr_pipeline = pipeline("automatic-speech-recognition", model=asr_model, tokenizer=asr_tokenizer)

# Load sexism detection model and tokenizer
sexism_model_id = "hackathon-pln-es/twitter_sexismo-finetuned-robertuito-exist2021"
sexism_model = AutoModelForSequenceClassification.from_pretrained(sexism_model_id)
sexism_tokenizer = AutoTokenizer.from_pretrained(sexism_model_id)
sexism_pipeline = pipeline("text-classification", model=sexism_model, tokenizer=sexism_tokenizer)

def transcribe_and_classify(audio_file):
  # Load and resample audio to 16kHz mono
  speech, sample_rate = librosa.load(audio_file, sr=16000, mono=True)

  # Transcribe audio to text with ASR model
  transcription = asr_pipeline(speech)["text"]

  # Classify text as sexist or not with sexism model
  sexism_result = sexism_pipeline(transcription)

  return transcription, sexism_result[0]["label"]

# Define interface components
audio_input = gr.Audio(source="microphone", type="filepath")
text_output = gr.Textbox(label="Transcripción")
label_output = gr.Textbox(label="Clasificación")

# Define interface layout
interface = gr.Interface(
    fn=transcribe_and_classify,
    inputs=audio_input,
    outputs=[text_output, label_output],
    title="Detección de Sexismo en Español",
    description="Grabe o suba un clip de audio en español para obtener la transcripción y averiguar si contiene lenguaje sexista.",
)

interface.launch()
```

## Step 4: Debug and Test the App

With our code complete, it‘s time to test it out! In your Space repo, click on "App" in the left sidebar which should open the Gradio interface.

If all goes well, you should see the app title, description, and an audio input component. Try recording a short audio clip in Spanish. After a few seconds, the transcribed text should appear, along with a label indicating whether the input was classified as sexist or not.

If you run into any issues, check the "Logs" tab to debug. Some common issues:

- Missing dependencies: Double check that all the required libraries are listed in `requirements.txt`
- CUDA out of memory errors: If you see a CUDA error, try setting `device="cpu"` when loading the models to run them on CPU instead of GPU.
- Incorrect model or tokenizer IDs: Make sure the model and tokenizer names in the code match the ones on the Hugging Face model hub.

Keep testing the app with different audio inputs to get a sense of its performance. You can also include some pre-recorded audio files in the repo to use as examples.

## Step 5: Share Your App on Hugging Face Spaces

Once you‘re happy with how the app is working, it‘s time to share it with the world! Go to the "Settings" tab in your Space repo and make sure the "Visibility" is set to "Public".

You can customize the appearance of your Space page by adding a logo, editing the README, and providing example audio files to showcase your app. The README is a great place to explain what your app does, provide usage instructions, and link to any relevant resources.

When you‘re ready, copy the URL of your Space page and share it out. Anyone with the link will be able to access and use your app without needing to install anything. Congrats, you‘ve just deployed your first Gradio app!

## Limitations and Future Directions

While our sexist audio detection app works reasonably well, there are some important limitations to keep in mind:

- ASR model performance: The accuracy of sexism detection is fundamentally limited by the accuracy of the ASR model. If the model mistranscribes words, it can lead to false positives or negatives in the subsequent classification.
- Domain mismatch: Both the ASR and sexism models were trained on specific datasets that may not fully represent the distribution of accents, background noise, topics, etc. in real-world audio. Performance may degrade on out-of-domain data.
- Binary labels: Sexism exists on a spectrum, but our model only outputs a binary label. It doesn‘t capture different types or severities of sexist language.
- Language limitations: Our app only works for Spanish audio. Expanding to other languages would require finding or fine-tuning ASR and sexism models in those languages.

There are many potential ways to improve and extend this app:

- Ensemble multiple ASR and sexism models to improve robustness and catch more edge cases
- Provide more granular labels (e.g. "potentially sexist", "highly sexist") or identify specific sexist phrases
- Return timestamps in the audio where sexist language was detected to help moderators quickly identify and review those segments
- Extend the app to other languages or language variants
- Allow users to flag errors and use that feedback to improve the models over time

Despite its limitations, audio-based sexism detection has many powerful applications. It can be used to automatically moderate audio content on social platforms at scale. Integrating it into AI assistants could help identify and confront users‘ biases in real-time. Teachers could use it to get feedback on unintentionally gendered language in the classroom.

As we develop more sophisticated AI models for understanding speech, it‘s important that we use them not just to transcribe audio, but to critically analyze the content of what is being said. By making it easier to identify sexist language in audio format, I hope this app can be one small step towards making our conversations more inclusive.

Thanks for following along with this tutorial! The complete code for this app is available at [link to Hugging Face Space]. If you have any questions or suggestions for improvement, please leave a comment below.

You can also connect with me on Twitter [@username] or GitHub [@username] to see my latest projects. Happy coding!

---

Source: [How to Build a Gradio App to Detect Sexist Audio Content](https://33rdsquare.com/gradio-app-for-detecting-whether-the-audio-content-is-sexist/)