The Ultimate Guide to Progress Bars in Python with TQDM (2026 Update)

If you‘ve ever run a long-running task in Python like processing a large dataset or performing complex computations, you know how frustrating it can be to stare at a blinking cursor wondering if any progress is being made. This is where progress bars come to the rescue by providing a visual indicator of how much work has been completed and how much remains.

In this guide, we‘ll dive deep into progress bars in Python with a focus on the excellent TQDM library. You‘ll learn what progress bars are, why they are useful, and get step-by-step instructions on implementing them in your own Python code. We‘ll cover everything from basic usage to advanced customization and explore best practices to keep in mind. Let‘s get started!

What are Progress Bars and Why Use Them?

A progress bar is a graphical control element used to visualize the progression of an extended computer operation, such as a download, file transfer, or installation. They provide at-a-glance information about how much of the task has been completed, how much remains, and potentially an estimated time to completion.

Imagine you have a script that needs to process 10,000 files. Without any feedback, the user has no idea if it will take 10 seconds or 10 hours! This is where a progress bar comes in. By periodically updating a progress bar during the task, the user gets that much-needed feedback and peace of mind. Some key benefits:

• Provides a visual indicator of progress to the user
• Gives an estimate of time remaining
• Makes long-running tasks less frustrating
• Helps identify if a task is stuck or frozen

Python doesn‘t include a built-in progress bar in the standard library, but luckily the TQDM library provides an excellent implementation that is both easy to use and highly customizable.

Introducing TQDM

TQDM (pronounced "tack-dum") is a Python library that provides fast, extensible progress bars with minimal overhead. Fun fact: "tqdm" is Arabic for "progress"!

Some key features of TQDM include:

• Integrates easily with for loops and any iterable
• Customizable appearance and output
• Supports nested progress bars for complex tasks
• Hooks for Jupyter Notebooks and Pandas integration
• Incredibly lightweight with no dependencies

Getting started with TQDM is a breeze. First, install it using pip:

pip install tqdm

Then import it into your Python script or Jupyter Notebook:

from tqdm import tqdm

That‘s it! You‘re ready to add progress bars to your Python code. In the next section, we‘ll walk through some examples to get you started.

Using TQDM in Python

The easiest way to use TQDM is to wrap any iterable with the tqdm() function. TQDM will automatically keep track of iterations and display a smart progress bar. Here‘s a basic example:

from tqdm import tqdm
import time

for i in tqdm(range(100)):
    time.sleep(0.1)

This will display a progress bar that fills up as the loop progresses:

76%|████████████████████████████         | 76/100 [00:07<00:02,  9.50it/s]

The bar shows:

• The current progress as a percentage
• How many iterations have been completed
• An estimate of time remaining
• The current iteration speed

You can easily use TQDM with any iterable, like lists:

items = [‘foo‘, ‘bar‘, ‘baz‘]
for item in tqdm(items):
    process(item)

Or even in list comprehensions:

results = [process(item) for item in tqdm(items)]

The real power of TQDM comes through its many customization options. For example, you can:

• Add a custom description message
• Change the unit from iterations to something more meaningful
• Modify the styling of the progress bar
• Output to a log file instead of standard output
• And much more!

Here are a few examples to give you a taste:

from tqdm import tqdm

# Add a description
for i in tqdm(range(100), desc=‘Processing‘):
    pass

# Use a different unit
size = 1024 * 1024 * 10  # 10 MB
with tqdm(total=size, unit=‘B‘, unit_scale=True, desc=‘Downloading‘) as pbar:
    with open(‘file.txt‘, ‘wb‘) as f:
        while True:
            chunk = download_chunk()
            f.write(chunk)
            pbar.update(len(chunk))

# Style the progress bar
bar_format = ‘{desc}{percentage:3.0f}%|{bar}|{n_fmt}/{total_fmt} [{elapsed}<{remaining}]‘
for i in tqdm(range(100), bar_format=bar_format, desc=‘Custom‘):
    pass

Consult the TQDM documentation for a full list of available options and examples.

TQDM in Jupyter Notebooks

TQDM works great in regular Python scripts, but what about Jupyter Notebooks? Luckily, TQDM provides a special tqdm_notebook function that integrates seamlessly with Jupyter‘s HTML output. Simply import and use it like normal:

from tqdm.notebook import tqdm_notebook
import time

for i in tqdm_notebook(range(100)):
    time.sleep(0.1)  

This will display an animated, full-color progress bar right in your notebook. The notebook version supports all the same options and customizations as the regular TQDM.

Tracking Pandas Progress with TQDM

If you work with Pandas dataframes, you know that operations like apply can take a long time on large datasets. Wouldn‘t it be nice to have insight into the progress of these operations? With TQDM, you can!

First, enable TQDM progress bars globally for Pandas:

import pandas as pd
from tqdm.notebook import tqdm_notebook

tqdm_notebook().pandas()

Then use the progress_apply function in place of the regular apply:

df = pd.read_csv(‘large_file.csv‘)
df[‘new_col‘] = df[‘old_col‘].progress_apply(slow_function)

TQDM will display a progress bar showing the completion percentage and estimated time remaining for the apply operation. This is incredibly handy for long-running Pandas tasks.

Best Practices for Using TQDM

While TQDM is very easy to use, there are a few best practices to keep in mind:

• Use tqdm_notebook in Jupyter and tqdm everywhere else. The regular tqdm may not display correctly in notebooks.
• Wrap the highest level iterable for the cleanest looking output. For deeply nested loops, you may want to only wrap the outer loop to avoid clutter.
• When wrapping code that writes a lot of output, use the position parameter to ensure the progress bar stays at the bottom.
• If you have a slow function that is called within a loop, consider refactoring it to return a generator and wrap that with tqdm instead.
• Be mindful of the overhead of progress bars on very fast iterations. TQDM is pretty lightweight, but if you‘re iterating over millions of items extremely quickly, the extra computation can add up. Consider using the mininterval parameter in these cases.

Comparing TQDM to Other Libraries

While TQDM is the most popular progress bar library in Python, it‘s not the only one. Other notable libraries include:

• progressbar2: A fork of the original progressbar library with Python 3 support and a few extra features. Comparable in features to TQDM.
• alive-progress: A newer entry that aims to be more lightweight and Pythonic than TQDM. Supports some unique spinners and styles.

In general, most Python developers tend to gravitate towards TQDM for its wide adoption, extensive documentation, and frequent updates. However, the other libraries may be worth checking out if you have specific needs.

What‘s New in TQDM for 2024?

The TQDM developers have continued to improve the library since its initial release. Some notable recent additions include:

• Support for rich text and emoji in progress bars via the colour and ascii parameters
• New contrib submodule with plugins for libraries like Keras, Telegram, and GUI toolkits
• Performance improvements, especially on Windows and for heavily nested progress bars
• Experimental synchronization support for multi-threaded use cases

Be sure to check the TQDM changelog for a full list of improvements in the latest versions.

Get Started with TQDM Today!

We‘ve covered a lot of ground in this guide, from the basics of progress bars to advanced usage of TQDM in Jupyter notebooks and Pandas. You should now have a solid grasp of how to use TQDM to add progress reporting to your own Python code.

Remember, a little bit of progress feedback goes a long way in creating a positive user experience, whether your users are data scientists, developers, or end-users. Try adding TQDM to your next Python project and see for yourself!

To learn more, check out these helpful resources:

• Official TQDM documentation
• TQDM GitHub repository
• TQDM vs other progress bars

Happy coding!