NVIDIA RTX 2080 Ti: A Breakthrough for Faster, More Powerful Deep Learning
Introduction
Deep learning has revolutionized the field of artificial intelligence in recent years, enabling machines to match or even surpass humans on tasks like image classification, speech recognition, and language translation. These incredible breakthroughs are powered by two key ingredients:
- Large neural network models with millions or even billions of parameters
- Massive amounts of data to train those models on
However, there is a third critical piece that is often overlooked – the hardware required to efficiently train those huge models on enormous datasets. More complex neural architectures and ever-expanding data can quickly overwhelm the memory and compute capacity of traditional CPUs. That‘s where GPUs like NVIDIA‘s RTX 2080 Ti come in.
GPUs have proven to be the ideal hardware platform for deep learning‘s demanding, parallelizable workloads. Their many cores and high memory bandwidth allow data scientists to iterate on models and gain insights from data significantly faster than would be possible with CPUs alone.
The RTX 2080 Ti in particular was a pivotal release that turbocharged AI research and development when it launched in 2018. In this article, we‘ll take an in-depth look at what makes the 2080 Ti so powerful for deep learning, backed by real-world benchmarks and examples. We‘ll also examine how it stacks up to other GPUs and alternative hardware platforms, as well as where it fits into the larger story of AI‘s rapid progress.
2080 Ti Benchmarks: Quantifying the Speed-Up
First, let‘s dive into the numbers. Exactly how much faster is deep learning on a 2080 Ti compared to previous-generation GPUs? To find out, we can look at benchmarks on some of the most common deep learning models and frameworks.
One popular benchmark is to measure training time for ResNet-50, a convolutional neural network architecture used for image classification, on the ImageNet dataset. According to TensorFlow benchmarks, training ResNet-50 on a single 2080 Ti takes around 29 hours, compared to 46.5 hours on a 1080 Ti. That‘s a 1.6x speedup for this important computer vision use case.
We can also look at inference performance – how quickly a trained model can generate predictions on new data. NVIDIA‘s own deep learning examples repo shows that a 2080 Ti can perform ResNet-50 inference on 1517 images/second compared to 1205 images/second on a 1080 Ti, a 1.26x boost.
Speedups can be even more significant on larger, more complex models. MLPerf benchmark results show that training the popular BERT-Large natural language processing model on the SQuAD question answering dataset takes 15.4 hours on an 8x 2080 Ti system, down from 23.8 hours on 8x 1080 Ti – a 1.55x improvement. Similarly, training the Mask R-CNN object detection model is about 1.4x faster on 2080 Ti.
These efficiency gains aren‘t just theoretical, they have real impact for practitioners. Cutting training times in half means AI researchers and engineers can run twice as many experiments in the same amount of time. They can tune more hyperparameters, optimize model architectures faster, and productize AI solutions more rapidly. For companies deploying deep learning in the real world, like self-driving cars or virtual assistants, that faster time-to-market is a huge competitive advantage.
New Tensor Core Architecture
So what‘s the secret sauce that makes the 2080 Ti so much more potent for deep learning? A big part of the story is the new Turing architecture‘s Tensor Cores.
Tensor Cores are specialized processing units designed specifically for the kind of 4×4 matrix math that is the backbone of deep neural networks. Multiplying matrices is the core computational building block of both training (forward and backward propagation) and inference (generating predictions from a trained model). The more Tensor Core performance, the faster a GPU can crunch through these critical operations.
NVIDIA claims Tensor Cores can achieve up to 12x higher peak FLOPS (floating point operations per second) for mixed precision matrix multiplication vs standard precision CUDA cores. In essence, they act as rocket boosters strapped to the GPU, propelling matrix math to new heights.
The RTX 2080 Ti packs 544 Tensor Cores under the hood, a huge increase over the non-existent Tensor Cores in the preceding 1080 Ti. This allows data scientists to take advantage of powerful techniques like mixed precision training, using Tensor Cores for 16-bit operations while maintaining 32-bit accuracy when needed. The result is significantly faster training times without sacrificing model quality.
Tensor Cores were a real game-changer and their impact can be seen in the blistering speeds achieved by 2080 Ti‘s on deep learning benchmarks compared to GPUs without them. They are now a standard feature in subsequent generations of NVIDIA GPUs.
Enabling Breakthrough Research
Beyond the numbers, it‘s instructive to look at real-world examples of groundbreaking AI research made possible by the 2080 Ti‘s impressive performance.
One prominent example is OpenAI‘s GPT-2 language model, one of the most sophisticated and realistic text generation models at the time of its release in 2019. GPT-2 was trained on an enormous corpus of 40GB of text data, with the largest version containing 1.5 billion parameters. This training required a huge amount of compute power – OpenAI reports using 256 2080 Ti GPUs running for a week to train the model. Without the 2080 Ti‘s speedy Tensor Cores and 11GB of high-bandwidth memory, training a model of this size and complexity would have been considerably slower and more difficult.
The 2080 Ti also powered DeepMind‘s AlphaStar system, which made headlines in 2019 by defeating top human professionals at the complex real-time strategy game StarCraft II. AlphaStar relied on a deep reinforcement learning approach, using a massive amount of computing power to train agents through self-play. DeepMind doesn‘t provide exact details, but does mention using "a league of agents playing on thousands of GPUs" to train AlphaStar over the course of many months. It‘s likely that 2080 Ti‘s made up a significant portion of that GPU cluster given their leading performance at the time.
More recently, the 2018 Ti was utilized in DeepMind‘s AlphaFold 2 system, which made a major breakthrough by predicting protein structures with unprecedented accuracy. The team trained the AlphaFold model on a dataset of 170,000 protein structures using 128 2080 Ti GPUs. The model‘s success at the 2020 CASP14 protein folding competition was a historic milestone for computational biology with huge implications for drug discovery and disease treatment.
These are just a few examples, but they demonstrate how access to powerful compute like the 2080 Ti accelerates the pace of discovery and unlocks new possibilities for AI applications. As deep learning models continue to scale in size and researchers take on more ambitious challenges, robust and efficient training hardware will only become more important.
Why GPUs for Deep Learning?
The 2080 Ti is undoubtedly an impressive feat of engineering but some may wonder – why are GPUs like it so dominant for deep learning compared to other types of processors? What makes GPUs uniquely well-suited for workloads like training enormous neural networks?
The key is that deep learning is an inherently parallel computing problem. Training involves applying the same mathematical operations over and over again to large amounts of data (often images, video, audio, or text). This work can be split up and performed simultaneously across hundreds or thousands of GPU cores.
In contrast, CPUs are designed for general-purpose computing and emphasize strong serial processing performance. While a CPU may have 8 or 16 powerful cores, a GPU can have thousands of simpler cores that excel at doing many calculations in parallel. It‘s the difference between a Swiss Army knife and a specialized tool optimized for a specific task.
Additionally, deep learning workloads benefit from the very high memory bandwidth provided by modern GPUs. The 2080 Ti‘s 616 GB/s of bandwidth allows it to quickly shuttle data between its high-speed memory and processing cores. More bandwidth means less time spent waiting for data to be fetched, and more time spent doing useful computation. Many deep neural network architectures are memory-bound, so a memory bottleneck can be a major performance limiter.
Price to Performance
Of course, all this bleeding-edge technology comes at a cost. At a launch MSRP of $999 for the base model and $1199 for the Founders Edition, the 2080 Ti was one of the most expensive consumer GPUs ever released. For cost-conscious AI researchers and startups, that premium pricing can be hard to justify.
To put the cost in perspective, a single RTX 2080 Ti is more expensive than a full gaming PC with a previous-generation GTX 1080 Ti GPU. So the question becomes – is the 20-50% performance boost worth the steep price increase? The answer depends on your specific needs and budget constraints.
For large tech companies with huge AI research divisions, like Google, Facebook, Microsoft, and Amazon, the productivity gains from faster hardware vastly outweigh the higher price tags. These companies are in an arms race to develop the most sophisticated AI systems and are willing to invest heavily in infrastructure to gain a competitive edge. In their cost equations, the bigger expense is often researcher salaries and the opportunity cost of slower progress.
For self-funded researchers or smaller organizations, the calculation may be different. Cheaper GPUs like the RTX 2070 or GTX 1080 Ti may offer better bang for the buck, even if they can‘t match the raw performance of a 2080 Ti. Alternatively, renting 2080 Ti‘s in the cloud from services like AWS or Google Cloud can provide access to high-end hardware with less upfront cost, though at the expense of long-term savings.
A Stepping Stone to the Future
As powerful as the 2080 Ti is, the rapid pace of progress virtually guarantees it won‘t be the pinnacle of deep learning hardware for long. AI models continue to grow exponentially in size and complexity, demanding more and more compute resources to train and run.
NVIDIA‘s own 2020 A100 GPU already offers a glimpse of that future. With 6912 CUDA cores and 432 Tensor Cores (compared to the 2080 Ti‘s 4352 and 544 respectively), the A100 delivers over 5 petaFLOPS of AI performance, 20 times that of its predecessor. Systems like NVIDIA‘s DGX A100 can combine 8 A100 GPUs to provide a staggering 40 petaFLOPS of AI horsepower. That‘s enough to train BERT-Large in under an hour – a far cry from the 2080 Ti‘s 15 hours.
One could argue that the 2080 Ti will soon be obsolete in the face of such blistering advances. However, another perspective is that the 2080 Ti is more like a stepping stone – a critical bridge between the deep learning hardware of the past and the increasingly specialized AI accelerators of the future.
The 2080 Ti‘s Tensor Cores were an important proof of concept, demonstrating the significant performance gains that could be achieved by adding custom silicon tailored for deep learning. It set the stage for the proliferation of domain-specific architectures in subsequent GPU generations as well as an explosion of startups building AI chips from the ground up.
So while the 2080 Ti may not be the fastest kid on the block for much longer, its impact on the field of deep learning is hard to overstate. It powered a generation of groundbreaking AI research and paved the way for a future in which specialized AI hardware is the norm rather than the exception.
Conclusion
The 2018 Ti was a remarkable feat of engineering that pushed the boundaries of what was possible with deep learning. Its potent combination of CUDA cores, Tensor Cores, and high-bandwidth memory delivered dramatic speedups on the most important deep learning benchmarks and enabled researchers to tackle ever-larger and more complex AI challenges.
But the 2080 Ti‘s significance goes beyond just its raw performance. It represented an inflection point in the deep learning hardware landscape – a moment when AI-specific accelerators transitioned from a nice-to-have to a must-have. It crystallized the need for specialized hardware to keep pace with the relentless growth of deep learning models and datasets.
In the years since the 2080 Ti‘s release, we‘ve seen an explosion of innovation in AI chips, both from incumbents like NVIDIA and Intel, and from a slew of startups racing to build the next generation of deep learning accelerators. The 2080 Ti provided a tantalizing glimpse of what was possible, and now the race is on to push those capabilities to their limits.
Nonetheless, it‘s important to remember that hardware is only one part of the equation. Progress in AI is driven by a virtuous cycle of algorithmic advances, growth in training data, and leaps in compute power. Continued research into more efficient neural network architectures, unsupervised and self-supervised learning, and techniques like transfer learning and meta-learning may help offset the need for raw compute. Innovations in software frameworks and libraries are making it easier than ever to parallelize and distribute workloads across whatever hardware is available.
Faster, more efficient hardware like the 2080 Ti is undoubtedly a key part of the recipe for advancing AI. But it‘s the complex interplay of hardware and software, algorithms and data, research and engineering that truly propels the field forward. As we look to the future, one thing is clear – deep learning is one of the most exciting and impactful technological frontiers of our time. We can‘t wait to see what breakthrough results await in the years ahead, as today‘s most powerful hardware is eclipsed by the next generation of deep learning accelerators.
