Teradata‘s Past, Present and Future: Insights from 20-Year Veteran Daniel Graham

Since its founding over 40 years ago, Teradata has been synonymous with massively parallel processing (MPP) data warehouses that power analytics at the world‘s largest companies. From retailers and banks to telcos and travel companies, Teradata‘s enterprise customers rely on its high-performance SQL engines to quickly generate insights from massive volumes of data.

I recently had the chance to speak with Daniel Graham, General Manager of Enterprise Systems at Teradata, to get his take on where the company has been, where it‘s going, and the trends shaping the future of analytics. Dan first joined Teradata in the late 1990s and has spent the bulk of his career there across a variety of engineering and product leadership roles. Here are some highlights from our wide-ranging conversation.

An Architectural Foundation for Analytics at Scale

Teradata‘s core technology dates back to research done at Caltech and Citibank in the late 1970s around MPP databases. The fundamental premise was to create a scalable architecture for parallel SQL processing across multiple nodes that could handle even the largest data volumes and most complex queries.

This required a complete rethinking of database design, from the I/O subsystem and file layout to the query optimizer and execution engine. The result was a platform capable of consistently delivering fast query response times against terabytes of data, something that was revolutionary at the time.

Over the ensuing decades, Teradata has continued to evolve its core database technology to take advantage of improvements in processing power, memory, storage density and networking. Recent benchmarks showcase the platform‘s ability to scan over 9 TB/hour/core and over 1 PB/hour in a single rack.

But it‘s not just about speed. Teradata has also optimized its database for mixed workloads that combine short tactical queries and longer-running deep analytics. Advanced workload management, MAPS (multi-temperature user data) and virtual storage technologies help ensure SLAs are met even in the face of unpredictable user demand.

Extending to the Cloud

Like virtually every enterprise software company, Teradata has had to evolve its offerings to reflect the growing adoption of public cloud infrastructure. But unlike application workloads that can be relatively easily "lifted and shifted" to the cloud, analytics workloads that require frequent data movement and processing present unique challenges.

Teradata has taken a multi-pronged approach to helping customers operationalize analytics in the cloud:

1. Vantage in the Public Cloud – Teradata now offers its full Vantage analytics platform as-a-service on AWS and Azure. This provides the flexibility and agility of the cloud while still delivering the performance and enterprise-grade management of Teradata‘s on-premises offering. Multi-cluster scaling, workload isolation and consumption-based pricing are also core to the cloud offering.

2. Hybrid Cloud Integration – For customers that want to keep an on-prem Teradata system but still access and combine data in the cloud, Teradata offers a suite of hybrid cloud integration capabilities. QueryGrid provides seamless query processing across on-prem and cloud systems, while Teradata‘s low-code ingestion and integration tools simplify data movement.

3. Cloud Data Lakes – Recognizing the growing popularity of cloud data lakes, Teradata has expanded its support for object stores like Amazon S3 and Azure Blob Storage. Customers can use these lower-cost storage options for colder data while still making it accessible for analytics. Teradata is also investing heavily in R&D around "intelligent multi-temperature data management" to automatically tier data across storage options based on access patterns.

Enabling Real-Time Analytics and Machine Learning

Another major trend reshaping the data and analytics landscape is the rise of real-time analytics and pervasive machine learning. Historically, Teradata was used primarily for strategic BI and reporting workloads where data latency of a few hours or even days was acceptable. But the world has changed.

"Today, many analytics use cases require access to data that is seconds or minutes old," Dan explained. "At the same time, the volume and types of data that organizations want to analyze has exploded with the growth of IoT sensors, mobile apps, and digital customer interactions."

To address the need for real-time data freshness, Teradata has heavily parallelized data loading and introduced "hash indexes" that enable super fast point queries. Enhancements to Teradata‘s integration with open source tools like Kafka and Spark have also made it easier to stream data into Teradata in real-time and trigger analytics workflows.

Teradata has also beefed up its machine learning capabilities. Its 4D Analytics framework provides a foundation for production ML at scale with support for in-database analytics in R and Python, native graph analytics, and automated model deployment and monitoring. Integration with popular ML tools like Dataiku and AWS SageMaker help data science teams be productive with their tools of choice.

Customer Success Through Collaboration

While the technology is certainly important, what really sets Teradata apart in Dan‘s view is the company‘s laser-focus on customer success and solving real business problems. He shared several anecdotes that illustrate Teradata‘s highly collaborative approach.

For example, one large retailer was struggling to get insights from its customer data fast enough to optimize inventory and pricing decisions. By working closely with the customer to understand its data and business processes, Teradata was able to design a solution that reduced query times from hours to minutes and generated millions in additional revenue.

In another case, a global manufacturer wanted to improve its supply chain efficiency but was hampered by data silos and lack of visibility across the entire production lifecycle. Teradata partnered with the customer to integrate data from multiple ERP and shop floor systems and build predictive models that optimized inventory levels and reduced waste.

"At the end of the day, our success is completely tied to our customers‘ success," said Dan. "We have a saying inside Teradata that ‘we are not successful until our customers are successful.‘ I think that mindset really permeates every aspect of how we work and interact with customers."

The Road Ahead for Data and Analytics

Having been at the forefront of the data warehousing and business intelligence wave, Teradata now finds itself navigating a rapidly evolving landscape with new technologies, platforms and competitors emerging all the time. I asked Dan where he sees the industry headed in the coming years.

"I think the explosion of data is only going to accelerate with 5G and the continued proliferation of connected devices," he said. "At the same time, the rise of cloud data warehouses like Snowflake and BigQuery means that analytics is becoming more accessible to more people across organizations."

According to Dan, this "democratization of analytics" will require platforms that can support a wide range of skill levels and use cases, from business users exploring data to data scientists building complex models. Teradata is investing in more intuitive, low-code interfaces and tighter integration with popular analytics and data science tools to meet these diverse needs.

Dan also expects to see continued adoption of open source technologies in the modern analytics stack. In his view, proprietary and open source tools will increasingly coexist and complement each other, with open source excelling at data transformation and model building while platforms like Teradata serve as the production backbone for analytics at scale.

Ultimately, Dan believes that Teradata‘s core strengths around scalable, high-performance analytics put it in a strong position to help enterprises transform into data-driven businesses. By continuing to innovate in areas like cloud, AI and real-time processing while staying true to its customer-centric roots, Teradata aims to be the trusted partner organizations turn to make smarter, faster decisions.

"The analytics use cases and technologies may evolve, but the fundamental need for enterprises to leverage data as a competitive asset endures," Dan said. "With our combination of deep engineering talent, business acumen and relentless customer focus, Teradata will be at the forefront of analytics innovation for another 40 years."