Driving Data Science at Lyft and Airbnb: Lessons from Dr. Alok Gupta
The fields of data science and machine learning have transformed industries and powered the growth of revolutionary companies over the past decade. Two prime examples are Airbnb and Lyft, which have leveraged cutting-edge AI/ML to disrupt the lodging and transportation sectors.
At the forefront of the data science efforts at these companies is Dr. Alok Gupta. Currently the Director of Data Science and Head of Growth Science at Lyft, Dr. Gupta previously spent four impactful years building the data science team and capabilities at Airbnb. His unique career path and leadership experiences offer valuable lessons for data scientists and organizations aiming to drive impact through data.
From Mathematics to Machine Learning
Dr. Gupta‘s journey to data science leadership began with a deep passion for mathematics. He earned his undergraduate degree in the field from the prestigious University of Cambridge before obtaining a Masters in Finance and Mathematics from Imperial College London. It was there that he first developed a fascination with stochastic calculus and statistics.
"I absolutely fell in love with statistics and its application to financial markets during my graduate studies," Dr. Gupta recalled in an interview on the DataHack Radio podcast[^1]. This newfound interest compelled him to pursue a Ph.D. focused on stochastic optimization at Oxford University, which he completed in 2010.
After graduating, Dr. Gupta took a role as a quantitative trader at Deutsche Bank during the aftermath of the global financial crisis. Here he designed and implemented algorithmic trading strategies, utilizing his expertise in statistics and modeling. "My role drew heavily on concepts from time series analysis, regression, optimization and other techniques that are core to machine learning," he noted.
It was through this experience that Dr. Gupta first recognized the parallels between quantitative finance and the burgeoning field of data science. Motivated to apply his skills to new domains, he began pursuing opportunities in the technology sector and in 2014, landed a role as a data scientist at Airbnb.
Laying the Foundation for Data Science at Airbnb
When Dr. Gupta joined Airbnb in 2014, the company had just reached 1,000 employees with a data science team of only 10. This gave him the unique opportunity to shape the team and establish best practices as the company underwent hypergrowth.
He started out building machine learning models for fraud detection and risk assessment to support the company‘s Trust & Safety initiatives. "Online marketplaces are ripe targets for bad actors, so identifying and preventing fraud is critical," explained Dr. Gupta. "We built sophisticated models drawing on techniques from anomaly detection, graph analysis, and more."
From there, Dr. Gupta went on to found and lead the data science team for Airbnb‘s Customer Support organization. At the time, the company had over 10,000 support agents globally handling inquiries across channels like phone, email, chat and social media. His team was tasked with optimizing the routing and allocation of tickets to improve efficiency and customer satisfaction.
"We treated the support optimization as a large-scale machine learning problem," said Dr. Gupta. "We considered factors like inquiry topic, agent skills, seasonality, locale and built models to predict metrics like resolution time and satisfaction scores." By implementing these models, Airbnb was able to significantly reduce average handle times while maintaining high CSAT.
For his final two years at Airbnb, Dr. Gupta shifted focus to user growth, leading teams that worked on guest acquisition and booking conversion. "Airbnb operates in a highly competitive market, so driving efficient growth through both paid and organic channels is key," he noted. His team built ML models to optimize spend across search, social and display advertising as well as to personalize content and offers on-site.
During Dr. Gupta‘s tenure, Airbnb‘s data science team grew over 10x to more than 100 members [^2]. Even more impactfully, he helped instill a culture of data-driven decision making and equip the company with the tools and talent to leverage machine learning across all functions.
Scaling Data Science and ML at Lyft
In 2018, Dr. Gupta joined Lyft to lead its rapidly growing data science organization. He is responsible for a 40+ person team spanning data science, machine learning engineering and data platform development that support all areas of Lyft‘s business [^3].
One of his initial mandates has been to scale Lyft‘s data science capabilities and align them with core company priorities around growth, engagement and marketplace efficiency. "Lyft is in a stage of hypergrowth, so we are focused on building the models and systems to power that growth sustainably," Dr. Gupta explained.
On the demand side, his teams are working on user acquisition, churn prevention, pricing, promotions and deep learning models for personalization. "We leverage Lyft‘s rich user data to build predictive models that help us match the right riders with the right offers at the right time," he said. Lyft has written about using techniques like gradient boosting and neural networks for these applications [^4].
Supply optimization is another key focus area, with data scientists building forecasting, simulation and behavioral models to predict and influence driver utilization. "Understanding the dynamics of a real-time, two-sided marketplace requires very sophisticated modeling," noted Dr. Gupta. "We consider factors like historical and real-time demand, road traffic, weather, special events and more to equip Lyft to make optimal decisions."
Across all these domains, Dr. Gupta has drawn on many of the best practices he established at Airbnb while adapting them for Lyft‘s needs. "Building cross-functional relationships, clearly defining success metrics, and investing in scalable tooling are all critical regardless of the company," he explained. "But the relative prioritization, tactical execution and domain-specific considerations certainly vary."
The Future of Data Science and Career Advice
Having built and led high-impact data science organizations at two of the most transformational companies of the past decade, Dr. Gupta has a unique perspective on the future of the function. "Data science and machine learning will only become more important to companies‘ competitiveness and fundamental operations," he asserted.
In particular, he sees techniques like deep learning and reinforcement learning gaining wider adoption and driving more intelligent, automated decision making. "As data scales and models become more sophisticated, data science will power more real-time, mission critical systems," Dr. Gupta predicted. "This will require data scientists to work even more closely with software and ML engineers."
Dr. Gupta also forecasts a democratization of machine learning that empowers employees outside of data science to leverage the technology. "Not everyone needs to become a machine learning expert, but basic data science literacy will become table stakes in many roles," he claimed. Based on his experience, best-in-class companies will invest in education and tooling to enable this.
As for aspiring data scientists looking to follow in his footsteps, Dr. Gupta offered a few key pieces of advice:
- Develop a strong technical foundation in statistics, programming and machine learning
- Cultivate business acumen and collaborate cross-functionally from the outset of your career
- Seek out fast-growing companies where data science can have an outsized impact
- Never stop learning as the tools, techniques and applications of data science evolve
"Data science and machine learning skills are highly applicable across industries and throughout the modern organization," Dr. Gupta concluded. "I‘ve been fortunate to leverage them to drive impact at amazing companies like Airbnb and Lyft, and I‘m excited to continue advancing the field through my work. For those entering the discipline, you have the opportunity to shape some of the most dynamic organizations and important technologies of our time."
[^1]: DataHack Radio episode with Dr. Gupta, 2019[^2]: Airbnb Technology Blog, "Using Data Science to Improve the Airbnb Experience", 2015
[^3]: VentureBeat, "Lyft hires new head of growth and head of data science", 2018
[^4]: Lyft Engineering Blog, "From Model to Action: Serving ML Predictions at Lyft", 2020
