DataHack Radio #22: Exploring the World of Computer Vision and Data Engineering with Dat Tran

The fields of computer vision (CV) and data engineering are rapidly advancing, with organizations increasingly leveraging these technologies to unlock insights and drive innovation. To shed light on how these cutting-edge techniques are being applied in the real world, I had the pleasure of speaking with Dat Tran, Head of Data Science at idealo internet GmbH, for the DataHack Radio podcast.

Dat is a true pioneer in the space, with years of experience building sophisticated CV systems and deploying ML models into production. In this in-depth interview, he shared his fascinating journey into data science, the challenges and opportunities his team is tackling at idealo, his philosophy on hiring, and much more. Let‘s dive in and explore the key insights and lessons learned.

From Investment Banking to Computer Vision Pioneer

Dat‘s path into data science is far from conventional. As an undergraduate, his focus was on investment banking, but he quickly realized it wasn‘t the right fit. Seeking a new direction, Dat pursued a Master‘s degree, which is when some friends introduced him to the burgeoning field of machine learning.

He was immediately drawn to the complexity and potential of ML. This passion led him to land a job in the advanced analytics department at Accenture, right as "big data" was becoming the industry‘s ultimate buzzword. Gartner estimates that global spending on big data and analytics will reach $274B by 2022, underscoring the huge demand for data science talent.

Recognizing the opportunity to gain more hands-on machine learning experience, Dat later moved to Pivotal Inc. as a data scientist. Over more than two years there, he worked on diverse projects spanning industries like automotive and airlines, building the deep knowledge and expertise that still serves him today. He even presented at multiple PyData conferences during this chapter – an impressive accomplishment for any data scientist.

Today, Dat leads the data science team at idealo, a successful e-commerce portal based in Berlin. His team is pushing the boundaries of what‘s possible with computer vision and data engineering at scale.

Pioneering Computer Vision at idealo

You might wonder what role computer vision plays at a price comparison site like idealo. As Dat explained, idealo has massive datasets of product and hotel images – around 2 million accommodations with an average of 130 images each. These images come from diverse sources, from owners‘ amateur photos to professionally shot visuals from luxury resorts.

Dat‘s team is tasked with analyzing and extracting value from this vast visual data. They‘ve built sophisticated CV pipelines powered by convolutional neural networks (CNNs) for:

• Image tagging: Automatically classifying images based on depicted features like bedroom, bathroom, reception area, etc. idealo‘s system can recognize over 100 distinct object categories.

• Image ordering: Intelligently reordering visuals for optimal user experience. Their algorithms consider factors like image quality, diversity, and relevance to create compelling visual narratives for each listing.

• Image upscaling: Transforming low-res photos into high-res using cutting-edge super-resolution techniques. By applying models like ESPCN and SRGAN, they can realistically enhance images 4x or more.

• Visual search: Enabling users to find products based on visual similarity. idealo‘s system uses deep metric learning to map images to a compact embedding space, where similar items cluster together.

The CNN architectures powering these applications include popular models like ResNet, Inception, and EfficientNet, implemented in frameworks like PyTorch and TensorFlow. Training these massive models requires significant compute resources – Dat‘s team makes heavy use of cloud platforms like AWS, with GPU clusters for accelerated training.

By applying CV in these innovative ways, idealo can enhance its user experience, improve listing quality, and uncover novel insights. A recent survey by IBM found that 57% of companies believe CV will play a significant role in shaping future customer experiences. idealo is ahead of the curve.

The Art and Science of Data Engineering

Deploying CV models into production is a complex challenge that requires deep data engineering expertise. Dat is a recognized leader in this space, known for his talk "Getting ML Models into Production the Cool Way" at PyData.

At idealo, Dat‘s team leverages an array of tools for data engineering, including:

• AWS: Used for model training, data storage, and more. Services like EC2, S3, and SageMaker enable teams to scale resources on demand.
• Kubernetes: An orchestration platform for deploying and managing containerized applications. idealo uses K8s to streamline model serving and ensure high availability.
• Apache Airflow: A platform for programmatically authoring, scheduling, and monitoring data pipelines. idealo‘s team uses Airflow to build robust ETL processes.
• MLflow: An open-source platform for managing the ML lifecycle, including experimentation, reproducibility, and deployment. MLflow helps idealo track and share models across the organization.

Dat emphasized the critical importance of software engineering fundamentals for data scientists – writing clean, well-documented code, using version control, implementing unit tests and integration tests, etc. He‘s found tools like pre-commit invaluable for enforcing code quality through automated checks.

In Dat‘s view, machine learning is ultimately just one part of a larger software system. Aspiring data scientists often underestimate the engineering skills required for real-world ML projects. His advice: focus on building a strong software foundation first, then layer on data science skills. It‘s this combination that will truly set you apart. The 2020 Kaggle ML & DS Survey found that 3 of the top 5 most commonly used tools by data scientists are software engineering staples: Git, the Unix shell, and Vim/Emacs.

Balancing Performance and Scalability

As powerful as advanced techniques like deep learning are, deploying them at scale introduces difficult trade-offs between model accuracy and system performance. Dat explained this challenge in the context of CNN model architectures used at idealo.

The most accurate CNNs like ResNet-152 or EfficientNet-B7 can have over 100 million parameters and billions of FLOPs (floating point operations) per inference. In a research setting, one can overlook the costs of such massive models – but in production, latency and efficiency are paramount to delivering business value.

Organizations are grappling with questions like:

• How do we select the right architecture for our use case?
• Can we optimize our models to be more lightweight without overly compromising accuracy?
• What‘s the right balance between complexity and speed for our product?

There‘s no one-size-fits-all answer, but the field is making progress. Techniques like model compression, quantization, knowledge distillation, and neural architecture search are enabling teams to create more efficient production-ready models. One study found that a compressed EfficientNet could achieve 98% of its full accuracy at just 20% the original model size.

Dat predicts we‘ll see continued innovation in systems for scalable CV/ML deployment in the coming years. Exciting projects to watch include:

• ONNX: An open format for representing ML models, enabling interoperability between frameworks.
• NVIDIA Triton: An optimized serving platform for deploying AI models at scale.
• TensorRT: A high-performance runtime for optimized inference on NVIDIA GPUs.
• TVM: A compiler stack for deep learning that enables models to run efficiently across diverse hardware.

Staying on the Cutting Edge

So how does a leader like Dat stay on top of the latest advancements in CV and data engineering? In his view, there‘s no substitute for hands-on experience.

"You best learn about these things when you do them yourself," he emphasized. There‘s nothing like wrestling with real data and models to develop true understanding and expertise.

That said, Dat also takes advantage of the wealth of learning resources available today – online courses, blog posts, podcasts, research papers, and more. A few of his go-to‘s:

• Papers With Code: A free resource that provides open access to ML papers, code, datasets, methods, and evaluation tables.
• Two Minute Papers: A YouTube channel that breaks down the latest AI research in accessible 2-5 minute videos.
• TDS: A Medium publication sharing concepts, ideas, and codes on data science and ML.
• deeplearning.ai: Andrew Ng‘s online deep learning courses, which combine theory and practice.

The challenge is that these materials often lack the structure and direct applicability of working on an actual project. Dat recommends using them to supplement and expand your on-the-job learning.

Another key strategy is to learn from other experts in the field. Dat is an active participant in data science communities and conferences, both as an attendee and a presenter. Building a strong peer network is invaluable for staying engaged with the latest trends and techniques.

Hiring World-Class Data Science Talent

As a data science leader, Dat also shared insights into his approach to hiring top talent. His process focuses on evaluating candidates in three key areas:

1. Machine learning fundamentals: Dat starts with 10 core ML concept questions that assess baseline knowledge – things like bias-variance tradeoff, cross-validation, regularization. Surprisingly, this is where many candidates drop off.

2. Practical skills: Next is a take-home machine learning assignment to evaluate the candidate‘s ability to actually implement concepts. Dat looks for clear, well-structured code that follows best practices.

3. Problem-solving and communication: Finally, Dat brings shortlisted candidates on-site to collaborate with his team on solving a realistic problem. He looks for strong critical thinking, research skills, and clear communication.

Throughout this process, Dat pays close attention to the candidate‘s software engineering abilities. Can they write clean, modular, well-documented code? Do they use version control effectively? How efficiently and scalably can they implement models?

Finding data scientists with this rare combination of ML knowledge and software engineering skill is a challenge, but it‘s well worth the effort to build a world-class team. Dat has found that focusing on these fundamentals yields candidates who can hit the ground running and make an impact from day one.

The Future of Computer Vision and Data Engineering

To wrap up our discussion, I asked Dat to share his predictions for the future of AI, ML, and computer vision. A few key trends he highlighted:

AutoML platforms that automate the model building process (e.g. Google AutoML, Microsoft Azure AutoML) will continue to gain adoption, making machine learning more accessible to non-experts. However, they‘re not a silver bullet – Dat believes there will always be a need for bespoke solutions and human expertise.

There will be a growing focus on developing explainable AI systems, whose reasoning and predictions can be understood and audited by humans. As AI is applied in increasingly high-stakes domains like healthcare and criminal justice, the ability to interpret and trust models is critical. The DARPA XAI program is funding research to create a new generation of explainable AI systems.

ML security and governance will come to the forefront, as companies seek to safeguard models against emerging threats like data poisoning, model inversion, and adversarial attacks. At the same time, there will be increasing scrutiny on the ethical development and deployment of AI to prevent unintended negative consequences. Dat applauds initiatives like the EU Trustworthy AI guidelines for providing a framework for responsible AI practices.

Dat is optimistic that ongoing research in areas like transfer learning, few-shot learning, self-supervised learning, and reinforcement learning will make AI systems more flexible and adaptable, while reducing dependence on massive labeled datasets. We‘re already seeing promising results:

• CLIP, a model trained on 400M image-text pairs, can be adapted to multiple downstream vision tasks with no additional training data.
• SimCLR, a self-supervised learning approach, achieved 85% of supervised performance on ImageNet using only unlabeled images.
• AlphaFold 2, a model trained via free-energy reinforcement learning, made a huge leap in solving the protein folding problem.

In the coming years, Dat predicts these and other advanced techniques will make their way from research into production, powering a new generation of intelligent applications. He encourages aspiring data scientists and ML engineers to dive deep into these areas – that‘s where the most exciting opportunities lie.

Conclusion

Throughout our conversation, Dat‘s deep passion for and command of his field shone through. His insights illuminate the state-of-the-art in applied computer vision and data engineering, while charting an exciting course for the future.

The key takeaways for aspiring data scientists and ML practitioners:

1. Build a strong foundation in software engineering and model deployment, not just ML theory.
2. Seek out diverse hands-on experience working with real-world data and models.
3. Stay on the cutting edge by learning from experts, papers, and courses – but always supplement with project work.
4. When evaluating tools and techniques, constantly balance tradeoffs between accuracy, efficiency, and scalability.
5. Develop skills in emerging areas like AutoML, explainable AI, ML security, and advanced learning paradigms – that‘s the next frontier.

Most importantly, cultivate curiosity and a willingness to learn. Dat‘s story shows that the path to CV and data engineering mastery is an ongoing journey that requires constantly updating one‘s skills and knowledge.

But for those willing to embrace the challenge, the reward is the opportunity to create groundbreaking intelligent systems that can solve some of the world‘s biggest problems. I, for one, left my conversation with Dat inspired to double down on my own learning journey – I hope you do too!

You can learn more about Dat Tran‘s work and connect with him on LinkedIn. And be sure to check out his talk "How you really get your data science models into production the cool way!" from PyData Berlin 2019 on YouTube.