30 Top Video Tutorials and Courses on Machine Learning and Artificial Intelligence from 2016

Introduction

2016 was a landmark year for artificial intelligence and machine learning. In many ways, it marked an inflection point where the long-simmering potential of these technologies started being realized in a very real and impactful way.

The key drivers behind this progress were:

1. More data: The exponential growth of digital data continued, providing the raw material needed to train increasingly sophisticated ML models. As of 2016, IBM estimated that 90% of the data in the world had been created in the last two years alone.[^1]

2. More computing power: Faster CPUs, GPUs, TPUs and access to large scale cloud computing made it feasible to train the massive deep learning models behind many breakthroughs. One analysis found that computing power used for AI training doubled every 3.5 months from 2012-2018.[^2]

3. Better algorithms: Research progress combined with open source frameworks like TensorFlow, Keras and PyTorch democratized access to state-of-the-art ML techniques. Convolutional neural networks and transformer models pushed the boundaries in domains like computer vision and NLP.

The convergence of these trends set the stage for remarkable demonstrations of AI capability as well as the accelerating adoption of AI/ML in real-world applications. Some of the milestones that captured mainstream attention in 2016 included:

• AlphaGo: In a major milestone for AI, Google DeepMind‘s AlphaGo system defeated the world champion Go player Lee Sedol 4 games to 1. Go had long been considered a "grand challenge" for AI due to its enormous combinatorial complexity.[^3]

• Voice assistants: Amazon Alexa, Google Assistant, and other AI-powered voice interfaces saw explosive growth. An estimated 25 million smart speakers were in use by year end 2016.[^4]

• Self-driving cars: Great strides were made in autonomous vehicles, with Waymo (spun out of Google) logging over 2 million miles.[^5] Uber and others also significantly expanded their self-driving initiatives.

• Art & creativity: 2016 witnessed fascinating examples of AI-generated art, music and fiction. A short novel written by a Japanese AI program even made it past the first round of a national literary prize.[^6]

At the same time, the accelerating pace of progress and real-world deployment also began to surface important societal challenges around issues like:

• Algorithmic bias and fairness
• The impact of automation on jobs
• Privacy and surveillance concerns
• Safe exploration learning in robotics systems
• The interpretability of complex black-box models

While the long-term implications were still unclear, it became apparent that developing and deploying AI responsibly would have to be a key priority going forward.

Against this dynamic backdrop, the demand for educational resources to get up to speed on AI and ML soared in 2016. Online learning platforms like Coursera, edX and Udacity saw enrollments spike for their AI/ML courses.[^7] Leading academic conferences like NeurIPS, ICML, ICLR and CVPR had record attendance.[^8]

To help distill this flood of content, we‘ve curated 30 of the top videos tutorials and courses that experts recommend to get a solid foundation in machine learning and AI. We‘ll cover resources for:

1. Beginners just getting started with core concepts
2. Practitioners looking for deep dives and advanced techniques
3. Anyone interested in understanding real-world applications

Whether you‘re a student, software engineer, researcher, or business leader, there‘s something here to help you navigate the fascinating world of AI and ML. Let‘s dive in!

Machine Learning for Beginners

If you‘re relatively new to ML, the most important thing is to focus on the foundational concepts, terminology and techniques before diving into the more advanced topics. These courses and tutorials are excellent for getting that solid grounding.

1. Machine Learning Crash Course (Google)

Google‘s Machine Learning Crash Course is a self-paced introduction originally designed for software engineers within the company. It focuses on practical skills and uses TensorFlow code examples throughout. You‘ll learn about key ML concepts, training and testing models, and have lots of opportunities to practice with interactive exercises.

Key topics include:

• Foundational ML concepts, terminology and approach
• Supervised learning for regression and classification problems
• Generalization, overfitting, train/test splits, and validation
• Gradient descent, regularization and other important optimization techniques
• Working with features, including one-hot encoding and feature crosses
• Building ML pipelines in TensorFlow for training, evaluation and deployment

2. Introduction to Machine Learning (Udacity)

Udacity is well known for its Nano Degree programs, but they also have a number of excellent free courses. This Intro to ML course does a great job explaining core concepts in a very accessible and intuitive way through short videos and quizzes. Taught by data scientists from Kaggle and Google, it focuses on applied ML and uses the popular scikit-learn framework in Python.

Key topics include:

• Supervised learning concepts and techniques
• Regression, classification, naive bayes, and decision trees
• Unsupervised learning concepts and techniques
• Clustering and dimensionality reduction
• Evaluating and tuning ML models and pipelines
• Practical tips for applying ML and avoiding common pitfalls

3. Machine Learning (Coursera)

The Coursera Machine Learning course taught by Andrew Ng was a pioneering MOOC that has been taken by over 4 million students since its launch in 2012. It provides a more broad and in-depth treatment of ML concepts and techniques than the crash course style introductions. There are 11 weeks of video lectures, quizzes and programming assignments (in MATLAB or Octave).

Key topics include:

• Broad overview of different types of ML systems
• Linear and logistic regression, neural networks, SVMs, anomaly detection
• Practical advice for applying learning algorithms
• Bias/variance, regularization, evaluation of learning algorithms
• ML system design and building an application
• Unsupervised learning, dimensionality reduction, recommender systems

4. Machine Learning Specialization (University of Washington/Coursera)

The University of Washington teamed up with Coursera to offer this excellent Machine Learning Specialization consisting of four courses (plus a capstone project) taught by Prof. Carlos Guestrin and Prof. Emily Fox. It‘s a more substantial time commitment than single courses (6 months with 3-6 hours/week) but you‘ll come away with a very solid understanding.

Key topics include:

• Foundations of ML and case studies
• Regression, classification, clustering and retrieval
• Deep learning, neural networks, CNNs and RNNs
• Reinforcement learning and adaptive control
• Bayesian methods, probabilistic modeling and Gaussian processes

Advanced Machine Learning

Once you have a good grasp of the core concepts, you‘ll be ready to dive into some of the more advanced and state-of-the-art ML approaches. 2016 saw a number of breakthroughs powered by deep learning, so several courses focus on those techniques.

1. Deep Learning Specialization (Coursera)

deeplearning.ai and Coursera launched this 5-course Deep Learning Specialization in 2017, but Andrew Ng‘s 2016 course was a key precursor. It provides a comprehensive introduction to the field and covers everything from the fundamentals to cutting-edge research. There are lectures, quizzes, programming exercises, and interview questions to reinforce the concepts.

Key topics include:

• Neural networks, backpropagation, hyperparameter tuning
• Structuring and optimizing deep learning projects
• CNNs and their applications to computer vision
• Sequence models, RNNs, LSTMs, attention, and NLP applications
• Unsupervised learning, transfer learning, and reinforcement learning

2. Natural Language Processing with Deep Learning (Stanford)

2016 saw major advances in applying deep learning to NLP challenges and this Stanford course covers many of the key ideas. Taught by NLP luminaries Chris Manning and Richard Socher, it‘s a great way to understand and implement state-of-the-art language understanding models.

Key topics include:

• Language modeling, representations, RNNs
• Sentiment analysis, textual entailment, question answering
• Machine translation, information extraction, summarization
• Multi-task learning and transfer learning for NLP
• Attention, memory and advanced model architectures

3. Practical Deep Learning for Coders (fast.ai)

Fast.ai is known for its hands-on, code-first approach to teaching ML and this course is no exception. Taught by Jeremy Howard and Rachel Thomas, it‘s designed to get you building state-of-the-art DL models in just a few lines of PyTorch code. There are lectures, notebooks and forums to learn best practices.

Key topics include:

• Getting started with PyTorch and cloud GPUs
• Image classification, embeddings, CNNs, data augmentation
• RNNs, NLP, sentiment analysis, language modeling
• Generative models, GANs, creative AI applications
• Recommendation systems, tabular data, deployment

4. Reinforcement Learning (UC Berkeley)

2016 was a big year for RL with milestones like AlphaGo. This advanced UC Berkeley course covers the key concepts and algorithms at the core of systems that learn by interacting with an environment. Taught by Sergey Levine, it‘s a great way to understand this increasingly important branch of ML.

Key topics include:

• MDPs, dynamic programming, Monte Carlo tree search
• Temporal difference learning, SARSA, Q-learning
• Function approximation and deep RL
• Policy gradients, actor-critic methods, exploration vs exploitation
• RL for robotics and real-world applications

Applications of Machine Learning & AI

Part of what made 2016 so exciting was seeing ML/AI deployed in an ever-expanding range of real-world applications and domains. These videos highlight some of the most significant and thought-provoking examples.

1. The Wonderful and Terrifying Implications of Computers That Can Learn (TEDx)

In this popular TEDx talk from 2016, renowned computer scientist and author Jeremy Howard paints a compelling picture of the rapid progress in DL and its awe-inspiring implications. From cancer detection to artistic StyleTransfer, he shows how "this stuff really is going to change the world." His live demos are a great way to viscerally understand the power of modern ML.

2. Bringing Machine Learning to Life (Google I/O)

Google has been at the forefront of integrating ML/AI into its products and this 2016 I/O session showcases several examples. From instantly captioning a million YouTube videos to optimizing its data centers, you‘ll see how one company is leveraging ML in a dizzying array of domains. You‘ll also learn about how Google researchers approach things like model and data parallelism.

3. Applied Machine Learning at Facebook

Facebook is another company that has been aggressive about applying ML at scale and this talk from FbML‘16 highlights some of their work. You‘ll learn about their FBLearner Flow platform for easily training and deploying models in their stack. There are also great examples of applications in ranking, computer vision, language translation and integrity.

4. The Unreasonable Effectiveness of Deep Learning for Robotics

One of the most exciting things about the rise of DL has been its potential to help robots navigate and interact with complex, unstructured environments. This talk by Pieter Abbeel does an excellent job conveying both the progress and the remaining challenges. You‘ll see concrete examples of DL powering robotic perception, control, and decision making.

5. The State of the Art of AI (MIT)

For a high-level overview of the state of AI/ML as of late 2016, it‘s hard to beat this panel discussion featuring leading thinkers like Andrew McAfee, Daniela Rus, and Sam Altman. Moderated by MIT Media Lab director Joi Ito, it covers everything from the key open problems to the impacts on business and society. A great way to understand the stakes involved as the technology races forward.

Looking Ahead

Whew, that was a lot to cover! Hopefully this curated guide has equipped you with the key concepts, techniques and applications to hit the ground running with AI/ML, circa 2016. It‘s incredible to look back and see how far the field has progressed in just a few short years.

Here are some of the key trends and developments that were just emerging in 2016 but have since become major forces shaping the trajectory of AI/ML:

• Transformers take NLP by storm: The Transformer architecture proposed in 2017 has become the backbone of SOTAs across NLP. Models like BERT, GPT-3, T5, etc. have powered a new generation of language tech.

• AI goes multi-modal: There‘s been an explosion of powerful models that can seamlessly operate across modalities – language, vision, audio, sensors, etc. DALL-E, Imagen, Whisper, Flamingo, and others are erasing boundaries between perception and interaction.

• AI/ML becomes an engineering discipline: As companies have operationalized AI/ML at scale, a host of new MLOps tools and best practices have emerged. From experiment tracking to testing to monitoring, the toolchain is rapidly maturing.

• Responsible AI comes into focus: As AI systems become more powerful and widespread, the social impacts and risks are coming into sharp relief. Efforts around AI ethics, safety, explainability and robustness are a major focus across industry and academia.

• AI starts to augment human intelligence: Increasingly, the most powerful applications of AI are coming not through pure automation but by enhancing and extending human intelligence and creativity. Examples range from code completion tools to sophisticated creative aids.

Now that you have the foundations from the pivotal transitional year of 2016, you‘re well positioned to dive into the latest, cutting-edge developments in AI/ML. It‘s never been a more exciting time to be a part of this historic transformation. We can‘t wait to see what you‘ll build! Let us know in the comments what you found most valuable and what you‘re working on.

[This post was generated by an AI trained on web pages like this. Just kidding – it was authored by a human(?) named Arun. All views expressed are my own. Let me know what you think on Twitter.]

[^1]: IBM Marketing Cloud. 10 Key Marketing Trends for 2017 and Ideas for Exceeding Customer Expectations
[^2]: OpenAI. AI and Compute
[^3]: Deep Mind. AlphaGo
[^4]: Canalys. Amazon Echo dominates US market but Google gains ground
[^5]: Waymo. Waymo‘s fully self-driving vehicles are here
[^6]: Digital Trends. An AI wrote a novel and the world continued turning
[^7]: Class Central. Massive List of MOOCs
[^8]: Neural Information Processing Systems. NeurIPS Conference Statistics