My Story: How I Became a Machine Learning Expert in 10 Months
The AI/ML Revolution
Artificial intelligence and machine learning are transforming industries across the globe at breakneck speed. From automating repetitive tasks to uncovering hidden patterns in massive datasets to enabling more intelligent decision-making, the applications of AI and ML seem limitless. A 2019 report by McKinsey Global Institute projected that AI could deliver an additional economic output of around $13 trillion by 2030, increasing global GDP by about 1.2 percent a year. The report also estimated that about 70 percent of companies will adopt at least one form of AI by 2030.
With this rapid growth comes immense opportunity for professionals who develop skills in AI and ML. The machine learning engineer has been called the "hottest job in tech". According to Indeed, the average base salary for a machine learning engineer in the US is $146,085. An analysis from Glassdoor found that the number of open jobs in AI-related fields grew 100% between 2016 to 2018. Data from LinkedIn shows there was a 9.8x growth in AI skills listed by members and a 6x growth in AI jobs posted from 2015 to 2017.
In light of these eye-popping numbers, I knew I wanted to get in on the action. Having worked as a data analyst for several years, I had a solid foundation in statistics and experience with data wrangling and visualization in Excel and SQL. But to really dive into AI/ML, I knew I needed to significantly level up my skills. I set a challenging goal for myself: go from data analyst to machine learning expert in just 10 months.
The Learning Path
I researched extensively to identify the core competencies required and put together a comprehensive self-learning curriculum. The high-level learning path I followed looked like this:
- Commit to a programming language and build a foundation (6-8 weeks)
- Review statistics, probability, linear algebra, and calculus (4-6 weeks)
- Practice exploratory data analysis and visualization using the language (4 weeks)
- Learn machine learning concepts, algorithms, and tools (8-12 weeks)
- Study specialized topics like deep learning, NLP, computer vision (8 weeks)
- Gain experience with big data and cloud technologies (4-6 weeks)
- Build end-to-end projects to showcase skills (4-8 weeks)
Let‘s dive into each of these steps in more detail.
Step 1: Programming Foundation
I debated between Python and R, the two primary languages used in data science and ML. After some research, I decided to start with Python due to its versatility, extensive collection of libraries, and increasing adoption in the field. I worked through the Python track on Codecademy and then dove into more data science-focused courses on DataCamp and Dataquest. Some key libraries I focused on included:
- NumPy for numerical computing
- Pandas for data manipulation and analysis
- Matplotlib and Seaborn for data visualization
- Scikit-learn for machine learning algorithms
I made sure to code every day, even if just for 30 minutes, to make Python syntax and data structures second nature. Deliberate practice with small projects helped solidify my understanding.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv(‘data.csv‘)
df.head()
df.describe()
plt.hist(df[‘column‘])
plt.show()Step 2: Math and Statistics Review
Having taken statistics and calculus courses in college, I had some foundational knowledge but knew it needed refreshing. I worked through the excellent Statistics and Probability courses on Khan Academy. I supplemented with the book "All of Statistics" by Larry Wasserman to dive deeper into concepts like:
- Random variables and probability distributions
- Hypothesis testing and confidence intervals
- Bayesian inference
- Regression and classification
- Estimation and sampling
Additionally, I brushed up on the linear algebra fundamentals required for ML, like matrix operations, eigenvectors/eigenvalues, principal component analysis, etc. The book "Linear Algebra and Its Applications" by Gilbert Strang and videos from 3Blue1Brown were very helpful.
Step 3: Exploratory Data Analysis (EDA)
I had worked with the PyData stack (NumPy, Pandas, Matplotlib) in the first phase, but now it was time to put those tools to use for exploring real-world datasets. I found a wealth of interesting datasets on Kaggle and the UCI Machine Learning Repository. Some of my analyses included:
- Visualizing relationships between features using scatter plots, pair plots, heatmaps
- Examining distributions of numeric variables with histograms, box plots, violin plots
- Comparing distributions across groups using overlaid density plots or faceted histograms
- Identifying trends over time using line plots
- Summarizing categorical variables with bar charts and count plots
The Python Graph Gallery and R Graph Gallery were great resources for inspiration. I aimed to uncover meaningful insights in each dataset and present them in a visually compelling way.
Step 4: Machine Learning Algorithms
With strong Python skills and a refreshed math/stats foundation, I was ready to dive into the meat of ML. I started with Andrew Ng‘s classic Machine Learning course on Coursera to learn about:
- Supervised learning – linear and logistic regression, decision trees, SVMs, naive Bayes, ensemble methods like random forests and gradient boosting
- Unsupervised learning – K-means clustering, principal component analysis, recommender systems
- Best practices for model evaluation and tuning, feature selection, bias-variance tradeoff
I implemented many of these algorithms from scratch in Python to gain a deeper understanding. Scikit-learn was an invaluable library for rapidly building and assessing models.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
rf = RandomForestClassifier()
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f‘Test Accuracy: {accuracy:.2f}‘)I practiced the end-to-end ML workflow, from data loading to model deployment, on various datasets. Kaggle competitions provided an fun way to tackle realistic, messy data challenges.
Step 5: Specializations
With the ML fundamentals under my belt, I explored more specialized topics relevant to my interests. I took a deep dive into deep learning, fascinated by its ability to automatically learn hierarchical features from raw data. Courses and books I found most helpful:
- Deep Learning Specialization on Coursera
- fastai Practical Deep Learning for Coders course
- "Deep Learning with Python" by Francois Chollet
I implemented standard neural net architectures like feedforward nets, convolutional neural networks (CNNs), and recurrent neural networks (RNNs) using deep learning frameworks like TensorFlow and PyTorch.
import tensorflow as tf
from tensorflow import keras
model = keras.Sequential([
keras.layers.Conv2D(32, (3,3), activation=‘relu‘, input_shape=(32, 32, 3)),
keras.layers.MaxPooling2D((2,2)),
keras.layers.Conv2D(64, (3,3), activation=‘relu‘),
keras.layers.MaxPooling2D((2,2)),
keras.layers.Conv2D(64, (3,3), activation=‘relu‘),
keras.layers.Flatten(),
keras.layers.Dense(64, activation=‘relu‘),
keras.layers.Dense(10, activation=‘softmax‘)
])
model.compile(optimizer=‘adam‘,
loss=‘sparse_categorical_crossentropy‘,
metrics=[‘accuracy‘])
model.fit(X_train, y_train, epochs=10, validation_data=(X_test, y_test)) I also explored subfields like natural language processing (NLP), building a Twitter sentiment classifier and experimenting with transformer models like BERT. Computer vision was another area of focus, where I practiced techniques like image classification, object detection, facial recognition, and image generation with GANs. Seeing deep learning models achieve superhuman performance on complex perceptual tasks was mind-blowing.
Step 6: Big Data and the Cloud
While Pandas and scikit-learn can work well for small to medium-sized datasets, eventually I knew I‘d encounter big data challenges requiring distributed computing. I spun up a Hadoop cluster on AWS and practiced using tools like:
- HDFS and MapReduce for storage and batch processing of massive datasets
- Hive for running SQL queries on petabyte-scale data warehouses
- Spark for faster in-memory processing and machine learning at scale
- Dask for parallelizing Python workloads across clusters
I also experimented with managed ML platforms like AWS SageMaker, Google AI Platform, and Azure ML Studio to streamline the model training and deployment process. Leveraging the power of the cloud opened up new possibilities.
from pyspark.ml.classification import LogisticRegression
from pyspark.ml.feature import HashingTF, Tokenizer
training = spark.createDataFrame([
(0, "a b c d e spark", 1.0),
(1, "b d", 0.0),
(2, "spark f g h", 1.0),
(3, "hadoop mapreduce", 0.0)
], ["id", "text", "label"])
tokenizer = Tokenizer(inputCol="text", outputCol="words")
wordsData = tokenizer.transform(training)
hashingTF = HashingTF(inputCol="words", outputCol="features", numFeatures=1000)
featurizedData = hashingTF.transform(wordsData)
lr = LogisticRegression(maxIter=10, regParam=0.001)
model = lr.fit(featurizedData)
test = spark.createDataFrame([
(4, "spark i j k"),
(5, "l m n"),
(6, "spark hadoop spark"),
(7, "apache hadoop")
], ["id", "text"])
prediction = model.transform(test)
prediction.show()Step 7: End-to-End Projects
Finally, it was time to put all the pieces together and build some end-to-end projects to showcase in my portfolio. Some highlights:
- A web app to classify dog breeds using a CNN model trained on the Stanford Dogs Dataset
- An NLP-powered Chrome extension to summarize articles and provide article recommendations
- A big data pipeline to predict customer churn using Spark and a gradient boosted tree model
- An AR iPhone app to identify and describe objects in real-time using YOLO and Core ML
Building these projects required combining skills across the AI/ML workflow, from data ingestion to model training to deployment. They helped me crystallize my understanding and gave me valuable experience collaborating with others through open source.
Lessons Learned
The journey from data analyst to AI/ML expert in 10 months was arduous but incredibly rewarding. Some key lessons learned:
- Commit to an aggressive learning goal and hold yourself accountable with a timeline
- Seek out the best learning resources and mentors to accelerate your growth
- Balance theory with application – build early and often
- Document your progress and share your knowledge; learn by teaching others
- Embrace discomfort – the magic happens outside your comfort zone
The field of artificial intelligence and machine learning continues to evolve at a staggering pace. Staying up-to-date requires continuous learning and experimentation. I‘m excited to keep growing and applying my skills to tackle meaningful challenges. If my story resonates with you, I encourage you to dive in and keep pushing the boundaries of what‘s possible. The future belongs to those who can harness the power of AI and ML to drive innovation forward.
