Out-of-Core Machine Learning: Efficiently Handling Massive Datasets

Machine learning has revolutionized many industries by enabling computers to learn patterns from data and make predictions. However, a key challenge arises when the dataset is too large to fit into a single machine‘s memory. Loading the entire dataset could cause the system to run out of RAM and crash.

Fortunately, out-of-core learning provides an efficient solution to this problem. Out-of-core learning is a technique that loads the data in chunks or batches from disk, processes each chunk, and then discards it to load the next one. This allows training machine learning models on datasets that are much larger than the available RAM.

In this article, we‘ll take a deep dive into out-of-core learning and a related technique called subsampling. We‘ll explore how to implement both using Python and libraries like pandas and Vaex. By the end, you‘ll have a solid understanding of how to tackle massive datasets in your own machine learning projects.

What is Subsampling?

Before we get into out-of-core learning, let‘s discuss a simpler but related approach to handling large datasets called subsampling. Subsampling involves taking a random sample of the data that is small enough to fit in memory. The idea is that a representative subset of the data can provide a reasonable approximation for training a model, especially if the data is homogeneous.

The benefits of subsampling are:

• Reduced memory usage since only a fraction of the data needs to be loaded
• Faster training times due to working with a smaller dataset
• Ability to experiment and iterate quickly

However, the downsides are:

• Potential loss of important patterns or outliers in the unsampled data
• Sampling bias if the sample is not truly random and representative
• Need to still handle massive datasets if even a fraction doesn‘t fit in memory

Subsampling with Pandas

The popular data analysis library pandas provides a convenient way to subsample data using the sample() function. Here‘s an example of how to take a 10% sample of a CSV file that‘s too large to read into memory:

import pandas as pd

# Read the large CSV file in chunks 
chunks = pd.read_csv(‘large_data.csv‘, chunksize=10000)

# Subsample 10% of each chunk
sampled_chunks = []
for chunk in chunks:
    sampled_chunk = chunk.sample(frac=0.1)
    sampled_chunks.append(sampled_chunk)

# Concatenate all the sampled chunks into a single DataFrame 
sampled_data = pd.concat(sampled_chunks)

In this code, we:

1. Read the CSV file in chunks of 10,000 rows at a time using chunksize parameter
2. For each chunk, take a random 10% sample using sample(frac=0.1)
3. Append each sampled chunk to a list
4. Concatenate all the sampled chunks together into the final subsampled DataFrame

The sample() function allows you to specify the number of rows to sample with the n parameter or the fraction of rows with the frac parameter. You can also set replace=True to enable sampling with replacement.

Out-of-Core Learning

While subsampling can be useful, it has limitations and isn‘t suitable for all large dataset problems. This is where out-of-core learning comes in. As mentioned earlier, out-of-core learning works by loading the data in chunks, processing each chunk, and then moving on to the next one. Only a subset of the data resides in memory at any given time.

The general out-of-core learning process looks like:

1. Load a chunk of data from disk into memory
2. Preprocess the data chunk (cleansing, normalization, feature extraction, etc.)
3. Partially train the model on the chunk
4. Discard the chunk and load the next one
5. Repeat steps 2-4 until the entire dataset is processed
6. Perform a final training iteration on the entire dataset if needed

Some machine learning algorithms are well-suited to out-of-core learning because they can be trained incrementally. These include:

• Stochastic Gradient Descent (SGD)
• Mini-batch gradient descent
• Vowpal Wabbit
• Tree-based methods like Random Forest or XGBoost

Depending on the size of the data chunks, out-of-core learning may be slower than training on the full dataset at once. However, it makes training possible on huge datasets where memory is a bottleneck. It‘s also quite scalable, since the chunk size can be tuned based on the available hardware.

Out-of-Core Learning with Vaex

Vaex is a high-performance Python library for lazy out-of-core dataframes and machine learning. It allows you to visualize, explore, and model datasets that are much larger than memory. Let‘s see how we can use Vaex to train a model on a massive dataset.

First, install Vaex:

pip install vaex

Next, let‘s assume we have a large CSV file called large_data.csv with 10 million rows and 5 columns. We can read it into a Vaex DataFrame like this:

import vaex

df = vaex.from_csv(‘large_data.csv‘)

Vaex will read the CSV in chunks and only load the data it needs into memory. We can get a sense of the dataset using df.describe():

df.describe()

#  column      count         mean          std        min       max
0    col1  10000000  5000000.500  2886751.539      1.000  10000000
1    col2  10000000        0.500        0.289      0.000      1.000
2    col3  10000000       50.000       28.868      0.100    100.000  
3    col4  10000000     1000.000      577.350      0.020   1999.999
4   label  10000000        0.500        0.500      0.000      1.000

To train a model using out-of-core learning, we can use Vaex‘s incremental learning functionality. Here‘s an example training a logistic regression model:

from vaex.ml.incr import IncrementalPredictor
from sklearn.linear_model import SGDClassifier

features = [‘col1‘, ‘col2‘, ‘col3‘, ‘col4‘] 
target = ‘label‘

model = SGDClassifier(loss=‘log‘) 
incremental = IncrementalPredictor(model=model, features=features, target=target, batch_size=10000)

incremental.fit(df=df, progress=‘widget‘)

In this code, we:

1. Imported the IncrementalPredictor class and SGDClassifier
2. Specified the feature columns and target column
3. Created an SGDClassifier model with log loss
4. Wrapped the model in an IncrementalPredictor with a batch size of 10,000
5. Trained the model on the DataFrame in batches using fit()

The batch_size parameter controls how many rows are loaded into memory at a time during training. You can tune this based on your available memory.

To get predictions from the trained model on a new Vaex DataFrame df_new:

predictions = incremental.predict(df_new)

Vaex also provides many other features like joining, grouping, aggregating, and plotting to work with and visualize large datasets efficiently. Check out the documentation to learn more.

Other Tools for Out-of-Core Learning

In addition to Vaex, there are other powerful tools in the Python ecosystem for out-of-core learning and distributed computing:

• Dask: Provides advanced parallelism and out-of-core computation with a familiar NumPy and pandas interface
• PySpark: Python API for Apache Spark, a distributed computing framework
• Rapids: Suite of open-source libraries for executing end-to-end data science pipelines on GPUs

Each of these has its own strengths and use cases. Dask integrates well with the existing PyData ecosystem, PySpark is great for massive datasets and Hadoop environments, and Rapids can achieve significant speedups on Nvidia GPUs.

Conclusion

In this article, we explored two key techniques for handling large datasets in machine learning: subsampling and out-of-core learning.

Subsampling is a quick way to create a representative subset of a large dataset that fits in memory. It‘s useful for exploratory analysis and iterating on models. However, it can miss important patterns and doesn‘t help if even a fraction of the data is too large.

Out-of-core learning, on the other hand, allows training models on datasets that are much larger than available memory by loading and processing the data in chunks. It‘s slower than in-memory training but makes it feasible to work with huge datasets. There are several Python libraries like Vaex, Dask, and PySpark that enable out-of-core learning.

When faced with a large dataset, consider the following:

• If a representative sample of the data can fit in memory and provide sufficient accuracy, use subsampling
• If the entire dataset is needed or even a fraction doesn‘t fit in memory, use out-of-core learning
• If you have a cluster of machines, use distributed computing frameworks like Spark
• If you have GPUs, consider GPU-accelerated libraries like RAPIDS

The key is to understand your data, hardware, and requirements to choose the right approach. With the tools and techniques covered here, you‘re well-equipped to tackle massive datasets and build powerful machine learning models!