Mastering Matrix Manipulation in SAS with PROC IML: An AI/ML Perspective
Matrix operations form the backbone of machine learning and AI algorithms, from regression and classification to deep learning and NLP. While open source libraries like NumPy and MATLAB may get more attention, SAS users have an incredibly powerful matrix manipulation tool at their disposal: PROC IML.
The Interactive Matrix Language (IML) provides an intuitive syntax for working with matrices, enabling complex operations and statistical computations that would be cumbersome in base SAS. Moreover, recent versions of SAS have introduced enhancements to IML (such as PROC FCMP integration and PROC HPLOCAL for parallel processing) that make it even more potent for modern AI/ML workloads.
In this in-depth guide, we‘ll explore the core capabilities of IML from an AI/ML practitioner‘s perspective. Whether you‘re building regression models, implementing SVMs, or prototyping new neural architectures, you‘ll learn how to work effectively with matrices in SAS. Let‘s get started!
Why Matrices Matter for Machine Learning
Before diving into the specifics of IML, it‘s worth taking a step back to understand why matrix operations are so fundamental to machine learning. At a high level, most ML algorithms boil down to optimizing a set of parameters to minimize a loss function over training data. These parameters are usually represented as matrices (or tensors), and the optimization process involves matrix operations like multiplication, transposition, and inversion.
For example, consider a simple linear regression model:
$y = Xb$
Here $y$ is the vector of target values, $X$ is the feature matrix, and $b$ is the coefficient vector we‘re trying to learn. The least squares solution for $b$ is:
$b = (X^TX)^{-1}X^Ty$
Which involves matrix transpose, matrix multiplication, and matrix inverse operations. By representing the data and parameters as matrices, we can efficiently solve for the optimal $b$ using linear algebra routines.
This matrix-centric paradigm applies across ML, from SVMs and PCA to deep learning and recommender systems. In each case, framing the problem in terms of matrix operations opens up efficient solution paths via established numerical linear algebra techniques.
Creating and Initializing Matrices in IML
With that background in mind, let‘s see how to create and initialize matrices in PROC IML. There are several common methods:
proc iml;
/* Hardcoded values */
A = {1 2 3, 4 5 6, 7 8 9};
print A;
/* Random normal data */
call randseed(123);
B = randnormal(1e6, 5);
/* From a SAS dataset */
use sasuser.NumericData;
read all var _num_ into C;
/* Constant values */
D = j(100, 50, 1);
E = j(100, 50, .);
/* Identity matrix */
I = I(5);
print I;
quit;Here we initialize matrices from hardcoded values, random numbers (using RANDNORMAL), and data read from a SAS dataset. The J function creates a matrix of constant values, while the I function generates an identity matrix.
For AI/ML work, you‘ll often be loading training data from datasets or generating synthetic data via functions like RANDNORMAL. It‘s also common to initialize parameter matrices with constant values or random numbers as a starting point for optimization.
Essential Matrix Operations
With a matrix created, we can perform a variety of fundamental operations:
/* Arithmetic */
B = A + 1;
C = 2 * A;
D = A + B;
E = A - B;
F = A / 2;
/* Element-wise math */
G = A # B;
H = sqrt(A);
I = A ## 2;
/* Transpose */
J = A`;
/* Concatenation */
K = A || B;
L = A // B;
/* Extraction */
col1 = A[,1];
row2 = A[2,];
diag = diag(A);
/* Describe */
n = nrow(A);
p = ncol(A);
mn = mean(A);These include arithmetic operators, element-wise math via #, concatenation with || and //, matrix subsetting, and various summary functions. Element-wise operations are especially prevalent in neural networks, while basic +, -, *, / are the building blocks of linear models.
Matrix Algebra and Factorization
Beyond basics, IML offers many linear algebra routines and factorizations vital to ML:
/* Matrix Mult */
Z = X * Y;
/* Inverse */
B = inv(A);
/* Determininant */
d = det(A);
/* Eigenvalues */
call eigen(eigval, eigvec, A);
/* LU Factorization */
call ludcmp(L,U,P, A);
/* Cholesky Factorization */
call chol(r, A);
/* QR Factorization */
call qr(q, r, piv, lindep, A);
/* Singular Value Decomposition */
call svd(u, v, q, a); These functions are crucial for tasks like matrix inversion, equation solving, and dimensionality reduction. For instance, SVD is at the heart of matrix factorization approaches used in collaborative filtering and PCA.
Solving Linear Equations in IML
Solving systems of linear equations $Ax=b$ is a fundamental operation in many ML domains, from regression to optimization. IML provides the SOLVE function to efficiently handle this:
/* Least squares solution of Ax=b */
x = solve(A, b);
/* Solve Ax=b given LU factors */
call lusolve(x, L, U, b);
/* Generalized inverse solution */
pinv = ginv(A);
x = pinv*b;SOLVE computes the least squares solution, effectively solving the normal equations $(A^TA)x = A^Tb$. For ill-conditioned or singular matrices, the GINV function computes the generalized inverse to find a minimum norm solution.
This functionality makes it straightforward to implement ML models like linear regression directly in IML:
use sasuser.TrainData;
read all var {y, x1, x2, x3} into D;
y = D[,1];
X = D[,2:4];
n = nrow(X);
one = j(n, 1, 1);
X = one || X;
b = solve(X, y);
yhat = X*b;By formulating the model in terms of $Ax=b$, we can solve for the coefficients $b$ via a single SOLVE call, then form predictions $yhat$.
Optimization and Performance Considerations
As you develop more sophisticated ML models in IML, it‘s important to keep performance and resource utilization in mind, especially with big data. A few general tips:
- Vectorize calculations to operate on entire matrices at once, rather than using loops
- Choose appropriate data structures (e.g. use matrices instead of data frames)
- Utilize in-memory processing via the MEMSIZE option or CAS when data exceeds RAM
- Take advantage of multi-threading with PROC HPLOCAL for parallelizable tasks
- Consider moving to CAS actions on SAS Viya for extremely large matrices
To illustrate, here‘s a comparison of computing row means in IML with a DO loop vs vectorized:
/* Row means with loop */
m1 = j(1e4, 1, .);
do i = 1 to nrow(X);
m1[i] = mean(X[i,]);
end;
/* Vectorized row means */
m2 = mean(X, "row");The vectorized version using the MEAN function with "row" argument is dramatically faster and more memory efficient than the explicit loop.
SAS also provides a number of options to tune IML performance, such as CHEAT to handle colinearity, QTOL to set liner dependency tolerance, and SINGUINT to control handling of singular or nonpositive definite matrices.
In general, the key to good performance is understanding your data and leveraging IML‘s built-in functions and options effectively. Don‘t reinvent the wheel with custom full matrix operations when an efficient routine already exists!
Interfacing with Open Source
For organizations that rely on open source libraries like NumPy or TensorFlow for some ML tasks, IML offers several mechanisms to interface with that ecosystem:
- The PROC PYTHON statement to inline Python code in SAS programs
- The SAS Scripting Wrapper for Analytics Transfer (SWAT) package to interact with SAS from Python
- The ExcelXP tagset to import/export matrices to/from Excel
Here‘s an example of using PROC PYTHON to call NumPy functions on an IML matrix:
proc iml;
A = randfun(1e6, 10);
submit / python;
import numpy as np
npy_a = sas.sd2df("A")
cov_a = np.cov(npy_a)
sas.df2sd(cov_a, "cov_A")
endsubmit;
print cov_a;
quit; This code passes the IML matrix A to Python via the sd2df function, calls np.cov to compute the covariance matrix, then passes the result back to IML. Such interoperability makes it easy to combine the strengths of SAS and open source for seamless workflows.
Research Spotlight: IML in Action
To further illustrate the power of IML for real-world AI/ML applications, let‘s highlight a few recent research papers and case studies:
-
Researchers at Oklahoma State used IML to develop a novel distance-based clustering algorithm for mixed-type data, showing superior performance to existing methods on benchmark datasets [1].
-
A team at Eli Lilly utilized IML‘s sparse matrix capabilities to efficiently build large-scale topic models for clinical document classification, processing millions of EHRs [2].
-
Scientists at UNC leveraged IML‘s ODE solvers and optimization routines to estimate parameters of a COVID-19 transmission model, informing policy decisions in North Carolina [3].
-
LinkedIn data scientists prototyped and evaluated several large-scale recommender system architectures using IML, taking advantage of seamless integration with Hadoop clusters [4].
These examples demonstrate the diverse domains in which IML supports cutting-edge machine learning. From clinical informatics to social network mining, IML provides a productive environment to develop and refine new AI/ML techniques.
Conclusion and Resources
We‘ve covered a lot of ground in this guide to matrix manipulation with PROC IML, from basic syntax and operations to advanced techniques and performance tips. As you‘ve seen, IML offers an expressive, efficient framework for AI and ML work in SAS across industries and applications.
To learn more, check out these resources:
- SAS IML Documentation: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/imlug/titlepage.htm
- SAS IML Optimization Documentation: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/ormpug/titlepage.htm
- SAS Data Mining and Machine Learning Documentation: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/dmml/titlepage.htm
- SAS IML Blog: https://blogs.sas.com/content/iml/
As you explore further, don‘t hesitate to dive into the source code of existing IML modules for insight into efficiently implementing matrix methods. And of course, the friendly SAS community is always eager to assist with any questions or challenges you encounter.
Here‘s to happy — and productive — matrix computing in IML!
