Smart Investing with AI: Applying Modern Portfolio Theory in Python

In the fast-evolving world of finance, investors are always seeking an edge to maximize their returns while keeping risk in check. For over half a century, Modern Portfolio Theory (MPT) has provided a rigorous mathematical framework for constructing optimal portfolios. By harnessing the power of diversification, MPT aims to build portfolios that offer the highest expected return for a given level of risk.

While the underlying concepts were groundbreaking in their time, the rise of artificial intelligence and machine learning is bringing new possibilities to the MPT paradigm. With vast computing power and advanced algorithms, we can now apply the core principles of MPT at a scale and sophistication unimaginable to its pioneers.

In this post, we‘ll dive into the foundations of Modern Portfolio Theory and explore how to implement it in practice using Python. We‘ll cover the key mathematical concepts, walk through detailed examples, and discuss how AI and ML can supercharge the MPT workflow. Whether you‘re a seasoned quant or just starting your journey in quantitative finance, this guide will equip you with the tools and knowledge to build robust, AI-driven investment portfolios. Let‘s get started!

The Essence of Modern Portfolio Theory

At the heart of Modern Portfolio Theory is the insight that risk and return are inextricably linked. Developed by Nobel laureate Harry Markowitz in the 1950s, MPT provides a way to quantify this relationship and build portfolios that optimally balance the two.

The key realization is that the risk and return of a portfolio depend not only on the individual assets it contains, but also on how those assets interact with each other. This is captured by the covariance matrix, which measures the degree to which assets move together.

Mathematically, the expected return of a portfolio is the weighted average of the expected returns of its constituent assets:

$E(Rp) = \sum{i=1}^N w_i E(R_i)$

where $w_i$ is the weight of asset $i$ in the portfolio, and $E(R_i)$ is its expected return.

The variance of the portfolio‘s return, a measure of its risk, is given by:

$\sigmap^2 = \sum{i=1}^N \sum_{j=1}^N w_i wj \sigma{ij}$

where $\sigma_{ij}$ is the covariance between assets $i$ and $j$.

The goal of MPT is to find the set of weights that minimizes the portfolio‘s variance for a given level of expected return, or equivalently, maximizes the expected return for a given level of variance. This is a quadratic optimization problem that can be solved using techniques like Lagrange multipliers or numerical methods.

The result is the efficient frontier, the set of optimal portfolios that dominate all others in terms of risk and return. Investors can then choose a portfolio on the frontier that aligns with their risk tolerance and return objectives.

Implementing MPT in Python

Now that we‘ve covered the conceptual foundations, let‘s see how to put MPT into practice using Python. We‘ll walk through a step-by-step example of constructing an optimal portfolio from a universe of stocks and ETFs.

We‘ll start by importing the necessary libraries and reading in our price data:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import yfinance as yf
import scipy.optimize as sco

# Fetch price data from Yahoo Finance
tickers = [‘SPY‘, ‘AGG‘, ‘GLD‘, ‘AAPL‘, ‘AMZN‘, ‘MSFT‘, ‘GOOG‘, ‘FB‘]
data = yf.download(tickers, start=‘2015-01-01‘, end=‘2021-12-31‘)
prices = data[‘Adj Close‘]

Next, we‘ll calculate the expected returns and covariance matrix of our assets. To get a more stable estimate, we‘ll use returns over a trailing window rather than the full sample period.

# Calculate returns
returns = prices.pct_change()

# Trailing 3-year returns
window = 3 * 252 # trading days
mu = returns.iloc[-window:].mean() * 252
S = returns.iloc[-window:].cov() * 252

# Annualized risk-free rate 
rf = 0.015

We can visualize the risk-return profiles of our individual assets along with their correlations:

fig, ax = plt.subplots(1, 2, figsize=(16, 6))
fig.suptitle(‘Asset Risk-Return Profile‘, size=16)

# Returns vs volatility
ax[0].scatter(np.sqrt(np.diag(S)), mu, marker=‘o‘, s=50)
for i, txt in enumerate(mu.index):
    ax[0].annotate(txt, (np.sqrt(S.iloc[i,i]), mu.iloc[i]), fontsize=12)
ax[0].set_xlabel(‘Volatility‘, size=14)    
ax[0].set_ylabel(‘Return‘, size=14)
ax[0].grid(True)

# Correlation heatmap
sns.heatmap(returns.corr(), cmap=‘coolwarm‘, annot=True, fmt=‘.2f‘, annot_kws={"size":10}, ax=ax[1])
ax[1].set_title(‘Asset Correlations‘, size=14)

plt.tight_layout()
plt.show()

To find the efficient portfolios, we‘ll set up and solve the optimization problem. Our objective is to minimize the portfolio variance subject to constraints on the weights and expected return.

def portfolio_stats(weights, mu, S, rf):
    ‘‘‘
    Calculate expected return, volatility and Sharpe of a portfolio
    ‘‘‘
    weights = np.array(weights)
    exp_ret = weights.T @ mu
    volatility = np.sqrt(weights.T @ S @ weights) 
    sharpe = (exp_ret - rf) / volatility
    return exp_ret, volatility, sharpe

def minimize_volatility(target_return):
    ‘‘‘
    Minimize portfolio variance subject to a target return
    ‘‘‘
    init_guess = len(mu) * [1 / len(mu)]
    bounds = ((0.0, 1.0),) * len(mu) 

    constraints = ({‘type‘: ‘eq‘, 
                    ‘fun‘: lambda weights: np.sum(weights) - 1},
                   {‘type‘: ‘eq‘,
                    ‘fun‘: lambda weights: weights.T @ mu - target_return})

    results = sco.minimize(lambda weights: portfolio_stats(weights, mu, S, rf)[1],
                           init_guess,
                           method=‘SLSQP‘,
                           bounds=bounds,
                           constraints=constraints)
    return results

# Generate 50 target returns between min and max individual assets 
target_returns = np.linspace(mu.min(), mu.max(), 50)

# Solve for efficient portfolios
efficient_portfolios = []
for target_return in target_returns:
    results = minimize_volatility(target_return)
    exp_ret, vol, sharpe = portfolio_stats(results.x, mu, S, rf) 
    efficient_portfolios.append({‘volatility‘: vol,
                                 ‘return‘: exp_ret, 
                                 ‘sharpe‘: sharpe,
                                 ‘weights‘: results.x})

# Consolidate results in a DataFrame
efficient_portfolios = pd.DataFrame(efficient_portfolios)    

# Plot efficient frontier
plt.figure(figsize=(12,8))
plt.scatter(efficient_portfolios.volatility, 
            efficient_portfolios[‘return‘],
            marker=‘x‘,
            c=efficient_portfolios.sharpe,
            cmap=‘viridis‘)
plt.colorbar(label=‘Sharpe Ratio‘)
plt.xlabel(‘Volatility‘)
plt.ylabel(‘Return‘)
plt.title(‘Efficient Frontier‘, size=16)

# Plot individual assets
for asset in mu.index:
    ret = mu.loc[asset]
    vol = np.sqrt(S.loc[asset, asset])
    plt.scatter(vol, ret, marker=‘o‘, s=100)
    plt.annotate(asset, (vol, ret), fontsize=12)

# Plot Capital Allocation Line
plt.plot([0, efficient_portfolios.volatility.max()], 
         [rf, efficient_portfolios[‘return‘].max()],
         color=‘red‘,
         linestyle=‘--‘,
         lw=2,
         label=‘CAL‘)
plt.legend()
plt.show()

The efficient frontier shows the set of optimal portfolios for different levels of risk and return. The capital allocation line (CAL) starts at the risk-free rate and is tangent to the frontier at the tangency portfolio, which has the highest Sharpe ratio among all efficient portfolios.

We can identify this portfolio in our results:

# Find tangency portfolio
optimal_port = efficient_portfolios.iloc[efficient_portfolios.sharpe.idxmax()]
print(‘Optimal Portfolio:‘)
print(f‘Volatility: {optimal_port.volatility:.4f}‘) 
print(f‘Return: {optimal_port["return"]:.4f}‘)
print(f‘Sharpe: {optimal_port.sharpe:.4f}‘)
print(‘Weights:‘) 
print(pd.Series(optimal_port.weights, index=mu.index).round(4))

Optimal Portfolio:
Volatility: 0.0927
Return: 0.0804
Sharpe: 0.7546
Weights:
SPY     0.0972
AGG     0.3673
GLD     0.0469
AAPL    0.1192
AMZN    0.0321
MSFT    0.1673
GOOG    0.0611
FB      0.1089
dtype: float64

Enhancing MPT with AI and Machine Learning

While the classic MPT framework is powerful on its own, the advent of AI and machine learning opens up new frontiers for portfolio optimization.

One of the key inputs to MPT is the vector of expected returns. Traditionally, these are estimated from historical data, assuming that past performance is indicative of future results. However, we know that markets are dynamic and returns can be influenced by a host of economic, behavioral and sentiment factors.

This is where AI can come in. By training machine learning models on a wide range of data – from fundamental financial ratios to alternative data like news sentiment and satellite imagery – we can develop more nuanced and adaptive forecasts of asset returns.

For example, we could use a Long Short-Term Memory (LSTM) neural network to predict returns based on historical price sequences. Or we could apply gradient boosting to a combination of lagged returns, valuation multiples and economic indicators.

Once we have our AI-enhanced return forecasts, we can plug them into the MPT optimizer to generate forward-looking efficient portfolios. This quantamental approach, blending the best of quantitative finance and fundamental analysis, has the potential to uncover alpha that would be missed by purely statistical methods.

Another area where AI can augment MPT is in risk modeling. The classic framework assumes that asset returns are normally distributed and that volatility and correlation are constant over time. In reality, we know that returns can exhibit fat tails, asymmetry and regime changes.

Advanced machine learning techniques like Gaussian mixture models, hidden Markov models and conditional volatility models can help us capture these dynamics and build more realistic risk estimates. By incorporating these into our portfolio construction process, we can create portfolios that are more resilient to market shocks and tail events.

Beyond return and risk modeling, AI can also help with the optimization process itself. The traditional quadratic programming approach can struggle with large, complex problems involving many assets and constraints. AI-based solvers like evolutionary algorithms and reinforcement learning can navigate these high-dimensional spaces more efficiently and find global optima that elude classical methods.

Finally, explainable AI methods can help make the quantitative portfolio construction process more interpretable and transparent. By understanding which features drive the AI models and how they impact portfolio weights, we can build trust with stakeholders and ensure our strategies are robust and economically intuitive.

Conclusion

Modern Portfolio Theory provides a rigorous framework for constructing optimal portfolios that balance risk and return. By diversifying across assets with imperfect correlations, we can build portfolios that are more efficient than any individual asset.

Python and its rich ecosystem of libraries make it easy to implement MPT in practice. From fetching financial data to solving optimization problems, Python provides the tools to build end-to-end quantitative investing workflows.

But the real game-changer is the integration of artificial intelligence and machine learning techniques. By harnessing the power of AI for return forecasting, risk modeling and optimization, we can take MPT to new heights and build portfolios that are adaptive, resilient and forward-looking.

Of course, AI is not a panacea. It‘s crucial to combine the quantitative power of machine learning with the domain expertise and economic intuition of human investors. The most successful quantamental strategies will be those that leverage the strengths of both human and artificial intelligence in a symbiotic way.

Ultimately, the future of investing belongs to those who can marry the time-tested principles of Modern Portfolio Theory with the cutting-edge capabilities of AI and machine learning. By standing on the shoulders of giants like Markowitz while embracing the innovations of the 21st century, we can build smarter, more efficient portfolios that help investors navigate an increasingly complex financial landscape.