The Best of ICLR 2018: Groundbreaking Advances in Machine Learning and AI

Introduction

The International Conference on Learning Representations (ICLR) is one of the world‘s premier gatherings for experts in machine learning, deep learning, representation learning, and artificial intelligence. Each year, the best and brightest minds in these rapidly evolving fields come together to share their latest research breakthroughs, exchange ideas, and discuss the most pressing challenges and opportunities that lie ahead.

ICLR 2018, which took place from April 30 to May 3 in Vancouver, Canada, was a groundbreaking event that showcased the incredible progress being made at the frontiers of AI and deep learning. The conference received an unprecedented 935 paper submissions—nearly double the number from 2017—out of which 337 were accepted. This is a testament to both the exploding interest in these fields and the exceptional quality and impact of the work being done.

In this post, we‘ll take a closer look at some of the most influential and thought-provoking papers presented at ICLR 2018. We‘ll dive into the key insights and innovations they offer, and explore what their findings could mean for the future of intelligent systems and the many domains they touch. We‘ll also highlight some of the dominant themes and trends that emerged, and share our perspective on where the field may be headed in the years ahead.

On the Convergence of Adam and Beyond

One of the Best Paper Award winners at ICLR 2018 was "On the Convergence of Adam and Beyond" by researchers at Google Brain. This work takes a critical look at adaptive gradient methods like Adam, which have become the default optimization algorithms of choice for deep learning. While powerful, the authors show that Adam can fail to converge to an optimal solution in some settings.

The paper does a deep theoretical analysis of why this happens. In simple terms, the exponential moving average of past squared gradients, which is a key component of the Adam update rule, can overshoot the optimal step size and lead to non-convergence in certain cases, especially for non-convex problems. The authors construct an example of a simple convex optimization problem where Adam provably fails to reach the global optimum.

More importantly, they don‘t just identify issues, but propose a new variant of Adam, called AMSGrad, that has guaranteed convergence to an optimal solution. AMSGrad keeps a current maximum of past squared gradients rather than an exponential average, which the authors prove ensures convergence. They show empirically that AMSGrad matches regular Adam‘s performance in standard deep learning settings, while improving robustness and stability in some cases.

This work highlights the importance of building our optimization methods on firm theoretical foundations, even as we push the limits of scale and complexity in deep learning. The convergence guarantees of classical stochastic gradient descent shouldn‘t be sacrificed for adaptive step sizes without careful analysis. At the same time, it shows there‘s still room for improvement in the core algorithms underlying modern AI systems.

Spherical CNNs

Another Best Paper was "Spherical CNNs" from the University of Amsterdam, which brings the power of convolutional neural networks (CNNs) to data represented on the sphere. While CNNs have been spectacularly successful for analysis of planar images and video, their application to inherently spherical data—like 360° photos/video or signals measured on the globe—has been limited.

The authors present an elegant mathematical framework for building spherical CNNs. The key insight is to swap out the standard translational convolutions and replace them with rotations on the sphere and convolutions in the spherical harmonic domain. They show how to define spherical convolutional layers, pooling, and other key building blocks.

To demonstrate the practical utility of this approach, the authors apply spherical CNNs to 3D object classification tasks. They show substantial improvements over previous methods, achieving over 99% accuracy on the SHREC‘17 benchmark. Even more impressive are the computational efficiency gains—spherical convolutions reduce the model size by an order of magnitude compared to conventional 3D approaches based on voxel grids.

The potential applications of spherical CNNs are immense. They could power more intelligent drones and autonomous vehicles by enabling efficient 360° perception. They could open up new frontiers in scientific domains like climate/weather modelling and computational chemistry. At a more fundamental level, they extend the hugely successful paradigm of deep learning to data that lives naturally on a sphere—an important step towards bringing learning representations closer to the underlying geometry of the world.

Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments

A third spotlight paper at ICLR 2018 was "Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments", a collaboration between UC Berkeley, OpenAI, UMass Amherst, and CMU. This work tackles the challenge of creating AI systems that can continuously learn and adapt in dynamic environments, an essential step towards truly autonomous agents.

The key idea is to leverage meta-learning—training a model that can itself learn quickly from limited experience. Specifically, the authors propose a gradient-based meta-learning algorithm called Optimization via Evolutionary Strategies (ES). The algorithm alternates between two phases: 1) collecting experience by rolling out multiple copies of the current policy in parallel, and 2) updating the policy parameters using ES to maximize the average return over the rollouts. Intuitively, this allows the model to learn a high-level strategy for adapting to new situations.

To test their approach, the authors introduce a new multi-agent competitive environment called RoboSumo, where the goal is to push the opponent out of the ring. The dynamics of the environment change over time, as agents update their strategies in response to each other, making it a challenging domain for continuous adaptation.

The results are impressive. Meta-learning enables much faster adaptation than traditional RL methods, achieving high reward in just a few adaptation steps. The learned adaptation strategy is also robust to the choice of opponent policy. Visualizations of the learned behavior show that the meta-trained agents develop sophisticated strategies like faking out the opponent with deceptive movements.

This work points to an exciting path forward for developing AI systems that can learn continuously and adapt rapidly in ever-changing real-world settings. It brings together ideas from meta-learning, evolutionary strategies, and multi-agent competition in a novel and effective way. As we look towards deploying autonomous agents in complex, open-ended environments, the ability to adapt on the fly will be increasingly critical.

Other Highlights and Themes

The three papers above are just a small sampling of the many groundbreaking works presented at ICLR 2018. Other notable papers explored topics like:

• Unsupervised learning of disentangled representations
• Generative models for program synthesis and induction
• Theoretical connections between deep learning and physics/dynamical systems
• Applications to healthcare, robotics, and computer vision

Some of the key trends and themes that emerged:

• A growing emphasis on unsupervised and self-supervised learning as a path towards richer, more generalizable representations
• Increased interest in hybrid approaches that combine deep learning with classical AI techniques like tree search or logic
• Applying deep learning to an ever-expanding range of domains, from medical diagnosis to materials design to automatic theorem proving
• A push towards building theoretical frameworks to understand when and why deep learning works (or doesn‘t)

Conclusion

ICLR 2018 provided an exhilarating glimpse into the future of AI and machine learning. The works presented demonstrated the field‘s astounding rate of progress—in the span of a few years, we‘ve gone from struggling to classify images to strategizing agents that adapt in real-time, mathematical frameworks for deep learning on spherical manifolds, and optimization methods with provable guarantees.

At the same time, they also underscored the magnitude of the challenges still ahead. We are far from having AI systems that can learn and reason in a truly human-like way, with the kind of flexible intelligence that generalizes seamlessly across tasks and domains. Achieving this will likely require fundamental breakthroughs in areas like unsupervised learning, transfer learning, causal inference, and common sense reasoning.

The path forward is uncertain, but there are many reasons for optimism. The researchers at ICLR and beyond are pushing the boundaries of what‘s possible with awe-inspiring dedication and ingenuity. By bringing together the world‘s brightest minds and providing an open platform for collaboration and discussion, venues like ICLR are accelerating the pace of discovery and innovation.

As we look ahead to ICLR 2019 and beyond, one thing is clear: the future of AI has never been brighter. We can‘t wait to see what groundbreaking advances the coming years will bring, and how they‘ll transform our world for the better. The revolution has only just begun.