AVBytes: The Week‘s Top AI & ML Advances – GPT-4 Powers Apps, Diffusion Models Generate Videos, and More

The world of artificial intelligence and machine learning continues to evolve at breakneck speed. Hardly a week goes by without major new developments from tech giants, leading academic labs, and AI startups across the globe. Keeping up with it all can feel like a full-time job.

That‘s why we started AVBytes – to give you a quick, curated take on the week‘s most important and exciting developments in AI and ML. This past week brought a flood of fascinating work, from the latest language models and multimodal systems to new approaches for few-shot learning, neural rendering, and embodied AI.

Here are some of the top stories and research papers that caught our eye over the last seven days. Be sure to subscribe to get AVBytes in your inbox every week!

GPT-4 Powers New Applications

Leading the pack this week was OpenAI‘s highly-anticipated release of GPT-4, the latest iteration of their groundbreaking large language model. Building on the success of ChatGPT and GPT-3, GPT-4 achieves human-level performance on a range of professional and academic benchmarks.

Even more impressive are the new capabilities GPT-4 enables. It can take visual inputs, allowing it to analyze images, generate captions, and even interpret diagrams and infographics. This has spawned a new wave of multimodal apps leveraging GPT-4‘s intelligence.

One striking example is Microsoft‘s announcement that GPT-4 will power the new Bing search engine. For the first time, web search will go beyond just retrieving relevant links to actually engaging in dialog, answering followup questions, and assisting with open-ended tasks. It‘s a major leap towards making search engines more intelligent and interactive.

Other notable GPT-4 apps announced this week include:

• Duolingo MAX, a new AI-powered language learning system
• Stripe‘s GPT-powered business intelligence and analytics tools
• Anthropic‘s Constitutional AI model built using GPT-4 as the base
• Casetext‘s CoCounsel legal assistant for analyzing documents

Microsoft and Google also unveiled new enterprise platforms for developing custom AI applications using their large language models. Between Azure Cognitive Services and Google Cloud‘s Vertex AI, companies of all sizes will be able to leverage cutting-edge NLP to build powerful language-based features.

All of these rapid developments underscore that the age of language models as a service has truly arrived. Over the coming months, expect GPT-4 and its siblings to enable a Cambrian explosion of new AI-powered products and experiences.

Diffusion Models Take Video Generation to the Next Level

While large language models stole many of this week‘s headlines, there were equally exciting advances in computer vision and generative AI.

One key theme was the impressive gains in high-resolution video generation using diffusion models. Google, Meta and others released new models that can generate coherent multi-minute videos from text prompts alone.

Google‘s Phenaki model in particular produced jaw-dropping results, including dynamic scenes, smooth camera movements and consistent character appearances. Meanwhile, Meta‘s Make-A-Video 2 supports stylized video generation across diverse domains like cartoons and paintings.

Video diffusion models open up exciting new possibilities for content creation, special effects, gaming, and virtual worlds. The ability to automatically animate characters and scenes with this level of fidelity was previously out of reach. Soon, game designers, filmmakers, and other creatives may be able to replace laborious CGI and motion capture pipelines with AI-powered tools.

Of course, these models also raise challenging ethical questions around issues like copyright, misinformation, and explicit content. As the technology becomes more accessible, it will be crucial to develop safeguards and norms around responsible development and deployment.

Few-Shot Learning Leaps Forward

Another exciting area of research this week involved few-shot learning – the challenge of learning new tasks from only a handful of examples.

Traditionally, deep learning has required massive labeled datasets to work well, limiting its applicability to data-scarce domains. But several new approaches are starting to close this gap impressively.

One key development was DeepMind‘s Adaptive Posterior Learning (APL) framework, which enables pretrained language models to rapidly adapt to new tasks using Bayesian inference. On benchmarks like SuperGLUE and ANLG, APL matched or exceeded leading techniques while using up to 16x less data.

Google also released MultiBERTs, a method for pretraining BERT models that improves few-shot performance on named entity recognition with as few as 10 labeled examples per entity type. By sharing parameters across related tasks, MultiBERTs learn more data-efficient representations.

For vision tasks, UC Berkeley researchers introduced PEERS, a framework for few-shot classification via prompting and reweighting the hidden representations of pretrained vision transformers. PEERS outperforms prior methods in the one-shot and five-shot settings on multiple benchmarks.

As few-shot capabilities improve, AI systems will be able to flexibly learn a greater range of tasks in the wild without expensive data collection or model retraining. This could make AI far more practical and scalable for long-tail and open-ended applications.

Robotics Simulate to Real World

Applying machine learning to robotics has historically been challenging due to the expense of real-world data collection and the difficulty of modeling complex physical dynamics. But researchers are starting to bridge the gap between simulation and reality more effectively.

This week, Google released RGB-Stacking, a simulated benchmark for vision-based robotic manipulation. By training agents in a high-fidelity simulation using reinforcement learning, Google was able to develop policies that transfer directly to real robot arms with minimal finetuning.

RGB-Stacking includes 50 diverse tasks like palletizing boxes, stacking bowls, and packing objects into corners. The benchmark provides a valuable test bed for studying where sim-to-real transfer works well and where it still struggles.

NVIDIA also announced Isaac Gym 2.0, a major upgrade to its open-source toolkit for photorealistc robotics simulation. The new version features improved physics, PBR rendering, and support for large-scale multi-robot domains. It also integrates tightly with NVIDIA‘s Omniverse platform for collaborative 3D simulation.

Armed with these better simulation tools, researchers and roboticists will be able to train more capable and robust systems before deploying them in the physical world. Over time, this could greatly accelerate the development of robots for manufacturing, logistics, service, and other impactful industries.

The Road Ahead for Responsible AI

With the rapid advancement of powerful AI systems, there is growing focus on ensuring they are developed and deployed in safe and beneficial ways. This week brought several notable developments on that front:

• The White House Office of Science and Technology Policy released an updated version of the Blueprint for an AI Bill of Rights, outlining principles for protecting civil liberties in an AI-driven world

• 200 tech leaders signed an open letter calling for a 6-month pause on training AI systems more powerful than GPT-4 to allow time for safety research and policy development

• The EU Parliament passed the AI Act, a comprehensive regulatory framework for AI that bans some applications and requires greater oversight for high-risk systems

• OpenAI, Google, Anthropic and others launched the Frontier Model forum to discuss best practices for developing large AI models responsibly

While there is still much uncertainty and debate around the path forward, it‘s encouraging to see both industry and policymakers taking proactive steps to get ahead of the challenges posed by advanced AI. Efforts to improve transparency, robustness, controllability and value alignment will be crucial to realizing the technology‘s positive potential while mitigating downside risks.

Onwards and Upwards

From language and vision to robotics and policy, this week demonstrated that AI progress is not slowing down anytime soon. The pace and scale of new breakthroughs is truly staggering.

At the same time, much work remains to translate these advances into real-world impact in a responsible and beneficial way. As more powerful models emerge and find their way into products and services that touch our daily lives, it will be crucial for the AI community to come together to shape the technology‘s trajectory.

One thing is clear though – there has never been a more exciting time to be working in AI and ML. The opportunities ahead are immense, and we can‘t wait to see what the coming weeks and months bring. Be sure to subscribe to AVBytes to stay up to date!