AVBytes: The Week‘s Top AI & ML News – Microsoft Achieves Human Parity in Machine Translation, Google‘s Latest Open Source Models, and More

Welcome back to another edition of AVBytes, your weekly roundup of the most noteworthy artificial intelligence and machine learning news. This week, advances in natural language processing take center stage, led by Microsoft‘s announcement that its NLP models have achieved human parity on machine translation tasks. We‘ll dive into the details behind this milestone and explore other significant NLP breakthroughs this week.

But that‘s not all that happened in the world of AI. From Google open sourcing powerful new deep learning models to AI being deployed for mental health and journalism, there‘s a lot to unpack. So let‘s get started with this week‘s top stories!

Microsoft‘s Machine Translation Models Reach Human Parity

In arguably the biggest AI news of the week, Microsoft announced that its machine translation models have reached human parity when translating news articles from Chinese to English. This means Microsoft‘s AI can translate text with quality that matches human translators.

To achieve this milestone, Microsoft researchers developed a deep learning model with dual learning and deliberation. The model uses two different approaches – a generative approach that tries to predict the next word, and a discriminative approach that uses attention to focus on relevant parts of the source text. The two components work together through multiple passes to refine the translations.

What‘s especially remarkable is the model was trained on a dataset of only 2000 sentences, which were translated by professional human translators. On this limited training data, Microsoft‘s model matched the average BLEU score (a common metric for evaluating translation quality) of the human translators. This demonstrates the power of the dual learning approach to perform well even with small datasets.

With this breakthrough, Microsoft takes a major step towards eliminating language barriers and making information accessible to everyone regardless of what language they speak. While machine translation still has a ways to go to match humans in more general scenarios beyond news articles, Microsoft‘s achievement shows the rapid progress happening in this space. I believe we‘ll continue to see NLP systems narrow the gap to human-level language understanding in the coming years.

DeepL and Baidu Release New Translation Models

Microsoft isn‘t the only company making waves in machine translation this week. DeepL, a German company known for its high-quality translation services, released a new NLP model called DeepL-52. Trained on over 50 billion parameters, it claims to outperform other top public translation models like Google Translate and Amazon Translate.

Meanwhile, Chinese tech giant Baidu open sourced ERNIE 3.0 (short for "Enhanced Representation through kNowledge IntEgration"), a massive model with 260 billion parameters trained on 4TB of clean data. ERNIE 3.0 achieves state-of-the-art results on a range of Chinese NLP tasks and enables few-shot learning in several domains like news classification, sentiment analysis, and natural language inference. This demonstrates the continued progress and democratization of large language models.

As someone who works with multiple languages, I‘m excited to try out these new translation models and APIs. Having access to high-quality, low-cost translation unlocks opportunities for everything from business to entertainment to academia. And as training data and compute power grows, I expect we‘ll continue seeing rapid improvements in cross-lingual AI systems.

Google Makes Music and Art with Machine Learning

Switching gears from language to multimedia, Google had a couple interesting announcements this week showcasing creative applications of machine learning:

First, Google released a new version of Magenta, an open source library for music generation with deep learning. Magenta 2.0 includes a new model called MusicLM that can generate high-fidelity music based on text descriptions like "smooth jazz with saxophone solo."

Google also open sourced the model behind the Pixel phone‘s portrait mode feature, which uses a neural network to add background blur to photos. By releasing the TensorFlow code on GitHub, Google enables developers to experiment with depth-based effects and potentially create new mobile camera features.

As someone interested in the intersection of AI and arts, I always geek out over projects like these. They demonstrate how machine learning can be a powerful tool for creative expression and inspire new forms of human-AI collaboration. At the same time, they raise important questions about data biases, artistic ownership, and the limits of AI creativity that will be important to grapple with as these systems advance.

AI for Mental Health and Journalism

This week also saw a couple examples of AI being applied for social good:

In the mental health space, a chatbot app called Woebot, which uses NLP to provide cognitive behavioral therapy, raised $90 million to expand its services. Backed by AI luminaries like Andrew Ng, Woebot aims to make mental health support more accessible and affordable. While it‘s not a replacement for human therapists, AI tools like Woebot can serve as a helpful supplement or entry point for those struggling with issues like anxiety and depression.

Meanwhile, Reuters announced it is developing an AI tool to help journalists analyze big datasets and spot newsworthy patterns. By quickly processing massive troves of data, the AI can surface leads for potential stories that human reporters can then investigate further. Reuters hopes this human-machine teaming will make investigative journalism more efficient without sacrificing quality.

As an AI practitioner, I‘m heartened to see these real-world applications aimed at improving well-being and uncovering truth. While there are certainly risks and limitations to be mindful of, I believe AI will increasingly be used as a positive force across healthcare, media, and other domains. The challenge will be ensuring these systems are designed and deployed responsibly, with input from diverse stakeholders.

AI for Improving Devices

Finally, Google provided a glimpse into how it‘s using AI to improve hardware devices over time. Researchers developed a deep learning pipeline to continuously optimize a phone‘s antenna signal by learning from user data. Deployed on Pixel phones via over-the-air updates, the system has improved median signal strength by nearly 20%.

This points to a future where our devices actually get better with age, rather than slowing down as hardware degrades. With more compute power shifting to the cloud, I can envision all sorts of ways that AI models could personalize and optimize everything from battery life to touch responsiveness to sound quality.

It‘s easy to take for granted how much "smarts" already goes into the devices we use every day, but this is really just the beginning. As federated learning frameworks and on-device AI becomes more sophisticated, I expect we‘ll see all kinds of products, from phones to cars to smart home appliances, that evolve and adapt to our individual needs over time. This is an area I‘ll be watching closely.

Closing Thoughts

From language to music to hardware, AI continued its relentless march forward this week. Microsoft‘s human parity milestone in particular stood out as a significant step towards machines that can understand and communicate in language as well as humans. But this is just one example of the rapid progress happening across the field, as this week‘s other announcements from Google, DeepL, Baidu, and others demonstrate.

At the same time, I believe it‘s important to maintain a balanced perspective. While the pace of breakthroughs can seem dizzying at times, today‘s AI systems are still narrow in scope and there‘s much work to be done to make them more robust, interpretable, and aligned with human values. Responsible development of AI will require ongoing collaboration between researchers, engineers, policymakers, and the broader public.

Still, weeks like this make me optimistic about the positive potential of AI to expand human knowledge and capabilities in incredible ways. I‘m grateful to play a small part in this fast-moving field, and I look forward to seeing what new milestones are on the horizon. Until next time, thank you for reading and if you found this valuable, please consider subscribing to receive these roundups every week!