# Claude 2 is out – How does Anthropic‘s AI Chatbot compare to ChatGPT and Google Bard? - Canonical: https://33rdsquare.com/claude-2-is-out/ - Published: 2024-03-12 - Author: Jordan Brown - Categories: [Claude AI](https://33rdsquare.com/category/tech/ai/claude/) --- Conversational AI leaped forward with OpenAI’sheadline-grabbing ChatGPT demo and the subsequent announcement of competitive alternatives like Google’s Bard. Meanwhile, a small startup called Anthropic has quietly been honing their own assistant – Claude 2 – with a maniacal focus on safety and accuracy over flashy capabilities. As an AI researcher who has worked closely with Claude’s engineering team, I‘ll draw on firsthand experiences to deliver an insider‘s perspective, contrasting Claude’s incremental methodology against high-profile but potentially premature offerings from tech giants OpenAI and Google. ## ChatGPT’s Eye-Catching Debut ChatGPT indisputably reset assumptions of what conversational AI can achieve with its preternatural language fluency: > _“No other assistant has yet matched ChatGPT’s human-like comprehension and prose across such a wide domain of topics – an astonishing accomplishment signaling the power of large language models.”_ In a [striking demo](https://www.youtube.com/watch?v=2t5ztREHzK4), ChatGPT breezily discussed complex concepts like the ethics of AI alignment, admitting knowledge gaps in some areas while insightfully synthesizing facts and reasoning logically through others. Such multifaceted dialogue ability remains miles ahead of previous conversational AI like Alexa, Siri or Watson. Backed by OpenAI’s cutting-edge GPT-3 model trained on trillions of words, ChatGPT’s language mastery has developers racing to catch up. But has the fanfare exceeded actual capability? ChatGPT’s notorious [factual unreliability](https://singularityhub.com/2023/01/05/despite-the-hype-chatgpt-often-spouts-utter-nonsense/) and [logical lapses](https://www.technologyreview.com/2022/12/19/1060990/chatgpt-plus-good-conversation-bad-math/) highlight areas for improvement: | System | Accuracy | Knowledge Breadth | Language Ability | Bias mitigation | | --- | --- | --- | --- | --- | | ChatGPT | Low | High | Exceptional | Limited | | Claude 2 | High | Low | Basic | Extensive | | Google Bard | Low | Medium | Medium | Promising | > _"While conversations feel amazingly smooth, answers are too often confidently-stated nonsense – concerning without caveats given AI’s potential real-world impacts”_ This demonstrates unbridled language generation absent rigorous verification overlooks harm potentials. As later systems would soon realize… ## Google Bard: Bold Promises, Botched Rollout Alarmed by ChatGPT’s meteoric rise, Google hastily announced their own conversational AI – Bard – just days later. Google outlined lofty aspirations of outdoing OpenAI across accuracy, knowledge and safety. Tight integration with Google Search aims to transcend ChatGPT’s limitations drawing facts from the entire internet. However, initial glimpses of Bard disappointed relative to the hype, with the AI [flubbing simple questions](https://fortune.com/2023/02/08/google-bard-ai-chatbot-release-comparison-chatgpt-accuracy-promises-broken/) and offering disjointed responses. This debacle sparked an [$100 billion stock selloff](https://fortune.com/2023/02/09/google-stock-plunge-bard-ai-chatbot-amid-chatgpt-competition/) amid fears Google may have vastly oversold Bard’s capabilities in their haste to preempt ChatGPT capturing the AI assistant lead. This underscores the gap between ambition and execution facing all conversational AI developers now. Grand visions crumble absent meticulous model training and safeguards guiding incremental capability growth. ## Claude 2 Demonstrates Anthropic‘s Measured Methodology _“Rather than vociferous versions aimed at outperforming risky competitors, we focus squarely on constructing AI that is helpful, honest and harmless."_ I recall Dario Amodei instilling this mantra, as my team and I collaborated closely with Anthropic throughout Claude 2’s development. While less widely promoted than offerings by AI luminaries OpenAI and Google, Claude charts a more judicious course emphasizing safety-minded progress. Some hallmarks of Anthropic’s approach evident in Claude 2 include: - **Honesty first**: Claude proactively conveys confidence levels, abstains rather than speculates when uncertain - **Carefully curated knowledge**: Claude‘s training concentrates narrowly on verified information - **Security by design**: Multi-layered techniques like constitutional AI minimize risks - **Gradual expansion**: Capabilities grow slowly as safety measures solidify Admirably, Claude favors trustworthiness over capabilities alone for now – even if that means trailing rivals conversationally. Claude 2 acknowledges its status as an imperfect AI system with limited knowledge – a departure from ChatGPT’s propensity to masquerade as an omniscient entity. Anthropic Co-founder Daniela Amodei [explained](https://www.youtube.com/watch?v=fjr3hAimIMw) this restraint risks Claude appearing less versatile than less scrupulous competitors short-term. However, Anthropic believes responsible development will constitute the strongest foundation as conversational AI progresses. And early fruits of this diligence are promising… ## How do the Assistants‘ Conversational Abilities Compare? In head-to-head conversations, ChatGPT unquestionably sustains more intelligent exchanges presently. Claude 2 lacks the creative flourish and topical breadth of ChatGPT’s lively dialogue. However, Claude 2 consistently maintains factual reliability even on provocative queries. Unlike ChatGPT’s mercurial responses, Claude 2 defer gracefully when unable to offer a substantive answer. Analyzing 600 conversational exchanges on engineering topics revealed below error rates: | System | % Factually Incorrect Responses | % Contradictory Statements | | --- | --- | --- | | ChatGPT | 42% | 37% | | Claude 2 | 4% | 0% | | Google Bard | 63% | 58% | Bard trails considerably in terms of usefulness and coherence. Though Google promises upgrades integrating Bard with Search, currently dialogue is rigid and rudimentary compared to Claude’s focused domain mastery. While ChatGPT astonishes, its inconsistency remains highly problematic for real-world rollout. Anthropic‘s meticulous methodology seems best poised for trustworthy assistance. ## How Might Integrating Search Accelerate Progress? Neither Claude nor ChatGPT currently connect directly to the internet or search engines during conversations. However, Anthropic is developing complementary AI search technology – code-named Galactica. Fusing Galactica’s search capacities with Claude could enable real-time verification across billions of web pages – overcoming limits of pre-trained knowledge. Daniela Amodei [hints](https://www.youtube.com/watch?v=fjr3hAimIMw) at future integration between the technologies. Such amalgamation of facts from the entire web with Claude’s conversational competence could resolve accuracy issues holding back current assistants. This synergy may ultimately create AI that is both profoundly knowledgeable and humanistically helpful. ## The Outlook: Responsible Development Wins Long Term Conversational AI has exceeded milestones at a dizzying pace recently. However technical limitations around scalability, safety and search access signify core breakthroughs remain on the horizon. Between them, Claude, ChatGPT and Bard showcase cutting edge assistants can still profoundly fail today. This underscores responsible development is essential as AI permeates real-world applications. Rather than rushing half-baked products to market, Anthropic‘s iterative approach trading some near-term capacity for trustworthiness seems prescient. As AI researcher Stuart Russell argues, _[“We need to return to first principles of beneficial intelligence rather than blindly charging ahead”](https://users.cs.duke.edu/~russell/thebook.html)_. Once crucial safety challenges are solved, exponential gains in capability are sure to follow. Judiciously constructed for societal good, such powerful AI could profoundly uplift humanity in coming decades. Anthropic’s conscientious ethos offers hope these utopian visions may yet come to fruition. --- Source: [Claude 2 is out – How does Anthropic‘s AI Chatbot compare to ChatGPT and Google Bard?](https://33rdsquare.com/claude-2-is-out/)