AI Shatters Speed and Accuracy Records in Reviewing Contracts, but Won‘t Replace Lawyers Anytime Soon

Introduction

In a landmark study, an artificial intelligence (AI) system developed by legal technology company LawGeex has beaten human lawyers at reviewing contracts both in terms of speed and accuracy. The AI achieved an average accuracy rate of 94% across five non-disclosure agreements (NDAs), compared to an average of 85% for the 20 experienced U.S. corporate lawyers who participated. Even more strikingly, the AI completed the review in just 26 seconds on average, while the human lawyers took 92 minutes on average.

This represents a watershed moment for the potential of AI to transform the $437 billion U.S. legal services industry by automating routine and time-consuming tasks like contract review. As AI technology continues to advance rapidly, it could help make legal services more efficient, affordable, and accessible. However, AI still has significant limitations and is unlikely to replace human lawyers entirely anytime soon.

How the LawGeex AI Works

At the core of the LawGeex AI are advanced natural language processing (NLP) and machine learning techniques that allow it to understand and analyze legal contract language. The system was trained on a dataset of tens of thousands of NDAs that had been annotated by legal experts to identify common clauses, key terms, and potential issues.

Through this training process, the AI developed its own complex statistical models of the patterns, semantics, and structures that typify NDAs. It can now take a new contract, break it down into discrete clauses and concepts, classify their types and functions, and evaluate them based on learned criteria like sentence length, word choice, and clause combinations.

Some of the specific NLP techniques used include:

• Tokenization to break the text into individual words and punctuation marks
• Part-of-speech tagging to identify the grammatical role of each word
• Named entity recognition to extract key terms like party names, jurisdictions, and effective dates
• Dependency parsing to map the syntactic structure of each sentence and the relationships between words
• Word embeddings to represent the semantic meaning of words and clauses in high-dimensional vector space

The LawGeex AI then applies a battery of machine learning classifiers and regressors to analyze each aspect of the contract based on its training data. This includes logistic regression to identify problematic clauses, support vector machines to score the overall risk level, and deep learning neural networks to identify clause boundaries and categorize clause types.

Finally, a hierarchical attention network architecture is used to assign different weights to each clause and section of the contract based on its learned importance to the overall meaning and risk profile. This allows the AI to surface the most salient issues for human review while still considering the entire contract in its proper context.

Implications for the Legal Profession

The LawGeex study has significant implications for the economics and practice of law. Currently, contract review is a major cost center for legal departments and a time sink for many lawyers. Large corporations like Microsoft and Cisco process tens of thousands of contracts per year, and even smaller businesses often have a steady flow of NDAs, vendor agreements, and other routine contracts.

Automating this kind of high-volume, low-complexity work could save businesses millions in legal fees while also freeing up lawyers to focus on more strategic and high-value tasks. According to the study, the human lawyers took an average of 92 minutes to review each NDA, billing at an average rate of $400 per hour. That translates to over $600,000 in legal fees per year for a company that processes just 1,000 NDAs annually.

The efficiency gains from AI could be especially disruptive for the many junior associates and paralegals who currently spend much of their time on routine document review and other mundane tasks. A 2016 McKinsey report estimated that 23% of a lawyer‘s job could be automated with existing technology. As AI improves, that percentage is likely to rise, potentially displacing tens of thousands of legal support roles.

However, AI is not a perfect substitute for human lawyers, and its use in the legal domain raises a number of challenges and risks. One major concern is the potential for algorithmic bias based on the data used to train the AI. If the training contracts come mostly from large enterprises in certain industries or jurisdictions, the AI may not perform as well on contracts from smaller businesses or different sectors and regions.

There are also questions about transparency and interpretability. Unlike human lawyers, AI systems cannot explain their reasoning or point to specific case law to justify their conclusions. This "black box" problem could make it difficult for lawyers to assess the reliability of AI recommendations or for judges to determine their admissibility in court.

Additionally, AI systems are only as good as the data they are trained on, and they can struggle with edge cases, ambiguity, and changing circumstances. For example, an AI trained on pre-pandemic contracts may not account for new force majeure clauses or risk factors introduced by COVID-19.

For these reasons, most experts believe that AI will augment rather than replace human lawyers in the near term. Lawyers will increasingly rely on AI assistants to streamline their work and provide data-driven insights. But they will still need to exercise professional judgment, provide nuanced advice, and build client relationships.

Over time, AI could help close the access to justice gap by making certain routine legal services more affordable and accessible to underserved populations. But it will be important to ensure that AI systems are designed and deployed in an equitable and ethical manner.

Other Applications of AI in Law

Beyond contract review, AI is being applied to a wide range of legal use cases, from e-discovery and due diligence to predictive analytics and legal research. Some examples include:

• Litigation prediction: Companies like Lex Machina and Ravel Law use machine learning to analyze past court decisions and predict the likely outcomes of new cases based on factors like the judge, jurisdiction, and case type. This can help lawyers decide whether to settle or litigate and how to craft their legal strategies.

• Legal research: AI-powered tools like ROSS Intelligence and Casetext can quickly search through millions of cases, statutes, and secondary sources to find relevant authorities and answer specific legal questions. This can save lawyers countless hours of manual research and help them uncover key insights faster.

• Contract management: Beyond just reviewing individual contracts, AI can also help businesses manage their entire contract lifecycles from drafting and negotiation to execution and renewal. Tools like Kira Systems and Evisort can automatically extract key data points from contracts, monitor compliance with obligations, and alert users to expiring deals or changing regulations.

• Intellectual property: AI can help automate many aspects of intellectual property law, from analyzing patent applications for novelty and non-obviousness to detecting trademark and copyright infringement online. Companies like TrademarkNow and Anaqua use machine learning algorithms to streamline IP filing, search, and protection processes.

• Compliance and risk management: By analyzing vast troves of structured and unstructured data, AI can help businesses identify potential legal and regulatory risks before they materialize. Companies like Merlon Intelligence and Ascent RegTech use natural language processing and machine learning to monitor employee communications, financial transactions, and regulatory filings for signs of fraud, misconduct, or non-compliance.

As these examples illustrate, the potential applications of AI in law are vast and varied. And as the technology continues to improve, it will likely be applied to even more legal domains and use cases over time.

Preparing for the Future of Legal Work

To stay competitive in an increasingly AI-driven legal market, lawyers will need to adapt their skills and knowledge for the 21st century. This means not only learning how to use AI tools effectively but also understanding how they work under the hood.

Just as lawyers today need to be proficient with legal research databases and e-discovery software, the lawyers of tomorrow will need to be conversant in data science, machine learning, and natural language processing. They will need to be able to assess the capabilities and limitations of AI systems, interpret their outputs, and communicate their implications to clients and colleagues.

This will require a significant shift in legal education and training. Law schools will need to incorporate more technology and data analytics courses into their curricula, and law firms will need to provide ongoing training and development opportunities for their attorneys.

At the same time, the legal profession will need to grapple with the ethical and societal implications of increasingly autonomous and opaque AI systems. As AI takes on more decision-making roles in areas like bail setting, sentencing, and legal aid allocation, it will be important to ensure that these systems are fair, transparent, and accountable.

Lawyers will also need to navigate the changing economics of legal services in an AI-enabled world. As more routine work is automated, the value proposition of many traditional legal services will likely decline. Lawyers will need to focus more on high-touch, high-value work that requires human judgment, creativity, and empathy.

This could accelerate the trend toward alternative legal service providers and new business models like legal subscription services, online dispute resolution platforms, and AI-powered legal marketplaces. It could also widen the gap between elite global law firms that can afford to invest heavily in AI and smaller regional firms that may struggle to keep up.

Conclusion

The LawGeex study is a remarkable demonstration of the potential for AI to transform the legal industry by performing certain tasks faster and more accurately than human lawyers. As AI continues to advance, it will likely be applied to an ever-wider range of legal use cases, from contract drafting and review to legal research and predictive analytics.

However, AI is not a replacement for human lawyers, and its use in the legal domain raises a number of technical, ethical, and economic challenges. Lawyers will need to learn how to effectively leverage AI tools while also understanding their limitations and implications.

The future of legal work will likely involve a hybrid model where humans and machines collaborate closely to deliver more efficient and effective legal services. Lawyers who can combine their deep legal expertise and judgment with the speed and scalability of AI will be well-positioned to thrive in this new era.

But to get there, the legal profession will need to undergo a significant transformation in terms of education, training, and business models. Law schools, law firms, and legal service providers will need to adapt to the changing technological landscape and the evolving needs of clients.

Ultimately, the goal should be to harness the power of AI to make legal services more accessible, affordable, and equitable for all. By augmenting human legal expertise with machine intelligence, we can create a more just and inclusive legal system that better serves the needs of society as a whole.