As technology advances, the media is full of stories about how robotics and artificial intelligence will soon replace humans in many professions. Testing this theory, a recent study by LawGeex set a machine-learning AI against 20 human lawyers to see how the technology would perform on the subject of contract law.
The US-based corporate lawyers had decades of experience in corporate law and contract review. The group comprised of law firm associates, sole practitioners, in-house lawyers, and General Counsel. Many of them worked at highly respected companies including Goldman Sachs and Cisco, and global law firms including Alston & Bird and K&L Gates.
Their mission was to spot problems in five Non-Disclosure Agreements (NDAs), a standard contract in most business deals. The layers were given four hours to study the contracts.
It took the lawyers an average of 92 minutes to complete the task. The longest time taken by a human lawyer was 156 minutes, and the shortest time was 51 minutes. However, the AI managed the work in just 26 seconds.
Crucially, while the lawyers achieved an average accuracy rate of 85%, the AI came out top again and was 94% accurate. The AI’s accuracy score tied with the highest scoring lawyer.
An independent panel of law professors judged the accuracy of the test.
The good news for humans is that the results don’t mean that lawyers are going to be obsolete anytime soon. Following the study, the lawyers reportedly had no fear of AI taking their jobs. Instead, the participants highlighted that high-volume and low-risk contracts took up too much of their time, and that automating work could make their jobs easier.
Zakir Mir, former regional counsel for BDP International, now at Allegiance International, and one of the participants in the study said: “It is crucial to make mundane contract work more efficient, especially when there are 50-100 pages of contracts for some major deals (M&A large tenders with agreements or multinational corporations). It can really help lawyers sift through these documents, and cut down on the sometimes-deliberate verbosity of these documents which can allow one party to mask core issues.”
Another participant, Deja Colbert, a contracts administrator at Omega Rail Management added that she found: “AI logical and credible for the purpose of reducing contract review time, thus, allowing prioritising other time-consuming tasks at hand.”
What do you think, is AI a threat or a benefit to the legal profession? Should law firms be looking to embrace this technology more in the future?