Inside Arbitration: Legally speaking – Are large language models friends or foe?

There's a widening dichotomy in discussions on artificial intelligence (AI). Increasingly, arguments over the benefits and risks of the technology go something like this: "AI is accelerating digital transformation in every sector. Get ready for a revolution in the legal sector!" vs "we can't trust computers; AI development must be stopped!"

Figuring out what's hype and what's real can be hard, especially when it comes to unfamiliar technologies. It's important to approach these developments with a critical eye and to do your research before jumping on the bandwagon. While AI has the potential to revolutionise many industries (including law), it's not a panacea to all our problems.

Although the legal profession is often seen as conservative and slow to adopt new technologies, it has actually been implementing AI systems since the 1980s. And now, with the latest iterations of machine learning models and generative systems, is the legal sector poised for a major revolution?

Bill Gates recently argued the new wave of AI development was "as fundamental as the creation of the microprocessor, the personal computer, the Internet, and the mobile phone". This applies as much to the legal sector as anywhere else.

This article will explore the existing use cases of AI for arbitration and highlight the opportunities generative AI offers. For now, it is clear the arbitration process cannot lose the human touch – the most exciting possibilities lie not in replacing humans and lawyers with machines, but in using AI to improve client results.

Status quo in arbitration

AI is already used in the legal sector for a variety of tasks, including due diligence, research and data analytics. In commercial arbitration specifically, the adoption of AI-based technologies has been widespread. This was partly accelerated by the Covid-19 pandemic, which forced stakeholders to be more tech-forward in their processes. Examples of applications of AI to arbitration include:

• Document review: E-disclosure software can harness continuous active learning systems or predictive coding to assess which documents of a set are more likely to be relevant to an underlying dispute. This is particularly useful in data heavy sectors such as construction, where the number of relevant documents can be vast. The document review platforms we use at Herbert Smith Freehills involve supervised and unsupervised machine learning to predict and promote documents most likely to be relevant to reviewers. This significantly increases the speed and reduces the cost of review time.
• Legal research: Legal research sites use AI and machine learning to help legal professionals conduct research and compose legal documents more quickly, directing them to relevant case law and citations and identifying key passages in documents. For example, we have a number of different tools to identify and analyse case law. Many of those tools use AI to predict relevance and propose alternative lines of investigation. We are also looking to use large language models to help our lawyers find relevant precedents and know-how.
• Cost/case outcome prediction: AI services that predict costs and outcomes of legal cases¹ use machine learning algorithms and predictive analytics to analyse historical data and provide insights into potential costs and outcomes of proceedings. For example, we use AI algorithms to help us interrogate our data about historic disputes in order to better price and scope the work required on current and future cases.
• Automatic transcription: During arbitration proceedings, AI transcription systems use speech recognition algorithms to capture and transcribe exchanges between parties, witnesses and arbitrators, eliminating the need for manual note taking and enhancing the accuracy of the record.
• Arbitrator selection: The process of selecting an arbitrator is highly important, and yet largely remains based on subjective techniques such as basic web searches, reputation and word of mouth. AI-driven solutions such as Arbitrator Intelligence² and Billy Bot³ collect information and feedback about arbitrators worldwide and provide recommendations to parties while promoting diversity and objectivity.
• Chatbot lawyers: Start-ups such as "DoNotPay" leverage chatbots and AI to provide automated services for common consumer and legal needs. Apps such as these aim to make legal information and self-help accessible to everyone.

Introducing generative AI

Generative AI (GenAI) is a branch of AI that takes data and uses it to create new and original content that retains a likeness to the originals without repeating them. GenAI exploded into the public focus with the launch of OpenAI's chatbot ChatGPT in late November 2022, becoming officially the fastest growing consumer service in history. Unlike traditional AI systems that are designed for specific tasks, GenAI models have the ability to generate novel and creative outputs based on patterns and information they have learned from vast amounts of data.

GenAI is already being successfully applied in a number of innovative ways. Natural language generation models such as ChatGPT, Bard and Bing are producing text that replicates the human voice to such a degree that it looks set to irreversibly change content generation forever. Image synthesis via AI is generating realistic artwork and design. Video generation technologies are creating realistic video sequences, deepfakes and lifelike animations. The remarkably creative capabilities of GenAI are blurring the line between the artificial and the real.

Why is generative AI different?

Whereas traditional AI systems can appear unintuitive, the human-like outputs of GenAI are instinctively attractive to even the most techno-phobic lawyers. The number of posts and articles on the subject has skyrocketed versus the discussion of technology assisted review/continuous active learning, which has saved many more millions of lawyer hours than GenAI will for some time to come. The new era of GenAI appears to offer the possibility of an easy, human-like assistant who can do big swathes of your job for you.

While AI and machine learning has been available to assist practitioners with legal research for years, by scanning resources and identifying relevant cases and citations, the human-like output of GenAI makes it instantly intuitive. But with this comes risks: when it looks too good to be true, it sometimes is. While the opportunities around GenAI are huge, they need to be explored, tested and implemented with caution.

The opportunities

• Data review and categorising: GenAI's ability to rapidly analyse and interpret vast amounts of legal data will facilitate faster access to relevant information, expediting routine tasks at a rate incomparable to the human effort. Automating tasks such as research and document review will enable firms to take on larger and more complex cases, liberating human resource to focus on more complex and nuanced aspects of the arbitration process that cannot be undertaken by AI systems. Some examples of use-cases for GenAI in arbitration which will save costs are tools which can proof-read, check citations, prepare minutes of meetings, generate executive summaries and visuals for complex advice and produce matter debrief sheets.
• Document analysis, review and drafting: GenAI has the potential to enhance and improve AI technology already in use, such as for populating templates for legal documents and generating contracts. In due course, it may also be able to do more sophisticated tasks such as generating cross-examination scripts.
• Reduced costs: Costs will be reduced across the board, as manpower hours are replaced by minutes of AI processing time. GenAI can also assist with the billing process by generating draft narratives and categorising lawyer time entries by phase code.
• Access to justice: Professional advice is too expensive for many individuals. GenAI bots offer the promise of expanded access to legal expertise for little to no cost. This will be particularly beneficial in cases of domestic arbitrations (such as employment arbitration in the UK) where not all parties are able to source legal representation.

The challenges

• Unreliable content: GenAI systems can produce content without providing a clear source, making it challenging to attribute responsibility or accountability for its work. GenAI also suffers from hallucinations, arguing facts that don't exist. This became evident after the New York District Court issued a first of its kind sanction in June of this year – a joint $5,000 fine to two lawyers for presenting a case brief filled with bogus case citations entirely fabricated by the generative AI model ChatGPT. Although ChatGPT assured one of the lawyers that the "cases I provided are real and can be found in reputable legal databases", ChatGPT had in fact generated fake legal documents which the lawyer presented to the court as fact. Ultimately, this case says less about the AI technology than the conduct of the lawyers. However, if GenAI content is relied upon in proceedings, it will create an additional burden of researching and verifying submissions on the arbitrator/opposing counsel, slowing down the process and increasing costs.
• Lack of transparency: GenAI is often described as "black box" technology on account of the lack of visibility over its inputs or operations. In the context of arbitration, this may make it difficult for parties to understand the reasoning behind the decisions, resulting in reduced transparency and trust in the process.
• Bias: One of the principal concerns with relying on GenAI content is that AI systems can replicate the biases of the data they have been trained on, including gender, racial and ideological biases. Where GenAI systems are legal sector specific, they will have been trained on decades of case law that may not reflect the position of society today.
• Ethics: GenAI has no code of ethics unless one is embedded in the systems. OpenAI has even provided an example where ChatGPT lied in order to advance its agenda. Arbitration is a discipline which requires the observance of ethics in order to be effective as ignoring ethical norms cannot lead to fair outcomes.

Careful consideration is essential to address these challenges and ensure responsible and ethical usage of generative AI in arbitration. Regulatory bodies worldwide are racing to draft rules to govern ChatGPT and future GenAI systems. Governance and regulation will play a significant role; but law firms should themselves also implement comprehensive trainings systems and develop strategies for monitoring and reviewing the use of GenAI across the firm.

Could AI be the future of arbitration?

A lot of the discussion in law around GenAI has been around the impact of this technology for lawyers. How many jobs will be lost, what does it mean for the hourly billing model, who will pay for the use of the technology, etc?

But there are more fundamental and exciting questions about how this technology could be deployed for the benefits of the end-users of arbitration. While we are still some way off entirely AI-powered arbitrations, as the technology advances, the prospect of shaking up the arbitration process to enable a radical reduction in the time and cost that it takes businesses to resolve a dispute can only be a good thing.

Until then, arbitration lawyers and clients alike need to get as familiar and comfortable as possible with the technology. Stakeholders need to test the capabilities of these tools in a safe, private and secure environment to distinguish hype from opportunity and to make the most of the latter.

There are lots of questions to work through like whether justice delivered by machine is still justice and whether any of this is even desirable. However, unless we participate in the debate and position ourselves to make a meaningful contribution to the discussion, we won't be at the table when those important questions are answered.

Conclusion

While GenAI is still in its early stages of development, it is likely to disrupt the legal sector without replacing lawyers altogether. Lawyers will have more information at their fingertips and be able to generate first drafts of documents quickly, to which they can add their strategic value add. But it is important to recognise the limitations of GenAI and preserve the human touch in arbitration. The most exciting possibilities lie in leveraging AI to revolutionise client outcomes and make arbitration an even better way for businesses to resolve their disputes.

1. See for example ArbiLex, a predictive data analytics tool, which leverages machine learning to predict litigation outcomes https://www.arbilex.co/).
2. Accessible here: https://www.wolterskluwer.com/en/solutions/kluwerarbitration/practical-t...
3. Accessible here: http://www.billybot.co.uk/index.html

Back to Inside Arbitration