Don’t believe the h(AI)pe?
In the legal industry (and no doubt other service industries) the received wisdom is that you can achieve two out of three things when doing a job for the client: do it quickly, do it well, do it cheaply. I was reminded of this the other day when discussing recent surveys by Deloitte (The future of legal work?) and Thomson Reuters (Future of professionals), both focusing on legal technology and in particular generative AI. The Deloitte survey canvassed ‘senior legal leadership’ at 43 of its ‘closest and largest clients’ and provides interesting insights into what those organisations are thinking and doing. The Thomson Reuters survey canvassed professionals in legal, tax, and accounting, receiving more than 2,200 responses.
Something I am always curious about is, while there is a belief generative AI will change things and that it should make things quicker, better, cheaper, it is not always clear how and why people believe this. For example:
- In the Deloitte report, 79% of respondents said they expected generative AI to have ‘moderate to significant long-term impact on how legal work is carried out’. That was despite the fact that 76% of its respondents reported ‘no current adoption of the technology’, although with the caveat that they were expecting some level of adoption in the next 2-3 years.
- Deloitte’s respondents also expect external law firms to adopt generative AI, and that it will produce benefits for clients (the prevailing expectations being that it will reduce costs or speed up turnaround time). However only 3 – 6% were seeing these benefits.
- A huge number of Thomson Reuters’ respondents (77%) believed AI would have a ‘high or transformational impact on their careers’. However, many professionals cannot articulate how to convey the benefits of use of AI to clients, with 46% saying they were ‘not very confident or not confident at all about articulating this’.
- Amongst those of Thomson Reuters’ respondents who use the technology, a fair number (28%) felt the output was ‘a strong starting point’ they would ‘just need to edit’, although most (50%) felt it to be a basic starting point where the user would still need to do ‘the majority of the work’, and 7% found the output to be poor.
- Interestingly, while pricing is one of the key areas where clients hope to see improvements, respondents to Thomson Reuters survey thought efficiencies would prompt a move away from pricing by reference to time taken and towards pricing the value of the work.
My guess is that a lot of people are searching for the Holy Grail: a tool that can respond quickly, accurately, and briefly to the question “Can we do this?”, which may get sent to in-house (or external) counsel late in a transaction, and all for less than the current legal spend. Despite all the hype and hope around generative AI this is not something that exists. If it did, most companies would be champing at the bit to replace their external lawyers with it.
Coincidentally, the same day these two surveys were discussed I also came across a study on the use of AI by police departments for report writing carried out by Ian T. Adams, Matt Barter, Kyle McLean, Hunter Boehme, and Irick A. Geary (No Man’s Hand: Artificial Intelligence Does Not Improve Police Report Writing Speed). This trial looked at whether AI tools reduced the time police officers spent writing reports. The findings were straightforward: use of AI did not result in any statistically significant reduction in time taken to create the reports. There were a few things that jumped out at me from this study:
- One reason put forward as a possible explanation for the lack of impact was that the reports were already largely based on templates. The aspects that were bespoke to a particular incident required substantial input from the officers involved, based on their experience of the particular incident. Use of templates may have already created substantial time savings and therefore only incremental savings were being made by using the new technology.
- The officers were reminded that the reports were for them to review, correct and issue under oath, and that the AI’s work could not be relied upon.
- Notwithstanding its findings, it did acknowledge that there might be other benefits to use of AI which remain untested, such as consistency of reporting or improved quality.
It is easy to see similarities between this study on the policing context and use of this technology in the legal context. Use of templates and precedents is the norm for law firms, as is a requirement that a firm stands behind its work. There is also the bigger picture point of marginal gains. For example, law firms also have the benefit of reference materials that succinctly and accurately summarise the law for them – why would they pay for something that is new, untested, and less reliable? Similarly, clients have been pushing for lower fees for longer than I have been a lawyer, and the side effect of that has been that firms embrace software solutions where they can, for example document automation for first drafts and cross-checking, or software to assist in disclosure exercises, and even for checking cross-references – the key question being do they work overall? When it comes to judgement calls and decision making, which is really what clients want from their advisors, the prevailing view seems to be that this is not within the capability of the technology, nor will it be any time soon.
I suspect it will take some time before we really know whether there are generative AI tools that will be able to deliver the Holy Grail of fast, cheap, and good. In the meantime, I wonder if most law firms might more wisely be asked (and consider) whether they are fully utilising existing software tools that may be less flashy but are more mature and more reliable.
Subcribe to news and viewsAlmost 70% of all professionals said that a higher or the same portion of work matters will be brought in-house within the next five years, compared to today.
https://www.thomsonreuters.com/content/dam/ewp-m/documents/thomsonreuters/en/pdf/reports/future-of-professionals-report-2024.pdf
