Buy vs build was once a niche area of debate for law firms, but it has now taken on special importance in a world of LLMs. Why? Because LLMs are just so powerful and so adaptive to legal sector tasks. Which is the way to go? (And what do steam engines have to do with it?)
The subject also came up during Legal Innovators California last week when John Scrudato, Director of AI and Innovation at Latham & Watkins and Tanguy Chau, CEO and co-founder of genAI-focused Paxton AI were on a panel together.
Scrudato made the point that law firms could indeed build plenty of the AI tools they needed without having to rely on lots of point solutions from vendors to do those tasks. Chau responded that the challenge was that AI technology moves on rapidly and you need specialists on hand to stay up-to-date; if you build your own there was a risk the DIY product may not be as good as it could be and/or may soon be behind the state of the art.
[ Note: rather than give a blow-by-blow account of the Legal Innovators California conference – which was a fantastic two days with some of the leading experts at the cutting edge of legal innovation – AL will take certain key themes and explore them each week. ]
Steam Engines and LLMs
One way to look at this issue is through an historical lens and the arrival of steam engines. Back in the 1800s, entrepreneurial industrialists would allow people to rent out the steam engines they’d manufactured to use in a variety of ways. This is rather how OpenAI, for example, allows us to subscribe to its LLM ‘engines’ to power the tasks we wish to undertake.
A steam engine could be used to drain a mine, power a spinning loom, as well as provide the force needed to drive a river boat or a locomotive train. They were powerful and highly adaptable. A cloth factory owner might get one on subscription, or perhaps a transport business, it didn’t matter. The key aspect was that anyone who could pay for this engine could have one and then apply it to a task of their choice.
Back then people had the same challenges with regard to buy vs build. Should they just wait until they could get an entire, pre-made, fully integrated steam-driven locomotive, rather than cobbling together a separate steam engine with a chassis, wheels, carriages and more? If they did build an entire infrastructure that fitted that rented steam engine, would the whole system soon be out of date? But, even if that happened, wasn’t it better to have total control over what you wanted to build?
The same is true with LLMs. Should you just tap into GPT-4 via Azure and with some thin wrapper engineering on top build your own carefully system-prompted and refined tool, for example to review complex finance documents? Or, is it better to wait for a legal tech vendor to come along – also using GPT-4 and prompts and refinement, plus some workflows – to do what you were going to build?
Questions that may be asked include:
- Which approach will be more cost efficient?
- If we build it, will it stay up-to-date? And how much will it cost us to keep it up-to-date?
- Who is going to do all the refinement and prompting? Does that mean bringing in consultants or hiring new staff to handle that? How much will that cost?
- Even if we can do all the work internally, what happens if the team leader leaves the firm? Who is going to own this?
- And in a busy legal innovation team, given that a vendor might have a solution for this need, then why not just buy it and focus on other tasks? Why go DIY when it’s not 100% necessary?
In short, people have been pondering this type of issue since the Industrial Revolution and now we are at a stage where professional services businesses in the 21st century are having the same debate.
Do You Want To Do The Work?
For Artificial Lawyer, it seems to come down to a central question: do you want to do the work, and then do you want to become a product manager and owner for many years to come?
Given the mass of legal tech solutions out there – which seem to be propagating like rabbits since the arrival of LLMs – is there a need to go DIY? If not, then it may be a false economy, as you will need to maintain this product indefinitely if it is to have lasting value. That means time and money invested by the firm. Will that cost be more or less than what a vendor would charge, i.e. a vendor that will keep it updated for you?
Likewise, there will be use cases where a vendor is not available, or good enough for your needs. Then the question is: do we wait until there is one, go all DIY, or get a consultant to build one?
But, even if a consultant builds one for you, eventually you will still have to take on ownership of it, so you are back to the central issue of product management. Plus, consultants are not working for peanuts. In short, once again: do you want to take on this work, this ownership?
Conclusion
The reality is: there is no absolute answer. Firms with sufficient resources will likely experiment and build plenty of things on top of LLMs and then perhaps as new vendor products come along retire those DIY tools in the years to come. For them, the experimentation was worth it and they won’t feel bad about the expense.
In other cases, some firms will have such a niche need that vendors will take many years to cover it well, and so DIY is really the best option. Also, it may even give some firms a competitive edge and / or allow them to do things for clients that would have been hard to do if dependent just on a vendor’s tool. I.e. where a firm builds a one-to-many tool for a niche group of clients, tapping internal experience and making something that is super-customised.
But, as noted, it all comes down to: do you want to put in the effort to build, to test, to maintain, to manage, to update and integrate?
Clearly, it can often be easier to buy off the peg, especially when the use case is a popular one and the tools out there are excellent. As you move to more and more specialised, niche capability needs then DIY makes more and more sense, in fact it can even be a route to build tools that create new revenue for the firm.
Ultimately, the decision comes down to what type of innovation team you have and the level of buy-in and support from the partners. If they want to be entrepreneurial and experiment and invest, then the world’s your LLM oyster. If they prefer to keep things simple and pared down to what is only necessary, then DIY is likely not going to happen.
Both approaches are valid, and many firms will do some of both at the same time, depending on the use cases, the level of appetite for expenditure of time and effort, the willingness to take ownership, and the desire to innovate on your own terms.
We have been having this debate since the 1800s, no doubt it will keep on going, whether it’s LLMs or steam engines.
Richard Tromans, Founder, Artificial Lawyer, June 2024