It’s gone totally wild on the news front this week because of the impending Nashville festival (see more below), but we start with news that Harvey has laid out the performance results of Tenet, its first post-trained open-weight model.
In a detailed analysis, the fast-growing legal AI platform explained that they’ve been focused over the past six months on two goals: ‘Building frontier legal intelligence using open-weight models; and creating systems to allow law firms to build their own specialized models and own their intelligence’.
And now part of the fruits of that labour is Harvey Tenet, which is a Kimi K3 base LLM that they post-trained together with Fireworks research for long-horizon legal work – and by that they mean setting agents on complex legal tasks with the hope they can get from start to finish without going astray.
This latter aspect is really what the future of the legal market is all about…but really the future. You can see how hard it is to do well here by looking at the performance data below across a range of benchmarks. The one that AL noticed most was Harvey’s own Legal Agent Bench – which is a tough assault course, based on the views of experienced lawyers as to whether an agent is capable of handling a complex legal task end to end. So, not a single prompt, or a simple database search, but acting just as a lawyer would on a multi-part task. I.e. this is really, really difficult to do right.
Now, those figures are low, 20% or lower, but then you have to remember how incredibly significant it will be once those numbers track upwards. And they will.
In the discussion with Zach below (see video), there’s an exploration of why, despite so much investment into legal AI, there is so little systemic change – at least in Big Law.
One possible reason is that law is still very much human to human, and AI itself is still ‘in day care’ and needs lots of lawyer supervision. If we can get agents up to around 95% on these benchmarks then we are in a whole different world. This is like moving from driving assist features – which is where we are now with legal AI, to approaching Waymo ‘self-driving’ levels of trust and accuracy.
And yes, the supposed untouchable ground of judgment would also be absorbed into this. Judgment is not sacred, AI can do it. It’s just a question of handling complexity, context, and enough relevant data. But, back to Tenet, what does this tell us?
Well, as AL covered before, TR is developing its own open weight model, Thomson, and they believe it’s done very well in terms of accuracy. In their case they’ve had a ton of legal data to help train it. Kirkland also appears to be trying to do this with their own data, at least for certain practice areas, with the help of Palantir.
In this case, Harvey’s Tenet, they don’t really spell out specific firms or inhouse teams that may have, or may not have, contributed data. In their paper, they state that: ‘We collaborated closely with Mercor and others to build and scale human expert datasets and for review and remediation of synthetically-generated data, which made training at this scale possible.’
It’s unclear if that data involved legal work (naturally anonymized) from those customers which wanted to help the experiment, or if it was developed internally by Harvey’s own many skilled lawyers, which it has hired over the years. They also used synthetic data to extend the collection of training information.
And they added there were cost savings on tokens as well.
So, is this a big deal? Yes. Why? For AL, aside from cost, or another aspect, it seems to be moving forward to reaching higher capabilities for legal agents that are critical to any chance of legal AI really becoming a game-changer.
At present, there is so much human oversight that AI remains trapped within the professional, artisanal ‘harness’ (to use an AI training term). Until agent performance hits such a high level that buyers of legal services absolutely trust the results on complex legal work alone, then we cannot move that far forward, at least within the confines of the Big Law current model.
In short, this may help with Harvey’s own internal economics, and it shows what can be done perhaps around AI sovereignty for law firms that want to build their own open weights models, but for AL the real prize is the incremental improvement in agentic performance. We are moving in the right direction. Congrats to Harvey and the team that worked on this, which included: Calvin Qi, Vasudha Rengarajan, Julio Pereyra, Niko Grupen, and Gabe Pereyra.
More about Tenet here.
–
OK and now for a news wrap – and wow… is there a lot of developments getting announced at the moment! Therefore, AL has super-summarised things. Hopefully after Nashville things will return back to normality.
- Hotshot, the learning platform for lawyers, has partnered with Traveling Coaches. They’re also working with Litera now.
- AI startup Space has announced a $2.4m pre-seed for its streaming filesystem, using zero disk space on computers allowing humans and agents to access petabytes of live data instantly. (Not a legal tech story, but it connects to it….)
- XTND Technologies, the billing and collections platform for insurance defense law firms, and Clio, have announced a strategic partnership.
- Legatics and Syntheia have also partnered to combine document comparison with Legatics’ deal platform.
- DISCO has enhanced its Auto Review solution to make ‘prompt iteration faster and easier and help teams refine their own AI review instructions without prompt-engineering expertise’.
- Truth Systems, the AI governance company for regulated industries, has launched Charter Insights, which gives law firms ‘a unified view of how AI is being used across their people and the platforms they rely on’.
- Harbor has found that: ‘Average annual law firm software spending rose 41% between 2021 and 2025, and overall technology spending now represents roughly 5% to 6% of firm revenue. Despite that scale of investment, zero participating firms could demonstrate a consistent way to measure whether AI is delivering business or client value.’
- Opus 2 has launched Ingenuity, the company’s agentic AI framework. Ingenuity runs on ‘the millions of insights’ generated in the Opus 2 platform. ‘In addition to retrieval and generative AI layers, Opus 2 Ingenuity includes separate layers of specialist AI agents that are applied underneath and on top of a matter record (structured data layer), keeping every case accurate by construction and every team reasoning from the truth,’ they explained.
- Clarra, a case management platform has formed an integration with Claude Cowork.
- And finally, Epiq Advisory and Legora have formed a partnership. Together, they will ‘enable corporate legal departments and law firms to deploy enterprise AI that delivers greater impact across legal workflows’.
So, there you go. And that’s not even all the news that was sent into AL in just the last couple of days. Next week is going to be just as busy as ILTACon kicks off.
And on the theme of Nashville, AL just wanted to recommend one restaurant. If you can manage to escape the Ikea-scale ILTACon hotel (which is strongly recommended) and get out into the balmy air of Tennessee, then book a table at Puckett’s in town. AL had a couple of meals there and at each sitting consumed well over 25,000 calories (just kidding)….but it felt like that. Here’s a couple of photos.
If you come from a culinary world where you don’t have grits, gumbo, and deep-fried everything, then this is the place to come for a taste of something very, very different! Enjoy…in moderation.
—
Right, that’s the food bit done. What next, ah yes, an excellent adventure. If you ever watched ‘The Big Short’ or wondered about Jevon’s Paradox and how it will function in the legal market, then listen in…!
–
And of course…
Two Major Legal Innovators Conferences this November
Come and join us in New York and London this November at Legal Innovators!
Legal Innovators UK – London, Nov 4 and 5
And, then Legal Innovators New York – Nov 17 and 18.
After another fantastic Legal Innovators California, where we had speakers from OpenAI, Y Combinator, Google, Meta, and many more pioneering organisations; and our stellar inaugural event in Paris this June, we are now looking forward to the landmark conferences in London and New York, both in November, and both across two days: Law Firm Day, and Inhouse Day.
–
That’s all folks. If you’re heading off to Nashville, have fun and get some grits and gumbo in you!
I’ll be in London, managing the avalanche of news from here. But my mind will be on the Puckett’s menu. Although, watch this space……there may be some special coverage of ILTACon, nonetheless…!
Richard Tromans, Founder, Artificial Lawyer
Discover more from Artificial Lawyer
Subscribe to get the latest posts sent to your email.