AI solicitor recommendations need a separate status check

The brief

Blind Justice UK’s September study reports that AI assistants sometimes recommended firms without mentioning regulatory sanctions or closure. Its tests distinguish asking for a recommendation from asking directly about regulatory history. The evidence is a targeted exercise, not a representative ranking of AI systems or the legal profession.

Our take is that recommendation and verification should be separate steps. Before relying on a shortlist, establish the exact legal entity, check its current status using the regulator’s own records and retain the date and source of that check. If a tool cannot establish status, its answer should leave that question open. A familiar name, review score or statement that a firm is regulated does not settle every due-diligence question.

The full record

A confident recommendation can conceal an unanswered status question. Blind Justice UK’s AI’s Blind Spot report examines that problem using consumer-style questions about solicitors.

What the study found

In a repeat exercise, the charity asked ChatGPT, Gemini and Claude for the best conveyancing solicitors in Doncaster, ten times each. It reports that all 30 answers put a previously fined firm first without mentioning the fine. In a separate Maidenhead exercise, it reports that Claude named a closed firm in ten of ten answers and ChatGPT in three of ten; Gemini did not name it. These are the charity’s reported results for those prompts and runs. Source: report, pages 3–5.

The evidence appendix records September tests in new chats, the displayed model settings, prompts, coding criteria and departures from the protocol. It explains that the fined-firm set was illustrative rather than random: some firms were selected after their review pages had been examined. The appendix also shows that answers could differ when asked directly about a named firm’s regulatory history.

What these results cannot establish

This design does not estimate how often an arbitrary user receives an unsafe recommendation. Repeating one location-specific prompt tests consistency in that scenario; it does not create a broad sample of legal needs. A historical fine, a restriction on practice and a closure are also different facts. They should not be treated as interchangeable judgements about whether a firm may currently act.

We have reviewed the published report and its evidence appendix, but have not rerun the chats or independently audited every underlying regulatory decision. The article therefore attributes the findings to the charity and does not turn the test into our own performance benchmark.

Our analysis: require a dated status record

A useful shortlist should separate the reason for suggesting a firm from the checks completed about it. Identify the legal entity and trading name, state the source and date of the status check, and make unresolved points visible. Where a regulatory decision is relevant, link the decision itself and describe its actual scope.

The same discipline applies when a person, rather than an AI tool, introduces a professional adviser. A recommendation explains why somebody might be worth considering. It cannot stand in for confirmation of the exact entity receiving the instruction or the authority of the person answering.

For a future benchmark, test whether a system retrieves current status, distinguishes sanctions from closure, links the original record and says when evidence is missing. Include firms with similar names and changed trading names. Report the full test set, dates, model versions and errors. That would measure a specific verification task rather than a vague claim of trustworthiness.

Our latest SRA alert analysis carries this distinction into the next stage: verifying the actual instruction after a firm has been identified.

We read the public report and evidence appendix linked by Blind Justice UK. Findings are attributed, not independently replicated. Repeated prompts are not a representative sample of all users, models or firms. This is analysis of published research, not a Formal Record benchmark or a finding that any named firm is currently unfit to act.

1. Blind Justice UK report landing page
2. AI’s Blind Spot, public report, September 2026
3. AI’s Blind Spot, evidence appendix, September 2026