How to keep legal terms readable when you mask client data

Learn why masking court names and article numbers breaks Claude's reasoning, and how Shield protects client privacy without removing legal context.

By RAGbase Published on Updated on 5 min read

A lawyer who wants to use Claude on a client file usually starts with one question: how much should be hidden before the text reaches Claude? The instinct is to mask everything, names, dates, amounts, even the court that issued the ruling. That instinct is wrong. A court name or an article number is not personal data, and once it disappears, Claude no longer reads the same file the lawyer has in hand.

Why court names break Claude’s reasoning

Claude works from the text it receives. If a brief cites “Cour de cassation” and a pseudonymisation tool turns that into a code, Claude is reasoning about a court it cannot name, in a ruling it cannot place. The same happens when an article number or a case reference changes. Claude can still produce an answer, but the answer stops tracking the actual legal reasoning in the file, because the legal reasoning depends on which court said what, under which article, on which date. Masking a name protects a client. Masking a court or a law does not protect anyone, it only removes information Claude needs to help.

Masking everything is easy, but then Claude cannot do the work. Shield masks your clients’ data and keeps the law readable: articles, courts, dates and amounts stay in clear. That is the design choice behind RAGbase Shield, and it is also what the public benchmark results show.

RAGbase measured how often each tool changes a legal term, meaning a court name, a law, or an article reference, on the benchmarks of its public results file. Three of the four benchmarks hold legal terms: MAPA, built from decisions of the Cour de cassation, REF, built for employment law, and Max, built from law firm files. On these three, Shield changes 0 to 4.2 % of the legal terms. Hexagone changes between 17.8 and 58.3 % of them, and Presidio between 15.3 and 55.6 %. On MAPA specifically, Shield changes 0 % of the legal terms in both of its modes, while Hexagone changes 43.1 % in Fast mode and 47.6 % in Precis mode, and Presidio changes 18.0 %.

BenchmarkShieldHexagonePresidio
MAPA0 % (both modes)43.1 % Fast, 47.6 % Precis18.0 %
REF4.1 % (both modes)17.8 % Fast, 47.1 % Precis15.3 %
Max2.1 % Fast, 4.2 % Precise47.2 % Fast, 58.3 % Precis55.6 %

When a tool changes the name of a court or the number of an article, the text that Claude receives no longer says what the file says. That is the court name problem in practice: a tool that masks aggressively to look safe ends up handing Claude a document that misquotes its own sources.

The overall score and why masking everything fails it

RAGbase also computes an overall score for each tool, the F1 computed by word, across four benchmarks in French: the two public ones, FR-PII-Bench and MAPA, and two built by RAGbase, REF and Max. The overall score drops when a tool leaves personal data readable, and it also drops when a tool masks text by mistake, including law it should have left alone. A tool that masks everything leaks nothing, but its overall score drops, because it masks the law too. This is why “mask more” is not the same test as “mask well”.

Shield has the highest overall score on the four benchmarks: FR-PII-Bench 96.9, MAPA 82.2, REF 97.1, Max 95.3. On these same four benchmarks, Hexagone scores between 37.9 and 80.7, and Presidio between 47.4 and 79.2.

BenchmarkShield FastShield PreciseHexagone FastHexagone PrecisPresidio
FR-PII-Bench96.996.280.780.766.5
MAPA80.482.243.837.963.5
REF97.197.166.761.179.2
Max95.391.854.048.947.4

The results show two Shield columns. Fast is the default, a local engine that finds names and identifiers with rules and dictionaries. Precise adds a local AI model. Both keep the legal terms readable, which is why both score close together on every benchmark above.

Getting Shield onto a lawyer’s computer

RAGbase Shield is free and open source, under the Apache-2.0 licence. It runs local, reversible pseudonymisation: names and numbers are replaced by codes before the text leaves the computer, and Claude’s answer is put back into clear names afterwards, also on the computer. Nothing goes to RAGbase. The source code, the evaluation scripts and the full results used above are public on GitHub, at https://github.com/ragbase/ragbase-shield/blob/main/evaluation/laptop/RESULTS_PUBLIC.md, so a firm’s AI lead can read the method and check the numbers directly rather than take a vendor’s word for it.

Adding Shield takes no technical skill. On the Install page, click “Add Shield to Claude on Mac” or “Add Shield to Claude on Windows”. In the Claude app, click “Sync”, “Add”, then “Continue”. Nothing else to install. You need the Claude app, with a paid Claude plan.

Then, in a new conversation, write “Protect a folder with Shield” and choose the folder of one client in the Shield window. Claude only ever sees codes for people and companies. The court names, dates, amounts and articles stay exactly as written in the file. Only a folder chosen in the Shield window is protected: do not attach a file or add a folder to the conversation.

One limit matters here, because this is about using the tool day to day: detection is never complete. Read the sensitive passages again, because a name on its own can be missed, and read the final document before you use it.

The best pseudonymisation engine on the market.

The limits of Shield are on the Limits and compliance page.

Sources

https://github.com/ragbase/ragbase-shield/blob/main/evaluation/laptop/RESULTS_PUBLIC.md; https://ragbase.ai

Tags: PseudonymisationLegal reasoningClient privacyData masking

Frequently asked questions

Why shouldn't I mask court names before sending text to Claude?
Court names and article numbers are not personal data. When they are masked, Claude cannot trace the legal reasoning in the file, because that reasoning depends on which court said what and under which law. Masking them removes information Claude needs to help, not information that protects anyone.
How does Shield decide what to mask and what to keep readable?
Shield masks names and company identifiers before the text leaves your computer, but keeps court names, dates, amounts, and article numbers in clear. Both Shield columns of the public results, Fast (default) and Precise (with AI model), follow this same design.
Where does my data go when I use Shield?
Shield runs local, reversible pseudonymisation on your computer. Names and identifiers are replaced with codes before text reaches Claude, and Claude answers with the codes. The real names come back only on your computer, in the final document that Shield saves in the "Shield - sorties" folder. Nothing is sent to RAGbase or stored elsewhere.
RAGbase
RAGbase Shield team

RAGbase builds RAGbase Shield, a free, open-source tool (Apache-2.0) for local, reversible pseudonymisation, and private AI systems installed at law firms.

Back to the blog