Connecting AI to your documents (without breaking confidentiality)
How to connect AI to the firm's legal sources so it answers citing the source, without inventing, and without letting client data leak outside.
You ask AI what recent case law says about a point in the case. It answers with a beautiful ruling, complete with number, reporting judge, and date. Invented. You almost file a brief citing a decision that never existed.
You ask public AI to summarize the company's promotion policy before a performance review, and paste the employee's performance history into the prompt, with name and salary, so it has more context. It spits out confident text, with deadlines and criteria that sound official, but it invented them, because it never read your real HR manual. Now there are two problems: the career rule you were about to communicate doesn't exist, and the employee's salary data just circulated outside the company's HR system. Connecting AI to the real manual, inside an environment the company controls, solves both ends: it answers citing the real policy, and the employee's data never leaves HR.
You ask public AI what the roadmap prioritized for next quarter, pasting the whole PRD into the prompt along with the contract number of a strategic client that drove the prioritization, so it understands the request better. It answers with convincing features and justifications, adoption numbers and all, but none of it came from your real PRD, it imagined it. Now there are two problems: the prioritization headed to the stakeholder meeting is a phantom, and the strategic client's name just traveled through an outside server nobody on the team controls. Connecting AI to the roadmap and the real metrics, inside an environment the company controls, solves both ends: it answers anchored in the true source, and the client's data never leaves the building.
You ask public AI to pull up the terms of the last proposal you sent that big account, pasting the negotiation history into the prompt with the agreed value and discount, so it remembers the context. It hands back values, discount, and timeline with total confidence, but it invented all of it, because it had never seen that proposal before you pasted it. Now there are two problems: the number you were about to repeat in the negotiation was never offered, and the account's real discount just circulated outside the company's CRM. Connecting AI to the pipeline and the real proposals, inside a controlled environment, solves both ends: it cites where each clause came from, and the sensitive sales data never leaves the building.
You ask public AI what the agreed SLA is with that logistics vendor, pasting the whole contract into the prompt with the value and the penalty clause, so it understands the case. It answers with delivery time and late penalty, all tidy, but it's a guess, because it had never read your supply contract before you pasted it. Now there are two problems: the penalty you were about to invoke isn't in the real agreement, and the vendor's commercial terms just traveled through a server outside the company. Connecting AI to your real contracts, inside a controlled environment, solves both ends: it answers pointing to the source clause, and the sensitive contract never leaves the building.
You ask public AI to list which controls cover personal data handling under your LGPD policy, pasting the company's entire risk map into the prompt, with system names and sensitive data volumes. It builds a flawless list, with articles and owners, but it invented it, because it had never seen your policy or your risk map before you pasted it. Now there are two problems: the control you were about to report to the audit doesn't exist, and the company's risk map just circulated outside the controlled environment, the very kind of data the policy is supposed to protect. Connecting AI to the real policies, inside an environment the company controls, solves both ends: it answers with traceability, and the risk map never leaves the building.
You ask public AI how the payments service's deploy flow works, pasting repository snippets into the prompt with an API key and production configuration, so it understands better. It describes a beautiful pipeline, with stages and validations that look like your team's, but they aren't, it invented them, because it had never read your repository before you pasted it. Now there are two problems: the step-by-step you were about to follow during an incident is a phantom, and the payments service's production key just traveled through an outside server nobody in engineering controls. Connecting AI to the real code and documentation, inside an environment the company controls, solves both ends: it answers citing the real file, and the sensitive key never leaves the building.
You ask public AI to summarize what the latest research found about the signup flow, pasting the full transcripts into the prompt with participants' names and emails, so it has the complete context. It delivers convincing insights, with percentages and named pain points, but it invented them, because it had never opened your research reports before you pasted them. Now there are two problems: the finding that was about to justify the redesign is a phantom, and the research participants' personal data just circulated outside the company's controlled environment. Connecting AI to your real studies and prototypes, inside a controlled environment, solves both ends: it answers anchored in the source, and the participants' data never leaves the building.
You ask public AI what the last competitive mapping concluded about that competitor, pasting the company's intelligence dossier straight in, with client name and contract figures, so it understands the context better. It hands back a market share reading that sounds confident and tidy, except most of it was invented, because it had never seen that dossier before you pasted it. The problem doubled: the number headed to the board is a phantom, and the confidential dossier just traveled through an outside server you don't control. Connecting AI to your real reports, inside an environment the company controls, solves both ends at once: it starts answering anchored in the real dossier, citing where each number came from, and the sensitive data never leaves the building.
Whoa, notice something: the risk in legal work has two faces, and they pull in opposite directions. On one side, AI running loose and generic invents law and cites phantom rulings. On the other, when you give it real context, comes the temptation to dump the client's entire contract into some public cloud, and then the problem stops being invention and becomes a confidentiality leak. This lesson is about doing both things at once: anchoring AI in the truth of the case AND keeping what belongs to the client inside the firm.
The core idea of this lesson. Connecting well takes AI out of "thinking it knows the law" and anchors it in YOUR sources (legislation, case law, contracts, and firm filings), answering with a traceable citation. But in legal work the connection has a non-negotiable lock: confidentiality and privilege. Client data and trade secrets cannot leave the firm. The gateway is the checkpoint that decides what goes out and what stays in. Connecting wrong leaks a secret or makes AI invent; connecting right does both things at once.
01Why loose AI invents law
Generic AI answers off the top of its head. It has never read your case, never opened your case file, never seen the contract on your desk. So it does what a rushed intern under pressure would do: talk nicely and guess with confidence.
Think about the difference between two legal advisors. The first answers from memory, confidently, and sometimes cites a ruling that doesn't exist. The second, before opening their mouth, gets up, goes to the file cabinet, pulls the applicable legislation, the case law, and the firm's filings, and answers with the folder open in front of them, pointing to where each thing came from.
Legal RAG is exactly that: turning AI into the second advisor. Instead of AI "thinking it knows the law," you connect it to your sources and it answers citing the source. The gain is direct and economic: fewer filings with phantom citations (less risk of sanctions), a checkable origin (you glance at the source), and less rework redoing what AI invented. Fair?
02What "connecting" means: AI starts working WITH your documents
Connecting isn't magic. It's the same RAG from lesson 2.4, now pointed at the firm's archive. You index the sources once (legislation, summaries, case law, templates, contracts, filings) and, with every question, AI searches only the right passages and answers anchored in them, with the citation in the footnote.
The difference from Ctrl+F is what makes this useful in law. Exact-word search only finds the term you typed. If the filing says "termination without cause" and you searched "breach of contract without just cause," Ctrl+F returns zero, even though it's the same idea. Meaning-based search finds by closeness of sense, so it brings back the right passage even when it's written with other words.
The message is simple: connecting well anchors AI in the truth of the case. But it's exactly when you connect the real archive that the problem legal work cannot afford shows up.
03The lock of law: confidentiality and privilege
In law, the raw material is confidential by nature. Client data, trade secrets, litigation strategy, documents protected by attorney-client privilege: none of that can escape the firm. Connecting AI to the archive is great for the quality of the answer, and that's exactly why it becomes a risk point: you're putting the firm's most sensitive material into circulation through a system.
The blind spot lives in a technical detail. When you use a public AI tool, the text you paste can leave your control: it travels to a third party's server, may end up in a log, may, depending on the terms of use, become training material. Pasting a confidential clause into that little chat box is the digital equivalent of forgetting the client's contract open on a café table.
Think of the physical rule: a confidential document doesn't leave the firm without need and without control. The digital world doesn't change the rule, it just changes the door the document can leak through. Before connecting any source, the question is always the same: can this piece leave the firm? Fair?
04How to connect without leaking: anonymize, run locally, and the gateway that blocks
Connecting safely isn't "don't use AI." It's deciding, piece by piece, what can go out and what has to stay. Three levers solve most cases.
The first is anonymization. Before sending a passage out, you swap names, IDs, amounts, and client-identifying data for neutral markers. AI keeps reasoning about the structure of the problem without seeing who the client is. For a generic legal thesis, that's usually enough.
The second is running locally. When the material is too sensitive to leave under any circumstances (trade secret, M&A, internal investigation), you use a model that runs on the firm's own infrastructure. The data never crosses the door. It costs more and takes more work, so you reserve it for what really requires it.
The third, and the one that ties it all together, is the gateway. The gateway is the checkpoint between the firm and the outside world: every AI call passes through it before going out. That's where you program the rules, like "if an ID number shows up, block it," "if the document is marked confidential, force the local model," "log who asked what." This connects directly to lesson 4.4 on execution governance and to the Stack + AI Gateway map: the gateway is the layer that turns a confidentiality rule on paper into a lock that actually blocks the data from leaving.
05Text calls for RAG, a contract with linked clauses may call for a graph
There's a memory decision that changes the outcome, and it connects to the "Which memory for your data" map from lesson 2.4. Not every legal archive gets solved the same way.
To ask about the content of a filing, a thesis, a passage of law, plain text is enough: meaning-based search pulls the right paragraph and you're done. That's the case for Vector RAG, the default, and where almost everyone starts.
But think of a contract full of clauses that reference each other: clause 12 refers to clause 4, which is limited by clause 7, which has an exception in clause 15. Here the question isn't "find the similar paragraph," it's "follow the links." That kind of chained question benefits from a graph, which stores the relationships between clauses, parties, and documents as lines connecting the dots. In practice, a mature firm runs hybrid: meaning-based search to find, graph to see how the pieces connect.
The practical takeaway: start simple by connecting the text with Vector RAG. When questions start crossing linked clauses or multiple contracts from the same client, flag that archive as a graph candidate. Choosing the wrong memory costs months of rework.
Do it now
Think of a real case from your practice where AI answered off the top of its head and got it wrong, or where you were afraid to paste a document into the tool: your real task.
- List 3 to 5 of your own sources that would contain the right answer (applicable legislation, case law, contract, filing, internal memo).
- For each source, mark a label: "can leave anonymized," "runs locally only," or "public, no restriction." Justify in one sentence who the confidential party is there.
- Write the rule your gateway would need to have to block a leak in that case (example: "block external sending if an ID number or client name appears").
- Decide the memory: does your question cross linked clauses or multiple documents from the same client? If not, Vector is enough. If so, note "graph candidate."
You've just designed a legal connection that anchors AI in the truth of the case without letting confidentiality slip out.
Practice
1. What is the core gain of connecting AI to the firm's sources (legal RAG) instead of letting it answer off the top of its head?
2. You need AI to reason about a legal thesis, but the contract involved has sensitive client data. What is the right approach to connect without breaking confidentiality?
3. What is the gateway's function in this legal connection design?
For the board
On the two faces of riskleft generic it invents law. Connected carelessly it becomes a breach of privilege.
On the designanonymising solves most of it. What cannot leave under any circumstance runs locally, never crossing the firm's door.
On the gatewayit turns the confidentiality rule on paper into a real lock: it blocks, forces the right model, and records who asked what.
Thanks for the feedback. It helps sharpen the next lesson.