Connecting AI to your numbers (without leaking anything)
How to make AI work with your real numbers, choosing the right path based on the data's format and connecting it safely, without leaking information or inventing values.
You ask AI what your contribution margin was last quarter. It answers with a flawless analysis, full of pretty finance terms. And made up. AI never opened your ERP or your closing, so it guesses a plausible number.
You ask AI to cross-reference the turnover rate, which lives structured in the payroll system, with the termination policy, which is plain text in a PDF. It hands back an answer full of people-management jargon, but it treated the policy as if it were a spreadsheet and the payroll data as if it needed meaning-based search. The team that was losing the most people got lost in the confusion, because neither source was read the right way. And before connecting the named termination records, there's still the question that matters most: can this data leave the company without being anonymized?
You ask AI which feature has the highest retention, cross-referencing the analytics dashboard, which is structured data, with the quarterly PDF report full of charts that the data team sends. It hands back a pretty number, but it read the PDF as plain text and lost the table, and treated the dashboard as if it needed meaning-based search instead of calculation. The feature that was really keeping users around was in a dashboard column it never added up correctly. Wrong format, wrong answer: structured data calls for reasoning over the schema, a report with a chart calls for reading as an image.
You ask AI to explain where the pipeline gets stuck the most, cross-referencing the CRM export, which is rows and columns, with the win-loss report in PDF that the sales team builds every quarter. It hands back a consultant-style answer, but it treated the CRM as text for meaning-based search and the chart-heavy PDF as a simple spreadsheet, losing the stage-by-stage conversion table. The real bottleneck was in qualification, hidden in a table it never read as an image. And before connecting the proposal history with customer data, there's a question missing: can this go to the model in the cloud, or does it need anonymization first?
You ask AI to find where the SLA blows up the most, cross-referencing the per-stage timing system, which is structured data, with the chart-heavy quality PDF report the hub sends you every week. It hands back an answer full of efficiency jargon, but it treated the PDF as plain text, losing the timing table, and treated the structured system as if it needed meaning-based search. The internal handoff that was causing the most delays was in a table it never read as a real image. Wrong format, wrong diagnosis: a structured system calls for reasoning over the schema, a report with a chart calls for reading as an image.
You ask AI to list which internal controls are most exposed, cross-referencing the risk matrix, which is rows and columns, with the data-handling policy, which is plain text, and the chart-heavy audit report in PDF. It hands back a generic list, but it scrambled all three sources: it treated the matrix as text, the policy as a spreadsheet, and the PDF lost the severity table. The control that was actually broken was in a policy it never read by meaning, and in a table it never read as an image. And before connecting any of these sources with customer data, there's the question that decides everything: can this leave the company, or does it need a closed environment?
You ask AI to find which service causes the most incidents, cross-referencing the structured monitoring log with the postmortem report in PDF, full of latency charts. It hands back a guess with the air of someone who knows infra, but it treated the log as text for meaning-based search and the PDF as a simple spreadsheet, losing the chart. The microservice that was really bringing the system down was in a table in the report it never read as an image. Wrong format, wrong root cause: a structured log calls for reasoning over the schema, a report with a chart calls for reading as an image.
You ask AI to find where the user abandons the flow the most, cross-referencing the funnel data, which is structured, with the research report in PDF, full of heat-map charts. It hands back a UX cliché, but it treated the funnel as plain text and the PDF as a spreadsheet, losing the heat map that showed the real friction point. The middle step of the flow, where the user actually got stuck, disappeared because neither source was read the right way. Wrong format, wrong finding: funnel data calls for reasoning over the schema, a report with a chart calls for reading as an image.
You ask AI which competitor is gaining market share fastest, combining your win-loss report in PDF, full of charts, with the quarterly sales spreadsheet in one go. It hands back a pretty answer, but it scrambled the two: it treated the PDF with the chart as plain text and lost the table, and treated the spreadsheet as if it needed meaning-based search. The small, agile competitor that was actually taking the most business from you was hiding in a spreadsheet row it never read as a real table. Wrong data format, wrong path: a PDF with a chart calls for reading as an image, a spreadsheet calls for reasoning over the schema, not the other way around.
Notice something: the problem is almost never that AI is bad at math. The problem is that it's answering off the top of its head, without ever having opened YOUR numbers. It's like hiring the best financial analyst in the market, sitting them in your office, and never giving them access to your ERP, your closing, your cost spreadsheet. They'll talk pretty and get it badly wrong, because they're guessing. Connecting AI to your data is exactly what takes it out of the generic. Except connecting it wrong is worse: either it leaks information that shouldn't have left, or it invents a number that looks true.
The core idea of this lesson. AI only becomes truly useful when it works WITH your numbers, and the FORMAT of the data decides the path. Structured data (spreadsheet, ERP, table) often doesn't even call for vector search: AI reasons over the schema, the statistics, and a sample. A text document (policy, contract) calls for RAG, search by meaning. A report with a chart and a table (a closing PDF, a statement) breaks traditional RAG, and that's where reading the page as an image comes in, without losing the numbers. And in finance, before anything else, comes the security question: what can leave the company and what can't.
01The format of the data decides the path
The first thing that changes everything is to stop treating "my data" as a single thing. There's no single path for connecting AI to your numbers. What exists is the format of the data, and the format dictates the road.
Think with me about three types of numbers that live in every company. There's structured data, which lives in rows and columns: the cost spreadsheet, the ERP table, the exported sales report. There's plain text documents: the credit policy, the supplier contract, the expense approval manual. And there's the visual report: the monthly closing PDF, the statement with a chart, the results presentation with a colorful table.
These are three different natures, and each one calls for a different tool. Trying to force all three down the same path is mistake number one. The @datasciencebrain study hits exactly this point: the format of the data decides the memory architecture, not the other way around.
02Structured data: AI often doesn't even need vectors
Here lies the most common mistake made by whoever just learned RAG: thinking that ALL data needs to become vector search. It doesn't. And in finance this is especially true, because a good chunk of your numbers live structured, in rows and columns.
When the data is a cost spreadsheet or an ERP table, AI gains almost nothing from turning every cell into a coordinate of meaning. What it needs is to understand the STRUCTURE: what the columns are, what each one means, how many rows there are, the sum of each column, the range of the values. Instead of "searching for similar snippets," it reasons over the schema, over the statistics (average, sum, max, min), and over a sample of a few representative rows.
Think about the practical difference. To answer "which product has the worst unit cost," AI doesn't need a fuzzy semantic search. It needs to look at the table, sort the unit cost column, and read the first row. That's reasoning over structured data, not vector search. It's cheaper in tokens, more precise, and doesn't carry the risk of the search bringing back the wrong record.
The takeaway is direct: before building an entire RAG for your numbers, ask whether the data isn't already structured. If it is, the path is to give AI the schema, the statistics, and a sample, and let it calculate. Fair enough?
03Document and visual report: two paths that don't mix
Now the other two formats, which many people treat the same way, and they're not.
The text document is RAG's natural territory, exactly as you saw in the lesson on RAG and memory (2.4). The credit policy, the supplier contract, the expense manual: all of that is plain text, and AI finds the right answer by searching for meaning. You ask "what's the payment term on this contract" and the search by meaning pulls the right passage, even if the contract writes "due date" instead of "payment term." For text, RAG solves it.
The visual report is where the trap lives. Take the PDF of your monthly close: it has a table, it has a bar chart, it has numbers aligned in columns, it has a footnote in the margin that changes the meaning of the line above. If you feed that PDF into traditional RAG, it extracts the text and SCRAMBLES everything: the table becomes a soup of loose numbers, the chart disappears, the relationship between the row and the column is lost. Traditional RAG breaks on the visual report, and the worst part is that it breaks silently: it hands back a number that looks right and is wrong.
The way out is to read the page as an IMAGE. Vision models (multimodal) look at the PDF like a photo of the page, see the table as a table, the chart as a chart, and preserve the relationship between the numbers. It's the difference between transcribing a statement blindly and actually looking at it.
04Security: in finance, it's the deadly point
Everything we've seen so far is about making AI get it right. Now the point that, in finance, is what can bring you down: what leaves the company.
The instant you connect AI to your numbers, you need to answer one question before any other: CAN this data leave the company? Some data can go to a cloud model with no problem at all. And some data can't leave under any circumstance: nominal payroll, customer data, a number that moves your stock price, a base subject to data-privacy law. Sending that out without thinking isn't optimization, it's a leak.
There are three layers of protection that go together. The first is anonymization: stripping what identifies before sending, swapping the customer's name for a code, masking the tax ID, aggregating what can be aggregated. The second is the choice of where it runs: sensitive data may call for a model running locally or in a closed environment, instead of the public cloud, and that's a risk decision, not a laziness decision. The third is the gateway, a gate that sits between you and the model and BLOCKS what can't go through, logging everything that did.
That gateway is the same concept as the Stack plus AI Gateway you saw in the Machine Room, and it connects straight into the lesson on execution governance (4.4). It's not bureaucracy: it's what separates "I used AI on my numbers" from "I leaked my numbers to a third party without noticing." In finance, that difference has a name: lawsuit, fine, and firing.
05Putting it all together: from generic number to your number
Now you can see the whole picture. Connecting AI to your data is what takes it out of the generic, it's what stops it from guessing and gets it answering with your closing right in front of it. But connecting is a double-edged sword: connecting it wrong either leaks information that shouldn't have left, or invents a number that looks true.
The mental sequence that protects you is always the same. First, look at the FORMAT of the data and choose the path: structured becomes reasoning over the schema, text becomes RAG, visual report becomes reading as an image. Second, before plugging in anything, run the security check: can this data leave, does it need anonymizing, does it run locally or in the cloud, is the gateway in the middle. Format first, security always.
Whoever understands this order stops treating AI as a magic box that guesses numbers and starts treating it as an analyst you give access to, with rules, for what it can see. It's the difference between an analysis you trust and one that puts you in a meeting with a ghost number. Fair enough?
Do it now
Take a real case from your finance area where AI answered off the top of its head and got it wrong, or where you haven't even tried using AI yet: your real task.
- List 3 sources of numbers that would contain the right answer (ERP, cost spreadsheet, closing PDF, contract, bank statement).
- For each source, classify the FORMAT: structured (rows and columns), text document, or visual report (table and chart on a page).
- Mark the path for each one: structured leads to reasoning over the schema, text leads to RAG, visual report leads to reading as an image.
- Do the security check for each source in one line: can this data leave the company? Does it need anonymizing? Does it run in the cloud or call for a closed environment?
- Point out which of the three sources is the most sensitive and write, in one sentence, what the gateway should block in it.
You just designed the connection of your numbers to AI through the right path and with the security gate in place. You're ahead of whoever just throws the closing PDF into a chat and prays.
Practice
1. You need AI to answer 'which product has the worst unit cost' from your cost spreadsheet. What is the most suitable path?
2. Why does throwing the monthly closing PDF (with tables and charts) into traditional RAG usually give a wrong answer?
3. Before connecting AI to sensitive financial data (nominal payroll, customer base), what is the correct stance?
For the board
On the diagnosisit is not bad at arithmetic. It is guessing, because it never opened your numbers.
On formatthe format of the data decides the route. A spreadsheet is already structured and often does not need vector search at all.
On the dangerous silencea visual report breaks traditional RAG without warning: it returns a number that looks right.
Thanks for the feedback. It helps sharpen the next lesson.