I built three AI assistants. Two of them would have leaked client data
Three POC builds in six months. Two had no privacy architecture. What would have happened the day they went to production — and the five-question test I now run before letting any AI assistant near a demo.
In the last six months I shipped three AI assistant POCs. An internal tool for a sales team, a customer-support copilot, and a Q&A bot over procurement documents. All three demoed beautifully. The clients were happy. Two of them would have leaked customer data the day they hit production.
I want to talk about the specific holes, because I see the same shape of mistake in every second AI POC people send me for review.
The pattern is always the same. Founder wants AI. Founder hires a vibe-coder (sometimes me, before I learned). Vibe-coder ships in two weeks. The thing works. Founder says "ship it." Lawyer says "wait."
Here's what I shipped and where each one had a hole.
Assistant #1 — sales copilot. Read sales notes from Notion, drafted follow-up emails, wrote them back to the CRM. Worked beautifully. Three problems I only caught on a careful re-read:
- The notes contained customer names, phone numbers, and contract values. All of that went into the OpenAI API on every request. The client's DPA with their own customers didn't permit it. We were one screenshot away from a real lawsuit.
- No audit log. If a customer ever asked "what data of mine did your AI see?", we couldn't have answered. Not "we don't want to" — we couldn't.
- Logs of every model response sat on a VPS in plain text. Default retention: forever. Nobody had decided otherwise. Nobody had thought about it.
Assistant #2 — support copilot. Suggested replies to support tickets. The vibe-coding tutorial says: pass the full ticket plus the customer's history. So we did. The history included addresses, failed-payment messages, and one case I'd rather not describe in writing. All of it went to the LLM. We never told the customer.
Assistant #3 — procurement Q&A. This one I built later, and built correctly. Two layers around the model — a PII scrubber on the way in, a redaction layer on the way out — plus an audit log of every prompt and response with a 30-day retention policy, plus SSO with role-based access so the legal team could see everything and the procurement team could see only their own queries.
The difference between #3 and the others wasn't AI skill. By the time I built it, I'd understood that the AI is the easy part. The plumbing around the AI is the part that decides whether you can sell the thing to a real enterprise or spend six months redoing it right before the deal closes.
Here is the test I now run on every AI assistant before I let a demo happen.
Where does the data go. Which fields actually leave your servers and end up in the LLM provider's logs? You should be able to point at the prompt template and say "this field, this field, not that one." If you can't — you're not ready.
What gets logged. Every prompt. Every response. With timestamps, user IDs, and a retention policy you've actually written down somewhere. If your CTO answers "we log everything to CloudWatch" — that's not an audit trail, that's a graveyard.
Who can see what. Role-based access on the AI interface itself, not just the underlying database. The intern shouldn't be able to query the CEO's email thread. The support agent shouldn't be able to surface the salary spreadsheet.
What happens on delete. When a customer asks to be forgotten, can you actually forget them — not just in the database, but in the embeddings, in the cached prompts, in the logs, in the backups? If your answer starts with "well, technically…" — start over.
What you promised the customer. The DPA, the privacy policy, the consent text. None of these are legal questions before they're engineering questions. The engineer who builds the data plane has to know what was promised.
I'm telling this story because the AI-compliance conversation in 2026 is happening almost entirely on the wrong end. Everyone is buying GDPR/AI policy templates from law firms. Almost nobody is building the data plane that makes those policies actually true.
A lawyer cannot make your AI assistant compliant. A lawyer can only tell you what you promised. The thing that has to deliver on the promise is the architecture — what gets sent, what gets logged, what gets masked, what gets deleted, who has access, what audit trail proves it.
I do this work as a service now. Engineering, not legal. Map the AI data flow. Mark every PII field. Build the scrubber, the audit log, the retention policy, the access model. Make the GDPR/LGPD/DPA promises true, not just signed.
It isn't the most exciting work in 2026. It's the work that keeps the exciting AI work from blowing up in your face nine months in.
If you're shipping an AI assistant and the five questions above make you uncomfortable, we should talk.