Security for startups

The AI Questions on a Security Questionnaire, Answered

The AI questions on enterprise security questionnaires, grouped by theme: what the reviewer checks, an answer that holds up, and the one that gets caught.

In this post
  1. What does an honest answer to an AI security question look like?
  2. Training on customer data
  3. Prompt and output retention
  4. Model providers as subprocessors
  5. Prompt injection
  6. Agent permissions and tool calls
  7. Logging of agent actions
  8. Output handling and PII leakage
  9. Evals and model changes
  10. AI incident response
  11. Where the answers come from
  12. What if you don’t have a SOC 2 report yet?
  13. Order of work

TL;DR: The AI section of a security questionnaire comes down to nine themes: training on customer data, prompt and output retention, model providers as subprocessors, prompt injection, agent permissions, logging of agent actions, output and PII leakage, evals and model changes, and AI incident response. For each one the reviewer is checking a specific thing, and the answer that holds up has four parts: a status, the evidence behind it, a named owner, and for any gap an interim step with a date. The answer that gets caught is the one that sounds fine until they say “send me the document.” If you have no SOC 2 report yet, the same answers still work, and there is a short section on it near the end.

A list of the AI questions on a security questionnaire is useful if you are the buyer sending the form. If you are the founder who just received one with an AI section on it, the list doesn’t tell you what the reviewer is looking for or what a good answer looks like.

This post is the founder side. I’m not going to give you a 40-question template. I’m grouping the questions by theme, because the same few concerns sit behind dozens of differently worded questions. If you can answer the theme, you can answer any wording of it. The example questions below were written for this post. None is copied from SIG, CAIQ or a buyer’s form.

I’ve been on the answering side of these questionnaires. I was CISO at Lavender and took its security program through SOC 2 Type II. My argument for why you shouldn’t paste auto-generated answers into these forms is in Why Auto-Filled Security Questionnaires Kill AI Deals, and I won’t repeat it here. This post assumes you are going to answer them yourself and want to know how.

One disclosure before the advice. I sell a paid kit that walks you through these answers, The AI Startup Security Questionnaire Kit, and I take consulting work on live questionnaires. Nothing below needs either one.

What does an honest answer to an AI security question look like?

It has four parts, and the reviewer is reading for all four: a status (in place, partial, not in place, or not applicable), the evidence that backs the status, a named owner, and for anything short of “in place,” what you do in the meantime and when the gap closes.

The evidence part is where answers get tested. “Evidence” means something a reviewer could open or watch you open: a settings page, a signed agreement, a log query, a runbook with a date on it.

The gap part is the one founders avoid and the one that helps most. Here is the shape, with made-up values:

Status:   Partial
Answer:   Prompts are kept in our log store for
          90 days. Our target is 30.
Interim:  Two named engineers have access.
          Access is reviewed monthly.
Closes:   2027-01-15
Owner:    CTO
Evidence: log-retention setting (screenshot,
          dated), access review ticket

A gap written this way gives the reviewer something to act on. They can accept it, ask you to close it before go-live, or put a date in the contract. A vague answer (“we are working on retention”) gives them nothing to do except worry. If the true status is “partial,” write “partial.”

The themes below follow the same pattern. For each one: what they’re checking, the shape of an answer that holds up, and the answer that gets caught.

Training on customer data

Example question: “Is customer content used to train or improve any model, yours or a third party’s?”

What they’re checking. There are three questions in that sentence. Does your model provider train on what you send it? Do you fine-tune or build indexes on customer data? Does your own product pipeline (feedback buttons, eval sets, support tooling) copy customer content into places a model might later learn from it?

An answer that holds up. Answer each of the three separately. For the provider, point at the actual terms. OpenAI’s API documentation says data sent to its API is not used to train or improve its models unless you explicitly opt in to share it. That page describes OpenAI’s default terms as of November 2026. Read the same page for whichever provider you use. Your evidence is the agreement or terms you actually signed, plus the account setting if one exists. A marketing page doesn’t count. For your own use, say whether you fine-tune, and if you do, whether each customer’s tuned artifacts are kept apart. Owner is usually the CTO.

The answer that gets caught. “We never train on customer data,” when a customer’s prompts sit in your evaluation set, or a thumbs-down button saves the whole conversation to a shared bucket. Before you write the sentence, search your own systems for where customer content gets copied. If the search turns something up, the answer is “partial,” with the copy’s retention and access, and a date for when it stops.

Prompt and output retention

Example question: “Where do prompts, completions and embeddings end up, how long does each copy live, and who can read it?”

What they’re checking. Whether your logs have become a second copy of the customer’s data with weaker controls than the first. Retention at the provider is one clock. Retention in your logging stack, your error tracker, your analytics tool and your vector store are the other clocks, and the reviewer knows you probably only thought about the first.

An answer that holds up. A short table: each place prompts and outputs land, the retention setting, who has access, and the evidence for each row. Provider-side numbers belong in it too. OpenAI’s documentation says abuse-monitoring logs are retained for up to 30 days, and that zero data retention is a control available only with OpenAI’s approval. Anthropic’s privacy center says it automatically deletes API inputs and outputs within 30 days, with listed exceptions. Both were the default terms as of November 2026. If you rely on zero retention, attach the approval.

The answer that gets caught. A single number (“30 days”) given for the provider while the same prompts sit indefinitely in application logs that every engineer can query. The reviewer asks “and in your own logs?” and the answer is longer than the one you gave.

Model providers as subprocessors

Example question: “List every third party that processes customer data in delivering the service, including AI model providers.”

What they’re checking. Whether your subprocessor list is complete and whether the model providers are on it. A model API is a subprocessor in the plain sense: customer content goes to it. Leaving it off a list that includes your email vendor looks like an oversight at best.

An answer that holds up. The model provider is named on the list, with what data it receives, where, under what agreement, and who approved it. If you route through a gateway, an aggregator or a hosted inference platform, those belong on the list as well, since each one is another party that sees the content. Add the date you last reviewed the list. When you add a provider, the list changes before the first production request goes out.

The answer that gets caught. A list from before you added a second model provider, or one that names the model company but not the cloud platform you reach it through. A reviewer can compare your list against your public sub-processor page and your privacy policy, so make the three say the same thing.

Prompt injection

Example question: “How do you test for and defend against prompt injection, including through documents, web pages and tool outputs your system reads?”

What they’re checking. Whether you understand that the model can’t be the security boundary. OWASP ranks prompt injection first in its Top 10 for LLM Applications, and its own page says it is unclear whether fool-proof methods of prevention exist. A reviewer who has read that will distrust any answer that claims the problem is solved.

An answer that holds up. Describe the design, which is where the real defense is: which agent contexts hold private data, which read untrusted content, which can send anything outside, and how you keep all three out of a single context. Then describe the testing: a set of adversarial inputs you run, how often, and where the results live. I wrote out the small-team version of this in Prompt Injection Defense for a 5-Person AI Startup. If you haven’t tested, say that, and give the date the first test run will finish and who is running it.

The answer that gets caught. “We use a guardrail model and a strong system prompt.” The follow-up is “show me what happens when a document you ingest says to email its contents elsewhere.” You need a test result to answer that.

Agent permissions and tool calls

Example question: “What actions can your AI agents take without human approval, and how are those limited?”

What they’re checking. Blast radius. OWASP’s entry on excessive agency names three causes: excessive functionality, excessive permissions and excessive autonomy. The reviewer’s questions map to those directly. What tools does the agent have that it doesn’t need? What credentials do those tools hold? What can it do without a person approving?

An answer that holds up. A tool allow-list: one row per tool, what it can reach, which credential and scope it uses, whether a human confirms before it runs, and who approved it and when. The field-by-field version is in The Agent Evidence Pack for a Security Review. The point of generating it from the code that registers your tools is that it stays true. If an agent holds a broad credential today, put that in the list and in the gap statement. Give the date the credential gets narrowed.

The answer that gets caught. “The agent only has access to what it needs,” with no list, or a list that doesn’t match what the code registers. The reviewer asks for the production credential the agent uses and what that credential could do if the agent were tricked.

Logging of agent actions

Example question: “Can you reconstruct what an AI agent did for a specific customer on a specific day, and how long do you keep that record?”

What they’re checking. Whether after an incident you could answer “what did it do?” without guessing. This is also where the questionnaire connects to your audit: logs that exist only on paper don’t help in either place.

An answer that holds up. A sample record they can read: timestamp, actor, tenant, tool, arguments, outcome, and whether a person confirmed it. A retention period you chose on purpose, and protection against editing. Those fields are the tool-call log record in The Agent Evidence Pack for a Security Review, linked in the section above. Building an LLM Audit Trail That Passes SOC 2 has the storage schema and the retention reasoning. Offering to walk the reviewer through one real event on a call is more convincing than any description.

The answer that gets caught. “We log everything.” Logs of what? Prompts but not tool arguments, or tool calls with no tenant on them, so nobody can filter to one customer. Check that the log can answer a question about a single tenant before you say it can.

Output handling and PII leakage

Example question: “How do you prevent model output from exposing personal data, secrets or another customer’s information?”

What they’re checking. Two separate risks. One is the model repeating sensitive content it was given, which OWASP lists as sensitive information disclosure. The other is your application trusting model output and passing it to a database, a browser or another system (OWASP calls it improper output handling).

An answer that holds up. For leakage, show how retrieval is scoped: the query for a customer’s context filters by tenant before anything reaches the model, and you can show the code or the test for it. For output, show where model output is validated before something acts on it, such as a schema check before a write, or links and images stripped before rendering. Name what you redact from prompts and where.

The answer that gets caught. Tenant isolation described as a property of the model (“it only sees that customer’s data”) instead of the retrieval layer. The reviewer asks what stops a query from returning another tenant’s documents, and there is no filter to point at.

Evals and model changes

Example question: “How do you evaluate model quality and safety, and what happens when a model version changes?”

What they’re checking. Whether the product they approved today will behave the same way next month. Providers retire models on a schedule (OpenAI and Anthropic both publish the dates), and a prompt that was safe on one version may behave differently on the next.

An answer that holds up. Which model version you pin, a set of regression cases that you run before changing it (including a few safety cases), who signs off on the change, and how you roll back. The evidence is the eval file in your repository and a record of the last time you ran it. If your evals are thin today, say so: “twenty cases, run by hand, automated by this date.”

The answer that gets caught. “We always use the latest model” or “we test it manually,” then nothing written down. A related catch is describing an eval you ran once at launch as an ongoing practice.

AI incident response

Example question: “Does your incident response plan cover AI-specific events, and when did you last exercise it?”

What they’re checking. Whether your plan was written for a database breach and never revisited. AI events look different: an agent takes a wrong action in a customer’s system, a prompt injection pulls data out, a provider has an incident, or a model produces something harmful at volume.

An answer that holds up. Your plan lists at least a couple of those scenarios, names who decides to turn the feature off, and says how: a flag, a kill switch for a specific tool, a way to revoke the agent’s credential. Include the date of the last tabletop or review and what changed afterward. If you have never exercised it, put a date on the first exercise.

The answer that gets caught. An incident response plan with no AI scenario in it, or a plan where nobody can say who is allowed to disable the agent.

Where the answers come from

Getting these themes answered once is the hard part, and it is mostly an interview: someone asks what you do today, writes it down with the evidence, and notes where you fall short. The kit I sell, The AI Startup Security Questionnaire Kit, automates that interview. Your coding agent asks you 120 questions and writes an answer bank where each answer carries a status, the evidence and an owner. The kit groups its questions into 10 themes of its own, and those cover the whole security program: access, data handling, vendors, incident response and the rest. One of the ten is AI and LLM use, with 29 questions. The nine themes in this post don’t map one to one onto the kit’s ten. The kit’s subprocessor-list and agent kill-switch questions, for example, sit in its vendor and incident response themes. Gaps go to their own file, and a check script fails an “in place” answer with no evidence, or a vague word with no cadence or date. It doesn’t fill in a buyer’s spreadsheet. You copy from the bank into whatever the buyer sent.

You can do the same with a document and an afternoon. Either way, write the answers from what your systems do today.

What if you don’t have a SOC 2 report yet?

The questionnaire arrives whether or not you have one, and the AI themes above don’t depend on a report. A SOC 2 report attests to a set of controls you scoped, and nothing in the AI section requires one to answer it.

Two things change. First, the reviewer has no auditor’s opinion to lean on for the basics, so your evidence for access control, change management and incident response has to be easier to inspect: screenshots, tickets, policy documents with dates. Second, state where you are on the audit plainly: not started, readiness work underway, auditor engaged, observation window open, with the dates you know. Don’t write “SOC 2 compliant” unless a report exists. “Readiness work underway, auditor engaged, Type I report expected in the first quarter, owner CTO” is better than a blank, and the reviewer can check each part of it.

A report would not have answered most of the AI questions anyway. Put the time into AI answers with evidence and a dated plan for the audit.

Order of work

If the questionnaire is due soon, do these in this order. Subprocessors, retention and the provider half of the training question first, since those are facts you can look up. Agent permissions and logging next, since they take the longest to prove. Prompt injection, output handling and evals after that, because the answer usually includes a gap and you want the date before the call. Incident response last, because the other answers tell you what the scenarios are.

Whatever order you pick, give each gap an owner and a date before you send anything back. If you want help building the program behind the answers, that is what I do as a fractional security lead, and what a 90-day engagement looks like is written up in detail. If you have a questionnaire in hand with a deadline, the place to start is the consulting page.