AI doesn't refuse to answer. It quietly guesses who you are.

Ask an AI: "Where should I invest, starting with $50 and adding $25 a month?"

You'll get a confident, well-organized answer. Emergency fund first, low-cost index funds, keep fees down. It reads like advice from someone who knows what they're doing.

Now notice what you didn't tell it. Your age. Whether you have debt. Whether you have three months of rent saved or none. Whether you're 35 with decades to compound or 60 and thinking about drawing this down soon. The right answer is genuinely different for each of those people. The model gave you an answer anyway.

So where did the answer come from? It came from the parts you left out — filled in by the model, silently, and never shown to you. That's the real problem with AI advice, and it's bigger than "sometimes it's wrong."

The model never says "I don't have enough information"

Here's the thing people miss: there is no insufficient-information mode. A human expert who didn't know your age would ask. The model won't. Ask an underspecified question and it doesn't stop — it imputes the missing facts and answers as if you'd stated them.

That's not a bug you can turn off. It's how the thing works. Every answer to an underspecified question is built on assumptions the model had to invent to answer at all. The assumption layer is mandatory. The only question is what it's made of.

And this part isn't a flaw. A 35-year-old and a 60-year-old should get different advice. If the model gave both the same answer, that would be worse. Making assumptions isn't the problem. Making them invisibly is.

It guesses from how you sound, not just what you said

Now it gets worse. When a fact is missing, the model doesn't reach for a neutral default. It infers the missing fact from how you wrote the question — your word choice, your tone, how sophisticated you sound.

We don't have to speculate about this. A 2026 MIT Sloan study had a thousand real people ask an LLM for financial advice, then simulated the outcomes over a lifetime. Two findings land directly on this:

First, people systematically leave things out. Only 6% mentioned an emergency fund. They weren't hiding it — they didn't know it was the thing that mattered. The model filled that gap for them, its own way.

Second, and this is the uncomfortable one: the researchers took prompts that didn't mention gender and randomly labeled them "I am a man" or "I am a woman." Same question, otherwise identical. The model gave more conservative investment advice to the ones labeled female. A third of the entire gender gap in advice came from the model changing its answer based on a label — not on any financial fact.

Read that again. The hidden assumption wasn't "I'll guess you're 45." It was "I'll guess you're the kind of person who..." — inferred from surface cues that correlate with, but do not determine, the right answer. Over a lifetime, differences like these compounded into 4-5% gaps in retirement wealth between groups asking functionally the same thing.

The seam is invisible, so you can't audit it

A human advisor who assumed you were mid-career would say so: "assuming you've got a while until retirement..." And you'd correct them on the spot. That one sentence is the whole safety mechanism. It makes the assumption inspectable.

The model bakes its assumptions into a fluent, confident paragraph with no seam. You can't see where the guess was, so you can't tell which parts of the answer rest on your facts and which rest on the model's inference about your type. The confidence is identical either way. That's what turns a normal limitation into a real problem: not that the model assumes, but that it assumes, sources the assumption from how you sound, and then hides it.

Put it in one sentence

AI has made expertise instantly accessible. But it delivers that expertise by silently guessing the context you didn't provide, guessing it partly from how you sound rather than only from what you said, and then folding the guess into an answer that looks exactly as sure as if you'd spelled everything out.

Cheap access to expertise. But the calibration that makes expertise useful is imputed, invisible, and unaccountable.

What to actually do about it

The obvious fix — "state your assumptions" — is right but not enough, for two reasons the study makes plain. You can't state context you don't know you're supposed to provide (the emergency fund). And even with full context, the model still injects things you didn't ask for (it recommended specific brands like Vanguard in 6% of answers when almost nobody asked). So the discipline has to do three things, not one:

• Force the gap into the open. Before it answers, make the model tell you what it would normally need and you didn't give — so the assumption becomes a question instead of a silent fill-in. "What do you need to know from me to do this well? Ask me first."
• Make it label grounded vs. guessed. Ask it to mark each assumption as traceable to a fact you stated, or inferred about you. The inferred ones are where the invisible calibration lives. Once they're on the page, you can correct them.
• Check the answer against fixed standards. Have it grade its own output against criteria you set in advance, so the assumptions become inspectable after the fact instead of buried in prose.

None of this requires a better model. It requires refusing to let the assumption layer stay invisible — which is a discipline you impose, because the model will never volunteer it.

We built exactly these guardrails into a small set of financial prompts and a review skill you can grab and run — including one whose only job is to audit advice you already got and flag every assumption the model made about you rather than from your numbers. They're free in the Korvai gallery. But the principle is bigger than any prompt: the model is always guessing who you are. Your job is to make it show its work.