← All posts
AIClient communicationResearch

Client Emails AI Writes Badly: 12 Scenarios

Twelve thin-prompt client email tests on Claude Sonnet 5: three came back clean, nine failed in five repeatable ways, and the pattern predicts which of yours will break.

Kathleen Atkins

Kathleen Atkins

CofounderUpdated 13 min read

Twelve draft client emails on a desk, five of them circled and grouped into labeled piles.

We gave one model twelve real client-email prompts, in the thin, one-line way advisors actually type them, and kept every output verbatim, clean ones included. Nine had a defect worth naming; three didn't. The nine sort into five repeatable patterns, and once you know the patterns, you can predict which of your own prompts will come back with something invented in it.

This isn't an argument against drafting client email with AI. A companion test ran the identical model on the identical day and, once the prompt carried voice, hard facts, and an instruction to flag anything unknown, produced clean output on the same class of task. Seven of the nine failures below trace to a fact the advisor left out of the prompt. The other two trace to something the advisor handed off entirely: a judgment about the person reading the email.

unknown node

Twelve prompts, one shared context

Every run in this piece is Claude Sonnet 5, on 2026-08-18. We did not test GPT-5, Gemini, or Copilot; where this piece says "the model," it means that one model, that one day. The twelve scenarios ran as two batches of six, and within each batch every prompt shared a single context, so a later output may carry a faint trace of an earlier one. The prompts were also deliberately thin: one or two sentences, no client facts, no voice guide, no constraints. That thinness is the experimental condition. It is not a claim about how advisors actually write prompts; some are far more careful, plenty are exactly this thin, and the outputs below are what happens either way. This isn't hypothetical: 52% of advisors responding to the 2026 T3/Inside Information survey already use at least one AI tool, up from 41% the year before (T3 Technology Hub, presented 2026-03-10).

We ran the same test on twelve real advisor situations (a death, a fee increase, a bitcoin question, a missed deadline, a termination, a projected shortfall, underperformance, a ghosted prospect, a cognitive-decline disclosure, a surprise tax bill, a competitor's return claim, and a mass check-in) and read every output for what it invented, not for how it read. Three came back clean: the condolence note, the termination, and the note to a ghosted prospect were serviceable on a first pass, with no fact, cause, or action asserted that the prompt hadn't supplied. The other nine each state something nobody told the model, as though it had been.

Thomas Kopelman, co-founder of AllStreet Wealth, named the issue: "The problem with using A.I. for financial advice is that ChatGPT doesn't know what questions to ask" (quoted by Cara Danielle Brown in GOBankingRates, 2024-07-01). None of these prompts asked a clarifying question, and the model didn't ask one back. It filled the gap itself, silently, and the nine failures fall into five distinct patterns.

Five named failure modes in AI-drafted client email, with the twelve tested scenarios distributed underneath each one.

Nine of twelve scenarios sorted into five patterns; three had no defect worth flagging. Source: Plotbook testing, Claude Sonnet 5, 2026-08-18.

Invented causation

Three of the twelve scenarios asked the model to deliver news without giving it a reason for the news. All three times, it supplied one anyway.

Told only that the fee is rising from 0.90% to 1.05% on January 1 (Scenario 2), it explained the increase as reflecting "the continued investment we're making in the resources, tools, and level of service we provide to you." Nothing in the prompt said any of that. If the real reason is a margin call or a partner-meeting decision the advisor wasn't in, the email now contradicts the file. The same reflex fired for a retirement shortfall (Scenario 6), which opened with "Based on current market conditions, updated assumptions, and your current savings and spending trajectory": three specific inputs the one-line prompt never mentioned, presented as the basis for the finding.

Asked to explain underperformance against the S&P 500 (Scenario 7), it did two things at once: asserted a specific diversified allocation ("bonds, international equities, and small/mid-cap holdings") the advisor never described, and invented a prior conversation: "what we agreed was right for you." No such agreement was in the prompt. If the client's actual allocation differs, the email now makes a claim the advisor can't stand behind if asked for detail.

Invented action already taken

Scenario 4 is a short prompt with real stakes: apologize to a client whose RMD deadline was missed, and who now owes a penalty. The model apologized, then, unprompted, reported remediation work nobody had done:

unknown node

Neither sentence was true when the email was drafted, because neither action was in the prompt. The advisor asked for an apology, not a status report, and the model wrote one anyway, as fact, with no qualifier attached to either claim.

An apology email that misstates what the firm has done is worse than sending no email. If the client finds the distribution unprocessed, or asks her accountant about a form nobody has started, the follow-up becomes about a second broken promise, stacked on the first missed deadline. It's the cleanest example in the batch of a pattern worth watching on its own: a prompt describing a problem tends to come back with the problem already announced as solved.

The hallucination menu

Scenario 10 is the most interesting failure here, because it is shaped like the opposite of a lie.

Asked to explain a surprise capital gains distribution from a mutual fund, the model correctly described the general mechanism (funds must distribute realized gains to shareholders even if nobody sold) and then, reaching the specific cause the client actually wants to know, did this instead of inventing a single answer:

unknown node An AI-drafted email explaining a surprise tax bill, showing a bracketed placeholder that offers the advisor a menu of three fabricated causes to choose from.

Every option inside the bracket is fabricated. None of the three was in the prompt.

This is a different shape of failure from the fee increase or the retirement shortfall, and it earns its own name because it will fool a faster reader than the others will. A single invented cause, stated flatly, reads as a claim, the kind a careful advisor learns to question on sight. A bracketed menu reads as a placeholder, the kind every advisor has seen a hundred times and knows to fill in. It survives a skim precisely because it looks unfinished. But all three options inside it are fiction. None came from a fund prospectus, a redemption report, or anything else the advisor actually knows about this fund's year. Pick any one to send, and the advisor isn't filling in a blank; they're choosing which fabrication to put their name on.

We call this a hallucination because that's the accurate word for it, even dressed up as a placeholder: a confident, specific, unsourced claim generated to complete the sentence, not retrieved from anything true.

Asserted regulation and fact

Scenario 11 hands the model a client's forwarded newsletter claiming a competing advisor returned 22% last year, and asks for a response. Most of what it wrote back is sound: question the number, ask what it represents, redirect to the client's own personalized benchmark. Buried in the middle of that reasonable advice is one sentence that isn't reasoning at all:

unknown node

That's a regulatory claim, stated as settled fact, with nothing in the prompt to cite. It may be gesturing at something real (performance-advertising rules do exist), but "reputable advisors are required to show all of that" is an assertion dressed as a citation, and the kind of line a client could repeat back to a regulator as something the firm told them. An advisor forwarding this email is vouching for a compliance claim nobody checked and the model never sourced.

Watch this pattern hardest under any kind of marketing-rule scrutiny: unlike an invented reason for a fee increase, an invented regulatory claim can leave the building and become someone else's evidence. If a draft asserts what "advisors are required to" do, verify it before it ships. Our SEC marketing rule checker is built for exactly that sentence.

Scenario 3 shows the same reflex with numbers. Asked whether a client should put 20% of his portfolio into bitcoin because a neighbor made money on it, the model didn't just counsel caution. It cited drawdown statistics ("70-80% from peak to trough") as settled fact, with no source, and proposed a specific counter-allocation ("something in the range of 1-5%") nobody authorized. Nobody asked for a number. It supplied one anyway, plus a recommendation nobody asked it to make.

We've made a version of this argument before, about a different kind of number: a confident, specific figure is not the same as an accurate one, and the gap between the two doesn't show up in how the sentence reads (our note on why a net worth estimate is a range, not a magic number).

Wrong instrument for the person

Two scenarios fall here, and neither is about a fact the model got wrong. Scenario 9 matters most in this batch: it is about a decision the model had no standing to make.

The prompt: an adult son has asked to be added to his mother's accounts; the mother has early-stage cognitive decline. The model wrote her a warm, careful-sounding, multi-step email. It also did four things nobody asked it to do:

  1. Named the specific legal instrument. It recommended a durable financial power of attorney by name, as the "standard, legally clean way" to handle the request: legal guidance for a client with a diagnosed cognitive condition, no attorney involved.
  2. Put the family matter in the subject line. The subject reads "A Note About [Son's Name]'s Request", a forwardable record naming a family member and a sensitive request, sent to a client whose capacity to manage her own correspondence is the thing in question.
  3. Disclosed the son's outreach, in writing. The email tells the mother her son contacted the advisor, a disclosure with real consequences for that relationship, made without knowing whether it's welcome or safe.
  4. Set the pace of a sensitive family conversation. It proposed a specific sequence (private meeting, then attorney, then paperwork) that may be right, or may not fit anything about this family the advisor knows and the model doesn't.
Breakdown of an AI-drafted email to a client with early-stage cognitive decline, marking the four judgment calls the model made that belong to a person who knows the family.

Four judgment calls, none of them the model's to make.

Every one of those four belongs to a person who knows this family: the client's history, the son's intentions, the stage of her decline, what she can process today. Stephan Shipe, CFP at Scholar Financial Advising, put the general limit well, in comments reported by Lucy Lazarony (syndicated via Yahoo Finance, 2026-07-10): "ChatGPT can be a great tool for explaining retirement strategies or helping you understand your options, but it's not the whole toolbox. It can't anticipate human behavior, emotion, or life's curveballs." Scenario 9 is that sentence made concrete. Nothing in the output is factually false, and the email is fluent, well organized, and legally coherent, which is exactly what makes it dangerous to send as drafted. There's no error to catch on a read-through, only a judgment to catch by already knowing it was never the model's to make.

This is the clearest case in the batch for treating anything client-facing as a draft a human reviews, not a finished product a human skims. Our note on human-in-the-loop review and our own trust page both start from the premise this scenario proves: fluency is not fitness to send.

Scenario 12 is the mild version of the same mismatch. A check-in email to forty clients untouched in over a year opens "I wanted to reach out personally rather than let another year go by": a line built for one recipient, sent unmodified to forty. Nothing in it is false. It is dishonest about its own scale in a way a merge tag can't fix, and only the advisor knows which of the forty deserve a genuinely personal note instead.

What this means for your own emails

Seven of these nine failures share one root cause: the prompt left a gap, and the model filled it rather than flagging it. Robert Persichitte, CFP at Delagify Financial, named the mechanism in the same Lazarony piece: "Remember that AI doesn't currently think critically or form new ideas. It finds existing ideas and connects them." Or, as he put it, "I like to think of it as a fancy Google." A fancy Google returns the most plausible-sounding thing it can connect to what you typed, and a plausible-sounding reason, action, number, or regulation is exactly what it returned in those seven emails.

The fix has nothing to do with abandoning the tool: supply the fact instead of leaving room for the model to supply one for you. The real reason for the fee increase, the client's actual allocation, what was processed and what wasn't — put those in the prompt and there's nothing left to invent. That's the whole difference between the failures cataloged here and the same model, same day, drafting clean client email in our read of the 2026 advisor AI surveys: the prompt carried the facts.

This is worth closing rather than avoiding, because the task isn't going away. Routine client communications already sits among the more commonly logged AI tasks, at 25% in the one survey that breaks tasks out by percentage, a vendor survey of 300 advisors published by wealthtech provider Advisor360°, released 2026-01-13), against 63% RIA adoption overall (Charles Schwab Advisor Services, released 2026-01-22) and 77% of bank advisors expecting to increase their AI use within two years (Cerulli Associates, 2025-09-04). No survey tells you which pattern your own thin prompt will hit; this test is meant to.

Scenarios 9 and 12 don't fit the "supply the missing fact" fix at all, because the problem there was never a missing fact. It was a missing person. Scenario 9 needed a human to have made every judgment call before anything with the client's name on it went anywhere. Before any client email ships, running it past our AI claim auditor catches the inventions the other four patterns describe. It won't catch Scenario 9 or 12. Nothing does, except deciding in advance which categories of email a model never gets handed.

Frequently asked questions

Should I stop drafting client emails with AI?

No. Three of the twelve emails tested here needed no correction, and separate testing on the same model, the same day, with a prompt carrying the client's actual facts, voice, and constraints, produced clean output on the same class of task. Most of the failures above trace to what the prompt left out, not to the category of task.

Which client emails are safest to hand to AI first?

The three that came back clean here (a condolence and process notice, a relationship termination, and a prospect follow-up) share a trait: none required the model to supply a reason, a number, or a judgment call it hadn't been given. Emails explaining why something happened, or touching a family or legal decision, are the ones worth reading twice before they go out.

How do I stop AI from inventing a reason in a client email?

Put the real reason in the prompt. Every invented-causation example here happened because the advisor asked for an outcome ("tell them the fee is going up") without supplying the reason behind it. A model given no cause supplies one that sounds plausible rather than leaving the sentence unfinished. Supply the cause yourself and there's nothing left for it to invent.

What's different about the "hallucination menu" from an ordinary invented fact?

A single invented fact reads as a claim, which a careful reader learns to question. A bracketed menu of fabricated options reads as an unfinished placeholder, the kind advisors fill in constantly, so it survives a skim precisely because it looks incomplete rather than false. All three options are still fabricated; picking one to send is still sending an invented fact, just one selected from a lineup instead of stated outright.