A survey that came out last week caught my eye: 76% of Americans now use AI tools regularly. But the majority of those same users say they don't trust the results.
So they use AI. Then manually verify.
This looks like a paradox. It's actually rational. But from a product perspective, it ends up somewhere strange: the user is using the tool to save time, and then spending extra time verifying the output. What's the net gain?
Trust isn't a legal problem.
Most AI products approach the trust issue like this: add a disclaimer, write "AI can make mistakes," include some fine print. Done.
That doesn't actually solve anything. Users already know AI can be wrong. The real question is: when is it wrong? Without knowing that, they approach every output with the same level of skepticism — high.
In healthcare software, this is especially visible. When a clinician sees a diagnosis suggestion on screen, how do they know whether to trust it? They're not going to read a disclaimer. They want to know how reliable the model is in that specific context.
Most products don't show that.
Calibrated confidence
The real design challenge in AI products isn't accuracy — it's communicating what the system knows versus what it doesn't, at the right level for the user.
This isn't about showing a confidence percentage. Most users don't know what to do with a number like "87% confidence."
It's more like:
- "This recommendation is based on 400 similar cases" vs. "This is based on 3 similar cases"
- "This information was updated last month" vs. "This data is from two years ago"
- When the model is uncertain, saying "I'm not sure about this" instead of confidently offering three options
Claude does this sometimes. It says "my knowledge here is limited — I'd recommend verifying this." And strangely, that makes me trust it more, not less.
What PMs actually see
When you do user research, people rarely say "I don't trust the AI output." They say "I double-check sometimes" or "it gets things wrong occasionally." But the underlying behavior is the same: they're re-evaluating every output from scratch.
That's a UX problem. And it's one product teams need to own.
Because trust debt creates a subtler problem than adoption resistance. A user who uses the product but doesn't feel like it's helping — they'll quietly leave. The churn shows up, but the real reason doesn't.
Two years from now, the AI products that win won't be the ones with the most accurate models. They'll be the ones that were honest about their limits.