A few months ago I was in a backlog refinement session when an engineer said, "We could use RAG for that." I nodded along. I had a rough sense of what RAG meant — retrieval-augmented generation, something about feeding documents to a model — but I couldn't have told you when it made sense versus fine-tuning, or what the cost implications were.
I left that meeting knowing I had a gap. Not a gap that required me to become a machine learning engineer. But a gap that was quietly limiting how useful I could be in decisions that increasingly involved AI.
That's the situation most product managers are in right now. And I think it's worth being honest about it.
Why "Delegate It to Engineering" No Longer Works
The default PM instinct is to treat AI as an engineering problem: describe what you want, hand it to the team, review the output. That worked fine when AI was a feature accent — autocomplete here, a recommendation widget there.
It doesn't work anymore when AI is a core capability of your product.
Here's the problem in practice. When you don't understand the underlying mechanics, you can't scope features accurately. You estimate AI tasks like any other development task, miss the data dependencies, and end up with a sprint that's blown two weeks before the demo. You also can't push back intelligently when engineering says something is hard — you don't know if "hard" means technically complex or just unfamiliar territory.
More importantly, you start making subtle product decisions that are wrong in ways you don't recognize. Promising features to users that require training data you don't have. Designing flows that assume a model will behave consistently when it won't. Building on capabilities that will become expensive to maintain at scale.
None of these are catastrophic individually. Compounded over a year of product decisions, they add up to a slower product and a team that trusts your judgment less.
The Three Questions You Need to Be Able to Ask
You don't need to understand backpropagation. You need to understand enough to ask the right questions in the room.
"Do we have the data for this?" Most AI features require training data, feedback loops, or structured inputs that don't exist yet. Asking this early — before anyone has written a line of code — surfaces a common source of delays that teams rarely discuss explicitly.
"Is this a prompt engineering problem or a model problem?" These have very different cost and timeline implications. Prompt engineering is fast and cheap to iterate. Custom model work is slow, expensive, and often unnecessary. Many AI features that get scoped as "train a model" can actually be solved with a well-designed prompt. You need to know enough to have this conversation.
"What happens when it's wrong?" Every AI system produces errors. The interesting question is: what does a wrong output look like to the user, and what's the recovery path? This is a product design question, not an engineering one. But you can only ask it meaningfully if you have a mental model of how the system can fail.
Where to Start (Without Going Deep)
The goal isn't fluency — it's enough literacy to be effective. Here's what's actually moved the needle for me:
Use the tools constantly. Not just for work tasks. Use ChatGPT, Claude, Perplexity, and whatever else is available in ways that push their limits. Ask them to do things they're bad at. The intuition you build about where models fail is directly applicable to product decisions.
Build a small vocabulary. You don't need a textbook. You need to know what a token is (and why it has cost implications), what an embedding is (and why it matters for search features), what hallucination means (and why it requires product guardrails), and what the difference is between RAG and fine-tuning. That's roughly four concepts. An afternoon is enough.
Talk to your engineers differently. Instead of handing off a spec and waiting, ask: "What's the trickiest assumption in this?" or "What would need to be true about our data for this to work?" Engineers generally appreciate PMs who engage at this level, and you'll surface blockers weeks earlier.
Read the error messages. When an AI feature behaves unexpectedly in testing, resist the instinct to immediately file a bug. Try to understand why it happened. This habit alone has changed how I write acceptance criteria for AI features.
The Real Risk of Staying Vague
The PM who treats AI as a black box doesn't disappear — they just become less and less relevant to the most important product decisions. Engineering makes calls that should involve product thinking. Roadmap commitments get made on shaky assumptions. The product drifts toward what's technically convenient rather than what users need.
AI literacy isn't about becoming technical. It's about staying in the room where the decisions are made.
The best time to start was a year ago. The second best time is now — and the learning curve is genuinely not steep.
If you're a PM who's been avoiding the AI deep-dive, what's held you back? I'm curious whether it's time, confidence, or something else entirely.