Someone asked me a question this week that I answered in thirty seconds and then could not stop thinking about: how do you know, as a product manager, where to implement AI in a product?
It sounds like a beginner question. It is actually the question, because every PM I talk to right now is squeezed between a CEO who wants AI on the homepage, a board that wants an AI story for investors, and users who mostly want the product to work.
You know where AI belongs by finding where your users stall, not where the demo shines. AI earns a place in your product only where users abandon a task, where a wrong answer is cheap to fix, and where no dumber solution does the job.
Everything else is decoration that you will pay for in tokens, complexity, and trust.
The Roadmap for Today
Two products, opposite answers: Why DraftKit has AI in one place and my advent calendar has none
95% of pilots fail at the same first step: What the failure data says about the question everyone asks backwards
The Stall Map: The five questions I run before any AI feature gets a sprint. And with a BONUS tool you can use today!
Running the Map on my own two products: Every question, both answers, side by side
The subtraction close: A diagnostic you can run on your roadmap tonight
Hey, I’m Elena! 👋 I’m an AI Product Manager and solo builder. I ship apps using Lovable, document what actually works, and write Prompt-Led Product for the PMs who want to build real things and lead through evidence instead of managing roadmaps.
🚀 If you build with AI or manage AI products, you are in the right place.
If you are new here, three posts to catch up:
I Break AI Tools So You Don’t Have To: how I stress-test the tools everyone else just reviews.
Why Claude.md Is the New PRD: the documentation shift that changed how I build.
Your AI Chatbot Is Lying to Customers: what happens when AI features ship without guardrails.
I Built Two Products and Gave Only One of Them AI
Here is the thing that makes me trust my own answer to this question. I run two live products, and they landed on opposite sides of it.
DraftKit is a collaboration platform for Substack writers. It has AI in one narrow spot: when two writers approve a collaboration, and don’t know how to pass the blank page, the system could help generating a first draft outline from both of their content libraries, so they never start from an empty page.
AI Advent Challenge is a 25-day interactive calendar that teaches AI skills through daily lessons and challenges. It was built with AI, it teaches AI, and it contains zero AI-powered features. Users never interact with a model. Not once.
A product about AI with no AI inside it, and a writing product with AI in exactly one place. Neither decision was an accident, and both came from the same test.
That test is what this post gives you. But first, the reason you need one at all.
95% of AI Pilots Fail Because the Question Was “Where Can We Use AI”
The industry ran a very expensive experiment on this question over the last two years, and the results are in.
MIT’s State of AI in Business 2025 report found that 95% of generative AI pilots delivered no measurable P&L impact, across an estimated $35 to 40 billion in enterprise spend.
S&P Global reported that 42% of companies abandoned most of their AI initiatives in 2025, up from 17% a year earlier. And Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing unclear business value as a lead cause.
Read those three numbers together and one pattern emerges. The projects did not fail because the models were weak. They failed because the placement decision was made before anyone defined the problem.
❌ What most builders assume: Pick the most visible surface in the product, add AI to it, and the value will reveal itself once users try it.
✅ What is actually true: AI placed at a visible surface instead of a friction point becomes a feature users try once, screenshot for LinkedIn, and never open again.
I see this in almost every product I audit. The AI feature lives where the demo needed it, on the dashboard, in the hero flow, next to the logo. The actual abandonment point, the place where users quietly give up and churn, has no AI near it and often no attention at all.
Strategic advice
Before you evaluate any AI feature request, ask the requester to name the user behavior it changes. If the answer describes a capability (”it can summarize anything”) instead of a behavior (”users stop abandoning setup at step three”), the feature is positioning, and positioning belongs on your landing page, at near-zero cost, instead of in your codebase.
Writers I respect in the PM space have been circling the same conclusion. Aakash Gupta has spent months telling AI PMs that the job is evaluating where models actually change a workflow’s outcome, and that this skill, not prompt writing, is what separates the role from regular PM work.
The Stall Map: Five Questions That Decide Where AI Belongs
I call my version of this decision the Stall Map, because it starts by mapping where users stall instead of where AI would look impressive. Five questions, in strict order. A feature must pass all five.
✅ 1. Where do users stall? Pull your funnel and find the step where completion drops hardest, or the task users start and abandon. AI candidates exist only at stall points. If nobody stalls there, AI has nothing to fix there.
✅ 2. What does a wrong answer cost? Models are probabilistic, so plan for the bad output, because it will happen. A wrong first draft costs one edit. A wrong billing amount, a wrong medical suggestion, or a wrong deletion costs trust you never get back. AI belongs where wrong answers are cheap and reversible.
✅ 3. Does a dumber solution work? If the fix can be a rule, a default, a template, or a reminder email, the deterministic version wins. It is cheaper, testable, and behaves the same way every time. AI only earns the slot when the input is too varied for rules to handle.
✅ 4. Can you afford the right answer? Every model call has a unit cost, and the cost scales with success. A feature that fires once per paying event is safe. A feature that fires on every action of every free user is a margin leak you approved yourself.
✅ 5. Can you name it by its outcome? If the only honest name for the feature is “AI-powered,” you built technology, and technology is nobody’s job to be done. If you can name what the user gets, the feature has a reason to exist.
What did I learn? 🤔
Deterministic code and probabilistic models fail differently, and that difference drives questions 2 and 3. Deterministic code, meaning ordinary if-this-then-that logic, fails loudly and identically every time, so you catch it once and fix it forever. A model fails quietly, differently each time, and often convincingly. You place each one where its failure style does the least damage.
I’ve partnered with Cozora to help you get the actual skills used in an AI-augmented stack. No jargon, just the technical execution required to ship. Get 10% off to start architecting the systems that actually run.
Running the Map on My Own Two Products
Theory is cheap, so here is the Map applied to both of my builds, question by question.
DraftKit’s stall point was brutal and specific. Two writers would match, agree to collaborate, get excited, open the shared workspace, and then face a blank page belonging to two people who had never written a single sentence together. Collaborations died right there, after every hard step was already done.
Run the questions against that moment:
Stall: confirmed, the drop-off sat between approval and first edit.
Wrong answer cost: one edit, the draft is a starting point humans rewrite.
Dumber solution: templates failed, a generic outline fits nobody’s two voices.
Unit cost: one generation per approved collaboration, a bounded and rare event tied to the product’s core value.
Outcome name: SMART Draft, a first draft, never “AI”.
5️⃣ Five passes. That is why DraftKit generates first drafts, and why that is nearly the only place a model runs in the entire product.
The Advent Calendar failed the Map at question one, everywhere. The core loop is open a door, read a lesson from a real practitioner, complete a challenge, keep a streak. The stall points in that loop are timing and motivation, and every one of them had a deterministic fix: unlock timers, streak logic, reminder emails.
An AI tutor inside the lessons would have failed question 2 as well, because a model that teaches AI concepts wrong damages the credibility of the 37 human contributors whose names sit on those doors.
And it would have failed question 4 spectacularly: thousands of players making daily model calls against a free product is a margin fire with no revenue on the other side.
❌ What most builders assume: A product in the AI space needs AI features to be credible.
✅ What is actually true: A product needs its core loop to work. Users keeping a 25-day streak never once asked where the AI was.
There is one more DraftKit decision hiding in question five that I want to make explicit. I renamed every AI feature in the product to SMART, and the word AI does not appear in the interface.
Users do not want AI, they want the draft, the match, the suggestion. The moment the technology becomes the feature’s name, you have admitted the feature has no outcome to be named after.
Want to evaluate your own feature? Use my Stall Map tool to find out! It’s completely free! 🥳
Want to build something by yourself?
Check out the Build Series. Part 1 | Part 2 | Part 3 | Part 4 | Part 5
The Best Answer to “Where Should We Add AI” Is Usually a Shorter List
When your CEO asks where the AI is, the pressure is to answer with additions. The Stall Map usually answers with subtractions, and that takes more nerve to present.
But here is what the exercise buys you.
Instead of walking into that conversation with “we could add AI to search, onboarding, and reports,” you walk in with
“our users stall at one moment, a wrong answer there costs nothing, no rule-based fix works, it fires once per paying event, and we can name what it does without saying AI.”
One of those PMs is guessing and determined to fit AI where it makes sense in the roadmap. The last one is the one actually running the product.
Tonight’s diagnostic: take the top three roadmap items that mention AI and write one line for each:
Stall it fixes: [the funnel step and the drop-off number]
Cost of a wrong answer: [what the user loses, and whether they can undo it]
The dumber fix you rejected: [the rule or template, and why it failed]
Any line you cannot fill in with real data is the line your CEO’s competitors also cannot fill in. That gap is your actual competitive position, in either direction.
If you would rather run it than write it, I built the interactive version: The Stall Map walks you through the five questions for one feature and hands you the verdict plus a PRD-ready justification paragraph you can paste straight into your roadmap doc. Free, no signup, and nothing you type leaves your browser.
And as always, stop guessing. Start building. 🚀
One question for you: Which feature on your roadmap has “AI” in its name right now, and what is the stall it claims to fix? Drop it in the comments. I will tell you exactly how I would run the Stall Map on your specific feature.
If you want me to run this evaluation across your whole product before your next planning cycle, reach out at hello@elenacalvillo.com.
One thing I'd add to the test: check whether the task involves judgment under ambiguity or just retrieval and formatting. AI features nobody asked for tend to cluster around the second category, because it's easy to demo and hard to justify skipping. The features users actually adopt usually replace a step where they were making a guess, not a step where they were just waiting for data to load.
So, so true - thank you Elena - I'm seeing so many leaders drawn in to launching EVERYTHING and doing MORE now that engineering is (seemingly) cheaper - but it creates a mess not just for maintenance but for users. Adding all 25 requested features is not a strategic path to deepening PMF or a coherent user experience. Applying energy to known stall points is a good guide!