Your Integration Works in Testing and Goes Blank for Real Users. This Is Why.
How I built a three-tier publication ID resolution chain for DraftKit's Network feature, and why a silent API failure costs more than a loud one.
Check out the Build Series. Part 1 | Part 2 | Part 3 | Part 4 | Part 5
Unlock the full breakdown + 100+ PM Prompts.
Paid subscribers get immediate access to every deep dive and full entry to the PM Prompt Vault. Stop writing prompts from scratch and steal the exact workflows used to ship real products.
Upgrade your subscription to keep reading and unlock the vault.
You ship an integration with a third-party API. Everything passes in testing. The demo looks clean. Three days after launch, a user messages you to say their page is empty.
You test the same flow on your own account and it loads immediately. You test it on a colleague’s account and it loads there too. By the time the user sends you a screenshot, their data has appeared and the issue looks resolved.
What actually happened is that the API has multiple response formats depending on how the underlying resource was originally created. Your code handled one format. The others returned nothing, silently, and you never knew.
This is exactly what happened when I built the Network feature in DraftKit. The feature pulls Substack recommendation data so creators can find potential collaborators from the writers they already recommend.
My first implementation worked correctly for most DraftKit creators and silently returned nothing for a meaningful portion of the rest.
Those creators opened the Network page, saw an empty list, and had no way to know whether the feature was broken or they just had no recommendations yet.
The fix required building a three-tier resolution chain inside the fetch-substack-recommendations edge function. Each tier is a different method for resolving a Substack publication ID.
E.g. If Tier 1 returns nothing, the function falls through to Tier 2. If Tier 2 fails, it falls through to Tier 3.
The entire chain runs in a single function call and the user never sees the fallback logic.
If your product has a third-party integration powering a user-facing feature, this pattern is worth building before you ship, not after. If you want me to look at where your integrations are failing silently, reach out at hello@elenacalvillo.com.
Substack Has 100K Earning Publications and No Single Standard ID Format
Backlinko reports that nearly 100,000 publications earn money on Substack as of April 2026, up from 50,000 in May 2025. The platform crossed 5 million paid subscriptions with 67% year-over-year growth. Every one of those publications has a Substack publication ID, and those IDs can be resolved in at least three different ways depending on when the publication was created, whether it uses a custom domain, and whether it has been indexed by Substack’s search infrastructure.
The oldest and most established publications tend to use a numeric ID format that resolves cleanly through the Substack Search API. Publications created after certain platform migrations have handles that resolve correctly through RSS metadata but return inconsistent results from Search. Publications on custom domains may not appear in Substack Search at all, even when the RSS feed is fully accessible via direct fetch.
The creator who reports an empty Network page is almost always the creator with the edge-case publication format. They are also often your most established user, which makes the failure more visible and the trust cost higher.
The same pattern appears across any platform that has evolved its data model over multiple years. Twitter (now X), LinkedIn, Shopify, and Substack all have legacy ID formats sitting in production that their own search APIs handle inconsistently. Building to a single endpoint is a choice to fail silently for the users who joined before your integration was designed.
The Reason PMs Do Not Catch This Before Launch
Most PM teams test integrations with accounts created during the development period. Those accounts use the current ID format, which is the one the primary API endpoint handles reliably. The legacy accounts, the custom domain accounts, and the accounts that migrated between platform versions are sitting in production with real users who have been on the platform for years.
Lenny Rachitsky has written about the gap between what PM teams test and what veteran users actually experience. The users who have been around longest are the ones most likely to surface edge cases, because their data predates the assumptions baked into your integration.
Testing with fresh accounts in a development environment will never reproduce a failure mode that only appears in accounts created under a different data model.
The QA pass tells you the feature works for users like you. It does not tell you what happens to users who joined before the current format was established.
The Approach I Follow
✅ Map every ID format the third-party API supports before writing integration code. For Substack, that meant cataloguing three resolution paths: the Search API, the RSS metadata route, and the direct URL parse. Spending an afternoon on this mapping cost me nothing. Skipping it cost me a meaningful silent failure rate in production across a portion of my creator base.
✅ Build the fallback chain as a single function with one entry point and one return type. In DraftKit, resolveSubstackPublicationId(substackUrl) is called once and returns either a resolved ID or null. The caller never needs to know which tier resolved it. Keeping the fallback logic encapsulated means it can be updated in one place without touching every call site in the codebase.
✅ Log which tier resolved each request. Tier distribution data tells you exactly how many of your users are on legacy paths. In DraftKit’s edge function, every successful resolution logs the tier used. That log showed me the exact shape of the problem and confirmed that Tier 3 correlated almost exactly with custom-domain publications.
✅ Set a timeout per tier and a hard timeout for the full chain. If Tier 1 takes more than 2 seconds to respond, you do not want the chain waiting for it before falling through to Tier 2. Each tier in DraftKit has its own AbortController with an independent timeout. The full chain times out at 8 seconds, at which point the function returns null and the UI surfaces a retry button rather than a blank page.
✅ Treat the null result as a distinct UI state, not an error. When the resolution chain returns null, the Network page shows “We could not find your recommendations yet” with a manual retry option. A retry button is not a great user experience, but it is better than a page that looks like it loaded successfully and contains nothing. The user needs to know something is different, even if the message is soft.
If your integration is live and you have not mapped its fallback behavior, I can audit it. Email hello@elenacalvillo.com.