[{"data":1,"prerenderedAt":126},["ShallowReactive",2],{"\u002Fwriting\u002Fwhy-import-pipelines-fail":3,"\u002Fwriting\u002Fwhy-import-pipelines-fail-related":120},{"id":4,"title":5,"body":6,"description":98,"extension":99,"featured":100,"kind":101,"meta":102,"navigation":100,"ogImage":103,"order":94,"path":104,"publishedAt":105,"relatedWork":106,"seo":107,"stem":108,"summary":109,"takeaways":110,"technologies":115,"videoDuration":103,"videoId":103,"__hash__":119},"articles\u002Fwriting\u002Fwhy-import-pipelines-fail.md","Why Ecommerce Product-Import Pipelines Fail",{"type":7,"value":8,"toc":89},"minimark",[9,14,18,22,29,35,41,47,52,56,70,74],[10,11,13],"h2",{"id":12},"they-rarely-fail-loudly","They rarely fail loudly",[15,16,17],"p",{},"The import pipelines that cause real damage don't crash — they succeed partially and quietly. A feed\ntruncated at 80% still \"runs successfully.\" A currency field silently defaulting to the wrong unit\nstill \"imports.\" The failure modes worth designing around are the ones that don't throw an\nexception.",[10,19,21],{"id":20},"common-failure-patterns","Common failure patterns",[15,23,24,28],{},[25,26,27],"strong",{},"Silent partial failures."," A job that processes 8,000 of 10,000 rows and reports success on the\n8,000 is often worse than a job that fails outright — it looks healthy in every dashboard while\nquietly corrupting the catalogue.",[15,30,31,34],{},[25,32,33],{},"Unowned unit and category mapping."," Every supplier has its own idea of units and categories.\nWithout an explicit, versioned mapping layer — not a pile of if-statements — these mismatches\ncompound for months before anyone notices the pattern.",[15,36,37,40],{},[25,38,39],{},"No rollback path."," Eventually a bad import gets published. If there's no way to revert to the\nlast known-good state quickly, that one bad run becomes a multi-day cleanup project instead of a\nfive-minute fix.",[15,42,43,46],{},[25,44,45],{},"Testing that doesn't reflect real chaos."," Import pipelines tested against clean, hand-crafted\nsample files pass every test and still break on day one against a real supplier's actual export —\nbecause real feeds are inconsistent in ways nobody thinks to fabricate in a test fixture.",[48,49],"architecture-diagram",{":steps":50,"title":51},"[{\"label\":\"Feed arrives\",\"detail\":\"Truncated, reordered, re-encoded\"},{\"label\":\"Parsed 'successfully'\",\"detail\":\"Partial data, no error thrown\"},{\"label\":\"Published\",\"detail\":\"Looks healthy in every dashboard\"},{\"label\":\"Discovered\",\"detail\":\"Days later, by a customer or a stockout\"}]","Where pipelines actually break",[10,53,55],{"id":54},"what-actually-holds-up","What actually holds up",[15,57,58,59,64,65,69],{},"Validate at every boundary, diff before publishing, own unit\u002Fcategory mapping explicitly and\nversion it, and keep a fast rollback path for when — not if — something bad gets published anyway.\nI go into how this plays out in practice in the\n",[60,61,63],"a",{"href":62},"\u002Fwork\u002Fcase-studies\u002Fcatalogue-automation","catalogue automation"," and\n",[60,66,68],{"href":67},"\u002Fwork\u002Fcase-studies\u002Fshopify-amazon-sync","Shopify\u002FAmazon sync"," case studies.",[10,71,73],{"id":72},"related","Related",[75,76,77,84],"ul",{},[78,79,80],"li",{},[60,81,83],{"href":82},"\u002Fwriting\u002Fmanaging-product-data-at-scale","How I Manage Product Data Across 200,000+ SKUs",[78,85,86],{},[60,87,88],{"href":62},"High-Volume Ecommerce Catalogue Automation",{"title":90,"searchDepth":91,"depth":91,"links":92},"",3,[93,95,96,97],{"id":12,"depth":94,"text":13},2,{"id":20,"depth":94,"text":21},{"id":54,"depth":94,"text":55},{"id":72,"depth":94,"text":73},"The recurring reasons ecommerce product-import pipelines break in production, from silent partial failures to unvalidated matching, and how to design against each one.","md",true,"article",{},null,"\u002Fwriting\u002Fwhy-import-pipelines-fail","2026-05-05",[62,67],{"title":5,"description":98},"writing\u002Fwhy-import-pipelines-fail","The failure modes I see repeatedly in supplier feed and product-import systems — and how to design around them.",[111,112,113,114],"Silent partial failures are more dangerous than loud total failures","Unit and category mapping problems compound over time if not owned explicitly","A pipeline without a rollback path will eventually need one","It worked in testing",[116,117,118],"Python","ETL design","Data validation","3TXeA4s4fGvXQuEsJx66q-hyUQjRWGJ9zlIygJIggwI",[121,123],{"path":62,"title":88,"summary":122},"Automating ingestion, cleanup, matching, and pricing for 200,000+ supplier SKUs.",{"path":67,"title":124,"summary":125},"Shopify and Amazon Marketplace Synchronization","Keeping inventory, pricing, and listings consistent across Shopify and Amazon SP-API.",1785688079476]