Every year I do a rough count of how many ADF pipelines I'm actively maintaining for clients. At the end of 2020, that number is significant enough that I'd feel the loss if ADF disappeared tomorrow. That was not true at the end of 2018.
Something shifted in 2020, and it wasn't just the platform maturing.
The Pandemic Accelerated What Was Already Coming
I had clients in March 2020 who were "evaluating cloud." By June, the same clients were in production. The pandemic removed the organizational inertia that had kept "cloud is the direction, but not yet" conversations alive for years. When your on-premises infrastructure team suddenly couldn't get to the data center, "not yet" became "now."
ADF benefited from this directly. Organizations that had been running ADF alongside SSIS -- using ADF for new projects but keeping SSIS for the established pipelines -- started retiring the SSIS infrastructure. The parallel operation phase ended. Cloud ETL went from "the new approach for new projects" to "the production system."
That's a meaningful maturity signal. Organizations don't decommission established production infrastructure for an experiment. They do it when the replacement has proven itself.
What Made the Platform Ready for That Trust
The managed infrastructure story was fully proven by 2020. I can point to ADF pipelines I deployed in 2018 that have been running without infrastructure intervention since. Zero server patches. Zero capacity planning. Zero 3am calls because a process died. The reliability of managed services at cloud scale is real.
The Managed Virtual Network feature shipped in 2020 and closed the last major enterprise objection: "we can't put our data sources on the public internet." Private endpoints through a managed VNet means ADF can now reach any Azure private resource without a Self-Hosted IR VM. That was the network security gap that kept some enterprise clients on SSIS longer than they otherwise would have been.
The connector breadth crossed 90 in 2020. The conversation in client scoping sessions stopped being about connectivity and started being about design patterns. That's where you want the conversation to be.
The Technical Wins That Compounded
Three years of CI/CD pattern refinement landed on a clean, stable approach: pre-deployment trigger stop, ARM template deployment with environment parameter files, post-deployment trigger restart. Teams that adopt this pattern have repeatable, auditable deployments across Dev/Test/Prod. The first deployment takes a day to wire up; after that, deploying pipeline changes takes minutes.
Data Flows reached a performance and feature level where they're a first choice for Spark-scale transformation, not a "let's try it and see" experiment. With cluster TTL keeping execution times predictable and the Optimize tab providing real tuning levers, I'm running production Data Flows on datasets that would have required a dedicated Databricks cluster two years ago.
The Open Question Heading Into 2021
The Synapse question is unresolved and it's the one that matters most for anyone making platform investment decisions. Synapse Pipelines is functionally equivalent to ADF, Microsoft's investment in Synapse is visible, and the long-term product consolidation trajectory seems clear even if the timeline isn't.
My prediction for 2021: Synapse Pipelines gets more features and better first-party documentation than ADF does. ADF gets maintained but the leading-edge investment goes to the Synapse surface. The product story consolidation continues toward whatever Microsoft calls the next version of this platform.
That doesn't mean stop using ADF. It means build portable, parameterized pipelines on well-understood patterns so that when the migration conversation comes -- if it comes -- the cost is low.
2020 was the year cloud ETL stopped being an experiment for most of my clients. 2021 is going to be about optimizing what's already in production and navigating the Synapse decision. I'll be writing about both. As always, I'm here to help.