A client's security team has disabled public network access on every Azure data resource in production. Azure SQL: private endpoint only. Storage account: no public access. Azure Key Vault: private endpoint required. The network is locked down correctly.
Now ADF needs to read from all of them.
The old answer: Self-Hosted IR on a VM inside the VNet, manage the VM, keep it patched, ensure it's available. The new answer: Managed Private Endpoints inside a Managed VNet. Here's how to set it up and what to expect in production.
The Architecture
When you enable Managed Virtual Network on an Azure Integration Runtime, ADF provisions a dedicated managed VNet under Microsoft's control, associated with your ADF instance. You then create Managed Private Endpoints from this VNet to your private Azure resources.
Data path for a Copy Activity reading from Azure SQL with public access disabled:
ADF Pipeline
-> Azure IR (running inside Managed VNet)
-> Managed Private Endpoint (inside Managed VNet)
-> Private Endpoint Connection (approved in Azure SQL)
-> Azure SQL Database (public access disabled)
End-to-end private. No public internet traversal. No VM to manage.
Setup Walkthrough
Step 1: Enable Managed Virtual Network on the Azure IR
In ADF Studio, go to Manage then Integration Runtimes. Edit your Azure IR (or create a new one). Under "Virtual network configuration," enable "Managed virtual network." Save.
Important: you cannot change this setting on an existing IR after it's been saved. If you have an existing Azure IR without Managed VNet and you want to add it, you need to create a new IR, migrate your linked services to use it, and delete the old one. Do this migration carefully in a dev environment first.
Step 2: Create a Managed Private Endpoint
In ADF Studio, go to Manage then Managed Private Endpoints, then New. Select the resource type (Azure SQL Database, Azure SQL Managed Instance, Azure Storage, etc.). Specify the subscription, resource group, and the specific resource. ADF submits a private endpoint connection request to that resource.
Step 3: Approve the Private Endpoint Connection
Navigate to your target resource in the Azure portal. For Azure SQL: Security then Private endpoint connections. You'll see a connection in "Pending" state from ADF. Select it and click Approve.
Approval propagation takes 2-5 minutes. After that, the managed private endpoint in ADF Studio shows "Approval state: Approved."
Step 4: Update the Linked Service to Use the Managed VNet IR
Edit your Azure SQL linked service. Under "Connect via integration runtime," select your Managed VNet-enabled IR. Test the connection. It should succeed without public network access enabled on the SQL resource.
Production Behavior: What to Expect
Cold Start Time
Managed VNet IRs have a longer cold start than standard Azure IRs. Standard: 1-2 minutes. Managed VNet: 3-6 minutes for the first run after the cluster has been idle. ADF is provisioning resources inside the managed VNet during this startup window.
Mitigation: enable Quick Reuse (TTL) on the IR. Set the TTL based on your pipeline frequency. For pipelines running every hour, a 30-minute TTL keeps the cluster warm. The second and subsequent runs within the TTL window start in under 30 seconds.
TTL Cost Model
You pay for the Managed VNet IR during the TTL window even without active pipeline runs. For a General Purpose 8-core cluster with 30 minutes TTL running 24 pipelines per day:
- Active execution: approximately 5 minutes per run times 24 runs = 120 minutes per day
- TTL idle: approximately 25 minutes per interval times 24 intervals = 600 minutes per day
- Total billed: 720 minutes per day, 84% idle
For batch workloads where runs are infrequent and long, a short TTL is more cost-efficient. Size the TTL to your actual run frequency.
Handling Multiple Private Resources
You can create multiple Managed Private Endpoints from the same Managed VNet IR -- one per private resource you need to reach. An IR can have private endpoints to Azure SQL, Azure Data Lake, Azure Key Vault, and Azure Service Bus simultaneously. Each private endpoint needs to be created and approved separately.
When You Still Need SHIR
Managed Private Endpoints cover Azure-to-Azure private connectivity. On-premises sources still require Self-Hosted IR. For hybrid architectures -- some cloud-private sources, some on-premises -- you'll run both: a Managed VNet Azure IR for the cloud-private resources and a SHIR for on-premises. Link the appropriate IR to each linked service and the architecture composes cleanly.
If you're building a private-network ADF architecture and want to review the design before implementing, I'm here to help.