The short answer
Securing data platforms in Azure requires isolating storage accounts, databases, and pipelines inside private networks to prevent data exfiltration and eliminate exposure to the public internet.
Private Endpoints and Azure Private Link
By default, Azure PaaS services like ADLS Gen2 and Azure SQL expose public internet endpoints protected only by firewall rules and authentication:
- Azure Private Endpoint: Creates a virtual network interface with a private IP address assigned from your virtual network subnet, attaching it directly to the target storage account or database.
- Disabling public access: Once Private Link is active, engineers set storage networking rules to deny all public traffic, forcing every request to traverse Microsoft private network backbone.
- Managed VNet in ADF: Azure Data Factory supports Managed Virtual Networks, allowing serverless runtimes to communicate with ADLS Gen2 through private endpoints without provisioning self-hosted proxy servers.
Azure Data Factory (Managed VNet)
| (Private Endpoint / Private IP)
v
ADLS Gen2 Storage Account (Public Internet Access = Disabled)
^
| (Private Endpoint / Private IP)
Azure Key Vault (Secrets & DB Credentials fetched via Managed Identity)Secret management with Azure Key Vault
must never store connection strings, API tokens, or encryption passwords in pipeline definitions or code repositories.
Azure Key Vault stores credentials centrally. Pipeline activities retrieve secrets dynamically at runtime by authenticating to Key Vault using their Entra ID managed identity, ensuring zero exposure of plaintext secrets.