In today’s data-driven world, securing access to your data resources is paramount. Azure Managed Identities provide a seamless and secure way to authenticate and authorize access to Azure services without the need for credentials in your code. In this blog, we’ll explore how to use Managed Identities to connect Azure Data Factory (ADF) with Azure Cosmos DB.
Introduction to Managed Identities
Managed Identities are a feature of Azure Active Directory (Azure Entra ID) that allows Azure services to authenticate to other Azure services without storing credentials in your code. This is particularly useful for services like Azure Data Factory, which often need to access data stored in Azure Cosmos DB. For more information about Managed Identities, refer the official documentation available from Microsoft at https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview.
Setting Up Managed Identities
To use Managed Identities with Azure Data Factory and Azure Cosmos DB, follow these steps:
Create a Managed Identity for Azure Data Factory
- Navigate to your Azure Data Factory instance in the Azure portal.
- Under the “Identity” section, enable the “System-assigned managed identity”.
Assign Roles to the Managed Identity
First, let’s identify the roles required for Azure Data Factory to effectively perform its various operations on Azure Cosmos DB for NoSQL. According to the official documentation, the following roles are necessary:
System-Assigned Managed Identity
- Reader Role: This role is necessary to read the metadata of the resources.
- Contributor Role: This role is required to manage the resources that the managed identity needs to access.
User-Assigned Managed Identity
- Managed Identity Operator Role: This role allows the management of the user-assigned managed identity.
- Reader Role: This role is necessary to read the metadata of the resources.
- Contributor Role: This role is required to manage the resources that the managed identity needs to access.
In this blog, we are mainly focusing on the System-Assigned Managed Identity. Hence, we will be configuring only the Reader and Contributor roles. Considering this, the following roles need to be assigned to the managed identity in Azure Cosmos DB for NoSQL:
- Reader Role: Cosmos DB Built-in Data Reader
- Contributor Role: Cosmos DB Built-in Data Contributor
For more details on these roles, you can refer to the https://docs.azure.cn/en-us/cosmos-db/nosql/security/reference-data-plane-roles
Challenges with Role Visibility
One common challenge when working with Azure Cosmos DB is that the roles “Cosmos DB Built-in Data Reader” and “Cosmos DB Built-in Data Contributor” may not be visible in the Azure portal. This is because these are data plane roles, and Microsoft Azure only displays control-plane roles in the user interface. To configure these roles, we need to use alternatives such as Azure CLI.
Assigning Roles Using Azure CLI
To assign the necessary roles using Azure CLI, follow these steps:
1. Open Azure CLI
- You can use the Azure Cloud Shell in the Azure portal or install Azure CLI on your local machine.
2. Assign the Cosmos DB Built-in Data Contributor Role
$account_name="<CosmosDBResourceName>"
$resource_group="<ResourceGroupName>"
$scope="/"
$principal_id="<ManagedIdentityPrincipalId>"
az cosmosdb sql role assignment create --account-name $account_name --resource-group $resource_group --scope $scope --principal-id $principal_id --role-definition-name "Cosmos DB Built-in Data Contributor"3. Assign the Cosmos DB Built-in Data Reader Role
$account_name="<CosmosDBResourceName>"
$resource_group="<ResourceGroupName>"
$scope="/"
$principal_id="<ManagedIdentityPrincipalId>"
az cosmosdb sql role assignment create --account-name $account_name --resource-group $resource_group --scope $scope --principal-id $principal_id --role-definition-name "Cosmos DB Built-in Data Reader"Replace <ManagedIdentityPrincipalId> with the principal ID of your managed identity, <CosmosDBResourceName> with the name of your Azure Cosmos DB account, and <ResourceGroupName> with the name of the Resource Group in which your Azure Cosmos DB is available.
Verifying Permissions
To verify that the roles have been assigned correctly, you can use the following Azure CLI command:
az role assignment list --assignee <principal-id> --scope <scope>Replace <principal-id> with the principal ID of your managed identity and <scope> with the scope of your Azure Cosmos DB account. This command will list all the role assignments for the specified managed identity, allowing you to confirm that the necessary roles have been assigned.
Conclusion
Using Managed Identities to connect Azure Data Factory with Azure Cosmos DB enhances security by eliminating the need for credentials in your code. This approach not only simplifies the authentication process but also aligns with best practices for securing access to your data resources.
By following the steps outlined in this blog, you can leverage Managed Identities to securely and efficiently manage data flows between Azure Data Factory and Azure Cosmos DB.