Installing a certificate from Azure KeyVault into a machine external to Azure

Why?

In Installing a certificate from Azure KeyVault into an Azure VM, a certificate was stored as a secret in a JSON format. The KeyVault was enabled for deployment so that the Microsoft.Compute resource provider has access. This configuration is needed to enable using Azure Powershell to install certificates on Azure hosted VMs. A similar technique can be used to install certificates stored in Azure KeyVault in machines external to Azure. The problem is an alternative agent to the Microsoft.Compute resource provider will be needed and that agent will need to be given a service principal.

What?

Azure Powershell will be used to manage ARM-based resources including Azure Active Directory (AD) and KeyVault. Login-AzureRmAccount needs to be called before resources can be managed. When settings up continuous integration (CI) and continuous deployment (CD) pipelines it is a best practice to use a service principal with Role Based Access Control (RBAC) rather than a user’s Microsoft Account. This technique follows the principle of least privilege. The service principal is used to access the secret stored in KeyVault. The secret JSON payload is deserialized to obtain the PFX certificate and password used to install into the target machine’s credential store.

Remember that Azure Powershell does not expose a direct way to export certificates that have been imported. This is why certificates need to be serialized with the password and stored in KeyVault as a secret using Set-AzureKeyVaultSecret. Secrets can be retrieved using Get-AzureKeyVaultSecret and deserialized to obtain the PFX certificate and password.

How?

The first step is to create an Azure AD application that will encapsulate the service principal and roles.

Figure 1: Create an Azure AD application [2]
$adApp = New-AzureRmADApplication `
-DisplayName "[name]" `
-HomePage "[app id uri]" `
-IdentifierUris "[app id uri]" `
-Password [password]

The -HomePage and -IdentifierUris are synonymous with the App ID URI shown in the Azure Portal for an Azure AD application. -Password is synonymous with the keys generated from the portal. By default the password will expire in 1 year. The password can be omitted. The Azure Portal can be used to generate more keys later.

The Azure Portal can also be used to create a new Azure AD application [1]. The client ID is the Azure AD application ID. This can be validated by calling Get-AzureRmADApplication. The following assumes an Azure Powershell session has been established by calling Login-AzureRmAccount.

Figure 2: Validate the Azure AD application ID is the same as the client ID [5].
PS> Get-AzureRmADApplication -ApplicationId [client ID]

DisplayName : [Azure AD application display name]
Type : Application
ApplicationId : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
ApplicationObjectId : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
AvailableToOtherTenants : False
AppPermissions : {}
IdentifierUris : {http://[application url]}
ReplyUrls : {http://[application url]}

The service principal can be created after the Azure AD application is created.

Figure 3: Create the service principal using the Azure AD application ID [2].
New-AzureRmADServicePrincipal -ApplicationId $adApp.ApplicationId

The Azure AD application name is also used as the service principal name. This can be validated by searching for the service principal by calling Get-AzureRmADServicePrincipal and supplying a -SearchString matching the application name. The application ID can also be specified using -ServicePrincipalName. The exact object id can also be specified using -ObjectId.

Figure 4: Get-AzureRmADServicePrincipal cmdlet syntax [6].
Parameter Set: ObjectIdParameterSet
Get-AzureRmADServicePrincipal -ObjectId <Guid> [ <CommonParameters>]

Parameter Set: SearchStringParameterSet
Get-AzureRmADServicePrincipal -SearchString <String> [ <CommonParameters>]

Parameter Set: SPNParameterSet
Get-AzureRmADServicePrincipal -ServicePrincipalName <String> [ <CommonParameters>]

The service principal must be granted access to the KeyVault. The following assumes a KeyVault has been created. See Setting up Azure KeyVault using Powershell for how to create one.

Figure 5: Grant the service principal access to KeyVault [7].
Set-AzureRmKeyVaultAccessPolicy `
-VaultName "[vault name]" `
-ServicePrincipalName "[client id]" `
-ResourceGroupName "[resource group name]" `
-PermissionsToSecrets 'get' `
-PassThru

The ByServicePrincipalName parameter set is used to specify a target KeyVault and Azure AD application client ID. The -ResourceGroupName is specified to improve search performance. -PermissionsToSecrets is used to grant only ‘get’ permissions to secrets. Finally, -PassThru is used to return the PSVault object. The PSVault object can be used to validate the operation had the intended effect.

Figure 6: Output of -PassThru switch.
Vault Name : [vault name]
Resource Group Name : [resource group name]
Location : Central US
Resource ID : /subscriptions/[subscription id]/resourceGroups/[resource group name]/providers/Microsoft.KeyVault/vaults/[vault name]
Vault URI : https://[vault name].vault.azure.net/
Tenant ID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
SKU : Standard
Enabled For Deployment? : True
Enabled For Template Deployment? : False
Enabled For Disk Encryption? : False
Access Policies : 
 Tenant ID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 Object ID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 Application ID : 
 Display Name : John Doe (jdoe@abc.com)
 Permissions to Keys : get, create, delete, list, update, import, backup, restore
 Permissions to Secrets : all
 
 Tenant ID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 Object ID : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 Application ID : 
 Display Name : [service principal name] ([Azure AD App ID URI])
 Permissions to Keys : 
 Permissions to Secrets : get

A successful operation will list the new access policy under “Access Policies”. An entry can be seen for the new [service principal name] with the ‘get’ permission to secrets. The other entry is the access policy for the account that created the KeyVault. This account has all permissions to keys and secrets.

Figure 7: Set-AzureRmKeyVaultAccessPolicy cmdlet syntax [7].
Parameter Set: ByServicePrincipalName
Set-AzureRmKeyVaultAccessPolicy `
[-VaultName] <System.String> `
[[-ResourceGroupName] <System.String> ] `
-ServicePrincipalName <String> `
[-PassThru] `
[-PermissionsToKeys {decrypt | encrypt | unwrapKey | wrapKey | verify | sign | get | list | update | create | import | delete | backup | restore | all}[] ] `
[-PermissionsToSecrets {get | list | set | delete | all}[] ] `
[ <CommonParameters>]

By now the Azure AD application and service principal have been created. The service principal has been granted access to a KeyVault. It is now time to validate the service principal can get secrets from KeyVault. Start by clearing all cached Azure accounts from the local machine. This is needed to ensure that operations are performed using the service principal and not another account.

Figure 8: Clearing cached Azure accounts [8].
Get-AzureAccount | ForEach-Object { Remove-AzureAccount -Name $_.Id -Force }

Get-AzureAccount is used to get a list of all cached Azure accounts on the current machine. This output is piped into a loop that calls Remove-AzureAccount on each account in the list output. Remove-AzureAccount requires the -Name be specified. The -Force switch is to skip user confirmation. The $_ token is used to reference the current item in the list being looped over. In this case it is an instance of a cached Azure account.

The service principal needs to be added once all cached accounts have been removed.

Figure 9: Adding a service principal account [11].
$clientId = "[client id]"
$key = ConvertTo-SecureString –String "[key]" –AsPlainText -Force

$credential = New-Object `
–TypeName "System.Management.Automation.PSCredential" `
–ArgumentList $clientId, $key
Add-AzureRmAccount `
-Credential $credential `
-Tenant 'xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx' `
-ServicePrincipal

The client ID and key for the service principal will be needed to create a PSCredential object. The credential object is passed into Add-AzureRmAccount along with the tenant ID. The -ServicePrincipal switch tells the cmdlet the credentials are for a service principal.

Figure 10: Add-AzureRmAccount cmdlet syntax [11].
Parameter Set: ServicePrincipal
Add-AzureRmAccount `
-Credential <PSCredential> ` # Required
-ServicePrincipal ` # Required
-TenantId <String> ` # Required
[-Environment <AzureEnvironment> ] `
[-EnvironmentName <String> ] `
[-SubscriptionId <String> ] `
[-SubscriptionName <String> ] `
[ <CommonParameters>]

Secrets can be queried from KeyVault using Get-AzureKeyVaultSecret to test whether the service principal was successfully added.

Figure 11: Query for a secret to validate the service principal was configured successfully [12].
$secret = Get-AzureKeyVaultSecret -VaultName "[vault name]" -Name "[secret name]"

$secret

Output:
Vault Name : [vault name]
Name : [secret name]
Version : a0624fde74ee48dcbdc1dfdb5a632af8
Id : https://[vault name].vault.azure.net:443/secrets/[secret name]/a0624fde74ee48dcbdc1dfdb5a632af8
Enabled : True
Expires : 
Not Before : 
Created : 7/5/2016 5:00:42 AM
Updated : 7/5/2016 5:00:42 AM
Content Type : 
Tags :

The service principal was only given ‘get’ permissions. Attempting to ‘list’ all secrets yeilds an error.

Figure 12: Listing secrets yields a permission error.
Get-AzureKeyVaultSecret -VaultName "[vault name]"

Output:
Get-AzureKeyVaultSecret : Operation "list" is not allowed
At line:1 char:1
+ Get-AzureKeyVaultSecret
+ ~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : CloseError: (:) [Get-AzureKeyVaultSecret], KeyVaultClientException
 + FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.GetAzureKeyVaultSecret

Once the service principal gets the target secret the JSON payload can be deserialized into a PFX certificate with the password.

Figure 13: Deserialize the secret JSON payload into a PFX certificate and password [13].
$jsonObjectBytes = [System.Convert]::FromBase64String($secret)
$jsonObject = [System.Text.Encoding]::UTF8.GetString($jsonObjectBytes)

$jsonObject

Output:
{
"data": "[base64 encoded certificate byte array]",
"dataType": "pfx",
"password": "[key]"
}

$customObject = ConvertFrom-Json $jsonObject
$customObject.data

Output:
[base64 encoded certificate byte array]

From here the “data” field can be deserialized into a PFX certificate that can be installed into a machine’s credential store.

Figure 14: Install the PFX certificate into the local machine’s credential store [15].
# Deserialize and save the PFX file.
$pfxBytes = [System.Convert]::FromBase64String($customObject.data)
[io.file]::WriteAllBytes("[path to]/[filename].pfx", $pfxBytes)

# Convert password to secure string.
$password = ConvertTo-SecureString -String $customObject.password -Force –AsPlainText

# Install the PFX certificate into the Cert:\LocalMachine\My certificate store.
Import-PfxCertificate `
–FilePath [path to PFX file] `
-CertStoreLocation cert:\localMachine\my `
-Password $password
Figure 15: Get-ChildItem can be used to verify the certificate has been added.
Get-ChildItem Cert:\LocalMachine\My

Works Cited

  1. Automating Azure on your CI server using a Service Principal
  2. Use Azure PowerShell to create a service principal to access resources
  3. Azure Key Vault Cmdlets
  4. Azure Powershell 1.5.1
  5. Get-AzureRmADApplication
  6. Get-AzureRmADServicePrincipal
  7. Set-AzureRmKeyVaultAccessPolicy
  8. How To Remove Azure Accounts (Cached Credentials) From PowerShell Remove-AzureAccount for ALL Accounts Step-By-Step
  9. Remove-AzureAccount
  10. Get-AzureAccount
  11. Add-AzureRmAccount
  12. Get-AzureKeyVaultSecret
  13. ConvertFrom-Json
  14. PFX Certificate in Azure Key Vault
  15. Import-PfxCertificate

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: