Skip to content

azure#

Some nice cicd bash common scripts

During CICD, we often have a large log output, it might be nice to have some common scripts to help us to format the log output, so that we can easily find the information we need.

Recently, when working with Sonar, I found that they have some scripts for such output formatting.

Hashing files

During CI/CD processes, and particularly during CI, we frequently hash dependency files to create cache keys (referred to as key input in Github Action actions/cache and key parameter in Azure pipelines Cache@2 task). However, the default hash functions come with certain limitations like this comment. To address this, we can use the following pure Bash shell command to manually generate the hash value.

Github Actions: copdips/get-azure-keyvault-secrets-action

Recently, I began a new project that requires migrating some process from Azure Pipelines to Github Actions. One of the tasks involves retrieving secrets from Azure Key Vault.

In Azure Pipelines, we have an official task called AzureKeyVault@2 designed for this purpose. However, its official counterpart in Github Actions, Azure/get-keyvault-secrets@v1, has been deprecated. The recommended alternative is Azure CLI. While Azure CLI is a suitable option, it operates in a bash shell without multithreading. If numerous secrets need to be fetched, this can be time-consuming.

Azure pipeline Windows agent UnicodeEncodeError

For people who encounter UnicodeEncodeError when using Windows Azure Pipeline agent, the issue might be here.

As per above link, or this email, the solutions could be:

  • You can override just sys.std* to UTF-8 by setting the environment variable PYTHONIOENCODING=UTF-8.
  • You can override all I/O to use UTF-8 by setting PYTHONUTF8=1, or by passing the command-line option -X utf8.