Blog#

bash-git-prompt tweaks

Some tweaks I made to bash-git-prompt. dynamic Python virtualenv path, new var GIT_MESSAGE, etc.

Rendering CSV as Markdown table in Mkdocs Material

Markdown table is not easy to write in IDE. So I want to write CSV and render it as Markdown table. Previously I used a pre-build script during CICD time to convert CSV to Markdown table, and referenced it as --8<-- "markdown_table_file_path.md". But it's not convenient to preview the result locally. So I want to find a way to render CSV as Markdown table in Mkdocs Material directly.

Migrating my blog from Jekyll Minimal Mistakes to Mkdocs Material

After using Jekyll Minimal Mistakes for years, I decided to migrate my blog to MkDocs Material, as it's written in Python and I'm more familiar with it.

Python local version identifiers

Python local version identifiers are used to distinguish between different builds of the same version of a package. They are used to indicate that a package has been modified in some way from the original source code, but should still be considered the same version.

Some nice cicd bash common scripts

During CICD, we often have a large log output, it might be nice to have some common scripts to help us to format the log output, so that we can easily find the information we need.

Recently, when working with Sonar, I found that they have some scripts for such output formatting.

Github actions: deploy static files to azure web app

Although Azure provides already a GitHub Actions for Azure Web App to deploy static files to Azure Web App, but we can also do it ourselves with a azure cli command.

Github Actions: Bash shell -e -o pipefail

Bash shell in Github actions by default is run with -e -o pipefail option. The full command used by Github actions is :

shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}

Hashing files

During CI/CD processes, and particularly during CI, we frequently hash dependency files to create cache keys (referred to as key input in Github Action actions/cache and key parameter in Azure pipelines Cache@2 task). However, the default hash functions come with certain limitations like this comment. To address this, we can use the following pure Bash shell command to manually generate the hash value.

Github Actions: copdips/get-azure-keyvault-secrets-action

Recently, I began a new project that requires migrating some process from Azure Pipelines to Github Actions. One of the tasks involves retrieving secrets from Azure Key Vault.

In Azure Pipelines, we have an official task called AzureKeyVault@2 designed for this purpose. However, its official counterpart in Github Actions, Azure/get-keyvault-secrets@v1, has been deprecated. The recommended alternative is Azure CLI. While Azure CLI is a suitable option, it operates in a bash shell without multithreading. If numerous secrets need to be fetched, this can be time-consuming.

Databricks Python pip authentication

Before the Databricks Unit Catalog's release, we used init scripts stored in DBFS to generate the pip.conf file during cluster startup, allowing each cluster its unique auth token. But with init scripts no longer available in the Unit Catalog's shared mode, an alternative approach is required.