azure#

Github Actions: Python

Github Actions: Cache

Github Actions: Custom Actions

Calling Azure REST API

This blog Calling Azure REST API via curl is pretty good. Just two more things.

Azure pipeline Windows agent UnicodeEncodeError

For people who encounter UnicodeEncodeError when using Windows Azure Pipeline agent, the issue might be here.

As per above link, or this email, the solutions could be:

• You can override just sys.std* to UTF-8 by setting the environment variable PYTHONIOENCODING=UTF-8.
• You can override all I/O to use UTF-8 by setting PYTHONUTF8=1, or by passing the command-line option -X utf8.

Azure pipeline delete blobs from blob storage

The example given by this post is for Azure Pipeline with the latest Ubuntu agent, for AzCli from local machine, removing the --auth-mode login part should work.

As it's a Linux pipeline agent, the pipeline task AzureFileCopy can not be used, it's written in Powershell, we should use the AzureCLI task instead.

Databricks cluster access mode

Azure pipeline System.AccessToken in shared pipeline

Azure pipeline jobs

Databricks job/task context

Suppose we're running following job/task in a Azure Databricks workspace:

jobId: "1111"
jobRunId: "2222"
taskRunId: "3333"
jobName: "ths job name"
taskName: "first-task"
databricksWorkspaceUrl: https://adb-4444444444.123.azuredatabricks.net/

Run below command in a Databricks job (task precisely):

dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson()