Most of Databricks management can be done from the GUI or CLI, but for Azure Service Principal, we can only manage it by the SCIM API. There's an open PR for adding support of SCIM API in Databricks CLI, but the lastest update is back to the beginning of 2021.
This post is to add some tips that not covered by the official API docs.
This post can be an extend to my previous post on variables and templates reuse
In fact, in addition to the variables and templates, I also need to reuse some non native Azure pipeline yaml files, for example some Python scripts defined in the shared template. If we use the same technic shown by the previous blog, the pipeline will throw error saying that it cannot find the Python script. This is because we need to checkout the remote repository at first before using the native pipeline yaml files.
In my project, I have several Azure pipelines that share some same variables, instead of declaring them in each pipeline, I would like to refactor it by using some central places to store the shared variables.
The official doc gives an explanation of all the predefined variables, but it lacks of some concrete examples. Hereunder some examples for my preferred variables.
Python unittest and Pytest is a big deal, this post just gives some small & quick examples on how to use Python unittest framework, especially with Pytest framework. This post is not finished yet.
Relative to pipenv, and poetry, if you're searching for some lightweight python package managers for a small project, I will introduce 2 handy tools for you: pipreqs and pip-tools.