OIDC (OpenID Connect) is an authentication layer built on top of the OAuth 2.0 protocol (authorization layer). It allows Web-based, mobile, and JavaScript clients to verify the identity of end-users based on the authentication performed by an authorization server (aka AS or IdP), as well as to obtain basic profile information about the end-user in an interoperable and REST-like manner. It eliminates storing and managing people's passwords.
There are two modern ways to generate an Azure OAuth2 access token using Python: one is by using the MSAL library, and the other is by using the Azure Identity library, which is based on the former.
There're also other ways to get the token, like using the requests or aiohttp libraries etc. to send a POST request to the Azure OAuth2 token endpoint, but it's not recommended. As the MSAL and Azure Identity libraries are the official libraries provided by Microsoft, they are more secure and easier to use. For e.g. they handle token caching, token refreshing, and token expiration automatically. Furthermore, some of the credential types are difficult (too many code) to be implemented by raw requests or aiohttp.
Before the Databricks Unit Catalog's release, we used init scripts stored in DBFS to generate the pip.conf file during cluster startup, allowing each cluster its unique auth token. But with init scripts no longer available in the Unit Catalog's shared mode, an alternative approach is required.