allow MI endpoint changing through environment variable

[jimdigriz]

Proposal to make local development work easier, for example where you are using SSH tunnelling to utilise the credentials assigned to an actual instance.

This allows the developer to run their code with something like this in one terminal point to a real Azure instance with an identity:

ssh -L 8000:169.254.169.254:80 192.0.2.1

Then their code could run using:

env IMDS_ENDPOINT=http://localhost:8000 func start

IMDS_ENDPOINT looks to make sense and I am hoping will not clash with anything; inspection of azure.identity looks like IDENTITY_ENDPOINT would also need to be set before anything goes off piste.

Let me know if this looks like it is something you would accept and what else it needs to be made suitable for inclusion.

[rayluo]

Your use case is meaningful. We would love to support that.

IMDS_ENDPOINT looks to make sense and I am hoping will not clash with anything; inspection of azure.identity looks like IDENTITY_ENDPOINT would also need to be set before anything goes off piste.

The name choice would potentially be problematic here, because there are many flavors of managed identity (MI) and those popular env vars would likely be used by some of them. Let's go with a new env var name which is unlikely to be used by something other than MSAL library.

Let's use MSAL_MANAGED_IDENTITY_ENDPOINT and define its content be a full url, so its usage would be something like MSAL_MANAGED_IDENTITY_ENDPOINT=http://localhost:1234/metadata/identity/oauth2/token.

Lastly, it would be great if you can also document this new behavior as a new Note (such as "If you want your local development to use a managed identity on your remote Azure VM, you can ..."), add it into line 136, so that it will be rendered into our docs.

[rayluo]

Need a different env var name, and documentation.

[jimdigriz]

Let's use MSAL_MANAGED_IDENTITY_ENDPOINT and define its content be a full url, so its usage would be something like MSAL_MANAGED_IDENTITY_ENDPOINT=http://localhost:1234/metadata/identity/oauth2/token.

The override here needs to be to be where to find the instance metadata server (orgin), not the full URL for the token endpoint.

For example as well as fetching identities elsewhere in code, I also need to query the IMDS for /metadata/instance too and it would be helpful to reuse the same variable elsewhere.

Of course, the "where my IDMS is" use case may be out of scope so a reasonable answer would be "sorry, you will need to use your own environment variable for the IMDS its-self and MSAL_MANAGED_IDENTITY_ENDPOINT for the identity endpoint".

Asking as this will affect documentation.

Actually MSAL_MANAGED_IDENTITY_ENDPOINT pointing to the identity endpoint works for me.

I forgot I already to use IMDS_ENDPOINT as Azure Arc enrolled systems use this and so my own can borrow this for local dev anyway.

Submitted an update, let me know if you want the comment wording amended or anything else.

Thanks for the prompt and useful feedback!

[jimdigriz]

@microsoft-github-policy-service agree company="coreMem Limited"

[jiasli]

This is a very useful feature.

However, even though the public doc already contains it: https://msal-python.readthedocs.io/en/latest/#managed-identity

it is still not available in any MSAL releases.

[bgavrilMS]

@jiasli @jimdigriz - SSH tunnelling will not work with the new improvements we are making to managed identity. Instead, we are looking at a different approach for testing.

[jimdigriz]

@bgavrilMS I hope part of that work is for real instances the SDK(s) is updated to check /sys/devices/virtual/dmi/id/chassis_asset_tag to determine if it is worthwhile even attempting to reach out to an IMDS?

On a side note, saying "this will not work" is somewhat concerning when then it is left up in the air hanging on what the actual alternative is likely to look like...when it will happen...what I need to do.... Mostly as it makes things in-actionable at my end other than "panic! panic! panic!" until an announcement is made...whenever that may be.

[rayluo]

@jimdigriz , sorry, there is not much detail available at this moment, but we will have to revert this change, at least for now. We will provide more updates when the new solution becomes available.

Meanwhile, since this previously merged PR has become part of this repository's permanent history, you could still use that change from its commit 7db6c2c or its branch, should you choose to do so.

In any case, this situation shall not "make things in-actionable" at your end, because this PR was meant to be used in some one-off testing only. It is not meant to be a foundation to build your in-production system upon.

On a side note, the future change - if any - will NOT be about check /sys/devices/virtual/dmi/id/chassis_asset_tag. That check requires sudo, but MSAL library will never require sudo to be run.

Thanks again for your willingness to contribute. We hope this new inconvenience will not prevent you from continue using MSAL library.

P.S.: Feel free to reach out to me on my email for further explanations.

[jimdigriz]

On a side note, the future change - if any - will NOT be about check /sys/devices/virtual/dmi/id/chassis_asset_tag. That check requires sudo, but MSAL library will never require sudo to be run.

/sys/devices/virtual/dmi/id/chassis_asset_tag is globally readable, maybe you are mixing it up with {chassis,product,board}_serial which is readable only by root?

alex@sarasti:~$ uname -a
Linux sarasti.home.arpa 6.1.0-27-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.115-1 (2024-11-01) x86_64 GNU/Linux
alex@sarasti:~$ ls -l /sys/devices/virtual/dmi/id/
total 0
-r--r--r-- 1 root root 4096 Nov 16 08:26 bios_date
-r--r--r-- 1 root root 4096 Nov 16 08:26 bios_release
-r--r--r-- 1 root root 4096 Nov 10 14:39 bios_vendor
-r--r--r-- 1 root root 4096 Nov 10 14:39 bios_version
-r--r--r-- 1 root root 4096 Nov 16 08:26 board_asset_tag
-r--r--r-- 1 root root 4096 Nov 10 19:19 board_name
-r-------- 1 root root 4096 Nov 16 08:26 board_serial
-r--r--r-- 1 root root 4096 Nov 10 14:39 board_vendor
-r--r--r-- 1 root root 4096 Nov 16 08:26 board_version
-r--r--r-- 1 root root 4096 Nov 16 08:26 chassis_asset_tag   <------------------
-r-------- 1 root root 4096 Nov 16 08:26 chassis_serial
-r--r--r-- 1 root root 4096 Nov 10 14:39 chassis_type
-r--r--r-- 1 root root 4096 Nov 16 08:26 chassis_vendor
-r--r--r-- 1 root root 4096 Nov 16 08:26 chassis_version
-r--r--r-- 1 root root 4096 Nov 16 08:26 ec_firmware_release
-r--r--r-- 1 root root 4096 Nov 10 14:39 modalias
drwxr-xr-x 2 root root    0 Nov 16 08:26 power
-r--r--r-- 1 root root 4096 Nov 10 19:19 product_family
-r--r--r-- 1 root root 4096 Nov 10 14:39 product_name
-r-------- 1 root root 4096 Nov 16 08:26 product_serial
-r--r--r-- 1 root root 4096 Nov 10 19:19 product_sku
-r-------- 1 root root 4096 Nov 16 08:26 product_uuid
-r--r--r-- 1 root root 4096 Nov 10 14:39 product_version
lrwxrwxrwx 1 root root    0 Nov 10 14:39 subsystem -> ../../../../class/dmi
-r--r--r-- 1 root root 4096 Nov 10 14:39 sys_vendor
-rw-r--r-- 1 root root 4096 Nov 10 14:39 uevent

Unless something has changed since between kernel 6.1 (what I am using) and the latest 6.11 which makes this no longer viable?

I have used this handy check in my own code for a while now, I assumed Azure offered this as a standard "root not needed" method to determine if running on an Azure instance (or Function) similarly to how GCP/AWS do too?

[rayluo]

On a side note, the future change - if any - will NOT be about check /sys/devices/virtual/dmi/id/chassis_asset_tag. That check requires sudo, but MSAL library will never require sudo to be run.

/sys/devices/virtual/dmi/id/chassis_asset_tag is globally readable ... Unless something has changed since between kernel 6.1 (what I am using) and the latest 6.11 which makes this no longer viable?

The link - which was quoted from your earlier message - brought me to a page mentioning something like sudo dmidecode, so I thought you were talking about something needing sudo. We can look into it. Regardless, that is really a "VM detection" topic which is different than the original topic in this PR, which deserves its own github issue. Please keep the current github PR discussion pertinent.

[rayluo]

Update:

• We ended up not using this PR as-is.

• But, as a byproduct of the POD Identity support shipped in MSAL Python 1.32, it is now possible to change Azure VM's Managed Identity endpoint through env var.
  For example, if you create your SSH tunnel this way (assuming your VM is on 192.0.2.1):

  ssh -L 8000:169.254.169.254:80 192.0.2.1
  

  Then your code could run locally using:

  env AZURE_POD_IDENTITY_AUTHORITY_HOST=http://localhost:8000 python your_script.py