Testing GPU access with Azure Local and AVD

· · , , , , ,

Tagged: , , , , ,

Recently I’ve been doing more testing with Azure Local, specifically with 23H2 and testing the deployment process, configuration options, and more. In this short post I wanted to write up on some testing I have been doing with GPU accelerated Virtual Machines.

Note that GPU assignment and passthrough are currently in preview – see here.

This will be the first in a series of posts covering Azure Local… hopefully!

To start this process I have the following deployed:

  • Azure Local 23H2 running on a single node deployment.
  • The Azure Local node enrolled to Azure Arc and the Azure Local deployment process completed.
  • 2x AVD Session Hosts deployed on Azure Local, running Windows 11 (win11-24h2-avd-01)
Setting up the Azure Local Node and GPU:

Within the Azure Portal this now shows as a ready to use Azure Local node:

And we can see our AVD VMs setup ready for use too:

A quick note when setting up Azure Local 23H2. I did experience an issue with Cluster Roles during deployment when running the deployment in Azure – it failed when creating the cluster. I was able to get around this by installing the roles first, and then retrying:

Enter-PSSession -ComputerName NAME
Install-WindowsFeature -Name "BitLocker", "Data-Center-Bridging", "Failover-Clustering", "FS-FileServer", "FS-Data-Deduplication", "Hyper-V", "Hyper-V-PowerShell", "RSAT-AD-Powershell", "RSAT-Clustering-PowerShell", "NetworkATC", "Storage-Replica" -IncludeAllSubFeature -IncludeManagementTools

GPU assignment on Azure Local depends on the GPU available – I’m testing with an L4 and thus will be using GPU-P:

Please Note – part of the below setup requires NVIDIA Drivers and Licensing. I have configured this test with a Trial License, which you can register for and download from NVIDIA. If you are moving to production you will need to source and procure appropriate licensing.

We then setup the required drivers on the host machine – this requires installing the drivers via PowerShell:

pnputil /add-driver .\nvgridswhci.inf /install /force

Further details on the driver install and the rest of this process, are available here: https://learn.microsoft.com/en-us/azure/azure-local/manage/gpu-preparation?view=azloc-24112#download-and-install-the-host-driver.

Once this process is completed, you can run the following command to see the GPU hardware:

Get-PnpDevice -Class Display | fl FriendlyName, ClusterId

Next we need to oartition the GPU – I’m using two partitions as I have two test VMs:

Set-VMHostPartitionableGpu -Name "GPU NAME" -PartitionCount 2

Then we can see this has been partitioned as requested, into two partitions:

We can now test using the nvidia-smi command:

Next up, we can setup our VMs and test!

Configuring the Azure Local Virtual Machines

We next need to attach the GPU to the Azure Local VMs – this requires the VM to be shut down, and then the GPU can be attached using the following command – this is being done across both of my AVD Session Hosts:

az stack-hci-vm gpu attach --resource-group "rg-local" --vm-name avd-0 --gpus GpuP
az stack-hci-vm gpu attach --resource-group "rg-local" --vm-name avd-1 --gpus GpuP

Once this process is completed, we need to install the NVIDIA drivers onto the VMs, and configure the licensing. Please refer to the NVIDIA documentation for this – as it may vary depending on your hardware configuration.

Once appropriately configured, including drivers and licensing – we can see the available GPU within Device Manager:

Testing

For testing I wanted to confirm three test scenarios, as this will confirm capability, and also prove the ability to run GPUs locally via Azure Local:

  1. GPU information via GPU Utilities (for example GPU-Z and nvidia-smi)
  2. GPU usage via Applications (for example, Google Earth)
  3. Local AI usage (for example using GPT4ALL).

To start all tests, I logged in via the Windows App, and launched a session, and ran step 1 of my tests:

Looking good so far – I then moved on to an application that could utilise the GPU, performing A/B testing to compare a GPU accelerated session to one without. Due to limitations in my own setup I was unable to capture these on video, but the results were clear – all graphical applications performed as expected with a GPU, and without a GPU they provided “choppy”, inconsistent and sluggish graphical performance, which was as expected.

I then moved onto testing with GPT4ALL, to allow the GPU to be put to use working with AI models. On an optimised machine – you can see the GPU working well and providing capability:

As you can see, the GPU is being used to generate responses within GPT4ALL – completing the basic tests I had set out.

Whilst the above is a general overview, followed by some basic testing, I hope it was useful – I will be blogging more on this in the coming weeks/months!

Resources / Wider Reading:
Skip to content