Azure Lab Services – creating an effective and reliable testing environment

Azure Lab Services (formerly DevTest Labs) is designed to allow for the rapid creation of Virtual Machines for testing environments. A variety of purposes and use cases can be serviced using DevTest Labs, for example, Development Teams, Classrooms, and various Testing environments.

The basic idea is that the owner of the Lab creates VMs or provides a means to create VMs, which are driven by settings and policy, all of which is configurable via the Azure Portal.

The key capabilities of Azure Lab Services are:

  • Fast & Flexible Lab Setup – Lab Services can be quickly setup, but also provides a high level of customization if required. The service also provides built in scaling and resiliency, which is automatically managed by the Labs Service.
  • Simplified Lab Experience for Users – Users can access the labs in methods that are suitable, for example with a registration code in a classroom lab. Within DevTest Labs an owner can assign permissions to create Virtual Machines, manage and reuse data disks and setup reusable secrets.
  • Cost Optimization and Analysis – A lab owner can define schedules to start up and shut down Virtual Machines, and also set time schedules for machine availability. The ability to set usage policies on a per-user or per-lab basis to optimize costs. Analysis allows usage and trends to be investigated.
  • Embedded Security – Labs can be setup with private VNETs and Subnets, and also shared Public IPs can be used. Lab users can access resources in existing VNETs using ExpressRoute or S2S VPNs so that private resources can be accessed if required. (Note – this is currently in DevTest Labs only).
  • Integration into your workflows and tools – Azure Lab Services provides integration into other tools and management systems. Environments can automatically be provisioned using continuous integration/deployment tools in Azure DevTest Labs.

You can read more about Lab Services here: https://docs.microsoft.com/en-us/azure/lab-services/lab-services-overview

I’m going to run through the setup of the DevTest Labs environment, and also run through a few key elements and run through the use cases for these:

Creating the Environment:

This can be done from the Azure Portal – just search for “DevTest Labs” and then we can create the Lab Account. Note – I have left Auto-shutdown enabled (this is on by default at 1900 with no notification):

Once this has been deployed, we are able to view the DevTest Labs Resource overview:

From here we can start to build out the environment and create the various policies and settings that we require.

Configuring the Environment

The first port of call is the “Configuration and Policies” pane at the bottom of the above screenshot:

I’m going to start with some basic configuration – specifically to limit the number of Virtual Machines that are allowed in the lab (total VMs per Lab) and also per user (VMs per user). At this point I will also be setting the allowed VM sizes. These are important configuration parameters, as with these settings in place we effectively limit our maximum compute cost:

[total number of VMs allowed in the lab] x [maximum/most expensive VM size permitted] = the maximum compute cost of the lab environment

This is done using the panes below:

First up, setting the allowed VM sizes. For this you need to enable the setting and then select any sizes you wish to be available in the Lab. I have limited mine to just Standard_B2s VMs:

Once we have set this up as required we just need to click “Save” and then we can move onto the “Virtual Machines per user” setting. I am going to limit my users to 2 Virtual Machines each at any time:

You’ll notice you can also limit the number of Virtual Machines using Premium OS disks if required. Once again – just click “Save” and then we can move onto the maximum number of Virtual Machines per Lab:

As you can see I have limited the number of VMs in the Lab to a maximum of 5. Once we have clicked “Save” we have now configured a few of the basic elements for our Lab.

Defining VM Images for our Lab

Next up – it’s time to configure some images that we want to use in our Lab. We have three options here – which provide a number of different configurations that suit different Lab requirements:

Marketplace images – these are images from the Azure Marketplace, much like we are used to selecting when creating Virtual Machines in the Portal

Custom images – these are custom images uploaded into the Lab, for example containing bespoke software or settings not available via the Marketplace or a Formula.

Formulas – these allow for the creation of a customised deployment based on a number of values. These values can be used as-is or adjusted to change the machine deployed. Formulas provide scope for customisation within defined variables and can be based on both Marketplace and Custom images.

For more information on choosing between custom images and formulas this is well worth a read: https://docs.microsoft.com/en-us/azure/lab-services/devtest-lab-comparing-vm-base-image-types

I’ve defined a single Marketplace Image from the Portal – and provided only Windows 10 Pro 1803:

Next, I am going to create a Formula based on this image, but also with a number of customisations. This is done by clicking on “Formulas” and then “Add”:

Next, we can configure the various settings of our Formula, but first we need to setup the Administrator username and password in the “My secrets” section of the lab. This data is stored in a Key Vault created as part of the Lab setup, so that they can be securely used in Formulas:

Next, I am going to create a Windows 10 Formula with a number of applications installed as part of the Formula, to simulate a client PC build. This would be useful for testing out applications against PCs deployed in a corporate environment for example. When we click “Formulas” and then  “Add” we are presented with the Marketplace Images we defined as available in the earlier step:

Marketplace Image selection as part of the Formula creation:

Once the base has been selected we are presented with the Formula options:

There are a couple of things to note here:

  • The user name can be entered, but the password is what we previously defined in the “My secrets” section
  • The Virtual Machine size must adhere to the sizes defined as available for the lab

Further down the options pane we can define the artifacts and advanced settings:

Artifacts are configuration and software items that are applied to the machines when built – for example, applications, runtimes, Domain Join options etc. I’m going to select a few software installation options to simulate a client machine build:

There are a few very useful options within other artifacts, which I feel deserve a mention here:

  • Domain Join – this requires credentials and a VNET connected to a Domain Controller
  • Download a File from a URI – for example if we need to download some custom items from a specific location
  • Installation of Azure PowerShell Modules
  • Adding a specified Domain User to the Local Admins group – very useful if we need all testing to be done using Domain Accounts and don’t want to give out Local Administrator credentials
  • Create an AD Domain – if we need a Lab domain spun up on a Windows Server Machine. Useful if an AD Domain is required temporarily for testing
  • Create a shortcut to a URL on the Public Desktop – useful for testing a Web Application on different client bases. For example we could test a specified Website against a number of different client builds.
  • Setup basic Windows Firewall configuration – for example to enable RDP or to enable/disable the Firewall

It is also worth noting that we can define “Mandatory Artifacts” within the Configuration and Policies section – these are artifacts that are applied to all Windows or Linux VMs created with the Lab:

After artifact selection we can specify the advanced settings for the Lab:

It is worth noting here that we can specify an existing VNET if required – this is particularly useful if we need to integrate the Lab VMs into existing environments – for example an existing Active Directory Domain. Here we can also configure the IP address allocation, automatic delete settings, machine claim settings, and the number of instances to be created when the formula is run.

Once the Formula is created we can see the status:

Granting access to the Lab

We can now provide access to end users – this is done from the Lab Configuration and Policies pane of the Portal:

We can then add users from our Azure Active Directory to the Lab Environment:

Visit this URL for an overview of the DevTest Lab Permissions: https://docs.microsoft.com/en-us/azure/lab-services/devtest-lab-add-devtest-user

Now we can start testing the Lab environment logged in as a Lab User.

Testing the Lab Environment

We can now start testing out the Lab Environment – to do this, head over to the Azure Portal and log in as a Lab User – in this case I am going to log in as “Labuser1”. Once logged into the Portal we can see the Lab is assigned to this user:

The first item I am going to do is to define a local username and password using the “My secrets” section – I won’t show this here but you need to follow the same process as I did earlier in this post.

Once we have accessed the Lab, we can then create a Virtual Machine using the “Add” button:

This presents the Lab user with a selection of Base Images – both Marketplace (as we previously defined) and Formulas (that we have previously setup):

I’m going to make my life easy – I’m a lab user who just wants to get testing and doesn’t have time to install any software… so a Formula is the way to go! After clicking on the “Windows10-1803_ClientMachine” Formula I just need to fill out a few basic details and the VM is then ready to provision. Note that the 5 artifacts we setup as part of the Formula and the VM size is already setup:

Once we have clicked create the VM is then built and we can see the status is set to “Creating”:

After some time the VM will show as Running:

Once the VM has been created we can connect via RDP and start testing. When creating this VM I left all of the advanced settings as defaults – which means as part of the first VM deployment, a Public IP and Load Balancer (so that the IP can be shared across multiple Lab VMs) has been created. When we now look at the VM overview window, we can just click connect as we normally would to an Azure VM:

Once we have authenticated, we can then use the VM as we would any other VM – note in the screenshot below, both Chrome and 7Zip (previously specified artifacts) are visible and have been installed (along with other items) for us before we access the VM:

When we have finished our testing or work on this VM – we have a number of options we can use:

  • Delete the VM – fairly self explanatory this one… the VM gets deleted
  • Unclaim the VM – the VM is then placed into the pool of Claimable VMs so that other Lab users can claim and use them. This is useful if you wish to simply have a pool of VMs that people use and then return to a Pool. For example – in a development team testing different OS versions or Browsers etc.
  • Stop the VM – this is the same as deallocating any Azure VM – we’d only pay for the Storage use when stopped

Hopefully this has been a useful overview of the DevTest Labs offering within Azure… congratulations if you made it all the way to the end of the post! Any questions/comments feel free to reach out to me via my contact form or @jakewalsh90 🙂

Azure CDN – Speeding up WordPress on Azure App Service and proving the results

It’s probably no secret that half of the IT blogs out there are running on WordPress or a similar Platform. WordPress is easy to use, simple, and requires little maintenance to run – what’s not to love?!

As with any Website – it doesn’t matter how good the back-end code or server hosting the site is, if your user is half way around the globe from where your site is hosted, the experience may not be that great… and not great experiences do not make for happy visitors. In the case of business/shopping websites this can mean international customers getting a bad experience for example, which is less than ideal if you are looking to grow and provide the same great service to users around the globe.

To combat this issue, a Content Delivery Network (CDN) is a great solution. A CDN essentially spreads your data across geographically separate servers across the world, and ensures that user requests are dealt with by the server that is closest to the end user. It is worth noting this means closest in networking distance, which is not always the same as physical distance. Azure has a great CDN offering with Points of Presence spread all over the world: https://docs.microsoft.com/en-us/azure/cdn/cdn-pop-locations – you would need to be in a VERY remote location to not have an almost local POP.

I’m going to test out WordPress running on Azure App Service natively, and then setup Azure CDN and compare the two – making use of Performance tests within Azure App Service to highlight the difference in metrics for both arrangements (Without CDN, and with CDN).

To start, I have deployed WordPress on Azure App Service with MySQL In App using the below template:

Head over to https://github.com/Azure/azure-quickstart-templates/tree/master/wordpress-app-service-mysql-inapp for the template.

Once this was deployed I completed the usual WordPress setup, and I now have a functioning site – but without any content. To create some content for testing, I used a plugin designed to generate posts and pages (with images) to give the site some content (including images) we can use when testing response times:

Once installed and run, this plugin gave me lots of posts and pages, to simulate the content of a real site, and all of these posts included an image:

Now I can start seeing how the site performs – without a CDN in place. Initially, I’m using a Performance Test to measure site performance – which can be accessed from the App Service pane in the Portal:

Creating a test is simple, I am just going to simulate 1000 users accessing in a 1 minute window:

As you can see, the metrics are coming back from the West EU test as follows with no CDN:

Average response time is probably the most key metric here – so 2.89 secs average from the West EU test region. To give an idea of the variation, I ran the test again, but this time from the East Asia region:

As you can see, there is a noticeable speed difference, albeit one that is to be expected. Based on the metrics (2.89s average West EU vs 5.81 average East Asia), we can see that the average response time for clients in the East Asia Region is around 200% that of those in the West EU region. So… about twice the waiting time for the page to load.

Configuring the Azure CDN

Configuring a CDN Endpoint for Azure Web Apps is extremely simple – it can be done from the Web App section of the Azure Portal:

For this test I have configured the CDN Endpoint as below. I’m using the Standard Akamai Offering for my test:

Once we have filled in the details the Endpoint is created:

Once the endpoint is created, we are presented with a new URL to access the CDN version of the site:

I then configured WordPress to integrate with the CDN using the CDN Enabler plugin:

To check the function – if we now have a look at the properties of an image on the page, we can see it is being sourced from the Azure CDN, and thus from a location geographically close to our users:

Because the plugin includes any content in wp-content, any image we upload to the Website will be provided to users via the CDN. Next up, I re-ran the performance tests to measure the performance differences now that we have switched some content to the CDN:

West EU:

East Asia:

Based on the above test results, when implementing the CDN endpoint, we saw the following differences in test results in terms of average speed increases:

without CDN with CDN % speed increase
West EU 2.89 1.52 47
East Asia 5.81 2.11 64

There are a few key results from this test:

  • Utilizing the CDN improved performance in both the local and remote regions – in both cases significantly
  • Remote regions saw the greatest performance boost at a 67% average load time speed increase
  • The performance in the remote (East Asia) region was better than the original test of the West EU region after we had added the CDN endpoint
  • Once configured both in Azure and in the Application (WordPress) there is no further configuration required
  • We can further improve the speed by taking more elements of the Web Application and bringing them into the CDN – for example theme files, static code, CSS etc. In my test I have only included the wp-content directory but there is more that could be added.

Hope this has been useful… until next time!

 

 

 

 

Testing out the Azure Firewall Preview

Azure Firewall was released for preview this week, so I thought I would give it a quick try and look at some of the features available. The firewall provides the following features at the current time:

  • Built-in high availability – built into the Azure Platform, so no requirement to create load balanced configurations
  • Unrestricted cloud scalability – the firewall can scale to meet your requirements and meet changing traffic demands
  • FQDN filtering – outbound HTTP/S traffic can be filtered on a specific set of domain names without requiring SSL termination
  • Network traffic filtering rules – centrally create allow or deny network filtering rules, based on IP, port, and protocol. Azure Firewall is fully stateful, and rules can be enforced and logged across multiple subscriptions and VNETs.
  • Outbound SNAT support – outbound virtual network traffic IP addresses are translated to the Azure Firewall Public IP so you can identify and allow VNET traffic to remote Internet Destinations
  • Azure Monitor logging –  All Firewall events are integrated with Azure Monitor. This allows archiving of logs to a storage account, streaming to Event Hub, or sending them to Log Analytics.

You can read more about the features here: https://docs.microsoft.com/en-us/azure/firewall/overview

Getting access to the Azure Firewall is easy – it’s built directly into the VNET Configuration window:

However, before we can use this, we need to enable the Public Preview for our Subscription with a few PowerShell commands:

You’ll need to wait upto 30 minutes at this point for the request to be enabled – see https://docs.microsoft.com/en-us/azure/firewall/public-preview for further information. You can run the following commands to check the status:

If all is well – it should look like this:

Finally, run the following command to complete the setup:

Before we can add a Firewall to a VNET, we need to create a subnet called “AzureFirewallSubnet” – this is to allow the firewall to communicate with addresses on the VNET. Once this is completed, we can setup the Firewall. This is just a case of filling in some basic details:

Once we have completed the basic details, we can review and complete the deployment:

Now that the Firewall is created, we are ready to start testing. In order to test the Firewall out, we need a subnet that is routed out via this Firewall. To do this, I used a route table that directs traffic to the Firewall IP:

We now have a Subnet within our VNET that is routed via the Azure Firewall – so now we can test out some rules. My lab environment is now setup as below (Note the jump VM in a separate Subnet that is NOT routed to the Firewall. This is to allow me to RDP to the test box as I have no VPN in place to test from etc.):

From the Test VM, internet access is now blocked – because there is no firewall rule in place to allow it. I am going to add an “Application Rule collection” which I will use to allow HTTPS access to jakewalsh.co.uk, but not HTTP access. This is configured from the Firewall management interface via the Azure Portal:

Then you will be presented with the following window:

Once I have clicked on “Add” the rule will be added to the Azure Firewall. From my test VM, access to https://jakewalsh.co.uk works, but note that HTTP does not:

HTTPS:

HTTP:

The same also works in reverse, so we can selectively block HTTP or HTTPS sites as we require.

As well as the Application Rules we can deploy, we can also create more traditional firewall rules (replace 0.0.0.0):

Overall, the Azure Firewall complements and extends the functionality of Network security groups and gives additional control over networks residing within Azure. The rules are simple to adjust and easy to work with. It will be promising to see how this feature develops over the coming months…

Azure VM Scale Sets and Remote Desktop Services?

When using any environment that provides virtual desktops at scale, it makes sense to have only the required number of resources running at the right time – rather than all of the resources all of the time. The usual approach to this is to use power management – so unused virtual machines are shut down when not in use.

With Azure we have another potential option designed for large workloads – to use Virtual Machine Scale Sets. This allows us to automatically scale up and down the number of Virtual Machines based on various factors and choices. This effectively allows us to ensure the most economical use of resources – as we never pay for more than we need to use, because the machines are de-allocated when not required. Scale Sets also provide a number of features around image management and VM sizing that could be useful for VDI environments.

In this post I am going to explore the validity and feasibility of VM Scale Sets for a Remote Desktop Services Environment. To start this post – I have the following environment configured, minus the scale set:

Note: if you need an RDS environment – this Azure template is awesome: https://azure.microsoft.com/en-gb/resources/templates/rds-deployment/ – I would advise using multiple infrastructure VMs for each role if this is a production service though.

Next – I configured a single server with the RDS Session Host role and all of the applications I require, as this will become our VM image. I then ran sysprep /generalize as per the Microsoft instructions for Image Capture in Azure. (See here). Once this is done we need to stop and de-allocate the VM, and then we need to turn this into an image we can use with a scale set:

Once this is done – we have a VM image saved:

So once we have an image – we can create Virtual Machines from this image, and create a Scale Set that will function as the means to scale up and down the environment. However – we need to do some more work first, as if we just scale up and down with a sysprepped VM, we end up with a VM off domain that won’t be of any use to us…. !

Usually – I just spin up Lab VMs using a JSON Template that creates the VM and joins it to an existing lab domain, using the JoinDomain extension. This saves me lots of time and gives me VMs deployed with minimal input (just a VM name is all I have to enter):

See https://github.com/Azure/azure-quickstart-templates/tree/master/201-vm-domain-join for more details and to use this template.

Now that we have a template – we are ready to go. I’m using Visual Studio to create the JSON for my deployment – and fortunately there is a built in scale set template we can use and modify for this purpose:

With the template up and running, we just need to add some parameters – and we can run a basic test deployment to confirm everything is working. My parameters for the basic template are shown below:

A quick test deployment confirms we are up and running:

However, there are a few issues with the template we need to correct – namely:

  • The machines are not joined to the Domain – and we need to place them into the correct OU for GPO settings too
  • A new VNET is created – we need to either use peering (prior to creation – or domain join operations will fail), or better an existing VNET already setup
  • The load balancer created is not required – we’ll be using the RDS Broker anyway

For this test – all I am concerned about is the domain join and VNET. The load balancer won’t be used so I can just discard this – however, the VNET and Domain Join issues will need to be resolved!

Issue 1 – using an existing VNET

To fix this, I am not going to reinvent the wheel – we just need some minor adjustment to the JSON file, based on this Azure docs article – https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-mvss-existing-vnet. In short, this will achieve the following:

  1. Add a subnet ID parameter, and include this in the variables section as well as the parameters.json
  2. Remove the Virtual Network resource (because our existing VNET is already in place)
  3. Remove the dependsOn from the Scale Set (because the VNET is already created)
  4. Change the Network Interfaces of the VMs in the scale set to use the defined subnet in the existing VNET

Issue 2 – joining the Scale Set VMs to an AD Domain

To get the VMs in the scale set joined to an AD Domain we need to make use of JsonADDomainExtension.

With this added to the JSON template for our deployment, we just need to add the variables and parameters (shown below) and then we are good to go:

Note: the first time I used this I had an issue with the Domain Join – it was caused by specifying only the domain admin username. When specified in the form above (domain\\adminusername) it then worked fine.

Now when we run the template, we get the usual Visual Studio output confirming success – but also a scale set, and, machines joined to the domain:

Because I have previously configured the image used in the Scale Set with the RDS Role, and the Software required – we just need the servers to use an RDS Broker that will manage inbound connections into the RDS Farm. This is where I encounter the first sticking point – these need to be added manually when the Session Collection is created 🙁

This wasn’t a massive issue for this test – so I went ahead and created a Session Collection and added in my VMs:

Next I tested the solution by launching a Desktop via Remote Desktop Web Access:

Bingo – I was then logged into an RDS Session. Note the RDS Connection Name (showing the Broker) and the Computer Name (showing the Session host). This confirms we are running as expected:

I’ve now demonstrated the RDS Farm up and running, utilizing machines created by a Scale Set, and also accessed via a connection broker. But – we aren’t quite done yet, as we have not looked how a scale set could enhance this solution. Below are a few ways we can improve the environment using Scale Sets, and a few limitations when used with RDS:

  • We have the option to Manually increase VM instances if we need more Session Hosts:

Note: this will require adding to the RDS Session collection manually (or via PowerShell)

  • We can scale the environment automatically using Auto Scale:

Below you can see a default scale rule (5 VMs in the Scale set) and then a rule that runs between 0600 and 1800 daily, and increases the VM Count up to 10 VMs if average CPU usage goes above 80%.

The rule for this Scale operation is shown below:

Note: this will still require machines adding to the Session Collection manually.

  • We can increase the size of the VMs

Once a new size has been selected – the existing VMs show as not up to date:

We would then need to upgrade the VMs in the scale set (requiring a reboot), but, does not require the VMs to be re-added to the Session Collection. With this option a drain, upgrade, drain, upgrade option would be available. This allows for a sizing upscale – without lots of reconfiguration or management required.

Overall, it would seem that although scale sets aren’t able to fully integrate with Remote Desktop Services collections, they are still very capable and powerful when it comes to managing RDS Workloads. Scale Sets can be used to size and provision machines, as well as to provide simple options to increase environment capacity and power. Purely using a scale set for the ability to spin up new VMs, or to manage sizing across multiple VMs is a logical step. We also have the option to reimage a VM – taking it back to a clean configuration.

Key Observations from my investigation:

  • We can scale an RDS environment very quickly, but RDS Servers can’t be automatically added to a session collection – the GPO settings for this don’t appear to support RDS post 2008R2 (whereby Session Collections and the new configuration method was introduced). This means servers have to be manually added when the Scale Set is scaled up
  • Scale sets can be used to increase VM size quickly – without reimaging servers (a reboot is all that is required)
  • Scaling can only look at performance metrics – we can’t scale on user count for example
  • Reimaging means we can take servers back to a clean build quickly – if a server has an issue we would just prevent logons and then reimage.
  • Scaling down can’t take logged on users into consideration – so we’d need a way of draining servers down first
  • Scale Sets will also allow us to scale up to very large environments with minimal effort – just increase VM count or size, and add the servers into the RDS Collection. A growing business for example – or one that provides a hosted desktop could scale from 10 servers to a few hundred with minimal effort.

Hope this helps, and congratulations if you have made it to the end of this article! Until next time!

Resources:

Azure Traffic Manager for NetScaler Gateway Failover

Azure Traffic Manager is designed to provide traffic routing to various locations based on a ruleset that you specify. It can be used for priority (failover), weighted distribution, performance, and geographic traffic distribution.

The failover option is similar to GSLB – and works in a similar way, so I am going to demonstrate that in this post. I’ve started with the following environment already configured:

  • Two Azure Locations (East US and South Central US), with a VPN between the sites to join the VNETs
  • 1 Domain Controller in each location
  • 1 NetScaler (standalone) in each location
  • 1 Citrix Environment spread across the two locations
  • NetScaler Gateway’s setup in both sites, and NAT’d out using Azure Load Balancer. (So that we have a public IP offering NetScaler Gateway services in both Azure Locations. Have a look at this Blog post if you require guidance on setting this up.)

Azure Lab Diagram

Before we setup the Azure Traffic Manager profile, we need to give our Public IP Addresses a DNS name label. To do this, browse to the Public IPs for your Load Balancers, and then click on “Configuration”. We need to give our Public IP addresses a DNS name label, as this is what Traffic Manager will be using to balance the endpoints.

Azure Public IP configuration

I have two public IPs so I have created two DNS Name Labels and given them appropriate names:

  • desktop-eus-jwnetworks = East US NetScaler Public IP
  • desktop-scus-jwnetworks = South Central US NetScaler Public IP

Next – it’s time to create the Azure Traffic Manager profile!

Traffic Manager profile creation

After we click create, we just need to populate a few basic details:

Traffic Manager profile creation

As you can see – I have given my Traffic Manager a name, selected Priority as the routing method (this gives us the failover in a similar manner to Active/Passive GSLB). Note: there are other options available:

Traffic Manager routing options

See here for an overview of the Traffic Routing Methods. Next – we need to configure some more settings on our Traffic Manager, to ensure that the Monitoring and Traffic Routing are going to work correctly. In the screenshot below I have adjusted the following:

  • DNS TTL – I’ve adjusted this to 60 seconds, this defaults to 300 seconds (5 minutes)
  • Protocol – HTTPS, this is because we are Monitoring the HTTPS NetScaler Gateway
  • Port – 443 as we are using this port for the NetScaler Gateway
  • Path – this is the path to the files that the monitor will be checking for, so in the case of NetScaler Gateway this is /vpn/index.html – if this page is not available then the service will be marked as unavailable.
  • Probing Interval – this is how often the endpoint health is checked. Values are either every 10 seconds or every 30 seconds
  • Tolerated number of failures – this is how many health check failures are tolerated before the endpoint is marked as unhealthy
  • Monitoring timeout – this is the time the monitor will wait before considering the endpoint as unavailable.

For more information on these configuration options – click here.

Traffic Manager configuration

Next – it is time to add our endpoints! To do this, click on Endpoints and then on Add:

Traffic Manager endpoints

We then need to add our Public IP addresses assigned to the Azure Load Balancers (where the NAT rules were created). Note – you will need to do this for BOTH endpoints:

Adding Endpoints to Traffic Manager

Once both are added, you will see the below in the Endpoints screen. Note that both Endpoints are shown as “Online” – this confirms our monitor is detecting the Endpoints as up. Also note that the Endpoints have priority – this means that under normal operation, all traffic will be sent to the “eus-desktop” endpoint (Priority 1), and in the event of a failure of the “eus-desktop” endpoint, all traffic will be directed to the “scus-desktop” Endpoint.

Endpoints

All that is left to do is test – however, first let’s make things neat for our users with a CNAME DNS Record. We are effectively going to CNAME our jwdesktop.trafficmanager.net record to something that users would be able to remember. You can find your record from the overview screen:

Traffic Manager overview

Next up I added a CNAME record in my Azure DNS Zone:

Add DNS Record

Add CNAME Record

Once this is created – we can start testing! But first, a diagram! Below is shown what we now have setup and working:

Solution Diagram

Note: in order to easily distinguish between my two Gateways, I set the EUS Gateway to the X1 theme, and the SCUS Gateway was left on the default NetScaler theme. When accessing https://desktop.jwnetworks.co.uk I am correctly shown the EUS Gateway:

EUS Gateway Test

Bingo – this all looks good to me! Next up, I disabled the Virtual Server for the EUS NetScaler:

Virtual Server Disabled

After around 30 seconds… the Monitor Status shows as Degraded:

[ for those interested in the maths (10 Second Probe interval + 5 second timeout)x1 tolerated failure (so effectively 15×2 attempts at connecting) ]

Endpoint Health

Next I refreshed the Page and we are presented with the SCUS Gateway page:

SCUS Gateway Page

As you can see, during a failure condition (the EUS Gateway vServer being taken down) the Traffic Manager directs traffic to our Priority 2 site, without any intervention from us. Any users would be able to refresh the page and then log back in. This can be used not only for NetScaler Gateway but for many internet facing services – for example OWA, SharePoint etc. There’s a great many services that can benefit from this type of failover and the resiliency that it offers.

Load Balancing Citrix StoreFront with Azure Load Balancer

Sometimes there is a requirement to Load Balance StoreFront using a method other than NetScaler. Although rare (in my experience!) this does occasionally happen when NetScaler is perhaps not being used for Remote Access –  in an internal only environment for example.

In this post I will explain how to Load Balance StoreFront using the native Azure Load Balancers. We start with a simple setup:

  • 1x Domain Controller
  • 2x Citrix StoreFront Servers – in an availability set called “EUS-StoreFront”
  • 1x Virtual Network (VNET)

All of the above is in the East US Azure Location.

We start by creating a new Azure Load Balancer. Note a few key settings here:

  • Type: Internal – this is because we are balancing traffic within our VNET (Internal Network only)
  • IP address – static… we don’t want the LB IP to change!

Once this is done – we can add the backend servers. We do this by targeting the Availability Set that the StoreFront Servers are in. For those familiar with NetScaler, this is similar to a Service Group:

Next – we need to configure some Health Probes. This allows us to determine the state of the StoreFront server and to confirm that the services we are load balancing are healthy and available. Note: at the current time Azure Load Balancer HTTP checks support relative paths only, so I have used /Citrix/CitrixWeb/monitor.txt – a simple text file (Static Content) I created to check that the Web Server is serving out content and thus working correctly. (https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/load-balancer/load-balancer-custom-probe-overview.md) I have configured by Health Probe as below:

Next – it’s time to create the Load Balancing Rule that will form the entry point for Load Balanced traffic. Note the Protocol (TCP), Ports (80 Frontend, and 80 Backend), Backend Pool (StoreFront Availability Set), Health Probe (our HTTP 80 monitor.txt check), Session Persistence (Client IP), and Idle Timeout (30 minutes is currently the maximum value):

We can then click OK and our Load Balancing Rule is created! Next I created a DNS A Record for StoreFront and pointed it at the Load Balancer IP. After this, I opened up a browser and typed in my newly created StoreFront DNS record. Bingo – we have a page!

To test that the Load Balancing was working. I shut down IIS on each server in turn, and then tested. Sure enough – even when only 1 out of 2 servers was running, the page stayed up and StoreFront was accessible.

This Load Balancer can be used for a variety of Web Applications, and is a simple way to Load Balance Azure based services as you require. Until next time… cheers!

GSLB for NetScaler Gateway across Azure Locations

In this post I’ll be going through how I have configured GSLB for NetScaler Gateway in Azure, and the various elements required for this type of configuration.

Firstly – I began by setting up the background infrastructure to demonstrate this test. Namely, 2x Active Directory DCs within two Azure Locations (Eastern US (EUS), and South Central US (SCUS)). These were on a Virtual Network setup for each location, joined with a Site to Site VPN utilizing Virtual Network Gateways. Next, I created a simple Citrix Environment that spanned the sites, ensuring I had resources in both sites – to properly demonstrate the failover.  An overview of the background infrastructure is shown below:

Next, I spun up two NetScaler BYOL versions:

These will form the bulk of the work where configuration will take place – in providing NetScaler Gateway via GSLB. I’m using Platinum Licensing, but Enterprise would also be fine (so that GSLB is available).

My NetScalers will all have multiple IP addresses assigned, for the SNIP, GSLB Site, and Gateway VIP:

Well worth a read at this point is CTP Gareth Carson’s awesome blog post around NetScaler deployment in Azure.

The next step for me was to setup 2x NetScaler Gateway vServers, which will be used for external access, and will be the vServers that will be provided by GSLB:

East US NetScaler:

South Central US NetScaler:

Next – I setup Authoritative DNS Listeners on both NetScalers, making use of the Subnet IP for this service (East US NetScaler shown below) :

So – next we can setup the GSLB Sites, to enable the synchronisation of GSLB information via Metric Exchange Protocol. It’s just a case of adding one local and one remote site on each NetScaler, and using the GSLB Site IPs. Once this is completed, the NetScalers will show as follows:

East US:

South Central US:

Once these are setup and showing as Active – we have communication between the NetScalers. This is carried out using Metric Exchange Protocol (MEP). Next – we need to configure GSLB, but first I am going to create the Public IP addresses for each site, as this makes the GSLB Setup easier! We will need two IP Addresses (one for each site), and TCP 443 (Gateway), and UDP 53 (ADNS) will need to be NAT’d through for this configuration to work. See the diagram below for an overview of this:

This is easy to configure – firstly we setup two Load Balancers, each with a Public IP (Static) assigned:

Ensure that Static is selected for the Public IP – otherwise this may change and then the solution will stop working:

Once this is created for both sites – we will have two Load Balancers ready to use for our NAT requirements:

At this point, I like to update the Network Security Groups on the NetScalers to ensure that the required inbound rules are in place. For both NetScalers we need to allow HTTPS and DNS inbound from anywhere (as these will be internet facing):

Once complete, we can then NAT through the required ports (TCP 443, and UDP 53) by creating an inbound NAT rule. Remember, you’ll need to do this twice on each Load Balancer – so both ports are forwarded to the NetScaler on the Load Balancer site.

Once these are all in place – we can test the NetScaler Gateway is up and accessible by visiting https:// and then the Load Balancer IP for each site. Before we configure GSLB on the NetScaler – we need to delegate the DNS Zone. This will vary depending on how your External DNS Servers are setup. Essentially – we need lookups for the Gateway URL to be handled by the NetScaler appliances. This means any lookups for the Gateway IP need to be delegated to the NetScaler appliances – so that they can provide the URL for the Active GSLB Site.

I’m using Azure DNS – so I have a zone setup. My URL is desktop.jwnetworks.co.uk – so I will be delegating control of this to the NetScalers. To start – create two A Records, one for each NetScaler, and these need to be pointed at the NAT’d IP. These will be used for the DNS Lookups. We then need to create a new NS Record for our Gateway Domain Name, with a 5 Second TTL, and pointing it at the A Records for the NetScalers we configured above:

Now that this is in place – it’s time to configure our GSLB Configuration! This can be done from one of the NetScalers and then propagated to the other via Configuration Sync. I’ll therefore carry this out on the EUS NetScaler – as you can see below, we have only the sites setup for GSLB (as we did this earlier):

We start by clicking the “Configure GSLB” button, and then run through the Wizard – I am going to run with an Active/Passive site:

We then click OK, and we are presented with the GSLB Sites pane – but we have already configured this:

We can click continue, and we then need to setup the GSLB Services. So these are the two NetScaler Gateways we are using to provide this service. The key here is that we need to make sure that the Public IP addresses are listed – EUS is shown below:

We then click Create, and then repeat this process for the SCUS Site – making sure that the Public IP address is entered again. Once this is done, we will have a Local and Remote Site Configured:

Next you will be prompted to create a GSLB Backup vServer – but I’m not going to create that as part of this Proof of Concept.

Next – we create the GSLB vServer. This is just the entry point for traffic being Load Balanced by GSLB. Note – ensure that you pick an appropriate load balancing method, and then click continue after filling out your details:

Next – click on Save, and the configuration is done! We can now sync the config across to the other NetScaler. This is done by clicking on “Auto Synchronisation GSLB” from the GSLB Dashboard:

Once this completes successfully – we can test our configuration! To start – we can do an nslookup, and set type=ns. This will tell us that the NameServers are correctly configured:

As you can see – the nslookup is returning all the expected information. Because we configured Active/Passive – the A record returned for a normal (A record) nslookup is that of the EUS NetScaler. Next – we can test that the Gateway Page is working and accessible:

Bingo – all good so far! Next – let’s try shutting down the EUS NetScaler and see if things are still working as expected. At this point the IP address returned should change from that of the EUS Load Balancer IP, to the SCUS Load Balancer IP:

Before EUS NetScaler Shutdown:

After EUS NetScaler Shutdown:

As you can see this works as expected, and after a page refresh, the NetScaler Gateway page is shown again:

This means that during a failure condition affecting the EUS NetScaler, requests for the Gateway URL will be directed (via DNS) to the SCUS Site. This provides Data Centre level failover for Gateway Services, making use of native Azure Load Balancers, and a single NetScaler on each site. This solution is suitable for pretty much any service accessed via a Web Browser – GSLB can be used in this way to fail NetScaler Gateway services over between Azure Sites or to distribute other traffic types as required.

 

 

 

 

Quick Post! – Using NetScaler responder policies/actions and backup vServers to notify users of Service Downtime

NetScaler, as I am sure you are aware, is a superbly powerful Application Delivery Controller. One of the most useful features is the use of Responder Policies/Actions, and Backup vServers to indicate that a service is down or to provide access to an alternative service for end users.

Let’s say – you have a load balancer configured that balances two Web Servers, which you then present for users to access. This load balancer could be configured with monitoring to check the health of the Web Servers, and also to ensure that the servers are loaded evenly with requests.

But – what happens when both servers are down? Maybe you have scheduled patching, or there is a fault. Leaving users with a blank page or one that times out is never ideal, and letting them know there is a fault is always best. This is especially relevant if the users are accessing pages of a commercial nature – for example, “The Online shop will be back open at 2pm” is a lot better than a blank page.

Both of the options below take less than 10 minutes configuration on a NetScaler – so if you need to get a page up quickly… these are very useful!

To do this – we can go down one of the two routes below:

Option 1

Use the Redirect URL option on the vServer – this redirects client requests to the custom URL when the service is down. So for example this could be a custom page on elsewhere, which explains that there is an issue.

This could be used for Scheduled patching for example – for times when we know the site will be down, we just add the URL to the vServer and any users who visit during the maintenance window will see the redirected page.

Option 2

Another option, is to use the Backup vServer feature within NetScaler – this is great because it can be used to direct traffic to a backup Data Center if our primary is unavailable. But also – we can use this feature to put up a NetScaler generated page informing the user that there is a problem with the backend service.

To do this – we need to create a new vServer to use for Maintenance. Create this with the following basic settings:

Then create a service for this – any local service bound to 127.0.0.1 will do. Assign a basic monitor so that the service shows as up.

Next – go to AppExpert, and then Responder, and create a Responder Action. Fill out the details as below – note: you will need to create a new HTML Page, this is the 2nd and 3rd screenshots below:

HTML Page:

Click on Done, and then Click Create for the Responder Action.

We can then go back to the vServer and assign this under the Policies option. Select the options as per the screenshot below and press Continue:

Next we are presented with the below screen, click on the Plus sign next to “Select Policy”:

Fill out the details as per the below:

Click on “Create” and then click on “Bind”. Finally, click “Done”. We can now apply this vServer as a backup vServer to others – so that if those vServers are down, this page will be shown to users.

This is configured on a per vServer basis as per the below:

Now when accessing the page, and with the services down – the following page is shown:

Obviously, you can customise the page a little more than I have – but hopefully this will help. It’s a quick step to setup and gives some extra information to users when there is scheduled work or an unexpected fault.

Upcoming CUGC User Share Webinar!

Improving the Resiliency of Your XenDesktop Environment – StoreFront Multi Site and NetScaler GSLB for StoreFront

Thursday, February 22, 2018 – 1:00 PM – 2:00 PM EST (6:00 PM to 7:00 PM GMT)

This Webinar focuses on improving XenDesktop environment resiliency using StoreFront Multi Site and NetScaler GSLB. This will cover the setup of these methods, the benefits and pitfalls, and show some practical demonstrations. There will be a Q and A afterwards. Dave Brett (CTP) will be joining as moderator for this webinar.

You can register for the Webinar here.

Testing out Project Honolulu

Recently I have been testing out something new – Project Honolulu from Microsoft. I first heard about this on Twitter (thanks to Eric @XenAppBlog), and was interested in what it could offer straight away. Project Honolulu is a new way to manage Windows Server – using a web based method, that does not rely on the traditional Server Manager GUI. Functionality is similar, and offers the usual range of configuration options, as well the ability to manage roles and features as you would normally expect.

You can download Project Honolulu here. Windows Server 2016 is supported natively, but for 2012R2 Support, you will need to install WMF5.0 (KB3134758).

Project Honolulu has a number of ways to deploy – but I went with a simple install on a single server within my lab. Once this is completed (it’s an easy next next next done install), you are presented with the following screen, which opens up in your default browser:

From here – we can add server connections. Note: Standalone, Failover Cluster, and Hyper-Converged systems are supported:

After I’d added a few servers from my lab, the main screen appeared as below. You can also import servers from a text file – so an export from AD is possible too:

From here, we can see the status of the servers I have added and then drill down further into the options by clicking on a server name. The overview screen gives the usual range of information we’d expect to see:

Particularly nice – is the metric display, which gives an overview of CPU, Memory, Ethernet, and Disk Activity. This is realtime data – but useful for monitoring key servers/clusters, perhaps on an Ops display board or large screen etc.:

As well as a range of metrics available, we have a range of management tools we can take advantage of. Particularly interesting is the ability to manage elements like Network Adaptors, Services, and Roles/Features, as well as to view Event Log entries and the Registry:

Management of Services is also a very useful feature – allowing services to be stopped and started (I wish it had a restart button though!) from the Web Console. This is particularly useful for Managed Service Providers – when the 2am call comes in that a failure has occurred, instead of a VPN into an RDP Session into another RDP Session, you can fire up a Web Interface and restart the service from there (NAT rules and an SSL cert required of course…) :

You’ll notice here that I’ve highlighted a couple of Citrix Services too – Project Honolulu allows you to manage all services running on a supported machine. So this is great for managing 3rd Party applications and services too. The lightweight nature of the system also means that this can be added to existing systems with ease (a single installer and a list of servers).

I’m really interested to see where this Project will go – in particular, it makes the use of Server Core much more accessible, because a familiar and common interface can be used for management of multiple servers. It also allows simple management of basic server configurations, as well as Service management for Microsoft and Third Party applications. Any environment could probably benefit from a single interface that allows basic configuration and Service restarts… the key questions is… where will this Project go next?

I’d really like to see support for more configuration changes, for example, customisable PowerShell options (e.g. this button in the interface runs this remote command) or support for a PowerShell session via the Web Interface. Also it would be great to see support for Third Party software – for example, additional modules that could be included to provide web based management of other software items on the server.