For several years I have worked with healthcare customers deploying, managing, and supporting Citrix environments that serve applications and desktops to thousands of users.
Citrix Virtual Apps and Desktops tends to be a popular choice for healthcare customers wanting to virtualize their applications and desktops, often due to the complexity of health IT, supporting many different applications from SaaS to modern thick apps and legacy apps, providing access out to other healthcare organizations, and keeping patient data secure at all times. Citrix performs well in this field and can solve many of the challenges and requirements mentioned.
With healthcare though, due to how healthcare staff work, their needs, and the typical applications that are used, there are particular issues that I have witnessed in this field that I haven’t so much witnessed in others, or at least not to the same scale. As such, I would like to highlight the three I have come across most recently and how you can help avoid or troubleshoot those issues more efficiently.
Audio and Dictation Issues
Dictation in particular is prominent in healthcare. With the many consultant visits by patients every day, the consultant dictates notes for that patient into their EHR app, for a secretary then to transcribe the audio to text, into a letter that can be sent to the patient or simply to be stored on the system. This method of work uses a consultant’s time efficiently.
I have seen in the past secretaries complain of poor audio quality in relation to these dictation audio files. Sometimes audio skips, slows down and then speeds up, and so on, or does not work at all.
Audio in a Citrix Virtual Apps and Desktops environment can be subject to poor latency, having a negative effect on how audio is delivered to the client. Otherwise, it could simply be impacted by missing best practice configuration.
To troubleshoot, if the issue is poor audio quality, firstly check the user’s latency when they are connected to Citrix. This is the most simple and quickest thing to check first. You can use a tool like Citrix Director, or your third-party solution if you have one. You should also check if the ICA connection is facing packet loss. This is a little more difficult when using a tool like Director, but can be achieved using third-party tools like the Goliath Performance Monitor from Goliath Technologies, or some packet captures between the endpoint and Citrix server. If the user is working remotely, you will want to make sure your third-party tool, if you have one, can still report on latency.
Secondly, you should analyse the consumption of the audio virtual channel. I have written an article previously explaining how incorporating this task into your troubleshooting steps could help you determine if audio bandwidth throughput is low due to a misconfigured Citrix policy, or due to other virtual channels contending for the user’s bandwidth, such as high video bandwidth consumption, which can then negatively impact the audio virtual channel, especially if no QoS is in place.
Finally, to avoid many of the audio performance problems your users may face in the first place, you should take a task to review your audio configuration for your Virtual Apps and Desktops deployment. As a starter, the Audio quality policy in Citrix Studio should be set to Medium. Secondly, you should adopt the use of Audio over RTP (Real-time Transport Protocol). This is the recommended protocol to use when delivering audio from a Citrix environment to the end user. It does require some configuration and firewall policies to be open but has most definitely in my experience solved many audio quality issues in the past.
Long Logon Times
Long logon times and associated user complaints are not unique to healthcare environments in any case, but healthcare has some of the highest user logon counts across any industry. Firstly, healthcare organizations typically run 24/7, and staff come and go as part of shifts. This means that throughout a day or week, the number of times that EHR app delivered via Citrix is launched enters the thousands. On the other hand, an individual user may launch a Citrix session multiple times per day. The hospital has many different kiosk or shared workstations that are used by many clinicians each day, and individual clinicians may launch Citrix apps from several PCs as they move between wards and so on.
That is why logon times are crucial for healthcare, because if those logon times are slow, they will most likely be realised in a healthcare environment, and the helpdesk are most likely to receive calls about it.
To troubleshoot, you really need the aid of a monitoring solution that can help track each step of the logon, and report on the time it takes to complete each step. The Citrix logon process contains many different steps such as Group Policy printing, logon script execution, printer and drive mappings, desktop load, brokering, user profile load, and so on. Without knowing which step is causing the overall process to be slow, you can’t begin to troubleshoot effectively.
However, that said, what I do recommend is that you take a process of elimination approach. Check for patterns and consistencies to the issue, such as if it’s a particular location that is impacted by long logon times, a particular set of users, users on a particular set of VDAs and so on. You will want to review the health of your entire infrastructure, from the network, to storage, hypervisor hardware, and the virtual infrastructure. Again, based on if you have the right monitoring solution in place or not will determine how manual and time intensive these steps are, and how many other different teams you need to involve.
Printing Issues
The final piece I wanted to talk about is printing. Clinicians or medical staff cannot print at all, or print jobs are not rendering as expected.
Again, like the number of logons, the healthcare industry prints more than any other industry I have worked with, which makes printing very important, and as it affects patient care, something that cannot go wrong!
However, it often does go wrong. Printing is complex. Print devices are deployed en-masse all over the environments in their hundreds, some wireless, others hardwired. Then you have the print servers that contain many different print drivers for each of your device types, and often those drivers are old or simply unstable and that can lead to things like spooler crashes.
To troubleshoot printing issues effectively, first understand the scope and impact of the problem. Are the issues affecting multiple people, what type of print jobs are the users trying to print, what devices are they trying to print from. For example, if it’s a single device, you are going to troubleshoot the issue from a different angle compared to if the issue was affecting multiple devices.
Also, keep in mind that no print job is the same. Some prints flow through Citrix, whilst others may flow through a healthcare organizations EHR infrastructure. So, knowing the type of print again will shape how you begin to tackle the issue, and where you begin to look.
If you have the ability through third-party monitoring solutions, track the print job as it flows through your infrastructure. Maybe the job is reaching the print server and spooling, but not arriving at the physical print device, or maybe it’s not spooling, or reaching the print server at all.
Asides understanding the scope and impact, and understanding the type of print job etc, you will want to check the health of your print servers and components a print job may travel through. Check to make sure there has not been any spooler crashes, typically due to bad drivers, and that print queues are responding to requests. You may even send engineers on-site to investigate the physical elements from the print device itself to the network connection piece. In many cases, a monitoring solution can automate many of these checks, so you can find route cause quicker.
Summary
If you support a Virtual Apps and Desktops environment in a healthcare organization, audio quality issues, slow logins, and printing woes are likely something you have come across during your time.
To be one step ahead in finding root cause, or catch issues before end-users are impacted, you ideally will have a monitoring solution that can monitor your entire infrastructure and has functionality specific for Citrix deployments.
Without the help of a monitoring solution, you are left with a more manual approach to troubleshooting, which can burn up many more hours than anticipated which isn’t ideal in any industry, not to mention healthcare were patient care could be impacted as a result.
Steve Greenberg
August 3, 2021Good info and definitely matches up with our health care experiences in the field