Table of Contents
Regular firmware updates are one of the most important maintenance tasks in a NetScaler ADC infrastructure. In addition to new features, current firmware releases include important bug fixes and security-related patches. Especially in the case of security advisories or actively exploited vulnerabilities, firmware updates should be scheduled promptly.
Because a firmware upgrade can affect production services such as NetScaler Gateway, Load Balancing, Content Switching, AAA, GSLB, or SSL Offloading, it should never be performed without proper preparation. A structured approach reduces downtime and minimizes the risk of unexpected issues.
This article describes the recommended upgrade process for production NetScaler ADC environments.
Preparation
Before every firmware upgrade, a number of basic checks should be performed first.
Review the Release Notes
Start by reviewing the release notes for the target version.
The following points are particularly important:
- supported upgrade paths
- known issues
- fixed issues
- changed default settings
- security advisories
- licensing requirements
Major version jumps, such as 13.1 → 14.1, often include changes that may affect existing configurations.
Download the Firmware
If the appliance is a VPX, a snapshot should be created immediately before the upgrade.
This makes it possible to restore the appliance within a few minutes if problems occur.
This step obviously does not apply to MPX appliances.
Save the Running Configuration
Before every upgrade, the current configuration should be saved.
|
1 |
save config |
Create a System Backup
In addition, a full system backup is always recommended.
CLI:
|
1 |
create system backup -level full |
GUI:
System → Backup and Restore
The backup is stored under
|
1 |
/var/ns_sys_backup/ |
and should also be downloaded and stored locally afterward.
Pre-Upgrade Functional Test
Before starting the update, it is recommended to test all production features that are currently in use.
This may include:
- NetScaler Gateway
- LDAP
- RADIUS
- TACACS
- SAML
- OpenID Connect
- Multi-Factor Authentication
- EPA / Endpoint Analysis
- StoreFront
- ICA Launch
- Load Balancing
- Content Switching
- SSL Offloading
- Rewrite Policies
- Responder Policies
- GSLB
- DNS
- NTP
- SNMP
Only when the environment is working correctly before the upgrade can any issues after the upgrade be clearly isolated.
Firmware Upgrade in a High Availability Cluster
In production environments, the Secondary Node should always be upgraded first.
The recommended procedure is as follows:
- Create a snapshot (VPX)
- Save the configuration
- Create a system backup
- Upgrade the Secondary Node
- Restart the Secondary Node
- Verify the firmware version
- Check the HA status
- Verify synchronization
- Perform a failover to the upgraded node
- Perform a functional test
- Upgrade the former Primary Node
- Restart the appliance
- Verify HA synchronization
- Optionally fail back to the original Primary Node
- Perform a final functional test
During the failover, existing Gateway connections may briefly be interrupted. ICA sessions usually remain active, although users may see a reconnect message for a few seconds.
Firmware Upgrade via the CLI
Although the firmware upgrade can also be performed through the web interface, in practice the CLI is generally the more reliable method.
Upload or extraction issues tend to occur less frequently when using the command line.
Connect to the NetScaler appliance via SSH.
Save the Configuration
|
1 |
save config |
Open the Shell
|
1 |
shell |
Create the Installation Directory
|
1 |
mkdir /var/nsinstall/<Version> |
Example:
|
1 |
mkdir /var/nsinstall/14.1-53.17 |
Copy the Firmware
Copy the downloaded firmware file (.tgz), for example using WinSCP, into the newly created directory.
Extract the Firmware
Enter your credentials when prompted.
|
1 2 3 |
cd /var/nsinstall/<Version> tar xzvf <Firmware>.tgz |
Start the Installation
|
1 |
./installns |
After the installation has completed successfully, leave the shell and restart the appliance.
|
1 2 3 |
exit reboot -warm |
After the reboot, verify that the expected firmware version is installed.
Known Certificate Digest Check Issue
With newer NetScaler ADC firmware versions, the SSL certificate pre-check may fail during the upgrade even though the certificates and private keys themselves are working correctly.
Typical error messages include:
|
1 2 3 4 5 6 7 8 |
Checking certificate digest... CertKeyname: Wildcard Company Digest Check: MISSING Certificate digest verification failed. Installation aborted. |
This behavior is now documented as a known issue. A common cause is spaces in the Certificate Key name or in the certificate and key file names. The SSL certificates themselves continue to work correctly, but the pre-upgrade validation fails and blocks the firmware upgrade.
Workaround
If it has been clearly established that the upgrade is failing only because of this digest check, the installation can be started with the following parameter:
|
1 |
./installns -C |
This skips only the Certificate Digest Check. All other installation steps and validation checks continue to run.
Note: Use this workaround only when the cause has been clearly identified and the certificates are known to be working correctly. After the upgrade, verify all SSL virtual servers, Certificate Keys, and HA synchronization.
Permanent Fix
Citrix recommends naming Certificate Keys as well as certificate and key files without spaces. For example, letters, numbers, underscores (_), and hyphens (-) can be used. This avoids the issue permanently.
Not Recommended
|
1 2 |
Wildcard Company.pem Company Key.pem |
Recommended
|
1 2 |
Wildcard_Company.pem Company_Key.pem |
or
|
1 2 |
Wildcard-Company.pem Company-Key.pem |
Firmware Upgrade via the GUI
Alternatively, the upgrade can also be performed entirely through the graphical user interface.
Navigate to:
System → Upgrade
Then:
- Select the firmware file
- Review the upgrade options
- Optionally disable the connection to NetScaler Console if it is not being used
- Start the upgrade
If the Reboot after successful installation option is enabled, no final success message may be displayed. Instead, the appliance restarts automatically immediately after the installation has completed.
After refreshing the browser, the new firmware version should already be displayed.
Storage Issues During the Firmware Upgrade
During a firmware upgrade, the installation may fail because there is not enough free disk space.
Typical error messages include:
|
1 2 3 4 5 6 7 8 9 10 11 |
Error: No space left on /var/ filesystem, aborting Error: No space left on /var/ filesystem, aborting installation… installns: [1455]: Error: No space left on /var/ filesystem, aborting installation… Error: No space left on /flash/ filesystem, aborting Error: No space left on /flash/ filesystem, aborting installation… installns: [1455]: Error: No space left on /flash/ filesystem, aborting installation… |
Appliances that have been in operation for several years often contain old firmware versions, trace files, or support bundles that can consume several gigabytes of storage.
Analyze Disk Usage
First, switch to the shell.
|
1 |
shell |
You can then identify the largest directories.
|
1 2 3 |
du -a /var | sort -n -r | head -10 du -a /flash | sort -n -r | head -10 |
This provides a quick overview of which directories are consuming the most storage.
Typical Directories to Clean Up
In most environments, the following directories are common candidates for cleanup.
| Directory | Content |
|---|---|
| /var/nstrace | Network traces |
| /var/ns_sys_backup | Old system backups |
| /var/tmp/support | Support files |
| /var/core | Core dumps |
| /var/crash | Crash dumps |
| /var/nsinstall | Old firmware versions |
| /flash | Old firmware files |
Free Up Disk Space
Examples:
|
1 2 3 4 5 6 7 8 9 |
rm -rf /var/nstrace/* rm -rf /var/ns_sys_backup/* rm -rf /var/tmp/support/* rm -rf /var/core/* rm -rf /var/crash/* |
Old firmware versions can also be removed.
|
1 2 3 |
rm -rf /var/nsinstall/<old version> rm -rf /flash/<old version> |
Important: Never delete the currently installed firmware version.
HA Synchronization Issues After the Upgrade
Newer NetScaler firmware versions use encrypted RPC communication between appliances by default.
This affects:
- High Availability
- Cluster
- GSLB
Communication uses TLS 1.2 on ports 3008 and 3009. When upgrading from older 12.1 or 13.0 builds, the internal RPC services should be configured for TLS 1.2 and it should be verified that these ports are not blocked by firewalls between the appliances.
A typical error message is:
|
1 |
Unable to connect to primary, please check the network connectivity from secondary to primary. |
Check the Internal Services
Navigate to:
Traffic Management → Load Balancing → Services → Internal Services
Check the following services in particular:
- nsrpcs-127.0.0.1-3008
- nskrpcs-127.0.0.1-3009
Verify that:
- TLS 1.2 is enabled
- an SSL profile is configured
- the service is UP
- the certificate is bound correctly
Check the Firewall
At a minimum, the following ports must be reachable between both appliances:
- TCP 3008
- TCP 3009
If these ports are blocked, configuration synchronization will fail.
Verify the HA Configuration
After every upgrade, the following points should be verified.
CLI:
|
1 2 3 4 5 |
show ha node show rpcNode show version |
Check that:
- both appliances are UP
- synchronization shows SUCCESS
- both appliances run the same firmware version
- both appliances run the same build
- the configuration is synchronized
Post-Upgrade Functional Test
After the upgrade, a complete functional test should be performed.
At a minimum, the following functions should be verified:
NetScaler Gateway
- User login
- ICA launch
- Workspace app
- Browser access
Authentication
- LDAP
- RADIUS
- TACACS
- SAML
- OpenID Connect
- Multi-Factor Authentication
Load Balancing
- HTTP
- HTTPS
- SSL Offloading
- Monitors
- Persistence
Content Switching
- Hostnames
- URL rules
- Redirects
- Rewrite Policies
- Responder Policies
Additional Features
- GSLB
- DNS
- NTP
- SNMP
- Syslog
- NetScaler Console
- EPA
- SSL certificates
The maintenance window should only be closed once all production functions have been tested successfully.
Rollback
Despite careful preparation, a firmware upgrade may still cause unexpected issues.
For this reason, a rollback strategy should be defined before every upgrade.
Recommended procedure:
- Fail over to the working HA partner
- Restore the VPX snapshot
- Restore the system backup if required
- Analyze the root cause
- Perform the upgrade again
A tested rollback procedure can save valuable time in the event of a failed upgrade.
Best Practices
The following practices have proven effective across numerous customer environments.
✔ Always upgrade the Secondary Node first.
✔ Create a snapshot and a system backup before every upgrade.
✔ Download firmware only from official sources.
✔ Review the release notes completely.
✔ Remove old firmware versions only after the upgrade has been completed successfully.
✔ Perform a functional test after every major step.
✔ Prefer the CLI for firmware upgrades whenever possible.
✔ Verify HA synchronization after the upgrade.
✔ Monitor the environment with PRTG, ControlUp, NetScaler Console, or your existing monitoring platform throughout the maintenance window.
✔ Document all changes.
✔ Validate new firmware in a test environment first whenever possible.
✔ Document the firmware version, build number, upgrade date, and relevant observations in the maintenance record.
Checklist
Before the Upgrade
- Release notes reviewed
- Maintenance window scheduled
- Users informed
- Snapshot created (VPX)
- Configuration saved
- System backup created
- Firmware downloaded
- SHA256 checksum verified
- Functional test completed
- Available disk space verified
After the Upgrade
- Firmware version verified
- HA status UP
- Synchronization SUCCESS
- Gateway tested
- LDAP tested
- SAML tested
- Load Balancing verified
- Content Switching verified
- SSL configuration verified
- Monitoring verified
- Event logs reviewed
- Backup updated
Conclusion
A NetScaler ADC firmware upgrade involves much more than simply installing a new software version. Careful preparation, a complete backup, a tested rollback strategy, and structured functional validation are all essential for a successful upgrade.
Changes introduced in recent firmware versions, such as encrypted RPC communication and additional certificate digest validation, demonstrate that modern upgrades require more attention than a simple version change. By considering these aspects during the planning phase, most upgrade-related issues can be avoided.
Organizations that follow a standardized firmware upgrade process can minimize downtime, improve operational security, and ensure that their NetScaler infrastructure remains secure, stable, and up to date.
Do you need support with a NetScaler firmware upgrade?
Deyda Consulting supports organizations with the planning, implementation, and troubleshooting of NetScaler ADC upgrades – from individual appliances to highly available enterprise environments.












