Switch and Router Firmware Upgrade: Process, Risks and Rollback Plan

Firmware upgrades of network devices are usually triggered by three reasons: patching security vulnerabilities, hardware compatibility requirements, and known defects in the version currently running. Whatever the reason, an upgrade is a change, and a change requires a process.

1. Pre-Upgrade Assessment

1. Confirm that the upgrade is necessary

A new release is not a reason to upgrade in itself. Confirm first: what problem does this version solve? What are the consequences of not upgrading? If the device is running stably and there is no security risk, choosing not to upgrade immediately is also a reasonable decision.

2. Select the target version

Give priority to a stable or recommended version rather than the newest release. Vendors usually mark the version they recommend. Avoid versions that have only just been released; wait a while and let the community and other users validate them.

3. Check the version upgrade path

Some devices do not support a direct upgrade across major versions and must be upgraded step by step. Always read the vendor's upgrade path documentation.

4. Confirm hardware compatibility

• Whether memory and storage space meet the requirements of the new version
• Whether line cards and modules are supported by the new version
• Whether the licence needs to be renewed

2. Pre-Upgrade Preparation

• Back up the current configuration file and the current firmware file
• Record key operating metrics of the current version (as a comparison baseline)
• Confirm that the out-of-band management channel is available (to avoid losing contact if the upgrade fails)
• Have the previous firmware file ready for rollback
• Confirm that no other change is running at the same time
• Notify the relevant business owners and agree the maintenance change window
• Define the rollback trigger conditions and the responsible engineer

Key reminder: out-of-band management (console / BMC / management port) must be tested and confirmed working before the upgrade. If the upgrade fails, the out-of-band channel is the only lifeline you have.

3. Upgrade Execution Process

1. Choose the timing

Schedule the work in an off-peak change window and allow plenty of time. Upgrades often take longer than expected, especially major-version upgrades and batch upgrades across many devices.

2. Run a pilot before the full rollout

With several devices of the same model, upgrade one first and observe it for 24 to 72 hours. Once it is confirmed to be stable, roll the upgrade out to the remaining devices.

3. Upgrade one device at a time, never in parallel

Core and aggregation devices must be upgraded one at a time, and you should confirm that the previous device has fully recovered before touching the next. Upgrading in parallel turns incident handling into chaos.

4. Verify immediately after the upgrade

• Whether the device boots normally and all line cards report Normal
• Whether the configuration has been fully retained
• Whether port status, VLANs, routing and link aggregation are normal
• Service connectivity testing (do not rely on device status alone)
• Check the alarms generated by the new version
• Review the logs for anything abnormal

5. Observation period

Keep observing for several days after the upgrade, focusing on CPU and memory, log anomalies and port flapping. Some problems do not surface at the moment the upgrade is performed.

4. Rollback Plan

Rollback is part of the upgrade plan, not an emergency response after failure.

Rollback trigger conditions (these should be written down before the upgrade):

• The device cannot boot normally
• The configuration cannot be fully restored
• A critical service is interrupted for longer than the agreed time
• Repeated alarms or performance degradation caused by the new version

Rollback steps:

1. Load the backed-up previous firmware version

2. Restore the backed-up configuration

3. Verify service connectivity

4. Record the reason for and symptoms of the rollback, and report them to the vendor

Note: the configuration format of some new versions is not compatible with that of older versions, so restoring the configuration directly during a rollback may fail. Confirm that the rollback path is workable before you upgrade.

5. Common Mistakes

• Upgrading without a configuration backup, then having to rebuild it by hand after a rollback
• Ignoring the version upgrade path and failing by jumping straight across major versions
• Running batch upgrades in parallel, so the affected scope cannot be determined when a problem occurs
• Looking only at device status and never testing services, so the fault is not found until users complain
• Having no rollback plan, so decisions are made on the spot and the time cost doubles
• Choosing the latest version instead of the stable one, and doing the vendor's testing for them

6. Summary

The risk of a firmware upgrade lies not in the technical difficulty but in process discipline. Backup, pilot, one at a time, verify, observe — with these five steps in place, an upgrade stops being a gamble and becomes a controlled operation. For devices carrying critical services, spend an extra maintenance window rather than skipping any one of these steps.

Note: This article is provided for technical exchange and experience sharing. Always refer to the original manufacturer's technical documentation and the conditions of your own environment. Maintenance and change operations should be carried out by qualified engineers.