Cisco Catalyst Stack Upgrade

Well… It will reboot your whole switch stack at once, In case you were wondering. But it has a neat feature of automatic rollback to the previous IOS XE version if something goes south with the newly upgraded switches.

The same goes for non-stacked Cisco Catalyst C9200 and C9300 switches, but the question was, and the answer is hard to find if the stack would reload members sequentially or it would just reload all members at once. The answer is of course the least good option which makes the upgrade impossible without network outage even if other devices are connected to the stack redundantly (to two or more stack members).

The whole procedure is fairly simple:

Switch Upgrade install mode

New Cisco switches are now usually shipped with install mode configuration for software installation. The other (older) bundle mode was simply a boot variable on the switch config that stated where the .bin files is saved on the switch flash and if multiple .bin files which one to load upon switch reboot.

In install mode, the switch is unpacking the .bin IOS file and crated an packages.conf file that states which multiple IOS unpacked files switch needs to load in its memory to book correctly. This is better because it uses less memory for running specific IOS and also speeds up switch boot which is still about 10-15 minutes for C9200/C9300 as well as for Nexus9K 3K 5K etc…

1. Check if the switch is in running in install mode by listing the output of show verison:

stack1#show ver

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 52 C9200L-48PXG-4X 17.03.05 CAT9K_LITE_IOSXE INSTALL
2 52 C9200L-48PXG-4X 17.03.05 CAT9K_LITE_IOSXE INSTALL
3 50 C9200L-48PXG-2Y 17.03.05 CAT9K_LITE_IOSXE INSTALL
4 50 C9200L-48PXG-2Y 17.03.05 CAT9K_LITE_IOSXE INSTALL

If your switch is in bundle mode you can change it to install mode following this nice guide

2. Those switches usually don’t have much flash, so if more than one IOS version is already expanded on the switch it could happen that the third .bin file will not even fit. With this command all .bin and packages files that the switch is not currently using will be deleted to make space for loading new IOS on the switch:

stack1#install remove inactive

After the cleaning step you can load your .bin file from accessible FTP or TFTP server (you should know USER and PASS of your FTP server):

stack1#copy ftp://USER:[email protected]/tftp/cat9k_lite_iosxe.17.03.05.SPA.bin flash:
Destination filename [cat9k_lite_iosxe.17.03.05.SPA.bin]?

Accessing ftp://*:*@10.252.102.16/tftp/cat9k_lite_iosxe.17.03.05.SPA.bin...!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

After the .bin image loads to the switch, you are ready to start the install add procedure which will copy the .bin file to all stack members, unpack it there in switch flash and create package.conf file for new IOS version.

stack1#install add file bootflash:cat9k_lite_iosxe.17.03.05.SPA.bin

After this is done you can check to see if your new version of IOS is listed as an unactivated version below the active old version on the switch by typing show install summary:

stack1#show install summary [ Switch 1 ] Installed Package(s) Information: State (St): I - Inactive, U - Activated & Uncommitted, C - Activated & Committed, D - Deactivated & Uncommitted -------------------------------------------------------------------------------- Type St Filename/Version -------------------------------------------------------------------------------- IMG C 17.03.04b.0.13 IMG I 17.03.05.0.6600

[ Switch 2 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.04b.0.13
IMG I 17.03.05.0.6600

[ Switch 3 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.04b.0.13
IMG I 17.03.05.0.6600

[ Switch 4 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.04b.0.13
IMG I 17.03.05.0.6600

As you see above, I have a switch stack with 4 members and they all have unpacked new IOS XE version 17.03.05 which is waiting to be activated and committed. At this time switches are still running the older version of IOS 17.03.04b and waiting for our next action.

Automatic rollback procedure

Auto abort timer: inactive is visible at the end of the output. It shows that the auto abort timer used for rollback of switch stack that didn’t boot correctly is not initiated. This timer starts after our next command when the switch stack reboots and starts to load the newly added IOS XE image.

Auto abort timer is by default 120 minutes and starts after install activate command and switch reboot. If the new IOS version is not committed after the switch boots back, the auto abort timer expires after 120 minutes, switch reboots again and boots the previous IOS version. I

t does that as a precaution and rollback procedure because no commit command was issued which usually means something went wrong and the administrator was not able to login back into the switch and confirm with the commit command.

Auto abort timer can be changed to some other value to make rollback faster. Keep in mind that switch upgrade with 4 stack members is around 25 minutes so try not to change this timer to something impossibly low or try not to change it at all.

stack1#install activate auto-abort-timer 45

Don’t forget to commit after reboot.

stack1#install activate

Install activate initiates switch stack reboot and loads new IOS version.
After switch reboots, check if the new software is running on all stack members:

stack1#show install summary
[ Switch 1 2 3 4 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.05.0.6600
--------------------------------------------------------------------------------
Auto abort timer: active
--------------------------------------------------------------------------------

It’s all good and we will commit new IOS to provide the switch with information that he booted correctly and it can stop auto abort timer.

stack1#install commit

install_commit: START Wed May 25 12:32:55 ROMANIA 2022
May 25 12:32:56.050 %INSTALL-5-INSTALL_START_INFO: R0/0: install_engine: Started install commit

install_commit: Committing PACKAGE

--- Starting Commit ---
Performing Commit on all members
[1] Commit package(s) on switch 1
[1] Finished Commit on switch 1
[2] Commit package(s) on switch 2
[2] Finished Commit on switch 2
[3] Commit package(s) on switch 3
[3] Finished Commit on switch 3
[4] Commit package(s) on switch 4
[4] Finished Commit on switch 4
Checking status of Commit on [1 2 3 4]
Commit: Passed on [1 2 3 4]
Finished Commit

That’s it, your switch is upgraded.

If you have trouble with getting an FTP from which load the .bin IOS XE to the switch, here is a simple guide on how to get one running inside Ubuntu VM. You can run this somewhere in the cloud, on-prem or on your machine as a VM. Just be sure it will be accessible from the switch for FTP.

Install an FTP Server

Get an FTP server where you can put your IOS image .bin file for the switch to download it to its flash:
Install an Ubuntu server and connect to it. Install vsftpd FTP service on it:

sudo apt update
sudo apt install vsftpd

Backup the original FTP config file so you can revert all the thing that you mess up while trying this:

sudo cp /etc/vsftpd.conf /etc/vsftpd.conf.orig

Create FTP user ftpuser you can name it differently of course:

sudo adduser ftpuser

Create an FTP folder for that user and give it permissions:

sudo mkdir /home/ftpuser/ftp
sudo chown ftpuser:ftpuser /home/ftpuser/ftp

Open FTP server config file and change some settings to match listed below:

sudo nano /etc/vsftpd.conf
# Allow anonymous FTP? (Disabled by default).
anonymous_enable=NO
#
# Uncomment this to allow local users to log in.
local_enable=YES
write_enable=YES
chroot_local_user=YES

Apply new config by restarting FTP service:

sudo systemctl restart vsftpd

Now the server is running and you can add your IOS image into /home/ftpuser/ftp to be accessible from switch.

Leave a Reply