Supervisor Upgrade

This past week, I upgraded one of our Sup-720s to the new Vs-720 which supports VSS (virtual switching system). I’m not using VSS at this time, but we will in the near future. We are only using a single Supervisor in each of our distribution switches. I did some work in the lab to make sure the supervisor upgrade would be as quick and painless as possible. Our access switches are all dual homed to another distribution so the impact would typically be minimal.

Here is what I did for the supervisor upgrade ::

I had the new supervisor running in a lab switch for a few weeks. Our production switches are fully loaded 6513 switches and what I was using in the lab was 6506 switches.

1. Let Supervisor burn in for 2-4 weeks in a lab.
2. Backup current config of production supervisor (this will be used to prep the new supervisor). Also record output of “show ip inter brief”
3. Establish base config for new supervisor.

You are probably wondering what constitutes a “base config”. Well, it’s basically everything except for the interface level configurations and aaa, plus a few miscellaneous pieces. If you try pasting in your interface configuration into a test supervisor, you have to make sure you have modules in those slots. In my case, I didn’t have the same switch with the same number of modules to use in a lab. For example, you can’t put a config in for module 10 (interface 10/x) if you don’t have anything in that module. I decided to exclude all interface configurations from my “base config”. You also want to take out all the aaa configs as well as line configurations (console, vty) and enable pass/secret. The reason for this is that you may find yourself locked out of your switch when there are no interface configurations to authenticate to TACACS. There are also a few miscellaneous configurations that will error if certain modules don’t exist (for example, monitor sessions).

4. At this point, you want to have 3 text files created (base_config, if_config, aaa). Simply cutting all the interface configs from the backup config into a new file will suffice. Also, when I say “interface configs”, leave any vlan and port-channel configs in the base_config file. These can be present regardless of what modules exist in switch. The only thing in the “if_config” file should be “interface x/y” configurations. For the aaa config, simply cut out all the aaa stuff and any enable secrets and line configs. You can also include those few miscellaneous configs that will error if certain modules don’t exist. A good rule of thumb for creating the base_config is cut the interface and aaa stuff out into it’s own files leaving everything else as base config.

5. Now test your base_config in the new supervisor. For this, you can copy and paste the base_config into a console terminal window. You will probably find certain commands that throw errors when pasting into the new supervisor. This is to be expected, and you will have to sort through them as they happen. be sure to watch the output while pasting. You may find it easier to paste certain sections at a time. You may have to tweak your terminal setting to prevent the window from getting overloaded and commands pasted start to fail. You can use the secure CRT settings below to slow it down.

6. You will notice that all your port-channel and vlan interfaces are shutdown in the config. Now go back to your base_config file and add “no shut” to what needs to be up/up based on the output you recorded from step 2.

7. Consider VLANs at this point. Now even though we use VTP, I still chose to make sure the VLANs were on the new supervisor before starting. I didn’t want to delay anything because vlans were still being learned. Create your config file you can use to create your vlans. Also keep in mind who has the vtp server/client roles currently. When I created the vlans, I decided to make the new supervisor a vtp client with a low revision number as well. Remember the higher revision number can cause other switches vlan config to be overwritten which is why I set to 0. Call it vlan_config or something. See example ::

vtp mode server
!
vlan 1
name default
!
vlan 2
name servers
!
exit
vtp mode client
! set config rev to 0
vtp mode transp
vtp mode client
!

8. At this point, you should have the base_config and vlan_config up and running on the new supervisor. You can audit your config as necessary against the backup config you took earlier. Feel free to reboot and erase the config and start from scratch as much as you like to make sure the configs are good to go. Once you feel good about them, save the config to nvram.

9. You need to add some commands at the very end of your “if_config” file. Once your new supervisor was to boot up with all the production modules that exist, you would find that all the interfaces are shutdown. Use the output from step 2 to make sure you “no shut” interfaces that should be up/up. Save your config. See example ::

!
interface range gig3/1 - 6, gig3/21 - 25
no shut
!
interface range gig4/1 - 16, gig5/1 - 16, gig6/1 - 16, gig8/1 - 16, gig9/1 - 16
no shut
!

10. Now that you have two separate files called “if_config” and “aaa”, you can upload them to the supervisor flash using tftp. If you have a few extra modules available, feel free to do some tests using a few of them. You can create a if_test_config file and take pieces of the if_config that would align with what modules you have. You can copy the interface test config into your supervisor just to make sure things look ok.

11. At this point, you have done enough testing in the lab with your configs and you’re ready to make the change. The supervisor that is ready to go should have the base_config and vlan_config currently running on switch. The if_config and aaa should be in flash. You should have reseated the supervisor several times in your testing to emulate what you would be doing during your change window.

12. Capture outputs of “show interface trunk”, “show vlan”, “show vtp stat”, “show ip interface brief”, “show etherchan sum”, “show ip eigrp neig” (or ospf), “show cdp neigh” on production switch.

13. Power off switch, insert new supervisor, boot up switch connecting to console. Allow time for all modules to be tested and functional before going any further. Check “show mod” output.

14. Paste in if_config. “copy sup-bootflash://if_config run”. Watch output and record any errors. Make sure you have a decent scroll-back buffer on terminal.

15. Wait about 5 minutes for everything to settle. Do an audit comparing outputs from step 12 to the new supervisor outputs. It’s a little tedious, but you can use some file compare utilities to help somewhat.

16. If everything looks good, save config, then paste in aaa config (”copy sup-bootflash://aaa run”). Try accessing switch via telnet/ssh, if everything logs in ok, save config for a final time.

17. Check “show process cpu history” for good measure. Also clear and check the logs.

Ted
CCIE No. 21785

This entry was posted on February 17, 2009 at 11:30 am and is filed under Routing&Switching, Cisco. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.

| Print