Andrew Travis's Blog

SNMP Perl Scripts

supergoop — Fri, 30 Dec 2011 19:31:34 +0000

Occasionally I need to access a switch with goofed up AAA configuration or a bunch of switches that I want to execute the same commands on. When I used to work for a big Fortune 500 company this became more of a necessity, so I wrote the following Perl script to copy and push configuration to the devices in question. I found myself needing this script again as I prepared hundreds of routers and switches for addition to LMS (CiscoWorks). The only information you need is the read-write community string to copy or update the configuration.

Copying the Configuration

This script copies the switch’s running configuration to the TFTP server. The usage is perl TFTP_Copy.pl 192.168.1.1 Core-Config.txt, where 192.168.1.1 is the router/switch and Core-Config.txt is the filename for the config file copied to your TFTP server. Here is the code:
use strict; use warnings; use lib "C:/Perl/site/lib"; use Cisco::CopyConfig ();


# declare variables

my $ipAddress = $ARGV[0];

my $SNMPcommunity = 'SyComCommunity';

my $configFile = $ARGV[1];

my $TFTPServer = '192.168.52.5';
# create a new Cisco::CopyConfig object

my $copyConfig = Cisco::CopyConfig->new(

	Host  => $ipAddress,

	Comm  => $SNMPcommunity,

);
# copy the running-configuration file to the TFTP server

print "Now copying the running-config on $ipAddress to the TFTP server... ";

if($copyConfig->copy($TFTPServer, $configFile)){

	print "success.\n";

}

else{

	die $copyConfig->error();

}

# free up memory $copyConfig = undef;

Once you open the copied configuration file, it will be an exact copy of the running configuration. This is useful for making backups of your device configurations or for examining the configuration to determine why you cannot login (AAA, incorrect local password, etc.).

Updating the Configuration

Here is where this script gets really useful. If you need to update the configuration, but can’t get in because of an erroneous AAA configuration or just want to save time over interactively logging into each device, you can utilize the merge function to merge a configuration that you push to the device. The usage is similar to before, but now we are specifying the configuration file to merge: perl SNMP_Update.pl 192.168.1.1 AAA_SNMP_Changes.txt. Here is the code:


use strict;

use warnings;

use lib "C:/Perl/site/lib";

use Cisco::CopyConfig ();
# declare variables

my $ipAddress = $ARGV[0];

my $SNMPcommunity = 'SyComCommunity';

my $configFile = $ARGV[1];

my $TFTPServer = '192.168.52.5';
# create a new Cisco::CopyConfig object

my $copyConfig = Cisco::CopyConfig->new(

	Host  => $ipAddress,

	Comm  => $SNMPcommunity,

);
# merge a configuration file into the running-config via TFTP

print "Now updating running-config on $ipAddress... ";

if($copyConfig->merge($TFTPServer, $configFile)){

	print "success.\n";

}

else{

	die $copyConfig->error();

}

# free up memory $copyConfig = undef;

Here is the AAA_SNMP_Config.txt contents that I pushed to the device:

! remove old AAA/TACACS+ configuration no tacacs-server host 192.168.52.20 key sycomisawesome no aaa group server tacacs+ nonexistantrsa no aaa authentication login vtyaccess group nonexistantrsa local enable no aaa authorization exec vtyaccess group nonexistantrsa local no aaa authorization commands 15 vtyaccess group nonexistantrsa local no aaa accounting exec default start-stop group nonexistantrsa line vty 0 4 no authorization commands 15 vtyaccess no authorization exec vtyaccess no login authentication vtyaccess exit ! ! configure SNMP access for LMS access-list 50 permit 192.168.52.233 snmp-server community SyCom rw 50 snmp-server community Technologies ro 50 ! ! configure new ACS server ip tacacs source-interface Vlan1 tacacs-server host 192.168.52.138 tacacs-server host 192.168.52.139 tacacs-server key howistheweather aaa new-model aaa authentication login default group tacacs+ local aaa authorization exec default group tacacs+ local end

I ran my TFTP server on my local laptop. The screenshot below illustrates the Solarwinds TFTP logs when merging the configuration changes with the running configuration:

I didn’t mention, but I guess it needs to be said… this method will not work when using a read-only string or if your SNMP community string is protected by an access-list and your machine (that you are running the script from) is not permitted by that access-list. This further emphasizes the need to utilize access-lists on your SNMP community strings and to choose difficult to guess strings. Even take it up a notch and use SNMPv3 instead of v1/v2c, since v3 is much more secure. If you leave your community strings unprotected by an access-list and, even worse, just use ‘public’ and ‘private’, all I have to do is run my script and I can either pull the entire switch configuration to my laptop for analysis or I can completely alter the switch configuration by pushing my own configuration to the switch. Protect yourself. I also didn’t go into detail in the post, but you can easily automate this with a seed CSV of devices and a while loop in the script. Ultimately, LMS does much of what these scripts do, but I often find myself in the position of preparing devices for LMS to manage (i.e. updating AAA and SNMP) and needing something automated up front to prepare them.

– Andrew

Now I’m With SyCom

supergoop — Wed, 05 Oct 2011 13:27:08 +0000

I rarely post on here about my employment and just keep it technical, but to the astute reader I thought it best to mention that I am no longer with Varrow and have joined SyCom. For me it was a move based on geography. My wife and I grew up in southwest Virginia and wanted to move back. The powers that be at Varrow were kind enough to let me move and just commute to customer sites predominately in North Carolina. This was good for a while, but I soon found myself exhausted with the travel. About the time I began looking for opportunities more local to where I lived, SyCom found me. After a series of interviews and an offer, I have joined SyCom as a Systems Engineer working out of their Roanoke office. I’ve only been with SyCom for a week and a half, but I’m loving every minute of it. I’ve transitioned into a hybrid of presales and implementation, focusing more on Cisco. I’m hoping this new move will find me updating this blog more frequently too. Thank you for reading.

- Andrew

Updates to the Cisco UCS

supergoop — Wed, 21 Sep 2011 20:05:50 +0000

Cisco has released some updates for the Cisco Unified Computing System, both in the form of hardware and software. The new Fabric Interconnects are the 6248s, replacing the 6120s and 6140s. At just 1U, they provide 48 ports of connectivity with the 16 port expansion module; each of these ports are unified, meaning they are configurable for TenGigabit Ethernet or Fibre Channel over Ethernet. Each of the ports in the 16 port expansion module can be configured for TenGigabit Ethernet, FCoE or Fibre Channel. Other updates in the 6248s include support for 4096 VLANs (over 1024) and 80Gbps half-width blade connectivity (over 20Gbps). Future enhancements to the 6248s will be a Layer 3 daughter card and support for FabricPath. More details to come on those…

Updates to the IO Modules, the 2208XP now have eight 10GigabitEthernet ports instead of four. This is the trick to providing 80 Gbps connectivity to each IO Module, 160Gbps connectivity per chassis (when utilizing 16 uplinks per chassis). This also quadruples the server bandwidth from 20Gbps (10Gbps on each IOM) to 80Gbps (40Gbps on each IOM)! Cisco has also released the second generation Virtual Interface Card, called the 1280. This VIC supports 40Gbps connectivity per slot and scales up the number of VM interfaces (with ESXi 5.0). There are no updates to the 5108 chassis, but that’s the beauty of modular design. You only need to replace the Fabric Interconnects, IO Modules and mezzanine cards to take advantage of these benefits — chassis and blades (apart from mezzanines) stay the same!

In order to support this new hardware, Cisco has released firmware version 2.0(1). This firmware also provides some additional features, such as iSCSI boot and VM-FEX integration with Red Hat Linux. Here are the other features included in 2.0(1):

Licensing – Updated information for new UCS hardware.
Firmware Bundle Option – Enables you to select a bundle instead of a version when updating firmware using the Cisco UCS Manager GUI.
Disk Drive Monitoring Support – Support for disk drive monitoring on certain blade servers and a specific LSI storage controller firmware level.
iSCSI Boot – iSCSI boot enables a server to boot its operating system from an iSCSI target machine located remotely over a network.
Pre-login Banner – Displays user-defined banner text prior to login when a user logs into Cisco UCS Manager using the GUI or CLI.
Unified Ports – Unified ports are ports on the 6200 series fabric interconnect that can be configured to carry either Ethernet or Fibre Channel traffic.
Upstream Disjoint Layer-2 Networks – Enables you to configure Cisco UCS to communicate with upstream disjoint layer-2 networks.
Virtual Interfaces – The number of vNICs and vHBAs configurable for a service profile is determined by adapter capability and the amount of virtual interface (VIF) namespace available on the adapter.
VM-FEX Integration for VMware – Cisco Virtual Machine Fabric Extender (VM-FEX) for VMware provides management integration and network communication between Cisco UCS Manager and VMware vCenter. In previous releases, this functionality was known as VN-Link in Hardware.
VM-FEX Integration for KVM (Red Hat Linux) – Cisco Virtual Machine Fabric Extender (VM-FEX) for VMware provides external switching for virtual machines running on a KVM Linux-based hypervisor in a Cisco UCS instance.

That’s about it as far as updates go. Check out M. Sean McGee’s blog post for even more detail. And if you’re wondering how hard it is to upgrade your environment, not hard at all; check out Cisco’s walkthrough to replacing your Fabric Interconnects and IO Modules with minimum downtime.

Thanks for reading.

- Andrew

Sources:

Cisco 6248UP Overview

Cisco 6200 Data Sheet

Firmware 2.0(1) Release Notes

UCS Express SRE-V 1.5 Released

supergoop — Fri, 15 Jul 2011 11:42:20 +0000

Cisco has released version 1.5 of their software, giving you the ability to add and modify users as well as install licenses in the GUI. Prior to this, you had to enter the SRE-V CLI (see my earlier post). Though, the most notable improvement is the ability to manage the UCS Express servers with vCenter! Prior, users had to connect their vSphere clients directly to each UCS Express server. Now those servers can be added to vCenter for better integration to the existing virtual environment. Cisco also fixed a bunch of bugs as is customary in software releases. Check out the release notes for all of the details: http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/sre_v/1.5/release/notes/sre_v_release.html.

For the vCenter integration, you will need to either buy a vCenter-managed license from Cisco or apply an existing Advanced/Enterprise/Enterprise Plus vSphere license. From talking with Jason Nash, buying a vCenter-managed license from Cisco is the cheaper option. The UCS Express managed by vCenter supports DRS, VMotion, etc., but is not supported by TAC. vCenter integration is a much welcomed (and requested) feature I’m glad to see added.

– Andrew

Updates to Cisco’s ACE Load Balancers

supergoop — Fri, 08 Jul 2011 17:57:56 +0000

Introducing ACE30 Module

Cisco has released some hardware updates to their ACE load balancers. The ACE20 has been replaced with the ACE30, offering increases in SSL transactions per second, L4/L7 connections per second and HTTP compression. See the table below for more metrics on how the ACE30 has surpassed the ACE20 module:

Testing Metric	ACE20	ACE30	Comparison
Throughput	16 Gbps	16 Gbps	Same
L4 CPS	325,000	500,000	54%
L7 CPS	NA	200,000	NA
Concurrent Connections	4M	4M	Same
SSL TPS	15,000	30,000	100%
SSL Bulk Throughput	3.3 Gbps	6 Gbps	82%
Max Compression	NA	6 Gbps	NA

Updates to ACE Software

Cisco is also updating the software for the ACE30 module and ACE 4710 appliance to A4(1.0). Going forward, Cisco will release one software release for both the ACE module and ACE appliance. A4(1.0) will include these new features:

In-band health checking of real servers for better and faster monitoring.
Hardware-based HTTP compression.
SSL Cipher-based load balancing to send clients with lower encryption to a separate serverfarm.
Many others like probe inheritance, buddy associations of cookies across multiple services, and better integration with GSS.

Updates to ANM

I’ve also seen a demo of Application Network Manager (ANM) 4.1 and it looks much sleeker than the past version of ANM. ANM is the GUI management system for ACE for those less-inclined to the CLI. Up till now, I’ve always been a CLI guy, but with the introduction of topology maps and historical/real-time graphs I may just have to come over to the dark side of ANM. Check out the screenshot below of the new interface:

Updated features to ANM include:

Visualizations – topology maps, historical graphs, realt-time graphs, VM details
VMware Integration – easy creation of real servers from VM information provided by vCenter
ANM Appliance – closed virtual appliance to deploy as a VM within vCenter
Operations API – automate exporting information and adding, removing and modifying servers as well as support for 3rd party tools
Scheduled ACE Backup – automated backup of ACE configurations

Conclusion

I love to see advancements in this area of Cisco’s arsenal. ACE load balancing combined with GSS site failover mixed with OTV is going to be big. Imagine automatically swinging the load from the local load balanced serverfarm to a remote serverfarm in a different data center when traffic bursts beyond the local serverfarm’s configured thresholds. The future is fast becoming reality.

– Andrew

Source and Source

Upgrading the UCS Express to v1.1

supergoop — Thu, 26 May 2011 05:11:22 +0000

Cisco released version 1.1 of their SRE-V software (more commonly referred to as UCS Express) earlier this month. We had a customer waiting for RAID1 support before they would build any VMs on their UCS Expresses. This post entails the steps involved in upgrading the software and setting up RAID1. Read on…

Upgrading the Software

First things first, download the software from cisco.com. I downloaded the sre-v-k9-upgrade.smv.1.1.1.zip file and unzipped it to my FTP server. Next, I SSH’d to the 3945 router and connected to a session on the SRE:

router#service-module sm2/0 session ucse-1# software install package url ftp://192.168.0.191/sre-v-k9.smv.1.1.1.pkg username varrow password huh-riiiight

This will kick off the software upgrade process. We were copying our installation files over a WAN to the branch office with the UCS Express, so it took some time (45 minutes) to copy all of the files up to the SRE. The system will reboot, loading the new software, and eventually reach this point:

STARTED: superthread_startup.sh Waiting 15 ... SYSTEM ONLINE ucse-1#

Congratulations, the software is installed. Pretty simple…

Configuring RAID 1 – The Easy Way

Now, we’re ready to configure RAID 1 on the UCS Express. There are two methods to converting the JBOD disks to RAID 1: keeping the VMs or blowing them away. Since our customer has two UCS Expresses, I tried both methods. First up (and my favorite), blowing it all away. For this method, you will need to backup any of your VMs to shared storage such as NFS. I would recommend this anyway just in case something goes wrong. At the hypervisor prompt, move the system into disk maintenance mode:

ucse-1# hypervisor set disk maintenance

This will reboot the SRE. Once it comes back up, go ahead and move the VMs to an NFS share if you haven’t already done so:

Inventory > Configuration > Storage > Browse Datastore
Right-click the VM folder and select Move To…
Move the VM to shared storage not on the UCS Express.

Now you can delete datastore1 and datastore2:

Inventory > Configuration > Storage > datastore1 > Delete
Inventory > Configuration > Storage > datastore2 > Delete

Now is the fun part, going into RAID setup to remove all of the drives and create a new logical RAID 1 volume. At the hypervisor prompt, enter RAID setup mode:

ucse-1# raid setup You are about to enter RAID management CLI console. !!! CAUTION !!! We recommend that you shutdown all the virtual machines, backup datastores, and move the system into disk maintenance mode (command 'hypervisor set disk maintenance') before changing any RAID configuration. Otherwise, it might damage the system. We also recommend that you first remove the datastore, and then remove the disk volume.

After modifying the RAID configuration, use the ‘Rescan All’ function in the vSphere Client GUI. Go to Inventory > Configuration > Storage. The rescan process might take several minutes. Next, display the logical drives:

raid-cli>logdrv ID RAID Disks Stripe Size(MB) DiskID Name 0 JBOD 1 N/A 476940.02 (1) JBOD on port 01 (00) 1 JBOD 1 N/A 476940.02 (2) JBOD on port 02 (00)

Delete all of the logical drives in the SRE (two drives):

raid-cli>logdrv -a clear raid-cli>logdrv Array with ID 0 in controller 0 does not exist.

Now that the logical drives have been deleted, we can create a new logical volume that is RAID 1:

raid-cli>logdrv -a add -i 0 -p 1,2 -e 0,0 -z "name=RAID1,raid=RAID1,init=quick" raid-cli>logdrv -v ******************************* Array ID : 0 Array name : RAID1 Array size : 476837.12 MB Array sector size : 512 Array raid mode : RAID1 Array write cache mode : Write Through Number of disks in array : 2 Disk members with ID in array : (1,2) Array activity status : Idle Array functional status : Online Driver Cache Mode: Write Thru Driver Lookahead threshold: 0 Driver Consolidation: enabled

Now that RAID 1 has been enabled, exit out of RAID setup mode and remove the system from disk maintenance mode:

raid-cli>exit ucse-1# hypervisor unset disk maintenance This command will move system out of disk maintenance mode. The system reboots after the scratch storage is switched to use a local datastore. We recommend that you shutdown all the virtual machines before you continue. Are you sure you want to continue?[confirm] No system reboot is required.

With the new RAID 1 logical volume, we need to rescan the storage. Open up your vSphere client to your UCS Express and navigate to Inventory > Configuration > Storage > Rescan All… Before, you probably saw two drives. Now we see only one, mirrored drive!

Follow through the steps to add the new storage. Finally, you’ll reach this screen where you can Finish.

Now that was easy!

Configuring RAID 1 – The “I Don’t Want to Move My VMs Off” Way

This second method assumes you don’t want to, or are unable to, move your VMs off of your local storage. There are many more hoops to jump through in this method since you have to convert the disks to RAID 0 and then RAID 1, but we’ll get you there. Go ahead and place the SRE into disk maintenance mode and once it reboots, move any VMs from datastore1 to datastore2:

Inventory > Configuration > Storage > Browse Datastore
Right-click the VM folder and select Move To…
Move the VMs to datastore2.

Next, we need to determine which JBOD volume you want to migrate to RAID 0:
Inventory > Configuration > Storage > datastore1 > Properties > Manage Paths…
The Runtime Name column provides information about the location of datastore1:
vmhba0:C0:T0:L0
vmhba0:C0:T1:L0
Looking at T0, T is the iSCSI target and 0 is the ID number of the JBOD volume. You’re now ready to delete datastore1 now that your VMs have been moved to datastore2. I had an issue with datastore1 being in use, so I was unable to delete it; I had to reboot the UCS Express to delete it.

Now, with only datastore2 remaining, enter RAID setup mode and delete the disk associated with datastore1:
ucse-2# raid setup raid-cli>logdrv ID RAID Disks Stripe Size(MB) DiskID Name 0 JBOD 1 N/A 476940.02 (1) JBOD on port 01 (00) 1 JBOD 1 N/A 476940.02 (2) JBOD on port 02 (00)

From the output above, take note of the value in the ID column. You will delete the logical disk with that ID. In our case, the disk that datastore1 was on was vmhba0:C0:T0:L0. That 0 in T0 matches the 0 in the ID column.

raid-cli>logdrv -a del -l 0

Rescan the storage in your vSphere client and then create the RAID 0 logical volume.

raid-cli>logdrv -a add -p 1 -e 0 -z "raid=raid0,name=RAID1,init=quick" raid-cli>logdrv ID RAID Disks Stripe Size(MB) DiskID Name 0 RAID0 1 128 476837.12 (1) RAID0 1 JBOD 1 N/A 476940.02 (2) JBOD on port 02 (00)

Once the RAID 0 logical volume is created, perform another storage Rescan in your vSphere client. Click Add Storage… and select the new volume. Navigate through the menu and finally click FInish. Now you’re ready to migrate your VMs from datastore2 to your newly created datastore. Once you’ve completed this, delete datastore2. Back in RAID setup mode, delete the disk associated with datastore2:

raid-cli>logdrv -a del -l 1 raid-cli>logdrv ID RAID Disks Stripe Size(MB) DiskID Name 0 RAID0 1 128 476837.12 (1) RAID0

Now you’re ready to merge the unused SATA drive to the RAID 0 volume. Refer to the output above for the disk ID (0 in our case) of our RAID 0 volume and refer to the earlier output above for the physical disk ID (2 in our case) of the disk we deleted.

raid-cli>migrate -a start -i 0 -l 0 -p 2 -s "raid=raid1" Migration is started on Array 0. raid-cli>migrate Migration is in progress 1% on logical drive with ID 0 in controller #0!


raid-cli>migrate

Migration is in progress 99% on logical drive with ID 0 in controller #0!
raid-cli>migrate

No activity for migration!
raid-cli>logdrv

ID    RAID Disks  Stripe   Size(MB)          DiskID Name

 0   RAID1     2     N/A  476837.12           (1,2) RAID0

raid-cli>logdrv -v ******************************* Array ID : 0 Array name : RAID1 Array size : 476837.12 MB Array sector size : 512 Array raid mode : RAID1 Array write cache mode : Write Through Number of disks in array : 2 Disk members with ID in array : (1,2) Array activity status : Idle Array functional status : Online Driver Cache Mode: Write Thru Driver Lookahead threshold: 0 Driver Consolidation: enabled

This process was crazy slow, but once it’s complete you’ll be ready to exit RAID setup mode and move the SRE out of disk maintenance mode.

raid-cli>exit ucse-2# hyper unset disk maintenance This command will move system out of disk maintenance mode. The system reboots after the scratch storage is switched to use a local datastore. We recommend that you shutdown all the virtual machines before you continue. Are you sure you want to continue?[confirm]

System is rebooting.

After exiting disk maintenance and rebooting, open up your vSphere Client and check the datastore name. I had to change mine as it was “snap…”. I opted for something more readable. I then browsed the datastore and added my VM(s) back to inventory.

Conclusion

We successfully upgraded two UCS Expresses today and setup RAID1. Moving the VMs around and creating the RAID 0 volume took a lot longer and I’m not sure it was worth the effort of not moving the VMs off of the UCS Express onto some form of shared storage. However, if you don’t have shared storage available, at least you can configure RAID 1 without losing your VMs. I’d like to hear how your experiences have been. Please post your results in the comments below. Thanks for reading.

- Andrew

Deploying vWAAS

supergoop — Wed, 30 Mar 2011 15:49:53 +0000

Overview

Earlier I blogged about deploying WAAS on a SRE. In our deployment, we used vWAAS to manage the WAAS appliances. vWAAS, as the v would imply, is a virtual machine. In our case, we deployed vWAAS as the Central Manager, dubbing it vCM. Tired of the acronyms yet? In this post, I’ll detail how to deploy vWAAS.

Deploying the OVA

I bet you’re wondering how to get a copy of vWAAS? So did I. At present, Cisco doesn’t have a license for you to activate vWAAS or even license how many WAAS appliances it can manage. You have to have the CD that came with your vWAAS order. Your Product Authorization Key (PAK) for vWAAS isn’t one you can activate to download a license or software — it’s just for record keeping.

Assuming you have the CD, login to vCenter and pop the CD in. In vCenter, go to File > Deploy OVF Template. Browse to the CD and you’ll see the *****.ova file. Select it, then name the VM. Casually navigate through the wizard until you get to the storage window. vWAAS is hungry and wants 254 GB of thick-provisioned storage. In our deployment, we had to carve up another LUN for him.

Click Finish to complete the installation and watch the progress bar, well, progress.

Close out the “Deployment Completed Successfully” window and power on the VM. Open a console session to the vWAAS:

Next, we’ll do some configuring…

Configuring vWAAS

VWAAS(config)# interface virtual 1/0
VWAAS(config-if)# ip address 192.168.0.239 255.255.255.0
VWAAS(config-if)# exit

Configure the default gateway:

VWAAS(config)# ip default-gateway 192.168.0.1
VWAAS(config)# ip primary-interface virtual 1/0

Ping the IP addresses of the default gateway to verify they can be reached. Add the Enterprise license using the license command:

VWAAS# license add Enterprise

Next, configure your WAAS appliances with the IP address of the Central Manager (central‐manager address 192.168.0.239, cms enable). You should see the devices listed in the next section.

Navigating the Interface

To access vWAAS, open your browser and enter the following: https://central-manager:8443/. Don’t forget the HTTPS or the port 8443. Login to vWAAS using the default credentials of “admin” and “default”.

Once you’ve logged in, you’ll be in My WAN > Dashboard. Navigate to Manage Devices and you should see all of your WAAS Appliances listed:

Finally, you can view graphs to your heart’s content under the Monitor section:

Conclusion

Deploying vWAAS was really simple due to the OVA file to deploy. There really isn’t too much to conclude other than that.

– Andrew

Configuring Cisco WAAS on a SRE

supergoop — Wed, 30 Mar 2011 14:14:51 +0000

Overview

In an earlier blog post, I had installed Cisco UCS on a Services Ready Engine (SRE). The SRE being just a server inside of an Integrated Services Router (ISR); in our case, it is a SRE-900 in the 3945 Generation 2 ISR. One of the available software packages to install on the SRE is Cisco’s very own WAAS. WAAS stands for Wide Area Application Services and accelerates traffic through TCP flow optimization, data redundancy elimination, compression, SSL optimization, and application-level optimization for web, email, and file sharing. Check this page for Cisco’s diagrams and details on how WAAS does this. Now that you know what WAAS is and how it runs on an SRE, let’s get into the configuration.

Understanding the SRE Interfaces

The SRE has three interfaces — two are internal (on the router backplane) and one is external:

Number in Diagram	SM-SRE Interface	Interface Numbering
1	Service-Module Interface to Router	SMx/0
2	Service-Module Interface to MGF	SMx/1
3	External Interface	Application Dependent

The MGF interface is not supported in WAAS, so just forget he’s there. You can use the external interface for management; we tried, but the layer 3 link between our router and the switch to which the WAAS interface connected caused a loop preventing acceleration.

Installing the WAAS Software

This section assumes you have a new SRE with no WAAS software on it at all. If you are just upgrading WAAS, you can use the Central Manager or the WAAS CLI instead of these steps. First, verify that the router is running IOS 15.0(1)M1 or later. Cisco recommends 15.0(1)M3 as of this writing. To download the latest WAAS software, navigate to Cisco’s downloads section. You’ll see two versions of the software; the NPE version stands for Non-Payload Encryption and is for use in countries where disk encryption is not permitted. Once you’ve downloaded the zip file, copy it to an FTP server and extract it. Six files will be extracted:

WAAS‐4.2.1‐K9.bin.srebootloader (Boot Loader)
WAAS‐4.2.1‐K9.bin.install.sre.header (TCL file signature)
WAAS‐4.2.1‐K9.bin.install.sre (TCL file)
WAAS‐4.2.1‐K9.bin (WAAS package)
WAAS‐4.2.1‐K9.key (WAAS application key)
WAAS‐4.2.1‐K9.installer

Now, configure the networking on the router for the SRE:

interface SM1/0
description WAAS
ip unnumbered Vlan1
service-module ip address 192.168.13.15 255.255.255.0
service-module ip default-gateway 192.168.13.1
ip route 192.168.13.15 255.255.255.255 SM1/0

Note that if you’re using the external interface on the WAAS, instead of “service-module ip address x.x.x.x y.y.y.y” use “service-module external ip address x.x.x.x y.y.y.y”. With the configuration above you won’t be able to manage the SRE using the “service-module sm1/0 session” command as we didn’t configure the backplane to talk to the SRE. Instead, you’ll be able to access the service-module using telnet or SSH.

Next, install the software on the SRE:

Router# service‐module sm1/0 install url
ftp://username:passwd@ftpserver/directory/waas‐accelerator‐4.2.3.9‐k9.bin

As the install progresses (about 10-15 minutes total), you can verify the status by executing “service-module sm1/0 status”.

Configuring the WAAS SRE

If you setup IP addresses to allow backplane communication, then execute “service-module sm1/0 session” to gain access to the WAAS SRE. Otherwise, telnet/SSH to the IP address you defined (in our case, 192.168.13.15). The first time you login, the username is “admin” and the password is “default”. Once you’re in, execute these commands:

restore factory‐default
device mode application‐accelerator
!
hostname clt-waas
central‐manager address 192.168.0.239
!
! Note: use Gi1/0 if using internal interface, otherwise use Gi2/0 if using external interface:
interface GigabitEthernet 1/0
ip address 192.168.13.15 255.255.255.0
!
primary‐interface GigabitEthernet 1/0
ip default‐gateway 192.168.13.1
!
ip name‐server 192.168.10.2
ip domain‐name varrow.com
ntp server 192.5.41.41
clock timezone EST -5 0
!
clear license Transport
license add Enterprise
!
cms enable

Now the SRE is configured. If it can’t reach the Central Manager, it’ll hang for a few minutes until it times out. Execute “show ip route” on the WAAS to make sure it has the correct default-gateway. Also run some pings and telnets to make sure you have connectivity between the WAAS and the Central Manager. You should be able to login to your Central Manager now and see the WAAS SRE under My Devices.

Configuring WCCP

For our deployment, we configured the default method: WCCP GRE Redirect along with Hash assignment. We entered the following configuration on the WAAS SRE:

wccp router-list 1 192.168.13.1
wccp tcp-promiscuous router-list-num 1
wccp version 2
egress-method negotiated-return intercept-method wccp

Next, we needed to identify which traffic to accelerate. We entered these commands on the ISR:

! define the access-list for traffic to omit from acceleration and traffic to permit for acceleration
ip access-list extended WAAS-Traffic_RemotetoDC
remark WAAS WCCP Mgmt Redirect List – Bidirectional
remark Deny VoIP Control Traffic
deny tcp any any eq 1300
deny tcp any any eq 2428
deny tcp any any eq 2000
deny tcp any any eq 2001
deny tcp any any eq 2002
deny tcp any any eq 2443
deny tcp any any eq 1718
deny tcp any any eq 1719
deny tcp any any eq 1720
deny tcp any any eq 5060
deny tcp any any range 11000 11999
remark Deny MGT Traffic
deny tcp any any eq telnet
deny tcp any any eq 22
deny tcp any any eq 161
deny tcp any any eq 162
deny tcp any any eq 123
deny tcp any any eq 8443
remark Deny Routing
deny tcp any any eq bgp
remark Deny Authentication Traffic
deny tcp any any eq tacacs
remark Accelerate All Traffic Over WAN
permit tcp any any
! note that we opted to accelerate all traffic going back to the DC (permit tcp any any)

Once we defined which traffic to accelerate, we enabled WCCP globally on the router:

ip wccp 61 redirect-list WAAS-Traffic_RemotetoDC
ip wccp 62 redirect-list WAAS-Traffic_RemotetoDC

Finally, we enabled WCCP on the relevant layer 3 interfaces where client-server traffic flow is expected. Note below that on the branch router, WCCP 61 is applied to the LAN interface. This will be swapped at the data center side where WCCP 61 will be applied to the WAN interface. Only complete this step once the aforementioned steps have been completed on the neighboring WAAS SRE.

interface GigabitEthernet0/1
description LAN
ip wccp 61 redirect in
!
interface Serial0/1/0
description T1 to Data Center
ip wccp 62 redirect in
!
interface Serial0/0/0.100 point-to-point
description MPLS to Data Center
ip wccp 62 redirect in

If you login to the Central Manager, it now should show traffic that is being accelerated.

On the data center side, we created one access-list to define acceleration traffic and will add subnets to it as we add WAAS accelerators to remote sites:

ip access-list extended WAAS-Traffic_DCtoRemote
remark WAAS WCCP Mgmt Redirect List – Bidirectional
remark Deny VoIP Control Traffic
deny tcp any any eq 1300
deny tcp any any eq 2428
deny tcp any any eq 2000
deny tcp any any eq 2001
deny tcp any any eq 2002
deny tcp any any eq 2443
deny tcp any any eq 1718
deny tcp any any eq 1719
deny tcp any any eq 1720
deny tcp any any eq 5060
deny tcp any any range 11000 11999
remark Deny MGT Traffic
deny tcp any any eq telnet
deny tcp any any eq 22
deny tcp any any eq 161
deny tcp any any eq 162
deny tcp any any eq 123
deny tcp any any eq 8443
remark Deny Routing
deny tcp any any eq bgp
remark Deny Authentication Traffic
deny tcp any any eq tacacs
remark Accelerate CLT Office to Data Center
permit tcp 192.168.10.0 0.0.0.255 192.168.0.0 0.0.0.255
permit tcp 192.168.11.0 0.0.0.255 192.168.0.0 0.0.0.255
permit tcp 192.168.12.0 0.0.0.255 192.168.0.0 0.0.0.255
permit tcp 192.168.13.0 0.0.0.255 192.168.0.0 0.0.0.255
remark Accelerate Data Center to CLT Office
permit tcp 192.168.0.0 0.0.0.255 192.168.10.0 0.0.0.255
permit tcp 192.168.0.0 0.0.0.255 192.168.11.0 0.0.0.255
permit tcp 192.168.0.0 0.0.0.255 192.168.12.0 0.0.0.255
permit tcp 192.168.0.0 0.0.0.255 192.168.13.0 0.0.0.255
!
ip wccp 61 redirect-list WAAS-Traffic_DCtoRemote
ip wccp 62 redirect-list WAAS-Traffic_DCtoRemote
!
interface GigabitEthernet0/1
description Data Center LAN
ip wccp 62 redirect in
!
interface Serial0/0/0
description T1 to CLT Office
ip wccp 61 redirect in
!
interface Serial0/1/0.100 point-to-point
description MPLS to CLT Office
ip wccp 61 redirect in

Conclusion

And that’s all there is to it! Needless to say, we could have tweaked the configuration to accelerate only certain protocols or only traffic destined for certain subnets. My next post will detail how to deploy vWAAS, a VM acting as Central Manager.

– Andrew

Installing UCS Express When the Boot Loader Fails

supergoop — Mon, 28 Feb 2011 21:05:06 +0000

I posted the other week about how to install the UCS Express. My second UCS Express installation actually failed. I had initiated the hypervisor installation, but it must have failed at some point in the process and the SRE was in a constant reboot cycle. I contacted Cisco TAC and they sent me the following instructions:

! copy the sre-v-installer.smv.x.x.x file to the router's flash router#copy ftp://varrow:######@192.168.0.191/sre-v-installer.smv.1.0.2 flash: Destination filename [sre-v-installer.smv.1.0.2]? Accessing ftp://*****:*****@192.168.0.191/sre-v-installer.smv.1.0.2... ! setup the tftp-server on the router to reference the sre-v-installer.smv.x.x.x file router(config)#tftp-server flash:sre-v-installer.smv.1.0.2 ! login to the SRE router#service-module sm3/0 session Initializing memory #1. Please wait... Initializing memory #2. Please wait... This may take a minute....


 Serial ATA Port 0 : Hitachi HTE545050B9A300

 Serial ATA Port 1 : Hitachi HTE545050B9A300

DDR Memory 4096 MB detected

Intel(R) Core(TM)2 Duo CPU     L9400  @ 1.86GHz

BIOS SM 3.52.6,  BIOS Build date: 08/03/2010

Intel(R) Core(TM)2 Duo CPU     L9400  @ 1.86GHz

BIOS SM 3.52.6,  BIOS Build date: 08/03/2010

System now booting...
Please wait...
Please press P to select Primary Boot Loader ...

          or S to select Secondary Boot Loader ...

          or wait to boot from default configuration ...

.......................................................................................................
SRE module get located and run....

Now booting from secondary boot loader....

Authenticating boot loader....

Secondary Boot Loader authenticated - booting....

Please enter '***' to change boot configuration:

***

 ServicesEngine Bootloader Version : 2.1.30

! the config command is optional, only if you need to change the IP address or TFTP server; you can keep the defaults

ServicesEngine boot-loader>  config

IP Address [1.1.1.6] >

Subnet mask [255.255.255.252] >

TFTP server [1.1.1.5] >

Gateway [1.1.1.5] >

Default Helper-file [sre-v-installer.smv.1.0.2] >

Ethernet interface[external|internal] [internal] >

Default Boot [none|disk|diag|chainloader] [disk] >

Default bootloader [primary|secondary] [secondary] >

ServicesEngine boot-loader> boot helper

Loading tftp://1.1.1.5/sre-v-installer.smv.1.0.2 ... done.
(output omitted)
     Welcome to Cisco Systems Service Engine Helper Software

Please select from the following

1       Install software

2       Reload module

3       Disk cleanup

4       Install License(s)

5       Check Installation

(Type '?' at any time for help)

Choice: 1

Package name: sre-v-k9.smv.1.0.2.pkg

RAID level: -1

Server url: ftp://192.168.0.191/

Username: varrow

Password: #######

Argument: 
Downloading ftp sre-v-k9.smv.1.0.2.pkg
(output omitted)
WARNING:: Software installation will clear disk contents

Continue [n]? y
(output omitted as SRE downloads files from FTP server, this part took about 15-20 minutes to install depending on bandwidth between SRE and FTP server)

Rebooting... Restarting system... System now booting...

At this point, the hypervisor is installed and the SRE is rebooting. Just let the hypervisor boot (it may take 5+ minutes). Don’t hit any keys no matter how tempted you are! Eventually you will see:

SYSTEM ONLINE se-1-1-1-6#

From this point, you can resume the setup to set the default gateway and activate the license.

– Andrew

Guide to Installing Cisco UCS Express

supergoop — Thu, 17 Feb 2011 22:00:12 +0000

I just completed my first installation of Cisco’s UCS Express, a server implanted in a ISR router. If you’re looking to get more information on what the UCS Express actually is, please check out my older post. Not surprisingly on the UCS Express is the lack of Cisco documentation available to configure and deploy it. After an exhaustive search, plenty of trial and error, and a ticket with Cisco TAC, I’ve written up the guide below.

Overview

The Cisco UCS Express is basically a hypervisor running on a Services-Ready Engine (SRE). The SRE is really just a server that fits into the new ISR Generation 2 routers and will run a purpose-built operating system. In this engagement, I worked with the SM-SRE-900-K9 (1.86GHz, 2x500GB SATA, 4GB RAM). The hypervisor is really called a SRE-V. Cisco and VMware partnered to create this hypervisor. Our customer purchased the Cisco 3945 ISR G2, which supports up to four SREs. In the remote router, we have two UCS Express SREs and one WAAS Express SRE (future post coming on the WAAS configuration). The SRE has one GigabitEthernet port on the front of the module and two GigabitEthernet ports on the backplane to the ISR chassis.

Design

Once I wrapped my head around the plethora of terms for this new technology, I began designing… This page, from a Cisco Lab, is hands-down the best resource for configuring UCS Express at present. When I called into TAC, they mentioned they used the same lab to prepare them for supporting UCS Express. When you execute a “show ip interface brief”, you’ll see the SRE interfaces listed as SMx/x:

router#sh ip int bri
Interface IP-Address OK? Method Status Protocol
GigabitEthernet0/0 unassigned YES NVRAM administratively down down
GigabitEthernet0/1 192.168.254.1 YES NVRAM up up
GigabitEthernet0/2 unassigned YES NVRAM administratively down down
Serial0/0/0 unassigned YES NVRAM up up
Serial0/0/0.100 x.x.x.x.x YES NVRAM up up
Serial0/1/0 unassigned YES NVRAM down down
SM1/0 unassigned YES NVRAM administratively down down
SM1/1 unassigned YES unset administratively down down
SM2/0 192.168.13.1 YES TFTP up up
SM2/1 unassigned YES unset up up
SM3/0 unassigned YES NVRAM administratively down down
SM3/1 unassigned YES unset up up
Vlan1 192.168.13.1 YES NVRAM up up

Thought must be placed into what IP address to assign the UCS Express:

In my case, we were simply going to use an IP address on the server subnet at the remote site. The gateway for this subnet resided on a Cisco Catalyst 4500 switch connected to the 3945 router. The problem with this is that the default gateway for the SRE (that you will see configured below) has to be an IP address on the router or nothing outside of the router will be able to reach the UCS Express. So, with that in mind, I set the UCS Express default gateway to the inside (LAN) interface of the router. This didn’t work either. I still couldn’t reach the UCS Express from anything that wasn’t connected directly into the router. Another option was to move the SVIs from the 4500 switch to the 3945 router. The problem with this is that all inter-Vlan routing must go across a 1GigabitEthernet link up to the router. This wasn’t ideal, so I opted to convert the link between the 4500 and the 3945 to a Layer 3 link, assigning IP addresses to each interface. This meant I needed to create a new subnet for the UCS Express servers on the 3945, using a new Vlan. I created a SVI for Vlan 13 (to match the third octet of my server subnet), but since no interfaces were configured for Vlan 13 the Vlan 13 SVI stayed down. Ultimately I had to set the default gateway for my UCS Express to the Vlan 1 SVI (which always stays up), using the subnet I created.

Configuration

interface Vlan1
description Cisco SRE Servers
ip address 192.168.13.1 255.255.255.0
!
interface SM2/0
description UCS Express
ip address 1.1.1.1 255.255.255.252
service-module ip address 1.1.1.2 255.255.255.252
!Application: SRE-V Running on SMV
service-module ip default-gateway 1.1.1.1
service-module mgf ip address 192.168.13.10 255.255.255.0
!
interface SM2/1
description Internal switch interface connected to Service Module
switchport mode trunk
!
ip route 1.1.1.2 255.255.255.255 SM2/0

Cisco TAC didn’t really have much advice on why it had to be configured this way, but through trial and error I found that it worked (and this is the only way I got it to work). Confused on why I have so many IP addresses on the SM2/0 interface? Me too. But the 1.1.1.0 /30 addresses are used for communication on the ISR backplane to the SRE. The 192.168.13.0 /24 addresses are routable and used for our UCS Express servers. The next step is to install the SRE-V software on the SRE. Go to cisco.com and download the latest SRE software. At time of writing, I downloaded sre-v-k9.smv.1.0.2.zip and unzipped it to my FTP server. Installing SRE-V requires a FTP or HTTP server serving up the files. I unzipped the files to my FTP server:

ftp> ls
200 PORT command successful. Consider using PASV.
150 Here comes the directory listing.
-rw-rw—- 1 538 538 141975 Feb 17 14:45 sre-v-installer-k9.smv.1.0.2.prt1
-rw-rw—- 1 538 538 14162632 Feb 17 14:45 sre-v-installer.smv.1.0.2
-rw-rw—- 1 538 538 916 Feb 17 14:45 sre-v-k9.smv.1.0.2.key
-rw-rw—- 1 538 538 250717 Feb 17 14:45 sre-v-k9.smv.1.0.2.pkg
-rw-rw—- 1 538 538 1692 Feb 17 14:45 sre-v-k9.smv.1.0.2.pkg.install.sre
-rw-rw—- 1 538 538 698 Feb 17 14:45 sre-v-k9.smv.1.0.2.pkg.install.sre.header
-rw-rw—- 1 538 538 65453522 Feb 17 14:47 sre-v-mgmt-k9.smv.1.0.2.prt1
-rw-rw—- 1 538 538 299803259 Feb 17 14:51 visor.smv.1.0.2.prt1
226 Directory send OK.

Next, I executed the following command on the 3945:

service-module sm 2/0 install url ftp://user:password@192.168.0.191/sre-v-k9.smv.1.0.2.pkg
Proceed with installation? [no]: yes
Loading sre-v-k9.smv.1.0.2.pkg.install.sre !
[OK - 1692/4096 bytes]

Service module installation
ios_version 15.1(3)T,
ios_image c3900-universalk9-mz
pkg_name sre-v-k9.smv.1.0.2.pkg
key_file sre-v-k9.smv.1.0.2.key
helper_file sre-v-installer.smv.1.0.2

Check target platform capabilities
cpu 1864
Resource check completed successfully. Proceeding to Install….

router#
*Feb 2 20:04:48.526: %SM_INSTALL-6-INST_RESET: SM2/0 is reset for software installation.

router#service-module sm2/0 status
Service Module is Cisco SM2/0
Service Module supports session via TTY line 131
Service Module is trying to recover from error
Service Module heartbeat-reset is enabled
Service Module is in fail open
Service Module status is not available

Module resource information:
CPU Frequency: 1864 MHz
Memory Size: 2530 MB
Disk 0 Size: 500107 MB
Disk 1 Size: 500107 MB
Disk 2 Size: 2055 MB

Install of ftp://*****:*****@192.168.0.191/sre-v-k9.smv.1.0.2.pkg in progress
Install status : File sre-v-k9.smv.1.0.2.key requested

Local Partition Info – (0 apps)
=====================
Retrieving partition information

*Feb 2 20:28:28.302: %SM_INSTALL-6-INST_PROG: SM2/0 PROGRESSING: Validating package signature ….
*Feb 2 20:28:28.426: %SM_INSTALL-6-INST_PROG: SM2/0 PROGRESSING: Parsing package manifest files ….
*Feb 2 20:28:30.338: %SM_INSTALL-6-INST_PROG: SM2/0 PROGRESSING: Starting payload download.
*Feb 2 20:28:34.226: %SM_INSTALL-6-INST_PROG: SM2/0 PROGRESSING: Starting payload download.
*Feb 2 20:28:45.614: %SM_INSTALL-6-INST_PROG: SM2/0 PROGRESSING: Performing Hot install ….

Install successful on SM2/0

*Feb 2 20:31:33.734: %SM_INSTALL-6-INST_SUCC: SM2/0 SUCCESS: install-completed.

Now the hypervisor is installed on the SRE. I can log into the module, verify the IP address, define the gateway, and activate the hypervisor license:

router# #service-module sm2/0 session
Trying 1.1.1.9, 2131 … Open
se-1-1-1-10#
se-1-1-1-10# show hypervisor ip
Hostname: localhost
Domain Name: None
IP Config: 192.168.13.100(Subnet Mask: 255.255.255.0)
169.254.1.1(Subnet Mask: 255.255.255.0)
Default Gateway: None
Preferred DNS Server: None
Alternative DNS Server: None
se-1-1-1-10# hypervisor set ip default-gateway 192.168.13.1
se-1-1-1-10# license activate sreVHost
Evaluation licenses are being activated in the device for the following feature(s):

Feature Name: SRE-V-HOST-LIC
……..
ACCEPT? [y/n]?y

License activation count saved for use at next reload

At this point you’ll want to reload the module for the license to take effect: router# service-module sm2/0 reload. Once the module has reloaded, you should be able to connect to it with your vSphere Client. But, just in case you can’t (and believe me, I couldn’t the first 4-5 times I tried setting this up) here are some useful commands to troubleshoot (taken from here):

se-192-168-13-10# sh ip route
Main Routing Table:
DEST GATE MASK IFACE
1.1.1.0 0.0.0.0 255.255.255.252 eth0
169.254.1.0 0.0.0.0 255.255.255.0 eth2
0.0.0.0 1.1.1.1 0.0.0.0 eth0

se-192-168-13-10# show hypervisor nics
Name PCI I/E Driver Link Speed Duplex MAC Address
vmnic0 0000:01:00.00 E e1000e Down 0Mbps Half c8:4c:75:2e:ce:41 (this is the physical port on the module front)
vmnic1 0000:02:00.00 I bnx2 Up 1000Mbps Full 50:3d:e5:0d:74:c1 (these are the logical ports on the module backplane)
vmnic2 0000:02:00.01 I bnx2 Up 1000Mbps Full 50:3d:e5:0d:74:c0

3 total nics (I – Internal nic, E – External nic)

se-192-168-13-10# show hypervisor vmknic
Intf. Portgroup/DVPort IP Address Netmask MAC
—————————————————————————
vmk0 Management Network 192.168.13.10 255.255.255.0 c8:4c:75:2e:ce:41
vmk1 CiscoReservedLocal 169.254.1.1 255.255.255.0 00:50:56:7f:2f:5c

2 total VMkernel nic(s)

se-192-168-13-10# show hypervisor vswitch
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 128 3 128 1500 vmnic2

PortGroup Name VLAN ID Used Ports Uplinks
VM Network 0 0 vmnic2
Management Network 0 1 vmnic2

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
ciscoSwitchLocal 8 3 8 1500

PortGroup Name VLAN ID Used Ports Uplinks
CiscoReservedLocal 0 2

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
ciscoSwitch 8 3 8 1500 vmnic1

PortGroup Name VLAN ID Used Ports Uplinks
CiscoReserved 0 1 vmnic1

When you’re ready to connect to the UCS Express, the default username is “esx-admin” and the default password is “change_it”:

Hopefully you’ll be able to connect. After I logged in and confirmed that I could in fact login, I went ahead and changed the default password. To do this, login to the SRE (service-module sm2/0 session) and enter “user update esx-admin password __________”. You can go wild creating additional accounts and groups (check this page), but I stuck to the one account for this simple deployment.

When you’ve logged in, you’ll see this familiar screen:

If you want to create an additional vSwitch, it must use the physical GigabitEthernet interface on the front of the module (that’s the only free interface left). You can create additional Virtual Machine Port Groups on vSwitch0. Remember how we defined sm2/1 to be a trunk? Now we can create multiple Port Groups in multiple Vlans on vSwitch0. For us, this is a simple deployment so we left the VM Network Port Group using Vlan 1 (since our VMs will be in the same subnet as the Service Console). Cisco has a pretty stern warning on modifying the ciscoSwitchLocal and ciscoSwitch vSwitches — don’t do it! At this point you can begin deploying Virtual Machines. Oh and don’t forget to write the SRE configuration:
se-192-168-13-10# write memory

Anyone else have experience (good or bad) deploying the UCS Express? Please share in the comments below.

– Andrew