Who needs Quagga? The Cisco Linux Router


I make several posts talking about how you can use Linux shells on your Cisco device. But what about opposite? In this post we’ll have a look at the Embedded Service Router which is running a full Cisco router application on a Linux host. This is mostly meant for IoT solutions but it is also a great lab solution if you can get your hands on it.

There are no prerequisites for the  ESR though if your running a 64 bit of Linux (and you should be!) you will need to install 32bit libraries since its a 32 bit app.

First we need to create a /opt/cisco folder

[root@rhel01 ~]# mkdir /opt/cisco

Then we need to upload the file to that folder and then extract the TAR

[root@rhel01 ~]# cd /opt/cisco/
[root@rhel01 cisco]# ls
[root@rhel01 cisco]# tar -xvf c5921i86-universalk9-tar.SPA.157-3.M

Once that is done we need to rename the folder to be c5921, yes I could have done it with the TAR command but here we are.

[root@rhel01 cisco]# mv c5921i86-universalk9-ms.157-3.M/ c5921/

When we get into the SWROPTIONS we need to map the interfaces between Linux and Cisco, lets have a look at ifconfig to see what is on my RHEL8 box.

[root@rhel01 c5921]# ifconfig
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet netmask broadcast
inet6 fe80::6196:43cc:3955:715c prefixlen 64 scopeid 0x20<link>
inet6 2001:cc1e:9989:0:286b:7faf:b0f2:87ab prefixlen 64 scopeid 0x0<global>
inet6 2001:cc1e:9988:0:fc24:75e7:e6c1:aa66 prefixlen 64 scopeid 0x0<global>
inet6 2001:1234:bbaa:0:a6c9:ac04:2ba4:9d5b prefixlen 64 scopeid 0x0<global>
ether 00:50:56:8a:46:3b txqueuelen 1000 (Ethernet)
RX packets 18772387 bytes 4560184387 (4.2 GiB)
RX errors 0 dropped 103001 overruns 0 frame 0
TX packets 5685854 bytes 597519104 (569.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens193: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:8a:6d:e8 txqueuelen 1000 (Ethernet)
RX packets 81112 bytes 45212072 (43.1 MiB)
RX errors 0 dropped 27 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:8a:14:83 txqueuelen 1000 (Ethernet)
RX packets 120891 bytes 7254330 (6.9 MiB)
RX errors 0 dropped 947 overruns 0 frame 0
TX packets 10 bytes 1470 (1.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens256: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:8a:98:0f txqueuelen 1000 (Ethernet)
RX packets 148571 bytes 49934321 (47.6 MiB)
RX errors 0 dropped 28 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Next we can rename SWROPTIONS.example.txt to SWROPTIONS and then edit the file so the interfaces match what we see in ifconfig

#Use launchapp=c5921i86-universalk9_npe-ms.SSA for NPE image


#This variable has to be set to true for kernel versions that has the patch(RHEL bug 1135347) and 
#to false for kernel versions that does not have the patch. 
#Not required for Centos Linux distributions except kernel version 2.6.36. 
#Mandatory for all other Liinux distributions
#For example for kernel 2.6.32-504.23.4 set flag as below





# Map Linux eth0 to IOS e0/0, type raw
# Set promiscuous true
# Make speed/duplex interface configs available for e0/0 in IOS.
# Monitor and Pull Linux interface changes like speed/duplex/MTU/MAC 
# of eth0 to IOS e0/0.
# Push changes like speed/duplex/MAC of IOS e0/0 to Linux eth0.


# Map Linux eth1 to IOS e0/1, type raw
# Set promiscuous true
# Make speed/duplex interface configs NOT available for e0/1 in IOS 
# Monitor and Pull Linux interface changes like speed/duplex/MTU/MAC 
# of eth1 to IOS e0/1.
# Push MAC changes of IOS e0/1 to Linux eth1.
# Speed/Duplex of e0/1 could not be changed by IOS and hence not 
# pushed to Linux eth1.


# Map Linux eth2 to IOS e0/2, type raw
# Set promiscuous true
# Make speed/duplex interface configs available for e0/2 in IOS 
# DO NOT Monitor/Pull Linux interface changes like speed/duplex/MTU/MAC 
# of eth2 to IOS e0/2.
# DO NOT Push IOS e0/2 changes to Linux eth2.




Once that is done we can start it up with c5921-swr-init.sh script

[root@rhel01 c5921]# ./c5921-swr-init.sh start
starting ./swr_reload...\n

[root@rhel01 c5921]# Loading Image:./c5921i86-universalk9-ms.SPA
./c5921i86-universalk9-ms.SPA running SWR the background, pid=23691, SWR=23692
Child process will exec swr image now....

Then we can access the virtual console with swrvcon and a number for the console session, I tend to use 100

[root@rhel01 c5921]# ./swrvcon 100

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

Cisco IOS Software, C5921 Software (C5921_I86-UNIVERSALK9-M), Version 15.7(3)M, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2017 by Cisco Systems, Inc.
Compiled Thu 27-Jul-17 01:38 by prod_rel_team

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

If you require further assistance please contact us by sending email to

Cisco C5921 (Intel-x86) processor with 370083K bytes of memory.
Processor board ID 100
8 Ethernet interfaces
512K bytes of NVRAM.

--- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]: no
Press RETURN to get started!


From this point this is a regular IOS router that we can setup.

Router(config)#hostname ESR01
ESR01(config)#ip domain-name testlab.com
ESR01(config)#line con 0
ESR01(config-line)#logg synch
ESR01(config-line)#line vty 0 15
ESR01(config-line)#logg synch 
ESR01(config-line)#transport input ssh 
ESR01(config)#username admin sec meowcatPass
ESR01(config)#aaa new
ESR01(config)#aaa authentication login default local
ESR01(config)#enable sec meowcatPass

We can give interfaces IPs, these are different from the IPs on the Linux interfaces.

ESR01(config)#int e0/0
ESR01(config-if)#ip add
ESR01(config-if)#no shut
ESR01(config)#ip route

We can go ahead and setup some routing, everything in IOS is supported.

ESR01(config)#router ospf 1
ESR01(config-router)#network area 0
*Sep 8 20:33:45.424: %OSPF-5-ADJCHG: Process 1, Nbr on Ethernet0/0 from LOADING to FULL, Loading Done
*Sep 8 20:33:45.424: %OSPF-5-ADJCHG: Process 1, Nbr on Ethernet0/0 from LOADING to FULL, Loading Done

A great thing for Collab people is that unlike IOL, the ESR fully supports all phone features, we can do everything from CUCME to call routing to even SAF.

ESR01(config-telephony)#max-dn 5
ESR01(config-telephony)#max-ephones 5
ESR01(config-telephony)#ip source-address
ESR01(config-telephony)#create cnf-file

*Sep 8 20:35:17.319: %EDSP-6-VEC_CHANGE: EDSP0's LES switching vector set to CEF switching
*Sep 8 20:35:17.320: %EDSP-6-VEC_CHANGE: EDSP0's LES switching vector set to CEF switching
*Sep 8 20:35:17.321: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0 added.
*Sep 8 20:35:17.330: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0.1 added.
*Sep 8 20:35:17.331: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0.2 added.
*Sep 8 20:35:17.331: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0.3 added.
*Sep 8 20:35:17.332: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0.4 added.
*Sep 8 20:35:17.332: %EDSP-6-IPV6_ENABLED: IPv6 on interface EDSP0.5 added.
*Sep 8 20:35:18.319: %LINEPROTO-5-UPDOWN: Line protocol on Interface EDSP0, changed state to up

The last thing we need to do is pick a license, otherwise it will stay at 8kbs.

ESR01(config)#license platform throughput level ?
c5921-x86-level0 5 Mbps throughput rate
c5921-x86-level1 10 Mbps throughput rate
c5921-x86-level2 25 Mbps throughput rate
c5921-x86-level3 50 Mbps throughput rate
c5921-x86-level4 100 Mbps throughput rate
c5921-x86-level5 200 Mbps throughput rate
c5921-x86-level6 500 Mbps throughput rate

ESR01(config)#license platform throughput level c5921-x86-level4
*Sep 8 20:39:25.235: %SMART_LIC-5-EVAL_START: Entering evaluation period
*Sep 8 20:39:25.235: %LICENSE_C5920-6-LICENSE_ACTIVATED: Installed license for feature c5921-x86-level4 now in use. Forwarding bandwidth limited to 100 Mbps

Right now we have a working Cisco router on linux! But you may be wondering how the Linux OS can route things through the router. We can use tap interfaces to get working.

If you recall the SWROPTIONS file we made, there was a tap interface defined for the e1/0 interface.


So all we need to do is configure that interface on the ESR and then the tap interface on my RHEL8 box.  If we want more taps we can just change the type in the SWROPTIONS file to tap and add a new tap interface.

[root@rhel01 c5921]# ifconfig tap0 up
[root@rhel01 c5921]# ifconfig tap0

ESR01(config)#int e1/0
ESR01(config-if)#ip add
ESR01(config-if)#no shut

At this point we can now ping the ESR through the tap interface!

[root@rhel01 c5921]# ping -c 5
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=255 time=0.745 ms
64 bytes from icmp_seq=2 ttl=255 time=0.641 ms
64 bytes from icmp_seq=3 ttl=255 time=0.724 ms
64 bytes from icmp_seq=4 ttl=255 time=0.729 ms
64 bytes from icmp_seq=5 ttl=255 time=0.572 ms

--- ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 95ms
rtt min/avg/max/mdev = 0.572/0.682/0.745/0.067 ms

Now we just need to add a static route that points to the ESR through the Tap interface, or we can get fancy and install FRR to do full routing with the ESR.  I’ll go with the static route this time.

[root@rhel01 ~]# ip route add via
[root@rhel01 ~]# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=255 time=0.698 ms
64 bytes from icmp_seq=2 ttl=255 time=0.754 ms
64 bytes from icmp_seq=3 ttl=255 time=0.727 ms

With all that in place we can go ahead and join our ESR to a DMVPN etc to give our Linux box full access to everything.



Leave a Reply