Taking a Rest(ful API)

Standard

Yesterday we did a quick discussion on what python is all about, I closed things out by showing a simple script that runs a show command and prints the results. Running a script that pushes commands and looks for certain output is called “screen scraping” since python doesn’t know or care what device it connects to and what the output should look like so it just “scraps” the results and lets you deal with it.Yesterday we did a quick discussion on what python is all about, I closed things out by showing a simple script that runs a show command and prints the results. Running a script that pushes commands and looks for certain output is called “screen scraping” since python doesn’t know or care what device it connects to and what the output should look like so it just “scraps” the results and lets you deal with it.


A better way to work with devices is by taking advantage of their APIs to more directly work with devices.

But Why?

Lets look at an screen scraping example where I want to collect the neighbor IPs from CDP output on this Nexus9k.

Here is what we are working with:

  NX9K01(config)# show cdp neighbors 
 Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
 S - Switch, H - Host, I - IGMP, r - Repeater,
 V - VoIP-Phone, D - Remotely-Managed-Device,
 s - Supports-STP-Dispute
 
 Device-ID Local Intrfce Hldtme Capability Platform Port ID
 R01.testlab.com Eth1/1 170 R B Gig0/2 
 R02.testlab.com Eth1/2 153 R B Gig0/2 
 R03.testlab.com Eth1/3 129 R B Gig0/2 
 R04.testlab.com Eth1/4 127 R B Gig0/2 
 R05.testlab.com Eth1/5 133 R B Gig0/2 
 R06.testlab.com Eth1/6 154 R B Gig0/2

We can see the IP information by looking at the show cdp neighbor detail section

  NX9K01(config)# show cdp neighbors details 
 ----------------------------------------
 Device ID:R01.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.1.11.1
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/1, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 148 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.1.11.1
 ----------------------------------------
 Device ID:R02.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.2.11.2
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/2, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 128 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.2.11.2
 ----------------------------------------
 Device ID:R03.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.3.11.3
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/3, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 142 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.3.11.3
 ----------------------------------------
 Device ID:R04.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.4.11.4
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/4, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 157 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.4.11.4
 ----------------------------------------
 Device ID:R05.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.5.11.5
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/5, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 144 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.5.11.5
 ----------------------------------------
 Device ID:R06.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.6.11.6
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/6, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 154 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.6.11.6

Let’s hop into the python shell on the Nexus and see if we can get each of the IPs, we will import the Cisco cli module and re for regex stuff.

  NX9K01(config)# python
 Python 2.7.5 (default, Nov 5 2016, 04:39:52) 
 [GCC 4.6.3] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> from cli import *
 >>> 
 >>> import re

First we will make a pretty common regex function that should pick up most IP addresses.

def get_ip (input):
      return(re.findall(r'(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)', input))

Then we need to capture the CDP output and store it in a variable

 >>> cdpdetail = cli('show cdp neighbor detail')
 >>> 
 >>> print cdpdetail
 ----------------------------------------
 Device ID:R01.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.1.11.1
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/1, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 159 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.1.11.1
 ----------------------------------------
 Device ID:R02.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.2.11.2
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/2, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 139 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.2.11.2
 ----------------------------------------
 Device ID:R03.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.3.11.3
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/3, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 152 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.3.11.3
 ----------------------------------------
 Device ID:R04.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.4.11.4
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/4, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 172 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.4.11.4
 ----------------------------------------
 Device ID:R05.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.5.11.5
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/5, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 150 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.5.11.5
 ----------------------------------------
 Device ID:R06.testlab.com
 
 Interface address(es):
 IPv4 Address: 10.6.11.6
 Platform: Cisco , Capabilities: Router Source-Route-Bridge 
 Interface: Ethernet1/6, Port ID (outgoing port): GigabitEthernet0/2
 Holdtime: 173 sec
 
 Version:
 Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)
 Technical Support: http://www.cisco.com/techsupport
 Copyright (c) 1986-2016 by Cisco Systems, Inc.
 Compiled Tue 22-Mar-16 16:19 by prod_rel_team
 
 Advertisement Version: 2
 Duplex: full
 Mgmt address(es):
 IPv4 Address: 10.6.11.6

Then we will run the cdp variable through our regex function and see what we get

  >>> get_ip(cdpdetail) 
 ['10.1.11.1', '1986-2016', '10.1.11.1', '10.2.11.2', '1986-2016', '10.2.11.2', '10.3.11.3', '1986-2016', 
 '10.3.11.3', '10.4.11.4', '1986-2016', '10.4.11.4', '10.5.11.5', '1986-2016', '10.5.11.5', '10.6.11.6', '1986-
 2016', '10.6.11.6']

Cool! Though we have two problems here, one is that our regex is picking up the Copyright 1986-2016 and the other is that it is the mgmt IP is being picked up and is duplicating our list.

So how can we fix this? The simple solution for the Copyright is simply to exclude it from the output to begin with.

 >>> cdpdetail = cli('show cdp neighbor detail | exclude Copyright')
 >>> get_ip(cdpdetail)
 ['10.1.11.1', '10.1.11.1', '10.2.11.2', '10.2.11.2', '10.3.11.3', '10.3.11.3', '10.4.11.4', '10.4.11.4', '10.5.11.5', 
 '10.5.11.5', '10.6.11.6', '10.6.11.6']

Awesome! Now we just have the duplicate IP issue. Can we just do another exclude command? Well it would be difficult because the Mgmt address section and it’s IPv4 Address are on two different lines so we would need to use something like the bash shell to try to easily sort that out.

Another solution

 >>> cdpsplit = cdpdetail.split()
 >>> 
 >>> cdpoutput = " ".join(sorted(set(cdpsplit), key=cdpsplit.index))
 >>> 
 >>> get_ip(cdpoutput) 
 ['10.1.11.1', '10.2.11.2', '10.3.11.3', '10.4.11.4', '10.5.11.5', '10.6.11.6']
 >>>

Now we have what we want and I can use the IP list to do whatever I wanted to do with it. But we can see this simple example took some experimentation and adjusting to make the output match how we want the data.

Restful API

Network devices typically give you access to their API through either REST API which uses HTTP to communicate or Netconf which works through SSH. We’ll play around with Rest API today.

Cisco has been working on improving their automation capabilities as part of their DNA roadmap, so modern Cisco devices support restful api and netconf though they are still playing catch up with Juniper and Arista a bit.

To enable Restful API we will make a user and give it privilege 15.

CSR01(config)#username restuser privilege 15 secret meowcat

Then we enable the csr_mgmt virtual-service (for CSRs anyway), we can either have the API use an interface IP or we can give it a separate one if we want. For now I’ll just use the shared IP option.

CSR01(config)#virtual-service csr_mgmt 
 CSR01(config-virt-serv)#ip shared host-interface g2
 CSR01(config-virt-serv)#activate 
 % Activating virtual-service 'csr_mgmt', this might take a few minutes. Use 'show virtual-service list' for progress.

Then we enable Restful API under remote management.

 CSR01(config)#remote-management 
 CSR01(cfg-remote-mgmt)#restful-api 
 RESTAPI is started
 CSR01#

Now we have the API up we need to interact with it, we can use the curl command on your platform of choice to do basic things, every action needs to be authenticated with a token that expires after a short while so we need to start by getting the token like so

  [root@centos01 ~]# curl -X POST https://10.0.123.1:55443/api/v1/auth/token-services -H 
 "Accept:application/json" -u "restuser:meowcat" -d "" --insecure 
 {"kind": "object#auth-token", "expiry-time": "Mon Aug 21 00:31:46 2017", "token-id": 
 "oFo29wybiwPnU9yw5dCOMiUus2T3KUXNfb5awVwY32Q=", "link": 
 "https://10.0.123.1:55443/api/v1/auth/token-services/6276842210"}[root@centos01 ~]#

Whelp that is a bit of a mess! Let’s see if we can use some linux wizardry to make it a bit more readable, first I’ll pipe it into python so we can use the json-tool to make it a bit better to read.

 [root@centos01 ~]# curl -X POST https://10.0.123.1:55443/api/v1/auth/token-services -H 
 "Accept:application/json" -u "restuser:meowcat" -d "" --insecure | python -mjson.tool
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
 100 206 100 206 0 0 124 0 0:00:01 0:00:01 --:--:-- 124
 {
 "expiry-time": "Mon Aug 21 00:31:46 2017",
 "kind": "object#auth-token",
 "link": "https://10.0.123.1:55443/api/v1/auth/token-services/6276842210",
 "token-id": "oFo29wybiwPnU9yw5dCOMiUus2T3KUXNfb5awVwY32Q="
 }

That is a little better! Now we are happy with our token output we can use it to query the router about the static routes that are configured

 [root@centos01 ~]# curl -sk -X GET -H "X-Auth-Token:y9wkJA4JKkIT29qhYSJHeO3+Nj60dEPy7eZuZqPbgLQ=" 
 \
 > https://10.0.123.1:55443/api/v1/routing-svc/static-routes | python -mjson.tool

REST API has four methods that we are interested in for network stuff, it can do:

  • GET: This is for retrieving information
  • POST: This is for creating new things such as adding a static route
  • PUT: This is for updating /replacing configuration
  • DELETE: This removes things.

We are doing a GET right now which is the -X GET in our curl command. We are also telling curl that it is json format and passing the token along with URL of the feature we want. Lastly we are piping it into python so it is nice and pretty. The difference between this and what we did above with CDP is that the output is always in a predictable format that we can easily work with.

 {
 "items": [
 {
 "admin-distance": 1,
 "destination-network": "0.0.0.0/0",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.1",
 "outgoing-interface": ""
 },
 {
 "admin-distance": 1,
 "destination-network": "100.100.100.0/24",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.100",
 "outgoing-interface": ""
 },
 {
 "admin-distance": 1,
 "destination-network": "200.200.200.0/24",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.200",
 "outgoing-interface": ""
 }
 ],
 "kind": "collection#static-route"
 }

Let’s see if we can add a static route with this method.

  [root@centos01 ~]# curl -sk -H "Accept:application/json" -H "content-type:application/json" -H "X-Auth-Token:1a2o5QgHvYAMqNbzf4nbHQ0cffhCnqZeY4pde7avqk8=" \
 > -X POST https://10.0.123.1:55443/api/v1/routing-svc/static-routes \
 > -d '{ "destination-network":"111.111.111.0/24","next-hop-router":"10.10.13.222" }'

Now if look at the router we have a shiny new static route!

 CSR01(config)#do sh run | in ip route 
 ip route 0.0.0.0 0.0.0.0 10.10.13.1
 ip route 100.100.100.0 255.255.255.0 10.10.13.100
 ip route 111.111.111.0 255.255.255.0 10.10.13.222
 ip route 200.200.200.0 255.255.255.0 10.10.13.200

So this is fancy but we need to figure out how to deal with the authentication token in a more useful fashion, one so lets see what we can do.We can stick with the linux side of things and do some more cli magic

  [root@centos01 ~]# curl -X POST https://10.0.123.1:55443/api/v1/auth/token-services -H "Accept:application/json" -u "restuser:meowcat" -d "" --insecure \
 > | python -mjson.tool | grep token-id | tr -d [:space:] | tr -d "\"," | cut -d ":" -f 2 > /var/tmp/ciscoauth
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
 100 206 100 206 0 0 186 0 0:00:01 0:00:01 --:--:-- 186
 1a2o5QgHvYAMqNbzf4nbHQ0cffhCnqZeY4pde7avqk8=
 
  [root@centos01 ~]# cat /var/tmp/ciscoauth 
 1a2o5QgHvYAMqNbzf4nbHQ0cffhCnqZeY4pde7avqk8=

We can shorten our commands a little bit by using backticks

 [root@centos01 ~]# curl -sk -X GET -H "X-Auth-Token:`cat /var/tmp/ciscoauth`" https://10.0.123.1:55443/api/v1/routing-svc/static-routes | python -mjson.tool 
 {
 "items": [
 {
 "admin-distance": 1,
 "destination-network": "0.0.0.0/0",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.1",
 "outgoing-interface": ""
 },
 {
 "admin-distance": 1,
 "destination-network": "100.100.100.0/24",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.100",
 "outgoing-interface": ""
 },
 {
 "admin-distance": 1,
 "destination-network": "111.111.111.0/24",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.222",
 "outgoing-interface": ""
 },
 {
 "admin-distance": 1,
 "destination-network": "200.200.200.0/24",
 "kind": "object#static-route",
 "next-hop-router": "10.10.13.200",
 "outgoing-interface": ""
 }
 ],
 "kind": "collection#static-route"
 }

Of course this isn’t the best way to handle it, instead we can make a python script that uses the request module to take the place of curl.

I won’t go into the details but here is the end result, this this we can reliably use restapi in our scripts to pull information or to configure the device.

 [root@centos01 ~]# python ciscorest.py --device 10.0.123.1 --user restuser --resource /routing-svc/static-routes
 Password: 
 /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
 InsecureRequestWarning)
 {u'items': [{u'admin-distance': 1,
 u'destination-network': u'0.0.0.0/0',
 u'kind': u'object#static-route',
 u'next-hop-router': u'10.10.13.1',
 u'outgoing-interface': u''},
 {u'admin-distance': 1,
 u'destination-network': u'100.100.100.0/24',
 u'kind': u'object#static-route',
 u'next-hop-router': u'10.10.13.100',
 u'outgoing-interface': u''},
 {u'admin-distance': 1,
 u'destination-network': u'111.111.111.0/24',
 u'kind': u'object#static-route',
 u'next-hop-router': u'10.10.13.222',
 u'outgoing-interface': u''},
 {u'admin-distance': 1,
 u'destination-network': u'200.200.200.0/24',
 u'kind': u'object#static-route',
 u'next-hop-router': u'10.10.13.200',
 u'outgoing-interface': u''}],
 u'kind': u'collection#static-route'}

Most network vendors worth a damn have restapi in most of their products nowadays, by way of example here is a Meraki one I put together to collect all the vlans in my network

 root@Home01:~# python meraki-vlan.py
 Prodnet, LAN, 1, 10.10.2.0/24
 Prodnet, Server, 13, 10.10.13.0/24
 Prodnet, Guest, 15, 10.10.15.0/24
 Prodnet, Wireless, 16, 10.10.16.0/24
 Prodnet, MEOWCAT-AP, 100, 10.10.100.0/24
 Prodnet, MEOWCAT-CORP, 101, 10.10.101.0/24
 Prodnet, MEOWCAT-GUEST, 102, 10.10.102.0/24
 Prodnet, MEOWCAT-BYOD, 103, 10.10.103.0/24
 Prodnet, Test-VLAN, 111, 172.16.0.0/24

 

SO CAN I USE RESTFUL API FOR EVERYTHING?

No, because it is an API we can only access the features the vendor makes available to us which can change from platform to platform and version to version. For example we started this by looking at CDP but there is no CDP access in a CSR’s API at the moment so we  still need to use screen scraping or a mix of other solutions. Most vendors will release a restful API reference guide for their platforms you can see what can be done natively.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s