Juniper, Zero Touch Provisioning, and Raspberry Pi

Recently, a client had the need to replace about 500 switches at multiple remote sites. ZTP came to mind as a possible shortcut to getting this done with the littlest possible work effort. After doing some reading about automation and Junos, I started playing around with python, jinja, and yaml. During this time period, it turned out that a colleague of mine was actually going through the same exercise with one of his clients. Special thanks to Vince Loschiavo, who came up with a way to build device-specific configuration files while also leveraging ZTP. His Git repository can be found here, which goes over all the steps one must take to get things working. I would highly recommend reading through his, and other documentation prior to moving forward, but in a nutshell, here's how ZTP works:

1. By default, when a new Juniper switch's management interface is plugged in, it attempts to go through the ZTP process.
1. It requests an IP address.
2. It attempts to download and upgrade the Junos OS.
3. It attempts to download and install a configuration.

 This process can be manipulated to satisfy more complex requirements. Below is a brief overview:

1. A user scans/enters the MAC address of each switch's management interface into a CSV file, along with other device-specific data (i.e. host name, IP addressing, VLANs, etc.).
2. This CSV file is then placed on a server somewhere on the network where python, jinja2, apache2, the required Junos OS version, and a DHCP server are installed/loaded.
3. Two jinja templates exist on this server that allow for the generation of device-specific configuration files.
1. A configuration template that contains variables for all data that differs per device.
2. A DHCP reservation template that contains variables for certain device-specific information (i.e MAC address of a switch's management interface).
4. A single python script exists on this server, which when executed, analyzes the CSV file, generates configuration files, as well as creates DHCP reservations for each switch.
5. Upon plugging each switch into the network, the ZTP process is triggered, and switches are deployment-ready in minutes.

This is awesome! However, in my case the customer did not have a management network built, and time constraints would not allow it...

Discussing my constraints with Vince and other team members, we came to the conclusion that a small, portable mini-computer just might work for this type of scenario. Enter Raspberry Pi!

Together we were able to validate that a Raspberry Pi gets the job done. It is super cheap and can be powered via a USB port on a switch. Once everything is loaded and the python script is executed, it is very plug and play. The Raspberry Pi can then be shipped from site to site to perform OS upgrades and provision configuration data.

In effort to make things a little simpler for network engineers that would like to try this out, I have taken a snapshot of my 8GB microSD with all the necessary applications installed and configured. It even has a sample CSV file, and generated samples from the python script. If you would like a copy just let me know. If you would like to build your own, here is what I installed:

• Snappy Ubuntu Core - https://wiki.ubuntu.com/ARM/RaspberryPi
• apache2 - used to transfer files via http
• python - used to generate files
• jinja2 - used to create templates
• isc-dhcp-server - used to serve IP addresses
• openssh-server - used to allow SSH access

 Enjoy!

Panorama, Templates, and Python Scripting

It has been quite some time since my last post. I have been super busy on different customer engagements as of late and its cutting into my posts!

I am a fan of the saying, "Centralize what you can, and only distribute what you must." The Panorama appliance from Palo Alto Networks does a great job of accomplishing this when deployed in environments with many geographically distributed locations. I recently utilized this appliance to deploy over 250 firewalls in about 2 weeks for a customer and found that it made my job a lot easier from a configuration perspective, but also made the security administrator's job much easier moving forward.

If you know anything about Panorama, you will know that Device Groups cover Policy and Objects information, while Templates cover Network and Device information (overview). For this particular deployment, there was one Device Group for the majority of the locations due to policies being standardized by the organization. Templates were a different story because IP addressing and other device-specific information will obviously vary from site to site. Palo Alto has a KB article that shows how to clone entire templates, which is what I utilized when originally deploying the firewalls. However, there isn't a very clear way to make changes to specific settings within all templates once the firewalls are deployed... For example, lets say I need to add the same syslog server profile to every firewall. Do I really have to enter the information into each template???

Maybe there is a better way to do this that I don't know about (UPDATE: there are now Template Stacks FYI), but I was able to utilize a Python, Jinja2, and csv file containing template names for each location. I wanted to share what I did because I believe it will help many other admins out there that run into issues with deployment. Please note that I am no way an expert at Python. I've merely utilized the automation tools that I've learned from previous endeavors and colleagues to apply them to this specific use case.

Step 1:

Using your favorite Linux flavor, install Python and Jinja2. Create a directory somewhere on the machine for the files you will be creating.

Step 2:

Create a csv file called "device_data.csv" with a column heading (i.e. location). The corresponding values should be equal to the template names in Panorama.

location
California
Texas
New York
Florida
Germany
Hong Kong
Beijing

Step 3:

Create a jinja2 file called "panorama_conf_template.j2" with the configuration parameters and include your variable (i.e. location).

set template {{ location }} config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER

Step 4:

Create your python script called "make_config.py" so that upon execution it will use the information from your jinja and csv files to create your config.

#!/usr/bin/python

# Import the necessary modules
import csv
import sys
import glob
import os
from jinja2 import Template


##################################################
# Begin: User defined variables
##################################################
# Path to configs
conf_path=""
# File name of your csv file
csv_filename="device_data.csv"

##################################################
# End: User defined variables
##################################################

# Read device_data.csv from the current directory
# csv.DictReader reads the first row as a header row and stores the column headings as keys
device_data = csv.DictReader(open(csv_filename))

# Loops through the device_data csv so we can perform actions for each row
for row in device_data:
    # Stores the contents of each "cell" as the value for the column header
    # key : value pair

    # The below example will print the value of the location column for the current row.
    # print row["location"]
    data = row
   
    # creates a filename variable for the template configuration based on the store in the CSV
    conffilename =  conf_path + row["location"] + ".txt";

    # Open the store config Jinja2 template file.
    with open("panorama_conf_template.j2") as t_fh:
        t_format = t_fh.read()

    # Set it up as a template
    template = Template(t_format)

    # Create the .txt file
    fout = open(conffilename, 'w')
    print fout

    # Write the conf file with the template and data from the current row
    # Performs a "search and replace"
    fout.write((template.render(data)))
    fout.close()

    # Print to SDOUT   
    #print (template2.render(data))

# finds all files ending in .txt and combines them into a single config file
read_files = glob.glob("*.txt")

with open("config_script.conf", "wb") as outfile:
    for f in read_files:
        with open(f, "rb") as infile:
            outfile.write(infile.read())

# deletes all .txt files as they are not needed once they are combined into the config file
filelist = glob.glob("*.txt")
for f in filelist:
    os.remove(f)

print "Good bye!"

Result:

Executing the script will result in the creation of the "config_script.conf" file which contains the following configuration data that can be entered (copy/paste) via SSH in Panorama:

set template California config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template Texas config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template New York config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template Florida config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template Germany config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template Hong Kong config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER
set template Beijing config shared log-settings syslog syslog-profile server syslog-1 transport UDP port 514 format BSD server 1.1.1.1 facility LOG_USER

Hopefully this is helpful to other engineers that want to save some time. This could be applied to really any configuration data that you are trying to automate.