When you think of networks, you probably think of physical things, routers, switches, firewalls,
and that kind of stuff. But increasingly, network engineers are managing massive networks that are
better managed via software than by admin applications. On this episode, you'll meet
David Barroso, who created Napalm, a vendor-neutral, cross-platform, open-source project that provides
a unified API to network devices. This is Talk Python to Me, episode 128, recorded July 17, 2017.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm,
and follow the show on Twitter via at talkpython. This episode is brought to you by Linode and
Rollbar. That's right. Welcome to Linode, who has joined Talk Python to Me as a major sponsor.
Be sure to check out what both of them are offering during their segments. It really helps support the
show. David, welcome to Talk Python. Thank you for having me here. It's been a long time coming that
we've talked about Python, all sorts of things, but no networking really, right? And so you're here to
help us fix that problem and talk about Python and network programming.
Yeah, here I am. Happy to follow.
Yeah, it's going to be great. So we're going to talk about your project, Napalm, and you're doing a bunch of
exciting stuff there. It seems like it's still going strong. It's been around for a few years. But before we get
into all that, let's start with your story. How did you get into programming, networking, Python, that sort of stuff?
Well, my background is network engineering, but I haven't always been a network engineer. I started like many, many years ago, more like systems engineer. So just dealing with Linux, Windows, Apache, Nginx, whatever. Then I was just doing, well, anything, Bash, Perl, Python, whatever you had to use to the job.
Then at some point, I had to move some data I had on a wiki page into something I could actually query. So I was just investigating what the best tool for that would be. And I came into some page that was actually talking about Django.
And I thought that was kind of like the perfect fit. And so I could actually say that Django was what really introduced me into Python and drove me to learn more about it. And then just like regular evolution, just career path, I just landed more into the networking side of things, which made me more focused on the network stuff, rather than on systems or actual Python or something.
But Python has been always there just trying to help me get the job done somehow.
I see. That's really cool. So you've done a lot of work with these systems engineering, with configuring services like Nginx and things like that. And just like Python has always played an important role in sort of making that work.
Yeah, I remember using CFEngine like more than a decade ago and doing a lot of Perl. Like back in the day, it was mostly Perl. But likely, it's not that much Perl nowadays. It's Python.
Yeah, I definitely think there's less Perl in the world than there used to be. That's good. And so what are you doing day to day? Like you're doing network stuff for your main job, right?
Yeah, so I'm a network systems engineer at Fastly. I spend most of my day both automating the network and building a control plane where we can actually integrate with the application.
So it's not the network is not only automated, but it's somehow directed by the application. The application can say like, oh, yeah, I want to send this packet via this path.
I want to send this other packet via this other path. So I'm working on these two areas.
Yeah, and that's pretty major stuff. I'm not sure everyone knows what Fastly is.
So maybe give them the quick elevator pitch.
Well, we are a CDN. The cool thing about Fastly is that we expose all of our services and configurations by an API.
So you can actually integrate with us. So it's not like the traditional CDN where you just send your objects and they catch them until you can send a new object and then they catch that as well.
So it's a matter of, yeah, I want to be able to respond in this way if the HTTP headers are like this. And you can actually do that in real time just by configuring it via the API.
That's kind of cool. You're a web developer. You should check it out.
Yeah, that sounds really cool. So you guys probably handle a little bit of network traffic, huh?
Yeah, yeah, yeah. We do a lot. Okay, I don't recall the numbers exactly, but I think it's about 5 million requests per second and a few terabits of bandwidth.
So yeah, it's quite a lot of traffic.
Yeah, that's crazy. You think about like a number of requests per second for like a RESTful API or something like that.
That might just be a couple of bytes response of JSON or something. But typically CDNs serve images and videos and files, right?
Like that's a crazy amount of traffic.
Yeah, we do all kind of traffic like video, music, images, JavaScript, even API. We have a few APIs behind us.
And that's because of the integration that I was mentioning before.
Yeah, yeah.
But yeah, that's not really my expertise.
No, no, but it's all built upon the network, which is super cool.
So traditionally people maybe configured networks by logging into like a Cisco router or switch or something like that.
And then, you know, maybe using the CLI there or using some tool.
But working with networks these days is becoming more and more a programming type of experience, right?
Yeah, I think that the network is following the same evolution that the system world experienced like 10, 15 years ago.
Like before you would just like edit your Apache.com from the virtual host files and all that kind of by hand and like CF engine came along, Puppet.
So the network is kind of now having that transition nowadays.
Like people was used to go through the console, SSH, or maybe some tools used to use SNMP to try to configure certain things.
But yeah, nowadays it's like tools like Puppet, Ansible, Solstack, Stagstock are starting to come to the network as well, which is interesting and cool.
Yeah.
Finally.
Yeah, that's great.
That sounds really exciting.
How's that affecting the whole ecosystem of network engineers?
Are people excited about this change?
Are they like, oh, my experience is no longer relevant.
It's not fun anymore.
Or like what's the general feel, do you think?
You will find both.
Like some people is like, oh, yeah, this sounds cool, a different way of doing things.
And I can just automate this and focus on more interesting stuff.
While other people is a bit more scared because, I don't know, maybe they are older engineers that they don't really have the time to pick up the new tools or the new programming languages.
So they might be a bit more averse to the change.
I think it's, yeah, you will find all sorts of people having all sorts of opinions.
Sure, of course.
So one of the terms that I hear frequently around the space is software-defined networks or SDNs.
What's the SDN?
That's mostly marketing.
Like vendors keep pitching SDN, but SDN doesn't really mean anything.
Like the original term was coined by some people at Stanford that were working with OpenFlow, which was like a super low-level API to try to program the switcher.
But that didn't translate well into hardware.
So it never took off.
OBS in the Linux stack uses OpenFlow, for example, for some things.
But that's software-based, which is fine because you don't have the hardware limitations to install all those flows.
But yeah, in the hardware world, like the real SDN never took off because of hardware limitation.
And now it's just used as a marketing term by, yeah, by many, many vendors.
Okay, of course we have software-defined networks now.
So there is this whole level of automation, though.
There is the ability to run scripts against like Cisco routers or various other types of routers.
And that's kind of where Napalm came in, right?
Previously, there was, for each vendor, each type of device you're going to work with, it was like a totally unique way of working with it, right?
Yeah, like each vendor has its own API.
Like if you go, you're logging into an iOS device by Cisco, for example, and you start typing commands.
The commands are going to be completely different from a Juniper device, for example.
And the CLIs, they're completely different.
They return different things to different commands.
They may have even different interfaces to actually connect to them programmatically.
Like, for example, iOS until very recently didn't have anything.
So you were using things like NetMiko, which is a Python library that works on top of Paramiko.
So it's basically just SSH expect-based, kind of.
Like Juniper used to have NetConf.
Well, it still has NetConf, which is actually a network protocol to configure devices.
And some other devices may have like a REST API.
Like there are just many ways.
So when you have to do something as simple as, I want to connect to my network and I want to retrieve the IP addresses that you have configured on the interface,
you have to start just like adding a lot of boilerplate.
Like, okay, if device equals to iOS, do this.
If device equals to this other vendor, just do this other thing.
So yeah, it used to be kind of a nightmare.
That's pretty much what I tried to solve with NAPALM.
Okay.
So NAPALM is an acronym, right?
What's it stand for?
Acronym is actually, it has actually an interesting story.
It was reversed, engineer.
Like I first came up with the acronym and then I tried to find some meaning to it.
Because, I mean, before it was all these if else statements that was mentioned, like if vendor this, if vendor that, right?
And I was writing sunscreen.
I was super pissed.
Like, I mean, this is just stupid.
This is just insane.
So, I mean, I want just to leave this on fire.
So the name came out of that and then I started just thinking, like, okay, now let's try to make some meaning out of it.
So, yeah.
So I came up with this network automation and programmability abstraction layer with multi-vendor support.
But, yeah, the name was actually reversed in here.
Yeah, that's pretty cool.
I mean, you have the important letters.
You got the N, the A, the P, and then you got to get creative about the ALM, right?
Yeah.
But, yeah, I wanted something with fire, basically, because that's actually my thing at the moment.
You're like, all right, we got to burn this down.
This is, like, messed up, right?
Yeah.
I've heard it.
Your co-creator of this library, was it, is her name Elsa?
I might be remembering.
Elisa Jasinska.
Elisa, Elisa.
That's right.
Yes.
You guys started this together, right?
Yeah.
So I started working on this when I was at Spotify.
And then I was just talking with her after a conference.
And she was like, oh, that's super cool.
I actually have exactly the same problem.
So then I just went back to my manager.
Like, okay, yeah, I've been talking with people.
And it turns out that there is some interest on collaborating on this.
So I just open sourced it.
And I started working with her.
And then other people came in.
And, yeah, now it's actually a pretty big community by now.
Yeah, it's going really strong now.
Yeah.
And I had heard there was even some places where people were using more or less like screen
scraping to get configuration out of some of these tools and some of these systems because
it was just so hard to do, right?
Yeah, there is a lot of screen scraping in devices like iOS, for example, from IA Bay Fisco.
Those devices traditionally didn't have any interface to interact with them.
So it was always like, yeah, SSH to the device.
And then just type this command.
Look for the prompt.
Now let's try to parse this thing.
And there is still a lot of that, to be honest.
Like, we do a lot of screen scraping with Napalm because, yeah, I mean, if they don't provide
the tools, we have to fix it ourselves.
But at least we fix it.
So when we are trying to solve the problem, like meaning what you are trying to solve at
your job, you don't have to care about parsing because we already fixed that problem.
Right.
So maybe something like that happens deep down when you call a function for a particular device,
but nobody knows and nobody cares.
I hope they care so they can act their vendors.
So the vendors return some JSON, XML, I don't care, just some structured data.
So a lot of what you guys do with Napalm is not write these vendor-specific communications
per se, right?
You might find libraries that are out there and just integrate them into Napalm, right?
Yeah.
What we try to do is just provide an abstraction layer, which means that if there is a library
already that can talk with the device, we will just use it.
Like, for example, for Juniper, they maintain a library to work with those devices.
So we just use that one.
That means that we only have to care about transforming data, for example, and providing
like common behaviors.
Like this method behaves in the same way, regardless if it's Juniper or Cisco.
And when you run this other method, the data that you return, it comes back normalized.
Right.
We don't really bother in dealing with the transport and connecting to the device.
Like someone else is caring about that.
We just provide the abstraction.
Yeah, that's really cool.
So you basically just do like an adapter to make it all look the same, the various libraries
that are out there.
What are some of the major libraries, Python libraries that people or that are maybe that
you're built upon?
So the two major ones, I would say that it's the Juniper library I mentioned, which is called
Pi EC, as Pi EZ.
And the other one is NetMiko.
That's quite popular, actually.
It's built on Paramiko.
And it's specifically to be able to interact with network devices because each one has different
idiosyncrasies.
Like one has this, I don't know, like the new line.
It's using this code, which is super old from the early Unix days.
This other one changes the prompt.
So you have to take into account.
So yeah, it builds on top of Paramiko, but it knows how to the tiny, gritty details of
each platform.
Yeah, yeah.
That sounds like something you don't want to write yourself that you would just like to
use.
Yeah, yeah.
So before we get into how it works, what hardware vendors do you support?
The list is getting long, right?
It used to be pretty short at the beginning.
Yeah, the original one was just four vendors because that's actually the ones that I and
Elisa had in our networks.
But nowadays it's, so from Fisco, it's iOS, iOS XR, and NXOS because if I have only one operating
system, you're going to have three.
Right, of course.
Why not?
It's Juniper.
It's Arista.
It's Fortinet.
It's, who else?
Microtech.
There's people working on Brocade as well.
Oh, there's Palo Alto as well.
There are probably a few more, but yeah, I don't recall them right now.
Yeah, of course.
No, but it's pretty comprehensive these days, huh?
Yeah, it's quite extensive.
And people keep working on adding more and more.
So one of the really important building blocks was these libraries that we talked about, the
actual communication with the various devices.
The other one is Ansible.
And you actually added SaltStack and StackStorm as well, right?
Yeah, I mean, we try to provide just a library.
We try not to have opinions on how people should be doing things.
So we try to just provide the basic Python library so people can either integrate with their own
framework or just write their own script.
So the three major tools that we integrate with are Ansible, SaltStack, and StackStorm.
Okay, that's really cool.
So let's just pick Ansible, for example, but it would be similar to others.
Can you maybe describe to me how we would use Napalm and Ansible?
And let's have some kind of goal, right?
Let's suppose you want to set up a load balancer, a couple of web front ends, and maybe a database server,
and some caching tiers, and try to build that network all together so that, you know,
only the right pieces can see each other and things like that.
How would that work?
So the way it would work is that you would have templates for your services, right?
But now with Ansible, you would have another template to map how this new service maps into
the network as well.
That might be like a new VLAN, a new IP address, somewhere else.
So you would just write another template as you did for Nginx and, I don't know, MySQL
or whatever you're using in your network.
And then you just have to, when you just compile this template with the data and you get like
an actual output, the only thing you have to do is tell Napalm to use that configuration.
file and apply it into the device.
And then you can do two things.
Either you apply it straight into the device or you just get a diff that you can actually
peer review or, you know, or something of the like.
That's a pretty nice experience that you can go to your network and reach out to all the
devices, right?
You'd set up your Ansible scripts and, you know, one for the load balancer, one for the
web tier, maybe things like that.
And you'd set them up and you could say, go query them the way they are now.
Figure out the changes that you would push to them and then generate a text diff of what
we're about to do to it, right?
Yeah.
I mean, the only difference between managing like your web server and your network with
Ansible and Napalm is going to be that instead of reloading the service, you're going to be
applying the configuration with Napalm into the device.
But the rest should be exactly the same workflow.
Just the template model, the data coming from your backend or YAML file.
And yeah.
Yeah.
It's really nice.
So that probably makes it pretty easy to store these in like GitHub or somewhere like that
and have like a history of the changes you've applied to your devices.
Yeah.
That's pretty much what everybody is doing.
Just like, yeah.
Just have it as any other template on your GitHub or whatever you're using.
Yeah.
And one thing you guys talked about is you push the entire configuration for the device over
to it.
But somehow the device knows to actually only apply the delta.
How does that work?
So, yeah.
So you can do both things.
You can either apply just a snippet of configuration.
Like if you just want to configure a VLAN, you can choose to configure that single VLAN.
Or you can apply the entire configuration.
The problem with applying snippets of configuration is that the way that devices work, you don't
have a single configuration file where you just like apply it on the device and then reload
the service.
And only what's on that file is what's going to be applied.
Actually, on a network device, you actually tell the network device how to do things.
Would be like going to a Linux box and start configuring the network using IP route commands.
IP route at this interface or IP route at this routing here.
So you're actually telling the device what to do and how to do it.
So if you're merging snippets of configuration, you have to be aware that if you want to remove
a VLAN, for example, you have to tell it remove this VLAN.
It's not good enough to just send the list of VLANs that you want.
That's why I'm trying to...
This is my kind of like a personal opinion.
I like instead just somehow compiling the entire configuration of the device, send it
to the device and tell the device, now reload the service, which is exactly how I would manage
Nginx and my SQL or something.
And just like, here's my config file.
Now reload.
I don't care what you had before.
Just do this.
All right.
This...
I want you to be like this.
Now be that way.
Yeah.
So that works well with most devices, but this feature for Napalm to use it has to be supported
natively.
And most of the devices nowadays, like iOS supports it, Arista supports it, Juniper, like the major
vendors support this, these features.
You're going to actually send a entire config file and reload the service.
It's a great quote, quote on the reload the service.
Yeah.
This portion of Talk Python to me is brought to you by Linode.
Are you looking for bulletproof hosting that is fast, simple, and incredibly affordable?
Look past that bookstore and check out Linode at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Plans start at just $5 a month for a dedicated server with a gig of RAM.
They have 10 data centers across the globe.
So no matter where you are, there's a data center near you.
Whether you want to run your Python web app, host a private Git server, or even a file server,
you'll get native SSDs on all the machines, a 40 gigabit network, 24-7 friendly support,
even on holidays, and a seven-day money-back guarantee.
Want a dedicated server for free for the next four months?
Use the coupon code python17 at talkpython.fm/Linode.
One of the consequences of the way Napalm works is that if somebody goes and manually starts changing the network settings or something, right?
They SSH in there and they make a few changes and then just log out.
And they don't save that anywhere, right?
Napalm will basically wipe that away, won't it?
If you're using this replace operation that I just described, yeah, that's exactly how it would work.
It's just like, yeah, I just created this VLAN by hand and you didn't save it in your template or in your database or something.
Yeah, it's going to be completely wiped out.
If you're using the other method where you're just applying snippets of configuration,
then unless you're explicitly removing the configuration, Napalm won't remove it.
Because Napalm won't even know that the configuration is in there.
Right.
Unless that snippet has exactly to do with that part that was changed, they might not never interact, right?
Yeah.
So some people are combining both manual with automated operations.
So they prefer to use this method where you just tell the device, apply this snippet of configuration.
So it depends a bit on what you are trying to accomplish and what your network looks like.
For some people, the network is so full of snowflakes that actual automating every tiny detail is literally impossible.
Yeah.
Yeah, of course.
I can imagine places like that.
It never was really built all at once.
It just kind of grew this way.
Oh, yeah.
Organic design.
Yes, of course.
Is there a way to go in reverse?
Like, I know I could come up with a bunch of Ansible scripts and I could generate a network out of it.
But is there a way to go to a network and say, now generate me the set of Ansible scripts that would reverse engineer it?
So that you could sort of start from an existing network, come up with these scripts, and then going forward, go in reverse, use Napalm to work with it.
So what I have done to migrate from this snowflake environment to a fully automated environment is I just copy the configuration file.
I mean, I just copy the configuration of the device and just put it into a static file.
And I just deploy that static file with Napalm.
And the only thing I'm doing is just override the configuration that it's already there.
The configuration is just completely static.
And then I start just taking bits of the configuration that is just defined statically into templates.
So that way I can just start migrating bit by bit.
Like, okay, now I have automated the VLANs.
So I can just remove the VLANs from the static configuration file and use a template for that.
The only thing I have to do is append one file into the other with Ansible.
And now I'm going to automate, like, the interface generation.
So I just take that away and automate that bit.
So you can start doing this step by step and just keeping your configuration statically defined somewhere else.
Yeah, that's a pretty good idea.
Just copy it over and go, this is how it is.
And then we're going to pull out the little pieces that change every now and then that we want to automate.
And everything else, we can just forget it, just leave it.
Okay.
Yeah, yeah, it makes a lot of sense.
So Napalm is written entirely in Python, right?
If I can trust GitHub's little measure of what languages are involved in this project, it's like all Python.
Yeah, it's all Python.
Yeah, yeah.
That's really cool.
You don't have too many performance considerations or anything where you might bring in Cython or other types of things, right?
It's more just quick communicate over the network to the device and the device has to do the heavy lifting.
Yeah, I mean, we are talking about something that you're interacting with.
I mean, you're interacting with the device over the network and you're probably using some API or method to interact with the device, which is not extremely fast.
So it's not like you're doing mathematical computation.
So I don't think that, yeah, that performance is a concern.
No one has brought that up at least.
Yeah.
And if it is, it's probably not your performance issue that you can work with, right?
It's like, well, this device is just slow when I do this to it.
Yeah.
I mean, most of the devices, like you type, like, I don't know, show interfaces.
And maybe the device has like 100 interfaces.
So the device takes like one or two seconds to actually return all the data.
So it's like, yeah, we could optimize like 10 milliseconds out of two seconds.
So, okay.
Yeah, of course.
Yeah, exactly.
Don't worry about it.
Yeah, I guess the real trick would be to make sure you're using the latest APIs and libraries from the various vendors.
Like if you were previously doing some weird screen scraping thing and now they have a JSON API, like switch to that, right?
Yeah, that's actually the main challenge we have.
Like trying to use the latest APIs, it usually don't go that well because, yeah, because vendors, I'm sorry, vendors, users still have like networks running operating system from the, I don't know, early 2000.
I mean, I have seen a lot of people with, yeah, my device has a 10 years uptime.
Like maybe you should count that on security advisories more than instead of in years.
That's really interesting.
So I'm paranoid about like security issues and things like that on my servers and on things like Nginx and stuff.
And if there's an update, like that thing is, even if it causes a reboot, it's getting applied because it's on the internet.
That's such a bad thing.
And so to think that a device is up for 10 years, that's probably needs some attention.
Well, the thing is that for most legacy networks, like reloading a device is super hard because you might bring down a lot of systems.
Like because everything uses the network.
It's the very foundation.
It's like taking away memory or the CPU or something, basically.
Exactly.
And most of the networks are actually completely isolated from the internet.
So the only way of actually trying to exploit like some bugs would be by first getting inside your network by just hacking the VPN or the bastion that you might have in your network.
So there is like certain degree of isolation in the network as well.
So upgrading is not.
It's not as critical as like a front end web server that's taking traffic off the internet.
Yeah.
That's one thing to pass the traffic along.
It's another to be the endpoint to like send it into your executing code.
Okay.
Interesting.
So one thing I wanted to ask you about is like there's a lot of cloud computing and other sort of programmable, automatable systems like virtual private servers, you know, things like Linode or DigitalOcean.
And then we've got EC2, we've got Azure and Heroku.
How does this whole network automation story fit with that?
Like, is there any place for say Napalm in AWS or is this really, I have my own data center.
I want to manage that or I would go do something with AWS or.
Yeah.
So the main focus of Napalm is actually like physical infrastructure.
So we don't really have any hooks with any of the, all the cloud companies, like none of them.
Like, yeah, we mostly operate on switches, routers, firewalls, all this kind of thing.
Yeah.
And you, you rarely have that low level of access in any of those places.
Yeah.
I mean, there could be like, maybe, yeah, I'm, I'm not a juicer myself of the public cloud.
So I'm not sure how they work, but there might be some room for like, I don't know, maybe integrating with how, with the firewall, their firewalls and load balancers.
But yeah, I don't know.
I haven't heard of anyone even considering something like this.
Sure.
I mean, there are APIs for the load balancers and APIs for the firewalls, but yeah, I'm not sure.
I'm not sure if it makes sense, but maybe, maybe it makes sense somehow.
Right.
But, but maybe not the same.
Right.
Maybe say, I want to connect these three machines and I want to have this in a virtual private network and whatnot.
Like that possibly could be a thing, but I'm not sure if it really makes sense to mix these tools.
I will say if someone comes up with something after hearing this podcast.
Yeah, exactly.
Maybe you'll get a pull request.
What about like Dockers?
If I'm a Docker, if I'm using Docker or like, I've got a bunch of them running together managed like by Kubernetes.
But let's say that's on my own network.
Well, if it's on your network, I'm pretty sure that you could somehow orchestrate configuring the network somehow with, with Napalm.
I don't know the details of Kubernetes, but I know people doing exactly that with OpenStack.
So their provisioning system just uses, uses Napalm to orchestrate the physical network with the rest of the, I don't know how it's called, Neutron.
I think it is the network plugin for, for OpenStack.
Interesting.
Yeah.
So what's the story between Napalm and OpenStack?
So OpenStack being like.
I don't know the details, to be honest.
I know that someone I was working on just like they have their, they're just in OpenStack to just deploy their virtual infrastructure.
And one of the pieces is actually configuring the network, right?
So someone had just written up plugin, which I don't know if it was just in production, actually, to just like configuring.
Like, for example, if I'm deploying this service, I need to configure this BGP station in here or this VLAN in there.
So it was just a matter of doing that with Napalm.
Okay.
Yeah.
There probably is some opportunities for some kind of integration there.
But, you know, I don't do enough with OpenStack, honestly.
I mean, yeah.
All right.
So can you maybe give us like the general workflow of working with Napalm?
So we start breaking down the devices into different services, right?
Like here's a firewall.
Here's a web server.
Things like that, right?
It depends on what you're doing, actually.
Because we try to just provide the API as generic as possible so you can write your own workflow.
So we do a set of methods, for example.
One set of methods just interact with the device configuration.
So you can say like, okay, I want to deploy this configuration.
Or I want to replace the existing configuration with this one.
This is the method that was mentioned before where you just wipe everything out, right?
Okay.
Now I have this configuration loaded.
I want to either discard it, commit it, or just get a diff back.
So these are just basic, primitive, so you can actually build your own workflow.
You can just integrate it with, I don't know, with Jenkins, for example, to stage the change and get a diff back, open a pull request and have someone validate it.
Like, for example, that's a common workflow.
And the other set of methods, it's getting data out of the network.
Like, I want to get the interfaces of my devices.
Or I want to get the BGP neighbors of my network.
Stuff like that.
So you can actually have into your Jenkins job, like after the change has been made.
Just get the data out and verify that the interface that you configure is actually configured as you want it.
So, yeah, we don't really have built-in workflow, but we try to just have the basic primitives so people can write their own workflows and just, yeah, integrate as they want.
Yeah, it's pretty agnostic to the way that the vendors want you to work.
You can come up with your own workflow, and because it'll talk to all the devices the same, that more or less should work, right?
Yeah.
Yeah, for example, I like more the workflow of just generating the entire config and send it to the device.
Because that way I know that what I told the device, it's going to be in there.
Rather than having to tell the device how to do things, like, yeah, I don't know how to do it.
Just get this done.
But other people prefer more like not all sorts of operations where they have like a self-service portal and they just click somewhere like, okay, create this, delete that.
So, yeah, it depends on your preference, I guess.
Yeah.
And probably how you're using your network and how often it changes and whatnot.
So, you mentioned Jenkins.
Like, what's the story with continuous integration?
Like, what are you seeing people do with Napalm and CI?
I would say that the common thing that people does with CI and Napalm is mostly automated deployment.
Like, for example, I just change some data on a YAML file, right?
And that triggers a Jenkins job that just runs an Ansible playbook, for example.
And Ansible playbook connects to the device, loads the new candidate configuration, and retrieves back a div.
And the div just gets just, it's posted into some PR sort of tool.
Could be Garrett or maybe another GitHub repo.
So, that's a common workflow.
Another workflow I have seen, it's a bit weird.
I don't remember who did this.
But we have a framework within Napalm where you can define certain rules.
Like, for example, I want to verify that all my BGP neighbors are sending me at least five prefixes.
We have a framework to describe things like that, for example.
Or I want to check that my interfaces have no errors.
It's just a YAML file where you can just describe certain data and validate it.
So, I've seen some people that when they deploy something on the application, they run this sort of validation tool to verify that the network is working fine.
Like, for example, we start this backup.
We want to check that we are not having interface errors at the same time.
So, I've seen people not connecting completely unrelated pieces and verifying stuff with CI as well.
Okay. And things like, I made a change to the network, but I don't expect the number of endpoints to change.
So, let's make sure that that is still a constant number or something like that, right?
Yeah, it was a kind of weird workflow.
They were trying to explain to me, I was like, okay, that sounds all confusing and cool.
So, I didn't really understand it very well.
Yeah, interesting.
This portion of Talk Python to Me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Relying on users to report errors, digging through log files, trying to debug issues, or getting millions of alerts just flooding your inbox and ruining your day.
With Rollbar's full-stack error monitoring, you get the context, insight, and control you need to find and fix bugs faster.
Adding Rollbar to your Python app is as easy as pip install Rollbar.
You can start tracking production errors and deployments in eight minutes or less.
Are you considering self-hosting tools for security or compliance reasons?
Then you should really check out Rollbar's compliant SaaS option.
Get advanced security features and meet compliance without the hassle of self-hosting, including HIPAA, ISO 27001, Privacy Shield, and more.
They'd love to give you a demo.
Give Rollbar a try today.
Go to talkpython.fm/Rollbar and check them out.
So to test this out, do you actually have to have the devices like a Juniper device or a Cisco device?
Or is there like the equivalent of virtual machines for these network devices?
Most platforms nowadays have a VM equivalent.
So we can use that for testing.
But we rely a lot on mocks.
Like for example, most of the time we, like most of the code is actually just normalizing data.
But just connecting to a device, retrieving the interface information and normalizing it into a common model.
So for those things, that's easy to mock.
You just need the original data, a few use cases, and just test that the parser, normalizer works fine.
So yeah, we can get away a lot with just mocks.
But yeah, there are certain things that you need, a VM or something.
Yeah, that makes sense.
It's like, so run some particular command on some test lab setup and just capture that text and go, now tell it that it gave you this.
Yeah.
With the mock layer in your CI.
Yeah.
The thing that you don't want to mock is the configuration management bit.
Because that's so critical.
If you miss something up in there, like you might be costing an outage.
So that we try to always, always test with VMs.
I want to play it safe there.
Is it stressful to work on this project?
About just to think like people might be running this against huge networks, either your own at Fastly or someone else's and just like, okay, I really have to get this right.
Because it could have real consequences if I mess it up.
No, I don't think it's stressful.
No, I don't think so.
Like the primitives that we provide with Napon to configure the network, it's actually five or six methods.
And those ones, they're written, they're written.
Like you don't really have to deal with them.
So yeah, I mean, it's a bit delicate when you're writing them for the first time.
But then once that's written, it's just a matter of, yeah, just keep testing.
I see what you're saying.
It's kind of low enough that the parts where it may get tricky or not quite right is actually at the person using Napalm, not Napalm itself.
And I mean, obviously, you're integrating into something as low level as your infrastructure.
You have to do your homework and actually do some integration testing yourself.
And make sure that, yeah, the exact version that you're running on the network, it actually works well with Napalm.
Like maybe you have like this super tiny bug that we have on the operating system of the network, I mean, that we haven't encountered before.
So that's just, yeah, basic integration testing, I guess.
Yeah, yeah.
Do you get requests for people to add support for particular devices that you don't have access to?
Oh, yeah, all the time.
And so how do you deal with evaluating those PRs?
For the configuration management part, we always ask to have proof that it's working.
And that could be as simple as he just creates a necessary user for me so I can actually validate the testing person while he's running the test on his machine.
Or I might be running the test on my machine.
But the thing is that if I don't have access to the device, at least the configuration bit, I need to be able to test it the first time myself.
Because then once it's tested and it's working in there, it will probably barely change.
And yeah, and what retrieves data from the device, that's easy.
That's just a matter of mocking the output.
So yeah, that's not really a worry.
Yeah, it just seems like when you work with all these different devices, it's tricky because you probably don't have access to every single variation of them, right?
Yeah, yeah, correct.
Yeah, so we just try to at least, at least the first time to validate, not take only the word of the contributor.
Could you do like screen sharing and just like watch them?
Yeah, I mean, anything that works for them.
I mean, if they don't want to share the screen, I mean, and they can just give me access to the device, I can just pull their branch and test it myself.
I mean, that's fine.
Yeah, cool.
So I just wanted to highlight a video presentation that you did at Nanog.
What was the first?
I might be getting it wrong.
Yeah, Nanog.
I thought that was really helpful.
It's a little bit dated, right?
That was, I think, from 2015, but it still shows a lot of the workflow and how people might do it.
So if they're really interested, I'll be sure to put that video in the show notes.
People can check it out.
Yeah, that's actually the video that we recorded when we went live with the project.
Like, here we are.
Yeah, here you are a couple years later, and I remember there was just the few devices you guys supported, and now there's a whole bunch.
Yeah.
Yeah, so do you want to tell us about some of the major deployments or users of Napalm?
So I don't really have an official list.
I never thought about that.
But I was actually going through the contributors and just trying to see where they work.
So this list I'm going to mention now is based on trying to check on where people work.
I don't know if they're actually using it on their networks.
It might be that the contributor is just, I don't know, doing it on its own spare time.
I don't know.
Just disclaimer here.
But I don't know.
I've seen contributions from people working at places like some trading firm that, I don't know,
friendly is called IMC, Nike, Netflix, obviously Spotify, which is where everything started, like eBay.
Yeah, you were at Spotify when you started working on this, right?
Yeah, right, right.
Yeah, I was working at Spotify when I started the project.
eBay is a big user of Python, so that makes a lot of sense.
Yeah, I know.
I have seen some contributions from people working there.
Again, I'm talking based on where I have seen they work.
I don't really know what they're doing.
There are a few IXPs like Lynx and Linux, which if you're a network engineer, you probably know who they are.
And Fastly, we use it as well.
And another big contributor is also Cloudflare, which is another CDN.
But yeah, there are a few contributors.
They seem to be working in interesting places.
Yeah, that's great.
Are you looking for more contributors?
People always ask me if there are good places to get started in open source that are kind of new.
They want to check this out.
So are you looking for competitors and what kind, I guess?
Anyone could help, actually.
There is a lot of work to do, both in, I don't know,
like something as basic as documentation.
That's kind of like even if you don't know Python or you're starting with Python, you can always help there.
And then also something you have to bear in mind is that this is mostly built by network engineers,
which means that it might not be the best Python.
So even if you don't know anything about networking, but you just want to help a project by just reviewing the code
and proposing improvements or something like that,
like, yeah, Napalm would be a great fit for that because, yeah, we are mostly network engineers.
But, yeah, we don't know what we are doing.
We're just trying to solve problems.
Hey, we made it work.
It's great.
So I asked you earlier about how you deal with the hardware that you don't know about.
But if somebody's got some new piece of hardware and they want to integrate it into Napalm,
what are all the steps that they got to go through for that to work?
So if you're a vendor, which actually has happened already, it's the same workflow as if you were just a regular contributor.
Like you just come to us proposing the idea.
That's just so we know that you're working on that because someone might already be working on us.
It's trying to avoid duplicating effort.
But I don't know.
We don't really have a formal structure.
Like there are certain ways that you have to integrate with the project.
But that's just part of how the project is kind of designed.
It's not really like formalities.
Like, oh, yeah, you have to support at least these three methods.
And if they're not supported, like it's a completely no-go.
Like, no, we don't really have strict rules like that.
I mean, if it's useful, it's useful.
Even if it's just like a couple of methods, someone else might just pick it up and improve it.
Right.
Just getting it started might be a big deal.
Nice.
Okay.
That's great.
So what's the feature, Nepal?
Like, what are you guys planning to add?
Or where is it going?
Now we just launched a new library where we tried to normalize syslog events, for example.
This is completely new.
The first release was launched like maybe a couple of weeks ago.
So because this, yeah, in the same way that every device has its own CLI and output from different calls.
Yeah.
In the same way that that happens, they also send like different syslog events with different data and everything.
So now we are trying to be able to normalize that.
So you can just start this service, use it as a syslog endpoint and just normalize the data and send it somewhere else, like whatever, log stash or whatever you're doing.
And I don't know.
We're trying now to integrate as well with OpenConfig.
OpenConfig is an industry effort to normalize the data that devices return, for example.
So when we started, OpenConfig didn't exist yet.
So we designed our own data models, to call them in some way.
And now we are trying to start integrating with this data models.
Yeah.
So if there's going to be a standard around all the data, then you guys definitely want to be part of it.
Yeah.
So now we're trying to integrate with that instead of just completely designing everything ourselves.
Like, yeah, if someone is solving that problem for me, why not?
Yeah.
And if people want to get in touch, you have both a Slack channel and a mailing list, right?
Yeah.
We're in Slack.
The Slack organization is called Network2Code.
We can probably add a link on the show notes to the page to register.
And there is also a mailing list, although it's not very active, to be honest.
Most people prefer Slack nowadays.
Looks like email is slowly dying.
And we actually launched today a webpage to keep all the news posted there and links to the Slack channel and everything.
So that's actually a good...
Wow, that's great.
Just today, huh?
Yeah, yeah.
Just like a couple of hours ago.
Nice.
Be sure to put that in the show notes that we're sharing so we can make sure everyone has that.
Yeah, because before we had just the GitHub.
But GitHub is not great if you want to keep people up to date into like, oh, yeah, we just launched this.
Or we have fixed this.
So make sure that you are updating your requirements and stuff like that.
So, yeah, we built this web too.
Yeah, nice.
Yeah.
And what's the Python 3 versus Python 2 story?
Do you support both or one or the other?
Yeah, we actually support both.
Kirk, which is one of the main contributors and happens to be also the creator of NetMiko, the tool that I mentioned before.
He really likes Python 3.
So he worked very, very hard.
And he got all the code working there.
That's great.
Very nice.
So you don't have to worry in a couple of years when Python 2 gets phased out.
That's good.
All right.
So Napalm seems like a really cool project.
If you're doing network automation, it seems really excellent.
So I wanted to ask you why I had you here.
A few more general questions just about networking type of things.
So what is the future for network?
What does it look like for network engineers, do you think?
More stuff like what you're doing with Napalm or is it changing or what do you think?
Million dollar question, I guess.
Everybody keeps asking that.
I don't know.
I mean, the way I see it is somehow the network engineer will transition into an SRE type of role.
Where you're not this CLI jockey that knows all in and outs of a network device, all the knobs.
Networks are becoming simpler and simpler because people keep just building bigger and bigger.
So there is less room for snowflakes.
So I see more like network engineers transitioning into an SRE type of role.
Okay.
And do you think things like this data standard you talked about and Napalm itself,
where you don't actually have to know the details as specifically about the devices?
Do you think it's more automation programmability, less specific knowledge about the individual devices?
Well, the thing is that networks traditionally were full of knobs.
Like you go and you configure OSPF and then you enable this super obscure option in here.
Then this other knob in that other place.
Like, yeah, when your network is not so small, you keep just trying to optimize like everything.
But now when you start building things at a bigger scale, like there is not that much room for those kind of thing.
Like you need to build something fast.
Regardless of if you're doing it manually or in an automated way, you need to start standardizing things.
So my guess is that eventually things will happen.
I mean, it's the same that happened with other services.
Like before you had like, I don't know, 20 instances of MySQL and each one was configured differently, trying to optimize every detail.
Now that you have like 100 MySQL, I don't look the same.
I didn't really care about anything.
Yeah, whatever.
Yeah, it's more about trying to manage 100 things than it is about tweaking little bits.
Because if you can manage 100, you can just add a few more and get better performance or whatever, right?
Yeah, the cloud is cheap, right?
So you just have more compute.
That's right.
The other thing I wanted to ask you about was IoT devices, right?
So we saw recently one of the, it was the biggest denial of service attack.
And the denial of service attack actually came from a bunch of hacked webcams or something like this, a bunch of these IoT things.
So it seems to me like having all these little crappy, unsecured, non-updated devices on networks is a problem.
It's going to be more and more of a problem as things go.
Like from a network perspective, like what can be done about these things?
The problem is where are those IoT devices located?
Are they located like behind a corporate network or are they located behind like someone's home?
Probably someone's home connected to a crappy firewall, a crappy NAT router that is like hasn't been updated in five years.
Yeah, I mean, then you have a problem because there is no actual way of automating that box that your provider gave you and didn't even provide you the user and passwords, right?
Yeah, I guess there is a little hope in there.
Not a whole lot of hope.
Do you have any hope for it?
Do you think like, what do you think is going to happen?
I don't know.
I hope that at some point, like the people building those devices actually take some responsibility and pride, I guess, on what they are building.
And they start building them properly.
I mean, that's actually the only thing we can hope for because you don't know where they are going to be.
Like if they're behind a corporate network, you can always secure the perimeter somehow.
But I mean, yeah, I mean, if it's at your, I mean, at home, like, I don't know.
Yeah, it's kind of kind of tough.
Even if you put them on like isolated networks, like my router will let me create multiple, like a guest network and another one, I could put it on there and it won't get to my stuff.
But still, it becomes a denial of service thing on the internet, right?
Which is not great.
Yeah, and you're tech savvy, right?
Like, I mean, if my parents try to, like, they buy like this Philips bulb that can do like a million things.
Like, I mean, they have no idea about computers.
They just bought it with the remote control that it comes and it just works.
Yeah, it seems to me that the silly cheap light bulbs and other things like this are going to have an interesting fix in the future.
And networking will probably somehow be part of it, but I don't know what it is yet.
All right, well, thanks for those questions.
So, I mean, we should wrap it up.
We're getting kind of into the show.
So, a couple of questions for you.
If you're going to write some Python code, what editor do you use?
BI, always.
BI, awesome.
Yeah, I saw you use it on your demo there in the video.
That's great.
And notable PyPI packages.
Aha.
So, obviously, we can pip install Napalm, right?
Yeah, correct.
Okay.
So, there's a good one.
Any others?
I would say Hammock, which builds on top of requests, and Enmico.
Yeah, those are probably the two I would mention.
Yeah, okay.
I don't know about Hammock.
That's cool.
What does it do?
It's mostly a wrapper on top of requests.
So, you can start doing, like, you create your instance, and then you do, like, myapi.api.v1.host.get.
And then the URL just builds itself.
So, it's kind of like...
Oh, interesting.
I see.
Nice to use.
It uses, like, the dynamic nature of types themselves to express the URL.
Okay, that's really cool.
I got to check that out.
All right.
So, it was very nice.
Thanks for being on the show.
Any final call for action?
If people want to get started, how do they do it?
So, well, if people want to get started, they can always visit our new webpage where you
can find links to our GitHub repo, to the documentation as well.
We have instructions on how to use VMs so you can actually start poking at Napalm without
having to deal with the actual infrastructure.
So, yeah, our website, it's probably the best place to begin with.
Okay.
Excellent.
Well, David, thank you so much for being on the show.
It was great to talk about networks with you.
Yeah, thank you for having me here.
It was a lot of fun.
You've got...
This has been another episode of Talk Python to Me.
Today's guest was David Barroso, and this episode has been brought to you by Linode and Rollbar.
Linode is bulletproof hosting for whatever you're building with Python.
Get your four months free at talkpython.fm/linode.
Just use the code python17.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might
have gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at rollbar.com slash
talkpythontome.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my Write Pythonic code course
at talkpython.fm/pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and
direct RSS feed at /rss on talkpython.fm.
Are you looking for a way to support the show?
Well, taking or recommending one of our courses is really the best way.
But if that's not for you, you can become a patron.
Visit patreon.com slash mkennedy for details.
You can give as little as $1 an episode.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.
you
Thank you.