WEBVTT

00:00:00.001 --> 00:00:04.780
One of the true superpowers of Python is the libraries over at the Python Package Index.

00:00:04.780 --> 00:00:10.660
They're all just a pip install away. And yet, like all code that we run on our systems,

00:00:10.660 --> 00:00:15.540
it is done with some degree of trust. How do we know that all those useful packages are

00:00:15.540 --> 00:00:21.380
trustworthy? That's the topic of this episode. Benz Tozer and John Speedmeyers are here to share

00:00:21.380 --> 00:00:26.660
their research into typosquatting on PyPI and other sneaky deeds. And we also get a chance to

00:00:26.660 --> 00:00:31.040
discuss some potential solutions, fixes, and tools to help solve this problem.

00:00:31.040 --> 00:00:36.640
This is Talk Python To Me, episode 319, recorded May 26, 2021.

00:00:36.640 --> 00:00:54.960
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem,

00:00:54.960 --> 00:00:59.420
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm at

00:00:59.420 --> 00:01:05.000
mkennedy. And keep up with the show and listen to past episodes at talkpython.fm. And follow the show

00:01:05.000 --> 00:01:10.860
on Twitter via at Talk Python. This episode is brought to you by Square and us over at Talk Python

00:01:10.860 --> 00:01:14.860
Training. Please check out what we're offering during our segments. It really helps support the show.

00:01:14.860 --> 00:01:19.900
Hey, all. I have a quick announcement. We've had transcripts for all of our episodes for a long

00:01:19.900 --> 00:01:24.440
time. But recently, we put more time and effort into making them more useful for you.

00:01:24.440 --> 00:01:29.100
Now, every show has a link to the transcripts right in your podcast player. And that transcript

00:01:29.100 --> 00:01:34.040
page lets you filter, search, and playback audio from exact moments within the transcript. Hope you

00:01:34.040 --> 00:01:39.660
enjoy the richer experience around using our episodes as reference materials. I'm also happy to announce

00:01:39.660 --> 00:01:46.760
a new sponsor of the show, Assembly AI. Assembly AI is a top-rated API for automatic speech-to-text.

00:01:46.900 --> 00:01:52.300
You can transcribe videos and audio files with human-level accuracy in just a few lines of code.

00:01:52.300 --> 00:01:57.140
To help us keep leveling up our transcript games, they're sponsoring the transcripts for our episodes

00:01:57.140 --> 00:02:02.400
going forward. So thank you to Assembly AI for higher quality transcripts and supporting the podcast.

00:02:02.400 --> 00:02:07.620
Check them out at talkpython.fm/assemblyai. Now, on to that conversation.

00:02:07.620 --> 00:02:10.760
Vince, John, welcome to Talk Python To Me.

00:02:10.760 --> 00:02:12.080
Thank you. Thanks for having us.

00:02:12.080 --> 00:02:12.560
Thank you.

00:02:12.680 --> 00:02:17.440
Yeah, it's great to have you both here. It's going to be exciting, unnerving, I might say,

00:02:17.440 --> 00:02:21.420
a little bit to have this conversation. But I think it's certainly high time.

00:02:21.420 --> 00:02:23.220
You'll never pip install the same way.

00:02:23.220 --> 00:02:29.580
Exactly, exactly. You just kind of hold your breath as you do at each time. And you know,

00:02:29.580 --> 00:02:34.220
I'm also, this is not a challenge that just the Python world faces.

00:02:34.220 --> 00:02:37.240
This is anyone that has a package manager.

00:02:37.240 --> 00:02:37.520
Yep.

00:02:37.780 --> 00:02:41.740
And the more open, the bigger the difficulties, I suppose, right? So NPM,

00:02:41.740 --> 00:02:44.160
gems, like you name it, right?

00:02:44.160 --> 00:02:47.640
Yep. If you're a software developer these days, it probably affects you.

00:02:47.640 --> 00:02:54.940
Absolutely. So before we get into the typo squatting, the supply chain issues and all the

00:02:54.940 --> 00:03:00.640
stuff in history and current problems and, you know, on the positive side, there are solutions

00:03:00.640 --> 00:03:04.660
and tools and things that we can talk about as well. Before we get into all that, let's start

00:03:04.660 --> 00:03:08.940
your story, maybe abbreviated version since there's a couple of you. Ben, how do you get

00:03:08.940 --> 00:03:09.940
into programming in Python?

00:03:09.940 --> 00:03:14.320
Programming I got into just as a kid. Got a computer when I was, I don't know, nine or

00:03:14.320 --> 00:03:18.880
10 and tinkered around with it, enjoyed it, ended up actually taking programming classes in

00:03:18.880 --> 00:03:22.940
high school, stuck with it in college, majoring in computer engineering, and was a software developer

00:03:22.940 --> 00:03:27.340
for the system engineer sort of stuff in the defense industry for 20 years.

00:03:27.340 --> 00:03:30.140
Yeah. What languages did you start in or mainly use?

00:03:30.320 --> 00:03:35.480
Originally started in C, actually originally in Pascal, then started in C, C++, and transitioned

00:03:35.480 --> 00:03:40.100
over to doing more active Python development. I just needed a web scraper, needed to collect

00:03:40.100 --> 00:03:43.220
some data and analyze it, and Python was the right tool for the job.

00:03:43.220 --> 00:03:44.600
You didn't want to do that in C++?

00:03:44.600 --> 00:03:51.160
I did not, no. And now, you know, Python is my preferred language for tinkering or back-end

00:03:51.160 --> 00:03:53.600
web development. Pretty much as much as I can use it for, I use Python.

00:03:53.600 --> 00:03:55.360
Yeah, fantastic. John, how about you?

00:03:55.360 --> 00:04:01.060
I don't have quite the classic story. I learned it programming through statistics classes in

00:04:01.060 --> 00:04:05.800
undergrad, specialized language called Stata that a lot of economists use. Good for legal

00:04:05.800 --> 00:04:11.700
trials, well-tested, but I didn't learn Python until I, in grad school, I took more data science

00:04:11.700 --> 00:04:17.260
classes and learned the typical NumPy, Panda, scikit-learn sort of stuff.

00:04:17.260 --> 00:04:20.940
Right. They're like, let us introduce you to probably, it was called IPython at the time.

00:04:20.940 --> 00:04:24.280
Exactly. And now, of course, Jupyter Notebooks, that sort of thing.

00:04:24.280 --> 00:04:30.060
Yeah. Fantastic. It's really interesting just to see all the broad and diverse ways that Python

00:04:30.060 --> 00:04:34.540
is growing and people are coming into it, you know? It's not that, well, I learned it for

00:04:34.540 --> 00:04:38.500
programming, you know, building an operating system and on I went. There's a lot of languages that are

00:04:38.500 --> 00:04:41.820
fairly, you know, or JavaScript. I built it to work on a website, right?

00:04:41.820 --> 00:04:42.080
Yeah.

00:04:42.080 --> 00:04:44.840
It draws people in from all sorts of things, which is awesome.

00:04:44.840 --> 00:04:45.680
It's a meeting ground.

00:04:45.680 --> 00:04:49.620
Yeah. Yeah. And I think that's one of the strengths, actually, kind of a sidebar is that

00:04:49.620 --> 00:04:53.680
we have all these people with different backgrounds and different motivations and interests and things

00:04:53.680 --> 00:04:57.780
they're trying to build rather than being more like, well, here's how I build my web app. How do

00:04:57.780 --> 00:05:03.080
you build your web app? Yeah. Very cool. And how about now? Vince, what are you up to day-to-day?

00:05:03.080 --> 00:05:07.200
So day-to-day, you know, kind of put down the keyboard, at least from the programming perspective.

00:05:07.660 --> 00:05:13.800
And I work as a cybersecurity subject matter expert for In-Q-Tel, which, so I guess my job

00:05:13.800 --> 00:05:18.440
there is to search for and then work with companies we find in the cybersecurity industry that have a

00:05:18.440 --> 00:05:23.600
high impact on national security, as well as providing kind of advisory services to our customers in the

00:05:23.600 --> 00:05:24.140
U.S. government.

00:05:24.140 --> 00:05:27.840
Okay, cool. So what's In-Q-Tel? Sorry, In-Q-Tel, I guess it is?

00:05:27.840 --> 00:05:28.100
Yes.

00:05:28.100 --> 00:05:31.320
Yeah. What's the company story there? Because you both are from the same company.

00:05:31.320 --> 00:05:37.460
Yeah. So it's a nonprofit, 501c3, stood up a little over 20 years ago by the CIA to basically help

00:05:37.460 --> 00:05:41.040
you know, originally the CIA, but now it's seen most of the intelligence community and

00:05:41.040 --> 00:05:47.100
elements of the DOD basically acquire and adopt and use cutting edge technology. They realized a

00:05:47.100 --> 00:05:51.020
little while ago, you know, around that time that a lot of innovation was moving into Silicon Valley

00:05:51.020 --> 00:05:55.760
and into other places in industry and startups. And the traditional acquisition model that federal

00:05:55.760 --> 00:05:59.480
government uses doesn't play well with those people. They don't understand it. So we kind of

00:05:59.480 --> 00:06:04.260
helped as a bridge working with startups, identifying them, and then helping them interact with the

00:06:04.260 --> 00:06:07.200
government and conversely helping the government, you know, adopt,

00:06:07.260 --> 00:06:09.880
said technologies and support their mission.

00:06:09.880 --> 00:06:13.940
So maybe, let me see if I can run a scenario by you. Maybe there's some Silicon Valley company

00:06:13.940 --> 00:06:21.800
that's created like a cool ML thing that identifies deforestation or something like that. And the

00:06:21.800 --> 00:06:26.620
government decides, oh, this might be really helpful for us for, I have no reason why. I have no idea

00:06:26.620 --> 00:06:31.120
why, but let me imagine there's a reason, right? You might help that company like work with the

00:06:31.120 --> 00:06:36.220
request for proposals and the whole crazy government side of things and get them more in line with what's

00:06:36.220 --> 00:06:39.820
needed there. Is that the story? Yeah, that's to an extent. Yeah. I mean, we actually invest in them,

00:06:39.820 --> 00:06:44.260
take equity, and that do help them learn how to interact with the government and also help them

00:06:44.260 --> 00:06:46.520
shape their product and meet our customer needs.

00:06:46.520 --> 00:06:50.600
Yeah. Okay, cool. Interesting. I had no idea such a company exists.

00:06:50.600 --> 00:06:51.720
John, how about yourself?

00:06:51.840 --> 00:06:58.400
I'm also at IQT. I work in what's called IQT Labs. It's an open source applied research and development

00:06:58.400 --> 00:07:02.840
lab where we do hands-on research, mostly in the open source, largely on GitHub.

00:07:02.840 --> 00:07:10.100
Cool. Sounds very, very fun. Now, let's talk about the supply chain issue, I guess, at a real broad level,

00:07:10.400 --> 00:07:16.160
right? And I don't know how you all feel. I suspect that you have a little more hesitancy

00:07:16.160 --> 00:07:21.480
or whatever as you interact with the computers and software and the internet and so on. You know,

00:07:21.480 --> 00:07:25.780
when you, oh, there's a cool new app, maybe I'll try that. Like, you might think a little more

00:07:25.780 --> 00:07:29.360
carefully about this than the average, you know, say, teenager or whatever.

00:07:29.360 --> 00:07:32.060
There's a little bit of paranoia that comes with this. It's true.

00:07:32.220 --> 00:07:37.040
Yeah, exactly. Exactly. That's what I'm getting at. And I feel like one of the more insidious

00:07:37.040 --> 00:07:42.120
aspects of this has been the supply chain side of things, right? Because it's one thing to say,

00:07:42.120 --> 00:07:47.040
that app looks shady. That site looks shady. Let me just not go there. Let me not click that link.

00:07:47.040 --> 00:07:54.900
Let me not install that. But if I were to install, you know, Office Suite X and I completely trust the

00:07:54.900 --> 00:08:01.140
company that makes that, but there's some library that they got from a third party and that third party

00:08:01.140 --> 00:08:07.000
had been hacked and they somehow Trojan'd that third party thing and no one's found out yet.

00:08:07.000 --> 00:08:11.720
I don't know. That's super scary. And that's kind of along the lines of some of the things that we're

00:08:11.720 --> 00:08:16.640
touching on. And so I think the most broad one of those in the recent times has got to be SolarWinds,

00:08:16.640 --> 00:08:20.980
right? That's certainly what's making the headlines these days. Still even, what,

00:08:20.980 --> 00:08:26.200
five, four or five months later. It's, yeah, still a topic of discussion around this theme.

00:08:26.200 --> 00:08:30.660
And yeah, I mean, that was a pretty challenging attack to pull off. I mean,

00:08:30.660 --> 00:08:35.860
it took nation state actors months, maybe years to plan, you know, laying the groundwork,

00:08:35.860 --> 00:08:40.440
getting things in place, you know, basically infiltrating SolarWinds development infrastructure.

00:08:40.440 --> 00:08:44.840
Pretty impressive, honestly, that they were able to do it. And obviously the impact was enormous.

00:08:44.840 --> 00:08:46.320
It was wildly successful.

00:08:46.320 --> 00:08:51.060
I think one thing that Vince and I have been interested in, though, while this sort of attack

00:08:51.060 --> 00:08:56.720
is very serious and obviously has rightly gathered a lot of attention, there are a number of other

00:08:56.720 --> 00:09:01.460
types of attacks, often focusing on open source software that are actually more numerous.

00:09:01.460 --> 00:09:08.840
How serious they are is actually open to debate. But we are still talking many people affected and can

00:09:08.840 --> 00:09:12.260
still have grave consequences, especially if you're the one that's hacked.

00:09:12.500 --> 00:09:18.940
So it's gotten less front of the newspaper attention, but Vince and I still think it's serious.

00:09:18.940 --> 00:09:22.840
Yeah, I think it's very serious. I started with this one because I feel like everyone has heard about this.

00:09:22.840 --> 00:09:24.480
Everyone can relate to this, right?

00:09:24.540 --> 00:09:33.180
And here's an example of a company that supplies network gear to many of the largest companies and government

00:09:33.180 --> 00:09:38.900
organizations around the world. And this was basically a way to get, you know, access to all of those.

00:09:38.900 --> 00:09:44.640
They think it's Russia's cozy beer crew, but who knows, right? And it almost doesn't matter.

00:09:44.640 --> 00:09:49.500
Another one that I think also is in the news really quick before we jump into the open source stuff.

00:09:49.500 --> 00:09:54.660
This is not open source at all, but was called Xcode Ghost. Have you two heard of this?

00:09:54.660 --> 00:09:59.000
Yeah. Yeah. So, yeah, I mean, basically what happened here was, you know, app developers,

00:09:59.000 --> 00:10:04.880
iOS developers in China don't like to download or can't download stuff from the Apple official

00:10:04.880 --> 00:10:10.560
Apple version of Xcode. Someone, you know, put a compromised version of Xcode up on some.

00:10:10.560 --> 00:10:12.340
So let's get it off BitTorrent or something.

00:10:12.340 --> 00:10:16.280
Yeah. I mean, also some Chinese file sharing site that app developers over there like to use

00:10:16.280 --> 00:10:20.700
because it's more convenient and they, they, it was compromised. There was a, basically a,

00:10:20.700 --> 00:10:25.480
something that would bake a backdoor into, you know, the ultimate compiled app that would go into

00:10:25.480 --> 00:10:27.820
the app store or variant of the app store.

00:10:27.820 --> 00:10:33.900
Yeah. So every app that was built and published to the app store with Xcode Ghost, which looked

00:10:33.900 --> 00:10:39.200
exactly like Xcode injected a backdoor malware type of thing into it. So there was something like

00:10:39.200 --> 00:10:46.880
2,500 applications, the iOS app store that yeah, affected like 128 million people. So that,

00:10:46.880 --> 00:10:48.720
that's bad kinds of things, right?

00:10:48.720 --> 00:10:49.260
Very bad.

00:10:49.260 --> 00:10:49.960
Not ideal.

00:10:49.960 --> 00:10:54.180
I mean, I guess attacking a compiler, I mean, developers trust their compiler, I would say,

00:10:54.180 --> 00:10:57.980
I mean, not being able to rely on that or feel like you have, and it's very hard to

00:10:57.980 --> 00:11:02.840
vet your company, especially for closed source or closed source, product like Xcode.

00:11:02.960 --> 00:11:07.260
It's very hard to see is my compiler compromised or not.

00:11:07.260 --> 00:11:12.000
Yeah. Yeah. And I think this actually is closer to the open source side of things,

00:11:12.000 --> 00:11:16.380
right? Because if you can start to infect the tools of the developers building the things,

00:11:16.380 --> 00:11:21.100
that's a problem. Yeah. So let's talk about the open source side, John, you pointed out,

00:11:21.100 --> 00:11:23.440
there's many known attacks over there.

00:11:23.440 --> 00:11:23.980
That's right.

00:11:23.980 --> 00:11:25.460
Set the stage. What's going on?

00:11:25.580 --> 00:11:29.660
There's actually a range of attacks, but I'll mention a couple here and we'll get into typosquatting.

00:11:29.660 --> 00:11:33.640
So there is certainly a typosquatting attack, which we'll discuss extensively today, which

00:11:33.640 --> 00:11:41.060
just like domain names, you might've heard someone is trying to go to a website and, mistypes it

00:11:41.060 --> 00:11:45.160
a little, or somehow gets confused about how it's spelled, maybe switching the order of words,

00:11:45.160 --> 00:11:50.400
and then ends up someplace that's malicious, either on the web, or if you're downloading a package,

00:11:50.400 --> 00:11:54.380
you download a package you think you want, but it's not actually.

00:11:54.380 --> 00:12:00.720
And sometimes not always, sometimes that contains malware and does things to your computer that you

00:12:00.720 --> 00:12:01.080
don't want.

00:12:01.080 --> 00:12:01.680
That's bad, right?

00:12:01.680 --> 00:12:08.360
Bad. Especially if there's arbitrary code execution, meaning they can do what they want because perhaps

00:12:08.360 --> 00:12:09.480
you've installed it as root.

00:12:09.480 --> 00:12:14.900
Right. And well, I think a lot of people who are getting into Python don't realize that when you

00:12:14.900 --> 00:12:21.380
pip install something, unless it's being installed as a wheel, as a binary wheel, it's running a setup.py

00:12:21.380 --> 00:12:27.020
as your account. So whatever your current account is able to do, like you said, if you run it as

00:12:27.020 --> 00:12:31.680
sudo, it's, it can do more, but even if it can just completely wreck your account and get your

00:12:31.680 --> 00:12:35.000
information for many people, that's plenty bad on your computer. You don't want to.

00:12:35.000 --> 00:12:39.140
Yeah. And it could be your computer. It could be your, your corporation's computer where you work

00:12:39.140 --> 00:12:43.900
or your company's computer. And this setup.py, you're exactly right. It is a key attack vector.

00:12:44.000 --> 00:12:48.540
For many people, it's simply a helpful way to install software. But unfortunately,

00:12:48.540 --> 00:12:51.020
some people abuse that specific resource.

00:12:51.020 --> 00:12:55.900
Yeah. I think it's been critical in the growth of how software is built. I remember,

00:12:55.900 --> 00:13:02.060
you know, Ben, you were talking about doing C++ programming. I remember back in 97, 98,

00:13:02.060 --> 00:13:07.880
99 doing C++ programming then. And it felt like whatever you wanted to do, almost everything you

00:13:07.880 --> 00:13:12.640
had to build from scratch. You want a library that does this kind of UI widgets? Well, how do I build

00:13:12.640 --> 00:13:17.600
that? You want a library that has this kind of data structure? Where do I either find or build that?

00:13:17.600 --> 00:13:22.540
Right. And now it's just pip install this thing, pip install that thing. And the building blocks

00:13:22.540 --> 00:13:28.660
that we have to compose are so much more effective, right? I can take a couple of libraries here and

00:13:28.660 --> 00:13:33.500
click them together. And all of a sudden I've got something absolutely incredible, like pulling data

00:13:33.500 --> 00:13:38.320
from different sources, creating amazing graphs. I mean, that is the power of modern software

00:13:38.320 --> 00:13:43.440
development, right? And yet, you know, I guess Corey Atkins out in the live stream has a nice

00:13:43.440 --> 00:13:48.680
sort of comment on this. Like he said, I didn't realize how naive I was thinking packages were

00:13:48.680 --> 00:13:52.320
vetted. You're not alone, Corey. And so you're not alone. Join the club.

00:13:52.320 --> 00:13:59.660
This portion of Talk Python To Me is brought to you by Square. Payment acceptance can be one of the

00:13:59.660 --> 00:14:04.420
most painful parts of building a web app for a business. When implementing checkout, you want

00:14:04.420 --> 00:14:10.740
it to be simple to build, secure, and slick to use. Square's new web payment SDK raises the bar

00:14:10.740 --> 00:14:16.060
in the payment acceptance developer experience and provides a best in class interface for merchants

00:14:16.060 --> 00:14:22.420
and buyers. With it, you can build a customized branded payment experience and never miss a sale.

00:14:22.420 --> 00:14:28.280
Deliver a highly responsive payments flow across web and mobile that integrates with credit cards and

00:14:28.280 --> 00:14:33.520
debit cards, digital wallets like Apple Pay and Google, ACH bank payments, and even gift cards.

00:14:33.520 --> 00:14:39.260
For more complex transactions, follow-up actions by the customer can include completing a payment

00:14:39.260 --> 00:14:44.540
authentication step, filling in a credit line application form, or doing background risk checks

00:14:44.540 --> 00:14:49.000
on the buyer's device. And developers don't even need to know if the payment method requires

00:14:49.000 --> 00:14:54.060
validation. Square hides the complexity from the seller and guides the buyer through the necessary

00:14:54.060 --> 00:14:59.440
steps. Getting started with a new web payment SDK is easy. Simply include the web payment SDK

00:14:59.440 --> 00:15:04.180
JavaScript, flag an element on the page where you want the payment form to appear, and then attach

00:15:04.180 --> 00:15:09.460
hooks for your custom behavior. Learn more about integrating with Square's web payments SDK at

00:15:09.460 --> 00:15:14.220
talkpython.fm/square, or just click the link in your podcast player's show notes.

00:15:14.300 --> 00:15:22.440
That's talkpython.fm/square. These incredible building blocks, these Lego pieces, there's a lot of faith

00:15:22.440 --> 00:15:27.280
out there that these are good building blocks. Not good in the sense they don't have bugs, but good in that

00:15:27.280 --> 00:15:28.180
they have a good intent.

00:15:28.180 --> 00:15:32.280
Well, I think that's one thing that's the key is that, and one of the things that's a challenge here is you have to

00:15:32.280 --> 00:15:38.020
kind of figure out the intent of the people you're trusting, and you are trusting them ultimately, and you have to

00:15:38.020 --> 00:15:41.520
hope they do not have malicious intent. Because inferring that is very challenging.

00:15:41.740 --> 00:15:47.760
It's a double-edged sword. I mean, I agree. It is a powerful change that you can download a couple

00:15:47.760 --> 00:15:53.680
libraries and have an amazing app, potentially in a few minutes, maybe an hour or two. And this is the

00:15:53.680 --> 00:16:00.360
dream of code reuse, come alive, finally. And it just so happens that there are sometimes downsides.

00:16:00.360 --> 00:16:06.940
They can be mitigated, but unfortunately to the unaware user, which it's all too easy to be unaware,

00:16:06.940 --> 00:16:10.400
it's difficult, actually. There are serious, there can be risks.

00:16:10.680 --> 00:16:16.240
Yeah, there definitely can. Kim Van Wick out of the live stream has an example. A benign example would

00:16:16.240 --> 00:16:22.240
be atter, A-T-T-R versus atters. Both are legitimate packages, but completely different.

00:16:22.240 --> 00:16:27.560
Another example would be if I want to install requests, but I actually just type request.

00:16:27.560 --> 00:16:30.940
I mean, even auditorily, they sound like requests.

00:16:31.200 --> 00:16:32.060
It's easy to do.

00:16:32.060 --> 00:16:38.420
It even sounds like very similar with the S versus no S there. And if somebody says,

00:16:38.420 --> 00:16:42.340
go install requests, you're like, oh, request, pip install request. God, I did it. Like, wait,

00:16:42.340 --> 00:16:43.280
no, no, no, no, don't do that one.

00:16:43.280 --> 00:16:47.540
Yeah. And it actually happened. You can find that that attack truly happened, affected,

00:16:47.540 --> 00:16:52.880
at least according to the article published about it, 20,000 users. So I don't know how many of them

00:16:52.880 --> 00:16:57.620
were actually affected. I haven't, we don't, this is unfortunately part of the problem. It's hard to

00:16:57.620 --> 00:17:03.360
track this data, but the example you brought up, I know you brought it up on purpose. It's serious.

00:17:04.120 --> 00:17:11.000
Yeah. And requests with the S is installed millions of times a week or a month. Many,

00:17:11.000 --> 00:17:12.540
many, many, many, many times, right?

00:17:12.540 --> 00:17:17.740
We'll talk about this later, but we found one called Pandar, like Pandas, but with an R.

00:17:17.740 --> 00:17:24.800
And, you know, it's not hard to imagine just by, either confusion or a mistake typing this.

00:17:24.800 --> 00:17:30.780
Yeah, absolutely. So another area I think that is a little bit interesting before we dive completely

00:17:30.780 --> 00:17:37.740
into the package management type of squatting and related type of issues has to do with a trusted

00:17:37.740 --> 00:17:44.060
open source thing becoming untrusted. And what I mean by that is there were some examples of things

00:17:44.060 --> 00:17:49.360
like Google Chrome extensions being put out there as proper extensions, and then someone taking over

00:17:49.360 --> 00:17:55.260
that project and then putting something maybe more adware in it, or something somewhat nefarious,

00:17:55.260 --> 00:18:01.500
if not actually malicious, or, you know, somebody who is running the request is not a great example

00:18:01.500 --> 00:18:08.240
because it's under the PSF organization, but many of the projects are under an individual, right? On

00:18:08.240 --> 00:18:13.840
their GitHub project. And so if somebody was able to break into that person's GitHub repo, and then they

00:18:13.840 --> 00:18:19.660
somehow sneak something into the code, well, does it look wrong? No, the person who made that change

00:18:19.660 --> 00:18:25.780
is the trusted benevolent person who runs this project, right? They are, if, you know, Guido

00:18:25.780 --> 00:18:30.400
van Rossum comes in and makes a change, well, who's going to look at that and go, oh, this is, this guy's

00:18:30.400 --> 00:18:35.080
sketchy. We better really, like, it's probably going to be fine, right? So if someone takes over an account,

00:18:35.080 --> 00:18:39.960
like, not only do they have access to the code and then how it gets pushed out to, you know, potentially

00:18:39.960 --> 00:18:45.540
gets into the stream that goes to PyPI. It's also done by the person who looks like they should be most

00:18:45.540 --> 00:18:51.940
trusted, right? So things like two-factor auth and just securing your GitHub and things along those

00:18:51.940 --> 00:18:56.360
lines seems extremely important as well. Absolutely. I mean, what you're describing with account takeovers

00:18:56.360 --> 00:19:01.920
happen numerous times. And there's variants on it too, where there's some single developer who's

00:19:01.920 --> 00:19:05.720
overworked, tired, doesn't use the project they create anymore. They just hand it over to someone

00:19:05.720 --> 00:19:10.180
who ends up, you know, putting a backdoor in it or some sort of malicious payload. I mean, that, that's

00:19:10.180 --> 00:19:13.900
happened. And then also people take advantage of the fact that not only do you have your GitHub

00:19:13.900 --> 00:19:18.620
profile secure, but you also have to have your PyPI or Ruby gems or, you know, where you actually

00:19:18.620 --> 00:19:23.560
publish your packages, people run. So there's kind of two areas for potential attack. And also people

00:19:23.560 --> 00:19:28.720
take advantage of the, you know, most people, at least me anyway, when I would vet software, I would go

00:19:28.720 --> 00:19:32.200
look at GitHub and then I would download, I wouldn't download it from GitHub. I would download it using

00:19:32.200 --> 00:19:37.740
PIP or whatever. And that kind of, dissonance or whatever you want to call it,

00:19:37.740 --> 00:19:41.680
there's another opportunity for, for confusion and malfeasance.

00:19:43.240 --> 00:19:49.020
Yeah. And so these things are hard to detect, but I guess the area that you all have done a lot of

00:19:49.020 --> 00:19:55.500
research in, you built some tools around and probably the biggest area is around the package

00:19:55.500 --> 00:19:57.640
management side of things, right? That's right.

00:19:57.640 --> 00:20:03.880
So we've talked about typosquatting and some of the challenges where people might mistype things.

00:20:03.880 --> 00:20:09.980
And you talked about some examples where you found packages that look like they were intended to be

00:20:09.980 --> 00:20:15.920
installed by accident, you know, to catch those. If there's 7 million people type, you know, 7 million

00:20:15.920 --> 00:20:20.720
times pip install requests is typed. Chances that a couple of those are misspelled or enough of those

00:20:20.720 --> 00:20:25.820
are misspelled is pretty high, but there were actually quite a few examples. Like for example,

00:20:25.820 --> 00:20:33.200
the register had an article. When was this? This was, this is recent, March, 2021. The title is Python

00:20:33.200 --> 00:20:41.100
Package Index nukes. 3,653 malicious libraries uploaded soon after security shortcomings highlighted.

00:20:41.100 --> 00:20:45.740
That's right. This is, there's really a longer historical narrative too, to include this.

00:20:45.740 --> 00:20:51.120
I'll call this a political activism, anti-typo squatting activism, where this,

00:20:51.120 --> 00:20:55.040
you could call it an attack, is really about drawing attention to this risk.

00:20:55.180 --> 00:20:59.480
Yeah. And I feel like a lot of these were people like, look, I'm proving to you this could actually

00:20:59.480 --> 00:21:06.200
happen. That's right. I'm proving by creating this thing that uploads as requests with the S&T.

00:21:06.200 --> 00:21:06.700
That's right.

00:21:06.700 --> 00:21:12.240
Swapped. But were there actually viruses put up there? Like what is the actual harm been?

00:21:12.240 --> 00:21:19.340
Yeah. So not all of these are. This one and a number of others, we can discuss those if we have time,

00:21:19.580 --> 00:21:25.340
were largely benevolent, but demonstrated the risk. But yes, there have been, at least by our

00:21:25.340 --> 00:21:31.740
calculations, 40 known malicious typo squatters on the Python Package Index, affecting thousands of

00:21:31.740 --> 00:21:37.020
users. We actually published a blog post on this, something like Python typo squatting is about more

00:21:37.020 --> 00:21:43.140
than typos. So yes, this has happened. I don't know the exact persons that it has affected. We just

00:21:43.140 --> 00:21:49.040
don't have that data. Sorry if it affected you. And we published this and got some debate on hacker news.

00:21:49.040 --> 00:21:53.480
And this is the point where Vince and I said, oh, there's really something here. There's a broad

00:21:53.480 --> 00:21:56.900
audience that hasn't had a voice that cares about this.

00:21:56.900 --> 00:22:02.320
Yeah. I mean, it could have been nothing, right? If I'm a student at a university and I install it on

00:22:02.320 --> 00:22:03.160
a lab computer.

00:22:03.160 --> 00:22:03.640
No big deal.

00:22:03.640 --> 00:22:06.860
No big deal. Like who trusts those lab computers, right?

00:22:06.860 --> 00:22:07.620
You shouldn't.

00:22:07.620 --> 00:22:13.920
I mean, not just because like somebody could have installed something bad on it, but there's,

00:22:13.920 --> 00:22:15.240
there are college students.

00:22:15.240 --> 00:22:15.680
Oh yeah.

00:22:15.680 --> 00:22:20.320
Who could be installing all sorts of just, you know, pranks and other kinds of stuff. So you

00:22:20.320 --> 00:22:22.020
should just treat those things with.

00:22:22.020 --> 00:22:23.020
Contaminated.

00:22:23.020 --> 00:22:29.080
Yes, they're fully connected. But on the other hand, if this is a data scientist working at like

00:22:29.080 --> 00:22:35.540
a major corporation or an agency and that happened to them, it could be the thing that opens the door

00:22:35.540 --> 00:22:40.480
to, you know, access to the entire network and all sorts of lateral movement, right?

00:22:40.480 --> 00:22:45.440
That's right. There's even one of the earliest pieces of anti-typo squadding activism comes from

00:22:45.440 --> 00:22:53.360
Nikolai Schocker, who was writing his undergraduate thesis at the time in Europe. And he showed that

00:22:53.360 --> 00:23:00.000
over a few weeks, he got over 17,000 downloads of a series of type of squad packages, including .mil,

00:23:00.000 --> 00:23:06.920
the military addresses of the United States military. So it is certainly possible that people in a more

00:23:06.920 --> 00:23:11.140
secure organization that really value security could accidentally be the victim of type of

00:23:11.140 --> 00:23:11.440
squatting.

00:23:11.440 --> 00:23:17.360
Yeah, absolutely. And the fact that it came out of a .mil domain shows that, yeah, that bad example

00:23:17.360 --> 00:23:21.500
could also happen. And also his thesis got covered on Ars Techno.

00:23:21.500 --> 00:23:23.880
That's right. Coolest undergrad thesis ever.

00:23:23.880 --> 00:23:27.180
Exactly. That's way better than anything I did in college.

00:23:27.180 --> 00:23:28.700
Oh yeah.

00:23:28.920 --> 00:23:33.600
Yeah. Fantastic. And then there was this project called Pyto Squatting.

00:23:33.600 --> 00:23:34.220
Yeah.

00:23:34.220 --> 00:23:34.840
Pyto Squatting.

00:23:34.840 --> 00:23:36.060
Yeah. It's a play on...

00:23:36.060 --> 00:23:39.080
Which actually has been... Yeah, like a play on typo squatting.

00:23:39.080 --> 00:23:43.420
It's a play on typo squatting. It's a clever one. And Benjamin Balderbach and Hano Beck,

00:23:43.420 --> 00:23:51.020
who are open source software activists, developers, also a journalist, they've really had a multi-year

00:23:51.020 --> 00:23:56.380
effort pointing out the dangers here. Not simply criticizing, but trying to help Python Software Foundation

00:23:56.380 --> 00:24:04.000
and the warehouse, our PyPI crew, raise money and build a consensus around trying to make

00:24:04.000 --> 00:24:05.280
this infrastructure safer.

00:24:05.280 --> 00:24:11.500
Yeah. Yeah. So they had this project called Pyto Squatting, but that actually got closed down.

00:24:11.500 --> 00:24:11.940
That's right.

00:24:11.940 --> 00:24:16.960
Yeah. Because they said that the PS... What do they call it?

00:24:16.960 --> 00:24:20.540
The PSRT, Python Security Response Team.

00:24:20.540 --> 00:24:25.960
That's it. PSRT. And I'm like, wait, there's a Python Security Response Team?

00:24:26.280 --> 00:24:26.820
That's cool.

00:24:26.820 --> 00:24:28.600
And they respond to emails too. They're good.

00:24:28.600 --> 00:24:34.480
Yeah. Okay. So this is an organization, a group of people under the PSF banner that basically

00:24:34.480 --> 00:24:37.420
triage these types of concerns, right?

00:24:37.420 --> 00:24:38.340
That's right. That's right.

00:24:38.340 --> 00:24:46.260
Okay. Yeah. I'll link to their page on python.org and they have their email there. They also have

00:24:46.260 --> 00:24:51.120
rules for different types of disclosure, like whether you should email them, do other things.

00:24:51.300 --> 00:24:56.720
That's right. And if you find a malicious package or even a package that you think is very suspicious,

00:24:56.720 --> 00:25:01.600
this is who to contact. And they're diligent and timely.

00:25:01.760 --> 00:25:06.620
So what do you two think about how this should be disclosed? People out there listening, they find

00:25:06.620 --> 00:25:12.580
something. Should they go to Hacker News and say, look, this horrible thing I found on PyPI or on

00:25:12.580 --> 00:25:21.100
NPM or whatever. Should they quietly disclose that to the security response team and then talk about it

00:25:21.100 --> 00:25:25.500
after it's been removed and fixed? What's the flow for disclosure?

00:25:25.500 --> 00:25:31.980
Seems like it would follow any other responsible disclosure process for traditional bugs, exploitable

00:25:31.980 --> 00:25:39.020
bugs that are with vulnerabilities, where it would be nice if you find a problem, contact maybe the

00:25:39.020 --> 00:25:43.520
Python security team, get in contact with the developer, get it fixed, probably get the package

00:25:43.520 --> 00:25:48.360
pulled down if in fact it is malicious. And then, yeah, it'd be nice to have some sort of reporting

00:25:48.360 --> 00:25:53.220
mechanism so that everyone who uses it could be identified. And the first part, John Speed was

00:25:53.220 --> 00:25:57.900
saying, you know, the Python South for Foundation and the PSR team do a good job or great job of

00:25:57.900 --> 00:26:03.300
being on top of it, being timely, being responsible. It's much harder to notify, you know, there's no

00:26:03.300 --> 00:26:06.200
authentication when you download one of these packages. So it's very hard to know who's been

00:26:06.200 --> 00:26:10.380
affected. So maybe just promoting that more would be helpful. But then people have to know where to

00:26:10.380 --> 00:26:13.280
look and that they need to look at all. It becomes challenging quickly.

00:26:13.280 --> 00:26:20.140
Well, it's like the Xcode ghost thing, you know, there was 2,500 apps that were backdoored.

00:26:20.560 --> 00:26:26.120
And I think only the top 25 were even disclosed. And it's like, if there was a list of 2,500 apps,

00:26:26.120 --> 00:26:31.260
are you going to go cross compare? No, you know, no normal person is going to cross compare that

00:26:31.260 --> 00:26:33.080
announcement with their phone.

00:26:33.080 --> 00:26:33.440
Right.

00:26:33.440 --> 00:26:33.800
Right.

00:26:33.800 --> 00:26:34.100
Right.

00:26:34.100 --> 00:26:37.780
And it's just such a challenge. And I feel like, you know, here we had the same thing,

00:26:37.780 --> 00:26:43.780
right? We had 3,653 packages removed. Well, are you going to go check if you had those? It's

00:26:43.780 --> 00:26:50.620
extra hard because it's, you didn't intend to ever have them. You didn't intend to swap the S and the

00:26:50.620 --> 00:26:55.880
T when you type requests, but you did. And you accidentally, almost unknowingly got it most

00:26:55.880 --> 00:27:02.560
likely. Right. And so I do think it's really hard to push this out as an awareness thing and like,

00:27:02.620 --> 00:27:06.320
hey, you should know that this happened. And so just go check, right? The checking,

00:27:06.320 --> 00:27:07.320
I think it's really tricky.

00:27:07.320 --> 00:27:11.140
Yeah. I mean, like many software problems, you need to solve it with more software.

00:27:11.140 --> 00:27:13.020
You got to solve it with AI probably.

00:27:13.220 --> 00:27:18.660
No, you definitely have to solve it with AI. I think one thing that's helpful and could be part

00:27:18.660 --> 00:27:26.220
of that process, but isn't always, unfortunately, is also taking a collection or taking that artifact

00:27:26.220 --> 00:27:31.120
that you found, let's say a Python package that was malicious and making sure it gets to somewhere

00:27:31.120 --> 00:27:36.560
where it can be studied and hopefully future attacks prevented. And so for Python and a couple

00:27:36.560 --> 00:27:41.020
of other languages, there is actually an interesting project. It has a very colorful name. It's called

00:27:41.020 --> 00:27:46.660
Backstabber's Knife Collection. Sounds very scary and malicious, but it is actually yet another

00:27:46.660 --> 00:27:51.340
enterprising grad student trying to collect malware samples, especially of interpreted languages.

00:27:51.340 --> 00:27:57.500
Python is one of them. And so that there can be a community researchers and hopefully then

00:27:57.500 --> 00:28:02.300
companies that can fight these packages. So that would be another thing I would add to the list.

00:28:02.300 --> 00:28:07.520
Yeah, there you go. Mark Ohm is the main person associated with that and has written some

00:28:07.520 --> 00:28:12.060
interesting papers and great stuff. And so I urge you, if you come upon this and you say,

00:28:12.060 --> 00:28:17.920
how do I act responsibly here? Do the things Bent says and also maybe grab a sample and give it to

00:28:17.920 --> 00:28:20.840
the Backstabber's Knife Collection or another similar repository.

00:28:20.840 --> 00:28:25.700
Yeah, interesting. Okay. Have I just messed up my computer by visiting this webpage as well? I wonder.

00:28:25.700 --> 00:28:27.340
I don't think so, but there is a...

00:28:27.340 --> 00:28:28.440
I'm just teasing. I'm just teasing.

00:28:28.440 --> 00:28:31.160
I mean, I can't guarantee anything though, but...

00:28:31.160 --> 00:28:35.640
No, of course, of course, of course. Before we get too far on, Corey Adkins also asked,

00:28:35.640 --> 00:28:39.740
when we were talking about messing up your computer, the lab computer, so on, he asked,

00:28:39.740 --> 00:28:46.540
could installing these types of things also affect shared server space? On my IaaS land,

00:28:46.540 --> 00:28:51.400
where I have a shared server running for however much someone else does something bad.

00:28:51.400 --> 00:28:55.480
I mean, theoretically, sure. It depends on the permissions, I would think. Yeah. If you install

00:28:55.480 --> 00:29:00.460
some dependency that has keylogger baked into it or, I don't know, or, you know, some sort of file,

00:29:00.460 --> 00:29:05.320
you know, collector, and it has permission to traverse all directories, then yeah, I mean,

00:29:05.320 --> 00:29:08.020
I could certainly see a scenario where that was possible. I mean, I haven't, you know,

00:29:08.020 --> 00:29:12.180
I haven't heard of that happening specifically, but there's nothing preventing it theoretically.

00:29:12.180 --> 00:29:17.340
Yeah. If you had a series of virtual machines, you know, it's pretty tricky from one virtual machine

00:29:17.340 --> 00:29:22.240
to escape to another, but I believe there have been examples, but those are exceedingly rare,

00:29:22.340 --> 00:29:26.020
those sorts of vulnerabilities, right? That's right. So while we're on this topic,

00:29:26.020 --> 00:29:29.760
I want to throw out an idea and then we'll talk about some of the tools you built, but I feel like

00:29:29.760 --> 00:29:34.860
we're right in the middle of this notification thing. Like we've got all these packages, they've

00:29:34.860 --> 00:29:39.080
been identified, they have been downloaded. We can see that we probably even have IP addresses,

00:29:39.080 --> 00:29:44.160
which you can reverse look up to DNS names as probably how those attributions were given,

00:29:44.160 --> 00:29:51.040
but it happens so often in so many different places, right? Like if I've got a continuous

00:29:51.040 --> 00:29:57.140
integration story that builds a Docker container that pushes to a Docker hub and then my production

00:29:57.140 --> 00:30:03.120
grabs that from that container, the place where the problem happened is not the place where the

00:30:03.120 --> 00:30:09.780
problem is, right? It's probably GitHub or some other CI pool. We have a really nice thing for this

00:30:09.780 --> 00:30:15.740
in the account space. Have I been pwned by Troy Hunt, which is a really nice project. I definitely

00:30:15.740 --> 00:30:19.240
recommend people go there and enter their email address.

00:30:19.240 --> 00:30:20.440
And prepare to be horrified.

00:30:20.440 --> 00:30:27.060
Yeah. And prepare to be horrified. There's 11.2, 11.3 billion accounts that have been breached,

00:30:27.060 --> 00:30:31.740
which is odd because it's more than all the humans, but we have more than one account. So there it is.

00:30:31.740 --> 00:30:37.140
But yeah, so you put your email in there and then in the future, well, historically as well,

00:30:37.140 --> 00:30:42.520
but then in the future, you say, if something has happened and your email appears in some kind of

00:30:42.520 --> 00:30:47.360
password dump, password breach or account informational breach, you'll get an email saying,

00:30:47.360 --> 00:30:51.540
Hey, we found something that should be concerning to you. Check it out. I would love to see something

00:30:51.540 --> 00:30:57.860
like this for pip, right? Something that says, I pip installed this thing and it just has a record of,

00:30:57.860 --> 00:31:02.520
here's my account. These are the things I've pip installed. If there turns out to be a problem

00:31:02.520 --> 00:31:05.440
with one of those, notify me that that had happened.

00:31:05.440 --> 00:31:07.720
That does sound really useful. I don't think it exists.

00:31:07.900 --> 00:31:11.840
I don't think it exists either. And it shouldn't just be a pip thing. It should be an npm thing.

00:31:11.840 --> 00:31:15.380
It should be a gem thing. It should be a crate thing. It should be something that like a,

00:31:15.380 --> 00:31:22.560
just a little bit of a wrapper that says, I would like to opt in to saying, here's my UUID.

00:31:22.560 --> 00:31:27.200
Here's my email address. And here's the list of things that I've installed. If it turns out that

00:31:27.200 --> 00:31:28.740
one of them is horrible, just let me know.

00:31:30.680 --> 00:31:32.920
It's so sensible. It makes me laugh.

00:31:32.920 --> 00:31:33.420
Yeah.

00:31:33.420 --> 00:31:38.880
Well, there's an idea out there as well, but this is way far down the line, right? This is,

00:31:38.880 --> 00:31:43.440
oh, we know this has happened. We know who's done it. We've, we know who's been affected and so on,

00:31:43.440 --> 00:31:50.160
but starting a little bit further back, you all have built some tools to go and start at the

00:31:50.160 --> 00:31:54.360
beginning and say, well, let's look and see what might be out there. That is bad, right? This is

00:31:54.360 --> 00:31:58.020
the tool you used to find pandar instead of pandas.

00:31:58.020 --> 00:32:03.000
That's right. I mean, I think the first idea you had is the crucial one, which is that

00:32:03.000 --> 00:32:08.180
you need to know that there's been a compromise in order to report it. And right now it's surprisingly

00:32:08.180 --> 00:32:14.100
hard to know that. So we're not the only one to have devised a tool or approach to finding

00:32:14.100 --> 00:32:19.020
malicious packages on the Python package index, but we took a particularly simple one and we said,

00:32:19.020 --> 00:32:23.420
can we use simply the metadata, especially the name, but some other information too,

00:32:23.840 --> 00:32:29.120
of packages. And then look at just the most downloaded packages and check who has names

00:32:29.120 --> 00:32:32.800
that are very similar to those packages. This is where AI comes in.

00:32:32.800 --> 00:32:37.880
Need crazy AI at this point. You do get a lot of false positives. People have similar names just

00:32:37.880 --> 00:32:42.960
because the packages are related. It's fine. There's no problem inherently with having a similar name,

00:32:42.960 --> 00:32:49.340
but we cracked open those packages too. This was some very boring Saturday mornings of mine

00:32:49.340 --> 00:32:54.480
and simply scanned through the code looking for anything that's suspicious. And lo and behold,

00:32:54.480 --> 00:32:59.740
we found one called Pandar that was actually doing key logging. It was a proof of concept. It's unlikely

00:32:59.740 --> 00:33:05.120
that it actually would have worked, but we reported it to the Python security response team, security at

00:33:05.120 --> 00:33:12.520
python.org. They said, yep, not good. Yanked it. And it was just an example of it's not that hard

00:33:12.520 --> 00:33:18.640
to find them. And we were showing yet again with a pretty simple demonstration that it's not that hard.

00:33:18.640 --> 00:33:24.220
Interesting. That's really cool. So basically the tool is about finding given popular packages,

00:33:24.220 --> 00:33:29.740
finding ones that are oddly similar. And then there's like a, let me go and see what this one's about.

00:33:29.860 --> 00:33:33.560
That's right. And so there's a couple additional checks to help anyone using it. And you can find

00:33:33.560 --> 00:33:39.340
it's an open source written in Python tool, command line tool. It also checks things like, for instance,

00:33:39.340 --> 00:33:46.460
is the description of the package on PyPI, is it very similar? So that what you are witnessing is

00:33:46.460 --> 00:33:52.920
someone who's trying to not only type with squat the name, but in some sense, like squat the broader

00:33:52.920 --> 00:33:55.380
metadata or almost like the copyright of the package.

00:33:55.380 --> 00:33:57.340
Right. Because you want it to look as similar as possible.

00:33:57.340 --> 00:33:58.780
Exactly. Like camouflage.

00:33:58.780 --> 00:34:01.660
Something that comes to mind. Yeah. Are you guys familiar with sneak?

00:34:01.660 --> 00:34:02.160
Yes.

00:34:02.160 --> 00:34:02.580
Yes.

00:34:02.580 --> 00:34:06.520
Is it package? There's a project. And geez, I'm forgetting.

00:34:06.520 --> 00:34:08.100
Are you thinking of the advisor project?

00:34:08.100 --> 00:34:10.320
Yes. Sneak package advisor.

00:34:10.320 --> 00:34:11.240
Yeah. It's neat.

00:34:11.240 --> 00:34:15.440
That's it. It's super neat. Yeah. That's what I was looking for. No, that's not how I want to spell it.

00:34:15.440 --> 00:34:21.380
Yeah. And so that thing is pretty cool. The reason I bring this thing up is you can come over here and I

00:34:21.380 --> 00:34:25.880
can type in a project like requests or whatever, and it'll tell us eventually,

00:34:26.340 --> 00:34:29.340
it'll tell us the package health score. Yeah.

00:34:29.340 --> 00:34:33.780
And it'll tell us things like there's this many PRs that have been open and closed. There's this many

00:34:33.780 --> 00:34:38.800
contributors. There's this many people participating that the maintenance looks like so on. One thing

00:34:38.800 --> 00:34:45.420
that I think would be cool would be to take this number plus a misspelling and say, if that number

00:34:45.420 --> 00:34:50.600
is really, really low for a package that should be really, really high, that's a challenge, right?

00:34:50.600 --> 00:34:56.000
If you look at the GitHub repo that is delivering this thing and it doesn't look right, if it's not

00:34:56.000 --> 00:35:00.360
associated with something that seems kind of hard to replicate, like a GitHub repo with many people

00:35:00.360 --> 00:35:04.760
participating over a long period of time, that seems like that could be a good flag as well.

00:35:04.880 --> 00:35:10.640
Yeah, certainly. It certainly seems like there's an abundant opportunity to build something into the

00:35:10.640 --> 00:35:17.380
actual download client to the pip or a wrapper around pip where it checks these sorts of things and

00:35:17.380 --> 00:35:23.420
create speed bumps for you as you are trying to download something or use a package so that

00:35:23.420 --> 00:35:29.680
says, hey, this looks suspicious. Have you thoroughly checked this? And I think your idea could contribute to

00:35:29.680 --> 00:35:36.340
exactly such a tool or tools. Yeah. Yeah. Very neat. You've been working on this and Martin

00:35:36.340 --> 00:35:43.360
Karnoguski created this thing called Aurora and also reached out and said, hey, I'm also working on this.

00:35:43.360 --> 00:35:49.020
And so, yeah, tell us about this thing called Aurora. Yeah. So we got an email last fall after publishing

00:35:49.020 --> 00:35:55.220
this blog post and he said, hey, I've been working on a similar tool. Not only does it check this metadata,

00:35:55.220 --> 00:36:01.500
but we even do static analysis of the entire Python package index. And we said, Martin, that's awesome.

00:36:01.500 --> 00:36:08.060
Let's work together. And so over the past six months, roughly now, in an open source collaboration

00:36:08.060 --> 00:36:14.680
between a number of us at IQT Labs and Martin Karnoguski, we have further refined Aurora, which

00:36:14.680 --> 00:36:20.500
truly is designed to do a static analysis of the entire Python package index open source tool. You can find it.

00:36:20.840 --> 00:36:26.580
He releases his data on a try as best as he can to release it regularly. We've also built a tool

00:36:26.580 --> 00:36:33.320
called Aura Borealis. That key thing, his aura produces 50 gigs of output when it's done scanning

00:36:33.320 --> 00:36:38.000
the entire Python package index. No human can wade through that. So.

00:36:38.000 --> 00:36:44.360
And I suspect also the IPA, the Python package authority folks probably don't want everyone

00:36:44.360 --> 00:36:50.220
downloading that much data all the time. No, it's exhausting and creates so many database issues and

00:36:50.220 --> 00:36:55.080
other things. So we've been working on a tool called Aura Borealis that you've pulled up that is a front

00:36:55.080 --> 00:37:02.760
end that makes it easier to use the data set that Martin creates this tool, Aura. This wouldn't

00:37:02.760 --> 00:37:08.840
necessarily be part of PyPI, though, of course, it could be. But we imagine this as a tool for

00:37:08.840 --> 00:37:15.200
organizations or persons that need to have global knowledge about either global knowledge about the

00:37:15.200 --> 00:37:20.220
entire Python package index and to rank and assess potential threats and go look, look for those,

00:37:20.220 --> 00:37:25.300
and then take appropriate action or even individual developers that are really curious about packages.

00:37:25.300 --> 00:37:30.820
This is makes it easy. The Aura Borealis isn't yet live, but we hope to make it live this summer.

00:37:30.820 --> 00:37:34.440
Aura is a in production tool. It works. So go check it out.

00:37:34.440 --> 00:37:39.100
Talk Python To Me is partially supported by our training courses.

00:37:39.100 --> 00:37:45.320
When you need to learn something new, whether it's foundational Python, advanced topics like async or web apps

00:37:45.320 --> 00:37:50.740
and web APIs, be sure to check out our over 200 hours of courses at Talk Python.

00:37:51.320 --> 00:37:57.520
And if your company is considering how they'll get up to speed on Python, please recommend they give our content a look. Thanks.

00:37:57.520 --> 00:38:05.920
This looks like it's really handy. You know, so the idea is basically it's going to run forever and that's going to generate tremendous amounts of data.

00:38:05.920 --> 00:38:11.360
Maybe just put a web front end on top of that static data for everyone to generate it over and over.

00:38:11.360 --> 00:38:16.320
Exactly. Instead of having generated over and over now having 50 gigs and having to write your own custom,

00:38:16.400 --> 00:38:20.540
probably Python script that's, you know, you'll have to optimize and blah, blah, blah.

00:38:20.540 --> 00:38:26.100
Yeah. So came out in the live stream just says I accidentally typed sync instead of sneak,

00:38:26.100 --> 00:38:30.780
which also is hard to spell anyway, because it's like a non-common spelling.

00:38:30.780 --> 00:38:34.700
So which is an excellent way to demonstrate making a typo of getting the wrong package.

00:38:34.700 --> 00:38:37.780
I have no idea what that's going to return. I'm not going to pull it up.

00:38:37.780 --> 00:38:41.780
Podcast imitates life, imitates art, imitates compromise.

00:38:43.580 --> 00:38:50.780
Exactly. All right. Well, this is really neat. How would I use the Aurora data and the Aurora Borealis project?

00:38:50.780 --> 00:38:53.640
I guess also we should talk about this from different angles, right?

00:38:53.640 --> 00:38:59.900
Maybe I'm a CISO at a company and I'm concerned that all my people are psyched about data science and Python

00:38:59.900 --> 00:39:06.320
or npm and web front ends and they just make me nervous all day and I want to get on top of it.

00:39:06.440 --> 00:39:11.960
So I want, as somebody who is concerned about, I would like to know what's happening in my software

00:39:11.960 --> 00:39:20.940
fly chain, or maybe I run, I maintain pandas and I'm really upset that pandar exists and I want to now be able to defend my package.

00:39:20.940 --> 00:39:24.480
Like it seems like there's different use cases and people out there.

00:39:24.480 --> 00:39:29.260
That's right. I think if you're a company and you have a group of software developers and

00:39:29.260 --> 00:39:34.140
you have the, let's say a security team that helps vet packages.

00:39:34.140 --> 00:39:39.880
So perhaps you put those packages in an internal repository so that the developers know that they're

00:39:39.880 --> 00:39:42.880
cleared to use or Borealis will help you do that.

00:39:42.880 --> 00:39:44.900
We're glad to set up pilots and discuss.

00:39:44.900 --> 00:39:47.740
You can email me, jmeyers at iqt.org.

00:39:47.740 --> 00:39:50.100
But there's also other angles too.

00:39:50.100 --> 00:39:53.400
There's just, you're a developer and you want to make an informed choice.

00:39:53.400 --> 00:39:57.260
The static analysis tool and its output can help you with that or Borealis.

00:39:57.500 --> 00:40:02.780
And I think there is also, you're right, there's a maintainer angle and also a PyPI administrator

00:40:02.780 --> 00:40:09.900
angle where you want to either protect a set of namespaces close to your package or you care

00:40:09.900 --> 00:40:11.480
about the health of the entire ecosystem.

00:40:11.480 --> 00:40:14.900
And those are all possible user types.

00:40:14.900 --> 00:40:15.260
Yeah.

00:40:15.260 --> 00:40:22.180
And we could probably use your PyPI scan to go and say, look, can I say, look for things

00:40:22.180 --> 00:40:23.700
similar to my package name?

00:40:23.700 --> 00:40:24.680
Yeah, that's right.

00:40:24.680 --> 00:40:27.100
And we built that into Aura Borealis too now.

00:40:27.260 --> 00:40:33.900
So in some ways, PyPI scan was a demo and still useful as a command line tool, but Aura Borealis

00:40:33.900 --> 00:40:36.100
and Aura has that now built in.

00:40:36.100 --> 00:40:38.060
Are you all going to put an API on top of this?

00:40:38.060 --> 00:40:38.720
Good question.

00:40:38.720 --> 00:40:40.720
That would be cool.

00:40:40.720 --> 00:40:45.160
The thing that's tricky, like everything in life, is it costs money and, you know,

00:40:45.160 --> 00:40:47.240
engineering resources and time.

00:40:47.740 --> 00:40:48.740
I certainly have a vision.

00:40:48.740 --> 00:40:48.900
I certainly have a vision.

00:40:48.900 --> 00:40:52.300
And, you know, if I don't do it, someone else should do it.

00:40:52.300 --> 00:40:53.220
Go make a lot of money.

00:40:53.380 --> 00:41:00.480
of creating a technical infrastructure that every single package and every single new version

00:41:00.480 --> 00:41:06.340
of every package, IPI, NPM, et cetera, gets scanned, a variety of scans, static analysis,

00:41:06.340 --> 00:41:08.280
dynamic analysis, metadata analysis.

00:41:08.280 --> 00:41:14.040
And that gets stored in a database that where you and I can go make API calls and get that

00:41:14.040 --> 00:41:16.640
information that we should on these packages.

00:41:16.800 --> 00:41:18.700
That could be, you know, there could be a free tier.

00:41:18.700 --> 00:41:21.880
And then if you really need to make a lot of calls, a paid tier.

00:41:21.880 --> 00:41:23.920
But someone should do it, I think.

00:41:23.920 --> 00:41:28.600
Yeah, it would be neat to know, like you said, integrate into, say, pip even.

00:41:28.600 --> 00:41:33.940
So if I pip install something, it could even flag it and say, hey, no, actually, we're going

00:41:33.940 --> 00:41:34.540
to block that.

00:41:34.540 --> 00:41:35.060
That's right.

00:41:35.060 --> 00:41:39.360
Preemptively, because it's got some low score, unless you do like a --force.

00:41:39.360 --> 00:41:41.060
Like, no, really, I mean, yeah, exactly.

00:41:41.060 --> 00:41:44.240
It's something that'll sort of slow it down, as you call them speed bumps.

00:41:44.240 --> 00:41:46.020
I hope someone does something similar to that.

00:41:46.020 --> 00:41:50.200
We have plans, but no active development underway.

00:41:50.200 --> 00:41:50.540
All right.

00:41:50.540 --> 00:41:55.580
So that sets the stage that some of the tools out there, at least to identify that there

00:41:55.580 --> 00:41:57.600
are potentially bad packages.

00:41:57.600 --> 00:42:02.820
And it's also, I guess, you know, worth pointing out that if we go over, say, to PyPI, there's

00:42:02.820 --> 00:42:05.640
over 300,000 packages over there.

00:42:05.640 --> 00:42:10.740
And if there are 40 actually malicious ones, right, the chances are low.

00:42:10.740 --> 00:42:12.340
They're not very high.

00:42:12.340 --> 00:42:17.320
But so people shouldn't be, you know, running for the hills and complete panic or anything,

00:42:17.320 --> 00:42:18.340
I don't think, from this.

00:42:18.340 --> 00:42:20.920
But at the same time, we should be careful.

00:42:20.920 --> 00:42:21.660
We should be cautious.

00:42:21.660 --> 00:42:23.600
So, you know, what can we do?

00:42:23.600 --> 00:42:24.780
That's the tough question.

00:42:24.780 --> 00:42:25.700
Vince, do you want to start?

00:42:25.700 --> 00:42:26.460
And you want me to go?

00:42:26.460 --> 00:42:26.880
Sure.

00:42:26.880 --> 00:42:28.900
I mean, there's a lot of things that we can do.

00:42:29.000 --> 00:42:33.900
I mean, John Speed's hit on a few of them about just kind of being more deliberate, you know,

00:42:33.900 --> 00:42:36.100
checking your work before you download something.

00:42:36.100 --> 00:42:40.600
And also, you know, when you're considering dependencies, I mean, you mentioned C++ and,

00:42:40.600 --> 00:42:41.700
you know, the late 90s.

00:42:41.700 --> 00:42:43.020
I vaguely remember those times.

00:42:43.020 --> 00:42:44.360
I remember when Boost came out, it was a big deal.

00:42:44.360 --> 00:42:44.800
Oh, yeah.

00:42:44.800 --> 00:42:46.360
You actually had a dependency that was...

00:42:46.360 --> 00:42:47.660
I remember reading more books.

00:42:47.660 --> 00:42:48.100
Right.

00:42:48.100 --> 00:42:50.880
Less internet, more books to make things...

00:42:50.880 --> 00:42:52.060
So, yeah, we moved on from that.

00:42:52.060 --> 00:42:55.140
But ultimately, you know, it is worth considering, do you actually need this dependency?

00:42:55.400 --> 00:42:58.320
You know, LeftPad, and npm is a funny, you know, canonical example.

00:42:58.320 --> 00:43:01.660
Broke the internet because people didn't feel like typing one line of their own code.

00:43:01.660 --> 00:43:04.720
They wanted to import a LeftPad-ing dependency.

00:43:04.720 --> 00:43:06.600
I do feel that's a really good example.

00:43:06.600 --> 00:43:12.160
And certainly, LeftPad came to mind, not as a malicious thing, but just as a supply chain

00:43:12.160 --> 00:43:14.360
Jenga tower type of thing.

00:43:14.360 --> 00:43:17.560
And somebody pulled too much on a part of the Jenga tower and it came down.

00:43:17.560 --> 00:43:25.140
I feel that the JavaScript community has way smaller Lego pieces than the Python community.

00:43:25.140 --> 00:43:28.440
The blocks that you click together here are larger.

00:43:28.440 --> 00:43:35.440
So, I feel like there's just fewer in number external dependencies on average in my Python

00:43:35.440 --> 00:43:37.080
experience than my JavaScript experience.

00:43:37.080 --> 00:43:37.980
Yeah, I think that's accurate.

00:43:37.980 --> 00:43:39.040
I mean, numbers vary.

00:43:39.040 --> 00:43:43.340
I've seen NPM, people who use NPMs are JavaScript developers.

00:43:43.340 --> 00:43:47.140
The average package in npm has like 94% dependency.

00:43:47.140 --> 00:43:50.980
You know, other dependencies, only 6% is your actual code you've written.

00:43:51.380 --> 00:43:56.740
Most of the modern languages, meaning JavaScript, Python, and some others are in like the 90-ish

00:43:56.740 --> 00:43:57.080
range.

00:43:57.080 --> 00:43:59.600
And then you see C and C++ are much lower.

00:43:59.600 --> 00:44:00.580
Java is somewhere in the middle.

00:44:00.580 --> 00:44:01.120
So, yeah.

00:44:01.120 --> 00:44:06.360
Python is, I would say, lower than JavaScript, but much higher than the kind of legacy languages

00:44:06.360 --> 00:44:08.540
that are historically used.

00:44:08.540 --> 00:44:13.800
So, be deliberate means things like don't just, as fast as you can, type pip install, whatever.

00:44:13.800 --> 00:44:17.500
Type pip install and then carefully type out the package name.

00:44:17.860 --> 00:44:19.880
Maybe give it a quick read before you hit go.

00:44:19.880 --> 00:44:20.980
Yeah, or just copy and paste.

00:44:20.980 --> 00:44:22.840
Don't type it all.

00:44:22.840 --> 00:44:23.140
Yeah.

00:44:23.140 --> 00:44:29.080
So, for example, if I'm over here on PyPI, there's a copy button I can click and it'll

00:44:29.080 --> 00:44:30.560
do exactly that.

00:44:30.560 --> 00:44:30.740
Right.

00:44:30.740 --> 00:44:31.040
Right.

00:44:31.040 --> 00:44:31.600
That's an option.

00:44:31.600 --> 00:44:31.960
Yeah.

00:44:31.960 --> 00:44:32.700
So, yeah.

00:44:32.700 --> 00:44:36.240
Just being a little more thoughtful and kind of, you know, looking at the dependency chain

00:44:36.240 --> 00:44:40.300
as well before you download something, which is much harder than it should be, to be

00:44:40.300 --> 00:44:41.160
completely fair.

00:44:41.160 --> 00:44:45.980
That's helpful to know that, you know, maybe the top level you are using Joski.

00:44:46.220 --> 00:44:47.400
That's how you pronounce that.

00:44:47.400 --> 00:44:49.120
I have no idea what Joski.

00:44:49.120 --> 00:44:49.520
I don't know.

00:44:49.520 --> 00:44:50.900
You should pip install that right now.

00:44:50.900 --> 00:44:51.760
Let's see what happens.

00:44:51.760 --> 00:44:52.700
Let's just see what happens.

00:44:52.700 --> 00:44:55.940
Here's an example of one of those that should rank lower.

00:44:55.940 --> 00:45:00.340
No offense if this is your project, but it literally has zero stars, zero forks.

00:45:00.340 --> 00:45:01.480
Its features are to do.

00:45:01.480 --> 00:45:02.740
Its requirements are to do.

00:45:02.740 --> 00:45:06.280
Its PyPI version banner is not found.

00:45:06.280 --> 00:45:08.660
And I mean, it is only four minutes old.

00:45:08.660 --> 00:45:09.480
They may be working.

00:45:09.480 --> 00:45:09.680
Yeah.

00:45:09.680 --> 00:45:10.140
Sure.

00:45:10.140 --> 00:45:10.480
Yeah.

00:45:10.480 --> 00:45:12.660
If it has dependencies, this one probably doesn't.

00:45:12.660 --> 00:45:13.980
But, you know, take a look at those two.

00:45:13.980 --> 00:45:16.920
Just makes them there's nothing egregiously wrong at a minimum.

00:45:16.920 --> 00:45:17.260
Yeah.

00:45:17.260 --> 00:45:22.380
That's one of the things that makes it kind of insidious and hard to see is the thing I

00:45:22.380 --> 00:45:24.160
directly look at may be fine.

00:45:24.160 --> 00:45:28.980
But the person who maintained that, did they make a mistake in the things that they depend

00:45:28.980 --> 00:45:29.280
upon?

00:45:29.280 --> 00:45:33.920
Or maybe, you know, transit, like follow that chain that graph down far enough.

00:45:33.920 --> 00:45:34.420
Right.

00:45:34.420 --> 00:45:37.500
There's a lot of layers that could be happening along the way.

00:45:37.580 --> 00:45:38.460
It ends up looking like a web.

00:45:38.460 --> 00:45:43.700
And not surprisingly, just because of that, most vulnerabilities inside of packages like

00:45:43.700 --> 00:45:48.440
this are in the transitive dependencies, the ones below the first layer, the dependencies

00:45:48.440 --> 00:45:48.860
are dependent.

00:45:48.860 --> 00:45:49.320
Interesting.

00:45:49.320 --> 00:45:51.420
So you can pip install the thing.

00:45:51.420 --> 00:45:52.780
What about pinning the version?

00:45:52.780 --> 00:45:58.340
I know there were some issues about having a private PyPI server, which I think is a good

00:45:58.340 --> 00:46:00.100
idea where you whitelist packages in.

00:46:00.100 --> 00:46:03.640
You say, we approve these things and only these things get installed.

00:46:03.640 --> 00:46:06.480
And if you want to use a new one, we've got to opt it in.

00:46:06.480 --> 00:46:08.140
And then now it's part of the organization.

00:46:08.140 --> 00:46:10.300
That seems like something you could do, right?

00:46:10.300 --> 00:46:15.340
There's PyPI server that you could set up that is a sort of pass-through layer there.

00:46:15.340 --> 00:46:19.120
But then there was also the vulnerability of the version mismatch.

00:46:19.120 --> 00:46:23.940
Like if there's a higher version of that thing on the public PyPI, then your local one.

00:46:23.940 --> 00:46:29.900
So people were putting in like data layer version 70, you know, and then it's like, oh, there's

00:46:29.900 --> 00:46:31.560
a newer version out there for me to go get.

00:46:31.700 --> 00:46:35.760
I'll get that, even though it was internal, meant to be internal only, right?

00:46:35.760 --> 00:46:37.140
So there's these challenges.

00:46:37.140 --> 00:46:39.560
But what do you think about a private whitelist server?

00:46:39.560 --> 00:46:45.420
It certainly seems valuable and seems like it's another speed bump, as John Speed was calling

00:46:45.420 --> 00:46:45.680
them.

00:46:45.680 --> 00:46:50.240
But yeah, I mean, then you run into scenarios like the one you described, where it's kind

00:46:50.240 --> 00:46:55.480
of, I guess, that's undefined behavior, potentially, or at least not well-known behavior that maybe

00:46:55.480 --> 00:46:56.640
isn't necessarily most intuitive.

00:46:56.640 --> 00:46:58.220
So even that might not be enough.

00:46:58.380 --> 00:47:00.620
So then, yeah, the pinning could help.

00:47:00.620 --> 00:47:05.760
Then, of course, there's the challenge of maintaining your pin at the proper level, which adds more

00:47:05.760 --> 00:47:10.420
effort on the developers to maintain up-to-date dependencies.

00:47:10.420 --> 00:47:11.720
At least publicly.

00:47:11.720 --> 00:47:12.360
Yeah.

00:47:12.360 --> 00:47:17.220
Publicly, we have Dependabot on GitHub, which is way more of a pain than it should be to use.

00:47:17.220 --> 00:47:22.580
Because if you've got 10 updates, it'll issue 10 PRs, which conflict with themselves.

00:47:22.940 --> 00:47:24.180
Anyway, that's a long story.

00:47:24.180 --> 00:47:27.360
But it's still at least some automation that says, hey, there's a new version of this.

00:47:27.360 --> 00:47:28.240
Here's the change log.

00:47:28.240 --> 00:47:33.540
And we also have the CVE security checks of Dependabot, which are really good.

00:47:33.540 --> 00:47:33.900
Yeah.

00:47:33.900 --> 00:47:38.620
Unfortunately, most of these typosquatting or just general supply chain attacks don't end

00:47:38.620 --> 00:47:40.120
up in the NVD as a CVE.

00:47:40.120 --> 00:47:40.580
Yeah.

00:47:40.580 --> 00:47:43.840
Who's going to study this one and then not just say, take it down, right?

00:47:43.840 --> 00:47:46.680
Like it's living under the, in the shadows, right?

00:47:46.680 --> 00:47:48.100
Of being unnoticed.

00:47:48.200 --> 00:47:48.960
To an extent, yeah.

00:47:48.960 --> 00:47:53.640
NPM does a good job with their advisory service of like saying, this is a malicious package

00:47:53.640 --> 00:47:55.080
and this is why we removed it.

00:47:55.080 --> 00:47:57.740
But not all package managers do that.

00:47:57.740 --> 00:48:00.580
And even so, then you have to go to all the most developers.

00:48:00.580 --> 00:48:03.740
These ones are developing multiple languages these days.

00:48:03.740 --> 00:48:04.640
So it's hard to keep track.

00:48:04.640 --> 00:48:04.880
Yeah.

00:48:04.880 --> 00:48:09.180
What about having isolated environments for trying out new packages?

00:48:09.180 --> 00:48:13.280
So for example, one of the things I'm trying to do is if I'm checking out any new package,

00:48:13.280 --> 00:48:14.240
I have to pip install.

00:48:14.240 --> 00:48:16.580
And maybe that happens in a Docker container.

00:48:16.820 --> 00:48:20.640
And then I throw away the container or possibly a VM with snapshotting on.

00:48:20.640 --> 00:48:22.760
And then I roll back the snapshot periodically.

00:48:22.760 --> 00:48:23.140
Yeah.

00:48:23.140 --> 00:48:29.580
Those both sound like great ways to have good hygiene and not isolate the potential blast radius

00:48:29.580 --> 00:48:33.040
of a potentially malicious package.

00:48:33.040 --> 00:48:33.300
Yeah.

00:48:33.300 --> 00:48:38.060
It's one thing to say, here's a thing we want you to check out and it's on PyPI and it's

00:48:38.060 --> 00:48:42.420
really well known, but it's, you know, you got to explore new things that aren't super well

00:48:42.420 --> 00:48:42.860
known yet.

00:48:42.860 --> 00:48:43.100
Right.

00:48:43.100 --> 00:48:44.860
And so how do you install that?

00:48:44.860 --> 00:48:45.300
Right.

00:48:45.300 --> 00:48:49.700
So I think some kind of blast blast store, like you said, like Docker, like a VM is not

00:48:49.700 --> 00:48:50.380
a terrible idea.

00:48:50.380 --> 00:48:50.660
Yeah.

00:48:50.660 --> 00:48:51.280
It's a good one.

00:48:51.280 --> 00:48:51.680
What else?

00:48:51.680 --> 00:48:55.740
There's the open source software found, security foundation.

00:48:55.740 --> 00:48:56.720
Yeah, that's right.

00:48:56.720 --> 00:48:58.340
This is open SSF.

00:48:58.340 --> 00:48:59.160
Open SSF.

00:48:59.160 --> 00:49:01.240
Clearly a reference to open SSL.

00:49:01.240 --> 00:49:01.820
Yeah.

00:49:01.860 --> 00:49:06.720
another well known software supply chain compromise that widespread impact.

00:49:06.840 --> 00:49:07.840
It's worth.

00:49:07.840 --> 00:49:07.840
It's worth.

00:49:07.840 --> 00:49:12.220
It's worth pointing out that this group for anyone who is comes very enthusiastic about

00:49:12.220 --> 00:49:17.180
open source software supply chain security in particular has become a meeting ground where

00:49:17.180 --> 00:49:21.540
both companies, but also persons interested in this sort of topics we've been discussing

00:49:21.540 --> 00:49:24.500
the day and more have set up a series of working groups.

00:49:24.500 --> 00:49:30.300
There's six roughly and, meet every few weeks, open community, fun, interesting people,

00:49:30.300 --> 00:49:35.240
either, interested in the topic or actively working to, give back and contribute.

00:49:35.240 --> 00:49:40.020
it's run by the Linux foundation and, we would highly recommend it as a place to

00:49:40.020 --> 00:49:41.540
find other like-minded persons.

00:49:41.540 --> 00:49:43.160
If you care about these sorts of topics.

00:49:43.160 --> 00:49:43.520
Yep.

00:49:43.520 --> 00:49:44.080
Fantastic.

00:49:44.080 --> 00:49:49.420
And then there's the further on down the road, which we've touched on a couple of times,

00:49:49.420 --> 00:49:54.600
but maybe we can encourage some enterprising person, people group out there to go after

00:49:54.600 --> 00:50:00.480
it like a, a hardened pip or, you know, we have things that are sort of on top of pip-tools.

00:50:00.480 --> 00:50:02.040
We've got pip ENV.

00:50:02.040 --> 00:50:03.980
We've got pip X.

00:50:03.980 --> 00:50:05.900
I'm a big fan of pip X, the isolation.

00:50:05.900 --> 00:50:07.240
And then that gives us kind of need.

00:50:07.400 --> 00:50:14.540
And just, I can see like, like a pip sec or something along those or PIPs, maybe a plural

00:50:14.540 --> 00:50:14.900
PIPs.

00:50:14.900 --> 00:50:18.740
I don't know for pip security, but something like that that incorporates some of these ideas.

00:50:18.740 --> 00:50:20.360
Maybe it, it checks in.

00:50:20.360 --> 00:50:25.880
You say like, I don't want to install any package that is not in the top 1000.

00:50:25.880 --> 00:50:26.500
Sure.

00:50:26.500 --> 00:50:32.200
Or a popular package, except for what I whitelist in on top of that or something or check with

00:50:32.200 --> 00:50:37.380
Aurora Borealis about the score or check with the, have I been PIP?

00:50:37.380 --> 00:50:39.800
Whatever that thing ever would become, right?

00:50:39.800 --> 00:50:42.380
So talk about like where you might see things going.

00:50:42.380 --> 00:50:42.820
Yeah.

00:50:42.820 --> 00:50:47.680
Well, there's been a couple, I'll call them starter projects in the hardened pip area.

00:50:47.680 --> 00:50:49.940
There even was one called pip sec.

00:50:49.940 --> 00:50:55.020
You can find it on PyPI, but it's really, there's nothing there, unfortunately, at least yet.

00:50:55.020 --> 00:50:56.400
That namespace is claimed.

00:50:56.400 --> 00:51:01.920
The maintainers who we mentioned, Benjamin Balderbach, especially are interested in doing something,

00:51:01.920 --> 00:51:04.660
just haven't had time, other busy priorities.

00:51:04.660 --> 00:51:10.960
And I think there is a lot of potential to build out that idea and create something that

00:51:10.960 --> 00:51:12.940
could be useful to the average developer.

00:51:12.940 --> 00:51:18.240
JavaScript has a tool that has at least some moderate popularity called MPQ that does this.

00:51:18.240 --> 00:51:23.400
And I think it's time for the Python community to see if there's something similar.

00:51:23.580 --> 00:51:24.700
I would love to see something like that.

00:51:24.700 --> 00:51:26.660
Another thing is Google.

00:51:26.660 --> 00:51:27.600
Thank you, Google.

00:51:27.600 --> 00:51:31.200
Has become a visionary sponsor of the PSF.

00:51:31.200 --> 00:51:37.740
And specifically, they want their funds to go towards critical supply chain security improvements,

00:51:37.740 --> 00:51:43.700
developing productized malware detection for PyPI, for a type of dynamic analysis infrastructure.

00:51:44.240 --> 00:51:55.200
So this sort of gets at the hit at, maybe there's something that the PyPA and PyPI.org could do on their end without even necessarily changing pip, right?

00:51:55.200 --> 00:51:57.300
PIP's going to go talk to some API there.

00:51:57.300 --> 00:51:59.240
And it goes, yeah, no, not this one.

00:51:59.240 --> 00:51:59.640
That's right.

00:51:59.640 --> 00:52:02.360
Or you're going to upload it, like with upload a new package.

00:52:02.360 --> 00:52:03.660
It goes, no, we don't want to accept it.

00:52:03.660 --> 00:52:15.860
And Dustin Ingram of the Python Software Foundation at PyCon just recently devoted his talk to talking about Python and the software supply chain issues that we've discussed today and writ large to include typosquatting.

00:52:15.860 --> 00:52:24.400
And it's clear that there is energy and willingness from even core members of Warehouse and Python Software Foundation to tackle these issues.

00:52:24.400 --> 00:52:26.400
So we're glad to see that.

00:52:26.400 --> 00:52:28.240
It'd be great to see something like that happening.

00:52:28.240 --> 00:52:29.820
I think layers as well, right?

00:52:29.820 --> 00:52:34.380
That's how you talk about security often is it's not just, well, you have a strong password and you're fine.

00:52:34.380 --> 00:52:36.460
Like, well, and maybe you have two-factor authentication.

00:52:36.460 --> 00:52:39.100
And maybe you run as lower permissions and, and, and, and, right?

00:52:39.100 --> 00:52:39.320
Yeah.

00:52:39.320 --> 00:52:39.940
Layers.

00:52:39.940 --> 00:52:43.580
So this could be one of the layers, but not necessarily all of them.

00:52:43.580 --> 00:52:43.800
Yeah.

00:52:43.800 --> 00:52:52.280
I should note that we even, a couple of us at IQT Labs even put in an issue recently that on Warehouse that might interested some parties here.

00:52:52.280 --> 00:52:54.180
It's issue 9527.

00:52:54.180 --> 00:52:59.760
You can also find it at short.iqt.org slash issue, just a redirect.

00:52:59.760 --> 00:53:05.480
And we essentially call for something like social distancing for the top Python package indexes.

00:53:05.480 --> 00:53:11.660
So that for very popular package names, the package names that are close by are blocked off.

00:53:11.660 --> 00:53:17.180
So that not saying that anybody who chooses those names is malicious, but just so malicious people can't choose them.

00:53:17.180 --> 00:53:18.660
Feel free to upvote that.

00:53:18.660 --> 00:53:22.540
We've been discussing this with some of the members of the Warehouse team.

00:53:22.540 --> 00:53:22.840
Yeah.

00:53:22.840 --> 00:53:28.380
So your proposal is that Pandar should not have even been allowed, right?

00:53:28.380 --> 00:53:28.740
That's right.

00:53:28.740 --> 00:53:41.360
Given that the package Pandas is so popular, minor variations on its spelling should basically be blocked or maybe redirect to Pandas and say with a warning, like, you tried to install Pandar.

00:53:41.360 --> 00:53:42.980
Did you mean to install Pandas?

00:53:42.980 --> 00:53:43.560
That's right.

00:53:43.560 --> 00:53:44.500
Something like that.

00:53:44.500 --> 00:53:44.940
That's right.

00:53:44.940 --> 00:53:51.200
So it's a way to build in guardrails so that the unwary don't fall prey to this.

00:53:51.200 --> 00:53:51.500
Yeah.

00:53:51.840 --> 00:53:54.140
Personally, my first impression is that that's a good idea.

00:53:54.140 --> 00:53:58.260
It's worth it that we don't need request and requests and requester.

00:53:58.260 --> 00:54:05.920
And, you know, the potential harm is higher than the value of, you know, reusing very, very similar names.

00:54:05.920 --> 00:54:06.940
Yeah, we agree.

00:54:06.940 --> 00:54:08.480
And there's obviously tradeoffs.

00:54:08.480 --> 00:54:08.840
Yeah.

00:54:08.840 --> 00:54:09.780
Vince, what do you think?

00:54:09.780 --> 00:54:11.280
You must agree with this, I suspect.

00:54:11.280 --> 00:54:12.180
I do agree with that.

00:54:12.180 --> 00:54:13.280
I definitely supported this.

00:54:13.480 --> 00:54:16.960
And I know one other thing that's under consideration that's relevant is namespacing.

00:54:16.960 --> 00:54:21.040
So you can, you know, Kenneth Wright is the request guy.

00:54:21.040 --> 00:54:22.200
He has his namespace.

00:54:22.200 --> 00:54:30.900
You go to his namespace, you're less likely to mistype that and have someone, the namespace and have someone who has claimed the same package within their own namespace.

00:54:31.240 --> 00:54:35.360
So possible, but, you know, it's another layer, I guess, as you were describing it.

00:54:35.360 --> 00:54:36.660
Yeah, it makes the commands.

00:54:36.660 --> 00:54:37.940
You got to type a little bit longer.

00:54:37.940 --> 00:54:40.380
But it makes it really clear where it's coming from.

00:54:40.380 --> 00:54:42.960
I mean, that's what the point of namespaces and programming is.

00:54:42.960 --> 00:54:46.760
It's really clear what library it comes from or what part of your code it comes from.

00:54:47.160 --> 00:54:47.860
And who?

00:54:47.860 --> 00:54:48.920
Grouped together in namespace.

00:54:48.920 --> 00:54:49.420
As well.

00:54:49.420 --> 00:54:52.580
I know Go has done, you know, used that to great success.

00:54:52.580 --> 00:55:01.600
Yeah, Kim Benwick out there put a cool comment that's sort of related to that, talking about the private IPI server that's, you know, redirecting out.

00:55:01.600 --> 00:55:09.540
It would help if the private PIs, if you had an option to prevent the account uploading images from or pulling images with a certain prefix.

00:55:09.660 --> 00:55:22.360
For example, if everybody named their packages ABC something at the company, you could say ABC is private, ABC star is private, and never, ever, you know, go look beyond here for that type of thing.

00:55:22.360 --> 00:55:23.680
I think that that's pretty interesting.

00:55:23.680 --> 00:55:24.900
Yeah, it's a good idea.

00:55:24.900 --> 00:55:27.700
Yeah, I think it seems super simple and a good idea.

00:55:27.700 --> 00:55:28.300
I agree.

00:55:28.300 --> 00:55:29.220
All right, gentlemen.

00:55:29.220 --> 00:55:31.540
Well, very cool to talk about this stuff.

00:55:31.540 --> 00:55:35.100
Like I said, it's going to make all of us a little bit more nervous, I suspect.

00:55:35.100 --> 00:55:37.440
You know, for example, Corey Adkins out there said,

00:55:37.520 --> 00:55:39.720
I also just found an article on malicious Docker images.

00:55:39.720 --> 00:55:41.220
Now I am paranoid, which.

00:55:41.220 --> 00:55:42.420
I'm sorry.

00:55:42.420 --> 00:55:43.060
Welcome.

00:55:43.060 --> 00:55:45.580
Yeah, yeah.

00:55:45.580 --> 00:55:46.440
I've been there for a while.

00:55:46.440 --> 00:55:47.060
All right.

00:55:47.060 --> 00:55:49.700
Before I let you two out of here, though, real quickly, let's answer.

00:55:49.700 --> 00:55:52.340
I'll ask you the two questions at the end of the show, of course.

00:55:52.340 --> 00:55:55.360
So if you're going to write some Python code, what editor do you use?

00:55:55.360 --> 00:55:57.680
I use Vim if I'm in the command line.

00:55:57.680 --> 00:56:01.380
But if I have the fortune to be outside of it, use Sublime.

00:56:01.380 --> 00:56:01.900
Right on.

00:56:01.900 --> 00:56:03.640
I suspect JupyterLab is also in there.

00:56:03.640 --> 00:56:04.700
Definitely Jupyter is in there.

00:56:04.700 --> 00:56:05.020
Yeah.

00:56:05.020 --> 00:56:05.780
And Ben's?

00:56:05.780 --> 00:56:06.760
Probably PyCharm.

00:56:07.160 --> 00:56:11.620
Yeah, I mean, I'll use Vim if I'm already in a command line.

00:56:11.620 --> 00:56:13.500
But yeah, that's not as often these days.

00:56:13.500 --> 00:56:15.900
So PyCharm is just my idea of choice.

00:56:15.900 --> 00:56:17.040
Right on.

00:56:17.040 --> 00:56:21.400
And then notable PyPI package, something that's like, oh, people should know about.

00:56:21.400 --> 00:56:22.720
Check out one called NetworkML.

00:56:22.720 --> 00:56:25.660
It's a package related to machine learning and network traffic.

00:56:25.660 --> 00:56:28.600
The lead maintainer is Charlie Lewis of IQT Labs.

00:56:28.600 --> 00:56:30.120
You can go find it on PyPI.

00:56:30.120 --> 00:56:30.440
Yeah.

00:56:30.440 --> 00:56:31.160
Fantastic.

00:56:31.160 --> 00:56:34.440
So machine learning plugins for network traffic.

00:56:34.440 --> 00:56:34.740
Yeah.

00:56:34.820 --> 00:56:38.080
So it identifies like anomalies and other weirdnesses like that?

00:56:38.080 --> 00:56:38.300
Yeah.

00:56:38.300 --> 00:56:39.800
It parses network traffic.

00:56:39.800 --> 00:56:45.400
And one of the cool things it does is it helps identify what sort of device is being observed.

00:56:45.400 --> 00:56:46.440
So is this thing a printer?

00:56:46.440 --> 00:56:47.900
Or is this thing a personal computer?

00:56:47.900 --> 00:56:49.400
Is it an active directory controller?

00:56:49.400 --> 00:56:49.960
Et cetera.

00:56:49.960 --> 00:56:51.120
Is it a canary?

00:56:51.120 --> 00:56:52.140
Is it a canary?

00:56:52.140 --> 00:56:52.540
Who knows?

00:56:54.600 --> 00:56:54.960
Awesome.

00:56:54.960 --> 00:56:55.440
All right.

00:56:55.440 --> 00:57:00.180
Well, thank you both for shedding the light on lots of what's happening, some of the things

00:57:00.180 --> 00:57:03.000
that are being done and what might also be done as well.

00:57:03.000 --> 00:57:04.300
So final call to action.

00:57:04.300 --> 00:57:07.640
People that want to get involved, maybe do more, become more aware.

00:57:07.640 --> 00:57:08.600
What do you all say?

00:57:08.600 --> 00:57:08.880
Yeah.

00:57:08.880 --> 00:57:10.120
I mean, there's plenty of work to be done.

00:57:10.120 --> 00:57:15.120
Open SSF is a very welcoming, relatively new organization that has a nice list of stuff

00:57:15.120 --> 00:57:15.560
to do.

00:57:15.560 --> 00:57:21.000
Python Software Foundation also actually has an active list of items they would like

00:57:21.000 --> 00:57:23.360
to work on, some of which are relevant to this topic.

00:57:23.360 --> 00:57:25.120
So that'd be two great places to start.

00:57:25.120 --> 00:57:27.820
I'll point you back towards that GitHub issue.

00:57:27.820 --> 00:57:30.040
Feel free to chime in.

00:57:30.040 --> 00:57:32.500
And I think there's definitely potential over the next few months.

00:57:32.500 --> 00:57:38.040
Additionally, we're actually working on a survey at IQT Labs called on secure code reuse.

00:57:38.040 --> 00:57:42.780
So if you want to help build the research foundations for this, you can find this survey at

00:57:42.780 --> 00:57:45.420
short.iqt.org slash survey.

00:57:45.420 --> 00:57:51.220
And we're trying to understand the developer or data scientists or other programming professional

00:57:51.220 --> 00:57:52.760
experience with package reuse.

00:57:52.760 --> 00:57:54.160
So that's another way.

00:57:54.160 --> 00:57:56.900
So hopefully this survey informs future tools.

00:57:56.900 --> 00:57:57.240
Yeah.

00:57:57.240 --> 00:57:57.860
Fantastic.

00:57:57.860 --> 00:57:59.960
Well, thanks for the work that you all are doing.

00:57:59.960 --> 00:58:00.920
And thanks for being on the show.

00:58:00.920 --> 00:58:01.560
Thanks for having us.

00:58:01.560 --> 00:58:02.340
Thanks for having us.

00:58:02.340 --> 00:58:02.580
Bye.

00:58:02.580 --> 00:58:06.460
This has been another episode of Talk Python To Me.

00:58:06.460 --> 00:58:10.020
Our guests on this episode were Ben Stoser and John Speedmeyers.

00:58:10.020 --> 00:58:14.660
It was brought to you by Square, us over at Talk Python Training, and the transcripts are

00:58:14.660 --> 00:58:16.320
brought to you by Assembly AI.

00:58:16.320 --> 00:58:22.100
With Square, your web app can easily take payments, seamlessly accept debit and credit cards, as

00:58:22.100 --> 00:58:23.400
well as digital wallet payments.

00:58:23.400 --> 00:58:29.000
Get started building your own online payment form in three steps with Square's Python SDK

00:58:29.000 --> 00:58:32.080
at talkpython.fm/square.

00:58:32.080 --> 00:58:34.300
Want to level up your Python?

00:58:34.560 --> 00:58:38.340
We have one of the largest catalogs of Python video courses over at Talk Python.

00:58:38.340 --> 00:58:43.540
Our content ranges from true beginners to deeply advanced topics like memory and async.

00:58:43.540 --> 00:58:46.200
And best of all, there's not a subscription in sight.

00:58:46.200 --> 00:58:49.100
Check it out for yourself at training.talkpython.fm.

00:58:49.100 --> 00:58:53.780
Be sure to subscribe to the show, open your favorite podcast app, and search for Python.

00:58:53.780 --> 00:58:55.100
We should be right at the top.

00:58:55.540 --> 00:59:00.260
You can also find the iTunes feed at /itunes, the Google Play feed at /play,

00:59:00.260 --> 00:59:04.460
and the direct RSS feed at /rss on talkpython.fm.

00:59:04.460 --> 00:59:07.880
We're live streaming most of our recordings these days.

00:59:07.880 --> 00:59:11.300
If you want to be part of the show and have your comments featured on the air,

00:59:11.300 --> 00:59:15.660
be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

00:59:16.220 --> 00:59:17.580
This is your host, Michael Kennedy.

00:59:17.580 --> 00:59:18.860
Thanks so much for listening.

00:59:18.860 --> 00:59:20.020
I really appreciate it.

00:59:20.020 --> 00:59:21.940
Now get out there and write some Python code.

00:59:21.940 --> 00:59:42.620
You're welcome.