New course: Agentic AI for Python Devs

Security of everything at PyCon 2026

Episode #557, published Sun, Aug 2, 2026, recorded Mon, Jul 6, 2026
0:00
01:07:60
Links Episode Deep Dive Subscribe Transcript
Guests and sponsors
Panelists
Panelists

Panelists

Juanita Gomez
Juanita Gomez
Mike Fiedler
Mike Fiedler
Seth Michael Larson
Seth Michael Larson
Security has always been the vegetables of software. Everyone agrees it matters, and somehow it never quite makes it onto the plate. At PyCon US this year, that changed. For the first time ever, security got its own dedicated, day-long track, one of just two at the whole conference, sitting right next to AI. And the room was packed to the back wall.

On this episode, I'm joined by the three people at the center of it. Seth Larson, Security Developer in Residence at the Python Software Foundation and, very recently, a CPython core developer. Juanita Gomez, a PhD researcher at UC Santa Cruz in open source security, who co-chaired the track. And Mike Fiedler, PyPI's Safety and Security Engineer, one of the very few people paid full-time to keep the packages you install safe.

We use the arc of the track's talks to take the temperature of Python security right now: supply chain attacks, dependency cooldowns, zero trust, SBOMs, and the push to bring Rust into CPython. And why not one of us thinks security is anywhere close to solved. Turns out that's the good news. It's why the room was full.

Watch this episode on YouTube
Play on YouTube
Watch the live stream version

Episode Deep Dive

Guests Introduction and Background

This episode brings together the three people at the center of the first-ever dedicated security track at PyCon US 2026, called "Trailblazing Python Security." Together they cover the full surface area of Python security, from the language runtime to the package index to the academic research frontier.

Seth Larson is the Security Developer-in-Residence at the Python Software Foundation and, very recently, a CPython core developer (he is also a PSF Fellow). His role is deliberately broad: he works on everything that is not the PyPI backend, including the Python language and runtime, the Python Security Response Team, packaging tools and standards, and the connective tissue between the wider security world and Python (things like CVEs and running a CVE Numbering Authority for the PSF). He co-chaired the security track, and he wrote the announcement post that gave the track its "Trailblazing" name. He blogs about security and, delightfully, retro video games, and he prefers to be reached on Mastodon or Signal.

Juanita Gomez is a Computer Science PhD researcher at UC Santa Cruz who co-chaired the security track. Her research focuses on the security and sustainability of scientific open source software, done in collaboration with the UC Santa Cruz Open Source Program Office (OSPO). She has been deeply involved in the scientific Python community for years, serves as Communications Chair on the SciPy Conference organizing committee, and is a Community Manager for the Scientific Python project. She has long tried to bring security into the conversation at SciPy through birds-of-a-feather sessions, so landing a full, dedicated security track at PyCon was personal for her. She noted in the episode that she is graduating and heading into industry, where she plans to keep working on open source security and sustainability. The best way to reach her is LinkedIn.

Mike Fiedler is the PyPI Safety and Security Engineer at the Python Software Foundation, based in New York City, and one of the very few people paid full-time to keep the packages you install safe. He was a volunteer PyPI admin and contributor for years before the paid role existed, and in this episode he is coming up on his third year in the job. He opened the track with a firsthand account of a real phishing campaign against PyPI maintainers. He describes himself as a "Code Gardener."

What to Know If You're New to Python

This episode lives in the world of Python packaging, authentication, and the software supply chain. You do not need to be a security expert to enjoy it, but a few mental models will help everything land.

  • PyPI and pip: PyPI (the Python Package Index) is the public warehouse of Python packages, and pip is the tool that downloads and installs them when you run "pip install." Almost everything in this episode revolves around how code gets from an author, through PyPI, onto your machine.
  • Dependencies and the supply chain: When you install one package, it usually pulls in many others you never chose directly, called transitive dependencies. The "supply chain" is that whole tree, and the episode repeatedly shows how a problem buried deep in it (a dependency of a dependency) can reach millions of users.
  • Two-factor authentication, and why the kind matters: Two-factor authentication (2FA) adds a second proof of identity beyond your password. A central point of the episode is that not all 2FA is equal: app-generated codes (TOTP) can be phished, while hardware keys and passkeys (WebAuthn) cannot, because they are tied to the real website's address.
  • CVEs and vulnerability databases: A CVE is a public, numbered record of a specific software vulnerability. Tools scan your dependencies against databases of these records to warn you, and the episode digs into where that system works well and where it falls short.
  • CPython and the standard library: CPython is the main implementation of Python, written mostly in C, and it ships with a large "standard library" of built-in modules. When the guests discuss rewriting parts of Python in Rust, this is the codebase they mean.

Key Points and Takeaways

Security finally got its own track at PyCon US, and the room was packed

For the first time ever, PyCon US devoted a full, day-long track to security, one of only two dedicated tracks at the conference, sitting right alongside AI. Seth Larson and Juanita Gomez co-chaired it, titled "Trailblazing Python Security," and curated seven talks into a single room so attendees never had to move. Several talks were standing room only, with people lined up against the back wall. As Seth put it, when a first-time event outgrows its room, you are doing something right. The significance is bigger than logistics: making security a track, rather than a few scattered talks, elevates it to a first-class topic in the Python community. The organizers already want to do it again next year, possibly with a more interactive panel format and, ideally, a bigger room.

Paying for security actually works, but capacity is still tiny

Mike and Seth are among a very small number of people paid full-time to work on Python's security, and they were candid about the tradeoffs. Mike's blunt framing: with volunteers, you get what you pay for, which is nothing, not because volunteers are not excellent, but because they do what they choose in their spare time. Hiring someone sets expectations and focuses effort on the unglamorous, high-impact work that volunteers tend to avoid, like monitoring for spam on a security response inbox or standing up a CVE Numbering Authority. The double edge is that a paid hire can cause volunteers to step back, concentrating pressure on the few full-timers who never have enough capacity. Seth described consciously leaving space for volunteers so the community "muscle" does not atrophy, avoiding both empire-building and low-value "snacking" on tasks others could do. The lesson, in Mike's words: if you pay for a focused security effort, you get good security outcomes.

Anatomy of a phishing campaign against PyPI maintainers

Mike opened the track by dissecting a real attack he handled firsthand. Rather than build a pixel-perfect clone of PyPI, the attackers simply stood up a transparent proxy in front of the real site at a look-alike domain, showing "pypj.org" (a lowercase j) instead of pypi.org, a difference almost invisible in a phone's font. They harvested maintainer email addresses that are publicly listed in GitHub profiles and package metadata, then sent legitimate-looking messages asking maintainers to take an action, exactly the kind of request PyPI itself sometimes sends. Because PyPI is open source, attackers could read the real email templates and copy them convincingly. Of more than a million accounts, only four maintainers fell for it and logged in, and only one package was confirmed compromised and republished with malicious code. Mike was emphatic that the humans did nothing wrong: they did what they were asked to do, and the failure was in the system, not the people.

Not all 2FA is equal: TOTP can be phished, passkeys cannot

The most practical takeaway from the phishing story is that two-factor authentication is not a monolith. Time-based one-time passwords (TOTP), the six-digit codes from an authenticator app or QR code, do not stop a real-time proxy attack, because the attacker simply relays the code the moment you enter it. Worse, once past login the attacker holds your session cookie, which is valid for hours, and can then mint an API token that, today, never expires, giving them durable publishing access without needing your password again. Phishing-resistant factors like WebAuthn, passkeys, and hardware keys such as YubiKey defeat this because the credential is cryptographically bound to the real site's address, so it will not hand anything to a look-alike domain. Anyone using a phishing-resistant factor was simply not affected by this attack. As a mitigation, PyPI now challenges TOTP-based sessions that suddenly change IP address, asking you to re-verify, though that can occasionally catch people bouncing between VPNs or cell towers.

The compromised package rode Hugging Face Transformers into millions of installs

The single package the attackers harvested was not a headline library, it was num2words, buried several layers down as a sub-dependency of Hugging Face's Transformers. Transformers is foundational to a huge amount of AI work and pulls tens of millions of downloads a day, and because num2words was not version-pinned inside it, any Transformers user would automatically pick up the newest, malicious release. This is the supply chain attack pattern in miniature: go after the weakest, least-watched link, not the famous front door, and let the dependency graph do the distribution for you. It also exposes a hard question for every developer, which is how far down the dependency chain you can realistically review. As Mike put it, none of us write everything ourselves, so we all inherit someone else's trust decisions.

Dependency cooldowns are a simple, powerful default for most people

One of the most talked-about ideas in the episode is the "dependency cooldown," which means refusing to install a package version until it has been public for some window, commonly three to seven days. The reasoning is elegant: in that window, the community's scanners and volunteer reporters have very likely chewed through a new release, and if it is malicious it has probably already been pulled. Michael pointed to cooldowns.dev, which cites cases like the LiteLLM incident where the large majority of malicious versions were taken down within a day, and nearly all within a week. Seth argued this is ideal for exactly the people who lack a corporate security team, such as students and solo developers, because you set it once and then quietly benefit from all the scanning and takedown work happening on PyPI. Juanita reinforced that many people running pip install have no idea what PyPI even is, so a safe default that requires no ongoing thought is a real win. The tooling is arriving fast: uv supports it with an exclude-newer option, and pip and other package managers are adding their own versions.

Why secure defaults are so hard at Python's scale

Michael shared a concrete pain point that shows why cooldowns cannot simply be turned on for everyone. He uses pinned dependencies with a compile-and-refresh workflow plus an exclude-newer window, and he added pip-audit to his Docker builds so a build fails if a package has a known CVE. The trap: a CVE would open inside his seven-day window, the fix would land one day later, but the cooldown would refuse to install the fixed version because it was too new, forcing a growing list of one-off exceptions. Seth called this the exact reason he cannot yet push cooldowns as a universal default, because it confuses users, produces baffling "that version does not exist" errors, and collides with other systems telling people to upgrade now. His framing is about operational maturity: if you signal maturity by using locks and exact versions, tools should trust you, and if you do not, a secure default should apply, but that default has to account for security fixes landing inside the window. The cleaner long-term fix may be to move more review to the index side, though PyPI's sheer volume means uploads are available instantly and scanned only after the fact.

Fast malware takedowns rely on volunteer reporters and quarantine

A big reason cooldowns work is the invisible army behind PyPI. Mike explained that PyPI has no commercial scanning relationship, yet a community of volunteer reporters, along with some businesses, continuously scan new uploads and flag suspicious packages. The best of them have earned such a low false-positive rate that PyPI lets their reports automatically move a package into a quarantine state, where it is no longer downloadable until a human like Mike clears it or removes it entirely. He singled out one volunteer, Kamil, based in Austria, whose tooling often flags malicious uploads within about thirty minutes of publish with essentially zero false positives. That combination of automated quarantine and trusted human reporters is what makes the "wait a few days" strategy so effective in practice.

Trusted publishing is the single easiest win for maintainers

When asked for the one thing a maintainer should do this week, Mike's answer was to adopt trusted publishing, convert at least one project to it, and then delete the old API token. Trusted publishing uses OpenID Connect so a CI system like GitHub Actions can exchange a short-lived identity token for a short-lived, scoped PyPI token at publish time, which removes the long-lived, never-expiring API tokens that made the phishing attack so damaging. The "delete the old token" part is essential, because leaving a legacy token active undermines the whole benefit. It is a concrete, high-leverage change that directly closes the exact hole attackers exploited in the incident Mike described.

Zero trust in 200 milliseconds: an identity for every transaction

Tristan McKinnon's talk took the room somewhere PyCon rarely goes, into the world of regulated, compliance-heavy industry. The idea is a zero trust model where, instead of holding a standing credential, your code requests a fresh, tightly scoped credential for each individual transaction, for example a key that is good for only a couple of seconds against AWS. The talk grounded this in FedRAMP, the US government's Federal Risk and Authorization Management Program, and showed how to implement identity-per-transaction in Python with familiar tools like the AWS SDK, then extrapolated to a backend-agnostic approach. Seth loved it precisely because this kind of federal-compliance and secure-controls work is done mostly inside companies and almost never discussed openly at open source conferences. It was a rare, valuable window into how the other half builds.

Bringing Rust into CPython is a technical and social marathon

Emma Smith's talk proposed rewriting parts of CPython in Rust for memory safety, starting small with a single standard library module and holding out the long-term possibility of more, potentially even parts of the language like the just-in-time compiler. The team's first target is zlib, a well-understood compression library that processes a lot of untrusted data, which makes it an ideal candidate for the memory-safety benefits of Rust. Both Juanita and Seth stressed that the hardest part is not the code, it is the people: CPython's maintainers are C developers who did not necessarily sign up to learn or maintain Rust, and open source moves by consensus, not decree. Juanita praised Emma for addressing that organizing-and-agreement challenge head-on, and noted a highlight of having Guido van Rossum in the audience asking Emma questions. Seth cautioned that rewriting in Rust is often treated as a punchline when it is genuinely difficult, and said the effort will be a long road, with more discussion coming at the EuroPython language summit.

SBOMs are only as good as they are accurate, and SBOMit aims to fix that

The "Asleep at the Wheel" talk by Sanchit Sahay and Abhishek Reddypalle tackled software bills of materials, the SBOM, which is a manifest of everything that went into a piece of software. Their key message was that many current SBOM tools are simply not accurate enough to be trusted for security decisions, and that closing that gap is real, unfinished work. Their project, SBOMit, records what actually happens during a build, including network calls and command invocations, and captures it as a verifiable record using in-toto (the "it" in the name), producing a far more trustworthy bill of materials. Juanita was proud that these student researchers connected their academic work to the real community by making the project part of the broader OpenSSF ecosystem so it is usable, not just publishable, and she noted they were brave enough to run a live demo that actually worked. Seth said the talk finally showed him how SBOMit works end to end, and he immediately imagined using it in a future PyPI build farm or recommended builder image.

Runtime SBOMs from memory ask a better question: is the vulnerable code even reachable?

Hala Ali and Andrew Case presented a deep dive into generating an SBOM from a live Python program's memory, going far into how modules are loaded and functions are called. Instead of trusting only what you declared as inputs, you inspect what is actually loaded and running, which can reveal an unexpected library or catch code an attacker slipped onto the machine. Seth was in awe of how deep it went, because it enables runtime analysis that shows whether a CVE's affected code path is even being used, without reading the source by hand. This lands on a real developer frustration Michael raised: most people never run an audit, and even when a tool flags a CVE, you still have to judge whether it matters, like a Windows-only, untrusted-input issue in a Linux Docker deployment. Mike noted a parallel effort in the open source vulnerability world to surface affected code paths so audits can be automatic, while cautioning that Python's dynamism, where you can import things without an import statement, makes true reachability genuinely hard to answer.

GitHub Actions are a sharp knife, and zizmor helps you not cut yourself

Andrew Nesbitt, the author behind the ecosyste.ms dataset, analyzed a huge number of PyPI-linked repositories for GitHub Actions security problems using zizmor, a static analyzer for workflow files. Mike's one-word review was "terrifying," because of how much zizmor surfaces, from easy-to-fix findings to serious injection risks. Michael and Mike zeroed in on the notorious pull_request_target trigger, which runs on forks with elevated privileges and enables attacks like cache poisoning, where an outside pull request with no rights can slip a malicious input into a trusted build. Mike's metaphor was that pull_request_target is a very sharp knife, and if you pick it up the wrong way you cut your hand, so zizmor flags it as probably-misused. The actionable advice is simple: run zizmor on your repository, better yet add it as a GitHub Action so it runs continuously, follow its recommendations, and move on. It was also Juanita's single recommendation for listeners.

The CVE system was never built for malware

The closing talk, "Breaking Bad (Packages)" by Shelby Cunningham and Madison Ficorilli, examined why traditional vulnerability tracking struggles with supply chain attacks. Seth, who has spent a great deal of time inside the CVE system, framed the core tension: a vulnerability is a flaw in legitimate software, but malware is not flawed software, it is the bad software itself, so treating every malicious package as a CVE could produce millions of records and overwhelm the system. Michael sharpened the distinction with an analogy: a booby-trapped document that turns Microsoft Word into a botnet is a CVE because Word is not meant to be malware, whereas a malware package simply is the threat. The talk walked through watering-hole attacks that pushed malware into open source ecosystems, including PyPI, and how vulnerability records like CVE and OSV responded. Seth's honest conclusion was that the CVE system feels so close to the right tool but not quite, and the community may need to build something new or do more thinking here.

Security is not solved, and the best thing you can do is support the upstreams you depend on

The episode closes on a hopeful, honest note: a packed room and a first-ever track are proof that security is a live, unfinished, exciting field, not a solved problem, and that is exactly why the room was full. Seth's parting recommendation was the most far-reaching: if you depend on Python and third-party packages, look upstream, recognize how much work is being done and how much remains, and support it. That support can be engineering time, such as joining a security response team or making security-focused contributions to projects you rely on, or it can be funding, such as donating to the Python Software Foundation, the nonprofit home for both Mike's and Seth's jobs. His blunt reminder is that security is not free, it costs real time to build the changes everyone benefits from, and it largely will not happen without internal advocates convincing their companies to fund the open source they depend on. It is a call to move from consuming open source to sustaining it.

Interesting Quotes and Stories

"It's shown that if you do pay for a focused security effort, you do get good security outcomes. And I think that's the takeaway for me and for anybody: if you pay for it, they will come." -- Mike Fiedler

"We had the plane, but now we're adding better sensors, better steering, better throttles, better toggles while we're still flying, and that flight is accelerating." -- Mike Fiedler

"We have to defend 100% of the time. They only have to be right once or twice." -- Mike Fiedler

"I have no qualms for the humans that fell for this. They did what they were supposed to do." -- Mike Fiedler

"You're only as strong as the weakest link in your chain. And we all depend on something and somebody else and somewhere." -- Mike Fiedler

"One of the nice things about this role being at a nonprofit is that you can be super, super open. You don't have to keep the secrets to yourself about what you've done and how you've seen success." -- Seth Larson

"Security is not free. It costs a lot of time to implement all of these changes that everyone ends up benefiting from anyways." -- Seth Larson

"Security is something that's parallel to everything in Python. If you're a maintainer for a package, having security somewhere on your mind should be a thing." -- Juanita Gomez

"It is a very sharp, sharp knife. And if you pick it up the wrong way, you have now cut your hand." -- Mike Fiedler, on the GitHub Actions pull_request_target trigger

There was also a lighter exchange that captured the mood of the day. When Michael joked that maybe security would just be solved by next year, Mike deadpanned the industry's favorite promise:

"AI will solve all of our security problems. It's what I've been told. It's what I've read." -- Mike Fiedler

The standout story of the episode is the phishing attack itself. The attackers did not bother building a byte-for-byte clone of PyPI. They stood up a transparent proxy in front of the real site at "pypj.org," a lowercase j where an i should be, a swap Mike pointed out is nearly impossible to catch by hovering over a link on your phone. They emailed maintainers using the very same kind of legitimate request PyPI sends, copied from PyPI's own open source email templates. Out of more than a million accounts, only four people logged in and only one package was confirmed compromised, but that package was a quiet sub-dependency of Hugging Face Transformers, so it had the reach to affect an enormous number of downstream users.

Another memorable thread was the human infrastructure behind the takedowns. Mike gave a heartfelt shout-out to Kamil, a volunteer reporter in Austria whose tooling often catches malicious uploads within about thirty minutes of publish, with essentially zero false positives, doing it all as unpaid community work. And Juanita shared a personal highlight from Emma Smith's Rust talk: the creator of Python, Guido van Rossum, sitting in the audience and asking the speaker questions, which she summed up as, isn't it cool that the creator of Python is asking you about your talk. On a lighter note, Michael's first act at PyCon in Long Beach was to rent a scooter and ride around the town and the beach for an hour just to get his bearings before check-in.

Key Definitions and Terms

  • Supply chain attack: Compromising software by targeting something it depends on, rather than the software itself, so the malicious code flows downstream to everyone who installs it.
  • Typosquatting: Registering a look-alike name or domain, such as pypj.org for pypi.org, to trick people who do not notice the difference.
  • TOTP (time-based one-time password): The rotating six-digit code from an authenticator app. It is a real second factor but is not phishing-resistant, because the code can be relayed to a fake site in real time. See datatracker.ietf.org/doc/html/rfc6238.
  • WebAuthn and passkeys: A public-key authentication standard that binds your credential to the real site's address, making it phishing-resistant. Hardware keys like YubiKey and modern passkeys use it. See w3.org/TR/webauthn-3.
  • Session cookie and API token: After you log in, a session cookie keeps you authenticated for hours, and an API token is a long-lived credential for automated publishing. Both can outlive the login step, which is why stealing them is so valuable to an attacker.
  • Trusted publishing: Publishing to PyPI using short-lived, scoped identity tokens exchanged via OpenID Connect from a CI system, instead of long-lived API tokens. See docs.pypi.org/trusted-publishers.
  • Dependency cooldown: A policy of not installing a package version until it has been public for a set window, so scanners and reporters have time to catch malicious releases first. See cooldowns.dev.
  • SBOM (software bill of materials): A manifest of all the components that went into a piece of software, used to understand and audit what you are actually running.
  • in-toto: A framework for cryptographically attesting to the steps of a software build so the resulting record can be verified, and the basis for SBOMit. See in-toto.io.
  • CVE and CNA: A CVE is a numbered public record of a specific vulnerability, and a CVE Numbering Authority (CNA) is an organization authorized to assign them. The Python Software Foundation is a CNA for Python and pip. See cve.org.
  • OSV: The Open Source Vulnerabilities database and schema, a machine-readable way to describe and distribute vulnerability data across ecosystems. See osv.dev.
  • Zero trust and identity-per-transaction: A model that assumes no standing trust and issues a fresh, narrowly scoped, short-lived credential for each individual action instead of holding one long-lived key.
  • pull_request_target: A GitHub Actions trigger that runs with elevated privileges and on forks, powerful but easy to misuse in ways that expose your repository to attack.

Learning Resources

If this episode sparked your interest, here are a few Talk Python Training courses to go deeper on the security, dependency, and packaging topics the guests covered. If you are newer to Python, start with the beginner course and build up from there.

  • Python Web Security: OWASP Top 10 with Agentic AI: Work through the entire OWASP Top 10 with real vulnerable-and-fixed examples in Flask, Django, and FastAPI, and build a reusable AI security agent plus a modern toolbox including pip-audit and Have I Been Pwned. The most direct follow-on to this episode's themes.
  • Managing Python Dependencies: Master pip, virtual environments, and dependency management best practices, which is the foundation for understanding pinning, cooldowns, and supply chain hygiene discussed throughout the show.
  • Python for Absolute Beginners: If terms like pip install and PyPI are new to you, this course starts at the very beginning and builds the core Python foundation you need before diving into security topics.

Overall Takeaway

The story of this episode is that Python security grew up in public. Giving security its own packed, day-long track at PyCon US, standing shoulder to shoulder with AI, is a signal that the community now treats it as core infrastructure rather than an afterthought, the vegetables finally making it onto the plate. What makes the conversation so encouraging is its honesty: a clever phishing proxy beat ordinary two-factor authentication, the CVE system was never designed for malware, SBOMs are not yet accurate enough, and even a smart idea like dependency cooldowns collides with real-world edge cases. None of that is cause for despair. It is a map of where the work is, and it is being done by a small, dedicated group of paid engineers and a much larger community of volunteers who scan, report, and contribute.

The practical throughline is that you do not have to be a security expert to make yourself and everyone downstream safer this week. Turn on phishing-resistant authentication, adopt trusted publishing and delete your old tokens, run zizmor on your repositories, and consider a dependency cooldown so you quietly benefit from the community's scanning. And if Python and open source create value for you or your company, close the loop by supporting the upstreams you depend on with time or money, because security is not free and it will not sustain itself. The room was full not because security is solved, but because it is not, and that is the most hopeful thing about it.

Links from the show

Guests
Juanita Gomez: linkedin.com
Mike Fiedler: miketheman.dev
Seth Michael Larson: sethmlarson.dev

Trailblazing Python Security: us.pycon.org
Everything Security at PyCon US 2026 (PSF blog): pyfound.blogspot.com
Dependency Cooldowns: cooldowns.dev
Anatomy of a Phishing Campaign (Mike Fiedler) Recording: www.youtube.com
FedRAMP: www.gsa.gov
Zero Trust in 200ms: Implementing Identity-Per-Transaction with Python & Serverless-Tristan McKinnon: www.youtube.com
Rust for CPython project: blog.python.org
pre-PEP: discuss.python.org
Rust for CPython: Making Python Safer and More Robust for Everyone - Emma Smith: www.youtube.com
SBOMit: github.com
Asleep at the Wheel: Getting your SBOMs to pay attention... - Sanchit Sahay, Abhishek Reddypalle: www.youtube.com
Volatility: volatilityfoundation.org
Post Incident Runtime SBOM Generation from Python Memory - Hala Ali: www.youtube.com
zizmor: docs.zizmor.sh
GitHub Actions security in Python packages (Andrew Nesbitt write-up): nesbitt.io
andrew/pycon: data & analysis for the GitHub Actions security talk: github.com
GitHub Actions Security in Python Packages - Andrew Nesbitt: www.youtube.com
gh-profiler: examine a GitHub user's profile to gauge their contributions: github.com
PyCon US YouTube channel: www.youtube.com
SBOMit: adding verification to SBOMs (OpenSSF): openssf.org
Ecosystems: ecosyste.ms

Watch this episode on YouTube: youtube.com
Episode #557 deep-dive: talkpython.fm/557
Episode transcripts: talkpython.fm

Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm
Mastodon: @talkpython@fosstodon.org
X.com: @talkpython

Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @mkennedy@fosstodon.org
Michael on X.com: @mkennedy

Want to go deeper on Python?

Take the next step beyond the episode with our Python courses at Talk Python Training: dozens of hands-on, self-paced courses taught by many of the same people you hear on the show.

Episode Transcript

Collapse transcript
WebVTT format On GitHub

00:00 Security has always been the vegetables of software.

00:03 Everyone agrees it matters, and somehow it never quite makes it onto the plate.

00:07 At PyCon US this year, that changed.

00:09 For the first time ever, security got its own dedicated day-long track, one of just two at the whole conference sitting right next to AI.

00:17 And the room was packed to the back wall.

00:20 On this episode, I'm joined by three people at the center of it.

00:23 Seth Larson, security developer in residence at the Python Software Foundation,

00:27 and very recently a CPython core developer, Juanita Gomez, a PhD researcher at UC Santa Cruz

00:34 in open source security who co-chaired the track and Mike Fiedler, PyPI's safety and security engineer,

00:41 one of the very few people paid full-time to keep the packages you install safe.

00:46 We use the arc of the track's talks to take the temperature of Python security right now.

00:51 Supply chain attacks, dependency cool-down, zero trust, S-bombs, and the push to bring Rust into CPython.

00:57 and why not one of us think security is anywhere close to solved.

01:01 And it turns out that's why the room was full.

01:04 This is Talk Python To Me, episode 556, recorded July 6th, 2026.

01:11 Talk Python To Me, yeah, we ready to roll.

01:14 Upgrading the code, no fear of getting old.

01:17 Async in the air, new frameworks in sight.

01:20 Geeky rap on deck.

01:21 Quarth crew, it's time to unite.

01:23 We started in Pyramid, cruising old school lanes.

01:26 Had that stable base, yes sir.

01:27 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists.

01:32 This is your host, Michael Kennedy.

01:34 I'm a PSF fellow who's been coding for over 25 years.

01:38 Let's connect on social media.

01:40 You'll find me and Talk Python on Mastodon, Bluesky, and X.

01:43 The social links are all in your show notes.

01:46 You can find over 10 years of past episodes at talkpython.fm.

01:49 And if you want to be part of the show, you can join our recording live streams.

01:53 That's right.

01:54 We live stream the raw uncut version of each episode on YouTube.

01:57 Just visit talkpython.fm/youtube to see the schedule of upcoming events.

02:02 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording.

02:07 This episode is brought to you by Sentry.

02:09 Don't let those errors go unnoticed.

02:10 Use Sentry like we do here at Talk Python.

02:12 Sign up at talkpython.fm/sentry.

02:16 Talk Python and Python Bytes both now have MCP servers.

02:20 Point your AI at 10 plus years of Python episodes, transcripts, and show notes.

02:24 Free.

02:25 Click MCP in the nav at talkpython.fm and at pythonbytes.fm.

02:31 Juanita, Mike, Seth, welcome all of you to Talk Python.

02:35 Hi.

02:36 Hello.

02:37 Awesome to have some of you back, some of you here for the first time.

02:41 Really, really great.

02:42 So we're going to talk about Python security.

02:46 And I believe, Seth, I saw that you wrote up a post article just thinking back on this kind of unique Python security track and this extra focus at PyCon US this year.

02:58 I thought, hey, that'd be great to just get everyone involved together, at least everyone involved in the track, not all eight, seven or eight speakers, whatever that number adds up to be many, many speakers.

03:08 And just kind of talk broadly about Python security and use this arc of the topics and talks covered at PyCon and the security track as maybe a backdrop.

03:19 So welcome, welcome.

03:21 Now, before we get into that exactly, let's just do quick introductions about who you are and how you're involved in stuff.

03:30 Juanita, you want to kick us off?

03:31 Sure.

03:32 My name is Juanita.

03:33 I am a PhD researcher right now at UC Santa Cruz.

03:37 I work with the hospital at my university doing research in open source security and sustainability.

03:42 And I've been involved in like the Python and the Sign2Pay Python community for a while.

03:46 So, yeah, I got invited to coaching on the track and that was very exciting.

03:51 Very cool. Where are you in your PhD? Are you past qualifying exams?

03:56 I'm graduating in August.

03:58 What? Oh, my God. That's awesome. Congratulations. You're far past it.

04:02 So what's the plans after?

04:05 I just got a job. I can't talk about it yet, but I just got a job. But I'm going to industry. I'm going to keep working on security and sustainability in open source mostly. But yeah, an industry.

04:18 Cool. Congratulations on both finishing the PhD and getting a job straight away. That's super cool.

04:23 Thank you.

04:24 Yeah. Mike, welcome back.

04:25 Hey, glad to be here. Mike Fiedler, PyPI Safety and Security Engineer, working for the Python

04:34 Software Foundation based out of New York City. I'm coming up on my third year here at the PSF,

04:40 which is a full milestone that I will be sharing with my compatriot here, Seth.

04:47 Very cool. And I know thanks for all the work. It's been, I think, pretty impactful in three

04:53 years already. Yeah, it's been kind of a wild ride of kind of incorporating this because I was a

05:00 volunteer PyPI admin and contributor long before this job opened up. And having the ability to sit

05:08 here and think about and work on security features and stuff for PyPI full-time has been a big eye

05:16 opener as to kind of where the gaps were and kind of where they remain, but also kind of what are

05:23 the very important things that we can do in the interim immediately while building up the kind of

05:29 the piece by piece bits that we need to get to these longer term ideas, but being able to kind

05:36 of still make progress while we're getting there. Again, it's that notion of like rebuilding the

05:41 plane while you're flying it. Like, okay, well, we had the plane, but now we're like adding better

05:45 sensors better kind of steering better throttles better better toggles while we're still flying

05:52 and that flight is accelerating yeah 100 and seat belts i think seat belts belong in the security

05:58 airplane analogy uh probably but again it's arguable sometimes seat belts are a problem

06:04 so like yeah it becomes a very good discussion of trade-offs and balances and kind of what what

06:10 actually will move a needle versus this is a nice to have uh you know ashtray yeah ashtrays oh my

06:16 gosh i remember i remember when there are ashtrays on airplanes yeah that was crazy i'm like why is

06:23 there still an ashtray on this airplane like i know they're not gonna let anyone use it fun fact

06:27 they still have them in airplane bathrooms because while it is against the law they still want you to

06:33 have a safe place to throw an ash your cigarette but if you're gonna violate the law so that way

06:38 you don't throw it into the trash can and set something on fire.

06:41 Seems reasonable. Seems reasonable. So as a way of a segue, Mike, I'll have you take this first,

06:48 and then Seth, you can pick this up. You pointed out being full-time on this project versus a bunch

06:54 of kind of contributors who get excited about part of it, but then there's this whole herding

06:59 cats across thousands, hundreds of thousands of projects and users and contributors. And

07:08 role to this versus just like a core dev type of situation, you know? Yeah, I think there's kind of

07:14 two facets here, right? And there is very much a trade-off, right? When it's full-time volunteers,

07:21 you get what you pay for, which is nothing, right? You haven't paid for anything. Volunteers are

07:25 going to do what volunteers want to do, and they're going to do their best effort. When you hire

07:29 somebody, then you set expectations and you try to make sure that you meet them, right? The double-edged

07:36 sword there is that by hiring full-time employees, volunteers often will take a step off because it's

07:42 like, okay, I don't need to volunteer as much as I used to because now someone's getting paid for

07:48 to do it. So it's both a good thing because it highlights and focuses effort on areas that need

07:55 kind of sanitization and focused effort to make progress, but it also gives volunteers a much

08:02 needed break, which then in turn puts more pressure on the full-time employees to execute,

08:09 which again, not enough capacity, not enough time. So we're doing our best here, but it's shown that

08:16 if you do pay for a focused security effort, you do get good security outcome. And I think that's

08:23 the takeaway for me and for anybody is if you pay for it, they will cut. Fair. Seth, welcome back.

08:30 Hey, Michael.

08:32 So quick intro for people who haven't listened to your episodes and then maybe your thoughts on this full-time thing.

08:38 Yeah, so I, everyone, Seth Larson, a security developer in residence at the Python Software Foundation.

08:44 I work primarily on kind of like everything that is not IPI backend.

08:49 So that's the Python language, the runtime, the security response team for Python and pip,

08:56 packaging tools and standards, and then kind of like how Curie World interacts with Python and vice versa.

09:01 Right. So like CBE and all that.

09:03 And recently a core developer, very recently.

09:06 So to answer the how does this role and all that interacting with volunteers,

09:13 My the way that I've tried to like because my space is like so large, my area of where I'm trying to focus.

09:21 What I end up doing is I try to find projects that are obviously like high impact, all of that.

09:27 Right. Like you don't want to be doing things. You don't want to be nibbling or what is the there's a snacking.

09:31 That's the phrase for it. You don't want to be snacking, but you also don't want to be doing things that volunteers could technically do.

09:39 Right. So if it's a job that is or a task or a project that a volunteer could do, maybe I will like pause and wait on that because I'll leave space for someone to potentially contribute there.

09:51 And then the things that are less likely to get volunteer contributions, things like spinning up a CBE numbering authority for the Python Software Foundation or being an admin on the PSRT where or the Python security response team where you're doing things like, OK, we're going to like be monitoring for spam being sent to our email thread.

10:08 Like volunteers maybe don't want to do that, but maybe they would want to be on the security response team contributing to coordinating vulnerability reports and then getting like acknowledged for their work.

10:19 Right. Like in the in the NCD record and advisory.

10:22 So that's kind of like how I think about it, because the area is so wide.

10:25 I try to make sure that I leave space for other people to contribute just in case that people do get around to it before I end up getting to it.

10:32 So we want to build the community volunteer muscle also, not just have it be on work.

10:38 Yeah, that's really cool. How much do you focus on popular third party projects like Django? I mean, Django already has full time people above Flask or something like that versus just pure Python?

10:50 Yeah, so I would say my time doesn't necessarily always like land directly on the project, but I do spend time responding to maintainers if they have questions.

11:02 So like, for example, David from Flask, him and I have talked many times about how to do like a security response program or a threat model or, you know, hey, I'm seeing this.

11:11 Like, can you help? Like, what what should I do in this situation?

11:14 Like, we've talked about that and I've talked about it with many other maintainers, not just Flask, just to try to give them like advice.

11:19 But then also whenever, like one of the nice things about this role being at like a nonprofit is that you can be super, super open.

11:28 You don't have to like keep the secrets to yourself about like what you've done and like how you've seen success.

11:33 So like we publish everything that we do and like where we see success and people in theory can just like take that copy paste it into their projects or their processes and have a better time than they were before.

11:45 So people can learn from each other.

11:47 I really do appreciate that part of the work.

11:49 That is a bit of a unique aspect of doing an open source security thing as opposed to,

11:53 I can't talk about it.

11:54 This is, no, we don't talk about anything.

11:56 So very interesting.

11:57 Let's talk about this track.

11:59 So there's this trailblazing Python security that talks about, for the first time ever,

12:05 a day-long track devoted to the latest in Python security.

12:09 And if you pull up the schedule of talks, I believe it was Saturday was the track.

12:15 And that was, there was just a whole room.

12:17 And what I also liked, it was a room, wasn't it?

12:19 That was like a fixed place, so you didn't have to keep moving.

12:22 Yeah.

12:22 So Juanita, you want to kick us off and talking about why this, why do you guys focus on this

12:28 this year?

12:28 And, you know, why did you decide it's going to take a specific slot in?

12:32 There's only two dedicated slots, security and AI.

12:36 And I think that's very fitting.

12:37 But tell us about this track.

12:38 Okay.

12:39 Well, I will let Seth talk about how did the idea come to place because I was not part

12:44 of that.

12:45 but I'm going to talk about why I think it's important and why it was important in the conference.

12:49 I think security is something that's parallel to everything in Python.

12:53 I think that if you're a maintainer for a package, I think having security somewhere on your mind should be a thing.

13:00 You should be thinking about security when you're developing a thing.

13:03 And I think there's a big gap in maybe the time that maintainers spend thinking about security or even the expertise.

13:13 And so I think this track was like a big opportunity for people that work in PyVon as maintainers or as users to just learn about some important security things when thinking about like their development processes.

13:27 I thought it was great that it was like a dedicated track because I think it highlights the importance of security.

13:33 Like it's not a bunch of talks that are kind of like in the schedule that people just kind of like figure out they want to go to.

13:38 But it's like we like made it a thing by making it like a full track.

13:42 And I think that elevates the importance of security within Python, which I think it's not necessarily always the case.

13:50 I think every time I talk to Mike, I'm like, why are you the only getting paid for doing Python security?

13:56 I think increasing the awareness of the importance of security is very important.

14:02 And I think that the track, I think, was meant to do that.

14:05 How much do you think what were in those seven talks is representative of the broader security

14:13 space, you know, getting your PhD, studying the stuff like you get maybe a little more

14:17 than just Python view?

14:19 Like, what did it capture?

14:21 And what did it miss?

14:21 I guess.

14:22 I think, honestly, we made it a track that was diverse enough that we could capture like

14:28 very different things within the security brother, like image.

14:33 And I also think that a lot of the things that we talked about are not even necessarily Python specific.

14:39 Like you should care about credentials.

14:41 You should care about phishing.

14:43 Like all of these things are things that you should care about, whether you do Python or C or any other like language development.

14:51 I think that something that maybe I was missing a little bit after I like saw maybe after the conference,

14:59 I was thinking about, I would have liked maybe like an intro version of security, something.

15:07 I think something that I work with in my research is actually like security practices,

15:13 mostly for like starting projects.

15:15 I work with like literally university projects.

15:19 And like the goal of my OSP is to improve the practices of the university projects.

15:23 And some of them are maybe not as advanced to take on some of the, like I would say more advanced security practices.

15:30 And so maybe something like that is something that maybe we did not capture as much,

15:35 like basic things, like literally just going to GitHub and like switching some switches

15:41 on the configurations of like your repository.

15:44 Some of those things maybe were missing.

15:46 And I think there's value on that for people that are like literally nowhere in the security like path.

15:52 But apart from that, I think the diversity was really good on the track.

15:55 I love it.

15:56 I think that'd be a great talk to kick off the track and like security 101.

16:00 Let's not get hacked the first week or something like that, you know?

16:04 Maybe I'll do it next year.

16:05 Yes, let's do it.

16:06 I would love to see that.

16:08 Like, for example, maybe just add a little bit of rate limiting and some logging.

16:13 So if something does happen, you could actually tell.

16:15 And, you know, these kinds of things, right?

16:17 It would be, you know, there's a whole list of that stuff, right?

16:19 Another idea that like came up in when we were thinking about how the track went and how the themes.

16:25 this is this is funny to his idea so i you're gonna have to correct me but uh the idea of like

16:30 having a little bit more interactivity uh to kind of shake things up as opposed to it being because

16:36 i mean with security with these talks maybe you're expecting it to i mean it's obviously like more

16:40 experts based but that ends up if you're in a talk format it's mostly talking to people but having

16:45 something like a panel where you start off with like a few questions to kind of set the theme

16:49 and then you kind of open it up to kind of get some vibes from the audience uh about like what

16:54 questions they're having, make it more interactive in that way. That seemed really interesting to me

16:59 as well. So we're going to see if we can experiment with that next year.

17:03 So am I hearing you say there's going to be a one next year?

17:06 We're hoping for it, for sure.

17:08 We think that we should have another one next year. I mean, there was no indication that we shouldn't do this again, is what we saw.

17:16 Honestly, I mean, security might be solved next year.

17:19 Oh, yeah.

17:19 That'd be great, wouldn't it?

17:20 For sure.

17:21 AI will solve all of our security problems.

17:23 It's what I've been told, what I've read.

17:25 Mike, you're trying to take us down a path that is a very different path.

17:28 Well, I mean, it's been a few minutes and we haven't said AI yet.

17:32 Well, no, you actually did because you introduced the other tracks.

17:35 Yeah.

17:36 Well, we only mentioned it in passing, but not with regard to this topic.

17:39 I do think actually maybe if we get some time, I do want to talk to you about it.

17:43 There's a lot to cover.

17:45 There is so much.

17:46 Maybe this could just be its own show that we could go into because there is a lot to cover.

17:50 I think there's like positives and negatives, and that kind of sums up AI right there, doesn't it?

17:55 It sums up everything, honestly.

17:56 It sums up a lot of life, yes.

18:00 Hey, one second.

18:01 Normally, this would be an ad break from Sentry, but not this time.

18:05 Let's just thank them for supporting the show and get right back to the conversation.

18:09 Also, visit talkpython.fm/sentry after the show.

18:13 Thanks, Sentry.

18:15 It's a lot of gray.

18:16 Yeah, not so much black and white, a lot of gray.

18:19 So it was like standing room only?

18:21 I mean, that sounds like that bodes well for our next year.

18:25 Yeah, next year, hopefully.

18:26 I mean, it'd be really great to even have a bigger room.

18:29 Or, yeah, we were quite happy with the turnout.

18:32 We had a few talks that basically it was standing room and there was nowhere else to come into the room.

18:37 So that's always great to see as an organizer for a first-time event is when the room is too small.

18:42 It means you're doing something right.

18:44 Exactly.

18:45 You've overachieved compared to what you were trying to work with or whatever.

18:49 Yeah, I guess just one in the test, what was your experience just co-sharing a track and so on?

18:57 What was it like being on that side of the conference?

19:00 I think it was, I thought it was great because I think, so I've actually been organizing SyPyre also for a couple of years.

19:08 And for the longest time, I've actually tried to bring security as a conversation at SyPyre.

19:13 So I've learned a couple of like reds of a feather session, but having a security track seems a little impossible.

19:19 I don't know, like it's so I don't know, like making making this happen at PyCon was it made me really, really happy because it gets you the opportunity to bring people to talk about the things that you really care about.

19:31 Like security in Python is really like personal to me.

19:35 I care a lot about it.

19:36 And I think part of the problem is the lack of awareness of some of the things that we discussed at the shark.

19:42 So I thought it was a great opportunity to just kind of bring that awareness into the table and making it happen.

19:48 Just like being part of that conversation.

19:51 I felt really proud of doing that.

19:53 And I think also, I think like Seth and I and other couple of people we had, I mean, I had a lot of fun reviewing some of the talks that people submitted.

20:02 We were unfortunately like limited to like eight slots.

20:06 But there was a lot of things that I learned by just reading some of the talk proposals.

20:11 So it was like, I think it was also like a learning experience for me to be part of the truck.

20:16 Yeah, I was going to say, compared to Juanita, my experience with organizing sessions and talks and conferences, this is like the very first time that I've like really dipped my toe into organizing at this level for a conference.

20:30 So it was a lot of learning for me.

20:32 And I thought it was a great experience.

20:34 I think other people could definitely like would enjoy doing this too.

20:39 So now I'm like just understanding and learning all of these things that are about how a conference track or a talk schedule is put together.

20:46 And for me, like the day of stuff was like so, so much.

20:52 It's you're running around and it was like very on the fly.

20:56 And, you know, they have all the recommendations that once the ball is rolling, it's like it's rolling.

21:01 And it was really, really fun.

21:02 I don't know. We had a lot of fun up there.

21:03 I think that it was a great session.

21:05 The energy was so positive.

21:07 People are always coming up to you afterwards and saying, oh, that was amazing.

21:11 Thank you for doing this, all of that.

21:12 And it was a really great experience.

21:15 Cool.

21:15 Yeah, PyCon was really fun this year, too.

21:17 I thought Long Beach was a pretty nice venue.

21:19 There's some nice food carts and other things to do there.

21:22 So that was really looking forward to back then.

21:25 Yeah, now that we know the cool little bar open, you know, rooftop bars or outside bars,

21:30 we can just go meet up and hang out with friends or the cool restaurant you want to go back to or whatever.

21:34 Yeah.

21:35 My first event there was to rent one of those like Lime or Bird scooters and just spend an hour riding around the town and the beach just to like get used to it and then going to check in.

21:45 So very fun.

21:46 Let's bring it over to the conference with the first of seven, I think we have seven, six or seven tracks that I put up for us to kind of riff on here.

21:56 And that would be by none other than Mike Fiedler, an anatomy of a fishing campaign.

22:01 What is this?

22:01 Oh, boy.

22:02 What is this?

22:03 So this was the first track of the day, wasn't it?

22:07 Yeah.

22:08 I'll tell you what this is.

22:09 This is why we can't have nice things, Mike.

22:11 This is why we can't have nice things.

22:13 It really is true, right?

22:15 Like there's, for me, it was a good opportunity to kind of do a little storytelling education

22:22 because these are cases that happen, right?

22:26 And this was a novel enough one that I was like, this deserves some disambiguation because

22:32 people hear phishing attack is like, oh yeah, yeah, yeah, yeah. This was different, right?

22:36 And I was very kind of excited to speak and be selected for the spot. And you mentioned standing

22:45 room only. Standing up there on the stage and looking out into a large room and seeing people

22:49 standing at the back wall and not seeing little dotted seats available throughout the room was a

22:56 little intimidating because it's like, you know, sometimes people don't sit in all the seats and

23:00 That's fine. Or like the first two rows are empty. But no, Juanita and Seth had basically packed this room with people, not intentionally, but purely purely by curating the timeline and like setting the tone of like, yeah, this is this is going to be a story.

23:16 And I was able to relate my personal firsthand experience of dealing with a phishing campaign that some of our users were tricked into visiting a website that was basically a proxy for PyPI.

23:34 In previous phishing campaign attempts, a lot of folks talk about building a byte for byte or pixel perfect version of a different website and steering traffic to that.

23:44 Here, they didn't even need to do that. They just threw up another proxy in front of it. And it was, you know, using email addresses that are like publicly available that maintainers put on their GitHub profiles and put in their, you know, Python package metadata. And they farmed out and emailed some unknowable number of maintainers.

24:04 What an interesting security issue, right?

24:07 Like you've got to be pretty sure that the login people use for their GitHub is probably the same email address that they've registered at PyPI.

24:15 And the packages that you want to take over from PyPI linked to their GitHub repo, right?

24:22 It's a really, it really does condense what you've got to do.

24:25 A little bit, right?

24:26 Like so in a lot of phishing campaigns, it's kind of a spray and pray, right?

24:30 They're casting a very wide net and they're going to email 2 million people.

24:34 And if they get two, that's a success for them, right?

24:37 Because we have to defend 100% of the time.

24:41 They only have to be right once or twice, right?

24:44 And they were in this case.

24:47 Again, PyPI is open source, right?

24:50 You can read on the entire code base.

24:52 You can see what our email templates look like.

24:55 And you can see that, yeah, we do ask maintainers now and then to do an action.

25:00 And they emailed maintainers and asked them to do an action.

25:03 And these maintainers did an action.

25:05 They did what they were asked to do.

25:08 For example, you emailed people a couple of years ago and said, hey, you need to turn on 2FA.

25:12 Yeah.

25:13 And that's totally legitimate.

25:16 And sometimes we will email people and say, hey, there's a problem with your metadata.

25:19 Go in and fix it.

25:20 We will occasionally tell people there's a problem.

25:24 This one in particular would tell them it wasn't using our email infrastructure.

25:29 You know, if you had anti-spam things, then like maybe it would have caught it.

25:34 But it didn't because the mail was coming from the domain advertised for this hack.

25:41 And it was basically showing instead of PyPI.org, PyPJ.org.

25:47 And I would challenge you at, you know, looking at an email on your phone to hover on a link and see that difference between an I and a J in the lowercase Arial font on your phone.

25:58 Like, you're not going to notice it, right?

26:00 Humans, I have no qualms for the humans that fell for this, right?

26:04 Like, they did what they were supposed to do.

26:06 What was annoying about this attack is we've been talking about 2FA for a very long time now,

26:13 and this is still phishing.

26:17 You know, you can still get phished with certain forms of two-factor authentication,

26:21 notably the time-based one-time password that were TOTP or what folks might think of your, like, QR code.

26:28 or Google Authenticator app, all of those things are the same thing.

26:32 Whereas if you used WebAuthn or the underlying technology for pass keys or like a YubiKey,

26:39 that's requiring a different sort of 2FA that is more phishing resistant.

26:45 So anybody who had the phishing resistant part was not subject to this attack.

26:50 And then I think it might be worth maybe just a quick conversation on that

26:55 because I totally agree because you could just proxy, they send over the thing,

26:59 and you've got 30, 15, whatever seconds to then, you know, curry those credentials over,

27:05 and you probably are fine to hack through with a proxy, right?

27:08 I mean, I think it's, I think the TOTP ones are more, like if somebody gets a hold of your credentials and they're leaked, right,

27:15 that's not enough to get in anymore, right?

27:19 Or if they're trying to brute force something, that can be a big challenge to limit that.

27:23 But for phishing, it really doesn't add a lot of safety, does it?

27:27 So it doesn't because the session token that you get once you have completed your username password, then you do your TOTP.

27:36 Now you have a session cookie, right?

27:38 This is what your browser uses instead of asking you for your password and to FA every single page view, right?

27:45 You have a session cookie that is good for, I don't know, eight hours or something like that.

27:50 So you don't have to do that all the time.

27:52 And this session cookie is something that they can use to then, they don't even need the TOTP within those 30 seconds.

27:59 They now have a session cookie and use that to log in as you.

28:03 And then the next part that they did was create an API token, publish.

28:08 API tokens today do not have any expiry.

28:11 So they were like, all right, I'm going to create a new API token.

28:14 And now I don't need your username, your password, your TOTP, the session token.

28:18 I don't need any of that.

28:19 I have an API token.

28:20 I can go ahead and do the things that I want to do on this website with this token.

28:26 So I think there's layers of security.

28:29 Something we added in the aftermath of this was if you now are using a session token and it was obtained via TOTP and it's coming from a different IP address, we're not going to accept that.

28:42 We're going to ask you to recertify that you are who you said you were because something is different.

28:49 But if you use WebOFN, then you're fine.

28:52 The people who will get impacted by this are folks who are like bouncing between VPNs

28:58 or if you're riding a train and you keep changing IP addresses as you kind of go to different cell towers,

29:04 which I don't think happens very often, but it can.

29:07 Those are the people who will get caught in that net.

29:09 But it's like, yeah, you know, we can't tell that you are who you said you were

29:13 based on those credentials.

29:14 Yeah.

29:14 Juanita, Seth, thoughts?

29:16 I want to say that this was one of my favorite talks of the track because it was very relatable.

29:22 Like a lot of the people that are sitting down in the room are maintainers who published their packages in YPI.

29:27 And just having Mike's perspective on like, oh, this can happen to you.

29:31 Like, you need to be aware of this.

29:34 I think it makes it much more impactful because it's like you are directly affected by some of these things.

29:41 And I think also Mike had a really, he not only told us a story about how all of these bad things happened, but also at the end, a call for action.

29:49 This is the things that you need to do in order to avoid these things from happening to you.

29:53 And so I thought that was really impactful for the people that were sitting down in that room.

29:58 Nice.

29:58 So one of the things I think is, I guess, I'm not sure if you mentioned it, but the result was there were some packages that got malicious code in them and then published a PyPI, right?

30:08 Just one that we know of, right?

30:10 We audited kind of a lot of the attacker signatures and kind of their behaviors.

30:14 We're only able to turn up one.

30:16 So only four of the user accounts of the, again, we have over a million user accounts on PyPI right now, but only four people fell for this phishing and like click through and actually logged in.

30:29 And only one package was kind of harvested and republished.

30:35 And again, that doesn't mean that others weren't created, you know, as a backup, but they went after one that is a dependency of a dependency of a dependency, right? They're working their way through the supply chain. They didn't go after anything that, you know, these other four, sorry, the other three maintainers had.

30:53 They went after the one maintainer who kind of fell for the fish.

30:58 And theirs was a dependency of Hugging Faces, Transformers library.

31:04 And if you're doing anything at all in AI land, which a lot of people are doing a lot in AI land, Transformers is kind of key.

31:12 It gets, I don't know, 30, 40 million downloads a day.

31:16 And Numb to Words is a sub-dependency of Transformers.

31:20 So by publishing a new version of Numb to Words, any Transformers user is going to get it.

31:26 Scary.

31:27 And it's, yeah, you know, and you maybe think I'm going to take on this dependency, so you'll review it.

31:32 But how far down the dependency chain do you follow, right?

31:35 I mean, the answer is like, you know, what was that TV show?

31:38 You are the weakest link.

31:39 Goodbye, right?

31:41 You're only as strong as the weakest link in your chain.

31:44 And we all depend on something and somebody else and somewhere, right?

31:48 None of us are writing 100% everything by ourselves based on like a C standards library.

31:54 Like you didn't write your compiler on your own.

31:56 You used somebody else's.

31:57 You didn't write the Python language on your own.

31:59 You used somebody else's.

32:00 And even if you are authoring Python, you're using tools that somebody else wrote, right?

32:05 You didn't write your editor on your own.

32:07 You didn't write your operating system on your own.

32:09 And again, if you are the one person who has done all of this, I commend you.

32:12 I also think you have put in a lot of effort that is commendable, but is unnecessary in

32:17 today's world.

32:18 but you are likely more secure as a result.

32:22 Yes.

32:24 This portion of Talk Python, I mean, is brought to you by our AI tools.

32:28 You know that thing where you ask an AI something about Python and it confidently tells you about the library version

32:34 from 18 months ago?

32:35 Well, we fixed that, at least for our shows.

32:38 Talk Python and Python Bytes both have MCP servers now.

32:42 Connect Talk Python and your AI can search over 550 episodes, full transcripts,

32:48 every guest in the entire course catalog of Talk Python courses, ConnectPython Bytes,

32:52 and he gets almost 500 episodes of Python news going back to 2016, including every link we've ever put in the show notes.

33:00 This means you can say things like, ask Talk Python what astral joining OpenAI means for uv,

33:07 or what has Python Bytes said about Locust, and get a real answer with real links, not a hallucination.

33:15 Name one of our shows in your prompt, and your AI knows exactly where to look.

33:19 And if you live in the terminal, Talk Python now has a CLI too.

33:22 One line, uvtoolinstalltalk-python-cli.

33:27 Then search the episodes, transcripts, guests, and courses without ever even opening a browser.

33:32 It's open source and it outputs text, JSON, or Markdown, so it also feeds AI tools that don't speak MCP.

33:40 And here's the real reason I built it.

33:41 Both shows cover around 10 years of Python history.

33:44 the people, the decisions, the packages that took over, and the ones that quietly didn't.

33:50 This enhanced access to all of our information is free. No account, no API keys, nothing to buy.

33:57 That history contained in these shows should be there for all of us. So visit talkpython.fm

34:02 and pythonbytes.fm and click the MCP link in the nav bar. Connect them right now to your agents

34:07 so that they will be accessible anytime they're needed in the future. Hope you all enjoy the access.

34:13 One thing that came to mind when I was looking at your talk is this whole concept of dependency

34:19 cooldowns, which is making some waves.

34:23 I think I noticed it pretty early in, put this up for everyone, pretty early, I guess

34:28 with uv, and I think pip is getting it as well, but also some of the other package managers.

34:34 But I've seen it sort of picking up steam in other areas unrelated to Python, other package

34:38 managers.

34:39 So I really like this idea.

34:42 I really like it.

34:43 I think cooldowns are a good idea for your average consumer. So in a different attack that Seth and I talk about later, we talk about how the people without using version pinning or deciding I want to use version 1.0.0 only, if you use unbounded versions, you're going to get the latest one that the registry has to offer.

35:09 And when attackers are looking to exploit you, that's kind of one vector they're looking to exploit to say these people are not pinning their versions.

35:20 Dumb to words was not pinned in Transformers.

35:23 So they're going to get whatever the latest one is.

35:26 Now, sub-dependency pinning and version bounds is a whole other set of conversations.

35:31 But as an end consumer, cooldowns is the easiest way for you to say, I don't want what the latest the registry has.

35:39 I want to wait three days or seven days because in the scope of those three to seven days,

35:44 it is very likely that some security scanners will have chewed through this and noticed

35:51 if there's a problem and reported it.

35:53 And if they have, there is a, again, a very high likelihood that that version is no longer

35:58 available for download in that time.

36:00 Yeah.

36:01 So like you look at cool, the thing I'm linking to is cooldowns.dev.

36:05 And in here, it pointed out, said, does it actually work?

36:09 And they gave some stats, you know, they talk about LiteLLM, some of the other issues that have come along.

36:13 They said, I think within a week, I think 90% of the vulnerable ones were taken down and most of them were taken down within a day or something like that, right?

36:24 Yeah. I mean, a lot of that on PyPI is thanks to our group of volunteer reporters.

36:30 And we don't have any kind of commercial relationship.

36:33 There's just people out there in the community who like scanning stuff.

36:36 There are businesses who do it, and some of them will report back to PyPI and say, hey, we found something fishy, something smelly.

36:43 We recommend you take a look.

36:44 Some of these folks have gotten such a very good or almost zero false positive rate that we kind of allow them on PyPI to put that into an automated quarantine status that requires a human like myself to review it and either clear it or tank it completely.

37:05 But in that interim state, it's no longer downloadable.

37:09 So those like our most notable one is Kamil Mankowski, who works out in Austria, and he

37:16 has gotten it down to like his reports are often within, you know, 30 minutes of publish.

37:22 His tooling has scanned it, analyzed it and reported it back with zero false positives.

37:27 And, you know, that only comes with curation and time.

37:30 And he's doing this as a volunteer effort.

37:32 So kudos out to Kamil.

37:34 It's a lot of data to be processing.

37:36 Yeah.

37:36 I wanted to comment on the cooldowns aspect too, like maybe about like the, who is it

37:40 for?

37:40 Cause I think this, this is always a challenge whenever we're building something at the scale

37:45 of Python, right?

37:47 Where it serves so many different communities and people and right.

37:50 There's people that have, you're on a developer team in a really large organization and you

37:54 have your own like security team who is deciding what your security policies are.

37:59 They're running a mirror of PyPI, right?

38:01 Like all this stuff is happening.

38:02 But then Python also needs to serve like students or people who this is their first programming

38:07 language.

38:08 And my thinking is, is that this sort of config where you set it once and then you don't have

38:14 to think about it and you benefit from this large apparatus that already exists out there

38:20 in PyPI, people that are scanning, people that are manually reviewing things, taking

38:24 them down, all of Mike's work, right?

38:25 Like you get to benefit from all of that work without having to have this massive, you know,

38:31 separate team or policy or having to necessarily think about like reviewing every single dependency

38:37 right like this is something that I don't think that a lot of developers are doing especially not

38:41 students right they want to run pip install with the thing that they want to install either for

38:46 their course or because they're learning they're not necessarily thinking about oh this is

38:50 potentially a way that malware is going to be installed on my system so set like a really

38:54 aggressive like seven days or something like that on your cooldown one time and then you're pretty

38:59 okay. Like, I think a lot of people should probably be using cooldowns, especially for like, developer

39:05 machines or a personal machine that you're running Python on where you're not, you don't have this

39:09 giant security apparatus working for you. Yeah, I want to second that and say that a lot of the

39:14 people that use Python packages have no idea of what is PyPI, where they're coming from, what does

39:20 pip install mean? And so having like, and like, like, seriously, but when I started doing all of

39:27 they say, I had no idea.

39:28 I was just running commands that I was reading somewhere in Reddit or somewhere else.

39:32 And I think just having some default ways of keeping security in my own developer environment

39:42 is, I think it's great.

39:44 For the most unexperienced developers, this is definitely more usable.

39:48 Yeah, I think it's a great idea.

39:50 I really like the idea.

39:51 So I adopted this to some degree and I thought it was amazing.

39:56 And then I ran into some problems.

39:58 And I'd love to hear your thoughts on this.

40:00 So I don't have unpinned dependencies.

40:04 But I use uvpipcompile, which will say, with an update, and say, OK, well, I've decided now I'm

40:10 going to release a new version.

40:11 Let's go ahead and do a refresh of the update of the dependencies and recompile.

40:16 And it will repin them to the newest.

40:17 And without this cooldown, it'll pin them to the very latest.

40:20 And so if, for some reason, something were to git pushed into PyPI an hour ago, and I ran that command, well, I might as well have

40:29 unpinned dependencies, right? So I'm going to put uv pip compile dash dash, like wait, whatever the

40:34 command is, exclude newer one week. And I even put it into the Docker build for all of my server

40:40 stuff so that the Docker build would fail if using a combination of pip audit and this. So if there's

40:47 a CVE, it will fail because there's a problem with one of the packages. And then I would use this

40:53 older ones but here's what i ran into is there would be a cve in something and then it would

40:57 it would window into the seven days ago and the fix would be like one day later but it wouldn't

41:03 allow it to install the one without the cve because that was excluding newer and so then i end up with

41:08 all of these like x ignore this one because this one you know what i mean like it's these two things

41:13 clashing it's kind of like one package says version has to be greater than two on a dependency

41:18 and one says version has to be less than two you're like well now what do i do yeah michael you're

41:21 You're touching on the exact reason why I would have a hard time, at least right this moment,

41:27 pushing for dependency cooldowns to be a default.

41:31 It confuses users, especially when there are other systems that are telling them that they need to resolve a problem.

41:39 And when they run the commands necessary that they know work because they've used them before

41:44 to upgrade a dependency to a specific version or whatever, they don't work.

41:47 Or they'll say something like, oh, that version doesn't exist because pip is like literally dropping

41:53 these versions from the candidate pool or something like this, right?

41:56 It'll say that version doesn't exist and you get these really confusing messages.

42:00 I mean, one day I do, I would really like to have something, it doesn't necessarily have to be cooldowns,

42:07 but some sort of system that the default is, if you are not, the way I like to think about it is like,

42:13 if you are not showing through your tool usage, whether or not you have like operational maturity

42:19 is kind of how I think about it, right?

42:21 Like you're not using a mirror, you're installing direct from latest.

42:24 If you're not showing that sort of operational maturity, like asking for a lock or an exact version or whatever,

42:29 then there is some sort of secure default.

42:32 But that secure default policy needs to take into account a lot of things.

42:36 And one of those things is whether a security vulnerability is being fixed

42:41 by something that is within this cooldown window or reviewing window.

42:44 And maybe this can be easier if there's more of this happening the index side. Like right now, IPI is mostly you upload it, it's immediately available, right? And

42:55 the scan comes after the fact, there's no intermediate step where it's like being reviewed

43:00 because we just can't, there's too much volume. And so like, if we get to a world where that is

43:05 happening, maybe tools can implement this sort of policy a little bit more easily, because it'll be

43:10 for the most part on the index side. But from a tool perspective, it's really difficult to get

43:14 a secure default that takes and respects all of these use cases perfectly right as a default.

43:21 It's very challenging.

43:23 So what you're saying is we need to hire a lot more people to scan packages before they

43:29 get published by BI.

43:30 That's what you say, right?

43:32 I'm open to emails.

43:34 Email Seth at Python.org for any interested parties solving this issue.

43:41 Indeed.

43:42 So earlier, Mike, you talked about API keys and how they're kind of outside the bounds of two-factor and so on.

43:49 So the next talk was Zero Trust in 200 Milliseconds, Identity Per Transaction with Tristan and McKinnon.

43:55 And that comes from FediRAMP, which is a U.S. government thing, the Federal Risk and Authorization Management Program that provides approach to security and risk and so on.

44:06 If I recall correctly, this is kind of a zero trust story where it's every single time you need to use an API key, you have to get a new version of an API key.

44:19 Like, I want to talk to AWS.

44:21 Great.

44:21 Here's a key you can use for two seconds or something, right?

44:25 Use it now.

44:27 Yeah.

44:27 What are your thoughts on this?

44:28 What are the takeaways from this one?

44:29 Anyone?

44:31 How do you feel about it?

44:31 So I was going to say, we're all pointing at each other.

44:34 I really love this talk.

44:36 because it talked about a side of Python and like development that a lot of people don't

44:42 necessarily experience, which is where you work in an industry that's like regulated or needs to

44:48 like implement FedRAMP, right? It is a tone of extra work to do this. And a lot of times it's

44:53 not really talked about a lot online because like you said, that's not necessarily like open source

44:59 communities that are doing this sort of work. It's mostly, you know, industry that is doing these

45:04 things. And so it's like really, really great to see this sort of talk where you're talking about

45:08 something like FedRAMP compliance and zero trust in Python with a lot of different like tools that

45:15 people already use. So like using the AWS SDK and how to implement with this with AWS and then

45:21 extrapolating that out to other different like backends. Right. So it's back end agnostic. I really

45:27 enjoyed this talk because of those reasons. Like this is just stuff that you don't necessarily see

45:31 at PyCon or at open source conferences, people talking about federal policy requirements

45:38 and how to implement secure controls like this.

45:41 So really, really interesting talk.

45:42 Yeah, it is a bit of a look in an area that you don't normally see.

45:45 And yeah, it's quite cool.

45:47 This next one, the next one is Rust.

45:51 And this by Emma Smith.

45:53 And this one certainly, it's interesting because you can debate it technically,

45:58 but then you also have to debate it from an organization perspective and so on. And the idea is, what if we could have more of CPython written in Rust, right?

46:09 But a lot of the people who work on CPython, hey, guess what? What is the name there? CPython,

46:16 as in the language, our C developers, and they don't necessarily want to rewrite stuff in Rust,

46:21 and they don't necessarily have an expertise in it, and a whole bunch of things like that. So one,

46:26 thoughts on the talk. Two, thoughts on Rust and Python.

46:29 I want to give my thoughts on the talk. I think I love this talk for two reasons. The first one is it was a talk about Rust in a Python conference, which shows the intersection between these two communities. And I think, I mean, I think Emma did a great job of explaining like the technical implications of the migration and like of why we need to do this.

46:54 But at the same time, I think she was very clear about what this means from like a community perspective. And I think part of the talk was actually talking about how can we actually make this happen? Not from a technical perspective, but like from organizing the people and like actually getting the people to like agree with doing this and like how do we make this happen?

47:16 Which I think is very on point with open source and how open source works.

47:21 You don't just make a decision and take a different path on something that you're doing.

47:27 There's a whole lot of people that are involved in making decisions for a specific community.

47:31 And I think Emma did a great job of illustrating how that specific aspect of open source works with this particular talk.

47:37 So I loved it.

47:38 And one of my highlights was that Wido was there and he ended up asking questions to Emma about this particular talk.

47:46 that was like, isn't it cool that you have the creator of Python asking you questions about your

47:51 talk? That was that was one of my highlights. Yeah, from like a security. So I am very

47:56 intrigued about this idea. I, it makes sense. But it's also like you said, like in a social sense,

48:04 extremely challenging. And also even like a technical sense, it's extremely challenging,

48:09 right? This is this is not an easy project. I think that there's maybe when people think about

48:14 rewriting and rust and all of this stuff, right? It's almost kind of like a joke, right? That you

48:18 can all just rewrite it in rust and oh, rust Python already exists. Why? Why would this be hard? Right?

48:24 Like this already exists out there. And it is extremely difficult. I am really excited to talk

48:31 about this at the language summit at EuroPython is a talk there also. And I think that Emma and

48:37 the whole team that's working on rust for Python have their work cut out for them. It's going to be

48:43 long journey. Talking about like just the technical aspect of this, I think that they

48:48 have made a really great choice in which library they're decided to potentially like attack first

48:55 as like a first go, which is Zlib, which is a compression library that's pretty understood,

49:01 processes a lot of like untrusted data, right? So there's the memory management, memory of security

49:09 vulnerability aspect. Not necessarily that Zlib has had a ton of issues in that area,

49:14 but it's a really good one to attack as a first module. Yeah, I'm going to be

49:21 Give us an idea of exactly what they're proposing here. Is this like, there are portions of the standard library that right now are pure Python or C, and we propose

49:30 rewriting some of them in Rust? So the proposal right now is to rewrite a single standard library

49:36 module in Rust with the long term goal being maybe more and even potentially the like parts

49:43 of the language being rewritten in Rust.

49:44 So like the I believe it was like legit.

49:47 The just in time compiler was another candidate for like a larger project or maybe like a here

49:53 you can cut your teeth on this.

49:54 We won't merge it, but we'll it'll it'll prove out whether or not the tooling that's around

49:59 this like compiling stuff to Rust and having it actually work for Python would work just

50:05 as an option.

50:05 this got discussed a while back too.

50:08 And so this has kind of been ongoing for quite a long time.

50:11 And so I'm like really excited that this talk, this is kind of like the first time that Emma and the team have like poked

50:15 their head out and be like, talk to the wider community about this is happening.

50:19 We're discussing this and what it could mean.

50:21 But yeah, it's going to be a long road ahead.

50:23 I have a feeling.

50:24 So yeah.

50:24 All right.

50:25 Next up, Asleep at the Wheel, S-Bombs for Python Builds.

50:28 Sanchit Seye and Abhishek Reddy Piley.

50:34 Sorry about that.

50:35 Yeah, so SBOMs.

50:37 What are the thoughts on SBOMs, folks?

50:39 We need more of them.

50:40 Yes, and better.

50:41 They need to be accurate.

50:42 Exactly.

50:43 I think, honestly, I was just going to comment really quick that I was very proud of this talk because Abby and Sancheid are both students.

50:53 I think Sancheid is a master's student.

50:55 Abby is a PhD student.

50:57 And as a PhD student, I love to see the research that people are doing in conferences are not academic conferences.

51:05 And I think they have done a really good work on developing their research, but also interacting with the outside community to figure out how to make their tools available and usable.

51:18 I love that they have made their project part of the OpenSSF, like, bigger ecosystem, because that kind of shows how they want to make sure that anything that they're doing is actually usable.

51:31 And yeah, like it's not just a paper about something that's very abstract and not really relevant, but like it's actually something that the community can use and adopt.

51:41 So I'm really proud of this talk.

51:43 Yeah, I was going to add that this talk is the talk that showed me what SBOMIT actually was, which is really, really cool.

51:52 I'd heard of it.

51:53 I kind of knew what it meant, right?

51:56 It's one of these things where there's just so many security tools and approaches and so much area to cover that you kind of like end up knowing about something.

52:06 And, you know, like when you'd maybe reach over there to learn more, but you don't know like how deep the iceberg goes.

52:13 right and how much work had to go into making this basically work like magic to record all of the

52:20 stuff that happened during your build all the network calls all the command invocations and

52:25 how it gets then exported into an s bomb but then also this verifiable record using i believe it's

52:31 called in toto that's what the it is an s vomit um and like how it all works end to end and so it

52:37 just gets my mind thinking about like if we were to release this like either like a build farm for

52:43 PyPI if we ever to work on that project this is the sort of thing that we would like reach for and

52:48 say like okay maybe we could integrate this technology into that or if we were going to

52:52 create a like a recommended builder like a docker image that did building like this would potentially

52:59 be a part of that story where you're recording network calls and then you can get this extremely

53:04 accurate software bill of materials document at the end that shows how a package was built.

53:10 And then you can have that be published with the package or you can use it as a user, however

53:14 you want to use it.

53:15 Like that sort of information is really important.

53:17 And it's really difficult to get after the fact, as opposed to like when it's in flight

53:21 and when the actual programs are doing the installations and the builds and all of that.

53:25 And I think you mentioned...

53:26 Oh, sorry, Godwin.

53:27 I was just going to say that this talk actually did a very good job of making people aware

53:32 that the current tools that generate S-BOMs are not as accurate.

53:36 And there's still a lot of work needed to make S-BOMs actually usable within a security perspective.

53:44 And they did a great job of just making that awareness.

53:48 And they also were brave enough to do a demo, which is like people now don't do demos because they all break,

53:54 but they did a demo and it worked.

53:55 That was really cool.

53:56 Very cool.

53:57 Seth, you theorized there might be some magical process involving cooldowns and pin dependencies and review.

54:04 And maybe SBOMs could be an input to that to say, well, this ain't got a problem,

54:09 but it has some source down in the supply chain that actually we've discovered is problematic.

54:14 So everything downstream from that, we need to check or flag or whatever.

54:19 Yeah, SBOM, the way I think about SBOM is mostly as just this third party.

54:25 It's not a part of an ecosystem, so it works really well for Python.

54:30 because we have so many Python packages out there are not just Python.

54:34 It's like C, Fortran, Rust, JavaScript, all of these different ecosystems, right?

54:40 GPU shaders.

54:41 Yeah, GPU shaders.

54:42 Throw them in there.

54:42 Why not?

54:43 I mean, Mike knows this even more than I do.

54:45 Like there's video.

54:47 It's just all sorts of great, great content on PyPI.

54:51 Yeah, not that we encourage that use case.

54:53 But there's a lot of stuff in Python packages and SBOMs are a great way to represent

54:59 like the provenance of all that software without it being Python specific.

55:04 For sure. All right. Picking up speed because we got a couple more to go through.

55:07 We've got maybe the follow on or the after case here is post incident runtime S-bomb generation

55:15 from Python memory by Hala, Ali and Andrew Case. And the general idea is there's what you've

55:20 declared to be your supply chain inputs and then there's what's actually there. So maybe look at

55:27 what is loaded into memory?

55:28 And you're like, oh, why is this library also loaded?

55:31 Or some other hacker type of thing got put on the machine and then loaded in.

55:36 And now we can detect that through this generation of looking at loaded modules and so on, right?

55:42 Yeah, this one, I was really intrigued about this because it went so deep into Python memory management

55:48 and how modules are loaded and how functions are called like extremely deep dive into this topic.

55:54 And so I was like super I was like in awe watching this talk.

55:58 I was like they just kept on being more cool stuff happening.

56:01 And so I was like looking at like runtime analysis.

56:03 You can see what functions are being called.

56:05 You can like to see whether or not like a CD is affecting your program.

56:10 So you can like introspect your program live while it's running and see whether

56:14 interesting like something is even being used.

56:16 You don't even have to like look at the source code to check or whatever.

56:20 You just look at your program and see if it's using that code.

56:23 Yeah, yeah, because if you bother, and I don't, maybe you all know better than I do, but if I were to speculate, I'd say less than 1% of people actually run pip audit or uv pip audit sort of thing, uv audit, whatever it is, on their code.

56:35 But even if you do, or you look at the CVE and you see one, or maybe get the PendaBot tells you, oh, there's a CVE in one of your dependencies, you better go fix it.

56:43 And then you got to go look and say, does it really matter?

56:46 I remember one of the packages I was using, it said, oh, on Windows, there's this issue that if you take untrusted input and it's a PDF and it's this way, it's going to do this terrible thing.

56:57 I'm like, yeah, but my deployment target is Linux on Docker and my dev machine is Mac.

57:03 And I don't think I've taken untrusted input whatsoever for this.

57:05 And so do I care?

57:07 I don't really care.

57:08 You know what?

57:09 I'm just going to not worry about it.

57:10 But PitBot will fill my build and then we're back to the cooldowns and all that.

57:15 But nonetheless, it's like, I think that's part of the problem.

57:18 And the fact, if you could say somehow combine the CVE and the CVE, actually, I can verify

57:23 it doesn't touch that bit of code.

57:25 That's super cool.

57:25 Yeah, on that front, I think there's an ongoing effort through the open source vulnerability

57:31 database to surface more rich details as to like code paths that are affected.

57:38 So that way you could automatically audit, do I use those code paths?

57:43 and then thus either fail or not fail your build.

57:46 But again, in languages like Python, where dynamicism is used, getting that an accurate representation

57:55 of am I using that code path is often a harder question to answer because you don't necessarily even have to import the library

58:03 via an import statement.

58:04 You can use import lib.

58:05 You can do all sorts of other different things.

58:08 So the challenge is how do we surface enough information make a confident decision without misleading you down the wrong path.

58:18 Indeed.

58:19 Let's keep moving on because time is a ticking.

58:23 The next one is GitHub Action Security and Python Packages by Andrew Nesbitt.

58:28 So, yeah.

58:29 Terrifying.

58:29 There's some really weird stuff.

58:31 That was very eye-ocaching.

58:32 Weird caching.

58:33 So Andrew's a nerd, right?

58:34 He's the best kind of nerd, right?

58:36 He loves researching things.

58:38 He loves accumulating data.

58:40 He's the author behind Ecosystems.

58:43 That's ecosystem.sys.ms.

58:47 And he analyzes lots of projects and was able to analyze a lot of GitHub project

58:54 IPI-linked repositories with Zismore, which he did not write.

58:58 But it is a very cool tool and great to audit your GitHub actions for security vulnerability.

59:05 It's scary what its surface is.

59:07 And when we're talking about folks getting started, this should be on that checklist of did you run Zizmor on your repository?

59:16 Better yet, did you add it as a GitHub action to run always on your repository? But, you know,

59:21 it surfaces a lot of easy to fix findings, and then sometimes surfaces the scarier ones of you've

59:27 left yourself open to an injection that leads to a exploit. And he talked about a couple of those.

59:33 Very interesting. Yeah, the GitHub actions have a lot of nuance and weird, there was some weird

59:38 cache poisoning that people could submit PRs that would trigger a GitHub Action build.

59:45 And even if they had no rights to the target system, their PR could point at a malicious

59:54 package somehow.

59:55 And then that would get cached into the actual build because the build ran in the context

01:00:01 of the target repo.

01:00:04 And hey, wouldn't it be nice if we could just cache this so we don't save it?

01:00:07 Maybe we should cache other people's inputs.

01:00:09 You said all the keywords there, Michael.

01:00:11 You said pull request and target.

01:00:13 That is the trigger for a lot of these.

01:00:15 GitHub Actions has a lot of different trigger keywords.

01:00:17 Push, pull request.

01:00:19 There is one called pull request target.

01:00:22 That one runs on forks.

01:00:24 It runs with elevated privileges.

01:00:26 It is dangerous.

01:00:28 Zismore calls that out as like, you are probably using this wrong because it is a very sharp, sharp knife.

01:00:35 If you pick it up the wrong way, You have now cut your hand and it's easy to pick up the wrong one.

01:00:41 So Runs is more.

01:00:42 Please start trying to juggle three of those.

01:00:44 Yeah.

01:00:44 Don't juggle three.

01:00:45 Runs is more.

01:00:46 Follow its recommendations.

01:00:47 Move on with your life.

01:00:49 All right.

01:00:49 One more here.

01:00:51 Breaking Bad packages.

01:00:53 Why traditional vulnerability tracking fails supply chain attacks.

01:00:56 Shelby Cunningham and Madison Figuerely.

01:00:59 So I really like this one because it talks about the CVE system, which is something that I've spent way too much time kind of digging around in.

01:01:06 And it's this challenge between intentional malicious software, like malware that we see on PyPI, and vulnerabilities.

01:01:17 And I think that there's this, sometimes we use CVEs for malware and sometimes we don't.

01:01:23 And because if we did, there'd be, you know, millions.

01:01:27 And it's like, do we, do we not?

01:01:29 Like, what are the trade-offs?

01:01:30 Does it overwhelm the system?

01:01:31 Is it even useful?

01:01:33 And it talked about all of these different watering hole attacks that ended up publishing malware to different open source ecosystems, including PIPI and how the this vulnerability record like CD or OSV responded in turn and like how people got the word out.

01:01:50 How did people respond to it?

01:01:53 And yeah, it's it's really challenging because it feels it feels so close to being the right system to use, but not quite.

01:01:59 And maybe that means that we need to build a new system.

01:02:02 Maybe that means that who knows what it means.

01:02:04 We need to do some more thinking in this area.

01:02:06 Yeah, I totally see it because malware is not a problem in valid software, right?

01:02:13 It's just a bad piece of software.

01:02:16 Whereas if I open up this PDF in Microsoft Word and it turns me into a botnet, that's a CVE.

01:02:21 Because Microsoft Word isn't intended to be malware.

01:02:24 All right, folks.

01:02:25 I think that might be it for the time that we've got.

01:02:30 So I guess final call to action here.

01:02:32 We have all these talks on PyCon US, and everyone links to them.

01:02:39 So every part of the show notes links up to them.

01:02:41 So I'll put the links in the show page for people listening and their podcast player and so on.

01:02:46 So go out and check out the interesting talks on YouTube.

01:02:49 I can put that out there.

01:02:51 But let's leave it with us.

01:02:53 Juanita, I'll give you the first word, and you can each answer this one.

01:02:56 So if everyone out there listening did one thing this week because of this episode,

01:03:02 because of what they heard us talking about.

01:03:03 What should they do?

01:03:04 What would you tell them?

01:03:04 Like, here's the one thing really I recommend most.

01:03:08 If it's only one, I'll just pick you, Sizmore.

01:03:11 Okay.

01:03:12 Mike?

01:03:13 Look at trusted publishing.

01:03:15 Convert at least one of your projects to use it.

01:03:18 And don't forget to delete the old token.

01:03:21 Got it.

01:03:21 Okay.

01:03:22 Spoken like a person who works on PyPI security.

01:03:25 Seth?

01:03:25 Yeah.

01:03:26 So my recommendation is if after hearing all of these security problems,

01:03:31 solutions like what's happening in security and how it's not this like solve thing to I'm assuming

01:03:38 that because you're watching this show that you have an interest in Python and you maybe you even

01:03:42 work with Python. I would recommend if you depend on Python or third party packages to look upstream

01:03:51 and to see that all of this work is being done and that there is so much more to do and support

01:03:58 those open source ecosystems that you depend on to enable them to do security. So whether that's

01:04:04 donating engineering time, so like working on a security team, making some of these security

01:04:09 focused contributions to open source projects that you care about, or whether that comes as

01:04:13 like funding, right? So like donating to the Python Software Foundation, for example,

01:04:18 nonprofit that is the home for Mike and I's job. Security is not free. It costs a lot of time to

01:04:25 implement all of these changes that everyone ends up benefiting from anyways. And it really would

01:04:30 not happen without internal advocates at companies and organizations working with their management and

01:04:36 all of that to convince them to support open source that they depend on. So that's my recommendation.

01:04:41 Perfect. And one final thing, maybe Juanita, people want to keep in touch with you.

01:04:46 Where's, you know, can they follow you on social? Do you got a blog?

01:04:50 I have a website where I guess I sometimes put things that I have done or I'm doing currently.

01:04:56 But honestly, the best way to reach me out is probably LinkedIn.

01:04:59 I try to post where I'm going to be if I'm giving talks, where to find me.

01:05:04 But if you just want to chat, just message me on LinkedIn.

01:05:07 I'm very happy to connect with people that are interested in any of the things that I'm working on.

01:05:11 Right on. Mike?

01:05:12 Yeah, I have a landing page at MikeTheMan.dev, and that will link you out to all of my other different locations, socials, blogs, etc.

01:05:22 And that is, yeah, I'm also on LinkedIn, but I also try to post on all the different socials of like, yep, I'm going to be at EuroPython.

01:05:29 Here's where I'm going to be.

01:05:30 So that way people can accost me on the hallway track and demand some new security feature, which is wonderful.

01:05:37 Love talking about it.

01:05:38 The most well-attended track of the conference indeed.

01:05:40 Seth.

01:05:41 Yeah, so you can find me online, sethamlarson.dev.

01:05:44 Links out to everywhere.

01:05:45 I prefer Mastodon, but I also respond to LinkedIn.

01:05:49 And there's even Signal there.

01:05:50 So if you want to just like privately ask me a security question or something like that,

01:05:53 or you just want to chat for a second, I respond to Signal.

01:05:57 So any of those locations work for me.

01:05:59 I publish a blog there as well.

01:06:00 So if you want to read anything about security or retro video games, those are your options.

01:06:05 Solid.

01:06:06 Great.

01:06:07 Thank you all for being here.

01:06:08 See you later.

01:06:09 Thank you.

01:06:10 This has been another episode of Talk Python To Me.

01:06:13 Thank you to our sponsors.

01:06:14 Be sure to check out what they're offering.

01:06:15 It really helps support the show.

01:06:17 Take some stress out of your life.

01:06:19 Get notified immediately about errors and performance issues in your web or mobile applications with Sentry.

01:06:25 Just visit talkpython.fm/sentry and get started for free.

01:06:30 Be sure to use our code talkpython26.

01:06:33 That's Talk Python, the numbers two, six, all one word.

01:06:38 Talk Python and Python Bytes both now have MCP servers.

01:06:42 Point your AI at 10 plus years of Python episodes, transcripts, and show notes.

01:06:47 Free.

01:06:48 Click MCP in the nav at talkpython.fm and at pythonbytes.fm.

01:06:52 If you or your team needs to learn Python, we have over 270 hours of beginner and advanced courses on topics

01:06:59 ranging from complete beginners to async code, Flask, Django, HTML, and even LLMs.

01:07:05 Best of all, there's no subscription in sight.

01:07:08 browse the catalog at talkpython.fm.

01:07:10 And if you're not already subscribed to the show on your favorite podcast player,

01:07:14 what are you waiting for?

01:07:15 Just search for Python in your podcast player.

01:07:17 We should be right at the top.

01:07:19 If you enjoy that geeky rap song, you can download the full track.

01:07:22 The link is actually in your podcast blur share notes.

01:07:24 This is your host, Michael Kennedy.

01:07:26 Thank you so much for listening.

01:07:27 I really appreciate it.

01:07:29 I'll see you next time.

01:07:41 I'm out.

Talk Python's Mastodon Michael Kennedy's Mastodon