Learn Python with Talk Python's 270 hours of courses

Python Packaging and PyPI in 2022

Episode #377, published Sat, Aug 13, 2022, recorded Thu, Aug 11, 2022

PyPI has been in the news for a bunch of reasons lately. Many of them good. But also, some with a bit of drama or mixed reactions. On this episode, we have Dustin Ingram, one of the PyPI maintainers and one of the directors of the PSF, here to discuss the whole 2FA story, securing the supply chain, and plenty more related topics. This is another important episode that people deeply committed to the Python space will want to hear.

Background noise warning: Just wanted to apologize for a bit of background noise on my end (Dustin had amazing audio). We had construction at our place, which would have been fine. But work started on the ceiling right under my desk making much more noise than expected. I think we generally have it cleaned up, but there may be a few sounds sneaking through. Thanks for the understanding. :)

Watch this episode on YouTube
Play on YouTube
Watch the live stream version

Episode Deep Dive

Guest background

Dustin Ingram is one of the maintainers of the Python Package Index (PyPI) and serves on the board of directors at the Python Software Foundation. He also works at Google on open-source security initiatives, focusing on projects that improve the security of Python’s packaging ecosystem. Prior to his current work, Dustin contributed to Python packaging tools and has been involved in a variety of efforts to modernize and secure PyPI. His unique dual perspective, as both a volunteer maintainer and a professional in open-source security, allows him to offer valuable insights into the growth, challenges, and solutions facing the Python community.

What to Know If You're New to Python

Here are a few basics that will help you understand this episode’s focus on packaging and security:

  • PyPI (Python Package Index): The main repository of Python libraries and frameworks that you install via pip install ....
  • pip: Python’s primary package installer, which fetches code from PyPI.
  • Virtual environments: A standard way to isolate your installed packages for each project (venv, conda, etc.).
  • 2FA (Two-Factor Authentication): An extra layer of account security often recommended for critical software services.

Key points and takeaways

  1. 2FA Requirement for Critical Packages
    PyPI has begun enforcing two-factor authentication for “critical projects,” defined by the top 1% of downloaded packages. This mandate aims to protect users from account takeovers and malicious package uploads, mitigating large-scale supply-chain attacks. While the community reaction was mixed, the PyPI maintainers underscored that 2FA dramatically reduces entire classes of attacks, such as phishing or domain-based account hijacking. Free hardware security keys were even provided (sponsored by Google) to help maintainers transition.

  2. Scale and Sponsorship of PyPI
    PyPI serves billions of daily requests, transferring over a thousand terabytes of data each day—an infrastructure burden that would cost millions if not for generous corporate sponsorships. Companies like Fastly donate their CDN services to keep package hosting free. The Python Software Foundation relies heavily on event revenue (notably PyCon) and sponsorships, underscoring that community support and sponsorships remain critical to the Python ecosystem’s sustainability.

  3. Volunteer-Driven Maintenance
    Maintaining PyPI involves a handful of volunteers and a small PSF staff, meaning new security requirements (like 2FA) come with significant overhead. Every support ticket for account recovery or package takedown adds up. The push toward making 2FA mandatory for high-impact packages is as much about reducing volunteer fatigue as it is about improving security.

  4. Atomic Writes Fiasco
    A maintainer of the atomicwrites package attempted to bypass PyPI’s “critical project” label by deleting and re-uploading the project. This caused a break for anyone using older versions. PyPI ultimately restored those deleted versions, but the incident highlighted the tension between maintainers’ autonomy and the broader ecosystem’s needs. It also spotlighted how package deletion can break countless downstream dependencies if not done thoughtfully.

  5. Lessons from NPM and Left-Pad
    The conversation touched on parallels with the “left-pad” incident in the JavaScript world, illustrating how one maintainer’s decision can send shockwaves across thousands of projects. PyPI’s stance has generally been to discourage package deletion, offering alternatives like yanking or archiving old releases. This approach aims to ensure continued stability for the broader Python community.

  6. James Bennett’s 2FA Response
    James Bennett wrote a widely shared article defending the PyPI team’s decision to require 2FA for popular packages. His commentary underscored that PyPI’s volunteer maintainers have every right to ask for minimal security from package owners. By cutting down on security emergencies, these volunteers can spend more time improving PyPI rather than constantly firefighting malicious uploads.

  7. New Upload API and Draft Releases (PEP 691+)
    PyPI historically used an older, monolithic post-based upload flow. Upcoming improvements, such as PEP 691, aim to introduce more modern, JSON-based interfaces and the concept of draft releases. Draft releases will let maintainers test and verify artifacts within PyPI before final publication, preventing accidental or rushed uploads from impacting the public index.

  8. Reducing Arbitrary Code Execution with PEP 621
    The conversation emphasized the shift away from setup.py scripts, which allow arbitrary code execution at install time. With PEP 621, Python packages can define metadata purely in pyproject.toml, reducing security risks. Modern tools like Flit, Poetry, and updated setuptools adhere to these standards, offering safer package installations.

  9. Supply Chain Security and OIDC
    Another improvement is adopting short-lived credentials via OpenID Connect (OIDC). By letting CI systems (like GitHub Actions) authenticate securely without storing long-lived API tokens, maintainers can reduce accidental credential leaks. This approach strengthens the integrity of each package release by cryptographically verifying which workflow performed the upload.

  10. pip-audit and SigStore
    The episode highlighted a couple of new security-oriented tools. pip-audit audits local environments (or requirements files) for known vulnerabilities based on databases like OSV (Open Source Vulnerabilities). Meanwhile, SigStore helps sign packages with ephemeral keys, making verification easier and bypassing the complexities of GPG’s “web of trust.” These are part of broader efforts to secure the Python supply chain.

Interesting quotes and stories

  • On community-driven infrastructure: “PyPI is as susceptible as a bank for phishing or domain-based attacks, but we have zero full-time support staff—it’s all volunteers.”
  • On 2FA backlash: “We never imagined there would be so much pushback for asking maintainers to enable two-factor. We’re just trying to protect them—and everyone else!”
  • Atomic Writes takedown: “Yes, you can delete your package from PyPI, but expect some serious fallout across the community.”

Key definitions and terms

  • PyPI (Python Package Index): Central repository for Python packages installable via pip. Pronounced "Pie - P - I."
  • Two-Factor Authentication (2FA): An added security measure requiring a second proof (e.g., app code, hardware key) beyond just a password.
  • Dependency Confusion: A supply-chain attack where internal packages are overshadowed by public packages of the same name.
  • Domain Resurrection Attack: Taking control of an expired domain to reset account passwords and gain unauthorized access.
  • PEP (Python Enhancement Proposal): A design document that describes new features or processes for Python.
  • OIDC (OpenID Connect): An identity layer built on OAuth 2.0 enabling short-lived, secure tokens for CI deployments.

Learning resources

If you want to deepen your Python skills—especially if you’re just getting started—check out these courses on Talk Python Training.

Overall takeaway

PyPI has grown from a simple code repository to a critical backbone for the entire Python community. Because of this, security and reliability measures—like mandatory 2FA for critical packages, safer metadata standards, and advanced tools for signing and vulnerability detection—are not merely nice to have. They’re essential for maintaining trust in Python’s ecosystem. By cooperating with the Python Software Foundation, sponsors, and volunteers, the community can balance user freedom with the security and stability required at Python’s scale.

Links from the show

Dustin on Twitter: @di_codes

Hardware key giveaway: pypi.org
OpenSSF funds PyPI: openssf.org
James Bennet's take: b-list.org
Atomicwrites (left-pad on PyPI): reddit.com
2FA PyPI Dashboard: datadoghq.com
github 2FA - all users that contribute code by end of 2023: github.blog
GPG - not the holy grail: caremad.io
Sigstore for Python: pypi.org
pip-audit: pypi.org
PEP 691: peps.python.org
PEP 694: peps.python.org
Watch this episode on YouTube: youtube.com
Episode transcripts: talkpython.fm

--- Stay in touch with us ---
Subscribe to Talk Python on YouTube: youtube.com
Talk Python on Bluesky: @talkpython.fm at bsky.app
Talk Python on Mastodon: talkpython
Michael on Bluesky: @mkennedy.codes at bsky.app
Michael on Mastodon: mkennedy

Talk Python's Mastodon Michael Kennedy's Mastodon