Running Python in Production

Episode Deep Dive

Guests introduction and background

Emily Morehouse is Director of Engineering at a digital product development company called Cuddlesocks. She’s also a Python core developer and was PyCon chair for multiple years (including the challenging COVID era). Emily works with diverse web, mobile, and IoT projects, touching everything from the front-end to DevOps.

Glyph is best known for creating and maintaining the Twisted network framework. He has worked on various production-grade Python deployments, most recently at Pilot.com. Glyph emphasizes thoughtful design of systems and is passionate about stability and reliability in Python.

Hynek has 14+ years of experience at a web hosting company based in Germany. He uses Python extensively for automating, securing, and scaling their platform. Hynek is also the maintainer of Argon2 CFFI for password hashing. He highlights topics around security, encryption, and hands-on Python production strategies.

What to Know If You're New to Python

If you’re just getting started, here are a few quick pointers from the discussion to keep you on track:

• Dependency Pinning: You’ll hear about pinning Python packages and managing them carefully. This ensures consistent, reliable deployments.
• Docker Basics: Many panelists recommend using Docker containers for portable, repeatable environments.
• Monolith vs. Microservices: The discussion underscores not overcomplicating your first project; it’s often safer to start with a single codebase.
• Security and Updates: They emphasized that staying current with Python’s minor versions and patch releases can protect you from known vulnerabilities.

Key points and takeaways

1. Running Python in Production at Different Scales While huge deployments at Instagram or Google get the most press, most Python teams run apps at small-to-medium scale. The guests highlighted that Python is "fast enough" for a majority of projects, and often performance bottlenecks come from database queries or third-party APIs rather than Python itself. They also emphasized that focusing on real, everyday production setups (rather than hyper-scale) can be more practical for most organizations.
   • Tools / Links:
     • DigitalOcean
     • Heroku
     • AWS and GCP
2. Start with a Monolith, Then Consider Microservices The guests agreed it’s often wise to begin with a single codebase—one “macroservice” or monolith—and split out services only if you have organizational capacity and genuine needs. Microservices introduce higher operational overhead (service discovery, monitoring, handling partial failures) that can be overkill for smaller teams.
   • Tools / Links:
     • Flask
     • Django
3. Cloud vs. On-Prem vs. Hybrid When deciding between cloud services and traditional hosting, you must weigh cost, operational overhead, and compliance constraints—especially in regions like the EU where data locality is important. Many small teams prefer fully managed platforms (like Heroku’s one-click approach), while larger or privacy-focused organizations might choose their own hardware or a hybrid.
   • Tools / Links:
     • AWS ECS
     • AWS RDS
     • Sentry (for centralized error tracking)
4. Docker for Portability and Simplified Deployments Packaging Python apps in Docker containers emerged as a major theme. Docker helps standardize local development and production environments. Emily highlighted that whether you’re on Heroku, AWS, or GCP, shipping a Docker image is a portable solution to “it works on my machine” problems.
   • Tools / Links:
     • Docker
5. Security: Defense in Depth and Staying Updated Hynek stressed “defense in depth” – assume your internal network might be compromised and encrypt as much traffic as possible (even internally). Everyone agreed that keeping dependencies updated is crucial, whether through automation like Dependabot or via a pinned-requirements workflow. They highlighted that big vulnerabilities (e.g., log4j) often reinforce how critical it is to keep pace with releases.
   • Tools / Links:
     • Dependabot
     • pip-compile (part of pip-tools)
     • Sentry
6. Performance and Load Testing Loading up your production-like environment with realistic data and scripts is crucial. The guests recommended using Python-based frameworks, such as Locust, which can mimic different user patterns (like “normal users” plus “admin users”) to see how your site performs under stress. They also noted how database bottlenecks often overshadow raw Python performance.
   • Tools / Links:
     • Locust
     • PageSpeed Insights (for front-end performance)
7. Database Query Optimization They stressed that many “slow web apps” are really “slow SQL queries.” PG Mustard was praised for simplifying EXPLAIN statements in PostgreSQL. Balancing the right indexes vs. too many indexes is important: you can speed up reads but risk slowing down writes.
   • Tools / Links:
     • PG Mustard
     • PostgreSQL
8. Handling Larger Teams As teams grow into the dozens or hundreds, monolithic codebases can become unwieldy with frequent merge conflicts. That is often when splitting into services or multiple repositories can help reduce friction—but only at that scale. The panelists gave real-life examples from large companies where microservices made sense.
9. Testing and Observability Cypress for front-end acceptance tests, or just limiting your max response time in development, can highlight potential performance or design issues. Pair this with robust real-time metrics and logging in production—so you discover issues before users do.
   • Tools / Links:
     • Cypress
10. Editor Choices and Developer Ergonomics A lively side note from each guest covered their editor picks: Emacs (with heavy customization), Vim, or VS Code. Emily credited Brett Cannon with finally convincing her to swap from Sublime to VS Code, praising VS Code’s built-in Python tools. While editors vary, the consensus was to pick something that boosts your efficiency and comfort.

• Tools / Links:
  • VS Code

Interesting quotes and stories

• On microservices vs. monoliths: “More moving parts are always harder to make reliable.” – Hynek, emphasizing that you need strong reasons (and a large enough team) to break your system apart.
• On security: “We treat our internal network as if it’s compromised … you can’t have enough layers.” – Hynek, describing the value of encryption and zero-trust approaches even inside your private LAN.
• On scaling: “Python is fast enough. You usually saturate your database before you saturate Python.” – A common refrain from all guests, reminding us where the real bottlenecks often are.
• On performance: “We had an issue because we added too many indexes. Sometimes you optimize reads at the expense of writes.” – Glyph, sharing real-world complexities of database tuning.

Key definitions and terms

• Monolith: A single, unified codebase or application that manages all application logic in one deployment.
• Microservices: A suite of smaller services, each focused on a specific function, communicating over the network.
• Defense in Depth: A security principle that layers protections (e.g., encryption, segmentation) so that a single failure doesn’t compromise the whole system.
• Dependency Pinning: Locking packages to a specific version for stability and reproducibility in production.
• Locust: A Python-based load testing framework that can simulate user scenarios against your web application.

Learning resources

If you’re just starting with Python, or you want a systematic, project-focused approach, check out:

• Python for Absolute Beginners: A thorough introduction to programming and Python fundamentals, perfect for new developers.

Additional on-topic courses from Talk Python Training that can help you dive deeper into production concepts:

• Getting started with pytest – Learn the most popular Python testing framework in depth.
• Full Web Apps with FastAPI – Build and deploy robust web apps that go beyond basic APIs.

Overall takeaway

Building reliable Python apps in production is more about making wise architectural and operational choices than about picking exotic technologies. Often, you can begin with a monolith, containerize your application, and focus on security fundamentals like dependency updates and defense in depth. As you scale, microservices and deeper cloud integrations might become essential, but keep complexity aligned with team capacity. Above all, monitor your real-world performance, be mindful of database queries, and stay excited about Python’s ever-growing ecosystem for modern production deployments.