Python Trends Episode 2024

Episode Deep Dive

Guests

• Jodie Burchell Jodie is a seasoned data scientist currently working as a Developer Advocate in Data Science at JetBrains. Before moving into the tech industry, she spent time in academia studying psychology. Her background includes several years of hands-on data science work, and she’s passionate about making complex data tasks more approachable for newcomers.
• Carol Willing Carol is a Python core developer and former steering council member, with deep involvement in Project Jupyter and open source education initiatives. Having worked in both startup environments and large-scale projects, she brings extensive experience in Python’s role in scientific computing, education, and community-driven development.
• Paul Everett Paul is a long-time Python developer and advocate focusing on the web. He also works at JetBrains and has been part of the Python ecosystem for years, going back to early Python web frameworks. Paul keeps a close eye on evolving Python trends, especially around performance and community building.

1. The Continuing Rise of Large Language Models (LLMs)

• LLMs Are Here to Stay: Despite some fatigue in hearing about them, the panel emphasized that interest and usage continue to rise.
• Open Source Fight Back: Libraries such as Hugging Face Transformers, LangChain, and LlamaIndex are gaining traction, pointing to more open solutions outside of proprietary systems.
• RAG (Retrieval Augmented Generation) and Vector Databases: Tools like Weaviate and Unstructured are part of the ecosystem helping LLMs handle specialized or domain-specific tasks with additional data sources.
• Education and “Hype” Awareness: The panel stressed the importance of understanding what LLMs really are (statistical language models rather than reasoning engines), encouraging better education and realistic expectations for beginners.

2. Python Performance and Concurrency

• No-GIL and Sub-Interpreters: Upcoming changes in CPython (removal of the Global Interpreter Lock and the possibility of sub-interpreters) could significantly affect how Python handles concurrency.
• Balancing Approachability vs. Performance: The group debated whether optimization efforts might add complexity and make Python less friendly for beginners. They agreed that Python’s “full spectrum” approach can still remain if advanced features are well-documented and carefully adopted.
• Typing, Async, and Modern Python: Mention of type hints, the growing reliance on Pydantic and FastAPI, and Async IO highlights how Python is evolving to handle large codebases while trying to preserve its readability.

3. Evolving Data Frame Ecosystem (Arrow, Polars, and Pandas 2.0)

• Arrow Integration: Apache Arrow is at the heart of the new data frame push, particularly with the release of Pandas 2.0. Arrow standardizes data formats and makes it easier to share data among different systems without heavy conversions.
• Polars: Polars is a Rust-based library offering a lazy evaluation engine and query optimization for higher performance than traditional Pandas in many scenarios.
• DuckDB: Mentioned as an in-process SQL OLAP database that can integrate well with Python-based analytics without blowing up memory usage, via duckdb.org. See episode 491: DuckDB and Python: Ducks and Snakes living together for a deep dive on DuckDB.

4. Python on the Front End and WebAssembly

• JupyterLite: The panel highlighted the ability to run Python in the browser via WebAssembly. JupyterLite is a project that enables running Jupyter notebooks client-side in the browser, opening new possibilities for easy onboarding and offline usage.
• PyScript and Pyodide: Tools like Pyodide (the backbone of many Python-in-the-browser projects) and PyScript make Python more accessible on the web and potentially reduce the barriers of environment setup.
• HTMX and the “Web Back in the Web”: Paul noted a growing trend to rely less on heavy JavaScript frameworks and more on simpler, server-driven or WebAssembly-driven solutions (e.g., htmx.org).

5. Rust’s Growing Role in the Python Ecosystem

• Performance Gains in Tooling: Rust-based tools like Ruff (a fast Python linter) are now common. They showcase how Rust’s safety and concurrency features can offer performance boosts to Python tooling.
• Underlying Implementations: Polars (mentioned above) is Rust-based, and other Python libraries have begun experimenting with Rust for core internals to handle performance-critical sections.

6. Community, Openness, and the Future of Python

• Inclusion and Education: Projects like “Humble Data” workshops, “PyOpenSci,” and the broader Python community events (e.g., PyCon) are key to welcoming new and non-traditional backgrounds into Python.
• Threat of Walled Gardens: The panel noted potential concerns about large corporations monetizing user-generated knowledge (e.g., search engines providing AI answers with no source links). This poses a challenge for open ecosystems.
• Planet Python, Blogs, and RSS: There was nostalgia for older ways of sharing knowledge openly (like RSS feeds), emphasizing that it may be time to revitalize community-controlled platforms rather than rely on large walled-garden sites.

Overall Takeaway

Across topics ranging from AI, data frames, and concurrency to Python-in-the-browser, the recurring theme was the power of community and openness. While Python faces new complexities—performance demands, the rise of LLMs, and changing tools—it still thrives on approachability and a culture of shared learning. The panelists agreed that continued focus on inclusive education, open tooling, and sustainable community practices remains essential for Python’s future success.