Building GPT Actions with FastAPI and Pydantic

Episode Deep Dive

Guest: Ian Moyer and His Background

Ian Moyer is a seasoned Python developer working at Genome Oncology, building precision oncology software for medical professionals and researchers. He was previously on the podcast in 2018 discussing Python for genomics and biology. Ian began his career creating e-commerce sites in Java before switching to Python. He has since used Python, FastAPI, and other modern tools to solve problems in genomic data processing and, more recently, to experiment with custom GPT (Generative Pre-trained Transformer) applications.

1. Ian’s Ongoing Python Journey in Genomics

• Moving from Java to Python: Ian described how creating a quick prototype in Django drastically sped up development, convincing his team to adopt more Python.
• Current Work at Genome Oncology: He focuses on building tools for precision oncology, including natural language processing solutions that help interpret medical and genetic data.

2. LLMs (Large Language Models) and Their Place in Development

• Initial Exposure to LLMs: Ian originally used NLP libraries like spaCy for text processing, but the release of ChatGPT highlighted the power of large models for tasks like summarization and information extraction.
• Developer Workflow with ChatGPT: Rather than using “autocomplete” coding tools exclusively, Ian prefers to collaborate with ChatGPT 4 for step-by-step problem solving, especially around edge cases and testing strategies.

3. Integrating LLMs in Python Apps

• APIs and Providers: The episode surveyed several LLM APIs (OpenAI, Azure OpenAI, Google Gemini, Anthropic Claude) and highlighted Simon Willison’s LLM library for switching between providers.
• Abstraction and Optionality: Ian stressed coding to an abstraction layer so you can swap LLM backends easily—just as you might swap databases or services in other parts of your application.

4. Running LLMs Locally (Open Source and On-Device Models)

• Local Models vs. Cloud APIs: Ian and Michael discussed projects such as LM Studio and LlamaFile.ai, which provide OpenAI-like local servers running models like Llama 2, Mistral 7B, and more.
• Prompt Engineering & Context Windows: They covered techniques like few-shot prompting, chunking text for embeddings, and the inherent limits of a model’s “context window.”

5. Retrieval-Augmented Generation (RAG)

• Basic Concept: Store documents or data in a vector database, retrieve relevant chunks via embeddings, then feed them into an LLM prompt to synthesize an answer.
• Vector Databases: They discussed solutions like PG Vector (for PostgreSQL) and how indexing methods can trade off speed versus perfect recall.

6. Custom GPTs from OpenAI

• Custom GPT Overview: A new feature from OpenAI that lets you name and brand a GPT, give it custom instructions, add PDF documents or knowledge sources, and share it with others.
• Actions and OpenAPI Specs: Formerly “plugins,” these GPT “actions” let your GPT call your custom API. Ian noted you can build that API with FastAPI plus an OpenAPI spec, effectively teaching the GPT how to interface with your code.

7. PyPI GPT Example

• FastAPI + Pydantic Demo: Ian’s example project (“PyPI GPT”) showcases how to serve an OpenAPI spec (via FastAPI) that a custom GPT can call. Although the demonstration is simple (fetching data about Python packages), it’s a working template for more advanced ideas.
• HTTPS & Hosting: If you want to replicate it, you’ll need a secure endpoint (e.g., via ngrok or Let’s Encrypt) so OpenAI can reach your API.

8. Tools, Libraries, and Projects Mentioned

Below are some references from the conversation, all of which Ian and Michael highlighted directly:

• LangChain: github.com/hwchase17/langchain
• LlamaIndex (formerly GPT Index): github.com/jerryjliu/gpt_index
• LM Studio: lmstudio.ai
• LlamaFile: llamafile.ai
• Simon Willison’s LLM: github.com/simonw/llm
• Sourcegraph Cody: about.sourcegraph.com/cody
• Marvin, Instructor, Outlines: Various open source Pydantic-based “extract-to-JSON” LLM libraries (mentioned for structured LLM outputs)
• VisiData: visidata.org – a CLI spreadsheet-like utility for CSVs and data
• btop: GitHub link – a colorful terminal-based system monitor

9. Medical Use Cases & Caution

• Leveraging GPTs in Oncology: Ian is actively experimenting with GPT solutions in genome interpretation, focusing on a “human-in-the-loop” approach where clinicians remain the final decision makers.
• Prompt Injection and Hallucination: They discussed the reality that LLMs can be tricked (prompt injection) or produce convincing but incorrect answers (hallucinations). Any serious medical or legal usage must account for these limitations.

Overall Takeaway

This episode takes a deep dive into building custom GPT apps using Python. It covers everything from best practices for LLM usage and prompt design to practical details on packaging a custom GPT with FastAPI and Pydantic. While GPTs can synthesize data and automate many tasks, Ian emphasizes the importance of safety, correctness, and a thoughtful integration approach—especially when reliability matters, such as in healthcare. By combining Python-based tooling with LLMs, developers can create targeted chat experiences that extend beyond the basic ChatGPT interface and potentially deliver high-impact solutions in specialized domains.