AI has taken the world by storm.
It's gone from near zero to amazing in just a few years.
We have chat GDP, we have stable diffusion.
What about Jupyter Notebooks and Pandas?
In this episode, we meet Justin Wagg, the creator of Sketch.
Sketch adds the ability to have conversational AI interactions
about your Pandas' data frames, code, and data
right inside of your notebook.
It's pretty powerful, and I know you'll enjoy the conversation.
This is Talk Python to Me, episode 410, recorded April 2nd, 2023.
Welcome to Talk Python to Me, a weekly podcast on Python.
This is your host, Michael Kennedy.
Follow me on Mastodon, where I'm @mkennedy,
and follow the podcast using @talkpython,
both on fosstodon.org.
Be careful with impersonating accounts on other instances.
There are many.
Keep up with the show and listen to over seven years
of past episodes at talkpython.fm.
We've started streaming most of our episodes live on YouTube.
Subscribe to our YouTube channel over at talkpython.fm
slash YouTube to get notified about upcoming shows
and be part of that episode.
This episode is brought to you by Brilliant.org
and us with our online courses over at Talk Python Training.
Justin, welcome to Talk Python to Me.
Thanks for having me.
It's great to have you here.
I'm a little suspicious.
I got to know, I really know how to test
whether you're actually Justin or an AI speaking as Justin.
What's the deal here?
Yeah, there's no way to know now.
No, there's not.
Well, apparently I've recently learned from you
that I can give you a bunch of Xs
and other arbitrary characters.
This is like the test.
It's like asking the Germans to say squirrel
in World War II sort of thing.
Like it's the test.
It's the tell.
There's always going to be something.
It's some sort of adversarial attack.
Exactly.
It's only going to get more interesting
with this kind of stuff for sure.
So we're going to talk about using generative AI
and large language models paired with things like Pandas
or consumed with straight Python
with a couple of your projects,
which are super exciting.
I think it's going to empower a lot of people
in ways that it hasn't really been done yet.
So awesome on that.
But before we get to it, let's start with your story.
How did you get into programming in Python and AI?
Let's see.
I got into programming in just like when I was a kid,
TI-83, learning to code on that.
And then sort of just kept it up as a side hobby
my whole life.
Didn't ever sort of choose it as my career path
or anything for a while.
It chose you.
Yeah, it chose me.
It just, I dragged it along with me everywhere.
It's just like the toolkit.
I got a, went to undergrad
and for physics, electrical engineering,
then did a physics PhD, experimental physics.
During that, I did a lot of non-traditional languages,
things like LabVIEW, Igor Pro,
just weird Windows, Windows hotkey
for like just trying to like automate things.
Yeah, sure.
So just was sort of dragging that along.
But along that path,
sort of came across GPUs
and used it for accelerating processing,
specifically like particle detection.
So it was doing some like electron counting
in some just detector experiments.
Is this like CUDA cores on NVIDIA type thing?
Precisely.
Stuff like that.
Okay.
And was that with Python
or was that with C++ or what?
At the time it was C++
and I made like a DLL
and then called it from LabVIEW.
Wow, that's some crazy integration.
It's like drag and drop programming too
on the memory GPU.
Exactly.
It was all over the place.
Also had,
it was a distributed LabVIEW project.
We had multiple machines
that were coordinating and doing this
all just to move some motors
and measure electrons.
But it got me into CUDA stuff,
which then at the time
was around the time
that the like AlexNet,
some of these like very first neural net stuff
was happening.
And so those same convolutional kernels
were the same exact code
that I was trying to write
to run like convolutions
on these images.
And so it's like,
oh, look at this like paper.
Oh, let me go read it.
It seems like it's got so many citations.
This is interesting.
And then like that sent me down
the rabbit hole of like,
oh, this AI stuff.
Oh, okay.
Let me go deep dive into this.
And then that just,
I'd say that like became
the obsession from them.
So it's been like eight years
of doing that.
Then sort of just
after I left academia,
tried my own startup,
then joined multiple others
and just sort of have been
bouncing around
as the sort of like
founding engineer,
early engineer at startups
for a while now.
And yeah,
Python has been the choice
ever since like late grad school
and on.
I would say it sort of like
came through the pandas
and NumPy part,
but then stuck for the scripting,
like just power,
just can throw anything together
at any time.
So it seems like
there were two groups
that were just
hammering GPUs,
hammering them,
crypto miners
and AI people.
but the physicists
and some of those
people doing large scale
research like that,
they were the OG
graphics card users,
right?
Way before
crypto mining existed
and really before AI
was using graphics cards
all that much.
When I was like looking
at some of the code,
like pre-CUDA,
there were some like
quant traders
that were doing
some like crazy stuff
off of shaders.
Like it wasn't even CUDA yet,
but it was shaders
and they were trying to like
extract the compute power
out of them from that.
So...
Look,
if we could shave
one millisecond off this,
we can short them all day,
let's do it.
But yeah.
Yeah.
The physicists,
I mean,
it's always been like,
yeah,
it's always the get
as much compute
as you can
out of the,
you know,
devices you have
because simulations are slow.
Yeah.
I remember when I was
in grad school
studying math,
actually senior year,
regular college,
my bachelor's,
the research team
that I was on
had gotten a used
silicon graphics computer
for a quarter million dollars
and some Onyx workstations
that we all were given to.
I'm like,
this thing is so awesome.
A couple years later,
like an NVIDIA graphics card
and like a simple PC
would crush it.
Like that's $2,000.
It's just,
yeah,
there's so much power
in those things
to be able to harness them
for whatever,
I guess.
Yeah.
As long as you don't have
too much branching,
it works really well.
Awesome.
So let's jump in
and start talking about,
let's start to talk about
ChatGP
and some of this AI stuff
before we totally get into
the projects
that you're working on,
which brings
that type of
conversational
generative AI
to things like Pandas,
as you said.
But to me,
I don't know how,
maybe you've been more
on the inside than I have,
but to me,
it looks like
AI has been one of those things
that's 30 years
in the future forever,
right?
It was like
the Turing test
and,
oh,
here's a chat,
I'm going to talk to this thing
and see if it feels human or not.
And then,
you know,
there was like OCR
and then all of a sudden
we got self-driving cars,
like,
wait a minute,
that's actually solving
real problems.
And then we got things
like ChatGP
where people are like,
wait,
this can do my job.
It seems like it,
just in the last couple of years,
there's been some inflection point
in this world.
What do you think?
Yeah,
I think there's sort of like
two key things
that have sort of happened
in the past,
I guess,
four or five years,
four years,
roughly.
One is the
attention is all you need
paper from Google,
sort of this transformer
architecture came out
and it's sort of a good,
very hungry model
that can just sort of
absorb a lot of facts
and just like a nice
learnable key value store
almost that's stuck.
So,
and then the other thing
is,
is the GPUs.
We were sort of just talking
about GPU compute,
but this has just been
really,
GPU compute has really been
growing so fast.
If you like,
look at the like
Moore's law equivalent
type things,
like it's just,
it's faster how much
we're getting flops
out of these things
like faster and faster.
So,
it's been really nice.
I mean,
obviously there'll be
a wall eventually,
but it's been good
riding this like
exponential curve for a bit.
Yeah,
is the benefit
that we're getting
from the faster GPUs,
is that because
people are able
to program it better
and the frameworks
are getting better
or because just
the raw processing power
is getting better?
All of the above.
Okay.
I think that
there was a paper
that tried to dissect this.
I wish I knew
the reference,
but I believe
that their argument
was that it was
actually more
the processing power
was getting better.
The actual like
physical silicon
were getting better
at making that
for specifically
this type of stuff.
But like
on exponentials,
but yeah.
the power that
those things take,
I have a gaming system
over there
and it has a
GeForce 2070 Super.
I don't know
what the Super
really gets me,
but it's better
than the not Super,
I guess.
Anyway,
that one still plugs
into the wall normal,
but the newer ones,
like the 4090s,
those things,
the amount of power
they consume,
it's like space heater
level of power.
Like,
I don't know,
800 watts or something
just for the GPU.
You're going to
brown out the house
if you plug in
too many of those.
Yeah.
Go look at those
DGX A100 clusters
and they've got
like eight of those
A100s just stacked
right in there.
They take really
beefy powers of mine.
It's built right
directly attached
to the power plant,
electrical power plant.
Nuts.
Okay,
so yeah,
so those things
are getting really,
really massive.
Here's the paper
Attention is All You Need
from Google Research.
What was the story of that?
How's that play into things?
Yeah,
so this came up
during like machine translation
sort of research
at Google
and the core thing
is they present this idea
of instead of just stacking
these like layers
of neural nets
like we're sort of used to,
they replace the like
neural net layer
with this concept
of a transformer block.
A transformer block
has this concept inside
that's an attention mechanism.
The attention mechanism
is effectively
three matrices
that you combine
in a specific order
and the sort of logic
is it
is that one of the vectors
takes you from some space
to keys
so it's almost like
it's like identifying labels
out of your data.
Another one is
taking you from your data
to queries
and then it like
dot products those
to find a weight
and then for the one
and then another one
finds weight values
for your things.
So it takes this
query and key,
you get the weights
for them
and then you take
the ones that were
sort of the closest
to get those values
from the third matrix.
Just doing it
sort of like
looks a little bit
like accessing an element
in a dictionary
like key value lookup
and it's a differentiable
version of that
and it did really well
on their machine learning
sorry,
on their machine translation
stuff.
This was,
I think it's
like one of the
first big one
is this BERT model
and that paper
sort of
the architecture
of the actual
neural net code
is effectively
unchanged
from this
to ChatGPT.
Like there's
a lot of stuff
for like
milking performance
and increasing
stability
but the actual
like core
essence of the
actual mechanism
that drives it
it's the same
thing since this paper.
Interesting.
It's funny that
Google didn't
release something
sooner.
It's wild
that they've had
they keep
showing off
that they've got
like equivalent
or better things
at different times
but then not
releasing it.
When Dolly
happened
they had Imogen
Imagine
I guess
I don't know
how you say it
and what was
the Party
as the two
they had two
different
really good
way better
than Dolly
way better
than stable
diffusion models
like the
that had
were out
and they like
showed it
demoed it
like but
never released
it to be used
so yeah
it's one of these
who knows
what's going to
happen with Google
if they keep
holding on to
these things.
Yeah well I think
there was some
hesitation
I don't know
holds up on
accuracy
or weird stuff
like that.
Sure.
Yeah now
cat's out of the
bag now
now it's happening.
Yeah the cat's
out of the bag
and people are
racing to do
the best they
can and it's
going to have
interesting consequences
for us both
positive and
negative I think
but you know
let's leverage the
positive once the
cat's out of the
bag anyway right?
Yeah.
Hopefully.
Might as well
like ask it
questions for
pandas.
so let's play a
little bit with
chat GDP and
maybe another
one of these
image type
things.
So I came
in here and I
stole this example
from a blog post
that's pretty nice
about not using
deeply nested
codes.
You can use a
design pattern
called a
guarding clause
that will look
and say if the
conditions are not
right we're going
to return early
instead of having
if something
if that also
if something else
so there's this
example that is
written in a poor
way and it says
like it's checking
for a platypus
so it says if
self.ismammal
if self.hasfur
if self.hasbeak
etc.
it's all
deeply nested
and just for people
who haven't played
with chat GDP
like I put that
in and I said
sure I told her
I wanted to call
this arrow
because it looks
like an arrow
and it says
it tells me a
little bit about
this so I'm
going to ask it
please rewrite
arrow to be
less nested
with girding
clauses right
this is like a
machine right
if I tell it
this what is it
going to say
let's see
it may fail
but I think
it's going to
get it
it's thinking
I put it
I mistakenly
put it into
chat GDP
4 which
takes longer
I might switch
it over to
3 I don't
know
but the
understanding
of these
things
there's a lot
of hype
about it
like I think
you kind of
agree with me
that maybe
this hype
is worthwhile
here we go
so look
at this
it rewrote
it said
if it's
platypus
if not
self is
man
will return
false
if not
has fur
and there's
no more
nesting
that's pretty
cool right
yep
I mean I'm
sure you've
you've played
with stuff
like this
right
yeah
big user
of this
I mean this
is kind
of interesting
right
like it
understood
there was
a structure
and it
understood
what these
were
and it
understood
what I
said
but what's
more
impressive
is like
please
rewrite
the program
to check
for
crocodiles
and you
know it
what is it
going to do
here
let's see
it says
sure no
problem
writes the
function
is crocodile
if not
self.is
reptile
if not
self.has
scales
if not
self.has
long snout
oh my
gosh
like it
not only
remembered
oh yeah
there's
this new
version I
wrote in
the
garden
clause
format
but then
it
rewrote
the tests
I mean
and then
it's
explaining
to me
why
it wrote
it that
way
it's
just
it's
mind
blowing
like how
how much
you can
have
conversations
with this
and how
much
it
understands
things
like
code
or
physics
or
history
what do
you think
yeah it's
really
satisfying
I love
that it's
such a
powerful
generalist
at these
like things
that are found
on the
internet
so if it
like if it
exists and
it's in
the training
data it
can do so
good at
synthesizing
composing
bridging
between them
it's really
satisfying
so it's
really fun
asking it
to as you're
doing rewriting
changing
language
I've been
getting into
a lot more
JavaScript
because I'm
doing a
bunch more
like front end
stuff and
just I
sometimes will
write a quick
one liner in
Python
that I know
how to do
with a
list
comprehension
and then I'll
be like
make this
for me
in JavaScript
because I
can't figure
out this
like how to
initialize an
array with
integers in
it
it's great
for just
like really
quick spot
checks
and it also
seems to
know a lot
about like
really popular
frameworks
so you can
ask it
things that
are surprisingly
detailed about
like a
how would you
do cores
with requests
in fast
API
and it can
help you
find that
exact
middleware
you know
it's like
boilerplate-y
but it's great
that it can
just be a
source for that
this portion
of Talk Python
to Me is
brought to
you by
brilliant.org
you're a
curious person
who loves
to learn
about technology
I know
because you're
listening to
my show
that's why
you would
also be
interested
in this
episode's
sponsor
brilliant.org
is entertaining
engaging
and effective
if you're
like me
and feel
that binging
yet another
sitcom series
is kind
of missing
out on life
then how
about spending
30 minutes
a day
getting better
at programming
or deepening
your knowledge
and foundations
of topics
you've always
wanted to
learn better
like chemistry
or biology
over on
brilliant
has thousands
of lessons
from foundational
and advanced
math to data
science
algorithms
neural networks
and more
with new
lessons added
monthly
when you sign up
for a free
trial
they ask a couple
of questions
about what
you're interested
in as well as
your background
knowledge
then you're
presented with a
cool learning
path to get
you started
right where
you should
be
personally
I'm going
back to some
science foundations
I love chemistry
and physics
but haven't
touched them
for 20 years
so I'm looking
forward to playing
with PV
equals NRT
you know
the ideal gas
law
and all the
other foundations
of our world
with brilliant
you'll get
hands-on
on a whole
universe of
concepts
in math
science
computer science
and solve
fun problems
while growing
your critical
thinking skills
of course
you could just
visit brilliant.org
directly
its url is
right there
in the name
isn't it
but please
use our
link because
you'll get
something extra
20% off
an annual
premium
subscription
so sign up
today at
talkpython.fm
slash brilliant
and start a
7 day free
trial
that's
talkpython.fm
slash brilliant
the link is
in your podcast
player show notes
thank you to
brilliant.org
for supporting
the show
it's insane
I don't know
if I've got it
in my history
here
we're rewriting
our mobile
apps for
talkbython
training for
our courses
in
Flutter
and we're
having a
problem
downloading
stuff
concurrently
using a
particular
library
in
Flutter
and so
I asked
it
I said
hey
I want
some help
with a
Flutter
and Dart
programs
what do you
want
it says
I'm using
the
dio
package
do you
know
it
oh yes
I'm
familiar
it does
HTTP
client
stuff
for
Dart
okay
I want
to
download
binary
video
files
and a
bunch
of
them
given
a
URL
I want
to do
them
concurrently
with three
of them
at a
time
write the
code for
that
and boom
it just
writes it
like using
that library
I told it
about not
just Dart
so that's
incredible that
we can get
this kind
of assistance
for knowledge
and programming
like you'll
never find
I mean I take
that back
you might
find that
if there's
a very
specific
stack
overflow
question
or something
but if
there's not
a write-on
question for it
you're not
going to find
it
I love
when you
know the
stack
overflow
would exist
for a
variant
of your
question
but the
exact one
doesn't
exist
and you
have to
go grab
the three
of them
to synthesize
and it's
just great
at that
it also
is pretty
good at
fixing errors
sometimes it
can walk
itself into
lying to
you
repeatedly
but
that's
so problematic
yeah
but you can
also ask
it
here's
my
program
are there
security
vulnerabilities
or do
you see
any
bugs
and it'll
find them
yep
yeah
it's
nuts
so people
may be
wondering
we haven't
talked yet
about your
project sketch
why I'm
talking so much
about chat
CP
so that is
kind of
the style
of AI
that your
project brings
to pandas
which we're
going to get
to
but I want
to touch
on two
more really
quick AI
things that
we'll dive
into it
the other
is this
just around
images
just the
ability to
ask questions
you've already
mentioned
three
dolly
imagine
and then
the other
one I don't
remember from
Google that
they haven't
put out yet
a mid
journey is
another
just the
ability to
say hey
I want a
picture of
this
no actually
change it
slightly like
that
it's
mind
blowing
they're a lot
of fun
they're great
for sparking
creativity
or having
idea and
just getting
to see it
in front of
you
I think
it's more
impressive to
me
than even
this chat
GTP
telling me
I want a
GTP
I want an
artificial
intelligence
panda
and it
came up
and I
want it
photorealistic
in the style
of National
Geographic
and so
it gave
me
this panda
you can see
beautiful whiskers
but just
behind the
ear
you can see
the fur is
gone
and it's
like
an android
type of
creature
that is a
beautiful
picture
it's pretty
accurate
it's nuts
that I can
just go talk
to these
systems
and ask
them these
questions
I find it
interesting
comparing
the chat
GPT
and the
mid-journey
style
I completely
get it
it's very
visceral
it's also
from another
perspective
I think
of the
weights
and the
scale
of the
model
and
these
image
ones
that
solve
all
images
are
so
much
smaller
in
scale
than
these
language
ones
that
have
all
this
other
data
and
stuff. So it's fascinating how complex language is. Yeah, I know the smarts is so much less,
but just something about it actually came up with a creative picture that never existed.
Yeah. Right. You could show this to somebody like, oh, that's an artificial panda. That's
insane. Right. But it's, but I just gave it like a sentence or two. Yeah. Yeah. Yeah. I don't know.
Yeah. This, it's a sort of a technical interpreter, but I, I love it because it's
like this, it's just phenomenal interpolation. It's like through semantically labeled space. So
like the words have meaning and it understands the meeting and can move sliders of like, well,
I've seen lots of these machine things. I understand the concept of gears and this metal and this,
like the shiny texture and then the fur texture and like, they're very good at texture. It's a,
yeah, really great how it interprets all of that just to fit the, you know, the small prompt.
Yeah. There are other angles of which it's frustrating. Like I want it turned, I want it
in the back of the picture, not the, no, it's always in the center. One more thing really quick.
And this leads me into my final thing is, is a GitHub copilot. GitHub copilot is like this in
your editor, which is kind of insane, right? You can just give it like a comment or a series of
comments and it will write it. I think chat GDP is maybe more open-ended and more creative, but this
is, this is also a pretty interesting way to go. I'm a heavy user of copilot. I, if there's a,
there's a weird crux and I'm like slowly developing like a need to have this in my browser. I was a,
on a flight recently and was with the internet and copilot wasn't working. And I felt the, like,
I felt the difference. I felt like I was like walking through mud instead of just like actually
running a little bit. And I was like, Oh, I've been disconnected from my distributed mind. I am broken
partially. Yeah. So incredible. So the last part I guess is like, you know, what are the ethics
of this? Like I went on very positively about mid journey, but how much of that is trained on
copyright material or there's GitHub copilot. How much of that is trained on GPL based stuff that was
in GitHub. But when I use it, I don't have the GPL any longer on my code. I might use it on commercial
code, but just running it through the AI, does that strip licenses or does it not? There's a GitHub
copilotlitigation.com, which is interesting. I mean, we might be finding out. There's also
think Getty, I think it's the Getty images. I'm not 100% sure, but I think Getty images is suing
one of these image generation companies. I can't remember which one I don't maybe mid journey. I
don't think it's mid journey. I think it's stable diffusion, but anyway, it doesn't really matter.
Like there's a bunch of things that are pushing back against us. Like, wait a minute,
where did you get this data? Did you have rights to use this data in this way? And I mean,
what are your thoughts on this angle of AI these days?
Yeah. I know it sounds like I don't worry too much about it in either direction. I think I
believe in personal ethics. I believe in open source things, availability of things,
because it just sort of like accelerates collective progress. But that said, I also believe in like
slightly different like social structures to help support people. Like I'm a, I guess,
a person believer in things like UBI or something like that on that direction.
So when you combine those, I feel like it, you know, things sort of work out kind of well,
but when we like, but it is still a thing that like,
Be Right exists and that there is this sense of ownership and this is my thing. And I wanted to
put licenses on it. And, I think that this sort of story started presumably that I wasn't really
having this conversation, but like when the internet came around and search engines happened and like
Google could just go and pull up your thing from your page and summarize it in a little blob on the
page is, was that fair? What if it starts, you know, your shop and it allows you to go buy that
same product from other shops. Like it, I think that the same things are showing up and in the same way
that the web, like in the internet sort of, it's sort of, it was a large thing, but then it sort of,
I don't know if it got quieter, but it sort of became in the background. We sort of found new
systems. It stopped being piracy and CDs and the music industry is going to struggle. And Hey,
things like Spotify exist and streaming services exist. And like, I don't know what the next way
is.
They're doing better than ever basically. Yeah. Yeah. Yeah. So I think it's just evolution.
And then like the, some things will change and adopt some things will like fall apart and new
things will be born. I, that's just a great, it's a good time for lots of opportunity, I guess is the
part that I'm excited about.
Yeah. Yeah. Yeah. For sure. I think that's definitely true. It probably, you're probably right. It probably
will turn out to be, you know, old man yells at cloud cloud doesn't care sort of story, you know,
in the end where it's like, on the other hand, if, if somebody came back and said,
you know, a court came back and said, you know what, actually anything trained on GPL
and then you use copilot on it, that's GPL. Like that would have instantly mega effects. Right.
Yeah. I, yeah. And I, I guess there's also stuff like the, I don't, I didn't actually read the
article. I only saw the headline and you know, that's the worst thing to do is to repeat a thing,
which is a headline. But, there was that Italy thing that I saw about, like, I don't know.
Yeah. That was really clickbaity, but I didn't get a time to look at it yet. So yeah. You probably
ask chat to be to summarize for you. If as long as it can be a Bing, I guess, get that updated.
Yeah. Yeah. Yeah. Yeah. There's a lot of, there's a lot of things playing in that space,
right? Some different places. Okay. So yeah, very cool. But as a regular user, I would say,
you know, regardless of kind of how you feel about this, at least this is my viewpoint right now.
It's like, regardless of how I feel about which side is right in these kinds of disputes,
this stuff is out of the bag. It's out there and available and it's a tool. And it's like saying,
you know, I don't want to use spell check or I don't want to use some kind of like code checking. I just
want to write like in straight notepad because it's pure, right? Like sure you could do that,
but there's these tools that will help us be more productive and it's better to embrace them and know
them than to just like yell at them, I suppose. Yeah. A lot of accelerant you can get.
Uh, really speed up whatever you want to get done. Yeah, absolutely. All right. So speaking of
speeding up things, let's talk pandas and not even my artificial pandas, but actual programming pandas
with this project that you all have from approximate. Yeah. Approximate labs called sketch. So sketch is
pretty awesome. Sketch is actually why we're talking today because I first talked about this on Python
bytes and I saw this was sent over there by Jake Furman and to me and said, you should check this
thing out. It's awesome. And, yeah, it's pretty nuts. So tell us about sketch. Yeah. So, even
though I use a copilot as I sort of described already, and it's become a crux I found in Jupyter
notebooks when I wanted to work with data, it just didn't, it doesn't actually apply that. So on one side,
it was sort of like missing the mark at times. And so it was sort of like, how can I get this
integrated into my flow? The way I actually work in a Jupyter notebook, if maybe I'm working a Jupyter
notebook on a remote server and I don't want to set up VS Code to do it. So I don't have copilot at all.
Like there's a bunch of different reasons that I was just like in Jupyter. It's a very different IDE
experience. It is. Yeah. It's super different, but also you might want to ask questions about the data,
not the structure of the code that analyzes the data, right? Exactly. Yeah. And so just a bunch of that
type of stuff. And then also at the other side, I was trying to find something that I could throw together
that I thought was strong demonstration of the value approximate labs is trying to chase, but wouldn't
take me too much time to make. So it was a, oh, I could probably just go throw this together pretty quickly.
I bet this is going to be actually useful and helpful. And so let's just do that. And so through on top of
the actual library I was using, it was sketch. I put this on it and then shift it. So sort of shifted what the
project was. Yeah. Yeah. So you also have this other project called Lambda Prompt. And so were
you trying to play around Lambda Prompt and then like see what you could kind of apply here to leverage
it? Or is that the full journey I can get into is started with data sketches. I left my last job
to chase bringing the algorithm, like combining data sketches with AI, but just like the vague,
like at that level. Tell us what data sketches is real quick. Sure. Yeah. So a data sketch is a
probabilistic aggregation of data. So if you have, I think the most common one that people have heard of
is hyperloglog and it's used to estimate cardinality. So estimate the number of unique
values in a column. A data sketch is a class of algorithms that all sort of like use roughly fixed
width in binary, usually representations. And then in a single pass, so their ON will look at each row
and hash the row and then update the sketch or not necessarily hash, but they update this sketch
object. Essentially. Those sketch objects also have another property that they are mergeable. So you
have this like really fast ON to go bring that like to aggregate up and you get this mergeability. So you
can map reduce it in, you know, trivial speeds. The net result is that this like tight binary packed
object can be used to approximate measures you were looking for on the original data. So you could look
at, if you do a few of these, they're like theta sketches, you can go and estimate not just the
unique count, but you can also estimate if this one column would join well with this other column,
or you can estimate, Oh, if I were to join this column to this column, then this third column that
was on that other table would actually be correlated to this first column over here. So you get these,
a bunch of different distributions, you get a whole bunch of these types of properties.
And each sketch is sort of just, I would say, algorithmically engineered, like very, very
engineered to be like information theory optimal at solving one of those like measures on the data.
And so tight packed binary representations.
All right. So you thought about, well, that's cool, but chat CTP is cool too.
Yeah.
What else?
The core thing was, so those representations aren't usable by AI right now. And when you actually go and
use GPT three or something like this, you have to figure out a way to build the prompt to get it to do
what you want. This was especially true in a pre instruction tuning world, you had to really like, you had to
play the prompt engineer role even more than you have to now. Now you could sort of get away with describing it to
ChatGPT. And one of the things that you really have to like, play the game of is how do you get all the
information it's going to need into this prompt in a succinct, but good enough way that it helps it do
this. And so what sketch was about was, rather than just looking at the context of the data, like the
metadata, the column names and the code you have, also go get some representations of representation of the
content of the data, turn that into a string, and then bring that string in as part of the prompt.
And then when it has that, it should understand much better at actually generating code, generating
answers to questions. And that's what that sketch was a proof of concept of that, that worked very well.
It really quickly showed how valuable actual data content context is.
Yeah, I would say people are resonating with people. It's got 1.5,000 stars on GitHub.
And it looks about six months old. So that's pretty good growth there.
Yeah, January 16th was the day I posted it on Hacker News. And it had three,
there was an empty repo at that point.
Okay, three stars. It's like me and my friends. Okay, cool. So this is a tool that basically patches
pandas to add functionality or functions, literally to pandas data frames that allows you to ask
questions about it, right?
Yep.
So what kind of questions can you ask it? What can it help you with?
Yeah, so there's two classes of questions you can ask, you can ask it, the ask type questions,
these are sort of from that summary statistics data. So from the general, you know, representation of your
data, ask it to like, give you answers about it, like, what are the columns here, you sort of have
a conversation where it sort of understands the general under like shape of the data, general
distributions, things like that, number of uniques, and like give that context to it, ask questions of that
system. And then the other one is ask it how to do something. So you specifically can get it to write
code to solve a problem you have, you describe the problem you want, and you can ask it to do that.
Right. I've got this data frame, I want to plot a graph of this versus that, but color by the other
thing.
Yep. And in the data space world, what I sort of decided to do is like in the demo here is just sort of
walk through what are some standard things people want to ask of data, like, like, what are those common
questions that you hear, like, in Slack between, you know, like, business team and an analyst team. And it's just
sort of like, Oh, can you do this? Can you get me this? Can you tell me if there's any PII? Is this safe to send?
Can I send the CSV around? Can you clean up this CSV? Oh, I need to load this into our catalog. Can you
describe each of these columns and check the data types all the way to can you actually go get me
analytics or plot this?
Yeah. Awesome. So and it plugs right into Jupyter Notebooks, so you can just import it and basically
installing Sketch, which is a pip or Conda type thing, and then you just import it, and it's good to go,
right? Yep. Using the Pandas extensions API, which allows you to essentially hook into their data
frame callback and register a, you know, a function. Interesting. So it's not as jammed on from the
outside. It's a little more, plays a little nicer with Pandas rather than just like, we're going to go
to the class and just tap on it. Yeah, yeah. I, yeah. Not full monkey patching here. It's a,
it's like hack supported, I think. I don't, I don't see it used often, but it is somewhere in the docs.
Excellent. But here it is. So what I wanted to do for this is there's a, an example that you can do,
like if you go to the repo, which obviously I'll link to, there's a video, which I mean,
mad props to you because I review so many things, especially for the Python Bytes podcast, where
there's a bunch of news items and new things we're just going to check out. And we'll, we'll find people
recommending GUI frameworks that haven't, not a single screenshot or other types of things. Like,
I have no way to judge whether this thing even might look like that. What does it even make?
I don't even know, but somebody put a lot of effort, but they didn't bother to post an image. And you
posted a minute and a half animation of it going through this process, which is really, really
excellent. So people can go and watch that one minute, one minute 30 video. But there's also a
collab opening Google collab, which gives you a running interactive variant here. So you can just
follow along, right? And play these pieces requires me to sign up on and run it. That's okay.
Let me talk people through some of the things it does. And you can tell me what it's doing,
how it's doing that, like how people might find that advantageous. So import sketch, import pandas
as PD standard. And then you can say pandas read CSV and you give it one from a, like a,
some example CSV that you got on your, one of your GitHub repos, right? Or in your account.
Yeah. I found one online and then added just random synthetic data to it.
Yeah. Like, Oh, here's a data dump. No, just kidding.
So then you need to go to that data frame called sales data. You say dot sketch dot ask as a string,
what columns might have PII personal identifying information in them?
Awesome. And so it comes, tell me how that works and what it's doing here.
So it does, I guess it has to build up the prompt, which is sent to GPT. So to open AI specific
completion endpoint, the building up the prompt, it looks at the data frame. It does a bunch of
summarization stats on it. So it calculates uniques and sums and things like that. There's two modes in
the backend that either does sketches to do those, or it just uses like DF dot describe type stuff.
And then it pulls those summary stats together for all the columns, throws it together with my,
the rest of the prompt I have, you can, we can go find it, but then it sends that prompt.
Actually, it also grabs some information off of inspect. So it sort of like walks the,
the stack up to go and check the variable name because the data frame is named sales data.
So it actually tries to go find that variable name in your call stack so that it can, when it writes
code, it writes valid code, puts all that together, send it off to open AI, gets code back, uses Python
AST to parse it, check that it's valid. If it's not valid Python code, or you tried to import something
that you don't have, it will ask it to rewrite once. So this is sort of like an iterative process. So it
takes the error or it takes the thing and it sends it back to open AI. It's like, Hey, fix this code.
And then it, or in this case, sorry, ask, it actually just takes this, it sends that exact
same prompt, but it just changes the last question to, can you answer this question off of the information?
This portion of talk Python me is brought to you by us over at Talk Python Training with our courses.
And I want to tell you about a brand new one that I'm super excited about. Python web apps that fly
with CDNs. If you have a Python web app, you want it to go super fast. Static resources,
turn out to be a huge portion of that equation. Leveraging a CDN could save you up to 75% of your
server load and make your app way faster for users. And this course is a step-by-step guide on how to do
it. And using the CDN to make your Python apps faster is way easier than you think. So if you've
got a Python web app and you would like to have it scaled out globally, if you'd like to have your users
have a much better experience and maybe even save some money on server hosting and bandwidth,
check out this course over at talkpython.fm/courses. It'll be right up there at the top.
And of course the link will be in your show notes. Thank you to everyone who's taken one of our courses.
It really helps support the podcast. I'm back to the show.
And so that sounds very, very similar to my arrow program. Rewrite it with garden clauses, redo it.
Like you kind of, I gave you this data in this code and I asked you this question and you can have a
little conversation, but at some point you're like, all right, well, we're going to take what it gives me
after a couple of rounds at it. Right.
Yeah. I take the first one that doesn't, that like passes an import check and passes AST linting.
There was a, the, when you use small models, you run into not valid Python a lot more, but with these
ones, it's almost always good.
It's ridiculous. Yeah. Yeah. Yeah. It's crazy. Okay. So it says the columns that might have PII
in them are credit card, SSN and purchase address. Okay. That's pretty excellent. And then you say,
all right, sales data dot sketch dot ask. Can you give me friendly name to each column and output this
as an HTML list, which is parsed as HTML and rendered in Jupyter notebook accurately. Right. So it says
index. Well, that's an index.
This one ends up being the same.
It's not a great, this one is not a great example because it doesn't have to like infer
because the names are like order space date, right? Instead of order, like maybe lowercase
O and then like attached a big D or whatever, but it'll give you some more information. You
can like kind of ask it questions about the type of data, right?
Yeah, exactly. I found this is really good at if you play the game and you just name all
your columns, like call one, call two, call three, call four, and you ask it, give me new column
names for all of these. It gives you something that's pretty reasonable based off of the data.
So pretty useful.
Okay. So it's like, oh, these look like addresses. So we'll call that address. And this looks like
social security numbers and credit scores and whatnot.
Yep. Yep. So it can really help with that quick first onboarding step.
Yeah. So everyone heard it here first. Just name all your columns. One, two, three, four,
and then just get help. Like AI, what do we call these? All right. So the next thing you did in this
demo notebook was you said sales data dot sketch dot. And this is different before I believe,
because before you were saying ask, and now you can say how to create some derived features from the,
from the address. Tell us about that.
Yeah. This is the one that actually is the code writing. It's essentially the exact same prompt,
but the change is the very end. It says like, return this as Python code that you can execute to do
this. So instead of answering the question directly, answer the question with code that will answer the
question.
Right. Write a Python line of code that will answer this question given this data, something like that.
Yep. Yep. Something like that. I don't remember exactly anymore. It's been a while, but yeah,
some I've iterated a little bit until it started working and I was like, okay, cool. And so,
uh, ask it for that. And then it spits back code. And that was, it sort of, it sounds overly simple,
but that was it. That was like, that was the moment. And I was just like, oh, I could just ask it to do my
analytics for me. And it's just all the, every other feature just sort of became like apparently
solvable with this. And the more I played with it, the more it was just, I don't have to think about,
I don't even have to go to Google or stack overflow to ask the question, to get the API stuff for me.
I could, from zero to I have code that's working is one step in Jupyter.
So you wrote that how to, and you gave it the question and then it wrote the lines of code and you just
drop that into the next cell and just run it. Right. And so for example, in this example, it said, well,
we can come up with city state and zip code and by writing a vector transform by passing a lambda,
that'll pull out, you know, the city from the string that was the full address and so on. Right.
Yeah. That's pretty neat.
Yeah. It's fun to see what it, what it does. Not again, not any of these things are always
probabilistic, but it also usually serves as a great starting point if, even if it doesn't get it
right.
Yeah. Sure. You're like, oh, okay. I see. Maybe that's not exactly right. Cause we have Europeans
in their city, maybe in their zip code or in different orders sometimes, but it gives you
something to work with pretty quickly. Right. By asking just a, what can I do?
And then another one, this one's a little more interesting instead of just saying like, well,
what other things can we pull out? It's like, this gets towards the analytics side, right? It says,
get the top five grossing states for the sales data. Right. And it writes a group by some sorts,
and then it does a head given five. And that's pretty neat. Tell us about this. I mean, I guess
it's about the same, right? Just ask more questions.
They all feel pretty similar to me. I think, I guess I could jump towards like things that
I wanted to put next, but I didn't, we're not reliable enough to like really make the cut.
I wanted to have it go like in my question was like, go build a model that predicts sales for the
next six months and then plot it on a 2d plot with a dotted line for the predicted plot. And like,
it would try, but it would always do something off. And I found I always had to break up the
like prompted to like smaller, smaller intern level code back. Yeah.
Yeah. It was fun getting it to train models, but it was also its own like separate thing. I sort of
didn't play with too much. And there's another part of sketch that I guess is not in this notebook. I
didn't realize. Yeah. Because you have to use the open AI API key, but it's the sketch apply. And
that's the, I'll say this one is another just like power tool. This one has like, I don't really talk
about, I don't even include it in the video because it's not just like as plug and play, you do have to
go set an environment variable. And so it's like, yeah, that's one step further than I want to,
I don't, it's not terrible, but it's a step. And so what it does is it lets you apply a completion
endpoint of whatever your design row wise. So every single row, you can go and apply and run something.
So if every row of your pandas data frame is a, some serialized text from a PDF or something,
or a file in your directory structure, and you just load it as a data frame, you can do dot
df.sketch.apply. And it's almost the exact same as df.apply. But the thing you put in as your function
is now just a Jinja template that will fill in your column variables for that row and then ask GPT to
continue completing. So I think I did silly ones, like here's a few states. And then the prompt is
extract the state for it. Or so I think, right, extract the capital of the state. Yeah. Yeah. So
just pure information extraction from it, but you can sort of like this grows into a lot more.
So does that come out of the data? Or is that coming out of open AI where like it sees what is
the capital of state and it sees New York? It's like, okay, well, all right, Albany.
Yeah. So this is purely extracting out of the model weights. Essentially, this is not like a factual
extraction. So this is probably a bad example because it's like it. But the thing that actually,
actually, the better example I did once was, what is like some interesting colors that are
good for each state? And it like just came up with a sort of like flaggish colors or sports team colors.
That was sort of fun when it wrote that as hex. You can also do things like if you have a large text
document or you can actually, I'll even do the more common one that I think everybody actually wants
is you have messy data. You have addresses that are like syntactically messy and you can say,
normalize these addresses to be in this form. And you sort of just write one example. It's a run
dot apply and you get a new column that is that cleaned up data. Yeah. Incredible. Okay. A couple
things here. It says I can use, you can directly call open AI and not use your endpoint. So at the
moment it kind of proxies through web service that you all have that somehow checks stuff or what does
that do? Yeah. It was just a pure ease of use. I wanted people to be able to do pip install and
import sketch and actually get it because I know how much I use things in, in a collab or in Jupyter
notebooks on weird machines and remembering an environment variable, managing secrets. It's like
this whole overhead that I want to deal with. And so I wanted to just offer a lightweight way if you
just want to be able to use it. But I know that that's not sufficient for secure. If people are going
to be conscious of this things and want to be able to, you know, not go through my proxy thing that's
there for help. So sure.
Offer this up.
What's next? Do you have a roadmap for this? Are you happy where it is and you're just letting it be or
do you have grand plans?
I don't have much of a roadmap for this right now. I'm actually, I guess there's like grand roadmap,
which is like at the company scale, what we're working on. I would say that if this, we're really trying to
solve data and with AI just in general. And so these are the types of things we hope to open source and
just give out there, like actually everything we're hoping to open source. But the starting place is
going to be a bunch of these like smaller toolkits or just utility things that hopefully save people
time or very useful. The grand thing we're working towards, I guess, is this more like the, it's the
full automated data stack. It's like the dream I think that people have wanted where you just ask it
questions and it goes and pulls the data that you need. It cleans it. It builds up the full pipeline.
It executes the pipeline. It gets you to the result and it shows you the result. And you look,
you can inspect all of that, that whole DAG and say, yes, I trust this. So we're working on getting
full end to end.
So when I went and asked about that Arrow program, I said, I think this will still do it. I think this
will probably work again. And it did, which is awesome. Just the way I expected. But, you know,
AI is not as deterministic as read the number seven. If seven is less than eight, do this,
right? Like what is the repeatability? What is the sort of experience of doing this? Like I ran it,
I ran it again. Is it going to be pretty much the same or is it going to have like, what's the mood
of the AI when it gets to you?
This is sort of a parameter you can, there's a little bit of a parameter you can set if you want
to play that game with the temperature parameter on these models at higher and higher temperatures,
you get more and more random, but it can also truly be out of left field random if you go too
high temperature.
Okay. But you get maybe more creative solutions.
Yeah. You could sometimes get that. And as you move towards zero, it gets more and more
deterministic. Unfortunately for really trying to do some like good provable, like sort of like
build chain type things with like hashing and caching and stuff. It's not fully deterministic,
even at zero temperature, but that's just, I think it's worth thinking about, but at the same time,
run it once, see the answers that it gives you comment that business out and just like put that
as markdown, you know, freeze it. It like us memorialize it in markdown because you don't need
to ask it over and over what columns have PII. Like, well, probably the same ones as last time.
We're just kind of like, right, these columns, credit card, social security and purchase address,
they have, have that. And so now, you know, right. Is that a reasonable way to think about it?
I think, yeah, if you, if you want to get determinism or the performance is a thing that
you're worried about, yeah, you can always cash. I think however you do it, comments or actually
with systems.
Sure. Sure. Sure. Sure. Or that like, like, how do I, you know, how do I do that group by sorting
business? Like you don't have to ask that over and over once it gives you the answer.
Yeah. Yeah. My workflow, when I use sketch, definitely I asked the question, I copy the
code and then I go delete the question or ask it a different question for my next problem that I have.
Yeah. I like, it's not code that it is a little bit like a vestigial when it, when you like save
your notebook at the end and you sort of want to go back and delete all the questions you asked because
you don't need to rerun it when you actually just go to execute the notebook later. But yeah,
that makes a lot of sense. And plus you look smarter if you don't have to show how you got
the answers.
Look at this beautiful code that's even commented.
Yeah, exactly. I guess you could probably ask it to comment your code, right?
Yeah. You can ask it to describe. There's been some really cool things where people will throw
like assembly at it and ask it to translate to different like languages so they can interpret
it. Or you could do really fun things like cross language, cross, I guess I'll say like levels
of abstraction. You could sort of ask it to describe it like at a very top level, or you can get really
precise, like for this line, what are all the implications if I change a variable or something like that?
Yeah, that's really cool. I suppose you could do that here. Can you can you converse with it? You
can say, okay, you gave me this. Does it I guess what's the word? Does it have like tokens in context
like chat HTTP does? Can you say, okay, that's cool. But but I want as integers, not as strings.
I don't know. Yeah, I did. I did not include that in this. There was a version that had something like
that, where I was sort of just keeping the last few calls around. But it quickly became it didn't align
with the Jupyter IDE experience, because you end up like scrolling up and down. And it you have too much
power over how you execute in a Jupyter notebook. So your context can change dramatically by just scrolling
up. And trying to via inspect look across different like, across a Jupyter notebook is just a whole
other nightmare. So yeah, I didn't try and like extract the code out of the notebook so that it
could understand the local context. You could go straight to chat HTTP or something like that,
take what it gave you and start asking it questions.
Okay, so another question that I had here about this. So in order for that to do its magic,
like you said, the really important thought or breakthrough or idea you had was like, not just
the structure of the pandas code or anything like that, but also a little bit about the data.
What is the privacy implications of me asking this question about my data? Suppose I have
super duper secret CSV. And should I not ask or how to on it? Or what is the story there?
What's the, if I work with data, how much sharing do I do of something I might not want to share if I
ask a question about it?
I'd say the same discretion you'd use if you would copy like a row or a few rows of that data into
a, into ChatGPT to ask it a question about it.
Okay.
Is the level of concern I guess you should have like on the specifically, I am not storing these
things, but I know is at least it was, it seems like they're going to start getting towards like a 30
day thing. But, so there's a little bit of, yeah, I mean, you're sending your stuff over the
wire, like over network, if you do this and to use these language models until they come local,
until these things like llama and alpaca get good enough that they're, yeah, they're going to be
remote. Actually, that could be a fun, sorry. I just now thought that could be a fun thing. Like
just go get alpaca working with a sketch so that it can be fully local.
Interesting. Like a privacy preserving type of deal.
Yeah. I hadn't actually, yeah, that's the, that's the power of these, smaller models that are
almost good enough. I could probably just like quickly throw that in here and see if it,
yeah, maybe it's a wider audience.
You have a option to not get through your API, but directly go to open AI. You could have another
one to pick other, other options, right? Potentially.
Yep. Yep. Yep. The, interface to these, one thing that I think is not, maybe it's talked
about it more than other places, but I haven't heard as much like excitement about it is that these,
uh, the APIs have gotten pretty nice for this whole space. the, they're all like the idea of a
completion endpoint is pretty straightforward. You send it some amount of text and it will continue
that text. And it's such a, it's so simple, but it's so generalizable. You could build so many
tools off of just that one API endpoint essentially. And so combine that with an embedding endpoint and
you sort of have all you need to, to make complex AI apps.
It's crazy. Speaking of making AI apps, maybe touch a bit on your, your other projects, Lambda.
So yeah, Lambda. Yeah.
But before you get into it, mad props for like Greek letter, like that's a true physicist or
mathematician that I can appreciate that there.
Yeah. That was, I was excited to put it everywhere, but then of course,
these things don't playing, playing games with character sets and websites. I'm the one that
causes, I both feel the pain, have to clean the data that I also put into these systems.
So yeah. Yeah. People are like a prompt and why is the a so italicized? I don't get it.
Yeah. Okay. Yeah. So yeah. Yeah. So this one came, I was, working with, this is pre GPT.
This is October. I guess it was right around ChatGPT coming out like around that time.
But I was, I was really just messing around a lot with, completion endpoints as we were talking.
And I kept rewriting the same request boiler over and over. And then I also kept rewriting
f-strings that I was trying to like send in. And I was just like, ah, Jinja templates solved this
already. Like there already is formatting for strings in Python. Let me just use that, compose that into a
function. And there's, let me call these completion endpoints. I don't want to think of them as like
API endpoint or RPC is a nice mental model, but I want to use them as functions. I want to be able to
put decorators on them. I want to be able to use them both async or not async in Python. I want to,
I just want to have this as a thing that I can just call really quickly with one line and just do
whatever I need to with it. And so through this together, it's very simple. Like, honestly, I mean,
like the hardest part was just getting all the layers of, there's actually two things you
can make a prompt that then, cause I wrap any function as a prompt. So not just, these
calls to GPT and then I do tracing on it. So as you like get into the call stack, every input and output
is you can sort of like get hooked into and trace with some like call traces. So there's a bunch of just
like weird stuff to make the utility nice, but functionally, as you can see here on it's,
you just import it, you write a Jinja template with the class, and then you use that object that
comes back as a function and your Jinja template variables get filled in. And your result is the
text string that comes back out of a GPT. Interesting. And people probably, some people
might be thinking like Jinja, okay, well I got to create an HTML file and all that, like not just a
string that has double curlies for turning stuff into like strings within the string, kind of a different
way to do f-strings as you were hinting at. Yeah. Yeah. There was a two pieces here. I realized as
I was doing this also, I think I sort of mentioned with a sketch I do. I really often was taking the
output of a language model prompt, doing something in Python, or actually I can do a full example of
the SQL writing like exploration we did, but, we would do these things, that were sort of run
GPT three to ask it to write the SQL. You take the SQL, you go try and execute it, but it fails for
whatever reason, or you, and you take that error, you say, Hey, rewrite it. So we talked about that
sort of pattern, which is sort of like rewriting. Another one of the patterns was increase the
temperature, ask it to write the SQL. You get like 10 different SQL answers in parallel. And this is where
the async was like really important for this. Cause I just wanted to use asyncio gather and run all 10
of these truly in parallel against the open eye endpoint, get 10 different answers to the SQL,
run all 10 queries against your database, then pull on what the most common, like of the ones that
successfully ran, which ones gave the same answer the most often, then that's probably the correct
answer. And, just chaining that stuff. It's like very pythonic functions. Like you can really
just imagine like, Oh, I just need to write a for loop. I just need to run this function, take the
output feed into another function, very procedural. But when you, all the abstractions in the
open at open AI API, the things like just everything else, there was nothing else really at the time,
but even the new ones that have come out like Lang chain that have sort of like taken the space by
storm now are not really just trying to offer the minimal ingredient, which is the function.
And to me, it was just like, if I can just offer the function, I can write a for loop. I can write,
I can store a variable and then keep passing it into it. You could do so many different
emergent behaviors with just starting with the function and then simple Python, scripting
on top of it.
And there's some interesting stuff here, land of prompt. So you can start, you can kind of start
it, set it. I don't know what chat GDP, you can tell it a few things. I'm going to ask you a question
about a book. Okay. The book is a choose your own adventure book. Okay. Now here I'm going to like,
you can prepare it, right? There's probably a more formal term for that, but you can do this here.
Like you can say, Hey system, you are a type of bot. And then you, that creates you an object that
you can have a conversation with. And you say, what should we get for lunch? And your type of bot is
pirate. And then so to say, as a pirate, I would suggest we have some hearty seafood or whatever,
right? Like that's, that's beyond what you're doing with sketch. I mean, obviously this is not so much
for code. This is like conversing with Python rather than in Python. I don't know. And your editor.
Yeah. This one was the open AI chat API endpoint came out and I was just like, Oh, I should support
it. So that's what this, I wanted to be able to Jinja template inside of the conversation. So you
can imagine a conversation that is prepared with like seven steps back and forth, but you want to hard
code with the conversation, like how the flow of the conversation was going. And you want to template
it so that like on message three, it put your new context problem on message four, it put the output
from another prompt that you ran on message five. It is this other data thing. And then you ask it to
complete this, the intent of like, it's arbitrarily complex, but still something like that
would be, you know, just three lines or so in Lambda prompt. The idea was that it would offer up a really
simple API for this. Well, other thing that's interesting is of an async and async version. So that's,
that's cool. People can, can check that out. Also a way to make it a hosted as a web service with say
like FastAPI or something like that. Yeah. And you can make it a decorator if you like an app prompt
decorator. Yeah. On any function you can just throw app prompt and it, it wraps it with the same class
so that all the, all the magic you get from that works. The server bit is I took, so FastAPI has
that sort of like inspection on the function, part. I did a little bit of middleware to get the
two happy together. And then all you have to do is import FastAPI and then run, you know, G unicorn
that app. And, it's two lines and any prompts you have made become their own independent rest
endpoint where you can just do a get or a post to it. And it returns the response from calling the prompt.
But these prompts can also be these chains of prompts. Like one prompt can call another prompt,
which can call another prompt. And those prompts can call async to not async back to async and things
like that. And it should work. Pretty sure this one actually, I did test everything as far as I know,
I'm pretty sure I've got pretty good coverage. So yeah, super cool. All right. Well get a little
short on time, but I, I think people are going to really, really dig this, especially sketch. I think
there's a lot of folks out there doing pandas that would love an AI buddy to help them
do things like not just analyze the code, but the data as well.
Yeah. Just, I think, anybody's, I know it's for me, but it's just like copilot in
VS Code ID, sketch in your Jupyter ID, takes almost nothing to add. And you,
whenever you're just sort of sitting there, you think you're about to alt tab to go to Google. You
could just try the sketch.ask and it's surprising how often that sketch.ask or sketch.howto gets you
way closer to a solution without even having to leave the, you don't even have to leave your,
your environment. It's like a whole other level of autocomplete for sure. And super cool. All right.
Now, before I let you out of here, you got to answer the final two questions. If you're going to write
some Python code and it's not a Jupyter notebook, what editor are you using? It sounds to me like you may
have just given a strong hint at what that might be. Yeah. I've switched almost entirely to VS Code.
and I've been really liking it with the remote development and, like it's just, I work
across like many machines, both cloud and local and some like five, six different machines are my
like primary working machines. And I use the remote, VS Code thing. And it just, I have a unified
environment that gives me terminal, the files and the code all in one and copilot on all of them.
Yeah. It's wild. All right. And then notable pipe UI package. I mean, pip install sketch,
you can throw that out there if you like. That's pretty awesome. But anything you've run across
you're like, Oh, this is people should know about this. Yeah. It doesn't have to be popular. Just
like, Oh, this is cool. In the, I guess these, these two are very popular, but, in the data
space, I really, I'm a huge fan of, Ray and, also arrow. Like I use those two tools as like
my backend bread and butter for everything I do. And so those have just been really great work.
Apache arrow. Right. And then Ray, I'm not sure. Yeah. Ray is a distributed, scheduling compute
framework. It's sort of like a, right. I don't know what they, yeah. I remember seeing about this.
Yeah. This is a, it is, I'm parsing, he didn't talk about other things, but I'm like parsing common
crawl, which is like 25 petabytes of data. And, Ray is great. It's just the workhorse. It power is
really useful. Like I find it's so snappy and good, but it offers everything I need in a distributed
environment. So I can write code that runs on a hundred machines and not have to think about it.
It works really well. That's, that's pretty nuts. Not as nuts as chat GDP and mid journey,
but it's still pretty nuts. So before we call it a date, do you want to tell people about approximate
labs? It sounds like you guys are making some good progress. I might have some, some jobs for people
to work in this kind of area as well. Yeah. So, we're, we're working at the intersection
of, AI and tabular data. So anything related to these training, these large language models,
and also, tabular data. So things with columns and rows, we are trying to like solve that problem,
try and bridge the gap here. Cause there's a pretty big gap. We have three main initiatives
that working on, which is we're trying to build up the data set of data sets. So just like the pile
or the stack or lay on five B these like big data sets that were used to train all these big
models. We're making our own on tabular data. We are training models. So this is actually
training large language models, doing these training, these full transformer models.
And then we're also building apps like sketch, like UIs, things that are actually there to help
make data more accessible to people. So anything that helps people get value from data and make it open
source. That's what we're working on. We just, raised our seed round. So we are now officially
hiring. So, looking for people who are interested in the space and who are enthusiastic
about these problems. Awesome. Well, very exciting demo libraries, I guess, however you call them.
But I think this, I think these are neat. People are going to find a lot of cool uses for them. So
excellent work and congrats on all the success so far. It sounds like you're just starting to take
off. Yeah. Thank you. All right, Justin, final call to action. People want to get started.
Let's pick sketch. People want to get started with sketch. What do you tell them?
Just, pip install it. Give sketch a, give sketch a try, pip install it,
import it, and then throw it on your data frame. Awesome. And then ask it questions or how tos.
Yeah. Yeah. Yep. Whatever you want. if you really, if you really want to, and you,
you trust the model, like throw some, applies and have it clean your data for you. Cool.
Awesome. All right. Well, thanks for being on the show.
Come in here and tell us about all your work. It's great. Yeah. Thank you. Yeah. See you later.
Thanks for having me.
This has been another episode of talk Python to me. Thank you to our sponsors. Be sure to check out
what they're offering. It really helps support the show. Stay on top of technology and raise your value
to employers or just learn something fun in STEM at brilliant.org. Visit talkpython.fm/brilliant
to get 20% off an annual premium subscription. Want to level up your Python? We have one of the largest
catalogs of Python video courses over at talk Python. Our content ranges from true beginners
to deeply advanced topics like memory and async. And best of all, there's not a subscription in
sight. Check it out for yourself at training.talkpython.fm. Be sure to subscribe to the show,
open your favorite podcast app and search for Python. We should be right at the top. You can also find
the iTunes feed at /itunes, the Google play feed at /play and the direct RSS feed at slash
RSS on talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of
the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at
talkpython.fm/youtube. This is your host, Michael Kennedy. Thanks so much for listening. I really
appreciate it. Now get out there and write some Python code.
I'll see you next time.