Have you heard of awesome lists? They're, well, pretty awesome, gathering up the most loved
libraries and packages for a given topic. While most lists cover awesome developer tools and
libraries, we don't have many examples of awesome applications, both for use and as examples to
draw from. That's why Mahmoud Hashemi decided to create awesome Python applications, and you're
about to dive headfirst into them. This is Talk Python to Me, episode 234, recorded September 24th,
2019.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm,
and follow the show on Twitter via at talkpython. This episode is brought to you by Linode and
Tidelift. Please check out what they're offering during their segments. It really helps support the
show. Mahmoud, welcome back to Talk Python to Me.
It's good to be back.
Man, it's great to be back. It wasn't that long ago that you were on Python Bytes,
and we still didn't get a chance to catch up because you were covering for me.
Yeah, but it was a great time. Yeah, happy to fill in any time. But really,
who could fill in your shoes?
Oh, man.
Thanks for that. You've been on the show before. I think the first time was quite early in the show's
history. You came to talk about a really interesting topic, Enterprise Python, right? Python being used
within the enterprise. We talked about a bunch of examples of that. And I feel like this is the
open source equivalent of that story just a little bit.
I was thinking about that today. Yeah, you're right. It is kind of similar,
different in a lot of ways. I think a lot more fun, but we'll get to that part. But yeah,
that was back in 2016. Time flies.
Yeah, time does fly. We've been at this stuff for a while.
I was at PayPal.
Yeah, that's right. And yeah, I mean, I guess that's probably a good time to just ask you,
what have you been up to these days? What's going on in your world?
I sort of had my fill of moving around different teams at PayPal, seeing how that whole business worked.
And it was cool to work in the enterprise. And I wanted to sort of like stay with that. So
I wanted to like kind of grow my own autonomy, as well as like just sort of see how startups worked.
I am in Silicon Valley, after all, you know, and PayPal is a startup in many ways, but it started
in like, what, 1998. So it's mostly started.
Yeah, it's mostly started at this point. You'd be surprised in some ways. But yeah.
Anyways, but basically, yeah, so I went to like a Series C company, did that for a while. It was
pretty cool. And then there's not too many teams, you can move around there. And like,
you know, so once you've learned it, you kind of learned it. And then I was like, okay, well,
let's come to a Series A company. And so for the past couple years, I've been at Simple Legal
here in Mountain View. And yeah, it's been really good. When I joined, the team was like four people
and or rather than we had like four engineers. And then now I think we're at like a dozen,
got a small team. I'm like principal engineer, do a lot of like code review,
architecture review, but also a fair amount of coding. And yeah, just having a blast.
That's cool. That looks like a fun product or service to work on. Is there a lot of Python
happening?
Best part, right? Like it's, it's not like PayPal where you're jockeying, like, you know,
for different technologies and so forth. It's like what we say goes, right? And we pick the
best technology, Python, Postgres, everything that everything we love. So we got that autonomy down.
But like, yeah, just to be clear, it's not like the most exciting company on the outside.
But I always tell people this, like, try to go for a boring company and then make it fun. Do it in
like, like choose to a boring company so that you can do things in a fun way. If you choose something
that's too exciting on the outside, then your day to day is just going to be a blur of boring stuff.
So yeah, we've done a lot of open source here. And I could go on about it for hours. But we got
other things to talk about.
Okay, some awesome stuff to talk about for sure. Yeah, I think there's a lot to that,
you know, people see sort of the company as the exciting thing, right? Like working for Tesla
or something like that. But, you know, if what you end up doing is writing C code to do a bunch of
internal boring stuff, well, then like, it doesn't matter how exciting the company is. That's not that
much fun.
No, exactly. Just bash scripts to shovel around logs. That's not exciting to me.
It doesn't matter if the logs are...
You know, I don't care how exciting the brand is.
Yes, exactly. Exactly. Cool. All right. Well, let's talk about some awesome things. Now,
there's been this history of awesome lists. Do you want to maybe just tell people about this,
this idea of awesome lists? I don't know how much of the history you know, but I know that you
must be involved because you've created one.
I'm actually not that great of an expert on these things because sort of when they started popping
up, I was already pretty well versed in, for instance, awesome Python. It's a huge repo,
like tons of contributors, been around a long time, got tons of content, and it's quite awesome in many
ways. But I rarely like refer to it or find much new there because I was sort of like, you know,
already along my Python path. But I'm sure it's out. It's a great resource for people out there. I refer
people to it all the time. So yeah, but this awesome list thing is a phenomenon. It's like a
meme on GitHub where it's like, I'm going to make a list of links, which are awesome for some definition
of awesome.
Yeah, for some definition of awesome. Exactly. And I don't even think it originated with Python. I feel
like there was some PHP stuff going on.
No, definitely not.
Yeah, awesome Python is really good. People should check that out. That's interesting. There's a bunch
of good libraries there. And there's also more focused ones. Like there's, I recently ran across
awesome ASGI for async server, like basically for async web frameworks and other AIO type of things,
asyncio things.
Yeah, it's just the default architecture for like linked content on GitHub, you know, if you want it
to be approachable. But yeah, there's this guy like Cinder Sorhus and basically, yeah, I'm pretty sure
that he's like node guy. Yeah. Anyways, he like has this sort of awesome authority. And it's like the
meta list. It's the list of lists that point to all the other awesome lists.
Oh, how cool. I didn't know there was an awesome list of awesome lists. That's super meta.
Oh, well, that's why they all have this badge with the cool sunglasses on it. And you know,
anyways, that's got like 120,000 stars, which is like, you know, it's good. Like, like, frankly,
so much of how we learn as a community is driven through streams of content that has to be manually
republished out. I'm thankful for anything out there that sort of constitutes a reference,
you know, that like a sort of institution I can go and look at rather than refreshing a Twitter stream
and hoping that somebody like has said something of value to me. Awesome lists are in the end on like
net. Pretty awesome.
I would say, yeah, they're aptly named. And I do like them. I feel like they're a little bit
vetted, right? Not it's not just a Yahoo of all Python things, right? It's not like PyPI or whatever.
It's these are the things people found to be extra good in these categories. I guess it's kind of like
Yahoo a little bit anyway, early, you know, 1996 Yahoo. So maybe tell us about your awesome list and like
how you came to come up with it and things like that.
Sure. So yeah, I didn't really set out to create an awesome list. I sort of backed my way into it.
So I started a couple of years back, like in I think 2017 or so, I started like giving conference
talks. I was already going to meetups and it was sort of the natural next step. Start giving talks.
I recommend everyone do the same. It's a good way to branch out.
Giving talks is a great step in like raising your profile and just breaking out of the mold of I'm
sort of an anonymous programmer, right?
Yeah. And just like writing blog posts, for instance, like, you know, once you actually sit down to
cover a topic, it exposes a lot of your own gaps. And so you end up filling in a lot of your
own knowledge. Yeah.
Just for personal benefit, I recommend it. So basically I was giving these talks and I was
covering topics like performance and packaging and testing and plugins and a bunch of architecture
stuff. And the thing is that like, like I just sort of alluded to, oftentimes when I'm starting out with
this talk, like I have some degree of like expertise in the area, but also I'm doing a lot of learning
myself. That's one of the reasons I like doing them. And so afterwards, people would have all these
questions, you know, I'd take it as a good sign. They thought the talk was good. They thought I know what I'm
talking about. That's great. But the fact is that like, when they're asking me about packaging
or plugins or, you know, performance, it's like, well, I only have my 10 years of experience to draw
from and I won't know every single answer, you know, to apply to their situation. I want to,
I desperately want to, but that's just not possible.
You can't always go through it, right? Like if you're packaging up an app and you know,
well, I got CX freeze to work for me. So now I can make it.
It's like, well, why are you going to go try all these other ones when you have one?
Exactly.
You've got a life to live and other apps to build, right? But other people can share their
experience and say, Oh, did you know about Pyox dodger? You're like, wait, what's that?
I certainly want that a hundred percent completion like stat, but it's just not going to happen. So
one thing I sort of wished I had was basically a way to refer them to a known working example,
just because it was available at the, at the conference at PyBay a couple of years back,
like the Zulip team was there and Zulip is like this chat application written in Python,
sort of like, it's sort of like a Slack, but kind of like blends in some email features. It's a pretty
cool design and they have a fully working application with a great community, a good onboarding process,
good docs, all this stuff. And so I'm like, look, this is a great exemplar. If you're writing a Django
server application and you're interested in say, introducing typing, they recently did that process and
you can go look at how they did it and you can get a lot better answer from looking at exemplars. And
you can, you know, for me, just spur of the moment. Right. Or even just some kind of talk where people
just talk about the concept of type annotations or type hints, but here's actually the GitHub
conversation and the issues and the PRs. And then this is the before and after and the trade-offs. And
it's just like a white paper, real world example of that. Right. Yeah. And the living maintainers who
like could answer those questions too. There's so much to draw from there. And so I was like,
okay, I'm going to make a list of Python applications just for me so that I can easily
refer to them. Like just off the top of my head, I was able to get together around 20 that I considered
pretty awesome. Can you think of like, I mean, Python is the biggest language in the world right
now, basically. So like, can you think of a few?
You would think that I would be able to start naming them. The real tricky part is these have
to be the open source ones, not just things written in Python. So Wagtail, for example,
is a CMS in Django, which is pretty nice. Reddit, Reddit is one. Trying to not cheat and think about
what is on your list, but these types of things.
How about you, the listener? Can you think of a application that is written in Python?
Okay. Time's up. Well, it turns out that there are more than the 20 I could come up with.
And when I started looking, it was sort of hard to find them. But once I found them,
like I sort of found it for myself in this sort of addictive cycle, like just always looking for more
because there was so much creativity there and often sort of underappreciated. So something I became
aware of pretty early on when I was developing my own, like, you know, open source Python application,
there's this contest that's actually happening now. It's called Wiki Loves Monuments. And
Wiki Loves Monuments is like a photography contest around the world. It's like the Olympics of
photography for free culture. And they needed a tool that would allow them to judge entries. So
I made one of those and, you know, had a nice team. There was some stuff we had to figure out on our
own. But as I was like building this, I was finding some other applications and they
had like a couple dozen stars, you know, and it's because GitHub is a place for developers.
It's for people who want to build software, not necessarily use software. Like, I don't know if
you remember the old days with like Two Cows and CNET and Download.com.
Two Cows was awesome. You were like, I need a thing. I mean, it was sort of the free and open app store,
right?
Right. It was freeware. It was shareware. It wasn't really like free software in that,
you know, it's the beer or whatever instead of the, you know, speech. But yeah, anyway, so basically,
I sort of like got a little bit addicted to finding these awesome applications, because
a lot of them were sort of diamonds in the rough, not getting as much appreciation as they probably
deserved. I mean, their users loved them, but not necessarily GitHub users.
Right. Well, GitHub's not exactly. Yeah.
It doesn't have the right incentives or the right sort of connective structure there, because GitHub is
all about connect. I feel like it's all about connecting you with libraries, either code you work
on or libraries you want to put into your application, but not just end user things that you could study or
make copies of, right?
Exactly. And so I ended up with like this sort of five point rubric for the things that I was looking for.
Basically, like it had to be free software. It had to have an online source repository. It's not much of a
reference if you can't see the up to date code that is actually shipped. It had to be using Python for a
significant part of its functionality, not necessarily pure Python. Let's be real, like realistic,
pragmatic applications are going to have a mix, you know, JavaScript, HTML, CSS, going to have to have
them. So until browsers run Python anyways. So yeah, then it had to be well known, or at least
prominent in its identifiable niche. There's some cool applications out there for like neuroscience and
maybe neuroscience people love them, or something like that doesn't have to be world famous. But
basically, it has to be maintained. This is another like really important thing. A lot of old wiki pages,
they sort of get stale knowns checking the links, and it just ends up being sort of a sad graveyard
after a few years. So these projects actually have to be maintained, the links have to be up,
they have to be functional on relevant platforms at the very least.
I've seen some of these types of applications you're talking about. And I'm like, found it on
GitHub, like, Oh, why is no one talking about this? This is great. And it says click here for a live
demo. And you click it. And it's 500. Crash. You're like, exactly. This cannot be getting that much love.
I'm setting out to sort of like make a list that doesn't do that. I'll get to that in a second. But
most importantly, and you alluded to it earlier, it's just that they have to be shipped applications,
not libraries or frameworks. So one of the first things I did was I went on the awesome Python list.
I'm like, surely there's some applications on this list I can use as exemplars. And I think I found like
three, right? Like Jupyter notebook. And it's not a huge surprise that Jupyter notebook's written in
Python. But awesome Python is all about the libraries. And I wanted a list of applications,
hence, awesome Python applications.
It's really good. And there was not really anything out there like this. And I think it's great.
When I was looking through it, you know, I expected a lot of web applications, and I'm sure we'll find
them and we'll talk about some of them. But there's also desktop GUIs, terminal, sort of ASCII type of
app, you know, ASCII, what a curses type applications, a lot of variety there.
There's so many ways to taxonomize it. And I could get into that if you'd like, because I've spent hours
mulling over it. But basically, we have the basic topic, like, you know, breakdown. But then we also got
sort of server versus like, if it's a server, like software, or if it's sort of like client or GUI
software. And then as far as the console is concerned, right, you have CLIs, which is just a
command line. And then you have a TUI, which is a text user interface, somewhat lesser known term.
And then you have sort of the interactive console style. Those are the three that work within the
terminal. And then you have like sort of some other interesting architectures out there too,
which I'm like, yeah, we'll cover it later. But the good news is that when I launched it on accident
on Brian Okkens testing code, is that I basically had found 180. Like I was turning over rocks. I was
like looking on Bitbucket, Launchpad, you know, this isn't just GitHub and Git, it's like Bazaar,
Mercurial, there's like Calathea and Peugeot, and like, which I hadn't really heard of, but I found
applications hiding on there. And of course, like all these Git labs out there that people self host
too, because there are all these great like sort of sub communities, the Debian sub community,
the Fedora, Red Hat, Gnome, and they all have their own cool organizational support and community. And
it's really interesting to just not just find the application, but also that community. So yeah,
we got 180 when I accidentally launched it sort of at the beginning of this year, or maybe like a little
bit before, like Merry Christmas to the Python community, I guess. But yeah, so I published a
blog post once that sort of started picking up, and that's still up on the site. So if you go to my blog,
like sedimental.org, it's just up there. I'm sure if you search awesome Python applications, it'll come
up. For sure. And I'll link to it in the show notes. I haven't blogged since then, because pretty much all
of my like, free time for content creation has just been going toward curating this. And I've been
learning a ton. I need to find a better way to share it. I guess that's sort of like why I'm on
the show. But the point is that at the beginning of this month, I was at something like 250 ish. And
I guess I should say at the beginning of September 2019, I was at the top of around 250 applications.
And now I'm at like 312. I'm aiming to get to 350 by end of September 2019. I say 350, not because it's a
particular goal, but because I have like a list of candidates that I need to evaluate.
A lot of our community, like sort of submitted is really like very time consumptive to find these.
So if people want to, you know, help me out, that'd be great.
Yeah, that's cool. So yeah, I think I'm going to do a PR for Doc Assemble for you, which is a legal
interviewing software based, I think it's based on Django.
I just looked it up and it actually looks very impressive. It's exactly the sort of polished
application that I think needs some more open source love. Anyways, I mean, one might think besides
my personal addiction, like why, why do this? And I have sort of like three goals that I'm trying to
hit, or at least trying to fulfill. I'm not sure if they'll ever really be hittable, then maybe not
like measurable enough. But I really want to like goal number one is I really want a better development
cycle. You know, like someone ideates idea for an app, you know, they go basically do maybe Django
start project, or they open up a blank editor, and they just start writing it and they start searching
on Stack Overflow almost immediately. You know, maybe they find a tutorial that gives them a to do or
something to get just barely get off the ground. But beyond that, it's all first principles.
And I just really want people to sort of benefit from all of the other discovery that these like
application developers have created. The way I put it in my high bay talk, like last month was that
Python is the biggest language in the world right now. But how do you personally benefit from that?
Yeah, you know, there's all this stuff out there. I mean, there's a bunch of great libraries we
benefit from. Absolutely. I do think one of the differentiators between a beginning programmer or a
beginner in an ecosystem. And somebody who's very experienced, some form of expert, I guess,
is that a lot of times the beginner will start coding and think everything has to be created from
scratch. Right? I need to, you know, load CSVs. So let me just like read the text of the file and start
splitting on, you know, commas or like weird stuff like that. Right? Whereas the more experienced person's
like, well, you know, pandas will read it, or there's also a CSV module in the library, like,
right, just use that. And I feel like this, it kind of helps the beginners close that gap to say,
I'm looking like I want to build something like that. Let me see what they did, right? Let me see
how they structured their file system, and they organize their code. And are they even using celery?
I heard I have to use celery. Do I have to use celery? I don't know. That's sort of the thing. If
you go on awesome Python right now, sure, you're going to find like hundreds of libraries. But what
are you supposed to use them all? Like what all ingredients goes into making sort of a complete
version of the app that you're sort of trying to build of that architecture? Yeah, I think that
basically, like my longer term goal here is like to have sort of a decision tree type interface.
So you can say like, well, look, I want to build a web application, you know, and then like,
you can sort of say, oh, I wanted to support this many users, or I wanted to basically use,
say, SQLAlchemy, or have a Docker image or something like that. And you can find your way to,
like an application that looks enough like the thing that you want, that you can just pull things
wholesale from it. And basically, not only get an application sooner, but also learn the best
practices without having to go through dozens of hours of conference talks and blog posts.
Where your users are, there's a data center for you. Whether you want to run a Python web app,
host a private Git server, or just a file server, you'll get native SSDs on all the machines,
a newly upgraded 200 gigabit network, 24 seven friendly support, even on holidays,
and a seven day money back guarantee. Need a little help with your infrastructure?
They even offer professional services to help you with architecture, migrations and more.
Do you want a dedicated server for free for the next four months?
Just visit talkpython.fm/Linode.
Compare this to the cookie cutter library of templates for us.
Yeah. I mean, that's actually, I hadn't really thought of that, but that's a very good point.
In a way, cookie cutter is trying to codify these architectures as well. I guess not to put too
fine of a point on it, right? But I mean, I have looked through 300 of these. I mean,
I've looked through over a thousand applications and very few of them, I like still bear the marks of
having started as a cookie cutter application. I think that cookie cutter certainly has its place
for getting a kickstart, especially for people who maybe don't want to write their own setup.py or
something like that. But yeah, I'm not sure that it's going to be enough architecture to really get
you completely off the ground. Like there are emerging technologies and containerization around like
that pack and say app image and snap and so forth, especially for these GUI applications on Linux,
just as an example. And maybe there's a cookie cutter for something like that. I don't think it's been
done enough times for it to like show up on that radar. And so basically you'll have to look for the
existing community within the existing, I mean, it's very hard to call the community actually,
within the existing ecosystem, like who has actually adopted this. And so that's like,
you know, I think I'll get to that in a minute here, but I did clone every single repository,
clocked in at around 30 gigabytes for 250 repos. I sort of ran a bunch of analysis and technology
discovery within it to figure out who's doing what. That's cool. Yeah. Like, in fact, let's just like
jump right into it because I'm, I've got these numbers and I just got to sort of get them out
there. Sure. Let me throw one quick comment though, on the cookie cutter bits while you're pulling up
the numbers. I feel like cookie cutter is great to help people jumpstart. And the idea is like,
you kind of get this prepackaged app, which is great. Like here's a way to do flask that already
talks to a database and a queue, or it already sends email or something. Right. That is a long ways away
from here's an actual application, right? Even when, as a developer, when you're building an
application, you think, Oh man, I'm almost done. I'm 80% done. You're like less than half done,
right? There's all these little edges you have to smooth off and these like edge cases that you don't
think about. And in these, these are real applications that shipped, which means they're way more polished.
And they're just, I don't even know that how comparable it is. They're both good for people
starting, but I feel like this is a much different set of things in the catalog.
And to be clear, they're polished for the end user. What I think is really interesting about them too,
is like all the rough edges they have. That can be really good for a programmer's ego to be like,
look, this worked. Okay. Like, I don't need to overthink this. Like, I'm just going to say,
this is good enough. If it's good enough for them, it's good enough for me. Ship it.
Yeah. It's easy to get hung up on like trying to make it perfect or trying to set up like infinite
scalability. So it's like Google and you're like, you have no users yet. Forget scalability.
Just get it out. Right? Exactly. And so I got my numbers up. Let's hear them. So yeah,
the quick methodology here is I just cloned every repo that's Mercurial, Bazaar, Git,
pulled them all. I have, you know, ransom slot count and like other analytics over the version
control history. We're looking at 19 million lines of code, 2 million commits with around 50,000
committers. That's an insane amount of effort. Like that is huge. And this is Python code.
I think I added it up and it was like hundreds of years of like maintainership or whatever you want
to call it, you know, like sometimes you go on maybe steam or something like that. And it sort of
aggregates for you how many hours was played on the game. And you sort of feel like you feel a little
bit terrible about humanity perhaps, but we all got to have fun anyways. But this, like I had like sort
of the opposite feeling, right? Like this, I think I actually experienced the true meaning of an awesome
list. Like I was in awe at the amount of like, you know, stuff I was looking at. And I sort of like
broke it down. So real quick thing, like about a third of them are server software with 64% being
like sort of desktop or CLI and just sort of like a quick breakdown there. I think the scariest thing for
me was basically Python compatibility, because we hear a lot of like doom and gloom at times,
but like contrasted with like a lot of excitement about Python 3. And I was worried that like, you
know, these maintainers are stretched so thin, they have not had time to add Python 3 support. And here
we are with something existential facing the Python application community. And I was very pleasantly
surprised to find that two thirds of these applications already support Python 3, like they run on Python 3.
That's great. It was a huge relief. This isn't the same thing as a library supporting Python 3. This is
them having converted a code base that is often quite large to Python 3, many of them. They didn't start
this way, because I think the average or sorry, the median application on the list is something like 10
years old. And some of them much older than that, like Mailman, the thing that Python uses and many other
organizations use for managing their email lists, that's written in Python, and it's made the jump,
you know, it supports Python 3. And that's pretty impressive. And if you look at sort of the
compatibility over time, you do see the Python, like two applications kind of tapering off, you haven't
had another Python, like I think the most recent Python 2 application was started in something like 2017,
versus, you know, Python 3 applications, which are being started today,
Right. There's probably a bunch more Python 3 apps starting now than there are Python 2,
I would expect.
You see a nice healthy trend there. Thankfully, I was super relieved.
I'm sure you'd either have to be crazy or depend on a library that is Python 2 only deeply to start
a Python 2 app these days.
And those are rarer and rarer. So it's a little bit harder to justify. There's still some interesting
case studies in here. So for instance, I think OilShell actually converted to Python 3 and then back to
Python 2. And he has a whole like, you know, blog post about why he did that, the lead maintainer
there. So that one's an interesting case study. And I think that's sort of what I hope to like,
one of the deliverables I hope from comes out of the list is just like all the interesting case studies.
I think we'll get to those too.
Yeah, there could be a bunch of stuff coming out of there. I think,
you know, another thing I think would be valuable, not that you don't already have enough going on,
but would be some kind of newsletter where just the stuff that gets added that month or something
comes in and just gets thrown out. That'd be great.
I'm too much of an engineer for that, Michael. That's why I've added an RSS feed.
You know, we just pull the data ourselves.
Yeah, exactly. Just pull the data yourself, right? If you want to do a newsletter and all that,
I fully support that. But yeah, I'm going to take a technical approach to this social problem.
Okay. It would be good to get there. It would be good to get there once the list sort of maybe
starts like plateauing a little bit more. And also once, yeah, once I get some more stuff in place to
keep the quality high, you know, because some of these projects might go like into an unmaintained
status and I'm not going to be the one vetting 350 projects every month, you know? So I need to
automate some things first.
How much automation could you do? Like, for example, every one of these has a link to their repo.
Could you look for unresponded to open issues or the lack of a commit for like one year puts it onto
like a warning list and then some contributor could come in and go, this looks suspicious.
That's almost exactly what I have planned. And I sort of have, I have a little command line
application I built for managing awesome lists. It's surprising how many awesome lists are just
manually maintained via PRs. But yeah, all my stuff is in a YAML file. I think I have something like
a thousand links in there. All of them are going to need to be checked for 404s and stuff like that.
But yeah, I do sort of have a informal bar of you need to have a commit in the last year.
There are exceptions to that. So for instance, you mentioned Reddit earlier. Reddit is a massively
important Python application for the internet. And they only have an archival version of the site up
from around 2017. But still, I mean, like, it was still a very big site in 2017 with a lot of
like real lessons. It is such an important site that I think it clearly deserves to be on there.
It actually surprised me that it was there. There are a couple other surprises in there too,
I think. I'll just sort of refer the audience maybe to the repo where we have a Jupyter notebook that has
all of the graphs in them. Like some of the like numbers I'm going to cite to you now are going to
be out of date in just a week or two when I give a PyGotham talk because I'm going to run it on 350
applications instead of 250. But we'll keep the Jupyter notebook in the repo up to date. But suffice
to say for now, like there's some really interesting findings in there.
Oh, I'm looking at it now. This is super well done. I'm so glad you put this up as a notebook so it can
just like live. That's great.
The most surprising trend, it doesn't affect me very much, but it was surprising to me just how like stark
it was the increase of QT based GUI applications compared to GTK based GUI applications. When you
look at the graphs in there, remember all of these applications have had a commit, like 95% of them
have had a commit in 2019. It may look like, oh, this is an old application, but it's maintained,
it's used, it is awesome and deserves some respect there. But like people are starting QT applications
these days, not GTK.
Yeah, yeah. I'm looking at that list. It's almost 50% QT and then 2% Pi game, 3% Kivi,
17% WX and 30% GTK. Yeah, pretty interesting. There's a bunch of graphs like that. This is a,
I don't know how long this is, but the scroll bar is very small.
That's great.
I'll do credit there. Like I was on a huge time crunch coming off of speaking tour in Africa.
Let's just say I gave a keynote in Tunisia, nothing too mysterious, but basically I was super time
crunched. And so that notebook right there is basically all my wife's work. She's in the commit
log and so forth, but credit where credit is due. I did not have time to do that stuff. And she really
saved my ass.
Yeah, she came through. That's awesome. Congrats to her. That's good work.
Yeah. Anyways, I could go on about the stats, but I think that probably at this point, people are,
people have got to be curious what other case studies are hiding on this list.
Yeah, they've got to be. So maybe pull out some highlights that stand out for you. And then I grabbed a
couple that maybe we can go more quickly through like more of them, just skip across to give people
a flavor of what we're talking about.
Absolutely. So one of the ones that I highlighted was OpenEDX. I know that Ned Batchelder is a big
deal in the Python community, and this is where he works, last I checked. But the EDX platform,
edX platform has something like 51,000 commits at the time of writing. And it's really interesting to me
because it is a mono repo. And it represents like a whole team's work. And so there are all these
dynamics that you can see happening there. And it's one of only three projects where no one developer
has more than 10% of the commit history.
Wow.
Yeah. It's the third largest Django project on the list with 300 committers. And it powers all of edX.org.
Yeah. And I think MIT's OpenCourseWare and a bunch of others as well.
Yeah. And just for contrast, 41% of applications are mostly written by one committer. How's that bizarre for you?
It is bizarre. But, you know, thinking about it, it makes sense to me, right? It seems like
somebody just wants this app to exist, so they're going to go create it. But then you see the things like Reddit or
OpenEdX or some of these others, you're like, well, these are built by large groups of people.
I sort of want to find a way to highlight these organizationally supported, foundation supported,
corporate supported applications, because they definitely follow a different sort of path
compared to your average just sort of side project.
What do you think about using folks out there listening, maybe they're PhD computer science folks
or other types of researchers? Do you think that this list might put together some interesting set
of data for people to go look at how apps are built?
I think that if there's someone who's doing some sort of combination, computer science, ethnography,
like, you know, studies, social science studies, there's absolutely a ton to learn from here.
I mean, I didn't sort of pick and choose applications based on these findings. These findings were
like, you can't exactly call it randomly selected, especially not with the Python qualifier in there,
right? But it is still a very interesting cross section. And if somebody wants to do that sort
of thing, I'm all for it. Happy to support that.
Yeah, that'd be cool. All right. So tell us some of the other ones.
Yeah, absolutely. So another one that I highlighted was Sentry. And so Sentry is very interesting because
it's so big and often promoted on podcasts and conferences and so forth. A lot of people are surprised
that their entire code base is just sitting on GitHub. It's got like it's 26,000 commits since
2008. For those who don't know, it's a web service and front end for cross platform application monitoring.
So it's sort of like New Relic and stuff like that. A little bit different than Datadog, though,
because it focuses on error reporting. But we're looking at a million lines of Python. I think it's one of the
very few projects that actually crossed that million line threshold. Because, you know, a million lines of
Python, it's like 10 million lines, 100 million lines of less efficient languages.
Yeah, exactly. A million lines of Python. That's a lot of code.
That does include about 120,000 vendored lines. Just so you know that I went looking for that
possibility. Like I went looking for libraries that maybe got vendored and tweaked and just copied in.
Maybe define that for folks, because not everyone will know exactly what you're talking about.
Yeah, of course. So vendoring is what you do when, you know, a library does something you don't want,
like drop support for intercompatibility with something else. And you're like, okay, well,
I'm just going to create my own little mini fork of this inside of my repo.
Right. Like if I didn't want to depend upon requests, theoretically, I could just jam the
source code for requests into my app.
Yeah. Like you probably didn't pay for it, but it's called vendoring because the person who like,
you know, publishes the library is a vendor. It's kind of like that.
You've taken responsibility for it, right?
Right.
Okay. But still, that's still a lot of code that is like, it's 880,000 lines. That's not
vendored.
Yeah. The largest Flask app I found in comparison is Pagure, P-A-G-U-R-E. And that's what's called a
forge. It's like a GitLab or a GitHub, but written in Python and it's written in Flask. So it's almost
like 10X smaller. So Sentry, its code is sitting all out there. And it's also BSD3 license, which is
very permissive for a for-profit application. So that's kind of interesting.
Yeah. That's super interesting.
The quirkiest thing I found though, the quirkiest like case study was this application called Gannetti.
And that's G-A-N-E-T-T-I. And it had like, I don't know, 100 or 200 like stars on GitHub or
something like that. And it was just like, what even is this thing? Because it was written in like
half Python, half Haskell. So like the very pure functional thing makes it with the very pragmatic
sort of systems thing. And it had 16,000 commits since 2007. So very mature. It's a cluster management
tool focused on long lived VMs used for workloads that don't have built in redundancy. So like web
servers, like, you know, web services, you can shut down the worker and start up a worker. The other
workers will take over. Right. But if you need like a job to not fail, and it's going to be very long
lived, right, you might use this. And so it was actually developed at Google. And that was also
very strange because Python and Haskell, especially Haskell, like these are not like super common
languages to find at Google for, you know, something that appears to be this important. And it's pretty
widely deployed. I found like a nice discussion of Wikimedia, like just talking, like, should we use OpenStack for
this? Or should we use Gennedy for this? And I think they ended up going with Gennedy. So it's still
pretty commonly used. That one was an oddity. Another one along those lines is a thing called
LocalStack. And that's a developer tool. It's useful because it sort of does like a mock service that
you can run locally of like AWS. And so if you're doing DevOps code, you can run a mini AWS locally and
like write tests against it. And that one is like, I think, a third Java is one of these blended
dead stories, right? Yeah, yeah. So like hybridizing with JavaScript and HTML and CSS, everyone expects
for a web service, right? But like to see things mixing with like Java and Haskell, these are some
pretty interesting case studies that have a whole story of their own, I'm sure.
This portion of Talk Python to Me is brought to you by Tidelift. Tidelift is the first managed open
source subscription, giving you commercial support and maintenance for the open source dependencies you
use to build your applications. And with Tidelift, you not only get more dependable software, but you pay
the maintainers of the exact packages you're using, which means your software will keep getting better.
The Tidelift subscription covers millions of open source projects across Python, JavaScript, Java, PHP,
Ruby, Ruby, .NET, and more. And the subscription includes security updates, licensing, verification,
and indemnification, maintenance and code improvements, package selection and version
guidance, roadmap input, and tooling and cloud integration. The bottom line is you get the
capabilities you'd expect and require from commercial software. But now for all the key open source software
you depend upon, just visit talkpython.fm/tidelift to get started today.
So which ones did you like?
Well, you know, I went through and I just wanted to pull like a couple that stood out to me on each
of the categories. So if we go to the awesome Python application list, there's a ton of categories.
You sort of put it into major categories and then the developer space just became a meta category,
right? So you've got internet and audio and graphics and productivity and education and science.
I pulled out the developer stuff compared to the non developer stuff because I think that the developer
stuff in general is a little bit easier to find. Like you're going to find blog posts and higher GitHub
stars for developer focused applications in many cases. It's not a super, super stark difference when
I ran the numbers, but definitely like, you know, most of the listeners have heard of Ansible,
you know?
Yeah, right, right. Or OpenStack or something like that. Yeah,
Exactly, exactly.
Yeah, cool. So there's a bunch of stuff under there. And I didn't, I didn't pull them out quite
like that. But maybe just to give people a sense of what's out there. Some of these are really small
and niche. And you tell they're like for somebody and others, they've been like the top 10 sites on
the internet.
Yeah, let's just go. I'll just go through the list. And we can just maybe give me some real quick
thoughts. So under the internet category, we have Deluge, which is a popular lightweight
cross platform BitTorrent client.
Deluge, which is how I pronounce it. I mean, I don't know how to pronounce it.
But
Deluge, okay.
But well, I don't know. It doesn't come with the pronunciation guide last I checked. But basically,
it's a BitTorrent client. And it has a few very interesting things about it. One,
it's a GUI application, it uses Twisted. If you just look like for best BitTorrent client,
you'll find it making those rankings. And people aren't saying like, this one's great,
because it's written in Python. You know, it's just a good application, very solid,
like the UI, etc. And it happens to be written in Python. And so that was one of the ones that
I got off the top of my head when I made my list of 20 because I am a Deluge user proudly backing up
my Linux images. The Deluge thing also has a little like hidden gem, which is that they managed to have
a web UI that is almost identical to their GUI UI. But it just like sort of like allows for remote
administration, it's still classified as a desktop application, because it's mainly written for
single user use. But it does have a web UI. And I'm not sure how they did it. But like,
the experience is the same. So like in the browser as using, I guess, like I'm not sure if they're QT or
GTK, I can check. Sure, sure.
But yeah, I'm a big fan of Deluge. I highly recommend checking out their repo.
Yeah, all these have links to the repo, it's a demo to their homepage, things like that. Another one is
the Lixire, a featureful file host and link shortener with API. So kind of like Bitly, maybe.
This one's an interesting one. Like it seems to be used by like, a community I don't fully
understand. But like, it gets a lot of use. So yeah, this is a recent addition. I'll probably
explore it a little bit more later. But yeah, it is like sort of like an imager combined with
like a Bitly, which sort of makes sense. Like if you're, you know, pasting things into an IRC chat
or something like that, it's got everything you need.
Yeah, super. The next one people might have heard of it's called Reddit.
Yeah. Yeah, we don't need to say a lot more about that other than this is sort of a snapshot from
2017. But I think it's still super cool that this is here because here's a high scale application. I
don't know exactly where it ranked, but it was in the top 20 sites at some point for destinations,
right? Definitely. I'm not sure where it's at now either. But yeah, definitely a major destination,
especially I'm sure for many listeners, whether you like it or not, you're going to like probably
find some useful stuff through Google search or something.
Yeah. But one interesting thing about Reddit is that they have a very interesting approach to their
like schema design. If you dig in, they're using what's called an OAV pattern or an EAV pattern. It's
an entity attribute value. And it's kind of like, it's sort of like object storage or document storage built
on top of relational database. And so some might consider it an anti-pattern to use it this to this
extent, because you're losing out on the constraints of a relational database. But it managed to work for
Reddit. So it can't be all bad. Anyways,
theory meets reality, right?
Yeah, exactly. And some might say like, oh, it's very slow compared to, you know, an optimized schema
with better indexing and so forth. But one, like databases have done some degree of optimization for
this pattern. It has a Wikipedia page. It's a pretty well known pattern. But also Reddit is fast because of
caching. That's the like other main learning there. If you look in there, it has layers and layers of caches.
A very interesting, like, you know, exemplar to learn from.
Yeah, I would definitely think so. All right, next category is audio. And I grabbed two of those out
of there. One's called Exile, something like this, which is a cross platform audio player and library
organizer tag editor type thing that looks pretty interesting.
I found this because I went on, everyone's familiar with Wikipedia. There's also this thing called
Wikidata. And Wikidata allows you to use a Sparkle query, which is sort of like a, I don't know how to
describe it exactly. But it's sort of like SQL, but extensible for like graph networks, kind of.
So I ran a query for all of the software that had Wikipedia pages that was known via like, you know,
Wikipedia or Wikidata to have been written in Python. And so I actually found it through there. I'm not
a user myself, but yeah, it's actively developed. It had a release come out just a few months ago. And
it's sort of like, kind of like Amarok, if anyone remembers that, but it's got like Last.fm support and
plugins and so forth. Seems pretty cool.
Nice. Another one, Music Brains Picard, which automatically identifies tags and organizes
musical albums, which sounds pretty cool.
It's funny name, but just like amazing software. Let me just say, I don't use it as much as I used
to back when I used to like DJ and stuff, but basically the way that Picard works.
So there's a Music Brains Foundation that develops it. And they also have a sort of like, it's kind of
like a Wikipedia, but for album information, it's kind of like Discogs or something like that,
but open. And what it'll do is you can like take one of your CDs, make a backup, right? And that
comes through and it's just as track one, track two, track three, track four. But it'll take the
number of tracks combined with the length of each track to generate kind of a fingerprint,
match that against an internet database of like album listings and so forth, and then automatically
bring in all the fully filled out like ID3 tags. So everything shows up very searchable.
Oh, that's awesome. Yeah. With like album art and all that. Yeah.
Man, Michael, I used to spend, I mean, you're in the audio world now too. Like I'm sure you know,
I used to spend so many hours, like, you know, getting those tags just right and still have typos.
And then this sweeps through and I press one button, 20 seconds later, it's just all perfectly
organizable and tagged and so forth. It made me feel like a fool, but also made me feel pretty magical.
Yeah. Well, I'm sure you appreciated it more than if you hadn't done that stuff by hand.
And then one day I find out it's written in Python and I'm like, that's going on the list.
Yeah, it's definitely on the list. All right. Switching from a music audio to video,
we have two editors, Flowblade and Openshot. Flowblade does multi-track, non-linear video
editing for Linux and Openshot is cross-platform video editor for like many of the platforms.
Yeah, that one like sports free BSD and Windows as well. So with these ones, like, I think what's
interesting is that a video editor is a very large application. It's going to have to have a lot of
codecs and other things. And those things are very touchy based on what platform they're running on.
They're incredibly finicky. It's super painful to develop that stuff. I've done it.
Exactly. Talk Python training. So very finicky stuff. And one sure shot way to like, you know,
ruin it too, is like to miss a dynamic library in your package. So being able to go in there and look
at how they do their packaging. Sure, it's a little bit like, you know, dirty, but I'm sure that you're
already deep in the dirty if you're looking for how to do this kind of application freezing. So it's
really useful to have those to refer to. Those are great examples. For the graphics world,
we have a free CAD for a general purpose CAD editor. That sounds pretty intense. And it's
awesome that that's in Python. So with free CAD, part of me was a little bit like sort of skeptical,
right? Like, could an open source community come up with something like CAD? Because I mean,
when you're dealing with the BIMS world, right, like, you know, it's like construction and design and so
forth. It's, it's just so much. And who could possibly have the side resources to do this? But you know,
they're doing it, they have a wiki, and they don't seem to be giving up. It's pretty impressive. I read
some sort of third party reviews. And it basically says, like, look, this may not be like top tier,
right? Like it may not be ready for like full professional shop usage and so forth. But it's
surprisingly usable. I'm not much of a CAD user myself. So I'll just have to take their word for it.
Right? If somebody if somebody listening is a CAD user, let me know, let me know if I should expand my
description here, write a review, support Python CAD. Speaking of supporting, some of these have
fund next to them. Exactly. So the sorts of links that I collect for each ones, basically, I'll do
repo, home, Wikipedia, if they have it, GitHub, if they're not on GitHub, but they do have a GitHub
mirror, demo, if there's a version of it running that you can like just try out docs, which is like
usually a read the docs link or something similar. And I added fund, which is basically going to be like
a Patreon link or a PayPal link. If I can find a way to show that these people are making into this
current generation of like approach to open source, which is like, yes, please do pay me. So I can spend
more time on this, then I will absolutely link it here. Because they deserve all the help they can get.
Yeah, absolutely. I think it's great that you're highlighting that. The other one in graphics is
Kuru image server. So I guess you give it like a large image and you can say I would like it at
right now I wanted a 250 by 250 or whatever. And it just, you don't have to keep redoing it. It just
automatically regenerates and probably caches that. Yeah, exactly. You upload an image and then it can
scale it to all different sizes. They say they have all sorts of tweaks and optimizations to do that sort
of stuff efficiently and performantly. And yeah, it has caching and stuff too. That's a pretty tough
task. You know, like that stuff can be pretty expensive. So I was like, maybe if someone's
interested in performance tweaks for images and so forth, this would be a good project to go see
how they achieve that. All right, let's go through the games real quick. There's a game section. I'll
just quickly go through them. You can just give me maybe some general thoughts. Sure. There's frets on
fire X, which I'm guessing is like one of those sort of music things come down and you hit them and
then you've got the streaming platform is like a homemade Twitch, which is pretty cool. You got Lucas
chest and unknown horizons, which is a cool strategy game. The game section is one that I definitely
wish was larger. A game is again, a very finicky, large application and one that's tough to undertake.
A lot of also there aren't a ton of open source and free ones. Like I know that Python was used in
one or two of the civilization games for scripting, but like that's not open source. So, but unknown
horizons is pretty similar to that sort of thing. It is an RTS. It makes extensive use of Python. So I'm
happy to see that there frets on fire is a little bit older, but from what I can tell,
it still works for some people. So I guess like dance dance revolution never gets old or maybe,
I don't know. Is it a rock band type thing? It says frets. I don't know. It's something like that.
Something like that. Yeah. Eve, Eve online is another good one to throw in there,
but obviously it's not open source. So, you know, you can't, can't really do that. Put it in there.
What else was second life? If you consider it a game, maybe, maybe you consider it life.
That's right. Maybe you do. All right. Under productivity, the top one here that I grabbed
is actually something I use to manage all of my servers and my infrastructure. And I love it.
Just love it. Glances. Yeah. It looks really good. I guess I don't manage enough servers to really
like me. I'm just like a, an H top user, but it's like H top, but better. Yeah. You get all sorts of
cool stuff. Like if I just run glances real quick, I guess I don't, there you go. It gives you graphs for
CPU memory swap, how much Ram you're using, like server load over one minute, five minute,
15 minutes. You can sort by memory usage, by CP usage. It shows you like your disc IO rates,
your network rate, all just like all sorts of stuff and great little hot keys for it. Yeah. It's,
it's like a shop, but yeah, like all the stuff you want.
Yeah. I'll have to give it a shot.
It's pretty good. Let's see. We also have bleach bit for like privacy. So it cleans up some stuff out
of your system, but also I'm guessing it like rewrites every empty bit of space or something like
that. Just giving the bleach part. I don't know. I used to work way back in the day, like in high
school for a brief time at like a computer repair shop in South Dakota, no less, but basically people,
they would come in with malware, spyware, all this stuff. And, you know, my computer's slow and I
have all these pop-ups. Yeah, exactly. I only installed nine toolbars on my IE 5.5 or whatever.
Like a 50 pixel bar in the middle where the actual content is.
Exactly. Oh man. You're putting too much of a, of an age on us here, Michael. Keep this content
evergreen. Anyways. All right. But basically what bleach bit is, I didn't realize it was written in
Python, but it is one of these cleaners that will go through and like, not just empty out your temp
directories and so forth, but also like clean up your registry because a lot of this software will make
access registry rights. And I guess, especially back in the day that would slow down windows.
So yeah, it removes like tracking things like increasingly that's become the focus to like
sort of clean out your browsers of like, you know, any weird flash cookies and that sort of thing.
Cool. Yeah. That's great. Yeah. Last one on the productivity space is GM vault, Gmail backup.
I guess we've sort of foregone, I mean, I personally have foregone some of this stuff in my Gmail,
but probably I should back it up, you know, I never know when big G is going to like, you know,
do something uncouth. Yeah. You never know. It is kind of nice to have that. I actually went and found
us, found out that if you go to the, the Google doc, just the Google data export, you can say export my
docs. And one of the problems, you can run Google drive and stuff and it'll have like your docs from
Gmail in there, but they're just a hyperlink back to Google docs. Right. But if you run the export,
you can say export it as word documents, Excel spreadsheets and a PowerPoint and actually get
the content of it, not just links to it back in drive. It's pretty cool. Yeah. This sounds like
this kind of sort of in that realm organization, a lot of archiving stuff in this world and library
bits. There's a funny one as well. Archive Matica, digital preservation and then archive box,
which is like self-hosted sort of way back machine. Exactly. Archive Matica. It's an interesting one
because it's sort of like, I think it's sort of targeted at like libraries and actual like archives,
whereas archive box is a little bit more like gorilla, like Yahoo says they're going to delete
geo cities. Okay. Let's like, you know, pull it all down. Like it's sort of those two schools,
which are definitely adjacent, but a little bit different. Yeah. Similar to that, I guess we have
open library, which is a web application for like a library catalog. So if for some reason you,
you have a small little library, you know, you don't have to like start from scratch.
And you'd be surprised like, yeah, libraries, I was at a library recently, like it only was open a few
hours a week, but they had a tremendous collection and they could definitely use something like this
because they didn't have anything to search through. You know, you just had to walk the stacks.
That's crazy. And the last one, the one that I said was funny, it's called I hate money.
There are some people who are very into sort of like personal finance management,
like I guess people who like sort of picked up wanting to balance the checkbooks epigenetically
somehow passed down to them. And, but no, there's like this whole movement of people who do plain text
accounting and get version. They're like, you know, they're, they're actual financial books.
I think I hate money sort of comes from that domain combined with the self hosting domain.
Yeah. Interesting. They're like, we're not doing meant forget meant.
This one seems to be about like sort of shared budgeting and stuff too. So this is like Fava
is the one that I was thinking of that I added recently, but I hate money is like basically
like you have roommates and you just want to keep track of who bought what when,
and it's like a little bit better than like a Google sheet.
Yeah. Yeah. That's pretty cool. All right. So I'll, I'll speed through a few more here real quick.
So communication with ask bot, which is very similar to stack overflow.
Also quite interesting and secure drop, which is like a whistleblower submission system
for media organizations. These are cool.
If you're into self hosting or you have a team that you don't want to pay for stack overflow or
something like that, then you can like run your own ask bot. And secure drop is a really important
one. That one was originally written by like Aaron Swartz and it's managed by freedom of the press
foundation. They just, I think it came out with a new release and yeah, that's huge for journalism in our
time.
Yeah, absolutely. One that I think is probably going to be really a welcome one for,
a lot of teachers and professors out there would be NB greater. This is under the education system,
which is Jupyter based notebook, basically create assignments in there and it will grade them
automatically for you. That sounds wonderful.
Not quite that level of teacher myself, but I think that it's pretty cool idea. Like instead of having
just workbooks, you can actually give a live notebook, have them fill in some blanks, do some things,
and then like have them submit the IPyNB and grade that. Yeah.
Another one that I like to point out, I don't know if it's under education, but a lot of people
seem to know about it, even if they're not developers, but sort of like education related
is called Anki, A-N-K-I. And it's basically like a flashcard program. And I meet all these lawyers
and doctors. They're like, oh man, I would not have made it through school without Anki, right?
It's like as important to them as Wikipedia or something like that, because it's sort of like
spaced repetition memorization tool written in Python.
If you're going to do anatomy or something like that, right?
You got to memorize it all.
There's no reason to rhyme. It's just that's part of the bone. It's called that. So we're going to
learn that. Yeah.
There's probably some reason, but yeah, it's a lot of memorization.
You can't Wikipedia thing during a surgery, I imagine.
Just hold real still. I'm just, I'm researching. Yeah. So the last one, speaking of this kind of stuff
is science that I want to dig into. And then I think we're going to be probably out of time for
touching on them. But I felt when I looked at the science area, I felt like there's about equal
amount as some of the other categories, but this is like really polished, really serious stuff. So we
have Ascend, which is mathematical chemical processing modeling. We have Cell Profiler, which is
interactive data exploration of biological image sets. We have SageMath, which is a competitor to
Matlab and Mathematica. Like these are, especially SageMath, these are real things.
No, SageMath is kind of a triumph, right? Like if you're maybe a Matlab user or something like that,
you should definitely check it out if you haven't already. But yeah, all of these science applications,
they get a lot of usage from their like academic counterparts, student counterparts and so forth.
But we don't often think to go find them as exemplars for like applications we might want to build.
But yeah, there's some really interesting ones in here. Like the CCAN was one that I like
jumped out at me because it is a data management system. So it's sort of like data hub that you can
host yourself. And so if you are running an organization like university or government,
you know, and you want to do anything with open data, you're going to need some sort of data hub,
some sort of portal for people to find that data and you need to manage your data on there.
And there's an open source one written in Python.
Yeah.
And I've done there's another one that's called Orange. It's like kind of like a component based
data mining software for graphical interactive data analysis and visualization, but you can train
machine learning models graphically. And it sort of has like a signal processing type metaphor. So
like you have this thing, and then you drag like a little curve into that thing and make like a little
flowchart. And then once it's working, you can export a Python program from that. And I've been using
that since I think 2012, 2013. So it's like from somewhere in Eastern Europe. And I think the professor
who like leads the lab that writes it, like I think I have his book as well, itself, like kind of a triumph
too. It also uses QT4 and QT5. So if you're undergoing a QT transition with your application, might be an
interesting one to look at too.
Oh, wow, that is a super interesting angle.
Yeah.
Yeah. Okay, great. So I think we're probably out of time for diving into anymore. But there's so much more
to cover. So there's the CMS category, the ERP category.
Business software.
Yeah, oh my goodness. SAS and all that. So static sites. And then there's the dev super category,
I'm calling it because there's 129 items in there and a bunch of subcategories like source control
and stuff. And people can just go through the list and find it. I think this is great.
Yeah, I could go on for hours. I really could. Each one's more exciting than the last.
These are so good. All right. I think we should sort of leave it there and people can go and they can
explore the cover. There's probably about 250 we haven't even touched on at least. And then people
also out there who are listening, maybe they want to, they maintain one of these projects or they use
one of these projects. They want to recommend it to you. What's the story there?
I have a GitHub issue template. Jump in there, like, you know, make sure it actually fulfills the criteria
of being like maintained and so forth. Again, I'm not super, super strict on that. But like,
if one particular category is like really overpopulated and it's like the seventh link
shortener or something like that, I got to be a little bit decisive there. If it's something that
is pretty like undernourished category, right? Like, you know, more is better, the more merrier. So if you
have like a game that you know of written in Python, it'll probably make its way in.
Yeah, super. Maybe someday the game category will break up into like,
tower defense and strategy or whatever. Who knows?
I really want to do a taxonomy, like sort of refactoring because some of these categories are
bursting at the seams. I recently added the storage category because I found all of these like,
database related things written in Python. I don't know, there's just so much.
Yeah, super cool. All right. Now before you get out here, the last two questions real quick.
Sure.
For you. When you write some Python code, what editor are you using these days?
Yeah, I'm still getting it done with Emacs. But you know, I'm open to experimentation. And one of
these days, maybe one of these other editors will get its hooks in me.
Yeah, cool, cool. Well, I'll keep asking you each time you come on the show.
And then notable PyPI package. You've got some good ones out there.
The thing that still dominates for me is Glom, right? So I have this Python package is called Glom.
You use it for deep getting into a dictionary, but also a variety of other things. It's sort of like
a data templating system. And more and more, it's becoming like a higher level programming
language almost. So right now we're building streaming support into it because a lot of
people have brought up like sort of the size of the data that they want to manipulate with Glom.
And so we got to have some sort of streaming metaphor in there. But I'll also point out that
a lot of these awesome Python applications are distributed through PyPI in some way. So I have
PyPI URLs on a bunch of them too. So shout out to them.
Yeah, yeah. Awesome. Awesome. All right. Well, final call to action. People want to get involved
in your project. What are the ways I already touched on some of it for submitting stuff,
but what are some of the ways people can get started or get involved?
Yeah, definitely check out the repo on GitHub. Check out the listing in the readme. Check out that
Jupyter notebook that's in there. Look for the RSS link as well. If you're an RSS reader user like me,
a bunch of interesting Python RSS readers. If you're not, you can just download one on that page too.
And yeah, then submit some more applications and also keep your eye out for the PyBay talk and probably
the PyGotham talk. All right. Excellent. Well, this was so much fun to talk to you about all this stuff.
And, you know, nice work on having this angle because I don't feel like it was covered and you've done a
really good job. It seems like it's taken off. It's got almost 10,000 stars on GitHub.
Yeah. I mean, and it's not really about that, but I am happy to see that it sort of gets out there
more. Speaking of giving it more coverage, I did technically start a YouTube channel. You can like
and subscribe if you'd like. It's yak.party, Y-A-K dot party. Maybe I'll post some APA stuff to it
in the coming future once I'm done with these talks.
That sounds great. All right. Well, thanks for being here as always.
Absolutely. Anytime, Mike.
You bet. Bye.
This has been another episode of Talk Python to Me. Our guest on this episode was Mahmoud Hashemi,
and it's been brought to you by Linode and Tidelift. Linode is your go-to hosting for whatever you're
building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E.
If you run an open source project, Tidelift wants to help you get paid for keeping it going strong.
Just visit talkpython.fm/Tidelift, search for your package, and get started today.
Want to level up your Python? If you're just getting started, try my Python Jumpstart by
Building 10 Apps course. Or if you're looking for something more advanced, check out our new
async course that digs into all the different types of async programming you can do in Python.
And of course, if you're interested in more than one of these, be sure to check out our
Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python. We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much
for listening. I really appreciate it. Now get out there and write some Python code.
you