Listeners often tell me one of the really valuable aspects of this podcast is the packages and libraries that they hear about and start using in their projects.
On this episode, I've invited Brian Okken, my co-host over on Python Bytes, to take this idea to 11.
We're going to cover the top 30 Python packages from the past year, a metric to be determined later in the show.
This is Talk Python to Me, episode 181, recorded October 3rd, 2018.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via at Talk Python.
This episode is brought to you by Linode and CloudBold.
Please check out what they're offering during their segments. Really helps support the show.
Brian, welcome back to Talk Python.
Oh, yeah, thanks.
Yeah, you've been here a bunch of times on this show.
And of course, we meet up quite often on Python Bytes, right?
Yeah, every week.
Every week. That's such a fun show.
And so we get to talk, you know, this is like the sort of extended edition of Python Bytes in some sense for our topic today.
Because on Python Bytes, we meet up, we talk about a couple of cool projects we found this week.
This one is, hey, here's a whole bunch of cool projects found throughout the year.
But this time we didn't pick it, did we?
No, it came from, where did it come from?
MyBridge AI.
It wasn't even from a person then.
No, so there's this app, this company, I guess, called MyBridge.
And they kind of like Zite or Flipboard curate a list of articles and other interesting reading.
But instead of just trying to do that for general population, they try to do that for professionals in various industries.
So one of the things they came up with was 30 amazing Python projects, the 2018 edition.
And we're just going to go through that list and just talk about these projects and sort of shine a light on some cool stuff that, well, this AI picked out for us.
Yeah.
And it's one of the interesting things is the, you can, I mean, like individuals can go in and do just the popularity of a, like the number of stars or the number of, maybe that number of downloads on PyPI or something.
But this is also, they also used the engagement, I'm guessing, how many, how often things get added to possibly and recency.
So if things haven't been edited for a long time, they're probably not going to be on the list.
Right.
Or stuff that maybe is new, but it has the same number of stars or something that's old, right?
That would make it more valuable.
Yeah.
So before we get into our main topic, though, let's just do a little background on you since it's been a while since you've been on this show.
Although I'm sure many people that listen to Talk Python also listen to Python Bytes.
A couple of things that you're notable for.
One is the pytest book, right?
What's the title of your book?
Python Testing with pytest.
Not very creative, but is exactly what it is.
Yeah, it's quite descriptive.
It starts out with an introduction just to get people's feet wet with pytest.
I've had a lot of people say that they didn't get lost with it, but a lot of the stuff there is stuff that they'll use down the road.
Like we get into building plugins and things like that.
Yeah, yeah.
Very nice.
And what do you do day to day?
Do something testing related, right?
Yeah, I make test equipment.
So I work for Roden Schwartz and make electronic test equipment.
And me specifically, I'm a manager for some C++ people that do embedded and a test team that does testing of the hardware.
Nice.
That's very cool.
All right.
So are you ready to get into our topics?
Yeah, definitely.
What's our first one?
The first one is Home Assistant at homeassistant.io.
And it is a open source automation, home automation that puts local control and privacy first.
So I haven't actually played with this, but it looks super awesome.
So there's a lot of different automated home automation things out there like Nest and others.
And then some do-it-yourself stuff like with Raspberry Pi and Arduino projects.
But this is sort of an overreaching one that interacts with all of these things.
And you can control multiple devices with one interface, track multiple devices, and then have some cross-device automation and notifications.
Like, for instance, you could monitor the soil moisture outside and then alert you, send you an email or something if it's too dry or something like that.
Even better, how about have it automatically turn on your sprinklers?
Because you have smart sprinklers, right?
You're right.
Only when it's needed because not when it rained.
Yeah.
One of the things I want to play with is the ability to, there's a location things where you can hook up through your, allow an app on your cell phone or something to know where you are.
And you can say, hey, when I leave my work, go ahead and turn the air conditioning up or the heat up in the house so that the temperature's right when I get home, regardless of when I left work.
So that'd be fun.
Yeah.
There's all sorts of cool stuff.
Another thing I want to give a shout out to here is Haas.
I guess the way you'd say it shortening would be Haas.io, H-A-S-S.io.
Or maybe Haas, like the avocado.
Yeah, I guess so.
Okay.
So this is a little thing that you can run on a Raspberry Pi that is like an embedded home assistant.
So you can just have a Raspberry Pi in your house and you have a private server that controls all of your devices.
And it's an iPhone app you can install that then talks to this private server and all sorts of cool stuff.
That's cool.
So you're not dependent on a cloud service or something for these sorts of things.
Right.
Exactly.
Yeah.
Isn't that cool?
Nice.
Yeah.
So Home Assistant, if you're doing anything with a smart home and you care at all about Python, there's tons of integration and it's all in Python, which is amazing.
Well, what's next?
Next up, we have PyTorch, which is a deep learning library, which is pretty awesome, right?
Yeah.
Is that from Microsoft or who's that from?
I think it's its own project.
Okay.
It's even coming out with a new version here pretty soon or maybe already.
Yeah.
So what's really cool about PyTorch, it's basically lets you do a lot of GPU accelerated, like tensor deep learning type of stuff, deep neural networks and so on.
But the new thing that's cool is in PyTorch 1.0 coming along, you can accelerate your Python machine learning code with native code.
So they've actually created their own little mini language called TorchScript.
And TorchScript is a subset of Python that can be compiled into either C++ or other high speed languages.
Nice.
This is very cool.
Yeah, it's pretty cool.
So it's already had like support for Cython to extend it, but now it even has support for its own special language coming soon.
So I think that's an RC1.
So if you're doing any machine learning, PyTorch, you probably already know about it, but quite cool.
And this little TorchScript subset thing is quite interesting as well.
Yeah.
Normally, I'm pretty happy about these new libraries, but sometimes, you know, not too much, right?
Yeah, I kind of get a little grumpy.
You can definitely get grumpy.
Well, grumpy is next on our list.
And grumpy is a Python to Go or Trans compiler.
Maybe it's the same thing.
Yeah.
And runtime.
So you can write Python code and have it run on a Go runtime.
Sure, why not?
That's pretty interesting.
Yeah, why not?
I mean, one of the things you get is pretty good support for concurrency because Go is all about that, right?
So that's kind of cool.
One of the things that kind of makes me grumpy about grumpy is that it's a legacy Python, not modern Python.
Oh, it's 2.7.
Yeah, the reason is this is a super focused project from the YouTube team.
And the YouTube team is running YouTube on Python 2.7.
And they're trying to make it go faster because they have requirements that I suspect most people don't have.
I mean, our podcast, Python Bytes, for example, gets a lot of traffic, right?
Yeah.
But, you know, if you look at YouTube, they get a million page views per second.
Yeah.
That's a lot of traffic.
That's a lot.
So actually, I had the grumpy guys on, well, the guy behind grumpy on Talk Python in episode 95.
And a lot of these topics we've covered in one place or another.
So I'll try to give a shout out to like more deep coverage on them if that's interesting to you.
But yeah, grumpy is kind of interesting.
I hope I would love to see this go somewhere.
Actually, I would love to see alternate core implementations.
Like I would love to see a Rust re-implementation of CPython, the runtime as well.
So pretty cool.
Yeah.
So Go has asynchronous stuff and so does Python.
And as of Python 3.5, we have async and await, right?
And that kind of unlocks a certain type of really good concurrency, which is the kind that is based on IO.
That's waiting on databases, waiting on microservices, waiting on the file system.
You know what websites do a lot of?
Wait on stuff.
Wait on stuff, right?
Wait on databases.
Wait on web services, et cetera.
And so Sanic is one of the big frameworks that came out pretty recently and is one of the really fast async and await based Python web frameworks.
And like it seems like all the new web frameworks that are coming out, they're like, yeah, we're basically the Flask API with a different implementation.
And this is no different.
Yeah.
One of the things that's kind of fun with Sanic is there's been a lot of like proof of concept things coming up.
But this one does look like it's gaining traction.
Yeah.
It's pretty cool.
And it's based on this thing called UV loop, which is an alternate implementation of asyncio in Python.
That's about twice to four times as fast.
So that's also pretty cool that it's based on that.
And also Sanic just announced, we'll probably cover this over on Python Bytes at some point, but they just announced that they're a community driven organization.
Used to be a project by a guy and now it's sort of its own thing.
A little bit like Palette's is to Flask.
There's that organization.
Oh, nice.
So if you're looking to contribute to projects, this is a good chance to do that.
They even, I don't know how many projects do this, but I think this is a great idea.
They say in there that you can go to GitHub and look through the various items and they'll put a help wanted tag on things they think would be a good thing for somebody to grab and contribute.
That's cool.
Yeah, it's pretty nice.
Yeah.
They're on fire, aren't they?
Yeah.
And they are on fire.
And next up is Python Fire.
So Python Fire is a, it's an interesting little project in, well, I don't know how little it is.
I haven't looked at the source code, but it's a different take on command line.
It's a little in scope though.
Yeah, a little in scope.
A different take on command line interface generation.
I'm familiar with Click and there's quite a few others, but Python Fire does it kind of a little different.
You put some boilerplate, not a whole bunch, but a little bit in one of your files.
And then you can, all the functions within that module get exposed as command line arguments.
And it's kind of magical how it works.
And it's fun.
It's a, if you're confused by generating command line interfaces, check this out.
It also, they kind of push it for people that are like, you just quick and dirty just for yourself or your small team.
These are, this might be a good thing.
Yeah, it's pretty cool.
Their little subtitle or whatever is automatically generating command line interfaces from absolutely any Python object.
That's pretty cool.
Yeah.
And then their documentation is beefed up recently.
So they, they've built a guide and there's a lot more easier ways to get into it now.
So that's cool.
Yeah, that's cool.
And this is from Google.
So, you know, you can pretty much trust.
It's probably pretty good, pretty nicely put together.
Yeah.
So going back to machine learning, our next one is Spacey.
When you think, when I think, at least when I think of machine learning and like text understanding, natural language understanding, I always thought of NLTP.
That seemed like the way to go.
But Spacey is the new fancy way to do that apparently.
And it's written in Python and Cython.
So it makes it super fast.
Yeah, this looks, this looks very interesting.
And natural language processing is actually, it's getting used quite a bit.
Yeah.
Anytime you want to take text and understand what the words mean, this is it, right?
If you say wanted to, maybe you were an algorithmic trader and you were studying the live flow of data on Twitter and trying to look for sentiment analysis around a particular stock and then having automatic trading happen on that.
You probably would put Spacey in there.
Yeah, sure.
Or do you want to?
Just grabbing an example out of the scenario.
Or maybe tracking the emotions of a certain chief of a country on Twitter.
You can maybe use this.
I'm not sure.
I'm not sure we'll understand that.
You can try.
All right.
So speaking of something I don't understand, and I want to understand it, help me understand PIP-ENV.
PIP-ENV?
PIP-ENV.
There's so many ways to create virtual environments and work with them and dependencies and so on.
And then there's this other way with PIP-ENV, which is really awesome, that applies some of the time.
It's done partly by Kenneth Wright.
So very cool.
But I don't know where it fits in my world still.
It's for applications.
And it's really good for coordinating all of the dependencies for application developers.
So if you've got a team of people working on an application, I think this is a good way to go.
Some of the ways to get to is built in work to separate deployment environments versus development environments, because you might need you might need extra things around like pytest and document generation and stuff like that.
If you're doing development that you don't need when you're just deploying, having those separated out is handled by PIP-ENV rather beautifully.
Yeah, nice.
So it creates and manages virtual environments.
It replaces requirements.txt and friends with the pip file and the pip file.lock.
And it sort of pins the versions and all of those kinds of things.
So pretty interesting.
I got to say, I'm still pip install dash R some stuff these days.
It also, it can generate your requirements file.
So it can play nice.
You can have a project play in both worlds at the same time.
Yeah, that's cool.
No, it's a good project.
And I just, I need to, you know, shake myself out of my rut and learn this as well.
And then I can decide better where it fits for me.
So one thing I've always wanted to do is create like a product that I could sell out of little tiny, like IoT smart things.
And I'm still trying to understand like what that might be.
But if I were to do that, I know what project.
Yeah, MicroPython.
Right on, MicroPython.
Tell us what that is.
No, it's your turn.
All right.
So MicroPython is basically a Python implementation of the operating system that runs microcontrollers.
So it's not like I have a Raspberry Pi, which is running some variation of Linux, and then I can install CPython on there.
It's like Python itself is the operating system.
And you can do insane things.
Like you can hook up a Lambda expression to a hardware.
Yeah, that's so cool.
Like that low level.
Isn't that amazing?
So you could like put this on like a $5 little embedded chip.
And it's got pretty good support.
It's Python 3.
And there's the Mu editor and other editors that can talk directly to it.
So developing in this world sounds fun.
It's been going for a few years.
And it's still going strong.
So I'm super excited for that.
Yeah.
A MicroPython evangelist.
Or maybe a prophet.
Ah, yes.
It's as if a prophet for it.
Indeed, prophet is the next one.
What's that?
I actually don't know.
I should have taken the last one.
It's a tool for producing high quality forecasts for time series data that has multiple seasonality with linear and nonlinear growth.
I don't even know what that I just read.
So, yeah.
So the idea is this is a project from Facebook.
So it's for predicting trends and time series data that might not be like stock prices.
Yeah.
Like stock prices or things that are not just completely linear, completely the same, right?
Like the seasonality and the nonlinear growth and things like that.
So it's just a really advanced library for predicting, given a time series of data, what's going to happen in the future.
Oh, there's a lot of uses for that.
That sounds neat.
Yeah.
It's pretty cool.
And from Facebook, I bet they do a little bit of predicting and analyzing and stuff like that with like your feed and whatnot.
Yeah.
And maybe for like predicting ad prices and things like that.
Yeah, exactly.
This portion of Talk Python to me is brought to you by Linode.
Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?
Look past that bookstore and check out Linode at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Plans start at just $5 a month for a dedicated server with a gig of RAM.
They have 10 data centers across the globe.
So no matter where you are, there's a data center near you.
Whether you want to run your Python web app, host a private Git server, or file server,
you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support,
even on holidays, and a seven-day money-back guarantee.
Do you need a little help with your infrastructure?
They even offer professional services to help you get started with architecture, migrations, and more.
Get a dedicated server for free for the next four months.
Just visit talkpython.fm/Linode.
The next one is something called Serpent AI.
And we actually talked about this back on Python Bytes number 50, but this is really cool.
It is cool.
It's like AI with game development, or actually, you better take this one too.
I'm not sure what this is.
So the idea is if you have an AI and you want to train it, you could construct like a fake world for it, right?
You could tell it, all right, you're in a box, you can move around, or like you're in a maze,
and you'd have to build that maze, right?
For your game.
You'd basically have to build a miniature game and then try to teach a character in the game a thing if you want to develop AI.
But we have all these real games that are super, super interesting, super nuanced, right?
Like games you might play.
Okay.
Racing games, first-person shooter games, whatever.
So Serpent AI attempts to let you control that game with Python, whatever that game is.
They claim you can turn any video game into a sandbox environment for AI and bot programming.
So imagine I'm trying to create a robot that can interact with the world.
You could try to create your own world, or you could stick that thing in Grand Theft Auto or some other game and make it run around and go,
okay, well, it's really good in this world.
So it's a little bit, let's use sidestep creating the environment for testing AI.
That's so cool.
Yeah.
It says it even works with like games based on Steam or stuff like that.
So it's pretty cool.
The guy who creates it also does a Twitch programming channel.
Okay.
It's kind of interesting, right?
Yeah.
So like he'll like fire it up and say, all right, I'm going to go and take this game.
I'm going to create an AI that's going to do this.
And he'll just like start from scratch.
So if, you know, if you got a Friday night and you don't got anything going on, you could watch that.
Oh, yeah.
That sounds great.
Because I don't really want to do it myself, but I'd like to watch somebody else do it.
Exactly.
Yeah.
All right.
So that's a good one.
If you're into like creating AIs and bots and stuff like that.
Pretty awesome.
Yeah.
The next one is just beautiful, right?
Yeah.
It is nice.
And this one I actually can understand.
So it's good.
Next one is Dash.
Tell us about it.
It's a framework for building analytical web applications.
But so it's kind of like you need plots and, you know, Python code interactions.
But you need some real-time stuff.
So you like you want to zoom in and look at new data and have data go back and forth.
And usually you do JavaScript or something like that and maybe React or something.
And this is, it's built on top of Plotly and React and Flask.
And it's just really, it's really a beautiful interface for data and for interacting with stuff on the web.
So I actually tried to use this.
And I think it would be fine for most applications.
But you do need to have either use their server or you can set up your own server.
But there needs to be a server to do a lot of the crunching.
So.
Right.
So basically you need a server that runs Flask.
Okay.
Their thing that's derived on top of Flash.
Yeah.
Right.
Yeah.
So if you go and open up the link, go to their main website, there's a part that says Dash is productive.
And it shows the amount of code to create this cool graph that's totally interactable, zoomable, drop down to choose different data sources.
And you basically have to say, I would like to graph these things and my data source is Google and my date range is this year.
And somehow it can just go and get like stock data and other stuff like straight from Google.
You don't even need to provide it the data.
Wow.
Yeah.
That's neat.
So there's a lot of, if you're trying to create visualizations on the internet and you're mostly happy writing in Python, this is a really pretty sweet thing.
Yeah, it is.
And it's really pretty.
Yeah, it's definitely pretty.
So, you know, another place where a lot of pretty stuff gets put is on Instagram.
Yeah.
This is actually an amusing little project called Instapy.
It's for trying to make your own Instagram bots.
And it's built just as if you were interacting with Instagram yourself through a webpage because it's built on Selenium and it's Python powered Selenium.
It's kind of a little pet project that might be used for, you know, like automatically farming out likes and commenting on certain pictures that have something in it or something like that or comments and followers.
There's a lot of stuff it's a good thing.
I'm not sure it's one of those.
Is this a good thing or a bad thing?
Yes, I know.
It was a little bit like that.
It's definitely interesting.
Yeah, it's definitely interesting.
Like I can see if you were, say, a social media company, like you did social media as a consulting project for large companies, you could write some cool automation that maybe automatically analyzes all the ways people interacted with you, store them, generate reports, stuff like that.
Yeah, so there's some good uses, but there's also, you know, bots in general.
I feel the same way about Twitter bots.
They kind of bug me, but what are you going to do?
It's the internet.
Yeah.
Yep.
Speaking of the internet, the internet's full of APIs.
And the next project is API Star, which this is a project that fully embraces Python 3.
So this is from Tom Christie, the guy who wrote Django REST Framework, but this is sort of his reimagining of what a REST Framework would look like in Python with Python 3.5 and above.
And it's really cool because it does things like the arguments to your API methods use type annotations to get their values.
So you can have like a variable or an argument colon and say this comes from the header and it's automatically extracted out of the header because of the type hint and things like that.
It's pretty crazy.
So yeah, this is a good one.
Yep.
Very good.
And you said you talked with him about it on one of your episodes, right?
I did back in episode 125.
I had Tom on there to talk about that.
So definitely a link to that as well.
Okay.
All right.
This next one, I'm going to let you do this because this one sounds simple.
Yeah, sure.
It's very simple.
I don't even know how to pronounce it.
It's spelled F-A-I-S-S.
Maybe F-A-I-S-S?
F-A-I-S?
Yeah.
F-A-I-S?
I'm going F-A-I-S because it has to do with similarity and stuff.
Maybe F-A-I-S.
I'm going F-A-I-S.
Oh, okay.
It's a library for efficient similarity search and clustering of dense vectors.
Of course.
I have to do that all the time.
Of course.
It's a data science-y thing for people that understand that.
You probably understand.
Contains algorithms that search in sets of vectors of any size, even ones that don't fit in your RAM.
So that is big.
It's written in C++, so it's going to be fast.
And it's written by Facebook.
Yeah, it's another one of the Facebook AI projects.
And it also runs on GPUs.
So it's another specialized AI system.
I feel kind of a little, I feel a little bit like I'm not entirely keeping up with the whole world of AI.
It's going so fast.
No, but now I get the name.
It's Facebook AI similarity search.
Yeah, there you go.
Right on.
Okay.
Yeah.
And who knows, maybe even when it auto tags you.
Who knows?
Maybe it's using this.
Maybe.
All right.
So another one that's interesting, sticking with the web for a little bit, is Mechanical Soup.
Yeah.
Actually, I hadn't heard of this before I ran across this list.
So this is neat.
I haven't either.
Yeah.
And that's why I like this little article project that we're covering.
Because I would say a good five or six of these.
I'm like, whoa, I'd never even heard of this.
Mechanical Soup is an automation library for interacting with websites.
So a little like Selenium, a little like Requests.
It'll keep cookies and send them around.
It'll follow redirects and interact with links and forms and all kinds of stuff, which is pretty awesome.
Yeah.
I want to give sort of notable mention as well to another one from Kenneth Wright's here called Requests HTML, which is somewhat similar.
So this is Mechanical Soup, like Beautiful Soup.
And Requests, you know, everyone knows what Requests is.
Well, Kenneth Wright's created another project called Requests HTML, which kind of merges Beautiful Soup and Requests as well.
So they're in the same space.
Yeah.
And, you know, if something goes wrong, you can get an exception.
But at least you can make them better, right?
Yes.
With better exceptions.
What's that?
The better exceptions is just it's a you kind of have to modify your code a little bit to make use of these.
But it intercepts the exception process so that you're the output, the tracebacks from your exceptions are easier to read and prettier.
There's colors in there and it tells you what values are and it kind of points the value to the variable.
And actually, it's kind of fun.
It's a neat thing to check out.
Yeah, it basically creates little like lines mapping parts of the code.
And one of the things that's actually super cool, if you look at the GitHub screenshot there, when it shows you the error, do you see, Brian, how it actually shows you the values of the functions that the values that were passed to the function?
That is so incredibly cool.
So you'll say this function was called and it crashed.
And not only does it just say, too bad it was on this line and color it in like indented, but like if it has a say, here's a function called deep and it takes A plus B and it'll have little visual lines going A is 2 and B is 15 in this crash.
Like that is awesome.
Yeah.
And it's even though it'll even go back a little bit and interpret things beforehand.
So I'm not sure how this is doing it, but it's pretty cool.
It's definitely cool.
It's definitely better, isn't it?
Yeah.
For me, it's similar to most of my, a lot of time I'm in the testing world and pytest does a lot of this stuff for you also.
It isn't as pretty with the colors and everything.
Yeah.
Quite cool.
Anyway.
So have you heard the saying that if you have a problem that you solve with regular expressions, then you solve it with regular expressions.
Now you have two problems.
Yeah.
This next one is meant to kind of make that go from one to zero, not one to two.
Tell us about it.
One to zero.
I don't get that.
Well, you have a problem and it's solved rather than I had a problem.
Now I have two problems.
Now you have two problems.
So it's flash text.
And it's a very focused solution for searching and search and replace and things that you would often use regex for and finding words within sentences and grouping them into.
You can have a set of phrases that all kind of phrases that all kind of mean the same thing.
If you want New York and big apple to mean the same thing.
You can search them together.
And it's like really, really fast.
Like if you look at the link that will conclude, it has some graphs on a search size compared to regex.
And it's kind of incredible.
Yeah.
It's really nice.
And so you can say it'll take like the sentence.
I love big apple and, you know, the big apple and the Bay area or something like that.
But if you've told it big apple is a stand in for New York and you ask, Hey, what keywords are here?
It says New York and Bay area.
Or you could also put San Francisco, right?
Yeah.
So, so quite cool.
And it's a way to sort of normalize text data for various things people might say.
I like it.
Yeah.
It's super simple and you don't have to write regular expressions.
So that's good.
Yeah.
I'm not really afraid of regular expressions, but yeah.
Anyway.
Yeah.
I'm not terribly afraid of them, but they're not the first thing I'd jump for.
Okay.
So I find dates in Python to be surprisingly not obvious.
How about you?
Well, they're really easy until they're not easy.
Exactly.
You're like, wait a minute.
That one has a time zone in it and that one doesn't.
You're done.
Too bad.
Good luck with that.
There's a lot of projects in Python to deal with, deal with date time.
And Maya is one that is, is from Kenneth Reitz.
Yeah.
And so, so it's a for humans application.
Yeah.
People feel better when whatever their project that they're using, like request HTTP for humans,
right?
So this is date times for humans.
Quite nice.
So if you're working with date times, it has a lot of, a lot of cool features and, and simplifies things around date times.
Some of the slang features are kind of fun.
So you can say, do things like give it a date and say a week ago or, and have it generate things like those sorts of slang versions of words.
And so interacting with putting out text, you don't want to use JavaScript.
You want to keep it on the Python side.
Maya is a good choice.
But anyway.
Yeah.
It's cool.
Yeah.
It's cool.
So you can say like slang time and it'll say that's, you know, 23 hours from now, or that's tomorrow, or it's yesterday, which is a real nice way to not be overly precise.
If you want to just put that on like a forum or something to that effect.
This portion of talk Python to me is brought to you by Cloudbolt.
Friends don't let friends violate PEP 8, nor do they let them spend their days in an unfulfilled work environment.
Good news.
Your friends at Cloudbolt want your help developing their state-of-the-art cloud management software.
Built with Django and ranked as the number one product of its kind, Cloudbolt is looking for talented engineers of all types.
Located in beautiful Portland, Oregon, Cloudbolt is an hour from the Pacific Ocean and Mount Hood.
You're not in Portland?
Not a problem.
Cloudbolt offers a relocation stipend to the Pacific Northwest and is also hiring solution engineers from everywhere around the globe.
Whether you're interested in containers, hypervisors, or just writing clean performant Python code, Cloudbolt would love to hear from you.
Visit talkpython.fm/Cloudbolt for more information.
That's talkpython.fm/Cloudbolt.
So I think this next one is right up your alley, Brian.
Really?
I do.
I'll take it for you, but I think it's up your alley.
So I don't know how to say it.
Mimesis?
I'm going to go with mimesis.
I don't know.
So it's another one of these fake.
It's another one of these faker libraries.
So the idea is you can go and say, I would like to create a person object.
And you can ask, what's their occupation?
How old are they?
You know, what's their name?
And you can tell things like, please answer me in English or German or whatever.
And it will basically generate fake test data.
So you could, you know, create a fake database and put it in a SQLite file, then use that
for testing or things like that.
So pretty cool.
Oh, yeah.
That does look neat.
And it looks like they have a page on like, well, why am I using this versus something else?
So they have a comparison to other libraries.
So it's good when people are trying to decide.
Yeah, it has a cool function called identifier.
And you can give it a mask, just like number, number, number, dash, dash, number, number,
slash, number.
And it'll like create one.
So if you need, say, like social security numbers or something else like that, it'll,
it's really easy to generate those.
Quite nice.
Yeah, I could see where that'd be useful.
Neat.
Yep.
So this next one is called open paperless for scanning documents.
I know, I don't know anything about this.
Tell me about that one.
Well, so this is a, I didn't know about this before reading this.
So apparently there's a, a document management system called the Mayan EDMS.
So I'm guessing Mayan electronic document management system, probably for some reason,
there's open paperless is a, even though the Mayan EDMS is a, is an open project.
Also the interface, apparently at least some people don't like the interface.
So open paperless is a Python interface to this system and a simplified interface to it.
So probably only have concern with, for people that interested in that, but maybe I should be.
Yeah.
Well, it's, it's got a lot of possibility for the automate the boring stuff type of things,
right?
Like I'm so tired of being in this office and doing this.
Like, can I write Python to make my job better?
Yeah, definitely.
Right.
Like I don't use paper for anything.
I'm not sure it'd be any value to me, but I can see how it would be for others.
Yeah.
And like things like places where you have to keep signed, keep copies of signed contracts
and receipts and all sorts of, there's a lot of jobs where you have to keep at least the,
a scan of the physical version.
So that would be good.
All right.
This next one is a shout out to my, one of my absolute favorite TV series.
And it scares me to death.
Really?
So F society.
Okay.
That's the hacker crew from Mr. Robot.
Okay.
It means a couple of things.
It's a play on like that little arcade where they, they all hung out where the U and the
N fell off, but also, you know, the more obscene version of it as well.
But so this one is a set of tools for hackers.
If you're doing penetration testing or things like that.
So you can check out, this is on GitHub and you can go and grab it.
So if you say wanted to turn this loose against your code or your infrastructure to see what
would happen if somebody else did that, you might want to give that a look.
Yeah.
And hopefully the people that you're trying to protect from aren't using the same tools,
but you know, that's, that's one of those good thing, bad things sort of things, but yeah,
for sure.
Yeah.
And it turns out that a Python is a big hit with hackers.
It's actually an interesting article about like the growth of Python adoption amongst hackers
was like 77% over the last year or something crazy.
like that.
Well, they're a segment of the population and Python's growing in all segments.
So yeah, that's right.
It's just, maybe it's because of all the opportunities to do live coding.
It definitely could be.
So this, you can take your program and you can hook up a debugger to it and sort of step
through.
That could be like PDB or that could be PyJarm or something like that.
But this live Python one is a little bit more of a, I would like to sit back and watch how
my program works.
Right.
Yeah.
And I'm still kind of a little confused by this one, but it's a, it's an application that's,
so you have to, it sits on your desktop and yeah, you kind of watch as your program's running
to tracing sort of thing.
And then you can see lots of stuff going on inside.
Yeah.
So it says, it lets you basically watch your program run like a movie.
So you can say, go and I'll sit back and it'll show like the execution through your program,
as well as like a stack of variables, like a watch window of variables just on the side
there, which is, it's pretty cool.
Yeah.
Maybe you could just have it.
You could have that just running while you're at lunch and people think you're working while
you're.
I'm sorry.
I'm not, I'm not coming back.
I'm compiling.
Yeah.
I don't know.
It might be fun.
I think it's a good idea.
I saw there was some coffee cup that you can get that it'll like heat itself up at like
six 30 in the morning and to like start to steam before you get into the office.
So it looks like you're in early, you just stepped out.
So this can go along with that.
Yeah.
So I told you I was confused about PIPF, right?
Yeah.
Still, still confused.
And Hatch is also pretty awesome, but it also makes me confused because there's a
there's all these different ways of what time might work with stuff.
So Hatch is another package manager, virtual environment manager for Python, which is pretty
cool and on our list.
Yeah.
And it also like they're taking over lots of other stuff too.
So it, it does things like start a project and like cookie gutter that kind of does in
it.
It also does things like push up to PIPI and, and a whole bunch of other things.
Right.
It has pytest support.
Yeah.
I don't have any issues with all the tools by themselves.
So I'm still, yeah, I don't, I don't think I need this, but you know, but a lot of people
use it and like it over other things.
Yeah.
It's cool.
And I think what this really means that there's all these different ways of doing this stuff.
It's like, it's just not really totally nailed down.
Like I recently interviewed the folks around the Python language summit, which is where the
core developers get together.
And one of, one of the topics at the core developers meeting in 2018 was virtual environments.
Do we need them?
Why are they so hard to teach?
Can we make them simpler?
Right.
I'm telling you, this is just like the fact that there's all these different solutions,
I mean, it's just not quite a solved problem.
So I guess that's, you know, it's good that people are trying to solve the problem.
Yeah, definitely.
This next one that came up is, I guess if you're in the right space, it'll be useful.
I'm not totally entirely sure, but number 24 is tangent, which is, it does source to source
debuggable derivatives.
Sure.
In pure Python.
Why not?
As in, you know, DYDX.
Yeah.
The first derivative of a thing.
So anyway, it's, if you have to do numerical differentiation, Python, it's pretty cool.
I probably don't want to say more than that about this one.
Sure.
So previously we had profit.
Did you foresee that Clairvoyant was coming?
Yeah, I knew that.
Because I wrote it in here, in the list.
So Clairvoyant, which is like kind of an awesome name for a project.
It's a software designed to identify and monitor social and historical cues for short-term stock
movement.
So it's, oh yeah, it's like kind of like profit, but focused on stocks.
I'm guessing.
Yeah.
Nice.
Yeah.
For all the algorithmic traders out there.
Here you go.
One that is pretty promising is called Monkey Type.
And that will come from Instagram, I believe, right?
Yeah, that's from Instagram.
So we have mypy, we have Monkey Type, and a couple of these other systems that are out
there trying to take code annotations and help us either evolve or better understand our code
or lent them to make sure they're correct based on their typing, right?
Yeah.
And so this one's kind of a runtime thing that it's, so if you, if you're not sure how your
code is being used, it can take a look at your, at how it's being used in runtime and
tell you what the types are.
What's really interesting is it'll generate a stub.
You know, the stubs, like it might say def add a colon int, b colon int, goes to int, and
then just dot, dot, dot instead of an implementation.
And then Python itself and mypy and PyCharm and whatnot can use that to actually validate
your code.
So it doesn't even have to modify your code.
It can generate these sort of parallel side type definition files, these stub files, and
put it on a type shed.
It'd be pretty cool.
Yeah.
And I think it might be kind of fun to use as a, just to, to go over some code to see
if what you think you're doing is really actually how it's being used.
Right.
And it's based on how it works at runtime.
So it looks at your code and says, well, every time you call this function, you pass an int.
So that's getting a colon int put on it right there.
Yeah.
You thought it was a string.
It's not.
It's an int.
No, it's an int.
That's why it's never equal to the string.
So, and you can even say, okay, do that and apply those back to my code and I'll put Python
three.
Yeah.
Annotations.
It'll rewrite your code that has no annotations with annotations guessed from runtime behavior.
That's pretty awesome.
Yeah.
That's nice.
And then you can just diff it and see what it does and whatever.
Yeah.
And then we have, there's so much stuff happening on the space.
These are not, this next year are not really on the list.
But we have mypy, which we talked about, right?
And there's mypyc, which will take this code here that we're talking about and compile it
to C. Standard Python with annotations compiled to C automatically.
Okay.
I'm not sure how that helps me.
That's a new project coming from Dropbox, actually.
Okay.
Because compiled Python needs annotations?
No.
If you want to, let's say you want to write a C extension.
One thing you could do is write the C code, compile it, and then import it.
The other thing you could do is just say this module, I would like to run through mypyc,
and now it's a C extension.
Oh, okay.
Now I get it.
Yeah?
Yeah.
I'm slow.
I know you work at a place where they do C and C++, but not everyone necessarily wants
to write it, right?
So this will let you skip that or maintain all Python, but still get this equivalent of what
had happened if you had done C compilation.
Why are we even writing C anymore if we could just have this?
I know.
And the other thing I want to throw in there is Cython will also take this code and compile
it to C as well with new...
Now it just does the Python 3 type annotations, and it would make that work as well.
So you could take monkey type and get yourself into a place where either of those things would
work on it.
It'd be pretty cool.
Nice.
They all fit together.
Yes.
You know, I don't know how many people listen to Python bytes, but if they did, they have
to know we've been on a GUI trip, right?
Yeah.
With the help of a lot of people telling us stuff.
We're like, oh, there's four.
They're like, no, no, no.
Let me tell you.
Let's count the ways.
Yeah.
So there's lots of ways to do user interfaces in Python.
One of the interesting ones is Eel.
Yeah.
Which I'm amused by the name because I got to make the obvious pun.
It's...
Or the...
Just state the obvious.
It's an electric Python.
Like Electron, but with Python.
Get it, Eel?
I love it.
The name is actually really awesome.
So for those who don't know, ElectronJS...
A lot of apps, probably one of the more notable ones is actually Slack, I believe.
I'm pretty sure Slack is certainly Visual Studio Code is an ElectronJS app.
Atom is an ElectronJS app.
Basically write in web stuff, HTML, JavaScript, CSS, and then run in on top of Node and embedded
Chrome.
So this lets you take Python and replace the Node side with it somewhat.
And it says it's for simple Electron-like apps, which is pretty cool.
So another one, speaking of listeners helping us out, that's pretty awesome, that I think
is more full-featured, is called Python Electron.
And I'm going to put a link to that in the show notes as well.
And there's actually some pretty cool interaction of here's how you write your Python code and use
zero MQ to communicate back and forth with the browser.
So pretty cool.
You can write just like standard Python and then you have an object and you just say,
this object is...
I would like to take this object and make it the API that my app can communicate with.
It's cool.
Yeah, neat.
I would say that was even surprising.
It was surprising.
And next up is Surprise.
So Surprise stands for Simple Python Recommendation System Engine.
The U and the R are silent.
A Python scikit...
Scikit's a noun?
A Python scikit for building and analyzing recommender systems.
Yeah, that's pretty cool.
So if you have any sort of e-commerce system or if you read this article, you might also
like that article.
Here you go.
Surprise lib.com, right?
Yeah.
Nice.
So we talked about mechanical soup.
The other one that we have previously talked about on Python Bytes not long ago is something
called Gain, which is a web crawler, much like the other ones we talked about.
But this one is special because it's based on AsyncIO, which is really cool, and UVloop and
AIOHTP and things like that.
Yeah.
It's a web crawler framework thing.
Yes.
You basically give it, you know, here is, let's say, a URL.
Start here.
And then go actually find all the other URLs and crawl them.
So basically, if you wanted to create DuckDuckGo, start here.
Yeah.
We need another one of those.
That's right.
But if you want to go to a website and sort of explore all the links and start downloading
and processing that stuff, Gain is pretty awesome because it does it super, super low latency
and in parallel with AsyncIO.
And actually, so I like projects like this, not necessarily to make commercial products out
of them, but for your own thing.
So if you've got a large website that you're maintaining or a company website or your own
personal one or whatever, and you want to make sure things are working right, being able
to crawl it and then interact with, instead of having hooks, you've got a tool for web crawling
that you can hook other Python up to.
There's a lot of stuff you can do with that and having it do it quickly on maybe your
limited little CPU that you've got playing around.
I think it's neat.
Yeah, that's awesome.
So yeah, that's a really good example as well is if I've got a large website, maybe it's even
like a CMS or something where I don't necessarily control all the stuff that goes into it.
You want to make sure there's no broken links.
You can easily do that with Gain.
That'd be cool.
Make that part of your automated build.
All right.
The last one here is one of these automate the boring stuff type things.
It's not interesting to many people, but if you need it, you really need it.
So if you would like to interact with PDFs and get data out of them, then PDF tab extract is
for you, right?
Yeah.
And the, I mean, the article and the, the, the, even the read me on this show some kind
of amazingly horrible scans of documents.
They're not straight.
Oh, they are horrible.
Oh my goodness.
They're so bad.
Yeah.
And there's like, I can hardly read them.
Yeah.
You can hardly read them.
And yet this has, this is a way to OCR and extract data and then create data sets out of,
out of like scans of things with tables in them.
So one of those things of, if you need to, if you need to, if this is your job, yeah, automating
this is a good idea.
That is so awesome because normally what you could do with some of the tools for PDFs is
get the text represented in the file.
But this, these examples are not that these are, here's your tabular data just scanned from
an image.
Now get the text out of it.
Yeah.
Cool.
So I'm sure that's going to make someone's day.
Right.
So that's our, our top 30 from 2018, which is really based a little bit on slightly older
data, but it was fun to cover with you, Brian.
It definitely was fun.
Um, and, similar feel is, Python bytes.
So if you, if you like this sort of a thing, we get to spend a little bit more time per
topic on Python bytes.
So head over to there and check us out there.
Yeah, absolutely.
That's a fun podcast for sure.
So do check that one out.
All right.
Before you go though, there's the two questions.
I always, always get to ask if you're going to write some code these days, what editor Python
code specifically.
PyCharm, always now has incredible pytest support.
And then I also got, I'm going to get this wrong, but I got, Oliver Best Walter keyed
me onto a project called like, power mode or something.
And, when you, when you type it's like sparks and fire comes out and it's just a blast.
So I'm using that all the time now.
So you want to feel powerful.
Like I'm kind of just low energy this morning.
All right.
Yeah.
Power mode, baby.
It's all.
When you, when, when you copy and paste a chunk of code also, or cut a big chunk of
code, it pops up like a bam, like a, on comics.
Um, it's neat.
Oh my gosh.
I have to find this.
You're going to change my, the way my, my, coding works.
This is going to be bad.
I had to turn off the flames and the shaking of the screen.
Cause I could, it can't work like that, but the sparks, I can, I can just work with those
sparks.
And so that's pretty excellent.
Normally I would ask you for a notable PI PI package, but we literally just covered like
35 PI packages.
Definitely.
So I think we're good.
I think we're good on that one, but let me swap that one out.
Just let, why don't you tell people about testing code, your other podcast, which you didn't give
a shout out to yet.
Oh yeah.
I also do another podcast called testing code.
That's a test and code T a two.
I can't spell today.
T E S T A N D C O D E.F.
That's not that FM.
Why am I doing that?
It's testing code.com.
That's it.
Yeah.
We cover all bunch of stuff.
We don't just do testing and cover a lot of other things too.
I know that we already have talk Python, but I do a little bit different take on things
and it's fun.
And I'm ramping things up.
We're doing weekly, weekly podcasts for the rest of the year.
That's awesome.
And you've had some pretty notable folks on there, so that's good.
Keep it up.
Thanks.
All right.
Well, hopefully people have found a couple of things that really apply to what they're
doing that they maybe hadn't heard of.
I know I had when I went through this.
Yeah, I did too.
It was a surprising.
It was very surprising.
My clairvoyance wasn't good.
My prophecies were not.
A lot to gain.
We're going to stop now.
Oh, sorry.
Oh my gosh, there was a lot to gain.
All right.
Let's just leave it there, Brian.
Thank you so much for coming.
Bye.
Thank you.
Bye.
This has been another episode of Talk Python to Me.
Our guest on this episode was Brian Okken, and it's been brought to you by Linode and Cloudbolt.
Linode is bulletproof hosting for whatever you're building with Python.
Get four months free at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Spend your work time fulfilled.
Write Python and Django code at Cloudbolt, developing their state-of-the-art cloud management software
in beautiful Portland, Oregon.
Visit talkpython.fm/Cloudbolt to join the team.
Want to level up your Python?
If you're just getting started, try my Python jumpstart by building 10 apps or our brand
new 100 days of code in Python.
And if you're interested in more than one course, be sure to check out the Everything Bundle.
It's like a subscription that never expires.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct
rss feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I'll see you next time.
Thank you.