Here's a question. What's the most common way to explore data? Would you say Pandas and Matplotlib?
Maybe you went a little broader and more general and said Jupyter Notebooks. How about Excel or
Google Sheets or Numbers or some other spreadsheet app? Yeah, my bet is on Excel. And while it has
many drawbacks, it makes exploring tabular data very accessible to many people, most of whom
aren't even developers or data scientists. On this episode, we're talking about a tool called
Mido. This is an add-in for Jupyter Notebooks that injects an Excel-like interface right into the
notebook. You pass it data via a Pandas data frame or some other source, and then you can explore it
as if you're using Excel. The cool thing is, though, just below that, in another cell, it's writing the
Pandas code. You need to actually accomplish that outcome in code. I think this will make Pandas and
Python data exploration way more accessible to many more people. If you've been intimidated by
Pandas or know someone who has, this could be what you're looking for. This is Talk Python to Me,
episode 343, recorded November 8th, 2021. Welcome to Talk Python to Me, a weekly podcast
on Python. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy,
and keep up with the show and listen to past episodes at talkpython.fm. And follow the show
on Twitter via at Talk Python. We've started streaming most of our episodes live on YouTube.
Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows
and be part of that episode. This episode is brought to you by Shortcut and Linode,
and the transcripts are sponsored by Assembly AI. Aaron, Nate, and Jake, welcome to Talk Python to Me.
Hey. Hey. Hey, y'all. Thanks, man. How's it going?
It's going really well. I'm excited to be on the data science side of the world today with you guys.
Cool. So are we.
Yeah, you built some really neat tools to help people get started and get up to speed and just be
more efficient than just solely writing Python code, but not, you know, excluding that either with your
product Mito. So that's super fun, and we're going to talk about that. But before we do, you know,
let's just kind of go around and how do you get interested in data science and working on this
Python tool? Aaron, you want to go first?
Yeah. So a little background. Jake and I, you can't tell just by the first names,
but we are twin brothers. So we've been working on projects together for a long time. Nate has been
our best friend since middle school. I think I didn't get invited to his eighth grade birthday.
So I think maybe starting in high school. And then he also went to college with us.
And I think like, yeah, we got our first like taste of data science at Penn. We all studied a mix of
computer science and business. And in the business classes, you do a lot of Excel based,
mostly, unfortunately, Excel based data analytics work and stat classes and finance classes and stuff
like that. So I think that's really where we got our first taste of data analytics or data science
work. And then we've each had some experiences through internships and jobs that we've had over
the past few years in the data science space as well. But really, I think it all goes back to the
beginnings of the courseworks that we did at Penn.
That's great that you all are able to stay together. I mean, obviously brothers,
continue to work together. And on this project, you know, business, business schools,
a whole business program just run on Excel, don't they?
It's kind of amazing the contrast because so Aaron and I specifically both got degrees in computer
science and in the business school. And so it was there was this transition where you'd be hanging out
in, you know, in class in the engineering school, and you'd be writing code. And then you would like
walk up campus into the business school, and it'd be like returning to the dark ages in some ways,
right? Yeah. And what's really cool, I think about Excel generally is that what we observed is that it
let, you know, our peers in school and us as well kind of complete really some amazing projects that we
might have not been able to do with code because our skills, you know, we were still learning at that
point. And so it's really kind of this really beginner friendly, amazingly powerful tool for what it is.
But then we would go back down to the engineering school and be like, Oh my God, there's all this
tooling here. We could have superpowers, but we don't know how to use this stuff. Right? So there was this
very direct contrast that we witnessed where there's very cool stuff happening all over the
place. But the tooling differential is pretty dramatic between between business school and
computer science world. And I think that's kind of what initially made us interested in this space.
Sure. You don't want to underestimate the power of just firing up Excel, selecting a section,
throwing up a graph or two. And that's incredible. And all of the functions and stuff. But you know,
it's if you think of bad programming practices, one of the worst ones has got to be like,
do these three things and then go to over here and then do a couple of things, then go over there and
then get that thing and then go back. You know, we've banished this type of programming from regular
programming long ago. And Excel is like that without even being able to see where the go to's point.
You know, it's really, it's really not very predictable, right?
Yeah. It's really amazing. And it's really, it's crazy, I think, because once you start thinking about
spreadsheets, you quickly realize, I mean, there's a couple of crazy things about spreadsheets that
people don't really acknowledge. We, you know, talk about them internally because we're
spreadsheet nerds at this point. And, you know, we like, you know, hyping up spreadsheets and stuff,
but it's like, you know, the original killer applications of computers were spreadsheets,
right? Oh yeah. And more than that, it's spreadsheets are the most successful programming
environment in the world. Hundreds of millions of people can program in Excel. The next leading
programming language has 10, 20 million. You know what I mean? So it's really, there's an order of
magnitude difference in how, you know, well adopted these things are, but you're totally right.
The number of foot guns in Excel and the amount of like 50 megabyte insane models that we've seen
where people are like, it's 75 tabs and they're all linked to each other in some crazy circular way.
It's really slow. I don't know why.
Well, I know why.
Not sure what's happening, but yeah. And we probably have built a couple of those ourselves,
probably even worse ones. But yeah, so, you know, we've seen that and it's definitely,
those are some of the problems of spreadsheets that we kind of initially,
we were like, hmm, maybe these are some things that we can, you know, try to help solve.
Yeah, absolutely. And Aaron and Nate, as you would go from business school back to the computer science
side, I guess specifically to the business school, did your business peers look at you like, oh,
these are the guys that have the power to make the thing happen. They can help us build the thing
that we can't quite automate or can't quite pull off.
I think in some ways it was like trying to work in a group in an Excel spreadsheet is a miserable
experience. I hope you haven't had to do it, but it's like you upload, you have a Google drive
and then you end up uploading new versions to the Google drive. And then it's usually a Google drive
paired with a text message group chat. And it's like, I just finished this sheet. Why don't you go up
and, you know, do download it again or something? Yeah, exactly. Make sure you're not doing it the same
time I'm doing it. And I think that I don't think Nate and I maybe solved, solve those collaboration
problems while we're at Penn, but I think it was those experiences and our thinking we've had,
our programming as a superpower made us want to start doing this. I don't know if it was
always recognized by everybody else though.
I think the big thing was maybe we weren't the best at attending class. So that was, it was hard to be a
good group member in the first place, but one day, one day will be helpful.
Yeah. Group work was always hard for me as well. Jake, how about you? How'd you get into this whole
project here?
Well, mostly by plug line. I'm Aaron's twin brother. So sort of like a, some sort of covenant,
I think comes through that. But no, we, so we, we started working really in this like Excel
collaboration space. My background, I worked at a software company during college, sort of on the
project management side of some data science projects. So I had, you know, not quite the business
side of it yet, but at least a little one separate from like the coding product side of it.
Yeah. And yeah, we started working on these collaboration issues and we built a few other
products before it had some, you know, modicums of success there, but really took a step back
at one point. We're looking at like, what are the biggest problems with spreadsheets? It's the speed,
it's the inability to play in the large data sizes, and it's the lack of repeatability though.
It just allows you to do, you know, repeatable processes in an efficient way. And so the place we
found that does all this really well is some kind of sort of very elite is, is Icon. So if the idea was to
stick a spreadsheet interface on top of Python, and that was sort of like a sentence we had written
down really, okay, now we need to backtrack and realize like, what does that mean? How do we do
that? How do we deal with that? And, opened the few thousand cans of worms doing so.
Yeah, I'm sure. But it's a super neat idea. You know, there's a lot of things that you can automate
with Python, but with what you guys built, we'll get to it in a little bit, but what you all built
lets you interact in this spreadsheet way, and then it writes the Python code. It doesn't
just sort of allow you to make changes. And then you got to stay in your tool, right? You
use the tool to write code that otherwise might be a little bit of a stretch for you.
Yeah, I think, and I can talk more high level, they can talk about exactly why and how we did
that. But like a lot of just from the business side, like a lot of other tools will try and
extract Python away. So we'll give you, we'll allow you to do the types of workflows that you
would do in Python, but in a DUI individual environment, we're much more tethered to the Python
or try to be more tethered to the Python and the notebook. It's really important to us that you're
staying in your Python environment, and you're not at any, now you're not at a disadvantage at
all because you don't have the code. The code is right there. It's being generated real time.
And that's important for yourself. If you're learning Python, if you're trying to use the code,
or if it's a communication layer, you want that code because you want to send that to a developer
who's working in Python as well. So really important for us to be tethered there.
Right. Yeah. It probably brings, allows you to bring more people into the actual project than
before.
Yeah. You're much less siloed by being in the environment.
Yeah, absolutely.
That kind of like mentality of when you're building tools for beginners or people that don't know,
like maybe the professional software, make it really point and click and kind of like hide a lot
of the complexity. I think that's something that we've experienced with tools that we've used.
For example, like we use Stripe and Stripe creates a bunch of dashboards for you. But the problem is
we have no idea what those dashboards, like what, what is the nitty gritty details of how those numbers
are calculated. And so we have all these metrics and we have really poor understanding of like,
what is this actually telling us? And so I think something interesting that we definitely try to do
is we give you, you know, people that are maybe less familiar with writing the syntax yourself,
the ability to, as you said, point and click and use the spreadsheet environment and then generate the
code. And then if you ever have questions about, oh, like what did this, what is this pivot table
that I created? What does it really mean? Then you can look at the generated code and see exactly
what's going on. And I think that kind of like understanding where users need help and where
users want the as professional as possible, you know, nitty gritty details is a stratification that
we've thought about and I think have a somewhat unique approach to when it comes to these like no code,
low code tools.
Yeah, absolutely. So I kind of want to set the stage by talking about some of the different
things people are doing with notebooks because notebooks have really taken over in the data
science space for a good reason. I think, you know, we had IPython notebooks and we had Jupyter and we
had JupyterLab, which is doing a little bit more than just Jupyter and people really love them,
I think. And while JupyterLab is great, I think there's even, there's a bunch of creative things
going on, trying to extend it and use it in different ways. And I feel like Mito falls in there. So I
wanted to throw out a couple and just see if you all have heard of these and if so, get your thoughts
on them. One of them is this thing called JUT, J-U-T, J-U-T, maybe something like that. And what it allows you
to do is it allows you to actually view notebooks in the browser. So have you guys, or not the
browser, in the terminal, have you guys seen this?
I haven't seen it, but I do have a lot of sympathy for the unknown pronunciation of JUT or JUT because
we get Mito and Mito a lot. So my heart goes out to the JUT for sure.
Yeah, I'm sure, I'm sure I'm messing it up, but yeah. Yeah. So here's a way to say like, well,
these notebooks are so popular. Let's see if we can show them in the terminal. Like if I'm SSH'd into a
remote machine and it has, you know, IPYNB files and I just want to see them, how do I look at them?
And so you just say JUT, I'm going to go with JUT. You just say JUT and then the file name and then
boom, it shows it right there using Rich, I think. Super cool. Yeah, I think that this is, and I'm sure
this comment will expand out on as we see more of these, but I think it really demonstrates something
that we've really observed, which is that notebooks are not like, they're just like Excel in many ways.
They're not a tool that's used by one person for one specific thing, right? So we use a tool called
Mixpanel, for example. It helps us tracks and metrics, for example. It's a product analytics
tool. Yeah. I've used it before. Yeah. It gives you insane amount of analytics of like, where did
these people come from? How did they find my product and stuff like that, right? Exactly. And you can
kind of imagine though, it's really in some ways, it's just for product analytics, right? In some ways.
And the really interesting thing about a lot of our users is that we, you know, one thing as we're
trying to learn about our users and work with them to improve the tool, one thing we really realized is
there's people from all over the place and they're interested in notebooks for 472 different
reasons, right? And so some people using notebooks are people who've never written any Python code
before in their life. Some people have two weeks under of experience and some people are 75 year old
developers who only use the terminal. And if you show them anything else, they'll try and fight you,
right? So it's, you know, it's really, I think this demonstrates that there's really an appetite for a
wide range of kind of ways of consuming these things and presenting these things and editing these things
on these notebooks specifically. Yeah. That's a great, yeah. Yeah. Great analysis.
I totally agree. That's another cool thing about a tool like this or tools like this more generally
is that I think what they sort of have to do for a product perspective is condense down. What are the
really powerful things about notebooks? Cause they're taking a notebook and bringing it out of a notebook
environment. And I think kind of what we're trying to do with a spreadsheet is like, what are the values
of a spreadsheet? How can we bring a spreadsheet into other environments? So I think a tool like Jude is,
is just trying to do that. I think it's a, it's an interesting way to think about product. It's like,
sort of like, what is the essence? What is it? What are the essential parts of a notebook? What
are the essential parts of a spreadsheet? How do we translate those and bring that value to other
environments? So I always, I'm interested in tools like that. Yeah, for sure. This is an interesting
one. Another one is they just came out with JupyterLab desktop version. And I suspect that you guys
could even integrate with the JupyterLab desktop app, right? I hope so. Most likely. Yes. The answer is
probably yes. But, you know, I'll be honest with you, like candidly, if I spend one more minute
on installation problems, I might, you know, chop my arms off or something. I don't know,
but it's, it's really, I mean, I'm sure I don't have to preach to you. I'm sure you've heard this a
million times, but the, the Python installation ecosystem environment issues is a, it's a massive
blocker. I knew it was a massive blocker from like an individual level. Oh, I bet if you're trying to
reach people who are going through it, like I would just want to use Excel and like have a little bit of
code. Yeah. Conduit virtual environments, hip versions of Python. Yeah. You probably get a
couple of questions about that every now and then. Yeah. Yeah, exactly. I think that this is a really
cool example of, I guess, you know, the Jupyter devs realizing that distribution is one of the primary
problems here. And certainly with us, it's, it's like the primary bottleneck and users trying our product
is not, they can't figure out how to use it. It's that they can't even get the thing installed in the
first place. Yeah. You know, making that as easy as possible. I'm, you know, I'm sure there's still
work to do here, but, you know, really hats off to them and definitely something that we're
interested in and working with them in the future. If we don't already, there's probably some hack to do
it currently, but I have not, I've not done anything in earnest with Jupyter and lab desktop,
but I have installed it and run it. And I think it comes pre assembled with Python and Conda.
You don't have to have, it basically comes all set and it, then it just hosts Jupyter
lab locally inside of an electron app. So it might even be better for you guys.
Yeah. I'm not totally sure, but if you can make it work at all, I bet it's better.
I think what Nate is talking about is it's a trade-off that any tool or product building
as an extension to Python is going to face, especially if it's one that's trying to make
parts of data science more accessible to a newer audience. If you're building it in a
Jupyter environment, there's a lot of freebies you get. There's a lot of great nuggets,
valuable things to the user you get, but, you know, installation can be such a nightmare that you might
be casting away at certain part top of the funnel just by doing it.
One of the benefits, I guess, that you all will receive is people can go Google for help setting
up notebooks and getting the notebook started and all that. And you don't have to be part of that,
right? Like there's a whole ecosystem of, of people running notebooks, people writing articles
about using notebooks for beginners. And so you can just sort of level up on top of that and say,
once you go through all what they show you over here, here's how you go. Right. I mean,
obviously you want to help people succeed because if they can't get Jupyter going,
they can't use Mito. Yeah. I was creating documentation for getting set up with Mito and
I went to the JupyterLab documentation and first things like creating a new sheet,
I just grabbed the JupyterLab YouTube video and put it in our documentation free.
Well, that's one of the things I like about your documentation and your site is it's sprinkled
with screencasts, like little examples of how to do stuff or how to demonstrate stuff. And
I think more places should do that, right? There's so many places or so many projects that I just
don't understand though. It'll be a UI framework and there won't be a single picture of anything.
Like what it's about picture. The whole purpose is pictures. Give us a picture, at least it's like a
gallery or something. And similarly with how do you use things and just, yeah, you know, hats off to you
guys for putting those in there. Cause I think it makes a big difference.
Yeah. One of the reasons I think we've done that and had some, some good videos is just in terms of
just growing the tool, we've partnered with a lot of people in the YouTube data science community,
and they're all really good at making demos. I think we've learned a lot from them. People like
the data professor, Chris Nick, or two we work with a lot, but they've sort of, I think, I think at least
myself learned a lot from how to present a tool in a video. And that's obviously really valuable for,
for going to teach.
Yeah. I prefer to just fire up a two or three minute video and watch it instead of reading
through and see what I really got to pay attention to.
This portion of talk Python to me is brought to you by shortcut, formerly known as clubhouse.io.
Happy with your project management tool. Most tools are either too simple for a growing engineering team
to manage everything or way too complex for anyone to want to use them without constant prodding.
Shortcut is different though, because it's worse. No, wait, no, I mean, it's better.
Shortcut is project management built specifically for software teams. It's fast, intuitive, flexible,
powerful, and many other nice positive adjectives. Key features include team-based workflows.
Individual teams can use default workflows or customize them to match the way they work.
Org-wide goals and roadmaps. The work in these workflows is automatically tied into larger company
goals. It takes one click to move from a roadmap to a team's work to individual updates and back.
Type version control integration. Whether you use GitHub, GitLab, or Bitbucket,
Clubhouse ties directly into them so you can update progress from the command line.
Keyboard-friendly interface. The rest of Shortcut is just as friendly as their power bar,
allowing you to do virtually anything without touching your mouse. Throw that thing in the trash.
Iteration planning. Set weekly priorities and let Shortcut run the schedule for you with
accompanying burndown charts and other reporting. Give it a try over at talkpython.fm
slash shortcut. Again, that's talkpython.fm/shortcut. Choose shortcut because you shouldn't
have to project manage your project management.
JupyterLab Desktop is one of these sort of interesting things. Another one is JupyterLite,
which is Jupyter written in WebAssembly that runs just in the front end of the browser.
You guys check this thing out?
No, but this sounds amazing. I don't know if I saw the WebAssembly version, but I saw a Python
compiled to JS version of this, etc. But I mean, it would be very cool if everything could happen
in the browser because I feel like installation problems would evaporate, which is the goal.
It's the goal at the end of the day. So this is super interesting. We'll definitely check this out.
Yeah, this might also be relevant to you guys because it's basically Jupyter running and then they run
in-browser WebAssembly language kernels. So a limited set of CPython in WebAssembly in the browser and
probably also Julia and R and all those things.
Maybe partially non-technical audience like myself. What is the benefit here?
I'll take a shot. Yeah. So how I would describe this is that normally the way JupyterLab works is
that it's a client server model, right? So your client is effectively the thing that you as a user
interact with. You have a notebook that you see, you write code in the notebook, you press shift enter
to run a notebook cell. What actually happens when you press shift enter is that code gets sent to
the server. The server in this case is a Python kernel and the Python kernel actually runs that
code and then sends you back the results. So there's what you see on your web browser and then
there's a server running usually on your command line or something and you send code to it. It executes
the code and sends you the results. The benefit of this thing is that you can get rid of that backend.
So there's no more installation and instead you only need this kind of notebook that lives on the front
end, WebAssembly is a tool that allows us to run the code on the front end. And so instead of taking
code, sending it to the backend and getting it back, we can do that all in one location.
Right. And the challenges you've already been talking about are the challenge of setting up
that backend to get Python to run on your machine with the environments and the dependencies. And this
would theoretically in principle, at least avoid all that. It's just loaded up and it goes.
Yeah. It's super cool. No, we'll definitely, we'll definitely look into it because this is a,
I mean, this is an area of continual research for us is how to improve our installation process.
Yeah. Very cool. All right. What else have I pulled up here? A paper mill. So paper mill lets
you treat notebooks like almost like functions that you can execute a notebook and then get a value out
for all sorts of things. This is probably the least relevant to you guys, the things that I pulled up
here, because it's exactly about making it not interactive and, you know, kind of skipping that,
but just another interesting thing that people are doing to kind of add more or do more with these
notebooks. Yeah. It seems super cool. Now I was just going to say, I've seen people,
we've seen people who have notebooks where they like call functions in other notebooks and all of that
syntax, you know, the Panda syntax is confusing, but then like linking notebooks and all of that is
confusing as well. So I think it's all interesting. It's all interesting to see, you know, I think just
like spreadsheets have so many different use cases and Nate touched on this earlier. Notebooks have
an incredibly large number of different use cases with people from like top level data scientists to
people just getting started. So the tooling ecosystem that's being created and it already
existed really quite diverse and really powerful. Absolutely. All right. I think I've got one more
for you here. This might be the one that you heard of Nate called notebook JS. No, this one renders,
parses them and then renders it down to HTML, which is, I guess I don't have one to open, but that's
all right. So yeah, that's, that's pretty neat that it'll basically allow you to turn notebooks into
HTML. And I think just to kind of draw a categorization model over what we've kind of seen
before, I think there's, there's a couple areas of focus and these are things that we focus on
internally. And we, I think we'll deems that will come up throughout the rest of the conversation.
So there's things that we'll call, let's say presentation, right? And presentation is this
looking at the outputs here in the terminal or looking at the outputs easily on a webpage without
running a server, right? And presentation is something that we encounter. How do we present data?
how do we present conclusions from analysis, et cetera. Then there's another thing, which I think
this paper mill thing that we looked at last is super relevant on, which is this idea of repeatability,
right? As anyone who's worked in notebooks knows, there are really great areas for being a scratch pad.
It can get pretty confusing sometimes when you've got like all these cells flying around,
you're executing things out of order.
That's the thing, you know, we have sort of solved the, the Excel, this cell refers to that cell.
And then, but you have this human aspect of notebooks, right? I can go, I want to try this,
try this, try this, go back and change this and then run this and the stuff below it, you know,
you can run them in different orders and even change them and then have some of the invisible
changes stuck in another unrun cell. Yeah. That's yeah. That can be a problem.
I mean, Aaron and I spend all day in notebooks and the amount of times where I'll go up to Aaron and be
like, Aaron, dude, look at this, this code makes no sense. What is this bug? I don't understand it.
And Aaron's like, you fool. You've been one, three, four, not one, two, three. What were you thinking?
You know, like just the amount of times that even we as people use notebooks all day,
that is really, you know, it's dramatic. And so I think that, it's, you know, that area of
reproducibility and repeatability is something that we spend a lot of time thinking about mostly
because a lot of our users are really interested in it and it's problems that they struggle with.
And, it's something I think we'll definitely get into, you know, as we kind of, you know,
what we actually ended up building. Yeah. Awesome. So speaking, get into it. Let's,
let's talk about Mido. So I think Mido lives in this realm of these interesting ways to do more
with notebooks. And as we've already hinted at, you basically take an Excel user interface,
stick it into a notebook, and then allow people to interact with the data within the notebook
in Excel style way. Right. But what's your elevator pitch?
Oh, I can do elevator pitch. Elevator pitch. Essentially, it's a spreadsheet interface for Python.
So everything you do in the spreadsheet is going to generate the equivalent code for you in the code.
So below, if you're watching the video right now, you're seeing a little demo happen, but we have
features for exploratory data analysis. We have features for data wrangling, data munching. I've
heard lots of different words for that type of process. So yeah, there, and then graphing as well.
And also we have the ability to save and replay analysis sort of like a macro. And in terms of
users, you know, we have people who are newer to Python using it to sort of introduce themselves to
the Python world. They're learning as they go. But the nice thing is that they're not held back by the
syntax at any point. So you're not Googling syntax, not going to stack row of flow. You're in your
notebook the entire time, sort of staying in a state of flow. And then we also have more advanced Python
users and people intermediate in the middle using it just to get their analysis done more quickly.
It's really fast to do something usually, especially like the code for graphing or pivot tables or
merging. Take a lot of time to type it out and get the syntax right. So in our tool, you just do it
in an Excel interface that you're used to and it spits out the correct code for you.
Yeah, it's cool. People definitely need to see the little video. So if you haven't checked it out,
you know, obviously I'll put a couple of videos in the show notes, but yeah, that's the idea is you
come in and you just basically inside the notebook as part of the cell, you know, you might be familiar
with having an interactive widget for like some kind of graph, right? Where it's got like some
sliders and stuff. It's like that, but it's Excel ish. Right? Yeah, exactly. Was this your first idea or
where did this whole idea start? We've worked on a lot of different like spreadsheet related tools
over, I guess, a year and a half ago or two years, a long time ago at this point, we started building
like a GitHub for Excel. We're building essentially like, you know, difference detection,
allowing you to merge, going back to those original problems that we talked about that we experienced
at school. So we were building kind of this GitHub for Excel platform. We were primarily talking to
investment bankers and people in private equity and these like really Excel power users. We built the
tool. We've interacted with a lot of them, eventually realized that wasn't maybe the most
helpful space that we could be working in and ended up finding our way to this where it was more
Python based Jupyter notebooks and just making them more accessible. And as Jake said, you know,
helping people across the spectrum from beginners to more advanced data analysts, get their analysis
done faster. Sure. I think there's two angles here. One is just, you could be really fast, right?
You could say, okay, I want to do, I see the data. I want to sort by this. Now I want to drop these
three columns and then I want to, you know, compute a, do a computer column of sort of thing,
or maybe join two pieces of data, like two data frames and create like a larger one out of that
and so on. You could be really quick with that. But I think maybe even more important to that is
helping people who are just stepping into the data science side of the world, right? They've,
they've been working in some other tools or no tools whatsoever, really. And then they just,
they hear, oh, I should go do Python and I should do notebooks. And then they're confronted
with pandas, which pandas is great and it's not that hard to use, but there's a zillion things
you can do with pandas and it's not super discoverable. What it is you should choose.
And also the notebooks don't really help as much as they could. I think it presenting
the features of an API, right? Like if I'm working in PyCharm or VS Code and I say dot,
like, boom, there's a bunch of descriptions. And if I get the mouse near it, it'll give me like
examples and documentation. And here it just lets you type unless you hit, you know, dot tab,
and you explicitly asked for it. And a lot of people don't, I suspect if they're coming from
economics, they don't know that, oh, like tab has this magic to show me what I can do and stuff like
that. Right.
Those people making the transition to the notebook environment are doing so at a, along a spectrum
of willingness as well. Some people are doing it because they're really excited about, you know,
up-leveling their skills, learning Python, making their analysis faster. And then some people are doing
it because, you know, upper level management wants me to do that. And I think those people are,
really grateful for having a tool that will help them, you know, meet the workflow requirements that
they're working in, but not necessarily having to, you know, I'm sure you've read at some point the
Pandas documentation. And it's, it's one of those that it has lots of great examples in fairness,
but no pictures. Yeah. It's not the most intuitive.
Probably not a lot of tutorial videos in there.
Yeah. So, I would add a sort of a third category to what Aaron was saying, which is like, it's, it's people not
forced by management, but it's not sort of out of their own interest in Python. It's more so they're just sort of
forced to, based on what they're trying to accomplish. If they have a certain data set of a certain size, you simply can't do it
in Excel or Google sheets. So there, I need a medium that's going to allow me to facilitate data analysis on this scale.
And it'd be really frustrating if you're thrown into pandas to do that and you have to spend all your time.
So it's kind of doing simple things that you're so used to doing in Excel, like adding columns, doing pivot tables,
whatever it is, using formulas. And now you're spending a new set of your time going through syntax and making sure that you're
typing out and capitalizing the right letters in a, in a, in a white code. That can be really frustrating. So our tools really
trying to allow you to stay focused on the actual analysis the entire time and less about getting your code.
And many people are Excel literate. Right. And so it kind of leverage that a little bit.
Yeah. And then for, you know, people who aren't even coming from Excel, but are just data scientists, they're a word. It's not, not Excel
literate, but they're sort of like visual literate. That's a horrible phrase, but what it means essentially is that they
understand how to do this. They understand, okay, these are the things I need to do it. Here are the butts. Do it here.
The options I could use. So it's still a really valuable environment for everyone, Excel or Python.
Nice. So question from the audience out there's phone says, hello, what are the limitations of Mito?
That's a great question. I think the, the honest answer is that Mito is certainly a work in progress and
there's a large portion of Panda's functionality that we don't currently support. So kind of, I think a
helpful context here is how we're actually developing this tool. So Mito is very much, I would say a
collaboration with our users. And what that practically means is that join our discord,
there's a feature request channel and anyone has the ability to show up and essentially say, Hey,
this is missing from my workflow. And we'll say, Oh, that's on a roadmap or, Oh, that wasn't on a
roadmap. We'll add that or help us engage with your workflow and understand where that's coming from.
But essentially what's happening is over the past six to eight to nine months or so, we've been working
really heavily with our users to kind of build out the core pieces of Panda's functionality that most of
our users work with one by one and kind of investing their workflows back into the tool.
Yeah. Which way do you take it? Do you go and say, all right, these are the things people need
to do in Pandas. How do we surface that in our interface? Or do you say, these are the Excel type
things people are happy with? How do I make that happen? And it's like, which direction do you find
yourself going?
It's a really great question. And Aaron, you can definitely speak to this more because this is
kind of part of our process internally that's been evolving, but really kind of what we try and do
at this point is we try and, we try and work very heavily with our users to understand, you know,
at the highest of levels, what are they trying to accomplish? Right? Someone really isn't usually
trying to make a pivot table. Usually what they're trying to do is conclude, should I tell a salesperson
to do X or Y?
What state are we spending, getting the most sales in? And all I have is zip codes or something
like that.
Exactly. Exactly. Right. There's really a lot of people that we work with and this isn't the only
thing, but one of the things, for example, is they're operating the level of, I'm looking to
predict this feature or understand what affects this, this piece of my data. And so what we do is we kind
of work with them to understand their workflow. And then we use that to internally figure out,
you know, what features drive this, what does Pandas support and how can we provide an interface
on that that really lets people, you know, get this done as quickly as possible in a way that gives
them as much flexibility at the end of the day, if they need to take this from Mito and
and go run with the code somewhere else.
Yeah.
And Eric, go for it.
Eric, go for it.
I was going to say, I feel like in the beginning, kind of when they were saying, we were very focused
on let's put Excel into Python. And it was like more about the Excel functionality. But I think over
time we've come to think more about the questions more, what is the best visual interface for Python,
for data science? And it's less so. And so some of those things are from Excel. Some of those things are
our own creations. It's still a spreadsheet interface, but the goal is certainly not to take
Microsoft Excel and give you all of that Python.
Interesting. Yeah. Cool. All right. I think maybe a good way to understand this, how this workflow
works and get a peek inside of the features is over here somewhere, I think in the documentation side of
things you've got, yeah, right at the beginning here, quick tutorial here, you've got this sort of,
I don't know, I'm not bouncing around, I'm not finding it quite where it was to tell you, but
there's this example where you go and load up a couple of CSV files and then like join them to
create a pivot table and stuff on that, like that. Maybe give us a talk through what working with that
data flow kind of looks like. You know, I talked about this idea of like turning zip codes into states
and that kind of stuff. What does work with Mito feel like? Give us a sense.
I can talk a little about that. So I think working with Mito feels hopefully very fast and very
intuitive, or I think two of the things and maybe robust or maybe like the three kind of things we
strive for. I think in terms of workflows that we see people doing, a lot of people kind of fit this
like feature creator and automator use case that we think about. So it's people that have some business
sort of question like where are all my sales coming from when I just have these zip codes.
And they're probably trying to do that in Python because they've been doing it in Excel over and
over again each month and they're looking for a more robust and more automatic way of doing that.
So I think a lot of, a lot of how, a lot of the ways that we see these workflows play out is people
either have a CSV file or they have like a snowflake connection to get some data. And then they start by
doing some simple EDA and trying to get a better sense of what their data actually looks like.
Right. Without Mito, it might be something like df.head or df.sample and just like kind of get a
visual look of a grid of data, right?
Exactly. Or even some people are, they want the more visual interface. So they're downloading it to
Excel initially and doing some manipulations or something like that there. But then that workflow is
very separate from their more automatic script that they're creating. And then they have to kind of
reconcile any changes that they've made back into the Python workflow. But yeah, so Mito has a bunch of
features. There's some here, you know, if you scroll down in the documentation on this left-hand side,
you might see some stuff that might be helpful. So there's things like summary stats right there,
which will show you like a graph, a distribution of the data in each column, a lot of those dot
describe functions and has a really intuitive filtering. So things like filter by value. So you
can see all the, all the unique values in your column, and then you can toggle them in and out of your data
set, or you can add more, you know, customized filters, whichever you would like. But then once you
move past this kind of like initial data cleaning, some people do write spreadsheet formulas. So we
have a bunch of Excel's most popular formulas, things like date manipulation and date parsing.
Here's a bunch of, here's a few of them. Once you kind of get a sense of what your data looks like,
you can do some of these transformations. And then ultimately you'll end up with this script that you can
use to run over and over again and never have to go back into the Excel world to and fight through the
manual process again. This portion of Talk Python to me is sponsored by Linode. Cut your cloud bills in
half with Linode's Linux virtual machines. Develop, deploy, and scale your modern applications faster
and easier. Whether you're developing a personal project or managing larger workloads, you deserve
simple, affordable, and accessible cloud computing solutions. Get started on Linode today with $100 in
free credit for listeners of Talk Python. You can find all the details over at talkpython.fm,
slash Linode. Linode has data centers around the world with the same simple and consistent pricing,
regardless of location. Choose the data center that's nearest to you. You also receive 24/7/365
human support with no tiers or handoffs, regardless of your plan size. Imagine that real human support
for everyone. You can choose shared or dedicated compute instances, or you can use your $100 in credit on
S3 compatible object storage, managed Kubernetes clusters, and more. If it runs on Linux, it runs
on Linode. Visit talkpython.fm and click the create free account button to get started. You can also
find the link right in your podcast player show notes. Thank you to Linode for supporting Talk Python.
"The DataFrame" to get data into this spreadsheet-like front-end that is Mido.
You basically just have to have a data frame. If you have a data frame, there's a ton of flexibility for
doing that. You could load a file, you could get it from the internet, you could do even read HTML off of
a URL and then go grab a table and then there's your data frame, work with that. There are simplifications,
things like when you're in Mido, you can hit file load equivalent and browse to the files and then it'll
write the pandas code, say like pd.readcsv, whatever you selected, right?
Yeah, exactly. One of the things I think would be really interesting to talk with you about is
generally in your survey of the Python ecosystem, it's interesting because Python is code, right?
And so in some ways we're on this boundary, this flexible border between pure code and low-code,
no-code tool. And there's been a hundred million low-code, no-code tools that have existed over the
past 25, 30 years, whatever. And some of them are around and some of them aren't. And really the
question for us that we kind of ask ourselves is, what unique do we bring to the table here? And as a
local tool, how do we differentiate ourselves? And it's that exact idea that you can also just pass a
data frame. We're not necessarily interested in, we don't want to stop you from writing Python code,
we want to enable you to write Python code as easily as you possibly can. And really that's
how we see ourselves, you know, manning that spectrum.
I think it's a really good place to be because my hesitation with all these low-code, no-code tools
is they usually, one, lock you into their thing, which is often a SaaS thing, right? So you're locked
into having your data there and continuing to subscribe to a thing, which don't get me started
on subscribing to so many things. I was suggested that I subscribe to my internet speed checker,
app, not pay for it.
You need to know, but like...
Once an hour, you need to check, right?
Yeah, yearly, I can subscribe to it. If I pay yearly for my speed checker, like,
what are you doing? Too many things that are subscribed to. Anyway, not that subscriptions
don't make sense for a lot of tools. But what I like about this is, like you said, you bring the
data from wherever and you could do this out of a Postgres database and you build up a data frame,
and then you could throw into this visual place that speeds you up. And then what comes out the
other end are more data frames, right? So you could even do multiple, I mean, maybe tell me this is
true. It seems like you could. You could do some regular Python code, some Mito that generates a
really interesting transformation, some more Python code, and then maybe another Mito block that takes
another bit of output and then brings in more data, does other stuff, right? You can kind of mix and match
throughout these, right? Throughout the notebook? Yeah, exactly. And actually, the other day, we
went into GitHub and we searched Mito sheet to see how people are using it. And you see people using
it in that exact way where they're importing data, they then generate some code using Mito. And then
they'll, you know, one of the things, admittedly, that the tool is maybe not the best that right now
is graphing. We have, you know, we support creating basic graphs, but not changing the colors,
changing the titles, all of that. What UI frameworks do you support? Map, Plotlib?
We use Plotlib. We use Plotlib. Okay. Yeah. We generate Plotlib code,
which has great documentation. It's really, you know, you can go in there, we give you a link,
and you can go in and make your edits. But this is one of those places where generating Python code,
and you know, in this case, Plotlib code is super helpful for everybody involved. For us,
it's helpful because, you know, we didn't have to recreate the Plotlib library, which is massive.
But for everybody else, it's also helpful because, you know, because we didn't generate the Plotlib,
we didn't recreate the Plotlib library. You are able to use, you know, the entire Plotlib ecosystem,
and you're not locked in at all. So for something like Alteryx, you know, where you can use their
graphing features, and if, you know, they don't have the graphs that you want to create, then you're kind
of out of luck because you're locked in, and there's not really an easy customizability path.
But you own your Python code that Mito generates. So it's up to you to do whatever you want.
Yeah, I suspect you guys don't recommend this. But you technically could delete out,
in a lot of cases, the Mito bits after you generate your Python code and keep going like,
okay, this was really helpful, but we actually don't need this anymore. There are cases where some of the
Excel-like functions really come from you guys, right? But a lot of it is what it's writing is pandas,
and NumPy and Plotly code, right? Just speaking to the lock-in or not the lock-in side of the story,
right? Yeah, I think we definitely support that. In Mito, actually, we have a button clear.
Data analysis is all about, it's very iterative, building your understanding of what is useful and
where you want to go. So one of the, actually, the most kindly done things in the tool is actually
clearing all of the edits that you've made to your analysis. And getting rid of Mito now that you have
that understanding and wanting to take it in another direction, I think we're huge supporters,
well, we're big supporters in our own development process of cleaning your workspace. Nate is proud
of how clean our code is. And, you know, if Mito isn't helpful for you right now, definitely get rid
of it so you have an easy notebook to clean up and debug. Yeah, I think that's a big contrast going to
some other no-code SaaS system. There is no, I don't want to use this exactly anymore, just let me carry
on with my analysis. There's none of that in most of these tools. And with yours, I think it's there to
support you and I can see it being incredibly valuable. But at the same time, it's not the
essence of what you're doing. It's the UI on top of Python, as Jake likes to say. Yeah, I was just going to
say to the point of deleting out the Mito sheets, we have a society of secret Mito users who are trying to
convince their bosses that they're really good at Python to get pay raises. So we support that workflow very,
very well. Exactly. So one of the challenges that you can have when you have a machine write code
is that it writes bad code that's hard to understand. And Nate, it sounds like you might have had some
input on this, that when you interact with Mito, you do certain operations. Like I do a filter,
I create a pivot table, or I filter out certain things. It'll actually write step one, you did this,
step two, you did that. And it writes what looks like pretty well formatted Python code with little
bits of documentation, like pivot the table, reset the column name and indexes and stuff like it'll
even comment your code that it writes. You want to speak to that a little?
Yeah, absolutely. I mean, this is a really interesting area that we've put a bunch of
research time in, if you can call it that. So there's a couple things that I want to,
I think, highlight here. And I'd love to also hear your thoughts on as well. So you're totally right.
Machines writing code, it's all the rage these days, in some ways, with these fancy,
you know, machine learning systems where you can, you know, write a little prompt and everything will get
written for you. Mito takes a little bit of a different approach where exactly as you say,
when you, for example, add a filter, it'll generate the line of code that corresponds to that filter.
Of course, immediately, the question becomes, this is not exactly how I want the code to be.
I didn't mean to do that. I meant to do something else. And so really, the way we think about it is
giving the user that code in the cleanest way that we possibly can. So just as an example of this,
if the user adds a column, and then immediately after renames that column, a feature that we're actually
releasing this week is that is going to get collapsed into just the adding of the column with the new name.
Right? Nice. Yeah.
That's ultimately what the user was intending to do. There's other more fancy things that you can do.
You can start getting into kind of code optimization, where it's like, you made a pivot table, then you
overwrote the pivot table. And those are all things that we're definitely interested in, and kind of are
on the roadmap for improving. But generally, you're totally right. If we want users to be able to learn
from this code, to be able to use this code, we need to generate clean semantic Python that really
works in the wild and is actually editable by the users. Otherwise, it's just a blob,
a mass that you can actually interact with.
Yeah, well, you get it, but at least from the examples I've seen, I would be happy to take that
and then start writing directly on that. Even though you don't see it on the screen,
it's like right above, I scrolled it right a little bit off there. But it says, don't edit this section.
This is Mido code. I guess that's if you still want to be able to use Mido on it.
Right. You know, exactly. Don't mess it up, right?
We're actually, that's a feature we're working on this week is improving that communication. You can
take Mido code and edit it and change it however you want. The only problem is Mido might have
trouble reinterpreting that if you try and then later replay an analysis. But that's also a work
in progress. And ideally, you know, the kind of perfect version of what we, you know, envision
long term, and maybe we'll get there. Maybe this is impossible. But one really cool thing that
we're kind of thinking about is edit a spreadsheet to generate Python.
Yeah, I was thinking if this could be a bi directional, that'd be fantastic.
Yeah, really cool. So really, it's this world where you're you're really fluidly writing code
and editing a spreadsheet. And if something's easier in code, go write the code. If something's easier
in the spreadsheet, go write the spreadsheet. And that's definitely a vision that we've seen
ourselves kind of realizing over time currently. And something that, as you mentioned, you can go
Mido Python, Mido Python and do that currently. But making that easier for our users to really use the
code in a dynamic and real way, as you would any other Python code that you write is definitely
something that we're actively investing in right now and really trying to improve.
Nice. Yeah, I like the idea of being able to have it do some optimizations rather than create a
variable and then overwrite the variable with some other thing and just do it all at once. I'm sure
there's a ton of stuff in pandas that could be done better rather than like a really naive,
straightforward, like multi step stage.
Yeah. And yet the other thing I will say is the one benefit actually to generating this code is that
and not to insult anyone because I'm the biggest offender of this, but most data science scripts you
see in the wild are not the like pinnacle of clean, well-kept code. It's usually out of order notebooks
where, you know, because it's such a dynamic process, it's just very hard in practice to keep
these things well organized. And so actually what we can do in practice is generate some documentation
for what's happening and help users save and manage these scripts in a more linear and organized way,
et cetera, and help users kind of adopt these best practices. And that's sort of the stuff that we've,
you know, we've been exploring recently and Aaron was actually working on some of this today,
improving some of this code generation stuff, but it's not the highest of bars to meet. And we
definitely think that we can, you know, continue to improve and surpass that and make sure that the
code we're generating is really great, great stuff and stuff you'd be happy to edit.
One thing that while I'm looking at this, I'll just throw this out of here as a piece of feedback with very
little actual experience. So take it with a grain of salt is you've got like step one,
step two, step three. It'd be cool if those were actually separate cells.
So like at the end of say step three, I could do like a pivot table dot head or something just to
like sort of touch on it and explore it a little bit along there.
Yeah, no, definitely. And a similar thing that we've also certainly thought of that that
multi-cell approach would enable is, you know, changing the order of steps, switching things
around and saying, oh, I actually want to filter first and then pivot versus pivoting and then
filtering. Yeah, certainly on the roadmap, but definitely, definitely something we
want to do, giving users more options on how they actually export this code and what they
do with it at the end.
Yeah, super neat. I've got a question out here that kind of leads into where I was going to
go with this anyway, and I'm going to switch the order so I don't cover your head, Jake.
Spawn also asked, does Mito support switching or you switching from or to or using Dask?
So, gotcha.
Yeah, not currently. No. So the one really cool thing is because of the Mito, the way Mito works
internally, which is something we can definitely get into depending on, you know, if you're,
you think your audience would be interested in it, what the appetite is. But we really have the
ability to kind of switch out, let's say, what the backends is of Mito, the kind of what code
do we actually end up generating is something that can we can leave up in the air. And really,
our interface can be a more general thing than just Python code or just pandas in Python code.
Yeah, especially for these frameworks that are near compatible, API compatible with pandas,
like Dask.
Like I think Dask is almost exactly, and many, for most of the basic operations,
it's, they've aligned on the pandas UI just, or the pandas, the pandas API just for, you know,
how to kind of handle things. So it's definitely something that in the future we could, you know,
we could support. And if we have users who are working with huge data sets and need Dask,
that it's definitely something we'd be interested in learning about and exploring.
You know, Dask, when I first think of Dask, I think large clusters, massively scaled out data.
But then at the same time, right over here, I have my MacBook Pro Max, which has, you know,
10 cores on it. And when I run Python code, I get 10% of that CPU, right? And Dask will allow you to
scale across your CPUs, even on your local machine, right, or scale larger than your memory and stuff.
And so I feel like it's Dask, even for just making your local work go better, is actually
probably under realized or under utilized. That's super cool. It's interesting that the range of
data set sizes that we actually see in practice, it's very, I would say, at least from my observations,
and Aaron, Jake, feel free to hop on this, but it's like very bimodal and that there's a lot of people
hanging out with 100,000 rows. And then there's some people are like, hey, I have, you know,
a hundred million records I'm looking to analyze. And we're like, well, you know,
good luck on your 2000 fall MacBook. You know what I got? It's going to take a bit.
Get a coffee.
Chill out, go to sleep, wake up tomorrow, hope it hasn't crashed, I think is the general strategy.
Yeah. So that does lead me towards my final two little areas I want to speak about before
we run out of time here. Now, on one hand, this is just writing Python code. So its performance and
its limitations are what effectively pandas can deal with. On the other, it is showing that stuff and
allowing you to sort it visually. So there might be some constraints on like amount of data you can
work with. What's the data size story?
It's a great question. We have a release coming out within the next two weeks. Generally,
but here's our, here's our motto. We obviously we're providing a visual interface. There's going
to be a little bit of overhead, but the way we like to think about what we do is that it's a tiny
little bit of flat overhead, no matter how big your data is.
Okay. For example, you won't try to show the entire hundred million rows in a grid or like
you'll do some sort of like virtual lazy load list or something like that.
Exactly. A lazy load list. And we actually have a lazy load of the entire data set. That's a
feature coming out within the next two weeks or so. It's all written. We just got to test it a bit
better in the wild. But yeah, effectively, it's, we're a very thin wrapper on top of pandas
functionality. And in practice, what that means, what that means is anything you can do in pandas,
you should be able to do in Mito from a data set size perspective. It's something that was
very important to us, especially because a lot of our users are in Python because of data set
size limitations in the first place. As I think Jake mentioned. Yeah. Yeah. Yeah. That's a good
point. This is something that we'd spent a lot of time. We were previously using AG grid as our
actual display unit. And it just wasn't made in combination of probably us not implementing it 100%
how they might implement it. And you know, them not optimizing for these ginormous data sense,
data sets. Nate in particular just spent a huge amount of time recreating the entire grid from
scratch. So we could have complete customizability over it and show as much as much data as possible.
Nice. I mean, sometimes you got to do that, right? You're like this, this control is amazing,
but we've outgrown it and bite the bullet and just do it right. Exactly. Yeah. Now,
interesting question from Samir out there. Hey Samir, can I use Mito and VS Code and I'll sort of
expand to that just a little bit. Can I use it in, in some of these other tools that are not exactly
notebooks. So we've got VS Code has its kind of own way of presenting and showing notebooks. We've got
PyCharm and we've got data spell, which is JetBrains new data science IDE thing. What's the story with
these environments? So unfortunately right now, Mito only works in future lab, but this question is
something that comes up all the time. We, I think the places we hear the most interest are VS Code and
Google collab. And we're definitely excited and really going to support those environments as well. And I
think we've, we've done a lot of work internally on how we design, how we design Mito to make it
extendable. It's now, you know, we have a lot of these like functionality that we're trying to pack
into the tool and then handling these new environments as, you know, a decent amount of development work as
well. So it's all a prioritization game at this point. Yeah, of course. Yeah. So short answer is we
don't support them now, but in the future we definitely will. Cool. All right. Two other areas quickly I want to
touch on. One, tell us a little bit about how this is implemented internally. I think it, I don't know
how it's implemented, but I'm, I'm guessing that it's somewhat like a lot of Jupyter stuff. Like I
want to do Jupyter things for Python, but I got to write them in JavaScript. Is that the story here as
well? You know, actually your earlier comment of, I imagine it's just like a slidey widget that you can
use in a graph in JupyterLab was spot on. The IPy interactive widget or whatever it's called?
We are actually just a very fancy IPy interactive widget. Okay. And practice how that actually works
for your audience. If they're interested is there's kind of two pieces to your code base. There's a
JavaScript front end and there's a Python backend. The JavaScript front end is the sheet that you see.
It's the buttons that you click. And what that actually does is it just, it's a very thin wrapper
that just then sends a message to this Mito sheet Python package in the back and says, Hey, I just
clicked this add column button. You should add a column, excuse me, to this data frame. And then that
Python processes that message and then responds to the front end and says, okay, great. Display the new data
frame and also write this code to the cell below. And that's kind of the high level of what happens
there. It gets, you know, as you can say, more complex than the nitty gritty. We're in practice.
We're a react code base. We use TypeScript because we like strong typing and we kind of hate Python's
weak typing. But do you use type annotations on your Python side? We are gradually adding them to our
code base. We don't currently type check. Actually, we mostly use them as like IDE support to make
things easier. That's the main thing I use them for as well, because a lot of times the
IDEs will show you the errors if you make them anyway, right?
Yeah. Yeah. No, you definitely, you get some support, but pandas, I'll say pandas typing
support. Obviously pandas is the main library we interact with. It's not perfect in all cases. It's
a very complex library, so for sure. But effectively in those cases, things kind of break down and the
errors that you get are maybe sometimes false positives, sometimes false negatives, and you can
shoot yourself in the foot sometimes. Yeah. Interesting. Okay. So it's a blend. It's
it's the JavaScript, React, TypeScript, front end, and then Python back end. Yep. Yeah, exactly.
That question of that, let's say that stratification in that architecture is that's exactly what's going
to have to evolve as we kind of move into other places like Google Colab, VS Code, etc. They all
have slightly different extension architectures. And so architecting our code bases, so these things are
separatable and reconfigurable and in the ways that other data science environments expect is something
that we've kind of been trying to do. But you know, a plea to everyone who's developing data
science IDEs, settle on one extension environment, please. I know it's never going to happen, but it'd
be really nice for us extension developers. We'd love it for sure. Right. Or making it an adapter.
Yeah. Right. If somebody created things that if you have a Jupyter UI and you want to put in
Google Colab, you just insert this thing and talk to it and then magic happens. Yeah. Shims when they work
are great. So if someone's done that, let us know. Please reach out and we'd really appreciate it.
Yeah. Yeah. Super cool. All right. Another thing I do want to make sure that we touch on a little bit
is up here at the top, I see plans. And so this is not for every possible use case of free tool,
right? You have a free version and you have a higher order paid version for teams, if I'm
understanding that correctly. Right. So we have, my tool is a free 3D tool. We have free users,
90 whatever percent of our users are for free to an end user. Please download it. You know,
you can, you can pivot solve and it's free. We work with some larger organizations in sort of more
of a scope manner doing cost of development, cost integration. And those are the payment happening
there. Sometimes it's some of those larger enterprises. What we're building out now is
sort of that middle piece where we, you know, we want to have sort of a plan for teams, maybe with better
security, some free development hours, things like that. And that business model is evolving, but we'll
probably be, what we want is a SaaS model there where you're paying, you know, 10 bucks a month
or something. It does seem like some kind of online system. I mean, you go to Jupyter through a browser
anyway, some kind of system that's like really already configured because you're helping people come into
Python who probably don't totally want to pip install and manage their path and activate virtual
environments and all those kinds of things. Yeah. Yeah. It's kind of like really helped them there.
Right? Totally. And that's one of the things we do when we work with some of these larger enterprises is
help them with the setup of their Drupal Hub environment, help them get the package they
need, help them get my to install, obviously. But I was going to say to your viewers, you know,
we're definitely looking to partner with more organizations or teams. If anyone wants to reach
out and make my email or something, we'll be in a link somewhere. Yeah. Yeah. We'll put your
contact information in the show notes. Sounds good. But yeah, we're definitely, you know,
we're looking to work with teams right now as we have a really good, really strong user base.
Yeah. Cool. Cool. So I'm glad that you guys have some kind of business model because a lot of
these things, they come and then people kind of lose interest and then they go. And there's a
real big difference of this is my job and my investment. So I'm really going to work on it
versus this is a thing I'm kind of excited about for a few months. So it's cool. You got a free plan
for people to use that. That's awesome. It's also cool that there's a path to support to just make it
better. Yeah. Yeah. We're here for the long haul and works out. The other thing I'll add here is it's,
and I know I mean this on a knock on the like hundreds of amazing data tools that are out there, but
there's, I think like this level of polish that we really feel a desire to reach with our tool.
It's like that, that kind of when you're delivering a tool to a paying customer, there's often a
different expectation that comes from the paying customer. And we do our very best to hold ourselves
to the highest standard possible. But when someone who's paying you reaches out and says, Hey, this
button doesn't look the way it should. That extra level of polish really kind of kicks the tool over
the edge. And all of those, all of that feature development ends up getting, you know,
pushbacks to the individual users. And so really the free users I'll say, and you know, most of the
people who use our tool. And so really, you know, we're trying to build the best tool that we can here
and making sure that we can do that sustainably long-term and really invest in, you know, what
we're doing and build a team around it is something that, you know, really is necessary if we're going to
deliver on what we think we can and the promise. Yeah. Cool. Another nice thing about the paying users
is just that we get to, we need to work in a much more close relationship with them. So there's a lot,
that's where we get to zone in on like specific use cases around financial services or around
bio research. So we've come to work on specific features and specific workloads that we definitely
wouldn't otherwise that I think made inspire going to really benefit everyone uses the tool.
Yeah. Yeah. Very cool. I'm fascinated with the different ways people are working and operating in
open source space or building on top of open source tools to create businesses. You know,
we've got the Anacondas, we've got the MongoDBs and stuff out there. So yeah, good,
good luck to you guys. I'd like to see you succeed here.
Yeah. Thanks so much. And then the last thing I'll say is, you know, we do our best to give back to
open source tools as well, especially the ones that we work on. So you'll see me sometimes being
annoying opening issues on GitHub. And I think that's another big piece of this is as we build
on open source tools, making sure we contribute back to them in ways that are meaningful and actually
helpful is, you know, certainly really important as well. And definitely something to say something.
Yeah. Fantastic.
All right. Well, I think that's about it for the time that we have to talk about extending notebooks and
Mido and all this really cool stuff that you all built. Before I let you out of here,
there's the final two questions, however many in what order and whatnot, you want to take this,
just jump on in notable PI PI package out there, something you've come across like,
oh, this library is awesome. It doesn't get enough attention. Anything come to mind?
Mido and Mido: Yes. Honestly, something that a lot of our users use and Jake,
actually feel free to hop in after me, but I would say a pandas profiling. It's a tool that does somewhat
similar stuff to us, but it's a super great tool for many of our users. And I don't think a ton of
people know about it. And it works right in JupyterLab as well. So it's called Lux, which is cool. I don't
even know if it's actually still being supported and developed at all, but I really love it.
LogTrestles. You look up Lux Python. It's cool. It does like automatic graph suggestions. So you
can pass in a data frame and it'll suggest sort of give you options of visualization to just click
on and use, which I think is a really quick thing. It's not the most like, why fully fledged package,
but for what it does, I think it's really good.
Mido and Mido: Now these little things that people don't know about, they're like,
oh, that's cool. I'm going to go check this out. It might help.
Mido and Mido: Yeah.
Mido and Mido: A Python API for intelligent visual discovery.
Mido and Mido: I may not could be exact more, but we're really close and we love the DeepNote
product. It's like, it's pushing notebooks forward. So trying to add collaboration,
like collaboration, like Google sheets and Google docs, it has a lot more, potentially a more friendly
interface than sending out the rest Jupyter notebooks that are a little bit bare gains at times.
Mido and Mido: Okay.
Mido and Mido: There's also another notebook.
Mido and Mido: Yeah, really cool. I've spoken to the DeepNote people just a little bit and
they're doing cool stuff for sure. Mido and Mido: Yeah, he's cool. Obviously,
there's another notebook called Hex. We had to talk to their founders recently
about some potential collaboration, but they're doing cool stuff as well.
Mido and Mido: Okay, fantastic. All right. And then if you're going to write some Python code,
on notebooks, obviously, if anything else, what editor do you use?
Mido and Mido: We're big VS Code users.
Mido and Mido: Okay. All three of you guys?
Mido and Mido: Jake when he dabbles. Mido and Mido: Yeah.
Mido and Mido: Yeah. Mido and Mido: Yeah.
Mido and Mido: Yeah. No, VS Code.
Mido and Mido: I'm in notebooks.
Mido and Mido: I was thinking about this yesterday recently. It's like,
I wish at school someone taught a class called actually doing software development in the real
world because I feel like I lived my whole school life writing Java code in Eclipse and
Mido and Mido and Mido and Mido and Mido and Mido and Mido and Mido and Mido and Mido and Mido. And I'm in the real world. And then I started using VS Code and it was like this transcendent moment of bliss where I was like, oh, programming can actually be fun. And it turns out the tools I was using just made my computer heat up to a thousand degrees and burn my lap.
Mido and Mido: Yeah, I really agree with that statement that that there's a bit of a mismatch of what is taught in sort of computer science and then what is expected of people when they get out in the real world. And it might not be as academically highly valued, but really good, like working with tools like VS Code and and PyCharm and these other tools that help you write code better quicker. And some of the software engineering science, I think that's really could be valuable.
Mido and Mido: No, totally. And also the other thing I'll say is for developers like us, maybe who came out of school and moved into a startup and, and, you know, didn't have a ton of experience, let's say writing Python in production in the wild. The other thing I would highly recommend is continuous integration. You can set it up through GitHub or GitLab. I'm sure if you use that as well, testing your code automatically on a server, huge productivity gains for us and really is increased our confidence that we're able to deliver like the best possible product and not something ever anyone ever told us about, you know, when we're in school. So.
Mido and Mido: No, go implement a database in Lisp. Okay. All right. And it's John out in the audience has a quick funny comment about the editors, Neo Vim, of course, that's starting to get some attention lately as well. Very cool.
Mido and Mido: I'm scared of anything with the word Vim and it scares me, but I'm sure you're a superhuman for using it. But,
Mido and Mido: You know how you generate a proper random number or character set is you get a first year computer science student into Vim and then you ask them to quit.
Mido and Mido: Yeah. I mean, you've seen the most, the most like Stack Overflow questions like I'm trapped in Vim. How the hell do I get out of here?
Mido and Mido: Yes, exactly. Mido: It's so funny. It's like half of what Stack Overflow does is answering just that question specifically.
Mido and Mido: Exactly. For sure. That's great.
Mido and Mido: All right. So final call to action. You guys, people are excited about this. How do they get started? Where do they go from here?
Mido and Mido: Dot Trimido.io, which is our documentation website. And if you're an organization enterprise,
just reach out to me at my email, cheek@sagacollab.com, which will have all links here. Or here, the planes page is a link there. But yeah, the documentation,
Mido and Mido: Yeah. Right on. Also, I'll throw out there while you're on the docs, watch the videos. That's a quick and easy way to really see what it's about. And before we wrap it up here, Mr. Hypermagnetic has a little comment like that Vim is the eighth. It's like deadly sins.
Mido and Mido: It's weird though, because it's not just the eighth of the deadly sins. It's the eighth of the deadly sins that also like 10% of the population swears is the greatest thing since sliced bread. And so it's like half the population, like me, I'm terrified of the damn thing. But my father is like, son, like my dad's dad from the eighties is like, son, have you heard of this? I'm like that, please. I can't take this right now.
Mido and Mido: Yeah, it's amazing. Yeah. My co-host in Python bites, Brian, he's all about Vim. Everything's Vim. It's great. But I did some Emacs and then I kind of did some other more UI oriented things. Awesome. All right. Well,
Jake, Aaron, Nate, it's been fun to have you here. Congratulations on this project. I think it's going to help a lot of people get into Python and data science quicker, more easy.
Mido and Mido: Awesome. Yeah. Thanks for having us. Great chatting.
Mido and Mido: Talk soon. Bye bye. Mido and Mido: Yep. Bye.
Mido and Mido: This has been another episode of Talk Python to Me. Thank you to our sponsors. Be sure to check out what they're offering. It really helps support the show. Choose Shortcut, formerly Clubhouse.io, for tracking all of your projects work, because you shouldn't have to project manage your project management.
Visit talkpython.fm/shortcut. Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines. Develop, deploy and scale your modern applications faster and easier. Visit talkpython.fm/linode and click the create free account button to get started.
Do you need a great automatic speech to text API? Get human level accuracy in just a few lines of code. Visit talkpython.fm/assemblyai.
Want to level up your Python? We have one of the largest catalogs of Python video courses over at Talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription in sight. Check it out for yourself at training.talkpython.fm. Be sure to subscribe to the show. Open your favorite podcast app. And start with your podcast.
a subscription in sight, check it out for yourself at training.talkpython.fm. Be sure to subscribe to
the show, open your favorite podcast app, and search for Python. We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm. We're live streaming most of our
recordings these days. If you want to be part of the show and have your comments featured on the air,
be sure to subscribe to our YouTube channel at talkpython.fm/youtube. This is your host,
Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write
some Python code.
Bye.
you
Thank you.