What do you do when you're a high caliber mathematician or a scientist and you want to
share your algorithms and code? Sounds like a job for GitHub, doesn't it? But the problem is often
this work is done on proprietary platforms such as Magma, MATLAB, Mathematica, and others.
Not only can you not share your licenses for, say, MATLAB, but there are often proprietary
separate libraries and tools for specialized work. These are expensive products. One example
from my distant past was using the wavelet toolbox on MATLAB. MATLAB itself is over $2,000
and the wavelet library is another $1,000. So to share my code, you must have both licenses
yourself. This is a problem. Well, if you're William Stein, you take this problem and you turn
it into an opportunity to build an open source competitor to MATLAB and related platforms.
This episode is all about SageMath, an open source feature-rich option for scientists and mathematicians
built by over 500 contributors and consisting of over 500,000 lines of Python and Cython code.
This is Talk Python to Me, episode 59, recorded May 10th, 2016.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm
and follow the show on Twitter via at Talk Python. This episode is brought to you by Hired and SnapCI.
Thank them for supporting the show on Twitter via at Hired underscore HQ and at Snap underscore CI.
During the conversation, we talked about running SageMath on Windows and some of the challenges there.
We mentioned that the new Ubuntu subsystem coming to Windows 10 this summer may make this easier.
Well, just after we recorded this episode, someone did indeed post a message to the SageMath mailing list
showing it running on the Ubuntu subsystem on Windows 10. So look for more goodness in that area down the line.
Now, let's chat with William.
William, welcome to the show.
Thank you.
Yeah, it's super exciting what we have to talk about today. We're going to talk about
building quite a large ecosystem of computational mathematical tools with Python and a bunch of
other related technologies, Cython, C, IPython, lots of stuff, right?
Mm-hmm.
Yeah, it'll be a lot of fun. But before we get into it, of course, tell me your story. How do you get
into programming and Python? You're a mathematician by training, right?
Yes. Yeah. So I've always been into programming since I was like five years old, but I got very
much into mathematics. As an undergraduate, I was a computer science major for a little while,
and then I switched to mathematics since I love it, especially number theory. And then I went to
graduate school at UC Berkeley from 1995 to 2000. And while I was at graduate school, I got heavily
involved into computing with various objects that come up in number theory. And so I had to write a
lot of code in order to compute with those objects. So I just wrote a huge amount of C++ code using a lot
of the libraries in the mid-1990s. It was really painful. It would take like 30 minutes or an hour to
compile, do lots of templating. And I wrote a little interpreter so I could create modular forms and
other mathematical objects and compute with them. And I was doing this sort of all in isolation without
using much beyond some C++ libraries. And then I found out about a computer algebra system called
Magma, which was extremely powerful for computing in abstract mathematical domains. It's kind of,
it's in a way, kind of a competitor to Mathematica or MATLAB, but aimed much more abstract mathematics.
So it has heavy support for group theory, abstract algebra, graph theory, etc. And very little support
for symbolic calculus. But it's what I needed for my PhD thesis work. So I worked really heavily on it.
It's extremely powerful system. And I wrote maybe 30 or 40,000 lines of code that are included with
that software. I went down and visited them in Sydney, Australia a few times. But one frustrating
thing about Magma is that it's closed source. I saw a talk by Manjul Bhargava, who recently won the
Fields Medal. But back in 2003, he was talking in Paris about using Magma for some research on quadratic
forms. And he was running into some brick walls due to it being closed source. And I also kind of took
stock of my own computing environment at the time. I used Linux, LaTeX, Emacs, and everything on my
computer was open source, except Magma, which was the one program that I cared about, and which really
mattered for my research. So I thought that was ridiculous.
Right. That's the most important thing. Yeah, the most important thing that you actually were working
with was the one that you had the least control or visibility into, right?
Yeah, exactly. And at that time, I was also using Python a lot, not for mathematics, but for
kind of scripting mathematics. So if I wanted to do a bunch of computations on a cluster, I would start
them. I would write little scripts via Python to start up the computations. I wrote websites using
Python. So I was using Python a lot, kind of as a complementary tool to mathematical computation.
And so I was at least familiar with Python because of this. And it was back in 2002, 2003, when Python
was more primitive than it is now.
Right. That's just a little before it started to get its scientific legs under it with things like
NumPy and SciPy. And it wasn't quite there, but it was on the verge of that breakout, right?
Yeah. But it still was a pleasant interpreter environment. And it had modern features like
multiple inheritance, you could define your own data types, had good exception handling, etc.
And so I really appreciated some of the functionality it had over, say, Magma. Magma,
though, has a massive library and is incredibly powerful. It misses a lot of basic features of a modern
programming language, which by modern, I mean, say, at least as good as C++ in 1993. I was used to at
least having exception handling and being able to define my own classes and so on.
Right. That's one of the challenges of not just Magma, I would say, but, you know, things like
R to some degree, MATLAB, Mathematica, all these various highly specialized tools that are great at
solving some type of computation or some kind of problem. But when you have to go and build a full
application out of it or something, it kind of hits a wall, right?
Yeah, exactly. There are a whole bunch of special purpose math languages designed by mathematicians.
And though they are pretty good for mathematics, they don't really compete with what you get when
you have computer science people systematically put full-time effort into designing a language
and then a whole ecosystem around that language. And so the difference between trying to write a
non-trivial program in Python versus Magma is that with Python, you have this massive ecosystem of
tools plus a language that's very general purpose. Whereas with Magma or Mathematica or MATLAB,
it's really good maybe for parts of the math things, but for the other parts, just manipulating strings
or whatever, it can be pretty painful.
Yeah, absolutely. So you were working with Magma and you couldn't really use it in the way that you
wanted to, sort of like a web front end or as the backend for a web front end. And that's one of the
problems you run into a lot with these types of software. I'm specifically thinking of MATLAB because
I have the most experience with it, but there's all sorts of little, you know, pay for this bit,
pay for that bit. It's registered on that machine. Like the rules are just crazy, right? And so you got
frustrated with this. Yeah.
Yeah. So in particular with Magma, the project director sent me in a message saying that I wasn't
allowed to use it as a general purpose compute engine on the backend for my websites. And that
was very frustrating. It wasn't just a matter of pay them or not pay them or get the right license
since they're a little more informal than that. And they don't really have kind of official licenses.
So that was like a huge wake up call that I really needed to do something. And so I decided I was going
to survey the open source landscape again for math software and see what's out there and then switch.
But when I looked around, it was really kind of frustrating back then. There were a number of
programs that kind of competed with Magma, but were much more specialized. And despite Magma's language
being fairly primitive compared to say Python, the other systems had languages that were much, much more
primitive yet. So, I mean, it's kind of hard to emphasize how primitive they are. It's kind of like
using, I don't know, assembly programming. I mean, things like different namespaces, that sort of thing is,
is an advanced feature compared to what's available in some of those languages. So I couldn't really bring
myself to go back to that. And also there's a huge amount of functionality that got implemented in Magma
over about a decade. And it just wasn't implemented anywhere in open source, which was a serious problem
as well. So there was just a massive lack of functionality. So what I decided naively first to
do is just choose a language. And I spent about six months to a year evaluating options like OCaml and
directly writing something myself in C++ and using Python. And at the end of the day, I chose Python
because the language was most similar to Magma's. And so I thought it would be easier to get users over
from Magma. And also the Python C API was kind of like what Magma had, but much cleaner as far as how
Magma itself is implemented. And so one thing about Python is that if you want to write a function that you
can use from the interpreter and is really fast, you can write it in C if you need to. And it's just as fast as if
you wrote it in C. And that's kind of a basic requirement if you're going to write code that will be used for
research, that you can write very, very fast code. So Python was good enough. And then I decided I would just sit down
in Python C and me, and I would look at maybe code from other projects and look up algorithms and implement
everything I needed. And this lasted about three hours. And I realized how insanely hard that would be to do.
And I completely switched courses and decided that purely out of laziness, I would try to wrap existing
code in whatever way I had to do so. So using P expect pseudo TTY type stuff, using C libraries,
and we're adding C libraries to existing computer algebra systems, etc.
So maybe not everyone knows expect. Actually, a lot of people don't know. What was it? They know
C, of course, but what does P expect? So one way if you have a, say, Python program,
and you'd like it to interact with some other kind of bizarre, let's just let's say there's a program
called Symmetrica or something, just some one researcher wrote it has a command line interface,
and it doesn't have any C library interface or anything. But you really want to call functions
in there and run some code there and get the results. What P expect will do is let you basically
simulate the terminal and use Python to programmatically feed code into the other program,
look at what output it produces, and then parse that output and return it. So it abstracts things away
and makes it look like you're calling some functions in a Python library. But really behind the scenes,
things are really more or less being copy pasted into a terminal. It's nice because you can interface
with a black, a closed source black box this way, if you need to. And the drawback is that it's brittle,
and it's potentially slow. There's a lot of latency. Instead of, well, there's a several millisecond
overhead to every single thing you do, which is painful. You really want the overhead to be a microsecond,
not a milliseconds. So there's like a order of magnitude of 1000 latency, which can be really
annoying. Interesting. So yeah, yeah. So you were, you were naive and hopeful and you just said,
I'm going to go write this. And then you basically just said, all right, forget that. Let's try to find
all the good building blocks that are already built and build upon them. And that's a very open source
way of problem solving as anyway, right? Yep. So I think about it this way. Like when I worked with Magma,
my team was all the other people, you know, working on that project. And when I switched to trying to do
something open source, the team was all other people in the entire open source sort of world,
everybody that contributed to open source. And definitely the easy way to go would be to
choose the best libraries for number theory and commutative algebra and graph theory and so on and
put them all together. And they weren't kind of built to be put together or to be called from Python,
Python. But it, no matter how hard it would be, it's way easier to provide a library interface to
one of those other systems that you can then use from Python very efficiently than to write one of
those systems from scratch. Like each of those systems took decades and decades of work to write.
So, I mean, total.
Yeah. And to debug them and fine tune them and, you know, that kind of stuff's very hard to know,
even if it's right some of the time.
And a lot of the algorithms, you really kind of have to be in the throes of a PhD
focused for years just to understand what the algorithms are and how they work. And
like, there's just no way one person can replicate that despite wanting to.
That's interesting. So in the end, you decided, all right, Magma is cool, but it's, it's got all
these restrictions and I'm going to go put this thing together. And what you built was called
SageMath, right?
Yes. Yep. So I built SageMath, which is, well, basically what it is, is it's a Python library,
but many of the dependencies are these tricky to compile C programs or Fortran programs or whatever,
which were written by mathematicians only to run on Linux, maybe just for their research.
And so in addition to just being a Python library, it's also a build system,
kind of like Anaconda, but obviously a little bit before that. And it targets a different type of
dependence, like different types of packages. It's a build system. It's a Python library,
library. And then to make it more friendly for mathematicians, it has a customized IPython
command line. And also around 2006, we wrote a web-based graphical interface, which is like the
IPython notebook, but we wrote it a few years earlier. And it was before things like WebSockets
and a lot of nice modern JavaScript functionality existed.
Back in the dark days of JavaScript, when it was earning its bad reputation.
Yep. It was, we started writing it right when I think Google Maps and Gmail appeared.
And there was this thing called Ajax, which let you update a web page without refreshing.
And so we had that little bit of functionality, but the thing we didn't get was a consistent
or a persistent connection. So instead we had to do polling techniques and all kinds of tricks that
were ugly.
Yeah. Very interesting.
In order to make it like, when you do a for loop in Python, you want to see the output as it appears.
And in order to do that, we had to do a lot of hacky stuff.
Yeah. So I think the way maybe to conceptualize it is this is an open source alternative to
something like Mathematica.
Exactly.
Or MATLAB maybe, but it's, it leverages a lot of the good data science, scientific tooling
of Python, like IPython notebooks, for example.
Exactly. So initially I just wanted it to be good enough for my number theory research, number
theory being my research area. But the second developer wasn't a number theorist. The second
person I could get on board is did research in coding theory. And so he wanted a bunch of
functionality that had little to do with number theories. And then he also wanted to teach
undergraduate courses like calculus and differential equations. And so I started adding a lot of
functionality related to those classes, just so that the range of applicability would get
bigger and I could get a few more developers and things sort of growing, started growing
that way. And so this, the system quickly went from being just for number theory to being much
more general. And in order to kind of focus development and decide what it should be, we chose the motto
or the mission statement to create a viable open source free alternative to Magma, Mathematica,
Matlab and Maple. Our focus, none of the developers really have a focus on numerical computation.
So we, in Sage itself, in the Sage library, focus more on algebraic aspects of things. So arbitrary
precision arithmetic, working with very quickly with large integers and rational numbers, doing linear algebra,
where you have no roundoff error, that sort of thing. And for the more numerical parts, the kind of viable
alternative to Matlab, we just incorporate libraries like NumPy, SciPy,
et cetera. Yeah.
Which the Python community has done an amazing job of developing over the years.
Right. And that's definitely gaining speed now, right?
Yeah. It's absolutely incredible now. I mean, we wanted all along to be a viable alternative
to Matlab. And I remember in 2006, trying to write statistics,
functionality, and all this kind of stuff for Sage directly. And just being like, I don't want to do
this at all, but it's part of what we're trying to do to get more users. So here I am doing it.
And I'm so happy that the community has come along and just made Python absolutely first rate in
statistics and numerical computing so that we can just incorporate that functionality.
Yeah, that's really great. One thing I think before we get too far into the details would be
interesting for the listeners is just the scale of this project, the number of developers, number of
lines of code, the technologies involved. Could you just give them like a quick sense of that?
Sure. So the project itself incorporates about 100 packages. We have done a lot of development at what
we call Sage Days workshops. And those we often have about 10 to 15 per year. We've had, I think,
80 Sage Days workshops so far. They're kind of inspired by the pypy workshops that happened about
a couple of years ago. But basically, we come together for a week and do lots of development.
The number of contributors to Sage is around 500. So those are people who have contributed
directly to the Sage library. Of course, Sage itself incorporates all these other open source
packages, which have their own contributor groups. The amount of source code, I don't know the exact
number, but it's on the order of several hundred thousand lines of actual code. And the code is about
maybe 60% Python and maybe 40% Cython code. So there's a very, very large amount of Cython code that we've
written. And functionality wise, we do cover very wide range of areas of mathematics. So almost anytime
somebody comes to use it, they find that there is something for the area of math that they're interested in.
This episode is brought to you by Hired. Hired is a two sided curated marketplace that connects the
world's knowledge workers to the best opportunities. Each offer you receive has salary and equity presented
right up front, and you can view the offers to accept or reject them before you even talk to the company.
Typically candidates receive five or more offers within the first week and there are no obligations
ever. Sounds awesome, doesn't it? Well, did I mention the signing bonus? Everyone who accepts a job
from Hired gets a thousand dollar signing bonus. And as Talk Python listeners, it gets way sweeter.
Use the link Hired.com slash Talk Python to me and Hired will double the signing bonus to $2,000.
Opportunities knocking. Visit Hired.com slash Talk Python to me and answer the call.
That's awesome. That's a really large scale project. And I think it's a cool success story.
So it's written in Python and Cython, which is kind of a flavor of Python. But you also chose Python as
basically the programming language of the system itself, right? Like MATLAB has its .in files with its own
language. Mathematica has their own language. But you sort of said, hey, Python is a good match for
the mathematicians.
Yes. So this is a decision that I made, I would say, purely out of laziness originally. As a CS undergrad,
I had written an interpreter. I took a course in writing interpreters. And I tried to do that again
for some of my number three research. And I also had just seen how hard it is to really develop a
programming language. And I really didn't want to do it. And moreover, the MAGMA group, I had often made
requests for them to improve their language. And they responded that though they could get grant funding
for the mathematical algorithms, they had a huge amount of trouble getting any funding for the actual
language because their funding sources were in mathematics. So I was kind of scared for a number of
reasons off from writing my own language. And basically, I'm like, well, I mean, it just follows
the same principle of reusing third party libraries and computer algebra systems to reuse a third party
language as well. I decided to use Python. It was criticized a lot by math people, because like literally
every other project like Sage that's ever happened up until that point used their made up their own
language was kind of like the thing you had to do. How has that criticism changed over time?
Do they still feel that way? Or has it become more accepted?
Way more accepted. And it's now considered a huge advantage. And often when people when I hear people
convincing other people to switch from Mathematica to Sage, at all levels, both at research and undergrad
teaching, one of the biggest, maybe the strongest arguments is that Python, the language is just a better
language. Because the syntax is easier for the students. And it's it prepares them for other
possible employment opportunities and down the line. So that's pretty interesting. Because when
I was studying math at university, I did study C++ and scheme and other things because I took some
computer science classes. But my math education really taught me just the math,
math lab language, whatever you call that. I'm sure there's a name for it. But you know,
that that's a very limited skill set to take away if I go out into the job market, right? But if people
as part of learning their their math and doing their research, their statistics, whatever, become
semi-proficient in Python, just sort of silently in the background, that's a really cool side effect.
Yeah. And I have had many PhD students who learned Python really, really well as a result of working
with me. And they really do use it in their jobs today. Like one of them, Robert Miller, who wrote a
huge amount of our graph theory functionality, now works as a data scientist at Google. And another one,
Simon Spicer, also works as a data scientist, but at Facebook. And they use Python, both of them use
Python really heavily in their jobs.
That's cool. Do you know if they use SageMath there?
I don't think either of them use SageMath for their jobs. But I do know that SageMath is used
at Google by some people, but not necessarily my students.
Okay, yeah, that's very cool. What would you say to people who are maybe just straight up using
NumPy or Matplotlib today? Maybe they're using an IPython notebook, like, should they consider using
SageMath? And what would it offer them?
It depends on what you're doing. So SageMath does symbolic things, and it can be very efficient at
doing them. Like you have a large matrix, and you want to do something with it, and you want to have
no rounding errors at all. Or you want to do something with linear algebra over finite fields or coding
theory. Basically, Sage provides a lot of functionality that's complementary to the numerical things that are
offered by NumPy and SciPy. It overlaps some with SymPy, S-Y-M-P-Y, though it has a lot of functionality that isn't in
SymPy. And in many cases, it can be a lot faster. So also, another remark is that Sage now uses Jupyter
Notebooks as our default graphical user interface. So when you download and install Sage, you get Jupyter
Notebooks. One big drawback, which I hope to address in the future, is that currently, by far, the easiest way to use
Sage is to install our self-contained distribution. And it's completely separate, really, from PyPy,
the Python packaging repository. And it's not something that you can just install in your own
Python environment. Right. There's no pip install SageMath. Yeah. Not yet. Yeah. I really, really hope
there will be in the future. But right now, that isn't the case. Right. And so that interferes with easy
adoption by existing Python users. That's true. But you do have a pretty decent way of getting it. You can go
like on the Mac, you can go download just a DMG, just a disk image. Or on Linux, you can get a tarball.
And it's kind of all self-contained there. So it's not too hard to get started with, right?
Yeah. A lot of our users just want to do math rather than deal with installation. And so we do put a huge
amount of effort into making it easy to just install. So we have Windows, Linux, and OS X binaries ready to go.
So the Windows one currently is a virtual machine that you run. And then it provides a server. And
then you get a Jupyter notebook that you can use Sage through.
I see. So maybe you run Internet Explorer, but then you're requesting back into like a Linux thing on a
virtual box or something?
Exactly. Yeah. That's how currently, that's our main supported environment in Windows right now.
Okay. Yeah. That's an easier way to get some of those libraries to compile for Windows, right?
Yeah. That is a seriously hard problem to get some of those libraries that were not built for Windows,
the C libraries or Fortran to compile there, right?
It's really, really hard. We've, we tried for years and are still trying to natively port Sage to Windows.
And I don't know if or when it will ever succeed.
Yeah. Yeah. Interesting.
It's like a, I don't know, in the Sage project, there's a lot of these sort of tall mountains that are
very difficult to climb and four or five people will try and fail. And then the sixth person will succeed.
Like writing a really good implementation of working with what are called finite abelian groups was one
thing where it just seemed like one person after the other tried and failed. And then eventually somebody got it.
And I hope porting Sage to Windows will be a similar thing. Some combination of
the right approach and Microsoft Windows getting more friendly to PostSix style programs.
Yeah. Well, you know, did you, I don't know if you heard, you are in Seattle, right? So maybe you heard
that Microsoft in this summer, they're shipping a new version of Windows 10 and it comes
with the ability to run native Linux binaries on the command line.
Yeah. So that's just awesome. And Microsoft is going in the right direction that we should
have something that works natively by maybe using, well, using that. It sounds really likely
actually.
It's, it was, it's gotta be a huge step in the right direction for you guys. So I don't
know if it'd fix it, but it's definitely not a negative.
Yep. Absolutely.
So one thing that's interesting, well, there's, there's a bunch of interesting things I want
to ask you about this. Let's start with this one. So doing math is obviously computational,
right? Computationally intensive and having more computational power is, is key on modern
hardware. That means parallelism, right?
Yep. It can.
How does it leverage, like multi-core machines? Does it support like parallel computational
parallelism and things like that in it?
So one thing that surprises me a lot over the years, at least in pure mathematics, the
sort of parallelism that we have to do is usually what's called embarrassingly parallel sometimes.
So somehow it's a lot easier than what people do in numerical computing with, you know, earthquake
simulation or whatever, where they have some big distributed computation and each step depends
on previous steps and there's lots of data flying all over the place and you have to use MPI or
something to do it. So the parallelization that typically comes up is you want to evaluate some
function F, 10,000 values of N. And so you just do it pretty easily and there's no real
dependencies between them. So basically using fork, just using stuff built around fork, you can do pretty
good parallelization that works pretty well for us as I've found.
Yeah. Okay. So in the embarrassingly parallel case, you just basically split it across processes
or something like that and a problem is solved, right? They've, they got their own guild, but who
cares? Cause it's theirs.
And you also, if you use a database, so you have your, you can have a process startup query a database
for some things that haven't been done, do them and put the results back in. So simple techniques
like that have pretty kept, I don't know, the community pretty happy. Some of our parallelization
issues aren't too ridiculously hard. There are, there are much more subtle, interesting, like low
level parallelization techniques that go into Sage where there's some C library or C++ library,
some Sage developer writes like David Harvey is a guy who wrote some super fast code for
polynomial multiplication and it uses parallelization techniques. And so Sage just kind of, or really Sage
via some, via Python just calls out to this library. The library happens to do things in parallel and
then give back a result. And so there's lots of little places like that where we have libraries
that just implicitly do things in parallel. They're like highly multi-threaded and you don't even know
they're, they're doing it unless you, you know, explicitly look right. So there's that level.
And then there's also the level of other, of forking. I wrote a decorator for Python called
parallel, which is included in Sage. So you put app parallel before a function. And then if you call
it with a list of inputs, it will, it'll fork the process, call the function with each of those inputs
up to the number of processors you have, and then get the results and give them back to you as a generator.
So there's some little things like that that are built into Sage and it takes care of some
possibly tricky surprises that would happen behind the scenes involving sub processes and so on.
That's really awesome. That's a very, Pythonic little statement you made there. I'm going to put a
decorator on it. It's going to, you know, fork this off and return it as a generator. I mean, that's,
that is really great.
Yeah. And it works really well. We have code that like nested functions written in Cython that just
happen to have an app parallel decorator on them and then they'll fork and regroup properly.
And the code itself is just probably two pages of code, maybe three pages. It's not a lot of code.
Yeah. It's just getting the, the, the right, concept or the Zen of the style to make it really,
um, expressive or whatever. Right.
Yep. there's a lot of, there's a ton of pieces of little things like that all over in Sage,
where I would like to separate them out as Python libraries, like individual standalone Python
libraries, and then have Sage pip install them as part of its kind of build process. So that would
make them much more widely available to the community. And then when they're running the context of Sage,
they would know to do the extra things that Sage needs and they wouldn't do those things.
otherwise. We also have a pre-parser. So for mathematicians, just an example, if you type two
slash three in Python two, which is what Sage uses, you get zero, which is floor division in Python three,
you get 0.666666. And neither of those are what pure mathematician expects. Pure mathematician
expects to get two thirds, which is like an exact rational number. So in Sage, we have a little pre-parser.
So when you use Sage directly via its own command line, which is this modified IPython
interactive command line, it will take each line of input and then do a little bit of parsing to it.
Um, for example, when you type two slash three, it replaces it by integer of two slash integer of three.
And then that allows you to make your own custom integer type. And then there's a bunch of other
things similar to that. would like, for example, when you use the carrot symbol, which in Python
means exclusive or a lot of mathematicians are used to that meaning exponentiation because that's what
it means in LaTeX.
Right. Not as opposed to double star or something like this. Yeah.
Yeah. It's really like, so in Sage, when you type the carrot symbol, it gets converted to double star
before it gets sent to Python. So there's, you don't do it. Mathematicians almost never do exclusive
or, but they do exponentiation constantly when typing in polynomials and all kinds of things.
so it's this optional thing that makes the Sage interface more friendly to mathematicians. It gives
us our own kind of new language, but in a very well-defined minimal way, which is just built on top of
Python. And in most cases, the things that get pre-parsed would be invalid Python code. And it would be nice to
connect us with Python packages. It would be nice if we could take this pre-parser and make it a standalone
Python package, which Sage just happens to use. And maybe like we have various interesting functionality in there,
like square brackets, one dot dot 10 gives you the list of numbers from one to 10. It's kind of like the notation you'd have in
MATLAB or Maple instead of using the range function. And there was a Python PEP to implement it, but it didn't get
accepted. But so many Sage people wanted it. Robert Bradshaw just implemented it anyways and added it to the pre-parser.
So it would be kind of neat if this whole pre-parser were made more generally available and then Sage just happened to use it.
Continuous delivery isn't just a buzzword. It's a shift in productivity that will help your whole team become more efficient.
With SnapCI's continuous delivery tool, you can test, debug, and deploy your code quickly and reliably.
Get your product in the hands of your users faster and deploy from just about anywhere at any time.
And did you know that ThoughtWorks literally wrote the book on continuous integration and continuous delivery?
Connect Snap to your GitHub repo and they'll build and run your first pipeline automagically.
Thanks SnapCI for sponsoring this episode by trying them for free at snap.ci slash talkpython.
Maybe it could even have different grammars for different areas.
Like maybe geologists do something different for the mathematicians. Who knows?
Yeah. And so maybe somebody is proposing some great matrix operations to include in Python eventually.
And maybe they get added to Python 3, but they're not in Python 2.
You could use the pre-parser and still have those.
And everything just gets converted to straight Python.
So you could convert everything and then stick it in a Python library if you need to.
That's cool.
You said that in your documentation you say you have a UI kit for adding user interfaces to calculations and the app.
That sounds pretty cool.
What's the UI technology there and how does that work?
Do you remember?
Yeah, I'm not quite sure what that refers to.
Yeah, okay. No worries.
Yeah, sorry.
That was written long ago, right? Okay.
And there's a lot of people.
The other problem is there's 500 contributors.
So I have very limited knowledge.
Sure, no worries.
I'll ask you another sort of math question.
One of the things when you're writing scientific and especially mathematical papers for academics and so on is you use LaTeX.
So you can very precisely express symbolic mathematics like integral signs and things like that.
And this is built right into it, right?
Yeah, so the connection between Sage and LaTeX is, of course, mathematicians all use LaTeX.
So we have a Dunder method.
Well, it's really just a single underscore method, underscore LaTeX underscore, and it's defined on most objects that you create in Sage.
And it gives back the LaTeX representation of that object.
It works a lot like the wrapper method in Python, but it's to give you a LaTeX representation rather than...
Right, right. That's awesome.
Yeah, so we have...
And it kind of recursively goes down the stack.
So if you make a matrix with entries that are rational functions over some other thing, then there's automatically a way to turn that entire matrix into a LaTeX representation.
And that gets used in the graphical user interface fairly automatically if you want, like in Jupyter Notebooks or in Sage worksheets.
Also, we have a LaTeX package called SageTec, which lets you write a LaTeX document and then use commands like backslash Sage right inside the LaTeX document.
So you can write something like consider and then in dollar signs backslash Sage.
And then in braces, you can put some expression and it will get evaluated using Sage whenever you LaTeX the document.
And then the result's cached.
And you can also put blocks of code and the output gets automatically put into your LaTeX document.
So basically it makes it so your LaTeX document can have executable Python code.
But it's really...
I mean, there's a similar thing like this for just Python.
But for Sage, it also gives functionality for graphics.
And it knows that Sage objects can be... have a LaTeX representation and it uses that automatically when typesetting the results.
Yeah, that's really cool.
So it's something that was written at the first Sage days way back in 2006.
Yeah, yeah, really nice.
Long, long ago.
So...
And it's been developed ever since.
So another part of this system that I think is really interesting is Cython.
And Cython actually...
This I had no idea about.
I just want to talk to you about SageMath.
But Cython actually came out of this project, right?
Yes, in the following sense.
So when I started Sage, I mentioned before that Python plus the Python C API was kind of like the killer combination that meant that I could implement what I wanted in Sage on top of Python.
So I had all these little benchmark programs and I would try to implement them in other languages like OCaml, like these really fast functional compiled languages.
And I would get something that was fast, but it wasn't quite as fast as I could write in C.
And if you kind of look, basically overall C lets you write really fast code and Python lets you use C code to write new functions.
But the Python C API is pretty challenging and potentially error prone.
You have to do manual reference counting all over the place.
And the different...
The ways in which the Python C API functions work can take a while to learn.
And I really envisioned a lot of people, including me when I'm tired or just really want to get work done, writing code.
And I wanted to write a huge amount of code.
And I didn't think it was really viable to write directly against the Python C API.
I would mess up some reference counting and get seg faults.
And that just wasn't what I wanted to do.
And so I started planning to write some Python program that would generate code against the Python C API.
And I got excited about the possibility of doing that.
And then I started searching around and found a program called Pyrex, which is like the glassware, P-Y-R-E-X, by Greg Ewing, who is a Python contributor.
And he wrote this amazing program where you could write code.
It looked almost like Python, but it would get converted into C extensions.
And those would get linked into Python.
And so that was really amazing.
But it lacked a lot of functionality.
Like it didn't have list comprehensions.
And there were just a lot of little things.
You couldn't do nested functions.
Also, he didn't use revision control or anything.
And he just kind of made it a zip archive every once in a while.
It seemed like he worked on it mainly during the Christmas break each year or something.
Those were the early days.
A lot ago, right?
So, yeah.
So this was like 2004, 2005.
And it was an absolutely awesome project.
But it kind of wasn't really going anywhere.
And so I started adding functionality I needed to Pyrex.
And that was...
I mean, I don't really like writing compilers very much.
It's not my thing.
I'm not very good at it.
But I had this PhD student, Robert Bradshaw, who I mentioned before, who now works at Google.
And he started diving in and writing a lot of really tricky stuff and doing all these optimizations to Pyrex.
And I thought that was pretty cool.
And then another guy, Stefan Benel, who is the developer of LXML, which is a Python library,
he was forking Pyrex to add all kinds of things as well.
And then at the same time, I made up a project called SageX for making Sage faster.
And it was just a combination of some of these Pyrex forks.
The name was really bad, though.
SageX.
And we were just distributing it inside of Sage.
So that was kind of stupid.
And so after about a year of that, I looked at this book by Fogel called Producing OSS.
It's on his website, producingoss.com.
And I think it's inspired by the Subversion Project.
It's a great book.
And I started...
You know, like when you read a book...
To me, like when I read a book, I kind of as an exercise think about a lot of stuff while I'm reading the book.
And so I started thinking about SageX and Pyrex.
And then I was brainstorming for a name for this thing and kind of thinking,
I should make this into a proper open source project.
And then I came up with this name, Cython, which is like Python, but C.
But I thought it was a better name than SageX, which is hard to say.
And sort of doesn't have anything to do with Python or C.
And so I Googled it.
And there was only one hit, which I could find.
And it was a picture of this punk rocker in England who called himself Cython flipping you off.
And so I thought, I'm safe.
You'll probably not lay claim to this, really.
Exactly.
So I thought, I'm safe.
I can choose this as a name.
And then I wrote to...
Or I talked to my PhD student, Robert, and to Stefan Benel.
And I'm like, so I'm going to start this project and make you the lead developer.
And each of them said no.
And so I was a little worried about that.
So I made them both lead developers and kind of like stepped back from it, made the website, bought the domain name.
And though they both sort of decided not to be the lead developer, they together did an amazing amount of work.
And then other people started contributing.
And I mean, it really has blown up a huge amount since then.
So I did try to push their direction to be, make it so that Cython can compile all of Python.
Like you could take the standard Python test suite and just build it and see how fast it is compared to Python.
And the systems really got deep because part of the test cases for Cython is compile all of Sage, which is huge, and then run the Sage test suite.
And also there are sort of things we need, like when you're running blocking C code, you want to be able to hit control C and have that interrupt the running code.
And so in Sage, we put a lot of work into writing something that would let you interrupt blocking C code that was written using Cython.
Martin Albrecht recently separated that out as a separate project that you can pip install.
So you can use it in your own Cython code.
Right. Okay. That's really cool.
So that's kind of where Cython came from.
And I think it's been steadily progressing over the years.
Very often there are things that come along which kind of sound like they do something similar, but Cython is just so battle-tested.
And no matter what, like if you want to write some fast code that you can use from Python and you're willing to understand the basics of C data types, you can use Cython and do it and get it done.
And if you want to make some code that's usable from C or C++, like a C or C++ library available to Python with no overhead, then you can do that via Cython.
So it's very different than SWIG, Standard Wrapper and Interface Generator, where SWIG will automatically take a C++ library and make it usable from Python.
But there's some Python code between the C++ library and you, and that slows things down a lot.
So with Cython, we do things like make it so you can use a third-party library that provides basic arithmetic addition and multiplication, say, in a finite field.
And we can make that so it's very, very fast.
So it just takes on the order of 100 nanoseconds to do additions and multiplications.
So it really, really eliminates overhead and makes using very, very low-level C code possible.
Yeah, that's really awesome.
How applicable is that in general?
Is this something you would only use on math?
Or if you've got a web service backend and there's a section that needs to go faster, would Cython be something you might consider?
I think it's really, really aimed towards very compute-heavy loads.
So it doesn't help much if your problem is IO-bound.
And then the new Python async stuff is probably way more useful, or something like Twisted.
Yeah, absolutely.
So it's really, really all about compute-heavy, non-asynchronous.
You have a synchronous thing like a for loop where you have a doubly nested for loop that's doing some calculation in the inside.
And you want to make that one part really, really fast, as fast as you could possibly write it in C.
That's what Cython lets you do.
Okay, very cool.
And that's the sort of thing that comes up a lot in math programming.
Yeah, of course it does.
Matrix multiplies and all sorts of things, right?
Yep.
So the sort of next step that you might take this whole project is rather than having people download 800 megs of all those packages and run it locally,
it's just fire up a web browser.
And so you started cloud.sagemath.com, right?
Yes.
Yep.
I started that in 2013.
And my initial motivation was that I had been teaching courses for just repeatedly every year for about a decade to undergraduates on how to use open source math software for kind of math.
So I get like 40 or 50 math or math CS stats econ undergrads.
And the whole course is about how to, you know, program in Python, program in Cython, use lots of functionality from Sage, use NumPy, use LaTeX a little bit.
And the installation challenges for the students were pretty bad.
And so for quite a while, they would use, they would often just use the Sage notebook server we had running, but that could only handle maybe a dozen people at once.
It really wasn't very scalable.
And all it did was give you Sage.
So you could type in Sage code and see the output, but it didn't have a terminal and it didn't have LaTeX.
And so I decided to write something that was much more all-inclusive where you could teach a whole course about anything related to basically open source scientific software and do it all in one unified place where the students in about, you know, 15 seconds are up and running.
So that was the motivation.
And something I've refined over the last few years.
I've taught using it a couple of times now and written some research papers and a book all within Sage Math Cloud.
And there have been several hundred other people who have done courses using it now.
Yeah, that's, that's really great.
I can just imagine the first day of these sort of computational math classes where it's like, all right, so what we're going to do is we're all going to download this.
Like my download canceled.
Mine won't install.
Mine installed, but it won't launch.
You're just like, ah, why?
This is not what this class is about, right?
But you've got to plow through that if it's on the machines, right?
So this, this is really cool.
And, you know, it's a really slick web app that you guys have built.
Thanks.
Yeah.
So I've, I've been working on an enormous amount for the last couple of years with some input from Jonathan Lee and Nicholas Rund, who are two UW undergraduates.
And also Harold Schilley, who's been working on Sage since about 2007 and who lives in Vienna, Austria.
So we've done, we've written this application and it has, I don't know, around about 4,000 users each day.
And it'll often have about 400 active users at once.
So it's a pretty heavily used.
And the usage is heavy because people are really running code.
And they're running lots of Jupyter notebooks and Sage worksheets and editing LaTeX documents and maybe running long-term calculations.
It all runs on top of Google Compute Engine, but it's also entirely top to bottom open source.
And some people download it and install it on their own computers.
Okay.
That's a really interesting component of it that you can get a local server version.
Yep.
And it provides, so what it gives you is real-time collaborative editing of terminals, Jupyter notebooks, LaTeX documents.
It also has a course management system built in for assigning homework to students and collecting it.
And everything's collaborative in that you can see other people editing, kind of like Google Docs.
There's also chats and stuff.
Yeah, that's fantastic.
And you've got a lot of features there.
Like, it's not just for, say, running Python code, but you can run C or Go, or there's the ability to open terminals, just Linux Bash or something like this, right?
Yep, absolutely.
So it's kind of like those online IDEs, but, you know, like cloud9.io or one of those.
But our target audience isn't at all programmers.
It's just people who, especially students who just want to get up and running quickly with open source math-related software.
And we also have really good support for R and Julia and Octave and so on, especially because Jupyter notebooks have kernels for all these languages.
Right.
You just pass it right along to Jupyter and let it deal with it, right?
Yep, exactly.
Nice.
So you've got a lot of cool stuff in your GitHub, which I'll put in the SageMath GitHub, and I'll put a link to that in the show notes.
And one of my stuff on there was stuff to do with Docker.
How does Docker fit into this?
First, there is a SageMath Docker image that Eric Bray, who I think is a guy in Europe, in France, who I think has the distinction of being the first full-time person to work on Sage.
So NSF grants for Sage have kind of dried up recently.
But in Europe, they just got this huge $8 million grant called Open Dream Kit.
It's across the European Union, and it supports open source math software, including Sage.
So the Europeans are really, really supporting open source math software.
And Eric created some really nice Docker images for running Sage.
So you just do like Docker run dash something SageMath, and you're up and running with Sage.
So it's yet another way of installing Sage.
And there's a Jupyter version of it.
So that's one thing.
So SageMath Cloud doesn't currently exactly use Docker.
We use Cgroups and a lot of the stuff that Docker is built on.
But I think it will use Docker extensively in the next few months under the hood.
You won't know the difference, but it will just make things run a little smoother and be more scalable.
Yeah, and let the Docker folks manage that kind of stuff and not you, right?
Yep.
Yeah, exactly.
Yeah, yeah.
Very cool.
All right.
We're getting kind of to the end of the show.
One thing I wanted to recommend for people is if this is all interesting to you guys, go to cloud.sagemath.com.
You can create an account there and check it out.
But there's also a video like right on the landing page there that gives the two-minute story of this.
If you want to share it, that's cool.
And it features you and a bunch of other people in there, which is cool.
And you've got some pretty sweet skating in there.
Yep.
I own the biggest skateboard ramp in the Northwest and I skate it a lot.
So I'm really into skating large half pipes.
Wow.
That's really cool.
Is it?
This is at your house?
It's at a friend of mine's house, thankfully.
It's not at my house.
Nice.
That's really cool.
When I was growing up, my brother built a quarter pipe.
So we periodically hurt ourselves.
So I think that was cool.
Yeah.
I got into skateboarding with my brother.
We skateboard together a lot still.
Oh, yeah.
That's great.
All right.
So I think it's probably a good time to wrap up our conversation.
So let me ask you two questions I always ask my guests.
First, even though you can't pip install SageMath, there's still a lot of packages that you guys
must use.
And so do you have a favorite PyPI package that is really helpful to you that maybe other
people don't know about?
Well, okay.
Let's see.
First, again, I really wish the answer was SageMath.
I did.
I posted my first ever package to PyPI.
How do you say it?
PyPI.
Guido says that PyPI and some of the core developers say PyPI and PyPI conflicts with PyPI the runtime.
So I'm with PyPI as well.
Okay.
So I posted to PyPI a PyGSL package.
So I'll say that that's a brand new one that wasn't there before.
I did it because a physicist who wanted to use it in their class on SageMath Cloud wanted
this.
GSL is the GNU scientific library, and they wanted these Cython bindings to the GNU scientific
library to be available.
And the only way to do it was to download some zip file off of SourceForge.
And that sucks.
So yeah, anything on SourceForge these days is a little suspect.
So I set up a PyPI account and figured out how to...
I took the zip file, then I pushed it.
So I myself became the kind of manager of the PyGSL package on PyPI.
Yeah, lovely.
It took me about two to three minutes to learn how to do so.
So my answer is that package or push your own package, especially some old thing on SourceForge
that should be pip installable and make it pip installable.
It's easy for anybody to make things pip installable.
Yeah, it's surprisingly easy to register a package and upload it and so on.
It's...
You just more or less create an account and just use the built-in tools and the set of tools
and stuff.
Yeah, it's really nice.
Yeah, absolutely.
Awesome.
And then what editor do you use when you write Python code?
So I dogfood everything.
So I do absolutely all of my development of SageMath Cloud and Sage from within SageMath Cloud.
And the editor of SageMath Cloud is...
It's built on top of Codemirror, which is a web, like a JavaScript-based code editor.
So I added things like multiple panes so you can split the view and see two points in the
document at once.
And also because the editing is synchronized, you can just open multiple browser tabs to
see multiple points in the same document at once.
And then I also have various plugins for the kind of functionality I felt like I really needed,
like deleting, trailing, white space, and so on.
So my answer is Codemirror, which is probably not the most popular answer, but that's my
answer.
Oh, oh, one other thing.
I think that's really cool.
One other thing is in SageMath Cloud, there's a history.
So whenever you edit, every keystroke almost is recorded.
And there's a slider that lets you slide back and forth and see the past, all past versions
of your document.
And so like when I'm editing code, I'll be like, oh, I wanted to look at what it was like
three minutes ago or 10 minutes ago and copy something from that.
And you can very easily do that with a little slider.
So you get kind of a whole third dimension of time when editing.
So it does have some benefits.
Yeah, that's a really nice feature.
I feel like a system has reached some level of maturity and well-roundedness and goodness,
basically, when it can create itself.
You know, when you compile C with a C compiler, when you write like PyPy, you know, you run
Python with Python, these types of things.
It's pretty cool.
So it's nice to hear you're doing that for your editor.
Awesome.
So before we say goodbye, what should people know about getting started with this?
If they want to check it out, what should they do?
So just type Sage or SageMath into Google and you'll get to the SageMath.org website.
And that has all kinds of documentation.
There's like probably 10 to 15,000 pages of documentation there, links to books and so on.
And there's a big link to, which just says SageMath online.
And if you click that, it takes you to the SageMath cloud site that we've been talking about.
And there's also something called SageMath cell, which gives you a single,
one single input box where you can type in a block of Sage code and see output.
And you can also embed those blocks inside of your own static website.
And there's an API for doing that.
So go to the website SageMath.org.
You can either download it or go to the cloud site and use it online.
It's all 100% open source, top to bottom.
William, this is quite an achievement.
I really love looking inside of it because you solved a lot of cool problems and a lot of
excellent technologies come out of it.
So congratulations.
Thank you.
Yeah.
It's been good to talk to you.
Take care.
Okay.
Thanks a lot.
Great talking to you.
Yeah.
Bye.
This has been another episode of Talk Python to Me.
Today's guest was William Stein.
And this episode has been sponsored by Hired and SnapCI.
Thank you guys for supporting the show.
Hired wants to help you find your next big thing.
Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity
presented right up front.
And a special listener signing bonus of $2,000.
SnapCI is modern, continuous integration and delivery.
Build, test, and deploy your code directly from GitHub.
All in your browser with debugging, Docker, and parallels included.
Try them for free at snap.ci slash Talk Python.
Are you or a colleague trying to learn Python?
Have you tried books and videos that left you bored by just covering topics point by point?
Well, check out my online course, Python Jumpstart by Building 10 Apps at talkpython.fm slash
course to experience a more engaging way to learn Python.
You can find the links from this show at talkpython.fm/episodes slash show slash 59.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and
direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
You can hear the entire song at talkpython.fm/music.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
I'm dating with my voice.
There's no norm that I can feel within.
Haven't been sleeping.
I've been using lots of rest.
I'll pass the mic back to who rocked it best.
I'm first, developers, developers, developers, developers, developers, developers, developers,
developers, developers, developers, developers, developers, developers, developers, developers,
you