The advances in astronomy over the past century are both evidence of and confirmation of the highest heights of human ingenuity.
We have learned by studying the frequency of light that the universe is expanding.
By observing the orbit of Mercury, that Einstein's theory of general relativity is correct.
It probably won't surprise you to learn that Python and data science play a central role in modern-day astronomy.
This week, you'll meet Jake Vanderplass, an astrophysicist and data scientist from the University of Washington.
Join Jake and me while we discuss the state of Python in astronomy.
This is Talk Python to Me, episode 81, recorded October 21st, 2016.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm.
And follow the show on Twitter via at Talk Python.
I'm very excited to announce this episode is sponsored by not one, but two new sponsors.
And they both have excellent offerings for Python developers.
Welcome GoCD by ThoughtWorks and DataSchool to the show.
Thank you both for supporting the show.
Jake, welcome to Talk Python.
Thanks.
Good to be here.
Yeah, it's great to have you.
I'm a huge fan of astronomy and science.
And I'd love to talk to you about how Python and astronomy interact and all the problems you're solving.
But before we get to those, let's start with your story.
How did you get into programming in Python in the first place?
Well, you know, I came to programming relatively late.
I had a little bit of early experience in like sixth grade with Hyperscript, HyperCard, but didn't do much.
I took a small programming class in high school, and I didn't really do much programming aside from evaluating physics stuff and Mathematica, simple things, until I was in grad school, actually.
So I arrived at grad school and started working with a research scientist who later became a faculty.
And I asked him, hey, this was around 2006, I asked him, hey, what programming language should I use?
Most of the people around were using IDL, this interactive data language.
It's a proprietary scripting language that's similar to MATLAB or Python in some ways.
And he was one of the only people using Python at the time, and he said, well, you should use Python.
That's the future.
Everyone's going to be doing that soon.
And so I decided to do it, and I learned Python.
I taught myself Python over winter break.
Sudoku was big at the time, so I wrote a Sudoku solver, and that was my way of learning how to do control flow and everything in Python.
Yeah, that's great.
I really think writing those little games like that are a great way to learn a language, or at least get started with it, right?
Because the problems are not so complicated.
There's not a lot of interaction.
It's not like, well, how do I talk to a database?
How do I do a UI?
How do I call the web?
Yeah.
It was super fun.
And being someone who didn't really have a background, any formal background in algorithms, it was a nice way to wrap my head around what sort of problems you can do in programming.
So pretty much Python from then on, you've been doing a lot of stuff.
You've been contributing to some machine learning libraries, to the whole Scikit area.
Yeah.
So where that came in is basically I started doing all this work in Python.
I was writing horrible little one-off scripts like most scientists do who don't have formal training.
And a couple years into my PhD program, I wrote my first paper.
And the first paper was pretty interesting.
It was using this relatively new at the time algorithm, a version of manifold learning called locally linear embedding.
It was using that to explore some astronomical spectra.
This algorithm was implemented out there.
The paper introducing it had a link to a little tarball of MATLAB code.
But I found pretty quickly that the code didn't scale to the size of the problem we had, which was hundreds of thousands or millions of spectra in several thousand dimensions.
And so I spent a summer basically looking at this science and nature paper, looking at this MATLAB software, and trying to figure out how to write a more scalable version of the algorithm.
And what came out of that was this C++ package.
And I published the paper and did this standard thing of putting the C++ package in a tarball on my website.
And I thought to myself, this is ridiculous.
And the next person who tries to use this in astronomy is going to have to spend, you know, they're going to have to hire another grad student to spend a summer and figure out how to implement this.
So I started asking people about how to make sure that your code can be used by other people.
And then I found out there's this whole catchphrase called reproducibility, open science and things like that.
So that was my foray into reproducibility and open science.
And as I was asking around, someone mentioned that there was this brand new package that might be interested in that algorithm called Scikit-learn.
And so I got in touch with Gael Varaku, who was getting Scikit-learn off the ground, and they thought it would be a good contribution.
So I started, I think that was 2010 or somewhere around there, I started contributing to Scikit-learn when I was really young.
And, you know, I haven't looked back.
I've really been turned on by this idea of open and reproducible science by making sure your software products that come out of your research are actually well-documented and reusable.
And this thing that was sort of a side project in the beginning has turned into most of what I do during my day-to-day work.
Isn't it funny how life takes those kinds of turns?
Like you plan to do one thing and you discover another and it really becomes something you're passionate about.
That's cool.
Yeah, and it turns out now I'm way more excited by general software tools than I am about the astronomy research that drew me into grad school.
Yeah, it sounds a lot like my story.
That's awesome.
Let's talk just really high level for just a moment about machine learning.
I know a lot of people out there are into data science and they know machine learning, but there's all sorts of listeners.
So, I mean, like how people used to solve problems and they would use like statistics or other types of things.
But then this whole machine learning seemed to formalize it, bring some algorithms together.
Like kind of give us an overview of what the whole story there is.
Yeah, so whenever I introduce machine learning, I always emphasize the fact that it's just a set of models to data.
You know, when you fit a line to data, you're doing machine learning.
When you take two clumps on a two-dimensional plot and draw a line between them to say this side is one type and that side is the other type,
that's a form of machine learning.
And where machine learning gets powerful is these algorithms that you can do by eye or by hand in two dimensions,
like drawing a line on a piece of paper.
Once you formalize those algorithms, you can scale them up to a large number of points and large number of dimensions.
You said you have a thousand dimensions in your previous problem.
Like you're not doing that by eye, right?
Yeah, yeah.
So, you know, a thousand dimensions, the equivalent is fitting a 999-dimensional hyperplane to split things into two groups.
And you can't really do that by eye.
But the key is that machine learning is nothing more complicated than fitting these models to data in a way that scales to large data sets and to high-dimensional data sets.
And, of course, it grew out of artificial intelligence and statistics in some sense.
But I think the core distinction between the machine learning way of doing things and the statistics way of doing things,
this distinction is put together, is described really well in this paper by Leo Briman called Statistical Modeling, the Two Cultures.
The overview summary of that is that, you know, in classic statistics, you're building models where you care about the model parameters.
Like you fit a line to the data and the slope is telling you something fundamental about the world.
Whereas in machine learning, you fit a line to data.
And you're not so much interested in the slope.
You're just interested in what that line can tell you about new data that you, you know, you want to predict something about.
I see.
So a lot of machine learning is about prediction in the future.
Like you create a model and then you want to ask it questions.
Yeah, absolutely.
And you're learning something about unknown data.
I mean, the distinction is not completely black and white there, but I think it's a useful way to think about machine learning versus statistics.
Yeah, that is an interesting way to put it.
What are some of the major tools in Python that people use?
Yeah, so scikit-learn is one of them.
This is the Python package that's built on NumPy and SciPy and kind of uses the classic tools.
It's really nice for doing sort of small to medium scale machine learning and modeling problems.
It doesn't have a particularly good scalability story.
There are some ways to kind of parallelize certain operations within scikit-learn.
But if you want to go to out-of-core data and things like that, there are other ways to do this.
So scikit-learn, to be honest, is for the bulk of what I end up doing in my work, I can use scikit-learn.
And for scaling to large data sets, often in the work I'm doing, I'm doing kind of massively parallel stuff where I can split the data into chunks
and run a small scikit-learn algorithm on one of those chunks and loop through them that way or parallelize them that way.
Yeah, that makes sense.
So maybe you're looking at some large part of the sky and you could break it into little grids or something.
Yeah, exactly.
I'm often looking at things object by object rather than trying to do things all at once.
If you need to do larger models that are doing things all at once, there are these interesting libraries recently built around things like Spark and TensorFlow.
TensorFlow. I'm not as experienced with those, but the TensorFlow stuff is interesting.
And in particular, there's this SKFlow package that I've been rather intrigued by that kind of builds a scikit-learn API around a TensorFlow backend.
Oh, that sounds like that's worth looking into. That sounds cool.
And there's also PySpark, which is interesting.
So where I'm in the process right now, it's been kind of fun.
I'm working with some computer scientists and some neuroimaging people and some database specialists to put together comparison between a number of Python-oriented approaches to doing scalable computation in a scientific setting.
And so hopefully that paper will be coming out in the next several months.
Oh, yeah. That sounds really interesting.
Give us some examples of how this whole machine learning story applies to astronomy.
So, like, what types of problems or things are people doing with this?
Yeah, so astronomy, we have a lot of areas where we want to predict certain aspects of things.
So one example, just to be concrete, is let's say we're looking for the distances to distant galaxies.
And the distances or red shifts of galaxies are important in constraining things about our understanding about the cosmology of the universe, the structure of the universe.
But getting an accurate distance to a galaxy is an expensive observation.
You have to do a spectral observation, which basically, you know, you look at an individual object and you split the light from that object using, like, a diffraction grading into its whole spectrum.
You know, red on one end, blue on the other end, and a thousand bins in between.
And given something like that, you can isolate certain emission lines or absorption lines and calculate its red shift, which is similar to its distance.
And that's really, really accurate.
But the problem is it's incredibly expensive because you have to look at individual objects and line up these diffraction gratings one by one.
And when we're looking at, when we're just taking pictures of the sky, we're getting thousands or millions of galaxies a night.
And it's, and we don't have the resources to take a spectrum of all of those.
So the question then is, can you, can you take a small set of objects where you have these very detailed spectral observations and learn something about them so that you can predict what the red shift might be from a more coarse photometric kind of, kind of picture observation of them.
And this maps pretty well onto a machine learning model, right?
You have, you, you take a picture of the whole sky, and so you get data about each object that way at a coarse level.
And then you take spectra of a certain collection of objects, and that gives you finer detail, more information about a subset of them.
And then you want to build a model that can predict that, you know, that more information, the red shift and the distance for, for all the rest of them.
So at first glance, machine learning seems to map pretty well onto astronomy data.
The thing that's difficult about it in practice is most machine learning models assume some sort of statistical similarity between your training set and your unknown set.
And in astronomy, unless you specifically design it that way, it's difficult to get that statistical similarity.
So for example, we tend to have spectra of nearby bright objects because they're easier to take spectra for.
If you're looking at distant, faint objects, the noise characteristics are different, the statistical distributions are different.
So a straightforward machine learning approach to that will miss some things, and you might not even know you're missing things.
Yeah, I can imagine.
You know, one of the things that just blows my mind is that we can see things so far away and so small and effectively so far in the past and still make intelligent statements about them.
Yeah, yeah.
It's pretty amazing.
It's unbelievable some of the things you guys are doing.
And the thing that blows me away, actually, about astronomy and astrophysics in general is the fact that these laws that we discovered in the laboratory here over the course of the centuries actually apply to what we see out there 10 billion light years away, right?
And it's not just that we're assuming they apply, it's that we can actually test and confirm that they apply.
One example is, you know, there's all these scientists in the 18th and 19th centuries studied the behavior of gases, right?
What happens if you blow up a balloon and how fast does the air come out?
And all that led to this formalized field of thermodynamics and statistical mechanics.
And as you go into more detail, like what happens if you're looking at ionized gases and things like that, we learned all this stuff in the lab.
And then in the mid-20th century, figured out that the cosmic microwave background, this light echo of the Big Bang, actually comes from a plasma in the early universe.
And we can understand the properties of the plasma there by the same laws.
And the reason that we know that the universe is 13 point, you know, I can't remember the decimals, but 13 point something billion years old with a very good accuracy.
One of the reasons we know that is because we understand the thermodynamics and statistical mechanics of the plasma in the early universe and can compute what that says about the cosmic microwave background.
And that story right there is just fascinating to me.
Yeah, it's totally fascinating.
And what I think is also fascinating is the guys who discovered it, was that Bell Labs in New Jersey?
I think.
Yeah, yeah.
Yeah, the guys who discovered that whole cosmic background radiation weren't looking for it.
They found it on accident and it was in their way, right?
Yeah.
And their first hypothesis, I guess, was that it was pigeon droppings on the detector.
And once they cleaned off all the pigeon droppings, they had to figure out it was something else.
Yeah, so they got the Nobel Prize for finding static in their instrument and realizing the static was significant.
Yeah, normally that would be a problem, right?
Yeah.
You want to get rid of it.
Very cool.
And so you gave a really interesting talk at PyData, I think 2015.
I'll be sure to link to the – it's up on YouTube.
I'll link to the video.
Mm-hmm.
And you talked about how distance is super important in astronomy.
And it relates to many of these big ideas that we hear about if we're sort of paying attention, I guess.
Yeah, absolutely.
Distance is fundamental to a lot of what we do.
And it's also really, really hard to figure out.
I mean, if you think about just looking at a dot of light in the sky, how do you tell how far away that is?
And so a big part of the story of astronomy over the past couple centuries has been people figuring out how to determine how far away things are.
So the first step that people figured out is we can do it geometrically.
You know, the same way as if you put your finger in front of your eye and close one eye in the other, your finger seems to jump around in front of the background.
That's called parallax.
And we can use a similar type of trick to find the distance to nearby stars because the Earth is on one side of the sun in June and on the other side of the sun in December.
And if you compare what the nearby stars look like compared to the background stars in June and December, you see them jump back and forth.
And you can use the geometry of that to figure out the distance to those stars.
I see.
So you measure the sky and you basically see which ones kind of move more and which ones are more or less fixed.
And then based on the parallax, you can say, well, these ones that move, they're five light years away or something.
Yeah.
And you can calculate that based on the angle and what we know about the Earth's orbit about the sun.
But that only works to within, well, up until a couple months ago, it was within maybe a few thousand light years.
There's this Gaia mission.
The data was just released in the last couple weeks.
And that one of the things that Gaia can do is it's going to give us really accurate parallax distances out to previously unheard of distances.
So we're going to really be able to figure out the three-dimensional structure of the stars in our galaxy.
But that parallax is not going to work when you go out to more distant galaxies.
So you have to come up with other ideas.
And one of the ideas that's been really fruitful is this idea of standard candles.
If I stick you on a street in the dark and I turn on a 100-watt light bulb and I put it right next to your eye, it's really bright.
But if I put it two blocks away down the street, it's really dim.
And that brightness and dimness, you can compute that because the apparent brightness is attenuated by a factor of one over the distance squared.
So if you know that, if you look at a light two blocks away and you know that it's a 100-watt light bulb and you have a very accurate photometer,
you can compute exactly how far away that light bulb is.
And this works with stars, too.
If we know the exact brightness of a star, the exact intrinsic brightness of a star, and we look at its apparent brightness, we can compute the distance very easily.
Now, the trick there is you need to know the intrinsic brightness of the star.
Yeah, that's the kind of stuff that amazes me because you look out at these things super far away.
And how do you know their intrinsic brightness, right?
Yeah, and it's really difficult.
One thing you can do is build off these things we learned from parallax, and you can look for certain classes of stars that are always around the same brightness.
And you know their brightness when you know the parallax distance.
And then you look for the same class of stars that are further out, and you can sort of infer their distance that way.
So this is why it's kind of called, in astronomy, it's known as the distance ladder.
You know, we have these direct methods that lead to more indirect methods of distances as we go further and further out.
And one of the coolest stories of this distance ladder is back in the early 20th century, there was this woman named Henrietta Leavitt.
And she was looking at variable stars.
So there are stars out there that get brighter and fainter with time.
And she was looking at particularly at this class of stars called Cepheid variables.
It was named after the fourth brightest star in the constellation Cepheus.
And she found something curious when she was looking at the variation of these.
They would get brighter and fainter with a period of, you know, somewhere between a day and a couple days, something like that.
And she found that the period of how fast they got brighter and dimmer was related to their intrinsic brightness.
And so there's this nice plot where she shows that.
And it's this roughly linear trend between period and intrinsic brightness.
And that's really nice because then you can, a period is something that you can find out in the sky.
So she looked out and, you know, found all these stars and confirmed that the period and the intrinsic brightness were related.
So then Hubble came along.
And you've probably heard of Hubble from the Hubble Space Telescope.
And what he did is he used the telescopes available to him and found more and more of these stars.
And based on this period brightness relation, was able to estimate the distances to all these stars.
And the thing that really completely blew open our understanding of the universe was when Hubble pointed his telescope at one of the, what they called then the spiral nebula.
So there were these spiral shaped clouds out in the sky that for a long time people thought were just clouds of dust in our galaxy.
But Hubble found individual Cepheid variables in the Andromeda spiral nebula and found that it wasn't in our galaxy.
It was about two and a half million light years away, farther away than anything we ever would have imagined existed.
So in one fell swoop, the study of variable stars led to us understanding that the universe is orders of magnitude bigger than we ever imagined.
That's really amazing how that ladders up there, right?
And beyond that, we also learned that the universe is not contracting or sort of static, but it's sort of going away from itself and accelerating, right?
Yeah.
Yeah, so at the same time, he found that not only were these galaxies really far away, but if he looked at all these galaxies,
they were the spiral nebula, and now we know them as galaxies because we know they're separate groups of stars.
He looked at all these and he found that there was a relationship between how far away they are and how fast they're receding from us.
We can measure their recession velocity by looking at the redshift of the light.
And it's kind of like the Doppler shift, when a siren goes by you, you hear it high at first and it passes and it goes low.
You know, this...
Yeah.
And that same Doppler shift, we could see that the light was shifting to a lower frequency,
just like the sound shifts to a lower frequency when a car goes away from you.
And you can measure the velocity.
And he found this relationship between the distance and the velocity,
which basically describes a uniformly expanding universe.
And, you know, right around the same time, Einstein's predictions, general relativity,
people were realizing that the general relativity equations that describe gravity
and explained the orbit of Mercury, among other things,
you could solve those in a way that led to an expanding universe.
So it was another confirmation of general relativity.
And this was all based on finding distances to galaxies.
This portion of Talk Python to me is brought to you by GoCD from ThoughtWorks.
GoCD is the on-premise, open-source, continuous delivery server.
With GoCD's comprehensive pipeline and model,
you can model complex workflows for multiple teams with ease.
And GoCD's value stream map lets you track changes from commit to deployment at a glance.
GoCD's real power is in the visibility it provides over your end-to-end workflow.
You get complete control of and visibility into your deployments across multiple teams.
Say goodbye to release day panic and hello to consistent, predictable deliveries.
Commercial support and enterprise add-ons, including disaster recovery, are available.
To learn more about GoCD, visit talkpython.fm/gocd for a free download.
That's talkpython.fm/gocd.
Check them out. It helps support the show.
I think one of the things that's super interesting about this is, you know,
this concept of variable stars and the work that woman did was very manual, right?
Like she would look at pictures and so on.
Yeah, she was measuring the way they measured brightness of stars back before CCDs
is you're looking at photographic plates and the brighter something is, the more it's saturated.
So you'd have to do a detailed measurement of the size of the dot on your photographic plate
and use that to compute the brightness of the star.
It's just, it's amazing to me that any of that work got done given how easy we have it now, you know?
Yeah, exactly. Exactly. It's such a different world.
But at the same time, we've kind of answered those questions for the simple small ones we focused on.
And now the amount of data that you guys are getting is so much larger that you have to start applying these machine learning algorithms
just to deal with it, right?
Yeah, absolutely.
So the project I've been involved in that's just starting to get off the ground,
first light is going to be in a couple of years,
is this project called the Large Synoptic Survey Telescope.
And you can think of this as an overview as it's a 10-year movie of the entire southern sky.
So it's a very wide-field camera that's going to be on a mountaintop in Chile in the Atacama Desert,
one of the driest places on the Earth, so we don't get, don't run into much weather.
And it'll be able to scan the entire night sky every three nights or so.
So get about 100 full sky frames in this movie per year and then do that for a decade.
And the big thing this is going to open up is more of the time domain.
You know, typically astronomers tend to treat the sky as this fixed thing.
There are individual times where we look at specific regions of the sky and see what has changed.
But we don't really have a global survey yet of the time domain of the sky.
And LSST is going to do this on a huge scale.
We're going to have 10 years of data with something like 30-ish terabytes per night of data coming through.
So the full survey size is going to be in the hundreds of petabytes by the end.
So it's really, it's bigger than anything that's been done before.
And it's really forcing astronomers to confront these old tool chains that they've had that don't really scale anymore.
You know, the stuff that you could do, you could sit down, 10 years ago you could sit down on a computer
and download, you know, all of the Sloan Digital Sky Survey and do some sort of local analysis.
I don't know if, even in 10 years, I don't know if we're going to have hundreds of petabyte size hard drives in our laptop, right?
We're going to have to do it a little bit differently.
Yeah, that's really a lot of data.
And the other thing you had said that was interesting is this data is being collected for everyone,
which means that it's not specifically focused on some type of answering some type of questions.
So the techniques and the tools and like the machine learning stuff you have to apply has a greater challenge.
Yeah, it has to be really, really general because this data, like you said, it's collected for everyone.
There's not really specific areas that it's addressing.
It's one of these discovery class missions, similar to the Hubble Space Telescope.
You know, you put it out there and you hope that the things you find are things that you're not going to be able to predict at the moment.
And what that means is that for any particular science case, you're not necessarily going to have the best data.
You know, if you were designing LSST to do one thing, like look for variable stars, you would do it very differently than if you're doing it in general
because you have to balance all these different concerns and different areas of research.
So, for example, going back to variable stars, one of the challenges with LSST is that rather than just observing in the same region of the spectrum every night,
that you'd want to do if you want to, you know, if you want to look at a variable star and see how the brightness changed from one night to the other,
you'd want to take the exact same observation.
But LSST is not taking the exact same observation every night.
It's getting a breadth of different bands throughout the spectrum, everything from infrared to,
in the near ultraviolet.
And what that means is that that's really good for things like determining the redshift of galaxies via machine learning, right?
But it's actually very bad for finding variable stars because now you have to model not only the variability,
but you have to model the spectral variability over the course of time, too.
And it gets much more challenging.
So as this data grows and as the heterogeneity of the data grows, having these sophisticated algorithms,
whether it's machine learning or some sort of forward modeling or some sort of nonparametric modeling,
that's becoming increasingly important.
And it's things that need to happen kind of in real time while you're observing the sky
because we want to be able to alert people within a minute or so.
If something changes on the sky and we find an interesting object,
there's going to be this alert stream so that somebody sitting in a telescope in another part of the world
could point their telescope there right away and catch this interesting phenomenon.
Yeah. Wow. I'm really excited to see what comes out of this. That's a big project.
Yeah, it's going to be huge. It's really going to define the way that we do astronomy over the course of the 2020s.
Yeah, for sure. So let's talk about some of the libraries that you might be using to answer questions here.
So the two major ones, and I guess one is kind of a subset of the other, is AstroPy and AstroML.
Yeah, so AstroPy is actually the big community standard.
And it's been a really cool project to watch and to be involved in because it started a few years ago
where everyone had their own little Python library to do things.
This grew out, I should step back.
Ten years ago, most people were using IDL.
And so the community evolved these sets of routines in IDL to do a lot of the common tasks.
And as more and more people moved over to Python because of – well, I'll go into that later.
But as more and more people moved into Python because of its advantages,
people built a whole bunch of different tools to do different things, and it was this sort of smattering.
And the Space Telescope Science Institute people, the folks behind Hubble,
came together in around 2012, 2011, and said,
we should consolidate all this and create one Uber package to rule them all.
And AstroPy was born, and it's actually accomplished its goal.
Pretty much everyone is using it now.
So that's an incredible package and really, really well done,
and awesome software engineering behind it.
Lots of buy-in from the community.
AstroML is something that I started along the same time.
I didn't have as broad of a vision, but I just wanted to bring together functionality
and examples of doing machine learning specifically for astronomy in Python.
And we actually wrote the package to accompany our book, which is –
it's a Princeton Press book on statistical modeling, machine learning, and et cetera in Python for astronomy.
Yeah, that's great.
And so what kind of things do you cover in your book about –
like what problems are solved or presented or data sets, things like that?
Yeah, in that book, what we do is we walk through a lot of the –
it's meant to be an intro graduate text on statistics and machine learning with astronomers in mind.
So we walk through all the basics of data mining, statistics, machine learning,
all the while using these data sets drawn from astronomy and problem situations that astronomical researchers will run into.
And along the way, we also provide code snippets and provide figures with the full figure source available online
so that if people want to actually use these techniques, they can grab our scripts and start modifying them from there and see where it goes.
So AstroML is what drives that a little bit.
In our next edition of the book, which might happen in the next year or so,
my big task is going to be to incorporate AstroPy
because we actually wrote that book before AstroPy existed.
And so it's already a little bit outdated and I want to make sure that everyone's –
I'm pointing everyone to the tools that are in AstroPy.
Yeah, that makes a lot of sense.
So you created the book and you're like,
we really should make this like a package that people can just use,
the techniques and whatnot, and then now it's a little more mature, right?
Yep.
Do you know of any discoveries that were done as a result of AstroML?
It's been referenced in a lot of papers.
I don't know offhand if there's anything that came exactly from that,
but it's definitely been used for a lot of the incremental building of knowledge over the last few years,
and it's been fun to see that.
Yeah, I'm sure that's really rewarding.
That's awesome.
Another big thing in the astronomy community is forward modeling and Bayesian approaches.
So I alluded to earlier the fact that machine learning is a little bit difficult
because the statistical similarity of the samples is not always a good assumption.
So the way that astronomers tend to get around that is to use forward modeling.
So you have some model for your system based on the physics that you know,
and you can look at the noise properties and the selection effects of your observations to constrain that model,
and then that model will tell you about the data that you observe.
And that tends to work really well in a Bayesian setting.
So a huge push in the last few years in astronomy has been to use tools like Markov Chain Monte Carlo
to do Bayesian analysis and to do these really large high-dimensional models to learn about the data.
So there's one package that's been pretty impactful there is the MC package, E-M-C-E-E.
And that's a package for doing Markov Chain Monte Carlo, doing Bayesian estimation,
written by an astronomer and used for, it's been cited, I think, thousands of times in the astronomy community
because so many people are doing that style of analysis.
Yeah, that's really amazing.
Yeah, I guess the whole how do you solve these prediction problems more quickly is really important.
And Monte Carlo simulations are really good at that.
Mm-hmm.
And particularly the Bayesian approaches.
Machine learning tends to be more of a frequentist approach,
and the Bayesian forward modeling approaches give you some advantage.
And when you have some a priori idea about what's driving your observations,
you can take advantage of that more in a Bayesian context than in a machine learning context.
So you wrote this book, The Statistics, Data Mining, Machine Learning, and Astronomy,
and you survived that process and you decided to come back for more.
And you're just about to finish up a book, right?
Another one.
Yeah, I'm just, I'm finishing one that's, it's an O'Reilly book.
So think, you know, cute little animal on the cover.
You've probably seen that.
What's your animal?
The animal is a Mexican bearded lizard.
Yeah, this one is the Python Data Science Handbook.
So the reason I did this is I, for years I've been approached by people who are, you know,
in research or in tech or something like that.
And they say, hey, I know how to use MATLAB.
I know how to use R.
But I want to learn how to do Python and I want to learn how to analyze data in Python.
And I hadn't found a really good resource to point them to except for kind of collections of videos online.
So I decided to write it.
And it's taken much longer than I thought because life gets in the way.
But we're at the point where it's, I'm doing the final edits right now.
So it should be released pretty soon.
Yeah, that's great.
Congratulations on that.
I'll be sure to link to it as well from the show notes so everyone can find it.
One thing I'm particularly excited about this book, I wrote it all in the form of Jupyter Notebooks
and got the publisher to agree to let me make the Jupyter Notebooks public.
So you can buy the printed version of the book or you'll be able to go on GitHub
and just work through the Jupyter Notebooks for free.
Wow, that really is cool.
Yeah.
Okay.
But you should buy the book to support all the work.
Definitely support the project.
That's cool.
But it's very cool that basically it's a live book, right?
Like if you have the data and you have the code and you can run it, you can explore it.
Yeah.
And we're working on, too, getting a hosted version of it up there on some cloud service.
So you could just basically click and have a live executable textbook at your fingertips.
Yeah, it's interesting.
I think a lot of things are going that way, right?
The days of just a printed book and a zip file are fading, let's say.
Yeah, yeah.
Because there's so much better ways of doing it now.
This portion of Talk Python to me is brought to you by DataSchool.
Have you thought about making a career change into the exciting world of data science, but
don't know how to get started?
DataSchool helps data science beginners like you to analyze interesting data sets and build
machine learning models to predict the future, all using Python.
You don't need a PhD or a background in mathematics, just a keen interest in using data to answer
your questions.
DataSchool has created a data science learning path exclusively for Talk Python listeners.
So visit talkpython.fm/DataSchool to launch your data science career.
DataSchool is run by my friend Kevin Markham, so I know that you're going to get excellent
content.
Check it out at talkpython.fm/DataSchool.
So let's talk a little bit about where you work and what you do, because you are breaking
some rules around how people in academia and scientists work with programming technology and how
programmers are involved.
And I think that's really interesting.
So you're at the University of Washington, but you're at this place called the eScience
Institute, right?
Yeah.
So I'm in the eScience Institute.
I've been here since the beginning of 2014.
And the goal of the eScience Institute is to basically further computational research around
campus.
And so it's existed for a while, but we really got a big boost in 2014 when I came on.
We got this joint grant between New York University, UC Berkeley, and UW.
And we all created some version of this Data Science Institute.
And so it's a five-year grant to support what we're doing.
And the goals are basically to see how we can reshape the culture of academia to take more
advantage of data science tools, to train people better, to provide career paths for software-focused
researchers.
And so, for example, the job that I have right now where I, what I do day-to-day is I spend
a lot of time consulting with researchers around the university, helping them figure out their
data challenges.
I mentor students who kind of have one foot in their home domain, their science, and one
foot in like a data science program.
And I work a lot on maintaining the software that astronomers and other scientists use.
And this is a position that's not really, it's, I feel like, sort of a stepchild in academia because
no one really understands that type of position.
It doesn't fit into the model of graduate student postdoc faculty.
So we have a number of people that are in a similar position to me that are working on
this.
And it's been super fun to see what comes out of this and the kind of novel trainings and
novel approaches to research that we can do.
And particularly fun because it's not only happening at UW, it's happening at NYU and UC Berkeley as
well.
And we can compare notes with those institutions and see how things are going.
That sounds like such a fabulous job.
Yeah, it's good for the time being.
I mean, I'm worrying that I'm peaking early because it's so fun.
I don't know what will come next.
It's all downhill from here.
No, no, that's really cool.
One of the things you pointed out in your PyData talk is that every field is entering a data-rich
era.
So there's all these biologists, sociologists.
You're basically there to help support like the biologists, sociologists, chemists, all the
people who are hitting the limits of how much data they can handle.
Yeah, absolutely.
And the way we're doing this is we have a number of different ways to engage with people on campus.
So one is we have these open office hours.
So just like you used to go see your professor during class, we have office hours that are oriented
towards researchers who, if they have a challenge, they can come talk to one of our people.
And we have people with expertise in everything from statistics, machine learning, to software
engineering, to cloud computing and scalability.
Another thing we do is we run these incubator programs.
So it's sort of designed off these startup incubators that are coming from the Silicon Valley, where
instead of incubating their startup idea, we're incubating their research idea and letting
researchers work shoulder to shoulder with a data scientist who has an expertise that complements
theirs.
And we also have graduate fellowships where students have one foot in their own department,
one foot in e-science and are taking not only, say, astronomy courses, but also database,
machine learning, statistics, computer science courses and getting a credit on their PhD for
that.
Yeah.
What I thought that was really fascinating is, you know, having gone through some part of
a PhD program, just there's so many things you've got to take and learn.
And you're so busy learning your specialty, right?
Like biology, if that's where your PhD was.
That is, it's really hard to be a good data science software type person as well.
Yeah, absolutely.
I think, you know, you said that these guys in these cohorts, they basically get half of
their requirements for their PhD program waived.
Yep.
So that they can focus the other half on sort of complementing this with data science and
programming, right?
Yeah, that's the idea.
And then what comes with that is they get, they have their home department advisor, but
they're also matched with a co-advisor that's more methodological.
And so it leads to not only the student growing a lot, but it leads to some interesting interdisciplinary
collaborations around campus.
And we've had a number of pretty cool grants that have been awarded based on some of these
partnerships.
Yeah, that sounds really quite amazing.
I wish that was around when I was in school.
Yeah, I do too.
I had to pick a lot of this stuff up on my own.
It would have been nice to have something like this.
Yeah.
If anyone out there is listening and they're maybe in a position where they're like, oh,
this is interesting.
How do we do this?
Right?
Like another thing that I thought you pointed out was really interesting is it's in a beautiful
location.
And you said that that was really important.
Yeah, yeah, definitely.
So we have this data science studio that it's an old library branch location on campus.
And we're on the sixth floor of this tower with where we have the whole floor with 360 degree
views looking out over Mount Rainier and the Olympic Mountains and things like this.
And it's important not just so that I can have an awesome view while I'm writing code,
but it's important because we want people around campus to interact with each other.
And so we want to be a place where people would like to come and just hang out.
So it's getting back to the, we call it the water cooler effect.
The people who are around in the 60s and 70s working on, working on computationally intensive
science, talk about the days when everyone would go to the mainframe on campus and you'd
be sitting there waiting to put your punch cards in.
And, you know, a hydrologist would be talking to an astrophysicist and finding out that they're,
they're solving the same equations with their programs.
So they would, you know, have that sort of talk and, and as the campus moved towards desktop
oriented computing, those sorts of opportunities went away.
And I think we're, we're better off if we can have that, that sort of connection.
So one of the cool things about our space here is it's a, it's a space that's open to anyone on
campus for, for just hanging out and working, but also for scheduling meetings.
So we have people from all different departments that are scheduling their group meetings here,
coming in, hanging out, having coffee, and then meeting someone from the other side of campus
who's solving the same differential equation in their completely different field.
Yeah, that's great.
Yeah.
Very, very nice.
Like I said, I wish that existed when I was in school.
All right.
So we're kind of getting near the end of the show and I have a couple of questions
that I wanted to run by you.
One is kind of almost metaphysical.
So I just heard the other day that there was some study done that there was, we underestimated
the number of galaxies by like a factor of 20 or something amazing, right?
Yeah.
That was really interesting that, you know, already there's so many galaxies out there and every
galaxy has, you know, so many stars and so many planets.
What do you think the chances that there's intelligent life outside the earth?
Not necessarily like there's people visiting the earth, right?
Just out there somewhere.
We will never meet them.
That's hard to say.
Um, you know, at this point, so, so one thing at UW, we have this, astrobiology group,
which sounds like a funny area of study because what are they studying?
Right.
But, but they're working on really interesting things, combining, what we know about biology,
about geophysics, about planetary astronomy, and looking for locations around the universe
where, life might exist.
And so they study extremophiles around here, like organisms that live on, on deep sea vents
and, in acidic boiling water environments, things like that.
Um, but one thing that's come out of that group is, this, this notion that, simple
life, you know, microbial life, probably could exist just about anywhere.
And I, I tend to think that, you know, I, I would be, I would be pretty surprised if we
don't find some sort of microbial life elsewhere in our, our solar system.
Uh, but the other thing to come out of that as we study the, the dynamics of planets and
things like this is that there are a lot of things about earth in particular that make
it very special and a lot of, coincidences that, that are, that would be hard to duplicate.
You know, things like the fact that Jupyter exists, keeps us from having a large number
of asteroid impacts on earth.
That's kind of a big shield.
And, you know, that, that asteroid impacts, as, as we know from geological paleontological
history is, they can be pretty bad for life on earth.
So, so the type of stability that we have on earth, particularly over the last, tens,
hundreds of millions of years, I, I suspect that's rather rare.
And that, that makes me think that intelligent life might be rather rare.
You know, I think, I think the seas of Europa, when we get, when we get something that can
burrow through the ice and look down there, I really hope there's something swimming around
down there because it would be pretty cool.
It'd be very cool.
I hope I, I hope I get to see that someday.
That'd be awesome.
All right.
So another one, my wife's a professor here at Portland State University.
So I hang out with some other, her colleagues and stuff.
And one of her colleagues had this student, she's teaching like a numerical methods for
partial differential equations, you know, something like that.
Her, her, you know, she's using Python and a lot of things like NumPy and so on in her
class for the computation.
And one of her students came and said, Hey, I just, I know MATLAB.
Can I just use MATLAB?
Why do I need to learn this Python thing?
What would you tell that student if you got that question?
Yeah, that's a good question.
Um, so number one, I think, use the tool that's most effective for your research.
And for example, if you, if there are programs in MATLAB that, don't exist in Python and,
and they were, they're required from your, for your research, there's no reason to learn a
new tool just because it's a new tool.
But on the other hand, there, there's some distinct advantages to Python.
And I alluded to this earlier when I talked about the field of astronomy shifting from 90% IDL over
the last 10 years to probably 90% Python.
And the advantages, that I see are number one, it's openness, right?
It's, it's open and it's free.
So that means, one thing that has come up with IDL, is there, there are site licenses
required for every instance that you run.
You know, it's, it's a pay to play type of operating system or type of, interpreter.
And so when people started running, running parallelized jobs, taking advantage of all the
computers in the department, there were times where a grad student would start a job and it
would use all the site licenses for the entire department and research in the department ground
to a halt, right?
Um, so you don't have that problem in Python because there's no site licenses with Python.
So Python can be cheaper to use.
Um, the other thing about it is that, to, to serve students well, you know, the, the
number of academic jobs versus the number of, undergrad degrees or PhDs granted is extremely
small.
So most of our students are going to be going out into the world and working somewhere other
than an academic department.
And, people in, in the outside world and the tech world, they're much more excited about
someone with Python chops than someone with IDL chops or MATLAB chops, just because that's,
you know, the way the world has gone.
So that, that's another good reason to move to Python.
Um, the other thing that I love about Python is the culture of open source.
So particularly now, 10 years ago, it was different, but now, just about anything you want to
do in Python, you can go out there and there's somebody who has made an open source library
for it, put it on a place like a GitHub or Bitbucket and, made it available.
And, and often these libraries are really, really well done.
There's just been this culture of, of well-designed open source, particularly in the scientific Python
community.
And it means that, you know, you can, you can do a, an amazing number of things just
out of the box with Python and the scientific installation.
Yeah.
What do you think that means for reproducibility?
Like I want to store this thing, of code, the interpreter, maybe even like a Linux Docker
image of the thing that I use to generate my paper.
Yeah, that's huge.
And, you know, it comes back to the, in the beginning, I was telling you how I got into
the Python open source world.
I realized that there was, I thought it was ridiculous that I'd spent this whole summer
building this tool that then no one was going to be able to use.
And, the tools in Python for enabling that sort of reproducibility, even having like an
executable paper are, are huge.
And I think it's really helping science drive, drive itself forward because we don't need
to reinvent the wheel every time we do a new study.
All right, Jake, I think that we have to leave it there for the topics, but I do have two final
questions for you before I let you go.
Okay.
I just saw on PyPI that we passed 90,000 distinct packages.
There's so many amazing things you can just pip install.
And, in your field, you probably get exposed to really interesting things that maybe not
everybody knows about, tell us about like one of your favorite PyPI packages that you might
recommend.
Well, I mentioned this earlier, but the MC, for Markov chain Monte Carlo, I think that's
just an incredible package and allows you to do so much as far as, Beijing modeling.
I could talk about one of my packages, but yeah, well, you know, it's your own packages are not
off limits.
Like Astro ML is all right, you know?
Yeah.
So one, one that I'm working on recently, which is a lot of fun is this, Altair package.
And what, what it is, is a Python interface to Vega light.
And Vega light is a, a visualization grammar aimed at statistical visualizations, that
basically, outputs interactive, JavaScript plots.
And so we've been, we've been writing this Python wrapper and trying to make a nice API to,
to create Vega light grammars and Vega light, visualizations.
I'm pretty excited about this because, you know, there's, there's so many options for plotting
out there right now.
There's, you know, there's matplotlib, there's bokeh, there's plotly, there's hollow views, there's
ggplot wrapper for Python.
There's, I'm going to miss something and someone's going to get mad at me.
Uh, seaborn, you know, things like that.
Um, but the, the one thing that, that Altair, that's interesting about Altair is that it interfaces
to this Vega light grammar.
And that grammar, I think has the, has the possibility of becoming sort of a lingua franca between these
various visualization packages.
And the Vega light, if you, if you've heard of D3, which is driving a lot of interactive
visualization on the web, Vega and Vega light are come out of the same research group.
So it's people who really know what they're doing as far as, as visualization design.
Yeah, that's cool.
That's a great pedigree.
Yeah.
So, you know, and I, now I get to write the, the Python classes that output this stuff and
it's pretty fun.
Great.
What's the package called?
It's called Altair, A-L-T-A-I-R.
All right.
Awesome.
All right.
And when you write some Python code, what editor do you use?
Uh, I, I go back and forth these days between Emacs and Atom.
Um, I actually like the Emacs key bindings, but I like the way that Atom, arranges an
entire project and lets you, lets you see all the files.
Yeah.
Those are both nice.
Cool.
All right.
So any final call to action?
I heard you had an announcement about your, PhD cohorts, like 50, 50 program.
Yeah.
So, we, we just put out this announcement for our 20, 2017 PhD fellowships or postdoctoral
fellowships.
So, this is looking for people who have recently finished their PhD who are interested
in continuing research in their own field, but also adding some sort of computational or
data science element to it.
And it's similar to the graduate program I described earlier.
You have, you apply to have one foot in your domain department, one foot in the e-science
institute with, with two advisors, one from the domain and one in a methodological area.
And we have just, a great set of postdocs here who are doing some, really phenomenal
work with that.
And it, so if you're, if you're, graduating PhD students and this e-science institute or data
science stuff sounds good.
Um, I'd encourage you to apply to applications or do sometime mid January.
All right.
That's plenty of time to get them in there.
Cool.
Yep.
All right.
And when's your book coming out?
Um, probably, probably January.
I think, I don't know at this point.
It depends how, how, how quickly I get this, this corrected manuscript back to them.
Yeah, of course.
I saw that you can get like an early access version of it, right?
Yeah.
The early access is there.
So if you want to take a look at the pre-release right now, you can, you can go buy it and they'll
update you to the released version when it comes out.
All right.
Sounds great.
So Jake, it's been super interesting talking about astronomy with you.
Thanks for coming on the show and sharing your story.
Yeah.
Thanks for having me.
You bet.
Bye-bye.
This has been another episode of talk Python to me.
Today's guest has been Jake Vanderplast.
And this episode has been sponsored by GoCD and data school.
Thank you both for supporting the show.
GoCD is the on-premise open source continuous delivery server.
Want to improve your deployment workflow, but keep your code and builds in-house.
Check out GoCD at talkpython.fm/gocd and take control over your process.
Data school is here to help you become effective with Python's data science tools quickly.
Skip years at the university.
Check out the talk Python to me learning path at talkpython.fm/data school.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course Python Jumpstart by building 10 apps at talkpython.fm/course to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my write Pythonic code course at talkpython.fm/pythonic.
You can find the links from this episode at talkpython.fm/episodes slash show slash 81.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
Stay tuned.
I'll be right back.
I'll pass the mic back to who rocked it best.
you
Thank you.