WEBVTT

00:00:00.001 --> 00:00:02.840
It's been an amazing year for Python and data science.

00:00:02.840 --> 00:00:07.420
It's time to look back at the major headlines and take stock in what we've done as a community.

00:00:07.420 --> 00:00:09.980
I've teamed up with the Partially Derivative podcast,

00:00:09.980 --> 00:00:15.500
and we're running down the top 10 data science stories of 2016 in this joint episode.

00:00:15.500 --> 00:00:21.280
This is Talk Python To Me, episode 91, recorded November 18, 2016.

00:00:21.280 --> 00:00:49.740
Welcome to Talk Python To Me, a weekly podcast on Python,

00:00:49.740 --> 00:00:52.800
the language, the libraries, the ecosystem, and the personalities.

00:00:52.800 --> 00:00:54.920
This is your host, Michael Kennedy.

00:00:54.920 --> 00:00:56.920
Follow me on Twitter, where I'm @mkennedy.

00:00:56.920 --> 00:01:00.800
Keep up with the show and listen to past episodes at talkpython.fm,

00:01:00.800 --> 00:01:03.320
and follow the show on Twitter via at Talk Python.

00:01:03.320 --> 00:01:07.840
This episode has been sponsored by Rollbar and Continuum Analytics.

00:01:07.840 --> 00:01:11.080
I want to say a special thank you to the folks at Continuum Analytics,

00:01:11.080 --> 00:01:15.600
you know, the Anaconda distribution people, for joining Talk Python as a sponsor.

00:01:15.600 --> 00:01:19.280
Thank them both for supporting the show by checking out what they have to offer

00:01:19.280 --> 00:01:20.300
during their segments.

00:01:20.300 --> 00:01:23.460
Jonathan, welcome back to Talk Python.

00:01:23.460 --> 00:01:24.920
Hey, thanks so much for having me.

00:01:24.920 --> 00:01:25.940
I'm really excited to be here.

00:01:25.940 --> 00:01:29.480
I'm so excited to have you back, because every time we do a show together,

00:01:29.480 --> 00:01:30.520
I have a great time.

00:01:30.520 --> 00:01:33.940
People seem to love it, and I know what we have on deck today.

00:01:33.940 --> 00:01:35.300
People are going to love the show.

00:01:35.300 --> 00:01:36.140
It's going to be really fun.

00:01:36.140 --> 00:01:37.200
Yeah, I think so, too.

00:01:37.200 --> 00:01:40.500
It's just, I love, I'm so glad, it was fun doing this last year,

00:01:40.500 --> 00:01:41.520
and I'm glad we're doing it again.

00:01:41.520 --> 00:01:45.620
It's just kind of cool to have an opportunity to look back over the past 12 months,

00:01:45.620 --> 00:01:49.460
and really, there's just so much news and so much data stuff that comes out over 12 months.

00:01:49.460 --> 00:01:51.220
It's an industry that's moving so fast.

00:01:51.220 --> 00:01:53.900
Just to have a chance to reflect a little bit is kind of cool.

00:01:53.900 --> 00:01:56.560
I remembered some things that I'd forgotten about the past year.

00:01:56.760 --> 00:01:57.720
Yeah, me too.

00:01:57.720 --> 00:01:58.880
Like, for example, Tay.

00:01:58.880 --> 00:02:00.320
I remember Tay.

00:02:00.320 --> 00:02:02.700
And I just, just because we brought it up.

00:02:02.700 --> 00:02:04.720
So we'll talk about Tay, the bot, later.

00:02:04.720 --> 00:02:06.380
Of course.

00:02:06.380 --> 00:02:08.440
Yeah, what have you been up to the last year?

00:02:08.440 --> 00:02:10.440
Like, you're still doing Partially Derivative.

00:02:10.440 --> 00:02:13.360
You got your business properly, your data science business going.

00:02:13.360 --> 00:02:14.540
Yeah, yeah, absolutely.

00:02:14.540 --> 00:02:17.700
So Partially Derivative, our podcast about data science,

00:02:17.700 --> 00:02:19.440
at least ostensibly about data science,

00:02:19.440 --> 00:02:22.740
kind of about drinking, largely about screwing around, is still there.

00:02:23.020 --> 00:02:26.960
So everybody who's interested in a little data nerdiness can go check that out.

00:02:26.960 --> 00:02:29.540
Yeah, and then the business has been going well.

00:02:29.540 --> 00:02:32.000
We're doing more projects, which is cool,

00:02:32.000 --> 00:02:36.480
kind of moving slowly and slowly away from the kind of startup mentality,

00:02:36.480 --> 00:02:38.480
which I think has been healthier for everybody.

00:02:38.480 --> 00:02:40.480
And been doing some really cool research projects,

00:02:40.480 --> 00:02:41.840
especially some natural language stuff.

00:02:41.840 --> 00:02:43.660
So yeah, it's been a really good year.

00:02:43.660 --> 00:02:44.220
How about you, man?

00:02:44.220 --> 00:02:45.840
How's your year in review?

00:02:45.840 --> 00:02:47.780
My year in review is amazing.

00:02:47.780 --> 00:02:51.100
You know, I went independent in February.

00:02:51.100 --> 00:02:55.220
I've been running this podcast and my courses as my primary business.

00:02:55.220 --> 00:02:58.120
And it has just been like a dream come true.

00:02:58.120 --> 00:02:58.560
It's amazing.

00:02:58.560 --> 00:03:00.060
So it's been a fabulous year.

00:03:00.060 --> 00:03:01.360
Yeah, I'm not surprised.

00:03:01.360 --> 00:03:03.240
The Python coursework is really awesome.

00:03:03.240 --> 00:03:06.840
I don't know how much you plug your own stuff on your show.

00:03:06.840 --> 00:03:08.680
So I'll just do it for you, for all of your listeners.

00:03:08.680 --> 00:03:12.320
If you're interested in learning Python and you haven't taken the courses yet,

00:03:12.320 --> 00:03:13.760
you should go do it right now.

00:03:13.760 --> 00:03:14.600
It's the best way to learn.

00:03:14.600 --> 00:03:15.400
Thank you so much.

00:03:15.400 --> 00:03:16.960
Yeah, it's really fun to do them.

00:03:17.460 --> 00:03:18.060
All right.

00:03:18.060 --> 00:03:21.940
So I think in our tradition, because it's happened one time,

00:03:21.940 --> 00:03:28.600
what we need to do at the end of the year is look through all of the interesting news stories

00:03:28.600 --> 00:03:34.640
that have to do with big data, with data science, with machine learning, with Python,

00:03:34.640 --> 00:03:37.160
and kind of do our take on them.

00:03:37.480 --> 00:03:43.300
So let's start at something that's been in the news a lot this year, the White House.

00:03:43.300 --> 00:03:43.760
Yeah.

00:03:43.760 --> 00:03:48.600
So very interestingly, so, you know, we've been maybe 18 months, a couple years,

00:03:48.600 --> 00:03:53.920
into the tenure of the country's, the U.S., our first chief data scientist.

00:03:53.920 --> 00:03:56.200
So DJ Patil came into the White House.

00:03:56.200 --> 00:04:01.200
We've had a CTO, a chief technical officer, in the White House for a little bit longer.

00:04:01.300 --> 00:04:07.280
And that group of folks has been really out in front on the way that we think about data

00:04:07.280 --> 00:04:10.140
and society, which has been a fascinating conversation.

00:04:10.140 --> 00:04:14.500
I think there's been like little trickles of information about what it means to do machine

00:04:14.500 --> 00:04:20.720
learning in an ethical way, how we avoid kind of algorithmic or bias in our models and our

00:04:20.720 --> 00:04:21.640
machine learning models.

00:04:21.640 --> 00:04:27.220
They did this report about how, as a society, we should think about the impacts of artificial

00:04:27.220 --> 00:04:31.600
intelligence, of machine learning, and big data to make sure that we're not taking some

00:04:31.600 --> 00:04:36.060
of the bias that's inherent in our society and therefore is inherent in the data and inherent

00:04:36.060 --> 00:04:39.660
in our models and is perpetuating some of that over time through technology.

00:04:39.660 --> 00:04:44.000
It was really a cool position for an administration to take, you know, sometimes government's a

00:04:44.000 --> 00:04:46.760
little bit behind technology or the technology industry.

00:04:46.760 --> 00:04:50.160
But in this case, I felt like they were really out in front, like kind of driving the conversation.

00:04:50.160 --> 00:04:51.480
So it was a cool story.

00:04:51.480 --> 00:04:52.820
Yeah, it is a very cool story.

00:04:52.820 --> 00:04:54.980
And of course, all the stories will be linked in the show notes.

00:04:55.080 --> 00:04:57.420
You can probably just flip over and you're a podcast player and click them.

00:04:57.420 --> 00:05:05.480
But what I find really interesting about this is we as technologists see technological progression

00:05:05.480 --> 00:05:09.140
almost always as rainbows and unicorns, right?

00:05:09.140 --> 00:05:17.960
It's like every new device that comes along that connects us or enables something, this is

00:05:17.960 --> 00:05:19.600
uniformly good, right?

00:05:19.600 --> 00:05:21.880
But that's not always the case, right?

00:05:21.880 --> 00:05:25.980
And actually, we'll get into some interesting twists later in some of those specific areas,

00:05:25.980 --> 00:05:27.600
how this can go wrong.

00:05:27.600 --> 00:05:31.940
But basically, they said, look, there's so many opportunities for data science and technology

00:05:31.940 --> 00:05:33.280
to do good for people.

00:05:33.280 --> 00:05:39.260
But at the same time, these like, let's take data science, the algorithms coming out of the

00:05:39.260 --> 00:05:44.180
machine learning could have a bias and not necessarily even a conscious bias, right?

00:05:44.440 --> 00:05:46.580
Yeah, that's actually one of the most interesting things about it.

00:05:46.580 --> 00:05:51.180
Because I think still, statistically speaking, and I think maybe even trending in this direction,

00:05:51.180 --> 00:05:57.780
the technology community and the data science community is still largely male and largely white.

00:05:57.780 --> 00:06:05.500
And so the interesting takeaway, I think, from a lot of these discussions about the way that bias is kind of infecting our technology

00:06:05.500 --> 00:06:12.900
or may not necessarily be this steady march to progress the way that we view it is because people often don't see

00:06:12.900 --> 00:06:16.840
or have a difficult time understanding the perspectives of people who aren't like them,

00:06:16.920 --> 00:06:18.580
which is kind of an obvious statement.

00:06:18.580 --> 00:06:24.320
But when we're encoding our worldview effectively into the technologies that we're developing,

00:06:24.320 --> 00:06:29.280
then we may not see the consequences of that technology.

00:06:29.280 --> 00:06:35.860
We're not intentionally encouraging racism or intentionally kind of encoding that institutional bias.

00:06:35.860 --> 00:06:41.960
But it's inevitable that that's going to be a byproduct of a community that's relatively homogenous still.

00:06:41.960 --> 00:06:45.400
And so I think it's just good that it's something that we're discussing.

00:06:45.400 --> 00:06:47.880
I think the only way to get past that is to have more awareness of it,

00:06:47.880 --> 00:06:50.880
and then ideally for more diversity in the technology industry.

00:06:50.880 --> 00:06:53.800
But that's sort of a separate and longer conversation.

00:06:53.800 --> 00:07:01.100
But yeah, so I think, again, it's cool that such a high-profile group of people who are leaders in the technology community

00:07:01.100 --> 00:07:03.600
took it upon themselves to initiate this conversation.

00:07:03.600 --> 00:07:04.280
Yeah, absolutely.

00:07:04.280 --> 00:07:08.100
So there's a report they released about this, and it's not all negative.

00:07:08.100 --> 00:07:09.280
It seems pretty balanced.

00:07:09.280 --> 00:07:11.440
Like, look, there's all these great things that we're going to be able to do,

00:07:11.440 --> 00:07:16.400
but there's also these safety checks we need to make sure are in the system.

00:07:16.400 --> 00:07:21.240
And at the end, they also put a little note that said they encourage you to follow along this summer and spring

00:07:21.240 --> 00:07:25.360
where they're hosting a series of public workshops on artificial intelligence and machine learning.

00:07:25.360 --> 00:07:30.940
Like, when did you think the White House would host workshops on artificial intelligence and machine learning?

00:07:31.940 --> 00:07:33.020
Yeah, it really is.

00:07:33.020 --> 00:07:34.660
It's a new world.

00:07:34.660 --> 00:07:35.620
It's pretty exciting.

00:07:35.620 --> 00:07:36.120
And I agree.

00:07:36.120 --> 00:07:36.920
That's good to point out.

00:07:36.920 --> 00:07:42.440
Like, we're having this – I feel like I've been framing this as if it was like a finger wagging or like an admonishment.

00:07:42.440 --> 00:07:43.600
And it's really not.

00:07:43.600 --> 00:07:46.820
It's actually – there's so much potential for these amazing technologies.

00:07:46.820 --> 00:07:52.760
Let's just make sure we're doing it in a way that it includes the entire society and not just the single viewpoint.

00:07:53.020 --> 00:07:53.560
Yeah, absolutely.

00:07:53.560 --> 00:07:54.000
Absolutely.

00:07:54.000 --> 00:07:55.000
All right.

00:07:55.000 --> 00:07:57.980
The next one up is this research paper.

00:07:57.980 --> 00:07:59.120
It's hard to call it paper.

00:07:59.120 --> 00:07:59.780
Digital paper.

00:07:59.780 --> 00:08:07.020
This research article called Social Bots Distort the 2016 U.S. Presidential Election Online Discussion.

00:08:07.020 --> 00:08:14.560
And if that sounds like a sort of tough title to say as an academic thing, it's because this is an academic paper.

00:08:15.020 --> 00:08:18.240
It's by Alessandro Bessi and Emilio Ferrara.

00:08:18.240 --> 00:08:21.840
And these are two professors or postdocs.

00:08:21.840 --> 00:08:23.500
I don't remember exactly their position.

00:08:23.500 --> 00:08:26.420
But in Southern California, it's some local universities there.

00:08:26.420 --> 00:08:30.080
And there's this place called firstmonday.org.

00:08:30.080 --> 00:08:33.240
And it's a peer-reviewed journal on the internet.

00:08:33.240 --> 00:08:36.440
And it's kind of a double meaning there.

00:08:36.440 --> 00:08:39.360
So it's a peer-reviewed journal that you can get for free on the internet.

00:08:39.360 --> 00:08:42.400
But it's a peer-reviewed journal about research on the internet.

00:08:42.400 --> 00:08:51.880
So it's pretty cool they've got a bunch of stuff about, like, how Reddit behaves and other sorts of things that we would probably care about, purely academic research.

00:08:51.880 --> 00:08:53.960
And it's super interesting what they found.

00:08:53.960 --> 00:09:00.360
So these guys, they created this thing called Bot or Not, which is a machine learning framework.

00:09:00.360 --> 00:09:06.600
And they basically set up a bunch of hashtags and a few keyword searches.

00:09:06.600 --> 00:09:11.460
And they said, we're going to monitor the Twitter firehose for these things, right?

00:09:11.460 --> 00:09:17.580
That's the real-time data flow coming out of Twitter for those particular things, which already is actually a challenge.

00:09:17.580 --> 00:09:22.060
And they talk about the technology of, like, consuming that much data, which is pretty interesting.

00:09:22.060 --> 00:09:23.680
It's written in Python.

00:09:23.680 --> 00:09:26.580
And you can actually get the Bot or Not thing on GitHub.

00:09:26.580 --> 00:09:35.740
And they say it has an accuracy of determining whether a thing, like a social thing, is a bot or is human at 95% or better.

00:09:35.740 --> 00:09:36.220
Whoa.

00:09:36.460 --> 00:09:37.400
That's pretty solid, right?

00:09:37.400 --> 00:09:38.340
That's kind of amazing.

00:09:38.340 --> 00:09:40.880
Yeah, that's a difficult distinction to make, I think, a lot of the time.

00:09:40.880 --> 00:09:41.320
That's cool.

00:09:41.320 --> 00:09:45.340
Yeah, they said they'd taken over 1,000 pieces of data to make that.

00:09:45.340 --> 00:09:46.280
Dimensions, I guess?

00:09:46.280 --> 00:09:48.780
To consider that, right?

00:09:48.860 --> 00:09:55.640
It was interesting to see the way that they – dimensions or the features that they built in order to make that model predictive.

00:09:55.640 --> 00:10:02.740
The kind of the behavioral things, the signatures that distinguish between a real-life person who just tweets a lot and a bot.

00:10:02.740 --> 00:10:06.300
It's interesting because it's – there's a lot of things that they were able to sort of distill down.

00:10:06.300 --> 00:10:07.380
But it is very interesting.

00:10:07.380 --> 00:10:09.180
And we don't want to go too much into the details.

00:10:09.180 --> 00:10:10.400
But they really write it up.

00:10:10.440 --> 00:10:11.860
It's like a 30-page paper.

00:10:11.860 --> 00:10:12.620
So that's cool.

00:10:12.620 --> 00:10:19.060
And kind of like we were just discussing, they have a similar take for social media as we were talking about with big data.

00:10:19.060 --> 00:10:24.300
And they say social media has been extensively praised for increasing democratic discussion, right?

00:10:24.300 --> 00:10:27.780
You think of the Arab Spring, for example, and things like that.

00:10:28.460 --> 00:10:37.320
But they say that you can also take this social media and use it for your own purposes, good or evil, right?

00:10:37.320 --> 00:10:48.120
So you can exploit these social networks to change the public discussion, to change the perception of political entities, or even try to affect the outcome of political elections.

00:10:48.120 --> 00:11:05.520
Yeah, and this is something – I mean, not to get us too off topic, but I have a little bit of a research background in understanding how highly motivated, like hyperactive users of a social media platform can basically form a group that's just large enough to seem just too big to really recognize.

00:11:05.520 --> 00:11:11.240
But if they act together, it's like let's say you have 1,000 people that are just really hyperactive and tweeting the same thing.

00:11:11.240 --> 00:11:12.240
Real people are bots.

00:11:12.240 --> 00:11:20.800
Those tweets or the hashtags that they promote or the content that they circulate seems as if it's gaining really widespread organic traction.

00:11:20.800 --> 00:11:24.480
And so you can effectively, like, force a narrative onto a social media.

00:11:24.480 --> 00:11:28.560
You can, like, hijack the mechanics of a social network using some of these techniques.

00:11:28.560 --> 00:11:33.780
And we're seeing it increasingly from groups that have some kind of ideological agenda.

00:11:33.780 --> 00:11:44.400
Everything from terrorist groups all the way to political organizations all the way to maybe foreign states that start with R trying to influence the outcome of the U.S. elections.

00:11:44.400 --> 00:11:50.760
Like, it's a really – it's both – like, from an academic or intellectual perspective, it's kind of fascinating.

00:11:50.760 --> 00:11:54.440
But at the same time, also a little ominous.

00:11:54.440 --> 00:11:55.220
It definitely is.

00:11:55.220 --> 00:11:59.600
Like, I really love social media, and I think it is a positive thing generally.

00:11:59.840 --> 00:12:03.700
But there are definitely examples, and this is one – I'm going to give you some stats here in a second.

00:12:03.700 --> 00:12:11.660
But another real clear example, which is not something trying to influence this, but it's just, you know, speaking of algorithms and unintended consequences,

00:12:11.660 --> 00:12:19.600
like Facebook and people living in bubbles and how they perceived the news this year and all those sorts of things are very interesting to study.

00:12:19.600 --> 00:12:19.920
Yeah.

00:12:20.040 --> 00:12:26.100
And in fact, at the time of this recording, it's not yet released, but probably by the time this airs, we will have published it.

00:12:26.100 --> 00:12:40.020
We've actually done some research that shows – that we think shows that the – when people in a particular community on Facebook share URLs from these fake news domains or hyper-partisan domains more often,

00:12:40.020 --> 00:12:43.900
it actually has a direct impact on the amount of bias that we see in their language.

00:12:44.380 --> 00:12:53.440
So that kind of – that loop where the community gets more biased, the news sites get more hyper-partisan or more extreme, and then the community gets more biased again,

00:12:53.440 --> 00:12:57.100
like that kind of feedback loop seems to be a real thing.

00:12:57.100 --> 00:13:05.440
And it's kind of a – how do you pull people back from an environment where they're literally not living in the same reality that you are, which is kind of strange.

00:13:05.440 --> 00:13:06.360
It's very strange.

00:13:06.560 --> 00:13:09.620
You know, especially when it comes to things that are outside their personal experience, right?

00:13:09.620 --> 00:13:15.900
So even though, like, we all have the same kind of jobs, we all, like, love our families and our kids, our day-to-day lives are mostly the same.

00:13:15.900 --> 00:13:24.660
But we can still get kind of whipped up into a frenzy about these things that are kind of at arm's length from us that get talked a lot about – that get talked about in kind of political campaigns.

00:13:24.660 --> 00:13:31.260
It's – it'll be – it's interesting that – it'll be interesting to see how these networks start to combat it now that they're aware of it.

00:13:31.260 --> 00:13:31.700
Yeah, yeah.

00:13:31.700 --> 00:13:32.800
I'm looking forward to that.

00:13:32.800 --> 00:13:33.460
I totally am.

00:13:33.460 --> 00:13:35.020
I'm looking forward to your research as well.

00:13:35.020 --> 00:13:35.680
That's cool.

00:13:36.080 --> 00:13:44.240
I feel like if the Twilight Zone were still a thing, the movie from the 50s and 60s, you know, there's a couple of episodes I could get from our news here.

00:13:44.240 --> 00:13:45.520
So are you ready for the conclusion?

00:13:45.520 --> 00:13:47.300
Did these bots have an effect?

00:13:47.300 --> 00:13:57.900
After all this research and a bunch of data analysis that they laid out, they said that bots are pervasively present and active in the online political discussion in the 2016 election.

00:13:58.640 --> 00:14:08.860
And they estimate 400,000 bots engaged in the discussion responsible for 3.8 million tweets or one-fifth of the entire political presidential conversation.

00:14:10.460 --> 00:14:10.900
Whoa.

00:14:10.900 --> 00:14:14.320
That's a huge percentage of the – it's weird.

00:14:14.320 --> 00:14:18.540
We think about it like, oh, this great forum for public discourse, but it's actually bots talking to each other.

00:14:18.540 --> 00:14:18.900
Yes, exactly.

00:14:18.900 --> 00:14:20.760
Like arguing amongst themselves.

00:14:20.760 --> 00:14:21.240
Yeah.

00:14:21.240 --> 00:14:22.360
It's probably true.

00:14:22.360 --> 00:14:23.620
I bet they did fight with each other.

00:14:25.120 --> 00:14:28.240
Actually, that was – I'm disappointed with myself because I won't remember the name.

00:14:28.240 --> 00:14:42.540
But if people listening to this Google kind of social activist bot fighting with trolls or, you know, something along those lines, there were a couple really interesting stories about people who would use – who wrote bots, kind of more activist artist types who wrote Twitter bots.

00:14:42.540 --> 00:14:48.560
And wrote them in a way that would start kind of banal online fights with people.

00:14:48.560 --> 00:14:54.180
So they'd find kind of like far-right trolls and they would say things like, I think your opinion is wrong.

00:14:54.180 --> 00:14:56.060
Like your argument isn't even valid.

00:14:56.060 --> 00:14:58.420
And they would just like content-free argument.

00:14:58.420 --> 00:15:00.060
But people would engage with them for hours.

00:15:00.060 --> 00:15:03.320
Like they would just fight with this bot for hours at a time.

00:15:03.320 --> 00:15:04.260
Oh, that's mean.

00:15:04.260 --> 00:15:05.280
Yeah.

00:15:05.280 --> 00:15:06.640
But I love it.

00:15:06.640 --> 00:15:09.440
Yeah.

00:15:09.440 --> 00:15:10.960
It is pretty fascinating.

00:15:10.960 --> 00:15:17.620
I think – and something – I mean, you kind of mentioned it, but I feel like it's worth reminding all of the listeners because everybody – many of your listeners will be developers.

00:15:17.620 --> 00:15:20.620
The code for this is released on GitHub.

00:15:20.620 --> 00:15:22.920
And there's an API that you can ping.

00:15:22.920 --> 00:15:31.940
So if you're doing any kind of research or if you're building an application that engages with Twitter, you can pretty easily check to see whether or not an account is a bot.

00:15:31.940 --> 00:15:38.700
So if that's something that is useful to you, like I really credit the researchers for making this available to the general public.

00:15:38.700 --> 00:15:39.540
That's a really cool service.

00:15:39.540 --> 00:15:39.760
Yeah.

00:15:39.760 --> 00:15:40.800
That's for sure.

00:15:40.800 --> 00:15:41.160
It's cool.

00:15:41.160 --> 00:15:41.660
It's on GitHub.

00:15:41.660 --> 00:15:42.000
It's cool.

00:15:42.000 --> 00:15:44.460
It's in Python and easily accessible to everyone listening here.

00:15:44.920 --> 00:15:49.940
So we have one more thing about the election and then I promise we'll move on, right?

00:15:49.940 --> 00:15:51.580
What's the deal?

00:15:51.580 --> 00:15:52.240
Absolutely.

00:15:52.240 --> 00:15:56.400
I mean, you know, elections are a big deal in the years that they happen.

00:15:56.400 --> 00:15:57.260
So, you know, fair.

00:15:57.260 --> 00:16:08.140
Well, and I think this one is especially interesting because it broke a lot of norms and the prediction just failed across the board in so many ways, in the media, in the pollsters, and so on.

00:16:08.140 --> 00:16:11.560
And so I think it's a looking back to see what do we need to fix?

00:16:11.560 --> 00:16:12.340
What do we need to change?

00:16:12.340 --> 00:16:13.740
So what's this next item about?

00:16:13.740 --> 00:16:14.000
Yeah.

00:16:14.000 --> 00:16:16.040
Well, it's about just that.

00:16:16.040 --> 00:16:23.460
So as anybody who was a poll watcher during the election cycle, which I think is most of the population these days.

00:16:23.460 --> 00:16:25.200
Because of the bots, you couldn't escape it.

00:16:25.200 --> 00:16:27.160
Yeah, exactly.

00:16:27.160 --> 00:16:29.520
Every time the polls changed, the bots wanted to talk about it.

00:16:29.520 --> 00:16:30.600
The polls were rigged.

00:16:30.600 --> 00:16:32.240
No, the polls show my candidate in the lead.

00:16:32.240 --> 00:16:33.780
Horse race, horse race, horse race.

00:16:34.020 --> 00:16:40.720
And what's interesting is that the vast majority of those polls and predictions were wrong, like dead wrong.

00:16:40.720 --> 00:16:53.800
And so there's been this kind of like, I don't know, huge reconciling in the data science community and kind of anybody who does this predictive forecasting on a number of things.

00:16:53.800 --> 00:16:57.380
Like there's the technical things that went wrong as in what's wrong with our models.

00:16:57.620 --> 00:17:06.400
There's what's wrong with our data, what's wrong with the polling process and on and on, like kind of into the weeds about what technically did we fail to get right in this process.

00:17:06.400 --> 00:17:10.840
And then there's these larger questions about why do we even do this?

00:17:10.960 --> 00:17:23.960
Like who actually benefits from that kind of checking 538 every day and seeing like, oh, Clinton's up 78% this week and oh, Trump's up 1% this week, you know, and like and just watching that change over the course of an election.

00:17:23.960 --> 00:17:29.940
And meanwhile, the politician, we don't hear very much about the policies of the individual politicians as voters.

00:17:29.940 --> 00:17:31.540
Are we actually informed more?

00:17:31.740 --> 00:17:35.920
So anyways, we actually did a whole episode about we did two, as a matter of fact.

00:17:35.920 --> 00:17:46.220
So we did one episode with a bunch of data scientists and people that are kind of on the inside of political campaigns and understand a lot about how campaigns pull people and how they make their predictions.

00:17:46.220 --> 00:17:53.640
We had people, Natalie Jackson, who is responsible for the Huffington Post model that was that was wrong, but so is everybody else's.

00:17:53.640 --> 00:18:01.000
And then we actually did a second episode with Mona Chalabi, who's now the director of data journalism for the Guardian for the US Guardian.

00:18:01.000 --> 00:18:15.200
But she used to work at 538 and has just some really, I think, really smart things to say about whether or not these polls help our public discourse, like whether or not data journalism actually should be involved in this kind of horse racing or, you know, this kind of horse race prediction.

00:18:15.200 --> 00:18:30.160
So and how much like human bias and interpreting the output of these models really impacted how they were published and how they were communicated to the public, because there's this weird thing happening based on the outcome of this election where data science is saying, hey, we made this huge mistake.

00:18:30.160 --> 00:18:38.260
And people are starting to fall back on their previous positions that I think felt comfortable for everybody, which was, well, I don't know how that thing works.

00:18:38.260 --> 00:18:39.680
I don't really trust the data.

00:18:39.680 --> 00:18:41.180
I'm going to go with how I feel.

00:18:41.340 --> 00:18:47.840
Here's an example where all of you guys thought you were right and you were clearly wrong, which reinforces my initial position that data is fallible.

00:18:47.840 --> 00:18:49.300
Therefore, I'm not going to trust it.

00:18:49.300 --> 00:18:57.540
And I think that's it might do a real harm to not just to data journalism, but to potentially society in general.

00:18:57.540 --> 00:19:01.120
That lack of faith in data is, I think, misplaced.

00:19:01.120 --> 00:19:06.940
There's a really there should be a lack of faith in the way that humans interpret the output of their models.

00:19:07.060 --> 00:19:12.780
Like, ultimately, there was a lot of human bias that was injected into that process that was pretty clearly important.

00:19:12.780 --> 00:19:13.680
Yeah, absolutely.

00:19:13.680 --> 00:19:15.820
And anyway, so obviously, I have a lot to say about this.

00:19:15.820 --> 00:19:17.720
I'm like, I'm like rambling on and on and on.

00:19:17.720 --> 00:19:18.660
But it's a huge story.

00:19:18.660 --> 00:19:20.160
Well, that's awesome.

00:19:20.160 --> 00:19:22.020
I'm really looking forward to checking out those episodes.

00:19:22.020 --> 00:19:23.580
And of course, I'll link them in the show notes.

00:19:23.580 --> 00:19:27.100
There's a couple of things that come to mind when I hear what you're saying.

00:19:27.240 --> 00:19:33.780
One is it feels a little bit like we've gotten ourself into this tight reinforcing loop that's super hard to get out of.

00:19:33.780 --> 00:19:39.920
And it's a little bit like quantum mechanics, where if you observe a thing, you change it.

00:19:39.920 --> 00:19:40.400
You know what I mean?

00:19:40.400 --> 00:19:43.860
Like, you try to measure the polls, but you tell people the polls so often.

00:19:43.860 --> 00:19:49.900
And then the various news groups and whatnot are like reinforcing their angle, but also the polls and opinion.

00:19:49.900 --> 00:19:54.780
And it's just like, is that the opinion or is that people's reaction to people's perceived opinion?

00:19:54.780 --> 00:19:57.480
I mean, it's just where do you detangle these things?

00:19:57.480 --> 00:19:58.900
Yeah, that's a really good point.

00:19:58.900 --> 00:20:07.200
Because in the media, we end up with these kind of meta narratives about the election, like Trump's a ridiculous candidate and Hillary Clinton is the inevitable president.

00:20:07.200 --> 00:20:08.580
We're just kind of waiting this thing out.

00:20:08.580 --> 00:20:10.100
I mean, that's an oversimplification.

00:20:10.100 --> 00:20:11.920
But in a lot of the media, that's what it was.

00:20:11.920 --> 00:20:20.680
And so I think it's interesting to see even people that are supposed to be objective, like journalists are always supposed to be objective, but data journalists especially, like believe in the numbers.

00:20:20.680 --> 00:20:22.540
That's the, you know, that should be the mantra.

00:20:22.540 --> 00:20:37.600
And yet I think every time that their data or their analysis or their technique showed that actually Trump might have a pretty decent chance of winning if you account for some of the inconsistencies in the polling, I think they all went, oh, that actually indicates a problem with my model.

00:20:37.600 --> 00:20:40.780
I should tweak it to make sure that it gives the results that are more correct.

00:20:41.560 --> 00:20:42.740
And whoops.

00:20:42.740 --> 00:20:50.140
It turns out that maybe that wasn't the right conclusion to draw when the models didn't perform as we were expecting.

00:20:50.140 --> 00:20:52.100
So it's been really fascinating.

00:20:52.100 --> 00:20:54.080
Well, fascinating.

00:20:54.080 --> 00:21:05.980
And perhaps, I don't know, it'll be interesting to see whether or not we continue to engage in this kind of like this kind of entertainment or, you know, I guess that's basically what it is, you know, like watching the score change from quarter to quarter.

00:21:05.980 --> 00:21:06.580
Exactly.

00:21:06.580 --> 00:21:07.720
Yeah.

00:21:07.720 --> 00:21:13.920
I think, you know, on one hand, you could make statements about humans and whether or not they'll just start to adjust.

00:21:13.920 --> 00:21:17.420
But there's such a commercial interest in the news.

00:21:17.420 --> 00:21:21.640
I'm thinking of like cable news, especially to just like continually cover this.

00:21:21.640 --> 00:21:23.860
So I'm not encouraged that we'll stop.

00:21:25.620 --> 00:21:26.200
That's true.

00:21:26.200 --> 00:21:29.420
Given that, maybe we should just get better at it before the next election.

00:21:29.420 --> 00:21:30.160
Exactly.

00:21:30.160 --> 00:21:30.600
Exactly.

00:21:31.940 --> 00:21:48.320
This portion of Talk Python To Me has been brought to you by Rollbar.

00:21:48.660 --> 00:22:00.060
One of the frustrating things about being a developer is dealing with errors, relying on users to report errors, digging through log files, trying to debug issues, or a million alerts just flooding your inbox and ruining your day.

00:22:00.060 --> 00:22:07.260
With Rollbar's full stack error monitoring, you'll get the context, insights, and control that you need to find and fix bugs faster.

00:22:07.820 --> 00:22:08.980
It's easy to install.

00:22:08.980 --> 00:22:13.220
You can start tracking production errors and deployments in eight minutes or even less.

00:22:13.220 --> 00:22:22.580
Rollbar works with all the major languages and frameworks, including the Python ones, such as Django, Flask, Pyramid, as well as Ruby, JavaScript, Node, iOS, and Android.

00:22:22.580 --> 00:22:31.800
You could integrate Rollbar into your existing workflow, send error alerts to Slack or HipChat, or even automatically create issues in Jira, Pivotal Tracker, and a whole bunch more.

00:22:31.800 --> 00:22:35.160
Rollbar has put together a special offer for Talk Python To Me listeners.

00:22:35.160 --> 00:22:40.860
Visit Rollbar.com slash Talk Python To Me, sign up, and get the bootstrap plan free for 90 days.

00:22:40.860 --> 00:22:43.900
That's 300,000 errors tracked all for free.

00:22:43.900 --> 00:22:47.620
But hey, just between you and me, I really hope you don't encounter that many errors.

00:22:47.620 --> 00:22:53.620
Loved by developers at awesome companies like Heroku, Twilio, Kayak, Instacart, Zendesk, Twitch, and more.

00:22:53.620 --> 00:22:55.240
Give Rollbar a try today.

00:22:55.240 --> 00:22:57.840
Go to Rollbar.com slash Talk Python To Me.

00:22:57.840 --> 00:23:11.000
Another big theme this year and last year, but especially this year, has been encryption, right?

00:23:11.000 --> 00:23:11.860
Yeah, absolutely.

00:23:11.860 --> 00:23:13.860
Maybe especially based on the election outcome.

00:23:13.860 --> 00:23:14.980
It'll become more of a concern.

00:23:14.980 --> 00:23:18.020
Actually, people going to places like ProtonMail.

00:23:18.020 --> 00:23:18.940
ProtonMail is awesome.

00:23:18.940 --> 00:23:22.860
It's like a super encrypted PGP type thing out of the guys at CERN.

00:23:22.860 --> 00:23:26.240
But it's kind of like Gmail, but with PGP and Switzerland.

00:23:26.240 --> 00:23:28.600
Anyway, things like that have been going up.

00:23:28.600 --> 00:23:29.720
Or like Signal, right?

00:23:29.720 --> 00:23:30.880
Like encrypted messaging.

00:23:30.880 --> 00:23:31.540
Yeah, for sure.

00:23:31.540 --> 00:23:35.500
And as well as the whole iPhone, Apple thing at the beginning of the year.

00:23:35.500 --> 00:23:37.060
I think that was 2016, right?

00:23:37.060 --> 00:23:38.820
Should they unlock it?

00:23:38.820 --> 00:23:40.040
Should they be made to unlock it?

00:23:40.040 --> 00:23:40.660
And so on.

00:23:40.840 --> 00:23:41.440
Oh, yeah.

00:23:41.440 --> 00:23:42.200
After that guy.

00:23:42.200 --> 00:23:43.380
Sam Bernardino.

00:23:43.380 --> 00:23:44.080
Yeah, that's right.

00:23:44.080 --> 00:23:44.960
Yeah, yeah.

00:23:44.960 --> 00:23:53.000
So Google decided to take this idea of encryption and use it for a really interesting AI experiment.

00:23:53.000 --> 00:23:55.320
So this is from the Google Brain team.

00:23:55.320 --> 00:23:59.060
And their slogan is, make machines intelligent, improve people's lives.

00:23:59.060 --> 00:24:03.140
So what they did is they started with three basic vanilla networks.

00:24:03.140 --> 00:24:03.780
Okay?

00:24:03.780 --> 00:24:08.100
And they named one Alice, one Bob, and one Eve.

00:24:08.100 --> 00:24:09.720
And you know where this is going, right?

00:24:09.720 --> 00:24:15.880
And all they did, all they did was said, Alice, the outcome they wanted to measure was, Alice

00:24:15.880 --> 00:24:17.480
has to send a secure message to Bob.

00:24:17.480 --> 00:24:18.680
Bob has to decrypt it.

00:24:18.680 --> 00:24:21.200
And Eve has to try to break the decryption.

00:24:21.200 --> 00:24:27.220
And the only advantage Alice and Bob have is they have a shared key that they can use for

00:24:27.220 --> 00:24:28.140
encrypting.

00:24:28.140 --> 00:24:33.060
But they didn't even tell the thing that you need to encrypt stuff or mention any sort of

00:24:33.060 --> 00:24:33.460
algorithm.

00:24:33.460 --> 00:24:38.840
All they did was give it a loss function that said, if Eve decrypts this, you lose.

00:24:38.840 --> 00:24:40.620
If Eve does not decrypt this, you win.

00:24:40.620 --> 00:24:42.020
And they just let it run.

00:24:42.020 --> 00:24:45.580
Yeah.

00:24:45.580 --> 00:24:46.460
It's kind of cool.

00:24:46.460 --> 00:24:46.960
Kind of cool.

00:24:46.960 --> 00:24:47.280
Yeah.

00:24:47.280 --> 00:24:47.920
But I wonder how they want to know.

00:24:47.920 --> 00:24:47.940
Yeah.

00:24:47.940 --> 00:24:50.960
So what they did is they did 25,000 trials of it.

00:24:50.960 --> 00:24:56.020
You know, how they go through a bunch of iterations each time to like train it up and teach it,

00:24:56.220 --> 00:24:59.380
you know, and let it try to basically invent encryption.

00:24:59.380 --> 00:25:00.540
Okay.

00:25:00.540 --> 00:25:04.600
So they said what they did is they more or less created this adversarial generative network.

00:25:04.600 --> 00:25:05.540
Okay.

00:25:05.540 --> 00:25:07.520
And then they just let it go.

00:25:07.520 --> 00:25:09.840
So the data that the networks were given.

00:25:09.840 --> 00:25:13.840
So Alice was given the key in plain text as the first layer of the neural network.

00:25:13.840 --> 00:25:17.540
Bob was given the key in cipher text as the input.

00:25:17.540 --> 00:25:19.280
And Eve only got the cipher text.

00:25:19.280 --> 00:25:20.740
So what do you think happened?

00:25:20.740 --> 00:25:23.160
Well, I know what happened.

00:25:23.280 --> 00:25:28.360
So I won't ruin the punchline, but it is a really fascinating result watching artificial

00:25:28.360 --> 00:25:30.000
intelligence battle each other.

00:25:30.000 --> 00:25:32.060
So let me ask a different question because, yeah, you do know it.

00:25:32.060 --> 00:25:35.020
Would you have expected this outcome?

00:25:35.020 --> 00:25:41.780
You know, for a problem that implies a kind of creativity, I wouldn't have actually.

00:25:41.780 --> 00:25:43.600
I'm not sure that I would have expected.

00:25:43.600 --> 00:25:50.820
I mean, maybe some version of this, but the outcome is what the AI that ultimately succeeded

00:25:50.820 --> 00:25:55.000
in encrypting its messaging actually, like the way in which it solved that problem, I think

00:25:55.000 --> 00:25:56.460
was, was what was actually startling.

00:25:56.460 --> 00:25:56.580
Yeah.

00:25:56.580 --> 00:25:59.000
It's very startling because it, they were not told about encryption.

00:25:59.000 --> 00:26:01.080
They were just told Eve is not supposed to know this.

00:26:01.080 --> 00:26:04.080
And I mean, it's not like a, it's not really smart, right?

00:26:04.080 --> 00:26:06.600
It's an, it's a, an artificial intelligence.

00:26:06.600 --> 00:26:08.580
It doesn't know hardly anything.

00:26:08.580 --> 00:26:13.740
And so the results, most of the time, Alice and Bob did manage to evolve a system that

00:26:13.740 --> 00:26:19.900
they could communicate with few errors and only a couple of times, well, relatively small

00:26:19.900 --> 00:26:20.180
percent.

00:26:20.180 --> 00:26:23.540
Let's say it looks like three, 4%.

00:26:23.540 --> 00:26:26.180
I don't know if you factor that out, area under the curve and whatnot.

00:26:26.760 --> 00:26:32.360
But a few percent of the time Eve showed an improvement over random guessing and actually

00:26:32.360 --> 00:26:33.060
figured it out.

00:26:33.060 --> 00:26:38.980
But soon as Eve did, Alice and Bob just ramped it up and like crushed, crushed her.

00:26:38.980 --> 00:26:40.060
She couldn't keep up anymore.

00:26:40.060 --> 00:26:41.120
So it's funny.

00:26:41.120 --> 00:26:46.660
It goes for like 6,000 iterations where Eve is basically losing and Alice and Bob are winning,

00:26:46.660 --> 00:26:47.600
but then it switches.

00:26:48.740 --> 00:26:52.060
And Eve kind of figures out for a minute and then they're just like, nope, we're going

00:26:52.060 --> 00:26:52.680
to change this.

00:26:52.680 --> 00:26:53.440
And then you're done.

00:26:53.440 --> 00:27:00.980
And what's interesting is that when Alice and Bob were ultimately successful, it's not as if

00:27:00.980 --> 00:27:08.540
they chose from like a buffet of kind of cryptology techniques or, you know, techniques for encryption

00:27:08.540 --> 00:27:12.420
or whatever, and then ultimately stumbled upon the one that was the most secure.

00:27:12.420 --> 00:27:17.540
They invented a new way to go about encrypting their messages.

00:27:17.540 --> 00:27:22.700
They invented a new kind of encryption in order to accomplish this goal, which is, that's

00:27:22.700 --> 00:27:26.260
to me the part that is startling, that they actually created something new.

00:27:26.260 --> 00:27:29.060
And of course, you know, the jokes on the internet were abound.

00:27:29.060 --> 00:27:34.060
Like, you know, basically two AIs have figured out how to talk to each other in a way that nobody

00:27:34.060 --> 00:27:34.800
else can understand.

00:27:34.800 --> 00:27:36.240
There's no problem here.

00:27:36.240 --> 00:27:37.180
Yeah, of course.

00:27:37.180 --> 00:27:38.540
There's no problem here.

00:27:38.540 --> 00:27:40.020
This will be fine.

00:27:40.020 --> 00:27:41.760
Wait, what's that thing in the sky?

00:27:41.760 --> 00:27:42.920
Yeah, so.

00:27:42.920 --> 00:27:46.160
Yeah.

00:27:46.340 --> 00:27:48.380
So this is how the Terminator starts.

00:27:48.380 --> 00:27:49.620
No, I don't actually think that.

00:27:49.620 --> 00:27:51.040
But I think this is super interesting.

00:27:51.040 --> 00:27:54.740
And I really do think the creativity aspect is what's so amazing.

00:27:54.740 --> 00:27:58.880
And I wonder if you really, I mean, we're talking about other encryption techniques, you know,

00:27:58.880 --> 00:28:01.780
PTP, ProtonMail, Signal, and so on.

00:28:01.780 --> 00:28:04.360
What if you really wanted to communicate secretly?

00:28:04.360 --> 00:28:09.020
You just get some super trained up AIs and with you and whoever you're trying to communicate

00:28:09.020 --> 00:28:09.280
with.

00:28:09.280 --> 00:28:11.200
And you just use whatever that thing does, right?

00:28:11.200 --> 00:28:12.060
Like, it's unknown.

00:28:12.060 --> 00:28:13.720
You don't even know what it does.

00:28:14.060 --> 00:28:17.500
Yeah, well, I mean, and to be totally frank with the audience, I think when it comes to

00:28:17.500 --> 00:28:20.980
this type, these types of deep learning techniques, like nobody knows what they do anyway.

00:28:20.980 --> 00:28:24.580
I mean, we know what they do mechanically, but nobody's quite sure.

00:28:24.580 --> 00:28:27.560
Nobody's proven why they're able to be as effective as they are.

00:28:27.560 --> 00:28:31.880
So we're kind of already in that territory where we're inventing things that are more complex

00:28:31.880 --> 00:28:34.060
than our brains can model or understand.

00:28:34.660 --> 00:28:35.020
Okay.

00:28:35.020 --> 00:28:41.480
And when you have those things that can generate themselves, I don't know, it's kind of interesting

00:28:41.480 --> 00:28:46.100
to imagine this future world where we don't actually rely on an encryption technique that

00:28:46.100 --> 00:28:46.640
we understand.

00:28:46.640 --> 00:28:51.400
We just have some AI that we think are smarter than everybody else's, and we just let them

00:28:51.400 --> 00:28:53.980
encrypt it however they see fit, pass the message.

00:28:53.980 --> 00:28:58.140
And then ultimately, any adversaries will be developing intelligence to try and break our

00:28:58.140 --> 00:28:58.480
encryption.

00:28:58.700 --> 00:29:02.220
And they'll just be kind of fighting it out in a world that we don't really understand.

00:29:02.220 --> 00:29:04.940
And hopefully our messages are, you know, secure.

00:29:04.940 --> 00:29:08.140
Did you just read from the back of like a William Gibson novel or?

00:29:08.140 --> 00:29:08.800
No, I'm just kidding.

00:29:08.800 --> 00:29:09.400
Doesn't it?

00:29:09.400 --> 00:29:09.920
Right?

00:29:09.920 --> 00:29:10.420
Right?

00:29:10.420 --> 00:29:11.380
I mean, it does.

00:29:11.380 --> 00:29:15.720
It sounds like we're, at least in some kind of, some of those like, those kind of seminal

00:29:15.720 --> 00:29:20.780
like 80s and 90s sci-fi authors, like this kind of far future that they predicted, at

00:29:20.780 --> 00:29:23.060
least certain aspects of it are starting to become a reality.

00:29:23.060 --> 00:29:26.580
The smarter that are, the more that algorithms can teach themselves.

00:29:26.820 --> 00:29:26.860
Yeah.

00:29:26.860 --> 00:29:27.920
It's super cool.

00:29:27.920 --> 00:29:32.160
I think it's, it's an uncertain future, but it's very interesting.

00:29:32.160 --> 00:29:32.720
It's very interesting.

00:29:32.720 --> 00:29:35.000
So the next item is actually about deep learning as well, right?

00:29:35.000 --> 00:29:35.600
Yeah.

00:29:35.600 --> 00:29:35.840
Yeah.

00:29:35.840 --> 00:29:40.360
I think just to continue on the conversation about deep learning, this was really the year

00:29:40.360 --> 00:29:42.540
that I think it came into its own.

00:29:42.540 --> 00:29:48.600
I feel like to give a quick overview for people who aren't familiar with either machine learning

00:29:48.600 --> 00:29:53.680
in general or this particular technique, basically it's a neural network and a neural network

00:29:53.680 --> 00:29:56.460
is kind of like a, well, let's not really worry about what it is.

00:29:56.460 --> 00:30:01.400
In theory, you're trying to, there's like neurons in a neural network and you kind of

00:30:01.400 --> 00:30:05.340
find a path through the neurons that allows your model to make a decision kind of in the

00:30:05.340 --> 00:30:06.540
same way that your brain works.

00:30:06.540 --> 00:30:09.920
Like you kind of light up a sequence of neurons in a very complicated pattern.

00:30:10.300 --> 00:30:14.160
And that sequence ultimately represents some kind of unique outcome.

00:30:14.160 --> 00:30:18.580
And in this case, it might be like, I don't know, tell me whether that person in the photograph

00:30:18.580 --> 00:30:22.340
is wearing a red t-shirt or a blue t-shirt, or tell me whether it's a man or a woman.

00:30:22.340 --> 00:30:28.060
And learning the kind of subtle patterns in the image that allow you to make that determination

00:30:28.060 --> 00:30:33.300
are the kind of lighting up of some sequence of neurons in a neural network.

00:30:33.440 --> 00:30:37.440
And deep learning is basically when you have many, many, many, many, many, many layers

00:30:37.440 --> 00:30:38.400
of your neural network.

00:30:38.400 --> 00:30:43.140
So much so that it's kind of difficult to understand what's happening in the middle.

00:30:43.140 --> 00:30:44.480
Like there's an input layer.

00:30:44.480 --> 00:30:46.140
We kind of know what goes into the neural network.

00:30:46.140 --> 00:30:48.380
There's an output layer where they tell us what happened.

00:30:48.380 --> 00:30:53.400
And then whatever happens in the middle, we kind of speculate about and make charts and

00:30:53.400 --> 00:30:54.220
kind of infer.

00:30:54.220 --> 00:30:57.640
Yeah, it feels a lot like the MRI sort of analysis.

00:30:57.640 --> 00:31:00.760
Well, creativity happens in this part of the brain.

00:31:00.760 --> 00:31:03.480
And when you're thinking about math, this part of the brain lights up.

00:31:03.480 --> 00:31:06.760
But like, that's the extent of our understanding to a lot of these, right?

00:31:06.760 --> 00:31:08.060
And this sounds a little like that.

00:31:08.060 --> 00:31:09.820
Yeah, yeah, it's exactly like that.

00:31:09.820 --> 00:31:14.040
But the gains from adopting these techniques have been really, really exciting.

00:31:14.040 --> 00:31:19.260
And I think over the next five years, we'll start to see how these technologies impact

00:31:19.260 --> 00:31:20.400
the products that we use.

00:31:20.540 --> 00:31:24.240
For the most part, I think the gains have been largely academic.

00:31:24.240 --> 00:31:26.020
There haven't been a lot of consumer applications.

00:31:26.020 --> 00:31:30.440
But the kind of things that neural networks have been tried or deep learning has been tried

00:31:30.440 --> 00:31:36.680
on, like a guy used a neural network to, it consumed all of the text from the first seven

00:31:36.680 --> 00:31:37.560
Harry Potter novels.

00:31:37.560 --> 00:31:39.160
And then it tried to write new ones.

00:31:39.160 --> 00:31:40.260
They were not good.

00:31:40.260 --> 00:31:41.640
They were quite bad, actually.

00:31:41.640 --> 00:31:43.120
But they were kind of hysterical.

00:31:43.120 --> 00:31:47.880
And then but plausible, like the language that the model used in order to generate these

00:31:47.880 --> 00:31:50.500
new novels was like structurally correct.

00:31:50.500 --> 00:31:52.860
Even if it didn't make any sense, if you know anything about the books.

00:31:52.860 --> 00:31:53.800
Yeah, that's really interesting.

00:31:53.800 --> 00:31:56.620
You know, I would love to see a slight variation on that.

00:31:56.620 --> 00:32:01.280
If you could abstract away a little bit more and not go straight down to the text, but just

00:32:01.280 --> 00:32:03.500
to the plot building blocks.

00:32:03.500 --> 00:32:05.720
There's Harry and there's Hermione.

00:32:05.720 --> 00:32:08.180
And Harry has this feeling.

00:32:08.180 --> 00:32:09.320
He did these actions.

00:32:09.320 --> 00:32:14.340
And then just go, OK, reorder that and have a writer put actual meaningful words to that

00:32:14.340 --> 00:32:15.040
like outcome.

00:32:15.040 --> 00:32:15.720
That would be cool.

00:32:16.060 --> 00:32:16.920
That would be super cool.

00:32:16.920 --> 00:32:17.580
Yeah.

00:32:17.580 --> 00:32:23.320
Because I think a lot of the what these networks are still losing is like this idea of kind

00:32:23.320 --> 00:32:24.240
of context.

00:32:24.240 --> 00:32:24.760
Yeah.

00:32:24.760 --> 00:32:29.420
Like like Google did a similar thing where they fed a neural network a bunch of romance

00:32:29.420 --> 00:32:29.780
novels.

00:32:29.780 --> 00:32:35.460
Although to its credit, it produced some poetry and the poetry read like a real poem, like the

00:32:35.460 --> 00:32:39.320
kind of thing that the romantically inclined among us might have written in high school.

00:32:40.460 --> 00:32:45.360
Kind of sappy, a little saccharine, sometimes unnecessarily dark.

00:32:45.360 --> 00:32:47.880
But yeah, you know, it's super, super interesting.

00:32:47.880 --> 00:32:48.980
But yeah, but it did.

00:32:48.980 --> 00:32:51.560
It's that it that does seem like the next evolution of it.

00:32:51.560 --> 00:32:54.560
Like we've we're kind of understanding language at a really fundamental level.

00:32:54.560 --> 00:33:00.180
But then how that how you we kind of build on how we use the building blocks inside language

00:33:00.180 --> 00:33:06.120
to form like larger concepts and ideas that maybe map over the course of hundreds of pages

00:33:06.120 --> 00:33:07.480
because they're that complex.

00:33:07.480 --> 00:33:11.700
That fortunately still seems to have escaped deep learning models.

00:33:11.700 --> 00:33:15.480
But when they figure that out, just imagine like we talked about all this election stuff.

00:33:15.480 --> 00:33:20.240
Could you imagine like a neural network crafting the story of an election over and then deploying

00:33:20.240 --> 00:33:23.980
thousands of bots communicating with each other in an encryption that we can't understand?

00:33:23.980 --> 00:33:25.660
Like that's when it happens, man.

00:33:25.660 --> 00:33:26.840
Yeah, it's all coming together.

00:33:26.840 --> 00:33:27.760
It's all coming together.

00:33:27.760 --> 00:33:31.000
I hope it's a benevolent AI.

00:33:31.000 --> 00:33:31.380
OK.

00:33:31.380 --> 00:33:35.420
Yeah, but but but there's not all it's not all it's not all potential benevolence and doom, right?

00:33:35.420 --> 00:33:38.940
There's actually some really exciting applications of data science, for example.

00:33:38.940 --> 00:33:39.220
Yeah.

00:33:39.220 --> 00:33:44.900
So, for example, the next thing I want to talk about is actually data sciences, data scientists,

00:33:44.900 --> 00:33:48.420
mathematicians, programmers doing good for the world.

00:33:48.420 --> 00:33:54.060
So one of the big challenges for humans still remains to be cancer, right?

00:33:54.060 --> 00:33:58.160
And one of the more common types is breast cancer.

00:33:58.160 --> 00:34:03.660
So there's this group that put together something called a dream challenge, the digital mammography

00:34:03.660 --> 00:34:04.560
dream challenge.

00:34:04.560 --> 00:34:05.220
All right.

00:34:05.220 --> 00:34:11.860
So the idea is the current state of the world is out of every thousand women screened, only

00:34:11.860 --> 00:34:15.860
five will actually have breast cancer, but 100 will be called back for further testing.

00:34:15.860 --> 00:34:19.380
And so it's not just, well, it's like another doctor visit.

00:34:19.380 --> 00:34:22.180
It's like you're told, hey, we found something in your scan.

00:34:22.180 --> 00:34:23.040
You need to come back.

00:34:23.040 --> 00:34:24.900
So there's all the concern and worry.

00:34:24.900 --> 00:34:26.080
You probably come back a week later.

00:34:26.080 --> 00:34:27.340
There's maybe a biopsy.

00:34:27.340 --> 00:34:28.520
Like you wait for the results.

00:34:28.520 --> 00:34:30.580
It's it's like really disrupting.

00:34:30.580 --> 00:34:30.800
Right.

00:34:30.800 --> 00:34:31.420
And expensive.

00:34:31.420 --> 00:34:37.720
So this group, a bunch of different groups came together and they're putting out a million

00:34:37.720 --> 00:34:43.600
dollar prize for anybody who can build a model that improves upon this and does better than

00:34:43.600 --> 00:34:45.040
the other people trying to do the same.

00:34:45.040 --> 00:34:51.160
So what I think is really interesting is the data and how you get access to the data.

00:34:51.160 --> 00:34:55.420
So fundamentally, what you'll do is you'll submit some sort of artificial intelligence machine

00:34:55.420 --> 00:34:59.220
learning type thing to process this data.

00:34:59.320 --> 00:35:06.840
And if if you can say, here's a bunch of images of scans and, you know, traditionally,

00:35:06.840 --> 00:35:11.460
there's been a certain amount of data available, but this is actually taken to an entirely new

00:35:11.460 --> 00:35:11.920
level.

00:35:12.360 --> 00:35:16.120
So you take this, this data, these scans, and you look at the pictures and you have

00:35:16.120 --> 00:35:18.460
to say, no, this actually is not cancer.

00:35:18.460 --> 00:35:19.460
Yes, this is cancer.

00:35:19.460 --> 00:35:24.080
And then they have the actual outcomes verified by biopsies.

00:35:24.080 --> 00:35:27.880
So you're given that as an input, but here's the deal.

00:35:27.880 --> 00:35:33.000
Normally the problem with doing medical research is you've got to anonymize the data.

00:35:33.000 --> 00:35:35.420
You've got to get permission to share the data and so on.

00:35:35.420 --> 00:35:37.460
So they don't share the data with you.

00:35:37.460 --> 00:35:38.400
Right.

00:35:38.400 --> 00:35:41.860
So the question is, how do you actually process this?

00:35:41.860 --> 00:35:42.900
How do you teach them or seeing anything?

00:35:42.900 --> 00:35:43.120
Right.

00:35:43.120 --> 00:35:47.540
Well, what they do is they give you like 500 pictures or something like that.

00:35:47.540 --> 00:35:48.260
So you can test.

00:35:48.260 --> 00:35:48.580
Right.

00:35:48.580 --> 00:35:49.580
And they give you the outcomes.

00:35:49.580 --> 00:35:50.320
This one was cancer.

00:35:50.320 --> 00:35:51.040
This one wasn't cancer.

00:35:51.040 --> 00:35:52.900
So you could kind of get it sort of working.

00:35:52.900 --> 00:35:58.060
And then they've set up this mechanism in the cloud and AWS using Docker.

00:35:58.060 --> 00:36:05.520
So what you do is you build your model into a Docker image using TensorFlow and a bunch of

00:36:05.520 --> 00:36:07.820
different capabilities that are available to you.

00:36:08.000 --> 00:36:12.080
You build your untrained model into a Docker image.

00:36:12.080 --> 00:36:19.320
You submit the Docker image to some cloud computing system running AWS and they train it on actual

00:36:19.320 --> 00:36:20.740
data and they teach it.

00:36:20.740 --> 00:36:21.520
Yes, this was cancer.

00:36:21.520 --> 00:36:22.220
No, that was cancer.

00:36:22.220 --> 00:36:22.920
Here's your prediction.

00:36:22.920 --> 00:36:23.560
Right, wrong.

00:36:23.560 --> 00:36:24.100
And so on.

00:36:24.600 --> 00:36:26.260
But you have no internet access.

00:36:26.260 --> 00:36:29.120
You can get like the logs or you can't actually ever see the data.

00:36:29.560 --> 00:36:40.920
And then they submit your trained on the real data that you never get to see to actually a huge amount of data, which they can use because nobody ever actually has access to it.

00:36:40.920 --> 00:36:50.880
So there's about 20 terabytes of 650,000 images, 640,000 images that you're going to run your model against to predict cancer.

00:36:50.880 --> 00:36:53.120
And then you'll be judged on your work against that.

00:36:53.380 --> 00:36:54.900
I find that really fascinating.

00:36:54.900 --> 00:37:08.080
So this idea that you basically build a model on your own, like just kind of speculate on what will or wouldn't work and then hand it over to be trained and tested on data that you never see.

00:37:08.080 --> 00:37:09.960
And then you just kind of know whether or not it worked.

00:37:09.960 --> 00:37:12.100
And then I guess tweak accordingly.

00:37:12.100 --> 00:37:21.720
I mean, it's a really awkward process, but at the same time, it's also a really novel solution to I think anybody who's ever worked with or been close to working with medical data.

00:37:21.720 --> 00:37:23.020
There's a lot.

00:37:23.020 --> 00:37:25.260
There's a huge need for this kind of work.

00:37:25.260 --> 00:37:39.220
But most of the people who do machine learning research don't have access to the data because they're not employed by the medical institution that has ownership of it and sort of has been given permission to use it and access it as they see fit.

00:37:39.220 --> 00:37:46.100
And so you almost always run into a wall right around that point in the conversation where it's like, OK, cool.

00:37:46.100 --> 00:37:48.480
We'll just, you know, give us as much data as you have.

00:37:48.480 --> 00:37:53.720
We'll go play around and we'll make a model and then we'll tell you how it goes and then we'll come together and blah, blah, blah.

00:37:53.720 --> 00:38:00.740
Like that's kind of a normal data science model building process where you say, give me whatever data you can and then we'll use that to figure it out.

00:38:00.740 --> 00:38:15.420
And so to come up with this technique, this kind of like double blind or triple blind or ultimately this kind of like, you know, blind trust, I guess, for using training and then using a model is kind of a novel solution.

00:38:15.420 --> 00:38:19.980
I think even if it's awkward, it's like it's a good first step to just get this kind of thing on the road.

00:38:20.200 --> 00:38:50.180
Right.

00:38:50.180 --> 00:38:51.540
one of N problems.

00:38:51.540 --> 00:39:00.880
I think there's there's so many interesting applications given that deep learning can now detect these really subtle patterns, these little really subtle distinctions from one image to the next.

00:39:01.140 --> 00:39:02.700
much better than a human being could.

00:39:02.700 --> 00:39:03.280
Yeah, absolutely.

00:39:03.280 --> 00:39:04.940
I think is it just has a ton of potential.

00:39:04.940 --> 00:39:08.940
So I'm glad that even if it's a little bit awkward that they're just pushing this forward, like let's just make it happen.

00:39:08.940 --> 00:39:10.580
However, we however we can do it legally.

00:39:10.580 --> 00:39:10.780
Right.

00:39:10.780 --> 00:39:15.120
It absolutely is working within the bounds of, you know, the privacy guidelines and so on.

00:39:15.120 --> 00:39:16.920
But it's, it's really interesting.

00:39:16.920 --> 00:39:18.000
And this is a framework.

00:39:18.000 --> 00:39:22.820
I believe this group is building out for future dream challenges, not just this one, right?

00:39:22.820 --> 00:39:25.220
This is like the first of many of these types of things.

00:39:26.480 --> 00:39:29.360
Let me take just a moment and tell you about a new sponsor of the show.

00:39:29.960 --> 00:39:32.540
This portion of Talk Python is brought to you by AnacondaCon.

00:39:32.540 --> 00:39:40.340
AnacondaCon 2017 is the inaugural conference for Anaconda users, as well as foundational contributors and thought leaders in the open data science movement.

00:39:40.340 --> 00:39:52.880
AnacondaCon brings together innovators in the enterprise open source community for educational, informative, and thought-provoking sessions to ensure attendees walk away with knowledge and connections they need to move their open data science initiatives forward.

00:39:53.700 --> 00:39:59.100
AnacondaCon will take place February 7th to 9th, 2017 in Austin, Texas.

00:39:59.100 --> 00:40:07.140
Attendees can expect to hear how customers and peers are using the Anaconda platform to supercharge the business impact of their data science work.

00:40:07.140 --> 00:40:12.320
In addition, attendees will have the opportunity to network with their peers in the open data science movement.

00:40:12.320 --> 00:40:19.760
To learn more, register for the event, or even sponsorship inquiries, please visit talkpython.fm/acon.

00:40:19.760 --> 00:40:23.500
That's talkpython.fm/acon, acon.

00:40:23.500 --> 00:40:28.180
So the other interesting thing about this is the hardware that you get to use.

00:40:28.180 --> 00:40:35.620
Because if you're going to process 20 terabytes of images and then apply machine learning to each one, that's going to be non-trivial, right?

00:40:35.620 --> 00:40:37.660
So they give you some hardware to work on.

00:40:37.660 --> 00:40:44.760
And in fact, your Docker image gets to run on servers powered by NVIDIA, Tesla, K80, GPUs.

00:40:44.760 --> 00:40:47.880
Which I think GPUs in machine learning is really interesting already.

00:40:47.880 --> 00:40:57.220
But just to give you some stats here, your machine gets to run on a server with 24 cores, one of these GPUs, and 200 gigs of RAM.

00:40:57.220 --> 00:40:59.500
And the GPUs are insane.

00:40:59.500 --> 00:41:14.280
Like they have almost 5,000 CUDA cores, 28 gigabytes of memory with 480 gigabytes per second transfer rate, and 8.7 teraflops of single precision computation power.

00:41:14.600 --> 00:41:17.180
Yeah, the stats on – they're just – it's mind-blowing.

00:41:17.180 --> 00:41:17.720
It's mind-blowing.

00:41:17.720 --> 00:41:28.360
And because I think that's something that sometimes gets lost in the discussion about deep learning is like the amount of calculations that take place in a deep learning, deep neural network are truly mind-boggling.

00:41:28.360 --> 00:41:37.040
I mean, training your kind of typical machine learning model might take somewhere between minutes to hours if it's complex or being trained on a lot of data.

00:41:37.040 --> 00:41:43.880
Deep learning models take days or weeks to train or months if you're doing it at like Google scale.

00:41:43.880 --> 00:41:46.800
I mean, the computations just take forever.

00:41:47.260 --> 00:41:53.100
And the reduction in computation time running on the GPU is phenomenal, like many orders of magnitude faster.

00:41:53.100 --> 00:42:02.260
And so the increasingly powerful hardware is really, I think, the untold story of how much it's accelerating the capacity of this type of machine learning.

00:42:02.260 --> 00:42:02.800
Yeah, absolutely.

00:42:03.180 --> 00:42:11.100
And I suspect these types of things where there's a million-dollar prize and the hardware to actually take a shot at it is quite interesting.

00:42:11.100 --> 00:42:12.480
Yeah, and it's expensive.

00:42:12.480 --> 00:42:21.840
I think that like at the moment, the most – this like high-end hardware that we're talking about, you know, it'd cost you $1,000 a day to run a single instance on AWS.

00:42:22.220 --> 00:42:23.520
But that's only going to come down.

00:42:23.520 --> 00:42:32.900
And just like we saw before with the kind of revolution of service-oriented architectures or kind of microservices where it was like kind of the idea of being like, ah, screw it, spin up a new instance.

00:42:32.900 --> 00:42:33.820
That's right.

00:42:33.820 --> 00:42:43.600
And like we lived in a world where we would spin up and kill instances all the time and never think about it for much more sophisticated and scalable and complex applications that lived on the web.

00:42:43.920 --> 00:42:53.140
It's only a matter of time before we have the same kind of mentality with these highly performant instances that are backed by GPUs.

00:42:53.140 --> 00:42:57.700
And I think that that'll – we're only just at the very beginning of that story, I think.

00:42:57.700 --> 00:42:58.560
Yeah, I totally agree.

00:42:58.560 --> 00:42:59.240
I think it's amazing.

00:42:59.240 --> 00:43:00.880
Like in 10 years, we'll be doing this on our watch.

00:43:00.880 --> 00:43:07.620
But speaking of things you don't want on your watch, Microsoft made a bot, and I don't want anywhere near my watch.

00:43:07.620 --> 00:43:13.080
Yeah, I'm not sure I want Microsoft's bot anywhere near my watch or my child.

00:43:13.080 --> 00:43:15.740
What are you saying the bot's a bad influence?

00:43:15.740 --> 00:43:19.260
I think it was a bad influence on all of us, on humanity perhaps.

00:43:19.260 --> 00:43:22.680
Actually, but the funny thing is that it was more like humanity was a bad influence on the bot.

00:43:22.680 --> 00:43:25.980
So we're talking about Tay, of course, Microsoft's Tay.

00:43:25.980 --> 00:43:34.320
So for those who missed this story, it was kind of a brief moment unless you're like a kind of hyperactive media consumer like I am.

00:43:34.320 --> 00:43:37.800
So Microsoft developed a chatbot and released it on Twitter.

00:43:37.800 --> 00:43:45.220
And the way that it worked is that the chatbot Tay would learn how to communicate based on how people communicated with it.

00:43:45.220 --> 00:43:53.720
So you could talk to Tay on Twitter and then Tay would kind of learn how it should respond given the context of what you asked.

00:43:53.720 --> 00:44:00.440
And it should learn how to construct language in a way that was consistent with the norms of this particular channel, which was Twitter.

00:44:00.600 --> 00:44:02.580
And it did a remarkably good job at that.

00:44:02.580 --> 00:44:06.940
When it responded to people, it largely responded in a way that made sense given what they asked.

00:44:06.940 --> 00:44:10.020
And it largely responded in a way that felt like a tweet.

00:44:10.020 --> 00:44:12.780
You know, like it started using weird abbreviations.

00:44:12.780 --> 00:44:16.380
It would say like, you know, see you, the letter C and the letter U to mean see you later.

00:44:16.380 --> 00:44:17.520
You know, things like that.

00:44:17.780 --> 00:44:20.500
And so in a lot of ways, it was a remarkable accomplishment.

00:44:20.500 --> 00:44:30.720
And I should point out that when Microsoft tested the same thing with a Japanese audience, the bot learned to be a sort of genial participant in normal conversation.

00:44:30.720 --> 00:44:41.040
But when they released the bot in English to a largely American audience, it learned very quickly to be a horrible racist.

00:44:41.480 --> 00:44:42.080
Oh, God.

00:44:42.080 --> 00:44:48.860
It was like, it's funny, but not, I mean, it was funny at the time, a little bit less funny now that we know more about like the alt-right.

00:44:48.860 --> 00:45:04.220
But at the time, basically, you know, the kind of Reddit 4chan crowd thought it would be funny as a prank to teach Tay that the way that human beings communicated with each other was to talk about like whenever it was asked, Tay was asked about what it thought about Mexicans.

00:45:04.220 --> 00:45:07.920
It would respond and say, Mexico is going to build the wall and it's going to pay for it.

00:45:07.920 --> 00:45:13.420
Or, you know, it would ask what its thoughts were about Jewish people and it would like apologize for the Holocaust.

00:45:13.420 --> 00:45:15.640
Like truly, truly offensive.

00:45:15.640 --> 00:45:16.740
That is offensive.

00:45:16.740 --> 00:45:17.160
Wow.

00:45:17.160 --> 00:45:19.660
Like breathtakingly offensive.

00:45:19.660 --> 00:45:23.500
And that's only kind of, I guess, is it even funny?

00:45:23.500 --> 00:45:29.400
I mean, there's an aspect of like the scale of the prank that's kind of funny or that like making a big corporation look stupid.

00:45:29.400 --> 00:45:31.880
Like I can see how it's funny in like a juvenile way.

00:45:31.880 --> 00:45:36.780
Anyway, it was just really interesting commentary on both like the sophistication of these technologies.

00:45:36.780 --> 00:45:47.740
Like anybody who's done any kind of natural language stuff knows that like has experienced, I think, how challenging it is to work with the language that people publish on Twitter because it's not really normal language.

00:45:47.740 --> 00:45:52.000
Like there's like a Twitter speak that's unique just to this weird little niche corner of the Internet.

00:45:52.000 --> 00:45:53.760
I guess it's kind of a big corner of the Internet.

00:45:53.760 --> 00:45:54.680
But you know what I mean?

00:45:54.680 --> 00:45:57.260
People speak differently on Twitter than they do anywhere else.

00:45:57.480 --> 00:46:00.780
And so for a machine to learn that is really cool.

00:46:00.780 --> 00:46:10.920
At the same time, it does speak a little bit to Internet culture that the first thing that people decided to do instead of like, like, again, like a Japanese audience, they treated it like kind of like a pet, like a fun friend.

00:46:10.920 --> 00:46:18.160
And, of course, it was immediately exploited to be kind of a horrible racist, misogynist, you know, like a Gamergate participant basically.

00:46:18.900 --> 00:46:21.520
I think it's really cool how well it did.

00:46:21.520 --> 00:46:24.380
But I think it's unfortunate that it was turned to evil.

00:46:24.380 --> 00:46:26.020
Oh, well.

00:46:26.020 --> 00:46:28.560
Yeah.

00:46:28.560 --> 00:46:29.440
So that was Tay.

00:46:29.440 --> 00:46:29.940
That was Tay.

00:46:29.940 --> 00:46:34.160
Go check it out if you're a machine learning researcher and a language computational linguist.

00:46:34.160 --> 00:46:35.960
Fascinating case study.

00:46:35.960 --> 00:46:40.780
And then also if you're interested in Internet culture and have a strong stomach, it's good for that, too.

00:46:40.780 --> 00:46:43.040
Just remember, it's a bot.

00:46:43.040 --> 00:46:45.180
It was made to be evil by the people.

00:46:45.180 --> 00:46:46.480
It wasn't designed that way.

00:46:47.060 --> 00:46:47.960
OK, that's a good point.

00:46:47.960 --> 00:46:49.280
And it's not like it learned to be evil.

00:46:49.280 --> 00:46:54.180
This is the only like people, of course, made jokes like, of course, you know, you release an AI on the Internet.

00:46:54.180 --> 00:46:57.600
And, of course, you know, within like four hours, it's a Nazi.

00:46:57.600 --> 00:47:00.760
And you're like, this is not bode well for the future of artificial intelligence.

00:47:00.760 --> 00:47:02.740
That's not really what's happening.

00:47:02.740 --> 00:47:05.720
It's not like bots want to kill all human beings.

00:47:05.720 --> 00:47:07.460
The AIs are not coming for us.

00:47:07.460 --> 00:47:08.500
Not yet.

00:47:08.500 --> 00:47:09.240
Not yet.

00:47:09.240 --> 00:47:12.380
But when they do, maybe we can turn them to our will.

00:47:12.380 --> 00:47:15.500
OK, so the next one has nothing to do with bots.

00:47:15.900 --> 00:47:21.780
In fact, this is a academics intersect open source intersect business stories.

00:47:21.780 --> 00:47:26.680
So William Stein is the guy that created this thing called Sage Math and Sage Math.

00:47:26.680 --> 00:47:27.480
Do you know Sage Math?

00:47:27.480 --> 00:47:28.660
I don't actually.

00:47:28.660 --> 00:47:30.120
Yeah, I was kind of surprised when I saw this.

00:47:30.120 --> 00:47:31.180
I'm interested to hear more about this.

00:47:31.180 --> 00:47:33.200
Sage Math is a really interesting project.

00:47:33.200 --> 00:47:45.740
It's direct competitor to MATLAB, Mathematica, MAGMA, some of these large commercial computational science sort of platforms that are not open.

00:47:45.740 --> 00:47:46.180
Right.

00:47:46.180 --> 00:47:53.960
Like you if you want to do machine learning on MATLAB, you probably got to buy some pack that's like two thousand dollars for every person who uses it and so on.

00:47:53.960 --> 00:47:54.140
Right.

00:47:54.140 --> 00:47:57.540
So it's really hard to share your work because you've got to have that like that extension back.

00:47:57.540 --> 00:48:15.800
So this guy, he's came out of Harvard, got his PhD there, I believe, and was at UCSD where he actually decided like everything I do in my computing life is open source, except for the one thing that I care most about where I do my math research, which is closed source.

00:48:15.920 --> 00:48:16.540
So that's it.

00:48:16.540 --> 00:48:18.320
I'm going to make make a competitor.

00:48:18.320 --> 00:48:21.380
And so fast forward 10 years or something like that.

00:48:21.380 --> 00:48:26.480
We have Sage Math, which is a really serious competitor to things like MATLAB and Mathematica.

00:48:26.480 --> 00:48:35.440
There's some interesting stuff that came out of it, like Cython, the compiled high performance version of Python came out of that project in some interesting ways.

00:48:35.440 --> 00:48:49.120
So this year he announced that he's decided that to run a successful open source project of that scale, doing that in an academic setting doesn't make sense if he really wants that to succeed.

00:48:49.120 --> 00:48:54.800
So he would say, you know, look, he built a great bunch of people at the University of Washington, where he is these days.

00:48:55.160 --> 00:48:58.660
But he would train these people to become great programmers and work on this project.

00:48:58.660 --> 00:49:05.420
And then they would be hired immediately by Google or some other places that, oh, you know, data science, you know, this computational stuff.

00:49:05.420 --> 00:49:07.020
We got a spot for you.

00:49:07.020 --> 00:49:08.080
And they would be off.

00:49:08.080 --> 00:49:16.560
So he decided to leave academia, leave his senior track job and start a company called Sage Math Cloud, which is like a cloud hosted version.

00:49:16.900 --> 00:49:26.260
You can do all sorts of like data science stuff there, run like Python notebooks, the whole Python scientific stack are and sort of share this across your classes.

00:49:26.260 --> 00:49:37.500
And I just think it's interesting to see this high profile professor leaving academics to venture out in the world to start a business based on open source.

00:49:37.500 --> 00:49:54.080
Yeah, I think that that's actually an interesting trend across the machine learning community, where sort of prior to this AI spring or whatever we're calling it, where pretty much everybody wants some kind of the need for machine learning and machine learning expertise is really high.

00:49:54.080 --> 00:50:03.540
This kind of work did come out of academia and the research labs associated with computer science departments at universities was where we expected a lot of this to come from.

00:50:03.540 --> 00:50:15.840
But now most of the large institutions, Microsoft Research, Google Research, IBM, some of the most of the really huge technology companies are effectively doing pure research.

00:50:15.840 --> 00:50:19.600
And so but pure research, not at academic salaries.

00:50:19.800 --> 00:50:36.740
So, you know, you've earned your PhD, maybe then a couple of years of teaching at machine learning at a high profile university, which it's kind of tough to turn down a couple hundred thousand dollars, $250,000 a year to go work with, you know, with huge resources at your disposal with some of the smartest people in the world.

00:50:36.740 --> 00:50:38.300
And universities are aware of this.

00:50:38.300 --> 00:50:52.440
I think that a lot of a lot of universities are really trying to rethink their relationship with their professors for just this reason, because they don't want to lose them completely to the private sector, but at the same time recognize that they'll never have the resources of the private sector.

00:50:52.440 --> 00:50:59.720
So maybe and you're seeing more people start to take a year off kind of ping back and forth between some of these research institutions and a university.

00:50:59.720 --> 00:51:03.800
It's kind of a new world on the and I'm not sure that there is anything wrong with it.

00:51:03.840 --> 00:51:13.300
I mean, as a as somebody who benefits a lot from this research, I think the way that the private sector is furthering this industry is really exciting.

00:51:13.300 --> 00:51:15.180
Actually, there's a lot of great things that are coming out.

00:51:15.180 --> 00:51:17.340
I see this as a positive news item.

00:51:17.340 --> 00:51:18.840
I'm super excited for William.

00:51:18.840 --> 00:51:21.900
I hope he succeeds in doing this because I think it's really great.

00:51:21.900 --> 00:51:28.900
I love to see open source projects that are become sustainable businesses or have sustainable businesses making the core project better.

00:51:28.900 --> 00:51:32.920
If you want to learn about SageMath on episode 59, I actually interviewed William there.

00:51:33.400 --> 00:51:41.400
Another, you know, on episode 81 of Talk Python, I interviewed Jake VanderPlas and he's at University of Washington as well, I believe.

00:51:41.400 --> 00:51:45.600
But there's no relationship between these two stories other than there.

00:51:45.600 --> 00:51:57.320
They've started this thing called the eScience Institute, which seems to be like a good balance of like maybe a modernization of people doing sort of industry stuff, but also academic computational stuff.

00:51:57.440 --> 00:52:06.200
I think if this story, if the story of people leaving academics to go do private stuff was told in the 90s, it might be a big negative, right?

00:52:06.760 --> 00:52:12.640
This guy went and started this private company where his smarts are bundled up in this commercial thing and hidden under IP.

00:52:12.640 --> 00:52:20.980
But there's so much open source that is coming out of this, even in the private space, although there's some kind of commercial component to it.

00:52:20.980 --> 00:52:24.260
You know, a lot of the stuff like SageMath, for example, is open source.

00:52:24.460 --> 00:52:29.240
So it's not like it's being lost to the world because it's going behind some corporate wall.

00:52:29.240 --> 00:52:30.940
Yeah, I think that that's a really good point.

00:52:30.940 --> 00:52:32.000
Like, I think that's true.

00:52:32.000 --> 00:52:34.100
This is mostly an open source story.

00:52:34.620 --> 00:52:39.880
Anaconda, which is now huge in the Python community, is built by a company called Continuum Analytics here in Austin.

00:52:39.880 --> 00:52:48.460
TensorFlow, which has now become sort of the de facto platform for building neural networks and deep learning models, came out of Google.

00:52:48.460 --> 00:52:49.600
And on and on and on.

00:52:49.600 --> 00:52:52.360
Like, I think – and SageMath is another great example of that.

00:52:52.460 --> 00:53:00.620
And it's cool to see focused on an area of research that is not necessarily computer science-y, you know?

00:53:00.620 --> 00:53:00.840
Yeah.

00:53:00.840 --> 00:53:07.440
Like, actually focusing on the kind of pure math aspects of it is a really valuable contribution.

00:53:07.440 --> 00:53:08.320
So I agree.

00:53:08.320 --> 00:53:11.280
I think it's kind of a cool trajectory.

00:53:11.280 --> 00:53:19.760
And I hope that the technology industry continues its commitment to open source because it really – I mean, not to sound hokey, but it really does benefit the world in a serious way.

00:53:19.760 --> 00:53:20.900
It's definitely a better place to be.

00:53:20.900 --> 00:53:21.560
I totally agree.

00:53:21.680 --> 00:53:23.340
So I'll give them a quick plug.

00:53:23.340 --> 00:53:27.660
Hopefully, you know, if you're a teacher or a professor out there, check out cloud.sagemath.com.

00:53:27.660 --> 00:53:30.260
There's a lot of cool stuff you can do for your classes and so on.

00:53:30.260 --> 00:53:31.480
All right.

00:53:31.480 --> 00:53:33.300
So AIs are smart.

00:53:33.300 --> 00:53:34.840
They can do lots of things.

00:53:34.840 --> 00:53:37.220
But there's just games they're never going to be able to solve like Go, right?

00:53:37.220 --> 00:53:38.920
Well, one would think.

00:53:38.920 --> 00:53:40.300
One would think.

00:53:40.300 --> 00:53:41.240
But actually, it's cool.

00:53:41.240 --> 00:53:48.240
We've talked a little bit about AIs being creative and kind of deep learning models actually coming up with kind of innovative approaches to solving a problem.

00:53:48.240 --> 00:53:51.160
And I think that that's been a big story this year.

00:53:51.620 --> 00:53:58.240
So, you know, we're kind of comfortable with the idea that machines beat us at games like chess, which as human beings we think are remarkably complex.

00:53:58.240 --> 00:53:59.260
There's so much strategy.

00:53:59.260 --> 00:54:00.860
There's so many potential moves.

00:54:00.860 --> 00:54:02.160
And that's true.

00:54:02.160 --> 00:54:11.000
I think a human being can really only hold – like grandmasters at chess can hold maybe eight permutations of the board in their head at any given time.

00:54:11.080 --> 00:54:14.800
Like they can see kind of eight moves ahead, what they'll do, what their opponent will do, what they'll do.

00:54:14.800 --> 00:54:17.440
And they can kind of keep that changing picture of the board in their head.

00:54:17.700 --> 00:54:19.460
Of course, a computer has no such limitations.

00:54:19.460 --> 00:54:23.100
They can play out almost – especially with the computational power that we have now.

00:54:23.100 --> 00:54:28.220
They can play out endless strategies and endless permutations and find the one that gives them the most likely chance of winning.

00:54:28.220 --> 00:54:33.340
We saw Watson basically kick everybody's butt at Jeopardy.

00:54:33.920 --> 00:54:40.560
You know, consume all the trivia knowledge of the universe, understand language, and figure out how to beat us at that game.

00:54:40.680 --> 00:54:44.160
I think that's super interesting because it's the natural language component of it.

00:54:44.160 --> 00:54:45.220
It's not like clear rules.

00:54:45.220 --> 00:54:46.860
This thing is on this square.

00:54:46.860 --> 00:54:48.320
It could move to those three squares.

00:54:48.320 --> 00:54:49.420
Yeah, yeah, absolutely.

00:54:49.420 --> 00:54:56.780
And being able to connect what was being asked in the question to the kind of like deep graph of knowledge that Watson has at its disposal, right?

00:54:56.780 --> 00:55:00.840
Kind of understanding the relationships between different contexts and blah, blah, blah, blah, blah.

00:55:00.840 --> 00:55:01.720
Very cool.

00:55:01.720 --> 00:55:09.540
The game that people thought probably would be inaccessible to computer or machines for kind of a long time is something called Go.

00:55:09.980 --> 00:55:15.280
And Go is kind of, for those who aren't familiar with it, and I'm not, like I'm not a Go player, so I might be getting this wrong.

00:55:15.280 --> 00:55:25.060
But it's basically like chess times 100, like really, really, really complicated chess because there's so many different ways that the board can change.

00:55:25.060 --> 00:55:26.440
Like there's so many different strategies.

00:55:26.440 --> 00:55:28.720
There's just, it's a lot more complex.

00:55:28.720 --> 00:55:34.020
The rules of the game are more complex and the possible outcomes of the game are more complex.

00:55:34.520 --> 00:55:48.340
And because it's so complex, it relies a little bit more on like, sure, some knowledge, but like strategy and intuition because it's a little bit difficult to understand the consequences of your move, like 10 moves down the line.

00:55:48.380 --> 00:55:55.760
So you kind of got to feel your way through it based on your expertise a little bit more than you can with a game like chess that you can pretty much keep all in your head at one time.

00:55:55.760 --> 00:56:01.420
And because of that, people thought, well, that's kind of a tough road to sell for an artificial intelligence.

00:56:01.840 --> 00:56:13.880
But not apparently because in March of this year, the world Go champion, Lee Sedal, basically had his butt handed to him by DeepMind, which was developed by Google.

00:56:13.880 --> 00:56:15.600
So it's a deep learning model.

00:56:15.600 --> 00:56:18.680
It understood how to play the game of Go in a five-game match.

00:56:18.680 --> 00:56:19.960
It cleaned the floor with him.

00:56:19.960 --> 00:56:24.960
And not only that, but it used like some really unorthodox techniques.

00:56:24.960 --> 00:56:30.980
Like it was basically an exceptionally creative and exceptionally intuitive player of the game of Go.

00:56:30.980 --> 00:56:36.780
So it was kind of like the last stand for human beings in terms of beating computers at games.

00:56:36.780 --> 00:56:38.480
And it wasn't really much of a contest.

00:56:38.480 --> 00:56:38.780
Yeah.

00:56:38.780 --> 00:56:42.060
So we don't want to go up against AI anymore, do we?

00:56:42.060 --> 00:56:45.460
I think it's really interesting.

00:56:45.460 --> 00:56:49.500
And again, I think the creative aspect of it is what's cool, right?

00:56:49.500 --> 00:56:50.960
The intuition, right?

00:56:50.960 --> 00:56:52.740
Those are the things we think that computers can't do that.

00:56:52.740 --> 00:56:55.280
Sure, they can map the entire problem space.

00:56:55.280 --> 00:57:03.080
And if they're fast enough, they could actually map out potential ways which you could not win if they followed this series of 100 steps or whatever.

00:57:03.080 --> 00:57:06.320
But if you're not doing that, then this gets even more interesting.

00:57:06.320 --> 00:57:06.820
Yeah.

00:57:06.820 --> 00:57:07.400
Yeah, absolutely.

00:57:07.400 --> 00:57:19.740
And just to kind of understand the difference in techniques, like anybody who's done maybe some kind of basic kind of first steps in machine learning, like a popular example is to have people try and beat a game of tic-tac-toe.

00:57:19.740 --> 00:57:26.380
Like to have your – to successfully win a tic-tac-toe using an artificial intelligence or to write a program that can do that.

00:57:26.380 --> 00:57:36.040
And the strategy is basically learn every possible outcome of the game and then at any given moment pick the path of all possible paths that gives you the best chance of victory.

00:57:36.040 --> 00:57:37.480
That's fairly straightforward.

00:57:37.480 --> 00:57:42.580
But on a tic-tac-toe board, the number of possible permutations of the game are really, really small.

00:57:43.000 --> 00:57:47.280
Nevertheless, if you haven't done that before, I mean, when I first did it, I found it to be very challenging.

00:57:47.280 --> 00:57:49.100
It's a challenging problem to go and solve.

00:57:49.100 --> 00:57:58.480
Extrapolating that to go I think just really demonstrates the huge leaps that we've made in this field over the past maybe decade.

00:57:58.480 --> 00:57:59.580
It's exciting.

00:57:59.580 --> 00:58:00.260
It is.

00:58:00.260 --> 00:58:05.020
Like what else – when released on the right problem, like what else can these models potentially figure out?

00:58:05.100 --> 00:58:11.360
What solutions can they see that are just unavailable to us because we just don't have the computational capacity in our brains?

00:58:11.360 --> 00:58:12.480
It's kind of exciting.

00:58:12.480 --> 00:58:12.660
Yeah.

00:58:12.660 --> 00:58:19.120
So what if we had way more self-driving cars and the game was to minimize traffic jams?

00:58:19.120 --> 00:58:20.060
That would be lovely, right?

00:58:20.060 --> 00:58:20.640
It would be.

00:58:20.640 --> 00:58:23.780
And what if – what if – think about this, my friend.

00:58:23.780 --> 00:58:27.020
What if the game was to simulate human existence?

00:58:27.020 --> 00:58:28.320
What about that?

00:58:28.560 --> 00:58:29.700
That's totally science fiction.

00:58:29.700 --> 00:58:33.300
That's like red pill, blue pill sort of thing, right?

00:58:33.300 --> 00:58:36.080
Well, one would think.

00:58:36.080 --> 00:58:57.840
But given the advances over the past – if we look at the past 10 years of video games and artificial intelligence and virtual reality, one presumes, or at least Elon Musk presumes – and this is our last story – that if you extrapolate that into the not-too-distant future, surely we should be able to simulate the entirety of human existence and play it out.

00:58:58.160 --> 00:59:03.800
And if we could do that, what's to say that we aren't the simulation of some future existence?

00:59:03.800 --> 00:59:15.460
And given how many simulations we'd probably run, like any sufficiently advanced society might run billions of simulations, and given that, what are the odds that we're the base reality and not just one of these simulations playing out?

00:59:15.460 --> 00:59:16.420
Pretty small, my friend.

00:59:16.420 --> 00:59:17.380
Pretty small.

00:59:17.380 --> 00:59:17.640
Yeah.

00:59:17.640 --> 00:59:18.560
Therefore –

00:59:18.560 --> 00:59:19.400
That is insane.

00:59:19.400 --> 00:59:20.520
Therefore, we're in the matrix.

00:59:20.520 --> 00:59:21.880
We are in the matrix.

00:59:21.880 --> 00:59:26.700
You know, on one hand, like it seems – okay, that's a really interesting thought experiment.

00:59:26.700 --> 00:59:30.960
You know, I mean, it reminds me of when I took philosophy in college, right?

00:59:30.960 --> 00:59:42.440
And my professor told me about Zenon's dichotomy paradox or whatever it was called, where in order to walk out of the classroom, you have to walk halfway to the door.

00:59:42.440 --> 00:59:47.340
And then you've got to walk halfway still and halfway of that and half of that.

00:59:47.340 --> 00:59:49.620
But that's actually an infinite series of steps.

00:59:49.840 --> 00:59:51.340
So how will you ever walk out of the door?

00:59:51.340 --> 00:59:55.080
I remember my mind being a little bit blown, like, how are we going to get out of there?

00:59:55.080 --> 00:59:58.080
Like, I understand that I walk out of here.

00:59:58.080 --> 01:00:02.100
But logically, like, how are you going to cross an infinite number of halves, right?

01:00:02.100 --> 01:00:02.920
That's crazy.

01:00:02.920 --> 01:00:08.980
But then, you know, of course, I went to calculus and realized, well, you're also at a rate of speed going infinitely faster.

01:00:08.980 --> 01:00:11.560
So it's like a limit that approaches, well, one, no big deal.

01:00:11.560 --> 01:00:15.500
And so when I hear this on one hand, I feel like Zenon's paradox.

01:00:15.500 --> 01:00:19.420
Like, you can set it up so you trick yourself to go, oh, oh my gosh, you're right.

01:00:19.420 --> 01:00:20.060
It's impossible.

01:00:20.060 --> 01:00:20.760
This is crazy.

01:00:20.760 --> 01:00:23.800
And then there's just like a moment of clarity where it just unlocks.

01:00:23.800 --> 01:00:25.260
Yeah, this is actually ridiculous.

01:00:26.060 --> 01:00:31.480
But I have huge respect for Elon Musk on one hand.

01:00:31.480 --> 01:00:35.660
I mean, more than almost like he is like Edison times 10 or something.

01:00:35.660 --> 01:00:36.280
I mean, he's amazing.

01:00:36.280 --> 01:00:50.900
And I just heard that Google is now releasing like structural map, street view type stuff for places like Las Vegas, where you can like literally map out in VR towns.

01:00:51.600 --> 01:00:56.020
So, you know, like put 50 years or a thousand years on that and what happens, right?

01:00:56.020 --> 01:00:56.440
Yeah.

01:00:56.440 --> 01:00:57.240
Yeah, absolutely.

01:00:57.240 --> 01:01:04.340
I mean, and the argument that seems to resonate with me most about this is kind of like, well, if that's true, who cares?

01:01:04.340 --> 01:01:04.940
Yeah.

01:01:04.940 --> 01:01:12.960
You know, like it's not as if it'd be one thing as if it was like actually the matrix and we were all living in a manufactured reality to our detriment.

01:01:12.960 --> 01:01:18.800
But if the idea is that like we're a simulation that's running itself, it's like, well, how is that really different than an actual reality?

01:01:18.800 --> 01:01:22.000
Like, why is that any different than a quote unquote base reality?

01:01:22.000 --> 01:01:26.480
Like it's, you know, that's kind of a biased definition of what reality is in the first place and on and on and on.

01:01:26.480 --> 01:01:27.680
And then, okay, fine.

01:01:27.680 --> 01:01:28.800
It's kind of like that.

01:01:28.800 --> 01:01:34.140
The first question that the only philosophy class that I ever took when the teacher walked in, it was like in very dramatic fashion.

01:01:34.140 --> 01:01:37.700
Like we all sat down and he just said, prove to me that I'm real.

01:01:37.700 --> 01:01:39.660
Yeah, absolutely.

01:01:40.100 --> 01:01:41.280
You know, and you're like, all right.

01:01:41.280 --> 01:01:41.880
Okay, cool.

01:01:41.880 --> 01:01:42.720
Dead Poets Society.

01:01:42.720 --> 01:01:47.320
But, you know, it's an interesting conversation.

01:01:47.320 --> 01:02:04.620
It seems more like really those advances will lead us to the point where like we might start to not care so much about whether or not we're like our surroundings are artificial, quote unquote, or like synthetic, you know, manufactured by a machine or biologically manufactured in the way that we're accustomed to today.

01:02:04.740 --> 01:02:05.020
Absolutely.

01:02:05.020 --> 01:02:07.880
And, you know, that's an interesting conversation to be had.

01:02:07.880 --> 01:02:17.600
Maybe a more useful one than whether or not we're like our universe is one of many in the multiverse as manufactured by a computer program hundreds of thousands of years in the future.

01:02:17.600 --> 01:02:18.380
But it is.

01:02:18.380 --> 01:02:18.800
It's cool.

01:02:18.800 --> 01:02:19.960
It's a fun thought experiment.

01:02:19.960 --> 01:02:30.460
It's like one of those things where like I applaud Elon Musk for basically like posing like a sci-fi philosophy question to the world, knowing that he basically had the world as an audience.

01:02:30.660 --> 01:02:35.660
And so for the next month afterwards, nerds like us were like, well, let's debate both sides.

01:02:35.660 --> 01:02:36.000
Yeah.

01:02:36.000 --> 01:02:39.440
Do you think he just woke up one day and said, you know what?

01:02:39.440 --> 01:02:41.320
I'm going to go to this place where I'm giving a speech.

01:02:41.320 --> 01:02:43.160
I'm just going to deadpan this sucker.

01:02:43.160 --> 01:02:45.780
I'm just I'm going to put it out there and just pretend this is.

01:02:45.780 --> 01:02:47.180
And let's just see what happens.

01:02:47.180 --> 01:02:48.620
Yeah.

01:02:48.620 --> 01:02:48.980
Yeah.

01:02:48.980 --> 01:02:50.880
At some point, you know, at some point you have to wonder.

01:02:51.260 --> 01:02:57.980
I get that, you know, like periodically if I'm like, you know, things are kind of cruising along like thing A, I'm doing is going pretty well.

01:02:57.980 --> 01:02:59.740
Thing B, there's some real fires to put out.

01:02:59.740 --> 01:03:01.740
Intellectually, I'm a little bit bored this week.

01:03:01.740 --> 01:03:02.540
You know what I mean?

01:03:02.540 --> 01:03:06.100
And of course, that goes away because, you know, whatever, we're all busy and we get consumed in our problems.

01:03:06.100 --> 01:03:09.560
But when Elon Musk has that kind of boredom, maybe this is what happens.

01:03:09.560 --> 01:03:10.520
It could be what happens.

01:03:10.520 --> 01:03:11.980
It was interesting.

01:03:11.980 --> 01:03:12.660
It's definitely interesting.

01:03:12.660 --> 01:03:17.040
And my I believe that we're not actually living in some kind of singularity yet.

01:03:17.040 --> 01:03:18.940
But I do think it's fun to think about.

01:03:18.940 --> 01:03:19.740
All right, Jonathan.

01:03:19.740 --> 01:03:20.240
Agreed.

01:03:20.240 --> 01:03:28.680
Yeah, I think we should leave everybody with this philosophical thought for the rest of the holiday till they come back to work and focus on actual things.

01:03:28.680 --> 01:03:30.220
Yeah.

01:03:30.220 --> 01:03:33.920
In the meantime, ponder your existence when you're thinking about your New Year's resolutions.

01:03:33.920 --> 01:03:34.360
Yeah.

01:03:34.360 --> 01:03:36.660
You got to come back to work next year or do you?

01:03:36.660 --> 01:03:39.640
What is work?

01:03:39.640 --> 01:03:40.660
What is meaning?

01:03:40.660 --> 01:03:41.300
Exactly.

01:03:41.300 --> 01:03:42.520
All right, man.

01:03:42.520 --> 01:03:45.020
Well, those were 10 really interesting stories.

01:03:45.200 --> 01:03:48.360
I think it's been a great year for data science and AI and things like that.

01:03:48.360 --> 01:03:49.100
Yeah, me too.

01:03:49.100 --> 01:03:50.500
It's been it's been a fascinating year.

01:03:50.500 --> 01:03:55.060
And I look forward to to 2017 being just as interesting and exciting.

01:03:55.060 --> 01:03:55.360
Thanks.

01:03:55.360 --> 01:03:57.720
Thanks so much for for having me on and for doing this.

01:03:57.720 --> 01:03:58.840
I think this has been a really fun episode.

01:03:58.840 --> 01:03:59.660
It's been great fun.

01:03:59.660 --> 01:04:00.060
You're welcome.

01:04:00.060 --> 01:04:04.840
So looking forward to 2017, everybody should be going to PyCon, right?

01:04:04.840 --> 01:04:05.960
Oh, absolutely.

01:04:05.960 --> 01:04:07.020
Absolutely.

01:04:07.020 --> 01:04:15.120
Because I heard a rumor that there may be some very exciting Python focused podcasts that are all hanging out waiting to talk to you.

01:04:15.120 --> 01:04:15.420
Absolutely.

01:04:15.420 --> 01:04:15.680
Is that right?

01:04:15.680 --> 01:04:16.480
That is absolutely right.

01:04:16.480 --> 01:04:20.900
So partially derivative, talk Python, Python Bytes, podcasting it.

01:04:21.100 --> 01:04:25.060
We're all getting together and we're doing a big group booth.

01:04:25.060 --> 01:04:27.240
You can come talk to all of us, meet all of us.

01:04:27.240 --> 01:04:29.280
We're going to be doing maybe some live recordings.

01:04:29.280 --> 01:04:34.520
We don't quite know what that looks like yet, but we're definitely putting together a group booth somewhere in the expo hall.

01:04:34.520 --> 01:04:41.680
So I think it's I'm not sure by the time this airs early bird discounts may be over, but don't wait till the end to buy your ticket.

01:04:41.680 --> 01:04:45.200
Buy them right away because they sold out last year and they were sad people.

01:04:45.200 --> 01:04:47.840
They reached out to me and wanted to come and I couldn't help them.

01:04:47.840 --> 01:04:48.160
Yeah.

01:04:48.160 --> 01:04:54.200
And if the trends are any indication, Python is only going to be more popular, only going to be more widely adopted.

01:04:54.200 --> 01:04:57.100
PyCon will only get bigger and more fully attended.

01:04:57.100 --> 01:04:57.940
So I agree.

01:04:57.940 --> 01:05:01.700
Get your tickets now and come hang out with your favorite podcasters.

01:05:01.700 --> 01:05:02.080
Yes.

01:05:02.080 --> 01:05:02.580
It'll be the best.

01:05:02.580 --> 01:05:03.000
It'll be great.

01:05:03.000 --> 01:05:03.820
It's going to be great fun.

01:05:03.820 --> 01:05:05.060
I'm looking forward to seeing you there.

01:05:05.060 --> 01:05:05.800
Yeah, me too.

01:05:05.800 --> 01:05:06.220
All right.

01:05:06.260 --> 01:05:06.620
Catch you later.

01:05:06.620 --> 01:05:07.060
All right.

01:05:07.060 --> 01:05:07.300
Thanks.

01:05:07.300 --> 01:05:07.740
Bye.

01:05:07.740 --> 01:05:12.400
This has been another episode of Talk Python To Me.

01:05:12.400 --> 01:05:18.840
Today's guest has been Jonathan Morgan, and this episode has been sponsored by Rollbar and Continuum Analytics.

01:05:18.840 --> 01:05:20.860
Thank you both for supporting the show.

01:05:20.860 --> 01:05:24.100
Rollbar takes the pain out of errors.

01:05:24.100 --> 01:05:31.800
They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course.

01:05:32.500 --> 01:05:38.960
As Talk Python To Me listeners, track a ridiculous number of errors for free at rollbar.com slash Talk Python To Me.

01:05:38.960 --> 01:05:50.940
Whether you want to hear the keynote by Rowan Curran from Forrester Research, meet with the guys behind Anaconda, or just mingle with high-end data scientists, you need to find your way to Austin, Texas for AnacondaCon this February.

01:05:50.940 --> 01:05:55.200
Start at talkpython.fm/Acon, A-C-O-N.

01:05:55.200 --> 01:05:58.140
Are you or a colleague trying to learn Python?

01:05:58.600 --> 01:06:02.780
Have you tried books and videos that just left you bored by covering topics point by point?

01:06:02.780 --> 01:06:11.420
Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course to experience a more engaging way to learn Python.

01:06:11.420 --> 01:06:18.740
And if you're looking for something a little more advanced, try my WritePythonic code course at talkpython.fm/Pythonic.

01:06:19.640 --> 01:06:24.260
You can find the links from this episode at talkpython.fm/91.

01:06:24.260 --> 01:06:25.340
That's right.

01:06:25.340 --> 01:06:31.720
Anytime you want to find a show in the show page and show notes, it's just talkpython.fm/episode number.

01:06:31.720 --> 01:06:33.940
Be sure to subscribe to the show.

01:06:33.940 --> 01:06:36.140
Open your favorite podcatcher and search for Python.

01:06:36.140 --> 01:06:37.380
We should be right at the top.

01:06:37.760 --> 01:06:46.700
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.

01:06:46.700 --> 01:06:51.780
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

01:06:51.780 --> 01:06:58.480
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.

01:06:58.680 --> 01:07:03.840
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.

01:07:03.840 --> 01:07:05.900
This is your host, Michael Kennedy.

01:07:05.900 --> 01:07:07.180
Thanks so much for listening.

01:07:07.180 --> 01:07:08.380
I really appreciate it.

01:07:08.380 --> 01:07:10.520
Smix, let's get out of here.

01:07:10.520 --> 01:07:10.520
Smix, let's get out of here.

01:07:10.520 --> 01:07:31.920
Smix, let's get out of here.

01:07:31.920 --> 01:07:32.420
Bye.

01:07:32.420 --> 01:07:32.620
.

01:07:32.620 --> 01:07:32.780
We'll be right back.

01:07:32.780 --> 01:07:33.780
Thank you.