The Python landscape is changing pretty dramatically.
Python's rapid growth over the past five years means it doesn't look the same as the early days.
On this episode, we'll take a deep look inside the state of the Python ecosystem
with Eva Jalowalska and Dmitry Filipov.
They led the recent PSF and JetBrains Python survey,
and they're here to dig into the results.
This is Talk Python to Me, episode 176, recorded August 27, 2018.
Welcome to Talk Python to Me, a weekly podcast on Python,
the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm,
and follow the show on Twitter via at talkpython.
This episode is brought to you by Linode and brilliant.org.
Check out what they're offering during their segments. It really helps support the show.
Eva, Dmitry, welcome to Talk Python.
Thanks for having us, Michael.
It's a pleasure to be on your show.
Oh, it's great that you're both here. I'm really glad that you came to talk about this project.
It's really important, I think, to take the pulse of the community every now and then,
and this is a perfect project for that.
So excited to share the results you guys got from this whole survey thing that you've done.
Before we get to that, though, let's start with your story.
How did you get interested in Python?
Eva, start with you, I guess.
Sure. So I actually started working on PyCon by itself,
and it was a very great experience.
And eventually, I left the employer that I was working through as a meeting planner and joined the PSF.
And that's how I got involved in the PSF and the Python community.
That's great.
Dmitry, how about you?
So I started my development career at Astronautics Corporation of America working on Boeing 777 and Boeing 787 projects.
So if you are on those airplanes, be sure they're safe.
I've put a lot of efforts into testing them.
Like actually working on such projects, you quickly learn that testing is an extremely important part of software development.
But my love story with Python actually began when I started working for Oracle about seven years ago.
I was responsible for interoperability between Oracle Linux and Oracle VM.
So back then, I would make sure that Oracle Linux runs smoothly as a guest virtual machine on Oracle VM.
So that required some Linux kernel hacking and writing a lot of automated tests to check that various Oracle Linux versions work properly.
So Python came in handy for writing tests and automation.
That's how I started with Python.
Later, I discovered an open position at JetBrains.
You thought you would go make some Python tools for Python people.
Exactly.
That's cool.
It must have been fun to work on the 777, especially the 787.
That's a really fantastic airplane.
Yes, it's a dream plane, dreamliner.
It totally is.
I've gotten to fly it a couple of times and I've always really enjoyed it.
I guess just to recap what you guys do today, day to day, Dubit, you're in charge of marketing and overseeing PyCharm at JetBrains.
Would you say that's your general role?
Yeah.
Basically, I'm responsible for many tasks around Python project.
PyCharm project, I mean.
I do some product management activities like making sure that PyCharm is released on time with everything ready by the release date.
I'm also responsible for everything marketing.
So, as an example, I coordinate content creation, our appearance at major Python events.
I run PyCharm blog, manage partnership campaigns, among many other things.
Yeah, that sounds like a fun job.
Eva, how about you?
So, I'm the director of operations at the Python Software Foundation.
So, that pretty much means that I oversee everything that the PSF does.
That includes Python, which is the conference that we run yearly here in the U.S.
And then our grants program, as well as any trademarks that we manage for the Python programming language.
And several other things, like managing our staff and day-to-day things like that.
Yeah, that's quite a bit.
And PyCon itself is a big project, right?
It absolutely is.
I would say at this point, it takes up two full-time employees at least eight months out of the year.
That's incredible.
But it's a huge event.
It's really fun.
So, speaking of PyCon, the reason I reached out to both of you is you did a really interesting presentation at PyCon for a project that had been maybe six months in the making, but previous to that, about this survey.
So, PSF and JetBrains put some efforts together to get a bunch of the Python community come fill out a nice questionnaire.
And you guys at JetBrains did a really great job on analysis of it.
So, we'll get to talk about all the results there and stuff, but let's start with how this project came together.
So, the story of the Python developer survey goes back in 2016, when I personally was fascinated by an idea of running a survey among Python developers.
By that time, I already had some experience with surveys among various PyStream users, but I wanted to run something on a completely different scale.
So, I gathered a working group comprised of people from the research team, where people have a lot of experience with math and stats.
People from the survey team who helped me to organize previous surveys.
People from the marketing department to help us to promote the survey designers, to create a nice-looking report.
And we did that survey, which was quite a success.
We gathered something around, I believe, 4,000 responses through our own channels, through advertisement on Google, Stack Overflow, and others.
This is the 2016 survey, right?
Yes, this is the 2016 survey.
So, it's telling the story prior to our latest 2017 survey.
Yeah, yeah, yeah.
So, back then, we've got to some nice results, and they were really good and of high quality.
I did a talk at PyGotham 2016 in New York, and the survey was well-received.
The only problem with that survey was that we had around 50% of PyTerm users among respondents, which was way higher than our estimation of PyTerm share among Python editors and IDs.
In some sense, you were surveying your customers rather than the broad.
Right.
Exactly, exactly.
Okay.
Exactly.
So, back then, I started thinking about how we can make the survey better, how to make it independent and unbiased.
So, later in 2016, I bumped into Eva, I believe, at PyCon APEC in South Korea, if I'm not mistaken.
Yeah.
And back then, I told her about the Python survey that we did, and together with Eva, we came up with an idea that Python Software Foundation can be the key to make the survey much better.
So, in 2017, we ran the survey in a different format.
Instead of promoting survey through JetBrains channels like we did in 2016, we promoted the survey only through the official Python channels and official Python Software Foundation channels.
Exactly.
So, we reached out via Python.org.
We put banners up on the download pages, which received the highest amount of feedback, as well as our Twitter, the PSF blog.
We reached out to several communities that we work with since the PSF gives out grants pretty much globally.
We're in touch with a lot of different communities.
So, we're going to continue to do a lot of outreach that way as well for the next version.
Yeah, that's really cool.
I think having this sort of vendor-neutral version really puts a lot of authority behind this survey, right?
It's one thing to say, well, look, PyCharm is really popular among PyCharm users.
Maybe, you know, Python 3 versus Python 2 usage varies on that because maybe more professionals use Python versus they don't.
I don't know.
I'm just making up this stuff, right?
But having it really neutral seems like a super valuable addition.
And we'll talk about where some of the participants came from, but it sounds to me like you really pulled that off quite well.
Yeah, we think it worked.
It worked very well.
Looking at the countries that we received responses from, we noticed that a lot.
We're missing a lot of feedback from people on the African continent, probably Indian continent, but we're going to move on that for the next version.
Yeah, and now you know, right?
Exactly.
Yeah.
There always has to be a starting point.
So you live and you learn.
Exactly.
So what were some of the goals of the survey?
With this survey, we wanted to solely, like, we did the survey for the purpose of mapping out an accurate landscape of the Python community.
We wanted to identify the latest trends and gather insight on how the Python development world looks today.
With this survey, we want to learn how Python is being used by diverse developers, what components complement its use,
and what makes it one of the most loved languages.
And from the PSF standpoint, it was important to us because, you know, the PSF is there to support the community.
So the more that we know of the direction that the community is going in, the more we'll be prepared to help them in the future.
So it's a very useful tool, not just for the community, in my opinion, but also for the nonprofit behind the language.
Yeah, certainly knowing where things are going and where the support needs to be.
And you learn some interesting lessons, like you were hinting at there, about, say, maybe, you know, the PSF does some outreach and some support for various communities.
And maybe some of these areas where it's somewhat underrepresented, you could, you know, maybe do more, right?
Exactly.
Yeah, that's great.
So, Dimitris, you said in your 2016 edition, you had about 4,000, 4,500 developers.
And now this survey had 9,500 developers from almost 150 different countries.
That's pretty awesome.
Yeah, it is.
And I hope in this year, actually, when we're going to run this survey next time, I hope we'll get even more respondents.
So it will be bigger and better in quality as well.
Yeah, absolutely.
I'll put this question out to both of you.
How can people, so last time it was in October 2017.
I think you're planning a similar timeframe for this one of 2018.
So that'd be in about a month and a half or so, the survey will come out again.
How do people, is there a way for people to, say, join a mailing list or get announcements so they don't have to, like, bump into a banner on python.org or something to know about it?
So we don't have a mailing list, but we'll definitely announce it via the PSF Twitter account, which is at the PSF.
And we'll be announcing it via our Slack channels and our blog, which is PyFound, blogspotpyfound.org.
How long was the survey open for, this one that you already did?
It was open for three weeks.
Three weeks.
So people have a decent amount of time.
And I'll definitely share it on my Twitter account as well and probably talk about it on the Python Bytes podcast and try to get the word out.
It'd be great to have as many people, you know, participate.
Because I know there's more than 9,000 Python developers out there.
Definitely.
Absolutely.
Eva, do you know the number, actually?
Do you have an official PSF number?
There is no PSF official data on that, but we're definitely hoping to get there with a survey like this.
Yeah, that's great.
I think it's really difficult to estimate the overall number of Python developers, and they can be official number for that.
Because in the end, like with the 2017 survey, we run into the problem because who is the Python developer?
It's a very difficult question.
It's a very difficult question because people use Python to a different extent.
Some of them use Python full-time as their primary language.
Some of them use them as an additional language.
And it's hard to say how many can be considered to be called Python developers.
Exactly.
Even for those people that are not, quote-unquote, developers in their day job, they might use Python as a tool in their day job.
It's definitely difficult to categorize.
Exactly.
For example, many people who wear, like, data analyst hats sitting in large banks doing a lot of data analysis.
Can we call them Python developers or data analysts if they use Python?
Yeah.
Does it got to do 20% other day in Python or 15%?
Right.
Right?
Yeah.
Yeah, exactly.
Yeah, that's a really good point.
Another thing, another idea I want to just throw out there as people think about this, this is not my idea.
This idea came from Scott Hanselman, who's really well-known in the developer space, has his own podcast.
But he has this really great idea talking about sort of reaching all of the developers.
And he says, look, there's this entire world of what he labeled dark matter developers.
We know they're out there, but we can't see them.
We can't detect them.
They don't come to conferences.
They're not on social media.
They just go to their job, and they work in their code, and they basically are not visible to the community in a sense.
There's tons of those people.
And that concept, I think, really makes it hard to know the entire extent.
But you guys are doing a pretty good job of trying to take that pulse anyway.
And we hope to improve it year on year.
Yeah, for sure.
So one of the things you did with this that I hinted at the beginning is you gave this as a presentation at PyCon in PyCon 2018.
So it was Python by the numbers.
So if people want to watch your presentation with some slides and graphs and stuff, they can check that out there.
And then, of course, the results of the survey are up.
And I'll put the link in the show notes.
And who was responsible for actually writing up the analysis of that?
Was that the JetBrains, the PSF, someone else?
We did it on the JetBrains side.
So our partnership with Python Software Foundation was that Python Software Foundation was responsible for promoting the survey through their official channels.
And JetBrains did all the rest.
So we set up the survey, set up the logic.
We then analyzed the results and created the report.
Yeah, congratulations, because that was a really, really nice write-up.
I love each section had like a here's the key takeaway from this section and would sort of, you know, not just show the stats, but put it into more colloquial terms, I guess.
Like, here's a bunch of stats.
And oh, by the way, what that means is there's, you know, four out of five Python developers use Python as their main language.
Or, you know, something to that effect would be like a thing that would be in there.
That was really cool.
Yeah, that was a cool feature of this report.
Yeah, I really found it super, super approachable.
So let's go back to first before we get into the results.
I want to make sure we make it really clear on who participated, where they're coming from, what groups are being measured.
Because, you know, one of the whole reasons I wanted to have you all on the show is there's so much hype around Python right now.
And there's so many surveys that are just putting Python at or near the top, or it has a, you know, really steep growth curve or something like that.
I'm thinking Stack Overflow surveys.
The IEEE survey just came out and had Python right at the top, things like that.
But those are all, people will say, well, yeah, that's what that survey said, but it's this slice of people or it's that slice of people.
So the people that took this survey were primarily coming from the Python community itself, right?
So let's talk about where, maybe Eva, you could cover this.
Where did the participants actually come from?
Like what sources did you drive them to the survey?
Sure.
So the sources that we used to get people to the survey were Python.org website, mainly through the downloads page and the homepage.
PSF Twitter account, the Python blog, LinkedIn, as well as I believe your podcast even brought some people in.
Yeah, that's, I saw you list Python podcasts and I'm like, wait, is that mine?
Yeah.
That's cool.
Yeah, that's great.
So 60% came from Python.org and then maybe social media pulled in.
Right.
Or you guys maybe pulled in another 30%.
So the vast majority was sort of from the PSF, right?
Correct.
Yeah.
And for the next version, we definitely want to do more one-on-one outreach to communities that might not necessarily have the need to go to Python.org.
Because mainly those are people that are downloading or they might not follow us on Twitter or whatnot.
So we're going to do a lot more one-on-one outreach as well.
Yeah, that's really tricky though, right?
Because if you do one-on-one outreach to certain communities, they might have really different views or like really different uses, right?
Different style.
It'll definitely give us the global outlook, right?
Yeah, absolutely.
All right.
So let's, I guess, maybe let's start talking through some of the results.
We could just start at some really high-level stuff like age.
Maybe give us a quick summary of the ages that people took this.
Was it like mostly young students or what kind of professional developers?
What did you see?
We definitely saw a younger audience.
Half were between the ages of 18 and 29 and a fourth were between the ages of 30 and 39.
Yeah, that's good.
There were still a fair number of people who were older.
I know a lot of folks feel like if they're, you know, 40, it's too late to learn the program.
But there was plenty of folks in those age ranges, right?
When you put all of the older categories together, they definitely added up to a significant amount.
Yeah, that's great to hear.
Because I feel like a lot of people are like, well, I missed my chance to be a developer.
And I just don't feel like that's really true.
Another interesting way that we were marketing the first one that we need to provide for the second one is to reach that older audience as well.
All right.
Absolutely.
The whole dark matter.
Thanks.
This portion of Talk Python to Me is brought to you by Linode.
Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?
Look past that bookstore and check out Linode at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Plans start at just $5 a month for a dedicated server with a gig of RAM.
They have 10 data centers across the globe.
So no matter where you are, there's a data center near you.
Whether you want to run your Python web app, host a private Git server, or file server,
you'll get native SSDs on all the machines.
A newly upgraded 200 gigabit network, 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.
Do you need a little help with your infrastructure?
They even offer professional services to help you get started with architecture, migrations, and more.
Get a dedicated server for free for the next four months.
Just visit talkpython.fm/Linode.
Another interesting fact that you guys had highlighted in your talk was that from younger developers who participated,
more disproportionately came from outside the U.S., I guess is what I'm trying to say.
We had a lot of people that came from India and China between that age range.
Yeah, I think it's interesting that maybe people see more opportunity or better opportunity by jumping into software development there,
whereas in more stable traditional markets like the U.S. or Europe, maybe there's less people going to software development.
I don't know.
It's an interesting takeaway, though.
Yeah, it could also depend on, you know, what schools are using.
It's an unknown at this point.
Yeah, for sure, for sure.
How about countries?
So speaking of coming from outside the U.S., like U.S. was probably still the most popular, right?
Yeah, the U.S. brought in 18%.
What about other ones?
Where else did people come from?
On the second place, we have India with 13%, China 7%, 6% for United Kingdom, 5% Germany,
then goes Brazil, France, Russia, Poland, Canada.
Yeah, it starts to tail off pretty quick, probably, at the far you go out.
Yeah, exactly, exactly.
So Brazil has 4%, and like the next France, it's 3%, and all the rest is 2% and 1%.
Yeah, interesting.
I wonder how much of this is a language thing, right?
To say that almost as many people came from India as they did from the U.S.,
whereas English is not the first language of folks in India.
So that's almost a stronger endorsement to have 13% coming from there, right?
True, true.
Pretty interesting.
How about an experience, like years of experience?
You said, sort of ask people how long they've been in programming.
What's the time that an average Python developer has been a Python developer?
So the Python survey respondents reported a wide range of experiences.
Like we had 22% of people with less than one year of experience in the IT industry.
We had the same percentage for people with more than 11 years of experience in software development.
And there was a smooth distribution in the middle.
Yeah, so some new people, which is great.
You've got to have new blood, or it's not going to be a vibrant community,
but also some experienced mentors around, right?
Yeah, exactly.
I think this shows how strong the Python language is growing to be.
Just like you said, Michael, you need newcomers,
and you need people that have been doing this for many years in order to be a successful community.
Yeah, absolutely.
Another thing that was interesting, you measured this in a couple of ways to try to get the sense of
how large of a team or what types of folks work there.
So you have, do you work by yourself or do you work in a team was one interesting thing you asked.
And then the other was, how large of a team do you work in?
So let's see, you guys want to speak to that?
Sure.
So in terms of working in teams versus working independently,
we found out that three out of five developers work on independent projects.
So 56% work on their own projects independently,
while 4% work on it as external consultants.
And then in terms of team sizes,
about three quarters of the developers who work in teams work in very small teams.
So 74% of their respondents in the survey said they work in team sizes between two and seven people.
And then the next highest group was 60% in eight to 12.
And that's already getting kind of large.
Yeah.
It's most of them right there.
I think it's really interesting to measure.
And I think it also measures something interesting about Python as a technology.
So when I think of like big corporate teams,
you know, I've done lots of training and I've traveled to lots of companies and worked with their various teams.
And you think of like these like .NET folks and Java folks on these like large teams that are building like enterprise software and stuff like that.
And those, I think, probably tend to be a little bit more than what you're measuring here.
And Python, I think one of its main advantages is it doesn't require a team of 100 people to do the same thing.
And I think you're kind of measuring that here.
Or there's evidence of that here.
Yeah.
It could definitely lead to evidence of that being the reason.
Absolutely.
Yeah, there's a really interesting story that came out from Mike Driscoll's Python interviews book.
And I'm sorry, I don't remember who he interviewed because there's like 10 people we talked about that actually talked about this history.
But there was, back in the day, there was Google Video and there was YouTube.
And YouTube was a little scrappy startup using Python.
And YouTube had a team of like 100 people using C++, I think they had said.
And little scrappy YouTube kept adding features and features faster and faster than this large C++ team at Google.
So Google eventually just bought them.
Just bought YouTube as a way to win.
But, you know, it's kind of, I feel like this part of the survey kind of captures that contrast there.
And I think it's pretty interesting.
So let's talk about jobs and what people are doing.
You ask about whether people were full-time employed, whether they were students, whether they're freelance or self-employed,
which is an interesting distinction.
I think it's a proper one, but it's interesting.
What did you find there?
So we found that the large majority of the respondents, 52%, are fully employed by a company or organization.
And then to what we were talking about earlier, 28% of the respondents are students.
Yeah, that's pretty high.
I think that's a good thing, though.
I mean, students now become professional developers a couple years later, right?
Yeah, absolutely.
It keeps that funnel being filled from newcomer to longtime contributor.
Yeah.
Well, it's also good to see that Python is finally being introduced in schools because that is something that we're continuously discussing, even at the PSP level.
Yeah, maybe we could talk about that just for a second.
And I'm sorry I only have, like, really U.S. visibility into this because I don't pay that much attention to schools all over the world.
But I know in the U.S., like, Python has been becoming or is the most taught first-year computer science language at universities, which is great.
And I think that that trend is generally global.
So, you know, I think that's a good thing.
And that's a good thing.
That's a good thing.
And that official program is, like, nationwide.
And I think it's still in Java, which makes it really, really, basically, just nobody seems to want to change that to catch up with the times because, you know, it's the way we've done things, right?
Do you ever address that or try to get around that or try to convince people maybe Java is not the best option for students to start with?
So this is a continuing discussion, I guess, debate, argument, whatever we want to call it.
So at PyCon, we even have a dedicated workshop called the Education Summit where educators get together and they discuss how to approach these situations and how to best convince people that Python is the right way to go, whether it's, you know, the tools.
That they have set up for children to learn Python or the type of support that's out there.
But mainly, I think one of the scenarios that we're running into is a lot of places just lack the instructors, maybe.
Yeah.
Who's going to reteach all the teachers that have spent all that time to learn Java, right?
They're not really developers.
They're probably, like, math teachers or chemistry teachers or something, right?
Exactly.
Interesting.
Yeah.
I'm sure that's a constant challenge that you guys deal with as a PSF.
All right.
How about salary range?
Like, one of the things that I think probably hasn't hurt Python's growth is you'll see a lot of different surveys from, like, Indeed or the job board type places where Python developers get paid above average, at least in the software space.
What do you find?
So we found that some are definitely above average.
I'm not sure what we want to call average at this point.
It varies so much by a region, right?
Right.
Exactly.
You live in the Midwest in a small town versus you live in San Francisco.
Like, one of the rich people, same salary in San Francisco, you might not be able to afford a house, right?
It's crazy.
Exactly.
And that is even a bigger range when you compare it to countries like India, right?
So I think that that even is evident in some of the other surveys that we see other communities put out.
But, yeah, it's hard to say what, which groups are above average and below without knowing that.
Sure.
So one of the things that you covered that I thought was really interesting was global job demand.
And if you take just the straight numbers, demand for Python developers in the U.S. is, I can't remember.
I'm trying to visualize the graph from the last time I saw it.
I think it was roughly like twice as much as any other country.
But that's partly because there's so much tech.
So what I found was really interesting was how much growth there was for Python jobs in different countries.
Yeah, exactly.
So I used the job board that's on python.org to gather some of those stats.
So I'm not sure how well those results, you know, actually blanket everything that's happening.
But it's definitely amazing to see what's happening in Brazil right now, how it's, you know, doubled in percentage in some of the areas.
Especially in countries that we might not consider as tech hubs.
But this is telling that maybe in the future we will.
Yeah, that's really interesting.
It definitely could be.
Yeah, so you had like year over year growth from 2016 to 2017, like four times the demand in Brazil, twice in China, and almost not quite twice, but quite a bit in India as well.
And those are all really good signs for people that know how to write Python.
Yeah, absolutely.
Probably this chart explains the numbers that we gathered with the survey on the distribution among countries.
Yeah.
So we also see here India has grown a lot, China, Brazil.
Those were all the ones where the participants were now coming from, right?
Yeah, exactly.
Yeah.
Yeah, great.
So, Dimitri, one of the things that you talked about on the survey that I thought was pretty interesting was, and we kind of touched on this earlier, is, are you a Python developer or a developer that uses Python?
So one way you tried to get at that was, is Python your main development language?
So what did you find out there?
Yeah.
So with this survey, we identified that almost four out of five Python developers use Python as their main language.
And comparing to the previous survey, it's 4% increase.
Right.
So we have more Python developers and the ones, and among them, more of them are choosing to make it their main language.
True.
Yeah.
So, yeah, and I think that this can be explained by rapidly growing popularity of Python among data scientists.
Yeah.
Because this segment of users is growing much faster than others.
Yeah, I guess if I was going to take, like, what is the one, if there was one biggest takeaway from your survey that you two did, it would have to be that, wow, data science is making an impact.
Absolutely.
True.
Right?
There was a lot of growth.
If you look at the growth curves of Python popularity over time, there seemed to be a pretty strong increase in the rate of growth 2012, 2013, and then it's gone on from there.
And I feel like that's when, you know, a lot of, you know, Jupyter is kicking in, the machine learning stuff in Python is kicking in.
There's just a lot of people are sort of moving away from R and those other languages and, you know, MATLAB and whatnot into Python.
So that definitely shows up here, doesn't it?
Mm-hmm.
Yeah, so you asked, what are the types of, like, what type of developer are you or what types of development do you do?
And you have 50% data analysis and 48 web developers.
Yeah, that's right there, isn't it?
Yeah.
Actually, in the survey, we asked two similar questions.
What do you use Python for where respondents could select multiple answers?
And what do you use Python for the most with only one possible answer?
So you just described the results from the first question where multiple answers were allowed.
What about the other one?
Yeah.
About, yeah.
About the other one, when we asked about the primary type of development, 26% of Python users identified as web developers.
Outpacing data analysis reported by only 18% by a wide margin.
However, when we combined, we had two options for data science, actually, data analysis and machine building.
And if we combine those two types of development, we come up to 27% of people doing primarily data science.
So actually, there is 26% of web developers and 27% of people doing data science.
So it's like one-to-one.
And you asked this question on the survey.
You said, you asked them, I can't remember exactly what the question was, but it was like, what do you expect people primarily do?
Or what ratio among data scientists and web developers?
Yeah.
And one out of 10 got that ratio correct.
Yeah, that was one of the most intriguing questions in this survey.
So we asked people to estimate the ratio between web developers and data scientists in the Python world.
Respondents could answer based just on their own experience, belief, and general gut feeling.
Because we explicitly asked about primarily types of development in the previous question, we could compare the community perception with the real stats that we've got from this survey.
So based on the data we've got from the question about main type of development, ratio is one-to-one.
And just as you said, only one in 10 people guessed this correctly.
Yeah.
And the most common answer for the community was there are five times as many web developers as there are data scientists.
Right.
Exactly.
That's way, it's not like a little off.
What I find most interesting about this, like this is already super interesting, but what I find even more interesting is half of the people who took this survey are data scientists.
And still, that was the most common answer.
So in some sense, I think it's interesting that even probably among data scientists, they feel like they're more underrepresented than they actually are.
That's a great point.
Yeah.
Isn't that crazy?
Yeah.
So I think, you know, maybe this will sort of shine a little light on that, right?
That people are like, wait, most people do what I do.
I thought I was the weird guy in the corner that they came to ask for graphs or whatever, right?
Well, hopefully it will bring many communities together.
Yeah.
You know, it makes me think of the keynote from PyCon 2017, I think it was, when Jake Vanderplass talked, you know, he did a great keynote there here in Portland, actually, and about Python, the Python community being a mosaic of all these different ways in which people use Python and their goals and what their whole training and what they want to do and the types of things they're building and just appreciating the differences.
And I think this is just one more reason we should appreciate that keynote.
Yeah, I agree 100%.
Even when we look at the 2018 keynote by Q, and she's talking about kids learning how to use Python to program Raspberry Pis so they can water their flowers outside.
It's a wide, wide range and a big mosaic.
So I'm glad you brought that up.
This portion of Talk Python to Me is brought to you by Brilliant.org.
Many of you have come to software development and data science through paths that did not include a full-on computer science or mathematic degree.
Yet, in our technical field, you may find you need to learn exactly these topics.
You could go back to university.
But then again, this is the 21st century and we do have the internet.
Why not take some engaging online courses to quickly get just the skills that you need?
That's where Brilliant.org comes in.
They believe that effective learning is active.
So master the concepts you need by solving fun, challenging problems yourself.
Get started today.
Just visit talkpython.fm/brilliant and sign up for free.
And don't wait either.
If you decide to upgrade to a paid account for guided courses and more practice exercises,
the first 200 people that sign up from Talk Python will get an extra 20% off an annual premium subscription.
That's talkpython.fm/brilliant.
So another hot button item in the community has got to be Python 2 versus Python 3, right?
Yeah, exactly.
So with this survey, we've learned that Python 3 is now a strong leader with 75%.
And Python 2 is used as the main interpreter by only 25% of people.
75% Python 3.
Yeah.
And the share of Python 3 is growing rapidly.
In 2016 survey, we had just only 16% of people using Python 2 and 40% for Python 3.
So it's two times increase for Python 3.
That's really great.
I think it's super good news for the community.
I feel like the story used to be I can't switch to Python 3 because library X, Y, and Z doesn't support Python 3.
And I need it, right?
And now I feel like the story is there's these three libraries I really would like to use, but they only support Python 3.
All the new features are only Python 3.
Yeah, true.
Another interesting finding from the same question that the distribution of Python 3 users is different comparing web developers and data scientists.
So among web developers, only 70% of people use Python 3.
Why do you think that is?
I think that is?
I think that this can be explained that web developers still have to maintain some legacy code while transitioning to Python 3.
And many data analysts and machine learning specialists.
They have joined the community just recently and went straight forward to Python 3.
Yeah, I think you're probably right.
This is a really great stat because I was talking to somebody on Twitter talking about a colleague saying, hey, my colleague says we should be using Python 2 because almost everybody uses Python 2.
There's this graph or that graph that they found somewhere.
And it's not just not does everybody, you know, it's not false that everybody uses Python 2.
It's like really false, right?
75% of the people are using Python 3.
And I think this number is going to rapidly increase as we get to January 1st, 2020 when Python 2 is no longer supported.
True.
So I'll throw this out to all the consultants out there.
So I have a theory that October, November, and especially January, right around that time frame.
So October, November, December of 2019, and January 2020, there are going to be a lot of executives and stuff going, wait a minute.
Are you telling me that our main platform for our website or our company actually no longer supported if there's bugs?
We can't get them fixed?
It's like, yeah, because you guys wouldn't let us rewrite it in Python 3.
So here we are.
And it's going to be kind of like the Y2K.
There's going to be all these consultants that are like, hey, I'm a specialist in converting from Python 2 to Python 3.
And there's going to be a lot of work around that.
That's a great comparison.
Yeah, exactly.
Great job opportunity.
Exactly.
People out there listening, you have about a year to get really good and market yourself as a Python 2 to 3 expert.
Yeah, it's pretty cool that this is out there.
All right.
So I guess we could talk a little bit about what languages and frameworks are popular.
What did you find there?
Yeah, the most popular languages used together with Python is JavaScript.
So 50% of Python developers use JavaScript.
You know, that ties almost exactly to the 49% of web developers.
True.
Right?
Another thing, we also found that HTML and CSS used by 49%.
So it's a pretty close number, which kind of confirms the same thing, like the same takeaway.
We also have 39% of people using SQL, which kind of also related to web development.
Bash and Shell, 39%.
Interestingly, 34% use C and C++.
23% Java.
14% PHP.
That's interesting.
11% C#.
Yeah, that's pretty interesting.
Certainly there's the web languages tie in there and the database stuff.
What about like popular libraries like Django versus Flask, those types of things?
Yeah, we identified that Django is the most popular framework.
It's used by 41% of Python developers.
It's not a big surprise, is it?
It's not.
So one thing that's interesting about your analysis here is it's kind of interactive.
You can see all the results are from a web developer view or a data science view.
So you have Django at 41%.
You have NumPy Pandas at 39%.
So they're almost tied in popularity.
But if you say, show that to me for web developers, it's 70%.
75% of web developers are using Django.
We're just a little bit of NumPy.
And data science, it's like you flip the graph, right?
Quite a bit.
We just kind of expect it, yeah.
Yeah.
You also have an analysis of like additional parallel technologies like Jupyter and Docker
and Anaconda and so on.
And similar stuff, right?
Like web developers, more Docker, data science, very much Jupyter.
Yeah, when asked about those additional technologies, Jupyter Notebook ranked highest at 31%,
which makes sense given the large number of developers involved in data science.
Yeah, I thought another interesting thing that you guys spoke about was cloud platforms.
Oh, yeah.
When we asked which one.
Yeah, which one do you use?
Like when I think of it, certainly the 800-pound gorilla in the room, the big thing is AWS, right?
True.
And you hear stats that Microsoft Azure is in second place among all the other ones.
But restrict that view to the Python community and that's not true, right?
Mm-hmm.
Exactly.
So Microsoft Azure ranked at 16%.
What about some of the other ones?
What was number two?
Yeah.
Number two was Google App Engine.
We had Heroku with 26%, DigitalOcean with 23%, and the fifth place is for Microsoft Azure.
Yeah.
Yeah, that's pretty interesting.
You could tell for sure that AWS plays an important role because when they have an outage, it seems like the internet itself is broken.
Why is everything down?
Oh, AWS Virginia is down.
I get it.
Yeah.
This question about popular cloud platforms.
We didn't list a lot of other popular options.
However, people answered in the other option.
So they mentioned Linode, Python Anywhere, OpenShift, and OpenStack.
So these platforms are also quite popular among Python developers, and we are going to include those options in the next iteration of our survey.
Oh, yeah.
That'll be really interesting.
Well, I also feel like some of these have gotten more momentum in the most recent times.
So I'm very interested to see what the next version of the survey is going to bring in comparison of this one.
Yeah.
You'll start to be able to develop these trends, which will actually be super interesting.
I think that would be the most interesting part.
Yeah, for sure.
Exactly.
It's going to be the main feature of our next survey.
Yeah.
So, Dimitri, since you brought up Python Anywhere, something that really surprised me a little while ago, well, quite a while ago, actually,
but I had gone to python.org so many times, and there's a little text box with, like, code right there and, like, a prompt-looking button.
But what I didn't realize is if you click that prompt, that will open an interactive shell on Python Anywhere where you can just start typing Python.
Yes, it will.
That's pretty cool.
I wonder how many people actually know about that.
We probably don't tweet about it as much as we should.
That's a good point.
Yeah, it's cool.
I'm looking at, like, a Bash shell or actually a Python REPL on C6 right here on Python.
Exactly.
3.6.
Very cool.
How cool is that?
Yeah, it's super cool.
All right.
So, let's see.
Maybe we can talk a little bit about tools and features.
One of the questions you asked were, here's a whole bunch of features that editors and other tools, like linters and source control,
how much do you use them or not use them?
Do you want to touch on that, Dimitri?
Yeah, sure.
So, the top spots were occupied by code up to completion, code refactoring,
writing unique tests, using virtual environments for Python projects.
On the other hand, no-scale databases, Python profilers, and code coverage tools were among the more rarely used tools and features.
Yeah.
So, I think it's too bad that, like, the profiler and code coverage weren't used that much.
But, you know, they're such a valuable thing that I think it's overlooked.
Yeah.
And, by the way, the results that they've got in 2017 is very similar to the data we've got in 2016.
So, the picture or the usage of those tools doesn't change much, seems to be.
Okay.
Interesting.
I should probably ask about this one.
Dimitri would probably be really more than happy to answer this one.
What about editors?
Well, for editors and IDs, we asked two similar questions.
So, one of them was, what editor or ID have you considered for use in your Python development?
With multiple answers allowed and a single answer option, a question, what is the main editor you use for your current Python development?
So, PyCharm is the most popular tool.
We have around...
You have 32% of the people...
Yeah, 32% for PyCharm.
Yeah.
For both editions, PyCharm community and PyCharm professional.
And that's, I mean, like, I think that number would be massively skewed if, like, you and JetBrains were the ones promoting this project.
But, like, we talked about the sources, right?
It wasn't really.
Yeah.
And then what are some of the other editors people use?
Some of the other popular editors are Sublime Text, Veeam, Idle, Atom, and VS Code.
Yeah.
I suspect the Idle speaks a lot to the students.
And I also think...
I feel like, you know, the survey was done leading up to October 2017.
I feel like VS Code will make a much higher appearance.
Yeah.
Right?
Like...
We are really looking forward for an updated results this year.
Yeah.
I think the VS Code team has, like, really put a lot of effort into Python.
I've started...
You know, I ask this question every time at the end of the podcast, what editor people use.
And I hear VS Code showing up more, a lot more than it used to.
I see a lot of great positivity around it on Twitter as well.
Yeah.
For sure.
Another thing...
Yeah.
Another thing that's interesting, I think, is to discuss how people get Python, right?
Like, Eva, I'm not sure what you can do, but if you could just get Microsoft to ship Python 3 with Windows, we'd all be in a good place.
But that doesn't happen.
So there's different ways in which we might get Python on your system.
So you asked that question, and some of the options were like, you go to Python.org and you click the download button.
Or use something like Homebrew or Aptitude on Linux or Anaconda or surprisingly many people build it from source.
7% Yeah.
True hackers.
Exactly.
That's a lot.
7%, right?
I mean, I know people do it, but that seems like a lot.
Anyway, so I guess we had 39% go and download it from Python.org, 31% get it through some kind of package manager in their OS, and 15% in Anaconda.
Correct.
Yeah.
Those are pretty interesting things to know, I think.
Well, it makes me happy that, you know, one of the main infrastructure items that the PSF helps maintain for the community is Python.org.
So I'm very happy to see that it's being used worldwide.
Yeah, absolutely.
I guess maybe the last question directly we can touch on is operating systems.
Because to me, if I had to be somehow stripped of my knowledge and just sort of walked around conferences or something, I would feel like, well, the majority of the Python world runs on Mac, right?
I would think so, yeah.
Because, like, everyone's carrying around a Mac.
They got all their stickers on it.
They're all cool.
But that's not exactly how it breaks down, is it?
Yeah, exactly.
So it feels like a very limited view.
Well, exactly.
That's why we think that the survey is more of a global outlook on how things are versus the way that we see them.
Yeah, so your survey found that 49% of the people use Windows, 19% Linux, 15% Mac, and then other people just, I don't know, other stuff.
Which is, I think, is pretty interesting.
And it really, you know, I try to drive the point home that Windows is super important in the Python space.
And I know it's a lot of, I think, yeah.
And I know it gets a lot of love.
But also, I feel like periodically packages people build, they're not tested on Windows, or they won't install on Windows.
Or if they do, they're hard.
Unlike Mac and Linux, it might be just pip install.
But on Windows, you've got to install this other weird thing to make it work, you know.
So I feel like Windows kind of gets lost in people thinking about their, where people are using Python.
But in fact, it's more than two times as popular as anything else.
Yeah.
Yeah, we're thinking about.
All right.
Well, I think that's probably it for the topics of the survey directly we want to cover.
Well done, you guys.
That was really nice.
Yeah, thank you.
Thank you.
People want to give you feedback?
How do they do it?
They should email us.
We have a mailing list.
And we also have a GitHub repo.
So the mailing list is surveys at python.org.
And that will go to Dimitri and I.
And the GitHub link, Michael, can you include that in the links below?
Awesome.
Yeah, I'll definitely include it.
So the link to the GitHub repo, which has an issue tracker where people can submit issues.
That's cool.
And then also...
Also feedback, not just issues.
But if they have any ideas for improvements or, you know, let's say they know of a community
that wants to get involved for next time, they could definitely post it there.
Yeah, interesting.
What about the raw data?
Like, Dimitri's team took this and came up with what I said was a really beautiful result.
But, you know, it is an interpretation of the data somewhat, right?
Exactly.
Can people do their own interpreting?
Yeah, absolutely.
So along with the report, we also shared the raw data in CSV format.
Okay, cool.
And yeah, that's...
You can find that off the linked survey results.
If you go by the link to the report, down in the bottom of the page, you can find the link
to the raw data.
Yeah.
I won't link to it directly because you have a couple of things like, please consider this
as you look at this data.
I'll let people read that first.
But yeah, nice.
Yeah.
Unfortunately, last year, we didn't provide Python scripts, which people can use to, like,
bootstrap their analysis with this raw data.
Yeah, that was a suggestion.
Yeah, that was a suggestion after our talk.
Right.
So maybe a Jupyter Notebook that does the analysis or something, right?
Exactly.
Yeah.
That'd be really sweet to have.
All right.
So you're doing this again?
Absolutely.
Very soon.
I'm not sure when you're going to be airing the episode, but we're planning on launching
in October.
Okay.
Yeah.
So this episode will be out really soon.
So people should have some time to get ready and prepare for its arrival.
But sometime in October, do you actually have a date when you're letting out or just general
timeframe?
We are going to start the survey in the beginning of October.
We don't have an exact date, but stay tuned for the beginning of October.
All right.
Cool.
And I'll try to spread the word again to all of the listeners when that happens.
Yeah.
All right.
Well, I think we probably should leave it there.
We've covered a lot of stuff.
And I really think this is a great service you all are doing for the community.
And like you, I think the trends are going to be as relevant as the actual numbers.
Yeah, I agree.
And I'm very happy that it's gaining some momentum and interest.
And thank you so much for inviting us to talk about it.
Yeah, absolutely.
So final call to action.
Maybe people could spread the word.
Like when you hear that the survey is out, tell everyone you know.
Encourage them.
Drop it in your internal company's Slack channel.
Things like that, right?
Yeah, exactly.
We encourage everyone to take part in this survey and promote it to everyone.
Post it on social media.
Exactly.
Take it to your user group.
Yeah, that's a good idea.
Take it to the user group.
Very cool.
All right.
Eva, Dimitri, it's been great to have you on the show.
Thanks for doing this project and sharing it with everyone.
Thank you.
And thank you for having us.
Yeah, thanks, Michael.
You bet.
Bye.
This has been another episode of Talk Python to Me.
Our guests on this episode have been Eva Jodlowska and Dimitri Filipov.
And it has been brought to you by Linode and Brilliant.org.
Linode is bulletproof hosting for whatever you're building with Python.
Get four months free at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Brilliant.org wants to help you level up your math and science through fun, guided problem solving.
Get started for free at talkpython.fm/brilliant.
Want to level up your Python?
If you're just getting started, try my Python jumpstart by building 10 apps or our brand new
100 days of code in Python.
And if you're interested in more than one course, be sure to check out the everything bundle.
It's like a subscription that never expires.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and
direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I'll see you next time.