#46: Python in Movies and Entertainment Transcript
00:00 What did you experience the last time you watched a movie in a theater? Were you captivated by fast-paced action and special effects? Deeply moved by the characters that came to life during those two hours when the outside world just melted away? Yeah, movies are still magical.
00:00 What was likely not top of mind was all the work that went into that movie, from the editing of the audio and video, the special effects, renderings, and coordination of maybe 100's of creative professionals. It turns out that Python plays a key role in coordinating all of this production work and that's what this episode is all about.
00:00 Join me as I talk with Rob Blau from Autodesk about Python in the movies and entertainment business. This is Talk Python To Me, episode number 46, recorded January 28th 2016.
00:00 [music intro]
00:00 Welcome to Talk Python to Me. A weekly podcast on Python- the language, the libraries, the ecosystem, and the personalities.
00:00 This is your host, Michael Kennedy. Follow me on twitter where I'm @mkennedy
00:00 Keep up with the show and listen to past episodes at talkpython.fm and follow the show on twitter via @talkpython.
00:00 This episode is brought to you by Snap CI and Opbeat. Thank them for supporting the show on twitter via @snap_ci and @opbeat.
00:00 Hi everyone. Two weeks ago I told you I had a huge announcement that I was really excited to share with you but couldn't.
00:00 Now that everything is in place, it's time to share my big plan with you.
00:00 When I started this podcast almost a year ago, I had modest expectations and more than a little uncertainty about how it would be received by the community. And honestly, every day I'm blown away by how many people's lives I touch and help educate in some part of Python that was less well known to them.
00:00 I get a huge amount of satisfaction spending time making Python more relevant to you through this podcast. I've been thinking about how I do more of that.
00:00 I believe the natural counterpart of this podcast which exposes people to new ideas and inspires them to learn more about Python is a comprehensive set of online courses to help you go from inspired to empowered, from new developer to highly effective developer, from a specialized developer to a well-rounded one.
00:00 Today I'm announcing the first course and taking the first step on a journey to build this resource for you and the whole Python community.
00:00 Over the next two years, I plan to release 20 high-quality online courses that will be between 3 to 10 hours each. These no-fluff courses will strive to make you effective with some corner of the Python ecosystem including the python language, web applications, databases, GUIs, parallelism, and more.
00:00 The first course will be about the Python programming language and will take a fun and playful approach of teaching you language details such as classes, loops, variables, and more. It's called Python Jumpstart by Building 10 Applications, it will be about 6 hours long, and will be a comprehensive introduction to the python language.
00:00 If this project resonates with you, then I need your help. As of today, I'm launching a kickstarter to make this first course, this first step on my journey, a reality.
00:00 Please, take a moment to visit the kickstarter to see what it's all about. Just go to talkpython.fm/course and you'll be redirected to the kickstarter page. There are many reward levels but main one is the $29 student reward which gives you lifetime access to the course at a significant discount.
00:00 If you believe in this project and want to help, please tweet about the kickstarter, share it with your coworkers, or even back it yourself.
00:00 I'm really excited to start down this path to build something amazing for the community and I can't do it without you. So any support or word-of-mouth endorsements you can give me will be deeply appreciated.
00:00 Please let me know you think. Send me a message over email,contact@talkpython.fm or on twitter via @talkpython.
00:00 I'm also doubling my efforts on the podcast as well. So keep listening and sending in suggestions and feedback. I'll do my best to bring you interesting and relevant episodes.
00:00 Now, let's hear from Rob Blau about making movies and video games with Python!
04:13 Rob, welcome to the show.
04:13 It's great to be here, thanks for having me.
04:16 You are welcome. Thanks for coming, I am super excited to talk about Autodesk. You guys kind of make a little bit of everything, there is so many different pieces of software that you write, and it sounds like Python plays some pretty cool part in that?
04:29 Yeah, definitely. So, Autodesk, the company mission is to help people build cool things, so pretty much software that helps people realize their creative vision, the largest pieces kind of like Autocad, like production, architecture side, and the piece of Autodesk that I work in is the media and entertainment division, so we get to work with animation analyses, visual effects houses and help build a software that they use to do the cool things they do to put pixels and games together.
05:09 That's really cool. If I jump on like a train and I go over to the airport and I get on a plane, fly somewhere and find my way to a movie theater, like how many of those different experiences are like modeled or somehow created with Autodesk?
05:28 The chances are every single step on the way, from the car that you jumped into, somewhere along the way from the factory that it was created in and the robots in it they were designed using some of the Autodesk software, the actual car itself, the models, all of the visualization that people do in order to figure out what it's going to be and then the actual instructions for building it, Autodesk makes products that do that. The city streets and like the planning, it's incredible, the places that the software is used, then at the airplane, the exact same thing. And then, you are going to the movie theater, and movie theater itself is probably done by an architect using Autocad in some way shape or form, or Redit which is used to kind of build bugger projects and make sure that all of the various components in the building, like the h back and the design and the wiring all talk to each other and will work once they actually build it. And then, you finally get the movie showing and you start seeing nowadays almost every single shot has some kind of visual effect on it, whether it's just a little bit of 2D cleanup that happens, up to the full 3D, the actual environment is completely created from scratch inside of some software and Autodesk makes some of the most widely used pieces of software for that, like Maya 3D studio Max, all of those are used to model and render the way we are seeing it up on screen.
07:00 That actually kind of blows my mind, that's even more intense than my original thought, that's awesome.
07:06 It's really cool. Fun mission to have, to just make it easier for people to realize their creative vision, and it's a nice one.
07:14 Yeah, I bet it is. So, let's totally dig into that, but before we do maybe you could sort of give me your background story, like how did you get into this place where you are working on basically building everything that makes up the synthetic world?
07:26 I had kind of a roundabout way there. I went to school for computer science, I was living in Boston, went to MIT and graduated around the first tech boom, did a couple of .coms at that point, and built up kind of just a varied background, I did a lot of scripting, a lot of database work, a lot of just systems admin kind of stuff, just build the startup culture and you did whatever you had to do to get going. And form that, I went to actually Linux conference, and saw a presentation form DreamWorks animation, where they were presenting how they were using Linux because they were one of the first companies to switch over to kind of big open source software as the foundation of the technology that they are using.
08:17 Interesting, what were they switching from, do you know? Silicon graphics?
08:21 Yeah, back in the days kind of silicon graphics, so they have kind of had deals with the Red Hat going and figuring our kind of enterprise Linux distros, they were early in adapting all that stuff.
08:32 So you intersected there when they were talking about how they were using Linux?
08:34 Exactly, and I got to talking with them and it turns out that they had interesting job that was doing, it's called pipeline work, in the visual techs industry, which is the software that helps tie together what all of the different departments do, so you've got an artist sitting there, they've got very specific job to do, it's almost like in an assembly line in a factory, you've got your station where you are doing your very specific thing except with an artist, it's a very creative process but they still have to take work done by somebody else, get it working for them, do what they are supposed to do to package it up for somebody else and then hand it to somebody else. And all of those pieces other than that creative work is kind of overhead, that it's nice to artists to not have to do, and that's where this idea of pipeline comes in. I was kind of fascinated by that right from this original conversation, plus I had about ten Boston winters and the idea of California sounded pretty good. So, I went to work for DreamWorks where I was for 5 years, and I was a member of their pipeline department, ended up being a supervisor in that department and that actually is where I first started using Python extensively, the pipeline at DreamWorks was Perl based, because back originally when you were doing text processing and having to glue together different software, Perl made a lot of sense.
10:07 Right, what year was that?
10:08 That was 2003 and 2004.
10:15 Yeah, Perl was still in a pretty strong position back then.
10:18 Exactly, and the pipeline that I am talking about had been in place for like a good probably 6, 7 years before that even, so when it was written like that was the choice. But, there is something that was kind of interesting that was happening with software in the 3D space, there was already a couple of pieces of software that had Python embedded in them, and Maya actually has one of the biggest- again taking it back to an Autodesk product was introducing a version that had an embedded Python and at that point, if one of the biggest applications that you are using in your pipeline speaks Python natively, there were pretty serious indications the Dream Works as a whole of that maybe committing to Python rather than Perl would be a great idea. Because all of the sudden, all of this integration work that we are doing doesn't have to end outside of the applications that we are trying to tie together, we get the chance to run inside the application with code that you are using to tie everything together, which is amazing, because all of the sudden you can speak the same language throughout the entire pipeline, and do a much more seamless hand off for the artists kind of, which is again, the purpose of all of this is to make it so that they can spend their time just doing as much creative work as possible.
11:45 Right, yeah, that's really cool, rather than trying to integrate Perl and Python and have some sort of handshake or hand off, just make a Python all the way through, right?
11:54 Yes. Which actually was at the time, controversial. It was a pretty big jump to do and it wasn't clear that Python was going to take hold in the entertainment industry the way that it has, but since then, nowadays it's a no-brainer, almost every new piece of software that comes out has an embedded Python interpreter and Python studios around the world is the defacto language that is used in order to do this kind of work, in order to tie stuff together, or even just tools that you write for artists almost it has to be compiled, it tend to gravitate towards Python to do it.
12:33 That totally makes sense to me. What I think is interesting, and I see it happening in data science, I see it happening in education, you tell me it happens here as well, which makes a lot of sense is- it seems like Python is becoming more and more popular, and that popularity is almost accelerating. Like, the rate of popularity seems to actually be growing, which is really cool to be kind of part of it as that happens- why do you think that is the case in the entertainment industry?
13:03 I think, I was thinking about this in advance. It starts by being so easy to embed and having kind of this kitchen sink approach, like those 2 things combined together made it so that with just a little bit of C code all of the sudden, the scripting environment that you could offer people became incredibly rich, and because of the ease of exposing your C, C++ libraries to Python even if you just go natively to the Python C API, it meant that all of these programs that didn't necessarily have a great scripting language, could have one without too much investment. Now Maya actually did have like incredibly embedded scripting language, it still does called MEL, but it was only for Maya and people that were MEL experts, it was a ton of work done in MEL, but it only, your world ended at the boundaries that Maya provided, and it didn't give you too much of a chance to extend outside of that. Whereas with Python, you get the exact same kind of scriptablity, but now you get it off of the rest of the world, rest of the universe, that's also speaking Python which is huge. So, I think the application developers saw that and it wasn't that huge of an investment to say like, "hey let's just put Python in there" and then it paid dividends, and then once that started happening it almost was a no-brainer, like if you are going to be doing this work, you do it in a language that all of the big applications that you are running speak natively and everything just kind of came together around that.
15:01 Yeah, it just kind of snowballs from there.
15:04 Exactly. And then once it kind of establishes the defacto, then everything else that comes in just has to speak it. In the industry, there has been a ton of new open source projects that are going on like a standard way to pass 3D geometry between applications, and standard way of describing what is going on inside of the 3D scene and every single one of those comes with Python bindings out of the box, because everybody knows that they are going to have to interact with it via Python.
15:40 Right. So if you don't have the Python bindings, it's almost like you don't have a mature scripting story or something like that.
15:47 Yes, and that can be the bulk of your scripting story. And if you don't write it then someone will write it really shortly because they will, people need it, if it going to be used, you need to speak Python.
16:01 That's interesting, you are talking about sort of you know, you had the scripting language MEL, and it was doing fine presumably, but it kind of- it could only do its thing, and I think that's also why Python is becoming one of the major reasons why it's becoming so popular in like data science, is there are languages like R and so on, but if you want to build like a full stack sort of ting, like I want to do my the data science part, I want to do my science, and my visualization, and my web app, and my database access- all in Python, you know, you can't do that in these other more specialized languages. So it's interesting to see the parallels over there.
16:40 Yeah. It is. I could actually say that the thing that I am seeing where Python isn't in play as much, comes from applications that were kind of originally developed for either like the very specific workflow or from outside of the industry and they are coming in, a lot of those actually start with nowadays Javascript engines built in because I think the same story has kind of repeated itself a little bit on the Javascript side where it is just so easy to embed at this point that for very little investment you can go ahead and have a Javascript interpreter running inside your app just like you can have a Python interpreter inside your application. Like, within the entertainment industry, those are actually some of the big areas where we kind of have a missing terms of getting everything to talk to each other, Photoshop is a good example of where there is various somewhat convoluted ways to try to get the Javascript that Photoshop speaks to talk to Python outside of Photoshop and various places are doing that but again, it's the idea of just how easy it is to create a scripting environment inside of these applications where there is a lot of value to being able to talk to it without writing compiled code.
17:58 You know, I think the whole V8, the thing happening with node, but the V8 engine itself, being a separate thing that you can host has done some pretty amazing stuff for Javascript. And I think some of the full stack like it's Python all the way through, like the whole node story again V8 is like it's Javascript all the way through.
18:20 Yeah, and it has a lot of the similar kind of parallels where it has a little bit more of the kitchen sink, not as much as Python does, but there is more and more Javascript libraries popping up all the time, just like their Python libraries- it's interesting, this thing that pattern replicate itself, it's kind of detrimental, from where I am sitting, because I would love it if all those things spoke Python, it would make my job easier but it's just kind of interesting, seeing that take hold again.
18:20 [music]
18:20 Continuous delivery isn't just a buzz word- it's a shift in productivity that will help your whole team become more efficient. With Snap CI's continuous delivery tool you can test, debug and deploy your code quickly and reliably. Get your product in the hands of your users, faster and deploy from just about anywhere at any time.
18:20 And did you know that Thoughtworks literally wrote the book on continuous integration and continuous delivery. Connect Snap to your Github repo and they'll build and run your first pipeline automagically. Thank SnapCI for sponsoring this episode by trying them for free at snap.ci/talkpython.
18:20 [music]
19:44 That is super interesting, it makes a lot of sense to say either Python or Javascript as a proposition for somebody who is not a developer, but needs to automate a thing. Right, to say like, "Your API is C++ and you've got to have this compiler with these headers and this libs, statically link," that just doesn't fly, does it?
20:04 No, the stuff that Python is used for tends to be a lot higher level, like workflow, if you are taking a workflow and translating it to code, so it tends to be pretty dynamic, it needs to be updated constantly, almost each project you will be tweaking the stuff to work how the people working on that project need to work to get their job done most efficiently. And it doesn't need to be 20:26 I guess
20:28 Yeah, because it's just orchestrating the things that are prior written in C.
20:31 Exactly.
20:32 Interesting, You were talking about Javascript, and having the Javascript try to interact with Python somehow, and obviously getting out of a Javascript engine can be pretty tricky, because they try to sandbox you a lot. But, have you looked at PyPy.js? Or one of these like embedded Javascript foundation Python implementations?
20:57 I haven't- I guess I should qualify like what I am doing at Autodesk is actually a group that is putting together one of these production pipelines that is just part of production management product that we make, so I am now not working at a studio doing all this pipeline work but working at a software company that is again, just trying to make it so that the various workflows that we see being done in the effects and animation, so just content creation in general, we make it really easy for people to get some of the ground work out of that. Even that said, we're not embedding the interpreters ourselves, like the 21:34 that I am working on we're trying to write the scripting code that's going to be able to talk to that application, so it's like Photoshop as Javascript, we are not going to be able to embed Python in it, although we could do plugin and embedded Python running, but then, there is a lot of threading issues and other stuff that- like we've tried to be as seamless as possible but you can only do so much with an application, it all depends on what kind of STK and API they offer you. So we don't have that level of control, but we are looking at some standard RPC kind of libraries, and like trying to figure out what does make a good bridge between those worlds.
22:18 Right, I think probably the easiest way to bridge Javascript to another thing is some kind of network layer, right?
22:25 Network layer or even if you are not going over the network, exactly, like RPC even if it's just talking to another process on the same machine, making that a really rich communication channel. And there are some impressive projects out there that do that, and we are actually taking off an effort to evaluate them and try to pick one that's going to work well for us.
22:52 One of the things you'd send over is you talked about sort of Python and games and what's the story there?
23:01 Sure games is kind of an interesting take on all of this, because the effects and Pixel production has been around longer, and has a bunch of different constraints than games have. So far, games have really been driven by the game engine and the developers has almost separate from the creation of the models and the shaters that show up in the game, like all of those need to play well together, but it's almost been more of an agile developer kind of style than the cascade of content flowing between different departments, which is what you see in animation and the effects mostly because the complexity of all of the environmental models that the effects has been playing around with, has been higher than what you have seen in games. But that's actually switching, a lot nowadays where the complexity of production in a game is a lot of the times higher, than what you would see in a featured film. The need for this kind of pipeline is showing up in games more and more. But they've got this whole coding side to the game where the artists are just piece of what's going on and they need the developers in order to tie everything that they are creating in, and it all comes together in the game engine that is what's actually going to be running all this stuff at the end of the day. What we've seen there is- there is some game engines that speak Javascript, coming out now, the tried and true has been .NET, just a lot of his has been pretty Microsoft centric, and the .NET code can speak Python via Iron Python that's out there, which has been kind of an interesting 25:01 for us, or LUA actually is another scripting language that a lot of the game engines tend to speak. But it's yet another kind of separate world, even the .NET side has limitations and lets us do kind of basic Python integration, but you start- you hit brick walls pretty quickly when you want to do cross platform networking stuff or guis especially have to start writing things specifically for the .NET framework as opposed to treating it just like another Python interpreter that's out there.
25:33 Right, yeah, you get very specific like WPF type stuff, you can't do like PySide, PyQt type of things that kind, which is much more cross platform. Can you give me just your feedback on like what you've done and what it's like to work with IronPython, I haven't actually, I've played with it and looked at it but I don't really know much about it so I'd be interested to hear.
25:55 I've actually found it to be pretty good, the boundaries that you hit are kind of what you would expect where Python itself starts having dependencies on third party libraries, like SSL documentation, or the GUI kind of side of the world, but I mean, personally, I've had fun little side projects like getting Iron Python up and running in a plugin for Microsoft Office projects, so that there is an API that's available only via Python, you can start interacting with it form within Excel in order to kind of get deep scripting ability within something like that without having to write visual Basics, and again, reuse that Python code base that you've got.
26:38 Yeah, that's quite interesting.
26:41 It's kind of Frankenstein kind of approach, but a lot of the integration that we end up doing has some element of that, because you are really trying to establish this rich Python programming environment, in all of these different spots, some of them you kind of have to jam it in.
27:04 They resist a little more, huh?
27:06 Exactly.
27:06 Iron Python was a Microsoft project, I think it was created by people inside Microsoft, maybe they created and promptly joined Microsoft like, I can't remember the exact chronology of it, but they a while ago, set it free and said, "we are not really doing anything more with IronPython but it's now just sort of open source project, the world can have it," and I just like, a few hours ago, although it's been out for a couple of months learned about this thing called PyJion have you heard of this?
27:38 I've heard the name, but I don't actually know much more than that.
27:42 It's kind of a competitor to PyPy which is a JIT compiled runtime implementation for Python but this is one that runs on the new open source cross platform version of .NET, so maybe there is something interesting there, coming in the future, I don't know, we'll see if it's interesting to see them taking kind of a second shot at something like that.
28:04 Cool, I'll check that out, because something like that could potentially open up like the game engines that speak .NET, the deep integration with kind of the rest of the creative process that share the pipeline from feature animation and the kinds of complex asset bills like the models and the various assets like you see on screen, that process for games...
28:30 Yeah, absolutely, I mean Iron Python seemed really cool, but kind of like you were saying like, its limitation is you are all in on Windows. From, like it only runs on Windows, you have to really program against the .NET so at the time Windows specific APIs, but with the cross platform plus this, you know, maybe, maybe that sort of opens up again, it's going to be really interesting I don't know where it is but...
28:54 Is that the mono environment that-
28:56 No, it's the mono environment was Miguel de Icaza's work on sort of re-implementing . NET which is herkulian, and they've gone on to do Zamron, right with the mobile stuff. So this is a new effort called the 29:13 by Microsoft to do a shell or a core, a kernel of .NET that's totally cross platform that you can do on OS10 and Linux, and sort of but the official not like sort of friend of me type project, or what they are doing, so... So it'll be interesting to see if this can bring a really rich version of Python to that story, kind of like we have Jython for Java, but that's not really opt in really quickly so maybe I don't know, I have high hopes but they make a dash, we'll see. I hope it comes out awesome, we'll see.
29:47 Definitely dig into that, thanks.
29:49 Yeah, you bet.
29:49 [music]
29:49 This episode is brought to you by Opbeat. Opbeat is application monitoring for developers, it's performance monitoring, error logging, release tracking, and workflow in one simple product. Opbeat is integrated with your codebase and makes monitoring and debugging of your production apps much faster and your code better.
29:49 Opbeat is free for an unlimited number of users and starting today, December1 st Opbeat is announcing that their Flask support is graduating from beta to a full commercial product. Visit opbeat.com/flask to get started today.
29:49 [music]
30:42 There sounds like a lot of positive news from all the cool things you are doing and the way you are able to get these pipelines together, I want to kind of talk about what you see the sticking points and so on a bit, but before you do, maybe just to give me- could you describe what the steps are and this pipeline that you talk about, like what are people doing, what is the software have to - just so we get a little more concrete idea there.
31:09 The kind of nature of this is that you can go down these giant rabbit holes, like you can go extremely deep into automating things, and traditionally, only fairly large studios that could afford having an R&D department, had the ability to build deep- lots of places had pipelines but they tend to be a pretty easy hands off, rather than tackling a lot of the deeper issues. It can start as early as a script, if you've got a script, you are thinking of making a movie out of the blue, if you need to do visual effects for, it tends to come in from some program, that stores it as an xml file, you can write a parser for that, in Python and use that in order to get a listing of here is all of the different scenes in the movie, and here is the characters and location things like that, which if you are doing this by hand, traditionally you will do a breakdown where you will just build up a spreadsheet of all those things so that you can start getting your head around what you have to build, you go ahead and you start storeyboarding things so that you got an idea of whatever thing is going to look like, and 32:19 software that is built around storyboarding, and the photoshop is kind of one of few players in that area. Even then you tend to just have these images that you are working on and then tracking those to say ok, here is the current storyboards, with that relate to the scene, you want to hand them off to editorial that's going to start cutting them together into the movie that we can watch to get a sense of how all displays together, there is lots of iterations there that go back and forth so you have to keep track of those files and know which ones are approved, what the notes were, so that whoever is going to be doing the next iteration on that, can take those into account, and it's really trying to put some organization around a very creative, quick iteration workflow, without getting in the way of that creative process. So then you start getting this editorial content together, where you are cutting together these images on a timeline ad you get a sense of timing, that tends to be a place where you need a lot of R&D to dig in, because a lot of the editorial software out there haven't been the based one, it tends to be a little bit of a black box, if your final cup which has an xmls back that you can start taking apart that you don't get a chance to understand too much of what's going on inside of editorial software.
33:38 Sure. And so, are you guys doing like actual processing of like mp4 files or raw assets in Python and look in them, that kind of stuff?
33:48 So an example of how Python would tie in over there, not with Autodesk but my previous job at a studio that's that does stop motion animation, actually there is a C library around this file format called AAF, that is what the 34:06 editorial software speaks and it's a pretty obscure format, very complicated, the API is very complicated, we wrap that, with Python bindings, so that we could take a export from 34:23 and really deconstruct it. Which meant that we actually were able to know which story boards in editor cut and where, and what characters were in that storyboard, so that when you are going ahead and putting together a shot, you would know generally the contents of that, and all the comments that came through, all of the notes that the director had for the storyboard artist, which is incredibly valuable because in the shot, you got to go ahead and take your 3D models and bring them in, so having the list of what has already been included from the storyboard artist gives your layout artist who is doing that work a leg up and you can get started and you can write a tool to automate them, the initial path at that and then let them actually do the creative piece of that which is where he should be spending his time rather than searching for the right models to put in. And the one that gets the animation, it's really valuable for the animator to be able to see all of the notes that the director had for the storyboard artist because they are very applicable to the job that he has to do. And, none of that would be available without being able to connect the dots between all of those different handoffs. So connecting those dots and managing those hands offs and making it so that you can deal with this explosion of data that happens throughout this process, in a kind of tractable meaningful way is what the pipeline is responsible for.
35:49 Ok, that gets me a really good view into I think it seems like there is all these different apps that the various people have to use, editors, artists and so on, and they don't necessarily talk to each other, right, so you are kind of writing this glue that can turn that into like one almost distributed data source, or something, right?
36:12 Right. It's kind of like if you are used to using excel, one front and you've got this text editor that you use on another front if you are able to kind of connect those together because they had knew that you are doing the counting work and was able to actually react to the files that you are saving from the text editor, parse them and automatically bring them into your spreadsheet, so that you didn't have to manually do that each time, you change one piece of information, and get the rest of it in sync with that. It's kind of that on a extremely grand scale.
36:50 I don't know if you have the answer off the top of your head, I don't really know, but I am thinking about all these different processes and all these different raw additions and then produced and sliced up editions, these files, you know, if I go and get like a movie, it's probably a couple of gigabytes, but what's the data the total data size for like a modern production movie, or something, do you have an idea?
37:15 e are in many terabytes land nowadays, yeah. It is terabytes and terabytes in order to do the models and the effects work and all of the simulations that need to happen, in order to get the quality that we are seeing on screen. It's incredible, the amount of data that goes into just those final pixels that you see, so the couple GB download that you get which is a really high quality video that you could watch, yeah, each of those you know, frames has gigabytes and gigabytes and gigabytes, like hundreds of GB behind it, alone. So we put it all together at 24 frame per second for a future length, it definitely adds up.
37:59 Yeah, now well that's crazy.
38:01 It is. The scale and especially the HD amount of data and number of people involved, and amount of handoffs that happened in order to achieve that thing that is on the screen is pretty amazing.
38:15 Yeah, I bet it is. Everything sounds really good, but you said that there is actually some issues that you were running into, like one of them was that using Python 3 in this world is not something you can just necessarily jump straight into.
38:29 A lot of the issues kind of around Python use almost come from how successful Python has been, and how it's embedded in so many different places, we just have the version 38:41 problem where currently the code that we are writing we actually support back to Python 2.4 because there is some versions of software that are there and been updated since, but studios are pretty notoriously slow to update if everything is still working, it had Python 2.4 embedded in it. There is definitely a lot of Python 2.5, 2.6 that is still out there, the majority of modern stuff is on Python 2.7 there is like 1 notable source project that's made the jump to Python 3 but because the nature of this Python work, is to glue stuff together when you had a jump that's not backwards compatible, it kind of makes it hard, because you have to have 2 different code bases, one that will run for all of the environments that are Python 2 whatever based, and one that will run for everything that is Python 3 based. And you can kind of work to minimize this but there is definitely like this hard jump that happens because not all of these pieces of software are going to update their Python interpreters at the same time, and there is nothing that says that you will be able to as studio, be able to update to that version, of that software at the same time. It gets them the whole lot of issues from just cost, support licensing, getting the bugs out of it, down to having the resources to update your code into running Python 3.
40:09 Wow, that's crazy.
40:11 That's going to be interesting 40:12 for the visual effects, animation world to get to the other side of, there are various attempts within the industry, to try to control this 40:24 artist issues because it's not just Python, there is all sorts of common libraries that are used throughout production and whatever they don't match it causes issues, so there needs to be effects reference platform, that tries to standardize things, Python 3 isn't even on the way, I think Python 2.7 or whatever is the target for the foreseeable future, I think it's probably going to be dropping support with the combination of security issues that just force updates that will be the thing that finally gets us an industry to the other side of that, but it is kind of unfortunate because of this wide range of versions that get the support with dependent libraries, and all of the complications that come up trying to develop software in that kind of an environment.
41:14 Yeah, that really that's quite challenging, it sounds like I mean, when I think of people that are using Python to and they say we can't switch to Python 3, in my mind I often think of I don't know, a couple of things, like one is we have a quarter million or more lines of code that was tested in as working on Python 2, and we are just not going to change that because it's too risky, and it's just, it's not broken, we don't really want to mess with it, that kind of thing, and then the other is we've got some key library that we depend on some package that itself is only in Python 2, so till that gets upgraded, you can only flip your app, right? But this is a whole another level, like-
41:56 Those two things they will fly
42:01 They are just one part, right, because you've got multiple compiled distributed ship to non updating potentially piece of the software that have a hosted versions packed into them, and you've got to fit to that, that's pretty crazy, and then you guys are building software that works with multiple versions, because you are not building the pipeline to make a movie, you are building software to give to the people that are making the movie, so it's just the few more dimensions there.
42:28 Actually, it's one of the things that really excites me about the project that I am working on, where almost for the first time rather than building one of these just for a particular studio or in open source projects, we are trying to do it with the resources of a company like Autodesk behind it, which all of a sudden like the jump Python 3 which for the industry as a whole feels like it's a huge issue. It gives us a place where if all of these vendors with the major applications got together and said ok, here is the version where we need to switch to Python 3 we could do a lot of leg work, but the industry as a whole gets stuff going and create that bridge, over the Python 3 which doesn't exist, if every studio had to do that themselves, and manage it themselves.
43:22 Yeah, that's really interesting, you are one of the few groups that live between all these pieces of software, where everyone else is just consuming them, right?
43:31 Exactly. And everyone, almost every company figures out a way to get that in between stuff written in order to automate stuff that they do, it's just too tedious otherwise, but we are kind of a central place where we can do that which gives these software developers someone to work with in order to figure out some of the issues. And we definitely wouldn't encounter all of the issues that studios out there could see with setup, that we at least can take it as big step further and prove that this kind of integration does work on the other side of that giant kazam, and here is how we did it, here is the issues we found and help other people solve it.
44:16 So another thing that you said that was kind of challenging, kind of like this version story, is PySide and PyQt can you maybe first of all tell people what that is? So that everyone knows and then what's the story there?
44:31 Sure. Qt is cross platform, C++ GUI tool kit, so you can build complicated applications using Qt as the components that you are using to deal with all the GUI widgets, and do it in a way that's cross platform, so it runs on Linux and on OSx and on Windows.
44:56 Yeah, and one thing I'd like to add about that is, that's one of the few cross platform UI frameworks that I think looks modern and really good on all of the different environments, like a lot of times you are like wow, that is definitely not native, that looks like a weird app or whatever. I think Qt is nice.
45:16 Yeah. It has a rich set of widgets that are very configurable, it ties in nicely with the native OS it's been around a while it's been used in a ton of places, I know some huge applications use Qt and it's funny that you say it looks modern, like there has actually been this divide int he Qt world between Qt 4 and Qt 5, similar to the kind of the jump to Python 3 where Qt 5 is 45:39 forward and that's one of the other things that entertainment industry is trying to navigate the jump from Qt 4 to Qt 5. And to go back to what is Qt and what is PySide, PySide and PyQt are two Python bindings through that Qt library so that you can build with these cross platform, good looking, complicated, UIs in complete Python, it's out there, it's great. But, PyQt is not open source, there is licencing restriction around it, and it makes it hard to build commercial product around that, even though kind of supporting it as runtime is completely doable but 46:18 if actually run PyQt app you need to be licensed if it's a commercial software. PySide it was the branch, it was another implementation of that Python binding in order to kind of get around those licensing restrictions to make it so that, within Python you could build these rich GUIs without having all those licensing constraints, that come with PyQt. Problem with PySide is that it's one of those open source projects that's kind of stalled out, and that has not been version that supports Qt 5. Ironically one of the big reasons that industry can't jump over to Qt 5 is because that Python is everywhere, people have used Python to build these really awesome production tools that are Qt 4 and there needs to be an upgrade path for all of that code. So, figuring out how there can bring some life into the PySide, project and get it going, as then the top of consideration and it's something that with the help of that be 47:23 effects reference platform, that I talked about earlier, because they are trying to drive this jump to Qt 5 because of there is a lot of improvements there, Qt 4 is starting show some age, they are trying to drive some life into the PySide project in order to get this update to be possible. It's an interesting thing, where again, this dependency and having all of these different apps coordinate this kind of an upgrade, it effects how we use Python and what's possible.
47:52 I think about I guess how that kind of paints you into a corner. I did play on with PySide, and PyQt and ran into this issue of well, the licensing on PyQt is, it seems overly restrictive, I am not really against commercial software, but it seems like- it just seems harder or more restricted than it should be. But then, again, I tried to use PySide and it was like almost working, it was frustrating.
48:18 Yeah.
48:18 But, I knew if I could get him to work, I'd get something really cool, but I just, I kind of gave up and did something else, all together.
48:24 Yeah, and we do a ton with PySide, it would be amazing to have it be in under more active development like, it's almost there, you can get a lot with it, there is definitely kind of low level issues with memory management, and things like that, that interface between the Qt world and Python world that if it was under active development we would be seeing some upgrades happen that would clean that up and make it easier to develop rich Python GUIs that are embedded in like these giant applications which is where like we actually realize a lot of just saving artists' time, so that they don't have to hunt around for files, or manually export things or knowing the setting that they do, we build the tool that pops up a GUI that does all of that for them and then they can just get back to doing the creative part of their job and all of that for a lot of places I must say 49:23 license for PyQt is done, in PySide.
49:29 Have you considered using some of the fully native options, like Pyobjc OS10 and Iron Python and WPF on Windows, and just having an separate UI layer-
49:43 I definitely thought about that, it's a lot of over head when something like PySide is there and you can write and once and have it work everywhere.
49:52 Yeah, it makes perfect sense, it's something I've been thinking about some native apps in Python and like well, you know how much, I guess it depends on how much of your code and complexity is in the UI itself, and how much you can push down the packages that those two distinct UIs that you have to write would consume, you know.
50:13 I mean, at that point, you are still wondering like would it make sense to have the GUI level written in actually objective C for OSX directly, and embed a Python interpreter and tie it in like that, like at that point, it being in Python you still wondering if having that layer be Python makes sense, because you are having to code it specifically for the operating system itself and for the GUI work you get the OSX code, in a really deep way
50:38 Right, right, maybe just use X code and like swift now, so that's pretty close to Python actually, something like that, right?
50:43 Yes.
50:45 But you would lose all the packages, anyway.
50:45 Exactly. And if we would talk to the rest of the pipeline again, like your general knowledge of what's going on, in a production and how to translate between the different applications is usually Python logic, so you've got to figure out a way to talk to that.
51:02 Yeah, it's super interesting sort of tradeoffs to consider, I personally would love to see PySide really come on strong but-
51:10 That would be amazing.
51:10 Yeah, yeah. All right, I have just enough time for a few more questions for you before you go. So, if you are going to write some Python code what editor do you open up?
51:19 That's funny. I have switched over to Sublime, I was kind of die hard for a long time and Sublime was the first thing that I saw that I was like, ok, those it's best enough to launch and best enough to get going, that if it competes and there is still definitely some stuff that I miss from them, but I kind of grown addicted to multiple cursors.
51:45 Yeah, yeah, ok, very cool, yeah, I like Sublime as well, it's nice. There is many thousands of packages out on PyPi, that people can go grab. What are some of your favorites, that maybe not everyone knows about, that they should check out.
52:00 I do think PySide can be able to build a GUI and complete Python and even package it up with like 52:13 flavor of that, is a pretty amazing thing for people to do, it's just kind of cool to be able to take something and actually hand it off as a self contained thing to somebody else to run.
52:26 Yeah, that is really cool, and, I don't think I or any of the guests have spoken about py2app your app or py2exe or Cx freeze or any of those categories, maybe you can just tell everyone out there, like those you can go pip install at least, or one or two of them, I think.
52:42 Yeah, and some of them just come with the Python distributions. So, it's a way of basically compiling down and freezing the Python interpreter and your code together as a native executable, so on OSX the py2app will actually create a bundle that has all of the Python embedded in it, and the same thing with py2exe you get the standalone exe that you can run on Windows, and it will run your code, so if you combine it with PySide then you get this exe that you double click on and you don't necessarily know that you are running Python under the hood, you've got to take the code that you've wrote and the speed with which you got to develop it because it's all scripted and you are able to put it together with something like PySide, and at the end of the day, you end up with the self contained thing that you can just 53:35 with somebody without having to setup any infrastructure under it and they will be able to run it.
53:40 Yeah, that's really cool, and it takes your packages and everything, right?
53:44 Yes. It embeds it all.
53:46 Yeah, I haven't had a real good use case with that because I mostly do web stuff, but it's really cool to think you know, instead of zipping up a bunch of scripts and sending them, you can, "oh yeah, I need to pip install this while you are at it." Just go here is your .app, double click it, here is your .exe, double click it. It's great. Nice, final call to cation or anything people should check out?
54:09 The call to action to people who are interested in open source development and the PySide I would recommend that it would be amazing to get people interested that again.
54:21 All right, awesome. And how do they find this project that you are working on? Is it something that's out or something you are working on still, that's going to come out?
54:27 It is out in beta form, there is production management software that Autodesk sells called Shutgon, you can go to shotgunsoftware.com and everything that I've been talking about is the pipeline developer section of that so the project is toolkit, where the pipeline toolkit or shotgun software.
54:49 All right sounds like a really interesting project. And, it definitely got a fresh look at the all the barbs that live in the different versions, all these things.
55:00 Yeah, I would say if people are kind of interested in this stuff, that I was talking about, the project that we work on is source shared, we've got public repos on git, the licenses you kind of have t o be using shotgun, as a product in order to use the project, but all of the Python code is there for people to look at and improve upon. We try to do it as a very open project, so that everybody that does this kind of work can join together as a community.
55:31 That's great. It's been really fun talking to you Rob, thanks for being on the show.
55:35 Thank you, it's been my pleasure.
55:36 Yeah, bye, bye.
55:36 This has been another episode of Talk Python To Me.
55:36 Today's guest was Rob Blau and this episode has been sponsored by SnapCI and Opbeat. Thank you guys for supporting the show!
55:36 Snap CI is modern continuous integration and delivery. Build, test, and deploy your code directly from github, all in your browser with debugging, docker, and parallelism included. Try them for free at snap.ci/talkpython
55:36 Opbeat is mission control for your Python web applications, keep an eye on errors, performance, profiling and more in your Django and Flask web apps.
55:36 Please take a moment to check out the online course kickstarter at talkpython.fm/course. You get a discount for the course by signing up early and the satisfaction of helping create something amazing.
55:36 You can find the links from the show at talkpython.fm/episodes/show/46
55:36 Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes and direct RSS feeds in the footer on the website.
55:36 Our theme music is Developers Developers Developers by Cory Smith, who goes by Smixx. You can hear the entire song on our website.
55:36 This is your host, Michael Kennedy. Thanks for listening!
55:36 Smixx, take us out of here.