00:00 Talk Python To Me
00:00 Episode 1, with guest Nicola Iarocci, recorded Tueday, March 31st, 2015
00:00 [music]
00:00 Hello and welcome to Talk Python To Me- a weekly podcast on Python- the language, the libraries, the ecosystem and the personalities. This is your host, Michael Kennedy, follow me on Twitter where I am at @mkennedy, keep up with the show and listen to past episodes at talkpythontome.com.
00:00 This episode we'll be talking to Nicola Iarocci about EVE. Let me introduce Nicola.
00:58 Nicola is a passionate Python and C# OpenSource hacker, based in Ravena, Italy. Back in 1991 he co-founded CIR2000, a small company focusing on delivering accounting software solutions for the small businesses. He is a creator and maintainer of a few OpenSource projects including EVE and the conference speaker who happens to be on sabathical. When he is not hacking on code or spending time with his three kids, he is usually busy running, swimming, hiking or reading a good old paper book.
01:29 Nicola, welcome to the show.
01:31 Hi Michael, thank you for having me.
01:32 Yeah, it's really great. Thanks for helping me kick off this podcast. So you and I have known each other for a few years and it's really nice to get a chance to speak with you again, I think it's been about a year since we last met on an aircraft carrier in New York city right?
01:47 Right.
01:48 It was a lot of fun. So, let's sort of start at the beginning. You like myself do Python and C#. So how did you get into Python?
02:00 Yes, as you say, I've actually always been a C# guy mostly, I worked like at these Windows desktop applications so you use and deploy your user kind servers and stuff like that. And as long as we were working in that context, we never really had the need to look anywhere else. C# in general was and the Microsoft stack in general was doing fine for us. Then, like everyone else, few years ago we started looking at moving to the web and mobile and stuff like that. And, I believe that is when I thought it was the perfect opportunity to move out of my comfort zone.
02:49 Then the Micro stack was still a good bet for all things related to the web, there was some access there you know, so I started looking at the Javascript first, and then Python and other languages, and there were so many options, and I really wanted to- I saw that was the perfect opportunity for me to start doing something else and looking around. And Python came- I don't really remember how, maybe I was just reading some tutorial online and I was immediately hooked by the language itself and the community and the OpenSource scene in general. It was great for incredible performance boost for example. For the first time I was playing around with Flask and something just didn't work as I expected, and I was stuck there, I couldn't make any further. And now I don't really remember what it was really, but something very stupid most likely, you know-
04:13 That's how it is when you are learning, right?
04:15 Yeah, I was just learning a new language and surely, it had to be something very stupid. And, yet this moment when I realized that I could just go a step further and look at the source code, from someone coming from the .Net work and closed source in general, it was really awesome to be able to go on the website, on the tab and check out the source code for Flask, and understand what actually was doing behind the scenes, and that's when Python and OpenSource in general really- I saw what the progression was. But then, in our world we were behind the iron curtain something like that, because- you had to buy licenses for programs, and most of the time even if you bought a license you still did not get any access to the source code. And so it was really an epiphany for me.
05:24 Yeah, it's fantastic.
05:27 Yeah. That's really awesome.
05:28 I think a lot of developers spend their time in worlds where much of what you do is a black box and you just can't know, and I think in the Python world we are kind of spoiled in that you can just keep looking farther down inside you know, and that's really great. So, how did you get- so you got into Python, and it sounded like you were getting started in Flask and then I guess EVE is kind of a natural extension. What is EVE actually?
05:56 So EVE is an OpenSource Python REST API framework designed for human beings, which means that it is supposed to be easy to use and powerful, and it is thought to allow any developer to effortlessly build and deploy highly customisable and fully see through web services. Or at least, that is the idea. It is powered by Flask, and it offers support for MongoDB out of the box, and there is also SQLAlchemy extension provided by the community, so you can use EVE both with MongoDB and Postgres or MySql or whatever.
06:40 So, yeah, that's fantastic. So, I kind of looked at it, I don't know all that much about Eve, I've just played with it a little, I saw some tutorials and checked them out, and it looks really interesting. I also picked up that it was inspired by the Request package so... That's cool. Basically the way it work- let me try to paraphrase how your library works is- I have some sort of data source that is by default MongoDB, which is an awesome data source by the way, or also SQLAlchemy so against a lot of the relational databases, and I can easily put like a restful API over my data, is that how it works?
07:15 Yeah, exactly. The idea basically, you have your data source somewhere and you bind it to an Eve instance, you can do it by just setting API configuration fine where you basically leave your API topology, your endpoints, how they work, to which collection of MongoDB or SQL each endpoint matches to, and then you are good to go. The idea basically is that you have a lot of features out of the box so you don't have to cold match in order to get started with your API. And then you can of course go in and call back function. So for example, when some request gets in, you can add your own code that makes whatever you need to do. But the basic idea is that if you just need a front end API for your data, you just have to set up the configuration file and you are good to go.
08:29 Yeah, it looks super easy to get started, looks great. And I think it's inspired by Request in that sense as well. So, are people using more sort of Javascript front ends and using that to get to their data or they are using this for more like back end services? Do you know like what the main consumption model is?
08:48 Yeah, well it is being used as for all kinds of use cases actually. We have a lot of websites consuming Eve instances, but we also have a lot of mobile clients for example. I myself am using Eve for my IOS, I've been drawing applications for example, which are being coded in C# actually. We are using Xamarin framework for writing our own mobile applications or so. You have all kinds of use cases, and also of course desktop application and all kinds of languages are consuming the instance, since you are just leading with Json basically which goes back and forth between the client and the server. So you can deploy to whatever you want actually.
09:43 Yeah, that's fantastic. So, you said MongoDB is the default back end; why did you choose MongoDB?
09:49 Well, Mongo DB has two interesting features for me, the first is the REST in REST world you basically are sending these back and forth between clients and servers, and MongoDB also stores Json or actually Bson which is evaluation of Json, but basically the same stuff, and then you have Python dictionaries which are the perfect match for Json as well. So when I was researching for building my own API, it came kind of the natural choice, I don't have to serialize and deserialize objects back and forth between the client and the REST server and the database, so everything is easy and the reason the lot of code in the middle doing conversions and serializations and stuff like that. You don't have anything like that when you are working with MongoDB.
10:55 And also, where REST APIs are stateless of this by default, and so you don't usually have to you know, your user is sending a small data pack for each request, and that's something that makes MongoDB lack of transactions and really are no issue because you are already dealing with the concept of being stateless between the test and the client that in some way has to be the fact that a single test can go wrong or can go fine and so what all this is- the concept of people going to use MongoDb is not really an issue in this particular use case.
11:46 It's an interesting point that the MongoDB has all this really positive features, scaleability, simplicity, lightweight, free, all this things, but one of it is- sometimes criticism is its lack of cross document transactions but if you are doing REST over HTTP that is a stateless protocol anyway so you've kind of- it's like you said, non issue, that's really interesting.
12:09 Yeah. And then, there is MongoDB being schemaless, which is really nice for me because when you are building, prototyping your API you are going to change your schema or if in API terms your endpoint configuration, your payload quite a lot you are adding fields and between API versions you are going to have new fields,remove odd fields, stuff like that, you are doing that constantly, at least at the beginning. And here MongoDB is very nicely used because it allows you to change your schema on the fly without having to update the database and so without any back end bad performance as well.
13:00 So it kind of made sense for what we were doing back then, and working with MongoDB on the back end and Python on the server side. And yeah, that's what a lot of people agree with, I believe because Eve has been adopted by a lot of people and also, there is as I was saying earlier, there is a SQL extension actually which connects Eve to SQLAlchemy so you can in fact use your standard SQL as well.
13:36 Yeah, that's really interesting, you just have to do a little bit more work. So, when you built the framework, who was the intended user that you were thinking of, like what is the most common use case?
13:51 I will say that the 13:57 user can benefit greatly from Eve because a nice thing about Eve is that you basically get a lot of design choices and implementation, so you can quickly prototype an API if you need to put your mobile or front end developers at work, you can probably give them an API, a functioning API in a few minutes or maybe in a few hours, and it is surely very interesting for a lot of people. I see there are PHP guys for example learning Python because they want to use Eve which is kind of a surprise to me but-
14:40 That's pretty amazing, so you must be kind of proud that you know, you are bringing people into the Python world literally because they want to use Eve, that's awesome.
14:50 Yeah, yeah, that's awesome. When I get emails from people who are learning Python because they want to use Eve that's totally awesome. It means that, you know kind of makes- that your goal has been achieved somehow and then you are also bringing new people to the Python world which is awesome. So I am really glad for it. Being based on Flask itself in fact we also have a lot of experience- Python guys working with Eve and building the extensions because you know, Eve is just a Flask subclass. So whatever works with Flask you can use with Eve.You can also get access to Flask documentation and whatever works there works with Eve as well.
15:25 So you can go on Stack Overflow or browse the Flask documentation, which is also excellent, and you can move in between layers, Eve on top and Flask is right behind Eve, and then eventually MongoDB or your SQL database. And so why it is very easy to start Eve. Probably if you are already using Flask you can enjoy Eve even so much more because you can do whatever you want with Eve, you can for example build a Flask extension and use it with Eve. Which is in fact what most people are doing, we have a few community extensions which are basically Flask extensions themselves. So, it is good for the beginners but is also good for the experienced developers I believe.
16:31 Yeah, that's really cool. So, all the great stuff that is out there for Flask you can just use it, and maybe even if you already have a Flask website for HTML you know, for like browsers to consume, you can add services to it pretty easily by adding Eve into your web app, yeah?
16:45 Exactly, that is exactly what people are doing and for example there is a guy who wrote Eve docs extension which basically what is doing is building API documentation and service them as static HTML pages, and what it does is leaving 17:09 the API configuration files and building a browser documentation for the user, so if you go to the example.com/docs, you get the API documentation for free, because it is being done as a Flask extension by these Eve docs extension itself so you can mix and match a HTML pages with REST APIs.
17:39 That's cool, so what this guy's project does is basically it's like a self documenting API or self documentation for API just opinion how you can figure Eve, yeah?
17:48 Yeah. I get a lot of questions about these actually Eve on Stackoverflow so how can I serve static pages along with my API, and that's actually very easy to do, you just have to add a route to Flask as you would do with proxy server, so it's very easy to pull out.
18:12 Very nice. So it sounds like, you know, this is a very popular project, and I looked on GitHub and it's got 2000 stars and 300 or so forks- that's a lot of people contributing. And this is not your full time job, right? This is like a second project, so how much work is it and challenges to balance making your awesome OpenSource project successful and actually keeping the day job running.
18:39 Yes, it's really becoming quite a challenge, because Eve is becoming a huge work. There are a lot of use cases and people want to do like all kind of stuff with Eve, which is super cool. The community is awesome, and we have about 70 contributors to the project at this moment, and they contribute with a lot of very cool features. On the other hand, you end up doing a lot of work for free. I mean for what we need Eve for we would be happy with what Eve was maybe one year ago.
19:22 And now, all the additions to the project are basically coming from the community, and they are either all adding the new features themselves or asking me to do something new, and interesting is I usually end up coding it for the community and for myself of course as well, but that takes a lot of time. So yeah, it's kind of a challenge because you have to balance your work life with your family and hobbies and OpenSource activity. It's not that easy but you also get a lot of gratifications for doing this, I get emails from all around the world, from people using Eve in so many strange, odd and amazing use cases. And so, it's hard to do, to put up, but it is also very well worth it for me.
20:26 Yeah, I am sure it's very rewarding, that's great. Are there other contributors to it, people who actually are putting a lot of effort into it, are you looking for new contributors or what is the status there?
20:38 Yes, as I was saying we have about 70 contributors of these, maybe 10 of them are very active and have been contributing to the project quite a lot lately. Some are very skilled with Flask and Eve. I still have to do a lot of work or most of the work but yes, I would really appreciate if more people would join the community and help with even with the simple things, like updating the documentation, fixing typos, as you can see I'm not an English speaker, so it is very easy to find and correct the mistakes in the documentation for example or in the docs in the Source code, stuff like that.
21:25 But also we have people who are adding amazing features like we had I think in 21:31 5 we had versioning feature added to Eve, so you can store multiple versions of your documents and your API and then you can get back the version that you need with simple action, and that is a very cool feature which was entirely contributed by the community from one guy actually. And, that's very cool. And so we had a lot of other additions like that from the community. And so yes, there is a lot of room for improvement, and I really would love to see more people jump in and help us with going further.
22:15 That's great. Yeah, I think people have a hard time realizing how much work it is to actually get something like Eve out the door and ready for the community. I mean, there is obviously the writing of the application or the library and maybe a sample, but there is the GitHub repository, the website, the documentation, all the little stuff, the little polish you have to put on there, it seems like you know, when you feel like you are done, you are half way there and you still have a ton of little things that are- somewhat uninteresting but have to be there like documentation.
22:49 Yeah, absolutely.
22:52 How's the experience being both a C# and Python and Javascript developer, how do you see the world living in this polyglot programming world?
23:05 It's been a wild ride actually, because you know, the languages are so different in so many ways. Python is a dynamic language whereas C# is a strongly typed language, and now I am playing with Go and C# on approaches to solving the same problem basically. And that's really interesting for me, because you don't always have one tool, one language, which is perfect for all kinds of problems, so it's great to actually being able to pick the right tool for the right job. And basically what I am now using is Python on the web side of things, so all my servers and websites are Python, while all my mobile applications and desktop applications are C# for example.
23:59 And I mean, it helps a lot because you get a very different skill set and you can just- you often find the same problem, you can solve with the different languages actually, have been solving with different methods and that really also means that you understand and experiment with different things even if you- I go back to C#, I often find that I am using the tricks which I learned while I was studying Python for example, and the contrary is also true, for example when I was looking at classes in Python it was very easy to pick them up because of course, I've been coding classes in C# before, like 20 years. And so it was very easy, it is a very trivial example but you get what I mean.
25:04 Yeah. Absolutely, I lived that life as well. I think one interesting aspect that is different is- the languages are actually not super different, right- the Python is obviously dynamic and C# is static, but in a lot of ways, like you said with the classes, and so on they are similar, where they start to get different is around the tooling and around the packages and the framework that you can use.
25:30 Yeah.
25:29 Yeah. What tools do you use for working with Python, do you have like editor that you are fond of or anything like that?
25:37 Yes. You know I am using Vim for coding in Python, and that's also something I never used when I was working with C#, so back in 2010 or something like that I started working with Python and I also pick Vim and I was Visual Studio guy and that being Visual Studio guy for a long time so coming to Python it also meant learning a lot of new tools, I started using Vim for example, or of course I am using pip for installing packages, and learning how Python packaging works which is totally different from C#. That's how very interesting aspect of using different languages.
26:31 And now what you have is that C# and .Net are in this new Microsoft era. We are getting to the point where even Microsoft is learning from the OpenSource experience and so we have, for me it is really exciting I think looking at how Microsoft is approaching and evolving this last maybe couple of years revising .Net framework as OpenSource for example is totally awesome and they are going cross platform as well, and this is all stuff that I've been seeing working on Python and there was always one I really would love to see this kind of approach in Microsoft world and Windows and C# and now it's happening, and it's really exciting.
27:23 It also means that Microsoft is learning from its let's call them mistakes maybe, and yeah, it's really cool and I am super excited because I see the OpenSource potential coming to the C# world. On the other hand, I must say that the OpenSource community in C# and .Net framework in general is still very in stages so to speak when I believe an OpenSource tool in C# I get a fraction of the attention that the same tool gets in Python for example. So there is still a lot of room for improvement.
28:07 Yeah. I totally agree. It's definitely a good sign that some of these closed source companies especially Microsoft are widely used is moving- I think they are learning a lot form Python and OpenSource. So it sounds like there is a lot of interesting pieces that you are bringing into Eve, like Cerberus and Flask Sentilen, things like that. Those are separate projects, separate packages that you are just using in Eve, right?
28:33 Yeah, I am using a lot of packages and I am also using Redis for example for some features. We have rate limiting which is a feature which is being within Eve, and it uses Redis actually. By the way Redis is also by an Italian developer Salvatore Sanfilippo who is a friend of mine. And yes, I am using Cerberus which is another OpenSource release of mine, it is used for data validation basically and Cerberus is being used even outside of Eve by a lot of people, since it allows basically to do validation for Python dictionaries. So we use it within Eve for data validation, but you can also, pip install it and use in for your own tool, this is another very interesting approach.
29:32 So, that is really interesting, I haven't used Cerberus but, basically what you can do is you can say what required fields and what required shape a Python dictionary should have and then you can say, "validate this particular dictionary". What kind of stuff can you validate there?
29:47 Yes, you can basically say, "hey, this is the name field is a string must have a maximum length of 10, and you must use this regular expression to validate it", and there are a lot of options actually you can say that the field must be unique or stuff like that. And this is true for the Eve by lot of course but also for any 30:18 and so people are using it everywhere and it is very nice.
30:23 Another tool that I ended up building for Eve is it also being used outside of Eve is Events, which is basically a package which allows you to bind the call back functions to any kind of event and so when Eve receives any kind of request, you can have a number of call back functions, which are being called by Eve and so you can do your own stuff with this payload which is coming in, and change it or store it elsewhere or call another API and validate it against it, stuff like that. And it all powered by Events which is another Python package that you can use for your own tools, for building your own tools.
31:18 That's fantastic. That sounds great. So another one that I saw in there was Flask Sentilen. What is that?
31:26 Yeah, Flask Sentinel is basically an implementation of OAuth for Flask, in particularly it is addressing the token authentication scheme so you can use it without Eve as well or you can use it with Eve to have OAuth2 on your API. So yes, you can basically look it up on GitHub it is being used for a lot of stuff.
32:03 That's cool. So, if I want to have like Twitter authentication or Google or Facebook or something like that, is that kind of what you are thinking?
32:12 Right now it only addresses basic token authentication. So it is not really suitable for that use case, it is not yet a full OAuth2 authentication scheme but the best use case I think is when you use it for your own mobile application and you want your user to be authenticated with your REST API framework, and you give him a token and then your mobile application uses this token instead of using the username and password. So yes, if kind of, just one of the many users of OAuth2- for me it is very difficult to pronounce that word, and yeah.
33:12 Ok, that is great. Yeah, so I can see how you use it, that's very interesting. Now, I noticed you also working on a project called Eve.net. Is that like a .Net spin off of this whole idea?
33:23 It is basically a REST client for .Net so what you use it for is for example if you are writing a mobile application for IOS and Android and want to connect to an Eve instance, you can use Eve.net. Since it knows very well how Eve performs, so you can send for example a class or an object instance to Eve and it deals with serializing the object and sending it to Eve and it handles the Eve responses.
33:59 And for example if you want to modify the object the user just say to eve.net "hey store these updates up to the server", and the client connects to Eve, sends the update and it reports back to the client and since it knows very well how Eve performs it can it hides a lot over the implementation behind the scene so you don't have to worry about conflicts, errors, validation, stuff like that. And it isn't interesting because it exactly a polyglot approach which we were speaking about earlier. You have C# and C# client working with the Python server and so it's kind of fun.
34:58 Yeah, because you know both sides of that story, you can make it really easy for the C# guys. I wonder if there are other languages out there like Swift or Java or something that might also benefit from that.
35:09 Yes, you can use everywhere you use .Net so it's very nice. Actually, one of the things that I am doing with the .Net framework at least here in Italy is encourage people to go out of their comfort zone and for example installing Eve instead of 35:29 framework or stuff like that on the server side and because they also get advantages of Eve and then they can use the C# client on the client on the mobile phone for example or on the desktop applications. Of course, Eve is totally language agnostic so you can use it equally with whatever language you want.
35:55 Yeah, that's beautiful. It uses the universal language of Json. Excellent.
36:00 Right.
36:00 Yeah, so are there any notable uses or really popular deployments of Eve that you can talk about?
36:09 Yes, there are 2 or 3 which I think can be interesting for your audience. For example we have Dailymotion which is, I don't know if you know about it, but they are I believe the second streaming website in the world right after YouTube, it's a French company which is streaming live contents on the web, and they are using Eve for some of their projects. Actually, they are also contributing to Eve so they are one of the main contributors to the project itself, which is totally cool for me. And then there is 36:55 which is a company they are they guys behind the SQL extension for example speaking of the community.
37:01 As you can see we have people who are using the tool itself and then contributing with new features and this is one special case because 37:12 basically took the SQL extension which was already being developed by other guys and you know, kind of completed it and they have now 100% I believe they are 100% feature complete, and so yeah. And then we have these very famous space agency but I didn't get any feedback from them, I wanted to mention them but I am not sure I can, let's just say it is a very famous space company led by a very well know entrepreneur who also work in car industry and stuff like that.
38:01 That's really fantastic. So, Eve might be going to space and back. That is really cool.
38:06 Yeah, that's my dream actually, it would be totally cool. I think they are still experimenting with this, they are contributors as well. One of their engineers is the guy who works on versioning feature for example. They are using it internally and I am not sure if they are using it for stuff in the space, I am not sure I want them to use Eve for for some things in the space.
38:31 It's the whole new level of responsibility. Yes it is.
38:38 Yes, it is being used by all kinds of people and that is what I find really interesting for me, it is really exciting, so many people with so many different use cases, who rely on your tool is you know, totally awesome.
38:58 Yeah, that's really great, I'm sure it's super rewarding and you know, thanks from everybody in the community for putting that together, that's cool. So, I think that is probably a good place to wrap up the show, so is there anything you want to mention or give a shout out to, give a little promotion to before we call it a day.
39:19 Not really, what I really would like to see is people join the community and as I said already, try the tool and if they have questions or contributions to add, we would be very happy to learn about them and see more people adopt the tool and the other the satellite tool like Flask- Sentinel or Cerberus and you know just keep sending feedback so that we can keep growing with the tool and the community itself.
39:55 Yeah, that is really great. Ok, so you can find Eve on the web at Python-Eve.org. Nicola thank you so much for your time, it's been a really interesting conversation.
40:06 Thank you, it's been great and thank you for inviting me it's been awesome.
40:10 You are welcome, hopefully we get to catch up on Aircraft carrier in New York city some time soon.
40:14 Ok.
40:17 Talk to you soon.
40:20 Looking forward to that, bye bye.
40:20 Bye.
40:20 [music]