Serverless Computing & Machine Learning

Here is a new couple of concepts (why two?) that happened to collide during my professional experience and that apparently work really well together.

Machine Learning has always been a passion since I started programming when I was 15, although back then I was not completely aware of its very existence. I just knew I wanted to build some kind of AI for my first games. :)

Serverless Computing, on the other hand, is quite a recent discovery that completely changed how I think about software development, whenever I plan to work on a new product or service.

I already had a chance to discuss what I’ve been working on with Jeff Barr on his blog (Machine Learning, Recommendation Systems, and Data Analysis at Cloud Academy) and during a Cloud Academy Webinar (Cloud Academy and AWS: How we use AWS for Machine Learning and Data Collection).

You may want to have a look at this webinar as well: Introduction to the Serverless Paradigm — by Austen Collins and me — and my last AWS Lambda Coding Session.

Here I would like to describe one more time the two main ideas that came up a few months ago and how I stumbled upon both concepts:

1. Machine Learning is a great use case for the Serverless Paradigm, with its own benefits and challenges.

2. Serverless Computing is an incredibly powerful deployment strategy for your Machine Learning models.

Who is NOT talking about ML these days?

My first encounter with Machine Learning

Well, I was 15 — still in high school — and I was about to learn that programming is not just about for-loops and if-else conditions. The first two games that I developed were Battleship and Pac-Man, in Visual Basic 6.0.

Needless to say, both games require some kind of AI, which is what makes the game interesting in the first place. Since it was still the beginning of the Internet era, the idea of an online multi-player version of Battleship was not even conceivable to me, therefore playing against my own computer sounded like the most interesting and feasible option.

Anyways, this was my very first “White Paper Syndrome” moment during a programming session in my life, when the following mystical question got stuck into my mind:

“How do you build a real Intelligence with plain code?”

My first 3 hours on “How the hell do I code Intelligence?”

I had to accept my inability to tackle this problem for quite a few years. Looking back at what happened in the next 4 or 5 years, that must be why I ended up studying Software Engineering, after all. :)

Back then, all my naive solutions were either too hard to beat (i.e. an omniscient intelligence) or way too dumb (i.e. random actions). This is why I quickly gave up and settled for a simple multi-player version of Battleship and a very user-friendly Pac-Man gameplay. The same kept happening again with many other games (Backgammon, Poker, etc).

Of course, later on I found out that Machine Learning is very useful not only to build decision-making AI, but also in many other interesting applications such as pattern recognition, data analysis, recommendation systems, computer vision, natural language processing, multimedia analysis, etc.

During college, my focus moved away from games and I fell in love with web technologies: HTML, CSS, PHP, JavaScript & Python.
This love became a real passion and a job as well, since 2011.
I joined Cloud Academy as Software Engineer and Frontend Developer in 2013, and I had a lot of fun with NoSQL (MongoDB), Python and Amazon Web Services.

What happened, then? Well, in the last few years Big Data earned the title of “ubiquitous buzzword” and Machine Learning went viral among the web community. The need of analysing and understanding huge amount of data became real: new tools and libraries appeared every day and most big companies started open-sourcing their own Machine Learning frameworks (Google’s Tensor Flow, Microsoft’s DMLT, Facebook’s FBLearner, etc.), plus many other open-source projects backed up by active communities (scikit-learn, Theano, PredictionIO, etc.).

Searches for “Machine Learning Python” (source: Google Trends)

Spoiler: your Data doesn’t have to be “Big” in order to be useful. Of course, you want to have enough data to build a reliable model that will work well when fed with new data, but I wouldn’t call it “Big” until you reach the Tera or Peta scale. Honestly, you need much less than that to build something useful.

Data-driven decisions will let you company survive.

If your company is not using data to answer its main questions and doubts, there is a very high chance that something will go wrong sometime soon. Being data-driven at every level of an organisation is gradually becoming not only a competitive advantage, but also the only way to survive in a very competitive environment, especially if you are a startup.

Ok, but what about Serverless?

Despite the proliferation of tools and libraries, you can’t just code machine learning models without some kind of mathematical and engineering knowledge. This is the skills-gap that MLaaS products (Machine Learning as a Service) have tried to solve, starting from Google Prediction API, Amazon Machine Learning, Azure Machine Learning, BigML, etc.

From a Developer & Software Engineer point of view, these services allow you to abstract most of the low-level details and let you focus on your data and your models, although most of them don’t allow much personalisation, if any.

Most MLaaS products don’t allow much personalisation.

This is why the Data Science community is still “stuck” to designing ad-hoc models with lower-level tools and frameworks. Which is great. Except that most of the time these ad-hoc models are developed by data scientists and tend to get stuck at the prototyping level. The outcome of this process will require a huge effort by some other developer or engineer, who probably can’t speak the data science language.

Wouldn’t it be great if prototypes developed by data scientists were production-ready without any further effort?

Earlier this year — at the 7th PyCon in Florence, Italy — I talked about why deploying a Machine Learning models is hard, compared to its design and prototypal implementation.

It was the fist time I publicly discussed and proposed Serverless technologies (i.e. AWS Lambda) as a successful and cost-effective deployment strategy for your machine learning models.

So why is Serverless such a good option for ML?

1. Isolation & Decentralisation. By following the Microservice Principles, your ML models will be exposed as a service, by means of a RESTFul interface (thanks to API Gateway). This way, it will be easier to decentralise your db/storage, isolate failure on a per-model level and hide every implementation details to the final user.
2. Zero Administration. Especially if you are a data scientist, managing servers shouldn’t be part of your daily tasks. With serverless, servers will conveniently disappear from your development and deployment workflow.
3. Scalability for free. As long as your models is deployed on AWS Lambda and correctly serves new predictions, you don’t have to worry about making it scale when your workload raises from 1 req/s to 50 req/s.
4. Never pay for idle. Indeed, there is no such thing with Serverless. You pay only for the number of executions and the corresponding duration: this is particularly interesting whenever your models are self-contained and can respond within a few milliseconds. In this case, your deployment strategy on AWS Lambda might be free up to 1M invocations per month, forever.
5. Event-driven & reusability. You may want to use your models for a big variety of tasks, and AWS made it very easy to connect Lambda to any other service, meaning that you could invoke your models when something happens within your AWS account (e.g. a new file on S3, a new record on DynamoDB, a new batch of records on Kinesis, etc.).
6. Function as the unit of scale. This is a perfect match with Machine Learning models, as you can think of each single model as an independent function that can be invoked, updated or deleted at any time without compromising the rest of the system.

Of course, Serverless won’t solve all your software design problems, and will most likely introduce new ones. But the community around it is growing exponentially every week (see the Serverless Framework and many other open-source projects).

I’m looking forward to a future where the abstraction of physical servers won’t be needed anymore, as well as most previous abstractions have already been replaced by better models. For example, we don’t see punched cards as the best way to think of computing anymore, as suggested by Tim Wagner during the first ServerlessConf in NY, a few months ago.

Reminder: come and say hi after my talk at the next ServerlessConf in Tokyo!