You may be surprised to hear that that the Classic watchdog is still used in production today, and many users do not care about this latency because of the value they get and the type of workloads they run. So we started to see an imbalance: Go and Python started instantaneously, but Node.js and Java took 5-10x times longer to start per request. A Node.js process is notoriously slow to start up, if you don’t believe me, try running it on a Raspberry Pi and you’ll see what I’m talking to. In fact I can recall a company reaching out to me that was using OpenFaaS and Node.js in production. Now that may still sound like a large number to you, but remember that it’s a lot better than multiple seconds! STDERR can either redirect to the container logs, or be combined into the function’s output.Ĭreating a process on a modern machine was many times faster than creating and scheduling a container on Kubernetes, we’re talking of < 100ms per request. Then the response for the HTTP response is read from STDOUT. The HTTP request body is marshalled into the process via STDIN, with the HTTP headers, Path and Method being turned into environment headers prefixed with X-Http. The OpenFaaS watchdog creates a process per request We then needed a way to marshal from our users to the processes, and back again and I used HTTP for that. In order to have a persistent container that could serve more than one request using process-level isolation, I needed a kind of init process, something that could watch over the container and launch these processes as required.Ī process needed a standard way to communicate and I picked up on the idea of UNIX pipes as a way to make this happen in every programming language and every kind of binary I was familiar with. Process-level isolation meant running one process per request as per the good old CGI days of the early internet. So after some experimentation I decided that the container should be created at deployment time, and perhaps we should use process-level isolation instead. Thus you get your 2 second latency for a cold-start. Why was that? Primarily because the health checks in Kubernetes at a minimum of every 1 second, so if you miss the first, you have to wait for the second to pass. So I tried it out, and found that this process would take several seconds, even for small container images, or images that were already cached on my cluster. Phil’s tests showed that for pre-pulled images, runc (the lowest level of a container in Kubernetes) could start a container in a fraction of a second. This was a problem that other people were clearly interested in solving. A health check has to pass for the container’s endpointĪnd when all that’s complete, you can probably serve a request.Īround the same time Phil Estes of (then at IBM) was writing a container benchmarking tool called Bucketbench.The container has to be started and have its endpoint registered for health checks.

The container has to be scheduled on a node with available resources.

The container image has to be fetched from a remote registry.

There’s several things that happen when a container is scheduled on Kubernetes: Let’s look at the lifecycle of a container, or as I like to call it “Why is this thing so slow?”

There’ll be plenty of links and resources if you want to go deeper, and a FAQ towards the end. In this article, I’ll cover the reason why containers are slow to start - particularly on Kubernetes, then take you through our research and development that ultimately led to not needing cold-starts, or not needing to see them in OpenFaaS. I knew that containers were going to be important for portability, but can you blame me for thinking that I would be able to run one container per HTTP request? That turned out to be a non-starter, but as a community we came up with something that’s used broadly in production today. I started out with OpenFaaS in 2016 when I was frustrated with the developer experience and portability of cloud functions. Import 3.Alex takes a look at the architecture of OpenFaaS and why you can say goodbye to coldstarts. It needs to fetch objects from S3 and I have therefore set up an IAM role and am building an S3 client in the Lambda's handler: import .runtime.Context I'm currently working on an AWS Lambda function written in Java.

Aws lambda slow startup