Detecting Cold Starts with IOpipe
Determining if a function invocation is a “cold start” can be quite helpful in troubleshooting a metric like latency.
Requirements
You must use the following versions of IOpipe client for cold start reporting to work properly:
- Node.js IOpipe library >= 0.2.0
- Python IOpipe library >= 0.3.1
If you’re not already an IOpipe user, you can sign up for a free 21 day trial, no credit card required.
What are cold starts?
Cold starts are essentially an invocation that takes longer to start up due to initialization of a container.
Even though it’s called “serverless”, there are of course servers under the hood. In AWS Lambda, containers are used under the hood to run your functions. Spinning up a new container is fast, but not as fast as an already warmed up container, nor as efficient to do for every single invocation.
So in order to provide low latency, AWS Lambda re-uses containers for your active Lambda functions so that they doesn’t have to “warm up” by loading dependencies or other resources each time they are invoked.
When do cold starts happen?
- Each time a new function or version is deployed it will incur a cold start.
- If a function has been inactive for a period of time, the “warm” container may be shut down. We are currently investigating this, but it is around 5 minutes of inactivity.
- When you have a sudden spike of invocations to a function, AWS Lambda will spin up additional containers to scale automatically. Each new container spun up will incur a cold start.
- We’ve identified that cold starts happen every 2–4 hours regardless of activity. AWS Lambda does some re-shuffling of resources to keep things running efficiently.
What are the effects of a cold start?
Cold starts will incur a higher latency to any request, as it takes time to initialize the container and pull in any dependencies before running your function. The amount of added latency can vary greatly depending on many factors. The largest factor tends to be pulling in external resources, and large dependencies loading.
How can I find cold starts with IOpipe?
You can see which invocations are a cold start by going to the function page in the IOpipe Dashboard, and looking at the invocation list. You can also sort by cold starts.
You can also see if an specific invocation is a cold start in the invocation page:
What can I do about cold starts?
If you find that cold starts are affecting your application negatively, you might be able to improve things.
- Reduce or eliminate any external resource loading, such as dependencies. This can have a pretty big improvement on cold start impacts.
- Keep your Lambda’s “warm”. If your Lambda isn’t called very often (more than 5 minutes between calls), you can setup a scheduled “ping” or schedule your Lambda to run every few minutes to keep it in a “warm” state. This will not eliminate cold starts completely, but it might help reduce the number of cold starts.
- Keep your lambda size down, the larger the lambda, the longer it will take to load it when it’s a cold start.
