ReadyToRun’s Impact on Cold and Lukewarm Starts
In a previous post I discussed an issue with .Net Lambda functions that I labelled “Lukewarm starts”. This effect is due to the JIT compiled nature of the .Net framework. I also discussed a number of possible ways of alleviating this issue. One of which was using the “ReadyToRun” compilation option that was introduced as part of .Net core 3.1. In this article, I’d like to dive into some details about the impact this feature has on both cold and lukewarm starts.
Making it Happen
The simplest way to compile and deploy your .Net core Lambdas is to use the dotnet cli coupled with the AWS Extensions for .Net cli. Using this you can simply execute the following command line
dotnet lambda deploy-function <MyReadyToRunLambda> --msbuild-parameters "/p:PublishReadyToRun=true --self-contained false"
This command ensures the binaries are built using ReadyToRun, and then deploys them to AWS Lambda.
The Impact
Using the very basic 2 branch application from my previous blog post, I was able to assess the various impacts of applying this compilation option. Of course, as stated in my previous post, real world lambdas may be significanlty more complex, but it gives us a good start to understand the impacts. As with before, the biggest impact will be seen when using the smallest memory size (and subsequently CPU allocation) of 128MB. Below are the X-Ray traces from the application for the cold and lukewarm start invocations running in a Lambda with 128MB.
Cold starts
Without ReadyToRun
With ReadyToRun
Lukewarm Starts
Without ReadyToRun
With ReadyToRun
Package size
The first and most obvious impact is the final package size. Without the ReadyToRun switch, the size of the package is just under 1MB (998.8Kb). Switching ReadyToRun on, increases this to 2.5MB. This will undoubtedly have an impact on the “initialization” phase of the Lambda cold start time. As can be seen from the above x-ray traces, the “Initialization” phase increases from 192ms to 198ms. This increase is barely noticeable however, with respects to the improvement made in the cold and lukewarm starts, which we’ll dive into next.
Cold and Lukewarm starts
The time for cold starts drops from 10.82s without ReadyToRun to 8.26s with ReadyToRun. That’s an improvement of almost 25%. With Lukewarm start, we go from 2.41s without ReadyToRun down to 1.13s with it, an improvement of more than 50%. This is huge, and easily dwarfs the ~3% degradation in the performance of the “initialization phase”. Performance improvements can also be observed for Lambdas running with more memory, but are not quite as impressive as they are with Lambdas using 128MB.
The Downside
Of course there is a cost for everything. While the ability to do ahead of time (AOT) compilation drastically improves your cold and luklewarm start times, it will ultimately not be able to produce as highly optimized code as a JIT compiled version that doesn’t have to make as many assumptions. This will result in slightly higher times for regular invocations. As an example the 95th persentile between the 2 experimental runs for the 128MB Lambda resulted in a P95 ~ 220ms without ReadyToRun, and a P95 ~ 430ms with ReadyToRun. Again this does become less of an issue as you increase the memory for the Lambda, but it may also depend on exactly what you are trying to optimize for. If cost is the main concern, and your workload will spend the vast majority of its time in warm state, then this could become a real issue. Especially given AWS now charge by the millisecond for Lambda invocations instead of rounding up to the nearest 100ms as they previously did.
Conclusion
As with any engineering, the most important thing to do is to have a good understanding of what is most important to you. This will help you decide what you are willing to trade off, and what you aren’t. If cold and lukewarm start time is important to you, then using ReadyToRun is pretty much a no-brainer. If cost or performance of your warm state is something you care far more about than cold or lukewarm start, then you may want to do some expreiments with your workload before using ReadyToRun.