When AWS says that the Amazon Linux kernel is optimized for EC2, they're not kidding
Have you ever noticed a change in performence when choosing between CentOS, RedHat or Fadora?
Path of Exile 2
Amazon Linux 2022/2023
Optimized performance
Amazon Linux 2 includes support for the latest Amazon Elastic Compute Cloud (Amazon EC2) instance capabilities and is tuned for enhanced performance. It includes packages that help ease integration with other AWS services.
Enhanced developer productivity
Amazon Linux 2 is offered as a virtual machine and container image for on-premises development and testing. Developers can speed up their application development by building, testing, and integrating on the same Linux distribution that is used in their production environment.
Peace of mind with long-term support
Amazon Linux 2 offers long-term support. Developers, IT administrators, and ISVs get the predictability and stability of a long-term support (LTS) release, but without compromising access to the latest versions of popular software packages.
Videos
Just thought I'd share an interesting result from something I'm working on right now.
Task: Run ImageMagick in parallel (restrict each instance of ImageMagick to one thread and run many of them at once) to do a set of transformations (resizing, watermarking, compression quality adjustment, etc) for online publishing on large (20k - 60k per task) quantities of jpeg files.
This is a very CPU-bound process.
After porting the Windows orchestration program that does this to run on Linux, I did some speed testing on c5ad.16xlarge EC2 instances with 64 processing threads and a representative input set (with I/O to a local NVME SSD).
Speed on Windows Server 2019: ~70,000 images per hour
Speed on Ubuntu 20.04: ~30,000 images per hour
Speed on Amazon Linux 2: ~180,000 images per hour
I'm not a Linux kernel guy and I have no idea exactly what AWS has done here (it must have something to do with thread context switching) but, holy crap.
Of course, this all comes with a bunch of pains in the ass due to Amazon Linux not having the same package availability, having to build things from source by hand, etc. Ubuntu's generally a lot easier to get workloads up and running on. But for this project, clearly, that extra setup work is worth it.
Much later edit: I never got around to properly testing all of the isolated components that could've affected this, but as per discussion in the thread, it seems clear that the actual source of the huge difference was different ImageMagick builds with different options in the distro packages. Pure CPU speed differences for parallel processing tests on the same hardware (tested using threads running https://gmplib.org/pi-with-gmp) were observable with Ubuntu vs Amazon Linux when I tested, but Amazon Linux was only ~4% faster.