Despite being what most people think about when mentioning AWS, I left their Elastic Compute Cloud (EC2) until now, mostly because personally I don’t use it all that much.
With the advent of Lambda, EC2 can feel a bit more laborious as there is a lot more set up involved than with serverless solutions.
These are virtual computing environments, called instances. You can set them up with the resources that you need, in terms of CPU, Memory, Storage, etc., so you don’t need a super machine if all you are doing is firing off cron jobs. But, you can also have access to some very powerful processing if you need it.
Within limits, you can resize your EC2 instance if your needs change, which, while requiring a shutdown of the instance, is a lot less effort than migrating all the code and config to a new server.
Pricing is a bit more complex than with Lambdas, but they can be billed by second or by hour, which is still very granular. The big difference is that an EC2 takes a lot longer to start up than a Lambda. If you have short spurts of activity happening every now and then, which need to happen in real-time, and can’t be queued up and processed once in a while, then a Lambda might be a better choice. However, if you are constantly doing a lot of processing, or you can collect a load of tasks to run once a month, and then fire up an EC2 and spend a couple of hours running them, an EC2 is possibly a better choice.
Also, the resource allocation on EC2s is a LOT more flexible. With Lambdas I have run into memory (3008MB*) and timeout (15 minutes*) limits when processing PDF data which has required ‘creative’ solutions to workaround. With an EC2 High Memory instance, you could allocate up to 24TB* of RAM!
One potential solution that was discussed was queueing up high memory tasks, and then when there is a specific number of tasks (or a certain time frame has passed), automatically spinning up an EC2 instance, running the processing, and then shutting it down. All of this is quite doable with the interconnectedness of the AWS products.
A more mundane example of an EC2 instance is a low resource affair executing cron jobs on Drupal sites pushing commands over Terminus for Pantheon. These trigger a series of events that then fire an SQS message from Drupal which in turn fires a Lambda to trigger the running of Data Pipelines.
* At the time of writing