Say that I have 5000 requests hitting an application every 5 seconds and I need to consume an object and write it to a database. Would a lambda with an http trigger that does an insert be more cost effective than having an ec2 instance that does the same thing?
Edit: Just want to say you all are awesome! I am coming from MSFT world and never experienced so many folks being so helpful. Makes me think I should have switched to AWS ages ago. Thanks everybody.
I know that there's a trend towards serverless architecture like Lambda.
I know Lambda is an on demand AWS service where you don't have to provision and allocate resources(EC2) based on forecasting. With Lambda, your compute resources can scale based on real-time demands.
In what situations would you want to use a non managed solution like EC2 vs lambda?
I'm under the impression that the less work you need to do in terms of operations the better - why AWS exists. If you use ec2 directly, you would have to define autoscaling rules, etc
Please help me work out the math here, as I think I am doing this wrong.
A Lambda of 128mb costs $0.0000000021/ms, this works out $0.00756/hour. A Lambda of 512mb costs $0.0000000083/ms, this works out $0.02988/hour.
Now if you look at EC2:
t4g.nano $0.0042/hour (0.5 GiB ram) t4g.micro $0.0084/hour (1GiB ram).
But... the Lambda will likely not run 100% of the time, and will stay warm for 10 minutes (not sure here?). And the RAM usage would be much better utilized if you got a function running, rather than an entire VPC.
Given all that, if the function can run with 128mb or less, it seems like a no-brainer to use Lambda.
However, if the function is bigger, it would only make sense to put it in an EC2 if it runs more than 30% of the time ($0.0084/hour cost of t4g.micro divided by 0.02988/h cost of 512mb lambda).
So why is everyone against Lambdas citing costs as the primary reason...?
Hi, I am building an application as a personal project for which I plan to use AWS services.
Without going into too much detail, the application is mostly just a CRUD application with the additional need to run a function on the database on the 1st of every month.
I will be using a dynamodb table for this because it is the cheapest option (A major requirement for me is low cost).
To build the application itself I have two choices:
-
Use API gateway and lambda to create all the endpoints I need, which I will call from my frontend which will be hosted as a static site on S3.
-
Build a Flask or Django app that interacts with dynamodb and deploy this on an EC2 instance. I can serve my frontend as static pages from here in this case.
Which option would you guys recommend?
I am not going to have too many users using this app. It is only going to be me. So there shouldn't be concurrent requests being made to the server.
Any help or advice would be appreciated.
I am working on a project, it's a pretty simple project on the face :
Background :
I have an excel file (with financial data in it), with many sheets. There is a sheet for every month.
The data is from June 2020, till now, the data is updated everyday, and new data for each day is appended into that sheet for that month.
I want to perform some analytics on that data, things like finding out the maximum/ minimum volume and value of transactions carried out in a month and a year.
Obviously I am thinking of using python for this.
The way I see it, there are two approaches :
-
store all the data of all the months in panda dfs
-
store the data in a db
My question is, what seems better for this? EC2 or Lambda?
I feel Lambda is more suited for this work load as I will be wanting to run this app in such a way that I get weekly or monthly data statistics, and the entire computation would last for a few minutes at max.
Hence I felt Lambda is much more suited, however if I wanted to store all the data in a db, I feel like using an EC2 instance is a better choice.
Sorry if it's a noob question (I've never worked with cloud before, fresher here)
PS : I will be using free tiers of both instances since I feel like the free tier services is enough for my workload.
Any suggestions or help is welcome!!
Thanks in advance
Hi guys, I'm extremely new to AWS. I watched a few of the Amazon AWS training modules and was introduced to Lambda and EC2. From my understanding, they are both computing services. I googled what the difference was between the two and what I read was that Lambda is used for smaller and quicker executions whereas EC2 is used for more high-performance executions. But why would you get lambda in the first place if all you're doing is computing simple tasks? Wouldn't it be easier to compute it locally? I would appreciate if someone could give examples of what kind of situation you would use each kind of resource. Sorry if the question is a bit elementary, I was just introduced to the subject.
Hey, everyone! I’m doing a small side project were I need to run a browser, login, click a button and that’s it. Maybe take a screenshot here and there but that’s about it. I’ve been debating myself with which one should be good for my use case. Lambdas seem like a good approach but I’m not so sure about scalability, whereas on the other hand we have EC2 and more memory to scale
I'm trying to determine whether using lambdas or an EC2 instance is right for my use case. Here are the details:
We're using a RDS t4g.small instance in production, and currently calls to the database are spread across all our code. I'm cleaning this up and creating a data access layer to prevent code duplication. I'd like the data access layer to be fast, but I know there are some limitations when working with RDS. With this instance size I believe 170 connections are allowed according to the AWS docs.
Option 1 - EC2: I could set this up as a persistent server with Express or the like routing request to the correct functions. On the plus side, it would be easy (I think) to restrict the number of database connections and ensure I don't receive errors due to no more connections being available. On the downside, it's a single instance and may slow down when there's a burst in traffic. In addition, I'd be paying for the instance to always be up, and if there were ever an uncaught exception, the database would be offline until I brought the server back up.
Option 2 - Lambda: I could set this up as a series of lambda functions, with each transaction type being an individual lambda. This would allow me to scale and provide very fast responses, even during high traffic bursts, however, with only 80 connections available, I'd need to manage connections to ensure errors aren't being thrown. There are some unknowns here for me. I'm looking into AWS proxy to allow connection pooling, and believe for my RDS instance type that would equate to roughly $25/month. I don't however know if there are other problems I haven't considered with this approach.
I'm new to AWS and have a feeling there are some things I'm not considering. Based on the above information, would you choose one of the above options or recommend something else?
I'd go with Lambda and use RDS Proxy to access RDS so bursts don't overwhelm connection counts.
Also, don't be too tight with your connection limits. Any Lambda bugs that leak connections will quickly hose the DB.
https://aws.amazon.com/rds/proxy/
First question: how is your workload? Any traffic patterns?
Depending on that you would choose either ec2 or lambda. But beware that ec2 may be a bottleneck.
On the other hand, the combo lambda + AWS RDS proxy scales fine, may be overkill if you dont receive much traffic, hence the first question.
Edit: I suppose you are creating a microservice around that rds instance and that it will expose a nice rest api.
Hello everyone, I was studying for my AWS CSAA exam and while I’ve sightly used both services and played around with them in lab environments, I can’t wrap my head around one question. When would I use a EC2 over a Lambda when Lambda is much cheaper and still provides scalability for a backend solution. For most simple web apps, a REST API served thru Lambda seems like a great option (esp when utilizing AWS API Gateway). When would I directly use EC2 (as well as ELB, ASG,etc)? I guess I’m trying to understand stand when a “serverless” web app makes sense. Thanks everyone for your time
EDIT: added serverless clarification
Hi everyone I have a question regarding cost between lambda and ec2 I am building a simple node application using puppeteer that will only act as an api. Obviously don’t expect heavy usage for now but overall Iv read that lambda will be cheaper to run up until heavy usage? Just wanted to get your thoughts if ec2 is better than lambda both in performance and cost. Again this will not be heavily used in either case. Also which ones performance will be better?
My traffic will be relative low. I'll start my project in June with v1. It will be a booking engine something similar to calendly. It is based on express.js. Do you recommend Lambda or EC2 based infrastructure?
I would use Lambda, but I worry about two things regarding to it:
-
When my API is somehow hit with DDoS then the expenses will explode because it is based on request count. I am not sure how reliable Lambda+CloudFront is, for example does it stop DDoS HTTP requests with always changing date ranges in the query string?
-
Cold start might be too slow.
While preparing for an Architect certificate exam, I got this example question from a website:
An application allows a manufacturing site to upload files. Each uploaded 3 GB file is processed to extract metadata, and this process takes a few seconds per file. The frequency at which the uploading happens is unpredictable. For instance, there may be no upload for hours, followed by several files being uploaded concurrently. Which architecture will address this workload in the most cost-efficient manner?
The answer ended up being Store the file in an S3 bucket. Use Amazon S3 event notification to invoke a Lambda function for file processing. However, this got me thinking that wouldn't using spot instances also be a good option since the file would be stored in an S3 bucket, and then spot instances could be spun up once or twice a day.
Which one would you choose to scale your dockerized fullstack application?
They all can scale your app. I'm curious about their performance differences.
I'm trying to implement a Build service (not a CI/CD service) -- a custom service that could remotely compile dart source to js source, returns it as a output.
The problem. this process requires to install flutter sdk on the machine, including the warm up, it could take up to 5 minutes.
Since it is a B2C service with a low latency is highly prioritized.
But i also don't want EC2 instance to handle this, since the build process itself can be quite havy.
Fit:
Lambda fits well all, except that the pre-warming process takes long
EC2 fits well that single warmup has consistency, but does not fit with the scenario of 'building' hundreads of applications in seconds.
Can i have a consistence storage (not s3, a computing storage) and can handle lots of requests with separated computing resource. thoughts?
----
I'm not familiar with EFS. can EFS work as a computing storage which contains executables, and packages? (think it of as node_modules storage)
I dont mind having redis / kafka on the middle for queing.
>>
Lambda + EFS VS Fargate VS EC2
If you want to pursue Lambda, you can workaround pre-warming with newly announced provisioned concurrency: https://aws.amazon.com/about-aws/whats-new/2019/12/aws-lambda-announces-provisioned-concurrency/
How long can a build process take, maximum, including the warmup? If it can go over 15 minutes, you don't want to use Lambda.
What kind of traffic pattern are you expecting? For example if you expect a lot of requests consistently throughout the day, a 5 min warmup won't be much of an issue because you'll always have some "warm" lambda instances.
I'm trying to determine whether using lambdas or an EC2 instance is right for my use case. Here are the details:
We fetch our data from an external source, process it, then store it in our database. For the external source, they allow a maximum of one request every 20ms, or 50 requests per second. Most of the processing that calls this system can be done asynchronously, and there are small bursts every 15 minutes along with larger bursts when we trigger a data fetch.
Option 1 - EC2: I could set this up as a persistent server with Express or the like routing the requests to fetch the correct data from the provider. On the plus side, it would be very easy to throttle my requests to a max of one every 20ms. On the downside, it's a single instance that needs to handle burst traffic, and may slow down. In addition, I'd be paying for the instance to always be up, and if there were ever an uncaught exception, fetching our source data would be stopped until I brought the server back up.
Option 2 - Lambda: I could set this up as a lambda function with a maximum scaling number of something like 5 to 10. No matter how I throttled each lambda function, I still couldn't ensure that two functions wouldn't fire a request within a 20ms period, as they're not aware of each other. When this happens, I'd receive a spike arrest error from the provider, and would need to add handling in my code to retry the call. On the plus side, I could save money with lambdas since the traffic bursts before falling off to nearly zero. On the downside, it would add a little complexity to my code and potentially upset the provider if they need to send me regular spike arrest errors.
I'm new to AWS and have a feeling there are some things I'm not considering. Based on the above information, would you choose one of the above options or recommend something else?
given lambda's 15 min time limit and other limitations, is lambda still a more cost effective option than ec2?
i count about 10 small scale projects off the top of my head for personal use. They will communicate with jenkins and become build jobs. for example, a personal website, a recipes webapp etc
feedback appreciated thanks