Maybe I'm missing something here, from an architectural point of view, I can't wrap my head on using node inside a lambda. Let's say I receive 3 requests, a single node instance would be able to handle this with ease, but if I use lambda, 3 lambdas with Node inside would be spawned, each would be idle while waiting for the callback.
Edit: Many very good answers. I will for sure discuss this with the team next week. Very happy with this community. Thanks and please keep them coming!
Can and should I use express.js in lambda?
AWS Lambda: Python vs Nodejs
Are there any performance/functionality differences for AWS Lambda functions written in node.js/Java - Stack Overflow
Go Lambda vs Node.js Lambda vs Python Lambda
Videos
I was looking at the basic template for Lambda functions and comparing the python and nodejs languages. The thing that stood out to me is that the nodejs template has the "async" functionality already listed while the python function, to my knowledge, is not able to use this type of functionality.
My question, are there pros and cons to using python or nodejs for lambda functions? I was under the impression you can complete the same tasks using either and choosing was based on preference.
Thank you
While Java has a slower startup time, it's runtime performance is a lot better than the one of node.js or Python. That means Java Lambda functions make sense if:
- They are Invoked regularly - AWS Lambda will reuse it's initialized containers and you can benefit from the better performance without the cold start latency
and/or
- They have a long runtime - If your function is running e.g. for 1 or 2 minutes, you do not really care about 5 seconds of startup if the function itself is running an order of magnitude faster than node.js
You may want to take a look at this github project that compares the cold start times.
One big factor of Java's bad cold starts is probably the fact that Java projects are often a lot bigger than e.g. a node.js function. In any way you should always try to keep functions as small as possible to reduce the initial latency.
And of course performance shouldn't be the only factor when choosing your programming language. I personally think node.js is extremely convenient when working e.g. with JSON data, this is why I use it in most of my functions.
Java functions on Lambda generally require more resources and take much longer to respond from a cold start, when compared to NodeJS functions. Otherwise there is no difference.
Are there any performance differences among them?
If I write Lambda serverless backend with Go, will it perform much faster than Node.js Lambda? Which language is the fastest for AWS Lambda?
Why do people make their functions like this:
const add = (a,b)=>{
console.log(a+b);
}
and not like this:
function add(a,b){
console.log(a+b);
}
What's the difference?
The answer to your question depends upon your current and future needs. I think you should always plan ahead and make sure that the current infrastructure that you will implement can be upgradable for future needs.
You should ask yourself those questions:
- In the future do I want to have some websocket connection?
- Do I need any proxy in my request routing?
- How big will become my application over time?
- Which AWS Service will I expect to use in the future
Scalability
Using Express.js in a lambda is not a good idea for many reasons:
- You will pay more for the execution of your Lambda because it will take more time to run and maybe more memory
- Latency is higher
- Doing a small modification means redeploying all your application code on 1 lambda so only 1 point of failure.
- Normally overtime your application's code base will grow as you add more features. The maintenance of that monolithic repo will be a pain in the ass and you will deploy less than you want because of the bugs you might encounter.
Cost effectiveness
Express.js on Lambda is more expensive because you need to proxy any method into your lambda using a API Gateway REST API rather then using API Gateway HTTP API
HTTP APIs are up to 71% cheaper compared to REST APIs
Latency
Lambda isn't magically executing your code without server even tho they market it like it is. When an event occur, AWS will launch a docker container, wait for it to fully load all your dependencies and then run your handler.
With a normal Node.js server on AWS EC2 or AWS ECS it's a one time cost because your server is always running and all your dependencies are already loaded but not on your lambda.
As AWS says:
This approach [Express.js and proxy all requests to your lambda] is generally unnecessary, and it’s often better to take advantage of the native routing functionality available in API Gateway. In many cases, there is no need for the web framework in the Lambda function, which increases the size of the deployment package. API Gateway is also capable of validating parameters, reducing the need for checking parameters with custom code. It can also provide protection against unauthorized access, and a range of other features more suited to be handled at the service level.
Best practices for organizing larger serverless applications
How to convert Express framework into plan Lambda
To simplify your life, I would suggest you to use SAM CLI. It's very simple to get started with it.
Install SAM CLI
If you're following the MVC pattern for your Express app. You only need to take your service files where your core logic live.
The folder structure I like to use for the lambda is the following
Suppose it's a small calendar app
──src-ts
├───handlers
│ getEvent.ts
│
├───tests
│ │ getEvent.tests.ts
│ │
│ └───utils
│ utils.ts
│
└───utils
utils.ts
validation.ts
It's important that your handler returns 3 thing
- Headers (JSON)
- statusCode (number)
- Body (stringified)
You also need a template.yml file to describe the infrastructure that your lambda need
AWSTemplateFormatVersion: 2010-09-09
Description: Describe the lambda goal
Transform:
- AWS::Serverless-2016-10-31
Resources:
# API Gateway
LambdaAPI:
Type: AWS::Serverless::Api
Properties:
StageName: StageName
Cors:
AllowMethods: "'POST, GET, OPTIONS'"
AllowHeaders: "'*'"
AllowOrigin: "'*'"
# IAM Role
LambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- 'sts:AssumeRole'
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
- arn:aws:iam::aws:policy/AmazonAPIGatewayInvokeFullAccess
GetEvent:
Type: AWS::Serverless::Function
Properties:
Runtime: nodejs12.x
Timeout: 180
FunctionName: youLambdaName
Handler: src/handlers/getEvent.handler
Role: !GetAtt LambdaRole.Arn
Events:
Get:
Type: Api
Properties:
RestApiId: !Ref LambdaAPI
Path: /events/{eventid}
Method: GET
Note
I used typescript and but when compiled it's creating an src folder
Some resource to help you more in depth:
- https://aws.amazon.com/blogs/compute/going-serverless-migrating-an-express-application-to-amazon-api-gateway-and-aws-lambda/
- https://dev.to/brightdevs/how-to-convert-an-express-app-to-aws-lambda--44gc
- https://medium.com/hackernoon/how-to-deploy-a-node-js-application-to-aws-lambda-using-serverless-ae7e7ebe0996
- https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
To Conclude
The upside of using lambdas without Express are:
- better scalability
- cost optimization
- lower latency
- higher availability because you have multiple lambda for each business logic instead of one that run it all
The downside of using lambdas without Express are:
- You need to modify your existing code
- Init time of your lambda needs to be part of your thinking when developing your logic
- You need to learn SAM yaml template and read the AWS doc when you wanna add more functionality to your API infrastructure.
Take advantage of the AWS infrastructure, don't try to go against it. All AWS Services are working together in a seamless and low latency way. You should remove Express from your infrastructure if you wanna go "Serverless".
If it's a bit hefty to be function as you said, you may think in creating a docker image for your app and run it using Fargate ECS which is considered a serverless option.