I've been losing myself in rewriting a bunch of lambda functions. I can't help but notice that there's a bunch of different opinions and philosophies that make up the bulk of the samples and documentation.
In my own team, one of the guys wrote his own test framework and schema validation library. Another wrote his own logging library and was a big fan of the Serverless framework. I've been using Lambda Power Tools and unittest myself but my lambdas are pretty vanilla. Three guys, three different development styles.
Everything feels so arbitrary and I'm not sure what actually constitutes a production-ready, well structured Python lambda.
FWIW We use terraform for initial project deployment with codebuild, and I have my own makefiles to deploy if I want to bother outside of it. Most of my lambdas are interfaced by AGW and interact with s3, dynamodb, and sagemaker.
Edit: to be clear - most of these are <200ms calls. I'm not trying to cram crazy things into a lambda. If anyone could share some code samples of some well-done Lambda's I'm happy.
amazon web services - How to package and deploy AWS python lambda functions automatically - Stack Overflow
Lambda function
End-to-End Tutorial on Combining AWS Lambda, Docker, and Python
AWS Lambda recently added support for Docker images with Python, where before you had to zip all your code and dependencies. With this update I wanted to make a full length tutorial to cover everything necessary to be productive.
The playlist is broken into six parts:
-
An introduction to the app that will be built and the various AWS components that will be used
-
Cloning the GitHub repo and doing necessary setup to test the application locally
-
Creating a Docker image of the Python script and running it on AWS Lambda, while also showing how painful it is to do even minor updates
-
Adding CI/CD with GitHub Actions to speed up code updates
-
Adding AWS Cloudwatch to create a cron job that schedules the Lambda function to run on periodic intervals
-
Walking through an AWS Cloudformation yaml template that will create the Lambda function and Cloudwatch rule automatically and covering all the benefits of using a Cloudformation template
This tutorial truly is end-to-end. If you enjoy the content, you can help me a ton by doing any or all of the following:
-
supporting me on Patreon
-
subscribing to my YouTube channel
-
liking and/or commenting on the video
-
sharing the video or channel on any platform (reddit, twitter, discord, etc.)
-
starring the GitHub repo
-
following me on GitHub
Any questions or requests, just leave a comment.
More on reddit.comLooking for a basic and simple Lambda tutorial
I'm working on an open source CLI / Python SDK called LaunchFlow that sets up the exact automation you described! All you need to do is supply a requirements.txt for the Python dependencies, and the tool will zip up your local workspace + Python dependencies and release it to your Lambda function.
The code looks like this:
import launchflow as lf
import numpy as np
def handler(event, context):
result = np.random.randint(0, 100)
return {
"statusCode": 200,
"body": f"Random Number: {result}",
}
api = lf.aws.LambdaService(
name="my-lambda-api",
handler=handler,
runtime=lf.aws.lambda_service.PythonRuntime(
requirements_txt_path="requirements.txt"
),
)
LambdaService Docs link
Just pip install launchflow[aws] + lf deploy to set up the automation with your local AWS credentials.
If you'd rather create your own automation, you could look at the LambdaService source code to see how we automate the Lambda build / release steps with boto3.
For the git piece: I'd recommend just checking in your main python file and a related set of requirements.txt files.
The two paths that I've used for easier Lambda deployments are Serverless and Terraform. I'll outline them briefly (there's also Amazon's CDK but I am less familiar with it).
- Serverless is basically a wrapper around Amazon's Infrastructure as Code tool (CloudFormation); either a set of JSON or YAML that define your Lambda and any associated resources. Here's a decent example of your setup i.e. a git tracked
handler.py, arequirements.txtfile and aserverless.ymlto deploy the lambda. - For Terraform, it's a more general purpose Infrastructure as Code tool that has plugins for each major cloud provider. The best example I could dig up was a simplified repo
Terraform's main advantage is for change tracking in a more generic way, but Serverless I think is simpler if you're not familiar with either one. (Serverless uses CloudFormation stacks for tracking changes, which are a bit funkier than Terraform's state management).
Most ideally; you'd want to have these things run in a pipeline on merge, so that your lamda is updated with new code changes after a code review and merge into a git repo. It might look like:
- Someone submits a Pull Request to your Git repository
- The request is reviewed/ merged
- On before/merge, some tests run to verify the code
- You run the serverless/terraform deploy to update the lambda
- Do some final, end to end verification on the new deployment
Let me know if you need me to clarify anything!