AWS Lambdas with Terraform

This project was used to quickly create some server-less functions with AWS Lambda and make them accessible via HTTP requests.

The steps you will need to do to create them in your AWS account:

• Login with AWS cli tools (instructions in README.md)
• Create bucket for the tfstate to be stored (in AWS S3)
• Add your bucket details to main.tf
• Run terraform init
• Run terraform apply, review, then confirm with yes

Once that completes you should be able to access the 2 new routes

# Echo your request back to you
https://<lambda_integration.app_id>.execute-api.<aws region>.amazonaws.com/hello

# Print some text
https://<lambda_integration.app_id>.execute-api.<aws region>.amazonaws.com/hello2

That's all you should need to do. However I advise against running code you randomly found on the internet against your AWS account without understanding what it does first. So take a look at the repo (it's not too big) and we will go through some of the details here.

The project was built to easily create Lambdas that could be accessible at different paths on the API Gateway. You don't need to use the API Gateway as a trigger though. As you'll be able to see in the README.md there are 2 blocks that you can add. The first is to create the actual Lambda:

module "lambda__hello_lambda" {
    source              = "./lambda"
    function_name       = "hello_lambda"
    function_directory  = "scripts/hello_lambda"
    function_handler    = "hello_lambda.lambda_handler"
    function_runtime    = "python3.8"
    iam_role            = module.iam_role.arn
}

The second is for the Gateway:

module "lambda_gateway__hello_lambda" {
    source = "./lambda_api_gateway"
    api_route_path = "/hello"
    api_gateway_id = module.api_gateway.id
    api_gateway_execution_arn = module.api_gateway.execution_arn

    lambda_arn = module.lambda__hello_lambda.arn
    lambda_invoke_arn = module.lambda__hello_lambda.invoke_arn
}

You can omit the 2nd if you are using a different trigger.

The code base is split up using modules to make it simpler to add new Lambdas, reuse code, and make the everything easier to understand. It's all rather simple once it's broken down into pieces.

IAM Role

The IAM Role is required to actually create anything.

resource "aws_iam_role" "iam_role_resource" {
    name = var.iam_role_name
    assume_role_policy = jsonencode({
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": "sts:AssumeRole",
          "Principal": {
            "Service": "lambda.amazonaws.com"
          },
          "Effect": "Allow",
          "Sid": ""
        }
      ]
    })
}

It only needs access to lambda.amazonaws.com. So this is fairly straight-forward.

Creating the Lambda

There are 2 stages involved in creating the Lambda itself.

data "archive_file" "lambda_zip" {
    type        = "zip"
    source_dir  = var.function_directory
    output_path = "${var.function_directory}/lambda.zip"
}

resource "aws_lambda_function" "lambda" {
    filename      = data.archive_file.lambda_zip.output_path
    function_name = var.function_name
    role          = var.iam_role
    handler       = var.function_handler

    source_code_hash = data.archive_file.lambda_zip.output_base64sha256
    runtime = var.function_runtime
}

We zip up the code you want using Terraforms archive_file tool. Then the aws_lambda_function will do the creation and the uploading.

The archive will be created in the function_directory e.g. scripts/hello_lambda/lambda.zip. These are generated by Terraform and don't need to be committed to version control. There's a .gitignore line excluding these.

At this point the Lambda exists, and you can Test trigger it from within the Lambda Web Interface.

Creating the API Gateway

Triggering via the Web Interface may be sufficient, but you also may want to let other people trigger it that don't have access to AWS. An easy solution to this is to create a HTTPS interface in front of your Lambda using API Gateway.

We have 2 separate components for this part. We have a single API Gateway that will be used by all of the Lambdas in this project. On top of that we have individual routes that will forward traffic to our functions using separate URL paths (the part after the /).

For the shared Gateway we need the Gateway itself and a stage. There's not a lot of lines required for this:

resource "aws_apigatewayv2_api" "lambda_api" {
    name          = "lambda-gateway"
    protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "lambda_stage" {
    api_id      = aws_apigatewayv2_api.lambda_api.id
    name        = "$default"
    auto_deploy = true
}

Then there's the module to create per Lambda API routes:

resource "aws_apigatewayv2_integration" "lambda_integration" {
    api_id           = var.api_gateway_id
    integration_type = "AWS_PROXY"
    integration_method   = "POST"
    integration_uri      = var.lambda_invoke_arn
    passthrough_behavior = "WHEN_NO_MATCH"
}

resource "aws_apigatewayv2_route" "lambda_route" {
    api_id             = var.api_gateway_id
    route_key          = "${var.method} ${var.api_route_path}"
    target             = "integrations/${aws_apigatewayv2_integration.lambda_integration.id}"
}

resource "aws_lambda_permission" "api-gw-AllowExecutionFromAPIGateway" {
    statement_id  = "AllowExecutionFromAPIGateway"
    action        = "lambda:InvokeFunction"
    function_name = var.lambda_arn
    principal     = "apigateway.amazonaws.com"

    source_arn = "${var.api_gateway_execution_arn}/*/*/*"
}

There's a bit more going on here, but basically we have 3 things:

• A permission to execute the Lambda
• An integration with the Lambda
• A route to identify this integration

The integration is done using AWS_PROXY and must be a POST. This is for the connection from Gateway to Lambda, you can still use GET for your API requests. I set the var.method variable to "GET" for the front facing interface (so I can easily access from the browser, or bookmark).

Enabling Logs

At this stage the API was working great. Then I tried scripts that were a little more complicated and was greeted with "Internal Server Error". When I looked at the Lambda Monitoring / Logs page AWS was telling me that more configuration needed to be done.

There are 2 blocks that are required for your Lambdas to be able to display logs.

Give the IAM CreateLogStream and PutLogEvents permissions:

# To Enable Logs
resource "aws_iam_role_policy" "log_writer" {
  name = "lambda-log-writer"
  role = aws_iam_role.iam_role_resource.id

  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ],
        "Resource": "arn:aws:logs:*:*:*"
      }
    ]
  })
}

and create the CloudWatch log group. The name is the important connection. It must be of this format and match your function name. Then everything will hook up nicely.

resource "aws_cloudwatch_log_group" "lambda_function" {
  name              = "/aws/lambda/${var.function_name}"
  retention_in_days = 14
}

Conclusions

And that's it. This process can seem pretty complex as a whole, but when broken down into it's individual pieces (and separated into modules) it's actually quite simple. We mostly use this for internal tooling. For example starting and stopping development machines via Slack. It allow us to API-ify some commands so that developers do not need AWS credentials to do some very specific, controlled actions.