Lambda Extension

The FireTail Logging Extension receives AWS Lambda events and response payloads and sends them to the FireTail Logging API. The extension utilizes a runtime-specific FireTail library in your function code, which outputs logs in a specific format. The FireTail extension then receives these logs from the Lambda Logs API.

Function Libraries

‍

Automated testing is set up using the testing package and github.com/stretchr/testify for shorthand assertions. You can run them with go test, or use the provided Makefile's test target, which is:

make test

This command outputs a coverage report (coverage.out) which you can view in your browser by using the go tool cover command:

go tool cover -html=coverage.out

Deployment

The FireTail Logging Extension is an external Lambda extension, published as a Lambda Layer. You can use the publicly accessible Lambda Layer published by FireTail or follow these steps to build, package, publish, and use the FireTail Lambda Extension.

Build the extension binary

The logging extension is a standard Go project and can be built by installing Go and using the go build command from the root directory of this repository. You will need to set the GOOS and GOARCH environment variables according to your target Lambda runtime's operating system and architecture. For more details, refer to the Environment variables section in the go command documentation. Here is an example command:

GOOS=linux GOARCH=amd64 go build

This command will create a binary with the same name as the root directory. If you have just cloned this repository, the binary will be named firetail-lambda-extension. The target in the provided makefile that corresponds to this step is build. It requires a target architecture (ARCH) that defaults to amd64. For example, you can:

make build ARCH=arm64

This will yield a build and build/extensions directory, and a binary within build/extensions named firetail-extension-${ARCH}.

Package the extension binary

To package the extension binary, place it into a directory named extensions and then zip the directory.

During the Init phase, Lambda extracts layers containing extensions into the /opt directory in the execution environment. Lambda looks for extensions in the /opt/extensions/ directory, interprets each file as an executable bootstrap for launching the extension, and starts all extensions in parallel.

The Makefile includes a target for this step named package, which depends on the build step. It requires a target architecture (ARCH) and an extension version (VERSION), which defaults to latest. For example, you can run:

make package ARCH=arm64 VERSION=v1.0.0

This command will create a .zip file in the build directory named firetail-extension-${ARCH}-${VERSION}.zip. This zip file contains the extensions directory and the binary within it, ensuring that when extracted into /opt, the extension binary will be located in the /opt/extensions/ directory as required by AWS Lambda.

Publish the package

To publish the package, you can use the AWS CLI's publish-layer-version command. You will need to repeat this process for every region in which you wish to use the layer. You will also need to specify the compatible architectures, and give the layer a name. The output of the command will provide you with the layer's ARN and layer version, which you may use to add it to your Lambdas.

If you reuse the same layer name multiple times, the layer version will be incremented. The approach taken in the provided makefile is to publish each extension version with a new layer name, so the layer version will almost always be 1.

The target in the provided makefile that corresponds to this step is publish. You must make the build target before the publish target. The publish target requires a target architecture (ARCH) and extension version (VERSION), which match that used when you made the package target; and a region in which to publish the layer (AWS_REGION). For example, you can:

make publish ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1

Make the layer public (optional)

To make the layer public, you may use the AWS CLI's add-layer-version-permission command. You will need to repeat this process for every layer you publish in every region. You will need to provide the layer name & layer version, a statement ID and region; and to make the layer public an action of lambda:GetLayerVersion and principal of *.

The target in the provided makefile corresponding to this step is public. You must make the publish target before the public target. The public target requires a target architecture (ARCH), extension version (VERSION) and AWS region (AWS_REGION) which match that used when you made the publish target, as well as the layer version created when you made the publish target (AWS_LAYER_VERSION). For example, you can:

make public ARCH=arm64 VERSION=v1.0.0 AWS_REGION=eu-west-1 AWS_LAYER_VERSION=1

Adding the Layer to a Lambda Function

There are several ways to add the published layer to your Lambda Function:

1. Using the AWS CLI.
2. Using An AWS Lambda Docker Build.
3. Using Terraform.

You will need to ascertain the layer ARN of the Lambda Layer containing the FireTail Lambda Extension that you wish to use. If you are not publishing your own FireTail Extension Lambda Layer, you may use the Lambda Layer published publicly by FireTail.

The latest extension version of the publically accessible Lambda Layer published by FireTail can be derived by taking the latest version tag in the Github Releases of this repository, and replacing the . characters with - characters. For example, v1.2.3 would become v1-2-3. You will also need to determine the architecture you need for your Lambda Runtime, which may be either arm64 or x86_64. When you have these two values, you can substitute them into ${VERSION} and ${ARCH} respectively in the following string:

arn:aws:lambda:us-east-1:247286868737:layer:firetail-extension-${ARCH}-${VERSION}:1

For example, for ARCH=amd64 and VERSION=v1-1-0 this should yield:

arn:aws:lambda:us-east-1:247286868737:layer:firetail-extension-x86_64-v1-1-0:1

Environment Variables

To properly configure the FireTail Lambda Layer for your Lambda Function, you must set at least one environment variable: FIRETAIL_API_TOKEN.

If you are not using the default region for FireTail SaaS platform, you will also need to set the FIRETAIL_API_URL environment variable accordingly. For example, if you are using the us.firetail.app region, the appropriate URL would be https://api.logging.us-east-2.prod.firetail.app/logs/bulk.

‍

Using the AWS CLI

To add the Lambda Layer to a Function, you may use the AWS CLI's update-function-configuration command. You will need to provide a region, the layer ARN and the name of the Function to which the layer is to be added.

The target in the provided makefile corresponding to this step is add. The add target requires the layer ARN (LAYER_ARN), the name of the Function to add the layer to (FUNCTION_NAME), and the AWS region in which both the layer and the Function must be found (AWS_REGION). For example, you can do:

‍

make add AWS_REGION=eu-west-1 
LAYER_ARN=your-layer-arn FUNCTION_NAME=your-function-name

‍

Using An AWS Lambda Docker Build

If your lambda is using a container image, you can add the layer to the image from within your Dockerfile. Relevant documentation can be found in this AWS Compute Blog post.

Below is a Docker stage you can add to your Dockerfile to download and unzip the Lambda Layer package. This snippet includes arguments for the ARN of the layer to fetch, the region from which to fetch the layer, and the AWS access key and secret access key to use when fetching the layer:

FROM alpine:latest as firetail-layer-copy

ARG AWS_LAYER_ARN=${AWS_LAYER_ARN:-""}
ENV AWS_LAYER_ARN=${AWS_LAYER_ARN}
ARG AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION:-"us-east-1"}
ENV AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
ARG AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-""}
ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
ARG AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY:-""}
ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}

RUN apk add aws-cli curl unzip
RUN mkdir -p /opt
RUN curl $(aws lambda get-layer-version-by-arn --region ${AWS_DEFAULT_REGION} --arn ${AWS_LAYER_ARN} --query 'Content.Location' --output text) --output layer.zip
RUN unzip layer.zip -d /opt
RUN rm layer.zip

You will then need to add the following step into the final stage in your Dockerfile to copy the extension from the firetail-layer-copy stage into the /opt/extensions directory in your final container image:

COPY --from=firetail-layer-copy /opt /opt

When these steps are complete you should be able to run your build process as before, except with the addition of the AWS_DEFAULT_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_LAYER_ARN build arguments.

Note: Ensure you securely use, and store these minimal access credentials

docker build . -t layer-image1:latest \
--build-arg AWS_DEFAULT_REGION=us-east-1 \
--build-arg AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
--build-arg AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
--build-arg AWS_LAYER_ARN=arn:aws:lambda:us-east-1:247286868737:layer:firetail-extension-x86_64-v1-0-0:1

‍

Using Terraform

Find below an example Terraform configuration that adds the FireTail extension to a Lambda Function as a layer:

resource "aws_lambda_function" "extensions-demo-example-lambda-python" {
        function_name = "LambdaFunctionUsingFireTailExtension"
        s3_bucket     = "lambda-function-s3-bucket-name"
        s3_key        = "lambda-functions-are-great.zip"
        handler       = "handler.func"
        runtime       = "python3.8"
        role          = aws_iam_role.lambda_role.arn

        environment {
                variables = {
                        FIRETAIL_API_TOKEN = "firetail-api-key",
                        FIRETAIL_API_URL = "https://api.logging.eu-west-1.prod.firetail.app/logs/bulk"
                }
        }

        layers = [
            "arn:aws:lambda::247286868737:layer:firetail-extension--:1"
        ]
}

‍