Set up AWS API Gateway logging with Kinesis FireHose

The AWS API Gateway Logging with Kinesis Firehose integration allows you to collect and analyze API Gateway log data by directing it into Amazon Kinesis Firehose. Using CloudFormation templates, this integration deploys the necessary resources, including IAM roles. To complete the setup, attach the logging configuration to the required APIs by following the steps detailed below. To set up the integration:

1. Navigate to Integrations in the FireTail platform. Select the Create Integration tab.
2. Click Set up Firetail API Gateway logging in an AWS Region with Kinesis.
3. Enter a name for the integration.
4. Select the AWS Region.
5. Select an application from the dropdown, or click Create to create a new application. This is the application that will be associated with the integration. Learn more about applications here.
6. Under App Token, click Create to create a token. Enter a name for the token. Click OK. Copy the app token. This is needed when the template is launched.

7. Log in to AWS.

8. Click Launch CloudFormation to launch the template. This opens in a new window.

9. Paste the app token into the HttpEndpointAccessKey field.

10. Select the checkbox; I acknowledge that AWS CloudFormation might create IAM resources. Click Create stack.

11. When the CloudFormation Stack has a status of CREATE_COMPLETE, copy the AWS Kinesis Firehose ARN and the AWS Role ARN from the Outputs tab.

12. Paste the above data into the AWS Kinesis Firehose ARN and AWS Role ARN fields respectively.

13. Click Submit.

Add logging to an API

When the integration has been created you must add logging every API Gateway API that you want the logging for. This needs to be done for logging to start.

1. In the FireTail platform, open the integration you have created. Copy the AWS Kinesis Firehose ARN value.

2. Log in to AWS, go to the API Gateway service. Select the API to be connected (must be v1 aka REST).

3. Click Stages in the left menu.

4. Scroll down to Logs and Tracing and click Edit.

5. Enable Custom Logging Access and fill in the following fields:

• Access log destination ARN - Paste the AWS Kinesis Firehose ARN from the Cloudformation template output in the integration into the field.
• Log-format - Paste the v1 log format below.

Log format:

{"version": "1.0.0-alpha", "metadata": {"region": "eu-west-1", 
"api_version": "1", "requestId": "$context.requestId", 
"user": "$context.identity.user", "caller": "$context.identity.caller", 
"stage": "$context.stage", "status": "$context.integration.status", "accountId": 
"$context.accountId", "apiId": "$context.apiId", "wafArn": 
"$context.webaclArn", "wafResponse": "$context.wafResponseCode", "domainPrefix": 
"$context.domainPrefix", "wafLatency": "$context.waf.latency", 
"integrationStatus": "$context.integrationStatus", "auth": 
{"authorizeLatency": "$context.authorize.latency", 
"authorizeStatus": "$context.authorize.status", "authorizerLatency": "$context.authorizer.integrationLatency", "authorizerStatus": 
"$context.authorizer.integrationStatus", "authenticateLatency": 
"$context.authenticate.latency", "authenticateStatus": "$context.authenticate.status"}, "customDomainBasePath": "$context.customDomain.basePathMatched"}, 
"request": {"httpProtocol": "$context.protocol", "ip": "$context.identity.sourceIp", 
"headers": {"User-Agent": ["$context.identity.userAgent"]}, 
"uri": "$context.domainName$context.path", "resource": "$context.resourcePath", 
"method": "$context.httpMethod"}, "response": {
"headers": {"Content-Length": ["$context.responseLength"]}, 
"statusCode": "$context.integration.status"}, 
"dateCreated": "$context.requestTimeEpoch", "executionTime": 
"$context.responseLatency", "oauth": {"sub": "$context.authorizer.claims['sub']"}}

6. Click Save.