Monitoring AWS Lambda logs with ClickStack using Rotel

Choose the Lambda layer that matches your Lambda runtime architecture. The {version} field is dependent on the AWS region that you’re deploying into. Check the releases page for the latest version numbers that correspond to your region.

Available regions:

• us-east-2, us-west-2
• eu-central-1, eu-north-1, eu-west-3
• ca-central-1
• ap-southeast-2, ap-northeast-2
• ap-south-1
• sa-east-1

In these examples replace {arch}, {region}, and {version} with the appropriate values above.

Option 1: AWS Console

1. Open the AWS Lambda console
2. Navigate to your Lambda function
3. Scroll to the Layers section and click Add a layer
4. Select Specify an ARN
5. Enter the Rotel layer ARN:

   arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}

6. Click Add

Option 2: AWS CLI

aws lambda update-function-configuration \
  --function-name my-function \
  --layers arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}

Option 3: AWS SAM

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      # ... other configuration ...
      Layers:
        - arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}

The Rotel Lambda Extension is configured using environment variables. You need to configure the OTLP exporter endpoint to point to your ClickStack instance. The examples assume your AWS Lambda function is able to reach the ClickStack instance.

Basic configuration (environment variables)

Add these environment variables to your Lambda function:

# Required: ClickStack OTLP endpoint
ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317

# Optional: Authentication headers
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=<YOUR_INGESTION_API_KEY>"

# Optional: Service name (defaults to Lambda function name)
ROTEL_OTEL_RESOURCE_ATTRIBUTES="service.name=my-lambda-api,service.version=1.0.0"

Advanced configuration (using .env file)

For more complex configurations, create a rotel.env file in your Lambda function bundle:

rotel.env:

ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=<YOUR_INGESTION_API_KEY>"
ROTEL_OTEL_RESOURCE_ATTRIBUTES="service.name=my-lambda-api,deployment.environment=production"

Then set the environment variable to point to this file:

ROTEL_ENV_FILE=/var/task/rotel.env

Using AWS Secrets Manager or Parameter Store

For production deployments, store sensitive values like API keys in AWS Secrets Manager or Parameter Store:

AWS Secrets Manager Example:

ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=${arn:aws:secretsmanager:us-east-1:123456789012:secret:clickstack-api-key-abc123}"

AWS Parameter Store Example:

ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=${arn:aws:ssm:us-east-1:123456789012:parameter/clickstack-api-key}"

Required IAM Permissions:

Add these permissions to your Lambda execution role:

For Secrets Manager:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:BatchGetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:clickstack-api-key-*"
    }
  ]
}

For Parameter Store:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameters"
      ],
      "Resource": "arn:aws:ssm:us-east-1:123456789012:parameter/clickstack-api-key"
    }
  ]
}

Invoke your Lambda function to verify logs are being sent to ClickStack:

aws lambda invoke \
  --function-name my-function \
  --payload '{"test": "data"}' \
  response.json

Check the Lambda logs for any errors:

aws logs tail /aws/lambda/my-function --follow

Once configured, log into HyperDX (ClickStack’s UI) and verify that logs are flowing:

Look for these key attributes in the logs:

• service.name: Your Lambda function name
• faas.name: AWS Lambda function name
• faas.invocation_id: Unique invocation ID
• cloud.provider: “aws”
• cloud.platform: “aws_lambda”

1. Open the AWS Console and navigate to AWS Lambda
2. Navigate to your Lambda function
3. Select Configuration → Permissions
4. Click the execution role name to open the IAM console
5. Edit the role and remove any logs:* actions:
   • If using a custom policy, edit to remove logs:CreateLogGroup, logs:CreateLogStream, and logs:PutLogEvents
   • If using the AWS managed policy AWSLambdaBasicExecutionRole, remove it from the role
6. Save the role

Invoke your function again and verify that:

1. No new CloudWatch log streams are created
2. Logs continue to appear in ClickStack/HyperDX

# This should show no new log streams after the policy change
aws logs describe-log-streams \
  --log-group-name /aws/lambda/my-function \
  --order-by LastEventTime \
  --descending \
  --max-items 5

AWS provides OpenTelemetry auto-instrumentation layers for multiple languages:

Find the latest versions in the AWS OpenTelemetry Lambda repository.

Add both the Rotel extension layer and the instrumentation layer:

aws lambda update-function-configuration \
  --function-name my-function \
  --layers \
    arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version} \
    arn:aws:lambda:{region}:901920570463:layer:aws-otel-nodejs-{arch}-ver-1-30-2:1

Set the AWS_LAMBDA_EXEC_WRAPPER environment variable to enable auto-instrumentation:

For Node.js:

AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-handler

For Python:

AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-instrument

For Java:

AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-handler

After invoking your function:

1. Navigate to the Traces view in HyperDX
2. You should see traces with spans from your Lambda function
3. Traces will be correlated with logs via trace_id and span_id attributes