Amazon CloudWatch Health Check

Gremlin offers several ways of creating AWS Health Checks: either adding an Amazon CloudWatch monitor or alarm, or by automatically creating Health Checks using Intelligent Health Checks. In both cases, the first step is to grant Gremlin permission to access your CloudWatch environment if you haven't already done so.

‍

Authenticating Gremlin to AWS

Please see our AWS integration documentation for instructions on authenticating Gremlin to your AWS account.

‍

Using Intelligent Health Checks

For AWS services that are mapped to an Application Load Balancer (ALB) or Classic Load Balancer, Gremlin can automatically create Health Checks for you. To enable Intelligent Health Checks:

1. Open the service's Settings page and navigate to Health Checks.
2. Under Intelligent Health Checks, click on the Type drop-down menu and select AWS. Then, click + Add.
   
3. Select the AWS account where your service's load balancer is deployed. If you haven't yet authenticated with AWS, follow the steps in Adding an AWS account in our AWS integration guide. Then, click Next.
4. Select the Application or Classic Load Balancer that you want to map to this service. Gremlin will use the selected load balancer's CloudWatch metrics as an indicator of service health. Then, click Next.
5. Click the checkbox next to Use Intelligent Health Checks for this service and click Save.

These Health Checks can be used instead of—or in tandem with—regular Health Checks.

‍

Adding an AWS CloudWatch alarm as a Health Check

Instead of using Intelligent Health Checks, you can also use any CloudWatch alarm as a Health Check. To add a CloudWatch alarm as a Health Check:

1. Open the Gremlin web app and navigate to Health Checks, or click this link.
2. Click + Health Check.
3. From the Observability Tool drop-down, select AWS. If you’ve already authenticated Gremlin to your AWS account, select your account ID from the AWS Account ID box. Otherwise, follow the instructions above. Click Next.
4. Enter a Name for the Health Check. We recommend using the same name that you use in CloudWatch.
5. Select Create a Health Check from an AWS CloudWatch Alarm URL.
6. Open the alarm you wish to use in the AWS Console, then copy its URL from your browser window.
7. Go back to the Gremlin web app and paste the URL into the Monitor or Alert URL box.
8. Click Test Health Check to confirm that Gremlin can access your monitor, and that it’s reporting back as healthy.
9. Click Create Health Check.

Alternatively, you can define custom success criteria for your Health Check by using the AWS API directly.

1. After entering the name of your Health Check, select Create a Health Check from AWS API.
2. Copy and paste the URL of your CloudWatch alert into the Monitor or Alert URL box.
3. Click Test Connection to confirm that Gremlin can access your monitor. Gremlin will also show the HTTP response code and the JSON body of the response.
4. Set the Success Evaluation Criteria. This is the criteria Gremlin will use to determine whether the alert is healthy, or if it’s in an alarm state. By default, Gremlin checks the value of .DescribeAlarmsResponse.DescribeAlarmsResult.MetricAlarms[0].StateValue to see if it equals OK. You can use any field here and compare it to any value. You can also specify the HTTP status code to look for, and set a maximum response timeout.
5. Click Create Health Check.

‍

Troubleshooting

My load balancer doesn't appear in the list of selectable load balancers

Only Application Load Balancers (ALBs) and Classic Load Balancers can be selected. Network Load Balancers (NLBs) and Gateway Load Balancers (GWLBs) are filtered out because they do not publish the application-level CloudWatch latency metrics that Intelligent Health Checks require.If your service sits behind an NLB, you can still create a Health Check manually from a CloudWatch alarm on NLB metrics (for example, UnHealthyHostCount or TCP_Target_Reset_Count) using the steps in Adding an AWS CloudWatch alarm as a Health Check.