Home About Me

Debugging an ECS Task That Passed Local Curl but Failed an ALB Health Check

2026-02-05

The health check failure pointed at the ALB, but the useful evidence was inside the task and on the node behind it. The checks below separate the container itself from the path in front of it.

Start inside the task

Log in to the EC2 instance that hosts the task container through a jump server, then use Docker to inspect and enter the container. Confirm that resources, the program, configuration, and processes are all up.

# Log in to the ECS node EC2 through the jump server
$ ssh -i ~/.ssh/your-jump-server-keys.pem [email protected] -J jump-server

# Run the docker command
$ docker ps

A note on finding the node IP: in the ECS cluster, open the task that needs investigation. The task detail view shows Launch type. The EC2 label includes an i-XXXX node ID; follow that link to the EC2 detail page. Under Networking, Private IPv4 addresses lists two IPs. Use the primary one that matches Private IP DNS name (IPv4 only). The jump server also needs network reachability to the ECS network, such as a VPC peering connection.

Some containers have neither curl nor apt available for installing it. In that case, download curl-static on the node EC2 and copy it into the container:

$ docker cp curl-aarch64 container-name:/tmp/
$ docker exec -it container-name sh
$ chmod +x /tmp/curl-aarch64
$ /tmp/curl-aarch64 http://127.0.0.1:3000

ECS tasks run their containers with Docker, so this is a direct way to inspect what is happening inside a task.

Check the listening address

Web services deserve extra attention here. npm start almost always binds to localhost unless told otherwise, so bind explicitly to 0.0.0.0. With Next, for example:

$ next start -H 0.0.0.0 -p 3001

Test the container IP from the node

Inside the container, request localhost:3000, assuming that is the listening port. A successful response confirms that the service started and is serving requests.

Inside the container, use ifconfig to get the task/container IP, for example 10.0.16.170, then exit the container. You can also get the IP from the service registered in the TargetGroup. From the EC2 instance that hosts the container, run curl against 10.0.16.170:3000. If that request fails, the security group attached to the task's Service has not allowed the port.

Confirm what TargetGroup registered

Once service access is confirmed, check the TargetGroup. The container IP and port registered by ECS should match expectations. At that point, the ALB-to-task-container path is clear.

What the checks showed

  1. In the path from TargetGroup to the ECS task container, Docker image EXPOSE and Ansible portMappings have no practical effect. Whether those settings are correct or incorrect does not change the access result. Ansible comes into this because ECS task deployment used an Ansible Playbook.

  2. In actual testing, whether requests work depends only on whether the ALB can reach the service inside the container through the container IP and container port. Two factors are most critical: the service inside the container listens on the right address (0.0.0.0), and the ECS Service security group allows the port being accessed.

Related Posts