CloudWatch Dashboard fails to report Lambda timeouts

[nadove-ucsc]

The CloudWatch dashboard uses the query string "Task timed out" to identify lambda timeouts. Currently, Lambda timeouts do not emit any logs containing this string, causing the dashboard to falsely report that no timeouts have occurred when they did in fact occur.

This query produces no results:

SOURCE "arn:aws:logs:us-east-1:465330168186:log-group:/aws/lambda/azul-indexer-anvilprod-mirror" START=2026-06-25T00:00:00.000Z END=2026-06-25T23:59:59.000Z |
filter @message like 'Task timed out'

This one reveals the actual occurrences:

SOURCE "arn:aws:logs:us-east-1:465330168186:log-group:/aws/lambda/azul-indexer-anvilprod-mirror" START=2026-06-25T00:00:00.000Z END=2026-06-25T23:59:59.000Z |
filter @message like 'Status: timeout'

[
    {
        "@timestamp": "2026-06-25 23:59:41.096",
        "@message": "REPORT RequestId: 4725a167-21a7-5731-99c2-c3d0e3dfd770\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1201 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:59:04.490",
        "@message": "REPORT RequestId: 820788a3-fb80-56f2-b287-ea144fc4d510\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1218 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:27:59.151",
        "@message": "REPORT RequestId: a5b090f1-91ed-54fd-9d24-f2463d166aa3\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1205 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:12:11.038",
        "@message": "REPORT RequestId: d3898837-7beb-5782-ac43-ea80d2ad95d8\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1220 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:06:53.883",
        "@message": "REPORT RequestId: db7a78a2-1f44-59ef-89e6-1a15b74cca3e\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1280 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:01:38.857",
        "@message": "REPORT RequestId: 5cc24f0c-8093-5d1f-8b71-2dbb265d41a6\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1280 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 22:00:37.546",
        "@message": "REPORT RequestId: c7952ecb-f9cf-53be-9f7a-4c7c5f5e9688\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1280 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 21:49:45.305",
        "@message": "REPORT RequestId: 3fe1d15a-f70f-502d-ac75-b3ed530abb0b\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1280 MB\tStatus: timeout\n"
    },
    {
        "@timestamp": "2026-06-25 21:42:15.570",
        "@message": "REPORT RequestId: 57b43aab-97ae-51c2-b4f2-581c35ed2660\tDuration: 180000.00 ms\tBilled Duration: 180000 ms\tMemory Size: 1280 MB\tMax Memory Used: 1220 MB\tStatus: timeout\n"
    }
]

[dsotirho-ucsc]

For demo, show Lambda timeouts reported in CloudWatch dashboard for anvilprod, retroactively for the timeframe during which this was originally observed.