1 year ago

#353506

test-img

Spenhouet

Grafana Loki LogQL: How to parse log lines with different log formats?

We have different type of logs for one and the same application. Some are coming from our application which logs in a JSON format and others are different types of log messages.

For example these 3 log lines:

"{\"written_at\": \"2022-03-30T07:51:04.934Z\", \"written_ts\": 1648626664934052000, \"msg\": \"Step 'X' started at 2022-03-30 07:51:04\", \"type\": \"log\", \"logger\": \"my-logger\", \"thread\": \"MainThread\", \"level\": \"DEBUG\", \"module\": \"my.module\", \"line_no\": 48}\n"
"                    ERROR    Data processing error: Did not work       \n"
"FileNotFoundError: [Errno 2] No such file or directory: '/local.json'\n"

To parse our application JSON logs we perform the following LogQL query:

| json log="log" 
| line_format "{{.log}}"
| json | line_format "{{.msg}}"
| __error__ != "JSONParserErr"

As our query already states, we can not parse the other line logs since they are not in JSON format.

Can we define different parsing and formatting depending on conditions? Or as fallback when the JSONParserErr happens?

grafana

grafana-loki

logql

0 Answers

Your Answer

Accepted video resources