1 year ago

#387703

test-img

Ptiseb

Multiple fluent bit parser for a kubernetes pod

I'm working on a K8S environment with Kibana, Elsastic Search & Fluent Bit for the log management.

To parse some of my application logs & extract some usefull data, I added a parser in the Fluent Bit configuration :

[PARSER]
    Name        my-parser-name
    Format      regex
    Regex       my-regex
    Types       a:string b:float c:integer

And I added the following annotation in the deployment.yaml file associated to the application:

metadata:
  annotations:
    fluentbit.io/parser-my-container-name: my-parser-name

At this step, everything's working fine.

I would like to add another parser to extract other data in my logs. I found this question: Can fluent-bit parse multiple types of log lines from one file? but it does not help me, I could not make it work.

I added another parser in my fluent bit configuration:

[PARSER]
    Name        my-new-parser-name
    Format      regex
    Regex       my-new-regex
    Types       d:integer

and I added the following filter:

[FILTER]
    Name        my-filter
    Match       *
    Parser      my-parser-name
    Parser      my-new-parser-name
    Key_Name    log

I restarted elastic search, fluent bit, created a new index pattern in Kibana, but I don't find the new data in Kibana. I also don't know what annotation should I put in my deployment.yaml file, should I remove it ? I'm a little stuck & lost on this post, can someone give me a hand on this topic :) ? I even don't know if I'm doing the right thing to get this new parser...

Thanks !

elasticsearch

kibana

fluent-bit

0 Answers

Your Answer

Accepted video resources