python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - fluentd

FluentD uploads new file, instead of rotated file to s3 for hourly log rotation of log files

We have a python application that generates hourly rotating log, and we have set up the time for each rotation as start of each hour, i.e. the log rotation would happen at 10:00, 11:00, 12:00 .... The...
test-img

Sameer Ranjha

amazon-web-services

kubernetes

amazon-s3

fluentd

Votes: 0

Answers: 1

Latest Answer

Here is our config in which we send logs to s3 every minute. type s3 <template> s3_bucket "mybucket" s3_region "my_region" path my_path/ s3_object_key_format %{path}%{t...
test-img

Riz

Handling multiline formats in splunk

I am not sure of how to set the BREAK_ONLY_BEFORE I have try setting BREAK_ONLY_BEFORE: date but it throws error My logs are : [2022-04-05 11:18:23,839] WARN Error while loading: connectors-versions...
test-img

Achu

splunk

multiline

fluentd

Votes: 0

Answers: 1

Latest Answer

The better method is to set LINE_BREAKER to a regular expression that defines the start of each event. These props.conf settings should do it. [mysourcetype] SHOULD_LINEMERGE = false LINE_BREAKER = (...
test-img

RichG

Fluentd - Kubernetes - ParserError error=&quot;pattern not matched with data&quot;

I'm trying to redirect Kubernetes logs from containers to OpenSearch. But there is always some error with the date. What am I doing wrong? Docker logs example: {"log":"time=\"2022-...
test-img

JDev

docker

kubernetes

logging

fluentd

fluent

Votes: 0

Answers: 1

Latest Answer

The date/time from your logs is 2022-04-01T09:54:33Z note no milliseconds. While in your config has time_format %Y-%m-%dT%H:%M:%S.%N%z %N - Fractional seconds digits, default is 9 digits (nanosecond)...
test-img

Piotr Malec

error_class=Fluent::Plugin::Buffer::BufferChunkOverflowError error=&quot;a xxxx bytes record (nth: 0) is larger than buffer chunk limit size (10240)&quot;

I have the following problem i am setting up a fluentd server but i get the following error: 2022-03-29 23:35:44 +0000 [error]: #0 unexpected error on reading data host="192.190.204.60" por...
test-img

Jose Roberto Nava Morales

linux

fluentd

opensearch

Votes: 0

Answers: 0

Posts

Questions

Blogs

Jobs