-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Description
When i am using syslog-ng and syslogng-kafka to push system logs to kafka Server. I notice that the messages are not json format, messages use single quote (which should be double-quotes ) and can not be recognized and parsed by logstash
reproduce
I use syslog-ng -F to debug and the logs be produced like this
1 2018-05-30T13:57:14.721616+08:00 www.test.com syslogng_kafka 32288 - - Message produced: {'FACILITY': 'cron', 'PRIORITY': 'info', 'HOST': 'www.test.com', 'PROGRAM': 'CROND', 'DATE': '1527659821', 'MESSAGE': '(root) CMD (/usr/lib64/sa/sa1 1 1)'}On logstash server i received the _jsonparsefailure error

syslog-ng conf
source s_sys {
file ("/proc/kmsg" program_override("kernel: "));
unix-stream ("/dev/log" max-connections(512));
internal();
};
destination syslog_to_kafka {
python(
class("syslogng_kafka.kafkadriver.KafkaDestination")
on-error("fallback-to-string")
options(
hosts("xxx:9092")
topic("system-log")
verbose("True")
display_stats("True")
)
);
};
log {
source(s_sys);
destination(syslog_to_kafka);
};
logstash conf
input {
kafka {
bootstrap_servers => "xxxx:9092"
topics => ["system-log"]
codec => json
}
}
......
Metadata
Metadata
Assignees
Labels
No labels