Fix logger in SdJournal, add optional filters and Docker journald tags

[marcogroppo]

Some fixes and additional features:

• Fix: SdJournal was still using the old logger, which has been removed.
• Fix: Check for nil-pointer in WriteBatch() before dereferencing it.
• Fix: IdleFunc() listener was logging too much because lastMetricTime was never updated.
• The user can now specify (optional) journald filters in the configuration file.
• Docker's journald log driver stores additional metadata tags (for example: CONTAINER_ID) with each log message. These have been added to the Record struct.

[toanctruong]

@marcogroppo Would you be able to compile the binary and make it available to download? I'm having some trouble building directly from your fork.

[marcogroppo]

@toanctruong If you are using Docker you can just docker run marcogroppo/systemd-cloud-watch. If you are not using Docker this is the exact sequence of commands I use to build the application: https://github.com/marcogroppo/systemd-cloud-watch/blob/master/Dockerfile

[fiunchinho]

hey @marcogroppo can you share a working systemd unit that uses the docker container of your fork to send journald logs to cloudwatch?
Thanks!!

[marcogroppo]

@fiunchinho sorry I'm not using it via systemd, I'm using it directly via docker-compose. For example: https://gist.github.com/marcogroppo/cd8b635f4639ccbf4a8f04bf2a98f2ae Please notice that you need to mount /var/log/journal inside the container (like I'm doing in the "volumes" section of the file). Ignore the filters if you just want to send everything; ignore the AWS_* environment variables and the ec2_instance_id if you're running on EC2. Aside from the logging driver the docker-compose configuration in the example should be equivalent to this docker command:

docker run -d -v ./conf/systemd-cloud-watch:/conf:ro -v /var/log/journal:/var/log/journal:ro -e AWS_ACCESS_KEY_ID='XXX' -e AWS_SECRET_ACCESS_KEY='YYY' marcogroppo/systemd-cloud-watch /conf/systemd-cloud-watch.config

[iskandar]

I extracted the binary like this:

$ docker run --rm --entrypoint "/bin/bash"  -v $(pwd):/foo -it marcogroppo/systemd-cloud-watch 
root@b232f9660574:/go/src# cp -v /go/bin/systemd-cloud-watch /foo

And now have the binary in my current directory:

$ file systemd-cloud-watch
systemd-cloud-watch: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, not stripped

[holyjak]

@marcogroppo can this be used to filter by unit name? I.e. if I add something like

filters = ["systemdUnit=cron.service"]

then I will only get logs from cron? Thanks!

PS: It would be nice to also update the README with the filters option and ideally also an example?

[marcogroppo]

@jakubholynet this should work (in theory):

filters = ["_SYSTEMD_UNIT=cron.service"]

but I have not tested it. The special fields are documented here:
https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html

[holyjak]

Thanks a lot, will try it out! lør. 13. jan. 2018 kl. 10:53 skrev Marco Groppo <notifications@github.com>:

…

@jakubholynet <https://github.com/jakubholynet> this should work (in theory): filters = ["_SYSTEMD_UNIT=cron.service"] but I have not tested it. The special fields are documented here: https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAmJPueBJeNygu_KNE-AoY4SXxaoxBihks5tKH0OgaJpZM4MjCFA> .

[analytically]

Is there an issue getting this merged?

[holyjak]

@RichardHightower hello, can this be merged and a new version released, please? Thank you!

[dokterbob]

poke

[donovan]

@RichardHightower @MammatusPlatypus ping