Exporting Failing Consul Checks to Prometheus

This weekend I hacked on a quick project to teach myself a bit more Go and maybe do something useful. I wrote a quick daemon to scrape failed Consul checks and make the data available for Prometheus scrape (this could tie in nicely with Alertmanager).

So, I played around a bit and made a simple tool with some nice CLI arg parsing via kingpin:

And a nice little struct to hold the Prometheus metrics and Consul client handler:

Of course, what good is a struct is without a constructor?

Our Exporter even implements prometheus.Collector!

And when it comes to the actual collection implementation, Consul’s Client provides wonderful retry logic and transport handling, all abstracted away, so we can focus on clean, simple code:

And now we need to tie it all together!

… of course, I put all this together and realize that the official Consul Prometheus exporter can do all this, and a lot more.

But no matter! This was a fun learning experience. Coming from Perl/Bash/OpenResty where everything is super concrete and there’s no baked in logic for grace (retries, keepalive, etc), writing this felt wonderfully simple, if not a bit odd- if the Consul server goes away and magically reappears, the code above gracefully handles it. Normally, writing sane OpenResty code to handle that needs a few day’s worth of testable boilerplate.

I also feel like I’m getting comfortable enough with Go that I can use godocs as meaningful learning material and not be completely overwhelmed, which made hacking on this super pleasant – I spent all my time in reference material and studying implementation a bit, not scratching my head over syntax and idiom.

Leave a Reply

Your email address will not be published. Required fields are marked *