Load Balanced DNS with dnsdist

In recent weeks I’ve found the need to configure and deploy a proper load balancing solution for an authoritative DNS cluster. Now for most solutions (up to a certain scale, and you’d know if you were there) a single-purpose authoritative DNS resolver doesn’t really need a balancing frontend; you can reasonably expect a decent-sized box running a modern kernel to handle several hundred thousand UDP packets per second, with a minimal amount of complimentary TCP traffic. Putting a frontend load balancing tier in front of an authoritative DNS cluster is really only necessary when either hardware redundancy or significant traffic shaping is a requirement, or the generation of authoritative data is expensive and needs to be horizontally scalable. I found myself needing to satisfy a few of these conditions, and have had a wonderful time playing and poking at a purpose-built FOSS DNS load balancing solution in dnsdist.

Continue reading

Enabling PCRE JIT in OpenResty on CentOS

I’m quickly finding that openresty is an excellent stack, bundling a large number of nginx modules and Lua functionality. One of the most useful features I’ve found so far is the nginx Lua API’s ability to perform PCRE matches (Lua offers string.find, using a search syntax that is similar to PCRE, but lacks robustness). Performance with PCRE can be increased by using PCRE JIT (just-in-time) compilation; this is required to be present within the underlying system’s PCRE package. JIT was introduce in PCRE 8.21, and, go figure, the upstream package on CentOS is PCRE 7.8.

Continue reading

ZOMG REMOTE SHELL EXPLOIT!!!! … Or, Not

A new post to the Full Disclosure mailing list today detailed a remote code exploit vulnerability within all available versions of the Nagios Remote Plugin Executor (NRPE), and provided a PoC that included gaining a reverse shell on the target server. Note that this reported exploit relies on a separate (but similar) vulnerability report in CVE-2013-1362. In today’s post, Dawid Golunski properly identifies a weakness within the NRPE argument sanitation that could allow a client to execute arbitrary commands on the remote host within the context of the NRPE user. At first this seems like a pretty serious vulnerability, but further analysis shows that successful exploitation hinges on a series of bad practices not related to NRPE code itself.

Continue reading

boto-rsync – Limitations and Workarounds

boto-rsync is a great tool for interacting with object storage systems like S3, but it’s not without limitation. We all know about the 5GB limit for a single PUT, which isn’t a problem for clients that can handle multipart upload. Sadly, boto-rsync doesn’t handle that, and until someone patches it, we need a way to break up large objects. This can crudely be done with split:

This disadvantage to this is that retrievals need to manually be catted together, which obviously isn’t always a good solution.

boto-rsync’s other weakness is in handling UTF8 filenames. Improperly-encoded filenames will throw a 400 Bad Request and cause the script to choke and die, rather than gracefully skipping the failing file and moving on. Re-encoding files with proper UTF8 fixes this:

Not pretty, but it works. Note that directories need to be checked and renamed first before handling files specifically.

UPDATE – These issues have both been addressed in https://github.com/dreamhost/boto_rsync

Coordinated Spam Effort

Yesterday we saw a large number of infected domains sending a massive spam run in what appeared to be a coordinated effort. Signs of a large-scale targeted effort included:

  • Sharp upticks of outbound spam requests occurring at once across multiple domains
  • The same script being used across disparate victims
  • A large sampling of IPs submitting malicious requests (a sample has been submitted to the ISC)

Continue reading