I’ve made some quick notes about this before, but I actually managed to forget the correct flags to make everything go zoom last night while doing some testing, so I’m writing a quick walkthrough for properly building in JIT support into OpenResty’s ngx.re.* calls.

Get the source OpenResty tarball:

The default PCRE libraries that I’ve seen on Debian and Redhat distros don’t have JIT support compiled in, so we’ll need to get our own copy and integrate it into the OpenResty build process. I’m using the following version:

We don’t need to build the PCRE package independently and install it system-wide (though you can); instead, we configure OpenResty to use our new PCRE source and integrate that into the finished build. We’ll also add the debug flag to double check our work once we’re done:

The output of the configuration should include a line similar to the following:

If the separate PCRE source wasn’t properly included, you’ll see configure indicating that the build will use the system PCRE library. Now we just need to build and install the binary and test it out:

We can use a few simple regex matches in a content_by_lua handler to confirm that OpenResty is caching the pattern and properly JIT compiling:

Because ngx.say will flush the output headers, we run both of our test patterns first, then output the results (it makes reading the debug log a little easier). In the debug log we can track how our build responded to the match call the first time after loading:

We see that the regex pattern is compiled and cached at the worker process level, which requires some additional memory allocation. We can also verify that we’re using JIT compilation by the log entry pcre JIT compiling result: 1 (the result would be 0 if we didn’t specify the ‘j’ flag, or our library didn’t support JIT compiling). The next time our Lua filter runs, we see much less debug output:

The worker process has the patterns cached, so it doesn’t need to compile them; this saves the overhead of memory (de)allocation on every request. JIT compiling regex patterns and caching them in-worker can be a huge performance boost, especially when expensive, complex patterns are used in a tight loop. One caveat to note is that a finite number of patterns can be cached, as defined by lua_regex_cache_max_entries; see the documentation for more details on tuning this and avoiding overflows (essentially, don’t cache dynamically-generated patterns that could have a large number of permutations, as eventually you won’t be able to cache any new patterns).