Safe Computing: Protecting the Flow of Execution

Most (though by no means all) exploitable security vulnerabilities in traditional client applications involve an attack disrupting the application’s flow of execution.  That is, instead of running the code that was intended, the CPU is directed somewhere else: perhaps just to a random memory address, in which case the application will usually crash, or perhaps to code provided by the attacker.

There are various techniques used to make it harder for an attacker to get his or her own code executed: for example, modern CPUs allow data pages to be marked not executable.  Unfortunately if an attacker can cause the CPU to jump to the wrong place, it is often possible to use legitimate code in malicious ways – this is known as a return-to-libc attack.  To counter this, a technique called address space layout randomization may be used, but even this is not always effective.

The most common cause of vulnerabilities involving the flow of execution is a buffer overflow on the stack, overwriting the return address of the affected subroutine with data of the attacker’s choosing.  Now, there are ways to reduce the risk of buffer overflows, and stronger techniques are possible if we adopt a blank-slate approach, but it is unlikely that the risk can be eliminated entirely.  Therefore, I suggest that the best approach would be to introduce separate code and data stacks, which I’m going to dub the dual-stack model.

In the simplest form of the dual-stack model, the code stack would contain fixed-size frames and the data stack variable-sized frames.  Local variables and call parameters would be on the data stack; the code stack would hold the return address and a pointer to the data stack frame associated with the code at the return address.  Ideally, the code stack would be located in a separate address space, or at least at an unpredictable address.

With a dual-stack model, buffer overflows on the data stack might corrupt the content of other local variables, including in parent procedures, but could not directly cause a flow-of-execution failure.

The other main situation in which the flow of execution is threatened is where a function pointer, interface, or polymorphic class is used.  From the CPU’s point of view these all boil down to jumping to a procedure whose address is contained in a variable rather than being fixed.  If the contents of the variable in question have been manipulated by an attacker, the correct flow of execution will be disrupted.

I don’t see any comprehensive way of preventing this, but the impact can be considerably mitigated if the code (or the CPU) can verify that the target address is, if not actually the correct procedure, at least a legitimate one.  This would require the variable to point not directly to the code, but into a trusted table containing a list of candidate procedures.  (Obviously this table would have to be located in read-only memory or nothing would be gained!)

As a further refinement, separate tables could be provided for different groups of function pointer calls.  In C, each table would correspond to a function signature; in object-oriented code, to a class or interface.  Only those procedures which are actually called in this way would need to be included in these tables.

It should be noted that these tables would have to be built at link time, which would be problematic for dynamic libraries.  I do not see this as a major issue, because one of the first things I’d do in designing a Safe Computing environment would be to eliminate dynamic libraries anyway – they cause reliability problems, and most of the reasons for using them are no longer compelling.  I’ll talk about this another time.



3 Responses to “Safe Computing: Protecting the Flow of Execution”

  1. harryjohnston Says:

    See also

    for an analysis of the current support for DEP/ASLR in popular third-party Windows applications.

  2. Anil Says:

    If you’re looking to redesign how computers work, then there are many fun tricks you can play. Your first solution is akin to having the stack grow upwards instead of downwards, which would severly limit any type of buffer overrun, vanilla or return-to-libc esque. It won’t change because of compatibility. The same benefits that allow you to inline x86 assembly code essentially means that no compiler is going to risk messing with the implicit memory layout because some program somewhere is going to break. Same thing with your second method, which is essentially hard-coded vtables. It would break too many things. COM goes right out the window with this change, for example.

    • harryjohnston Says:

      I’m mostly interesting here in writing about ideas for a hypothetical new computing platform, so backwards compatibility isn’t mandatory. 🙂

      However, in this particular case, I don’t think the dual-stack proposal introduces a significantly greater risk of incompatibility than DEP or ASLR. Did you have any particular issue in mind? In the context of a compiler targeting an existing operating system, it would of course have to be opt-in, if only because it would limit the target executable to versions of the OS with dual-stack support.

      Linking single-stack and dual-stack code together might be a bit tricky, but I expect something could be worked out.

      (I had considered mentioning the idea of having the stack grow upwards, but decided it would cause unnecessary confusion! It isn’t really the same idea, although I admit it is a simpler approach that would be very nearly as effective.)

      It’s pretty much the same situation with the dispatch-table idea. I don’t see any problem with a compiler offering support for this, provided the programmer is aware that it shouldn’t be turned on if COM support is required, or if the code will be calling through a function pointer into an external library. (It would be more elegant, though, to build support directly into the CPU.)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: