Most (though by no means all) exploitable security vulnerabilities in traditional client applications involve an attack disrupting the application’s flow of execution. That is, instead of running the code that was intended, the CPU is directed somewhere else: perhaps just to a random memory address, in which case the application will usually crash, or perhaps to code provided by the attacker.
There are various techniques used to make it harder for an attacker to get his or her own code executed: for example, modern CPUs allow data pages to be marked not executable. Unfortunately if an attacker can cause the CPU to jump to the wrong place, it is often possible to use legitimate code in malicious ways – this is known as a return-to-libc attack. To counter this, a technique called address space layout randomization may be used, but even this is not always effective.
The most common cause of vulnerabilities involving the flow of execution is a buffer overflow on the stack, overwriting the return address of the affected subroutine with data of the attacker’s choosing. Now, there are ways to reduce the risk of buffer overflows, and stronger techniques are possible if we adopt a blank-slate approach, but it is unlikely that the risk can be eliminated entirely. Therefore, I suggest that the best approach would be to introduce separate code and data stacks, which I’m going to dub the dual-stack model.
In the simplest form of the dual-stack model, the code stack would contain fixed-size frames and the data stack variable-sized frames. Local variables and call parameters would be on the data stack; the code stack would hold the return address and a pointer to the data stack frame associated with the code at the return address. Ideally, the code stack would be located in a separate address space, or at least at an unpredictable address.
With a dual-stack model, buffer overflows on the data stack might corrupt the content of other local variables, including in parent procedures, but could not directly cause a flow-of-execution failure.
The other main situation in which the flow of execution is threatened is where a function pointer, interface, or polymorphic class is used. From the CPU’s point of view these all boil down to jumping to a procedure whose address is contained in a variable rather than being fixed. If the contents of the variable in question have been manipulated by an attacker, the correct flow of execution will be disrupted.
I don’t see any comprehensive way of preventing this, but the impact can be considerably mitigated if the code (or the CPU) can verify that the target address is, if not actually the correct procedure, at least a legitimate one. This would require the variable to point not directly to the code, but into a trusted table containing a list of candidate procedures. (Obviously this table would have to be located in read-only memory or nothing would be gained!)
As a further refinement, separate tables could be provided for different groups of function pointer calls. In C, each table would correspond to a function signature; in object-oriented code, to a class or interface. Only those procedures which are actually called in this way would need to be included in these tables.
It should be noted that these tables would have to be built at link time, which would be problematic for dynamic libraries. I do not see this as a major issue, because one of the first things I’d do in designing a Safe Computing environment would be to eliminate dynamic libraries anyway – they cause reliability problems, and most of the reasons for using them are no longer compelling. I’ll talk about this another time.