A major theme of our research has been the ``holy grail'' among operating systems researchers -- efficient cross-domain communication. This is targeted at making it feasible for complex software systems to be structured as independent modules for reliability without sacrificing performance. Examples of the need for fast communication between protected modules include small-kernel operating systems and closely integrated multi-vendor applications such as Microsoft OLE or Apple OpenDoc. Our research involved four separate efforts over as many years, culminating in an aggregate reduction of at least two orders of magnitude in RPC performance over previous approaches.
Fast RPC using hardware protection
We have developed two approaches to the problem
of reducing the cost of cross-domain RPC. The first was
called Lightweight Remote Procedure Call (LRPC).
This work showed that by simplifying every aspect of an RPC -- control
transfer, data transfer, linkage, and
stubs, it was possible to achieve a factor of three performance improvement
over the best traditional approaches based on independent threads exchanging
messages. The techniques demonstrated in LRPC are now in use
in Microsoft's latest operating system, Windows NT.
The second approach was called User-Level Remote Procedure Call (URPC). URPC moves responsibility for cross-address space communication from the kernel into each application's address space; in doing so, URPC exploits the advantages of multiprocessors to achieve another factor of two improvement over LRPC. The techniques demonstrated in URPC have more recently been used in building fast RPC implementations over local area networks, cited in SIGCOMM and SOSP.
Protection within an address space
We have also developed two techniques for providing for protection
between modules in the same address space.
The traditional approach to isolating faults between cooperating modules is
to place each in its own address space, with protection implemented by
hardware page tables. For tightly-coupled modules, however, this can incur
prohibitive context switch overhead for cross-domain communication,
even using LRPC or URPC techniques.
First, we developed a method that uses anonymity instead of hardware page tables for protection. Logically independent memory segments are placed at random locations in the same address space and protection domain. With the onset of 64-bit virtual addresses, it is unlikely that a process will be able to locate any other segment by accidental or malicious memory probes; it impossible to corrupt a segment without knowing its location. Knowing the location of a memory segment is thus equivalent to having a capability to modify the segment. The benefit is that a cross-domain RPC need not involve a hardware context switch.
More recently, we developed a different approach to providing efficient, language-independent, software-based fault isolation. In our approach, we enforce protection in software, by modifying the object code of a distrusted module so that it can never write or branch to an illegal address outside its domain. Once we have done this, we can load distrusted domains into the same address space, without sacrificing protection; communication between distrusted modules can then be implemented as little more than a normal procedure call. Our approach poses a tradeoff relative to hardware fault isolation: substantially faster communication between fault domains, at a cost of slightly increased execution time within a domain. This tradeoff benefits frequently communicating modules.