An efficient, general-purpose framework
for dynamic software updating in C
Kitsune is a simple framework that C programmers can use to write software that can be updated on the fly. Such dynamic software updates preserve in-memory state and active connections, thus avoiding the disruption of shutdown-and-restart. Updates take place quickly, and require no special system support (like redundancy).
We have used Kitsune to retrofit a half-dozen open-source programs to support dynamic updating. These include Snort, Tor, Redis, Memcached, Icecast, and vsftpd. These modified versions are available on our Github Page. Code changes to these programs were small, and experiments show that Kitsune updating support adds essentially no performance overhead.
Why is dynamic sofrware updating (DSU) using Kitsune valuable?
DSU is valuable for services with persistent uptime requirements, or for services of which a disconnection or long interruption would be disruptive to users.
How do I get started using Kitsune?
The best way to learn Kitsune is to follow the tutorial located in the /doc directory of the repository. This will walk you through the practical aspects of using Kitsune and introduce you to the Kitsune runtime APIs and xfgen language. After completing the tutorial, you should be able to use Kitsune to add dynamic updating support to other projects.
What benefits does DSU using Kitsune have over other updating strategies, like having redundant instances or using checkpoint and restart?
Redundancy-based updates require extra hardware resources, which may not otherwise be needed, and can require significant changes to application code. Checkpointing to and restoring from disk can introduce a long delays (tens of seconds to minutes). Kitsune requires no extra hardware resources (except perhaps a modest increase in memory) and performs updates in millseconds.
How much programmer effort is required to use Kitsune?
Adding support for dynamic updating using Kitsune is essentially adding a feature to your application. Depending on the programmer's familiarity with the application, and the application's own degree of complexity, adding support for DSU using Kitsune can range from mild to difficult.
In our own experience, virtually all of the work required to add Kitsune integration with an existing project occurs in the first version. After this point, the Kitsune support code becomes a separate concern. This code changes very little over time.
In all cases, the lines of code added to our benchmark programs to support Kitsune accounts for less than 1% of the total code.
What programs can be updated with Kitsune?
Potentially, any C program can be updated with Kitsune. Kitsune doesn't change process ID's on an update, so any resources allocated by the operating system will persist after an update.
At face value, this does not seem to differentiate Kitsune from other C updating systems, like Ginseng or the more recent UpStare. However, both of those systems require target programs to pass a static analysis that may be over-conservative. They also perform significant source-to-source compilation of the original program, which adds overhead and can inhibit compiler optimizations. Kitsune is much gentler on the host application, requiring no static analysis, and very little (potentially no) source-to-source rewriting.
Can Kitsune update multithreaded programs?
Yes. Half of Kitsune's benchmark programs are multithreaded: Memcached, Icecast, and Snort.
How much overhead does Kitsune add to normal execution?
In our experience, Kitsune introduces essentially zero overhead while your program is executing normally.
How long does it take to update a program from one version to the next?
In general, updates take place very quickly. In our benchmarks, the time from when a program first signals that an update is required, to the time the program transfers control into Kitsune, to the time when the new version of the program has taken over and finished initializing all state, is generally less than 200ms, and less than 500ms in the worst cases.
In general, the time for a dynamic update depends on the structure of the program, and the nature of the change. Some programs are very easy to update with Kitsune because they either have little in-flight state, or have data structures that are quick for Kitsune to traverse. Memcached, for example, updates quickly regardless of how full its caches are. Update times for Redis scale nearly linearly with database keys, on the opposite end of the spectrum. That said, a small change to Redis makes its updates essentially constant-time too. See our paper for details.
Can Kitsune update programs that implement distributed systems?
Kitsune can update programs that do not introduce a backwards-incompatible protocol change; e.g., we have updated Tor, a distributed overlay router. Additional support is required if a node's protocol chnages due to an update; e.g., to at least present the illusion of updating the entire distributed system at once. This is future work.
Can Kitsune update C++ programs?
With some more development effort, it should, but at this point, no.
Can Kitsune fix memory leaks, stack or heap corruption, or perform other data structure repair?
Kitsune provides no native support for these operations. Programmers may be able to hand-write code that fixes an application's state.
Can Kitsune update itself?
The Kitsune "driver" program (about 100 lines of C code) cannot change, but the Kitsune runtime library can. It is statically linked to a target program, and two copies of the runtime library exist when both versions of a program are loaded in memory (to transfer existing state). Kitsune's internal state is transferred using the same mechanisms available to target programs.
Could Kitsune update across ABI versions?
Probably not. For Kitsune to update a program, the old and new versions (compiled to shared objects) must be loaded into the same process image. This is probably impossible across ABI versions.
Can Kitsune update shared libraries?
Yes, in some cases, but this feature is not directly supported. See our paper for details.
What license is Kitsune released under?
Kitsune is licensed under the LGPLv3 license.