KRANK — hyper converged real-time OS in user-space Linux

The idea is to make real-time OS sitting in user-mode of Linux host. The hardware such as cpu cores, ram, network and disk is taken away from Linux exclusively.

There is a bridge, a Linux kernel driver that talks to KRANK and allows KRANK to access Linux VFS, IPC, and/or network stack etc. On KRANK side this looks like paravirtualized drivers that are slower then the monopolized direct equivalents.

Programs are loaded as ELF with KRANK’s custom dynamic linker “krank-ld” to load it up and verifiy that it doesn’t:

• use libc other then a limited Musl fork for KRANK (w/o syscall wrapper and POSIX API)
• - has no “int” or syscall related instructions in .text section
• - all sections are properly protected and KRANK doesn’t allow changing EXECUTE protections.
• - JIT is supported through a special API of KRANK, where a JIT doesn’t get to make code executable. w/o same checks for allowed instructions on the memory. Also on “open to codegen” protections are bounced to READ|WRITE on “close for codegen” validated and bounced to EXECUTE. This kills the power of lazy stub hot-patching though…

More choices:

• kernel is a user-mode binary using DPDK to monopolize a NIC
• - SPDK is an option for storage
• - single (user-mode) address space for all aplications
• - kernel runs applications as threads (process is a Linux thread and to reduce confusion called isolate) and there is only logical protection between KRANK isolates
• - isolates talk via cheap zero-copy IPC in KRANK, same interface also works with Linux apps over bridge and requires a round-trip through kernel (kdbus, Linux kernel DBUS)
• - Linux processes are “foreing” isolates, KRANK can manipulate them as well via round-trip to Linux kernel
• - KRANK syscalll API is defined in Protobuf 3 over direct memory mapped queues: request queue and response queue. Multiple queues are allowed, with 1 per CPU being a typical choice.

The idea is that we use Linux for management tasks but KRANK for a few (1 typically) high-perf programs to run on a node.