RUMPUSER(3) NetBSD Library Functions Manual RUMPUSER(3)
NAME
rumpuser -- rump kernel hypercall interface
LIBRARY
rump User Library (librumpuser, -lrumpuser)
SYNOPSIS
#include <rump/rumpuser.h>
DESCRIPTION
The rumpuser hypercall interfaces allow a rump kernel to access host resources. A hypervisor implementation must implement the routines described in this document to allow a rump kernel to run on the host. The implementation included in NetBSD is for POSIX-like hosts (*BSD, Linux, etc.). This document is divided into sections based on the func- tionality group of each hypercall. Since the hypercall interface is a C function interface, both the rump kernel and the hypervisor must conform to the same ABI. The interface itself attempts to assume as little as possible from the type systems, and for example off_t is passed as int64_t and enums are passed as ints. It is recommended that the hypervisor converts these to the native types before starting to process the hypercall, for example by assigning the ints back to enums.
UPCALLS AND RUMP KERNEL CONTEXT
A hypercall is always entered with the calling thread scheduled in the rump kernel. In case the hypercall intends to block while waiting for an event, the hypervisor must first release the rump kernel scheduling con- text. In other words, the rump kernel context is a resource and holding on to it while waiting for a rump kernel event/resource may lead to a deadlock. Even when there is no possibility of deadlock in the strict sense of the term, holding on to the rump kernel context while performing a slow hypercall such as reading a device will prevent other threads (including the clock interrupt) from using that rump kernel context. Releasing the context is done by calling the hyp_backend_unschedule() upcall which the hypervisor received from rump kernel as a parameter for rumpuser_init(). Before a hypercall returns back to the rump kernel, the returning thread must carry a rump kernel context. In case the hypercall unscheduled itself, it must reschedule itself by calling hyp_backend_schedule().
HYPERCALL INTERFACES
Initialization int rumpuser_init(int version, struct rump_hyperup *hyp) Initialize the hypervisor. version hypercall interface version number that the kernel expects to be used. In case the hypervisor cannot pro- vide an exact match, this routine must return a non-zero value. hyp pointer to a set of upcalls the hypervisor can make into the rump kernel Memory allocation int rumpuser_malloc(size_t len, int alignment, void **memp) len amount of memory to allocate alignment size the returned memory must be aligned to. For exam- ple, if the value passed is 4096, the returned memory must be aligned to a 4k boundary. memp return pointer for allocated memory void rumpuser_free(void *mem, size_t len) mem memory to free len length of allocation. This is always equal to the amount the caller requested from the rumpuser_malloc() which returned mem. Files and I/O int rumpuser_open(const char *name, int mode, int *fdp) Open name for I/O and associate a file descriptor with it. Notably, there needs to be no mapping between name and the host's file system namespace. For example, it is possible to associate the file descriptor with device I/O registers for special values of name. name the identifier of the file to open for I/O mode combination of the following: RUMPUSER_OPEN_RDONLY open only for reading RUMPUSER_OPEN_WRONLY open only for writing RUMPUSER_OPEN_RDWR open for reading and writing RUMPUSER_OPEN_CREATE do not treat missing name as an error RUMPUSER_OPEN_EXCL combined with RUMPUSER_OPEN_CREATE, flag an error if name already exists RUMPUSER_OPEN_BIO the caller will use this file for block I/O, usually used in con- junction with accessing file sys- tem media. The hypervisor should treat this flag as advisory and possibly enable some optimiza- tions for *fdp based on it. Notably, the permissions of the created file are left up to the hypervisor implementation. fdp An integer value denoting the open file is returned here. int rumpuser_close(int fd) Close a previously opened file descriptor. int rumpuser_getfileinfo(const char *name, uint64_t *size, int *type) name file for which information is returned. The namespace is equal to that of rumpuser_open(). size If non-NULL, size of the file is returned here. type If non-NULL, type of the file is returned here. The options are RUMPUSER_FT_DIR, RUMPUSER_FT_REG, RUMPUSER_FT_BLK, RUMPUSER_FT_CHR, or RUMPUSER_FT_OTHER for directory, regular file, block device, character device or unknown, respectively. void rumpuser_bio(int fd, int op, void *data, size_t dlen, int64_t off, rump_biodone_fn biodone, void *donearg) Initiate block I/O and return immediately. fd perform I/O on this file descriptor. The file descrip- tor must have been opened with RUMPUSER_OPEN_BIO. op Transfer data from the file descriptor with RUMPUSER_BIO_READ and transfer data to the file descrip- tor with RUMPUSER_BIO_WRITE. Unless RUMPUSER_BIO_SYNC is specified, the hypervisor may cache a write instead of committing it to permanent storage. data memory address to transfer data to/from dlen length of I/O. The length is guaranteed to be a multi- ple of 512. off offset into fd where I/O is performed biodone To be called when the I/O is complete. Accessing data is not legal after the call is made. donearg opaque arg that must be passed to biodone. int rumpuser_iovread(int fd, struct rumpuser_iovec *ruiov, size_t iovlen, int64_t off, size_t *retv) int rumpuser_iovwrite(int fd, struct rumpuser_iovec *ruiov, size_t iovlen, int64_t off, size_t *retv) These routines perform scatter-gather I/O which is not block I/O by nature and therefore cannot be handled by rumpuser_bio(). fd file descriptor to perform I/O on ruiov an array of I/O descriptors. It is defined as follows: struct rumpuser_iovec { void *iov_base; size_t iov_len; }; iovlen number of elements in ruiov off offset of fd to perform I/O on. This can either be a non-negative value or RUMPUSER_IOV_NOSEEK. The latter denotes that no attempt to change the underlying objects offset should be made. Using both types of offsets on a single instance of fd results in undefined behavior. retv number of bytes successfully transferred is returned here int rumpuser_syncfd(int fd, int flags, uint64_t start, uint64_t len) Synchronizes fd with respect to backing storage. The other arguments are: flags controls how synchronization happens. It must contain one of the following: RUMPUSER_SYNCFD_READ Make sure that the next read sees writes from all other parties. This is useful for example in the case that fd represents memory to write a DMA read is being performed. RUMPUSER_SYNCFD_WRITE Flush cached writes. The following additional parameters may be passed in flags: RUMPUSER_SYNCFD_BARRIER Issue a barrier. Outstanding I/O operations which were started before the barrier complete before any operations after the barrier are per- formed. RUMPUSER_SYNCFD_SYNC Wait for the synchronization operation to fully complete before returning. For exam- ple, this could mean that the data to be written to a disk has hit either the disk or non-volatile memory. start offset into the object. len the number of bytes to synchronize. The value 0 denotes until the end of the object. Clocks The hypervisor should support two clocks, one for wall time and one for monotonically increasing time, the latter of which may be based on some arbitrary time (e.g. system boot time). If this is not possible, the hypervisor must make a reasonable effort to retain semantics. int rumpuser_clock_gettime(int enum_rumpclock, int64_t *sec, long *nsec) enum_rumpclock specifies the clock type. In case of RUMPUSER_CLOCK_RELWALL the wall time should be returned. In case of RUMPUSER_CLOCK_ABSMONO the time of a mono- tonic clock should be returned. sec return value for seconds nsec return value for nanoseconds int rumpuser_clock_sleep(int enum_rumpclock, int64_t sec, long nsec) enum_rumpclock In case of RUMPUSER_CLOCK_RELWALL, the sleep should last at least as long as specified. In case of RUMPUSER_CLOCK_ABSMONO, the sleep should last until the hypervisor monotonic clock hits the specified absolute time. sec sleep duration, seconds. exact semantics depend on clk. nsec sleep duration, nanoseconds. exact semantics depend on clk. Parameter retrieval int rumpuser_getparam(const char *name, void *buf, size_t buflen) Retrieve a configuration parameter from the hypervisor. It is up to the hypervisor to decide how the parameters can be set. name name of the parameter. If the name starts with an underscore, it means a mandatory parameter. The manda- tory parameters are RUMPUSER_PARAM_NCPU which specifies the amount of virtual CPUs bootstrapped by the rump ker- nel and RUMPUSER_PARAM_HOSTNAME which returns a prefer- ably unique instance name for the rump kernel. buf buffer to return the data in as a string buflen length of buffer Termination void rumpuser_exit(int value) Terminate the rump kernel with exit value value. If value is RUMPUSER_PANIC the hypervisor should attempt to provide something akin to a core dump. Console output Console output is divided into two routines: a per-character one and printf-like one. The former is used e.g. by the rump kernel's internal printf routine. The latter can be used for direct debug prints e.g. very early on in the rump kernel's bootstrap or when using the in-kernel rou- tine causes too much skew in the debug print results (the hypercall runs outside of the rump kernel and therefore does not cause any locking or scheduling events inside the rump kernel). void rumpuser_putchar(int ch) Output ch on the console. void rumpuser_dprintf(const char *fmt, ...) Do output based on printf-like parameters. Signals A rump kernel should be able to send signals to client programs due to some standard interfaces including signal delivery in their specifica- tions. Examples of these interfaces include setitimer(2) and write(2). The rumpuser_kill() function advises the hypercall implementation to raise a signal for the process containing the rump kernel. int rumpuser_kill(int64_t pid, int sig) pid The pid of the rump kernel process that the signal is directed to. This value may be used as the hypervisor as a hint on how to deliver the signal. The value RUMPUSER_PID_SELF may also be specified to indicate no hint. This value will be removed in a future version of the hypercall interface. sig Number of signal to raise. The value is in NetBSD sig- nal number namespace. In case the host has a native representation for signals, the value should be trans- lated before the signal is raised. In case there is no mapping between sig and native signals (if any), the behavior is implementation-defined. A rump kernel will ignore the return value of this hypercall. The only implication of not implementing rumpuser_kill() is that some application programs may not experience expected behavior for standard interfaces. As an aside,the rump_sp(7) protocol provides equivalent functionality for remote clients. Random pool int rumpuser_getrandom(void *buf, size_t buflen, int flags, size_t *retp) buf buffer that the randomness is written to buflen number of bytes of randomness requested flags The value 0 or a combination of RUMPUSER_RANDOM_HARD (return true randomness instead of something from a PRNG) and RUMPUSER_RANDOM_NOWAIT (do not block in case the requested amount of bytes is not available). retp The number of random bytes written into buf. Threads int rumpuser_thread_create(void *(*fun)(void *), void *arg, const char *thrname, int mustjoin, int priority, int cpuidx, void **cookie) Create a schedulable host thread context. The rump kernel will call this interface when it creates a kernel thread. The scheduling policy for the new thread is defined by the hypervisor. In case the hypervisor wants to optimize the scheduling of the threads, it can perform heuristics on the thrname, priority and cpuidx parameters. fun function that the new thread must call. This call will never return. arg argument to be passed to fun thrname Name of the new thread. mustjoin If 1, the thread will be waited for by rumpuser_thread_join() when the thread exits. priority The priority that the kernel requested the thread to be created at. Higher values mean higher priority. The exact kernel semantics for each value are not available through this interface. cpuidx The index of the virtual CPU that the thread is bound to, or -1 if the thread is not bound. The mapping between the virtual CPUs and physical CPUs, if any, is hypervisor implementation specific. cookie In case mustjoin is set, the value returned in cookie will be passed to rumpuser_thread_join(). void rumpuser_thread_exit(void) Called when a thread created with rumpuser_thread_create() exits. int rumpuser_thread_join(void *cookie) Wait for a joinable thread to exit. The cookie matches the value from rumpuser_thread_create(). void rumpuser_curlwpop(int enum_rumplwpop, struct lwp *l) Manipulate the hypervisor's thread context database. The possible opera- tions are create, destroy, and set as specified by enum_rumplwpop: RUMPUSER_LWP_CREATE Inform the hypervisor that l is now a valid thread context which may be set. A currently valid value of l may not be specified. This operation is informational and does not mandate any action from the hypervisor. RUMPUSER_LWP_DESTROY Inform the hypervisor that l is no longer a valid thread context. This means that it may no longer be set as the current context. A currently set context or an invalid one may not be destroyed. This operation is informational and does not man- date any action from the hypervisor. RUMPUSER_LWP_SET Set l as the current host thread's rump kernel context. A previous context must not exist. RUMPUSER_LWP_CLEAR Clear the context previous set by RUMPUSER_LWP_SET. The value passed in l is the current thread and is never NULL. struct lwp * rumpuser_curlwp(void) Retrieve the rump kernel thread context associated with the current host thread, as set by rumpuser_curlwpop(). This routine may be called when a context is not set and the routine must return NULL in that case. This interface is expected to be called very often. Any optimizations per- taining to the execution speed of this routine should be done in rumpuser_curlwpop(). void rumpuser_seterrno(int errno) Set an errno value in the calling thread's TLS. Note: this is used only if rump kernel clients make rump system calls. Mutexes, rwlocks and condition variables The locking interfaces have standard semantics, so we will not discuss each one in detail. The data types struct rumpuser_mtx, struct rumpuser_rw and struct rumpuser_cv used by these interfaces are opaque to the rump kernel, i.e. the hypervisor has complete freedom over them. Most of these interfaces will (and must) relinquish the rump kernel CPU context in case they block (or intend to block). The exceptions are the "nowrap" variants of the interfaces which may not relinquish rump kernel context. void rumpuser_mutex_init(struct rumpuser_mtx **mtxp, int flags) void rumpuser_mutex_enter(struct rumpuser_mtx *mtx) void rumpuser_mutex_enter_nowrap(struct rumpuser_mtx *mtx) int rumpuser_mutex_tryenter(struct rumpuser_mtx *mtx) void rumpuser_mutex_exit(struct rumpuser_mtx *mtx) void rumpuser_mutex_destroy(struct rumpuser_mtx *mtx) void rumpuser_mutex_owner(struct rumpuser_mtx *mtx, struct lwp **lp) Mutexes provide mutually exclusive locking. The flags, of which at least one must be given, are as follows: RUMPUSER_MTX_SPIN Create a spin mutex. Locking this type of mutex must not relinquish rump kernel context even when rumpuser_mutex_enter() is used. RUMPUSER_MTX_KMUTEX The mutex must track and be able to return the rump kernel thread that owns the mutex (if any). If this flag is not specified, rumpuser_mutex_owner() will never be called for that particular mutex. void rumpuser_rw_init(struct rumpuser_rw **rwp) void rumpuser_rw_enter(int enum_rumprwlock, struct rumpuser_rw *rw) int rumpuser_rw_tryenter(int enum_rumprwlock, struct rumpuser_rw *rw) int rumpuser_rw_tryupgrade(struct rumpuser_rw *rw) void rumpuser_rw_downgrade(struct rumpuser_rw *rw) void rumpuser_rw_exit(struct rumpuser_rw *rw) void rumpuser_rw_destroy(struct rumpuser_rw *rw) void rumpuser_rw_held(int enum_rumprwlock, struct rumpuser_rw *rw, int *heldp) Read/write locks provide either shared or exclusive locking. The possi- ble values for lk are RUMPUSER_RW_READER and RUMPUSER_RW_WRITER. Upgrad- ing means trying to migrate from an already owned shared lock to an exclusive lock and downgrading means migrating from an already owned exclusive lock to a shared lock. void rumpuser_cv_init(struct rumpuser_cv **cvp) void rumpuser_cv_destroy(struct rumpuser_cv *cv) void rumpuser_cv_wait(struct rumpuser_cv *cv, struct rumpuser_mtx *mtx) void rumpuser_cv_wait_nowrap(struct rumpuser_cv *cv, struct rumpuser_mtx *mtx) int rumpuser_cv_timedwait(struct rumpuser_cv *cv, struct rumpuser_mtx *mtx, int64_t sec, int64_t nsec) void rumpuser_cv_signal(struct rumpuser_cv *cv) void rumpuser_cv_broadcast(struct rumpuser_cv *cv) void rumpuser_cv_has_waiters(struct rumpuser_cv *cv, int *waitersp) Condition variables wait for an event. The mtx interlock eliminates a race between checking the predicate and sleeping on the condition vari- able; the mutex should be released for the duration of the sleep in the normal atomic manner. The timedwait variant takes a specifier indicating a relative sleep duration after which the routine will return with ETIMEDOUT. If a timedwait is signaled before the timeout expires, the routine will return 0. The order in which the hypervisor reacquires the rump kernel context and interlock mutex before returning into the rump kernel is as follows. In case the interlock mutex was initialized with both RUMPUSER_MTX_SPIN and RUMPUSER_MTX_KMUTEX, the rump kernel context is scheduled before the mutex is reacquired. In case of a purely RUMPUSER_MTX_SPIN mutex, the mutex is acquired first. In the final case the order is implementation- defined.
RETURN VALUES
All routines which return an integer return an errno value. The hypervi- sor must translate the value to the the native errno namespace used by the rump kernel. Routines which do not return an integer may never fail.
SEE ALSO
rump(3) Antti Kantee, "Flexible Operating System Internals: The Design and Implementation of the Anykernel and Rump Kernels", Aalto University Doctoral Dissertations, 2012, Section 2.3.2: The Hypercall Interface. For a list of all known implementations of the rumpuser interface, see http://wiki.rumpkernel.org/Platforms.
HISTORY
The rump kernel hypercall API was first introduced in NetBSD 5.0. The API described above first appeared in NetBSD 7.0. NetBSD 8.3 August 24, 2014 NetBSD 8.3
Powered by man-cgi (2024-08-26). Maintained for NetBSD by Kimmo Suominen. Based on man-cgi by Panagiotis Christias.