0.21.0 --- openkal 0.20: transfers at an offset, openkal.region, a program's and a unit's memory, a supervised limit; the clock through the vDSO

.github/workflows/ci.yml

@@ -250,7 +250,19 @@ jobs:
           # examined here, where a build of the modules exists: the test names
           # every entity SURFACE.txt lists, so a name the modules do not export
           # fails to compile and the diagnostic names it.
-          list="$(grep -vE '^[[:space:]]*(#|$)' .spec/SURFACE.txt | sort -u)"
+          # THE NAMES OF THE INTERFACES THIS IMPLEMENTATION PROVIDES, not every
+          # name: since openkal 0.20 SURFACE.txt carries `openkal.prof' and
+          # `openkal.debug', which are a loader's (clause 12.3), and an interface
+          # an implementation does not provide is absent by clause 6.1. The set is
+          # the manifest's `provides-interfaces', which the step above checks
+          # against the artefact, and the self-description every implementation
+          # exports.
+          provided="$(sed -n '/^provides-interfaces/,/^]/p' mcpp.toml | grep -oE 'openkal\.[a-z]+'; echo openkal.version)"
+          list="$(awk -v keep="$provided" '
+              BEGIN { n = split(keep, k, "\n"); for (i = 1; i <= n; i++) want[k[i]] = 1 }
+              /^# openkal\.[a-z]+( |$)/ { group = $2; next }
+              /^[[:space:]]*(#|$)/ { next }
+              group in want { print }' .spec/SURFACE.txt | sort -u)"
           {
             # THE MODULE LIST IS DERIVED, NOT WRITTEN OUT.
             #

README.md

@@ -5,10 +5,10 @@ for Linux, written on the kernel's own system-call interface.
 
 ```toml
 [dependencies]
-openkal = "0.19.0"
+openkal = "0.20.0"
 
 [target.'cfg(os = "linux")'.dependencies]
-openkal-linux = "0.20.0"
+openkal-linux = "0.21.0"
 ```
 
 ## Why it does not use a C library
@@ -42,7 +42,8 @@ second and third.
 
 ## Interfaces provided
 
-All eight. `tools/check-surface.sh --complete` in the specification package
+Every interface of `SURFACE.txt` except `openkal.prof` and `openkal.debug`, which
+are a loader's (clause 12.3). `tools/check-surface.sh --complete` in the specification package
 compares the exported names against `SURFACE.txt`.
 
 ## The stack a started context is given
@@ -101,22 +102,57 @@ A program that carries a runtime of its own answers from that runtime instead
 (`pthread_getattr_np`), because in that arrangement the runtime is what owns the
 stacks and is what a program above openkal would ask for itself.
 
-## A defect this version repairs
-
-0.16.0 added the storage this implementation keeps for the region a context
-stands on, and that storage made a latent defect of `src/tls.h` visible: the
-thread-local segment's alignment was clamped up to sixteen where the block's
-SIZE is computed, and the size has to be the one the linker measured every
-variable's offset against. On a segment stating `p_align = 8, p_memsz = 56` the
-linker laid the variables out at `tp - 56` while the region was built 64 bytes
-deep, so the image of the program's thread-local storage sat eight bytes below
-the variables that name it. A program whose thread-local variables are
-initialised then reads another variable's bytes, and a C++ program's
-`thread_local` object can find its guard byte non-zero and never be constructed
---- which is how openkal-llvm-runtime's own probe reported it.
-
-The clamp now applies to the ALLOCATION, which wants sixteen, and not to the
-size, which is the linker's.
+## What 0.21 adds (openkal 0.20)
+
+**Transfers at an offset.** `kal_fs_read_at` and `kal_fs_write_at` are the
+kernel's `pread64` and `pwrite64`: one call each, and the file's position is
+neither used nor moved. A write at an offset to a file opened for appending lands
+at the end, which is this kernel's answer and one the specification names.
+
+**Regions.** `openkal.region` is provided. With the kernel's default treatment of
+memory (`vm.overcommit_memory` 0 or 1) a reservation is a mapping usable at once
+(`MAP_NORESERVE`), `kal_region_commit` makes no call, returning the backing is
+`madvise(MADV_DONTNEED)` and discarding is `madvise(MADV_FREE)`;
+`KAL_REGION_PROP_OVERCOMMIT` is claimed. Where the kernel charges memory when it
+is mapped (`vm.overcommit_memory` 2), a reservation is inaccessible and uncharged,
+backing is `mprotect`, and returning it is a fresh inaccessible mapping over the
+part; the property is not claimed. The mode is read once from /proc.
+
+**The clock without a system call.** `kal_time_monotonic` and `kal_time_wall`
+call the `clock_gettime` the kernel places in every program (the vDSO, found
+through `AT_SYSINFO_EHDR`), and the system call remains where there is none.
+
+**A volume is asked once.** Whether a file may be started is a property of its
+volume as well as of its node (`KAL_INFO_EXECUTABLE`), and the volume's answer
+(`fstatfs`) is kept by the device number the node reports: a `stat` that wants the
+field makes one system call, not two. Eight volumes are kept.
+
+**What a program and a unit use.** `kal_process_info` reports `KAL_PINFO_MEMORY`:
+`VmRSS` and `VmHWM` of /proc/<pid>/status for a running program, and for a
+started program that has ended its peak from the record the kernel keeps until it
+is collected. `kal_process_wait` therefore waits without collecting
+(`waitid(WNOWAIT)`) and `kal_process_close` collects: a caller that waits and
+never releases the handle leaves the program in the kernel's table until it ends
+itself, which the handle's ownership (clause 6.7) already asked it not to do.
+`kal_timeout_wait_process` waits on the program's descriptor (`pidfd_open`, 5.3
+and later) instead of polling.
+
+A unit is a process group, and its use is the sum over the entries of /proc
+whose group is the unit's --- every program in it, including those its members
+started; one read per program on the machine, made only when a caller asks.
+`KAL_PROCESS_PROP_JOB_USAGE` is claimed.
+
+**A limit on a unit is supervised.** `kal_process_job_limit` starts, with the
+first limit, a context that observes the limited units and ends a unit with
+SIGKILL when it sees its use over the limit: every 500 ms while the use is below
+half the limit, 200 ms below four fifths, 100 ms above. The longest interval is
+therefore 500 ms. A program a limit ended is reported by `kal_process_wait` with
+`KAL_TERMINATED_LIMIT`. `KAL_PROCESS_PROP_JOB_LIMIT_SUPERVISED` is claimed and
+`KAL_PROCESS_PROP_JOB_LIMIT_HARD` is not: the kernel's own ceiling is a control
+group, a program may make one only where the hierarchy was delegated to it, and
+whether it was is the service manager's record rather than the kernel's --- on an
+ordinary desktop the program's own slice is writable and is not delegated. A
+program that holds a delegation and wants the kernel's ceiling uses it directly.
 
 ## Conformance
 

mcpp.toml

@@ -1,7 +1,7 @@
 [package]
 namespace   = "mcpplibs"
 name        = "openkal-linux"
-version     = "0.20.0"
+version     = "0.21.0"
 description = "The reference implementation of openkal for Linux, written on the kernel's own system-call interface so that it can be placed beneath a C library as well as above one."
 license     = "Apache-2.0"
 
@@ -61,10 +61,11 @@ provides-interfaces = [
     "openkal.space",
     "openkal.timeout",
     "openkal.map",
+    "openkal.region",
 ]
 
 [dependencies]
-openkal = "0.19.0"
+openkal = "0.20.0"
 
 # The package contributes definitions and no modules. The interface it
 # implements is declared by the specification package, which this package

src/fs.cpp

@@ -220,6 +220,34 @@ namespace okl {
 // The volume's properties, for the interfaces outside this file that ask one
 // (openkal.net's rendezvous at a name).
 kal_uintptr volume_props_of(int fd) { return volume_props(fd); }
+
+// THE PROPERTIES OF A VOLUME, ONCE PER VOLUME (0.21). An enquiry that wants
+// whether a file may be started asks its volume (a format that records no such
+// bit answers nothing), and asked `fstatfs' once per enquiry: a compiler run as
+// an image made 71 thousand of them in one warm start, one beside every `stat'.
+// A volume's format does not change under a device number while it is mounted,
+// so the answer is kept by the device the kernel reports for the node; a slot
+// a later device needs is taken over (eight are kept).
+struct volume_answer { okl_u64 dev; kal_uintptr props; bool known; };
+volume_answer g_volumes[8];
+unsigned g_volume_next = 0;
+volatile int g_volume_lock = 0;
+
+kal_uintptr volume_props_on(okl_u64 dev, int fd) {
+    while (__atomic_exchange_n(&g_volume_lock, 1, __ATOMIC_ACQUIRE)) okl::sys(okl::nr_sched_yield);
+    for (auto const& v : g_volumes)
+        if (v.known && v.dev == dev) {
+            const kal_uintptr props = v.props;
+            __atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
+            return props;
+        }
+    __atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
+    const kal_uintptr props = volume_props(fd);
+    while (__atomic_exchange_n(&g_volume_lock, 1, __ATOMIC_ACQUIRE)) okl::sys(okl::nr_sched_yield);
+    g_volumes[g_volume_next++ % 8] = volume_answer{ dev, props, true };
+    __atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
+    return props;
+}
 }  // namespace okl
 
 namespace {
@@ -236,7 +264,8 @@ namespace {
 // The exception is KAL_INFO_EXECUTABLE, which is a property of the volume as
 // well as of the node: a format that stores no mode reports execute bits the
 // mount chose, and reporting those would be answering for the volume. The
-// volume is asked only when the caller wants the field, and only for a file.
+// volume is asked only when the caller wants the field, and only for a file,
+// and once per volume.
 void fill_info(const okl::kstat& st, kal_u32 wanted, kal_node_info* out,
                int volume_fd) {
     const kal_u32 self = out->self_size;
@@ -259,7 +288,7 @@ void fill_info(const okl::kstat& st, kal_u32 wanted, kal_node_info* out,
         __builtin_offsetof(kal_node_info, executable) + sizeof v.executable);
     if ((wanted & KAL_INFO_EXECUTABLE) != 0 && self >= reaches
         && v.kind == kal_node_file
-        && (volume_props(volume_fd) & KAL_FS_PROP_EXECUTABLE) != 0) {
+        && (okl::volume_props_on(st.dev, volume_fd) & KAL_FS_PROP_EXECUTABLE) != 0) {
         v.present   |= KAL_INFO_EXECUTABLE;
         v.executable = (st.mode & 0111u) != 0 ? 1 : 0;
     }
@@ -487,6 +516,43 @@ int kal_fs_truncate(kal_file f, kal_u64 size) {
     return okl::failed(r) ? okl::translate(r) : kal_ok;
 }
 
+// TRANSFERS AT AN OFFSET (openkal 0.20): the kernel's own `pread64' and
+// `pwrite64', which neither use nor move the position. A write at an offset to
+// a file opened for appending lands at the end, which is this kernel's answer
+// and one the specification names. The write is completed as a stream's is,
+// and an interruption is retried rather than reported (clause 7.5).
+kal_intptr kal_fs_read_at(kal_file f, void* buf, kal_uintptr len, kal_u64 offset) {
+    const int fd = okl::unpack(f.h);
+    if (fd < 0) return -kal_err_invalid;
+    for (;;) {
+        const okl_long r = okl::sys(okl::nr_pread64, fd, reinterpret_cast<okl_long>(buf),
+                                    static_cast<okl_long>(len), static_cast<okl_long>(offset));
+        if (okl::interrupted(r)) continue;
+        if (okl::failed(r)) return -okl::translate(r);
+        return static_cast<kal_intptr>(r);
+    }
+}
+
+kal_intptr kal_fs_write_at(kal_file f, const void* buf, kal_uintptr len, kal_u64 offset) {
+    const int fd = okl::unpack(f.h);
+    if (fd < 0) return -kal_err_invalid;
+    const auto* p = static_cast<const unsigned char*>(buf);
+    kal_uintptr done = 0;
+    while (done < len) {
+        const okl_long r = okl::sys(okl::nr_pwrite64, fd, reinterpret_cast<okl_long>(p + done),
+                                    static_cast<okl_long>(len - done),
+                                    static_cast<okl_long>(offset + done));
+        if (okl::interrupted(r)) continue;
+        if (okl::failed(r)) {
+            if (done != 0) return static_cast<kal_intptr>(done);
+            return -okl::translate(r);
+        }
+        if (r == 0) break;
+        done += static_cast<kal_uintptr>(r);
+    }
+    return static_cast<kal_intptr>(done);
+}
+
 int kal_fs_info(kal_dir base, const char* name, kal_uintptr len,
                 kal_uintptr flags, kal_u32 wanted, kal_node_info* out) {
     const int b = okl::unpack(base.h);