Skip to content

Commit 4ee4157

Browse files
0.21.0 --- openkal 0.20: transfers at an offset, openkal.region, a program's and a unit's memory, a supervised limit; the clock through the vDSO (#45)
Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent c3ab303 commit 4ee4157

11 files changed

Lines changed: 859 additions & 81 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,19 @@ jobs:
250250
# examined here, where a build of the modules exists: the test names
251251
# every entity SURFACE.txt lists, so a name the modules do not export
252252
# fails to compile and the diagnostic names it.
253-
list="$(grep -vE '^[[:space:]]*(#|$)' .spec/SURFACE.txt | sort -u)"
253+
# THE NAMES OF THE INTERFACES THIS IMPLEMENTATION PROVIDES, not every
254+
# name: since openkal 0.20 SURFACE.txt carries `openkal.prof' and
255+
# `openkal.debug', which are a loader's (clause 12.3), and an interface
256+
# an implementation does not provide is absent by clause 6.1. The set is
257+
# the manifest's `provides-interfaces', which the step above checks
258+
# against the artefact, and the self-description every implementation
259+
# exports.
260+
provided="$(sed -n '/^provides-interfaces/,/^]/p' mcpp.toml | grep -oE 'openkal\.[a-z]+'; echo openkal.version)"
261+
list="$(awk -v keep="$provided" '
262+
BEGIN { n = split(keep, k, "\n"); for (i = 1; i <= n; i++) want[k[i]] = 1 }
263+
/^# openkal\.[a-z]+( |$)/ { group = $2; next }
264+
/^[[:space:]]*(#|$)/ { next }
265+
group in want { print }' .spec/SURFACE.txt | sort -u)"
254266
{
255267
# THE MODULE LIST IS DERIVED, NOT WRITTEN OUT.
256268
#

‎README.md‎

Lines changed: 55 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@ for Linux, written on the kernel's own system-call interface.
55

66
```toml
77
[dependencies]
8-
openkal = "0.19.0"
8+
openkal = "0.20.0"
99

1010
[target.'cfg(os = "linux")'.dependencies]
11-
openkal-linux = "0.20.0"
11+
openkal-linux = "0.21.0"
1212
```
1313

1414
## Why it does not use a C library
@@ -42,7 +42,8 @@ second and third.
4242

4343
## Interfaces provided
4444

45-
All eight. `tools/check-surface.sh --complete` in the specification package
45+
Every interface of `SURFACE.txt` except `openkal.prof` and `openkal.debug`, which
46+
are a loader's (clause 12.3). `tools/check-surface.sh --complete` in the specification package
4647
compares the exported names against `SURFACE.txt`.
4748

4849
## The stack a started context is given
@@ -101,22 +102,57 @@ A program that carries a runtime of its own answers from that runtime instead
101102
(`pthread_getattr_np`), because in that arrangement the runtime is what owns the
102103
stacks and is what a program above openkal would ask for itself.
103104

104-
## A defect this version repairs
105-
106-
0.16.0 added the storage this implementation keeps for the region a context
107-
stands on, and that storage made a latent defect of `src/tls.h` visible: the
108-
thread-local segment's alignment was clamped up to sixteen where the block's
109-
SIZE is computed, and the size has to be the one the linker measured every
110-
variable's offset against. On a segment stating `p_align = 8, p_memsz = 56` the
111-
linker laid the variables out at `tp - 56` while the region was built 64 bytes
112-
deep, so the image of the program's thread-local storage sat eight bytes below
113-
the variables that name it. A program whose thread-local variables are
114-
initialised then reads another variable's bytes, and a C++ program's
115-
`thread_local` object can find its guard byte non-zero and never be constructed
116-
--- which is how openkal-llvm-runtime's own probe reported it.
117-
118-
The clamp now applies to the ALLOCATION, which wants sixteen, and not to the
119-
size, which is the linker's.
105+
## What 0.21 adds (openkal 0.20)
106+
107+
**Transfers at an offset.** `kal_fs_read_at` and `kal_fs_write_at` are the
108+
kernel's `pread64` and `pwrite64`: one call each, and the file's position is
109+
neither used nor moved. A write at an offset to a file opened for appending lands
110+
at the end, which is this kernel's answer and one the specification names.
111+
112+
**Regions.** `openkal.region` is provided. With the kernel's default treatment of
113+
memory (`vm.overcommit_memory` 0 or 1) a reservation is a mapping usable at once
114+
(`MAP_NORESERVE`), `kal_region_commit` makes no call, returning the backing is
115+
`madvise(MADV_DONTNEED)` and discarding is `madvise(MADV_FREE)`;
116+
`KAL_REGION_PROP_OVERCOMMIT` is claimed. Where the kernel charges memory when it
117+
is mapped (`vm.overcommit_memory` 2), a reservation is inaccessible and uncharged,
118+
backing is `mprotect`, and returning it is a fresh inaccessible mapping over the
119+
part; the property is not claimed. The mode is read once from /proc.
120+
121+
**The clock without a system call.** `kal_time_monotonic` and `kal_time_wall`
122+
call the `clock_gettime` the kernel places in every program (the vDSO, found
123+
through `AT_SYSINFO_EHDR`), and the system call remains where there is none.
124+
125+
**A volume is asked once.** Whether a file may be started is a property of its
126+
volume as well as of its node (`KAL_INFO_EXECUTABLE`), and the volume's answer
127+
(`fstatfs`) is kept by the device number the node reports: a `stat` that wants the
128+
field makes one system call, not two. Eight volumes are kept.
129+
130+
**What a program and a unit use.** `kal_process_info` reports `KAL_PINFO_MEMORY`:
131+
`VmRSS` and `VmHWM` of /proc/<pid>/status for a running program, and for a
132+
started program that has ended its peak from the record the kernel keeps until it
133+
is collected. `kal_process_wait` therefore waits without collecting
134+
(`waitid(WNOWAIT)`) and `kal_process_close` collects: a caller that waits and
135+
never releases the handle leaves the program in the kernel's table until it ends
136+
itself, which the handle's ownership (clause 6.7) already asked it not to do.
137+
`kal_timeout_wait_process` waits on the program's descriptor (`pidfd_open`, 5.3
138+
and later) instead of polling.
139+
140+
A unit is a process group, and its use is the sum over the entries of /proc
141+
whose group is the unit's --- every program in it, including those its members
142+
started; one read per program on the machine, made only when a caller asks.
143+
`KAL_PROCESS_PROP_JOB_USAGE` is claimed.
144+
145+
**A limit on a unit is supervised.** `kal_process_job_limit` starts, with the
146+
first limit, a context that observes the limited units and ends a unit with
147+
SIGKILL when it sees its use over the limit: every 500 ms while the use is below
148+
half the limit, 200 ms below four fifths, 100 ms above. The longest interval is
149+
therefore 500 ms. A program a limit ended is reported by `kal_process_wait` with
150+
`KAL_TERMINATED_LIMIT`. `KAL_PROCESS_PROP_JOB_LIMIT_SUPERVISED` is claimed and
151+
`KAL_PROCESS_PROP_JOB_LIMIT_HARD` is not: the kernel's own ceiling is a control
152+
group, a program may make one only where the hierarchy was delegated to it, and
153+
whether it was is the service manager's record rather than the kernel's --- on an
154+
ordinary desktop the program's own slice is writable and is not delegated. A
155+
program that holds a delegation and wants the kernel's ceiling uses it directly.
120156

121157
## Conformance
122158

‎mcpp.toml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22
namespace = "mcpplibs"
33
name = "openkal-linux"
4-
version = "0.20.0"
4+
version = "0.21.0"
55
description = "The reference implementation of openkal for Linux, written on the kernel's own system-call interface so that it can be placed beneath a C library as well as above one."
66
license = "Apache-2.0"
77

@@ -61,10 +61,11 @@ provides-interfaces = [
6161
"openkal.space",
6262
"openkal.timeout",
6363
"openkal.map",
64+
"openkal.region",
6465
]
6566

6667
[dependencies]
67-
openkal = "0.19.0"
68+
openkal = "0.20.0"
6869

6970
# The package contributes definitions and no modules. The interface it
7071
# implements is declared by the specification package, which this package

‎src/fs.cpp‎

Lines changed: 68 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,34 @@ namespace okl {
220220
// The volume's properties, for the interfaces outside this file that ask one
221221
// (openkal.net's rendezvous at a name).
222222
kal_uintptr volume_props_of(int fd) { return volume_props(fd); }
223+
224+
// THE PROPERTIES OF A VOLUME, ONCE PER VOLUME (0.21). An enquiry that wants
225+
// whether a file may be started asks its volume (a format that records no such
226+
// bit answers nothing), and asked `fstatfs' once per enquiry: a compiler run as
227+
// an image made 71 thousand of them in one warm start, one beside every `stat'.
228+
// A volume's format does not change under a device number while it is mounted,
229+
// so the answer is kept by the device the kernel reports for the node; a slot
230+
// a later device needs is taken over (eight are kept).
231+
struct volume_answer { okl_u64 dev; kal_uintptr props; bool known; };
232+
volume_answer g_volumes[8];
233+
unsigned g_volume_next = 0;
234+
volatile int g_volume_lock = 0;
235+
236+
kal_uintptr volume_props_on(okl_u64 dev, int fd) {
237+
while (__atomic_exchange_n(&g_volume_lock, 1, __ATOMIC_ACQUIRE)) okl::sys(okl::nr_sched_yield);
238+
for (auto const& v : g_volumes)
239+
if (v.known && v.dev == dev) {
240+
const kal_uintptr props = v.props;
241+
__atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
242+
return props;
243+
}
244+
__atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
245+
const kal_uintptr props = volume_props(fd);
246+
while (__atomic_exchange_n(&g_volume_lock, 1, __ATOMIC_ACQUIRE)) okl::sys(okl::nr_sched_yield);
247+
g_volumes[g_volume_next++ % 8] = volume_answer{ dev, props, true };
248+
__atomic_store_n(&g_volume_lock, 0, __ATOMIC_RELEASE);
249+
return props;
250+
}
223251
} // namespace okl
224252

225253
namespace {
@@ -236,7 +264,8 @@ namespace {
236264
// The exception is KAL_INFO_EXECUTABLE, which is a property of the volume as
237265
// well as of the node: a format that stores no mode reports execute bits the
238266
// mount chose, and reporting those would be answering for the volume. The
239-
// volume is asked only when the caller wants the field, and only for a file.
267+
// volume is asked only when the caller wants the field, and only for a file,
268+
// and once per volume.
240269
void fill_info(const okl::kstat& st, kal_u32 wanted, kal_node_info* out,
241270
int volume_fd) {
242271
const kal_u32 self = out->self_size;
@@ -259,7 +288,7 @@ void fill_info(const okl::kstat& st, kal_u32 wanted, kal_node_info* out,
259288
__builtin_offsetof(kal_node_info, executable) + sizeof v.executable);
260289
if ((wanted & KAL_INFO_EXECUTABLE) != 0 && self >= reaches
261290
&& v.kind == kal_node_file
262-
&& (volume_props(volume_fd) & KAL_FS_PROP_EXECUTABLE) != 0) {
291+
&& (okl::volume_props_on(st.dev, volume_fd) & KAL_FS_PROP_EXECUTABLE) != 0) {
263292
v.present |= KAL_INFO_EXECUTABLE;
264293
v.executable = (st.mode & 0111u) != 0 ? 1 : 0;
265294
}
@@ -487,6 +516,43 @@ int kal_fs_truncate(kal_file f, kal_u64 size) {
487516
return okl::failed(r) ? okl::translate(r) : kal_ok;
488517
}
489518

519+
// TRANSFERS AT AN OFFSET (openkal 0.20): the kernel's own `pread64' and
520+
// `pwrite64', which neither use nor move the position. A write at an offset to
521+
// a file opened for appending lands at the end, which is this kernel's answer
522+
// and one the specification names. The write is completed as a stream's is,
523+
// and an interruption is retried rather than reported (clause 7.5).
524+
kal_intptr kal_fs_read_at(kal_file f, void* buf, kal_uintptr len, kal_u64 offset) {
525+
const int fd = okl::unpack(f.h);
526+
if (fd < 0) return -kal_err_invalid;
527+
for (;;) {
528+
const okl_long r = okl::sys(okl::nr_pread64, fd, reinterpret_cast<okl_long>(buf),
529+
static_cast<okl_long>(len), static_cast<okl_long>(offset));
530+
if (okl::interrupted(r)) continue;
531+
if (okl::failed(r)) return -okl::translate(r);
532+
return static_cast<kal_intptr>(r);
533+
}
534+
}
535+
536+
kal_intptr kal_fs_write_at(kal_file f, const void* buf, kal_uintptr len, kal_u64 offset) {
537+
const int fd = okl::unpack(f.h);
538+
if (fd < 0) return -kal_err_invalid;
539+
const auto* p = static_cast<const unsigned char*>(buf);
540+
kal_uintptr done = 0;
541+
while (done < len) {
542+
const okl_long r = okl::sys(okl::nr_pwrite64, fd, reinterpret_cast<okl_long>(p + done),
543+
static_cast<okl_long>(len - done),
544+
static_cast<okl_long>(offset + done));
545+
if (okl::interrupted(r)) continue;
546+
if (okl::failed(r)) {
547+
if (done != 0) return static_cast<kal_intptr>(done);
548+
return -okl::translate(r);
549+
}
550+
if (r == 0) break;
551+
done += static_cast<kal_uintptr>(r);
552+
}
553+
return static_cast<kal_intptr>(done);
554+
}
555+
490556
int kal_fs_info(kal_dir base, const char* name, kal_uintptr len,
491557
kal_uintptr flags, kal_u32 wanted, kal_node_info* out) {
492558
const int b = okl::unpack(base.h);

0 commit comments

Comments
 (0)