Skip to content

Commit 513b28a

Browse files
authored
0.14.1 --- a copy of the space can start a thread (#29)
Fixes mcpplibs/openkal-musl#55. `kal_space_start` made the copy with the duplication primitive alone. This system's `fork` does more: around the primitive each part of the system library takes its locks, and in the copy each makes its state true of the new process. Without that, the copy's first thread failed with `EAGAIN`. Measured on macOS 14.8.9, in a program that duplicated itself with the primitive and asked the copy for a thread: | preparations | the copy's `pthread_create` | | --- | --- | | none | `EAGAIN` | | the two child preparations alone | the copy was killed | | and the thread library's locks around the primitive | 0 | | the whole set, in the library's order, five rounds of twenty threads | 0 | The routine `fork` itself calls (`_libSystem_atfork_*`) is not exported; the parts it calls are (dyld, malloc, libc, the thread library, and `_mach_fork_child`). They are called in the library's order, under names no C library in this ecosystem defines, so they are reachable beneath one. `port/libSystem.tbd` lists them for a link made elsewhere, and `-U` leaves them for the loader in a native link, whose development stub need not list them. Handlers a program registers with `pthread_atfork` are not run: a copy made here is not a `fork` the program asked for. `tests/conformance_space_task.cpp` starts a context, joins it and starts one with a least stack size in a copy, three times, and reads the outcome through `kal_process_wait`.
1 parent d48dc4f commit 513b28a

6 files changed

Lines changed: 174 additions & 8 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,15 @@ jobs:
329329
# first is deliberate: `pthread_self' IS a name musl defines, and
330330
# this system's library would be handed a thread record of
331331
# another library's layout.
332+
# _dyld_atfork_*, _dyld_fork_child,
333+
# _malloc_fork_*, _libc_fork_*,
334+
# _pthread_atfork_*, _mach_fork_child the preparations this
335+
# system's `fork' performs around the duplication primitive, part
336+
# by part, which kal_space_start calls so that a copy can start a
337+
# thread (src/space.cpp, mcpplibs/openkal-musl#55). The routine
338+
# `fork' calls them from is not exported; each part is, under a
339+
# name no C library in this ecosystem defines. None of them calls
340+
# back into a C library above.
332341
# __libc_start_main, main, _main the hand-over, undefined here by
333342
# construction.
334343
# kal_* the interface itself.
@@ -348,7 +357,7 @@ jobs:
348357
# ___dso_handle likewise the format's: emitted
349358
# beside the weak references above, defined by the image's own
350359
# start files.
351-
permitted='^_?(memcpy|memmove|memset|memcmp|bzero|clock_gettime_nsec_np|pthread_create_from_mach_thread|pthread_create|pthread_join|_pthread_self|_pthread_get_stackaddr_np|_pthread_get_stacksize_np|__libc_start_main|main|kal_[a-z_]+|__stack_chk_guard|__stack_chk_fail|GCC_except_table.*|_ZN3okm.*|__Unwind_Resume|__dso_handle|section\$(start|end)\$__TEXT\$__(eh_frame|unwind_info))$'
360+
permitted='^_?(memcpy|memmove|memset|memcmp|bzero|clock_gettime_nsec_np|pthread_create_from_mach_thread|pthread_create|pthread_join|_pthread_self|_pthread_get_stackaddr_np|_pthread_get_stacksize_np|_dyld_atfork_(prepare|parent)|_dyld_fork_child|_(malloc|libc)_fork_(prepare|parent|child)|_pthread_atfork_(prepare|parent|child)|_mach_fork_child|__libc_start_main|main|kal_[a-z_]+|__stack_chk_guard|__stack_chk_fail|GCC_except_table.*|_ZN3okm.*|__Unwind_Resume|__dso_handle|section\$(start|end)\$__TEXT\$__(eh_frame|unwind_info))$'
352361
# Reported with the object that references it. A symbol without the
353362
# object it came from names a fault and not a place, and the first
354363
# time this check fired the answer was in the object rather than in

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ written on the kernel's own calls.
88
openkal = "0.16.0"
99

1010
[target.'cfg(os = "macos")'.dependencies]
11-
openkal-macos = "0.14.0"
11+
openkal-macos = "0.14.1"
1212
```
1313

1414
Its purpose is as much to test the specification as to be used. A specification

‎mcpp.toml‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22
namespace = "mcpplibs"
33
name = "openkal-macos"
4-
version = "0.14.0"
4+
version = "0.14.1"
55
description = "An implementation of openkal for macOS, written on the kernel's own calls. Its purpose is as much to test the specification as to be used."
66
license = "Apache-2.0"
77

@@ -70,6 +70,31 @@ flags = [
7070
"-fno-stack-protector", "-fno-builtin"] },
7171
]
7272

73+
# THE PREPARATIONS FOR A DUPLICATE ARE LEFT FOR THE LOADER TO FIND.
74+
#
75+
# kal_space_start calls the parts of the system library that this system's
76+
# `fork' calls around the duplication primitive (src/space.cpp,
77+
# mcpplibs/openkal-musl#55). They are exported, and they are not part of the
78+
# interface the system documents, so a development stub need not list them.
79+
# `-U' states that a name may be left for the loader to find among the images
80+
# it has loaded, which is where it is; a name the stub does list is bound as
81+
# usual. A link made elsewhere reads port/libSystem.tbd, which lists them. The
82+
# flags reach every program above this package: mcpp forwards a dependency's
83+
# `ldflags' to its consumer's link.
84+
ldflags = ["-Wl,-U,__dyld_atfork_parent",
85+
"-Wl,-U,__dyld_atfork_prepare",
86+
"-Wl,-U,__dyld_fork_child",
87+
"-Wl,-U,__libc_fork_child",
88+
"-Wl,-U,__libc_fork_parent",
89+
"-Wl,-U,__libc_fork_prepare",
90+
"-Wl,-U,__mach_fork_child",
91+
"-Wl,-U,__malloc_fork_child",
92+
"-Wl,-U,__malloc_fork_parent",
93+
"-Wl,-U,__malloc_fork_prepare",
94+
"-Wl,-U,__pthread_atfork_child",
95+
"-Wl,-U,__pthread_atfork_parent",
96+
"-Wl,-U,__pthread_atfork_prepare"]
97+
7398
[features]
7499
default = []
75100

‎port/libSystem.tbd‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,13 @@ targets: [ arm64-macos, x86_64-macos ]
2828
install-name: '/usr/lib/libSystem.B.dylib'
2929
current-version: 1351.0.0
3030
#
31-
# FIVE NAMES ARE CALLED AND ONE IS NOT.
31+
# EIGHTEEN NAMES ARE CALLED AND ONE IS NOT.
3232
#
33-
# The first four are names this implementation CALLS, and the enumeration above
34-
# found them. The three that 0.13.0 added answer the stack of the calling
33+
# The first eighteen are names this implementation CALLS. The thirteen that
34+
# 0.14.1 added are the preparations this system's `fork' performs around the
35+
# primitive, part by part, which a duplicate made by kal_space_start needs
36+
# before it can start a thread (src/space.cpp, mcpplibs/openkal-musl#55).
37+
# The three that 0.13.0 added answer the stack of the calling
3538
# thread (openkal 0.15, clause 11 entry 21), and they follow the same rule as
3639
# the two before them: this system's library defines them and no C library in
3740
# this ecosystem does, which is what makes them reachable from an
@@ -54,7 +57,12 @@ current-version: 1351.0.0
5457
# evidence about a format.
5558
exports:
5659
- targets: [ arm64-macos, x86_64-macos ]
57-
symbols: [ _clock_gettime_nsec_np, _pthread_create_from_mach_thread,
60+
symbols: [ __dyld_atfork_parent, __dyld_atfork_prepare, __dyld_fork_child,
61+
__libc_fork_child, __libc_fork_parent, __libc_fork_prepare,
62+
__mach_fork_child, __malloc_fork_child, __malloc_fork_parent,
63+
__malloc_fork_prepare, __pthread_atfork_child, __pthread_atfork_parent,
64+
__pthread_atfork_prepare,
65+
_clock_gettime_nsec_np, _pthread_create_from_mach_thread,
5866
_pthread_get_stackaddr_np, _pthread_get_stacksize_np,
5967
__pthread_self, dyld_stub_binder ]
6068
...

‎src/space.cpp‎

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,63 @@
1919

2020
extern "C" {
2121

22+
// THE SYSTEM LIBRARY'S OWN PREPARATION FOR A DUPLICATE, WITHOUT WHICH THE
23+
// DUPLICATE CANNOT START A THREAD.
24+
//
25+
// This system's `fork' is not only the primitive. Around it each part of the
26+
// system library takes its locks, and in the copy each makes its state true of
27+
// the new process --- among the rest the name of the task, through which the
28+
// thread library allocates every stack, and the thread library's own record of
29+
// which threads exist. The primitive alone left the copy holding the original's
30+
// state, and the first thread it asked for failed (mcpplibs/openkal-musl#55).
31+
//
32+
// MEASURED on macOS 14.8.9, in a program that duplicated itself with the
33+
// primitive and asked the copy for a thread:
34+
//
35+
// nothing EAGAIN
36+
// the two child preparations alone the copy was killed
37+
// and the thread library's locks around 0
38+
// the whole set below, five times twenty 0
39+
//
40+
// The library's own `fork' calls these from a routine it does not export, so
41+
// the parts are called directly, in its order: locks before the primitive from
42+
// the outermost part inward, released in the original in the reverse order,
43+
// and the state renewed in the copy. Each is exported under a name no C library
44+
// in this ecosystem defines, so it is reachable beneath one as above one. The
45+
// handlers a program registers with `pthread_atfork' are not run: a copy made
46+
// here is not a `fork' the program asked for.
47+
void _dyld_atfork_prepare(void);
48+
void _dyld_atfork_parent(void);
49+
void _dyld_fork_child(void);
50+
void _malloc_fork_prepare(void);
51+
void _malloc_fork_parent(void);
52+
void _malloc_fork_child(void);
53+
void _libc_fork_prepare(void);
54+
void _libc_fork_parent(void);
55+
void _libc_fork_child(void);
56+
void _pthread_atfork_prepare(void);
57+
void _pthread_atfork_parent(void);
58+
void _pthread_atfork_child(void);
59+
void _mach_fork_child(void);
60+
61+
} // extern "C"
62+
63+
namespace {
64+
65+
void prepare() {
66+
_dyld_atfork_prepare(); _malloc_fork_prepare(); _libc_fork_prepare(); _pthread_atfork_prepare();
67+
}
68+
void in_the_original() {
69+
_pthread_atfork_parent(); _libc_fork_parent(); _malloc_fork_parent(); _dyld_atfork_parent();
70+
}
71+
void in_the_copy() {
72+
_mach_fork_child(); _dyld_fork_child(); _pthread_atfork_child(); _malloc_fork_child(); _libc_fork_child();
73+
}
74+
75+
} // namespace
76+
77+
extern "C" {
78+
2279
int kal_space_start(void (*entry)(void*), void* arg, void* stack_top,
2380
kal_process* out) {
2481
if (entry == nullptr || out == nullptr) return kal_err_invalid;
@@ -31,10 +88,12 @@ int kal_space_start(void (*entry)(void*), void* arg, void* stack_top,
3188
(void)stack_top;
3289

3390
bool is_duplicate = false;
91+
prepare();
3492
const okm_long child = okm::duplicate(is_duplicate);
35-
if (okm::failed(child)) return okm::translate(child);
93+
if (okm::failed(child)) { in_the_original(); return okm::translate(child); }
3694

3795
if (is_duplicate) {
96+
in_the_copy();
3897
entry(arg);
3998
// THE ENTRY IS NOT REQUIRED TO RETURN, AND IF IT DOES THE CONTEXT ENDS.
4099
//
@@ -46,6 +105,7 @@ int kal_space_start(void (*entry)(void*), void* arg, void* stack_top,
46105
for (;;) okm::sys(okm::nr_exit, 0);
47106
}
48107

108+
in_the_original();
49109
*out = kal_process{ static_cast<kal_uintptr>(child) };
50110
return kal_ok;
51111
}

‎tests/conformance_space_task.cpp‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
// Conformance: a context started in a copy of the space can start a context.
2+
//
3+
// mcpplibs/openkal-musl#55: in a copy made by kal_space_start, the first
4+
// thread the copy asked for failed with EAGAIN, while the original started one.
5+
// The duplication primitive alone leaves the system library's state the
6+
// original's, and this system's `fork' prepares it around the primitive; the
7+
// implementation now does what `fork' does. Observed through the status, the
8+
// only channel a separate space has: 3 is a context started, joined and given
9+
// its least stack size in the copy, and any other value is the step that
10+
// failed.
11+
import openkal.types;
12+
import openkal.task;
13+
import openkal.space;
14+
import openkal.process;
15+
import openkal.stream;
16+
import openkal.abort;
17+
18+
namespace {
19+
int failures = 0;
20+
void check(bool ok, const char* what) {
21+
if (ok) return;
22+
++failures;
23+
const char pre[] = "FAIL: ";
24+
kal::write(kal::err(), pre, sizeof(pre) - 1);
25+
kal_uintptr n = 0; while (what[n]) ++n;
26+
kal::write(kal::err(), what, n);
27+
kal::write(kal::err(), "\n", 1);
28+
}
29+
30+
volatile int g_ran = 0;
31+
void ran(void*) { g_ran = 1; }
32+
33+
void in_the_copy(void*) {
34+
kal_task t{};
35+
if (kal_task_start(ran, nullptr, &t) != kal_ok) kal_exit(4);
36+
if (kal_task_join(t) != kal_ok || g_ran != 1) kal_exit(5);
37+
kal_task s{};
38+
if (kal_task_start_sized(ran, nullptr, 16u << 20, &s) != kal_ok) kal_exit(6);
39+
kal_task_join(s);
40+
kal_exit(3);
41+
}
42+
43+
alignas(16) unsigned char g_stack[64 * 1024];
44+
}
45+
46+
int main() {
47+
for (int round = 0; round < 3; ++round) {
48+
kal_process p{};
49+
const int rc = kal_space_start(in_the_copy, nullptr, g_stack + sizeof g_stack, &p);
50+
check(rc == kal_ok, "a copy of the space starts");
51+
if (rc != kal_ok) break;
52+
int status = 0, terminated = 0;
53+
check(kal_process_wait(p, &status, &terminated) == kal_ok, "the copy is waited for");
54+
kal_process_close(p);
55+
check(terminated == 0 && status == 3, "the copy started a context, joined it, and started one with a least stack size");
56+
if (status != 3) {
57+
char line[] = "copy status: X\n";
58+
line[13] = static_cast<char>('0' + (status % 10));
59+
kal::write(kal::err(), line, sizeof line - 1);
60+
}
61+
}
62+
if (failures == 0) kal::write(kal::out(), "openkal-macos: a copy of the space starts contexts\n", 51);
63+
return failures == 0 ? 0 : 1;
64+
}

0 commit comments

Comments
 (0)