Skip to content

Commit 6eb3353

Browse files
committed
fix(daemon): explain foreign-owned cache ancestors and warn on network filesystems (DeusData#1717)
A cache root whose resolved path runs through a directory owned by another non-root account (typical on shared/HPC filesystems where ~/.cache is a symlink into a shared tree) is refused by the private-directory walk. The refusal itself is correct: the owner of an ancestor can rename or replace everything below it after the walk, so it could redirect the cache. But posix_directory_transition_secure() set no validation detail, so the caller printed "ancestry component validation failed (errno 17)" -- a stale EEXIST from mkdirat() that named neither the directory nor the reason. - The refusal now names the directory, its owner uid and account name, the caller's uid, and the remedy (a location with a self/root-owned ancestry, e.g. via CBM_CACHE_DIR). The path is shortened from the left so the remedy always fits the 384-byte detail buffer. - On Linux the account name is read from /etc/passwd, not getpwuid(). The release binary is statically linked against glibc, and getpwuid() there loads the host's NSS modules at runtime, which can crash on a glibc version mismatch (a real risk on sssd/LDAP cluster hosts). Accounts that exist only in a directory service are shown by uid. - New advice-only check: on Linux, a cache root on NFS, CIFS, SMB2, Lustre or GPFS (statfs f_type) logs one daemon.cache_root_network_fs warning per process. SQLite WAL is unreliable on network filesystems. Nothing is refused. - docs/CONFIGURATION.md gains a "Shared / HPC filesystems" section. The ownership policy is unchanged, and there is still no override (same stance as DeusData#1830). Test seams: one seam reports a directory (matched by dev/ino) as foreign-owned, and another supplies the statfs magic. The refusal, the message and the once-per-process warning are therefore testable without root and on every platform. Refs DeusData#1717 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 0f52d30 commit 6eb3353

5 files changed

Lines changed: 458 additions & 4 deletions

File tree

‎docs/CONFIGURATION.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,33 @@ be indexed and later returned. And the credential list is a denylist, so it
235235
raises the cost of a mistake rather than closing the class — a directory it does
236236
not name is permitted.
237237

238+
### Shared / HPC filesystems
239+
240+
On clusters and other shared machines, home directories often live on a network
241+
filesystem (NFS, Lustre, GPFS, SMB/CIFS), and `~/.cache` is sometimes a symlink
242+
into a shared tree. Three rules keep CBM working there:
243+
244+
- **Put the cache and the rendezvous on local storage.** Set `CBM_CACHE_DIR` (indexes,
245+
`_config.db`) and, if needed, `CBM_RUNTIME_DIR` (the daemon rendezvous) to a
246+
local disk you own, for example `/local/scratch/$USER/cbm` or `/tmp/$USER-cbm`.
247+
Set the same values in your MCP client's environment and your shell.
248+
- **Do not route the cache into another account's tree.** A symlinked `~/.cache` is
249+
resolved and allowed, but every directory on the *resolved* path must be owned
250+
by you or by root. If a parent such as `/shared/fs` belongs to another account,
251+
that account can rename or replace anything below it, so CBM refuses the path
252+
and names the directory and its owner:
253+
254+
```text
255+
... (cache-private) - '/shared/fs' is owned by uid 1234 (alice), not by you (uid 1000) or root; ...
256+
```
257+
258+
This check has no override. Choose a location with a safe ancestry instead.
259+
- **Avoid network filesystems for the cache.** The indexes are SQLite databases in
260+
WAL mode, which needs shared-memory locking that network filesystems do not
261+
provide reliably. On Linux, CBM logs one `daemon.cache_root_network_fs` warning
262+
at startup when the cache root is on NFS, Lustre, GPFS, CIFS or SMB2. The cache
263+
still works there, but it can be slow, and a database can be corrupted.
264+
238265
## 5. Agent and Editor Integration Files
239266

240267
The `install` command can also write MCP entries and instruction blocks into agent/editor config files such as Claude Code, Codex, Gemini, VS Code, Cursor, Zed, and others.

‎src/daemon/ipc.c‎

Lines changed: 205 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@
2121
#include <stdio.h>
2222
#include <stdlib.h>
2323
#include <string.h>
24+
#if defined(__linux__)
25+
#include <sys/vfs.h>
26+
#endif
2427

2528
enum {
2629
CBM_DAEMON_IPC_SEND_TIMEOUT_MS = 5000,
@@ -54,6 +57,76 @@ const char *cbm_daemon_ipc_validation_detail(void) {
5457
return ipc_validation_detail_buffer;
5558
}
5659

60+
/* #1717: a cache root on a network filesystem (a shared HPC home, an NFS or
61+
* SMB mount) is ACCEPTED -- this is advice, never a refusal. SQLite's WAL mode
62+
* relies on shared-memory locking that network filesystems do not provide
63+
* reliably, so indexes there can be slow or corrupt. Detection is statfs
64+
* f_type, Linux only; other platforms never warn. */
65+
static const char *ipc_network_fs_name(unsigned long magic) {
66+
static const struct {
67+
unsigned long magic;
68+
const char *name;
69+
} kinds[] = {
70+
{0x6969UL, "NFS"}, {0xFF534D42UL, "CIFS"}, {0xFE534D42UL, "SMB2"},
71+
{0x0BD00BD0UL, "Lustre"}, {0x47504653UL, "GPFS"},
72+
};
73+
for (size_t index = 0; index < sizeof(kinds) / sizeof(kinds[0]); index++) {
74+
if (kinds[index].magic == magic) {
75+
return kinds[index].name;
76+
}
77+
}
78+
return NULL;
79+
}
80+
81+
static atomic_bool g_ipc_network_cache_warned;
82+
#ifdef CBM_ENABLE_TEST_SEAMS
83+
static bool g_ipc_fs_magic_override_active;
84+
static unsigned long g_ipc_fs_magic_override;
85+
void cbm_daemon_ipc_set_fs_magic_for_test(bool active, unsigned long magic) {
86+
g_ipc_fs_magic_override_active = active;
87+
g_ipc_fs_magic_override = magic;
88+
atomic_store(&g_ipc_network_cache_warned, false);
89+
}
90+
#endif
91+
92+
static bool ipc_path_fs_magic(const char *path, unsigned long *magic_out) {
93+
#ifdef CBM_ENABLE_TEST_SEAMS
94+
if (g_ipc_fs_magic_override_active) {
95+
*magic_out = g_ipc_fs_magic_override;
96+
return true;
97+
}
98+
#endif
99+
#if defined(__linux__)
100+
struct statfs status;
101+
if (statfs(path, &status) != 0) {
102+
return false;
103+
}
104+
/* f_type is a signed word on some ABIs; the magics are 32-bit. */
105+
*magic_out = (unsigned long)status.f_type & 0xFFFFFFFFUL;
106+
return true;
107+
#else
108+
(void)path;
109+
(void)magic_out;
110+
return false;
111+
#endif
112+
}
113+
114+
bool cbm_daemon_ipc_warn_if_network_cache_root(const char *cache_root) {
115+
unsigned long magic = 0;
116+
const char *kind = NULL;
117+
if (!cache_root || !cache_root[0] || !ipc_path_fs_magic(cache_root, &magic) ||
118+
!(kind = ipc_network_fs_name(magic))) {
119+
return false;
120+
}
121+
if (atomic_exchange(&g_ipc_network_cache_warned, true)) {
122+
return false; /* one warning per process */
123+
}
124+
cbm_log_warn("daemon.cache_root_network_fs", "path", cache_root, "fs", kind, "reason",
125+
"sqlite_wal_unreliable_on_network_fs", "hint",
126+
"set CBM_CACHE_DIR to local storage (docs/CONFIGURATION.md)");
127+
return true;
128+
}
129+
57130
static cbm_daemon_ipc_listen_failure_t ipc_listen_failure;
58131

59132
/* The failure recorder and its reset are only reached from the POSIX socket
@@ -401,6 +474,9 @@ int cbm_daemon_ipc_wait_pending(const cbm_ipc_pending_ops_t *ops, uint32_t timeo
401474
#include <poll.h>
402475
#include <pthread.h>
403476
#include <stddef.h>
477+
#if !defined(__linux__)
478+
#include <pwd.h>
479+
#endif
404480
#include <sys/file.h>
405481
#include <sys/socket.h>
406482
#include <sys/stat.h>
@@ -1788,14 +1864,138 @@ static bool posix_directory_parent_secure(int directory_fd) {
17881864
* never chmod'd or ACL-rewritten. The euid-only enforcement that stops a
17891865
* squatted private directory lives in the created/final/snapshot checks, not
17901866
* here — this only walks ancestors. */
1791-
static bool posix_directory_transition_secure(int parent_fd, int child_fd) {
1867+
#ifdef CBM_ENABLE_TEST_SEAMS
1868+
/* #1717 seam: report one directory (by dev/ino) as owned by a foreign uid, so
1869+
* the foreign-owned-ancestor refusal and its diagnostic are testable without
1870+
* root (an unprivileged test cannot chown a directory to another account). */
1871+
static bool g_posix_foreign_owner_active;
1872+
static dev_t g_posix_foreign_owner_dev;
1873+
static ino_t g_posix_foreign_owner_ino;
1874+
static uid_t g_posix_foreign_owner_uid;
1875+
bool cbm_daemon_ipc_posix_set_foreign_owned_dir_for_test(const char *directory_path,
1876+
unsigned long foreign_uid) {
1877+
struct stat status;
1878+
g_posix_foreign_owner_active = false;
1879+
if (!directory_path || lstat(directory_path, &status) != 0 || !S_ISDIR(status.st_mode)) {
1880+
return false;
1881+
}
1882+
g_posix_foreign_owner_dev = status.st_dev;
1883+
g_posix_foreign_owner_ino = status.st_ino;
1884+
g_posix_foreign_owner_uid = (uid_t)foreign_uid;
1885+
g_posix_foreign_owner_active = true;
1886+
return true;
1887+
}
1888+
void cbm_daemon_ipc_posix_clear_foreign_owned_dir_for_test(void) {
1889+
g_posix_foreign_owner_active = false;
1890+
}
1891+
#endif
1892+
1893+
static uid_t posix_directory_observed_owner(const struct stat *status) {
1894+
#ifdef CBM_ENABLE_TEST_SEAMS
1895+
if (g_posix_foreign_owner_active && status->st_dev == g_posix_foreign_owner_dev &&
1896+
status->st_ino == g_posix_foreign_owner_ino) {
1897+
return g_posix_foreign_owner_uid;
1898+
}
1899+
#endif
1900+
return status->st_uid;
1901+
}
1902+
1903+
enum { POSIX_OWNER_NAME_CAP = 33, POSIX_PASSWD_LINE_CAP = 1024 };
1904+
1905+
/* The account name for uid, or "" when it cannot be named. Diagnostic only.
1906+
*
1907+
* Linux reads /etc/passwd directly instead of calling getpwuid(): the Linux
1908+
* release binary is statically linked against glibc, where getpwuid() loads the
1909+
* HOST's NSS modules at runtime -- a version-mismatch crash hazard exactly on
1910+
* the sssd/LDAP hosts (#1717's shared HPC filesystems) this message serves.
1911+
* A directory-service account is then shown by uid only. */
1912+
static void posix_owner_name(uid_t uid, char out[POSIX_OWNER_NAME_CAP]) {
1913+
out[0] = '\0';
1914+
#if defined(__linux__)
1915+
FILE *passwd = cbm_fopen("/etc/passwd", "r");
1916+
if (!passwd) {
1917+
return;
1918+
}
1919+
char line[POSIX_PASSWD_LINE_CAP];
1920+
while (fgets(line, sizeof(line), passwd)) {
1921+
char *name_end = strchr(line, ':');
1922+
char *password_end = name_end ? strchr(name_end + 1, ':') : NULL;
1923+
if (!password_end || name_end == line) {
1924+
continue;
1925+
}
1926+
char *uid_end = NULL;
1927+
errno = 0;
1928+
unsigned long line_uid = strtoul(password_end + 1, &uid_end, 10);
1929+
if (errno == 0 && uid_end != password_end + 1 && *uid_end == ':' &&
1930+
line_uid == (unsigned long)uid) {
1931+
(void)snprintf(out, POSIX_OWNER_NAME_CAP, "%.*s", (int)(name_end - line), line);
1932+
break;
1933+
}
1934+
}
1935+
(void)fclose(passwd);
1936+
#else
1937+
struct passwd entry;
1938+
struct passwd *found = NULL;
1939+
char storage[POSIX_PASSWD_LINE_CAP];
1940+
if (getpwuid_r(uid, &entry, storage, sizeof(storage), &found) == 0 && found && found->pw_name) {
1941+
(void)snprintf(out, POSIX_OWNER_NAME_CAP, "%s", found->pw_name);
1942+
}
1943+
#endif
1944+
}
1945+
1946+
/* The foreign-owned-ancestor refusal. The remedy is the part the reader acts
1947+
* on, so it must survive the fixed detail buffer: the path is shortened from
1948+
* the LEFT ("...") to fit, which keeps the refusing component at its end. */
1949+
static void posix_foreign_owner_detail_set(const char *child_path, uid_t owner) {
1950+
char name[POSIX_OWNER_NAME_CAP];
1951+
posix_owner_name(owner, name);
1952+
char owner_text[64];
1953+
if (name[0]) {
1954+
(void)snprintf(owner_text, sizeof(owner_text), "uid %ld (%s)", (long)owner, name);
1955+
} else {
1956+
(void)snprintf(owner_text, sizeof(owner_text), "uid %ld", (long)owner);
1957+
}
1958+
char tail[sizeof(ipc_validation_detail_buffer)];
1959+
int tail_length = snprintf(tail, sizeof(tail),
1960+
"' is owned by %s, not by you (uid %ld) or root; that account "
1961+
"controls this path. Use a location whose parent directories are "
1962+
"yours or root's (e.g. set CBM_CACHE_DIR)",
1963+
owner_text, (long)geteuid());
1964+
if (tail_length < 0) {
1965+
return;
1966+
}
1967+
size_t used = (size_t)tail_length + 1; /* the opening quote */
1968+
size_t room = used < sizeof(ipc_validation_detail_buffer)
1969+
? sizeof(ipc_validation_detail_buffer) - 1 - used
1970+
: 0;
1971+
size_t path_length = strlen(child_path);
1972+
const char *ellipsis = "";
1973+
const char *shown = child_path;
1974+
if (path_length > room) {
1975+
ellipsis = "...";
1976+
shown = child_path + path_length - (room > 3 ? room - 3 : 0);
1977+
}
1978+
ipc_validation_detail_set("'%s%s%s", ellipsis, shown, tail);
1979+
}
1980+
1981+
static bool posix_directory_transition_secure(int parent_fd, int child_fd, const char *child_path) {
17921982
struct stat parent;
17931983
struct stat child;
17941984
if (parent_fd < 0 || child_fd < 0 || !posix_directory_parent_secure(parent_fd) ||
17951985
fstat(parent_fd, &parent) != 0 || fstat(child_fd, &child) != 0 ||
17961986
!S_ISDIR(parent.st_mode) || !S_ISDIR(child.st_mode) ||
1797-
!posix_directory_ancestor_owner_trusted(parent.st_uid) ||
1798-
!posix_directory_ancestor_owner_trusted(child.st_uid)) {
1987+
!posix_directory_ancestor_owner_trusted(posix_directory_observed_owner(&parent))) {
1988+
return false;
1989+
}
1990+
uid_t child_owner = posix_directory_observed_owner(&child);
1991+
if (!posix_directory_ancestor_owner_trusted(child_owner)) {
1992+
/* #1717: this refusal used to set no detail, so the caller printed
1993+
* "ancestry component validation failed (errno 17)" -- a stale EEXIST
1994+
* from the mkdirat() on the existing component, naming neither the
1995+
* directory nor the reason. The policy is unchanged: an account that
1996+
* owns an ancestor can rename or replace everything below it after
1997+
* this walk, so it could redirect the private directory. Name it. */
1998+
posix_foreign_owner_detail_set(child_path, child_owner);
17991999
return false;
18002000
}
18012001
return true;
@@ -1870,7 +2070,8 @@ static int private_directory_tree_open(const char *directory_path) {
18702070
: -1;
18712071
struct stat status;
18722072
ok = next_fd >= 0 && fd_set_cloexec(next_fd) && fstat(next_fd, &status) == 0 &&
1873-
S_ISDIR(status.st_mode) && posix_directory_transition_secure(current_fd, next_fd);
2073+
S_ISDIR(status.st_mode) &&
2074+
posix_directory_transition_secure(current_fd, next_fd, path);
18742075
if (ok && created) {
18752076
ok = status.st_uid == geteuid() && fchmod(next_fd, 0700) == 0 &&
18762077
cbm_macos_extended_acl_fd_clear(next_fd) &&

‎src/daemon/ipc.h‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,17 @@ bool cbm_daemon_ipc_private_directory_secure(const char *directory_path);
9292
* append it to their error messages; policy decisions never read it. */
9393
const char *cbm_daemon_ipc_validation_detail(void);
9494

95+
/* #1717: advice, never a refusal. When cache_root lives on a network
96+
* filesystem (Linux statfs: NFS, CIFS, SMB2, Lustre, GPFS), log one
97+
* `daemon.cache_root_network_fs` warning per process pointing at local
98+
* storage. Returns true only when this call emitted that warning. */
99+
bool cbm_daemon_ipc_warn_if_network_cache_root(const char *cache_root);
100+
#ifdef CBM_ENABLE_TEST_SEAMS
101+
/* Report `magic` as the cache root's statfs f_type on every platform (active),
102+
* or restore real detection; either way re-arm the once-per-process warning. */
103+
void cbm_daemon_ipc_set_fs_magic_for_test(bool active, unsigned long magic);
104+
#endif
105+
95106
/* This process's most recent listener-publication failure: the stage that
96107
* refused, the errno the failing step reported (0 when the step reported
97108
* none), and the artifact path it was operating on ("" when no path applies).
@@ -127,6 +138,12 @@ bool cbm_daemon_ipc_win_sid_trusted_for_testing(void *sid);
127138
* so they can be tested without a chown-able overflow-owned directory. */
128139
void cbm_daemon_ipc_posix_set_ancestor_overflow_uid_for_test(bool active,
129140
unsigned long overflow_uid);
141+
/* #1717 seam (POSIX): report the directory at directory_path (matched by
142+
* dev/ino) as owned by foreign_uid during the ancestor walk. False when the
143+
* path is not a directory. Clear restores real ownership. */
144+
bool cbm_daemon_ipc_posix_set_foreign_owned_dir_for_test(const char *directory_path,
145+
unsigned long foreign_uid);
146+
void cbm_daemon_ipc_posix_clear_foreign_owned_dir_for_test(void);
130147
bool cbm_daemon_ipc_posix_uid_map_is_single_uid_for_test(const char *uid_map, unsigned long euid);
131148
bool cbm_daemon_ipc_posix_ancestor_stat_ok_for_test(unsigned long owner, unsigned int mode,
132149
unsigned long euid, bool overflow_active,

‎src/main.c‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1455,6 +1455,8 @@ static main_build_identity_status_t main_build_identity(cbm_daemon_build_identit
14551455
if (!cbm_daemon_ipc_private_directory_secure(canonical_cache)) {
14561456
return MAIN_BUILD_IDENTITY_CACHE_PRIVATE;
14571457
}
1458+
/* #1717: advice only -- a network-filesystem cache root is accepted. */
1459+
(void)cbm_daemon_ipc_warn_if_network_cache_root(canonical_cache);
14581460
/* Every cache consumer in this process must use the exact path whose
14591461
* fingerprint joins the account-wide cohort. Keeping an original symlink
14601462
* spelling in the environment would let a later retarget move storage

0 commit comments

Comments
 (0)