Skip to content

Commit e24b200

Browse files
cknittclaude
andcommitted
Keep watching a source folder when an overlapping one is removed
Rust rewatch (`rescript-rust`) watches each source folder separately, and source folders may overlap. On a full rebuild it unwatched the paths that went away and registered only the new ones. With inotify on Linux, notify keeps one watch per directory, shared by every registered path that reaches it, so unwatching a removed recursive folder also dropped a folder that is still configured, and edits there were no longer compiled. Outside macOS, every configured path is now registered again after the stale ones are unwatched; registering an already watched directory only updates its watch, so unchanged watches have no gap. macOS keeps registering only the new paths, because FSEvents restarts its event stream on every watch change and its watches don't depend on each other. Example (text): rescript.json of a package, while `rescript-rust watch` runs: "sources": [{"dir": "src", "subdirs": true}, "src/retained"] changed to "sources": ["src", "src/retained"] then src/retained/Retained.res is edited. Before (text): The edit isn't compiled (Linux). After (text): The edit is compiled. Source: rewatch/tests/watch/08-watch-overlapping-source-removed.sh, run against master's rescript-rust (fails 2 of 2) and this branch's (passes). Changelog entry: - Rewatch: keep watching a source folder on Linux when an overlapping recursive source folder is removed from `rescript.json`. #8839 Review: Is OCaml rewatch (`rescript`) affected? No. It resolves every watched folder to its canonical directory, keeps one libuv handle per directory, and on a refresh closes only the directories no longer wanted. The new test runs against both executables in the shared suite and passes for `rescript` on master already. Review: Why not keep the incremental update everywhere? Watches can share inotify state through overlapping folders, `..` and symlinks, including directory symlinks inside a recursive folder; working out every way an unwatch can drop another watch took several review rounds. Registering every path again needs no such reasoning and costs no gap on inotify. Review: Why does macOS keep the incremental update? #8730 introduced it because FSEvents restarts its event stream on every watch or unwatch call and loses the events in between. FSEvents watches are independent paths, so an unwatch never drops another watch there. Area: rewatch tests Changelog: fix Output: unchanged Tests: +rewatch/tests/watch/08-watch-overlapping-source-removed.sh ~rewatch/tests/suite.sh PR: #8839 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BC5F5QPeQh5cHBjGBnvJ2T Signed-off-by: Christoph Knittel <ck@cca.io>
1 parent 735a485 commit e24b200

4 files changed

Lines changed: 145 additions & 5 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Rewatch: keep watching a source folder on Linux when an overlapping recursive source folder is removed from `rescript.json`. https://github.com/rescript-lang/rescript/pull/8839

‎rewatch/src/watcher.rs‎

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -216,12 +216,32 @@ fn register_watches(
216216
registered
217217
}
218218

219-
/// Brings the registered watches from `current` to `next`, touching only the paths that changed.
219+
/// Whether `update_watches` registers only the new paths, leaving unchanged watches alone, rather
220+
/// than registering every path again (see `update_watches` for why this differs per platform).
221+
const REGISTER_ONLY_NEW_WATCHES: bool = cfg!(target_os = "macos");
222+
223+
/// Brings the registered watches from `current` to `next`, which happens on every full rebuild:
224+
/// after a `rescript.json` change, but also when a source file is added, removed or renamed.
225+
///
226+
/// Stale watches are unwatched first: those whose path is no longer in `next`, and those whose
227+
/// directory was replaced since it was registered. What happens to the other watches depends on
228+
/// how the platform's watcher backend (from the `notify` crate) treats overlapping watches:
229+
///
230+
/// - **macOS (FSEvents):** every `watch`/`unwatch` call restarts the event stream, and events that
231+
/// happen during the restart are lost. Each registered path is watched independently, so
232+
/// unwatching one path never affects another. Only the new paths are registered, so a full
233+
/// rebuild that keeps the same paths doesn't restart the stream at all.
220234
///
221-
/// Some backends restart their event stream on every `watch`/`unwatch` call (FSEvents on macOS
222-
/// does), and events that happen during the restart are lost. Leaving unchanged watches alone
223-
/// avoids those gaps entirely in the common case where a full rebuild keeps the same paths.
224-
/// Watches whose directory was replaced since registration are registered again.
235+
/// - **Linux (inotify) and other platforms:** notify keeps one watch per directory, shared by
236+
/// every registered path that reaches it: overlapping source folders, other spellings of a
237+
/// folder (`..`, symlinks) and directory symlinks inside a recursive folder. Unwatching a stale
238+
/// path can therefore also drop a watch that is still needed (#8825). So every path in `next` is
239+
/// registered again after the unwatches. Registering a directory that is already watched only
240+
/// updates its existing watch, so watches that weren't dropped keep reporting events without a
241+
/// gap, and those that were dropped are restored.
242+
///
243+
/// Windows takes this path too. It has no stable directory identity, so every watch counts as
244+
/// stale there and is unwatched and registered again on each full rebuild.
225245
///
226246
/// Returns the watches that are registered afterwards, which excludes paths that failed to register.
227247
fn update_watches(
@@ -236,6 +256,9 @@ fn update_watches(
236256
for watch in stale {
237257
let _ = watcher.unwatch(&watch.path);
238258
}
259+
if !REGISTER_ONLY_NEW_WATCHES {
260+
return register_watches(watcher, next);
261+
}
239262
let added: WatchPaths = next
240263
.iter()
241264
.filter(|(path, mode)| {

‎rewatch/tests/suite.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,7 @@ fi
150150
./watch/05-watch-ignores-non-source.sh &&
151151
./watch/06-watch-missing-source-folder.sh &&
152152
./watch/07-changed-interface-after-failed-implementation.sh &&
153+
./watch/08-watch-overlapping-source-removed.sh &&
153154

154155
# Lock tests
155156
./lock/01-lock-when-watching.sh &&
Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
#!/bin/bash
2+
cd $(dirname $0)
3+
source "../utils.sh"
4+
cd ../../testrepo
5+
6+
bold "Test: Watcher keeps watching a folder when an overlapping source folder is removed"
7+
8+
error_output=$(rewatch clean 2>&1)
9+
if [ $? -eq 0 ];
10+
then
11+
success "Repo Cleaned"
12+
else
13+
error "Error Cleaning Repo"
14+
printf "%s\n" "$error_output" >&2
15+
exit 1
16+
fi
17+
18+
DEP01_CONFIG=packages/dep01/rescript.json
19+
RETAINED=packages/dep01/src/retained
20+
21+
cleanup() {
22+
git restore --worktree -- "$DEP01_CONFIG"
23+
rm -rf "$RETAINED"
24+
rm -f rewatch.log
25+
}
26+
27+
# src/retained is watched both through the recursive src folder and as a
28+
# source folder of its own
29+
mkdir -p "$RETAINED"
30+
echo 'let value = 1' > "$RETAINED/Retained.res"
31+
node -e "
32+
const fs = require('fs');
33+
const config = JSON.parse(fs.readFileSync('$DEP01_CONFIG', 'utf8'));
34+
config.sources = [{dir: 'src', subdirs: true}, 'src/retained'];
35+
fs.writeFileSync('$DEP01_CONFIG', JSON.stringify(config, null, 2) + '\n');
36+
"
37+
38+
retained_output() {
39+
find "$RETAINED" -name 'Retained.*js' | head -n 1
40+
}
41+
42+
rewatch_bg watch > rewatch.log 2>&1 &
43+
success "Watcher Started"
44+
45+
wait_for_retained_value() {
46+
local value=$1
47+
local timeout=30
48+
while [ "$timeout" -gt 0 ]; do
49+
output=$(retained_output)
50+
if [ -n "$output" ] && grep -q "$value" "$output"; then
51+
return 0
52+
fi
53+
sleep 1
54+
timeout=$((timeout - 1))
55+
done
56+
return 1
57+
}
58+
59+
if ! wait_for_retained_value "= 1"; then
60+
error "Initial build did not compile the retained folder"
61+
cat rewatch.log
62+
exit_watcher
63+
cleanup
64+
exit 1
65+
fi
66+
success "Initial build completed"
67+
68+
completed_builds=$(grep -c "Finished .*compilation" rewatch.log 2>/dev/null || true)
69+
70+
# Remove the recursive src folder while keeping src itself (non-recursive)
71+
# and src/retained
72+
node -e "
73+
const fs = require('fs');
74+
const config = JSON.parse(fs.readFileSync('$DEP01_CONFIG', 'utf8'));
75+
config.sources = ['src', 'src/retained'];
76+
fs.writeFileSync('$DEP01_CONFIG', JSON.stringify(config, null, 2) + '\n');
77+
"
78+
if ! wait_for_pattern_count rewatch.log "Finished .*compilation" \
79+
$((completed_builds + 1)) 30; then
80+
error "No rebuild after the configuration change"
81+
cat rewatch.log
82+
exit_watcher
83+
cleanup
84+
exit 1
85+
fi
86+
87+
echo 'let value = 2' > "$RETAINED/Retained.res"
88+
if wait_for_retained_value "= 2"; then
89+
success "Edit in the retained folder was compiled"
90+
else
91+
error "Edit in the retained folder was not picked up"
92+
cat rewatch.log
93+
exit_watcher
94+
cleanup
95+
exit 1
96+
fi
97+
98+
if ! exit_watcher; then
99+
cleanup
100+
exit 1
101+
fi
102+
cleanup
103+
104+
# Rebuild to regenerate the outputs removed by `rewatch clean`
105+
rewatch build > /dev/null 2>&1
106+
107+
if git diff --exit-code . > /dev/null 2>&1 && [ -z "$(git ls-files --others --exclude-standard .)" ];
108+
then
109+
success "No leftover changes"
110+
else
111+
error "Leftover changes detected"
112+
git diff .
113+
git ls-files --others --exclude-standard .
114+
exit 1
115+
fi

0 commit comments

Comments
 (0)