Replace lib/bs and lib/ocaml with a flat lib/rescript (#9245)

* Write each package's build artifacts flat to lib/rescript

Why:
    Every artifact existed in lib/bs, nested like the sources, and again in
    lib/ocaml, flat, and lib/bs also held copies of the sources and the
    generated JavaScript (#9151). The build now writes one flat lib/rescript.

Layout modified (artifact directory of a package):
Before (text):
    lib/bs/src/A.ast  lib/bs/src/A.cmi  lib/bs/src/A.res  lib/bs/src/A.js
    lib/bs/.compiler.log  lib/bs/compiler-info.json  lib/bs/.sourcedirs.json
    lib/bs/build.ninja
    lib/ocaml/A.ast  lib/ocaml/A.cmi  lib/ocaml/A.cmj  lib/ocaml/A.cmt  lib/ocaml/A.res
    lib/build.lock  lib/watch.lock
After (text):
    lib/rescript/A.ast  lib/rescript/A.cmi  lib/rescript/A.cmj  lib/rescript/A.cmt
    lib/rescript/.compiler.log  lib/rescript/compiler-info.json  lib/rescript/.sourcedirs.json
    lib/rescript/build.lock  lib/rescript/watch.lock

Before (text): bsc runs in lib/bs with -I ../ocaml; the stdlib is <runtime>/lib/ocaml
After (text): bsc runs in lib/rescript with -I .; the stdlib is <runtime>/lib/rescript

Publication modified (a compiled module's ASTs, interface and typed tree):
Before (text): written to lib/bs by bsc, then copied to lib/ocaml
After (text):
    held in the build file store until the request succeeded, then
    1. the interface, only when it changed
    2. the typed tree, which marks the module compiled
    each renamed into place, or written in place when the rename fails
    a failed compile: interface forgotten, AST dated back, partial typed tree published

Freshness modified (Build_freshness, Compiler_info, namespace maps):
Before (representation):
    source removed or must be parsed again ↦ AST removed
    no compiler-info.json, lib/ocaml/.compiler.log exists ↦ clean
    Ns.mlmap changed or Ns outputs missing ↦ compile namespace
After (representation):
    source must be parsed again ↦ AST dated back to time 1, always parsed again
    compiler-info.json written once the clean decision is made
    compile namespace ↦ Ns.cmt removed until the compile publishes it
    lib/bs or lib/ocaml exists ↦ clean, which removes them

The compiler keeps early cutoff with one copy of each interface: it
replaces an interface it cannot read instead of failing, and a .cmj cut
short after its digest no longer counts as unchanged. Stale public
JavaScript is removed when an AST proves the build generated it; the JS
mirror and the build.ninja marker are gone. Clean empties lib/rescript but
for the lock files.

Review: Why hold ASTs as well as interfaces? A failed parse used to leave its AST in lib/bs only; with one copy it would look current and the next build would skip the parse and lose the error.
Review: Why rename into place, with an in-place write only as a fallback? A build killed during an in-place write leaves a truncated file, and a truncated typed tree with a fresh time counts as compiled for good. On Windows the rename fails while an editor has the file open without delete sharing, so the file is then written in place.
Review: Why date an AST back instead of removing it? A later compile of the same build or watch session may still read it. Time 1 is below any real source time except the epoch, which is why such an AST is always parsed again.
Review: What about a removed module's working .cmi, which cleanup deferred? It went to lib/bs, outside the include path. In lib/rescript it would let dependents compile against a deleted module, so it is removed at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gGBR9g3DD93vrWbYzU
Signed-off-by: Christoph Knittel <ck@cca.io>

* Read compiled files from the flat lib/rescript in analysis and gentype

Why:
    The tools found a module's compiled files through the nested lib/bs
    mirror of the sources and the source copies in lib/ocaml, which the
    previous commit removes.

Each tool now looks a module up by its compiled file name:
- analysis reads <pkg>/lib/rescript/<Module>[-Ns].cmt, the stdlib's compiled
  files from <runtime>/lib/rescript and its sources from the runtime package
  root, .sourcedirs.json from lib/rescript, and incremental files from
  lib/rescript/incremental;
- reanalyze scans each package's lib/rescript, and its -cmt-root walk skips
  the editor's incremental directory;
- gentype finds a dependency module's source directory in the header of its
  AST, read once per build, and maps a namespace module to its package root;
  it looks an imported module of the same package up as <Name>[-Ns].cmt,
  trying the compiler's spelling first, and reads typed trees through the
  build file store, which holds the one being compiled, without keeping
  them.

Review: Why does the reanalyze snapshot move one line? The scan visited the nested source directories first and the build root last; one flat directory is scanned in name order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gGBR9g3DD93vrWbYzU
Signed-off-by: Christoph Knittel <ck@cca.io>

* Ship the runtime's lib/rescript and sources, and update CI, build tests and docs

Why:
    The runtime package shipped lib/ocaml with copies of its sources; it now
    ships lib/rescript and its own sources at the package root, which
    analysis reads (#9151).

The release artifact, its CI upload and the playground's cmij extraction
take lib/rescript, its build state stays out of the package, and the
ignore files name lib/rescript. Build tests read the in-source outputs that
the JS mirror used to copy, and look for the debug IR, the belt interfaces
and post-build paths under lib/rescript; the installation test looks for
the watch lock there. The changelog fragment lists the new paths for tools
that read the old ones and says that lib/rescript belongs in .gitignore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gGBR9g3DD93vrWbYzU
Signed-off-by: Christoph Knittel <ck@cca.io>

---------

Signed-off-by: Christoph Knittel <ck@cca.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Author: cknitt

.github/workflows/ci.yml

@@ -326,15 +326,16 @@ jobs:
           name: binaries-${{ matrix.node-target }}
           path: packages/@rescript/${{ matrix.node-target }}/bin
 
-      - name: "Upload artifacts: lib/ocaml"
+      - name: "Upload artifacts: lib/rescript"
         if: matrix.upload_libs
         uses: actions/upload-artifact@v7
         with:
-          name: lib-ocaml
+          name: lib-rescript
           path: |
-            packages/@rescript/runtime/lib/ocaml
-            !packages/@rescript/runtime/lib/ocaml/*.ast
-            !packages/@rescript/runtime/lib/ocaml/*.iast
+            packages/@rescript/runtime/lib/rescript
+            !packages/@rescript/runtime/lib/rescript/*.ast
+            !packages/@rescript/runtime/lib/rescript/*.iast
+            !packages/@rescript/runtime/lib/rescript/compiler-info.json
 
       - name: Select API docs website branch
         id: api-docs-plan
@@ -423,7 +424,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v8
         with:
-          pattern: "@(binaries-*|lib-ocaml)"
+          pattern: "@(binaries-*|lib-rescript)"
 
       - name: Move artifacts into packages
         run: .github/workflows/moveArtifacts.sh

.github/workflows/moveArtifacts.sh

@@ -26,7 +26,7 @@ check_statically_linked() {
 }
 
 # rescript
-mv lib-ocaml packages/@rescript/runtime/lib/ocaml
+mv lib-rescript packages/@rescript/runtime/lib/rescript
 
 # @rescript/{target}
 chmod +x binaries-*/*.exe

.github/workflows/publish.yml

@@ -112,7 +112,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v8
         with:
-          pattern: "@(binaries-*|lib-ocaml)"
+          pattern: "@(binaries-*|lib-rescript)"
 
       - name: Move artifacts into packages
         run: .github/workflows/moveArtifacts.sh

.gitignore

@@ -49,7 +49,7 @@ node_modules
 _coverage/
 bisect*.coverage
 
-lib/ocaml
+lib/rescript
 tests/build_tests/*/lib/
 #ignore temporary directory
 *.goog.js

CONTRIBUTING.md

@@ -262,7 +262,7 @@ To view the typed tree of the file run:
 ### Project
 
 ```sh
-make artifacts # Build compiler and standard library and populate lib/ocaml
+make artifacts # Build compiler and standard library and populate lib/rescript
 npm link
 cd myProject
 npm install

analysis/bin/main.ml

@@ -64,7 +64,7 @@ Options:
 
   createInterface: print to stdout the interface file for src/MyFile.res
 
-    ./rescript-editor-analysis.exe createInterface src/MyFile.res lib/bs/src/MyFile.cmi
+    ./rescript-editor-analysis.exe createInterface src/MyFile.res lib/rescript/MyFile.cmi
 
   format: print to stdout the formatted version of the provided file
 
@@ -127,10 +127,10 @@ let main () =
     let uri = Uri.from_path root_path in
     match Packages.find_root ~uri (Hashtbl.create 0) with
     | Some (`Bs root_path) -> (
-      match Build_system.get_lib_bs root_path with
+      match Build_system.get_artifact_dir root_path with
       | None -> print_endline "\"ERR\""
-      | Some lib_bs ->
-        Cache.delete_cache (Cache.target_file_from_lib_bs lib_bs);
+      | Some artifact_dir ->
+        Cache.delete_cache (Cache.target_file_from_artifact_dir artifact_dir);
         print_endline "\"OK\"")
     | _ -> print_endline "\"ERR: Did not find root \"")
   | [_; "completion"; path; line; col; current_file] ->

analysis/reanalyze/README.md

@@ -17,7 +17,7 @@ Dead code analysis and other experimental analyses for ReScript.
 rescript-tools reanalyze -config
 
 # Run DCE analysis on specific CMT directory
-rescript-tools reanalyze -dce-cmt path/to/lib/bs
+rescript-tools reanalyze -dce-cmt path/to/lib/rescript
 
 # Run all analyses
 rescript-tools reanalyze -all

analysis/reanalyze/src/paths.ml

@@ -133,14 +133,15 @@ type cmt_scan_entry = {
     This is a v2 extension produced by `rewatch` to support monorepos without requiring
     reanalyze-side package resolution.
 
-    The scan plan is a list of build roots (usually `<pkg>/lib/bs`) relative to the project root,
+    The scan plan is a list of build roots (`<pkg>/lib/rescript`) relative to the project root,
     plus a list of subdirectories (relative to that build root) to scan for `.cmt/.cmti`.
+    Build roots are flat, so the subdirectories are empty and each build root is scanned itself.
 
     If missing, returns the empty list and callers should fall back to legacy behavior. *)
 
 let read_cmt_scan () =
   let source_dirs_file =
-    ["lib"; "bs"; ".sourcedirs.json"]
+    ["lib"; "rescript"; ".sourcedirs.json"]
     |> List.fold_left Filename.concat run_config.bsb_project_root
   in
   let get_fn key fn json =

analysis/reanalyze/src/reanalyze.ml

@@ -15,6 +15,9 @@ let collect_cmt_file_paths ~cmt_root : string list =
       let skip_dir =
         let base = Filename.basename dir in
         base = "node_modules" || base = "_esy"
+        (* The editor's copies of compiled files. *)
+        || base = "incremental"
+           && Filename.basename (Filename.dirname dir) = "rescript"
       in
       if (not skip_dir) && Sys.file_exists abs_dir then
         if Sys.is_directory abs_dir then

analysis/src/build_system.ml

@@ -40,9 +40,12 @@ let get_runtime_dir root_path =
         None))
   | true -> Some root_path
 
-let get_lib_bs path = Files.if_exists (path /+ "lib" /+ "bs")
+(* A package's compiled files, flat in lib/rescript. *)
+let get_artifact_dir path = Files.if_exists (path /+ "lib" /+ "rescript")
 
+(* The runtime's compiled files. Its sources are at the root of the runtime
+   package, see [get_runtime_dir]. *)
 let get_stdlib base =
   match get_runtime_dir base with
   | None -> None
-  | Some runtime_dir -> Some (runtime_dir /+ "lib" /+ "ocaml")
+  | Some runtime_dir -> Some (runtime_dir /+ "lib" /+ "rescript")