Skip to content

Commit 00e3f40

Browse files
authored
Skip unneeded work in occur checks and primitive specialization (#9193)
* Try a cheap bounded walk before the full occur checks Why: Every variable link in unification runs `occur`, and the Tvar/Tconstr cases also run `deep_occur`. `occur` builds an AVL set of visited nodes along every path and `deep_occur` marks every node and walks the type again to unmark it, although almost all of these walks see a handful of nodes. On a 552 module React app they make 943k and 494k calls, about 0.5 s of 14 s single domain CPU. Both checks now first walk the type with no visited set and no marks, giving up after 256 nodes (which also stops cycles and heavy sharing); when the walk gives up, the unchanged full walk runs. - `occur` in expression mode: a walk over the edges `occur_rec` follows that finishes without meeting the variable proves `occur_rec` would neither raise nor expand anything. Pattern mode, which looks types up, always runs the full walk. - `deep_occur`: the walk only answers "absent". It gives up on meeting the variable and on a node an enclosing walk already marked, so the marks left behind are the same as the full walk's. Review: Can an answer, a diagnostic or an artifact change? No: each fast walk either proves the full walk would do nothing more than it did, or gives up and the full walk runs from scratch. The new fixtures pin the errors, including one past the budget, and the corpus (5,398 artifacts and 1,147 .cmt/.cmti) is byte identical to master. Signed-off-by: Jono Prest <jjprest@gmail.com> * Skip scraping a primitive's type when no specialization reads it Why: `specialize_primitive` scraped the primitive's whole function type (a deep copy plus head expansion) on every use, but only unified operators and comparisons look at it. For the other 95% of uses (on a 552 module React app, 21,582 of 22,770) the result was thrown away and `find_builtin` answered. For a function type whose primitive is in neither table, it now calls `find_builtin` directly. Review: Could the skipped scrape have had an effect? Scraping a function type looks nothing up: only a Tconstr head is expanded. What is left is a copy with fresh type ids, made during Lambda translation after the .cmi and .cmt are written, whose result nothing read. The corpus artifacts and .cmt files are byte identical to master. Signed-off-by: Jono Prest <jjprest@gmail.com> * Build the occur walk's closure once per call Why: `occur_reaches` passed `occur_reaches budget ty0` to `iter_type_expr` at every node, so the fast walk allocated a partial application per node it visited. `occur_reaches` now defines its walk once per call, capturing the budget and the variable, as `deep_occur_fast_absent` already does. The edges it follows and the exceptions it raises are unchanged. Measured with scripts/compilerBenchmarks.js against the previous commit: allocated words -1.58% in total, stress/TypeAliasChain.res -53%, Belt and runtime files -2% to -2.7%. On a 552 module React app (clean build, one domain) the build allocates 0.05% fewer words. A version that returns a bool instead of raising allocated more than this one (-1.27% in total), because `List.for_all` takes a closure per list. Signed-off-by: Jono Prest <jjprest@gmail.com> * Walk the occur fast paths without allocating Why: The bounded walks that `occur` and `deep_occur` try first allocated a ref for their budget and a closure for the walk on every call, and `iter_type_expr` allocated another closure at each arrow type they visited. Unification runs `occur` for every variable it links, mostly on small types, so this allocation was a large share of what the fast paths cost. `occur_reaches` and `deep_occur_fast_absent` now walk the type with mutually recursive functions that take the budget left and return what remains, following the edges of `iter_type_expr` written out by constructor. The match on `type_desc` lists every constructor, so a new one is a compile error here rather than a missed edge. Only `deep_occur_fast_absent` at a `Tvariant` still goes through `iter_type_expr` with a closure, for the row's fields. The nodes visited, the order, the budget and the exceptions raised are unchanged. Measured with scripts/compilerBenchmarks.js against the previous commit: allocated words -1.03% in total (-1.14% without the project builds), stress/TypeAliasChain.res -8.3%, Belt map and set modules -2.7% to -3.4%. On a 552 module React app (clean build, one domain) the build allocates 0.99% fewer words. Keeping the closures and only skipping the walk for a leaf type (a variable or a constructor without arguments) gave -0.57% in total and -0.54% on the app. Review: Can a result change? No: a walk that finishes visited exactly the nodes the closure version visited, within the same budget; one that gives up still falls back to the full check. Signed-off-by: Jono Prest <jjprest@gmail.com> --------- Signed-off-by: Jono Prest <jjprest@gmail.com>
1 parent dcedd5d commit 00e3f40

9 files changed

Lines changed: 1082 additions & 4 deletions
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Build: skip unneeded work in the type checker's occur checks and when specializing primitives.

‎compiler/ml/ctype.ml‎

Lines changed: 104 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1380,6 +1380,7 @@ let is_contractive env p =
13801380

13811381
exception Occur
13821382

1383+
(* [occur_reaches] must follow the same edges in expression mode. *)
13831384
let rec occur_rec env allow_recursive visited ty0 = function
13841385
| {desc = Tlink ty} -> occur_rec env allow_recursive visited ty0 ty
13851386
| ty -> (
@@ -1413,7 +1414,57 @@ let type_changed () = Domain.DLS.get type_changed_key
14131414

14141415
let merge r b = if b then r := true
14151416

1416-
let occur env ty0 ty =
1417+
(* Cheap first passes for [occur] and [deep_occur]: walks with no visited
1418+
set and no marks that give up (raise [Give_up]) after [walk_budget]
1419+
nodes, which also stops cycles and heavy sharing, or when they cannot
1420+
decide; the caller then runs the full walk. A pass that finishes has
1421+
visited every node the full walk would. *)
1422+
exception Give_up
1423+
1424+
let walk_budget = 256
1425+
1426+
(* Must follow the same edges as [occur_rec] in expression mode. Finishing
1427+
proves [occur_rec] would neither meet [ty0] nor find a [Tconstr] cycle,
1428+
its only ways to raise, and it expands types only after raising. The
1429+
edges are those of [iter_type_expr], spelled out so that the walk
1430+
allocates nothing: it returns the budget left instead of keeping it in a
1431+
ref that a closure captures. *)
1432+
let rec occur_reaches_budget ty0 budget ty =
1433+
match ty.desc with
1434+
| Tlink ty -> occur_reaches_budget ty0 budget ty
1435+
| desc -> (
1436+
if ty == ty0 then raise Occur;
1437+
let budget = budget - 1 in
1438+
if budget < 0 then raise Give_up;
1439+
match desc with
1440+
| Tobject _ | Tvariant _ | Tvar _ | Tnil | Tunivar _ -> budget
1441+
| Tarrow (params, ret) ->
1442+
occur_reaches_budget ty0 (occur_reaches_params ty0 budget params) ret
1443+
| Ttuple l | Tconstr (_, l, _) | Tpackage (_, _, l) ->
1444+
occur_reaches_list ty0 budget l
1445+
| Tfield {typ; rest} ->
1446+
occur_reaches_budget ty0 (occur_reaches_budget ty0 budget typ) rest
1447+
| Tlink ty | Tsubst ty -> occur_reaches_budget ty0 budget ty
1448+
| Tpoly (ty, tyl) ->
1449+
occur_reaches_list ty0 (occur_reaches_budget ty0 budget ty) tyl)
1450+
1451+
and occur_reaches_list ty0 budget = function
1452+
| [] -> budget
1453+
| ty :: l -> occur_reaches_list ty0 (occur_reaches_budget ty0 budget ty) l
1454+
1455+
and occur_reaches_params ty0 budget = function
1456+
| [] -> budget
1457+
| {typ} :: l ->
1458+
occur_reaches_params ty0 (occur_reaches_budget ty0 budget typ) l
1459+
1460+
let occur_reaches ty0 ty = ignore (occur_reaches_budget ty0 walk_budget ty)
1461+
1462+
let occur_cannot_fail ty0 ty =
1463+
match occur_reaches ty0 ty with
1464+
| () -> true
1465+
| exception (Occur | Give_up) -> false
1466+
1467+
let occur_full env ty0 ty =
14171468
let allow_recursive = !(umode ()) = Pattern in
14181469
let old = !(type_changed ()) in
14191470
try
@@ -1432,6 +1483,12 @@ let occur env ty0 ty =
14321483
| Occur -> Unify []
14331484
| _ -> exn)
14341485

1486+
(* In pattern mode [occur_rec] looks types up ([is_contractive]), which can
1487+
load interfaces, so only expression mode takes the shortcut. *)
1488+
let occur env ty0 ty =
1489+
if !(umode ()) = Expression && occur_cannot_fail ty0 ty then ()
1490+
else occur_full env ty0 ty
1491+
14351492
let occur_in env ty0 t =
14361493
try
14371494
occur env ty0 t;
@@ -1657,8 +1714,48 @@ let mkvariant fields closed =
16571714

16581715
(**** Unification ****)
16591716

1660-
(* Return whether [t0] occurs in [ty]. Objects are also traversed. *)
1661-
let deep_occur t0 ty =
1717+
(* Must follow the same edges as [deep_occur_marked]. Answers only "absent":
1718+
it gives up on meeting [t0] (where the marked walk stops early) and on a
1719+
node an enclosing walk marked (which the marked walk skips but then
1720+
unmarks in [unmark_type]), so it never leaves marks that differ. Like
1721+
[occur_reaches_budget], it spells out the edges of [iter_type_expr] so
1722+
that it allocates nothing, except at a [Tvariant]. *)
1723+
let rec deep_occur_budget t0 budget ty =
1724+
let ty = repr ty in
1725+
if ty.level < lowest_level then raise Give_up;
1726+
if ty == t0 then raise Occur;
1727+
let budget = budget - 1 in
1728+
if budget < 0 then raise Give_up;
1729+
match ty.desc with
1730+
| Tvar _ | Tnil | Tunivar _ -> budget
1731+
| Tarrow (params, ret) ->
1732+
deep_occur_budget t0 (deep_occur_params t0 budget params) ret
1733+
| Ttuple l | Tconstr (_, l, _) | Tpackage (_, _, l) ->
1734+
deep_occur_list t0 budget l
1735+
| Tobject ty | Tlink ty | Tsubst ty -> deep_occur_budget t0 budget ty
1736+
| Tfield {typ; rest} ->
1737+
deep_occur_budget t0 (deep_occur_budget t0 budget typ) rest
1738+
| Tpoly (ty, tyl) -> deep_occur_list t0 (deep_occur_budget t0 budget ty) tyl
1739+
| Tvariant _ ->
1740+
let budget = ref budget in
1741+
iter_type_expr (fun ty -> budget := deep_occur_budget t0 !budget ty) ty;
1742+
!budget
1743+
1744+
and deep_occur_list t0 budget = function
1745+
| [] -> budget
1746+
| ty :: l -> deep_occur_list t0 (deep_occur_budget t0 budget ty) l
1747+
1748+
and deep_occur_params t0 budget = function
1749+
| [] -> budget
1750+
| {typ} :: l -> deep_occur_params t0 (deep_occur_budget t0 budget typ) l
1751+
1752+
let deep_occur_fast_absent t0 ty =
1753+
match deep_occur_budget t0 walk_budget ty with
1754+
| _ -> true
1755+
| exception (Occur | Give_up) -> false
1756+
1757+
(* [deep_occur_fast_absent] must follow the same edges. *)
1758+
let deep_occur_marked t0 ty =
16621759
let rec occur_rec ty =
16631760
let ty = repr ty in
16641761
if ty.level >= lowest_level then (
@@ -1674,6 +1771,10 @@ let deep_occur t0 ty =
16741771
unmark_type ty;
16751772
true
16761773

1774+
(* Return whether [t0] occurs in [ty]. Objects are also traversed. *)
1775+
let deep_occur t0 ty =
1776+
(not (deep_occur_fast_absent t0 ty)) && deep_occur_marked t0 ty
1777+
16771778
(*
16781779
1. When unifying two non-abbreviated types, one type is made a link
16791780
to the other. When unifying an abbreviated type with a

‎compiler/ml/translcore.ml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -412,7 +412,7 @@ let specialize_comparison
412412
(* Specialize a primitive from available type information,
413413
raise Not_found if primitive is unknown *)
414414

415-
let specialize_primitive p env ty (* ~has_constant_constructor *) =
415+
let specialize_typed_primitive p env ty (* ~has_constant_constructor *) =
416416
let fn_expr = is_function_type env ty in
417417
let unified =
418418
match fn_expr with
@@ -429,6 +429,20 @@ let specialize_primitive p env ty (* ~has_constant_constructor *) =
429429
| None -> Primitive table.objcomp
430430
with Not_found -> find_builtin p.prim_name)
431431

432+
(* Only unified operators and comparisons read the primitive's type, so the
433+
others skip scraping it (a copy of the whole function type) when it is a
434+
function type: scraping one then looks nothing up (only a [Tconstr] head
435+
is expanded, and only a [Tconstr] result is looked up), so the copy is
436+
all the skipped work. *)
437+
let specialize_primitive p env ty =
438+
match (Btype.repr ty).desc with
439+
| Tarrow _
440+
when not
441+
(Hashtbl.mem Unified_ops.index_by_name p.prim_name
442+
|| Hashtbl.mem comparisons_table p.prim_name) ->
443+
find_builtin p.prim_name
444+
| _ -> specialize_typed_primitive p env ty
445+
432446
(* [is_unit] excluded: unit shares the undefined constant but is not a
433447
[%null] / [%undefined] literal, and comparing it never suppressed the
434448
warning. *)
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
We've found a bug for you!
3+
/.../fixtures/occurs_check_function_type.res:4:22
4+
5+
2 │ contains it. */
6+
3 │ let f = (x: 'a) => {
7+
4 │ let y: 'a => int = x
8+
5 │ y
9+
6 │ }
10+
11+
This value might need to be wrapped in a function that takes an extra
12+
parameter of type 'a
13+
14+
Here's the original error message
15+
This has type: 'a
16+
But it's expected to have type: 'a => int
17+
The type variable 'a occurs inside 'a => int

0 commit comments

Comments
 (0)