Repository navigation
dev-env: ARM-Mac VSCode dev environment with F5 debugging (OTP 26/27) - #62
Merged
Merged
Conversation
Scaffold Matt Pocock's engineering-skills config: AGENTS.md with the Agent skills block plus docs/agents/ for issue-tracker (GitHub via gh), triage-labels (canonical defaults), and domain-docs (single-context).
Add a platform-wide glossary (CONTEXT.md) of the load-bearing OpenKazoo concepts, plus the first two ADRs recording the hard fork from 2600hz/kazoo and the Erlang/OTP 27 runtime target.
Salvaged from PR #10 and corrected against a real ARM Mac run: - pin couchdb:3.5.2 and rabbitmq:3.13.7-management (both arm64; rabbitmq matches the tag CI already uses) - drop the obsolete `version:` key and CouchDB's 5986 port, removed in CouchDB 3.2 - reconcile CouchDB credentials to admin/admin, per the dev-env map - add a couchdb-init one-shot creating the _users / _replicator / _global_changes system databases (n=1 q=1), without which a fresh CouchDB 3 never reaches cluster_finished - pin the RabbitMQ hostname so Mnesia keeps its node name and the volume survives container recreation - healthcheck CouchDB on /_up rather than /, which answers before ready Part of #27. Closes #30.
The prod profile strips debug_info (no_debug_info), which int-based step debugging needs, so a debuggable local release needs its own profile. Add a dev profile keeping debug_info, with relx dev_mode so applications are symlinked into the release (edit + recompile without reassembling) and include_erts false so the node runs on the OTP already on PATH -- the same runtime the editor's language server uses. Wire it to a build-dev target producing _build/dev/rel/kazoo. Refs #31
…se start types (#40) Supplies the three files the dev release needs in order to boot, and fixes the release manifest defect that stopped it booting at all. config/config-dev.ini - Kazoo runtime config pointed at docker-compose.dev.yml (AMQP guest@127.0.0.1:5672, CouchDB 127.0.0.1:5984 admin/admin), selected via KAZOO_CONFIG. config/vm.args.dev - fixed longname kazoo_apps@127.0.0.1 + cookie, so remote_console and the els_dap attach are deterministic. Selected via VMARGS_PATH. config/sys-dev.config - lager handlers matching the names kz_log looks for, so the INI's [log] levels actually apply. Selected via RELX_CONFIG_PATH. Per the decision on #32 these are passed as environment variables at launch rather than wired into rebar.config as relx pointers, leaving the release itself unchanged. Three things that are easy to get wrong, all documented in the files: - All three paths must be ABSOLUTE. The start script cd's to the release root before exec'ing erlexec, so a relative path resolves against _build/dev/rel/kazoo. A relative VMARGS_PATH appears to work because the script's own awk pass reads it relative to the shell -- it then fails at boot. - sys-dev.config, not sys.config.dev: erl's -config appends ".config" unless the name already ends in it, so sys.config.dev is unloadable. - vm.args' -setcookie must match [kazoo_apps] cookie. It is only the initial cookie; kazoo_apps_init:set_cookie/0 resets the live one from the INI at boot, so a mismatch locks remote_console out from that point on. The rebar.config change is separate from the above and fixes a defect that affects the prod release equally: crypto, public_key, ssl, asn1, runtime_tools, syntax_tools and compiler were all marked `none` in the relx release list, which includes an application without starting it. OTP will not start an application whose dependencies are merely loaded, so crypto being unstarted stopped kazoo_stdlib, and with it every kazoo app; syntax_tools/compiler likewise stopped goldrush and so lager. The boot script reported {started,started} throughout, so `bin/kazoo foreground` produced a bare Erlang node with no Kazoo in it and no log line explaining why. os_mon stays `none` -- nothing in the release declares it. Verified on a clean boot against docker-compose.dev.yml on an ARM Mac: 133 applications running, config-dev.ini parsed, AMQP connected, CouchDB reachable (kz_datamgr:db_info/0 lists the databases), node name and live cookie both kazoo_apps@127.0.0.1 / kazoo_dev_cookie, and console/error/crash logs written.
Kazoo-level database bootstrap for the local dev node: register views,
refresh every database, create the master account. Re-runnable -- an
existing master account is reported and left alone.
Two pieces. scripts/dev-init.sh is the entry point: it reads the node
name and cookie out of config/vm.args.dev so there is one source of
truth, and needs none of the KAZOO_CONFIG / VMARGS_PATH /
RELX_CONFIG_PATH triple that booting the node requires.
scripts/dev_init.escript does the talking.
The toolchain is checked, never installed -- a missing escript fails
with the requirement (OTP 27) rather than a package-manager incantation.
Nothing here is platform-specific.
Neither `sup' nor `bin/kazoo eval' could do this job:
* core/sup/priv/sup builds its target as Node ++ "@" ++
net_adm:localhost(), so it aims at whatever hostname the machine
advertises and cannot address the dev node, which is named on a
literal IP (kazoo_apps@127.0.0.1). It picks longnames/shortnames
off that same hostname too. The escript takes the host from the
node name it was given instead.
* bin/kazoo eval and rpc go through erl_call, which leaves the
evaluated expression's group leader on the target node --
kapps_maintenance:refresh/0 prints a line per database and every
one of them lands in the node's console log rather than the
caller's terminal. rpc:call/5 from a real Erlang node carries the
group leader across, so progress streams to whoever ran the task.
The CouchDB system databases stay out of scope here: the couchdb-init
one-shot in docker-compose.dev.yml creates them, because CouchDB never
finishes cluster setup without them.
.vscode/tasks.json holds only this task for now; the rest of the flow
arrives with the erlang-ls wiring that owns .vscode/ as a whole.
Remove eight entries from the relx release list. Two different reasons. Seven are auto-resolved: syntax_tools, compiler, runtime_tools, asn1, crypto, public_key and ssl are each a declared dependency of something already in the release -- crypto ~30 kazoo apps declare it public_key kazoo_auth, kazoo_couch, kazoo_attachments, ... asn1 kazoo_web (and the ssl chain) ssl the same chain syntax_tools goldrush (-> lager) and recon compiler goldrush (-> lager) runtime_tools rabbit_common (-> amqp_client) -- so relx resolves them itself and starts them in dependency order. Listing an application only overrides its START TYPE; it is never what gets it into the release. Deleting these seven leaves the generated .rel byte-identical: same 153 applications, all permanent. This finishes what c3ee4ce started. They were introduced, with the rest of the relx block, by acf6475 ("add OTP24 support") -- the rebar2 -> rebar3 migration -- as `{App, none}'. That marking is right for the whapps, which kapps_controller starts at run time via ensure_all_started, but wrong for system libraries the boot script must start before any kazoo app can: `none' overrode their start type to "never start" and broke the dependency chain while still reporting {started,started}. c3ee4ce fixed the boot by listing them as started; not listing them at all is the smaller fix, and it removes config that now reads as if these apps were special when they are not. os_mon goes for the opposite reason: it is the one entry nothing depends on AND nothing uses. It watches the host and reports by raising SASL alarms -- cpu_sup and memsup each spawn an OS port program, disksup shells out to `df' on a timer -- and no module under core/ or applications/ calls cpu_sup, memsup, disksup or os_mon, so those alarms had no consumer. Carried as `{os_mon, none}' it shipped ~8 modules and 2 port binaries that never ran. Since nothing declares it, dropping the entry drops it from the release: 152 applications, and cpu_sup is non_existing on the node. Verified by rebuilding and booting: the node still reaches the same 133 running applications, with all seven auto-resolved apps up and os_mon absent.
Wraps _build/dev/rel/kazoo/bin/kazoo with the environment a dev node
needs, so the launch environment lives in one place instead of being
restated by every VSCode task, launch config and shell.
It carries the KAZOO_CONFIG / VMARGS_PATH / RELX_CONFIG_PATH triple
(absolute, because the start script cd's to the release root before
exec'ing erlexec), and it makes step-debugging possible WITHOUT putting
an interpreter in the release.
`int' -- the code interpreter every Erlang breakpoint runs on -- lives
in OTP's `debugger' application, which the release does not include and
should not: shipping an interpreter to prod is not wanted. Reaching it
from the dev node takes two things, both set only by this script and so
neither able to leak into prod:
-pa <debugger ebin> puts its modules on the code path, resolved at
run time from the same OTP the release was built
against (include_erts=false), so nothing here is
machine-specific.
CODE_LOADING_MODE the release start script defaults to `embedded',
where only modules named in the boot script are
ever loaded. -pa alone is NOT enough there:
code:which(int) resolves while
code:ensure_loaded(int) still answers
{error,embedded}, and els_dap fails with a bare
`undef'. `interactive' restores load-on-demand,
which the edit -> recompile -> reload loop wants
anyway.
Verified on an ARM Mac by driving els_dap over DAP stdio exactly as the
VSCode extension does, with this script as the launch command: a
breakpoint in kz_mochinum:digits/1 is verified, hit, and reports
inspectable locals -- with rebar.config untouched.
Part of #27.
The Makefile's CHECK_TOOLS only asks whether `erl' and `rebar3' exist. That is not enough for the dev environment: the dev release is built with include_erts=false, so it symlinks whichever OTP is on PATH, and the prebuilt erlang-ls is built for OTP 27 -- a debug session needs the language server and the node on one runtime. A newer OTP sails through the presence check and then fails much later and much less legibly. Add scripts/dev-toolchain.sh, which checks OTP major 27 and rebar3 3.27.0 (CI's pin) and reports a mismatch with the requirement. It installs nothing and names no platform -- provisioning stays the developer's call. Both pins are overridable via KAZOO_OTP_VERSION / KAZOO_REBAR3_VERSION for deliberate excursions. The pins are deliberately not read from ci.yaml: OTP 27 is wanted here for a dev-tooling reason CI knows nothing about, so a CI bump must not silently drag the dev environment along. The two are cross-checked instead, and a divergence warns rather than fails -- the local toolchain is fine either way, it is the repo that needs a decision. Wired to `make dev-toolchain', and `make build-dev' now runs it first. CI never calls build-dev, so CI is unaffected. README gains the platform-specific install path that the script deliberately omits: the versioned `erlang@27' formula (plain `brew install erlang' gives OTP 28), the keg-only PATH step it needs, and `rebar3', which is not vendored and has no bootstrap escript -- `make' cannot run at all without it. Also records a consequence of include_erts=false: the release's lib/ symlinks point at the exact Cellar patch version it was built against, so a `brew upgrade erlang@27' leaves them dangling and requires a rebuild. Part of #27. Closes #35.
… (#47) * dev-env: name the dev node after the host it advertises (#43) The dev node was named kazoo_apps@127.0.0.1, chosen so its identity would not depend on whatever name the machine advertises. That determinism cost us both of the tools this dev environment exists for. `sup' composes its target as Node ++ "@" ++ net_adm:localhost() and has no flag for the host half -- `-n' supplies only the name -- so every `sup <module>_maintenance <fn>' in Kazoo's docs missed the node and answered "Connection to service failed!". els_dap, the step-debug backend, composes the same host the same way, and its own runtime config is unreachable in practice: erlang-ls parses erlang_ls.config as YAML while els_dap reads it with file:consult/1, so one file cannot satisfy both. There F5 did not fail, it hung silently in setBreakpoints. Both compose the same string, so naming the node after that string makes both work at once -- and nothing machine-specific is checked in, because the name is computed at launch. The name cannot be injected at the command line: unlike the -pa in dev-node.sh, `-name' in ERL_FLAGS loses to the one in the args file ("Multiple -name given to erl, using the first"). So dev-node.sh generates _build/dev/vm.args from config/vm.args.dev, rewriting only the -name line, and points VMARGS_PATH there. The checked-in file stays plain, readable and still bootable by hand. relx's own RELX_REPLACE_OS_VARS was the alternative and was not taken: with RELX_OUT_FILE_PATH unset it resolves the output back to the input path, overwriting a tracked file and leaving a .orig beside it. scripts/dev-hostname.sh holds the one definition of the node's identity. It mirrors sup:long_or_short_name/1 -- a dotted host means -name, a bare one means -sname -- because a shortnames `sup' cannot reach a longnames node at all, so a host with no resolver domain would otherwise just move the near-miss somewhere else. The emulator is happy either way; `erl -name foo@dotless' boots fine on OTP 27. els_dap's use_long_names is a static boolean in launch.json, correct for any dotted host and a one-line toggle on a dotless one; that residue goes to #34. dev-init.sh and dev_init.escript now compute the node name through the same helper instead of reading a literal, so there is one implementation rather than two that drift. The cookie is still read from vm.args.dev -- it is a genuine constant and the one value that must agree with config-dev.ini. Verified on the driver's ARM Mac: node boots to 133 applications as kazoo_apps@N70H4HP3HN.local, `sup kazoo_apps_maintenance ready' connects and returns "all kazoo applications have been started", and dev-init.sh re-initializes the database against the renamed node. * dev-env: add scripts/sup, a working sup entry point (#43) Naming the dev node after the host it advertises made it reachable, but core/sup/priv/sup still could not run against it, for two reasons that have nothing to do with the node's name. No code path. core/sup/priv/sup runs sup.escript through `bin/kazoo escript', and relx_escript() in the generated start script execs a bare escript with only RELEASE_ROOT_DIR exported -- no -pa and no ERL_LIBS into the release's own lib/. `sup:main/1' is therefore undefined before it ever tries to connect, and the failure surfaces as an escript stack trace rather than anything recognisably about SUP. This is upstream relx/Kazoo behaviour and hits the prod release identically. No config, so the wrong cookie. sup resolves its cookie through kazoo_config_init:read_cookie/1, and get_cookie/2 lets that value beat an explicit `-c'. With no KAZOO_CONFIG it reads the packaged default and tries `change_me', so even `sup -c kazoo_dev_cookie' fails authentication -- which reads exactly like a cookie mismatch the caller has no way to fix. scripts/sup supplies both, and nothing else: ERL_LIBS into the release lib dir, KAZOO_CONFIG at config-dev.ini, VMARGS_PATH at the generated vm.args. It runs the same sup:main/1 and so keeps sup's whole surface -- its flags, its output and its exit codes. Arguments reach it through -extra rather than string interpolation, so quoting survives. It passes no -n: sup composes Node ++ "@" ++ net_adm:localhost() and the node is now named on exactly that. core/sup is deliberately left alone -- it is a core module and this is a dev-env concern. Both defects above are upstream-shaped and belong upstream. Verified on the driver's ARM Mac with no environment set: `scripts/sup kazoo_apps_maintenance ready' returns "all kazoo applications have been started" (exit 0); `scripts/sup crossbar_maintenance find_account_by_realm master.dev.local' finds the account dev-init.sh created; a bogus module exits 1 and a maintenance function returning false exits 2.
…#34) (#48) Replace PR #10's placeholder .vscode/* (nonexistent make targets, a YAML config path, a `change_me` cookie) with a working erlang-ls set, and add the root erlang_ls.config: - extensions.json recommends erlang-ls.erlang-ls (the #37 decision). - settings.json teaches VSCode Kazoo's file types (so opening one activates the extension) and flags erlang_ls.config as YAML, not Erlang terms. - erlang_ls.config points apps_dirs at core/* + applications/* (source, so navigation works before build-dev) and deps at _build/dev/lib. - tasks.json chains Start Infra -> Build Dev -> (boot) with Initialize Database, using only targets/paths that exist. Start Infra uses the two-call `up -d` + `--wait couchdb rabbitmq` pattern so the couchdb-init one-shot's clean exit does not read as a failure (#30). - launch.json is an els_dap config (type erlang) whose whole environment lives in scripts/dev-node.sh; projectnode is the bare `kazoo_apps` + use_long_names true, which els_dap composes to the same host dev-node.sh names the node after, and cookie matches config-dev.ini (#43). Also extend scripts/dev-toolchain.sh to check pkg-config (and that it can resolve openssl, the real Mac keg-only failure) and go -- both are build-dev prerequisites (kazoo_auth's OpenSSL RSA driver; martini/libsecsipid via `go build`) that CHECK_TOOLS misses, so the build failed deep instead of fast. Verified end-to-end on an ARM Mac: F5 flow boots the node (133 apps) and a breakpoint in kz_mochinum:digits/1 is hit with inspectable locals.
…#51) A booted dev release lists every whapp as {App, none} -- on the code path but not started -- and leaves kapps_controller to start them at runtime. Its highest-priority selector is the KAZOO_APPS env var, so dev-node.sh sets it, deterministic on the very first boot before any DB bootstrap. The list is not hand-kept: a whapp is exactly an application whose .app.src marks {is_kazoo_app, true} (the predicate kapps_controller:is_kapp/1 uses at runtime), so it is derived from the tree -- 36 apps today, incl. ecallmgr, which ?DEFAULT_KAPPS omits because upstream runs it on its own node. Verified live on an ARM Mac: 149 OTP apps up, 35/36 whapps auto-start, ecallmgr idle-stable with no FreeSWITCH connected (ecallmgr_fs_nodes:connected() => []). call_inspector is the sole hold-out -- it hardcodes /var/log/kazoo, tracked separately.
…#53) docs/dev-environment.md documents the full clone -> open an Erlang file -> F5 -> Initialize Database -> breakpoint flow, the scripts behind each step (dev-node.sh / dev-init.sh / dev-hostname.sh / sup), inspection via remote_console / sup / observer / recon, and the ARM-Mac caveats. The erlang-ls activation caveat (F5 is inert until an .erl is open) leads the document rather than sitting in a footnote -- it is the first thing a new developer trips over. README corrections: * "Erlang Language Server" -> erlang-ls (id erlang-ls.erlang-ls), and the note that the extension bundles its own erlang_ls/els_dap escripts so nothing extra is installed. * pkg-config (+ keg-only OpenSSL on PKG_CONFIG_PATH) and Go added to the macOS requirements -- both are checked by make dev-toolchain and needed by make build-dev. * Setup & Running: correct launch config name ("Start Kazoo (debug)"), add the open-an-.erl-first step and the Initialize Database step, and point at docs/dev-environment.md. Closes #36.
…ib (#49) (#54) The dev release is dev_mode, so under _build/dev/lib the 87 project apps are real dirs whose src/include/priv symlink back to core/* + applications/* source. The old deps_dirs glob (_build/dev/lib/*) therefore re-indexed every project module a second time -- once as an app, once as a dep -- roughly doubling the project files walked at cold index and making go-to-definition ambiguous between source and the _build/dev/lib copy. erlang-ls 0.0.46 has no exclude_paths key, so the fix is to point deps at _build/default/lib, which holds only the 52 third-party deps (project apps never build into default), each carrying real src/ for navigation. include_dirs deps entry moved to match; project-app includes are already covered by core/*/include + applications/*/include. Live cold-index timing + parse_transform false-positive tuning tracked on #49.
…55) A step-debug breakpoint interprets the whole module (int:i/1), and interpreted code runs 1-2 orders of magnitude slower -- every call, from every process, routes through one meta-interpreter. Measured on the live dev node (149 apps, ~2200 procs, OTP 27 / ARM): kz_mochinum (leaf) ~100x per-call ~140x under 8-way concurrency kz_json (hot data) ~22x per-call ~66x under 8-way concurrency The concurrency column is the hazard: int serializes, so the penalty grows with concurrent callers -- interpreting a hot-path module turns it into a global bottleneck. Setting the breakpoint is cheap (int:i/1 is 4-16ms even for a 1600-line module). - docs/dev-environment.md: new "Where not to set a breakpoint on a live node" section under Step-debugging -- safe vs avoid, with the numbers. - scripts/int-bench.escript: reproducible harness (rpc + erl_eval so timing runs in-node and output carries back, per dev_init.escript); numbers are hardware-specific, re-run per machine. Leaves nothing interpreted.
…e umbrella (#56) * dev-env: fix erlang-ls include_dirs so include_lib resolves across the umbrella erlang-ls's compiler diagnostic is a plain compile:file/2 with {i,Dir} options and no code:lib_dir override, so include_lib("app/include/x.hrl") resolves only when a {i,Dir} joins with the FULL "app/include/x.hrl" -- i.e. Dir must be the PARENT of the apps, not the leaf app/include dir. With only the leaf globs every cross-app include_lib failed, orphaning every record/type/macro each app-root header defines. Measured (basic_validation, erlang-ls's exact options) over a 140-module sample: leaf globs only : 4/140 clean, 2207 errors parents + src : 138/140 clean, 9 residual errors Add the app parents (core, applications, _build/default/lib) for include_lib and the app src roots (core/*/src, applications/*/src) for plain -include from src/<subdir>/ modules; keep the leaf include/ globs for navigation. include_dirs feeds only compile {i,_} + include-POI resolution, never indexing (els_indexing walks apps_paths/deps_paths/otp_paths), so this cannot reintroduce a double-index. The 9 residual errors are irreducible parse_transform noise -- erlang-ls applies no lager_transform/kazoo_ast, so their generated funs/specs read as undefined. Refs #49 * dev-env: document erlang-ls include_dirs + residual diagnostic noise in the runbook Correct the stale deps line (deps come from _build/default/lib, not _build/dev/lib, since PR #54), add a note on why include_dirs carries the app parents + src roots, and characterise the irreducible parse_transform false positives in section 7. Refs #49
Relax the single-major OTP 27 pin to 26-or-27, proven end to end (build, boot, F5 step-debugging) on both -- issues #57 / #58 / #59. - scripts/dev-toolchain.sh: accept OTP major 26 or 27 (was exact-27, via a case over an accepted set), keeping the KAZOO_OTP_VERSION override to force a single major. Parse the rebar3 version from the `rebar <vsn>' phrase instead of line 1: on OTP 26 a failed rebar3_lint plugin-load (it needs OTP 27+) prints its own colour-escaped error to stdout ahead of the version line, which line 1 read as "OTP". The CI cross-check now warns when ci.yaml's otp-version is outside the accepted set. - README.md, docs/dev-environment.md: document 26-or-27 support; correct the erlang-ls rationale (the bundled els_dap/erlang_ls escripts are portable precompiled beams that load on both majors -- no per-OTP artifact); and add the rule that switching OTP *major* needs a full `rm -rf _build`, since stale deps compiled under the other major abort the VM with `size_object: matchstate term not allowed`.
ruhnet
requested changes
Sep 19, 2026
ruhnet
left a comment
ruhnet
left a comment
Member
There was a problem hiding this comment.
Some relatively minor comments to help agents be more accurate in their terminology during development.
Address review feedback on PR #62: - CONTEXT.md: distinguish carrier (supplies numbers), resource (offnet call routing), and provider (number services like CNAM/e911) as three independent concepts; add Resource and Provider entries. - CONTEXT.md: stepswitch routes offnet calls to resources, not carriers. - CONTEXT.md: Crossbar system-wide routes live under /v2/, not only /v2/accounts/{ID}/. - ADR 0002: retitle/rename to Target Erlang/OTP 26 and 27.
Contributor
Author
|
Thanks @ruhnet — pushed 99ab1e0 addressing all four:
|
ruhnet
approved these changes
Sep 20, 2026
Member
There was a problem hiding this comment.
Looks good!
Disclaimer: I didn't do a super-deep review, as I don't use VSCode nor MacOS, and AI output is pretty wordy.... However, I think in the interest of moving things along, and given that there is not really any actual running Kazoo code in this PR, a full in depth review wasn't really necessary. Merge it! 🦖
The benchmark script did not add enough value to maintain; remove it along with the doc references in dev-environment.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ARM-Mac VSCode dev environment
Delivers the destination of Map: ARM-Mac VSCode dev environment for OpenKazoo (#27) — all 18 tickets closed. On a clean ARM Mac a developer can now
git clone→ open in VSCode → bring up Docker infra, build a debug dev release, initialize the database, and boot a single node running all whapps (incl.ecallmgr), with erlang-ls navigation/diagnostics and F5 step-through debugging with inspectable locals on both OTP 26 and OTP 27.What's here
Local infra & build
docker-compose.dev.yml— pinned CouchDB 3.5.2 + RabbitMQ 3.13.7 (arm64), with acouchdb-initone-shot so a fresh CouchDB reachescluster_finished.devrebar3 profile +make build-dev— keepsdebug_infoand relxdev_mode(required forintstep-debugging);~35sincremental reassembly.scripts/dev-toolchain.sh(+make dev-toolchain) — enforces OTP 26 or 27 and rebar3 3.27.0, and up-front checkspkg-config/openssl andgo(needed by the RSA driver and the STIR/SHAKENmartinidep).Node config & launch
config/config-dev.ini,config/sys-dev.config,config/vm.args.dev— plus a fix to seven release apps wrongly markednonethat had silently broken boot (this hitprodtoo).scripts/dev-node.sh— the launch entry point; names the node after its host (kazoo_apps@<host>) so bothsupand els_dap can address it, and setsKAZOO_APPSso every whapp autostarts. Cold boot runs 149 OTP applications, 35/36 whapps,ecallmgridle-stable without FreeSWITCH.scripts/dev-init.sh+ Initialize Database VSCode task;scripts/supwrapper sosupworks against the dev node.Debugging
.vscode/+erlang_ls.config— erlang-ls language server and els_dap debug adapter wired for F5, reachingintthrough the launch environment (-pa debugger ebin+CODE_LOADING_MODE=interactive) so theprodrelease is untouched. Live breakpoint hit inkz_mochinum:digits/1on both OTP 26 and 27.Docs & groundwork
docs/dev-environment.mdrunbook + README updates (supersedes and retires PR openkazoo: add OpenTelecom VSCode and docker compose additions #10), including therm -rf _buildrule when switching OTP major versions.CONTEXT.mdglossary, seed ADRs, and agent-skills configuration.Closes #27.