Skip to content

Commit eb62861

Browse files
authored
Merge pull request #172 from Univers42/develop
Merge pull request #171 from Univers42/164-bug-incomplete-output-from-curl-installations
2 parents 26cae57 + a3cad05 commit eb62861

204 files changed

Lines changed: 10609 additions & 806 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude‎

Submodule .claude added at bf956c0

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,8 @@ jobs:
254254
# debug build runs this sweep from the pty job; this is its release run.
255255
- name: ${...} quoting sweep against the RELEASE build
256256
run: python3 tests/expansion_quote_sweep_test.py build/bin/hellish
257+
- name: curl installer output regression (#164)
258+
run: python3 tests/curl_install_output_test.py build/bin/hellish
257259
# The suite diffs hellish against bash byte-for-byte, so anything that
258260
# steers bash steers the verdict -- and the people most likely to run it
259261
# are the people whose rc exports the most. A ~/.hellishrc framework

‎.gitignore‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ out.dot
99
tests/results.log
1010
tests/bench_results.txt
1111
tests/agnostic_results.tsv
12-
.claude/
1312

1413
# secrets / credentials — never commit
1514
.env
@@ -44,8 +43,5 @@ tests/glob-zoo/tab file
4443

4544
# Host-runnable static binaries produced by `make static`.
4645
dist/
47-
.claude/*
48-
.claude
49-
.CLAUDE
50-
*CLAUDE*
5146
site/
47+
*.env

‎.gitmodules‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,6 @@
1212
path = tests/inception
1313
url = ../Inception.git
1414
branch = feat/scripts-run-with-launching-shell
15+
[submodule ".claude"]
16+
path = .claude
17+
url = ../claude-deal-with-the-devil.git

‎CLAUDE.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
hellish is a from-scratch POSIX(+bashisms, + an opt-in zsh dialect) shell in C, written as a 42 project. Its behaviour is defined by diffing against a **pinned bash 5.3.9** (`bash --posix`) and, for the zsh dialect, a pinned zsh 5.9. In-depth docs: `DEV_DOC.md`, `CONTRIBUTING.md`, and `src/<module>/README.md` for each module.
6+
7+
## Setup
8+
9+
- Submodules are required: `vendor/libft` (libft + the `ft_malloc` allocator) and `vendor/scripts` (dev tooling). If the build can't find libft, run `git submodule update --init --recursive`.
10+
- Build the oracles once: `make oracle` (→ `~/bash-5.3.9`) and `make zsh-oracle` (→ `~/zsh-5.9`). The golden suite compares against these, not the system bash.
11+
- `make` with no target prints the help page, which lists every target with a description.
12+
13+
## Build
14+
15+
| Command | Result |
16+
|---|---|
17+
| `make all` | `MODE=debug`: `-O0 -g3`, ASan+LSan, libc malloc (`SAFE=1`) → `build/bin/hellish` |
18+
| `make OPT=1 all` / `MODE=release` | `-O3 -flto`, `ft_malloc` (`SAFE=0`), no sanitizers |
19+
| `MODE=relwithdebinfo` | `-O2 -g`, `ft_malloc` |
20+
| `make re` / `make re OPT=1` | full rebuild; both must build cleanly before a PR |
21+
22+
- `-Wall -Wextra -Werror`: a warning fails the build. Add flags with `EXTRA_CFLAGS=…`. Passing `CFLAGS=…` replaces the whole flag set.
23+
- Each mode/SAFE combination has its own object dir (`build/obj-<mode>-<alloc>`), but they all link to the same binary path.
24+
- Sources are found with `find src` (excluding `src/platform/`), so a new `.c` file needs no Makefile edit. Only `src/platform/$(TARGET)/` (currently `posix`) is compiled in. Platform-specific code goes in per-file implementations there, not in `#ifdef`s.
25+
- Debug knobs: `--debug=lexer|parser|ast`, `HELLISH_ALLOC_STATS=1` (live bytes at exit, the leak oracle on `SAFE=0`), `HELLISH_BANNER=0`, `HELLISH_NO_UPDATE_CHECK=1`.
26+
27+
## Test
28+
29+
A green `make test` alone isn't enough. The test layers are:
30+
31+
- **Golden suite**: `make test`. Each category in `tests/` is a file with one command per line. Every case runs through `hellish -c` and the pinned `bash --posix`, and stdout, exit status and any written files are diffed.
32+
- Run one category: `cd tests && ./tester <category>` (`-v` for verbose).
33+
- **A new category file must be added to `test_lists` in `tests/tester`**, or it silently never runs.
34+
- The tester `chmod 000`s `tests/test_files/invalid_permission`. If a run is interrupted, `chmod 755` it before git operations.
35+
- **pty/interactive tests**: `make pty-test`. It discovers every `tests/*.py` automatically. Run a subset with `tests/pty_suite.sh -- <pattern>`. Each `.py` takes the shell path as its only `argv[1]`; keep it that way.
36+
- Many `make *-test` targets run one pty test; see `make help`.
37+
- **Whole scripts**: `tests/run_scripts.sh`. **Hard corpus**: `tests/hard/run.sh`.
38+
- **Allocator parity**: `cd tests && ./verify_alloc.sh` builds and diffs both `SAFE=1` and `SAFE=0` against bash. Run `make test-release` after touching memory code (ASan and `-O3` can disagree).
39+
- `make arena-stress`: parse arena under ASan with tiny chunks. `make help-test`: every builtin must have a `help` entry.
40+
- `make norm` runs 42 norminette (it only reports, and always exits 0).
41+
- Pre-PR gates: `make re`, `make re OPT=1`, `make test`, `tests/run_scripts.sh`, `tests/hard/run.sh`, `make pty-test`, `cd tests && ./verify_alloc.sh`, `make norm`.
42+
43+
Leak checking: ASan/LSan only work on `SAFE=1`. On `SAFE=0` use `HELLISH_ALLOC_STATS=1`.
44+
45+
## Architecture
46+
47+
One command's path through the shell (`src/core/shell.c`: `main` → `on()` → `repl_shell` → `off()`):
48+
49+
1. **Input** (`src/infrastructure/input*.c`, readline in `rl_*`/`history*`): one unified line buffer (`t_rl`) for tty, file, `-c` and pipe input, plus multi-line joining and heredoc collection (`src/heredoc/`).
50+
2. **Alias expansion** (`src/alias/`) produces `state->alias_exp`, which is what gets lexed.
51+
3. **Lexer** (`src/lexer/`) and **parser** (`src/parsing/`) build the AST (`incs/ast.h`, `src/infrastructure/ast*.c`).
52+
4. **Reparser** (`src/word_splitting/`) runs between parsing and expansion. It splits coarse `AST_WORD` tokens into quote- and variable-aware sub-tokens and re-tags assignment words.
53+
5. **Expander** (`src/expander/`) handles parameter/arith/command/process substitution, IFS splitting, brace expansion, globbing (`src/glob/`) and redirect construction. `expand_zsh_*` is the zsh dialect.
54+
6. **Execution** (`src/execution/`, fork/job parts in `src/platform/posix/`, `src/job_control/`) runs builtins (`src/builtins/`, dispatch table plus `help_data*.c`), functions, pipelines and compound commands.
55+
7. After each REPL turn, the per-command AST, redirects and input are freed, so memory stays flat over long scripts.
56+
57+
**`t_shell`** (`incs/shell.h`) holds all the shell's state and is passed everywhere. New state goes there, or in a struct you pass down. **Never add a global variable.** The few legacy globals (word slab, parse arena, env index, readline completion iterators, dir stack) are scheduled for cleanup and aren't precedent.
58+
59+
**Memory model** (`src/helpers/README.md`):
60+
- All allocation goes through `xmalloc`/`xfree`. They route to either libc or `ft_malloc` at compile time, and a pointer must never cross heaps.
61+
- The **parse arena** (`parena_*`) is a bump allocator, open only around the main cycle's parse. Nested parses (`eval`, `source`, cmdsub bodies) fall through to the heap. `parena_free` is a no-op for arena pointers. Heap memory attached to an arena tree must call `parena_note_attach()`. Before sharing a struct between nodes, check `parena_owns()`; issue #94 was a double free caused by skipping that check.
62+
- The **word slab** (`word_slab_push`) holds argv strings. Values that can escape into the environment (assignments, `for` values) must live on the general heap.
63+
- Shutdown goes through `free_all_state`. Its ordering is deliberate.
64+
- Bugs in libft/ft_malloc get fixed in the submodule's own repo, then the pointer gets bumped here. Don't work around them from the shell.
65+
66+
## Code style (42 Norm, enforced)
67+
68+
- At most 5 functions per file and 80 columns per line. No `for` loops and no ternaries. Every file starts with the standard 42 header. Because files hit the 5-function limit, modules have many `foo2.c`, `foo3.c` files; continue that pattern.
69+
- Comments are block comments placed **before** a function or at file scope, never inside a function body. They explain the *why* (root cause, invariant, issue number). Complex functions get a Doxygen block (`@brief`/`@param`/`@return`, plus the hard part).
70+
71+
## Workflow
72+
73+
- Every fix ships with a test that covers the bug and nearby cases.
74+
- Branch from `develop` (named `type/short-description`). PRs target `develop`; `main` and `develop` are protected.
75+
- Commits use Conventional Commits (`type(scope): imperative subject`, types `feat|fix|refactor|perf|test|docs|chore|style`), enforced by the hook from `./vendor/scripts/install-hooks.sh`. The body explains the root cause and what the test proves.
76+
- **Don't add `Co-authored-by:` trailers for AI tools.**
77+
- After changing builtins, regenerate the wiki page with `make docs-builtins`.

‎USER_DOC.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,52 @@ A syntax error in any file hellish loads is reported once, with the file
9797
and line, and the line itself — the same two lines bash prints — and the
9898
lines before it still run.
9999

100+
## The prompt
101+
102+
`PS1` is the prompt, and it is bilingual: bash's backslash escapes (`\u`,
103+
`\w`, `\$`, `\[ \]`…) and zsh's percent escapes (`%~`, `%F{cyan}`,
104+
`%(?.a.b)`…) work in the same string. `PROMPT` is the zsh spelling with
105+
zsh's exact rules, and wins when both are set; `RPROMPT` is the right
106+
prompt. The full escape list is section 1 of `hellishrc.example`, and
107+
[wiki/interactive.md](wiki/interactive.md#prompt-) has the rest.
108+
109+
What is live at every render, and what is not:
110+
111+
| in the prompt | rendered? |
112+
|---|---|
113+
| `$VAR`, `${VAR}`, `${VAR:-x}`, `$?` and the other specials | yes, every time the prompt is drawn |
114+
| `$((…))` | yes |
115+
| `$(…)` and backquotes | **no** — the text stays on screen as written; it would fork on every redraw |
116+
117+
For a computed segment, set a variable from a hook and put the variable in
118+
the prompt. `HELLISH_PRECMD_FUNCS` runs once before each prompt
119+
(`hellishrc.example`, section 5b):
120+
121+
```sh
122+
_branch() { BR=$(git branch --show-current 2>/dev/null); }
123+
HELLISH_PRECMD_FUNCS=(_branch)
124+
PS1='${BR:+($BR) }\w \$ '
125+
```
126+
127+
**`prompt` is a shell function, not a builtin.** The installer seeds it
128+
into `~/.config/hellish/rc.d/40-prompt-switch.hsh`, next to the 29 themes
129+
in `~/.config/hellish/themes/`. `type prompt` says whether it is loaded;
130+
from a checkout, `HELLISH_THEMES=share/themes; . share/rc.d/40-prompt-switch.hsh`
131+
loads it by hand.
132+
133+
**`\A` is not bash's 24-hour clock here.** It is the animated glyph, and it
134+
was there first. A bash `PS1` pasted in with `\A` shows the glyph — or
135+
nothing, since the animation is off by default — where bash shows `14:30`.
136+
Write `\D{%H:%M}` or `%T` for the clock.
137+
138+
**The animation is off by default, on purpose.** `HELLISH_ANIM=spinner`
139+
(or `pulse`, `ember`) makes `\A` repaint the prompt row about ten times a
140+
second while the shell is idle. That repaint has been reported to corrupt
141+
the prompt on some terminals: an escape sequence printed as text
142+
(`8;2;90;96;106m`), or a cut glyph shown as `�`. It could not be
143+
reproduced on demand, so it ships off. If you turn it on and see that,
144+
`HELLISH_ANIM=off` is the fix.
145+
100146
## Check that it works
101147

102148
```sh
@@ -111,11 +157,62 @@ conf list # plugin framework: what is on and off
111157
If a terminal ever misbehaves after an experiment: `reset` — and the
112158
escape hatches above get you back to your previous shell at any time.
113159

160+
## Completing `make` targets
161+
162+
`make <TAB>` / `make cl<TAB>` completes target names from the Makefile in the
163+
current directory (also `gmake`, `/usr/bin/make`, and behind `sudo`, `env X=1`,
164+
`&&`, `(`; a wrapper option that takes an argument, as in `sudo -u root make`,
165+
falls back to filenames). The file is parsed in the shell itself; make is never run.
166+
167+
- Files read: the first regular file of `GNUmakefile`, `makefile`, `Makefile`
168+
(GNU make's order, symlinks followed), at most 4 MiB. No such file, a
169+
directory, or an unreadable file means plain filename completion.
170+
- Offered: the names on `target:` / `a b c:` / `a::` lines. Not offered:
171+
variable assignments, directives, recipe lines, `define` blocks, names with
172+
`%` or `$`, names starting with `.`; text after the colon is ignored.
173+
- `include` / `-include` / `sinclude` of literal file names are followed ONE
174+
level deep (what an included file includes is ignored). Changing only an
175+
included file does not refresh the cache; touch the Makefile.
176+
- Cache: the targets are reused while the Makefile's name, inode, mtime and
177+
size are unchanged (one `stat` per TAB).
178+
- A word starting with `-` or containing `=` or `/` keeps filename completion,
179+
and so does any `make` line with `-C`, `--directory`, `-f`, `--file` or
180+
`--makefile` before the cursor, or a prefix that matches no target
181+
(`make out.txt`).
182+
- A `complete` spec for `make` (under `shopt -s progcomp`) wins over all of
183+
this; `complete -r make` brings the targets back.
184+
114185
## Knobs
115186

116187
| variable | effect |
117188
|---|---|
118189
| `HELLISH_BANNER=0\|1` | welcome panel off / on |
190+
| `HELLISH_HISTORY_PREFIX=0\|off` | Up/Down plain history; default is prefix search, read at every prompt |
191+
| `HELLISH_AUTOSUGGEST=1\|on` | inline suggestion, off by default: the newest history line that starts with what you typed, shown dim after the cursor (see below); read at every prompt |
192+
| `HELLISH_SUGGEST_MAX=N` | how many of the newest history lines the suggestion looks at (default 1000; not a number, or below 1, is the default); read at every prompt |
119193
| `HELLISH_NO_UPDATE_CHECK=1` | never check for updates, no badge |
120194
| `HELLISH_NO_ANIM=1` | no startup animation |
121195
| `HELLISH_NO_EXEC=1` | user-install hook: skip for this one session |
196+
197+
### Inline suggestion
198+
199+
With `HELLISH_AUTOSUGGEST=1` (set it at the prompt or in the rc file) the
200+
newest history line that starts with what you have typed appears dim after
201+
the cursor. It is a hint, not part of the line: **Right**, **End** or
202+
**Ctrl-F** at the end of the line insert it; anywhere else, and when there
203+
is no hint, those keys do what they always did. Enter runs only what you
204+
typed, and the hint is wiped from the screen. `unset HELLISH_AUTOSUGGEST`
205+
(or `=0`) turns it off at the next prompt. A key you bound yourself
206+
(`bind` / `bindkey` / `.inputrc`) is never taken over.
207+
208+
Never suggested: a line with a newline, a tab or any other control character
209+
(an old `printf '\e[31m'` cannot repaint your prompt), and nothing at all
210+
while you search (Ctrl-R), in vi command mode, on a continuation prompt, with
211+
the cursor inside the line, or on a line that would not fit the terminal's
212+
width (the hint is cut at the edge, never wrapped). A right prompt
213+
(`RPROMPT`) is hidden while the hint reaches it and comes back when it does
214+
not.
215+
216+
For zsh widgets (`zle -N`), the variable `POSTDISPLAY` is the text shown
217+
after the cursor: read it in a widget to get the current hint, assign it
218+
(`POSTDISPLAY=' <hint>'`) to show your own until the line next changes.

‎docker/Dockerfile‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,17 +44,25 @@ FROM ${BASE}
4444
# in was a red rung on a commit the rung had built fine an hour earlier.
4545
# A mirror that keeps failing four times still fails the build -- it is
4646
# the transient, not the broken, that is retried.
47+
# apt gets the same loop, update included: an Ubuntu mirror mid-sync serves
48+
# an index that disagrees with its files ("File has unexpected size ...
49+
# Mirror sync in progress?", or a 404 on a .deb the index already moved
50+
# past), and only a fresh update a little later cures it. It turned the
51+
# ubuntu 22.04 rung red on PR #141 before a line was compiled.
4752
ARG CC=cc
4853

4954
RUN set -eux; \
5055
if command -v apk >/dev/null 2>&1; then \
5156
apk add --no-cache build-base readline-dev ncurses-dev git bash \
5257
coreutils python3 clang; \
5358
elif command -v apt-get >/dev/null 2>&1; then \
54-
apt-get update; \
55-
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
56-
build-essential libreadline-dev git ca-certificates bash \
57-
python3 clang; \
59+
n=0; until apt-get update -o Acquire::Retries=3 \
60+
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
61+
-o Acquire::Retries=3 --no-install-recommends \
62+
build-essential libreadline-dev git ca-certificates bash \
63+
python3 clang; do \
64+
n=$((n + 1)); [ "$n" -ge 4 ] && exit 1; sleep $((n * 15)); \
65+
done; \
5866
rm -rf /var/lib/apt/lists/*; \
5967
elif command -v pacman >/dev/null 2>&1; then \
6068
pacman -Sy --noconfirm --needed base-devel readline git bash \

‎hellishrc.example‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,15 @@
7070
#
7171
# \S is usually the nicer answer than $? anyway: it shows " ✘N" only after a
7272
# failure and stays invisible when everything is fine.
73+
#
74+
# $((...)) is live too. $(...) is NOT: command substitution in a prompt is
75+
# never run -- it stays on screen as the text you wrote -- because it would
76+
# fork on every redraw. Compute the value in a hook (section 5b) and put the
77+
# variable in the prompt; the hook runs once per prompt, not per redraw:
78+
#
79+
# _branch() { BR=$(git branch --show-current 2>/dev/null); }
80+
# HELLISH_PRECMD_FUNCS=(_branch)
81+
# PS1='${BR:+($BR) }\w \$ '
7382

7483
# ── ANIMATION: OFF BY DEFAULT, AND THAT IS DELIBERATE ──────────────────
7584
# A live style makes \A repaint the prompt rows in place ~10x/second

0 commit comments

Comments
 (0)