Skip to content

Commit d833200

Browse files
sp00ktoberclaude
andauthored
feat(mhfu): one memory interface and the overlay and EBOOT analysis on rabbitizer (#6)
* feat(mhfu): memory, typed views, overlays, BOOT.BIN, file ids, MIPS core, one CLI Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): an overlay's text section starts at 0x40; its first 64 bytes are zero Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): drop EM_OVERLAY_BASE; below an em overlay lies game_sub's bss, not the overlay's Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): a struct can give its table stride; psutil is a direct dependency Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): shared MIPS analyses on rabbitizer mips.Code decodes a text range once and answers what the old tools each re-implemented: function bounds, calls (direct, virtual by vtable slot, tail), lui/lo pairs through rabbitizer's RegistersTracker as spimdisasm walks it, jump tables with their bound and bias, and the constant a register holds at a site. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): the em package for big-monster overlay analysis Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): stage runtime, hit volumes, attack tables, model injection files mhfu.stage reads a stage overlay's parameter object (exits, spheres, the raw surface table), the map table, and the live resident-file, gathering-spot and small-monster spawn tables. mhfu.hitzone and mhfu.hitbox read the part system and the attack tables through the species row and the setter call. mhfu.inject places finished files for the framework's live injection. files.py covers all 282 stage PACs, the 16 st046 lobby variants included. CLI: mhfu stage ids|maps|exits|surfaces|spots|spawns, mhfu hitzones, mhfu hitboxes, mhfu inject. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): relocate an MWo3 overlay by a 64 KB-aligned delta References come from mhfu.mips: j/jal targets, lui/lo pairs as rabbitizer's RegistersTracker pairs them (matching spimdisasm on every em, stage, game_task and game_sub overlay) and data words into the overlay's own image and bss. The header's load address and ctor list move with it; the bss of game_sub below an em overlay does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(mhfu): an overlay maps as the game holds it; Struct.step, Species and LOBBY_TASK in one place each Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): Code.table; relocate from the overlay's raw file Code.table gives the table an indexed load reads (lw v0, 0(table + index)), the one idiom hitbox needed that Code.constant does not cover. relocate() now starts from Overlay.file, so a moved overlay keeps its disc padding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(mhfu): hitbox and stage read code through mips.Code hitbox.setter_calls replaces calls_to, Code.constant the lui/addiu window, and Code.table the try-every-register loop for the volume table; the stage parameter object is what any function returns in v0. Identical results: 22 attack tables with their 355 volume sets and all 267 stage parameter objects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(mhfu): hitbox and stage tests on mips.Code The synthetic setter call stores a table entry the way the game does, lw v0, 0(table + index), instead of relying on a scan of every register. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): mhfu dis, xref, switch and reloc The game's code from BOOT.BIN or an overlay (a file id, game_task, game_sub, emNN or stageNNN); an address may be a name from the table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): follow the move pseudo-instruction in constants and switch operands rabbitizer reports or rd, rs, $zero as cpu_move; move_source names the copied register for it and for addu/or with $zero. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): the effects a big-monster overlay spawns, and the species effect-id bias Addresses for the intel section: the framed and positional effect spawns, the entity factory and the run of attack-node constructors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): the attack id each big-monster handler spawns, and how it fits the table Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): the engine-to-overlay interface: entity vtables, factory, dispatch sites Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): the measured behaviour-pair census from the framework log Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): mhfu effects, attacks, abi and census Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): species intel blocks: credited effects and attacks, parts, attack tables, census The per-pair join waits for the moveset, phase and chain analyses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): big-monster moves, what ends them and what follows, from the overlay Moveset reads the action tick's (main, sub) pairs, their handlers and the animation ids each hands the executor; phases the gates a handler waits on; chain the pair a handler enters when it ends, through enter-action and the per-main translators. One path walker interprets handlers for both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): data flow across labels; tail-called functions are entries source, constant and the virtual-call slot now take every path into a site within its function, jump-table edges included, instead of stopping at the first label: a vptr loaded on both arms of a branch-likely now gives its slot (three more vt+0x28 calls in game_task). A j to a run no branch reaches starts a function when it lies in another function or right after the j, and a prologue a j reaches is no longer taken for a second frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): parity with the old em tools; tail calls, facts and per-path values Gates read the handler's body over tail jumps and each frame on every path; budget seeds through walker probes. The walker follows tail calls as continuations, learns entity cells from stores and branches, and the chain treats a tail-calling thunk as the wrapper it leads to. An enter-action id that sets nothing is told apart from a computed one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(mhfu): CALL_CLOBBERS public, chain on Call.slot alone, Live reads raise Unmapped Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(mhfu): README for the memory interface, views and the analysis commands Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(mhfu): abi reads the vtable slot from Call.slot, which now sees past labels Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(mhfu): mhfu intel, the per-species join (schema mhfu.species_intel/1) An inline case keeps its own note instead of the census-only one; the overlay's text is the section's true range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(mhfu): drop the empty address sections Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(mhfu): one shared MIPS encoder module for the test code modkit_testing.mips replaces nine per-file encoder sets; each encoder is checked against rabbitizer's decode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(mhfu): test_phases and test_cli_binary on the shared encoders Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mhfu): the census skips forced transitions and the visits they cut short Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 2d5a118 commit d833200

59 files changed

Lines changed: 8118 additions & 34 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/mhfu/README.md‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# mhfu
22

3-
What the modkit knows about Monster Hunter Freedom Unite (PSP, EU release ULES01213) at runtime.
3+
What the modkit knows about Monster Hunter Freedom Unite (PSP, EU release ULES01213): the game's
4+
addresses and structs, and the analysis of its code. Everything reads through one memory
5+
interface, so the same code runs on the running game, a RAM dump or the game's own files.
46

57
## Addresses
68

@@ -17,10 +19,52 @@ addresses.ENTITY.HP # <HP +0x2E4 u16>, the field's offset
1719
```
1820

1921
```bash
20-
python -m mhfu.addresses c -o addresses.gen.h # MHFU_SCREEN_STATE, MHFU_ENTITY_HP, ...
21-
python -m mhfu.addresses lua -o addresses.gen.lua # mhfu.addr.SCREEN_STATE, mhfu.addr.ENTITY.HP, ...
22+
mhfu addresses c -o addresses.gen.h # MHFU_SCREEN_STATE, MHFU_ENTITY_HP, ...
23+
mhfu addresses lua -o addresses.gen.lua # mhfu.addr.SCREEN_STATE, mhfu.addr.ENTITY.HP, ...
2224
```
2325

2426
Product code in this repository never writes an address as a number: a pre-commit check rejects
2527
one. For a value that only looks like an address, such as a MIPS jump opcode, end the line with
2628
a `noaddr` comment.
29+
30+
## Memory and typed views
31+
32+
```python
33+
from mhfu.files import Extracted
34+
from mhfu.memory import Image, Live
35+
from mhfu.structs import Entity
36+
37+
game = Extracted.find("extracted/") # the output of `mhp-formats extract`
38+
em75 = game.em(75) # the Tigrex overlay, mapped where the game loads it
39+
em75.u32(em75.text.start)
40+
41+
Entity(Image(dump, base), address).hp # from a RAM dump
42+
Entity(Live(client), address).hp = 1 # in the running game, via ppsspp_debug.Client
43+
```
44+
45+
A view's fields take their offset and type from `addresses.toml`; one that disagrees fails at
46+
import.
47+
48+
## Overlay and EBOOT analysis
49+
50+
The game's code is plain MIPS in `BOOT.BIN` and in its overlays (`game_task`, `game_sub`, one
51+
`emNN` per big monster, one per stage). `mhfu.mips` decodes it with
52+
[rabbitizer](https://github.com/Decompollaborate/rabbitizer) and finds functions, calls, switches
53+
and constants; the commands below are built on it. They read an extracted game from `--data` or
54+
`MHFU_DATA`.
55+
56+
| Command | What it prints |
57+
|---|---|
58+
| `mhfu dis ADDR [COUNT]`, `xref ADDR`, `switch ADDR` | Disassembly, references, jump tables, in BOOT.BIN or `--overlay` |
59+
| `mhfu reloc OVERLAY DELTA -o OUT` | An overlay moved to another address |
60+
| `mhfu moveset`, `phases`, `chain SPECIES` | A big monster's moves, what ends each, what follows it |
61+
| `mhfu effects`, `attacks`, `abi` | The effects and attacks each move spawns; the engine-to-overlay interface |
62+
| `mhfu hitzones`, `hitboxes SPECIES` | Where a monster can be hit, and where it hits |
63+
| `mhfu census --log framework.log` | How long each move lasted in a real game, from the framework's log |
64+
| `mhfu intel --all` | All of the above per species, as `species/emNN.json` |
65+
| `mhfu stage ids\|maps\|exits\|surfaces` | Stage files, the map table, area exits |
66+
| `mhfu inject FILE_ID FILE` | Places a finished file for the framework's live injection |
67+
68+
## Licence
69+
70+
MIT.

‎packages/mhfu/pyproject.toml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
[project]
22
name = "mhfu"
33
version = "0.1.0"
4-
description = "Monster Hunter Freedom Unite (PSP) runtime knowledge: addresses and struct layouts."
4+
description = "Monster Hunter Freedom Unite (PSP) at runtime: addresses, live structs, automation, overlay analysis."
55
readme = "README.md"
66
requires-python = ">=3.11"
77
license = "MIT"
8-
dependencies = []
8+
dependencies = ["ppsspp-debug", "rabbitizer>=1.16", "pyelftools>=0.31", "psutil>=6"]
9+
classifiers = ["Typing :: Typed"]
10+
11+
[project.scripts]
12+
mhfu = "mhfu.cli:main"
913

1014
[build-system]
1115
requires = ["uv_build>=0.11,<0.12"]

‎packages/mhfu/src/mhfu/addresses.py‎

Lines changed: 29 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212

1313
from __future__ import annotations
1414

15-
import argparse
1615
import functools
1716
import re
1817
import sys
@@ -65,9 +64,18 @@ class Struct:
6564
doc: str
6665
fields: dict[str, Field]
6766
size: int | None = None
67+
stride: int | None = None
68+
69+
@property
70+
def step(self) -> int:
71+
"""Bytes from one of these to the next in a table: `stride`, else `size`."""
72+
step = self.stride or self.size
73+
if step is None:
74+
raise TypeError(f"struct {self.name} has no size or stride in addresses.toml")
75+
return step
6876

6977
def __getattr__(self, name: str) -> Field:
70-
fields = self.__dict__.get("fields", {})
78+
fields: dict[str, Field] = self.__dict__.get("fields", {})
7179
if name in fields:
7280
return fields[name]
7381
raise AttributeError(f"struct {self.__dict__.get('name')} has no field {name}")
@@ -119,13 +127,16 @@ def parse(data: dict[str, Any]) -> Table:
119127
where = f"struct.{name}"
120128
if not _NAME.match(name):
121129
problems.append(f"{where}: names are UPPER_SNAKE_CASE")
122-
if not isinstance(entry, dict) or set(entry) - {"doc", "size", "fields"}:
123-
problems.append(f"{where}: expects doc, fields and an optional size")
130+
if not isinstance(entry, dict) or set(entry) - {"doc", "size", "stride", "fields"}:
131+
problems.append(f"{where}: expects doc, fields and an optional size or stride")
124132
continue
125-
size = entry.get("size")
133+
size, stride = entry.get("size"), entry.get("stride")
126134
if size is not None and (not isinstance(size, int) or size <= 0):
127135
problems.append(f"{where}: size must be a positive integer")
128136
size = None
137+
if stride is not None and (not isinstance(stride, int) or stride <= 0):
138+
problems.append(f"{where}: stride must be a positive integer")
139+
stride = None
129140
_doc_ok(where, entry.get("doc"), problems)
130141
fields: dict[str, Field] = {}
131142
by_offset: dict[int, str] = {}
@@ -142,14 +153,14 @@ def parse(data: dict[str, Any]) -> Table:
142153
fields[fname] = Field(offset, fname, fentry["type"], fentry["doc"])
143154
if not fields:
144155
problems.append(f"{where}: has no fields")
145-
structs[name] = Struct(name, entry.get("doc", ""), fields, size)
156+
structs[name] = Struct(name, entry.get("doc", ""), fields, size, stride)
146157

147158
# every generated name (Python and Lua namespace, C macro) must be unique
148159
for name in addresses.keys() & structs.keys():
149160
problems.append(f"{name}: is both an address and a struct")
150161
macros: dict[str, str] = {name: f"address.{name}" for name in addresses}
151162
for s in structs.values():
152-
for member in [*s.fields, *(["SIZE"] if s.size else [])]:
163+
for member in [*s.fields, *(["SIZE"] if s.size else []), *(["STRIDE"] if s.stride else [])]:
153164
macro = f"{s.name}_{member}"
154165
if macro in macros:
155166
problems.append(
@@ -201,6 +212,8 @@ def render_c(t: Table) -> str:
201212
out += ["", f"/* struct {s.name}: {s.doc} */"]
202213
if s.size:
203214
out.append(f"#define MHFU_{s.name}_SIZE 0x{s.size:X}u")
215+
if s.stride:
216+
out.append(f"#define MHFU_{s.name}_STRIDE 0x{s.stride:X}u")
204217
for f in s.fields.values():
205218
out += [f"/* {f.type}: {f.doc} */", f"#define MHFU_{s.name}_{f.name} 0x{int(f):X}u"]
206219
out += ["", "#endif", ""]
@@ -222,6 +235,11 @@ def render_lua(t: Table) -> str:
222235
out += [f" ---struct: {s.doc}", f" {s.name} = {{"]
223236
if s.size:
224237
out += [" ---the struct's size in bytes", f" SIZE = 0x{s.size:X},"]
238+
if s.stride:
239+
out += [
240+
" ---bytes from one row to the next",
241+
f" STRIDE = 0x{s.stride:X},",
242+
]
225243
for f in s.fields.values():
226244
out += [f" ---{f.type}: {f.doc}", f" {f.name} = 0x{int(f):X},"]
227245
out.append(" },")
@@ -247,20 +265,10 @@ def __dir__() -> list[str]:
247265

248266

249267
def main(argv: list[str] | None = None) -> int:
250-
ap = argparse.ArgumentParser(
251-
prog="python -m mhfu.addresses",
252-
description="Print the address table as a C header or a Lua module.",
253-
)
254-
ap.add_argument("lang", choices=sorted(RENDER))
255-
ap.add_argument("-o", "--output", type=Path, help="write here instead of stdout")
256-
ap.add_argument("--table", type=Path, help="read this TOML instead of the packaged one")
257-
args = ap.parse_args(argv)
258-
text = RENDER[args.lang](load(args.table) if args.table else table())
259-
if args.output:
260-
args.output.write_text(text, encoding="utf-8")
261-
else:
262-
sys.stdout.write(text)
263-
return 0
268+
"""`python -m mhfu.addresses c|lua`, the same as `mhfu addresses c|lua`."""
269+
from .cli import main as cli
270+
271+
return cli(["addresses", *(sys.argv[1:] if argv is None else argv)])
264272

265273

266274
if __name__ == "__main__":

0 commit comments

Comments
 (0)