Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
130 lines (99 loc) · 5.62 KB

CONTRIBUTING.md

File metadata and controls

130 lines (99 loc) · 5.62 KB
Raw

Contributing

Thanks for looking. This started as the artifact behind a paper and is being built out into something people can actually use, so there is a lot of useful work available at every size.

Getting set up

git clone https://github.com/19PINE-AI/transparent-offload
cd transparent-offload
sudo apt install build-essential libssl-dev zlib1g-dev   # Debian/Ubuntu
make && make check

make check should print 13 passing tests in well under a minute, with no GPU and no root. If it does not, that is a bug worth an issue on its own — tell us your distribution, kernel and glibc version.

Where help is most useful

Roughly in order of how much difference it would make:

  • Try it on a real server and tell us what happened. Especially a thread-per-connection server we have not tested. Whether it worked or not is equally interesting; TOFFLOAD_STATS=stderr:1 output makes the report useful.
  • A backend for hardware we do not have. FPGA, DPU, QAT, an inference server. The contract is four functions — docs/writing-a-backend.md.
  • Reduce the detector's overhead. Always-on page protection costs about 40% on safe applications because it re-protects on every fiber resume. Only re-protecting dirtied pages is the obvious fix and nobody has written it.
  • Portability. The context switch supports x86-64 and aarch64. Everything else assumes Linux, glibc and epoll. musl and other libcs are untested.
  • The event-loop gap. Event-driven servers run safely but gain nothing. Giving them overlap means the runtime owning the event loop. This is the biggest open design question in the project — see ROADMAP.md.

Issues tagged good first issue are scoped to be finishable in an evening.

Making a change

  1. Branch from main.
  2. Make the change, with a test if it is a behaviour change.
  3. make check must pass. If you touched the runtime, run make demo too and say in the PR whether the numbers moved.
  4. Open a PR describing what changed and why. If it fixes a bug, say how you reproduced it.

CI runs the build and the suite on gcc and clang, plus ASan and UBSan builds.

Code style

Match the file you are editing. The runtime (src/libtransparent.c) is written in a deliberately dense one-statement-per-line style; new code there should look like its neighbours rather than like a style guide. Newer files (src/config.c, the tests) are more conventional. .clang-format describes the newer style and make format applies it — but do not reformat existing dense code as part of an unrelated change, because it makes the diff unreviewable.

What we do care about:

  • The tree builds warning-free with -Wall -Wextra. Keep it that way.
  • Comments explain why, not what. The code already says what.
  • No new global state in the runtime without a note on how it behaves across fork(), which servers do at startup.

Touching the runtime

src/libtransparent.c runs inside other people's processes, in a constructor, before their main(). A few rules that are not obvious:

  • Interposers must degrade to passthrough. If something is not resolvable or not applicable, call through to the real symbol. The runtime's failure mode should be doing nothing, never doing something wrong.
  • accel_done and the scheduler loop are hot. Every scheduler iteration touches every parked fiber. Do not add syscalls there.
  • Symbol versioning is load-bearing. Interposing pthread_cond_* without matching glibc's GLIBC_2.3.2 versions crashes MariaDB at startup. The version script src/transparent.map and the dlvsym calls exist for that reason; if you touch them, test against MariaDB.
  • Anything holding a lock across a yield needs an exit path. The handler lock deadlocked the runtime exactly once because a handler that closed its connection never reached the write() that released it.

Adding a test

tests/run.sh is a plain bash suite. Add a block guarded by want "your-name" so it can be run alone with tests/run.sh your-name, use run_client rather than calling the client directly (it applies the stall timeout), and assert something that would actually break if the feature regressed. Unit tests are C files in tests/ with a matching Makefile rule.

A test that asserts a mitigation works should first assert the hazard is real — see hazard-real in the suite. Otherwise a change that accidentally disables the hazard makes the mitigation test pass for the wrong reason.

Measurements and claims

This project came out of a paper, and the standard for numbers is the same as it was there:

  • Say what hardware, what concurrency, what the baseline was.
  • Repeat and report a median. make demo does three repetitions per cell because a single run produced a physically impossible table.
  • If a comparison involves cores, equalise the core budget or say clearly that you did not.
  • Never present a number you did not measure. An honest "we did not test this" is worth more than a plausible guess.

Results files live in bench/results/ and record the setup alongside the numbers. Follow that pattern.

Reporting bugs

Include:

  • what you ran, exactly, including every TOFFLOAD_* and ACCEL_* variable
  • TOFFLOAD_STATS=stderr:1 output from the run
  • distribution, kernel, glibc, compiler
  • whether TOFFLOAD_NOFIBER=1 makes it go away — that one line usually localizes the problem immediately

A response failing verification, a lost update where the application holds a lock, or a stall are all correctness bugs. Please label them as such.

Licence

MIT. By contributing you agree your contribution is licensed under it.