TactiWeave-VP

中文

TactiWeave-VP classifies fabrics from aligned tactile image sequences and six calibrated physical signals. The model combines a shared single-channel ResNet-18, a physical-state MLP, frame-wise feature fusion, a causal TCN, attention pooling, and a classifier.

Data contract

The project uses one dataset layout only:

data/
├── train/
│   └── <class>/
│       ├── image/<frame>.png
│       └── features/meta.json
├── val/
│   └── <class>/
│       ├── image/<frame>.png
│       └── features/meta.json
└── test/
    └── <class>/
        ├── image/<frame>.png
        └── features/meta.json

train, val, and test must all exist, contain data, and have exactly the same class-directory names. Class names are sorted and mapped to the compact range 0..C-1; this mapping is saved in every checkpoint. Keep data.classes: null in the training configuration so the complete class set is discovered and validated from the split directories.

The repository's val and test directories are currently empty placeholders. Training will report an error until all three splits have been collected. Nothing automatically creates, splits, or migrates existing data.

See Data pipeline for the complete on-disk and preprocessing contract.

Installation

Python 3.10 or newer is required.

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Choose the PyTorch wheel appropriate for the target CPU/CUDA environment if the pinned wheel in requirements.txt is not suitable.

Acquisition and real-time inference import the vendor package as sightac_sdk. Place the complete SDK directory at the repository root as sightac_sdk/ and retain its internal layout. Install the sensor driver and any native camera dependencies required by the SDK before connecting the device.

Quick start

Collect each class separately for every split. The data root is explicit:

python srcs/data_acquisition.py --data-root data --dataset train --fabric 1 --frame-count 2000
python srcs/data_acquisition.py --data-root data --dataset val   --fabric 1 --frame-count 400
python srcs/data_acquisition.py --data-root data --dataset test  --fabric 1 --frame-count 400

Press S to start or pause recording and Q or Esc to stop. Starting a new acquisition for the same split/class replaces that session's PNG files and meta.json; it does not append to the old session.

After all three splits contain the same classes, train with:

python srcs/train.py --config srcs/training_config.yaml --data-root data

Run offline inference on one direct session directory such as data/train/1:

python srcs/offline_inference.py `
  --checkpoint results/<run>/best.pt `
  --input-dir data/train/1 `
  --output results/<run>/offline.json

--input-dir is not a dataset root: do not pass data/. Run real-time inference with:

python srcs/realtime_inference.py --checkpoint results/<run>/best.pt --usb-id 1

Training creates a unique directory below results/ containing effective configuration, logs, metric history, summaries, and best.pt/last.pt checkpoints. Offline inference prints the session prediction and can write JSON; real-time inference displays rolling predictions or prints them with --no-display.

Only load checkpoints you trust. PyTorch checkpoint loading uses pickle-compatible serialization, and a malicious checkpoint may execute code. The inference entry points accept schema-v1 checkpoints produced by this project's srcs/train.py.

Documentation

• Usage
• Data pipeline
• Model structure

License

See LICENSE.