[New Model] Add TabLDM foundation model

[occamsX]

Adds TabLDM: a ~70M-parameter in-context-learning tabular foundation model with a dual-stream column embedder and a sparse Mixture-of-Experts (MoE1) backbone, trained on large-scale synthetic tabular data. Like TabPFN, fit does not update weights; it only preprocesses the context and loads the pretrained checkpoint, and prediction runs through a single forward pass.

Wrapper: models/tabldm/model.py
Codebase: https://github.com/occamsX/tabarena
Checkpoints: https://huggingface.co/occams/Xiaomi-TabLDM
Technical report: not released yet (no paper/BibTeX published upstream).

⚠️ Not yet benchmarked on the cluster

has_raw/has_processed/has_results are all False and verified=False in info.py — no benchmark run exists yet.

Local fit is now verified for binary: a CPU-forced smoke fit (ag_args_fit={"num_gpus": 0}) ran _fit's full path end to end (resource negotiation, preprocessing, model_cls(...).fit(X, y)) for both a single-fold fit + refit and a 2-fold bagged fit (sequential_local fold-fitting) + refit, each completing with a real validation accuracy and no errors. Reaching this point needed two environment fixes, unrelated to the wrapper code: the sandbox sits behind a corporate TLS-intercepting proxy whose CA isn't in the venv's bundled certifi store (fixed by pointing REQUESTS_CA_BUNDLE/SSL_CERT_FILE at the system bundle, which already trusts it), and occams/Xiaomi-TabLDM is a gated HF repo ("gated": "auto"), so the checkpoint download needs an HF token from an account that requested and was granted access. TODO(user): confirm benchmark/CI nodes will have such a token available (e.g., HUGGING_FACE_HUB_TOKEN), since without one the checkpoint download returns 403s regardless of network setup.

Changes

• models/tabldm/{model,hpo,info,init}.py — the wrapper + ModelInfo (auto-discovered by the registry; method="TabLDM", ag_key="TA-TABLDM", can_hpo=False, verified=False).
• models/tabldm/_vendor/ — the tabldm inference package (30 files) copied from xiaomi-tabldm, plus enhanced classifier/regressor estimators wrapping the base MoE1 model with an ensemble/calibration pipeline.
• models/init.py — TabLDMModel added to _LAZY_CLASSES.
• website/website_format.py — classified as a foundation model.
• pyproject.toml — tabldm = ["einops"] extra + added to extended.
• ruff.toml — _vendor/ excluded from lint (vendored code, kept as-imported).

Notes

• fit ignores X_val/y_val/time_limit (in-context-learning model, no training loop); preprocessing is handled by the vendored estimator itself.
• Weights and code are Apache-2.0 — no non-commercial restriction, unlike some other recent foundation-model submissions.
• refit_folds=True + sequential_local fold-fitting (matching TabICL/TabSwift/LimiX): refitting one model on all data gives faster inference at similar quality to a bagged ensemble, and sequential fitting avoids fold contention on the shared HF checkpoint cache.
• Only the MoE1 architecture (2 routed experts top-1 + 1 shared expert) is supported by the current inference package; the default config uses the vendored estimators' own checkpoints (clf ≈ 70.1M params/reg ≈ 71.1M params).
• No search space (can_hpo=False) and no warmup() override yet — AbstractTorchModel covers generic torch/CUDA warmup; prefetch_weights() already pre-downloads both checkpoints ahead of the timed fit. TODO(user): decide whether checkpoint prefetch/load should also happen during warmup once a run is possible.
• No tests/tabarena/models/smoke_configs.py override yet. TODO(user): once the checkpoint download is unblocked and a toy fit is verified, decide whether the default config needs a lighter override.
• date="2026-08-31" in info.py is a placeholder (today's date, not a run date). verified=False and no suite/cache_kwargs yet — filled in by the upload flow once a benchmark run exists.

[LennartPurucker]

Heyho, very cool, thank you for your contribution!

Can you report back once you have benchmark results for your method? I will then work on integrating the model and confirming your results.

Moreover, I suggest not vendorizing your own model. This is mostly a workaround for when we don't control the model's codebase. Here, please try to keep your model and interface in their own GitHub codebases controlled by you, make them pip-installable, and then install/import them in the PR.

Also, do you have a blog post or documentation on your method?

[LennartPurucker]

very clean otherwise, cool!

[LennartPurucker]

You can add the install of your own package here

[LennartPurucker]

I am not sure you need this function, the default wrapper should to about the same