Skip to content

ATTRIBUTIONS.md

Latest commit

 

History

History
613 lines (444 loc) · 29 KB

ATTRIBUTIONS.md

File metadata and controls

613 lines (444 loc) · 29 KB
Raw

Attributions

This document acknowledges sources of vendored code, runtime artifacts, test documents, and baseline data used in the Xberg project.

Pandoc Test Suite

Test documents and reference baseline outputs derived from the Pandoc test suite:

  • Source: https://github.com/jgm/pandoc
  • License: GPL-2.0-or-later
  • Usage: Test documents and reference baselines only (no code copied from Pandoc)
  • Attribution: John MacFarlane and Pandoc contributors
  • Purpose: Baseline reference testing - used to validate our native Rust extractors work correctly on the same documents that Pandoc processes

Test Documents from Pandoc

The following test documents were copied from the Pandoc repository to /test_documents/:

Org Mode

  • org-select-tags.org - SELECT_TAGS and EXCLUDE_TAGS testing
  • pandoc-tables.org - Org Mode table formats
  • pandoc-writer.org - Comprehensive Pandoc test suite in Org Mode format

Typst

  • typst-reader.typ - Fibonacci sequence with mathematical formulas
  • undergradmath.typ - Comprehensive undergraduate mathematics document (16KB)

DocBook

  • docbook-chapter.docbook - Recursive section hierarchy (7 nested levels)
  • docbook-reader.docbook - Comprehensive DocBook 4.4 test suite (36KB, 1704 lines)
  • docbook-xref.docbook - Cross-reference (xref) functionality testing

JATS

  • jats-reader.xml - Comprehensive JATS (Z39.96) Journal Archiving test document (38KB, 1460 lines)

FictionBook

  • test_documents/fictionbook/pandoc/ - 13 FictionBook test files including:
    • basic.fb2 - Basic FictionBook structure
    • images-embedded.fb2 - Embedded base64 images
    • math.fb2 - Mathematical content
    • meta.fb2 - Document metadata testing
    • reader/emphasis.fb2 - Text emphasis testing
    • reader/epigraph.fb2 - Epigraph/quote elements
    • reader/meta.fb2 - Document metadata and title info
    • reader/notes.fb2 - Footnotes/endnotes with cross-references
    • reader/poem.fb2 - Poem/verse structure
    • reader/titles.fb2 - Section titles and heading hierarchy
    • And others

OPML

  • opml-reader.opml - OPML 2.0 outline structure (US states example)
  • pandoc-writer.opml - Comprehensive Pandoc test suite in OPML format

Baseline Outputs Generated

For each test document listed above, three baseline outputs were generated using Pandoc 3.8.3:

  1. Plain Text (*_pandoc_baseline.txt) - Raw text content extraction
  2. JSON Metadata (*_pandoc_meta.json) - Full Pandoc AST with document structure and metadata
  3. Markdown (*_pandoc_markdown.md) - Markdown representation for format comparison

Total: 132 baseline files for 44 documents across 6 formats

GPL Compliance Statement

We acknowledge that Pandoc is licensed under GPL-2.0-or-later. We have:

  • ✓ Used Pandoc's test documents (test data is allowed under GPL)
  • ✓ Generated baseline outputs using Pandoc for comparison purposes
  • ✓ NOT copied any Pandoc source code
  • ✓ Implemented our extractors independently in Rust
  • ✓ Used Pandoc only as a behavioral baseline for testing

Our Rust extractors are independently implemented and do not contain any GPL-licensed code from Pandoc.

Verification

Test documents and baselines can be regenerated at any time using:

./generate_pandoc_baselines.sh

This script processes all test documents and generates fresh baselines using the installed version of Pandoc.

docx-lite

DOCX XML parser vendored into crates/xberg/src/extraction/docx/parser.rs:

  • Source: https://github.com/v-lawyer/docx-lite
  • License: MIT OR Apache-2.0
  • Authors: V-Lawyer Team
  • Version: 0.2.0 (vendored with modifications)
  • Usage: DOCX text extraction parser inlined into xberg core
  • Modifications:
    • Fixed Paragraph::to_text() joining text runs without whitespace (#359)
    • Adapted to xberg's quick-xml v0.39 and zip v7.x APIs
    • Removed file-path based APIs (only bytes/reader needed)

xberg-gliner

Span-mode GLiNER preprocessing, prompt construction, tensor construction, and decoding logic vendored into crates/xberg-gliner/:

  • Source lineage: gline-rs → xberg-gliner
  • Source URLs: https://github.com/fbilhaut/gline-rs, https://github.com/xberg-io/gline-rs-fork
  • Source crate/version: xberg-gliner, vendored from gline-rs 0.2.1 (fork at xberg-io/gline-rs-fork)
  • License: Apache-2.0
  • Author: Frédérik Bilhaut
  • Usage: Local ONNX named-entity recognition backend for xberg
  • Scope: Span-mode text splitting, zero-shot prompt construction, Hugging Face tokenizer adapter, GLiNER tensor construction, logits decoding, and greedy overlap filtering
  • Excluded: ORP pipeline code, examples, CSV helpers, relation extraction, token-mode runtime, memory profiling, and generic pipeline abstractions
  • Modifications:
    • Adapted crate identity to xberg-gliner
    • Removed ORP dependency and replaced the pipeline wrapper with direct ort session creation and synchronized inference
    • Integrated with xberg's model catalog/cache and xberg-io/gliner-models artifact layout
    • Added focused unit tests for validation, splitting, prompt construction, tensor shapes, decoding, and overlap filtering

GLiNER Models

GLiNER ONNX runtime artifacts consumed by xberg are exported and governed through xberg-io/gliner-models. Their source model lineage is the gliner-community Hugging Face organization.

  • Upstream source: https://huggingface.co/gliner-community
  • Runtime artifact repository: https://huggingface.co/xberg-io/gliner-models
  • Default license: Apache-2.0 unless a model manifest declares otherwise
  • Original GLiNER authors: Urchade Zaratiana, Nadi Tomeh, Pierre Holat, Thierry Charnois
  • Usage: ONNX weights, tokenizer files, manifests, checksums, and notices for xberg NER
  • Policy: Model weights are not committed to this repository; verified artifacts are downloaded into the xberg model cache

hwpers

Vendored HWP text extraction code from the hwpers crate:

  • Source: https://github.com/Indosaram/hwpers
  • License: MIT OR Apache-2.0
  • Authors: HWP Parser Contributors
  • Vendored Version: 0.5.0
  • Location: crates/xberg/src/extraction/hwp/
  • Purpose: Text extraction from Korean Hangul Word Processor (.hwp) files
  • Scope: Minimal subset — CFB reader, binary record parser, text extraction only
  • Excluded: HWPX (XML/ZIP), writer, renderer, crypto, preview modules

paddle-ocr-rs

Vendored source code from the paddle-ocr-rs crate for PaddleOCR via ONNX Runtime integration:

Vendored Files

The following source files were vendored from paddle-ocr-rs:

  • ocr_lite.rs - Core OCR pipeline and high-level API
  • db_net.rs - DBNet text detection network
  • crnn_net.rs - CRNN text recognition network
  • angle_net.rs - Text angle detection network
  • base_net.rs - Base network trait
  • ocr_utils.rs - Image preprocessing utilities
  • ocr_result.rs - Result type definitions
  • scale_param.rs - Scaling parameter calculations
  • ocr_error.rs - Error type definitions

Modifications

The vendored code has been modified for Xberg integration:

  • Updated to Rust 2024 edition
  • Aligned with Xberg workspace dependencies
  • License changed to MIT with dual copyright (original author retained)

License Compatibility

The original Apache-2.0 license is compatible with MIT relicensing. The original copyright and attribution are preserved in the vendored crate's LICENSE file.

fastembed-rs

Text embedding inference pipeline vendored into crates/xberg/src/embeddings/engine.rs:

  • Source: https://github.com/Anush008/fastembed-rs
  • License: Apache-2.0
  • Author: Anush008 and contributors
  • Vendored Version: Based on 0.2.x
  • Location: crates/xberg/src/embeddings/engine.rs
  • Purpose: ONNX-based text embedding inference with thread-safe concurrent embedding generation

Modifications

The vendored code has been modified from the original fastembed-rs:

  • Changed embed() method signature from &mut self to &self for thread-safe concurrent inference without mutex contention
  • Adapted to Xberg's ONNX Runtime integration and error handling
  • Integrated with Xberg's embedding configuration and model management

License Compatibility

The original Apache-2.0 license is fully compatible with Xberg's Elastic License 2.0 (ELv2). The original copyright and attribution are preserved in the vendored code's comments.

numbers-parser Test Fixtures

Test documents derived from the numbers-parser test suite:

  • Source: https://github.com/masaccio/numbers-parser
  • License: MIT
  • Author: Jon Connell (masaccio)
  • Usage: Test documents and reference baselines only (no code copied)
  • Modifications: Fixtures downloaded directly for integration testing.
  • Location: test_documents/iwork/

yake-rust

YAKE keyword extraction algorithm vendored into xberg:

  • Source: https://github.com/quesurifn/yake-rust
  • License: MIT
  • Authors: Kyle Fahey, Anton Vikstrom, Igor Strebz
  • Vendored Version: 1.0.3
  • Location: crates/xberg/src/keywords/yake/
  • Purpose: YAKE (Yet Another Keyword Extractor) statistical keyword extraction

Modifications

  • Replaced segtok dependency with custom memchr-based sentence splitter (fixes #676 BacktrackLimitExceeded on large files)
  • Integrated with xberg's stopwords module (64 languages vs original 34)
  • Replaced hashbrown with ahash, inlined streaming-stats and levenshtein
  • Optimized punctuation checks with byte lookup tables
  • Removed itertools dependency (manual dedup)

License Compatibility

The original MIT license is compatible with Xberg's Elastic License 2.0 (ELv2).

text-splitter (inlined)

The chunking submodule crates/xberg/src/chunking/text_splitter/ is a trimmed inline copy of text-splitter v0.30.1 by Benjamin Brandt. We inlined it because upstream pins tokenizers = "0.22", which conflicts with xberg's direct tokenizers 0.23 dependency and pulls a duplicate copy of tokenizers into the build graph (breaking the Tokenizer: ChunkSizer bound in chunking::core).

Modifications

  • Dropped the code (tree-sitter) splitter — xberg has its own tree-sitter integration and does not use the upstream code splitter.
  • Dropped the tiktoken-rs sizer — unused.
  • Rebuilt against tokenizers 0.23.
  • Renamed feature gate tokenizers → chunking-tokenizers; the markdown splitter is always available because pulldown-cmark is already a non-optional xberg dependency.
  • Tightened visibility on internal types to pub(crate).
  • Path rewiring: upstream crate::* paths inside the inlined module rewritten relative to the new submodule root.

License Compatibility

The MIT license is compatible with Xberg's Elastic License 2.0 (ELv2). The full upstream license text is reproduced below:

MIT License

Copyright (c) 2023 Benjamin Brandt

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Test Documents from text-splitter

The following test inputs were copied from the text-splitter repository to /test_documents/text_splitter/:

  • text/romeo_and_juliet.txt — Shakespeare, public domain (Project Gutenberg)
  • text/room_with_a_view.txt — E. M. Forster, public domain (Project Gutenberg)
  • markdown/commonmark_spec.md — CommonMark spec, CC-BY-SA-4.0
  • markdown/github_flavored.md — GitHub Flavored Markdown spec, CC-BY-4.0

libheif-rs

Safe Rust bindings around libheif-sys for decoding HEIF / HEIC / AVIF containers, vendored as the xberg-libheif crate:

  • Source: https://github.com/Cykooz/libheif-rs
  • License: MIT
  • Author(s): Kirill Kuzminykh (Cykooz) and contributors
  • Vendored Version: 2.7.0
  • Location: crates/xberg-libheif/
  • Purpose: Decode HEIF-family containers (HEIC, HEIF, AVIF, HEICS, AVCS) to interleaved RGBA pixels and expose EXIF / XMP metadata blocks for xberg's image-extraction and OCR pipeline.

Vendored Files / Scope

  • Full upstream src/ tree (LibHeif, HeifContext, ImageHandle, Image, decoder, encoder, color profile, regions, metadata, reader, security limits, track, utils, and the optional image-crate integration module).
  • Vendored verbatim from upstream v2.7.0 so the public API mirrors libheif-rs exactly; we continue to depend on upstream libheif-sys from crates.io for the underlying C bindings.

Modifications

  • Workspace dependency alignment (libc, image pinned via the workspace).
  • Rust 2024 edition (upstream is 2021).
  • Replaced upstream include_str!("../README.md") doc shim with an inline module-level vendoring header pointing at this file.
  • Workspace clippy / rust lints applied via [lints] workspace = true.

License Compatibility

MIT is permissive and compatible with re-distribution alongside Xberg's Elastic License 2.0 (ELv2) workspace. The upstream MIT license text is preserved verbatim at crates/xberg-libheif/LICENSE.

System Library Requirement

libheif-rs is a safe wrapper around the C library libheif, which in turn requires libde265 (HEVC) and libaom (AV1). These must be available at build and runtime. libheif is LGPL-licensed and is linked dynamically only (never statically vendored). Pixel decoding is therefore native-only — the heic feature is excluded from xberg's wasm-target and android-target aggregate features. EXIF extraction via nom-exif is pure Rust and works on every target.

pdfium-render

High-level idiomatic Rust wrapper around Pdfium, forked and vendored for xberg:

  • Source: https://github.com/ajrcarey/pdfium-render
  • License: MIT OR Apache-2.0 upstream; the fork takes the MIT option and is redistributed under MIT, retaining the upstream copyright notice below.
  • Author: Alastair Carey (alastair@alastaircarey.com), Copyright (c) 2022-2026
  • Forked Version: 0.8.x-era, with bindings generated against pdfium 7678 (src/bindings/version.rs, src/bindgen/pdfium_7678.rs); the runtime binary pin used by scripts/download_pdfium_runtime.sh and .task/languages/rust.yml is a newer, verified-compatible 7881
  • Location: crates/xberg-pdfium-render/
  • Purpose: PDF structure-tree and content-mark access, text extraction and rendering via Google's Pdfium library

Modifications

  • Updated to Rust 2024 edition and workspace conventions
  • Added musl static linking support in build.rs
  • Removed examples and test data (kept in the upstream repo)
  • Added struct-tree and content-mark wrappers (~1,036 lines across struct_element.rs, struct_tree.rs, content_mark.rs, content_marks.rs) that upstream does not provide — upstream's PdfPageStructureTree exposes no public constructor and has no PdfStructElement type at all

License Compatibility

The upstream MIT OR Apache-2.0 dual license permits redistribution under MIT alone. The upstream author and copyright are recorded in this file, which is distributed with the source.

PDFOxide

Pure-Rust PDF parsing, extraction and rendering — xberg's default PDF engine, forked and vendored:

  • Source: https://github.com/yfedoseev/pdf_oxide
  • License: MIT OR Apache-2.0 upstream; this fork elects the MIT arm and is redistributed under MIT, retaining the upstream copyright notice below.
  • Author: Yury Fedoseev, Copyright (c) 2025-present
  • Forked Version: forked pre-1.0, published independently as xberg-pdf-oxide 1.0.1 on crates.io, then vendored here from that repository's development branch at commit 196ec71a1cfd6e818e1498053584ff93c794bde1. No git history was carried across, so that SHA is the provenance link; the source repository has since been retired.
  • Location: crates/xberg-native-pdf/ (crate xberg-native-pdf, lib xberg_native_pdf)
  • Purpose: text extraction, reading order, tables, forms, annotations, XFA, encryption and page rendering — everything behind PdfBackend::Native

Modifications

  • Renamed to xberg-native-pdf on vendoring. Upstream's trademark policy permits describing this crate as a fork of PDFOxide but not using upstream's names as the name of a distribution, so the rename is a compliance improvement as well as a naming one. This file, crates/xberg-native-pdf/ATTRIBUTIONS.md, and crates/xberg-native-pdf/CHANGELOG.md are the places that document the upstream project by design. Beyond those, the identifier pdf_oxide (or a variant spelling) survives, deliberately, in a small number of live, commented, load-bearing spots — confirm with grep -rlniE 'pdf[_-]?oxide' crates/:

    • crates/xberg/src/core/config/pdf.rs (PdfBackend's rejection test): the old spelling must appear verbatim so the test can assert it is rejected, not silently aliased.
    • crates/xberg-native-pdf/src/document.rs (a /Producer allowlist entry): matches PDFs already written by the pdf_oxide/xberg-pdf-oxide lineages, which are still arriving in the wild, so recognizing them as trusted authoring tools is intentional.
    • crates/xberg-native-pdf/src/decoders/flate.rs (an environment-variable back-compatibility chain): PDF_OXIDE_MAX_DECOMPRESS_MB and XBERG_PDF_OXIDE_MAX_DECOMPRESS_MB keep older deployments' env vars working.
    • crates/xberg-native-pdf/tests/test_signed_pdf_opens_and_renders.rs: a verbatim quotation of a third-party signing tool's own error message (PdfOxide.Exceptions.SignatureException), not a reference to this codebase.

    The same grep also turns up crates/xberg-php/ and crates/xberg-node/ (alef-generated binding sources) and every docs-site/src/content/docs/reference/api-*.md / cli.md/configuration.md/types.md page: these are mechanically generated from the Rust doc comments above and were stale (pre-dating the xberg-native-pdf rename in the comments they mirror) at the time of writing; they carry no independent attribution and refresh on the next alef generate, not by hand-editing. Historical entries in the root CHANGELOG.md (and its docs-site/src/content/docs/changelog.md mirror) also name pdf_oxide/xberg-pdf-oxide; those describe releases from when the crate really was named that, and changelogs are an append-only historical record, so they are left as written rather than retroactively edited.

  • Stripped upstream's language bindings, CLI, MCP server, and its OCR, ML, digital-signature, WASM, barcode, HTML/CSS-to-PDF and PDF/A subsystems

  • Replaced ttf-parser with the fontations/skrifa stack

  • Migrated diagnostics from the log facade to tracing; the target root is exported as xberg_native_pdf::LOG_TARGET_ROOT so consumers filter on a value rather than a literal

  • Rust 2024 edition and workspace conventions

  • Per-release detail in crates/xberg-native-pdf/CHANGELOG.md

Vendored and bundled third-party code

crates/xberg-native-pdf/ATTRIBUTIONS.md is the authoritative record and ships with the crate. It covers src/vendor/fontdb/ (derived from fontdb 0.24.0, Yevhenii Reizner, MIT) and the fonts embedded via include_bytes! under src/fonts/assets/: DejaVuSans and DejaVuSans-Bold (Bitstream Vera / DejaVu licence), DroidSansFallbackFull.ttf (Apache-2.0) and NotoEmoji-Regular.ttf (SIL OFL 1.1), the latter two behind the cjk-form-fonts feature. The DejaVu licence requires renaming modified fonts; these are unmodified and must stay so.

License Compatibility

Electing the MIT arm of an MIT-OR-Apache-2.0 dual offer is permitted by both licences. The upstream copyright notice is preserved verbatim at crates/xberg-native-pdf/LICENSE, which is distributed with the source. No endorsement by or affiliation with the upstream project is implied.

libwpd + librevenge

Native WordPerfect document libraries built from source and statically linked into the xberg-libwpd crate:

  • Source: https://libwpd.sourceforge.net/
  • License: MPL-2.0 OR LGPL-2.1+ (xberg builds under the MPL-2.0 arm)
  • Authors: Document Liberation Project (LibreOffice/AbiWord ecosystem) contributors
  • Versions: librevenge 0.0.6, libwpd 0.10.3
  • Location: vendored as committed .tar.gz archives under crates/xberg-libwpd/vendor/, decompressed and compiled by crates/xberg-libwpd/build.rs (checksum-verified at extract time); consumed via crates/xberg-libwpd/src/shim.cpp
  • Purpose: WordPerfect (.wpd) document extraction
  • Modifications: none; both libraries are built from unmodified upstream source

License Compatibility

See THIRD_PARTY_LICENSES.md for the full MPL-2.0 vs LGPL-2.1+ linking analysis. In short: xberg statically links under the MPL-2.0 arm (file-level copyleft, static linking explicitly permitted), so no copyleft obligation reaches xberg's MIT-licensed core or language bindings.

Boost C++ Libraries (build-time, xberg-libwpd)

A header-only subset of Boost, used at build time by the xberg-libwpd C++ shim:

  • Source: https://www.boost.org/
  • License: BSL-1.0 (Boost Software License 1.0)
  • Author: Boost contributors
  • Location: bcp-extracted subset vendored as crates/xberg-libwpd/vendor/boost-subset.tar.gz, decompressed by crates/xberg-libwpd/build.rs
  • Purpose: header-only dependency of libwpd/librevenge (build-time only; no Boost code is redistributed in xberg binaries)
  • Modifications: none; subset extracted verbatim with bcp

jhqxxx/aha

Rust-native VLM-OCR backends vendored into crates/xberg-candle-ocr/:

  • Source: https://github.com/jhqxxx/aha
  • License: Apache-2.0
  • Author: jhqxxx
  • Vendored Version: e29ddc589d089042afd66ab8ea76409d8d33f701 (jhqxxx/aha @ HEAD on 2026-06-17)
  • Location: crates/xberg-candle-ocr/src/vendor/aha/ (shared infra), crates/xberg-candle-ocr/src/models/deepseek_ocr/ (DeepSeek-OCR + Qwen2 decoder), crates/xberg-candle-ocr/src/models/paddleocr_vl/ (PaddleOCR-VL 1.5 upgrade)
  • Purpose: Rust-native VLM-OCR backends (PaddleOCR-VL 1.5, DeepSeek-OCR) including their Qwen2 decoder dependency and shared SigLIP/CLIP/MRoPE infrastructure subsets.

Vendored Files

File Source Notes
(populated as Phases 3-4 land each file)

Modifications

  • anyhow::Result<T> → Result<T, CandleOcrError> throughout
  • assert!/assert_eq!/panic!/unwrap() on user-controlled values → typed CandleOcrError returns
  • Candle 0.9.2 → Candle 0.10 API migration
  • Generation loop routed through xberg_candle_ocr::models::glm_ocr::mtp::generate_mrope (with a new forward_step_with_position_ids trait extension per A3) — NOT aha's common/generate.rs
  • rustdoc on every pub item
  • Dead code blocks dropped (specific examples: aha modules.rs:214-230, 265-266)
  • paddleocr_vl::model::get_rope_index: vision-block position ids restored to aha's (t, h, w) mrope layout after the vendoring rewrite transposed the h index and dropped the t row (crashed on non-square grids); images always carry a single frame so the t row is constant
  • paddleocr_vl::model::forward: restored aha's seqlen_offset-based position continuation for cache_position-less decode steps (dropped during vendoring, failing every decode step with a dtype mismatch)
  • paddleocr_vl::engine::generate: rewritten from a full-sequence re-feed (which double-appended the KV cache each step and read the argmax at the wrong rank) to aha's prefill + cached single-token step loop
  • paddleocr_vl::model::Ernie4_5Model::forward: restored aha's causal attention mask for the multi-token prefill (dropped during vendoring; bidirectional prefill degenerated generation into single-token repetition)

License Compatibility

Apache-2.0 is compatible with Xberg's Elastic License 2.0 (ELv2). Original Apache-2.0 copyright headers are preserved at file-top in each vendored file.

Last Updated: 2026-06-17

Self-hosted retrieval models

xberg re-hosts ONNX weights for 17 upstream retrieval models across four xberg-io/* Hugging Face repos and serves them as embedding/reranker/sparse/late-interaction presets. Weights are re-hosted unmodified (unchanged upstream weights, xberg-added ONNX export/manifest/tokenizer artifacts only) under the xberg-io org. All upstream licenses are permissive (Apache-2.0 or MIT), consistent with xberg's permissive-only dependency policy.

xberg-io/embedding-models

Dense sentence-embedding presets:

Upstream model License Preset
sentence-transformers/all-MiniLM-L6-v2 Apache-2.0 fast
BAAI/bge-base-en-v1.5 MIT balanced
BAAI/bge-large-en-v1.5 MIT quality
intfloat/multilingual-e5-base MIT multilingual
Alibaba-NLP/gte-modernbert-base Apache-2.0 gte-modernbert-base (default)
minishlab/potion-base-8M MIT lightweight (model2vec static)
Snowflake/snowflake-arctic-embed-m-v2.0 Apache-2.0 arctic-embed-m-v2.0
Qwen/Qwen3-Embedding-0.6B Apache-2.0 qwen3-embedding-0.6b

xberg-io/reranker-models

Cross-encoder reranker presets:

Upstream model License Preset
BAAI/bge-reranker-base MIT —
BAAI/bge-reranker-v2-m3 Apache-2.0 —
jinaai/jina-reranker-v1-turbo-en Apache-2.0 —
Qwen/Qwen3-Reranker-0.6B Apache-2.0 —
cross-encoder/ettin-reranker-150m-v1 Apache-2.0 balanced (default alias)

xberg-io/sparse-embeddings

SPLADE-family sparse embedding presets:

Upstream model License Preset
prithivida/Splade_PP_en_v1 Apache-2.0 splade
opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill Apache-2.0 opensearch-v3-distill (default)

xberg-io/late-interaction-models

ColBERT-family late-interaction presets:

Upstream model License Preset
answerdotai/answerai-colbert-small-v1 Apache-2.0 colbert
lightonai/GTE-ModernColBERT-v1 Apache-2.0 gte-moderncolbert (default)

Policy

Model weights are not committed to this repository; verified artifacts are downloaded into the xberg model cache at runtime, matching the GLiNER Models policy above.

Last Updated: 2026-07-08

Last Updated: June 17, 2026 Pandoc Version Used: 3.8.3 Baseline Generation Date: December 6, 2025