Direct Translation from Biblatex to CSL

[Drodt]

Builds on #489.

This PR introduces direct translation from Biblatex entries to CSL variables. This offers more control over the taxonomy of CSL variables and hopefully loses less information than going the route of Biblatex -> hayagriva -> CSL.

Plan

• Finish taxonomy
• Add tests building on Zotero's Better BibTeX

[YDX-2147483647]

I conducted a test against the example entries in the standard GB/T 7714—2025 with https://github.com/citation-style-language/styles/blob/995f064bd45846c000080286186d81d0b97c96cc/china-national-standard-gb-t-7714-2025-numeric.csl and https://github.com/typst-doc-cn/bib-csl-dev-data/blob/42e5c083a0fbc07aa96c15a5a5746b2804c88a9c/data/GB-T_7714—2025.better.bib.
The output of this PR at present (869a88f) is actually worse than https://github.com/typst/hayagriva/releases/v0.10.1.

This PR:

v0.10.1:

State of the art:

The meaning of the metrics, taking the last as an example:

After normalizing punctuation encodings (e.g., full/half-width), the outputs of 164 entries are identical to the standard, and 4 entries are not identical but only differed in letter cases;
For the remaining entries, count the number of different characters (Levenshtein distance) with letter case ignored: 80 entries differ by only one or two characters, 49 entries differ by three to nine characters, 42 entries diff by 10 to 42 characters, and 5 entries diff by more than 42 characters. (42 is 1.5 times the typical number of characters used for recording a DOI in GB/T 7714.)

Test code

The results of v0.10.1 are generated from https://github.com/YDX-2147483647/gb7714-bench/blob/b4ab37c50a2cc9ca9e0a89e10ded6feac1339eda/processors/typst.nu.

The results of this PR are generated by the following codes.

diff --git a/crates/hayagriva-biblatex/Cargo.toml b/crates/hayagriva-biblatex/Cargo.toml
index 40eaee2..976e84c 100644
--- a/crates/hayagriva-biblatex/Cargo.toml
+++ b/crates/hayagriva-biblatex/Cargo.toml
@@ -11,6 +11,7 @@ keywords.workspace = true
 
 [dependencies]
 hayagriva-core = { path = "../hayagriva-core", features = ["biblatex"] }
-biblatex = { workspace = true, features = ["unic-langid"] }
+biblatex = { workspace = true, features = ["unic-langid", "serde"] }
 citationberg = { workspace = true }
 unic-langid = { workspace = true }
+serde_json = "1"
diff --git a/crates/hayagriva-biblatex/src/lib.rs b/crates/hayagriva-biblatex/src/lib.rs
index 958a915..870fd56 100644
--- a/crates/hayagriva-biblatex/src/lib.rs
+++ b/crates/hayagriva-biblatex/src/lib.rs
@@ -14,8 +14,19 @@ use hayagriva_core::{
 };
 use unic_langid::LanguageIdentifier;
 
+pub use biblatex;
+
+#[derive(Debug, Clone, PartialEq, Eq)]
 pub struct Entry(pub biblatex::Entry);
 
+// `biblatex::Entry` doesn't derive `Hash`.
+impl std::hash::Hash for Entry {
+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
+        let bytes = serde_json::to_vec(&self.0).unwrap();
+        bytes.hash(state);
+    }
+}
+
 impl EntryLike for Entry {
     fn resolve_number_variable(
         &self,

# crates/foo/Cargo.toml
[package]
name = "foo"
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
readme.workspace = true
keywords.workspace = true

[dependencies]
hayagriva-archive = { path = "../hayagriva-archive" }
hayagriva-biblatex = { path = "../hayagriva-biblatex" }
hayagriva-csl = { path = "../hayagriva-csl" }
hayagriva-format = { path = "../hayagriva-format" }

// crates/foo/src/main.rs
use hayagriva_archive::locales;
use hayagriva_biblatex::{Entry, biblatex::Bibliography};
use hayagriva_csl::{
    BibliographyDriver, BibliographyRequest, CitationItem, CitationRequest,
};
use hayagriva_format::citationberg::IndependentStyle;

mod strings;
use strings::{GB_7714_2025_NUMERIC_COMPLIANT_CSL, GB_T_7714_2025_BETTER_BIB};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let locales = locales();
    let style = IndependentStyle::from_xml(GB_7714_2025_NUMERIC_COMPLIANT_CSL)?;
    let entries: Vec<Entry> = Bibliography::parse(GB_T_7714_2025_BETTER_BIB)?
        .into_iter()
        .map(|entry| Entry(entry))
        .collect();

    let mut driver: BibliographyDriver<'_, Entry> = BibliographyDriver::new();
    driver.citation(CitationRequest::new(
        entries.iter().map(CitationItem::with_entry).collect(),
        &style,
        None,
        &locales,
        Some(1),
    ));

    let result = driver.finish(BibliographyRequest {
        style: &style,
        locale: None,
        locale_files: &locales,
    });

    let mut output = String::new();
    for row in result.bibliography.map(|b| b.items).unwrap_or_default() {
        if let Some(prefix) = row.first_field {
            output.push_str(&format!("{prefix:#} "));
        }
        output.push_str(&format!("{:#}\n", row.content));
    }
    println!("{output}");

    Ok(())
}

crates/foo/src/strings.rs (renamed as .txt)

Detailed comparison

1. Many [J] (journal in GB/T 7714, article-journal in CSL) become [PP] (preprint in GB/T 7714, article in CSL), because this PR makes many entries match multiple CSL entry types, and the CSL style cannot handle that reliably.

left = v0.10.1 and right = this PR.

2. The nonstand bib @standard entry type is still not mapped to the CSL standard entry type. (Consider adding all non-standard BibLaTeX types biblatex#62, All (Misc?) entries with URL are recognized as webpage (in gb-7714-2015-numeric?) #312)

   GB/T 7714 uses [S] for standards and [EB] for web pages.

3. Some fields get lost.

left = v0.10.1 and right = this PR.

4. The URLs are no longer normalized.

left = v0.10.1 and right = this PR.

However, this does not deny the value of direct bib→csl translation; it merely indicates that this approach requires much more testing.

[Drodt]

Thanks for setting up those tests but this PR is in a very early state. I do not recommend wasting your time on tests just yet. At this point, this PR just marks what we have planned and to collect some input.

Tbh, I'm surprised the PR works that well in your tests. I don't even consider parents of entries at the moment. I'll keep working on it and it will be a higher priority once #489 is merged.

And to selfishly defend my approach here: I already find the new approach easier to manage. To write this PR, I had to look at the current state. Which means understanding the very complex biblatex -> Hayagriva translation and Hayagriva's CSL taxonomy and connect the two.