Fix failing tests due to timeout because of LaTeX update breaking tufte doc

Somehow we have some failing test in some situations

running 2 tests from ./smoke/render/render-page-layout.test.ts
[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf ...'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (1m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (2m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (4m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (8m0s)
 FAILED (10m0s)
[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html ...'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (1m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (2m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (4m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (8m0s)
 FAILED (10m0s)

Ful log

running 2 tests from ./smoke/render/render-page-layout.test.ts
[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf ...'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (1m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (2m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (4m0s)
'[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf' has been running for over (8m0s)
 FAILED (10m0s)
[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html ...'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (1m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (2m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (4m0s)
'[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html' has been running for over (8m0s)
 FAILED (10m0s)

ERRORS 

[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf => ./test.ts:346:8
error: AssertionError: Failed assertion: 

--------------------------------------------------------------------------------
[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf
          ./run-tests.sh smoke/render/render-page-layout.test.ts

[verify] > Output Created

No output created message
AssertionError: No output created message
    at Module.assert (https://jsr.io/@std/assert/0.224.0/assert.ts:18:11)
    at assert (https://jsr.io/@std/testing/0.224.0/asserts.ts:608:11)
    at Object.verify (file:///home/runner/work/quarto-cli/quarto-cli/tests/verify.ts:316:7)
    at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:256:25)
    at eventLoopTick (ext:core/01_core.js:218:9)
    at async innerWrapped (ext:cli/40_test.js:180:5)
    at async outerWrapped (ext:cli/40_test.js:123:14)

OUTPUT:
    - The project is out-of-sync -- use `renv::status()` for details.
    
    
    
    processing file: tufte-pdf.qmd
    
    1/29                        
    
    2/29 [fig-margin]           
    
    3/29                        
    
    4/29 [fig-fullwidth]        
    
    5/29                        
    
    6/29 [fig-main]             
    
    7/29                        
    
    8/29 [fig-main-margin-cap]  
    
    9/29                        
    
    10/29 [unnamed-chunk-1]      
    
    11/29                        
    
    12/29 [fig-two-together]     
    
    13/29                        
    
    14/29 [fig-two-separate]     
    
    15/29                        
    
    16/29 [fig-margin-together]  
    
    17/29                        
    
    18/29 [unnamed-chunk-2]      
    
    19/29                        
    
    20/29 [fig-margin-separate-a]
    
    21/29                        
    
    22/29 [unnamed-chunk-3]      
    
    23/29                        
    
    24/29 [unnamed-chunk-4]      
    
    25/29                        
    
    26/29 [unnamed-chunk-5]      
    
    27/29                        
    
    28/29 [unnamed-chunk-6]      
    
    29/29                        
    
    output file: tufte-pdf.knit.md
    
    
    pandoc 
    to: latex
    output-file: tufte-pdf.tex
    standalone: true
    pdf-engine: lualatex
    variables:
      graphics: true
      tables: true
    default-image-extension: pdf
    reference-location: margin
    
    metadata
    documentclass: scrartcl
    classoption:
      - DIV=11
      - numbers=noendperiod
    papersize: letter
    header-includes:
      - \KOMAoption{captions}{tableheading}
    block-headings: true
    title: A Quarto Page Layout Example
    subtitle: Inspired by Tufte Handout, Using Quarto
    date: today
    citation-location: margin
    bibliography:
      - skeleton.bib
    
    
    Rendering PDF
    running lualatex - 1
    timed out after 10 minutes
    
    Stack trace:
        at asErrorEx (file:///home/runner/work/quarto-cli/quarto-cli/src/core/lib/error.ts:74:12)
        at logError (file:///home/runner/work/quarto-cli/quarto-cli/src/core/log.ts:348:15)
        at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:239:13)
        at eventLoopTick (ext:core/01_core.js:218:9)
        at async innerWrapped (ext:cli/40_test.js:180:5)
        at async outerWrapped (ext:cli/40_test.js:123:14)
    throw new AssertionError(msg);
          ^
    at assert (https://jsr.io/@std/assert/0.224.0/assert.ts:18:11)
    at Module.fail (https://jsr.io/@std/assert/0.224.0/fail.ts:17:3)
    at fail (https://jsr.io/@std/testing/0.224.0/asserts.ts:663:11)
    at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:320:11)

[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html => ./test.ts:346:8
error: AssertionError: Failed assertion: 

--------------------------------------------------------------------------------
[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html
          ./run-tests.sh smoke/render/render-page-layout.test.ts

[verify] > Output Created

No output created message
AssertionError: No output created message
    at Module.assert (https://jsr.io/@std/assert/0.224.0/assert.ts:18:11)
    at assert (https://jsr.io/@std/testing/0.224.0/asserts.ts:608:11)
    at Object.verify (file:///home/runner/work/quarto-cli/quarto-cli/tests/verify.ts:316:7)
    at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:256:25)
    at eventLoopTick (ext:core/01_core.js:218:9)
    at async innerWrapped (ext:cli/40_test.js:180:5)
    at async outerWrapped (ext:cli/40_test.js:123:14)

OUTPUT:
    timed out after 10 minutes
    
    Stack trace:
        at asErrorEx (file:///home/runner/work/quarto-cli/quarto-cli/src/core/lib/error.ts:74:12)
        at logError (file:///home/runner/work/quarto-cli/quarto-cli/src/core/log.ts:348:15)
        at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:239:13)
        at eventLoopTick (ext:core/01_core.js:218:9)
        at async innerWrapped (ext:cli/40_test.js:180:5)
        at async outerWrapped (ext:cli/40_test.js:123:14)
    throw new AssertionError(msg);
          ^
    at assert (https://jsr.io/@std/assert/0.224.0/assert.ts:18:11)
    at Module.fail (https://jsr.io/@std/assert/0.224.0/fail.ts:17:3)
    at fail (https://jsr.io/@std/testing/0.224.0/asserts.ts:663:11)
    at fn (file:///home/runner/work/quarto-cli/quarto-cli/tests/test.ts:320:11)

 FAILURES 

[smoke] > quarto render docs/page-layout/tufte-pdf.qmd --to pdf => ./test.ts:346:8
[smoke] > quarto render docs/page-layout/tufte-html.qmd --to html => ./test.ts:346:8

FAILED | 0 passed | 2 failed (20m0s)

This is due to timeout and it makes our CI fail but no always/

Main is passing
Latest PR on main is passing
I have one PR on 1.8 not passing due to this
Our timing of tests (sequential timed test run one after the others) that fails because of this (https://github.com/quarto-dev/quarto-cli/actions/runs/19023126991/job/54321943546)

Locally documents renders fine and test seems to ./smoke/render/render-page-layout.test.ts at least on windows.

Investigation required.

So it works locally too on Ubuntu.

Somehow we have similar issue in our broken smoke-all non parallelized. Some random test like are failling because running forever.

[smoke] > quarto render docs/smoke-all/book/htmlbook/appendix-2.qmd --to asciidoc ... FAILED (10m0s)
[smoke] > quarto render docs/smoke-all/book/htmlbook/chap-1-1.qmd --to asciidoc ...'[smoke] > quarto render docs/smoke-all/book/htmlbook/chap-1-1.qmd --to asciidoc' has been running for over (1m0s)
'[smoke] > quarto render docs/smoke-all/book/htmlbook/chap-1-1.qmd --to asciidoc' has been running for over (2m0s)
'[smoke] > quarto render docs/smoke-all/book/htmlbook/chap-1-1.qmd --to asciidoc' has been running for over (4m0s)
'[smoke] > quarto render docs/smoke-all/book/htmlbook/chap-1-1.qmd --to asciidoc' has been running for over (8m0s)

So it is has if at some point running lots of Quarto document as part of Deno.test() if failing us in some situation.

This will be a hard one to understand as I can reproduce locally - at least on individual test. And running all the tests is really time consuming !!

Progress ! I can reproduce locally when running the bucket

readarray -t my_array < <(echo '["./smoke/authors/author-name.test.ts","./smoke/filters/editor-support.test.ts","./smoke/inspect/inspect-cleanup.test.ts","./smoke/inspect/inspect-recursive-include.test.ts","./smoke/jupyter/issue-11990.test.ts","./smoke/render/render-callout.test.ts","./smoke/render/render-format-extension.test.ts","docs/smoke-all/2023/01/12/knitr-options-yaml.qmd","docs/smoke-all/2023/10/10/issue-7187.qmd","docs/smoke-all/2023/10/11/unicode-subref-labels.qmd","docs/smoke-all/2023/10/18/7264.qmd","docs/smoke-all/2023/09/26/6977.qmd","docs/smoke-all/2023/09/14/6833.qmd","docs/smoke-all/2023/05/24/5657-raw.qmd","docs/smoke-all/2023/11/09/3650.qmd","docs/smoke-all/2023/04/27/5316.qmd","docs/smoke-all/2023/04/20/callout-test.qmd","docs/smoke-all/2023/08/22/issue-6584.qmd","docs/smoke-all/2023/12/04/7784/about.qmd","docs/smoke-all/2023/07/17/echo-fenced-annotation.qmd","docs/smoke-all/2023/03/07/callout-footnote.qmd","docs/smoke-all/2023/02/01/4174.qmd","docs/smoke-all/2023/02/05/4200-1.qmd","docs/smoke-all/conditional-content/meta.qmd","docs/smoke-all/2025/01/08/7260.qmd","docs/smoke-all/2025/05/18/issue12766-1.qmd","docs/smoke-all/2025/04/09/issue-11695.qmd","docs/smoke-all/2025/03/31/issue-12338/about.qmd","docs/smoke-all/2024/01/27/8464/about.qmd","docs/smoke-all/2024/01/03/8071.qmd","docs/smoke-all/2024/01/18/8331.qmd","docs/smoke-all/2024/10/28/issue-10567/index.qmd","docs/smoke-all/2024/10/23/issue-10829/posts/welcome/index.qmd","docs/smoke-all/2024/09/13/issue-10340.qmd","docs/smoke-all/2024/05/09/9631.qmd","docs/smoke-all/2024/05/06/9593.qmd","docs/smoke-all/2024/05/31/9816.qmd","docs/smoke-all/2024/06/21/issue-8987/index.qmd","docs/smoke-all/2024/06/12/9972-a.qmd","docs/smoke-all/2024/06/07/9903-3.qmd","docs/smoke-all/2024/04/11/9335.qmd","docs/smoke-all/2024/07/19/10334.qmd","docs/smoke-all/2024/07/02/10004-b.qmd","docs/smoke-all/2024/02/22/8814/summary.qmd","docs/smoke-all/2024/02/12/issue-8712/README.md","docs/smoke-all/code-line-numbers/code-line-numbers-highlight.qmd","docs/smoke-all/brand/logo/sidebar/brand-override-alt/conclusion.qmd","docs/smoke-all/brand/logo/document-brand-precedence.qmd","docs/smoke-all/brand/logo/brand-icon-small-favicon-book/references.qmd","docs/smoke-all/brand/logo/explicit-dark-light-paths.qmd","docs/smoke-all/article-layout/tables/tufte.qmd","docs/smoke-all/jupyter/execution-env-project/execution-env-var.qmd","docs/smoke-all/jupyter/error/display-error-true-global-cell-false.qmd","docs/smoke-all/typst/brand-yaml/color/exper/brand-color.qmd","docs/smoke-all/typst/brand-yaml/typography/nobrand/brand-typography.qmd","docs/smoke-all/typst/block-divs.qmd","docs/smoke-all/dark-mode/ggplot-brandless.qmd","docs/smoke-all/dashboard/3-valuebox-static.qmd","docs/smoke-all/dashboard/2-layout-complex.qmd","docs/smoke-all/dashboard/9-no-title.qmd","docs/smoke-all/listings/image-lazy-loading/blog/post-2.qmd","docs/smoke-all/2022/09/30/author-crossref-conditions/crossref.qmd","docs/smoke-all/2022/12/12/code-annotation.qmd"]' | jq -rc '.[]')
haserror=0
for file in "${my_array[@]}"; do 
  echo ">>> ./run-tests.sh ${file}"
  shopt -s globstar && ./run-tests.sh $file
  status=$?
  [ $status -eq 0 ] && echo ">>> No error in this test file" || haserror=1
done

So this is good base to understand what is happening..

This is something related to the TeX Live environment.
With a clean TinyTeX install, the document renders. But run as part of the bucket, something as installed inside the LaTeX environment, and it makes the document hang. Really weird, but not the first time I encounter this problem.

So the problem is ./smoke/render/render-format-extension.test.ts running before. it installs some packages that creates the hang for tufte.qmd

It happens with academic/document.qmd when rendering with elsevier-pdf format.

Something happens that then make the hanging with other render. It is probably related to lualatex engine... 🤔

This could also be something related to very old elsarticle in Quarto Journals!

But it passes in a clean environment - I added CI there: https://github.com/quarto-journals/elsevier/actions/runs/19108165036/job/54597685613

I am thinking of testing those formats in the repo there instead of in this CI to avoid this mess.

Root cause found!

After extensive binary search testing (narrowing down from 51 files), I've identified the exact cause: the outdated elsarticle.cls bundled in the test extension.

The problematic file:

tests/docs/extensions/format/academic/_extensions/quarto-journals/elsevier/elsarticle.cls

Version mismatch:

Bundled version: v3.3 (2020/11/20)
TinyTeX version: v3.4c (2025/01/11)

What's happening:

When ./smoke/render/render-format-extension.test.ts renders with elsevier-pdf format:

The old bundled v3.3 class is used with lualatex
This triggers TinyTeX package updates (amsmath, latex, tools, etc.)
Updates regenerate lualatex format files expecting modern T1 font encoding
Old class without explicit T1 encoding requirement creates mismatch
TinyTeX environment gets corrupted
Subsequent lualatex renders (like tufte.qmd) hang

Key difference between versions:

The critical fix in v3.4c is:

\RequirePackage[T1]{fontenc}

This was missing in v3.3, causing font encoding issues with modern TinyTeX.

Confirmed with testing:

Fresh TinyTeX each time:

Test with old v3.3: render elsevier-pdf → tufte.qmd hangs ❌
Test with new v3.4c: render elsevier-pdf → tufte.qmd passes (33s) ✅

Solution:

Update the bundled elsarticle.cls from v3.3 to v3.4c in the test extension. The current version can be pulled from a fresh TinyTeX install or CTAN.

Updating elsevier format in

This should help !

Fix confirmed! ✅

After updating the extension locally with:

cd tests/docs/extensions/format/academic
quarto update extension quarto-journals/elsevier --no-prompt

The extension was updated from elsarticle.cls v3.3 (2020/11/20) → v3.4c (2025/01/11).

Verification test:

Fresh TinyTeX install
Run ./smoke/render/render-format-extension.test.ts (with updated v3.4c extension)
Run docs/smoke-all/article-layout/tables/tufte.qmd immediately after

Result:

Format extension test: PASSED
tufte.qmd: PASSED (33s) - no hang!

The extension update successfully resolves the TinyTeX corruption issue. The fix will be available once the upstream PRs (quarto-journals/elsevier#38 and quarto-journals/elsevier#40) are merged and the extension needs to be updated in the quarto-cli test suite.

Update journal extensions before running format extension tests

It seems like we have issue with JSS.

Update: Root cause identified

Used binary search to narrow down the issue:

Culprit test: smoke/extensions/extension-render-journals.test.ts
Specific format: JSS (jss-pdf)

Root cause: JSS render triggers tlmgr update which regenerates lualatex format files. The old jss.cls v3.3 (2021) is incompatible with the regenerated formats, causing subsequent lualatex renders to hang.

This is the same pattern as the previous elsevier issue - likely needs similar fix (adding \RequirePackage[T1]{fontenc} to jss.cls).

Continuing investigation Monday to confirm and implement fix.

After more investigation, it could be related to really recent update of tabular and array environment

https://texlive.mycozy.space/macros/latex/required/tools/array.pdf

But I can reproduce the problem reliably while

Updating TeX Live environment to latest
Rendering our test with article layout feature => This fails !

So I have for now opt-out the tests

test - temporary opt out some article layout test due to latex update issues #13664

This will be while waiting this LaTeX solves itself or that we solve it on our side.

Root Cause Identified

The hang is caused by updating both latex and tools packages to version 76708 together. This is a LaTeX Project upstream bug.

Minimal reproduction:

# Fresh TinyTeX works fine
quarto install tinytex --no-prompt
quarto render docs/smoke-all/article-layout/tables/tufte.qmd  # ✅ Works

# Update both packages
tlmgr update --self
tlmgr update latex tools

# Now hangs indefinitely
quarto render docs/smoke-all/article-layout/tables/tufte.qmd  # ❌ Hangs at "running lualatex - 1"

Updating either package individually causes errors but not hangs. The combination causes an infinite hang.

Confirmed on both Linux and Windows.

Add flaky test debugging methodology to dev-docs

Navigation Menu

Repository navigation

Fix failing tests due to timeout because of LaTeX update breaking tufte doc #13647

Update: Root cause identified

Root Cause Identified

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fix failing tests due to timeout because of LaTeX update breaking tufte doc #13647

Description

Activity

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 5, 2025

cderv commented on Nov 7, 2025

cderv commented on Nov 7, 2025

Update: Root cause identified

cderv commented on Nov 10, 2025

cderv commented on Nov 11, 2025

cderv commented on Nov 11, 2025

Root Cause Identified

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions