Skip to content

Commit b9df9be

Browse files
jr-rkclaude
andcommitted
fix(mediafilter): log unparsable-PDF filter-media errors as WARN, not ERROR
A corrupt/malformed source PDF made PDFBoxThumbnail and TikaTextExtractionFilter let a parse IOException escape, which MediaFilterServiceImpl re-logged at ERROR -- flooding the nightly filter-media job (~4,655 ERROR lines/night) and tripping alerting, even though the job already skips the file and continues. Reimplemented for dtq-dev (PDFBox 3 / Loader.loadPDF; the TUL PDFBox-2 diff does not apply as-is): - PDFBoxThumbnail: catch the parse IOException, log WARN, return null (skip). - TikaTextExtractionFilter: read the bitstream into memory first so a genuine assetstore read failure still propagates, then catch IOException/TikaException from the in-memory parse only -> WARN + return null. This narrows the catch so real I/O errors are not swallowed (the TUL Tika variant caught all IOException). Port of dataquest-dev/dspace-customers#903 (item 4). Source: customer/TUL 7035a4c (reimplemented). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 6114281 commit b9df9be

2 files changed

Lines changed: 20 additions & 6 deletions

File tree

dspace-api/src/main/java/org/dspace/app/mediafilter/PDFBoxThumbnail.java

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
package org.dspace.app.mediafilter;
99

1010
import java.awt.image.BufferedImage;
11+
import java.io.IOException;
1112
import java.io.InputStream;
1213

1314
import org.apache.logging.log4j.Logger;
@@ -79,6 +80,13 @@ public InputStream getDestinationStream(Item currentItem, InputStream source, bo
7980
} catch (InvalidPasswordException ex) {
8081
log.error("PDF is encrypted. Cannot create thumbnail (item: {})", currentItem::getHandle);
8182
return null;
83+
} catch (IOException ex) {
84+
// A malformed/non-standard PDF (bad %PDF- header, missing xref, truncated file, etc.) is a
85+
// data-quality problem in the source bitstream, not a DSpace fault. Skip the thumbnail
86+
// instead of failing the whole filter-media run. See dataquest-dev/dspace-customers#752.
87+
log.warn("PDF could not be parsed by PDFBox. Cannot create thumbnail (item: {}): {}",
88+
currentItem::getHandle, ex::getMessage);
89+
return null;
8290
}
8391

8492
// Generate thumbnail derivative and return as IO stream.

dspace-api/src/main/java/org/dspace/app/mediafilter/TikaTextExtractionFilter.java

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -79,16 +79,22 @@ public InputStream getDestinationStream(Item currentItem, InputStream source, bo
7979
// Get maximum size of structure that Tika will try to buffer.
8080
int maxArray = configurationService.getIntProperty("textextractor.max-array", DEFAULT_MAX_ARRAY);
8181
IOUtils.setByteArrayMaxOverride(maxArray);
82+
// Read the bitstream fully first so a genuine assetstore read failure surfaces here and is
83+
// not mistaken for a parse error in the catch below.
84+
byte[] sourceBytes = source.readAllBytes();
8285
try {
8386
// Use Tika to extract text from input. Tika will automatically detect the file type.
8487
Tika tika = new Tika();
8588
tika.setMaxStringLength(maxChars); // Tell Tika the maximum number of characters to extract
86-
extractedText = tika.parseToString(source);
87-
} catch (IOException e) {
88-
System.err.format("Unable to extract text from bitstream in Item %s%n", currentItem.getID().toString());
89-
e.printStackTrace(System.err);
90-
log.error("Unable to extract text from bitstream in Item {}", currentItem.getID().toString(), e);
91-
throw e;
89+
extractedText = tika.parseToString(new ByteArrayInputStream(sourceBytes));
90+
} catch (IOException | TikaException e) {
91+
// The bytes are already in memory, so this can only be a malformed/non-standard source
92+
// file (corrupt PDF header, missing xref, truncated content) -- a data-quality problem in
93+
// the bitstream, not a DSpace fault. Skip extraction instead of failing the whole
94+
// filter-media run. See dataquest-dev/dspace-customers#752.
95+
log.warn("Unable to extract text from bitstream in Item {}: {}",
96+
currentItem.getHandle(), e.getMessage());
97+
return null;
9298
} catch (OutOfMemoryError oe) {
9399
System.err.format("OutOfMemoryError occurred when extracting text from bitstream in Item %s. " +
94100
"You may wish to enable 'textextractor.use-temp-file'.%n", currentItem.getID().toString());

0 commit comments

Comments
 (0)