Skip to content

Commit 1cda0dc

Browse files
milanmajchrakjr-rkMatusBekeclaude
authored
CLARIN-DSpace v9/Port #1409 + #1407 + #1408 (filter-media WARN, SWORDv2 double-delete guard, LDAP hardening) to the v9 base (#1430)
* Port #1409 to dtq-dev-9-base: fix(mediafilter): log unparsable-PDF filter-media errors as WARN, not ERROR (#1409) Source: b0c4850 (dtq-dev PR #1409) A corrupt or malformed PDF makes PDFBoxThumbnail and TikaTextExtractionFilter throw a parse IOException that MediaFilterServiceImpl re-logs at ERROR, flooding the nightly filter-media job and tripping log-based alerting even though the job already skips the file and continues. Both filters now catch the IOException, log at WARN and return null, so the bitstream is skipped cleanly. The encrypted -PDF branch (InvalidPasswordException) stays at ERROR. Applied verbatim -- both files are byte-identical with vanilla 9.3 on this branch, and the v9 `Loader.loadPDF(new RandomAccessReadBuffer(source))` rewrite did not disturb the catch chain the new block attaches to. No test on either branch covers this path (the source PR has none either). Co-authored-by: MatusBeke <matus.beke7@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Port #1407 to dtq-dev-9-base: [Port to dtq-dev] fix: SWORDv2 item double-delete with WorkflowManagerDefault (#1407) Source: f241c47 (dtq-dev PR #1407) ContainerManagerDSpace.doContainerDelete() now checks, before the final itemService.delete(), whether a DELETE event for the same item UUID is already queued on the context -- i.e. the item was deleted earlier in this transaction -- and skips the second delete. Historically that second delete threw when the item came in through WorkflowManagerDefault. Both delete paths keep deleteWrapper(); the guard is the explicit safety net the source PR settled on, in its NPE-hardened form (itemUUID.equals(event.getSubjectID()), constrained to Event.DELETE on Constants.ITEM subjects). As the source PR states, the current base does not double-delete on any path, so this is behaviour-neutral today. Applied verbatim: the file is byte-identical with vanilla 9.3 on this branch and Context.getEvents() / Event.getEventType / getSubjectType / getSubjectID are unchanged in v9. The new java.util.UUID import lands after java.util.TreeMap and org.dspace.event.Event after org.dspace.core.LogHelper, so checkstyle import order (com < jakarta < org) is preserved. Known gap carried over from the source PR: the negative branch of the guard (delete skipped because a DELETE event is already queued) has no test on any branch. The two existing Swordv2IT delete tests only cover the positive path. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Port #1408 to dtq-dev-9-base: [Port to dtq-dev] fix: harden LDAP auth (groupmap guard, DN logging) (#1408) Source: 6615b01 (dtq-dev PR #1408) Two of the source commit's three hunks are ported; the third is already here. Ported (both in assignGroups()): - A groupmap entry without a ':' separator, or with a blank left or right part, no longer reaches `t[1]` (ArrayIndexOutOfBoundsException) and no longer leaves an empty ldapSearchString -- which made containsIgnoreCase(dn, "" + ",") match essentially every DN and assign the mapped group to all LDAP users. The entry is now logged at ERROR with its index and skipped, and scanning continues at the next index. Parsing uses split(":", 2) so a colon inside a DSpace group name is preserved instead of truncated. - System.out.println("dn:" + dn) becomes log.debug(LogHelper.getHeader(context, "assignGroups", "dn=" + dn)) -- a semi-sensitive DN off the default log level and out of stdout. NOT ported -- VANILLA-COVERED: the e-mail fallback hunk (the 5-arg setEpersonAttributes overload plus its two call sites). That fix went upstream as 23b999e (PRs DSpace#11293/DSpace#11331, authored by DataQuest) and is in dspace-9.3: git merge-base --is-ancestor 23b999e dspace-9.3 -> 0 git diff dspace-9.3 origin/dtq-dev-9-base -- .../LDAPAuthentication.java -> empty On this branch setEpersonAttributes already has both overloads, both call sites already pass `email`, and `StringUtils.isNotEmpty(email)` occurs exactly twice -- the same count as on dtq-dev. Re-applying the hunk would have duplicated it. Added beyond the source commit: LDAPAuthenticationTest, 5 unit tests over the groupmap parsing (no LDAP server; ConfigurationService and GroupService mocked, assignGroups reached by reflection because it is private). The source PR lists the missing test under its own "Open points"; card BE-06's AC-3 needs exactly this behaviour proven, and with no LDAP server anywhere in the estate a unit test is the only way to prove it. Worth backporting to dtq-dev. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: jurinecko <juraj.roka@dataquest.sk> Co-authored-by: MatusBeke <matus.beke7@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2c1c1dd commit 1cda0dc

5 files changed

Lines changed: 162 additions & 8 deletions

File tree

dspace-api/src/main/java/org/dspace/app/mediafilter/PDFBoxThumbnail.java

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
package org.dspace.app.mediafilter;
99

1010
import java.awt.image.BufferedImage;
11+
import java.io.IOException;
1112
import java.io.InputStream;
1213

1314
import org.apache.logging.log4j.Logger;
@@ -79,6 +80,13 @@ public InputStream getDestinationStream(Item currentItem, InputStream source, bo
7980
} catch (InvalidPasswordException ex) {
8081
log.error("PDF is encrypted. Cannot create thumbnail (item: {})", currentItem::getHandle);
8182
return null;
83+
} catch (IOException ex) {
84+
// A malformed/non-standard PDF (bad %PDF- header, missing xref, truncated file, etc.)
85+
// is a data-quality issue in the source bitstream, not a DSpace fault. Skip the
86+
// thumbnail instead of failing the whole filter-media run.
87+
log.warn("PDF could not be parsed by PDFBox. Cannot create thumbnail (item: {}): {}",
88+
currentItem::getHandle, ex::getMessage);
89+
return null;
8290
}
8391

8492
// Generate thumbnail derivative and return as IO stream.

dspace-api/src/main/java/org/dspace/app/mediafilter/TikaTextExtractionFilter.java

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,10 +85,13 @@ public InputStream getDestinationStream(Item currentItem, InputStream source, bo
8585
tika.setMaxStringLength(maxChars); // Tell Tika the maximum number of characters to extract
8686
extractedText = tika.parseToString(source);
8787
} catch (IOException e) {
88-
System.err.format("Unable to extract text from bitstream in Item %s%n", currentItem.getID().toString());
89-
e.printStackTrace(System.err);
90-
log.error("Unable to extract text from bitstream in Item {}", currentItem.getID().toString(), e);
91-
throw e;
88+
// A malformed/non-standard source file (e.g. a PDF with a corrupt header, missing
89+
// xref, or truncated content) is a data-quality issue in the bitstream, not a
90+
// DSpace fault. Skip text extraction for it instead of failing the whole
91+
// filter-media run.
92+
log.warn("Unable to extract text from bitstream in Item {}: {}",
93+
currentItem.getHandle(), e.getMessage());
94+
return null;
9295
} catch (OutOfMemoryError oe) {
9396
System.err.format("OutOfMemoryError occurred when extracting text from bitstream in Item %s. " +
9497
"You may wish to enable 'textextractor.use-temp-file'.%n", currentItem.getID().toString());

dspace-api/src/main/java/org/dspace/authenticate/LDAPAuthentication.java

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -610,7 +610,7 @@ public String getName() {
610610
*/
611611
private void assignGroups(String dn, ArrayList<String> group, Context context) {
612612
if (StringUtils.isNotBlank(dn)) {
613-
System.out.println("dn:" + dn);
613+
log.debug(LogHelper.getHeader(context, "assignGroups", "dn=" + dn));
614614
int groupmapIndex = 1;
615615
String groupMap = configurationService.getProperty("authentication-ldap.login.groupmap." + groupmapIndex);
616616
boolean cmp;
@@ -619,7 +619,16 @@ private void assignGroups(String dn, ArrayList<String> group, Context context) {
619619
// groupmap contains the mapping of LDAP groups to DSpace groups
620620
// outer loop with the DSpace groups
621621
while (groupMap != null) {
622-
String t[] = groupMap.split(":");
622+
String t[] = groupMap.split(":", 2);
623+
if (t.length < 2 || StringUtils.isBlank(t[0]) || StringUtils.isBlank(t[1])) {
624+
log.error(LogHelper.getHeader(context, "assignGroups",
625+
"malformed groupmap entry at index " + groupmapIndex + ": " + groupMap +
626+
" - expected '<ldapSearchFragment>:<dspaceGroupName>' with both parts non-empty"));
627+
groupMap = configurationService.getProperty(
628+
"authentication-ldap.login.groupmap." + ++groupmapIndex);
629+
continue;
630+
}
631+
623632
String ldapSearchString = t[0];
624633
String dspaceGroupName = t[1];
625634

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
/**
2+
* The contents of this file are subject to the license and copyright
3+
* detailed in the LICENSE and NOTICE files at the root of the source
4+
* tree and available online at
5+
*
6+
* http://www.dspace.org/license/
7+
*/
8+
package org.dspace.authenticate;
9+
10+
import static org.mockito.ArgumentMatchers.any;
11+
import static org.mockito.Mockito.mock;
12+
import static org.mockito.Mockito.never;
13+
import static org.mockito.Mockito.verify;
14+
import static org.mockito.Mockito.when;
15+
16+
import java.lang.reflect.Method;
17+
import java.util.ArrayList;
18+
19+
import org.dspace.AbstractUnitTest;
20+
import org.dspace.core.Context;
21+
import org.dspace.eperson.Group;
22+
import org.dspace.eperson.service.GroupService;
23+
import org.dspace.services.ConfigurationService;
24+
import org.junit.Before;
25+
import org.junit.Test;
26+
27+
/**
28+
* Unit tests for the {@code authentication-ldap.login.groupmap.N} parsing in
29+
* {@link LDAPAuthentication#assignGroups(String, ArrayList, Context)}. No LDAP server is
30+
* involved: the configuration and the group service are mocked and only the mapping logic runs.
31+
*/
32+
public class LDAPAuthenticationTest extends AbstractUnitTest {
33+
34+
private static final String GROUPMAP = "authentication-ldap.login.groupmap.";
35+
private static final String DN = "uid=jdoe,ou=staff,dc=example,dc=org";
36+
37+
private LDAPAuthentication ldapAuthentication;
38+
private ConfigurationService configurationService;
39+
private GroupService groupService;
40+
41+
@Before
42+
public void setUp() {
43+
ldapAuthentication = new LDAPAuthentication();
44+
configurationService = mock(ConfigurationService.class);
45+
groupService = mock(GroupService.class);
46+
ldapAuthentication.configurationService = configurationService;
47+
ldapAuthentication.groupService = groupService;
48+
}
49+
50+
/**
51+
* assignGroups is private, and the branch under test is the one taken when the caller has no
52+
* list of LDAP groups, so the second argument has to be a real null.
53+
*/
54+
private void assignGroups() throws Exception {
55+
Method method = LDAPAuthentication.class
56+
.getDeclaredMethod("assignGroups", String.class, ArrayList.class, Context.class);
57+
method.setAccessible(true);
58+
method.invoke(ldapAuthentication, DN, null, context);
59+
}
60+
61+
@Test
62+
public void assignGroupsSkipsEntryWithoutSeparator() throws Exception {
63+
when(configurationService.getProperty(GROUPMAP + 1)).thenReturn("ou=staff");
64+
65+
assignGroups();
66+
67+
verify(groupService, never()).findByName(any(), any());
68+
}
69+
70+
@Test
71+
public void assignGroupsSkipsEntryWithBlankSearchPart() throws Exception {
72+
when(configurationService.getProperty(GROUPMAP + 1)).thenReturn(":Admins");
73+
74+
assignGroups();
75+
76+
verify(groupService, never()).findByName(any(), any());
77+
}
78+
79+
@Test
80+
public void assignGroupsSkipsEntryWithBlankGroupPart() throws Exception {
81+
when(configurationService.getProperty(GROUPMAP + 1)).thenReturn("ou=staff:");
82+
83+
assignGroups();
84+
85+
verify(groupService, never()).findByName(any(), any());
86+
}
87+
88+
@Test
89+
public void assignGroupsKeepsScanningAfterAMalformedEntry() throws Exception {
90+
when(configurationService.getProperty(GROUPMAP + 1)).thenReturn(":Admins");
91+
when(configurationService.getProperty(GROUPMAP + 2)).thenReturn("ou=staff:Staff");
92+
when(groupService.findByName(context, "Staff")).thenReturn(mock(Group.class));
93+
94+
assignGroups();
95+
96+
verify(groupService).findByName(context, "Staff");
97+
}
98+
99+
@Test
100+
public void assignGroupsKeepsAColonInsideTheDSpaceGroupName() throws Exception {
101+
when(configurationService.getProperty(GROUPMAP + 1)).thenReturn("ou=staff:Staff:Local");
102+
when(groupService.findByName(context, "Staff:Local")).thenReturn(mock(Group.class));
103+
104+
assignGroups();
105+
106+
verify(groupService).findByName(context, "Staff:Local");
107+
}
108+
}

dspace-swordv2/src/main/java/org/dspace/sword2/ContainerManagerDSpace.java

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
import java.util.List;
1414
import java.util.Map;
1515
import java.util.TreeMap;
16+
import java.util.UUID;
1617

1718
import org.apache.logging.log4j.Logger;
1819
import org.dspace.authorize.AuthorizeException;
@@ -25,6 +26,7 @@
2526
import org.dspace.core.Constants;
2627
import org.dspace.core.Context;
2728
import org.dspace.core.LogHelper;
29+
import org.dspace.event.Event;
2830
import org.dspace.workflow.WorkflowItem;
2931
import org.dspace.workflow.WorkflowItemService;
3032
import org.dspace.workflow.factory.WorkflowServiceFactory;
@@ -755,14 +757,18 @@ protected void doContainerDelete(SwordContext swordContext, Item item,
755757
WorkflowTools wft = new WorkflowTools();
756758
if (wft.isItemInWorkspace(swordContext.getContext(), item)) {
757759
WorkspaceItem wsi = wft.getWorkspaceItem(context, item);
760+
// remove only the workspace wrapper row; the item itself is deleted below.
758761
workspaceItemService.deleteWrapper(context, wsi);
759762
} else if (wft.isItemInWorkflow(context, item)) {
760763
WorkflowItem wfi = wft.getWorkflowItem(context, item);
761764
workflowItemService.deleteWrapper(context, wfi);
762765
}
763766

764-
// then delete the item
765-
itemService.delete(context, item);
767+
// then delete the item, unless an upstream method already queued its deletion
768+
// in this transaction (safety net against a double itemService.delete()).
769+
if (!isItemAlreadyDeleted(context, item.getID())) {
770+
itemService.delete(context, item);
771+
}
766772
} catch (SQLException | IOException e) {
767773
throw new DSpaceSwordException(e);
768774
} catch (AuthorizeException e) {
@@ -788,4 +794,24 @@ private Item getDSpaceTarget(Context context, String editUrl,
788794

789795
return item;
790796
}
797+
798+
/**
799+
* Returns true if a DELETE event for this item is already queued on the context
800+
* (i.e. the item was deleted earlier in this transaction), so the caller can skip
801+
* a second {@code itemService.delete()} that would otherwise fail.
802+
*/
803+
private boolean isItemAlreadyDeleted(Context context, UUID itemUUID) {
804+
if (context.getEvents() == null) {
805+
return false;
806+
}
807+
808+
for (Event event : context.getEvents()) {
809+
if (event.getEventType() == Event.DELETE
810+
&& event.getSubjectType() == Constants.ITEM
811+
&& itemUUID.equals(event.getSubjectID())) {
812+
return true;
813+
}
814+
}
815+
return false;
816+
}
791817
}

0 commit comments

Comments
 (0)