UFAL/Release hotfix 2026 02 26 - Merge PR (#1260)

* UFAL/Fixed failing integration test (ufal#1332) (#1249)

* Add debug messages to fauling test

(cherry picked from commit 4cc3694)

Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>

* [Port to dtq-dev] Fix OpenAIRE integration: null handling and HTTP client lifecycle (#1248)

* Fix OpenAIRE integration: null handling and HTTP client lifecycle (ufal#1330)

* Add test for OpenAIRE connector

* Add null check for OpenAIRE response to prevent NullPointerException

Co-authored-by: kosarko <1842385+kosarko@users.noreply.github.com>

* Fix HTTP client lifecycle to prevent premature connection closure

Co-authored-by: kosarko <1842385+kosarko@users.noreply.github.com>

* Keep the try with resources but copy the response

into an in memory stream and return that

* license:check

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kosarko <1842385+kosarko@users.noreply.github.com>
(cherry picked from commit 02984db)

* Handle NumberFormatException in OpenAIREFundingDataProvider.getNumberOfResults and use explicit UTF-8 charset in OpenAIRERestConnectorTest

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kosarko <1842385+kosarko@users.noreply.github.com>
Co-authored-by: milanmajchrak <milan.majchrak@dataquest.sk>

* UFAL/Added a comment to do not forget mounting the file which is changed via ocnfiguration feature (#1247)

* UFAL/Issue 1315: Store file preview to database when file preview is created on Item Page load. (ufal#1316) (#1241)

* Issue ufal/clarin-dspace1315: Store file preview to database when file preview is created on item page load

* PR comments: commit context only when any of the file preview is successfully created

(cherry picked from commit aab626b)

Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>

* UFAL/Issue 1313: fixed error when file preview is not generated for bitstream with store_number = 77 (ufal#1318) (#1240)

* Issue ufal#1313: fixed error when file preview is not generated for bitstream with store number = 77

(cherry picked from commit 04d64f7)

Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>

* UFAL/Nw version metadata issues (#1236)

* Issue ufal#1266: dc.date.available and dc.relation.replaces metadata not cleared properly (ufal#1307)

* Issue ufal#1266: dc.date.available and dc.relation.replaces metadata not cleaned properly in new item version

* resolve MR comments - update ignoredMetadataFields in versioning-service.xml

* update ClarinVersionedHandleIdentifierProviderIT test to check dc.identifier.uri metadata for new version

(cherry picked from commit 7ffaf9a)

* Issue 1319: do not copy dc.identifier.doi metadata when new item version is created

(cherry picked from commit 1b7ed17)

---------

Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>

* UFAL/Fix: add bitstream download-by-handle endpoint for curl instructions (#1252)

* fix: add bitstream download-by-handle endpoint for curl instructions

Adds GET /api/core/bitstreams/handle/{prefix}/{suffix}/{filename} endpoint
that directly serves bitstream content by item handle and filename.

This resolves the issue where curl download instructions generated by the
UI produced URLs pointing to non-existent backend endpoints, resulting in
404 errors when users attempted to download files via command line.

The new endpoint resolves the handle to an Item, finds the bitstream by
exact filename in ORIGINAL bundles, and streams the raw content with
correct Content-Type and Content-Disposition headers.

Refs: dataquest-dev/dspace-angular#1210

* Fixed compliing errors

* Small refactoring - use constants and removed unnecessary changes

* added comments, return 404 status instead of 402

* unauthorized instead of forbidden

* fix: use RFC 5987 Content-Disposition for non-ASCII filenames

curl -J on Windows cannot create files with non-ASCII characters (e.g.
diacritics like e/a) from a raw UTF-8 Content-Disposition filename header.

Uses filename*=UTF-8''percent-encoded-name (RFC 5987/6266) which curl
properly decodes. Also includes an ASCII fallback in filename param.

* fix: move context.complete() after streaming to prevent truncated downloads

context.complete() was called before bitstreamService.retrieve(), closing
the DB connection and causing 'end of response with X bytes missing' errors.
Now context.complete() is called only after the full content has been streamed.
For S3 redirect and HEAD paths, context.complete() remains before return
since no streaming is needed.

* fix: use real UTF-8 filename in Content-Disposition instead of ASCII fallback

The filename parameter now contains the original name (with diacritics like
e/a) instead of replacing non-ASCII chars with underscores. Characters in
the ISO-8859-1 range are transmitted correctly by Tomcat and understood by
curl on Western/Central-European systems. The filename* parameter still
provides RFC 5987 percent-encoded UTF-8 for modern clients (curl 7.56+).

* fix: revert to ASCII fallback in Content-Disposition, add edge-case tests

Content-Disposition filename parameter now uses ASCII fallback (non-ASCII
replaced with underscore) per RFC 6266. Modern clients use filename* (RFC
5987) which has the full UTF-8 name. The curl command no longer relies on
Content-Disposition at all (uses -o instead of -OJ).

New integration tests for edge cases:
- Multiple dots in filename (archive.v2.1.tar.gz)
- Double quotes in filename (escaped in Content-Disposition)
- CJK characters (beyond ISO-8859-1)
- Same filename in ORIGINAL and TEXT bundles (only ORIGINAL served)

* fix: resolve compilation errors and fix IT test assertions

- Remove duplicate HttpStatus import (apache vs spring)
- Add missing MediaType import (spring)
- Fix Content-Type assertion to include charset=UTF-8
- Use URI.create() for pre-encoded URLs in tests to prevent
  double-encoding (%25) rejection by StrictHttpFirewall

All 15 integration tests pass.

* test: add complex filename test (diacritics, plus, hash, unmatched paren)

New IT test for filename 'Media (+)#9) ano' verifying correct URL decoding,
Content-Disposition encoding, and content delivery. 16/16 tests pass.

* fix authorization, comments, tests

* fix: change expected status from 401 to 403 for authenticated non-admin user

The test downloadBitstreamByHandleUnauthorizedForNonAdmin uses getClient(token)
which means the user IS authenticated. The controller correctly returns 403
(Forbidden) for authenticated users without access, not 401 (Unauthorized).
401 is only for anonymous/unauthenticated requests.

---------

Co-authored-by: Paurikova2 <michaela.paurikova@dataquest.sk>

---------

Co-authored-by: Ondřej Košarko <ko_ok@centrum.cz>
Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kosarko <1842385+kosarko@users.noreply.github.com>
Co-authored-by: Paurikova2 <michaela.paurikova@dataquest.sk>

Author: 5 people

checkstyle-suppressions.xml

@@ -8,4 +8,5 @@
          on JMockIt Expectations blocks and similar. See https://github.com/checkstyle/checkstyle/issues/3739 -->
     <suppress checks="Indentation" files="src[/\\]test[/\\]java"/>
     <suppress checks="Regexp" files="DSpaceHttpClientFactory\.java"/>
+    <suppress checks="Regexp" files="OpenAIRERestConnectorTest\.java"/>
 </suppressions>

dspace-api/src/main/java/org/dspace/external/OpenAIRERestConnector.java

@@ -207,8 +207,11 @@ public InputStream get(String file, String accessToken) {
                         break;
                 }
 
-                // do not close this httpClient
-                result = getResponse.getEntity().getContent();
+                // the client will be closed, we need to copy the response stream to a new one that we can return
+                try (InputStream is = getResponse.getEntity().getContent()) {
+                    byte[] bytes = is.readAllBytes();
+                    result = new java.io.ByteArrayInputStream(bytes);
+                }
             }
         } catch (MalformedURLException e1) {
             getGotError(e1, url + '/' + file);

dspace-api/src/main/java/org/dspace/external/provider/impl/OpenAIREFundingDataProvider.java

@@ -169,7 +169,19 @@ public int getNumberOfResults(String query) {
         String encodedQuery = encodeValue(query);
 
         Response projectResponse = connector.searchProjectByKeywords(0, 0, encodedQuery);
-        return Integer.parseInt(projectResponse.getHeader().getTotal());
+        if (projectResponse == null || projectResponse.getHeader() == null) {
+            return 0;
+        }
+        String total = projectResponse.getHeader().getTotal();
+        if (StringUtils.isBlank(total)) {
+            return 0;
+        }
+        try {
+            return Integer.parseInt(total);
+        } catch (NumberFormatException e) {
+            log.error("Failed to parse search result count from OpenAIRE: {}", e.getMessage());
+            return 0;
+        }
     }
 
     /**

dspace-api/src/main/java/org/dspace/storage/bitstore/SyncBitstreamStorageServiceImpl.java

@@ -7,6 +7,7 @@
  */
 package org.dspace.storage.bitstore;
 
+import java.io.File;
 import java.io.IOException;
 import java.io.InputStream;
 import java.sql.SQLException;
@@ -185,12 +186,17 @@ public Map computeChecksumSpecStore(Context context, Bitstream bitstream, int st
     }
 
     @Override
-    public InputStream retrieve(Context context, Bitstream bitstream)
-            throws SQLException, IOException {
+    public InputStream retrieve(Context context, Bitstream bitstream) throws SQLException, IOException {
         int storeNumber = this.whichStoreNumber(bitstream);
         return this.getStore(storeNumber).get(bitstream);
     }
 
+    @Override
+    public File retrieveFile(Context context, Bitstream bitstream) throws IOException {
+        int storeNumber = whichStoreNumber(bitstream);
+        return this.getStore(storeNumber).getFile(bitstream);
+    }
+
     @Override
     public void cleanup(boolean deleteDbRecords, boolean verbose) throws SQLException, IOException, AuthorizeException {
         Context context = new Context(Context.Mode.BATCH_EDIT);

dspace-api/src/main/java/org/dspace/versioning/DefaultItemVersionProvider.java

@@ -202,7 +202,7 @@ protected void copyRelationships(
      */
     private void manageRelationMetadata(Context c, Item itemNew, Item previousItem) throws SQLException {
         // Remove copied `dc.relation.replaces` metadata for the new item.
-        itemService.clearMetadata(c, itemNew, "dc", "relation", "replaces", null);
+        itemService.clearMetadata(c, itemNew, "dc", "relation", "replaces", Item.ANY);
 
         // Add metadata `dc.relation.replaces` to the new item.
         // The metadata value is: `dc.identifier.uri` from the previous item.

dspace-api/src/test/java/org/dspace/builder/WorkspaceItemBuilder.java

@@ -249,6 +249,29 @@ public WorkspaceItemBuilder withFulltext(String name, String source, InputStream
         return this;
     }
 
+    /**
+     * Add bitstream with specific store number.
+     *
+     * @param name          bitstream name
+     * @param source        bitstream test source location
+     * @param is            input stream of the bitstream
+     * @param storeNumber   store number
+     *
+     * @return this WorkspaceItemBuilder
+     */
+    public WorkspaceItemBuilder withBitstream(String name, String source, InputStream is, int storeNumber) {
+        try {
+            Item item = workspaceItem.getItem();
+            Bitstream b = itemService.createSingleBitstream(context, is, item);
+            b.setStoreNumber(storeNumber);
+            b.setName(context, name);
+            b.setSource(context, source);
+        } catch (Exception e) {
+            handleException(e);
+        }
+        return this;
+    }
+
     /**
      * Create workspaceItem with any metadata
      * @param schema metadataSchema name e.g. `dc`

dspace-api/src/test/java/org/dspace/external/OpenAIRERestConnectorTest.java

@@ -0,0 +1,63 @@
+/**
+ * The contents of this file are subject to the license and copyright
+ * detailed in the LICENSE and NOTICE files at the root of the source
+ * tree and available online at
+ *
+ * http://www.dspace.org/license/
+ */
+package org.dspace.external;
+
+import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.spy;
+import static org.mockito.Mockito.when;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+
+import eu.openaire.jaxb.model.Response;
+import okhttp3.mockwebserver.MockResponse;
+import okhttp3.mockwebserver.MockWebServer;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.dspace.app.client.DSpaceHttpClientFactory;
+import org.junit.Test;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+
+public class OpenAIRERestConnectorTest {
+
+    @Test
+    public void searchProjectByKeywords() {
+        try (InputStream is = this.getClass().getResourceAsStream("openaire-projects.xml");
+               MockWebServer mockServer = new MockWebServer()) {
+            String projects = new String(is.readAllBytes(), StandardCharsets.UTF_8)
+                    .replaceAll("( mushroom)", "( DEADBEEF)");
+            mockServer.enqueue(new MockResponse().setResponseCode(200).setBody(projects));
+
+            // setup mocks so we don't have to set whole DSpace kernel etc.
+            // still, the idea is to test how the get method behaves
+            CloseableHttpClient httpClient = spy(HttpClientBuilder.create().build());
+            doReturn(httpClient.execute(new HttpGet(mockServer.url("").toString())))
+                    .when(httpClient).execute(Mockito.any());
+
+            DSpaceHttpClientFactory mock = Mockito.mock(DSpaceHttpClientFactory.class);
+            when(mock.build()).thenReturn(httpClient);
+
+            try (MockedStatic<DSpaceHttpClientFactory> mockedFactory =
+                         Mockito.mockStatic(DSpaceHttpClientFactory.class)) {
+                mockedFactory.when(DSpaceHttpClientFactory::getInstance).thenReturn(mock);
+                OpenAIRERestConnector connector = new OpenAIRERestConnector(mockServer.url("").toString());
+                Response response = connector.searchProjectByKeywords(0, 10, "keyword");
+                // Basically check it doesn't throw UnmarshallerException and that we are getting our mocked response
+                assertTrue("Expected the query to contain the replaced keyword",
+                        response.getHeader().getQuery().contains("DEADBEEF"));
+            }
+        } catch (IOException e) {
+            e.printStackTrace();
+        }
+    }
+}

dspace-api/src/test/java/org/dspace/external/provider/impl/OpenAIREFundingDataProviderTest.java

@@ -14,7 +14,9 @@
 import java.util.List;
 import java.util.Optional;
 
+import eu.openaire.jaxb.model.Response;
 import org.dspace.AbstractDSpaceTest;
+import org.dspace.external.OpenAIRERestConnector;
 import org.dspace.external.factory.ExternalServiceFactory;
 import org.dspace.external.model.ExternalDataObject;
 import org.dspace.external.provider.ExternalDataProvider;
@@ -102,4 +104,21 @@ public void testGetDataObjectWInvalidId() {
 
         assertTrue("openAIREFunding.getExternalDataObject.notExists:WRONGID", result.isEmpty());
     }
+
+    @Test
+    public void testGetNumberOfResultsWhenResponseIsNull() {
+        // Create a mock connector that returns null
+        OpenAIREFundingDataProvider provider = new OpenAIREFundingDataProvider();
+        provider.setSourceIdentifier("test");
+        provider.setConnector(new OpenAIRERestConnector("test") {
+            @Override
+            public Response searchProjectByKeywords(int page, int size, String... keywords) {
+                return null;
+            }
+        });
+
+        // Should return 0 when response is null, not throw NullPointerException
+        int result = provider.getNumberOfResults("test");
+        assertEquals("Should return 0 when response is null", 0, result);
+    }
 }

dspace-api/src/test/java/org/dspace/identifier/ClarinVersionedHandleIdentifierProviderIT.java

@@ -0,0 +1,155 @@
+/**
+ * The contents of this file are subject to the license and copyright
+ * detailed in the LICENSE and NOTICE files at the root of the source
+ * tree and available online at
+ *
+ * http://www.dspace.org/license/
+ */
+package org.dspace.identifier;
+
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.CoreMatchers.startsWith;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Calendar;
+import java.util.List;
+import java.util.TimeZone;
+
+import org.dspace.AbstractIntegrationTestWithDatabase;
+import org.dspace.builder.CollectionBuilder;
+import org.dspace.builder.CommunityBuilder;
+import org.dspace.builder.ItemBuilder;
+import org.dspace.builder.VersionBuilder;
+import org.dspace.content.Collection;
+import org.dspace.content.Item;
+import org.dspace.content.MetadataValue;
+import org.dspace.content.factory.ContentServiceFactory;
+import org.dspace.content.service.InstallItemService;
+import org.dspace.content.service.ItemService;
+import org.dspace.kernel.ServiceManager;
+import org.dspace.services.factory.DSpaceServicesFactory;
+import org.dspace.workflow.WorkflowItem;
+import org.dspace.workflow.WorkflowItemService;
+import org.dspace.workflow.factory.WorkflowServiceFactory;
+import org.junit.Before;
+import org.junit.Test;
+
+/**
+ * Unit Tests for ClarinVersionedHandleIdentifierProvider
+ *
+ * @authorMilan Kuchtiak
+ */
+public class ClarinVersionedHandleIdentifierProviderIT extends AbstractIntegrationTestWithDatabase {
+    private IdentifierServiceImpl identifierService;
+    private InstallItemService installItemService;
+    private ItemService itemService;
+    private WorkflowItemService workflowItemService;
+
+    private Collection collection;
+
+    @Before
+    @Override
+    public void setUp() throws Exception {
+        super.setUp();
+        context.turnOffAuthorisationSystem();
+
+        ServiceManager serviceManager = DSpaceServicesFactory.getInstance().getServiceManager();
+        identifierService = serviceManager.getServicesByType(IdentifierServiceImpl.class).get(0);
+
+        itemService = ContentServiceFactory.getInstance().getItemService();
+        installItemService = ContentServiceFactory.getInstance().getInstallItemService();
+        workflowItemService = WorkflowServiceFactory.getInstance().getWorkflowItemService();
+
+        // Clean out providers to avoid any being used for creation of community and collection
+        identifierService.setProviders(new ArrayList<>());
+
+        parentCommunity = CommunityBuilder.createCommunity(context)
+                .withName("Parent Community")
+                .build();
+        collection = CollectionBuilder.createCollection(context, parentCommunity)
+                .withName("Collection")
+                .build();
+    }
+
+    @Test
+    public void testNewVersionMetadata() throws Exception {
+        registerProvider(ClarinVersionedHandleIdentifierProvider.class);
+        Item itemV1 = ItemBuilder.createItem(context, collection)
+                .withTitle("First version")
+                .build();
+
+        // new item "dc.relation.replaces" metadata has to be set to this value
+        String itemV1HandleRef = itemService.getMetadataFirstValue(itemV1, "dc", "identifier", "uri", Item.ANY);
+
+        // set "dc.relation.replaces" metadata on itemV1
+        itemService.addMetadata(context, itemV1, "dc", "relation", "replaces", null, "some_value");
+        // replace "dc.date.available" metadata on itemV1 to some old value
+        itemService.clearMetadata(context, itemV1, "dc", "date", "available", Item.ANY);
+        itemService.addMetadata(context, itemV1, "dc", "date", "available", null, "2020-01-01");
+        // simulate itemV1 having a DOI identifier assigned
+        itemService.addMetadata(context, itemV1, "dc", "identifier", "doi", null,
+                "https://handle.stage.datacite.org/10.5072/dspace-1");
+
+        Item itemV2 = VersionBuilder.createVersion(context, itemV1, "Second version").build().getItem();
+
+        // check that "dc.date.available", metadata is not copied to itemV2
+        assertThat(itemService.getMetadata(itemV2, "dc", "date", "available", Item.ANY).size(), equalTo(0));
+
+        // check that "dc.identifier.uri", metadata is not copied to itemV2
+        assertThat(itemService.getMetadata(itemV2, "dc", "identifier", "uri", Item.ANY).size(), equalTo(0));
+
+        // check that "dc.identifier.doi", metadata is not copied to itemV2
+        assertThat(itemService.getMetadata(itemV2, "dc", "identifier", "doi", Item.ANY).size(), equalTo(0));
+
+        // check that "dc.relation.replaces" points to itemV1
+        List<MetadataValue> metadataValues = itemService.getMetadata(itemV2, "dc", "relation", "replaces", Item.ANY);
+        assertThat(metadataValues.size(), equalTo(1));
+        assertThat(metadataValues.get(0).getValue(), equalTo(itemV1HandleRef));
+
+        WorkflowItem workflowItem = workflowItemService.create(context, itemV2, collection);
+        Item installedItem = installItemService.installItem(context, workflowItem);
+
+        // get current date
+        Calendar calendar = Calendar.getInstance();
+        calendar.setTimeInMillis(System.currentTimeMillis());
+        calendar.setTimeZone(TimeZone.getTimeZone("UTC"));
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
+        String date = sdf.format(calendar.getTime());
+
+        // check that "dc.relation.replaces" points to itemV1
+        metadataValues = itemService.getMetadata(installedItem, "dc", "relation", "replaces", Item.ANY);
+        assertThat(metadataValues.size(), equalTo(1));
+        assertThat(metadataValues.get(0).getValue(), equalTo(itemV1HandleRef));
+
+        // Check that itemV2 has the correct "dc.date.available" metadata set to current date
+        metadataValues = itemService.getMetadata(installedItem, "dc", "date", "available", Item.ANY);
+        assertThat(metadataValues.size(), equalTo(1));
+        assertThat(metadataValues.get(0).getValue(), startsWith(date));
+
+        // check "dc.identifier.uri" metadata has new value different from itemV1
+        metadataValues = itemService.getMetadata(installedItem, "dc", "identifier", "uri", Item.ANY);
+        assertThat(metadataValues.size(), equalTo(1));
+        assertThat(metadataValues.get(0).getValue(), not(itemV1HandleRef));
+    }
+
+    private void registerProvider(Class type) {
+        // Register our new provider
+        IdentifierProvider identifierProvider =
+                (IdentifierProvider) DSpaceServicesFactory.getInstance().getServiceManager()
+                        .getServiceByName(type.getName(), type);
+        if (identifierProvider == null) {
+            DSpaceServicesFactory.getInstance().getServiceManager().registerServiceClass(type.getName(), type);
+            identifierProvider = (IdentifierProvider) DSpaceServicesFactory.getInstance().getServiceManager()
+                    .getServiceByName(type.getName(), type);
+        }
+
+        // Overwrite the identifier-service's providers with the new one to ensure only this provider is used
+        identifierService = DSpaceServicesFactory.getInstance().getServiceManager()
+                .getServicesByType(IdentifierServiceImpl.class).get(0);
+        identifierService.setProviders(new ArrayList<>());
+        identifierService.setProviders(List.of(identifierProvider));
+    }
+}