Skip to content

Commit 5ca7562

Browse files
milanmajchrakPaurikova2kosarkoKasinhouMatus Kasak
authored
New Release merge - 2025/July/24 (#1024)
* UFAL/DOI - Added type of resource to data cite (#975) * UFAL/The process output is not displayed because of S3 direct download (#971) * The S3 direct download is provided only for the files located in the ORIGINAL bundle * Use constant for the ORIGINAL string value * Check if type is html (#983) * check if type is html * added test for html mime type * used static string for text/html, added check * Ufal dtq sync062025 (#985) * we should identify as clarin-dspace Fix test (cherry picked from commit 6cdf2d1) * update email templates to use dspace.shortname dspace.name can be a long string not fit for Email subjects nor signatures (cherry picked from commit 98d60dd) * match v5 submission (cherry picked from commit 4a2b65f) * get rid of lr.help.phone Phone is now conditional in the templates. Use `mail.message.helpdesk.telephone` if you want it. The change in the *.java files is to preserve the params counts. The relevant templates are getting the phone directly from config (cherry picked from commit cba5695) * Add option to configure oai sample identifier some validators use this value, should be a real id in prod deployments (cherry picked from commit 912f13f) * NRP deposit license (cherry picked from commit ba23878) * Fix ufal#1219 Get rid of setting the jsse.enableSNIExtension property which causes issues with handle minting (cherry picked from commit 7d03173) * UFAL/Improve file preview generating (#972) * get name and size from metadata and header of file, avoid input stream using * remove temp file, checkstyle, do not load full file * add { } after if * added check for max preview file * used ZipFile and TarArchived for filepreview generating * added removed lines * used 7z for zip and tar files * removed 7z and used zip and tar entry * improved file previrew generating speed, used string builder, xml builder, authorization only if is required * checkstyle, return boolean from haspreview and previrews from getPreview, replaced return with continue * fix problem with hibernate session * fix .tar.gz generating * skip fully entry for tar * added indexes for speed up queries * added license header * named constant by upper case * inicialized fileInfo, refactorization of code based on copilot review --------- Co-authored-by: milanmajchrak <90026355+milanmajchrak@users.noreply.github.com> * Fix the file preview integration test (#989) * The hasPreview method has been changed, but the IT wasn't updated correctly * Use the correct checkbox for the input field - use repeatable (#991) * UFAL/EU Sponsor openaire id should not be required (#1001) * EU Sponsor openaire id should not be required * Not required also in the czech submission forms * Logging error message while emailing users (#1000) * Logging error message --------- Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk> Co-authored-by: milanmajchrak <milan.majchrak@dataquest.sk> * UFAL/Teaching and clariah submissions does not have clarin-license (#1005) * UFAL/Fix logging in LogoImportController (#1003) * fix logging * used formatter for msg * UFAL/Update the resource policy rights when changing submitter (#1002) * removed res policies for submitter and created newones when item is shared * avoid magic number, use constant * set submitter in existing res policies * removed not used shared link * UFAL/Added date to title when creating new version (#984) * added date to versioned item title * used more modern approach for getting current time * renamed test * used var for reusing * UFAL/Item handle info in email after download request (#1006) * Added item handle to email * Exception when item not found * Checked grammar * Handled multiple items found by bitstream * Using PID instead of handle --------- Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk> * UFAL/Incorrect password hash funct used during migration (#999) * password in request is already hashed, used different password hash funct * renamed password param in eperson endpoint * [devOps] labelling reviewing process * [devOps] labelling reviewing process * UFAL/New version keeps the old identifier * UFAL/Send email to editor after submitting item (#1016) Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk> * UFAL/Local file size is 0 for file with no zero size (#1017) * update item metadata after the bitstream size has changed * issue 1241: ItemFilesMetadataRepair script implementation (#1243) (#1021) * issue 1241: ItemFilesMetadataRepair script implementation * extend script to be applicabble for all items, and for items with files metadata that have missing bitstreams (files) * implement dry-run option * option description fix * Improve error message * Use "0" instead of "" + 0 * Improve error message (cherry picked from commit 706f6f6) Co-authored-by: kuchtiak-ufal <kuchtiak@ufal.mff.cuni.cz> * UFAL/Refbox upgrade (#1015) * Created integration test * Created an endpoint for complete ref box information like in the v5 * Added integration tests for formatting authors * Removed double semicolon * Fetch the metadata value following the current locale * Updated firstMetadataValue because it did return empty string instead of null * Use DEFAULT_LANGUAGE instead of current locale * UFAL/Added doc - issue link (#1023) --------- Co-authored-by: Paurikova2 <107862249+Paurikova2@users.noreply.github.com> Co-authored-by: Ondřej Košarko <ko_ok@centrum.cz> Co-authored-by: Kasinhou <129340513+Kasinhou@users.noreply.github.com> Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk> Co-authored-by: jurinecko <95219754+jr-rk@users.noreply.github.com> Co-authored-by: jm <jm@maz> Co-authored-by: kuchtiak-ufal <kuchtiak@ufal.mff.cuni.cz>
1 parent da6b0ad commit 5ca7562

14 files changed

Lines changed: 1019 additions & 14 deletions

File tree

‎.github/workflows/PM-label-review-process.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Reviewer Label Management
2+
3+
on:
4+
pull_request:
5+
types: [review_requested]
6+
pull_request_review:
7+
types: [submitted]
8+
9+
permissions:
10+
pull-requests: write
11+
12+
jobs:
13+
manage-reviewer-labels:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Apply reviewer labels
17+
uses: mazoea/ga-maz/label-review@master
18+
with:
19+
target-reviewer: 'vidiecan'
20+
assigned-label: 'REVIEW-in-progress'
21+
completed-label: 'REVIEW-done'
22+
github-token: ${{ secrets.GITHUB_TOKEN }}

‎dspace-api/src/main/java/org/dspace/app/itemupdate/ItemFilesMetadataRepair.java‎

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
/**
2+
* The contents of this file are subject to the license and copyright
3+
* detailed in the LICENSE and NOTICE files at the root of the source
4+
* tree and available online at
5+
*
6+
* http://www.dspace.org/license/
7+
*/
8+
package org.dspace.app.itemupdate;
9+
10+
import java.util.Iterator;
11+
import java.util.List;
12+
import java.util.UUID;
13+
14+
import org.apache.commons.cli.CommandLine;
15+
import org.apache.commons.cli.CommandLineParser;
16+
import org.apache.commons.cli.DefaultParser;
17+
import org.apache.commons.cli.HelpFormatter;
18+
import org.apache.commons.cli.Options;
19+
import org.apache.commons.cli.ParseException;
20+
import org.apache.commons.collections4.CollectionUtils;
21+
import org.apache.logging.log4j.LogManager;
22+
import org.apache.logging.log4j.Logger;
23+
import org.dspace.content.Bundle;
24+
import org.dspace.content.Collection;
25+
import org.dspace.content.Item;
26+
import org.dspace.content.MetadataValue;
27+
import org.dspace.content.factory.ClarinServiceFactory;
28+
import org.dspace.content.factory.ContentServiceFactory;
29+
import org.dspace.content.service.CollectionService;
30+
import org.dspace.content.service.ItemService;
31+
import org.dspace.content.service.clarin.ClarinItemService;
32+
import org.dspace.core.Constants;
33+
import org.dspace.core.Context;
34+
import org.dspace.eperson.EPerson;
35+
import org.dspace.eperson.factory.EPersonServiceFactory;
36+
37+
/**
38+
* Documentation for this class: https://github.com/ufal/clarin-dspace/pull/1243#issue-3236707035
39+
*/
40+
public class ItemFilesMetadataRepair {
41+
42+
private static final Logger log = LogManager.getLogger(ItemFilesMetadataRepair.class);
43+
44+
private ItemFilesMetadataRepair() {
45+
}
46+
47+
public static void main(String[] args) throws Exception {
48+
log.info("Fixing item files metadata started.");
49+
50+
Options options = new Options();
51+
options.addRequiredOption("e", "email", true, "admin email");
52+
options.addOption("c", "collection", true, "collection UUID");
53+
options.addOption("i", "item", true, "item UUID");
54+
options.addOption("d", "dry-run", false, "dry run - with no repair");
55+
options.addOption("h", "help", false, "help");
56+
options.addOption("v", "verbose", false, "verbose output");
57+
58+
CommandLineParser parser = new DefaultParser();
59+
try {
60+
CommandLine line = parser.parse(options, args);
61+
if (line.hasOption('h') || !line.hasOption('e')) {
62+
printHelpAndExit(options);
63+
}
64+
String adminEmail = line.getOptionValue('e');
65+
String collectionUuid = line.getOptionValue('c');
66+
String itemUuid = line.getOptionValue('i');
67+
boolean verboseOutput = line.hasOption('v');
68+
boolean dryRun = line.hasOption('d');
69+
run(adminEmail, collectionUuid, itemUuid, dryRun, verboseOutput);
70+
} catch (ParseException e) {
71+
System.err.println("Failed to parse command line options: " + e.getMessage());
72+
printHelpAndExit(options);
73+
}
74+
75+
log.info("Fixing item files metadata finished.");
76+
}
77+
78+
private static void run(String adminEmail,
79+
String collectionUuid,
80+
String itemUuid,
81+
boolean dryRun,
82+
boolean verboseOutput) throws Exception {
83+
84+
System.out.println("ItemFilesMetadataRepair Started.\n");
85+
86+
try (Context context = new Context(Context.Mode.READ_WRITE)) {
87+
ItemService itemService = ContentServiceFactory.getInstance().getItemService();
88+
ClarinItemService clarinItemService = ClarinServiceFactory.getInstance().getClarinItemService();
89+
90+
EPerson eperson = EPersonServiceFactory.getInstance().getEPersonService().findByEmail(context, adminEmail);
91+
context.turnOffAuthorisationSystem();
92+
context.setCurrentUser(eperson);
93+
context.restoreAuthSystemState();
94+
95+
String messagePrefix = dryRun ? "Found incorrect files metadata in" : "Updated";
96+
if (itemUuid != null) {
97+
// fixing only one item
98+
Item item = itemService.find(context, UUID.fromString(itemUuid));
99+
if (item == null) {
100+
throw new IllegalArgumentException("Item not found with the provided UUID");
101+
}
102+
boolean updated = updateItem(item, context, clarinItemService, itemService, dryRun, verboseOutput);
103+
if (updated) {
104+
System.out.println(dryRun ? "Files metadata are incorrect." : "Files metadata were updated.");
105+
} else {
106+
System.out.println("Files metadata are correct.");
107+
}
108+
} else if (collectionUuid != null) {
109+
// fixing items in collection
110+
CollectionService collectionService = ContentServiceFactory.getInstance().getCollectionService();
111+
Collection collection = collectionService.find(context, UUID.fromString(collectionUuid));
112+
if (collection == null) {
113+
throw new IllegalArgumentException("Invalid Collection UUID");
114+
}
115+
Iterator<Item> itemIterator = itemService.findAllByCollection(context, collection);
116+
Results results =
117+
updateItems(itemIterator, context, clarinItemService, itemService, dryRun, verboseOutput);
118+
System.out.printf("Checked %d items in Collection: \"%s\".\n",
119+
results.getItemsCount(), collection.getName());
120+
System.out.printf("%s %d items.\n", messagePrefix, results.getUpdatedItemsCount());
121+
} else {
122+
// fixing all items
123+
Iterator<Item> itemIterator = itemService.findAll(context);
124+
Results results =
125+
updateItems(itemIterator, context, clarinItemService, itemService, dryRun, verboseOutput);
126+
System.out.printf("Checked %d items.\n", results.getItemsCount());
127+
System.out.printf("%s %d items.\n", messagePrefix, results.getUpdatedItemsCount());
128+
}
129+
context.complete();
130+
}
131+
132+
System.out.println("\nItemFilesMetadataRepair Finished");
133+
}
134+
135+
private static Results updateItems(Iterator<Item> itemIterator,
136+
Context context,
137+
ClarinItemService clarinItemService,
138+
ItemService itemService,
139+
boolean dryRun,
140+
boolean verboseOutput) throws Exception {
141+
int itemsCount = 0;
142+
int updatedItemsCount = 0;
143+
while (itemIterator.hasNext()) {
144+
itemsCount++;
145+
boolean updated =
146+
updateItem(itemIterator.next(), context, clarinItemService, itemService, dryRun, verboseOutput);
147+
if (updated) {
148+
updatedItemsCount++;
149+
}
150+
}
151+
152+
return new Results(itemsCount, updatedItemsCount);
153+
}
154+
155+
private static boolean updateItem(Item item,
156+
Context context,
157+
ClarinItemService clarinItemService,
158+
ItemService itemService,
159+
boolean dryRun,
160+
boolean verboseOutput) throws Exception {
161+
boolean updated = false;
162+
163+
List<MetadataValue> filesCountValues =
164+
itemService.getMetadata(item, "local", "files", "count", Item.ANY);
165+
List<MetadataValue> filesSizeValues =
166+
itemService.getMetadata(item, "local", "files", "size", Item.ANY);
167+
List<MetadataValue> hasFilesValues =
168+
itemService.getMetadata(item, "local", "has", "files", Item.ANY);
169+
170+
int filesCount = 0;
171+
String filesCountValue = "undefined";
172+
if (!filesCountValues.isEmpty()) {
173+
filesCountValue = filesCountValues.get(0).getValue();
174+
try {
175+
filesCount = Integer.parseInt(filesCountValue);
176+
} catch (NumberFormatException ex) {
177+
// filesCount = 0
178+
}
179+
}
180+
long filesSize = 0;
181+
String filesSizeValue = "undefined";
182+
if (!filesSizeValues.isEmpty()) {
183+
filesSizeValue = filesSizeValues.get(0).getValue();
184+
try {
185+
filesSize = Long.parseLong(filesSizeValue);
186+
} catch (NumberFormatException ex) {
187+
// filesSize = 0
188+
}
189+
}
190+
String hasFiles = hasFilesValues.isEmpty() ? "no" : hasFilesValues.get(0).getValue();
191+
192+
List<Bundle> originalBundles = item.getBundles(Constants.CONTENT_BUNDLE_NAME);
193+
if (!CollectionUtils.isEmpty(originalBundles)) {
194+
Bundle bundle = originalBundles.get(0);
195+
boolean hasBitstreams = !CollectionUtils.isEmpty(bundle.getBitstreams());
196+
if (hasBitstreams && (filesCount == 0 || filesSize == 0 || !"yes".equals(hasFiles))) {
197+
if (verboseOutput) {
198+
String message = "Incorrect metadata: [files.count: %s, files.size: %s, has.files: %s], " +
199+
"in item '%s' with files.";
200+
System.out.printf((message) + "%n", filesCountValue, filesSizeValue, hasFiles, item.getHandle());
201+
}
202+
if (!dryRun) {
203+
clarinItemService.updateItemFilesMetadata(context, item, bundle);
204+
}
205+
updated = true;
206+
} else if (!hasBitstreams && (filesCount > 0 || filesSize > 0 || "yes".equals(hasFiles))) {
207+
if (verboseOutput) {
208+
String message = "Incorrect metadata: [files.count: %s, files.size: %s, has.files: %s], " +
209+
"in item '%s' without files.";
210+
System.out.printf((message) + "%n", filesCountValue, filesSizeValue, hasFiles, item.getHandle());
211+
}
212+
if (!dryRun) {
213+
itemService.clearMetadata(
214+
context, item, "local", "has", "files", Item.ANY);
215+
itemService.clearMetadata(
216+
context, item, "local", "files", "count", Item.ANY);
217+
itemService.clearMetadata(
218+
context, item, "local", "files", "size", Item.ANY);
219+
itemService.addMetadata(
220+
context, item, "local", "has", "files", Item.ANY, "no");
221+
itemService.addMetadata(
222+
context, item, "local", "files", "count", Item.ANY, "0");
223+
itemService.addMetadata(
224+
context, item, "local", "files", "size", Item.ANY, "0");
225+
}
226+
updated = true;
227+
}
228+
}
229+
return updated;
230+
}
231+
232+
private static void printHelpAndExit(Options options) {
233+
// print the help message
234+
HelpFormatter myHelp = new HelpFormatter();
235+
myHelp.printHelp("dsrun org.dspace.app.itemupdate.ItemFilesMetadataRepair \n", options);
236+
System.exit(0);
237+
}
238+
239+
private static class Results {
240+
private final int itemsCount;
241+
private final int updatedItemsCount;
242+
243+
public Results(int itemsCount, int updatedItemsCount) {
244+
this.itemsCount = itemsCount;
245+
this.updatedItemsCount = updatedItemsCount;
246+
}
247+
248+
public int getItemsCount() {
249+
return itemsCount;
250+
}
251+
252+
public int getUpdatedItemsCount() {
253+
return updatedItemsCount;
254+
}
255+
}
256+
257+
}

‎dspace-api/src/main/java/org/dspace/xmlworkflow/XmlWorkflowServiceImpl.java‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -264,7 +264,11 @@ public void alertUsersOnTaskActivation(Context c, XmlWorkflowItem wfi, String em
264264
mail.addArgument(argument);
265265
}
266266
for (EPerson anEpa : epa) {
267-
mail.addRecipient(anEpa.getEmail());
267+
String email = anEpa.getEmail();
268+
if (email != null && email.contains(";")) {
269+
email = email.split(";")[0].trim();
270+
}
271+
mail.addRecipient(email);
268272
}
269273

270274
mail.send();

‎dspace-server-webapp/src/main/java/org/dspace/app/rest/ClarinBitstreamImportController.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
import org.dspace.content.service.BundleService;
3434
import org.dspace.content.service.ItemService;
3535
import org.dspace.content.service.clarin.ClarinBitstreamService;
36+
import org.dspace.content.service.clarin.ClarinItemService;
3637
import org.dspace.core.Constants;
3738
import org.dspace.core.Context;
3839
import org.springframework.beans.factory.annotation.Autowired;
@@ -72,6 +73,8 @@ public class ClarinBitstreamImportController {
7273
private Utils utils;
7374
@Autowired
7475
private MostRecentChecksumService checksumService;
76+
@Autowired
77+
protected ClarinItemService clarinItemService;
7578

7679
/**
7780
* Endpoint for import bitstream, whose file already exists in assetstore under internal_id
@@ -197,6 +200,9 @@ public BitstreamRest importBitstreamForExistingFile(HttpServletRequest request)
197200
throw new AccessDeniedException("You do not have write rights to update the Bundle's item");
198201
}
199202
if (item != null) {
203+
// Update item file metadata after the bitstream size has changed
204+
clarinItemService.updateItemFilesMetadata(context,
205+
item, bundle);
200206
itemService.update(context, item);
201207
}
202208
bundleService.update(context, bundle);

0 commit comments

Comments
 (0)