[Port to dtq-dev] Issue dspace-customers#903: stop ItemConverter writing to the DB on every item GET - #1411
Conversation
ItemConverter.convert() -- invoked during read-only REST serialization -- called ClarinItemService.updateItemDatesMetadata(), which runs clearMetadata/addMetadata. That triggered Hibernate dirty-checking and DB writes on every item GET (rolled back by DSpaceRequestContextFilter), plus per-request log noise. Split derivation from persistence: - ClarinItemService/Impl: extract deriveDateIssuedFromApproximateDate(Item) (pure, returns the derived value or null); updateItemDatesMetadata() delegates to it and only writes when the value actually changes. dtq-dev's log.debug for the empty approximate-date case is kept (the WARN->DEBUG change already landed via 84e9f3a), which is why the source cherry-pick conflicted here. - ItemServiceImpl.update(): derive + persist inside the existing isMetadataModified guard, so the write happens on PATCH/PUT, not on read. - ItemConverter.convert(): drop the write-on-read call; instead override dc.date.issued on the REST DTO from the derived value (display-only, honouring metadataExposureService.isHidden) so stale DB values still display correctly. Port of dataquest-dev/dspace-customers#903 (item 1). Source: customer/zcu-data 24f31c3 (adapted; cherry-pick conflicted on the WARN->DEBUG divergence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR ports dspace-customers#903 to dtq-dev to eliminate unintended database writes during REST GET /items/{id} serialization by removing entity mutation from the read/DTO conversion path and relocating persistence to the item update path.
Changes:
- Removed the
ItemConverter.convert()write-on-read call toupdateItemDatesMetadata()and replaced it with a DTO-only override ofdc.date.issuedderived fromlocal.approximateDate.issued. - Added a pure derivation method
deriveDateIssuedFromApproximateDate(Item)toClarinItemService/ClarinItemServiceImpl, and updatedupdateItemDatesMetadata()to only write when the stored value differs. - Updated
ItemServiceImpl.update()to persist the derived date only when item metadata is modified (PATCH/PUT).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/ItemConverter.java | Removes write-on-read and derives dc.date.issued for display on the REST DTO only. |
| dspace-api/src/main/java/org/dspace/content/service/clarin/ClarinItemService.java | Adds a pure derivation API to support read-only DTO overrides. |
| dspace-api/src/main/java/org/dspace/content/ItemServiceImpl.java | Persists derived date only during update flows when metadata changes. |
| dspace-api/src/main/java/org/dspace/content/clarin/ClarinItemServiceImpl.java | Extracts derivation logic and avoids unnecessary metadata writes when already up-to-date. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- ClarinItemServiceImpl.updateItemDatesMetadata(): skip the dc.date.issued write only when the field holds exactly one value equal to the derived value. A multi-valued field is now still normalized to the single derived value instead of being skipped on a first-value match (Copilot review #2). - ItemConverter: set the derived dc.date.issued DTO value via MetadataRest#put instead of getMap().put(Collections.singletonList(...)), so place is normalized and list semantics stay consistent with other fields; drops the now-unused Collections import and manual setPlace(0) (Copilot review #1). - Add ClarinItemServiceImplTest: pure Mockito unit tests covering deriveDateIssuedFromApproximateDate (empty/blank/non-numeric/sequence) and the skip/normalize guard (single-equal skip, multi-value normalize, differ). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Addressed both Copilot review comments in
Added |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (1)
dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/ItemConverter.java:97
- Avoid the magic number
-1for metadata confidence. DSpace definesChoices.CF_UNSETfor the default/unset confidence (andMetadataValuedefaults confidence to -1), so using the constant improves readability and reduces the chance of inconsistency if the value ever changes.
MetadataValueRest dateRest = new MetadataValueRest(derivedValue);
dateRest.setConfidence(-1);
// MetadataRest#put normalizes place and keeps Arrays.asList semantics consistent with other fields
target.getMetadata().put("dc.date.issued", dateRest);
…er writing to the DB on every item GET (#1411) (#1429) Source: 99074f2 (dtq-dev PR #1411) ItemConverter.convert() runs during read-only REST serialization, yet it called ClarinItemService.updateItemDatesMetadata(), which does clearMetadata + addMetadata on dc.date.issued. Every single item GET therefore dirtied the Hibernate session and wrote to the database (the write was rolled back by DSpaceRequestContextFilter, but the cost and the provenance noise were real). Derivation is now separated from persistence: - ClarinItemService / ClarinItemServiceImpl: new pure deriveDateIssuedFromApproximateDate(Item) returning the derived value or null; updateItemDatesMetadata() delegates to it and writes only when dc.date.issued does not already hold exactly that single value (a multi-valued field is still normalized down to the single derived value). - ItemServiceImpl.update(): new @Autowired ClarinItemService and a guarded clarinItemService.updateItemDatesMetadata(context, item) as the first statement inside the existing `if (item.isMetadataModified() || item.isModified())` block, under its own `if (item.isMetadataModified())` -- deliberately NOT merged into the vanilla event-dispatch `if` a few lines below. - ItemConverter.convert(): the write-on-read call is gone; dc.date.issued is instead overridden on the REST DTO via MetadataRest#put, honouring metadataExposureService.isHidden, so stale DB values still display correctly. - New ClarinItemServiceImplTest: 9 pure Mockito unit tests. Behaviour change (same as on dtq-dev): the derived dc.date.issued is now persisted lazily -- on the next PATCH/PUT of the item -- instead of on every GET. There is no backfill, so an item that is never edited keeps its stale stored value in the database while the REST response shows the derived one. Adaptations for the v9 base: - ItemServiceImpl: only the ClarinItemService field was taken from the source's field hunk. dtq-dev also declares ClarinMatomoBitstreamTracker and ProvenanceService there; neither exists on dtq-dev-9-base, so re-adding them would not compile. update() itself uses Instant.now() on this branch; the new statement was placed above it, unchanged. - ItemConverter: 32 added / 12 removed instead of the source's 32 / 13, because dtq-dev has a blank line after the removed try/catch block and this branch does not. - ClarinItemServiceImpl, ClarinItemService and ClarinItemServiceImplTest applied verbatim -- the WARN->DEBUG divergence the source commit had to resolve is already on this branch, so no conflict arose there. Co-authored-by: jurinecko <juraj.roka@dataquest.sk> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Problem
Every REST
GETof an item triggered database writes.ItemConverter.convert()— invoked during read-only serialization — calledClarinItemService.updateItemDatesMetadata(), which doesclearMetadata/addMetadata. This caused Hibernate dirty-checking and DB writes on read (rolled back byDSpaceRequestContextFilter.abort()), plus per-request log noise, on every item GET.Port of dataquest-dev/dspace-customers#903 (item 1 of a 6-item umbrella). Source:
customer/zcu-data24f31c330d— cherry-pick conflicted (see below), applied as an adapted port.Root cause
Date-derivation (turn
local.approximateDate.issuedintodc.date.issued) is a write that was being run from a read path so that stale DB values would still display correctly. The write belongs inupdate(); the display correction belongs on the DTO.Change set
Split derivation (pure) from persistence (write):
ClarinItemService(interface) — addderiveDateIssuedFromApproximateDate(Item).ClarinItemServiceImpl— extract the parsing intoderiveDateIssuedFromApproximateDate()(returns the derived year,"0000", ornull);updateItemDatesMetadata()delegates to it and writes only when the stored value differs — skipping the write only whendc.date.issuedalready holds exactly one value equal to the derived one (multi-valued fields are still normalized to the single derived value). Kept dtq-dev'slog.debugfor the empty-approximate-date case (the WARN→DEBUG change already landed on dtq-dev via84e9f3a2c2; the source commit still hadlog.warn, which is why the cherry-pick conflicted here).ItemServiceImpl.update()— callclarinItemService.updateItemDatesMetadata()inside the existingisMetadataModified()guard, so the date is derived and persisted only when metadata actually changes (PATCH/PUT).ItemConverter.convert()— remove the write-on-read call; instead overridedc.date.issuedon theItemRestDTO from the derived value viaMetadataRest#put(display-only, honouringmetadataExposureService.isHidden). No entity mutation, no DB write.Review updates (
b238a0b397)Both Copilot review comments addressed and unit coverage added:
ClarinItemServiceImpl) — skip the write only for a single already-derived value; a multi-valueddc.date.issuedis still collapsed to the single derived value (was skipping on a first-value match).ItemConverter) — build the derived value viaMetadataRest#put(place normalization +Arrays.asListsemantics, consistent with other fields) instead ofgetMap().put(Collections.singletonList(...)); dropped the manualsetPlace(0)and the now-unusedCollectionsimport.ClarinItemServiceImplTest(9 pure-Mockito unit tests) covering derivation (empty / blank / non-numeric / sequence / single) and the skip/normalize guard.Validation
The no-write property is covered structurally (the converter no longer calls
updateItemDatesMetadata) plus the unit-level skip/normalize guard. Full-context runtime (docker / DSpace test-kernel) is not bootstrappable in this environment. CI exercises the approximate-date display path viaItemRestRepositoryIT— note those ITs assert the displayed value; they are not a dedicated "no DB write on GET" assertion.Risk & rollback
Touches core CLARIN item serialization and
ItemServiceImpl.update(). For the standard read/display path, behaviour is preserved: items withlocal.approximateDate.issuedstill show the deriveddc.date.issuedin REST responses, and the value is now persisted on metadata change (previously the read-path write was always rolled back, so nothing was ever persisted). Regression focus: approximate-date handling. Revert = single commit.Behaviour notes
local.approximateDate.issuedpresent it is authoritative: any metadata-modifyingupdate()(PATCH/PUT) derives and overwritesdc.date.issued, so a client-supplieddc.date.issuedis replaced. This matches what the REST API already displayed."0000".Notes / assumptions
Conflict resolution kept dtq-dev-side changes (the
log.debugdowngrade). The four-file split matches the source commit's intent; the display-only override reuses the shared derivation rather than duplicating parsing.