test: add CLARIN consumer-contract coverage + branch-differential CI

The CLARIN/UFAL surface was merged onto dtq (main -> dtq, 2026-08-17) with
zero tests, while three main-lineage consumers depend on it. This lands the
counterpart to test_repo_usage_contract.py for that surface and makes the
main+dtq merge validatable instead of assumed.

Defects fixed (characterization-tested; each crashed or mis-answered a real
consumer call before this change):
- get_bundle_by_name / get_resource_policy / get_items_from_collection: guard
  the None from a failed fetch instead of a NoneType subscript (D1/D3/D6).
- get_resource_policy: empty policy list -> None, not IndexError (D2).
- get_user_by_email: a 404 miss -> None, not a truthy uuid-less User that
  slips past `if user:` (D4).
- create_submit_group: a 201 with no body -> None, not Group(None) (D5).
- Group(None) / User(None): construct an empty object, not TypeError (D7).
- get_owningCollection: on non-200 return None (not an empty truthy
  Collection) so src/repo/_audit.py's `owning_col is None and
  last_err.status_code == 401` reauth retry actually fires (D8, found via the
  consumer contract).

Tests: +69 (68 -> 137). New suites test_clarin_read/write, test_models_clarin,
test_clarin_usage_contract, test_transport_hardening; _helpers gains CLARIN
builders; TestGetItems gains the no-arg form; test_repo_usage_contract gains
the owningCollection reauth contract.

CI: tests.yml now builds on dtq/main/feat/**/fix/**; a differential-contract
job runs the CLARIN suite against BOTH the dtq and main implementations
(@pytest.mark.dtq_only marks the fixes/deltas skipped on the main leg) as the
merge gate; a coverage floor (--cov-fail-under=70) and secret-gated
consumer-smoke jobs are added. pytest.ini registers the markers.

Verified locally: leg A (dtq) 137 pass; leg B (main impl) 45 shared contracts
pass, 14 dtq_only deselected; the 14 dtq_only CLARIN tests fail on the main
impl, confirming the deltas are real.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: jr-rk

.github/workflows/tests.yml

@@ -2,8 +2,11 @@ name: Tests
 
 on:
   push:
-    branches: [ dtq ]
+    # dtq is the mainline; main is the CLARIN branch we are validating a merge
+    # into. feat/** and fix/** get CI before they open a PR.
+    branches: [ dtq, main, 'feat/**', 'fix/**' ]
   pull_request:
+  workflow_dispatch:      # manual validation of a merge candidate
 
 permissions:
   contents: read
@@ -37,4 +40,125 @@ jobs:
           pip install -r requirements-test.txt
 
       - name: Run tests
-        run: python -m pytest tests/ -v
+        # Coverage floor guards against the CLARIN surface silently sliding back
+        # toward the zero it had before test/clarin-usage-coverage landed.
+        run: >
+          python -m pytest tests/ -v
+          --cov=dspace_rest_client --cov-report=term-missing
+          --cov-fail-under=70
+
+  differential-contract:
+    # THE MERGE GATE. Run the CLARIN consumer-contract suite against BOTH the
+    # dtq implementation (this checkout) and the main implementation (swapped in
+    # from origin/main). A test green on both proves the merge preserves that
+    # behaviour. Tests that deliberately encode a dtq fix or behaviour change
+    # are marked @pytest.mark.dtq_only and are skipped on the main leg.
+    #
+    # Leg selection targets the four CLARIN test files explicitly: the DQ test
+    # modules import dtq-only symbols (ResourcePolicy, ...) at module scope, so
+    # collecting them against the main implementation would be an import error.
+    name: CLARIN contract vs ${{ matrix.impl }} impl
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        impl: [dtq, main]
+
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0        # need origin/main to swap the implementation in
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.10"
+          cache: pip
+          cache-dependency-path: requirements-test.txt
+
+      - name: Install test deps
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-test.txt
+
+      - name: Swap in the ${{ matrix.impl }} implementation
+        if: matrix.impl != 'dtq'
+        run: git checkout "origin/${{ matrix.impl }}" -- dspace_rest_client/
+
+      - name: Run CLARIN contract suite
+        run: |
+          FILES="tests/test_clarin_read.py tests/test_clarin_write.py \
+                 tests/test_models_clarin.py tests/test_clarin_usage_contract.py"
+          if [ "${{ matrix.impl }}" = "dtq" ]; then
+            python -m pytest $FILES -v            # full CLARIN surface, incl. dtq_only
+          else
+            python -m pytest $FILES -v -m "not dtq_only"   # shared contract only
+          fi
+
+  # ---- Consumer smoke jobs -------------------------------------------------
+  # Turn "the API surface is a superset" into "the consumers still import/run".
+  # Gated on CONSUMER_READ_TOKEN: until that read-scoped token for the private
+  # consumer repos exists, the gate job reports enabled=false and consumer-smoke
+  # is skipped (a clean green), per plan §6.3.
+  check-consumer-token:
+    runs-on: ubuntu-latest
+    outputs:
+      enabled: ${{ steps.probe.outputs.enabled }}
+    steps:
+      - id: probe
+        env:
+          TOKEN: ${{ secrets.CONSUMER_READ_TOKEN }}
+        run: echo "enabled=${{ env.TOKEN != '' }}" >> "$GITHUB_OUTPUT"
+
+  consumer-smoke:
+    needs: check-consumer-token
+    if: needs.check-consumer-token.outputs.enabled == 'true'
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - repo: DSpace-ISstag-integration
+            ref: main
+            smoke: python -m pytest tests/ mcp/tests/ -q
+          - repo: dspace-rest-test
+            ref: master
+            smoke: python -c "import dspace_rest_client.client"
+          - repo: dspace-import-clarin
+            ref: main
+            smoke: python -c "import dspace_rest_client.client"
+          # dspace-item-importer is intentionally omitted until its .gitmodules
+          # is repointed off the deleted `dtq-dev` branch (plan §6.3 / brief §5).
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          path: candidate
+
+      - uses: actions/checkout@v6
+        with:
+          repository: dataquest-dev/${{ matrix.repo }}
+          ref: ${{ matrix.ref }}
+          token: ${{ secrets.CONSUMER_READ_TOKEN }}
+          submodules: recursive
+          path: consumer
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.10"
+
+      - name: Point the consumer submodule at this candidate commit
+        run: |
+          rm -rf consumer/libs/dspace-rest-python
+          cp -r candidate consumer/libs/dspace-rest-python
+
+      - name: Install and smoke
+        working-directory: consumer
+        run: |
+          python -m pip install --upgrade pip
+          pip install ./libs/dspace-rest-python
+          if [ -f requirements.lock ]; then pip install -r requirements.lock; fi
+          if [ -f libs/dspace-rest-python/requirements-test.txt ]; then
+            pip install -r libs/dspace-rest-python/requirements-test.txt
+          fi
+          ${{ matrix.smoke }}

.gitignore

@@ -13,3 +13,5 @@ __pypackages__/
 env/
 venv/
 .idea/
+.coverage
+coverage.xml

dspace_rest_client/client.py

@@ -1096,7 +1096,12 @@ def get_owningCollection(self, item_uuid):
         url = f'{self.API_ENDPOINT}/core/items/{item_uuid}/owningCollection'
         try:
             r = self.api_get(url, None, None)
-            self.verify_response(r, f"item:{item_uuid}",  True)
+            # On a non-200, verify_response records self._last_err and returns
+            # False - return None here (not an empty, truthy Collection) so the
+            # caller's `owning_col is None and last_err.status_code == 401`
+            # reauth path (src/repo/_audit.py) actually fires.
+            if not self.verify_response(r, f"item:{item_uuid}", True):
+                return None
             r_json = parse_json(response=r)
             return Collection(r_json)
         except ValueError:
@@ -1257,7 +1262,12 @@ def create_submit_group(self, collection):
         url = f'{self.API_ENDPOINT}/core/collections/{collection.uuid}/submittersGroup'
         r = self.api_post(url, json={}, params=None)
         if r.status_code == 201:
-            return Group(parse_json(r))
+            # a 201 with an empty/invalid body would make Group(None) here; return
+            # None instead so a caller's `if not group` guard fires cleanly rather
+            # than passing a uuid-less Group into add_member()
+            j = parse_json(r)
+            if j:
+                return Group(j)
         return None
 
     def add_member(self, group, eperson):
@@ -1350,7 +1360,9 @@ def get_items_from_collection(self, collection_id, page=0, size=1000):
         items = list()
         r = self.api_get(url)
         r_json = parse_json(r)
-        if '_embedded' in r_json:
+        # a failed request parses to None; return the empty list rather than
+        # crashing on `'_embedded' in None`
+        if r_json and '_embedded' in r_json:
             if 'searchResult' in r_json['_embedded']:
                 if '_embedded' in r_json['_embedded']['searchResult']:
                     for item_resource in r_json['_embedded']['searchResult']['_embedded']['objects']:
@@ -1367,7 +1379,9 @@ def get_bundle_by_name(self, name, item_uuid):
         """
         url = f'{self.API_ENDPOINT}/core/items/{item_uuid}/bundles'
         r_json = self.fetch_resource(url, params=None)
-        if '_embedded' in r_json:
+        # fetch_resource returns None on any non-200 (records self._last_err);
+        # guard so a failed lookup is a clean None, not a NoneType subscript crash
+        if r_json and '_embedded' in r_json:
             if 'bundles' in r_json['_embedded']:
                 for bundle in r_json['_embedded']['bundles']:
                     if bundle['name'] == name:
@@ -1381,9 +1395,13 @@ def get_resource_policy(self, bundle_uuid):
         url = f'{self.API_ENDPOINT}/authz/resourcepolicies/search/resource?uuid={bundle_uuid}&embed=eperson&embed=group'
         r = self.api_get(url)
         r_json = parse_json(r)
-        if '_embedded' in r_json:
-            if 'resourcepolicies' in r_json['_embedded']:
-                return r_json['_embedded']['resourcepolicies'][0]
+        # guard against a failed request (r_json is None) and against an empty
+        # policy list - both must be a clean None, not a TypeError/IndexError
+        if r_json and '_embedded' in r_json and 'resourcepolicies' in r_json['_embedded']:
+            policies = r_json['_embedded']['resourcepolicies']
+            if policies:
+                return policies[0]
+        return None
 
     def create_resource_policy(self, resource_uuid, data, group_uuid=None, eperson_uuid=None):
         """
@@ -1475,7 +1493,15 @@ def get_user_by_email(self, email):
         params = {'email': email}
         try:
             response = self.api_get(url, params=params)
+            # a miss returns 404 (with a JSON error body): building a User from
+            # that yields a truthy, uuid-less object that passes `if user:` and
+            # fails much later. Treat any non-200 as "no such user" -> None.
+            if response.status_code != 200:
+                _logger.error(f"No user for email {email}: {response.status_code}")
+                return None
             user_data = parse_json(response)
+            if not user_data:
+                return None
             return User(user_data)
         except Exception as e:
             _logger.error(f"Error retrieving user by email {email}: {e}")

dspace_rest_client/models.py

@@ -407,6 +407,7 @@ def __init__(self, api_resource=None):
         @param api_resource: API result object to use as initial data
         """
         super().__init__(api_resource)
+        api_resource = api_resource or {}
         self.type = 'group'
         if 'name' in api_resource:
             self.name = api_resource['name']
@@ -442,6 +443,7 @@ def __init__(self, api_resource=None):
         @param api_resource: API result object to use as initial data
         """
         super().__init__(api_resource)
+        api_resource = api_resource or {}
         self.type = 'user'
         if 'name' in api_resource:
             self.name = api_resource['name']

pytest.ini

@@ -0,0 +1,9 @@
+[pytest]
+# Markers used to slice the suite for the branch-differential CI run
+# (see .github/workflows/tests.yml :: differential-contract). A pytest.ini
+# takes precedence over any [tool.pytest.ini_options] a future pyproject.toml
+# might add, so the marker registry stays in one place regardless of packaging.
+markers =
+    dtq_only: behaviour introduced on dtq; not expected to hold on the main implementation
+    clarin: exercises the CLARIN/UFAL surface (main-lineage consumers)
+    dq: exercises the DQ integration surface (DSpace-ISstag-integration)

requirements-test.txt

@@ -5,3 +5,5 @@
 pytest>=7.0
 requests-mock>=1.11
 requests
+# coverage floor is enforced in CI (tests.yml :: test job, --cov-fail-under)
+pytest-cov>=4.0

tests/_helpers.py

@@ -32,6 +32,10 @@
 COLLECTION_UUID = "22222222-2222-2222-2222-222222222222"
 BITSTREAM_UUID = "9f54ef33-c454-4d8e-a5fe-79d8291045ba"
 ANON_GROUP_UUID = "6ecfd145-3b7d-429e-ab31-ef6905a05763"
+# Used by the CLARIN-side suites (eperson/group lookups, submit groups).
+EPERSON_UUID = "33333333-3333-3333-3333-333333333333"
+GROUP_UUID = "44444444-4444-4444-4444-444444444444"
+BUNDLE_UUID = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
 
 
 def make_client(api_endpoint: str = API) -> DSpaceClient:
@@ -112,3 +116,65 @@ def policy_json(pid: int = 1, action: str = "READ", group_name: str = "Anonymous
     if start_date is not None:
         d["startDate"] = start_date
     return d
+
+
+def raw_policy_json(pid: int = 1, action: str = "READ", **extra) -> dict:
+    """A resource policy in the *raw* shape the CLARIN ``get_resource_policy``
+    returns (a plain dict the caller subscripts as ``["id"]``), not a model."""
+    d = {"id": pid, "action": action, "type": "resourcepolicy"}
+    d.update(extra)
+    return d
+
+
+def group_json(uuid: str = GROUP_UUID, name: str = "Anonymous",
+               permanent: bool = False, **extra) -> dict:
+    d = {"uuid": uuid, "name": name, "type": "group", "permanent": permanent}
+    d.update(extra)
+    return d
+
+
+def user_json(uuid: str = EPERSON_UUID, email: str = "tester@dspace.test",
+              name: str = "Tester", netid: str = None, can_login: bool = True,
+              **extra) -> dict:
+    d = {"uuid": uuid, "type": "eperson", "name": name, "email": email,
+         "canLogIn": can_login}
+    if netid is not None:
+        d["netid"] = netid
+    d.update(extra)
+    return d
+
+
+def label_json(lid: int = 10, label: str = "PUB", title: str = "Publicly available",
+               icon: str = "pub.png", extended: bool = False) -> dict:
+    return {"id": lid, "label": label, "title": title, "icon": icon,
+            "extended": extended}
+
+
+def license_json(lid: int = 1, name: str = "CC-BY",
+                 definition: str = "https://creativecommons.org/licenses/by/4.0/",
+                 confirmation: int = 1, required_info: str = "SEND_TOKEN",
+                 label: dict = None, extended: list = None) -> dict:
+    d = {"id": lid, "name": name, "definition": definition,
+         "confirmation": confirmation, "requiredInfo": required_info}
+    if label is not None:
+        d["clarinLicenseLabel"] = label
+    if extended is not None:
+        d["extendedClarinLicenseLabels"] = extended
+    return d
+
+
+def clarin_allowance_json(aid: int = 1, **extra) -> dict:
+    d = {"id": aid, "type": "clarinlruallowance"}
+    d.update(extra)
+    return d
+
+
+def search_envelope(items: list) -> dict:
+    """The ``discover/search/objects`` HAL envelope, wrapping each item as an
+    ``indexableObject``. Used by ``get_items_from_collection`` and
+    ``search_objects``.
+    """
+    return {"_embedded": {"searchResult": {
+        "page": {"totalElements": len(items)},
+        "_embedded": {"objects": [
+            {"_embedded": {"indexableObject": it}} for it in items]}}}}