CMR-11422: Add parameter to only allow validate keyword usage by provider (Enforcement list)

[eudoroolivares2016]

Overview

What is the objective?

Adding a parameter store to be able to prevent providers in an enforced list e.g. the esdis providers +- to no longer be able to use the validate-keyword: false header which has allowed unsearchable metadata into the system

There is a temporary carveout for the KMS API itself until we resolve the cache delay issue either by removing the CMR cache entirely or having KMS issue a cache refresh to CMR CMR-11524.

What are the changes?

Added a new parameter store that is a comma sep list, pulled that into the conditional logic of the validate-keyword value, refactoring KMS lookup function.

What areas of the application does this impact?

CMR ingest and metadata validation

Testing

1. Create multiple providers if in SIT, set this parameter store in the ingest-app redeploy CMR
2. Ensure that the providers on that list are unable to use validate-keyword header. If they do it will always resolve to being true. Ensure that the KMS API if correcting a metadata keyword issue is still able to make sure of it. (SIT only testing possible) https://cmr.sit.earthdata.nasa.gov/kms/metadata_correction
3. Delete local keyword cache. Ensure that running curl -i -X POST -H "Authorization: mock-echo-system-token" http://localhost:3006/caches/refresh/kms regenerates all the caches as it does on main

Required Checklist

• New and existing unit and int tests pass locally and remotely
• clj-kondo has been run locally and all errors in changed files are corrected
• I have commented my code, particularly in hard-to-understand areas
• I have made changes to the documentation (if necessary)
• My changes generate no new warnings

Additional Checklist

• I have removed unnecessary/dead code and imports in files I have changed
• I have cleaned up integration tests by doing one or more of the following:
  • migrated any are2 tests to are3 in files I have changed
  • de-duped, consolidated, removed dead int tests
  • transformed applicable int tests into unit tests
  • reduced number of system state resets by updating fixtures. Ex) (use-fixtures :each (ingest/reset-fixture {})) to be :once instead of :each

[eereiter]

Please remove debugging code.

[eereiter]

So if the CMR_KEYWORD_ENFORCED_PROVIDERS parameter store does not exist an empty array [] is returned. What if an empty string is defined in the parameter store ""

When the input % is an empty string "", this snippet will evaluate to a list containing a single empty keyword: (:).

Step-by-Step Breakdown
(string/split "" #",")
Splitting an empty string does not return an empty list. It returns a vector containing the empty string itself: [""].
(string/trim "")
Trimming an empty string simply returns the empty string: "".
(keyword "")
Passing an empty string to keyword produces an empty keyword printed as : in Clojure.

Why This Is Dangerous
An empty keyword (:) is valid Clojure syntax, but it is rarely intended.
It can cause silent bugs because:It evaluates to truthy in conditional checks.
It will not match a proper keyword like :provider-id.It consumes a tiny amount of memory in the keyword table that cannot be garbage collected.

;; Option 1: Remove blanks before processing (Recommended)
(->> (string/split % #",")
(remove string/blank?)
(map (comp keyword string/trim)))
;; Input: "" => Output: ()

;; Option 2: Use keep to filter out blanks during the map step
(keep #(when-not (string/blank? %)
(keyword (string/trim %)))
(string/split % #","))
;; Input: "" => Output: ()

[eereiter]

I have a couple of small changes

[codecov-commenter]

Codecov Report

❌ Patch coverage is 46.00000% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.25%. Comparing base (0b364bd) to head (5bcd401).

Files with missing lines	Patch %	Lines
ingest-app/src/cmr/ingest/api/collections.clj	16.66%	14 Missing and 1 partial ⚠️
...app-lib/src/cmr/common_app/services/kms_lookup.clj	70.37%	8 Missing ⚠️
ingest-app/src/cmr/ingest/config.clj	20.00%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2490      +/-   ##
==========================================
+ Coverage   29.20%   29.25%   +0.04%     
==========================================
  Files        1014     1014              
  Lines       71257    71199      -58     
  Branches     1265     1268       +3     
==========================================
+ Hits        20811    20827      +16     
+ Misses      49244    49167      -77     
- Partials     1202     1205       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sylvesterkaczmarek]

Provider enforcement is bypassed by the mere presence of cmr-send-kms-metadata-fixer; the new tests even accept the value false. Nothing in get-validation-options authenticates that request header, so an enforced provider can set it and regain the normal validate-keywords=false path. Gate this carveout on trusted KMS caller identity rather than a client-supplied header, and add a spoofing regression.