Skip to content

Add Eclipse P2 mirror to avoid download.eclipse.org outages (neural-search) - #1940

Merged
heemin32 merged 2 commits into
opensearch-project:mainfrom
peterzhuamazon:eclipse-p2-mirror
Aug 13, 2026
Merged

Add Eclipse P2 mirror to avoid download.eclipse.org outages (neural-search)#1940
heemin32 merged 2 commits into
opensearch-project:mainfrom
peterzhuamazon:eclipse-p2-mirror

Conversation

@peterzhuamazon

Copy link
Copy Markdown
Member

Description

Add withP2Mirrors to Spotless eclipse formatter to use ci.opensearch.org/eclipse/ CloudFront mirror instead of hitting download.eclipse.org directly. This prevents CI failures when Eclipse's download server is down or slow.

Issues Resolved

opensearch-project/opensearch-build#6421 (comment)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@peterzhuamazon

Copy link
Copy Markdown
Member Author

expected mirror update.

@github-actions

Copy link
Copy Markdown

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit f285ee5.

Hard block: Issues at High severity or above will block this PR from merging.

PathLineSeverityDescription
formatter/formatting.gradle10highAdds a P2 mirror that redirects Eclipse plugin downloads from the official 'https://download.eclipse.org/' to 'https://ci.opensearch.org/'. This modifies the source from which build-time Eclipse formatter artifacts (plugins/features) are fetched. Per mandatory supply chain rules, any change to package source URLs or build plugin registries must be flagged regardless of whether the target domain appears legitimate. A compromised or attacker-controlled mirror at ci.opensearch.org could serve malicious Eclipse plugins executed during the build process.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 1 | Medium: 0 | Low: 0


Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.


⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

@peterzhuamazon peterzhuamazon moved this from Backlog to In review in OpenSearch Engineering Effectiveness Aug 13, 2026
@peterzhuamazon peterzhuamazon added the skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis. label Aug 13, 2026
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.43%. Comparing base (746c7df) to head (d0f19a4).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1940      +/-   ##
============================================
+ Coverage     83.42%   83.43%   +0.01%     
- Complexity     3883     3886       +3     
============================================
  Files           291      291              
  Lines         13821    13821              
  Branches       2295     2295              
============================================
+ Hits          11530    11532       +2     
  Misses         1457     1457              
+ Partials        834      832       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit d0f19a4)

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

PR Scope Mismatch

The PR description states this change adds withP2Mirrors to the Spotless eclipse formatter to use the ci.opensearch.org/eclipse/ CloudFront mirror. However, the diff only modifies .github/workflows/CI.yml (adding fail-fast: false and re-indenting the integMultiNodeTest job) and contains no Gradle/Spotless configuration change. The stated fix for the Eclipse P2 outage issue is not present in this diff.

jobs:
  Get-CI-Image-Tag:
    uses: opensearch-project/opensearch-build/.github/workflows/get-ci-image-tag.yml@main
    with:
      product: opensearch

  Check-neural-search-linux:
    needs: Get-CI-Image-Tag
    strategy:
      fail-fast: false
      matrix:
        java: [21, 25]
        os: [ubuntu-latest]

    permissions:
      contents: read
      id-token: write

    name: Gradle Check Linux
    runs-on: ${{ matrix.os }}
    container:
      # using the same image which is used by opensearch-build team to build the OpenSearch Distribution
      # this image tag is subject to change as more dependencies and updates will arrive over time
      image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
      # need to switch to root so that github actions can install runner binary on container without permission issues.
      options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

    steps:
      - name: Run start commands
        run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4

      - name: Setup Java ${{ matrix.java }}
        uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
        with:
          distribution: 'temurin'
          java-version: ${{ matrix.java }}
          cache: 'gradle'

      - name: Run build
        run: |
          chown -R 1000:1000 `pwd`
          su `id -un 1000` -c "./gradlew check --parallel -x integTest"

      - name: Upload Coverage Report
        if: ${{ !cancelled() && contains(matrix.os, 'ubuntu') && contains(matrix.java, '21') }}
        uses: codecov/codecov-action@75cd11691c0faa626561e295848008c8a7dddffe # v5
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          use_oidc: ${{ env.ACTIONS_ID_TOKEN_REQUEST_URL != '' }}
          use_pypi: true

  Check-neural-search-windows:
    strategy:
      fail-fast: false
      matrix:
        java: [21, 25]
        os: [windows-latest]

    name: Gradle Check Windows
    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4

      - name: Setup Java ${{ matrix.java }}
        uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
        with:
          distribution: 'temurin'
          java-version: ${{ matrix.java }}
          cache: 'gradle'

      - name: Run build
        run: |
          ./gradlew check --parallel -x integTest

  Precommit-neural-search-linux:
    needs: Get-CI-Image-Tag
    strategy:
      fail-fast: false
      matrix:
        java: [21, 25]
        os: [ubuntu-latest]

    name: Pre-commit Linux
    runs-on: ${{ matrix.os }}
    container:
      # using the same image which is used by opensearch-build team to build the OpenSearch Distribution
      # this image tag is subject to change as more dependencies and updates will arrive over time
      image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
      # need to switch to root so that github actions can install runner binary on container without permission issues.
      options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

    steps:
      - name: Run start commands
        run: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-command }}

      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4

      - name: Setup Java ${{ matrix.java }}
        uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
        with:
          distribution: 'temurin'
          java-version: ${{ matrix.java }}
          cache: 'gradle'

      - name: Run build
        run: |
          chown -R 1000:1000 `pwd`
          su `id -un 1000` -c "./gradlew precommit -x integTest --parallel"

  integTest:
    needs: Precommit-neural-search-linux
    strategy:
      fail-fast: false
      matrix:
        java: [ 21, 25 ]
        os: [ ubuntu-latest ]
    name: Integ Test JDK${{ matrix.java }}, ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
      - name: Set up JDK ${{ matrix.java }}
        uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
        with:
          java-version: ${{ matrix.java }}
          cache: 'gradle'
          distribution: temurin
      - name: Build and Run Tests
        run: |
          ./gradlew integTest --parallel

  integMultiNodeTest:
    needs: Precommit-neural-search-linux
    strategy:
      fail-fast: false
      matrix:
        java: [ 21 ]
        os: [ ubuntu-latest ]
        configureNodeRoles: [ false, true ]
    name: Multi-Node Integ Test JDK${{ matrix.java }}, With Role Assignment ${{ matrix.configureNodeRoles }}
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
      - name: Set up JDK ${{ matrix.java }}
        uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
        with:
          java-version: ${{ matrix.java }}
          distribution: temurin
          cache: 'gradle'
      - name: Build and Run Tests
        run: |
          ./gradlew integTest --parallel -PnumNodes=3 -PconfigureNodeRoles=${{ matrix.configureNodeRoles }}

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit d0f19a4

@heemin32
heemin32 merged commit f4e975e into opensearch-project:main Aug 13, 2026
102 of 103 checks passed
@github-project-automation github-project-automation Bot moved this from 👀 In Review to ✅ Done in Engineering Effectiveness Board Aug 13, 2026
heemin32 pushed a commit that referenced this pull request Aug 14, 2026
…earch) (#1940) (#1945)

* Add Eclipse P2 mirror to avoid download.eclipse.org outages



* Fix indentation in CI.yml integMultiNodeTest job



---------


(cherry picked from commit f4e975e)

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com>
Co-authored-by: Peter Zhu <zhujiaxi@amazon.com>
@opensearch-ci-bot

Copy link
Copy Markdown
Contributor

The backport to 3.8 failed. Please backport manually. See failed workflow run: https://github.com/opensearch-project/neural-search/actions/runs/31832669174

@peterzhuamazon
peterzhuamazon deleted the eclipse-p2-mirror branch August 14, 2026 19:21
yizheliu-amazon added a commit to yizheliu-amazon/neural-search that referenced this pull request Aug 25, 2026
…2 mirror

The withP2Mirrors(... -> https://ci.opensearch.org/) mirror added in opensearch-project#1940
started timing out at project-configuration time:

  A problem occurred configuring root project 'neural-search'.
  > java.io.IOException: Failed to load eclipse jdt formatter:
        java.lang.RuntimeException: java.net.SocketTimeoutException: timeout

Because Spotless resolves the formatter during configuration, every Gradle
task (integTest, BWC, pre-commit, ...) fails on any runner that cannot reach
the mirror, independent of the change under test. Drop withP2Mirrors and let
the formatter resolve directly from download.eclipse.org, matching the
OpenSearch 3.x resolution documented in opensearch-project/OpenSearch#20826.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>
yizheliu-amazon added a commit to yizheliu-amazon/neural-search that referenced this pull request Aug 25, 2026
…rg P2 mirror

Same change as opensearch-project#1964, applied here so opensearch-project#1919 CI can run past the eclipse-jdt
SocketTimeoutException that the ci.opensearch.org P2 mirror (added in opensearch-project#1940)
is currently throwing at configuration time. Revert once opensearch-project#1964 lands on main.
See opensearch-project/OpenSearch#20826.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>
martin-gaievski pushed a commit that referenced this pull request Aug 25, 2026
…lve model id from cluster settings (#1919)

* Add model_selection parameter to semantic field to resolve model id from cluster settings (#1918)

Introduces a `model_selection` nested object (language_option + model_type) on the
semantic field. The model id is resolved from operator-configured cluster settings
`plugins.neural_search.model_selection.model_id.<model_type>.<language_option>`
(an affix setting, flexible for future dimensions). No model is auto-deployed and
there is no cache: the operator deploys the model and configures the setting.

- ModelSelection DTO validates language_option (ENGLISH|MULTILINGUAL) and
  model_type (SPARSE|DENSE).
- ClusterSettingSemanticModelResolver reads the configured model id, fails with a
  clear error when unset, and verifies the model exists and its type matches the
  requested model_type.
- SemanticMappingTransformer resolves model_selection fields; when both model_id and
  model_selection are provided they must resolve to the same model id, otherwise the
  request is rejected.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>

* [temporary] CI: resolve Eclipse JDT formatter without ci.opensearch.org P2 mirror

Same change as #1964, applied here so #1919 CI can run past the eclipse-jdt
SocketTimeoutException that the ci.opensearch.org P2 mirror (added in #1940)
is currently throwing at configuration time. Revert once #1964 lands on main.
See opensearch-project/OpenSearch#20826.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>

---------

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>
sharp-pixel pushed a commit to sharp-pixel/neural-search that referenced this pull request Aug 25, 2026
…lve model id from cluster settings (opensearch-project#1919)

* Add model_selection parameter to semantic field to resolve model id from cluster settings (opensearch-project#1918)

Introduces a `model_selection` nested object (language_option + model_type) on the
semantic field. The model id is resolved from operator-configured cluster settings
`plugins.neural_search.model_selection.model_id.<model_type>.<language_option>`
(an affix setting, flexible for future dimensions). No model is auto-deployed and
there is no cache: the operator deploys the model and configures the setting.

- ModelSelection DTO validates language_option (ENGLISH|MULTILINGUAL) and
  model_type (SPARSE|DENSE).
- ClusterSettingSemanticModelResolver reads the configured model id, fails with a
  clear error when unset, and verifies the model exists and its type matches the
  requested model_type.
- SemanticMappingTransformer resolves model_selection fields; when both model_id and
  model_selection are provided they must resolve to the same model id, otherwise the
  request is rejected.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>

* [temporary] CI: resolve Eclipse JDT formatter without ci.opensearch.org P2 mirror

Same change as opensearch-project#1964, applied here so opensearch-project#1919 CI can run past the eclipse-jdt
SocketTimeoutException that the ci.opensearch.org P2 mirror (added in opensearch-project#1940)
is currently throwing at configuration time. Revert once opensearch-project#1964 lands on main.
See opensearch-project/OpenSearch#20826.

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>

---------

Signed-off-by: Yizhe Liu <yizheliu@amazon.com>
peterzhuamazon added a commit that referenced this pull request Aug 26, 2026
…earch) (#1940) (#1958)

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 3.8 enhancement release skip-changelog skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis. v3.9.0

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants