Merge pull request #6 from pko89403/refactor/yagni-cleanup

YAGNI cleanup and internal module renames

Author: pko89403

README.ko.md

@@ -78,7 +78,6 @@ Strategy를 설정한 뒤 `AzureOpenAIReranker`에 전달합니다.
 from ranksmith import AzureOpenAIReranker, ListwiseStrategy
 
 strategy = ListwiseStrategy(
-    algorithm="rankgpt_sliding_window",
     window_size=20,
     stride=10,
     max_document_chars=4000,
@@ -116,7 +115,7 @@ reranker = AzureOpenAIReranker(
     api_key="...",
     azure_endpoint="https://example.openai.azure.com",
     azure_deployment="gpt-4o-mini",
-    strategy=TourRankStrategy(rounds=2, group_parallelism=1),
+    strategy=TourRankStrategy(rounds=2),
 )
 ```
 
@@ -145,7 +144,6 @@ reranker = AzureOpenAIReranker(
         target_rank=10,
         window_size=20,
         max_adaptive_reranker_calls=20,  # 선택적 adaptive phase budget cap.
-        batch_parallelism=2,  # 선택 사항. provider thread-safety가 불확실하면 1 유지.
     ),
 )
 ```
@@ -157,14 +155,14 @@ prior로 사용합니다. score가 전혀 없으면 standard TrueSkill prior를
 후보 수가 작을 때는 `target_rank`를 문서 수로 자동 제한합니다.
 `max_adaptive_reranker_calls`는 adaptive refinement phase만 제한하며, 선택적
 initial pass 호출은 결과 metadata에서 별도로 함께 집계됩니다.
-`batch_parallelism`은 같은 AcuRank iteration 안의 독립 batch를 병렬 호출하되,
-posterior update는 deterministic batch order로 적용합니다.
+`AsyncAcuRankStrategy`의 `batch_parallelism`은 같은 iteration 안의 독립 batch를
+동시에 호출하되, posterior update는 deterministic batch order로 적용합니다.
 
-> **참고**: `strategy`를 명시하지 않으면 기본적으로 `ListwiseStrategy(algorithm="rankgpt_sliding_window")`가 자동으로 적용됩니다. Pairwise PRP, Setwise, TourRank-r, AcuRank는 기본 listwise보다 LLM 호출 수가 많을 수 있으므로 live benchmark 전 호출 수를 확인해야 합니다.
+> **참고**: `strategy`를 명시하지 않으면 기본적으로 `ListwiseStrategy()`(RankGPT sliding window)가 자동으로 적용됩니다. Pairwise PRP, Setwise, TourRank-r, AcuRank는 기본 listwise보다 LLM 호출 수가 많을 수 있으므로 live benchmark 전 호출 수를 확인해야 합니다.
 
 ## 커스텀 Strategy
 
-커스텀 reranking 메소드는 `ListwiseStrategy.algorithm`에 새 문자열 값을 추가하는
+커스텀 reranking 메소드는 내장 Strategy 클래스를 수정하는
 방식보다, 새 Strategy 클래스로 구현하는 방식을 권장합니다. Strategy는 정규화된
 `Document` 목록, model client, 선택적 `top_k`를 받아 `RerankResult` 목록을 반환합니다.
 
@@ -253,9 +251,6 @@ reranker = AzureOpenAIReranker(
 )
 ```
 
-`OpenAIProvider`, `AnthropicProvider`, `GeminiProvider`는 향후 SDK 구현을 위한
-public stub입니다. 호출하면 `RerankProviderError`로 fast fail 합니다.
-
 ## 비동기 지원 (Async Support)
 
 대규모 트래픽이나 FastAPI 같은 비동기 웹 프레임워크를 위해 async reranker를

README.md

@@ -80,7 +80,6 @@ Configure a strategy and pass it to `AzureOpenAIReranker`.
 from ranksmith import AzureOpenAIReranker, ListwiseStrategy
 
 strategy = ListwiseStrategy(
-    algorithm="rankgpt_sliding_window",
     window_size=20,
     stride=10,
     max_document_chars=4000,
@@ -118,7 +117,7 @@ reranker = AzureOpenAIReranker(
     api_key="...",
     azure_endpoint="https://example.openai.azure.com",
     azure_deployment="gpt-4o-mini",
-    strategy=TourRankStrategy(rounds=2, group_parallelism=1),
+    strategy=TourRankStrategy(rounds=2),
 )
 ```
 
@@ -147,7 +146,6 @@ reranker = AzureOpenAIReranker(
         target_rank=10,
         window_size=20,
         max_adaptive_reranker_calls=20,  # Optional adaptive-phase budget cap.
-        batch_parallelism=2,  # Optional; keep 1 if your provider is not thread-safe.
     ),
 )
 ```
@@ -158,17 +156,17 @@ TrueSkill prior. Partial score metadata and boolean score values fail fast.
 
 For small candidate sets, `target_rank` is clipped to the number of documents.
 `max_adaptive_reranker_calls` limits only the adaptive refinement phase; the
-optional initial pass is counted separately in result metadata.
-`batch_parallelism` parallelizes independent batches within the same AcuRank
-iteration, while posterior updates are still applied in deterministic batch
-order.
+optional initial pass is counted separately in result metadata. On
+`AsyncAcuRankStrategy`, `batch_parallelism` runs independent batches within the
+same iteration concurrently, while posterior updates are still applied in
+deterministic batch order.
 
-> **Note**: If `strategy` is not provided, it defaults to `ListwiseStrategy(algorithm="rankgpt_sliding_window")`. Pairwise PRP, Setwise, TourRank-r, and AcuRank can use more LLM calls than basic listwise reranking, so check call estimates before live benchmarks.
+> **Note**: If `strategy` is not provided, it defaults to `ListwiseStrategy()` (RankGPT sliding window). Pairwise PRP, Setwise, TourRank-r, and AcuRank can use more LLM calls than basic listwise reranking, so check call estimates before live benchmarks.
 
 ## Custom Strategies
 
 Custom reranking methods should be implemented as new strategy classes instead
-of adding new string values to `ListwiseStrategy.algorithm`. A strategy receives
+of patching the built-in strategy classes. A strategy receives
 the normalized `Document` objects, a model client, and optional `top_k`, then
 returns `RerankResult` objects.
 
@@ -257,10 +255,6 @@ reranker = AzureOpenAIReranker(
 )
 ```
 
-`OpenAIProvider`, `AnthropicProvider`, and `GeminiProvider` are reserved public
-stubs for future SDK-backed implementations. Calling them fails fast with
-`RerankProviderError`.
-
 ## Async Support
 
 `ranksmith` provides first-class asynchronous support for high-throughput
@@ -368,9 +362,8 @@ Runnable examples live in the `examples/` directory.
 This repo ships a [Claude Code](https://code.claude.com/docs) plugin,
 `ranksmith-advisor`, that helps you choose a reranking strategy for your use
 case and returns working, CI-verified snippets. It encodes ranksmith-specific
-guardrails, so the suggested code follows the library's real contracts (no
-calls into unimplemented providers, no `algorithm` string hacks, no treating
-`confidence` as a reranker).
+guardrails, so the suggested code follows the library's real contracts (Azure
+is the only bundled provider, and `confidence` is not a reranker).
 
 Use it from Claude Code:
 

benchmarks/__init__.py

@@ -8,7 +8,7 @@
 from pathlib import Path
 from typing import Literal, cast
 
-from ranksmith._metrics import mrr_at_k, ndcg_at_k, recall_at_k
+from benchmarks.metrics import mrr_at_k, ndcg_at_k, recall_at_k
 
 SCHEMA_VERSION = 1
 CandidateStrategy = Literal["candidate_file", "oracle_plus_random"]
@@ -81,7 +81,6 @@ def load_beir_cases(
     seed: int = 13,
     dataset_name: str = "BEIR/SciFact",
     fixture_prefix: str = "beir-scifact",
-    license_text: str = "See upstream dataset license metadata.",
 ) -> list[BenchmarkCase]:
     _validate_positive("candidate_count", candidate_count)
     if max_cases is not None:
@@ -132,7 +131,7 @@ def load_beir_cases(
                 fixture_id=f"{fixture_prefix}-{split}-{query_id}",
                 dataset=f"{dataset_name} {split}",
                 source=f"cache:{cache_dir}",
-                license=license_text,
+                license="See upstream dataset license metadata.",
                 query_id=query_id,
                 query=queries[query_id],
                 documents=documents,

src/ranksmith/_benchmark.py benchmarks/benchmark.pysrc/ranksmith/_benchmark.py renamed to benchmarks/benchmark.py

@@ -1,14 +1,13 @@
 from __future__ import annotations
 
 import hashlib
-import json
 import math
-from collections.abc import Mapping, Sequence
+import statistics
+from collections.abc import Sequence
 from dataclasses import dataclass
-from pathlib import Path
-from typing import Any, Literal, TypeVar
+from typing import Literal, TypeVar
 
-from ranksmith._metrics import map_score, mrr_at_k, ndcg_at_k, recall_at_k
+from benchmarks.metrics import map_score, mrr_at_k, ndcg_at_k, recall_at_k
 from ranksmith.strategies import TourRankStageConfig
 
 T = TypeVar("T")
@@ -30,13 +29,6 @@ class MtebRerankingSample:
     candidates: tuple[MtebRerankingCandidate, ...]
 
 
-@dataclass(frozen=True)
-class ParsedRanking:
-    ranking: tuple[int, ...]
-    valid: bool
-    failure_type: str | None
-
-
 @dataclass(frozen=True)
 class PriceConfig:
     input_token_price_per_1m: float
@@ -141,68 +133,6 @@ def _parse_tourrank_method(method: str) -> MethodConfig:
     )
 
 
-def parse_ranking_with_failure_type(
-    raw_response: str,
-    expected_count: int,
-) -> ParsedRanking:
-    try:
-        data = json.loads(raw_response)
-    except json.JSONDecodeError:
-        return ParsedRanking((), False, "json_parse_failure")
-    if not isinstance(data, dict) or "ranking" not in data:
-        return ParsedRanking((), False, "missing_ranking")
-    ranking = data["ranking"]
-    if not isinstance(ranking, list):
-        return ParsedRanking((), False, "missing_ranking")
-    if not all(type(item) is int for item in ranking):
-        return ParsedRanking((), False, "non_integer_rank")
-    parsed = tuple(ranking)
-    if len(parsed) != expected_count:
-        return ParsedRanking(parsed, False, "length_mismatch")
-    if any(rank < 1 or rank > expected_count for rank in parsed):
-        return ParsedRanking(parsed, False, "out_of_range_rank")
-    if len(set(parsed)) != len(parsed):
-        return ParsedRanking(parsed, False, "duplicate_rank")
-    return ParsedRanking(parsed, True, None)
-
-
-def apply_integer_permutation(
-    items: Sequence[T],
-    ranking: Sequence[int],
-) -> tuple[T, ...]:
-    return tuple(items[index - 1] for index in ranking)
-
-
-def rankgpt_window_ranges(
-    *,
-    document_count: int,
-    rank_start: int,
-    rank_end: int,
-    window_size: int,
-    step: int,
-) -> tuple[tuple[int, int], ...]:
-    if document_count < 1:
-        return ()
-    if rank_start < 0 or rank_end < 1 or window_size < 1 or step < 1:
-        raise ValueError("rank_start, rank_end, window_size, and step are invalid")
-    if step > window_size:
-        raise ValueError("step must be less than or equal to window_size")
-    if rank_end <= rank_start:
-        raise ValueError("rank_end must be greater than rank_start")
-
-    effective_end = min(rank_end, document_count)
-    ranges: list[tuple[int, int]] = []
-    end = effective_end
-    max_iterations = (effective_end - rank_start) // step + 2
-    for _ in range(max_iterations):
-        start = max(rank_start, end - window_size)
-        ranges.append((start, end))
-        if start == rank_start:
-            return tuple(ranges)
-        end -= step
-    raise RuntimeError("rankgpt_window_ranges exceeded iteration cap")
-
-
 def compute_query_metrics(
     *,
     sample: MtebRerankingSample,
@@ -225,9 +155,7 @@ def compute_query_metrics(
 
 
 def mean(values: Sequence[float]) -> float:
-    if not values:
-        return 0.0
-    return sum(values) / len(values)
+    return statistics.fmean(values) if values else 0.0
 
 
 def percentile(values: Sequence[float], percentile_value: float) -> float:
@@ -259,32 +187,6 @@ def estimate_cost(
     )
 
 
-def write_jsonl(path: Path, rows: Sequence[Mapping[str, Any]]) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    with path.open("w", encoding="utf-8") as handle:
-        for row in rows:
-            handle.write(json.dumps(dict(row), sort_keys=True) + "\n")
-
-
-def completed_result_keys(path: Path) -> set[tuple[str, str, str, str]]:
-    if not path.exists():
-        return set()
-    completed: set[tuple[str, str, str, str]] = set()
-    for line in path.read_text(encoding="utf-8").splitlines():
-        if line.strip() == "":
-            continue
-        row = json.loads(line)
-        completed.add(
-            (
-                str(row["task"]),
-                str(row["split"]),
-                str(row["query_id"]),
-                str(row["method"]),
-            )
-        )
-    return completed
-
-
 def tourrank_stage_configs_for_candidate_count(
     candidate_count: int,
 ) -> tuple[TourRankStageConfig, ...]:

src/ranksmith/_metrics.py benchmarks/metrics.pysrc/ranksmith/_metrics.py renamed to benchmarks/metrics.py

@@ -33,12 +33,6 @@ LLM 기반 reranking을 위한 작고 신뢰성 있는 Python 패키지를 만
 - `AsyncAzureOpenAIReranker`
 - `AzureAOAIProvider`
 - `AsyncAzureAOAIProvider`
-- `OpenAIProvider`
-- `AsyncOpenAIProvider`
-- `AnthropicProvider`
-- `AsyncAnthropicProvider`
-- `GeminiProvider`
-- `AsyncGeminiProvider`
 - `ModelClient`
 - `AsyncModelClient`
 - `ModelProvider`
@@ -92,7 +86,7 @@ LLM 기반 reranking을 위한 작고 신뢰성 있는 Python 패키지를 만
 - `tolerance`: `0.01`
 - `uncertain_threshold`: `10`
 - `initial_pass`: `True`
-- `score_metadata_key`: `score`
+- score prior metadata key: `score` (고정)
 
 ## Codex 읽기 순서
 1. `docs/wiki/00_context.md`

src/ranksmith/_mteb_eval.py benchmarks/mteb_eval.pysrc/ranksmith/_mteb_eval.py renamed to benchmarks/mteb_eval.py

@@ -20,25 +20,23 @@ src/ranksmith/
   parsing.py               # LLM response contract parser
   strategies/
     __init__.py            # public strategy exports
-    _common.py             # shared validation/capability guards
-    _listwise.py
-    _pairwise.py
-    _setwise.py
-    _tourrank.py
-    _acurank.py
+    common.py              # shared validation/capability guards
+    listwise.py
+    pairwise.py
+    setwise.py
+    tourrank.py
+    acurank.py
   providers/
     __init__.py            # public provider exports
-    _azure.py              # Azure OpenAI implementation
-    _stubs.py              # unimplemented provider stubs
-  _providers.py            # backward-compatible re-export layer
+    azure.py               # Azure OpenAI implementation
 ```
 
 외부 사용자는 root import 또는 `ranksmith.strategies`, `ranksmith.providers`의 public export를 사용한다.
-`strategies/_*.py`, `providers/_*.py`는 내부 구현 모듈로 취급한다.
+strategies/, providers/ 하위 개별 모듈은 내부 구현으로 취급한다.
 
 ## ModelProvider
 실제 SDK 호출은 Azure OpenAI만 구현한다.
-OpenAI, Anthropic, Gemini provider는 향후 구현을 위한 public stub이며 호출 시 fast fail 한다.
+다른 vendor는 사용자 정의 `ModelProvider` 구현으로 연결한다.
 
 Provider는 `ModelRequest`를 받아 `ModelResponse`를 반환한다.
 Provider는 ranking 도메인 prompt의 의미를 알지 않는다.

docs/wiki/00_context.md

@@ -84,7 +84,6 @@ def main() -> None:
             target_rank=3,
             window_size=3,
             max_adaptive_reranker_calls=1,
-            batch_parallelism=1,
         ),
     )
 

docs/wiki/02_architecture.md

@@ -48,7 +48,6 @@ async def async_main() -> None:
         azure_deployment=deployment,
         api_version="2024-10-21",  # 필요한 API 버전에 맞게 수정
         strategy=AsyncListwiseStrategy(
-            algorithm="rankgpt_sliding_window",
             window_size=10,
             stride=5,
         ),