Support Codegrep models through System One

README.md

@@ -240,6 +240,78 @@ Use `await client.factcheck(...)` with `AsyncNimble`. Both clients accept
 `model` is not an argument to this method. The legacy
 `client.minicheck.factcheck.create(claim=..., context=...)` is unchanged.
 
+## Code relevance scoring
+
+Use `system_one` with a Codegrep model name and your existing `state` and named
+`questions`. The model selects the effort, validation limits, and billing rate.
+
+```python
+from bespokelabs import nimble
+
+with nimble.Nimble() as client:
+    result = client.system_one(
+        state={
+            "query": "Find where failed uploads are retried",
+            "items": [
+                {
+                    "id": "n0",
+                    "path": "upload.py",
+                    "kind": "file",
+                    "filePreview": {"text": "def retry_upload(): ..."},
+                }
+            ],
+        },
+        questions={
+            "q0": {
+                "type": "noul",
+                "instructions": "Is this file useful for answering the query?",
+            }
+        },
+        model="nimble-codegrep-medium",
+    )
+    print(result.nouls["q0"].noul)
+    print(result.details["q0"].overflow)
+    print(result.escalation_skipped)
+```
+
+`state` is a string or JSON object, with at most 2 million characters under the
+service's state-size check. `questions` contains 1–128 `noul` or `boolean`
+questions. Instructions may be text, an object, or a list; optional `criteria`
+describe `true` and `false`. The complete JSON request must fit within 1 MiB.
+
+For a state containing multiple items, question `q0` addresses item `n0`, `q1`
+addresses `n1`, and so on. With a single item, any question ID is accepted. A state
+without items is used whole for every question. The SDK does not generate IDs
+or questions.
+
+- `nimble-codegrep-low` scores each question with the small model.
+- `nimble-codegrep-medium` rechecks uncertain questions with the larger model.
+- `nimble-codegrep-high` uses the larger model for every question.
+
+Answers are calibrated relevance scores: values above 0.5 mean keep the item at
+the model's calibrated operating point. They are not probabilities of factual
+truth. Each question has typed `details` containing `raw`, `overflow`, `escalated`,
+and per-model `scores`. If an item exceeds the model's context limit, its answer
+is 1.0 and `overflow=True`, with `raw=None`: it was kept without being scored.
+
+Medium may return a successful response with `escalation_skipped=True` when the
+larger model is unavailable. The smaller model's answers remain available;
+callers can inspect the flag before relying on the result.
+
+Input rates are $0.02 / $0.04 / $0.12 per million tokens for low / medium / high.
+Input usage counts the actual per-item prompts, including extra model calls at
+medium effort. Identical prompts can be deduplicated by the service. Output usage
+is reported but carries no charge. A medium fallback is charged for its returned
+input usage at the medium rate. Failed requests have no usage charge.
+
+Codegrep returns `SystemOneResponse` with typed metadata; `CodegrepResponse` is an
+alias for it. General System One responses have empty `details` and
+`escalation_skipped=False` when those fields are absent.
+
+Use `system_one` with sync or async clients, request options, and existing API
+keys and retry settings. Raw and streaming response wrappers also support
+Codegrep models.
+
 ## Async usage
 
 Simply import `AsyncNimble` instead of `Nimble` and use `await` with each API call:

api.md

@@ -53,3 +53,20 @@ billable input usage, support decisions, and available escalation metadata.
 
 `BespokeLabs` / `AsyncBespokeLabs` and `Client` / `AsyncClient` are compatibility
 aliases. The nested `client.nimble.system_one` interface remains available.
+
+## Codegrep models
+
+```python
+from bespokelabs.nimble.types import CodegrepQuestion, CodegrepDetail, CodegrepResponse
+```
+
+- `client.system_one(state=..., questions={...}, model="nimble-codegrep-medium") -> SystemOneResponse`
+- `await async_client.system_one(state=..., questions={...}, model="nimble-codegrep-medium") -> SystemOneResponse`
+
+Use `nimble-codegrep-low`, `nimble-codegrep-medium`, or `nimble-codegrep-high`
+through `POST /v1/nimble/systemone`. Codegrep supports 1–128 `noul`/`boolean`
+questions and string/object state, within the shared 1 MiB request-body limit.
+The response preserves question IDs in `answers` and typed `details`, with usage
+and an `escalation_skipped` flag for medium fallback. Overflow answers are 1.0
+with `raw=None` and `overflow=True`. Normal, raw, and streaming response access
+works with both synchronous and asynchronous clients.

src/bespokelabs/nimble/_client.py

@@ -124,14 +124,14 @@ def system_one(
         self,
         *,
         state: Content,
-        questions: dict[str, Question],
+        questions: Mapping[str, Question],
         model: str = "nimble-latest",
         extra_headers: Headers | None = None,
         extra_query: Query | None = None,
         extra_body: Body | None = None,
         timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,
     ) -> SystemOneResponse:
-        """Evaluate up to 64 typed questions against shared state in one request."""
+        """Evaluate typed questions; model selects the service and its limits."""
         return self.nimble.system_one(
             state=state,
             questions=questions,
@@ -340,14 +340,14 @@ async def system_one(
         self,
         *,
         state: Content,
-        questions: dict[str, Question],
+        questions: Mapping[str, Question],
         model: str = "nimble-latest",
         extra_headers: Headers | None = None,
         extra_query: Query | None = None,
         extra_body: Body | None = None,
         timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,
     ) -> SystemOneResponse:
-        """Evaluate up to 64 typed questions against shared state in one request."""
+        """Evaluate typed questions; model selects the service and its limits."""
         return await self.nimble.system_one(
             state=state,
             questions=questions,

src/bespokelabs/nimble/_codegrep.py

@@ -0,0 +1,87 @@
+"""Keep Codegrep's explicit state/question contract and validate its responses."""
+
+from __future__ import annotations
+
+import copy
+from typing import Any
+
+from ._utils import is_list, is_mapping
+from ._models import validate_type
+from ._factcheck import _count, _probability
+from .types.codegrep import CodegrepResponse
+from .types.factcheck_response import Effort
+
+CODEGREP_MODELS = {f"nimble-codegrep-{effort}": effort for effort in ("low", "medium", "high")}
+
+
+def prepare_codegrep(*, state: object, questions: object, effort: Effort) -> dict[str, Any]:
+    if effort not in ("low", "medium", "high"):
+        raise ValueError("effort must be 'low', 'medium', or 'high'")
+    if not isinstance(state, str) and not is_mapping(state):
+        raise ValueError("state must be a string or object")
+    if isinstance(state, str) and not state.strip():
+        raise ValueError("state must not be blank")
+    if len(state if isinstance(state, str) else str(state)) > 2_000_000:
+        raise ValueError("state must not exceed 2000000 characters")
+    if not is_mapping(questions) or not 1 <= len(questions) <= 128:
+        raise ValueError("questions must contain 1 to 128 named questions")
+    for qid, question in questions.items():
+        if not isinstance(qid, str) or not is_mapping(question):  # pyright: ignore[reportUnnecessaryIsInstance]
+            raise ValueError("questions must map string IDs to question objects")
+        if set(question) - {"type", "instructions", "criteria"}:
+            raise ValueError("unknown question field")
+        if question.get("type") not in ("noul", "boolean"):
+            raise ValueError("Codegrep supports only noul and boolean questions")
+        instructions = question.get("instructions")
+        if not (isinstance(instructions, str) or is_mapping(instructions) or is_list(instructions)):
+            raise ValueError("instructions must be a string, object, or list")
+        if isinstance(instructions, str) and not instructions.strip():
+            raise ValueError("instructions must not be blank")
+        criteria = question.get("criteria")
+        if criteria is not None and (
+            not is_mapping(criteria)
+            or set(criteria) - {"true", "false"}
+            or any(not isinstance(v, str) for v in criteria.values())
+        ):
+            raise ValueError("criteria must map true and false to strings")
+    # Preserve caller IDs and snapshot nested data before an async request starts.
+    return copy.deepcopy({"state": state, "questions": dict(questions), "effort": effort})
+
+
+def parse_codegrep(response: object, *, question_ids: list[str]) -> CodegrepResponse:
+    if not is_mapping(response):
+        raise ValueError("Invalid Codegrep response")
+    answers, details = response.get("answers"), response.get("details")
+    if (
+        not is_mapping(answers)
+        or not is_mapping(details)
+        or set(answers) != set(question_ids)
+        or set(details) != set(question_ids)
+    ):
+        raise ValueError("Invalid Codegrep response: question IDs do not match")
+    for qid in question_ids:
+        answer, detail = answers[qid], details[qid]
+        if not is_mapping(answer) or answer.get("type") != "noul" or not _probability(answer.get("noul")):
+            raise ValueError("Invalid Codegrep response: invalid answer")
+        if not is_mapping(detail) or any(type(detail.get(k)) is not bool for k in ("overflow", "escalated")):
+            raise ValueError("Invalid Codegrep response: invalid details")
+        raw = detail.get("raw")
+        if "raw" not in detail or (raw is not None and not _probability(raw)):
+            raise ValueError("Invalid Codegrep response: invalid raw score")
+        if detail["overflow"]:
+            if raw is not None or answer["noul"] != 1.0:
+                raise ValueError("Invalid Codegrep response: invalid overflow answer")
+        elif raw is None:
+            raise ValueError("Invalid Codegrep response: missing raw score")
+        scores = detail.get("scores")
+        if not is_mapping(scores) or any(v is not None and not _probability(v) for v in scores.values()):
+            raise ValueError("Invalid Codegrep response: invalid model scores")
+    usage = response.get("usage")
+    if not is_mapping(usage) or any(not _count(usage.get(k)) for k in ("input_tokens", "output_tokens")):
+        raise ValueError("Invalid Codegrep response: invalid usage")
+    if type(response.get("escalation_skipped")) is not bool:
+        raise ValueError("Invalid Codegrep response: missing fallback status")
+    try:
+        return validate_type(type_=CodegrepResponse, value=response)
+    except ValueError as error:
+        raise ValueError("Invalid Codegrep response: invalid metadata") from error

src/bespokelabs/nimble/resources/nimble.py

@@ -2,14 +2,19 @@
 
 from __future__ import annotations
 
+from typing import Mapping, cast
+from functools import partial
+
 import httpx
 
 from .._types import NOT_GIVEN, Body, Query, Headers, NotGiven
 from .._utils import (
+    is_mapping,
     maybe_transform,
     async_maybe_transform,
 )
 from .._compat import cached_property
+from .._codegrep import CODEGREP_MODELS, parse_codegrep, prepare_codegrep
 from .._resource import SyncAPIResource, AsyncAPIResource
 from .._response import (
     to_raw_response_wrapper,
@@ -19,6 +24,7 @@
 )
 from .._base_client import make_request_options
 from ..types.nimble import system_one_params
+from ..types.factcheck_response import Effort
 from ..types.nimble.system_one_params import Content, Question
 from ..types.nimble.system_one_response import SystemOneResponse
 
@@ -49,7 +55,7 @@ def system_one(
         self,
         *,
         state: Content,
-        questions: dict[str, Question],
+        questions: Mapping[str, Question],
         model: str = "nimble-latest",
         # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs.
         # The extra values given here take precedence over values defined on the client or passed to this method.
@@ -64,9 +70,9 @@ def system_one(
         Args:
           state: Text, structured JSON, or a text transcript to evaluate.
 
-          questions: Named Noul, Choice, or Score question dictionaries.
+          questions: Named question dictionaries. Codegrep supports noul/boolean questions.
 
-          model: Hosted model name or alias.
+          model: Hosted model name or alias, including nimble-codegrep-low/medium/high.
 
           extra_headers: Send extra headers
 
@@ -76,6 +82,33 @@ def system_one(
 
           timeout: Override the client-level default timeout for this request, in seconds
         """
+        overrides: Mapping[str, object] = extra_body if is_mapping(extra_body) else {}
+        selected_model = overrides.get("model", model)
+        if isinstance(selected_model, str) and selected_model.startswith("nimble-codegrep"):
+            if selected_model not in CODEGREP_MODELS:
+                raise ValueError("Unknown Codegrep model; use nimble-codegrep-low, -medium, or -high")
+            body = prepare_codegrep(
+                state=overrides.get("state", state),
+                questions=overrides.get("questions", questions),
+                effort=cast(Effort, CODEGREP_MODELS[selected_model]),
+            )
+            body.pop("effort")
+            body["model"] = selected_model
+            return cast(
+                SystemOneResponse,
+                self._post(
+                    "/v1/nimble/systemone",
+                    body=body,
+                    cast_to=object,
+                    options=make_request_options(
+                        extra_headers=extra_headers,
+                        extra_query=extra_query,
+                        extra_body=extra_body,
+                        timeout=timeout,
+                        post_parser=partial(parse_codegrep, question_ids=list(body["questions"])),
+                    ),
+                ),
+            )
         return self._post(
             "/v1/nimble/systemone",
             body=maybe_transform(
@@ -117,7 +150,7 @@ async def system_one(
         self,
         *,
         state: Content,
-        questions: dict[str, Question],
+        questions: Mapping[str, Question],
         model: str = "nimble-latest",
         # Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs.
         # The extra values given here take precedence over values defined on the client or passed to this method.
@@ -132,9 +165,9 @@ async def system_one(
         Args:
           state: Text, structured JSON, or a text transcript to evaluate.
 
-          questions: Named Noul, Choice, or Score question dictionaries.
+          questions: Named question dictionaries. Codegrep supports noul/boolean questions.
 
-          model: Hosted model name or alias.
+          model: Hosted model name or alias, including nimble-codegrep-low/medium/high.
 
           extra_headers: Send extra headers
 
@@ -144,6 +177,33 @@ async def system_one(
 
           timeout: Override the client-level default timeout for this request, in seconds
         """
+        overrides: Mapping[str, object] = extra_body if is_mapping(extra_body) else {}
+        selected_model = overrides.get("model", model)
+        if isinstance(selected_model, str) and selected_model.startswith("nimble-codegrep"):
+            if selected_model not in CODEGREP_MODELS:
+                raise ValueError("Unknown Codegrep model; use nimble-codegrep-low, -medium, or -high")
+            body = prepare_codegrep(
+                state=overrides.get("state", state),
+                questions=overrides.get("questions", questions),
+                effort=cast(Effort, CODEGREP_MODELS[selected_model]),
+            )
+            body.pop("effort")
+            body["model"] = selected_model
+            return cast(
+                SystemOneResponse,
+                await self._post(
+                    "/v1/nimble/systemone",
+                    body=body,
+                    cast_to=object,
+                    options=make_request_options(
+                        extra_headers=extra_headers,
+                        extra_query=extra_query,
+                        extra_body=extra_body,
+                        timeout=timeout,
+                        post_parser=partial(parse_codegrep, question_ids=list(body["questions"])),
+                    ),
+                ),
+            )
         return await self._post(
             "/v1/nimble/systemone",
             body=await async_maybe_transform(

src/bespokelabs/nimble/types/__init__.py

@@ -2,6 +2,12 @@
 
 from __future__ import annotations
 
+from .codegrep import (
+    CodegrepState as CodegrepState,
+    CodegrepDetail as CodegrepDetail,
+    CodegrepQuestion as CodegrepQuestion,
+    CodegrepResponse as CodegrepResponse,
+)
 from .factcheck_response import (
     Effort as Effort,
     FactcheckResult as FactcheckResult,

src/bespokelabs/nimble/types/codegrep.py

@@ -0,0 +1,18 @@
+"""Parameters and results for code relevance scoring."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Union
+
+from .nimble.system_one_params import NoulQuestion, BooleanQuestion
+from .nimble.system_one_response import (
+    CodegrepDetail as CodegrepDetail,
+    SystemOneResponse,
+)
+
+CodegrepState = Union[str, Dict[str, Any]]
+
+
+CodegrepQuestion = Union[NoulQuestion, BooleanQuestion]
+
+CodegrepResponse = SystemOneResponse

src/bespokelabs/nimble/types/nimble/__init__.py

@@ -4,6 +4,7 @@
     NoulQuestion as NoulQuestion,
     ScoreQuestion as ScoreQuestion,
     ChoiceQuestion as ChoiceQuestion,
+    BooleanQuestion as BooleanQuestion,
     SystemOneParams as SystemOneParams,
 )
 from .system_one_response import (