protocol update

Author: TYLERSFOSTER

assets/protocol_assets/system/graph_population/v001/graph_population_control_protocol.md

@@ -2626,6 +2626,14 @@ ask for that approval and continue after approval. Do not convert absent tool
 approval into a graph-population blocker. If approval is denied, stop with a
 source-approval failure that names the denied capability.
 
+For ordinary `MAKE-GRAPH`, source-backed means backed by admissible persisted
+source evidence, not by a single privileged adapter. A generated bundle must
+not treat one public source family outage, throttle, timeout, malformed
+response, or shallow result as global source exhaustion while admissible source
+adapter candidates remain. Unless the human explicitly restricted sources,
+source adapter expansion is a normal Markdown recovery loop and does not
+require human approval merely because the next adapter is different.
+
 If the manifest explicitly says `no_live_lookup`, `bundle_only`,
 `scaffold_only`, `smoke_only`, or `no_population`, and no source batches are
 provided, source-backed population cannot proceed and the executor must stop
@@ -2645,6 +2653,10 @@ Before source lookup, identify:
 - disallowed sources;
 - source adapter ID;
 - fallback source adapter order;
+- source adapter expansion policy;
+- source adapter registry path;
+- source adapter candidate frontier path;
+- same-source-family exhaustion rule;
 - timeout, retry, and backoff policy;
 - batch size and batch runtime budget;
 - repo-local source batch cache path;
@@ -2656,10 +2668,15 @@ Before source lookup, identify:
 If any source boundary is missing, do not crawl. Stop with
 `missing_source_boundary`.
 
-If any source adapter, timeout, retry, fallback, or batch policy required by
-the loop is missing, do not crawl. Stop with
+If any source adapter, timeout, retry, fallback, adapter-expansion, or batch
+policy required by the loop is missing, do not crawl. Stop with
 `missing_source_execution_policy`.
 
+If an ordinary `MAKE-GRAPH` loop's fallback order contains only variants of one
+source family and lacks a source-adapter expansion policy/frontier, stop with
+`incomplete_source_adapter_expansion_policy` before treating that source family
+as exhaustive.
+
 If the loop has no repo-local source batch cache path, do not crawl. Stop with
 `missing_source_cache_path`.
 
@@ -2694,9 +2711,10 @@ When trouble occurs:
 
 ```text
 timeout -> retry_if_budget_remains -> fallback_if_declared -> recovery_child_loop_if_declared -> recovery_exhausted_stop
-rate_limited -> backoff_if_declared -> fallback_if_declared -> recovery_child_loop_if_declared -> recovery_exhausted_stop
-malformed_response -> retry_if_budget_remains -> fallback_if_declared -> recovery_child_loop_if_declared -> recovery_exhausted_stop
-insufficient_results -> persist_partial_success -> update_frontier -> source_strategy_recovery_child_loop_if_budget_remains -> recovery_exhausted_stop
+rate_limited -> backoff_if_declared -> fallback_if_declared -> source_adapter_expansion_child_loop_if_same_family_exhausted -> recovery_exhausted_stop
+malformed_response -> retry_if_budget_remains -> fallback_if_declared -> source_adapter_expansion_child_loop_if_same_family_exhausted -> recovery_exhausted_stop
+source_adapter_unavailable -> checkpoint -> source_adapter_expansion_child_loop_if_candidates_remain -> recovery_exhausted_stop
+insufficient_results -> persist_partial_success -> update_frontier -> source_strategy_or_adapter_expansion_child_loop_if_budget_remains -> recovery_exhausted_stop
 insufficient_depth_for_declared_fields -> persist_partial_success -> field_recovery_child_loop_if_budget_remains -> field_recovery_exhausted_stop
 insufficient_relation_family_diversity -> relation_recovery_child_loop_if_budget_remains -> relation_discovery_recovery_exhausted_stop
 insufficient_pair_evidence_for_edges -> edge_evidence_recovery_child_loop_if_budget_remains -> edge_evidence_recovery_exhausted_stop
@@ -2715,17 +2733,26 @@ pair-specific evidence are satisfied.
 
 If source lookup cannot yet support declared type fields, declared relation
 fields, source-backed instances, or pair-specific edge evidence, the executor
-must run the declared recovery ladder before stopping. Only after recovery is
-exhausted may it stop with the precise limitation. Do not fill target counts
-with deterministic local completion, placeholder values, source-adapter-only
-identity rows, endpoint pairings, or scaffold records unless the human
-explicitly requested scaffold or smoke mode.
+must run the declared recovery ladder before stopping. For ordinary
+`MAKE-GRAPH`, that ladder includes source-adapter expansion unless the human
+explicitly restricted sources or the adapter frontier is exhausted. Only after
+recovery is exhausted may it stop with the precise limitation. Do not fill
+target counts with deterministic local completion, placeholder values,
+source-adapter-only identity rows, endpoint pairings, or scaffold records unless
+the human explicitly requested scaffold or smoke mode.
 
 Failure and event codes:
 
 ```text
 source_batch_timeout
 source_rate_limited
+source_adapter_unavailable
+source_adapter_expansion_required
+source_adapter_expansion_started
+source_adapter_candidate_accepted
+source_adapter_candidate_rejected
+source_adapter_expansion_exhausted
+source_scope_user_restricted
 source_malformed_response
 source_fallback_used
 source_partial_success
@@ -2734,8 +2761,42 @@ source_depth_limited
 field_richness_limited
 edge_evidence_limited
 missing_source_execution_policy
+incomplete_source_adapter_expansion_policy
+```
+
+### 15.2 Source Adapter Expansion Recovery
+
+For ordinary `MAKE-GRAPH`, a source adapter is a recoverable execution choice,
+not the definition of the domain. If one adapter family fails, the executor
+must ask whether admissible adapter candidates remain before marking the run
+blocked.
+
+The next legal action after same-family source failure is:
+
+```text
+1. checkpoint the failing batch;
+2. persist any partial source cache and Markdown report;
+3. update cursor.active_recovery with source_adapter_expansion_required;
+4. create or update runs/<run_id>/source_adapter_candidate_frontier.md;
+5. create or update a Markdown batch packet for adapter evaluation;
+6. evaluate the next admissible adapter candidate;
+7. persist its probe result under runs/<run_id>/source_batches/;
+8. resume the parent source loop if the adapter satisfies the evidence shape.
 ```
 
+A new adapter candidate may be accepted without Project Owner approval when it
+is public, legal to access under ordinary live lookup, inside the user-provided
+or generated source scope, and can be persisted/reported through the run
+artifact tree. Human approval is required only for credentialed, paid,
+private/personal, terms-of-service-sensitive, or explicitly out-of-scope
+sources.
+
+The executor may stop with `source_strategy_recovery_exhausted` or
+`source_depth_limited` only when the adapter frontier has been exhausted,
+source scope was explicitly restricted by the human, or every remaining adapter
+candidate requires approval that has been denied or cannot be requested in the
+current run.
+
 ### 15.2 Markdown Batch Execution Discipline
 
 Any loop that crawls sources or may process many graph records must execute in

assets/protocol_assets/system/graph_population/v001/graph_population_protocol_schema.md

@@ -1203,6 +1203,8 @@ Recoverable semantic failures include:
 ```text
 insufficient_results
 source_rate_limited
+source_adapter_unavailable
+source_adapter_expansion_required
 source_depth_limited
 field_richness_limited
 relation_family_diversity_unmet
@@ -1225,6 +1227,8 @@ narrow_query
 broaden_query
 switch_source_adapter
 switch_source_class
+discover_new_source_adapter
+evaluate_source_adapter_candidate
 seed_from_authoritative_index
 extract_seed_list
 split_current_type_or_edge_work_into_batches
@@ -1240,9 +1244,15 @@ directory. Recovery may not be hidden in Python, shell scripts, notebooks,
 model memory, terminal history, or temporary files.
 
 A recovery ladder is exhausted only when the generated attempt budget, source
-fallback order, query rewrite policy, source-class expansion policy, and
-frontier expansion policy have all been tried or explicitly ruled out. Only
-then may the run stop with a recovery-exhausted limitation such as:
+fallback order, query rewrite policy, source-adapter expansion policy,
+source-class expansion policy, and frontier expansion policy have all been
+tried or explicitly ruled out. For ordinary `MAKE-GRAPH`, an outage, timeout,
+malformed response, rate limit, sparse result, or depth limit in one source
+adapter or one source family does not exhaust recovery. It must route to the
+explicit Markdown source-adapter expansion child loop unless the human
+explicitly restricted the source scope or all admissible source-adapter
+candidates have already been evaluated and rejected in repo-local reports.
+Only then may the run stop with a recovery-exhausted limitation such as:
 
 ```text
 source_strategy_recovery_exhausted
@@ -1914,7 +1924,12 @@ Minimum candidate shape:
     "live_population_lookup_authorization": "authorized_by_make_graph",
     "population_without_live_or_supplied_sources": "not_applicable_live_lookup_authorized",
     "explicit_no_live_lookup_overrides_make_graph_default": false,
-    "source_fallback_order": []
+    "source_fallback_order": [],
+    "source_adapter_registry_path": "runs/run_001/source_adapter_registry.md",
+    "source_adapter_candidate_frontier_path": "runs/run_001/source_adapter_candidate_frontier.md",
+    "source_adapter_expansion_allowed_by_default": true,
+    "source_adapter_expansion_requires_human_approval": false,
+    "same_source_family_exhaustion_rule": "do_not_treat_one_source_family_failure_as_global_source_exhaustion"
   },
   "field_policy": {
     "field_tier_contract_defined": true,
@@ -2341,6 +2356,83 @@ stop_with_source_failure
 
 The executor must not invent this policy during the run.
 
+### 7.1.1 Domain-Discovered Source Adapter Expansion Contract
+
+For ordinary `MAKE-GRAPH`, source-backed means backed by admissible persisted
+source evidence. It does not mean backed by one privileged source adapter.
+Wikidata, a public API, a search result page, an official organization page, an
+authority file, a catalog, a repository, or another public source family can be
+an adapter if the generated protocol records how it is queried, filtered,
+persisted, and interpreted through Markdown reports.
+
+Unless the human explicitly restricts sources, source adapter expansion is an
+internal generated recovery loop. It does not require human approval merely
+because the next adapter is different from the first adapter. Human approval is
+required only when the candidate adapter needs credentials, payment, private or
+personal data access, terms-of-service-sensitive scraping, a non-public source,
+or a source family outside an explicit user-provided source scope.
+
+Every ordinary `MAKE-GRAPH` bundle must define a source adapter registry and an
+adapter expansion frontier:
+
+```text
+source_adapter_registry_path:
+source_adapter_candidate_frontier_path:
+source_adapter_expansion_policy:
+source_adapter_expansion_allowed_by_default: true
+source_adapter_expansion_requires_human_approval: false unless credentialed/private/restricted
+source_scope_restriction_rule:
+adapter_candidate_evaluation_criteria:
+same_source_family_exhaustion_rule:
+```
+
+A source fallback order that contains only variants of one source family, such
+as two endpoints for the same public database, is not a complete recovery
+policy for ordinary `MAKE-GRAPH`. It is only the first adapter-family retry
+plan. If that source family rate-limits, times out, malforms responses, or is
+too shallow, the next legal recovery action is to create or advance a Markdown
+source-adapter expansion child loop.
+
+The generated source-adapter expansion loop must be domain-agnostic in shape
+and domain-specific only in its discovered candidates. It may consider source
+families such as:
+
+```text
+official domain or institution pages
+public APIs
+public catalogs or collection databases
+authority files or controlled vocabularies
+public indexes or registries
+entity pages from the current source family
+source pages cited by already accepted records
+search/discovery results that lead to persisted source batches
+human-supplied source files or URLs
+```
+
+Before a new adapter can affect graph JSON, the generated Markdown child loop
+must record:
+
+```text
+adapter_candidate_id:
+source_family:
+why_relevant_to_current_domain:
+allowed_query_or_navigation_shapes:
+evidence_shape:
+expected_record_types:
+coverage_probe_plan:
+rate_limit_or_timeout_policy:
+persistence_plan_under_runs/<run_id>/source_batches:
+markdown_report_path:
+accept_reject_defer_decision:
+reason:
+```
+
+If one adapter fails but admissible adapter candidates remain unevaluated, the
+run must not stop with `source_depth_limited`, `source_rate_limited`, or
+`source_strategy_recovery_exhausted`. It must stop only the current batch,
+checkpoint state, create/update the source-adapter expansion child loop, and
+continue there when execution budget permits.
+
 ## 7.2 Markdown Batch Partition Contract
 
 Every crawl loop that can touch an external source or a large graph surface
@@ -3975,6 +4067,11 @@ child_loop_generation_rule:
 recovery_attempt_budget:
 source_strategy_fallback_order:
 query_rewrite_policy:
+source_adapter_expansion_policy:
+source_adapter_registry_path:
+source_adapter_candidate_frontier_path:
+source_adapter_candidate_evaluation_criteria:
+same_source_family_exhaustion_rule:
 source_class_expansion_policy:
 frontier_expansion_policy:
 resume_parent_condition:
@@ -4000,6 +4097,9 @@ max_sources_checked:
 evidence_threshold:
 source_adapter_id:
 source_fallback_order:
+source_adapter_expansion_policy:
+source_adapter_registry_path:
+source_adapter_candidate_frontier_path:
 timeout_seconds:
 max_retries:
 backoff_seconds:
@@ -4132,6 +4232,7 @@ pair_evidence_review_rule: edge instances need exact source-relation-target evid
 counter_reconciliation_rule: accepted counters must match graph_build_target using accepted primitive families
 status_rule: only exact semantic_acceptance_status: passed completes the run
 generated_script_rejection_rule: generated population scripts are forbidden workflow drivers
+source_adapter_expansion_rule: one adapter-family failure routes to Markdown adapter expansion, not global source failure
 recovery_rule: if recoverable gaps remain, create/update Markdown child loops and continue
 ```
 
@@ -4234,6 +4335,7 @@ Compatibility requires:
 - path reconciliation fields;
 - source crawl boundaries;
 - source adapter fallback policy;
+- source adapter expansion policy and adapter candidate frontier for ordinary `MAKE-GRAPH`;
 - recoverable semantic failure policy;
 - child-loop generation policy for source, field, relation, instance, and edge-evidence recovery;
 - batch execution policy;
@@ -4483,7 +4585,8 @@ A generated protocol bundle is acceptable only if:
 - every semantic loop spec defines a recovery ladder, child-loop generation rule, recovery budget, resume condition, exhaustion condition, and proxy-substitution ban;
 - every loop spec defines source boundaries;
 - every source-crawling loop defines source adapters, fallback behavior,
-  timeout/retry behavior, semantic recovery behavior, and batch execution;
+  source-adapter expansion behavior, timeout/retry behavior, semantic recovery
+  behavior, and batch execution;
 - every batch-capable loop defines
   `batch_execution_meaning: markdown_batch_partition_not_generated_code_loop`;
 - every batch-capable loop defines `batch_plan_path` and