[Capability][Server] Add element sources and isolate loading failures

CHANGELOG.md

@@ -8,6 +8,9 @@ All notable changes to `mcp/sdk` will be documented in this file.
 * [BC Break] `Builder::setLazyLoading(true)` now also defers loading into a registry passed to `setRegistry()`, which any `RegistryInterface` supports. A registry you read directly stays empty until the server first reads it; without the call it is still loaded at build.
 * [BC Break] Detect the server capabilities when a request asks for them, so `initialize`, `server/discover` and `subscriptions/listen` run a pending load. The server advertises what the registry holds, and no longer every kind for a custom loader or discovery that was not loaded yet. While a load is still running, it advertises all kinds. `Configuration::$capabilities` is replaced by `Configuration::capabilities()`, and the constructor also takes a closure.
 * Add `DeferredRegistryInterface`, so the builder loads a custom registry without announcing the load as list changes, and no longer depends on the concrete `Registry`.
+* Add `ElementSourceInterface`, `ElementCollection` and `Builder::addSource()`. A source returns its elements as a value.
+* [BC Break] An exception from an element loader or discovery no longer aborts loading: `ChainLoader` logs it as an error, runs the remaining loaders and throws a `PartialLoadException`. The registry serves what loaded and retries on the next read, the server advertises all kinds until then, and `Builder::build()` no longer throws for such a failure. A `ConfigurationException`, e.g. from a manual element with a missing handler class, still aborts loading.
+* Deprecate `DiscoveryLoader`, use `SourceLoader` over a `DiscoverySource` with `onlyMissing: true` instead.
 * Split `RegistryInterface` by role into `ElementCatalogInterface`, `ElementResolverInterface` and `ElementWriterInterface`, which it extends. Request handlers depend on the role they use, so a handler can run against a catalog or resolver alone.
 
 0.10.0

docs/run/server-builder.md

@@ -508,6 +508,7 @@ $server = Server::builder()
 | `setReferenceHandler()` | referenceHandler | Replace the class that calls the handlers of tools, resources and prompts |
 | `setRegistry()` | registry | Use your own registry. It is loaded when the server is built, unless you call `setLazyLoading()` |
 | `setLazyLoading()` | lazyLoading? | Load elements on the first registry read, which is at the latest when a client asks for the capabilities (default without `setRegistry()`), or, with `false`, when the server is built. Your own registry stays empty until the server first reads it |
+| `addSource()` | source | Add an element source, see [Element sources](../servers/registration.md#element-sources) |
 | `addLoader()` | loader | Add a custom loader that registers elements |
 | `addLoaders()` | loaders | Add several custom loaders |
 | `addRequestHandler()` | handler | Prepend a single custom request handler |

docs/servers/registration.md

@@ -256,6 +256,62 @@ Use `add()` when the metadata cannot be inferred from a handler class via reflec
 prefer `addTool/addResource/addResourceTemplate/addPrompt`, which can derive metadata from the handler's signature and
 docblock.
 
+## Element sources
+
+Your tools, resources or prompts may come from a database or another service. An element source reads them and
+returns them as a collection:
+
+```php
+use Mcp\Server\Capability\Registry\Source\ElementCollection;
+use Mcp\Server\Capability\Registry\Source\ElementSourceInterface;
+use Mcp\Server\Capability\Registry\ToolReference;
+use Mcp\Schema\Tool;
+
+final class ProductToolSource implements ElementSourceInterface
+{
+    public function __construct(private readonly ProductRepository $products)
+    {
+    }
+
+    public function provide(): ElementCollection
+    {
+        $tools = [];
+        foreach ($this->products->findPublished() as $product) {
+            $tool = new Tool('product_'.$product->id, $product->title, $product->inputSchema, $product->description, null);
+            $tools[] = new ToolReference($tool, [ProductTools::class, 'run']);
+        }
+
+        return new ElementCollection(tools: $tools);
+    }
+}
+```
+
+Register the source on the builder:
+
+```php
+$server = Server::builder()
+    ->setServerInfo('Product Server', '1.0.0')
+    ->addSource(new ProductToolSource($products))
+    ->build();
+```
+
+The server calls `provide()` when it loads the elements.
+
+An element from a source replaces one with the same identifier from a custom loader. Elements registered on the
+builder replace those of a source, and discovery never replaces an existing element.
+
+If a source throws an exception, the server logs it as an error and serves the other elements. The server calls the
+source again on each read until it succeeds. Until then, it doesn't call the sources that already succeeded. The registry
+doesn't announce elements that arrive on a later attempt, so clients see them the next time they list.
+
+While a source fails, the server advertises tools, resources and prompts, because it can't know what the source provides.
+
+A custom loader can write elements before it fails. While a custom loader fails, the server also runs everything that
+comes after it on each read, including the sources.
+
+A `ConfigurationException` stops the load instead, because it reports a mistake in your setup, e.g. a
+handler class that doesn't exist. The request that triggered the load fails with `-32603`.
+
 ## Hybrid Approach
 
 Combine discovery with manual registration for maximum flexibility:

src/Server/Builder.php

@@ -26,7 +26,6 @@
 use Mcp\Schema\Tool;
 use Mcp\Schema\ToolAnnotations;
 use Mcp\Server\Capability\Completion\ProviderInterface;
-use Mcp\Server\Capability\DeferredRegistryInterface;
 use Mcp\Server\Capability\Discovery\CachedDiscoverer;
 use Mcp\Server\Capability\Discovery\Discoverer;
 use Mcp\Server\Capability\Discovery\DiscovererInterface;
@@ -37,12 +36,14 @@
 use Mcp\Server\Capability\Registry\ElementReference;
 use Mcp\Server\Capability\Registry\LazyRegistry;
 use Mcp\Server\Capability\Registry\Loader\ChainLoader;
-use Mcp\Server\Capability\Registry\Loader\DiscoveryLoader;
 use Mcp\Server\Capability\Registry\Loader\ExplicitElementLoader;
 use Mcp\Server\Capability\Registry\Loader\LoaderInterface;
 use Mcp\Server\Capability\Registry\Loader\ReflectedElementLoader;
+use Mcp\Server\Capability\Registry\Loader\SourceLoader;
 use Mcp\Server\Capability\Registry\ReferenceHandler;
 use Mcp\Server\Capability\Registry\ReferenceHandlerInterface;
+use Mcp\Server\Capability\Registry\Source\DiscoverySource;
+use Mcp\Server\Capability\Registry\Source\ElementSourceInterface;
 use Mcp\Server\Capability\RegistryInterface;
 use Mcp\Server\Exception\InvalidArgumentException;
 use Mcp\Server\Exception\LogicException;
@@ -276,6 +277,11 @@ final class Builder
      */
     private array $loaders = [];
 
+    /**
+     * @var ElementSourceInterface[]
+     */
+    private array $sources = [];
+
     private bool $hasCustomRegistry = false;
 
     private ?bool $lazyLoading = null;
@@ -938,6 +944,17 @@ public function addLoaders(iterable $loaders): self
         return $this;
     }
 
+    /**
+     * Register an element source. Its elements win over those of custom loaders and discovery,
+     * and lose to elements registered on the builder. A failing source is logged and skipped.
+     */
+    public function addSource(ElementSourceInterface $source): self
+    {
+        $this->sources[] = $source;
+
+        return $this;
+    }
+
     /**
      * Stop serving multi round-trip handlers to handshake-era clients.
      *
@@ -1143,10 +1160,9 @@ private function resolve(): array
             $this->gcDivisor,
         );
 
-        // ExplicitElementLoader and ReflectedElementLoader run before DiscoveryLoader so manual entries are seen first;
-        // DiscoveryLoader's identity check then preserves them against same-name discovered entries.
         $loaders = [
             ...$this->loaders,
+            ...array_map(static fn (ElementSourceInterface $source): SourceLoader => new SourceLoader($source), $this->sources),
             new ExplicitElementLoader(
                 $this->explicitTools,
                 $this->explicitResources,
@@ -1158,25 +1174,22 @@ private function resolve(): array
 
         if (null !== $this->discoveryBasePath) {
             $discoverer = $this->discoverer ?? $this->createDiscoverer($logger);
-            $loaders[] = new DiscoveryLoader($this->discoveryBasePath, $this->discoveryScanDirs, $this->discoveryExcludeDirs, $discoverer, $this->discoveryNamePatterns, $logger);
+            $loaders[] = new SourceLoader(
+                new DiscoverySource($this->discoveryBasePath, $this->discoveryScanDirs, $this->discoveryExcludeDirs, $discoverer, $this->discoveryNamePatterns),
+                true,
+                $logger,
+            );
         }
 
-        $chainLoader = new ChainLoader($loaders);
+        $chainLoader = new ChainLoader($loaders, $logger);
 
-        if ($this->hasCustomRegistry && true === $this->lazyLoading) {
-            $registry = new LazyRegistry($this->registry, $chainLoader);
-        } elseif ($this->hasCustomRegistry) {
-            $registry = $this->registry;
-            if ($registry instanceof DeferredRegistryInterface) {
-                $registry->loadFrom($chainLoader);
-            } else {
-                $chainLoader->load($registry);
-            }
-        } else {
-            $registry = new Registry($eventDispatcher, $logger, loader: $chainLoader);
-            if (false === $this->lazyLoading) {
-                $registry->load();
-            }
+        $registry = $this->hasCustomRegistry
+            ? new LazyRegistry($this->registry, $chainLoader)
+            : new Registry($eventDispatcher, $logger, loader: $chainLoader);
+
+        // A supplied registry is read directly by its owner, so it only waits when that is asked for.
+        if ($this->hasCustomRegistry ? true !== $this->lazyLoading : false === $this->lazyLoading) {
+            $registry->load();
         }
 
         $messageFactory = MessageFactory::make(additional: $this->extensionMessages);

src/Server/Capability/DeferredRegistryInterface.php

@@ -21,7 +21,7 @@
 interface DeferredRegistryInterface
 {
     /**
-     * Runs a pending load, if there is one. A failed load stays pending.
+     * Runs a pending load, if there is one. A load that fails or is incomplete stays pending.
      */
     public function load(): void;
 

src/Server/Capability/Registry.php

@@ -27,6 +27,7 @@
 use Mcp\Server\Event\ResourceTemplateListChangedEvent;
 use Mcp\Server\Event\ToolListChangedEvent;
 use Mcp\Server\Exception\InvalidCursorException;
+use Mcp\Server\Exception\PartialLoadException;
 use Mcp\Server\Exception\PromptNotFoundException;
 use Mcp\Server\Exception\ResourceNotFoundException;
 use Mcp\Server\Exception\ToolNotFoundException;
@@ -90,7 +91,11 @@ public function load(): void
             return;
         }
 
-        $this->loadFrom($this->loader);
+        try {
+            $this->loadFrom($this->loader);
+        } catch (PartialLoadException) {
+            return;
+        }
 
         $this->loaded = true;
     }

src/Server/Capability/Registry/LazyRegistry.php

@@ -19,6 +19,7 @@
 use Mcp\Server\Capability\DeferredRegistryInterface;
 use Mcp\Server\Capability\Registry\Loader\LoaderInterface;
 use Mcp\Server\Capability\RegistryInterface;
+use Mcp\Server\Exception\PartialLoadException;
 
 /**
  * Defers a loader over any registry to the first read, leaving the registry itself unaware of it.
@@ -49,6 +50,8 @@ public function load(): void
             try {
                 $this->loadFrom($this->loader);
                 $this->loaded = true;
+            } catch (PartialLoadException) {
+                // what loaded is served, and the load stays pending
             } finally {
                 $this->loading = false;
             }

src/Server/Capability/Registry/Loader/ChainLoader.php

@@ -12,27 +12,75 @@
 namespace Mcp\Server\Capability\Registry\Loader;
 
 use Mcp\Server\Capability\RegistryInterface;
+use Mcp\Server\Exception\ConfigurationException;
+use Mcp\Server\Exception\ElementSourceException;
+use Mcp\Server\Exception\PartialLoadException;
+use Psr\Log\LoggerInterface;
+use Psr\Log\NullLogger;
 
 /**
  * Composes multiple loaders into a single one. Child loaders run in the order they were given;
- * for conflicting keys, the loader that runs later wins (last-write-wins).
+ * for conflicting keys, the loader that runs later wins (last-write-wins). A failing loader is logged
+ * and skipped, and the chain reports the incomplete load once the others ran. A
+ * {@see ConfigurationException} is a mistake in the server's own setup that no retry fixes, so it
+ * ends the load instead.
+ *
+ * The next load of that registry only retries what failed. A loader that recovers, or that may have
+ * written before it failed, is followed by all loaders after it, so that last-write-wins gives the
+ * result of an uninterrupted run. An {@see ElementSourceException} means that nothing was written.
  *
  * @author Christopher Hertel <mail@christopher-hertel.de>
  */
 final class ChainLoader implements LoaderInterface
 {
+    /**
+     * @var \WeakMap<RegistryInterface, array<int, true>>
+     */
+    private \WeakMap $failed;
+
     /**
      * @param LoaderInterface[] $loaders
      */
     public function __construct(
         private readonly array $loaders,
+        private readonly LoggerInterface $logger = new NullLogger(),
     ) {
+        $this->failed = new \WeakMap();
     }
 
     public function load(RegistryInterface $registry): void
     {
-        foreach ($this->loaders as $loader) {
-            $loader->load($registry);
+        $retry = $this->failed[$registry] ?? null;
+        $runRemaining = null === $retry;
+        $failed = [];
+
+        foreach (array_values($this->loaders) as $position => $loader) {
+            if (!$runRemaining && !isset($retry[$position])) {
+                continue;
+            }
+
+            try {
+                $loader->load($registry);
+                $runRemaining = true;
+            } catch (ConfigurationException $e) {
+                unset($this->failed[$registry]);
+
+                throw $e;
+            } catch (\Throwable $e) { // loading runs inside request handling, so nothing a source throws may end it
+                $failed[$position] = true;
+                $runRemaining = $runRemaining || !$e instanceof ElementSourceException;
+                $this->logger->error(\sprintf('Loading elements from "%s" failed: %s', $loader::class, $e->getMessage()), ['exception' => $e]);
+            }
+        }
+
+        if ([] === $failed) {
+            unset($this->failed[$registry]);
+
+            return;
         }
+
+        $this->failed[$registry] = $failed;
+
+        throw new PartialLoadException('Some element loaders failed.');
     }
 }

src/Server/Capability/Registry/Loader/DiscoveryLoader.php

@@ -13,13 +13,16 @@
 
 use Mcp\Server\Capability\Discovery\DiscovererInterface;
 use Mcp\Server\Capability\Discovery\DiscoveryState;
+use Mcp\Server\Capability\Registry\Source\DiscoverySource;
 use Mcp\Server\Capability\RegistryInterface;
 use Psr\Log\LoggerInterface;
 use Psr\Log\NullLogger;
 
 /**
  * @internal
  *
+ * @deprecated use {@see SourceLoader} over a {@see DiscoverySource} with `onlyMissing: true`
+ *
  * @author Antoine Bluchet <soyuka@gmail.com>
  */
 final class DiscoveryLoader implements LoaderInterface