[Capability][Server] Add element sources and isolate loading failures

Refs #495

Author: chr-hertel

CHANGELOG.md

@@ -29,6 +29,9 @@ All notable changes to `mcp/sdk` will be documented in this file.
 * [BC Break] `ReferenceHandler` resolves a string handler that is not an invokable class from the container, so a loader can register a service id as handler. The container goes first, so a function handler no longer runs if the container has an entry of the same name.
 * [BC Break] `Builder::setLazyLoading(true)` now also defers loading into a registry passed to `setRegistry()`, which any `RegistryInterface` supports. A registry you read directly stays empty until the server first reads it; without the call it is still loaded at build.
 * Add `DeferredRegistryInterface`, so the builder loads a custom registry without announcing the load as list changes, and no longer depends on the concrete `Registry`.
+* Add `ElementSourceInterface`, `ElementCollection` and `Builder::addSource()`. A source returns its elements as a value and declares its element kinds, so the server announces only those capabilities.
+* [BC Break] An exception from an element loader or discovery no longer aborts loading: `ChainLoader` logs it as an error, runs the remaining loaders and throws a `PartialLoadException`. The registry serves what loaded and retries on the next read, and `Builder::build()` no longer throws for such a failure.
+* Deprecate `DiscoveryLoader`, use `SourceLoader` over a `DiscoverySource` with `onlyMissing: true` instead.
 * Split `RegistryInterface` by role into `ElementCatalogInterface`, `ElementResolverInterface` and `ElementWriterInterface`, which it extends. Request handlers depend on the role they use, so a handler can run against a catalog or resolver alone.
 * Add `Psr6SessionStore` and `Psr6NotificationBus`, which work like `Psr16SessionStore` and `Psr16NotificationBus` over a PSR-6 cache pool. `Builder::setDiscovery()` also accepts a PSR-6 pool as `$cache`.
 * Fix a nullable tool parameter with a default (e.g. `?string $category = 'books'`) receiving the default when the client explicitly sends `null`: `ReferenceHandler` now passes the `null` through.

docs/run/server-builder.md

@@ -489,6 +489,7 @@ $server = Server::builder()
 | `setReferenceHandler()` | referenceHandler | Replace the class that calls the handlers of tools, resources and prompts |
 | `setRegistry()` | registry | Use your own registry. It is loaded when the server is built, unless you call `setLazyLoading()` |
 | `setLazyLoading()` | lazyLoading? | Load elements on the first registry read (default without `setRegistry()`) or, with `false`, when the server is built. Your own registry stays empty until the server first reads it |
+| `addSource()` | source | Add an element source, see [Element sources](../servers/registration.md#element-sources) |
 | `addLoader()` | loader | Add a custom loader that registers elements |
 | `addLoaders()` | loaders | Add several custom loaders |
 | `addRequestHandler()` | handler | Prepend a single custom request handler |

docs/servers/registration.md

@@ -241,6 +241,63 @@ Use `add()` when the metadata cannot be inferred from a handler class via reflec
 prefer `addTool/addResource/addResourceTemplate/addPrompt`, which can derive metadata from the handler's signature and
 docblock.
 
+## Element sources
+
+Your tools, resources or prompts may come from a database or another service. An element source reads them and
+returns them as a collection:
+
+```php
+use Mcp\Capability\Registry\Source\ElementCollection;
+use Mcp\Capability\Registry\Source\ElementKind;
+use Mcp\Capability\Registry\Source\ElementSourceInterface;
+use Mcp\Capability\Registry\ToolReference;
+use Mcp\Schema\Tool;
+
+final class ProductToolSource implements ElementSourceInterface
+{
+    public function __construct(private readonly ProductRepository $products)
+    {
+    }
+
+    public function provide(): ElementCollection
+    {
+        $tools = [];
+        foreach ($this->products->findPublished() as $product) {
+            $tool = new Tool('product_'.$product->id, $product->title, $product->inputSchema, $product->description, null);
+            $tools[] = new ToolReference($tool, [ProductTools::class, 'run']);
+        }
+
+        return new ElementCollection(tools: $tools);
+    }
+
+    public function kinds(): array
+    {
+        return [ElementKind::Tool];
+    }
+}
+```
+
+Register the source on the builder:
+
+```php
+$server = Server::builder()
+    ->setServerInfo('Product Server', '1.0.0')
+    ->addSource(new ProductToolSource($products))
+    ->build();
+```
+
+The server calls `kinds()` to announce its capabilities and calls `provide()` when it loads the elements.
+
+An element from a source replaces one with the same identifier from a custom loader. Elements registered on the
+builder replace those of a source, and discovery never replaces an existing element.
+
+If a source throws an exception, the server logs it as an error and serves the other elements. The server calls the
+source again on each read until it succeeds. Until then, it doesn't call the sources that already succeeded. The registry
+doesn't announce elements that arrive on a later attempt, so clients see them the next time they list.
+
+A custom loader can write elements before it fails. While a custom loader fails, the server also runs everything that
+comes after it on each read, including the sources.
+
 ## Hybrid Approach
 
 Combine both methods for maximum flexibility:

src/Capability/DeferredRegistryInterface.php

@@ -21,7 +21,7 @@
 interface DeferredRegistryInterface
 {
     /**
-     * Runs a pending load, if there is one. A failed load stays pending.
+     * Runs a pending load, if there is one. A load that fails or is incomplete stays pending.
      */
     public function load(): void;
 

src/Capability/Registry.php

@@ -22,6 +22,7 @@
 use Mcp\Event\ResourceTemplateListChangedEvent;
 use Mcp\Event\ToolListChangedEvent;
 use Mcp\Exception\InvalidCursorException;
+use Mcp\Exception\PartialLoadException;
 use Mcp\Exception\PromptNotFoundException;
 use Mcp\Exception\ResourceNotFoundException;
 use Mcp\Exception\ToolNotFoundException;
@@ -89,7 +90,11 @@ public function load(): void
             return;
         }
 
-        $this->loadFrom($this->loader);
+        try {
+            $this->loadFrom($this->loader);
+        } catch (PartialLoadException) {
+            return;
+        }
 
         $this->loaded = true;
     }

src/Capability/Registry/LazyRegistry.php

@@ -14,6 +14,7 @@
 use Mcp\Capability\DeferredRegistryInterface;
 use Mcp\Capability\Registry\Loader\LoaderInterface;
 use Mcp\Capability\RegistryInterface;
+use Mcp\Exception\PartialLoadException;
 use Mcp\Schema\Page;
 use Mcp\Schema\Prompt;
 use Mcp\Schema\ResourceDefinition;
@@ -48,6 +49,8 @@ public function load(): void
         try {
             $this->loadFrom($this->loader);
             $this->loaded = true;
+        } catch (PartialLoadException) {
+            // what loaded is served, and the load stays pending
         } finally {
             $this->loading = false;
         }

src/Capability/Registry/Loader/ChainLoader.php

@@ -12,27 +12,67 @@
 namespace Mcp\Capability\Registry\Loader;
 
 use Mcp\Capability\RegistryInterface;
+use Mcp\Exception\PartialLoadException;
+use Psr\Log\LoggerInterface;
+use Psr\Log\NullLogger;
 
 /**
  * Composes multiple loaders into a single one. Child loaders run in the order they were given;
- * for conflicting keys, the loader that runs later wins (last-write-wins).
+ * for conflicting keys, the loader that runs later wins (last-write-wins). A failing loader is logged
+ * and skipped, and the chain reports the incomplete load once the others ran.
+ *
+ * The next load of that registry only retries what failed. A loader that recovers, or that may have
+ * written before it failed, is followed by all loaders after it, so that last-write-wins gives the
+ * result of an uninterrupted run. A {@see SourceLoader} fails before it writes.
  *
  * @author Christopher Hertel <mail@christopher-hertel.de>
  */
 final class ChainLoader implements LoaderInterface
 {
+    /**
+     * @var \WeakMap<RegistryInterface, array<int, true>>
+     */
+    private \WeakMap $failed;
+
     /**
      * @param LoaderInterface[] $loaders
      */
     public function __construct(
         private readonly array $loaders,
+        private readonly LoggerInterface $logger = new NullLogger(),
     ) {
+        $this->failed = new \WeakMap();
     }
 
     public function load(RegistryInterface $registry): void
     {
-        foreach ($this->loaders as $loader) {
-            $loader->load($registry);
+        $retry = $this->failed[$registry] ?? null;
+        $runRemaining = null === $retry;
+        $failed = [];
+
+        foreach (array_values($this->loaders) as $position => $loader) {
+            if (!$runRemaining && !isset($retry[$position])) {
+                continue;
+            }
+
+            try {
+                $loader->load($registry);
+                $runRemaining = true;
+            } catch (\Throwable $e) { // loading runs inside request handling, so nothing a source throws may end it
+                $failed[$position] = true;
+                $runRemaining = $runRemaining || !$loader instanceof SourceLoader;
+                $this->logger->error(\sprintf('Loading elements from "%s" failed: %s', $loader::class, $e->getMessage()), ['exception' => $e]);
+            }
         }
+
+        if ([] === $failed) {
+            unset($this->failed[$registry]);
+
+            return;
+        }
+
+        $this->failed[$registry] = $failed;
+
+        throw new PartialLoadException('Some element loaders failed.');
     }
 }

src/Capability/Registry/Loader/DiscoveryLoader.php

@@ -13,13 +13,16 @@
 
 use Mcp\Capability\Discovery\DiscovererInterface;
 use Mcp\Capability\Discovery\DiscoveryState;
+use Mcp\Capability\Registry\Source\DiscoverySource;
 use Mcp\Capability\RegistryInterface;
 use Psr\Log\LoggerInterface;
 use Psr\Log\NullLogger;
 
 /**
  * @internal
  *
+ * @deprecated use {@see SourceLoader} over a {@see DiscoverySource} with `onlyMissing: true`
+ *
  * @author Antoine Bluchet <soyuka@gmail.com>
  */
 final class DiscoveryLoader implements LoaderInterface

src/Capability/Registry/Loader/SourceLoader.php

@@ -0,0 +1,78 @@
+<?php
+
+/*
+ * This file is part of the official PHP MCP SDK.
+ *
+ * A collaboration between Symfony and the PHP Foundation.
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Mcp\Capability\Registry\Loader;
+
+use Mcp\Capability\Registry\Source\ElementSourceInterface;
+use Mcp\Capability\RegistryInterface;
+use Psr\Log\LoggerInterface;
+use Psr\Log\NullLogger;
+
+/**
+ * Writes the elements of a source into a registry.
+ *
+ * @author Christopher Hertel <mail@christopher-hertel.de>
+ */
+final class SourceLoader implements LoaderInterface
+{
+    /**
+     * @param bool $onlyMissing keep what the registry already holds under the same name or URI
+     */
+    public function __construct(
+        private readonly ElementSourceInterface $source,
+        private readonly bool $onlyMissing = false,
+        private readonly LoggerInterface $logger = new NullLogger(),
+    ) {
+    }
+
+    public function load(RegistryInterface $registry): void
+    {
+        $collection = $this->source->provide();
+
+        foreach ($collection->getTools() as $reference) {
+            if ($this->mayWrite('tool', $reference->tool->name, $registry->hasTool(...))) {
+                $registry->registerTool($reference->tool, $reference->handler);
+            }
+        }
+
+        foreach ($collection->getResources() as $reference) {
+            if ($this->mayWrite('resource', $reference->resource->uri, $registry->hasResource(...))) {
+                $registry->registerResource($reference->resource, $reference->handler);
+            }
+        }
+
+        foreach ($collection->getResourceTemplates() as $reference) {
+            if ($this->mayWrite('resource template', $reference->resourceTemplate->uriTemplate, $registry->hasResourceTemplate(...))) {
+                $registry->registerResourceTemplate($reference->resourceTemplate, $reference->handler, $reference->completionProviders);
+            }
+        }
+
+        foreach ($collection->getPrompts() as $reference) {
+            if ($this->mayWrite('prompt', $reference->prompt->name, $registry->hasPrompt(...))) {
+                $registry->registerPrompt($reference->prompt, $reference->handler, $reference->completionProviders);
+            }
+        }
+    }
+
+    /**
+     * @param callable(string): bool $exists
+     */
+    private function mayWrite(string $kind, string $key, callable $exists): bool
+    {
+        if (!$this->onlyMissing || !$exists($key)) {
+            return true;
+        }
+
+        $this->logger->debug(\sprintf('Ignoring %s "%s" from an element source: the registry already holds it.', $kind, $key));
+
+        return false;
+    }
+}

src/Capability/Registry/Source/DiscoverySource.php

@@ -0,0 +1,48 @@
+<?php
+
+/*
+ * This file is part of the official PHP MCP SDK.
+ *
+ * A collaboration between Symfony and the PHP Foundation.
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Mcp\Capability\Registry\Source;
+
+use Mcp\Capability\Discovery\DiscovererInterface;
+
+/**
+ * Provides the elements a {@see DiscovererInterface} finds in a directory tree.
+ *
+ * @author Christopher Hertel <mail@christopher-hertel.de>
+ */
+final class DiscoverySource implements ElementSourceInterface
+{
+    /**
+     * @param string[] $scanDirs
+     * @param string[] $excludeDirs
+     * @param string[] $namePatterns
+     */
+    public function __construct(
+        private readonly string $basePath,
+        private readonly array $scanDirs,
+        private readonly array $excludeDirs,
+        private readonly DiscovererInterface $discoverer,
+        private readonly array $namePatterns = DiscovererInterface::DEFAULT_NAME_PATERNS,
+    ) {
+    }
+
+    public function provide(): ElementCollection
+    {
+        $state = $this->discoverer->discover($this->basePath, $this->scanDirs, $this->excludeDirs, $this->namePatterns);
+
+        return new ElementCollection($state->getTools(), $state->getResources(), $state->getResourceTemplates(), $state->getPrompts());
+    }
+
+    public function kinds(): array
+    {
+        return ElementKind::cases();
+    }
+}