docs: document LangChain 1.x import rules and parallel doc maintenance

Expand sync-langchain-docs to cover all six doc surfaces, drop README-as-
source-of-truth wording, and record which imports use langchain vs langchain_core.

Co-authored-by: Cursor <cursoragent@cursor.com>

Author: shixish

.claude/skills/audit-langchain-docs/SKILL.md

@@ -8,7 +8,7 @@ allowed-tools: Bash(python3:*), Bash(grep:*), Bash(find:*), Bash(make:*), Read,
 
 This skill checks every Diffbot documentation file on the LangChain docs site against this repo's public API, runs the docs repo's own validators (Vale, etc.), and propagates any findings back into this repo — `README.md` and Python source files (docstrings, comments). The docs repo's validation standards apply repo-wide here.
 
-## Source of truth (this repo)
+## Package authority (this repo)
 
 | Artifact | What it defines |
 |----------|----------------|
@@ -17,17 +17,19 @@ This skill checks every Diffbot documentation file on the LangChain docs site ag
 | `langchain_diffbot/retrievers/` | Retriever constructor parameters, return types |
 | `langchain_diffbot/chat_models/` | Chat model constructor parameters |
 | `langchain_diffbot/document_loaders/` | Loader constructor parameters |
-| `README.md` | Canonical prose, API table, auth model, examples |
+| `README.md` | GitHub/PyPI package reference — prose, API table, auth model, examples |
 
 ## Documentation files to audit (langchain-ai/docs)
 
 A local `langchain-ai/docs` checkout is expected at the sibling **`../langchain-docs`** by default (override with `$LANGCHAIN_DOCS_REPO`).
 
 | File | What it should reflect |
 |------|----------------------|
-| `src/oss/python/integrations/providers/diffbot.mdx` | Overview hub: API table, install, auth, component table, links to tools/retrievers pages |
+| `src/oss/python/integrations/providers/diffbot.mdx` | Overview hub: API table, install, auth, component table, links to detail pages |
 | `src/oss/python/integrations/tools/diffbot.mdx` | All 7 tools: `DiffbotExtractTool`, `DiffbotWebSearchTool`, `DiffbotKnowledgeGraphTool`, `DiffbotEntitiesTool`, `DiffbotAskTool`, `DiffbotOntologyTool`, `DiffbotDQLProbeTool` |
 | `src/oss/python/integrations/retrievers/diffbot.mdx` | Both retrievers: `DiffbotKnowledgeGraphRetriever`, `DiffbotWebSearchRetriever` |
+| `src/oss/python/integrations/chat/diffbot.mdx` | `ChatDiffbot` usage |
+| `src/oss/python/integrations/document_loaders/diffbot.mdx` | `DiffbotExtractLoader`, `DiffbotCrawlLoader` |
 | `src/oss/python/integrations/providers/all_providers.mdx` | Card entry for Diffbot |
 | `src/oss/python/integrations/tools/index.mdx` | Row in Search table + card in All tools and toolkits |
 | `src/oss/python/integrations/retrievers/index.mdx` | Rows in External index table + card in All retrievers |
@@ -165,5 +167,6 @@ uv run pytest tests/unit_tests/test_readme_parity.py tests/unit_tests/test_readm
 ## Notes
 
 - Fixes to `README.md` and Python source are applied directly by this skill. Fixes to `langchain-docs` files must be made there separately (this repo has no write access to the remote).
-- To sync the provider hub after fixing README.md: run `/sync-langchain-docs`.
-- To fix tools/retrievers deep-dive pages: edit them directly in `langchain-ai/docs`.
+- To align langchain-docs pages with README.md after fixes here: run `/sync-langchain-docs`.
+- To fix detail pages: edit them directly in `langchain-ai/docs`.
+- **LangChain 1.x imports:** Flag `langchain.documents`, `langchain.prompts`, `langchain.runnables`, or `langchain.output_parsers` as stale — those modules do not exist. Use `langchain.messages` for message types and `langchain_core` for `Document`, prompts, parsers, and runnables. See the `sync-langchain-docs` skill for the full table.

.claude/skills/sync-langchain-docs/SKILL.md

@@ -1,23 +1,51 @@
 ---
 name: sync-langchain-docs
-description: "Keep all Diffbot docs in sync: README.md in this repo and the three LangChain docs pages (providers, tools, retrievers). Use when code changes, when any doc page drifts, or when the user says: sync langchain docs, update the diffbot docs, sync the integration pages."
+description: "Keep all Diffbot docs in sync: README.md in this repo and the LangChain docs pages (providers, tools, retrievers, chat, document loaders). Use when code changes, when any doc page drifts, or when the user says: sync langchain docs, update the diffbot docs, sync the integration pages."
 allowed-tools: Bash(python3:*), Bash(git:*), Bash(gh:*), Bash(make:*), Read, Edit, Write
 ---
 
 # Sync Diffbot docs across all pages
 
 ## What this skill does
 
-Four documents describe the `langchain-diffbot` package. Keep them all accurate and consistent — update whichever ones need it, not just one.
+Six documents describe the `langchain-diffbot` package. Keep them all accurate and consistent — update whichever ones need it, not just one.
 
 | File | Audience | Owns |
 |------|----------|------|
 | `README.md` (this repo) | GitHub / PyPI readers | Complete package reference — install, auth, all classes, examples |
 | `providers/diffbot.mdx` (langchain-docs) | Docs site visitors landing on Diffbot | Overview only: install, auth, components table, links to detail pages |
 | `tools/diffbot.mdx` (langchain-docs) | Docs site visitors looking for tools | Full tool documentation with examples |
 | `retrievers/diffbot.mdx` (langchain-docs) | Docs site visitors looking for retrievers | Full retriever documentation with examples |
+| `chat/diffbot.mdx` (langchain-docs) | Docs site visitors looking for chat models | `ChatDiffbot` instantiation, invocation, streaming, chaining |
+| `document_loaders/diffbot.mdx` (langchain-docs) | Docs site visitors looking for loaders | `DiffbotExtractLoader`, `DiffbotCrawlLoader` |
 
-**Link, don't duplicate within langchain-docs.** The provider hub names every class and links to the tools/retrievers pages; the detail pages don't repeat install/auth. The README is for a different audience and channel (GitHub/PyPI) — it can be complete without violating this rule.
+**Link, don't duplicate within langchain-docs.** The provider hub names every class and links to detail pages; detail pages don't repeat install/auth. The README is for a different audience and channel (GitHub/PyPI) — it can be complete without violating this rule.
+
+## LangChain import rules (LangChain 1.x)
+
+Assume readers use current `langchain` and `langchain-core` releases. The slim `langchain` package (1.3+) only ships `agents`, `chat_models`, `embeddings`, `messages`, `rate_limiters`, and `tools`. Paths like `langchain.documents` or `langchain.prompts` do not exist and will fail if copied.
+
+Use this split in **every** README and MDX example:
+
+| Symbol | Import from |
+|--------|-------------|
+| `HumanMessage`, `AIMessage`, `ToolMessage`, other message types | `langchain.messages` |
+| `create_agent`, `tool`, `init_chat_model`, agent middleware | `langchain.agents`, `langchain.tools`, `langchain.chat_models` |
+| `Document`, `ChatPromptTemplate`, `StrOutputParser`, `RunnablePassthrough`, LCEL primitives | `langchain_core.*` |
+
+Examples:
+
+```python
+from langchain.messages import HumanMessage
+from langchain_core.documents import Document
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.runnables import RunnablePassthrough
+```
+
+**Docs CI note:** The langchain-docs **Check PR Imports** job only flags `langchain_core` imports that `langchain` actually re-exports (mostly `messages` and `tools`). `langchain_core.documents` and `langchain_core.prompts` are allowed and correct — do not "fix" them to nonexistent `langchain.*` paths.
+
+**Package source code** (`langchain_diffbot/*.py`) continues to import from `langchain_core` directly; these rules apply to user-facing documentation and examples only.
 
 ## Where the docs repo lives
 
@@ -33,20 +61,22 @@ If that errors, ask the user for its path or to clone `git@github.com:langchain-
 
 <Steps>
 
-### Read all four files
+### Read all six files
 
 Read every file before touching any of them:
 
 - `README.md`
 - `$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)/src/oss/python/integrations/providers/diffbot.mdx`
 - `$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)/src/oss/python/integrations/tools/diffbot.mdx`
 - `$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)/src/oss/python/integrations/retrievers/diffbot.mdx`
+- `$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)/src/oss/python/integrations/chat/diffbot.mdx`
+- `$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)/src/oss/python/integrations/document_loaders/diffbot.mdx`
 
 Also check what triggered the sync — git diff, the user's description, or a specific change — so you know what actually changed and can limit edits to what's necessary.
 
 ### Identify what needs updating
 
-For each of the four files, decide independently whether it needs a change. Common triggers:
+For each file, decide independently whether it needs a change. Common triggers:
 
 - **New or renamed class** → update the components table in README + provider hub; add documentation to the appropriate detail page (tools or retrievers); update any import examples.
 - **Behavior change to a tool or retriever** → update README + the matching detail page.
@@ -69,7 +99,7 @@ uv run pytest tests/unit_tests/test_readme_parity.py tests/unit_tests/test_readm
 
 **providers/diffbot.mdx** — hub only. Structure:
 1. Frontmatter (`title`, `description`)
-2. Sync comment (keep it — see current file for wording)
+2. Maintenance comment (keep it — see current file for wording; README and MDX pages are peers, not source/derived)
 3. Short intro + API → class mapping table
 4. `## Installation` as `<CodeGroup>` with `pip` and `uv` tabs
 5. `## Authentication` — prose + `db = Diffbot(...)` snippet only; no usage examples
@@ -80,12 +110,27 @@ uv run pytest tests/unit_tests/test_readme_parity.py tests/unit_tests/test_readm
 
 **retrievers/diffbot.mdx** — full retriever docs. Include both retriever classes, output shaping, LCEL chain usage. Link back to the provider hub for install/auth. Do not repeat tool content.
 
+**chat/diffbot.mdx** — `ChatDiffbot` only. Instantiation, invocation, streaming, chaining. Follow the LangChain import rules above (`langchain.messages`, `langchain_core.prompts`).
+
+**document_loaders/diffbot.mdx** — `DiffbotExtractLoader` and `DiffbotCrawlLoader` only. Link back to the provider hub for install/auth.
+
 MDX formatting rules (Vale enforces these — violations block the docs CI):
 - Em dashes: no surrounding spaces (`word—word`, not `word — word`)
 - `prebuilt` not `pre-built`
 - Install blocks: `<CodeGroup>` with `pip` and `uv` tabs
 - Relative links to this repo become absolute `https://github.com/diffbot/langchain-diffbot/...` URLs
 
+### Validate imports and prose in langchain-docs
+
+If you changed any MDX in the docs repo, run the import checker (flags only `langchain_core` imports that should be `langchain.*` re-exports) and Vale:
+
+```bash
+DOCS="$(python3 .claude/skills/sync-langchain-docs/sync.py --repo)"
+cd "$DOCS"
+uv run scripts/check_pr_imports.py
+make lint_prose FILES="<space-separated changed .mdx paths>"
+```
+
 ### Lint every changed MDX file
 
 The docs repo has its own Vale setup. Run it against each changed MDX:

.claude/skills/sync-langchain-docs/sync.py

@@ -1,11 +1,10 @@
 #!/usr/bin/env python3
-"""Resolve the Diffbot provider page inside a local langchain-ai/docs checkout.
+"""Resolve Diffbot docs paths inside a local langchain-ai/docs checkout.
 
-README.md in this repo is the single source of truth for the Diffbot provider
-page on the LangChain docs site; the `sync-langchain-docs` skill *generates* the
-page (`.mdx`) from it. The generation itself is agent-driven (prose → house
-style), so this script does not copy anything — it just resolves where the page
-lives, so the skill never hardcodes the path:
+README.md and the LangChain docs pages are maintained in parallel—not generated
+from one another. Import rules for examples (LangChain 1.x) live in
+`.claude/skills/sync-langchain-docs/SKILL.md`. This script only resolves where
+those pages live so the skill never hardcodes paths:
 
     # Resolve the docs repo from --docs-repo, $LANGCHAIN_DOCS_REPO, or the
     # sibling ../langchain-docs, then print:

CLAUDE.md

@@ -53,17 +53,21 @@ Methods like `crawl_list_jobs`, `crawl_get_job`, `crawl_delete_job`, and `dql_re
 
 ## Documentation
 
-Four documents describe this package; keep them all accurate when things change:
+Six documents describe this package; keep them all accurate when things change:
 
 | File | Audience | Owns |
 |------|----------|------|
 | `README.md` | GitHub / PyPI readers | Complete reference — install, auth, all classes, examples |
 | `providers/diffbot.mdx` (langchain-docs) | Docs site — Diffbot landing page | Overview only: install, auth, components table, links to detail pages |
 | `tools/diffbot.mdx` (langchain-docs) | Docs site — tools reference | Full tool documentation with examples |
 | `retrievers/diffbot.mdx` (langchain-docs) | Docs site — retrievers reference | Full retriever documentation with examples |
+| `chat/diffbot.mdx` (langchain-docs) | Docs site — chat model reference | `ChatDiffbot` usage |
+| `document_loaders/diffbot.mdx` (langchain-docs) | Docs site — loader reference | Extract and crawl loaders |
 
 The langchain-docs pages link to each other rather than duplicate content. Use the `sync-langchain-docs` skill to update whichever pages need it. The local docs checkout is expected at sibling `../langchain-docs` (overridable via `$LANGCHAIN_DOCS_REPO`).
 
+**LangChain 1.x imports in docs:** Use `langchain.messages` for message types; use `langchain_core` for `Document`, prompts, output parsers, and runnables. Do not use `langchain.documents` or `langchain.prompts` — those modules do not exist in current `langchain` releases. See the `sync-langchain-docs` skill for the full table and CI notes.
+
 **The docs repo leads on prose quality.** If validation fixes are made to the docs pages (Vale, CI, editorial review), propagate those improvements back to `README.md` — don't let the README drift to a lower standard.
 
 Two unit suites keep the README in lockstep with the package: `tests/unit_tests/test_readme_parity.py` (the `## Components reference` table matches `__all__`, every class is documented, every example builds a client) and `tests/unit_tests/test_readme_examples.py` (every executable example runs under `respx` mocks; the crawl example is documented but not executed — flagged via `tests/readme.py`'s `is_executable`). `tests/integration_tests/test_readme_examples.py` runs the same blocks live.

tests/readme.py

@@ -1,12 +1,12 @@
 """Shared helpers for testing the python code blocks in README.md.
 
-README.md is the single source of truth for this package's docs: the LangChain
-provider page is *generated from it* by the `sync-langchain-docs` skill. Three
-suites consume these helpers: `unit_tests/test_readme_examples.py` runs the
-blocks against respx mocks (deterministic, no token),
-`integration_tests/test_readme_examples.py` runs them against the live Diffbot
-API, and `unit_tests/test_readme_parity.py` checks the page stays in lockstep
-with the package surface (`__all__`). The execution suites decide which blocks
+README.md is maintained alongside the LangChain docs pages (provider hub, tools,
+retrievers, chat). Three suites consume these helpers:
+`unit_tests/test_readme_examples.py` runs the blocks against respx mocks
+(deterministic, no token), `integration_tests/test_readme_examples.py` runs them
+against the live Diffbot API, and `unit_tests/test_readme_parity.py` checks the
+README components table stays in lockstep with the package surface (`__all__`).
+The execution suites decide which blocks
 to run by inspecting each block's imports and whether it is executable.
 """
 

tests/unit_tests/test_readme_examples.py

@@ -1,11 +1,9 @@
 """Execute the Python code blocks in README.md against mocked Diffbot endpoints.
 
-README.md is the single source of truth for this package's docs: the
-docs.langchain.com provider page is *generated from it* by the
-`sync-langchain-docs` skill. This guards against the README (and therefore that
-page) drifting from the package: a renamed class, a changed kwarg, or a wrong
-import surfaces here as a failing exec rather than as a user copy-pasting a
-broken snippet.
+README.md is maintained alongside the LangChain docs pages. This guards against
+the README drifting from the package: a renamed class, a changed kwarg, or a
+wrong import surfaces here as a failing exec rather than as a user copy-pasting
+a broken snippet.
 
 The blocks are run under `respx` mocks of every Diffbot endpoint — the same
 canned-response style the rest of `unit_tests/` uses — so the suite stays

tests/unit_tests/test_readme_parity.py

@@ -1,9 +1,8 @@
 """Keep README.md in lockstep with the package surface.
 
-README.md is the single source of truth for this package's docs; the LangChain
-provider page is generated from it by the `sync-langchain-docs` skill. These
-tests fail if the README drifts from `__all__`, so a class added, renamed, or
-removed cannot silently miss the docs (and therefore the published page).
+README.md is maintained alongside the LangChain docs pages. These tests fail if
+the README drifts from `__all__`, so a class added, renamed, or removed cannot
+silently miss the README components table.
 """
 
 from __future__ import annotations