feat(seo): add per-page descriptions and canonical metadata (#1531)

## Why
Every page currently receives the same site-wide description, and
`og:url` always points at the homepage. This makes chapter and
static-page metadata misleading to search engines and link previews. It
also has to be correct on both `essentialcsharp.com` and
`dev.essentialcsharp.com`.

## What changed
- Give public MVC actions page-specific descriptions. Chapter pages read
ParserCli-generated chapter descriptions when present, with a
heading-based fallback when the generated file is absent.
- Emit exactly one canonical `<link>` and a matching `og:url` on every
successful, non-Identity page, excluding query strings. The origin
always comes from `SiteSettings:BaseUrl`.
- Chapter pages keep the path from the parser-generated canonical (so
aliases such as `/hello-world` point to `/introducing-c`), re-based from
the parser's hardcoded production origin onto the configured origin.
  - The parser's own canonical tag is stripped from the injected head.
- Share the selected description across HTML, Open Graph, and Twitter
metadata, while retaining the layout fallback for pages without an
override.
- Cover canonical and description output for both production and dev
origins with functional tests; document the metadata contract.

## Supersedes #1130
This reconciles #1130 and addresses its
review findings:
- **Duplicate/conflicting canonical on aliases:** fixed. There is one
canonical, and it points to the primary page.
- **`X-Forwarded-Host` without an allowlist:** avoided. The request
`Host` is not used for canonicals, so `XForwardedHost` is not enabled.
That flag would also have let a trusted proxy poison emailed
reset/confirmation links.
- **`XForwardedHost` is a no-op under
`ASPNETCORE_FORWARDEDHEADERS_ENABLED=true`:** moot for the same reason.

Also addresses this PR's own review comment: metadata tests now pin
`SiteSettings:BaseUrl` instead of depending on
`appsettings.Development.json`.

## Deployment note
Each environment must set `SiteSettings__BaseUrl` to its own origin, for
example `https://dev.essentialcsharp.com` for dev. Production already
depends on this for the www→apex redirect.

## Validation
The full Web test suite passes locally (186 tests), built with
`/p:SkipFrontendBuild=true`. The local frontend build is blocked by an
unrelated stale `@zxcvbn-ts/core` install.

Author: BenjaminMichaelis

EssentialCSharp.Web.Tests/ChapterDescriptionServiceTests.cs

@@ -0,0 +1,45 @@
+using EssentialCSharp.Web.Services;
+using Microsoft.AspNetCore.Hosting;
+using Moq;
+
+namespace EssentialCSharp.Web.Tests;
+
+public class ChapterDescriptionServiceTests
+{
+    [Test]
+    public async Task GeneratedDescriptionsOverrideHeadingFallback()
+    {
+        string directory = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString());
+        Directory.CreateDirectory(Path.Combine(directory, "Chapters"));
+        try
+        {
+            File.WriteAllText(Path.Combine(directory, "Chapters", "sitemap.descriptions.json"),
+                """[{"Keys":["example-page","alias"],"Value":"A tailored chapter description."}]""");
+            Mock<IWebHostEnvironment> environment = new();
+            environment.SetupGet(e => e.ContentRootPath).Returns(directory);
+            ChapterDescriptionService service = new(environment.Object);
+            SiteMapping mapping = CreateMapping("alias", "Examples");
+
+            await Assert.That(service.GetDescription(mapping)).IsEqualTo("A tailored chapter description.");
+            await Assert.That(service.GetDescription(CreateMapping("missing", "Other Examples"))).Contains("Other Examples");
+        }
+        finally
+        {
+            Directory.Delete(directory, recursive: true);
+        }
+    }
+
+    [Test]
+    public async Task WithoutGeneratedDescriptionsUsesPageHeading()
+    {
+        Mock<IWebHostEnvironment> environment = new();
+        environment.SetupGet(e => e.ContentRootPath).Returns(Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString()));
+        ChapterDescriptionService service = new(environment.Object);
+
+        await Assert.That(service.GetDescription(CreateMapping("sample", "Sample Heading"))).Contains("Sample Heading");
+    }
+
+    private static SiteMapping CreateMapping(string key, string heading) =>
+        new([key], key, ["Chapters", "01", "Pages", "01.html"], 1, 1, 1,
+            "Chapter One", heading, null, 1);
+}

EssentialCSharp.Web.Tests/FunctionalTests.cs

@@ -1,4 +1,9 @@
 using System.Net;
+using HtmlAgilityPack;
+using EssentialCSharp.Web.Services;
+using Microsoft.AspNetCore.Hosting;
+using Microsoft.AspNetCore.Mvc.Testing;
+using Microsoft.Extensions.DependencyInjection;
 
 namespace EssentialCSharp.Web.Tests;
 
@@ -42,6 +47,72 @@ public async Task WhenPagesAreAccessed_TheyReturnHtml(string relativeUrl)
         await Assert.That(content).Contains("<html", StringComparison.OrdinalIgnoreCase);
     }
 
+    [Test]
+    [Arguments("/about?rid=abc", "https://essentialcsharp.com/about", "Learn about Essential C#")]
+    [Arguments("/home", "https://essentialcsharp.com/home", "Explore Essential C#")]
+    [Arguments("/cookie-policy", "https://essentialcsharp.com/cookie-policy", "uses cookies")]
+    [Arguments("/guidelines", "https://essentialcsharp.com/guidelines", "coding guidelines")]
+    [Arguments("/announcements", "https://essentialcsharp.com/announcements", "announcements")]
+    [Arguments("/TermsOfService", "https://essentialcsharp.com/TermsOfService", "terms of service")]
+    [Arguments("/mcp-setup", "https://essentialcsharp.com/mcp-setup", "Model Context Protocol")]
+    [Arguments("/introducing-c", "https://essentialcsharp.com/introducing-c", "Introducing C#")]
+    [Arguments("/hello-world", "https://essentialcsharp.com/introducing-c", "Hello")]
+    public async Task PagesHaveOneCanonicalAndPageSpecificDescriptions(string path, string expectedCanonical, string expectedDescription)
+    {
+        await using WebApplicationFactory<Program> factory = CreateFactoryWithBaseUrl("https://essentialcsharp.com");
+        await AssertPageMetadata(factory, path, expectedCanonical, expectedDescription);
+    }
+
+    [Test]
+    [Arguments("/about", "https://dev.essentialcsharp.com/about", "Learn about Essential C#")]
+    [Arguments("/introducing-c", "https://dev.essentialcsharp.com/introducing-c", "Introducing C#")]
+    [Arguments("/hello-world?rid=abc", "https://dev.essentialcsharp.com/introducing-c", "Hello")]
+    public async Task CanonicalUrlsUseConfiguredDeploymentOrigin(string path, string expectedCanonical, string expectedDescription)
+    {
+        await using WebApplicationFactory<Program> factory = CreateFactoryWithBaseUrl("https://dev.essentialcsharp.com/");
+        await AssertPageMetadata(factory, path, expectedCanonical, expectedDescription);
+    }
+
+    private WebApplicationFactory<Program> CreateFactoryWithBaseUrl(string baseUrl) =>
+        Factory.Inner.WithWebHostBuilder(builder => builder.ConfigureServices(services =>
+            services.PostConfigure<SiteSettings>(settings => settings.BaseUrl = baseUrl)));
+
+    private static async Task AssertPageMetadata(WebApplicationFactory<Program> factory, string path, string expectedCanonical, string expectedDescription)
+    {
+        using HttpClient client = factory.CreateClient(new WebApplicationFactoryClientOptions { AllowAutoRedirect = false });
+        using HttpResponseMessage response = await GetFollowingGetRedirectsAsync(client, path);
+        await Assert.That(response.StatusCode).IsEqualTo(HttpStatusCode.OK);
+
+        HtmlDocument document = new();
+        document.LoadHtml(await response.Content.ReadAsStringAsync());
+        HtmlNode head = document.DocumentNode.SelectSingleNode("//head") ?? throw new InvalidOperationException("Missing head element");
+        HtmlNodeCollection canonicals = head.SelectNodes("link[@rel='canonical']") ?? throw new InvalidOperationException("Missing canonical link");
+        await Assert.That(canonicals.Count).IsEqualTo(1);
+        await Assert.That(canonicals[0].GetAttributeValue("href", "")).IsEqualTo(expectedCanonical);
+        await Assert.That((head.SelectSingleNode("meta[@property='og:url']") ?? throw new InvalidOperationException("Missing OG URL")).GetAttributeValue("content", "")).IsEqualTo(expectedCanonical);
+
+        string description = (head.SelectSingleNode("meta[@name='description']") ?? throw new InvalidOperationException("Missing description")).GetAttributeValue("content", "");
+        await Assert.That(description).Contains(expectedDescription);
+        await Assert.That((head.SelectNodes("meta[@name='description']") ?? throw new InvalidOperationException("Missing description")).Count).IsEqualTo(1);
+        await Assert.That((head.SelectSingleNode("meta[@property='og:description']") ?? throw new InvalidOperationException("Missing OG description")).GetAttributeValue("content", "")).IsEqualTo(description);
+        await Assert.That((head.SelectSingleNode("meta[@name='twitter:description']") ?? throw new InvalidOperationException("Missing Twitter description")).GetAttributeValue("content", "")).IsEqualTo(description);
+    }
+
+    [Test]
+    public async Task NotFoundPageKeepsDescriptionFallbackWithoutCanonical()
+    {
+        using HttpClient client = CreateClientWithoutAutoRedirect();
+        using HttpResponseMessage response = await GetFollowingGetRedirectsAsync(client, "/non-existing-page1234");
+        await Assert.That(response.StatusCode).IsEqualTo(HttpStatusCode.NotFound);
+
+        HtmlDocument document = new();
+        document.LoadHtml(await response.Content.ReadAsStringAsync());
+        HtmlNode head = document.DocumentNode.SelectSingleNode("//head") ?? throw new InvalidOperationException("Missing head element");
+        await Assert.That(head.SelectNodes("link[@rel='canonical']")).IsNull();
+        await Assert.That((head.SelectSingleNode("meta[@name='description']") ?? throw new InvalidOperationException("Missing description")).GetAttributeValue("content", ""))
+            .Contains("Accelerate your development knowledge");
+    }
+
     [Test]
     public async Task WhenTheApplicationStarts_NonExistingPage_GivesCorrectStatusCode()
     {

EssentialCSharp.Web/Controllers/HomeController.cs

@@ -13,7 +13,7 @@
 
 namespace EssentialCSharp.Web.Controllers;
 
-public class HomeController(ILogger<HomeController> logger, IWebHostEnvironment hostingEnvironment, ISiteMappingService siteMappingService, IHttpContextAccessor httpContextAccessor, IRouteConfigurationService routeConfigurationService, IOptions<SiteSettings> siteSettings) : BaseController(routeConfigurationService, httpContextAccessor)
+public class HomeController(ILogger<HomeController> logger, IWebHostEnvironment hostingEnvironment, ISiteMappingService siteMappingService, ChapterDescriptionService chapterDescriptionService, IHttpContextAccessor httpContextAccessor, IRouteConfigurationService routeConfigurationService, IOptions<SiteSettings> siteSettings) : BaseController(routeConfigurationService, httpContextAccessor)
 {
     [EnableRateLimiting("content")]
     public IActionResult Index()
@@ -40,10 +40,29 @@ public IActionResult Index()
                     styleNode.SetAttributeValue("nonce", nonce);
                 }
             }
+            // The parser emits a canonical with a hardcoded production origin. Keep its path (so alias routes
+            // still point at the primary page) but re-base it on this deployment's configured origin, and
+            // remove it from the injected head so the layout renders exactly one canonical.
+            string canonicalPath = "/" + siteMapping.PrimaryKey;
+            foreach (HtmlNode canonicalNode in doc.DocumentNode.SelectNodes("/html/head/link[@rel='canonical']")?.ToList() ?? [])
+            {
+                string href = canonicalNode.GetAttributeValue("href", "");
+                if (Uri.TryCreate(href, UriKind.Absolute, out Uri? absolute))
+                {
+                    canonicalPath = absolute.AbsolutePath;
+                }
+                else if (href.StartsWith('/'))
+                {
+                    canonicalPath = href.Split('?', '#')[0];
+                }
+                canonicalNode.Remove();
+            }
             string headHtml = doc.DocumentNode.Element("html")?.Element("head")?.InnerHtml ?? string.Empty;
             string html = doc.DocumentNode.Element("html")?.Element("body")?.InnerHtml ?? string.Empty;
 
             ViewBag.PageTitle = siteMapping.IndentLevel is 0 ? siteMapping.ChapterTitle + " " + siteMapping.RawHeading : siteMapping.RawHeading;
+            ViewData["Description"] = chapterDescriptionService.GetDescription(siteMapping);
+            ViewData["CanonicalUrl"] = $"{siteSettings.Value.BaseUrl.TrimEnd('/')}{Request.PathBase}{canonicalPath}";
             ViewBag.NextPage = FlipPage(siteMapping!.ChapterNumber, siteMapping.PageNumber, true);
             ViewBag.CurrentPageKey = siteMapping.PrimaryKey;
             ViewBag.PreviousPage = FlipPage(siteMapping.ChapterNumber, siteMapping.PageNumber, false);
@@ -63,40 +82,46 @@ public IActionResult Index()
     public IActionResult TermsOfService()
     {
         ViewBag.PageTitle = "Terms Of Service";
+        ViewData["Description"] = "Read the terms of service for using Essential C# and its online C# learning resources.";
         return View();
     }
 
     [Route("/cookie-policy", Name = "CookiePolicy")]
     public IActionResult CookiePolicy()
     {
         ViewBag.PageTitle = "Cookie Policy";
+        ViewData["Description"] = "Learn how Essential C# uses cookies and manages your privacy preferences.";
         return View();
     }
 
     [Route("/Announcements", Name = "Announcements")]
     public IActionResult Announcements()
     {
         ViewBag.PageTitle = "Announcements";
+        ViewData["Description"] = "Read the latest Essential C# announcements, updates, and news about the online C# guide.";
         return View();
     }
 
     [Route("/about", Name = "about")]
     public IActionResult About()
     {
         ViewBag.PageTitle = "About";
+        ViewData["Description"] = "Learn about Essential C#, the free online C# tutorial and reference by Mark Michaelis and Benjamin Michaelis.";
         return View();
     }
 
     [Route("/home", Name = "home")]
     public IActionResult Home()
     {
+        ViewData["Description"] = "Explore Essential C#, a free online C# tutorial and reference with practical examples for developers.";
         return View();
     }
 
     [Route("/guidelines", Name = "guidelines")]
     public IActionResult Guidelines()
     {
         ViewBag.PageTitle = "Coding Guidelines";
+        ViewData["Description"] = "Explore Essential C# coding guidelines and best practices for writing clear, maintainable C#.";
         FileInfo fileInfo = new(Path.Join(hostingEnvironment.ContentRootPath, "Guidelines", "guidelines.json"));
         if (!fileInfo.Exists)
         {

EssentialCSharp.Web/Controllers/McpSetupController.cs

@@ -21,6 +21,7 @@ public McpSetupController(IRouteConfigurationService routeConfigurationService,
     public IActionResult Index()
     {
         ViewBag.PageTitle = "MCP Setup";
+        ViewData["Description"] = "Connect AI coding tools to Essential C# with the Model Context Protocol setup guide.";
         var toolInfos = _tools
             .OrderBy(t => t.ProtocolTool.Name)
             .Select(t =>

EssentialCSharp.Web/Program.cs

@@ -267,6 +267,7 @@ private static void Main(string[] args)
         builder.Services.AddOutputCache();
         builder.Services.AddCaptchaService(builder.Configuration.GetSection(CaptchaOptions.CaptchaSender));
         builder.Services.AddSingleton<ISiteMappingService, SiteMappingService>();
+        builder.Services.AddSingleton<ChapterDescriptionService>();
         builder.Services.AddSingleton<IRouteConfigurationService, RouteConfigurationService>();
         builder.Services.AddSingleton<IListingSourceCodeService, ListingSourceCodeService>();
         builder.Services.AddSingleton<IBookToolQueryService, BookToolQueryService>();

EssentialCSharp.Web/Services/ChapterDescriptionService.cs

@@ -0,0 +1,47 @@
+using System.Text.Json;
+
+namespace EssentialCSharp.Web.Services;
+
+public sealed class ChapterDescriptionService
+{
+    private readonly Dictionary<string, string> _Descriptions;
+
+    public ChapterDescriptionService(IWebHostEnvironment environment)
+    {
+        string path = Path.Combine(environment.ContentRootPath, "Chapters", "sitemap.descriptions.json");
+        if (!File.Exists(path))
+        {
+            _Descriptions = new Dictionary<string, string>();
+            return;
+        }
+
+        using FileStream stream = File.OpenRead(path);
+        List<ChapterDescription> entries = JsonSerializer.Deserialize<List<ChapterDescription>>(stream)
+            ?? throw new InvalidDataException($"No chapter descriptions found in {path}");
+        Dictionary<string, string> descriptions = new(StringComparer.OrdinalIgnoreCase);
+        foreach (ChapterDescription entry in entries)
+        {
+            foreach (string key in entry.Keys)
+            {
+                descriptions.Add(key, entry.Value);
+            }
+        }
+        _Descriptions = descriptions;
+    }
+
+    public string GetDescription(SiteMapping mapping)
+    {
+        foreach (string key in mapping.Keys)
+        {
+            if (_Descriptions.TryGetValue(key, out string? description) && !string.IsNullOrWhiteSpace(description))
+            {
+                return description;
+            }
+        }
+
+        string heading = mapping.IndentLevel == 0 ? mapping.ChapterTitle : mapping.RawHeading;
+        return $"Learn about {heading} in Essential C# with practical examples and explanations from the online C# guide.";
+    }
+
+    private sealed record ChapterDescription(List<string> Keys, string Value);
+}

EssentialCSharp.Web/Views/Shared/_Layout.cshtml

@@ -12,24 +12,35 @@
 @inject SignInManager<EssentialCSharpWebUser> SignInManager
 @inject IConfiguration Configuration
 @inject IOptions<CaptchaOptions> _CaptchaOptions
+@inject IOptions<SiteSettings> SiteSettings
 <!DOCTYPE html>
 <html lang="en">
 <head>
     @{
         const string imageUrl = "https://essentialcsharp.com/images/icon.png";
-        const string description = "Accelerate your development knowledge with C# expert Mark Michaelis and Benjamin Michaelis' free, online comprehensive C# tutorial and reference that is updated through C# 11.0";
+        const string defaultDescription = "Accelerate your development knowledge with C# expert Mark Michaelis and Benjamin Michaelis' free, online comprehensive C# tutorial and reference that is updated through C# 11.0";
+        string description = ViewData["Description"] is string pageDescription && !string.IsNullOrWhiteSpace(pageDescription)
+            ? pageDescription
+            : defaultDescription;
+        string canonicalUrl = ViewData["CanonicalUrl"] as string
+            ?? $"{SiteSettings.Value.BaseUrl.TrimEnd('/')}{Context.Request.PathBase}{Context.Request.Path}";
         string title = $"Essential C#{(string.IsNullOrEmpty(ViewBag.PageTitle) ? string.Empty : $": {ViewBag.PageTitle}")}";
     }
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <meta property="og:type" content="website" />
-    <meta property="og:url" content="https://essentialcsharp.com/" />
+    <meta property="og:url" content="@canonicalUrl" />
+    @if (Context.Response.StatusCode < 400 && !Context.Request.Path.StartsWithSegments("/Identity"))
+    {
+        <link rel="canonical" href="@canonicalUrl" />
+    }
     <meta property="og:image:secure_url" content="@imageUrl" />
     <meta property="og:image" content="@imageUrl" />
     <meta name="twitter:image" content="@imageUrl" />
     <title id="page-title">@title</title>
     <meta property="og:image:type" content="image/png" />
-    <meta name="description" property="og:description" content="@description" />
+    <meta name="description" content="@description" />
+    <meta property="og:description" content="@description" />
     <meta name="twitter:description" content="@description" />
     <meta name="twitter:card" content="summary" />
     <meta name="twitter:site" content="@@IntelliTect" />

README.md

@@ -10,4 +10,10 @@ For any bugs, questions, or anything else with specifically the code found insid
 
 For information on setting up your local development environment, please see the [Getting Started Guide](docs/getting-started.md).
 
+## Page metadata
+
+The shared layout uses `ViewData["Description"]` for the HTML, Open Graph, and Twitter descriptions; pages without an override use the site-wide fallback. Public page actions set their own descriptions. Chapter pages use `Chapters/sitemap.descriptions.json` when available (generated by ParserCli), or a description based on the chapter heading when the file is absent. Views can also set `ViewData["Description"]` before the layout renders.
+
+The layout renders exactly one canonical link (and a matching `og:url`) for every successful, non-Identity page. Its origin always comes from `SiteSettings:BaseUrl`, so each deployment must set it to its own origin (for example `SiteSettings__BaseUrl=https://dev.essentialcsharp.com` for the dev environment). The request `Host` header is deliberately not used, to avoid host-header poisoning. For chapter pages, the path from the parser-generated canonical is kept (so alias routes such as `/hello-world` point to the primary page) and re-based onto the configured origin; the parser's own `<link rel="canonical">` is removed from the injected head. Other pages use the request path. Query strings are excluded.
+
 Please use issues or discussions to report issues found.