Skip to content

Latest commit

 

History

History
545 lines (447 loc) · 23.7 KB

File metadata and controls

545 lines (447 loc) · 23.7 KB

☁️ Azure Log Analytics Workspace Terraform Module

Manage a Log Analytics workspace with its solutions and data-export rules as one unit, private by default, on hashicorp/azurerm ~> 4.0.

Terraform azurerm module type resources

🧩 Overview

  • 📊 Creates one azurerm_log_analytics_workspace — the central log/metric store for diagnostics and Application Insights.
  • 🧩 Installs azurerm_log_analytics_solution entries (for example ContainerInsights) from a keyed map.
  • 📤 Configures azurerm_log_analytics_data_export_rule entries to a storage account or event hub.
  • 🔒 Secure by default: internet ingestion and query are off (private via Azure Monitor Private Link Scope), SKU PerGB2018, retention 30 days.

💡 Why it matters: the workspace is the sink nearly every other module wires diagnostics into. A private-by-default, tagged workspace with explicit retention keeps observability data governed and off the public internet.

❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!

🗺️ Where this fits in the family

flowchart LR
  rg["terraform-azurerm-resource-group"]
  ws["terraform-azurerm-log-analytics-workspace: the family keystone, also owns data-export rules and solutions"]
  cluster["terraform-azurerm-log-analytics-cluster: dedicated cluster for CMK and commitment-tier pricing"]
  cmk["terraform-azurerm-log-analytics-cluster-customer-managed-key: bind the key AFTER the identity is granted on the vault"]
  qp["terraform-azurerm-log-analytics-query-pack"]
  qpq["terraform-azurerm-log-analytics-query-pack-query: name is a GUID, display_name is the label"]
  ls["terraform-azurerm-log-analytics-linked-service: Automation Account OR Cluster, two unrelated links in one resource"]
  lsa["terraform-azurerm-log-analytics-linked-storage-account: redirect ONE data class, by Resource ID, NO keys"]
  si["terraform-azurerm-log-analytics-storage-insights: READ storage. REQUIRES a full-control account KEY."]
  ss["terraform-azurerm-log-analytics-saved-search: query or reusable FUNCTION. Every field force-new."]
  wt["terraform-azurerm-log-analytics-workspace-table: per-table retention and PLAN. The only one of the three with plan."]
  wtcl["terraform-azurerm-log-analytics-workspace-table-custom-log: CREATES a _CL table and owns its columns"]
  wtms["terraform-azurerm-log-analytics-workspace-table-microsoft: six Microsoft tables only. NO plan argument."]
  dwe["terraform-azurerm-log-analytics-datasource-windows-event: LEGACY agent path"]
  dwp["terraform-azurerm-log-analytics-datasource-windows-performance-counter: LEGACY agent path"]
  aa["terraform-azurerm-automation-account"]
  sa["terraform-azurerm-storage-account"]
  kv["terraform-azurerm-key-vault: soft delete AND purge protection REQUIRED"]
  dcr["terraform-azurerm-monitor-data-collection-rule: the MODERN replacement for both datasource modules, and how data reaches a _CL table"]
  ds["terraform-azurerm-monitor-diagnostic-setting: the KEYLESS alternative to storage-insights"]

  rg -->|"resource_group_name, location"| ws
  ws -->|"id, BY ID"| lsa
  ws -->|"id, BY ID"| si
  ws -->|"id as log_analytics_workspace_id"| ss
  ws -->|"id, BY ID"| wt
  ws -->|"id, BY ID"| wtcl
  ws -->|"id, BY ID"| wtms
  ws -->|"id, BY ID"| ls
  ws -->|"name plus resource_group_name, BY NAME"| dwe
  ws -->|"name plus resource_group_name, BY NAME"| dwp
  qp -->|"id, BY ID"| qpq
  aa -->|"id as read_access_id"| ls
  cluster -->|"id as write_access_id, workspace sku must be LACluster"| ls
  cluster -->|"id, plus identity_principal_id feeds the vault grant"| cmk
  kv -->|"key versionless_id, prefer unpinned"| cmk
  sa -->|"ids, no keys"| lsa
  sa -->|"id PLUS a full-control account key"| si
  dcr -->|"prefer this for anything new"| dwe
  dcr -->|"prefer this for anything new"| dwp
  dcr -->|"custom stream writes into the table"| wtcl
  ds -->|"prefer this to avoid the key"| si
  wt -->|"OVERLAP: both own retention on the same table. Pick ONE owner."| wtms

  classDef me fill:#0078D4,stroke:#004578,color:#fff;
  classDef keystone fill:#004578,stroke:#001f3f,color:#fff;
  classDef sib fill:#eef2f7,stroke:#b8c4d0,color:#1b1b1b;
  class ws keystone;
  class cluster,qp,ls,lsa,si,ss,wt,wtcl,wtms,cmk,qpq,dwe,dwp me;
  class rg,aa,sa,kv,dcr,ds sib;
Loading

🧬 What this module builds

flowchart LR
  I1["name / location / resource_group_name"]
  I2["sku = PerGB2018, retention = 30"]
  I3["internet ingestion/query = false (private)"]
  I4["solutions (map)"]
  I5["data_export_rules (map)"]

  LAW["azurerm_log_analytics_workspace.this"]
  SOL["azurerm_log_analytics_solution.this<br/>for_each = solutions"]
  EXP["azurerm_log_analytics_data_export_rule.this<br/>for_each = data_export_rules"]

  O1["id / name / workspace_id"]
  O2["solution_ids / data_export_rule_ids"]
  O3["primary_shared_key (sensitive)"]

  I1 --> LAW
  I2 --> LAW
  I3 --> LAW
  I4 --> SOL
  I5 --> EXP
  LAW --> SOL
  LAW --> EXP
  LAW --> O1
  SOL --> O2
  LAW --> O3

  classDef this fill:#0078D4,color:#ffffff,stroke:#004578,stroke-width:2px;
  classDef key fill:#004578,color:#ffffff,stroke:#004578;
  class LAW key;
  class SOL this;
Loading

Resource inventory

Resource Cardinality Role
azurerm_log_analytics_workspace.this 1 (keystone) The workspace.
azurerm_log_analytics_solution.this 0..N (for_each) Installed solutions.
azurerm_log_analytics_data_export_rule.this 0..N (for_each) Continuous export rules.

✅ Provider / Versions

Requirement Value
Terraform >= 1.12.0
Provider hashicorp/azurerm ~> 4.0
Provider block None in this module — the caller configures provider "azurerm" { features {} }, auth, and subscription.

Schema notes that bite

  • name, resource_group_name, and location are immutable.
  • With internet_ingestion_enabled = false (the default), agents must reach the workspace via an Azure Monitor Private Link Scope (AMPLS) — otherwise ingestion fails silently. Set it true for public ingestion.
  • With internet_query_enabled = false (the default), portal/API queries from the public internet are blocked — query via AMPLS/VPN or set it true.
  • Lowering retention_in_days deletes data older than the new window.
  • primary_shared_key / secondary_shared_key are secrets — emitted as sensitive; prefer Entra ID / managed-identity ingestion.

🔑 Required Azure RBAC Roles / Permissions

  • Log Analytics Contributor on the target resource group (workspace, solutions, export rules), plus write access on the data-export destination (storage account / event hub).

Azure Prerequisites

  • The Microsoft.OperationalInsights resource provider registered on the subscription (and Microsoft.OperationsManagement for solutions).
  • An existing resource group.
  • For data export, the destination storage account or event hub already exists.
  • For private access, an Azure Monitor Private Link Scope and linked private endpoint.
  • The caller configures the provider "azurerm" { features {} } block, auth, and subscription; the module declares none of these.

📁 Module Structure

terraform-azurerm-log-analytics-workspace/
├── providers.tf     # required_version + azurerm ~> 4.0; no provider block
├── variables.tf     # name, rg, location, sku, retention, network toggles, solutions, data_export_rules, ...
├── main.tf          # azurerm_log_analytics_workspace.this + for_each solutions + export rules
├── outputs.tf       # id, name, workspace_id, solution_ids, data_export_rule_ids, shared keys (sensitive)
├── README.md        # this document
├── SCOPE.md         # cross-module contract
├── LICENSE          # MIT
└── .gitignore       # canonical Terraform ignore set

⚙️ Quick Start

provider "azurerm" {
  features {}
}

module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-platform-prod-eastus2"
  resource_group_name = "rg-monitor-prod-eastus2"
  location            = "eastus2"
  retention_in_days   = 90
}

ℹ️ The empty call is private (no internet ingestion/query). Pin the module by tag (?ref=v1.0.0), never a branch.

🔌 Cross-Module Contract

Consumes

Input Type From
resource_group_name string terraform-azurerm-resource-group (name)
location string caller / resource group (location)
data_export_rules[*].destination_resource_id string storage account / event hub (id)

Emits

Output Description Consumed by
id Workspace Resource ID diagnostic settings, workspace-based App Insights
name Workspace name diagnostics, tagging
workspace_id Customer ID GUID agents
solution_ids / data_export_rule_ids child maps audit

📚 Example Library

1 · Minimal (private, PerGB2018, 30 days)
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-min-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
}

🔒 Internet ingestion and query are off — reach it via AMPLS.

2 · Longer retention (compliance)
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-audit-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
  retention_in_days   = 730
}
3 · Daily ingestion cap
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-capped-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
  daily_quota_gb      = 5
}
4 · Public ingestion (opt-in)
module "law" {
  source                     = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                       = "law-public-eastus2"
  resource_group_name        = "rg-monitor-eastus2"
  location                   = "eastus2"
  internet_ingestion_enabled = true
  internet_query_enabled     = true
}

⚠️ Opening public ingestion/query removes the private-only guardrail — do so deliberately.

5 · ContainerInsights solution
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-aks-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
  solutions = {
    container = {
      solution_name = "ContainerInsights"
      plan          = { publisher = "Microsoft", product = "OMSGallery/ContainerInsights" }
    }
  }
}
6 · Multiple solutions
solutions = {
  container = { solution_name = "ContainerInsights", plan = { publisher = "Microsoft", product = "OMSGallery/ContainerInsights" } }
  vm        = { solution_name = "VMInsights",        plan = { publisher = "Microsoft", product = "OMSGallery/VMInsights" } }
}
7 · Data export to storage
data_export_rules = {
  to-archive = {
    destination_resource_id = var.archive_storage_account_id
    table_names             = ["AzureActivity", "SigninLogs"]
  }
}
8 · Data export to event hub (disabled)
data_export_rules = {
  to-siem = {
    destination_resource_id = var.eventhub_namespace_id
    table_names             = ["SecurityEvent"]
    enabled                 = false
  }
}
9 · Managed identity (for CMK / secured export)
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-mi-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
  identity            = { type = "SystemAssigned" }
}
10 · Force Entra-only ingestion
module "law" {
  source                       = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                         = "law-entra-eastus2"
  resource_group_name          = "rg-monitor-eastus2"
  location                     = "eastus2"
  local_authentication_enabled = false
}

🔒 Disables workspace-key auth; agents must use Entra ID / managed identity.

11 · Capacity reservation SKU
module "law" {
  source                             = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                               = "law-commit-eastus2"
  resource_group_name                = "rg-monitor-eastus2"
  location                           = "eastus2"
  sku                                = "CapacityReservation"
  reservation_capacity_in_gb_per_day = 100
}
12 · for_each — a workspace per environment
module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  for_each            = toset(["dev", "prod"])
  name                = "law-${each.key}-eastus2"
  resource_group_name = "rg-monitor-eastus2"
  location            = "eastus2"
}
13 · 🏗️ End-to-end composition

A resource group, a workspace, and a diagnostic setting sending a resource's logs into it.

provider "azurerm" {
  features {}
}

module "rg" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"
  name     = "rg-monitor-prod-eastus2"
  location = "eastus2"
}

module "law" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-log-analytics-workspace.git?ref=v1.0.0"
  name                = "law-platform-prod-eastus2"
  resource_group_name = module.rg.name
  location            = module.rg.location
  retention_in_days   = 90
}

module "diag" {
  source                     = "git::https://github.com/microsoftexpert/terraform-azurerm-monitor-diagnostic-setting.git?ref=v1.0.0"
  name                       = "to-law"
  target_resource_id         = var.some_resource_id
  log_analytics_workspace_id = module.law.id
}

💡 Every module in the suite that emits platform logs points its diagnostic setting at module.law.id.

📥 Inputs

Name Type Required Default Description
name string Workspace name. Immutable.
resource_group_name string Containing resource group. Immutable.
location string Azure region. Immutable.
sku string "PerGB2018" Pricing SKU.
retention_in_days number 30 7–730.
internet_ingestion_enabled bool false Public ingestion.
internet_query_enabled bool false Public query.
local_authentication_enabled bool null Workspace-key auth.
daily_quota_gb number null Daily cap.
identity object null Managed identity.
solutions map(object) {} Installed solutions.
data_export_rules map(object) {} Export rules.
tags map(string) {} Tags.
timeouts object null Optional timeouts.
Full variable schemas
variable "solutions" {
  type = map(object({
    solution_name = string
    plan = object({
      publisher      = string
      product        = string
      name           = optional(string)
      promotion_code = optional(string)
    })
  }))
  default = {}
}

variable "data_export_rules" {
  type = map(object({
    name                    = optional(string)
    destination_resource_id = string
    table_names             = list(string)
    enabled                 = optional(bool, true)
  }))
  default = {}
}

🧾 Outputs

Output Description Kind
id The Azure Resource ID of the Log Analytics workspace Passthrough
name The name of the Log Analytics workspace Passthrough
location Azure region the resource is deployed in, in the canonical form Azure uses Passthrough
workspace_id The workspace (customer) ID GUID used by agents and some integrations Passthrough
solution_ids Map of solution key => the solution's Azure Resource ID Derived
data_export_rule_ids Map of data export rule key => the rule's Azure Resource ID Derived
primary_shared_key The workspace primary shared key Passthrough
secondary_shared_key The workspace secondary shared key Passthrough
sku The SKU actually in effect Passthrough
retention_in_days Interactive retention in effect Passthrough
daily_quota_gb The daily ingestion cap in GB, or -1 for unlimited Passthrough
internet_ingestion_enabled Whether logs may be sent over the public internet Passthrough
internet_query_enabled Whether the workspace may be queried over the public internet Passthrough
identity_principal_id The principal ID of the workspace's managed identity, if configured (else null) Derived
identity_tenant_id The tenant ID of the workspace's managed identity, if configured (else null) Derived
is_reachable_from_the_public_internet True when EITHER ingestion or query is open to the public internet Passthrough
changing_sku_usually_destroys_the_workspace Always true, and the most consequential fact about this resource Constant
standard_and_premium_skus_are_accepted_then_refused Always true Constant
capacity_reservation_starts_a_31_day_commitment Always true Constant
daily_quota_stops_collection_rather_than_billing Always true, and it reads the wrong way round Constant
workspace_id_is_not_the_resource_id Always true, and the single most common wiring mistake against this module Constant
deleting_the_workspace_reserves_its_name_for_14_days Always true Constant
shared_keys_are_stored_in_terraform_state Always true Constant

🧠 Architecture Notes

  • Private by default. internet_ingestion_enabled and internet_query_enabled both default to false; use Azure Monitor Private Link Scope, or set them true to open public access. Neither blocks creation.
  • The module owns solutions and export rules. Both are for_each maps keyed by a stable identifier; adding or removing one never re-indexes the rest.
  • Retention is destructive downward. Lowering retention_in_days deletes older data.
  • Secrets flagged. The shared keys are sensitive outputs; Entra ID / managed-identity ingestion is preferred (local_authentication_enabled = false).
  • features {} dependence. No provider {} block here; the caller configures provider "azurerm" { features {} }.

🧱 Design Principles

Concern Secure default (empty call) Opt-out / opt-in
Public ingestion internet_ingestion_enabled = false set true
Public query internet_query_enabled = false set true
Local (key) auth Platform default set local_authentication_enabled = false to force Entra
SKU / retention PerGB2018 / 30 days raise retention; change SKU
Secret outputs Shared keys marked sensitive

🚀 Runbook

cd terraform-azurerm-log-analytics-workspace
terraform init -backend=false
terraform validate
terraform fmt -check
Remove-Item -Recurse -Force .terraform -ErrorAction SilentlyContinue

Pin the module by tag (?ref=v1.0.0), never a branch. Plan-only during authoring; a human runs plan/apply from CI.

🧪 Testing

The offline proof gate — terraform init -backend=false, terraform validate, terraform fmt -check — proves the configuration is type-correct against the pinned azurerm ~> 4.0 schema (including the sku and retention validations) and canonically formatted, with no cloud calls. What it does not exercise: AMPLS reachability, solution/product name validity, and export destination existence — those surface only under terraform plan/apply against real credentials from CI.

💬 Example Output

$ terraform output
id           = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-prod-eastus2/providers/Microsoft.OperationalInsights/workspaces/law-platform-prod-eastus2"
name         = "law-platform-prod-eastus2"
workspace_id = "c0ffee00-1234-5678-9abc-def012345678"
solution_ids = {}

🔍 Troubleshooting

Symptom Cause Fix
No logs arriving internet_ingestion_enabled = false with no AMPLS Configure AMPLS/private endpoint, or set the flag true.
Portal query blocked internet_query_enabled = false Query via AMPLS/VPN, or set it true.
AuthorizationFailed Missing Log Analytics Contributor Grant it on the resource group.
Data older than N days gone retention_in_days lowered Retention reductions are destructive; plan accordingly.
Agent auth fails local_authentication_enabled = false but agent uses a key Use Entra/MSI ingestion, or re-enable local auth.
Solution create fails Wrong plan.product/publisher Use the exact OMSGallery product string.

🔗 Related Docs


💙 "Infrastructure as Code should be standardized, consistent, and secure."