reddit-scraper-n8n-workflow.json

{
  "name": "Extract and score Reddit Scraper & Media API: Posts, Users & Comments data to Google Sheets, CSV, and Slack with Apify",
  "nodes": [
    {
      "parameters": {
        "content": "## Reddit Scraper & Media API: Posts, Users & Comments — Automated Sheets, CSV & Slack Pipeline\n\nBuilt for market research, e-commerce, and growth teams. This workflow runs on a schedule to extract fresh data with the **[reddit-scraper Apify Actor](https://apify.com/unitbytes/reddit-scraper?fpr=939u3w&fp_sid=n8n_io)** ($0.001 / item), skips previously seen records, scores each item, logs results to **Google Sheets** and **CSV**, and posts **Slack** alerts for top matches.\n\n### How it works\n1. **Every 6 Hours** triggers the run and loads search settings from **1. Configure Search & Rules**.\n2. **2. Run UnitBytes (reddit-scraper)** executes the actor via Apify's synchronous endpoint.\n3. **3. Skip Seen Records** deduplicates items by `id` across runs.\n4. **4. Clean & Score Items** and **5. Map Export Fields** normalize fields (`itemType, id, title, subreddit, author`) and compute a 0–100 `priorityScore`.\n5. **6a. Log to Google Sheets** and **6b. Export Backup CSV** store all rows, while **6c. Filter High-Priority Items** sends top matches to **7. Send Slack Alert**.\n\n### Setup\n1. Copy your API token from [reddit-scraper on Apify Store](https://apify.com/unitbytes/reddit-scraper?fpr=939u3w&fp_sid=n8n_io) into the **HTTP Header Auth** credential (`Authorization: Bearer <TOKEN>`) on node 2.\n2. Edit `actorInput`, `sheetTab`, and `minEngagementScore` in **1. Configure Search & Rules**.\n3. Connect **Google Sheets** and **Slack** credentials (or test immediately via CSV export).\n\n### Customization\n- Tune `minEngagementScore` or `priorityKeyword` in node 1 to adjust Slack alert sensitivity.\n- Full schema reference: [https://unitbytes.com/actors/reddit-scraper/](https://unitbytes.com/actors/reddit-scraper/)",
        "height": 760,
        "width": 560,
        "color": 1
      },
      "id": "a0cf1c82-a4fd-5e05-9d24-19021779914f",
      "name": "Overview",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -640,
        -80
      ]
    },
    {
      "parameters": {
        "content": "## 1. Schedule & Search Settings\nTriggers on schedule and sets target search parameters, Google Sheets tab, and alert thresholds.",
        "height": 380,
        "width": 440,
        "color": 7
      },
      "id": "5ad07754-1869-5753-ae5c-944e8f15a8e7",
      "name": "Section 1 — Schedule & Settings",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        0,
        60
      ]
    },
    {
      "parameters": {
        "content": "## 2. Extract & Deduplicate\nRuns the UnitBytes Apify Actor and skips records already processed in previous runs.",
        "height": 380,
        "width": 440,
        "color": 7
      },
      "id": "292ba9e3-b3e2-5492-9089-dc50351ad608",
      "name": "Section 2 — Extract & Deduplicate",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        460,
        60
      ]
    },
    {
      "parameters": {
        "content": "## 3. Clean, Score & Flatten\nNormalizes fields, parses numeric metrics, and calculates a 0–100 priority score.",
        "height": 380,
        "width": 440,
        "color": 7
      },
      "id": "8cb547ea-006d-5cf2-b021-871313de0415",
      "name": "Section 3 — Clean, Score & Flatten",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        920,
        60
      ]
    },
    {
      "parameters": {
        "content": "## 4. Sync to Sheets, CSV & Slack\nLogs all new records to Google Sheets and CSV, and routes high-priority matches to Slack.",
        "height": 660,
        "width": 540,
        "color": 7
      },
      "id": "308ed96a-7178-53a1-9690-8b3e53b05411",
      "name": "Section 4 — Sheets, CSV & Slack",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        1380,
        -80
      ]
    },
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "hours",
              "hoursInterval": 6
            }
          ]
        }
      },
      "id": "b783ccea-f7e0-55a5-bc88-436542ffcbfa",
      "name": "Every 6 Hours",
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.2,
      "position": [
        40,
        220
      ]
    },
    {
      "parameters": {
        "assignments": {
          "assignments": [
            {
              "id": "f149eda6-977e-5cc0-bc66-8e71dbfd56fb",
              "name": "actorInput",
              "value": "{\"startUrls\": [], \"searchQuery\": \"artificial intelligence\", \"subreddit\": \"\"}",
              "type": "object"
            },
            {
              "id": "7604fce7-52b1-59d5-b174-fab0d4161ec6",
              "name": "sheetTab",
              "value": "CommunityFeed",
              "type": "string"
            },
            {
              "id": "21442156-46af-5dfb-bfad-53beb1f07243",
              "name": "minEngagementScore",
              "value": 50,
              "type": "number"
            },
            {
              "id": "f88462d2-9479-57b8-87b2-8f0a9388bde0",
              "name": "priorityKeyword",
              "value": "",
              "type": "string"
            }
          ]
        },
        "options": {}
      },
      "id": "14e5a646-dfd7-5bea-98e1-2aa0c6b409d4",
      "name": "1. Configure Search & Rules",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        260,
        220
      ]
    },
    {
      "parameters": {
        "method": "POST",
        "url": "https://api.apify.com/v2/acts/unitbytes~reddit-scraper/run-sync-get-dataset-items?format=json&clean=true",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ]
        },
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "={{ JSON.stringify($json.actorInput) }}",
        "options": {
          "timeout": 300000
        }
      },
      "credentials": {
        "httpHeaderAuth": {
          "id": "58e4a8b3-3798-5837-9e9c-c5513711f4d1",
          "name": "Apify API Token (Header Auth)"
        }
      },
      "id": "e39592aa-86de-5e63-b431-b2200e43d7f5",
      "name": "2. Run UnitBytes (reddit-scraper)",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        500,
        220
      ]
    },
    {
      "parameters": {
        "operation": "removeItemsSeenInPreviousExecutions",
        "dedupeValue": "={{ $json.id || $json.id || $json.url }}",
        "options": {}
      },
      "id": "a6548e3b-a1d1-5fd6-9561-7c8356ddec32",
      "name": "3. Skip Seen Records",
      "type": "n8n-nodes-base.removeDuplicates",
      "typeVersion": 2,
      "position": [
        720,
        220
      ]
    },
    {
      "parameters": {
        "jsCode": "// Normalize fields, parse numeric metrics, and score items for Slack alerts\nconst config = $('1. Configure Search & Rules').first().json;\nconst threshold = Number(config.minEngagementScore ?? 50);\nconst keywordFilter = String(config.priorityKeyword || '').trim().toLowerCase();\nconst primaryKeys = [\"itemType\", \"id\", \"title\", \"subreddit\", \"author\", \"score\"];\n\nreturn $input.all().map((item) => {\n  const raw = item.json || {};\n  const cleaned = { ...raw };\n\n  for (const k of primaryKeys) {\n    if (typeof cleaned[k] === 'string') {\n      cleaned[k] = cleaned[k].replace(/\\s+/g, ' ').trim();\n    }\n  }\n\n  const recordId = String(cleaned.id || cleaned.id || cleaned.url || '');\n  const titleText = String(cleaned.title || cleaned.content || cleaned.body || cleaned.keyword || cleaned.name || cleaned.sellerNick || recordId).slice(0, 180);\n  const itemUrl = String(cleaned.url || cleaned.productUrl || cleaned.taobaoUrl || cleaned.resolved_url || cleaned.input_url || cleaned.contributor_url || '');\n  const rawMetric = cleaned.price ?? cleaned.promotionPrice ?? cleaned.priceMin ?? cleaned.score ?? cleaned.likedCount ?? cleaned.likeCount ?? cleaned.value ?? cleaned.review_stars ?? cleaned.local_guide_level ?? 0;\n  const numericValue = Number(String(rawMetric).replace(/[^0-9.-]/g, '')) || 0;\n\n  let priorityScore = 50;\n  let isHighPriority = false;\n  const mode = 'engagement';\n\n  if (mode === 'price_deal') {\n    if (numericValue > 0 && numericValue <= threshold) {\n      priorityScore = Math.min(100, Math.round(70 + ((threshold - numericValue) / Math.max(threshold, 1)) * 30));\n      isHighPriority = true;\n    }\n  } else if (mode === 'review_audit') {\n    if (cleaned.is_removed === true || cleaned.is_edited === true || (numericValue > 0 && numericValue <= threshold)) {\n      priorityScore = 95;\n      isHighPriority = true;\n    }\n  } else {\n    if (numericValue >= threshold) {\n      priorityScore = Math.min(100, Math.round(70 + Math.min(numericValue / Math.max(threshold, 1), 3) * 10));\n      isHighPriority = true;\n    }\n  }\n\n  if (keywordFilter && titleText.toLowerCase().includes(keywordFilter)) {\n    priorityScore = Math.min(100, priorityScore + 20);\n    isHighPriority = true;\n  }\n\n  return {\n    json: {\n      ...cleaned,\n      recordId,\n      titleText,\n      itemUrl,\n      numericValue,\n      priorityScore,\n      isHighPriority,\n      alertSummary: `${titleText} (Metric: ${numericValue} | Score: ${priorityScore}/100)`,\n      scrapedAt: new Date().toISOString(),\n    },\n  };\n});"
      },
      "id": "3391d3e4-f89d-5818-8d95-aec553fdec54",
      "name": "4. Clean & Score Items",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        960,
        220
      ]
    },
    {
      "parameters": {
        "assignments": {
          "assignments": [
            {
              "id": "5421638a-f02f-54b6-94b9-15500aee199a",
              "name": "itemType",
              "value": "={{ $json.itemType }}",
              "type": "string"
            },
            {
              "id": "64f07873-fcfb-5f5a-acab-7f9a6b65ee96",
              "name": "id",
              "value": "={{ $json.id }}",
              "type": "string"
            },
            {
              "id": "d4f1caae-2341-5b73-a4ed-69c24993ccc0",
              "name": "title",
              "value": "={{ $json.title }}",
              "type": "string"
            },
            {
              "id": "0232b987-e7ce-5d57-b446-ebb6a109e786",
              "name": "subreddit",
              "value": "={{ $json.subreddit }}",
              "type": "string"
            },
            {
              "id": "00a0b2d9-f3ce-50ac-9992-880c4c7157fc",
              "name": "author",
              "value": "={{ $json.author }}",
              "type": "string"
            },
            {
              "id": "375e03ad-cd4d-5aef-8cdd-93547efdc864",
              "name": "score",
              "value": "={{ $json.score }}",
              "type": "string"
            },
            {
              "id": "d029ebda-136c-51a9-83b9-8e3ea190955b",
              "name": "numericValue",
              "value": "={{ $json.numericValue }}",
              "type": "number"
            },
            {
              "id": "d97ffbd2-20cf-5a11-a1ee-b5b704911c6a",
              "name": "priorityScore",
              "value": "={{ $json.priorityScore }}",
              "type": "number"
            },
            {
              "id": "58c835bf-ae3b-5e4d-8e7a-cb6350273ed1",
              "name": "isHighPriority",
              "value": "={{ $json.isHighPriority }}",
              "type": "boolean"
            },
            {
              "id": "825a6464-0de9-5938-b385-c9412af5c149",
              "name": "alertSummary",
              "value": "={{ $json.alertSummary }}",
              "type": "string"
            },
            {
              "id": "f7723cf6-4221-5470-9010-973c1e06deec",
              "name": "scrapedAt",
              "value": "={{ $json.scrapedAt }}",
              "type": "string"
            }
          ]
        },
        "includeOtherFields": true,
        "options": {}
      },
      "id": "88a5fef6-83ea-5438-ae0d-a67b439da4db",
      "name": "5. Map Export Fields",
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        1180,
        220
      ]
    },
    {
      "parameters": {
        "resource": "sheet",
        "operation": "append",
        "documentId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        },
        "sheetName": {
          "__rl": true,
          "mode": "name",
          "value": "={{ $('1. Configure Search & Rules').first().json.sheetTab }}"
        },
        "columns": {
          "mappingMode": "defineBelow",
          "value": {
            "itemType": "={{ $json.itemType }}",
            "id": "={{ $json.id }}",
            "title": "={{ $json.title }}",
            "subreddit": "={{ $json.subreddit }}",
            "author": "={{ $json.author }}",
            "score": "={{ $json.score }}",
            "numericValue": "={{ $json.numericValue }}",
            "priorityScore": "={{ $json.priorityScore }}",
            "isHighPriority": "={{ $json.isHighPriority }}",
            "alertSummary": "={{ $json.alertSummary }}",
            "scrapedAt": "={{ $json.scrapedAt }}"
          },
          "matchingColumns": [],
          "schema": [
            {
              "id": "itemType",
              "displayName": "itemType",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "id",
              "displayName": "id",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "title",
              "displayName": "title",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "subreddit",
              "displayName": "subreddit",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "author",
              "displayName": "author",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "score",
              "displayName": "score",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "numericValue",
              "displayName": "numericValue",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "priorityScore",
              "displayName": "priorityScore",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "isHighPriority",
              "displayName": "isHighPriority",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "alertSummary",
              "displayName": "alertSummary",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "scrapedAt",
              "displayName": "scrapedAt",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            }
          ]
        },
        "options": {
          "cellFormat": "RAW"
        }
      },
      "credentials": {
        "googleSheetsOAuth2Api": {
          "id": "392f8733-93e7-5818-aa0c-b1fd35cc5d9c",
          "name": "Google Sheets account"
        }
      },
      "id": "7136d251-bbc7-5e4a-a3fd-889a7fa8ff43",
      "name": "6a. Log to Google Sheets",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 4.7,
      "position": [
        1440,
        -20
      ]
    },
    {
      "parameters": {
        "operation": "csv",
        "options": {
          "fileName": "reddit-scraper-dataset-export.csv"
        }
      },
      "id": "1a13eb47-f7ff-5350-8541-cfe8450bfb30",
      "name": "6b. Export Backup CSV",
      "type": "n8n-nodes-base.convertToFile",
      "typeVersion": 1.1,
      "position": [
        1440,
        160
      ]
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "loose",
            "version": 2
          },
          "conditions": [
            {
              "id": "5cb210b0-2ac9-54d4-9f97-a6135c208792",
              "leftValue": "={{ $json.isHighPriority }}",
              "rightValue": true,
              "operator": {
                "type": "boolean",
                "operation": "true",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      },
      "id": "cb6bc77a-322c-5148-aefc-1e35f0c7e3b0",
      "name": "6c. Filter High-Priority Items",
      "type": "n8n-nodes-base.filter",
      "typeVersion": 2.2,
      "position": [
        1440,
        360
      ]
    },
    {
      "parameters": {
        "resource": "message",
        "operation": "post",
        "select": "channel",
        "channelId": {
          "__rl": true,
          "mode": "list",
          "value": ""
        },
        "messageType": "text",
        "text": "=:bell: *New High-Priority Match (Reddit Scraper & Media API: Posts, Users & Comments)*\n*<{{ $json.itemUrl }}|{{ $json.titleText }}>*\n• *Priority Score:* {{ $json.priorityScore }}/100 ({{ $json.numericValue }})\n• *Record ID:* `{{ $json.recordId }}`\n• *Scraped At:* {{ $json.scrapedAt }}",
        "otherOptions": {
          "includeLinkToWorkflow": false,
          "unfurl_links": false
        }
      },
      "credentials": {
        "slackApi": {
          "id": "6c67b0db-bde1-5e74-9d67-b43b6bfdb500",
          "name": "Slack account"
        }
      },
      "id": "a1fe7cd4-8022-537a-b6fb-e266c6fd907f",
      "name": "7. Send Slack Alert",
      "type": "n8n-nodes-base.slack",
      "typeVersion": 2.3,
      "position": [
        1680,
        360
      ]
    }
  ],
  "connections": {
    "Every 6 Hours": {
      "main": [
        [
          {
            "node": "1. Configure Search & Rules",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "1. Configure Search & Rules": {
      "main": [
        [
          {
            "node": "2. Run UnitBytes (reddit-scraper)",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "2. Run UnitBytes (reddit-scraper)": {
      "main": [
        [
          {
            "node": "3. Skip Seen Records",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "3. Skip Seen Records": {
      "main": [
        [
          {
            "node": "4. Clean & Score Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "4. Clean & Score Items": {
      "main": [
        [
          {
            "node": "5. Map Export Fields",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "5. Map Export Fields": {
      "main": [
        [
          {
            "node": "6a. Log to Google Sheets",
            "type": "main",
            "index": 0
          },
          {
            "node": "6b. Export Backup CSV",
            "type": "main",
            "index": 0
          },
          {
            "node": "6c. Filter High-Priority Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "6c. Filter High-Priority Items": {
      "main": [
        [
          {
            "node": "7. Send Slack Alert",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "settings": {
    "executionOrder": "v1"
  },
  "meta": {
    "templateCredsSetupCompleted": false,
    "instanceId": "030f9420-0ca8-5240-8fe8-208e287a8b19"
  },
  "tags": [
    {
      "name": "Web Scraping"
    },
    {
      "name": "Apify"
    },
    {
      "name": "UnitBytes"
    },
    {
      "name": "Social Media & Community Data"
    }
  ]
}