crawl/START-HERE.txt at main · vishnupriyajha2026/crawl · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
CRAWL V1 - START HERE
=====================

SUPPORTED COMPUTER

Crawl v1 is the Mac release. It supports:

  - Apple-chip Macs: M1, M2, M3, M4 and newer
  - Intel Macs
  - macOS 13 or newer

This ZIP is not the Windows release. Do not try to run run-crawl.command on
Windows. A separate Windows version needs to be tested on a real Windows
computer before it is shared.


BEFORE YOU START

You need:

  1. An Apify account and token
  2. Either a Gemini API key, or a Google Cloud project with billing/free trial
  3. Your Google Cloud Project ID if you choose the free-credit Vertex option

Watch Vishnupriya's setup video first, then double-click run-crawl.command.


CREATE YOUR GOOGLE CLOUD PROJECT FOR THE FREE-CREDIT OPTION

1. Open https://cloud.google.com/free and activate the Google Cloud free trial.
   Google may ask for a payment method to verify your identity. Eligibility and
   the available credit depend on your Google account and country.

2. Open https://console.cloud.google.com.

3. Click the project selector at the top, then click New Project.

4. Name the project Crawl. Leave the billing account, organisation and parent
   resource at their default values, then click Create.

5. Open the new Crawl project and copy its Project ID. Copy the ID, not the
   project name or project number.

Your ID will look like:

  crawl-123456

Keep it ready. Crawl will ask you to paste it during Vertex setup.


WHAT CRAWL INSTALLS AUTOMATICALLY

If your Mac does not already have what Crawl needs, the launcher downloads
private copies into this private tools folder in your user account:

  ~/.crawl-tools

It can automatically install:

  - Node.js, used to run Crawl
  - Python 3.13, required by Google Cloud CLI
  - Google Cloud CLI, used to connect Crawl to your Google Cloud project

These installs do not require an administrator password and do not replace the
Mac's system Python. The first setup can take several minutes.


VERTEX / GOOGLE CLOUD SETUP

When Crawl asks how to power the AI:

  a = paste a Gemini API key
  b = use Google Cloud credit through Vertex

If you choose b, Crawl will:

  1. Install any missing setup tools
  2. Open Google login in your browser
  3. Ask for your Google Cloud Project ID
  4. Enable Vertex AI and Cloud Storage
  5. Create a storage bucket for the videos
  6. Save the settings locally for future runs

Your Project ID looks like crawl-123456. It is not the project name or number.

When the Google login page opens, use the Google account that owns the Crawl
project. Crawl uses one login to authorize both Google Cloud setup and the tool.


IF DOUBLE-CLICKING IS BLOCKED

1. Right-click run-crawl.command.
2. Click Open.
3. Click Open again.

If macOS still blocks it, open Terminal, type "bash " with a space, drag
run-crawl.command into Terminal, and press Return.


IF AUTOMATIC NODE INSTALLATION FAILS

Install the current LTS version from:

  https://nodejs.org

Then close Terminal and double-click run-crawl.command again.


IF AUTOMATIC PYTHON INSTALLATION FAILS

Install Python 3.13 for macOS from:

  https://www.python.org/downloads/macos/

Then close Terminal and double-click run-crawl.command again. Crawl will detect
the new Python automatically.


IF GOOGLE LOGIN DOES NOT OPEN

Look in the Terminal window for a long Google URL. Copy it into your browser,
log in with the same Google account used for the Cloud project, click Allow,
then return to Terminal.


IF IT STILL FAILS

Take one screenshot that includes the first ERROR line and the lines immediately
above it. Do not include API keys or tokens. Send that screenshot to Vishnupriya.