docs: improve HealthyCode deployment docs

Signed-off-by: Keshav Priyadarshi <git@keshav.space>

Author: keshav-space

deploy/README.md

@@ -50,6 +50,15 @@ vm.max_map_count=262144
 
 Then run `sudo sysctl -p` to reload the kernel runtime parameters configuration.
 
+### Clone HealthyCode
+
+```
+git clone https://github.com/aboutcode-org/healthycode.git /opt/healthycode/ && cd /opt/healthycode/
+```
+
+Open `/opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml` in a text editor,
+and set the credentials as suggested in following steps.
+
 ### Ports and networks
 
 The main GrimoireLab services needed from outside its docker compose network
@@ -73,7 +82,7 @@ are minimally documented here.
 
 This various values for credentials are configured using environment variables
 and defaults in docker compose file at:
-docker-compose/docker-compose-grimoirelab2.yml
+`/opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml`
 
 You can update that file as needed.
 
@@ -108,93 +117,90 @@ NOTE that this is not a good practice for production services, yet only
 containers inside this compose project can reach them. 
 
 The good practice is to set passwords using the following variables for a
-production enviroment:
+production environment:
 
- - For mariadb, set enviroment variables ending in:
+ - For mariadb, set environment variables ending in:
    - `*_DB_USER`
    - `*_DB_PASSWORD`
 
- - For Valkey, set enviroment variables ending in:
+ - For Valkey, set environment variables ending in:
    - `*_REDIS_PASSWORD`
 
 ## GrimoireLab Startup
 
-Once the initial configutaion above is completed, the next step if launch the
+Once the initial configuration above is completed, the next step if launch the
 containers with these commands, from a clone of this repository:
 
+
 ```
-cd docker-compose/
+cd /opt/healthycode/deploy/docker-compose/
 docker compose -f docker-compose-grimoirelab2.yml up -d
 ```
 
-## Installation of with PurlDB and ScanCode.io 
-
-### Base installation
+## Installation of PurlDB and ScanCode.io
 
 The first step is to install PurlDB and ScanCode.io using their standard
 documentation.
 
-- Setup PurlDB using code and documentation at https://github.com/aboutcode-org/purldb
-
-Once completed, make sure that PurlDB is up and running.
-
-
-- Setup ScanCode.io as a PurlDB scan worker using code and documentation at 
-https://github.com/aboutcode-org/scancode.io
-
-### Add shared GrimoireLab docker network to ScanCode.io
-
-There is a temaplte for this docker network configuration in:
-etc/scancodeio/network.yml. We copy that to the host:
+### Install and setup PurlDB
 
 ```
-cd healthycode-deploy/
-sudo cp ./etc/scancodeio/network.yml /etc/scancodeio/network.yml
+git clone https://github.com/aboutcode-org/purldb.git/ /opt/purldb/ && cd /opt/purldb/
+make envfile
 ```
 
-### Edit ScanCode.io systemctl config to run with healthycode profile
+Open `/opt/purldb/.env` in a text editor and populate with this config:
 
 ```
-sudo vi /etc/systemd/system/scancodeio.service
+ALLOWED_HOSTS=<your host>
+SITE_URL=https://<your host>/
+PURLDB_PUBLIC_SETUP=True
+REST_API_ENDPOINTS=scan_queue,health,packages
 ```
 
-Update ExecStart as below:
+Build and run PurlDB
 ```
-ExecStart=/usr/bin/docker compose \
-  -f /opt/scancodeio/docker-compose.purldb-scan-worker.yml \
-  -f /etc/scancodeio/networks.yml \
-  --profile healthycode \
-  up --scale worker=${NUM_WORKERS}
+sudo docker compose -f /opt/purldb/docker-compose.yml build
+sudo docker compose -f /opt/purldb/docker-compose.yml up -d
 ```
 
+Once completed, make sure that PurlDB is up and running.
+
+
+### Setup  ScanCode.io as a PurlDB scan worker
+
 ```
-sudo systemctl daemon-reload
+git clone https://github.com/aboutcode-org/scancode.io.git /opt/scancodeio/ && cd /opt/scancodeio/
+make envfile
 ```
 
-### Add PurlDB base URL and API key to ScanCode.io
+#### Add PurlDB base URL and API key to ScanCode.io
 
 Get or create a PurlDB API key for the scanworker:
 ```
-# on purldb
-docker compose -f /opt/purldb/docker-compose.production.yml exec web python \
-  manage_purldb.py create-scan-queue-worker-user <username>
+sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
+  create-scan-queue-worker-user scancode-worker <username>
 ```
 
+Copy the scancode-worker API Key, and open `/opt/scancodeio/.env` in a text editor.
+
 ```
-sudo vi /etc/scancodeio/.env
+sudo vi /opt/scancodeio/.env
 ```
 
-Populate with this config, using localhost assuming eveything is co-located on
+Populate with this config, using localhost assuming everything is co-located on
 one server:
+
 ```
-PURLDB_URL=
-PURLDB_API_KEY="scanworker-api-key"
+PURLDB_URL=<your host>
+PURLDB_API_KEY="scancode-worker-api-key"
 ```
 
-### Configure GrimoireLab environment in ScanCode.io
+
+#### Configure GrimoireLab environment in ScanCode.io
 
 ```
-sudo vi /etc/scancodeio/.env
+sudo vi /opt/scancodeio/.env
 ```
 
 Populate with this config:
@@ -203,20 +209,27 @@ GRIMOIRELAB_URL="http://grimoirelab-nginx:8000/"
 GRIMOIRELAB_OPENSEARCH_URL="https://grimoirelab-opensearch:9200/"
 GRIMOIRELAB_ECOSYSTEM="npmlargerecosystem"
 GRIMOIRELAB_PROJECT="NPMMostUsed"
+GRIMOIRELAB_OPENSEARCH_INDEX="events"
 ```
 
 Populate these fields based on the same secret that may have been set in
-`./docker-compose/docker-compose-grimoirelab2.yml`
+`/opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml`
 
 ```
+# Populate this using value from GRIMOIRELAB_SUPERUSER_USERNAME
+# and GRIMOIRELAB_SUPERUSER_PASSWORD set earlier in
+# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
 GRIMOIRELAB_USERNAME=""
 GRIMOIRELAB_PASSWORD=""
-GRIMOIRELAB_OPENSEARCH_INDEX=""
+
+# Populate this using value from GRIMOIRELAB_ARCHIVIST_STORAGE_USERNAME
+# and GRIMOIRELAB_ARCHIVIST_STORAGE_PASSWORD set earlier in
+# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
 GRIMOIRELAB_OPENSEARCH_USERNAME=""
 GRIMOIRELAB_OPENSEARCH_PASSWORD=""
 ```
 
-### Expose ScanCode.io on port 8443
+#### Expose ScanCode.io on port 8443
 
 This is optional, and enables access to the ScanCode.io scan worker.
 If using this, make sure you configure authentication in ScanCode.io first.
@@ -227,19 +240,125 @@ sudo vi /opt/scancodeio/.env
 
 Add the below env variables:
 ```
+ALLOWED_HOSTS=<your host>
+CSRF_TRUSTED_ORIGINS=https://<your host>:8443
+
 NGINX_PUBLISHED_HTTP_PORT=8081
 NGINX_PUBLISHED_HTTPS_PORT=8443
 ```
 
-### Restart ScanCode.io
+If this is enabled ScanCode.io will be accessible at: `http://<your host>:8443`
+
+
+#### Start ScanCode.io
+
+Remove any old healthycode volume.
+
+```
+docker volume rm scancodeio_healthycode_venv
+```
+
+Pull prebuilt ScanCode docker images
+
 ```
-sudo systemctl restart scancodeio
+sudo docker compose -f /opt/scancodeio/docker-compose.purldb-scan-worker.yml pull
 ```
 
+Start ScanCode with HealthyCode profile
+
+```
+sudo docker compose \
+  -f /opt/scancodeio/docker-compose.purldb-scan-worker.yml \
+  -f /opt/healthycode/deploy/etc/scancodeio/networks.yml \
+  --profile healthycode \
+  up --scale worker=2 -d
+```
+
+Note: we attach shared GrimoireLab network to ScanCode service using `networks.yml`
+
 ## Verify everything is working end to end
 
 
-Visit the PurlDB health API at: https://<your host>/api/health/
+Visit the PurlDB health API at: `https://<your host>/api/health/`
+
+Make a test request with `https://<your host>/api/health/?purl=pkg:npm/lodash&format=json`
+
+
+## Some useful commands
+
+### Start and Stop ScanCode.io service
+
+Start ScanCode.io
+
+```
+sudo docker compose \
+  -f /opt/scancodeio/docker-compose.purldb-scan-worker.yml \
+  -f /opt/healthycode/deploy/etc/scancodeio/networks.yml \
+  --profile healthycode \
+  up --scale worker=2 -d
+```
+
+Stop ScanCode.io:
+
+```
+sudo docker compose -f /opt/scancodeio/docker-compose.purldb-scan-worker.yml down
+```
+
+### Start and Stop PurlDB service
+
+Start PurlDB:
+
+```
+sudo docker compose -f /opt/purldb/docker-compose.yml up -d
+```
+
+Stop PurlDB:
+
+```
+sudo docker compose -f /opt/purldb/docker-compose.yml down
+```
+
+### Start and Stop GrimoireLab service
+
+Start GrimoireLab:
+
+```
+docker compose -f /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml up -d
+```
+
+Stop GrimoireLab:
+
+```
+docker compose -f /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml down
+```
+
+
+## Enable authentication for ScanCode.io (Optional):
+
+Add this to `/opt/scancodeio/.env`
+
+```
+SCANCODEIO_REQUIRE_AUTHENTICATION=True
+```
+
+Run this command to create username and password required to login to ScanCode:
+
+```
+docker compose -f /opt/scancodeio/docker-compose.yml exec web ./manage.py scanpipe create-user <user-name>
+```
+
+Restart ScanCode.io service.
+
+## SSL/TLS Configuration
+
+Add `fullchain.pem` and `privkey.pem` for your host inside:
+
+- /opt/purldb/etc/nginx/conf.d/
+- /opt/scancodeio/etc/nginx/conf.d/
+
+Restart PurlDB and ScanCode.io service.
 
-Make a test request with https://<your host>/api/health/?purl=pkg:npm/lodash&format=json
+## Reference documentation
 
+- https://scancodeio.readthedocs.io/en/latest/installation.html
+- https://purldb.readthedocs.io/en/latest/getting-started/install.html