hdx-scraper

This application is using the HDX and the Healthsites Global Mappging Project APIs to automate the process of country data extraction from healthsites and sharing to HDX platform.

See:

• HDX Python Library API documentation
• Healthsites API documentation

for more information.

Download

git clone https://github.com/healthsites/hdx-scraper.git

Dependencies

The script requires ogr2ogr binary which is a part of the GDAL package

Installation

Open a terminal and run the installation script. This will install a virtualenv and install all the requirements.

cd hdx-scraper
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Adjust HDX environment in the .env file. Optionally create a .env.local file (git-ignored) with values that override .env, e.g. real API keys on the server.

HDX API key

Get into HDX website and save your API in your home directory. The full process for getting the key is well documented in the HDX API repository : Usage -> Getting started -> Obtaining your API Key.

Healthsites.io API Key

Obtain you API key from Healthsites.io and add it to the .env file

Running the script

Open a terminal and execute the following command (it takes ages, so wrap it into screen or similar)

python main.py

Deleting a country from HDX

To remove a single country's dataset (and all its resources) from HDX:

python delete_country.py "Country Name"

The country name must match the one used in countries.csv. Remove the country's row from countries.csv as well, otherwise the next run of main.py will recreate the dataset.

Data

The generated datasets are available in hdx from this link : https://data.humdata.org/organization/healthsites