Basic Overview

Easy deployment of TheyBuyForYou tools and services into a local environment.

Quick Start

This process has been tested in an environment with 4 CPUs and 4GB RAM

1. Install Docker and Docker-Compose

2. Clone this repo

   git clone https://github.com/TBFY/platform.git
   

3. Download the latest TBFY data dump from Zenodo into a temporal folder (e.g /tmp).
   

4. Run the platform by:

   docker-compose up -d
   

5. Wait for all services to be available (e.g. Started Application in xx.xx seconds). The first time it may take a few minutes to download the Docker images.

   docker-compose logs -f
   

6. Initialize the RDF repository

   1. Log into the Fuseki administration GUI, http://localhost:3030, using the admin user and the password that you set in the fuseki.env file.
   2. Create a new dataset tbfy with the dataset type Persistent (TDB2).
   3. Upload the NACE and OpenCorporates identifer system data files from here.
   4. Follow these instructions to publish the TBFY procurement data. First set the environment variable TBFY_FUSEKI_URL to:

       export TBFY_FUSEKI_URL=http://localhost:3030
      

7. Initialize the Document repository

   1. Unzip the TBFY_DATA_DUMP_JSON.zip file from the temporal folder. The content is organized by month in different folders.

      unzip /tmp/TBFY_DATA_DUMP_JSON.zip 
      

   2. Load the tender descriptions using the index_tenders.py python script. Depending on the number of folders selected (e.g /tmp/* or /tmp/2020-04-* or /tmp/2020-04-30), the operation may take minutes or hours.:

      ./script/index_tenders.py -i /tmp/2020-04-30
      

8. That's all!

Services

Once the platform is up and running, all TBFY services are available from a single entry point:

service	description
/industryCodes	industry codes
/jurisdictions	jurisdictions
/ocds	OCDS instances
/brands	organization names
/triples	SPARQL Endpoint
/documents	tender descriptions
/organisation	organizations
/contract	contracts
/tender	tenders
/award	awards
/contractingProcess	contracting processes

Resources

TBFY services are supported by the following resources:

resource	description
http://localhost:8085	English Topic Model
http://localhost:8086	Spanish Topic Model
http://localhost:8087	French Topic Model
http://localhost:8088	Italian Topic Model
http://localhost:8089	Portuguese Topic Model
http://localhost:8983	Document Repository
http://localhost:8983/solr/banana	Dashboard
http://localhost:3040	SPARQL Query Interface
http://localhost:3030	RDF Repository

Contributing

Please take a look at our contributing guidelines if you're interested in helping!