Commit bc89eed7 authored by Alexandre Delanoë's avatar Alexandre Delanoë

Merge remote-tracking branch 'origin/365-documentation-improve-readme' into dev

parents 81ae8313 24d1c8d2
...@@ -40,55 +40,85 @@ You must have the following installed: ...@@ -40,55 +40,85 @@ You must have the following installed:
Clone both the backend (`haskell-gargantext`), and the frontend (`purescript-gargantext`) at the root of the backend. Clone both the backend (`haskell-gargantext`), and the frontend (`purescript-gargantext`) at the root of the backend.
```shell ```shell
git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git $ git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
cd haskell-gargantext $ cd haskell-gargantext
git clone https://gitlab.iscpif.fr/gargantext/purescript-gargantext.git $ git clone https://gitlab.iscpif.fr/gargantext/purescript-gargantext.git
cd .. $ cd ..
``` ```
#### Enter a Nix shell #### The Nix shell
Enter a Nix shell. This will take a long time the first time you run it: In what follows, many commands need to be executed from within the Nix shell. To make that clear, those will be prefixed with `n$`, *but you must not actually type `n$` before the commands*.
To enter a Nix shell, run the following (this will take a moment the first time you run it, be patient):
```shell ```shell
nix-shell $ nix-shell
``` ```
Once you are in a Nix shell, you can run commands like you would in any other shell.
You can exit a Nix shell at any point with `exit`. At any point, you can exit a Nix shell and go back to your regular shell by running `exit`.
In what follows, many commands need to be executed from within the Nix shell. To make that clear, those will be prefixed with `n$`, *but you must not actually type `n$` before the commands*.
If for some reason you do not want to enter a Nix shell, you can still run a command from outside: If for some reason you do not want to enter a Nix shell, you can still run a command from outside: running the following in a non-Nix shell
```shell ```shell
nix-shell --run "my command" $ nix-shell --run "my command"
``` ```
is equivalent to running `my command` from within a Nix shell. is equivalent to running `my command` from within a Nix shell.
#### Disable optimization flags #### (Optional) Disable optimization flags
If you are developing Gargantext, you might be interested in disabling compiler optimizations.
This speeds up compilation, but the compiled program itself will be less efficient.
To disable compiler optimizations, copy the file `cabal.project.local_toCopy` (which contains the flags that disable optimizations) into `cabal.project.local` (which will be read by Cabal):
```shell
$ cp cabal.project.local_toCopy cabal.project.local
```
#### Build the frontend
Make a file `cabal.project.local` that will tell Cabal to turn off optimizations:
```shell ```shell
cp cabal.project.local_toCopy cabal.project.local $ cd purescript-gargantext/
$ ./bin/install
$ cd ..
``` ```
#### Install backend dependencies #### Build the backend
*Note: This project can be built with either stack or cabal. We keep the `cabal.project` up-to-date, which allows us to build with cabal by default but we support stack thanks to thanks to `cabal2stack`, which allows us to generate a valid `stack.yaml` from a `cabal.project`. Due to the fact gargantext requires a particular set of system dependencies (C++ libraries, toolchains, etc) we use nix to setup an environment with all the required system dependencies, in a sandboxed and isolated fashion.* *Note: This project can be built with either stack or cabal. We keep the `cabal.project` up-to-date, which allows us to build with cabal by default but we support stack thanks to thanks to `cabal2stack`, which allows us to generate a valid `stack.yaml` from a `cabal.project`. Due to the fact gargantext requires a particular set of system dependencies (C++ libraries, toolchains, etc) we use nix to setup an environment with all the required system dependencies, in a sandboxed and isolated fashion.*
*This documentation shows how to build with cabal. For information related to stack, see `docs/using_stack.md`.* *This documentation shows how to build with cabal. For information related to stack, see `docs/using_stack.md`.*
**From within the Nix shell**, run: Depending on your situation, there are several ways to build the project:
1. **Simple build**
*This will build the project and install the executables `gargantext-cli` and `gargantext-server` somewhere on your system.
Depending on your Cabal configuration, this is probably `~/.local/bin/` or `~/.cabal/bin/`.*
From within the Nix shell, run:
```shell ```shell
n$ cabal update n$ cabal update
n$ cabal install n$ cabal install
``` ```
#### Build the backend and frontend 2. **Full build**
*Same as "simple build" above, but also runs tests and builds documentation.*
Just run the `install` script:
```shell ```shell
./bin/install $ ./bin/install
cd purescript-gargantext/ ```
./bin/install
cd .. 3. **Build and run**
*Builds and runs the Gargantext server. This has the advantage of letting you run Gargantext without having to know where on your machine the executable is.*
*Since you will be running Gargantext,* **you need to have gone through initialization first;** *see "Initializing and running" below.*
From inside a Nix shell:
```shell
n$ cabal run gargantext-server -- --ini gargantext.ini --run Prod
``` ```
### Initializing and running ### Initializing and running
...@@ -96,8 +126,8 @@ cd .. ...@@ -96,8 +126,8 @@ cd ..
#### Start containers for database and NLP software bricks #### Start containers for database and NLP software bricks
```shell ```shell
cd devops/docker $ cd devops/docker
docker compose up $ docker compose up
``` ```
The initialization schema should be loaded automatically from `devops/postgres/schema.sql`. The initialization schema should be loaded automatically from `devops/postgres/schema.sql`.
...@@ -105,7 +135,7 @@ The initialization schema should be loaded automatically from `devops/postgres/s ...@@ -105,7 +135,7 @@ The initialization schema should be loaded automatically from `devops/postgres/s
#### Create configuration file #### Create configuration file
```shell ```shell
cp gargantext.ini_toModify gargantext.ini $ cp gargantext.ini_toModify gargantext.ini
``` ```
> `.gitignore` excludes this file, so you don't need to worry about committing it by mistake, and you can change the passwords in `gargantext.ini` safely. > `.gitignore` excludes this file, so you don't need to worry about committing it by mistake, and you can change the passwords in `gargantext.ini` safely.
...@@ -121,24 +151,25 @@ The master user's name is automatically set to `gargantua`, but you will be prom ...@@ -121,24 +151,25 @@ The master user's name is automatically set to `gargantua`, but you will be prom
#### Running #### Running
From inside a Nix shell: Make sure you know where `gargantext-server` is (probably in `~/.local/bin/` or `.cabal/bin/`). If the location is in your `$PATH`, just run:
```shell ```shell
n$ cabal run gargantext-server -- --ini gargantext.ini --run Prod $ gargantext-server -- --ini gargantext.ini --run Prod
``` ```
(If the location is not in your `$PATH`, just prefix `gargantext-server` with the path to it.)
If you are working on the backend, you might want to use the `./start` script: it rebuilds the backend, starts the docker containers, and launches the Gargantext server at once. You might want to use the `./start` script: it rebuilds the backend, starts the docker containers, and launches the Gargantext server at once.
### Running tests ### Running tests
From nix shell: From nix shell:
``` ```shell
cabal v2-test --test-show-details=streaming n$ cabal v2-test --test-show-details=streaming
``` ```
Or, from "outside": Or, from "outside":
``` ```shell
nix-shell --run "cabal v2-test --test-show-details=streaming" $ nix-shell --run "cabal v2-test --test-show-details=streaming"
``` ```
### Working on libraries ### Working on libraries
...@@ -161,8 +192,8 @@ When a devlopment is needed on libraries (for instance, the HAL crawler in https ...@@ -161,8 +192,8 @@ When a devlopment is needed on libraries (for instance, the HAL crawler in https
### Multi-User with Graphical User Interface (Server Mode) ### Multi-User with Graphical User Interface (Server Mode)
``` sh ``` shell
~/.local/bin/stack --docker exec gargantext-server -- --ini "gargantext.ini" --run Prod $ ~/.local/bin/stack --docker exec gargantext-server -- --ini "gargantext.ini" --run Prod
``` ```
Then you can log in with `user1` / `1resu` Then you can log in with `user1` / `1resu`
...@@ -172,23 +203,23 @@ Then you can log in with `user1` / `1resu` ...@@ -172,23 +203,23 @@ Then you can log in with `user1` / `1resu`
#### Simple cooccurrences computation and indexation from a list of Ngrams #### Simple cooccurrences computation and indexation from a list of Ngrams
``` sh ``` shell
stack --docker exec gargantext-cli -- CorpusFromGarg.csv ListFromGarg.csv Ouput.json $ stack --docker exec gargantext-cli -- CorpusFromGarg.csv ListFromGarg.csv Ouput.json
``` ```
### Analyzing the ngrams table repo ### Analyzing the ngrams table repo
We store the repository in directory `repos` in the [CBOR](https://cbor.io/) file format. To decode it to JSON and analyze, say, using [jq](https://shapeshed.com/jq-json/), use the following command: We store the repository in directory `repos` in the [CBOR](https://cbor.io/) file format. To decode it to JSON and analyze, say, using [jq](https://shapeshed.com/jq-json/), use the following command:
``` sh ``` shell
cat repos/repo.cbor.v5 | stack exec gargantext-cbor2json | jq . $ cat repos/repo.cbor.v5 | stack exec gargantext-cbor2json | jq .
``` ```
### Documentation ### Documentation
To build documentation, run: To build documentation, run:
```sh ```shell
stack build --haddock --no-haddock-deps --fast $ stack build --haddock --no-haddock-deps --fast
``` ```
(in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`). (in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`).
...@@ -233,24 +264,24 @@ Playground is located at http://localhost:8008/gql ...@@ -233,24 +264,24 @@ Playground is located at http://localhost:8008/gql
https://www.cloudytuts.com/tutorials/docker/how-to-upgrade-postgresql-in-docker-and-kubernetes/ https://www.cloudytuts.com/tutorials/docker/how-to-upgrade-postgresql-in-docker-and-kubernetes/
To upgrade PostgreSQL in Docker containers, for example from 11.x to 14.x, simply run: To upgrade PostgreSQL in Docker containers, for example from 11.x to 14.x, simply run:
```sh ```shell
docker exec -it <container-id> pg_dumpall -U gargantua > 11-db.dump $ docker exec -it <container-id> pg_dumpall -U gargantua > 11-db.dump
``` ```
Then, shut down the container, replace `image` section in `devops/docker/docker-compose.yaml` with `postgres:14`. Also, it is a good practice to create a new volume, say `garg-pgdata14` and bind the new container to it. If you want to keep the same volume, remember about removing it like so: Then, shut down the container, replace `image` section in `devops/docker/docker-compose.yaml` with `postgres:14`. Also, it is a good practice to create a new volume, say `garg-pgdata14` and bind the new container to it. If you want to keep the same volume, remember about removing it like so:
```sh ```shell
docker-compose rm postgres $ docker-compose rm postgres
docker volume rm docker_garg-pgdata $ docker volume rm docker_garg-pgdata
``` ```
Now, start the container and execute: Now, start the container and execute:
```sh ```shell
# need to drop the empty DB first, since schema will be created when restoring the dump $ # need to drop the empty DB first, since schema will be created when restoring the dump
docker exec -i <new-container-id> dropdb -U gargantua gargandbV5 $ docker exec -i <new-container-id> dropdb -U gargantua gargandbV5
# recreate the db, but empty with no schema $ # recreate the db, but empty with no schema
docker exec -i <new-container-id> createdb -U gargantua gargandbV5 $ docker exec -i <new-container-id> createdb -U gargantua gargandbV5
# now we can restore the dump $ # now we can restore the dump
docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump $ docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump
``` ```
### Upgrading using ### Upgrading using
...@@ -258,24 +289,24 @@ docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump ...@@ -258,24 +289,24 @@ docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump
There is a solution using pgupgrade_cluster but you need to manage the clusters version 14 and 13. Hence here is a simple solution to upgrade. There is a solution using pgupgrade_cluster but you need to manage the clusters version 14 and 13. Hence here is a simple solution to upgrade.
First save your data: First save your data:
``` ```shell
sudo su postgres $ sudo su postgres
pg_dumpall > gargandb.dump $ pg_dumpall > gargandb.dump
``` ```
Upgrade postgresql: Upgrade postgresql:
``` ```shell
sudo apt install postgresql-server-14 postgresql-client-14 $ sudo apt install postgresql-server-14 postgresql-client-14
sudo apt remove --purge postgresql-13 $ sudo apt remove --purge postgresql-13
``` ```
Restore your data: Restore your data:
``` ```shell
sudo su postgres $ sudo su postgres
psql < gargandb.dump $ psql < gargandb.dump
``` ```
Maybe you need to restore the gargantua password Maybe you need to restore the gargantua password
``` ```shell
ALTER ROLE gargantua PASSWORD 'yourPasswordIn_gargantext.ini' $ ALTER ROLE gargantua PASSWORD 'yourPasswordIn_gargantext.ini'
``` ```
Maybe you need to change the port to 5433 for database connection in your gargantext.ini file. Maybe you need to change the port to 5433 for database connection in your gargantext.ini file.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment