Skip to content

haskell-gargantext
haskell-gargantext
Commit afdbe31d authored by Grégoire Locqueville's avatar Grégoire Locqueville
2 Browse files
Options

Improve onboarding experience 

Edited the README to have a linear path to installation.
Also made scripts to run the executables without having to worry
whether they are on our path.
parent 3030272d
Pipeline #6826 failed with stages
          in 48 minutes and 16 seconds
          Hide whitespace changes
          Inline Side-by-side
          Showing with 121 additions and 171 deletions
          README.md
          View file @ afdbe31d
          ......@@ -7,10 +7,9 @@
          #### Table of Contents
          1. [About the project](#about)
          2. [Installation and development](#install)
          3. [Uses cases](#use-cases)
          4. [GraphQL](#graphql)
          5. [PostgreSQL](#postgresql)
          2. [Installation](#install)
          3. [Use cases](#use-cases)
          4. [Notes for developers](#develop)
          ## About the project <a name="about"></a>
          ......@@ -21,107 +20,136 @@ This software is free (as "Libre" in French) software, developed by the CNRS Com
          GarganText Project: this repo builds the backend for the frontend server built by [backend](https://gitlab.iscpif.fr/gargantext/haskell-gargantext).
          ## Installation and development <a name="install"></a>
          ## Installation <a name="install"></a>
          Disclaimer: since this project is still in development, this document remains in progress. Please report and improve this documentation if you encounter any issues.
          ### Prerequisites
          You must have the following installed:
          You need to have the following installed:
          * [GHCup](https://www.haskell.org/ghcup/)
          * [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
          * [Nix](https://nixos.org/download/)
          * [Docker Compose](https://docs.docker.com/compose/install/).
          - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
          - [Curl](https://everything.curl.dev/index.html)
          - [Nix](https://nixos.org/download/)
          - [Docker Compose](https://docs.docker.com/compose/install/)
          ### Building
          You will need to add yourself as a member of the `docker` group for Docker
          Compose to work (replace `myusername` with your username in the following
          command):
          ```bash
          sudo usermod -a -G docker myusername
          ```
          #### Clone the projects
          ### Getting the source code
          Clone both the backend (`haskell-gargantext`), and the frontend (`purescript-gargantext`) at the root of the backend.
          ```shell
          $ git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
          $ cd haskell-gargantext
          $ git clone https://gitlab.iscpif.fr/gargantext/purescript-gargantext.git
          $ cd ..
          ```bash
          git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
          cd haskell-gargantext
          git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
          cd ..
          ```
          #### The Nix shell
          ### Building Gargantext
          In what follows, many commands need to be executed from within the Nix shell. To make that clear, those will be prefixed with `n$`, *but you must not actually type `n$` before the commands*.
          To enter a Nix shell, run the following (this will take a moment the first time you run it, be patient):
          ```shell
          $ nix-shell
          #### Building the Backend
          Run the install script from inside the `haskell-gargantext/` directory:
          ```bash
          ./bin/install
          ```
          Once you are in a Nix shell, you can run commands like you would in any other shell.
          This will build Gargantext, run the tests, and build the documentation.
          At any point, you can exit a Nix shell and go back to your regular shell by running `exit`.
          #### Building the Frontend
          If for some reason you do not want to enter a Nix shell, you can still run a command from outside: running the following in a non-Nix shell
          ```shell
          $ nix-shell --run "my command"
          Move into the `purescript-gargantext/` directory, then run the install script:
          ```bash
          cd purescript-gargantext
          ./bin/install
          cd ..
          ```
          is equivalent to running `my command` from within a Nix shell.
          #### (Optional) Disable optimization flags
          ### Initialization
          Rename the `gargantext-settings.toml_toModify` into `gargantext-settings.toml`,
          and fill in the missing fields.
          If you are developing Gargantext, you might be interested in disabling compiler optimizations.
          This speeds up compilation, but the compiled program itself will be less efficient.
          > `.gitignore` excludes this file, so you don't need to worry about committing
          > it by mistake, and you can change the passwords in `gargantext-settings.toml`
          > safely.
          To disable compiler optimizations, copy the file `cabal.project.local_toCopy` (which contains the flags that disable optimizations) into `cabal.project.local` (which will be read by Cabal):
          **NOTE** If you had the `gargantext.ini` file before, you can automatically
          convert it into a file `gargantext-settings.toml` by running the following
          from a Nix shell:
          ```shell
          $ cp cabal.project.local_toCopy cabal.project.local
          n$ ./bin/cli ini
          ```
          #### Build the frontend
          ### Launching and Initializing the Database
          ```shell
          $ cd purescript-gargantext/
          $ ./bin/install
          $ cd ..
          Launch the docker image from inside the dedicated directory:
          ```bash
          cd devops/docker/
          docker compose up
          ```
          #### Build the backend
          Then initialize the database using the dedicated command: from inside the
          `haskell-gargantext/` directory, run
          ```bash
          ./bin/cli init
          ```
          *Note: This project can be built with either stack or cabal. We keep the `cabal.project` up-to-date, which allows us to build with cabal by default but we support stack thanks to thanks to `cabal2stack`, which allows us to generate a valid `stack.yaml` from a `cabal.project`. Due to the fact gargantext requires a particular set of system dependencies (C++ libraries, toolchains, etc) we use nix to setup an environment with all the required system dependencies, in a sandboxed and isolated fashion.*
          And provide a name and a password for the master user as requested.
          *This documentation shows how to build with cabal. For information related to stack, see [the note about Stack](https://write.frame.gargantext.org/s/2b50f559075be33faad884e13899a66fc6865c41b97d87772f8bc447234995f7) in the developer documentation.*
          ### Running Gargantext
          Depending on your situation, there are several ways to build the project:
          From inside the `haskell-gargantext/` directory, run
          ```bash
          ./bin/run
          ```
          1. **Simple build**
          ## Use Cases <a name="use-cases"></a>
          *This will build the project and install the executables `gargantext-cli` and `gargantext-server` somewhere on your system.
          Depending on your Cabal configuration, this is probably `~/.local/bin/` or `~/.cabal/bin/`.*
          ### Multi-User with Graphical User Interface (Server Mode)
          From within the Nix shell, run:
          ```shell
          n$ cabal update
          n$ cabal install
          ``` shell
          $ ~/.local/bin/stack --docker exec gargantext-server -- --run Prod
          ```
          2. **Full build**
          Then you can log in with `user1` / `1resu`
          *Same as "simple build" above, but also runs tests and builds documentation.*
          Just run the `install` script:
          ### Command Line Mode tools
          ```shell
          $ ./bin/install
          #### Simple cooccurrences computation and indexation from a list of Ngrams
          ``` shell
          $ stack --docker exec gargantext-cli -- CorpusFromGarg.csv ListFromGarg.csv Ouput.json
          ```
          3. **Build and run**
          ### Analyzing the ngrams table repo
          We store the repository in directory `repos` in the [CBOR](https://cbor.io/) file format. To decode it to JSON and analyze, say, using [jq](https://shapeshed.com/jq-json/), use the following command:
          *Builds and runs the Gargantext server. This has the advantage of letting you run Gargantext without having to know where on your machine the executable is.*
          ``` shell
          $ cat repos/repo.cbor.v5 | stack exec gargantext-cbor2json | jq .
          ```
          ### Documentation
          *Since you will be running Gargantext,* **you need to have gone through initialization first;** *see "Initializing and running" below.*
          To build documentation, run:
          From inside a Nix shell:
          ```shell
          n$ cabal run gargantext-server -- --toml gargantext-settings.toml --run Prod
          $ stack build --haddock --no-haddock-deps --fast
          ```
          #### Upgrading haskell packages
          (in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`).
          ## Notes for developers <a name="develop"></a>
          ### Developer Documentation
          If you want to contribute to Gargantext, there is [some documentation geared to developers](https://write.frame.gargantext.org/s/805e1ee2bae24079554d24cdbc8ef5ba5c4ef7d83218c1e86c84af8ac269031d).
          ### Upgrading Haskell Packages
          We use `gargantext.cabal`, `cabal.project` and `cabal.project.freeze`
          as the source of truth. Ouf ot that, we generate the `stack.yaml` file
          ......@@ -173,54 +201,7 @@ with `cabal v2-build all`).
          Also, here is a relevant discussion:
          https://discourse.haskell.org/t/whats-your-workflow-to-update-cabal-dependencies/9475
          ### Initializing and running
          #### Start containers for database and NLP software bricks
          ```shell
          $ cd devops/docker
          $ docker compose up
          ```
          The initialization schema should be loaded automatically from `devops/postgres/schema.sql`.
          #### Create configuration file
          ```shell
          $ cp gargantext-settings.toml_toModify gargantext-settings.toml
          ```
          **NOTE** If you had the `gargantext.ini` file before, you can automatically
          convert it into a file `gargantext-settings.toml` by running the following
          from a Nix shell:
          ```shell
          n$ cabal v2-run gargantext-cli -- ini
          ```
          > `.gitignore` excludes this file, so you don't need to worry about
          > committing it by mistake, and you can change the passwords in
          > `gargantext-settings.toml` safely.
          #### Create master user
          From within the Nix shell:
          ```shell
          n$ gargantext-cli init
          ```
          The master user's name is automatically set to `gargantua`, but you will be prompted for their password and email address.
          #### Running
          Make sure you know where `gargantext-server` is (probably in `~/.local/bin/` or `.cabal/bin/`). If the location is in your `$PATH`, just run:
          ```shell
          $ gargantext-server --run Prod --toml gargantext-settings.toml
          ```
          (If the location is not in your `$PATH`, just prefix `gargantext-server` with the path to it.)
          You might want to use the `./start` script: it rebuilds the backend, starts the docker containers, and launches the Gargantext server at once.
          ### Running tests
          ### Running Tests
          From nix shell:
          ......@@ -265,7 +246,7 @@ export TMPDIR=$(pwd)/_build
          ### Working on libraries
          When a devlopment is needed on libraries (for instance, the HAL crawler in https://gitlab.iscpif.fr/gargantext/crawlers):
          When a development is needed on libraries (for instance, the HAL crawler in https://gitlab.iscpif.fr/gargantext/crawlers):
          1. Ongoing devlopment (on local repo):
          1. In `cabal.project`:
          ......@@ -280,56 +261,13 @@ When a devlopment is needed on libraries (for instance, the HAL crawler in https
          > Note: without `stack.yaml` we would have to only fix `cabal.project` -> `source-repository-package` commit id. Sha256 is there to make sure CI reruns the tests.
          ### Tooling info
          Once you get Gargantext to compile and run on your machine, you will likely want
          the following:
          - Language support (intellisense) in your editor; see `docs/editor_setup.md`
          - Being able to send commands to the Gargantext server from GHCI; see `docs/running_commands.md`
          ## Use Cases <a name="use-cases"></a>
          ### Multi-User with Graphical User Interface (Server Mode)
          ``` shell
          $ ~/.local/bin/stack --docker exec gargantext-server -- --run Prod
          ```
          Then you can log in with `user1` / `1resu`
          ### Command Line Mode tools
          #### Simple cooccurrences computation and indexation from a list of Ngrams
          ``` shell
          $ stack --docker exec gargantext-cli -- CorpusFromGarg.csv ListFromGarg.csv Ouput.json
          ```
          ### Analyzing the ngrams table repo
          We store the repository in directory `repos` in the [CBOR](https://cbor.io/) file format. To decode it to JSON and analyze, say, using [jq](https://shapeshed.com/jq-json/), use the following command:
          ``` shell
          $ cat repos/repo.cbor.v5 | stack exec gargantext-cbor2json | jq .
          ```
          ### Documentation
          To build documentation, run:
          ```shell
          $ stack build --haddock --no-haddock-deps --fast
          ```
          (in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`).
          ## GraphQL <a name="graphql"></a>
          ### GraphQL
          Some introspection information.
          Playground is located at http://localhost:8008/gql
          ### List all GraphQL types in the Playground
          #### List all GraphQL types in the Playground
          ```
          {
          ......@@ -341,7 +279,7 @@ Playground is located at http://localhost:8008/gql
          }
          ```
          ### List details about a type in GraphQL
          #### List details about a type in GraphQL
          ```
          {
          ......@@ -356,9 +294,9 @@ Playground is located at http://localhost:8008/gql
          }
          }
          ```
          ## PostgreSQL <a name="pgsql"></a>
          ### PostgreSQL <a name="pgsql"></a>
          ### Upgrading using Docker
          #### Upgrading using Docker
          https://www.cloudytuts.com/tutorials/docker/how-to-upgrade-postgresql-in-docker-and-kubernetes/
          ......@@ -383,7 +321,7 @@ $ # now we can restore the dump
          $ docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump
          ```
          ### Upgrading using
          #### Upgrading using
          There is a solution using pgupgrade_cluster but you need to manage the clusters version 14 and 13. Hence here is a simple solution to upgrade.
          ......@@ -409,14 +347,3 @@ Maybe you need to restore the gargantua password
          $ ALTER ROLE gargantua PASSWORD 'yourPasswordIn_gargantext-settings.toml'
          ```
          Maybe you need to change the port to 5433 for database connection in your gargantext.ini file.
          ## `haskell-language-server`
          If you want to use `haskell-language-server` for GHC 9.4.7, install it
          with `ghcup`:
          ```shell
          ghcup compile hls --version 2.7.0.0 --ghc 9.4.7
          ```
          https://haskell-language-server.readthedocs.io/en/latest/installation.html
          bin/cli 0 → 100755
          View file @ afdbe31d
          # The following line is more portable than just /bin/bash:
          #!/usr/bin/env bash
          # A couple hygienic options
          set -e -u
          # The following command will run `cabal run gargantext-cli --` followed by the
          # options provided by the user, from inside a Nix shell. For instance,
          # if the user types
          # $ ./bin/cli someCommand "some string argument"
          # the following will be run from inside a Nix shell:
          # $ cabal run gargantext-cli -- someCommand "some string argument"
          # It's a little convoluted because we want to keep spaces that were enclosed in
          # quotes or escaped by the user.
          nix-shell --run "$(printf "%q " cabal run gargantext-cli -- "$@")"
          bin/run 0 → 100755
          View file @ afdbe31d
          # The following line is more portable than just /bin/bash:
          #!/usr/bin/env bash
          # A couple hygienic options
          set -e -u
          echo "Launching Gargantext..."
          nix-shell --run "cabal run gargantext-server -- --run Prod --toml gargantext-settings.toml"
          Markdown is supported
          0% or
          You are about to add 0 people to the discussion. Proceed with caution.
          Finish editing this message first!
          Please register or to comment