Merge branch 'dev' into 175-dev-doc-table-count-temp

CHANGELOG.md

+## Version 0.0.6.9.4.5
+
+* [BACK][CLEAN] Switch to Hackage version of hstatistics
+* [BACK][CLEAN] Drop unused servant-static-th
+* [FRONT][CLEAN] README 
+* [FIX][CRALER] ArXiv API
+
+## Version 0.0.6.9.4.4
+
+* [BACK][Add optional Accelerate dependency on Darwin for pkgs.nix](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/merge_requests/128)
+* [BACK][Crawlers] pubmed clean unused deps removed
+* [BACK][CLEAN][Explore cutting down forks/extra-deps of libraries (#180)](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/issues/180)
+* [BACK][DOC] README
+
+## Version 0.0.6.9.4.3
+
+* [BACK][DOC][Welcome: Door To enter the project (#177)](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/issues/177)
+* [FRONT][DOC][Door to entry: Welcome! (#269)](https://gitlab.iscpif.fr/gargantext/purescript-gargantext/issues/269)
+* [BACK][REFACT][Doc Table: count (#175)](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/issues/175)
+
 ## Version 0.0.6.9.4.2
 
 * [FRONT][FIX][Regression : on graph (#496)](https://gitlab.iscpif.fr/gargantext/purescript-gargantext/issues/496)

README.md

+<div align="center"><img height="180" src="https://gitlab.iscpif.fr/gargantext/main/images/logo.png"></div>
+
+&nbsp;
 # Gargantext with Haskell (Backend instance)
 
-## About the project
+![Haskell](https://img.shields.io/badge/Code-Haskell-informational?style=flat&logo=haskell&color=6144b3)&nbsp;&nbsp;![Stack](https://img.shields.io/badge/Tools-Stack-informational?style=flat&logo=&color=6144b3)&nbsp;&nbsp;![GHC](https://img.shields.io/badge/Tools-GHC-informational?style=flat&logo=&color=2E677B)&nbsp;&nbsp;![Nix](https://img.shields.io/badge/Package%20manager-Nix-informational?style=flat&logo=debian&color=6586c8)&nbsp;&nbsp;![Docker](https://img.shields.io/badge/Tools-Docker-informational?style=flat&logo=docker&color=003f8c)
+
+#### Table of Contents
+1. [About the project](#about)
+2. [Example2](#example2)
+3. [Third Example](#third-example)
+4. [Fourth Example](#fourth-examplehttpwwwfourthexamplecom)
+
+## About the project  <a name="about"></a>
 
 GarganText is a collaborative web-decentralized-based macro-service
 platform for the exploration of unstructured texts. It combines tools
-from natural language processing, text-data-mining tricks, complex
-networks analysis algorithms and interactive data visualization tools to
-pave the way toward new kinds of interactions with your digital corpora.
+from natural language processing, text-data-mining bricks, complex
+networks analysis algorithms and interactive data visualization tools
+to pave the way toward new kinds of interactions with your textual and
+digital corpora.
 
-This software is free software, developed and offered by the CNRS
-Complex Systems Institute of Paris Île-de-France (ISC-PIF) and its
+This software is free (as "Libre" in French) software, developed by the
+CNRS Complex Systems Institute of Paris Île-de-France (ISC-PIF) and its
 partners.
 
-GarganText Project: this repo builds the
-backend for the frontend server built by
+GarganText Project: this repo builds the backend for the frontend server built by
 [backend](https://gitlab.iscpif.fr/gargantext/haskell-gargantext).
 
 
 ## Installation
 
-Disclaimer: this project is still in development, this is work in
-progress. Please report and improve this documentation if you encounter issues.
+Disclaimer: since this project is still in development, this document
+remains in progress. Please report and improve this documentation if you
+encounter any issues.
 
-### Stack setup
+### Prerequisite
 
-You need to install stack first:
+Clone the project.
+```shell
+git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
+cd haskell-gargantext
+```
+### 1. Install Stack
+
+Install [Stack (or Haskell Tool Stack)](https://docs.haskellstack.org/en/stable/):
 
 ```shell
 curl -sSL https://get.haskellstack.org/ | sh
@@ -33,152 +52,85 @@ curl -sSL https://get.haskellstack.org/ | sh
 Verify the installation is complete with
 ```shell
 stack --version
+Version 2.9.1
 ```
 
-### With Nix setup
+### 2. Install Nix
 
-First install [nix](https://nixos.org/guides/install-nix.html): 
+Install [Nix](https://nixos.org/download.html):
 
 ```shell
-curl -sSL https://nixos.org/nix/install | sh
+$ sh <(curl -L https://nixos.org/nix/install) --daemon
 ```
 
-Verify the installation is complete
+Verify the installation is complete with
 ```shell
-$ nix-env
-nix-env (Nix) 2.3.12
-```
-And just build:
-``` sh
-stack --nix build --fast
+$ nix-env --version
+nix-env (Nix) 2.12.0
 ```
 
-### Build Core Code
+> **NOTE INFO (upgrade/downgrade if needed)**
+> Gargantext works with Nix 2.12.0 (older version than current 2.13.2). To downgrade your Nix version:
+> `nix-channel --update; nix-env -iA nixpkgs.nixVersions.nix_2_12 nixpkgs.cacert; systemctl daemon-reload; systemctl restart nix-daemon`
+> Upgrading Nix: https://nixos.org/manual/nix/unstable/installation/upgrading.html
+
+
+### 3. Build Core Code
 
 NOTE: Default build (with optimizations) requires large amounts of RAM
 (16GB at least). To avoid heavy compilation times and swapping out your
-machine, it is recommended to `stack build` with the `--fast-` flag,
+machine, it is recommended to `stack build` with the `--fast` flag,
 i.e.:
 
 ``` sh
 stack --nix build --fast
 ```
-or
-
-``` sh
-stack --docker build --fast
-```
-
-#### Docker
-
-``` sh
-curl -sSL https://gitlab.iscpif.fr/gargantext/haskell-gargantext/raw/dev/devops/docker/docker-install | sh
-```
-
-#### Debian
 
-``` sh
-curl -sSL https://gitlab.iscpif.fr/gargantext/haskell-gargantext/raw/dev/devops/debian/install | sh
-```
-
-#### Ubuntu
-
-``` sh
-curl -sSL https://gitlab.iscpif.fr/gargantext/haskell-gargantext/raw/dev/devops/ubuntu/install | sh
-```
+If the build is finishing without error, you are ready to launch
+GarganText! See next step.
 
-### Add dependencies
+&nbsp;
+&nbsp;
+&nbsp;
+&nbsp;
+&nbsp;
 
-1. CoreNLP is needed (EN and FR); This dependency will not be needed soon.
-
-``` sh
-./devops/install-corenlp
-```
-### Launch Gargantext (It should have been initialized first)
-Run
-haskell-gargantext/devops/docker$ docker-compose up
-
-Then 
-``` sh
-stack --nix install
-~/.local/bin/gargantext-init "gargantext.ini"
-```
-Or for Docker env, first create the appropriate image:
-
-``` sh
-cd devops/docker
-docker build -t cgenie/stack-build:lts-18.12-garg .
-```
-
-then run:
-
-``` sh
-stack --docker exec gargantext-init -- gargantext.ini
 
 ### Initialization
 
-#### Docker
-
-Run PostgreSQL first:
+Docker-compose will configure your database and some NLP bricks (such as CoreNLP):
 
 ``` sh
+# If docker is not installed:
+# curl -sSL https://gitlab.iscpif.fr/gargantext/haskell-gargantext/raw/dev/devops/docker/docker-install | sh
 cd devops/docker
-docker-compose up
+docker compose up
 ```
-
 Initialization schema should be loaded automatically (from `devops/postgres/schema.sql`).
 
-#### Gargantext
-
-##### Fix the passwords
-
-Change the passwords in gargantext.ini_toModify then move it:
-
-``` sh
-cp gargantext.ini_toModify gargantext.ini
-```
-(`.gitignore` avoids adding this file to the repository by mistake)
-
-
-##### Run Gargantext
-
-Users have to be created first (`user1` is created as instance):
-
+Then install:
 ``` sh
 stack --nix install
-~/.local/bin/gargantext-init "gargantext.ini"
-```
-
-For Docker env, first create the appropriate image:
-
-``` sh
-cd devops/docker
-docker build -t cgenie/stack-build:lts-18.12-garg .
 ```
 
-then run:
-
+Copy the configuration file:
 ``` sh
-stack --docker exec gargantext-init -- gargantext.ini
+cp gargantext.ini_toModify gargantext.ini
 ```
+Do not worry, `.gitignore` avoids adding this file to the repository by
+mistake, then you can change the passwords in gargantext.ini safely.
 
-### Importing data
-
-You can import some data with:
+Users have to be created first (`user1` is created as instance):
 ``` sh
-docker run --rm -it -p 9000:9000 cgenie/corenlp-garg
-stack exec gargantext-import -- "corpusCsvHal" "user1" "IMT3" gargantext.ini 10000 ./1000.csv
+~/.local/bin/gargantext-init "gargantext.ini"
 ```
 
-### Nix
-
-It is also possible to build everything with [Nix](https://nixos.org/) instead of Docker:
+Launch GarganText:
 ``` sh
-stack --nix build
-stack --nix exec gargantext-import -- "corpusCsvHal" "user1" "IMT3" gargantext.ini 10000 ./1000.csv
 stack --nix exec gargantext-server -- --ini gargantext.ini --run Prod
 ```
 
+
 ## Use Cases
 
 ### Multi-User with Graphical User Interface (Server Mode)
@@ -212,7 +164,7 @@ cat repos/repo.cbor.v5 | stack --nix exec gargantext-cbor2json | jq .
 To build documentation, run:
 
 ```sh
-stack --docker build --haddock --no-haddock-deps --fast
+stack --nix build --haddock --no-haddock-deps --fast
 ```
 
 (in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`).

gargantext.cabal

@@ -5,7 +5,7 @@ cabal-version: 1.12
 -- see: https://github.com/sol/hpack
 
 name:           gargantext
-version:        0.0.6.9.4.2
+version: 0.0.6.9.4.5
 synopsis:       Search, map, share
 description:    Please see README.md
 category:       Data
@@ -76,6 +76,7 @@ library
       Gargantext.Core.Viz.Graph.Index
       Gargantext.Core.Viz.Graph.Tools
       Gargantext.Core.Viz.Graph.Tools.IGraph
+      Gargantext.Core.Viz.Graph.Types
       Gargantext.Core.Viz.Phylo
       Gargantext.Core.Viz.Phylo.API
       Gargantext.Core.Viz.Phylo.API.Tools
@@ -234,7 +235,6 @@ library
       Gargantext.Core.Viz.Graph.Legend
       Gargantext.Core.Viz.Graph.PatriciaTreeTypes
       Gargantext.Core.Viz.Graph.Tools.Infomap
-      Gargantext.Core.Viz.Graph.Types
       Gargantext.Core.Viz.Graph.Utils
       Gargantext.Core.Viz.LegacyPhylo
       Gargantext.Core.Viz.Phylo.Example
@@ -431,7 +431,6 @@ library
     , matrix
     , monad-control
     , monad-logger
-    , monad-logger-aeson
     , morpheus-graphql
     , morpheus-graphql-app
     , morpheus-graphql-core
@@ -480,7 +479,6 @@ library
     , servant-mock
     , servant-multipart
     , servant-server
-    , servant-static-th
     , servant-swagger
     , servant-swagger-ui
     , servant-xml

nix/pkgs.nix

@@ -32,7 +32,9 @@ rec {
     icu
     graphviz
     llvm_9
-  ];
+  ] ++ ( lib.optionals stdenv.isDarwin [
+       darwin.apple_sdk.frameworks.Accelerate
+       ]);
   libPaths = pkgs.lib.makeLibraryPath nonhsBuildInputs;
   shellHook = ''
     export LD_LIBRARY_PATH="${pkgs.gfortran7.cc.lib}:${libPaths}:$LD_LIBRARY_PATH"

package.yaml

@@ -6,7 +6,7 @@ name: gargantext
 #         | | | +-----  Layers * : New versions with API additions
 #         | | | | +---  Layers * : New versions without API breaking changes
 #         | | | | |
-version: '0.0.6.9.4.2'
+version: '0.0.6.9.4.5'
 synopsis: Search, map, share
 description: Please see README.md
 category: Data
@@ -101,6 +101,7 @@ library:
   - Gargantext.Core.Viz.Graph.Index
   - Gargantext.Core.Viz.Graph.Tools
   - Gargantext.Core.Viz.Graph.Tools.IGraph
+  - Gargantext.Core.Viz.Graph.Types
   - Gargantext.Core.Viz.Phylo
   - Gargantext.Core.Viz.Phylo.API
   - Gargantext.Core.Viz.Phylo.API.Tools
@@ -215,7 +216,6 @@ library:
   - matrix
   - monad-control
   - monad-logger
-  - monad-logger-aeson
   - morpheus-graphql
   - morpheus-graphql-app
   - morpheus-graphql-core
@@ -264,7 +264,6 @@ library:
   - servant-mock
   - servant-multipart
   - servant-server
-  - servant-static-th
   - servant-swagger
   - servant-swagger-ui
   - servant-xml

src-test/Graph/Clustering.hs

@@ -13,7 +13,8 @@ Portability : POSIX
 module Graph.Clustering where
 
 import Gargantext.API.Ngrams.Types (NgramsTerm(..))
-import Gargantext.Core.Viz.Graph (Graph(..), Strength(..))
+import Gargantext.Core.Viz.Graph
+import Gargantext.Core.Viz.Graph.Types
 import Gargantext.Core.Viz.Graph.Tools (doSimilarityMap)
 import Gargantext.Core.Viz.Graph.Tools.IGraph (spinglass)
 import Gargantext.Prelude

stack.yaml

@@ -42,10 +42,10 @@ extra-deps:
   commit: 10a416b9f6c443866b36479c3441ebb3bcdeb7ef
 - git: https://gitlab.iscpif.fr/gargantext/hlcm.git
   commit: 6f0595d2421005837d59151a8b26eee83ebb67b5
-- git: https://github.com/delanoe/hstatistics.git
-  commit: 90eef7604bb230644c2246eccd094d7bfefcb135
 - git: https://github.com/paulrzcz/HSvm.git
   commit: 3fe28b683aba5ddf05e3b5f8eced0bd05c5a29f9
+- git: https://github.com/delanoe/hstatistics.git
+  commit: 90eef7604bb230644c2246eccd094d7bfefcb135
 
   # API libs
 - git: https://github.com/haskell-servant/servant.git
@@ -58,8 +58,6 @@ extra-deps:
     - servant-auth/servant-auth/
     - servant-auth/servant-auth-client/
     - servant-auth/servant-auth-server/
-- git: https://github.com/delanoe/servant-static-th.git
-  commit: 8cb8aaf2962ad44d319fcea48442e4397b3c49e8
 - git: https://github.com/alpmestan/ekg-json.git
   commit: fd7e5d7325939103cd87d0dc592faf644160341c
 
@@ -74,8 +72,7 @@ extra-deps:
 
   # External Data API connectors
 - git: https://gitlab.iscpif.fr/gargantext/crawlers/pubmed.git
-  commit: 31cb4d28dcb5d17274cede5e67b2a01914379129
-  #commit: 364885c891cbadcd4d8a623d2e41394b09f653aa
+  commit: 4ade495751eaf31d3ca1ac8b0ae13d3538c6e18c
 - git: https://gitlab.iscpif.fr/gargantext/crawlers/istex.git
   commit: a34bb341236d82cf3d488210bc1d8448a98f5808
 - git: https://gitlab.iscpif.fr/gargantext/crawlers/hal.git
@@ -83,7 +80,7 @@ extra-deps:
 - git: https://gitlab.iscpif.fr/gargantext/crawlers/isidore.git
   commit: 3db385e767d2100d8abe900833c6e7de3ac55e1b
 - git: https://gitlab.iscpif.fr/gargantext/crawlers/arxiv-api.git
-  commit: a2d78abeaec9315be765b90d5e51a4a50c48e7b8
+  commit: 2d7e5753cbbce248b860b571a0e9885415c846f7
 #- git: https://gitlab.iscpif.fr/cgenie/arxiv-api.git
 #- arxiv-0.0.3@sha256:02de1114091d11f1f3ab401d104d125ad4301260806feb7f63b3dcefc7db88cf,1588
 
@@ -178,3 +175,4 @@ ghc-options:
   hmatrix: -O2 -fsimpl-tick-factor=10000 -fdicts-cheap -fdicts-strict -flate-dmd-anal  -fno-state-hack
   sparse-linear: -O2 -fsimpl-tick-factor=10000 -fdicts-cheap -fdicts-strict -flate-dmd-anal  -fno-state-hack
   gargantext-graph: -O2 -fsimpl-tick-factor=10000 -fdicts-cheap -fdicts-strict -flate-dmd-anal  -fno-state-hack
+  "$locals": -fwrite-ide-info -hiedir=".stack-work/hiedb"

weeder.dhall

+{ roots = [ "^Main\\.main\$"
+          , "^Paths_.*"
+          ], type-class-roots = True }