Squashed commit of the following:

commit 27413399
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 15 16:17:10 2024 +0200

    [VERSION] +1 to 0.0.7.3.3

commit f1d6a083
Merge: 0143f243 afdbe31d
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 15 15:45:40 2024 +0200

    Merge remote-tracking branch 'origin/improve-onboarding-experience' into dev

commit 0143f243
Author: Alfredo Di Napoli <alfredo.dinapoli@gmail.com>
Date:   Mon Oct 14 13:45:12 2024 +0200

    Fix step1c again

commit 43901901
Author: Alfredo Di Napoli <alfredo.dinapoli@gmail.com>
Date:   Mon Oct 14 12:39:43 2024 +0200

    Fix bug in GargPorter implementation

commit d44831d0
Author: Alfredo Di Napoli <alfredo.dinapoli@gmail.com>
Date:   Mon Oct 14 10:50:31 2024 +0200

    Introduce SpecContext, replicate search issue

    This commit refactors the test code ever so slightly to add a proper
    type called `SpecContext` rather than anonymous pairs to carry around
    the spec context/data. It also replicates the search problem around #415
    via a test.

commit b4260d9b
Author: Alfredo Di Napoli <alfredo.dinapoli@gmail.com>
Date:   Mon Oct 14 09:11:51 2024 +0200

    Add soysauce docs list to test-data

commit afdbe31d
Author: Grégoire Locqueville <gregoire.locqueville@cnrs.fr>
Date:   Thu Oct 10 18:51:46 2024 +0200

    Improve onboarding experience

    Edited the README to have a linear path to installation.
    Also made scripts to run the executables without having to worry
    whether they are on our path.

commit 3030272d
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Thu Oct 10 18:31:19 2024 +0200

    [VERSION] +1 to 0.0.7.3.2

commit bd33dd6c
Merge: 1d3417d9 592d966c
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Thu Oct 10 18:10:51 2024 +0200

    Merge remote-tracking branch 'origin/dev-websockets-node-update' into dev

commit 592d966c
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Thu Oct 10 18:06:20 2024 +0200

    [notifications] add missing test/Test/Core/Notifications.hs

commit 1d3417d9
Merge: 163304df a48fe0c8
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Thu Oct 10 17:54:39 2024 +0200

    Merge remote-tracking branch 'origin/dev-websockets-node-update' into dev

commit a48fe0c8
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Thu Oct 10 10:01:10 2024 +0200

    [ws] rename AsyncUpdates to Notifications

    This is bit more clear

commit cd831db4
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Oct 9 16:00:48 2024 +0200

    [tests] first working notification test

commit 81af005d
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Oct 9 11:05:11 2024 +0200

    Squashed commit of the following:

    commit 163304df
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 18:39:54 2024 +0200

        [FIX] conflict

    commit 82c68074
    Merge: f7b76918 5623161c
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 18:28:55 2024 +0200

        Merge remote-tracking branch 'origin/dev-websockets-node-update' into dev

    commit f7b76918
    Merge: fe7a92cc 88655f68
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 18:28:53 2024 +0200

        [FIX] conflicts

    commit fe7a92cc
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 17:19:53 2024 +0200

        fix: no longer update graphs and phylos on corpus update

    commit f775d4a3
    Merge: 76b557ea d2f4b89d
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 16:27:53 2024 +0200

        Merge remote-tracking branch 'origin/dev-guidelines-update' into dev

    commit 76b557ea
    Merge: 2925d008 50c77ea2
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 16:27:27 2024 +0200

        Merge remote-tracking branch 'origin/304-dev-pubmed-api-not-in-toml' into dev

    commit d2f4b89d
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 15:37:54 2024 +0200

        DEVELOPER_GUIDELINES: update about git amend

        This is the result of Autumn workshop 2024

    commit 50c77ea2
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 15:15:28 2024 +0200

        [notifications] fix for send

        sendNonblocking threw an error initially. I just do a compromise and
        timeout the normal send (which blocks infinitely sometimes)

    commit 025b80b6
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 14:10:56 2024 +0200

        [docker] fix network: host, fix caddyfile

    commit 2925d008
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 10:34:17 2024 +0200

        fix arbitrary instance

    commit e8fb3db6
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 10:13:40 2024 +0200

        fix: re-add lost instances

    commit b86d2e61
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 10:09:18 2024 +0200

        [toml] remove pubmed api key from config

        It's set up in user settings instead and has been for a long time.

    commit c06de5ef
    Merge: ab710337 a0ec337b
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 09:35:55 2024 +0200

        Merge remote-tracking branch 'gitlab/dev' into cm/update-corpus-button

    commit ab710337
    Author: Christian Merten <christian@merten.dev>
    Date:   Fri Apr 26 22:32:33 2024 +0200

        feat: update corpus endpoint

commit d4a9200e
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Oct 9 11:01:43 2024 +0200

    [ws] notification action on node share

commit 163304df
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 8 18:39:54 2024 +0200

    [FIX] conflict

commit 82c68074
Merge: f7b76918 5623161c
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 8 18:28:55 2024 +0200

    Merge remote-tracking branch 'origin/dev-websockets-node-update' into dev

commit f7b76918
Merge: fe7a92cc 88655f68
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 8 18:28:53 2024 +0200

    [FIX] conflicts

commit 5623161c
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 18:18:23 2024 +0200

    [ws] implement node update (rename, move) with notifications to parents

commit 88655f68
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 18:05:25 2024 +0200

    Squashed commit of the following:

    commit f775d4a3
    Merge: 76b557ea d2f4b89d
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 16:27:53 2024 +0200

        Merge remote-tracking branch 'origin/dev-guidelines-update' into dev

    commit 76b557ea
    Merge: 2925d008 50c77ea2
    Author: Alexandre Delanoë <devel+git@delanoe.org>
    Date:   Tue Oct 8 16:27:27 2024 +0200

        Merge remote-tracking branch 'origin/304-dev-pubmed-api-not-in-toml' into dev

    commit d2f4b89d
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 15:37:54 2024 +0200

        DEVELOPER_GUIDELINES: update about git amend

        This is the result of Autumn workshop 2024

    commit 50c77ea2
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 15:15:28 2024 +0200

        [notifications] fix for send

        sendNonblocking threw an error initially. I just do a compromise and
        timeout the normal send (which blocks infinitely sometimes)

    commit 025b80b6
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 14:10:56 2024 +0200

        [docker] fix network: host, fix caddyfile

    commit 2925d008
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 10:34:17 2024 +0200

        fix arbitrary instance

    commit e8fb3db6
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 10:13:40 2024 +0200

        fix: re-add lost instances

    commit b86d2e61
    Author: Przemysław Kaminski <pk@intrepidus.pl>
    Date:   Tue Oct 8 10:09:18 2024 +0200

        [toml] remove pubmed api key from config

        It's set up in user settings instead and has been for a long time.

    commit c06de5ef
    Merge: ab710337 a0ec337b
    Author: Christian Merten <christian@merten.dev>
    Date:   Tue Oct 8 09:35:55 2024 +0200

        Merge remote-tracking branch 'gitlab/dev' into cm/update-corpus-button

    commit ab710337
    Author: Christian Merten <christian@merten.dev>
    Date:   Fri Apr 26 22:32:33 2024 +0200

        feat: update corpus endpoint

commit fe7a92cc
Author: Christian Merten <christian@merten.dev>
Date:   Tue Oct 8 17:19:53 2024 +0200

    fix: no longer update graphs and phylos on corpus update

commit f775d4a3
Merge: 76b557ea d2f4b89d
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 8 16:27:53 2024 +0200

    Merge remote-tracking branch 'origin/dev-guidelines-update' into dev

commit 76b557ea
Merge: 2925d008 50c77ea2
Author: Alexandre Delanoë <devel+git@delanoe.org>
Date:   Tue Oct 8 16:27:27 2024 +0200

    Merge remote-tracking branch 'origin/304-dev-pubmed-api-not-in-toml' into dev

commit d2f4b89d
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 15:37:54 2024 +0200

    DEVELOPER_GUIDELINES: update about git amend

    This is the result of Autumn workshop 2024

commit 50c77ea2
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 15:15:28 2024 +0200

    [notifications] fix for send

    sendNonblocking threw an error initially. I just do a compromise and
    timeout the normal send (which blocks infinitely sometimes)

commit 025b80b6
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 14:10:56 2024 +0200

    [docker] fix network: host, fix caddyfile

commit ee0db8c1
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 12:39:12 2024 +0200

    [nix] add nanomsg to nix pkgs

    Also, fix ./start to use gargantext-settings.toml

commit 2925d008
Author: Christian Merten <christian@merten.dev>
Date:   Tue Oct 8 10:34:17 2024 +0200

    fix arbitrary instance

commit e8fb3db6
Author: Christian Merten <christian@merten.dev>
Date:   Tue Oct 8 10:13:40 2024 +0200

    fix: re-add lost instances

commit b86d2e61
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Oct 8 10:09:18 2024 +0200

    [toml] remove pubmed api key from config

    It's set up in user settings instead and has been for a long time.

commit c06de5ef
Merge: ab710337 a0ec337b
Author: Christian Merten <christian@merten.dev>
Date:   Tue Oct 8 09:35:55 2024 +0200

    Merge remote-tracking branch 'gitlab/dev' into cm/update-corpus-button

commit ab710337
Author: Christian Merten <christian@merten.dev>
Date:   Fri Apr 26 22:32:33 2024 +0200

    feat: update corpus endpoint

CHANGELOG.md

+## Version 0.0.7.3.3
+
+* [FRONT][FIX][Display graph parameters in legend (#706)](https://gitlab.iscpif.fr/gargantext/purescript-gargantext/issues/706)
+* [BACK][FIX][Document Search (#415)](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/issues/415)
+* [BACK][DOC+Scripts][Improving onboarding
+  experience](https://gitlab.iscpif.fr/gargantext/haskell-gargantext/merge_requests/360)
+
 ## Version 0.0.7.3.2
 
 * [FRONT][FIX][[Node Graph] Legend tab improvements (#689)](https://gitlab.iscpif.fr/gargantext/purescript-gargantext/issues/689)

bin/cli

+# The following line is more portable than just /bin/bash:
+#!/usr/bin/env bash
+
+# A couple hygienic options
+set -e -u
+
+# The following command will run `cabal run gargantext-cli --` followed by the
+# options provided by the user, from inside a Nix shell. For instance,
+# if the user types
+# $ ./bin/cli someCommand "some string argument"
+# the following will be run from inside a Nix shell:
+# $ cabal run gargantext-cli -- someCommand "some string argument"
+# It's a little convoluted because we want to keep spaces that were enclosed in
+# quotes or escaped by the user.
+nix-shell --run "$(printf "%q " cabal run gargantext-cli -- "$@")"

bin/run

+# The following line is more portable than just /bin/bash:
+#!/usr/bin/env bash
+
+# A couple hygienic options
+set -e -u
+
+echo "Launching Gargantext..."
+nix-shell --run "cabal run gargantext-server -- --run Prod --toml gargantext-settings.toml"

gargantext.cabal

@@ -5,7 +5,7 @@ cabal-version: 3.4
 -- see: https://github.com/sol/hpack
 
 name:           gargantext
-version: 0.0.7.3.2
+version: 0.0.7.3.3
 synopsis:       Search, map, share
 description:    Please see README.md
 category:       Data
@@ -32,23 +32,24 @@ data-files:
     ekg-assets/bootstrap-1.4.0.min.css
     ekg-assets/chart_line_add.png
     ekg-assets/cross.png
-    test-data/ngrams/GarganText_NgramsTerms-QuantumComputing.json
     test-data/ngrams/GarganText_DocsList-nodeId-177.json
     test-data/ngrams/GarganText_NgramsTerms-nodeId-177.json
+    test-data/ngrams/GarganText_NgramsTerms-QuantumComputing.json
     test-data/ngrams/simple.json
     test-data/ngrams/simple.tsv
+    test-data/phylo/187481.json
     test-data/phylo/bpa_phylo_test.json
     test-data/phylo/cleopatre.golden.json
-    test-data/phylo/nadal.golden.json
+    test-data/phylo/issue-290-small.golden.json
     test-data/phylo/nadal_docslist.golden.tsv
+    test-data/phylo/nadal.golden.json
     test-data/phylo/nadal_ngramslist.golden.tsv
-    test-data/phylo/issue-290-small.golden.json
     test-data/phylo/open_science.json
-    test-data/phylo/small-phylo.golden.json
+    test-data/phylo/phylo2dot2json.golden.json
     test-data/phylo/small_phylo_docslist.tsv
+    test-data/phylo/small-phylo.golden.json
     test-data/phylo/small_phylo_ngramslist.tsv
-    test-data/phylo/187481.json
-    test-data/phylo/phylo2dot2json.golden.json
+    test-data/search/GarganText_DocsList-soysauce.json
     test-data/stemming/lancaster.txt
     test-data/test_config.ini
     test-data/test_config.toml
@@ -734,6 +735,7 @@ common testDependencies
     , epo-api-client
     , extra ^>= 1.7.9
     , fast-logger ^>= 3.2.2
+    , filepath ^>= 1.4.2.2
     , fmt
     , gargantext
     , gargantext-prelude
@@ -809,6 +811,7 @@ test-suite garg-test-tasty
       CLI.Phylo.Common
       Paths_gargantext
       Test.API.Private.Share
+      Test.API.Private.Table
       Test.API.Authentication
       Test.API.Routes
       Test.API.Setup
@@ -869,6 +872,7 @@ test-suite garg-test-hspec
       Test.API.Notifications
       Test.API.Private
       Test.API.Private.Share
+      Test.API.Private.Table
       Test.API.Routes
       Test.API.Setup
       Test.API.UpdateList

src/Gargantext/Core/Text/Terms/Mono/Stem/Internal/GargPorter.hs

@@ -122,9 +122,14 @@ afterStep1b word = fromMaybe word result
 step1b :: [Char] -> [Char]
 step1b = either identity afterStep1b . beforeStep1b
 
+-- Issue #415: According to the Porter stemming rules, we need to replace `y` with `i` only if there
+-- are no other vocals at the end.
 step1c :: [Char] -> [Char]
-step1c word = fromMaybe word result
-    where result = replaceEnd containsVowel word "y" "i"
+step1c word
+  | length word > 2 && List.last word == 'y' && isConsonant word (List.length word - 2)
+  = List.init word <> "i"
+  | otherwise
+  = word
 
 step1 :: [Char] -> [Char]
 step1 = step1c . step1b . step1a

test-data/search/GarganText_DocsList-soysauce.json

+{
+  "documents": [
+    {
+      "document": {
+        "id": 1101563,
+        "hash_id": null,
+        "typename": 4,
+        "user_id": 58,
+        "parent_id": null,
+        "name": "THE EFFECT OF ANTIOXIDANTS ON FROZEN GROUND PORK",
+        "date": "1956-01-01T00:00:00Z",
+        "hyperdata": {
+          "abstract": "The relative effectiveness of monosodium glutamate (MSG), soybean flour, and butylated hydroxyanisole (BHA) as antioxidants for ground pork stored raw, and pork cooked prior to freezer storage, was studied. Peroxide determinations were made at intervals through 18 months of storage, and organoleptic judgments at corresponding intervals through 12 months. Peroxide determinations indicated that soybean flour, BHA, and the cooking process alone inhibited fat oxidation, but MSG did not. On palatability tests, soy-treated pork was rated down on flavor. Samples with MSG received the best scores, but showed rapid peroxide development after 12 months when stored raw. None of the samples became rancid during the first 12 months of storage. At 15 and 18 months, peroxide numbers indicated rancidity in the untreated and in the MSG treated pork stored raw.",
+          "authors": "NEILL, J;  PAGE, L",
+          "bdd": "WOS",
+          "language_iso2": "EN",
+          "publication_date": "1956-01-01 00:00:00 UTC",
+          "publication_day": 1,
+          "publication_month": 1,
+          "publication_year": 1956,
+          "source": "FOOD TECHNOLOGY",
+          "title": "THE EFFECT OF ANTIOXIDANTS ON FROZEN GROUND PORK"
+        }
+      },
+      "ngrams": {
+        "ngrams": [],
+        "hash": ""
+      },
+      "hash": ""
+    },
+    {
+      "document": {
+        "id": 1101539,
+        "hash_id": null,
+        "typename": 4,
+        "user_id": 58,
+        "parent_id": null,
+        "name": "INFLUENCE OF DIETARY PROTEIN LEVEL AND AMINO ACID COMPOSITION ON CHICK;  PERFORMANCE",
+        "date": "1965-01-01T00:00:00Z",
+        "hyperdata": {
+          "abstract": "Studies were designed to investigate the effects of altering dietary protein levels and/or amino acid composition on chick growth and feed efficiency. Contradictory observations in chick performance were made among a series of diets in which the crude protein content was increased from 18 to 22%. Chick weight and feed efficiency was unaffected as dietary protein was increased by replacing cellulose with monosodium glutamate or L-glutamic acid. Chick performance was improved, however, by supplementation of the deficient essential amino acids. Increasing dietary protein concomitantly with essential amino acid supplementation had no effect on chick weight or feed efficiency. In contrast, significant improvements in chick performance were observed in a series of diets where dietary protein was increased by replacing corn with soybean meal.",
+          "authors": "ASKELSON, CE;  BALLOUN, SL",
+          "bdd": "WOS",
+          "doi": "10.3382/ps.0440193",
+          "language_iso2": "EN",
+          "publication_date": "1965-01-01 00:00:00 UTC",
+          "publication_day": 1,
+          "publication_month": 1,
+          "publication_year": 1965,
+          "source": "POULTRY SCIENCE",
+          "title": "INFLUENCE OF DIETARY PROTEIN LEVEL AND AMINO ACID COMPOSITION ON CHICK;  PERFORMANCE"
+        }
+      },
+      "ngrams": {
+        "ngrams": [],
+        "hash": ""
+      },
+      "hash": ""
+    },
+    {
+      "document": {
+        "id": 1102103,
+        "hash_id": null,
+        "typename": 4,
+        "user_id": 58,
+        "parent_id": null,
+        "name": "DILUENT SENSITIVITY IN THERMALLY STRESSED CELLS OF;  PSEUDOMONAS-FLUORESCENS",
+        "date": "1977-01-01T00:00:00Z",
+        "hyperdata": {
+          "abstract": "Thermally injured cells of P. fluorescens cannot produce colonies on Trypticase soy agar (TSA) after dilution with 0.1% peptone. Nutritional exigency could not be used as the criterion for this injury, since varying the composition of the plating medium had little effect on the number of colonies that developed. The injured cells had no requirement for compounds known to leak out during the heat treatment in order to recover. The cells did not exhibit injury if dilution preceded heat treatment on the plating medium, demonstrating that the heat treatment sensitized the cells to the trauma of dilution. Substitution of 0.1% peptone with growth medium as the diluent largely offset the previously observed drop in TSA count. Little difference in survival was observed when monosodium glutamate or the balance of the defined medium was used as the diluent. The diluent effect was ionic rather than osmotic. The presence of cations was important in maintaining the integrity of the injured cell, and divalent cations enhanced this protective effect. The role of these cations at the level of the cell envelope is discussed.",
+          "authors": "GRAY, RJH;  ORDAL, ZJ;  WITTER, LD",
+          "bdd": "WOS",
+          "doi": "10.1128/AEM.33.5.1074-1078.1977",
+          "language_iso2": "EN",
+          "publication_date": "1977-01-01 00:00:00 UTC",
+          "publication_day": 1,
+          "publication_month": 1,
+          "publication_year": 1977,
+          "source": "APPLIED AND ENVIRONMENTAL MICROBIOLOGY",
+          "title": "DILUENT SENSITIVITY IN THERMALLY STRESSED CELLS OF;  PSEUDOMONAS-FLUORESCENS"
+        }
+      },
+      "ngrams": {
+        "ngrams": [],
+        "hash": ""
+      },
+      "hash": ""
+    },
+    {
+      "document": {
+        "id": 1101179,
+        "hash_id": null,
+        "typename": 4,
+        "user_id": 58,
+        "parent_id": null,
+        "name": "GROWTH OF BACILLUS-CEREUS IN MEDIA CONTAINING PLANT SEED MATERIALS AND;  INGREDIENTS USED IN CHINESE COOKERY",
+        "date": "1980-01-01T00:00:00Z",
+        "hyperdata": {
+          "abstract": "Growth and sporulation of enterotoxigenic strains of B. cereus in media containing 20 different plant seed flours and meals, with and without added infusions of beef, pork, chicken and shrimp, monosodium glutamate (MSG) and soy sauce, were studied. Suspensions (2%; pH 7.1) of seed flours and meals from diverse botanical origins were excellent sources of nutrients for growth. No correlations could be made between composition of seed materials and rate of cell division. Mean generation times of B. cereus cultured in soy, peanut and rice flour media supplemented with animal flesh infusions were significantly faster (P .ltoreq. 0.05) than those of respective controls. Monosodium glutamate (1-2%) and soy sauce (5-10%) stimulated the rate of growth of B. cereus in rice flour medium. Test flours supporting slower growth rates appeared generally to support higher rates of sporulation.",
+          "authors": "BEUCHAT, LR;  MALIN, CFA;  CARPENTER, JA",
+          "bdd": "WOS",
+          "doi": "10.1111/j.1365-2672.1980.tb01028.x",
+          "language_iso2": "EN",
+          "publication_date": "1980-01-01 00:00:00 UTC",
+          "publication_day": 1,
+          "publication_month": 1,
+          "publication_year": 1980,
+          "source": "JOURNAL OF APPLIED BACTERIOLOGY",
+          "title": "GROWTH OF BACILLUS-CEREUS IN MEDIA CONTAINING PLANT SEED MATERIALS AND;  INGREDIENTS USED IN CHINESE COOKERY"
+        }
+      },
+      "ngrams": {
+        "ngrams": [],
+        "hash": ""
+      },
+      "hash": ""
+    }
+  ],
+  "garg_version": "0.0.7.3.1"
+}

test/Test/API/Authentication.hs

@@ -20,7 +20,7 @@ import Prelude qualified
 import Servant.Auth.Client ()
 import Servant.Client
 import Test.API.Routes (auth_api)
-import Test.API.Setup (withTestDBAndPort, setupEnvironment)
+import Test.API.Setup (withTestDBAndPort, setupEnvironment, SpecContext (..))
 import Test.Database.Types
 import Test.Hspec
 import Gargantext.API.Routes.Named
@@ -32,7 +32,7 @@ cannedToken = "eyJhbGciOiJIUzUxMiJ9.eyJkYXQiOnsiaWQiOjF9fQ.t49zZSqkPAulEkYEh4pW1
 tests :: Spec
 tests = sequential $ aroundAll withTestDBAndPort $ do
   describe "Prelude" $ do
-    it "setup DB triggers" $ \((testEnv, _), _) -> setupEnvironment testEnv
+    it "setup DB triggers" $ \SpecContext{..} -> setupEnvironment _sctx_env
   describe "Authentication" $ do
     baseUrl <- runIO $ parseBaseUrl "http://localhost"
     manager <- runIO $ newManager defaultManagerSettings
@@ -41,15 +41,15 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
     -- testing scenarios start here
     describe "GET /api/v1.0/version" $ do
       let version_api = gargVersionEp . gargAPIVersion . mkBackEndAPI $ genericClient
-      it "requires no auth and returns the current version" $ \((_testEnv, port), _) -> do
-        result <- runClientM version_api (clientEnv port)
+      it "requires no auth and returns the current version" $ \SpecContext{..} -> do
+        result <- runClientM version_api (clientEnv _sctx_port)
         case result of
           Left err -> Prelude.fail (show err)
           Right r  -> r `shouldSatisfy` ((>= 1) . T.length) -- we got something back
 
     describe "POST /api/v1.0/auth" $ do
 
-      it "requires no auth and authenticates the user 'alice'" $ \((testEnv, port), _) -> do
+      it "requires no auth and authenticates the user 'alice'" $ \(SpecContext testEnv port _app _) -> do
 
         -- Let's create the Alice user.
         void $ flip runReaderT testEnv $ runTestMonad $ do
@@ -66,7 +66,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
 
         result `shouldBe` Right expected
 
-      it "denies login for user 'alice' if password is invalid" $ \((_testEnv, port), _) -> do
+      it "denies login for user 'alice' if password is invalid" $ \(SpecContext _testEnv port _app _) -> do
         let authPayload = AuthRequest "alice" (GargPassword "wrong")
         result <- runClientM (auth_api authPayload) (clientEnv port)
         putText $ "result: " <> show result

test/Test/API/Errors.hs

@@ -15,7 +15,7 @@ import Servant.Auth.Client ()
 import Servant.Client
 import Servant.Client.Generic (genericClient)
 import Test.API.Routes (mkUrl)
-import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob)
+import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob, SpecContext (..))
 import Test.Hspec
 import Test.Hspec.Wai.Internal (withApplication)
 import Test.Utils (protected, withValidLogin, protectedNewError)
@@ -26,7 +26,7 @@ tests :: Spec
 tests = sequential $ aroundAll withTestDBAndPort $ do
   describe "Errors API" $ do
     describe "Prelude" $ do
-      it "setup DB triggers and users" $ \((testEnv, port), _) -> do
+      it "setup DB triggers and users" $ \(SpecContext testEnv port _app _) -> do
         setupEnvironment testEnv
         baseUrl <- parseBaseUrl "http://localhost"
         manager <- newManager defaultManagerSettings
@@ -41,7 +41,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
 
     describe "GET /api/v1.0/node" $ do
 
-      it "returns the old error by default" $ \((_testEnv, port), app) -> do
+      it "returns the old error by default" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             res <- protected token "GET" (mkUrl port "/node/99") ""
@@ -52,7 +52,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
                     statusCode `shouldBe` 404
                     simpleBody `shouldBe` [r|{"error":"Node does not exist","node":99}|]
 
-      it "returns the new error if header X-Garg-Error-Scheme: new is passed" $ \((_testEnv, port), app) -> do
+      it "returns the new error if header X-Garg-Error-Scheme: new is passed" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             res <- protectedNewError token "GET" (mkUrl port "/node/99") ""

test/Test/API/GraphQL.hs

@@ -10,7 +10,7 @@ module Test.API.GraphQL (
 import Gargantext.Core.Types.Individu
 import Prelude
 import Servant.Auth.Client ()
-import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob)
+import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob, SpecContext (..))
 import Test.Hspec
 import Test.Hspec.Wai.Internal (withApplication)
 import Test.Hspec.Wai.JSON (json)
@@ -21,10 +21,10 @@ tests :: Spec
 tests = sequential $ aroundAll withTestDBAndPort $ do
   describe "GraphQL" $ do
     describe "Prelude" $ do
-      it "setup DB triggers" $ \((testEnv, _), _) -> setupEnvironment testEnv
+      it "setup DB triggers" $ \SpecContext{..} -> setupEnvironment _sctx_env
 
     describe "get_user_infos" $ do
-      it "allows 'alice' to see her own info" $ \((testEnv, port), app) -> do
+      it "allows 'alice' to see her own info" $ \(SpecContext testEnv port app _) -> do
         createAliceAndBob testEnv
 
         withApplication app $ do
@@ -34,7 +34,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
             protected token "POST" "/gql" query `shouldRespondWithFragment` expected
 
     describe "nodes" $ do
-      it "returns node_type" $ \((_testEnv, port), app) -> do
+      it "returns node_type" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             let query = [r| { "query": "{ nodes(node_id: 2) { node_type } }" } |]
@@ -42,21 +42,21 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
             protected token "POST" "/gql" query `shouldRespondWithFragment` expected
 
     describe "check error format" $ do
-      it "returns the new error if header X-Garg-Error-Scheme: new is passed" $ \((_testEnv, port), app) -> do
+      it "returns the new error if header X-Garg-Error-Scheme: new is passed" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             let query = [r| { "query": "{ languages(id:5) { lt_lang } }" } |]
             let expected = [json| {"errors": [{"locations":[{"column":13,"line":1}],"message":"Unknown Argument \"id\" on Field \"languages\"."}] } |]
             protectedNewError token "POST" "/gql" query `shouldRespondWithFragment` expected
 
-      it "returns the old error (though this is deprecated)" $ \((_testEnv, port), app) -> do
+      it "returns the old error (though this is deprecated)" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             let query = [r| { "query": "{ languages(id:5) { lt_lang } }" } |]
             let expected = [json| {"errors": [{"locations":[{"column":13,"line":1}],"message":"Unknown Argument \"id\" on Field \"languages\"."}] } |]
             protected token "POST" "/gql" query `shouldRespondWithFragment` expected
 
-      it "check new errors with 'type'" $ \((_testEnv, port), app) -> do
+      it "check new errors with 'type'" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "gargantua" (GargPassword "secret_key") $ \_clientEnv token -> do
             let query = [r| { "query": "mutation { delete_team_membership(shared_folder_id:1, team_node_id:1, token:\"abc\") }" } |]

test/Test/API/Private.hs

@@ -9,26 +9,25 @@ module Test.API.Private (
 
 import Gargantext.API.Routes.Named.Node
 import Gargantext.API.Routes.Named.Private
-import Gargantext.Core.Types (Node)
 import Gargantext.Core.Types.Individu
+import Gargantext.Core.Types (Node)
 import Gargantext.Database.Admin.Types.Hyperdata (HyperdataUser)
 import Gargantext.Prelude hiding (get)
 import Network.HTTP.Client hiding (Proxy)
-import Network.Wai
 import Servant.Auth.Client ()
 import Servant.Client
 import Servant.Client.Generic (genericClient)
 import Test.API.Private.Share qualified as Share
+import Test.API.Private.Table qualified as Table
 import Test.API.Routes (mkUrl)
-import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob)
-import Test.Database.Types
+import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob, SpecContext (..))
 import Test.Hspec
 import Test.Hspec.Wai hiding (pendingWith)
 import Test.Hspec.Wai.Internal (withApplication)
 import Test.Hspec.Wai.JSON (json)
 import Test.Utils (protected, shouldRespondWithFragment, withValidLogin)
 
-privateTests :: SpecWith ((TestEnv, Int), Application)
+privateTests :: SpecWith (SpecContext a)
 privateTests =
   describe "Private API" $ do
     baseUrl <- runIO $ parseBaseUrl "http://localhost"
@@ -38,7 +37,7 @@ privateTests =
     describe "GET /api/v1.0/user" $ do
 
       -- FIXME(adn): unclear if this is useful at all. Doesn't do permission checking.
-      it "doesn't allow someone with an invalid token to show the results" $ \((testEnv, port), _) -> do
+      it "doesn't allow someone with an invalid token to show the results" $ \(SpecContext testEnv port _ _) -> do
 
         createAliceAndBob testEnv
 
@@ -49,7 +48,7 @@ privateTests =
         length result `shouldBe` 0
 
       -- FIXME(adn): unclear if this is useful at all. Doesn't do permission checking.
-      it "allows 'alice' to see the results" $ \((_testEnv, port), _) -> do
+      it "allows 'alice' to see the results" $ \(SpecContext _testEnv port _app _) -> do
 
         withValidLogin port "alice" (GargPassword "alice") $ \clientEnv _token -> do
           let gargAdminClient = (genericClient :: GargAdminAPI (AsClientT ClientM))
@@ -60,33 +59,33 @@ privateTests =
 
     describe "GET /api/v1.0/node" $ do
 
-      it "unauthorised users shouldn't see anything" $ \((_testEnv, port), app) -> do
+      it "unauthorised users shouldn't see anything" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           get (mkUrl port "/node/1") `shouldRespondWith` 401
 
-      it "allows 'alice' to see her own node info" $ \((_testEnv, port), app) -> do
+      it "allows 'alice' to see her own node info" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
             protected token "GET" (mkUrl port "/node/8") ""
               `shouldRespondWithFragment` [json| {"id":8,"user_id":2,"name":"alice" } |]
 
-      it "forbids 'alice' to see others node private info" $ \((_testEnv, port), app) -> do
+      it "forbids 'alice' to see others node private info" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
             protected token "GET" (mkUrl port "/node/1") "" `shouldRespondWith` 403
 
     describe "GET /api/v1.0/tree" $ do
-      it "unauthorised users shouldn't see anything" $ \((_testEnv, port), app) -> do
+      it "unauthorised users shouldn't see anything" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           get (mkUrl port "/tree/1") `shouldRespondWith` 401
 
-      it "allows 'alice' to see her own node info" $ \((_testEnv, port), app) -> do
+      it "allows 'alice' to see her own node info" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
             protected token "GET" (mkUrl port "/tree/8") ""
               `shouldRespondWithFragment` [json| { "node": {"id":8, "name":"alice", "type": "NodeUser" } } |]
 
-      it "forbids 'alice' to see others node private info" $ \((_testEnv, port), app) -> do
+      it "forbids 'alice' to see others node private info" $ \(SpecContext _testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
             protected token "GET" (mkUrl port "/tree/1") "" `shouldRespondWith` 403
@@ -96,7 +95,9 @@ tests :: Spec
 tests = do
   sequential $ aroundAll withTestDBAndPort $ do
     describe "Prelude" $ do
-      it "setup DB triggers" $ \((testEnv, _), _) -> setupEnvironment testEnv
+      it "setup DB triggers" $ \SpecContext{..} -> setupEnvironment _sctx_env
     privateTests
   describe "Share API" $ do
     Share.tests
+  describe "Table API" $ do
+    Table.tests

test/Test/API/Private/Share.hs

@@ -43,12 +43,12 @@ shareURL token =
 tests :: Spec
 tests = sequential $ aroundAll withTestDBAndPort $ do
   describe "Prelude" $ do
-    it "setup DB triggers" $ \((testEnv, _), _) -> do
-      setupEnvironment testEnv
+    it "setup DB triggers" $ \SpecContext{..} -> do
+      setupEnvironment _sctx_env
       -- Let's create the Alice user.
-      createAliceAndBob testEnv
+      createAliceAndBob _sctx_env
 
-    it "should fail if no node type is specified" $ \((_testEnv, serverPort), app) -> do
+    it "should fail if no node type is specified" $ \(SpecContext _testEnv serverPort app _) -> do
       withApplication app $ do
         withValidLogin serverPort "alice" (GargPassword "alice") $ \clientEnv token -> do
           url <- liftIO $ runClientM (shareURL (toServantToken token) Nothing Nothing) clientEnv
@@ -57,7 +57,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
               -> liftIO $ (CL8.unpack $ responseBody res) `shouldSatisfy` (T.isInfixOf "Invalid node Type" . T.pack)
             _ -> fail "Test did not fail as expected!"
 
-    it "should fail if no node ID is specified" $ \((_testEnv, serverPort), app) -> do
+    it "should fail if no node ID is specified" $ \(SpecContext _testEnv serverPort app _) -> do
       withApplication app $ do
         withValidLogin serverPort "alice" (GargPassword "alice") $ \clientEnv token -> do
           url <- liftIO $ runClientM (shareURL (toServantToken token) (Just NodeCorpus) Nothing) clientEnv
@@ -66,7 +66,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
               -> liftIO $ (CL8.unpack $ responseBody res) `shouldSatisfy` (T.isInfixOf "Invalid node ID" . T.pack)
             _ -> fail "Test did not fail as expected!"
 
-    it "should return a valid URL" $ \((testEnv, serverPort), app) -> do
+    it "should return a valid URL" $ \(SpecContext testEnv serverPort app _) -> do
       withApplication app $ do
         withValidLogin serverPort "alice" (GargPassword "alice") $ \clientEnv token -> do
           cId <- liftIO $ newCorpusForUser testEnv "alice"
@@ -77,7 +77,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
             Right (ShareLink _)
               -> pure ()
 
-    it "should include the port if needed (like localhost)" $ \((testEnv, serverPort), app) -> do
+    it "should include the port if needed (like localhost)" $ \(SpecContext testEnv serverPort app _) -> do
       withApplication app $ do
         withValidLogin serverPort "alice" (GargPassword "alice") $ \clientEnv token -> do
           cId <- liftIO $ newCorpusForUser testEnv "alice"

test/Test/API/Private/Table.hs

+{-# LANGUAGE BangPatterns #-}
+{-# LANGUAGE TypeApplications #-}
+{-# LANGUAGE TypeFamilies #-}
+
+module Test.API.Private.Table (
+    tests
+  ) where
+
+import Gargantext.API.HashedResponse
+import Gargantext.Core.Text.Corpus.Query
+import Gargantext.Core.Types
+import Gargantext.Core.Types.Individu
+import Gargantext.Prelude
+import qualified Gargantext.API.Ngrams.Types as APINgrams
+import qualified Gargantext.Database.Query.Facet as Facet
+import Servant.Client
+import Test.API.Routes
+import Test.API.Setup
+import Test.API.UpdateList (createDocsList, checkEither)
+import Test.Hspec
+import Test.Hspec.Wai.Internal (withApplication)
+import Test.Utils
+
+
+tests :: Spec
+tests = sequential $ aroundAll withTestDBAndPort $ do
+  describe "Prelude" $ do
+    it "setup DB triggers" $ \SpecContext{..} -> do
+      setupEnvironment _sctx_env
+      -- Let's create the Alice user.
+      createAliceAndBob _sctx_env
+
+    beforeAllWith createSoySauceCorpus $ do
+      it "should return sauce in the search (#415)" $ \SpecContext{..} -> do
+        let corpusId = _sctx_data
+        withApplication _sctx_app $ do
+          withValidLogin _sctx_port "alice" (GargPassword "alice") $ \clientEnv token -> do
+            liftIO $ do
+              (HashedResponse _ tr1)
+                <- checkEither $ runClientM (get_table token
+                                                       corpusId
+                                                       (Just APINgrams.Docs)
+                                                       (Just 10)
+                                                       (Just 0)
+                                                       (Just Facet.DateDesc)
+                                                       (Just $ RawQuery "sauce")
+                                                       Nothing
+                                            ) clientEnv
+              length (tr_docs tr1) `shouldBe` 1
+
+      it "should return soy in the search (#415)" $ \SpecContext{..} -> do
+        let corpusId = _sctx_data
+        withApplication _sctx_app $ do
+          withValidLogin _sctx_port "alice" (GargPassword "alice") $ \clientEnv token -> do
+
+            liftIO $ do
+              (HashedResponse _ tr1)
+                <- checkEither $ runClientM (get_table token
+                                                       corpusId
+                                                       (Just APINgrams.Docs)
+                                                       (Just 10)
+                                                       (Just 0)
+                                                       (Just Facet.DateDesc)
+                                                       (Just $ RawQuery "soy")
+                                                       Nothing
+                                            ) clientEnv
+
+              length (tr_docs tr1) `shouldBe` 3
+
+createSoySauceCorpus :: SpecContext () -> IO (SpecContext CorpusId)
+createSoySauceCorpus ctx@SpecContext{..} = do
+  withApplication _sctx_app $ do
+    withValidLogin _sctx_port "alice" (GargPassword "alice") $ \clientEnv token -> do
+      corpusId <- createDocsList "test-data/search/GarganText_DocsList-soysauce.json" _sctx_env _sctx_port clientEnv token
+      pure $ const corpusId <$> ctx

test/Test/API/Setup.hs

 {-# LANGUAGE BangPatterns #-}
 
-module Test.API.Setup where
+module Test.API.Setup (
+    SpecContext(..)
+  , withTestDBAndPort
+  , withTestDBAndNotifications
+  , withBackendServerAndProxy
+  , setupEnvironment
+  , createAliceAndBob
+  ) where
 
 import Control.Concurrent.Async qualified as Async
 import Control.Concurrent.MVar
@@ -51,6 +58,21 @@ import Test.Database.Types
 import UnliftIO qualified
 
 
+-- | The context that each spec will be carrying along. This type is
+-- polymorphic so that each test can embellish it with test-specific data.
+-- 'SpecContext' is a functor, so you can use 'fmap' to change the 'a'.
+data SpecContext a =
+  SpecContext {
+    _sctx_env   :: !TestEnv
+  , _sctx_port  :: !Warp.Port
+  , _sctx_app   :: !Application
+  , _sctx_data  :: !a
+  }
+
+instance Functor SpecContext where
+  fmap f (SpecContext e p a d) = SpecContext e p a (f d)
+
+
 newTestEnv :: TestEnv -> Logger (GargM Env BackendInternalError) -> Warp.Port -> IO Env
 newTestEnv testEnv logger port = do
   tomlFile@(SettingsFile sf)      <- fakeTomlPath
@@ -94,7 +116,7 @@ newTestEnv testEnv logger port = do
 
 -- | Run the gargantext server on a random port, picked by Warp, which allows
 -- for concurrent tests to be executed in parallel, if we need to.
-withTestDBAndPort :: (((TestEnv, Warp.Port), Application) -> IO ()) -> IO ()
+withTestDBAndPort :: (SpecContext () -> IO ()) -> IO ()
 withTestDBAndPort action =
   withTestDB $ \testEnv -> do
     -- TODO Despite being cautious here only to start/kill dispatcher
@@ -123,7 +145,7 @@ withTestDBAndPort action =
              env <- newTestEnv testEnv ioLogger 8080
              makeApp env
     let stgs = Warp.defaultSettings { settingsOnExceptionResponse = showDebugExceptions }
-    Warp.testWithApplicationSettings stgs (pure app) $ \port -> action ((testEnv, port), app)
+    Warp.testWithApplicationSettings stgs (pure app) $ \port -> action (SpecContext testEnv port app ())
 
 withTestDBAndNotifications :: D.Dispatcher -> (((TestEnv, Warp.Port), Application) -> IO ()) -> IO ()
 withTestDBAndNotifications dispatcher action = do

test/Test/API/UpdateList.hs

@@ -7,12 +7,14 @@
 
 module Test.API.UpdateList (
     tests
-  , newCorpusForUser
+  -- * Useful helpers
   , JobPollHandle(..)
+  , newCorpusForUser
   , pollUntilFinished
-
-  -- * Useful helpers
   , updateNode
+  , createDocsList
+  , checkEither
+
   ) where
 
 import Control.Lens (mapped, over)
@@ -57,11 +59,12 @@ import Gargantext.Prelude hiding (get)
 import Network.Wai.Handler.Warp qualified as Wai
 import Paths_gargantext (getDataFileName)
 import qualified Prelude
+import System.FilePath
 import Servant
 import Servant.Client
 import Servant.Job.Async
 import Test.API.Routes (mkUrl, gqlUrl, get_table_ngrams, put_table_ngrams, toServantToken, clientRoutes, get_table, update_node)
-import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob)
+import Test.API.Setup (withTestDBAndPort, setupEnvironment, createAliceAndBob, SpecContext (..))
 import Test.Database.Types
 import Test.Hspec
 import Test.Hspec.Wai.Internal (withApplication, WaiSession)
@@ -114,13 +117,13 @@ uploadJSONList port token cId pathToNgrams = do
 tests :: Spec
 tests = sequential $ aroundAll withTestDBAndPort $ do
   describe "UpdateList API" $ do
-    it "setup DB triggers and users" $ \((testEnv, _), _) -> do
+    it "setup DB triggers and users" $ \(SpecContext testEnv _port _app _) -> do
       setupEnvironment testEnv
       createAliceAndBob testEnv
 
     describe "POST /api/v1.0/lists/:id/add/form/async (JSON)" $ do
 
-      it "allows uploading a JSON ngrams file" $ \((testEnv, port), app) -> do
+      it "allows uploading a JSON ngrams file" $ \(SpecContext testEnv port app _) -> do
         cId <- newCorpusForUser testEnv "alice"
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
@@ -142,7 +145,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
                                             ]
                                         } |]
 
-      it "does not create duplicates when uploading JSON (#313)" $ \((testEnv, port), app) -> do
+      it "does not create duplicates when uploading JSON (#313)" $ \(SpecContext testEnv port app _) -> do
         cId <- newCorpusForUser testEnv "alice"
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \clientEnv token -> do
@@ -206,7 +209,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
 
     describe "POST /api/v1.0/lists/:id/csv/add/form/async (CSV)" $ do
 
-      it "parses CSV via ngramsListFromCSVData" $ \((_testEnv, _port), _app) -> do
+      it "parses CSV via ngramsListFromCSVData" $ \(SpecContext _testEnv _port _app _) -> do
         simpleNgrams <- liftIO (TIO.readFile =<< getDataFileName "test-data/ngrams/simple.tsv")
         ngramsListFromTSVData simpleNgrams `shouldBe`
           Right (Map.fromList [ (NgramsTerms, Versioned 0 $ Map.fromList [
@@ -214,7 +217,7 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
                                 , (NgramsTerm "brazorf", NgramsRepoElement 1 StopTerm Nothing Nothing (MSet mempty))
                               ])])
 
-      it "allows uploading a CSV ngrams file" $ \((testEnv, port), app) -> do
+      it "allows uploading a CSV ngrams file" $ \(SpecContext testEnv port app _) -> do
         cId <- newCorpusForUser testEnv "alice"
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \_clientEnv token -> do
@@ -257,12 +260,12 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
 
     describe "POST /api/v1.0/corpus/:id/add/form/async (JSON)" $ do
 
-      it "allows uploading a JSON docs file" $ \((testEnv, port), app) ->
+      it "allows uploading a JSON docs file" $ \(SpecContext testEnv port app _) ->
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \clientEnv token -> do
             void $ createFortranDocsList testEnv port clientEnv token
 
-      it "doesn't use trashed documents for score calculation (#385)" $ \((testEnv, port), app) -> do
+      it "doesn't use trashed documents for score calculation (#385)" $ \(SpecContext testEnv port app _) -> do
         withApplication app $ do
           withValidLogin port "alice" (GargPassword "alice") $ \clientEnv token -> do
             corpusId <- createFortranDocsList testEnv port clientEnv token
@@ -336,21 +339,28 @@ tests = sequential $ aroundAll withTestDBAndPort $ do
                                             ) clientEnv
               length (_ne_occurrences fortran_ngram') `shouldBe` 1
 
-
-
-createFortranDocsList :: TestEnv -> Int -> ClientEnv -> Token -> WaiSession () CorpusId
-createFortranDocsList testEnv port clientEnv token = do
+createDocsList :: FilePath
+               -> TestEnv
+               -> Int
+               -> ClientEnv
+               -> Token
+               -> WaiSession () CorpusId
+createDocsList testDataPath testEnv port clientEnv token = do
   folderId <- liftIO $ newPrivateFolderForUser testEnv "alice"
   ([corpusId] :: [NodeId]) <- protectedJSON token "POST" (mkUrl port ("/node/" <> build folderId)) [aesonQQ|{"pn_typename":"NodeCorpus","pn_name":"TestCorpus"}|]
   -- Import the docsList with only two documents, both containing a \"fortran\" term.
-  simpleDocs <- liftIO (TIO.readFile =<< getDataFileName "test-data/ngrams/GarganText_DocsList-nodeId-177.json")
-  let newWithForm = mkNewWithForm simpleDocs "GarganText_DocsList-nodeId-177.json"
+  simpleDocs <- liftIO (TIO.readFile =<< getDataFileName testDataPath)
+  let newWithForm = mkNewWithForm simpleDocs (T.pack $ takeBaseName testDataPath)
   (j :: JobPollHandle) <- checkEither $ fmap toJobPollHandle <$> liftIO (runClientM (add_file_async token corpusId newWithForm) clientEnv)
   let mkPollUrl jh = "/corpus/" <> fromString (show $ _NodeId corpusId) <> "/add/form/async/" +|_jph_id jh|+ "/poll?limit=1"
   j' <- pollUntilFinished token port mkPollUrl j
   liftIO (_jph_status j' `shouldBe` "IsFinished")
   pure corpusId
 
+createFortranDocsList :: TestEnv -> Int -> ClientEnv -> Token -> WaiSession () CorpusId
+createFortranDocsList testEnv port =
+  createDocsList "test-data/ngrams/GarganText_DocsList-nodeId-177.json" testEnv port
+
 updateNode :: Int -> ClientEnv -> Token -> NodeId -> WaiSession () ()
 updateNode port clientEnv token nodeId = do
   let params = UpdateNodeParamsTexts Both

test/Test/Database/Operations/DocumentSearch.hs

@@ -135,6 +135,8 @@ stemmingTest :: TestEnv -> Assertion
 stemmingTest _env = do
   stem EN GargPorterAlgorithm "Ajeje"        `shouldBe` "Ajeje"
   stem EN GargPorterAlgorithm "PyPlasm:"     `shouldBe` "PyPlasm:"
+  stem EN GargPorterAlgorithm "soy"          `shouldBe` "soy"
+  stem EN GargPorterAlgorithm "cry"          `shouldBe` "cri"
   -- This test outlines the main differences between Porter and Lancaster.
   stem EN GargPorterAlgorithm "dancer"       `shouldBe` "dancer"
   stem EN LancasterAlgorithm  "dancer"       `shouldBe` "dant"