This gigantic commit ports the existing DB operations in GGTX to use the
transactional API, meaning that we can now compose DB operations and
they will all run in the same Postgres transaction using the same
connection, which will eliminate those class of bugs where concurrent DB
access might result in an inconsistent state.

On top of that, we simplify some parts of the API, for which a summary
is given below:

1. The `NodeStoryEnv` management has been greatly simplified; in the new
   API we don't need an external connection pool to be passed and we
   don't have to pass IO actions, we can just pass DB operations,
   therefore we can greatly simplify the API to just pass mostly pure
   values;

2. Due to the fact that our `DBTx` monad can't do arbitrary IO (which is
   a good thing) we cannot fire Central Exchange notifications
   immediately. Rather that happens now is that we collect the
   `CEMessage` to be sent and we fire them in the relevant concrete
   monad after we finished with the DB transaction. This means that in
   principle there would be a small delay between the DB operation
   taking place and the notification firing but in practice the latency
   should be negligible and bear in mind this is typically what we want:
   if we have a long DB Tx that triggers an error in the middle we don't
   want to be sending out CE messages prematurely if the overall
   operation didn't succeed!

3. There are still a few places in the codebase where we couldn't make
   things fully compositional with regards to the DBTx API, because we
   had Servant handlers which had DB operations mixed with other IO
   effectful computations (or other things like the notification from
   the `MonadJobStatus`). For now we are splitting these functions by
   manually running the partial DB operations, and while this is not
   ideal it can be fixed in subsequent merge requests.

4. The `WorkerEnv` doesn't use `IOException` as its `MonadError`
   anymore, as for consistency we can just use `BackendInternalError` by
   adding a `InternalWorkerError` data constructor accepting the
   `IOException` triggered by the Worker monad.

More testing is needed, with particular attention to performance
(regression) but this should hopefully offer a decent baseline.

All calls were basically:

nlpServer <- view (nlpServerGet lang)
addDocumentsToHyperCorpus nlpServer ...

commit be879b1e
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Thu Jan 30 18:22:44 2025 +0100

    [ngrams] code fixes according to review

    Related MR:
    !378

commit bf89561b
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Jan 22 21:11:47 2025 +0100

    [test] notification on node move

    Also, some small refactorings.

commit 3d5d74ab
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Jan 22 20:13:44 2025 +0100

    [tests] add notifications func comment, fix core/notifications indent

commit b8ea3af2
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Jan 22 19:13:35 2025 +0100

    [update-project-dependencies]

commit 1217baf4
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Wed Jan 22 19:09:17 2025 +0100

    [tests] notifications: test async notifications for update tree

    Related to
    #418

commit 874785e9
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 21 06:06:27 2025 +0100

    [refactor] unify Database & ExternalIDs

    These types are the same, except for Database.Empty

    I managed to have backwards compatibility with the frontend format,
    hence the frontend doesn't need any changes.

commit e7b16520
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 21 06:05:57 2025 +0100

    [cabal] upgrade haskell-bee to fix TSRetry and ESRepeat issues

commit ad045ae0
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Jan 20 06:32:49 2025 +0100

    [cabal] upgrade haskell-bee tag

commit b3910bb4
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 14 10:56:12 2025 +0100

    [test] move some Arbitrary instances to Test/Instances.hs

commit bb282d02
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 14 09:17:23 2025 +0100

    [test] WithQuery offline test (with EPO constructor)

commit c0fe2e51
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 14 06:59:45 2025 +0100

    [query] move EPO user/token into the datafield

    This simplifies the WithQuery structure even more

commit 93586adc
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Jan 13 17:45:42 2025 +0100

    [tests] fix WithQuery frontend serialization test

    Also, add WithQuery pubmed test (with api_key)

commit bc29319c
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Jan 13 10:13:15 2025 +0100

    [ngrams] simplify WithQuery json structure

    There is now only a 'datafield' field, no need for duplicated
    'database'.

    Related to #441

commit e6fdbee4
Merge: 95dc32b3 13457ca8
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Fri Jan 10 12:03:59 2025 +0100

    Merge branch 'dev' into 224-dev-understanding-ngrams

commit 95dc32b3
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 7 20:01:11 2025 +0100

    [ngrams] refactor PubMed DB type (to include Maybe APIKey)

commit baa2491f
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 7 18:09:04 2025 +0100

    [refactor] searx search refactoring

commit fcf83bf7
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Tue Jan 7 11:14:03 2025 +0100

    [ngrams] more types annotations

commit 0d8a77c4
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 16:15:07 2024 +0100

    [ngrams, test] refactor: Count -> Terms

commit 85f1dffe
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 14:35:05 2024 +0100

    [ngrams] refactor opaque Int into TermsWeight newtype

commit a81ea049
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 14:34:39 2024 +0100

    [CLI] fix limit removal

    It wasn't used anyways.

commit d1dfbf79
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 11:35:41 2024 +0100

    [ngrams] one more simplification in ngramsByDoc'

commit fcb48b8f
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 11:33:33 2024 +0100

    [ngrams] some more simplification of ngramsByDoc'

commit ab7c1766
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 30 11:00:19 2024 +0100

    [ngrams, tests] understanding ngramsByDoc better

commit 35c2d0b0
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 23 21:20:29 2024 +0100

    [ngrams] small simplification to docNgrams function

commit 161ac077
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Mon Dec 23 18:35:59 2024 +0100

    [ngrams] annotate types of ngrams algorithms

commit 08c7c91c
Author: Przemysław Kaminski <pk@intrepidus.pl>
Date:   Sat Dec 21 09:45:00 2024 +0100

    [ngrams] improve function documentation, add types, add unit tests

    I want to understand ngrams algorithms better.

Also, some import refactoring

Now everything is in Core/Config

I moved HasConfig to Core/Config instead of it being in Database.Prelude

This commit moves `GargConfig` and the other config-related
data structures back into gargantext, so that they can be edited
and expanded without needing to worry about the prelude project.

This was (User, Either CorpusName, [CorpusId]) before, but the case of
UserMaster doesn't make sense with these parameters, so I rewrote the
function to accept only correct datatypes as inputs.

This reverts commit f4284b4e.

Removing the `All` data constructor from  the `Lang` datatype has
several benifits, the main one being that we now have total functions
for things like `toISO693`.

This will make possible to write an uniform interface for stemming
`stem :: Lang -> StemmingAlgorithm -> T.Text -> T.Text`, as now
`Lang` will always map to a valid (ISO693) language.

This reverts commit efbe327f, reversing
changes made to 6d9bcd07.

We have contexts.hash_id which is enough (computed by postgres as a
trigger).

Basically, Gargantext.Prelude exports all of Protolude now.

This commit refactors the codebase to add a new function called
`addDocumentsToHyperCorpus` which generalises the common pattern of
calling `insertMasterDocs` and later adding the documents via the DB
`Doc` functions.

It also "lowers" some functions to run into a plain `DBCmd`.

It's not fully wired in, `convertQuery` for Arxiv is implemented as
`undefined`.

Also, added a basic GraphQL endpoint.

This big commit improves the jobs API in a way that now we can
completely abstract away over a JobLog.

Doesn't compile yet.

This is somewhere in Flow.hs in types for ConduitT () a m ()
where m monad is IO.

One issue is that it creates new list/docs/board/graph nodes.

Database.PostgreSQL.Simple.Time.Printer.year:  years BCE not supportedCallStack (from HasCallStack):  error, called at src/Database/PostgreSQL/Simple/Time/Internal/Printer.hs:83:23 in postgresql-simple-0.6.4-JK0smXwdTUMDl7m3Ne57PV:Database.PostgreSQL.Simple.Time.Internal.Printer