• Alfredo Di Napoli's avatar
    Port DB operations to transactional API · 53512f89
    Alfredo Di Napoli authored
    This gigantic commit ports the existing DB operations in GGTX to use the
    transactional API, meaning that we can now compose DB operations and
    they will all run in the same Postgres transaction using the same
    connection, which will eliminate those class of bugs where concurrent DB
    access might result in an inconsistent state.
    
    On top of that, we simplify some parts of the API, for which a summary
    is given below:
    
    1. The `NodeStoryEnv` management has been greatly simplified; in the new
       API we don't need an external connection pool to be passed and we
       don't have to pass IO actions, we can just pass DB operations,
       therefore we can greatly simplify the API to just pass mostly pure
       values;
    
    2. Due to the fact that our `DBTx` monad can't do arbitrary IO (which is
       a good thing) we cannot fire Central Exchange notifications
       immediately. Rather that happens now is that we collect the
       `CEMessage` to be sent and we fire them in the relevant concrete
       monad after we finished with the DB transaction. This means that in
       principle there would be a small delay between the DB operation
       taking place and the notification firing but in practice the latency
       should be negligible and bear in mind this is typically what we want:
       if we have a long DB Tx that triggers an error in the middle we don't
       want to be sending out CE messages prematurely if the overall
       operation didn't succeed!
    
    3. There are still a few places in the codebase where we couldn't make
       things fully compositional with regards to the DBTx API, because we
       had Servant handlers which had DB operations mixed with other IO
       effectful computations (or other things like the notification from
       the `MonadJobStatus`). For now we are splitting these functions by
       manually running the partial DB operations, and while this is not
       ideal it can be fixed in subsequent merge requests.
    
    4. The `WorkerEnv` doesn't use `IOException` as its `MonadError`
       anymore, as for consistency we can just use `BackendInternalError` by
       adding a `InternalWorkerError` data constructor accepting the
       `IOException` triggered by the Worker monad.
    
    More testing is needed, with particular attention to performance
    (regression) but this should hopefully offer a decent baseline.
    53512f89
Name
Last commit
Last update
.gitlab/issue_templates Loading commit data...
bench Loading commit data...
bench-data/phylo Loading commit data...
bin Loading commit data...
devops Loading commit data...
docs Loading commit data...
ekg-assets Loading commit data...
nix Loading commit data...
src Loading commit data...
src-doctest Loading commit data...
test Loading commit data...
test-data Loading commit data...
.clippy.dhall Loading commit data...
.envrc Loading commit data...
.ghci Loading commit data...
.gitignore Loading commit data...
.gitlab-ci.yml Loading commit data...
.mailmap Loading commit data...
CHANGELOG.md Loading commit data...
CODE_OF_CONDUCT.md Loading commit data...
DEVELOPER_GUIDELINES.md Loading commit data...
LICENSE Loading commit data...
README.md Loading commit data...
cabal.project Loading commit data...
cabal.project.freeze Loading commit data...
cabal.project.local_toCopy Loading commit data...
gargantext-settings.toml_toModify Loading commit data...
gargantext.cabal Loading commit data...
gargantext.ini_toModify Loading commit data...
hie.yaml Loading commit data...
run Loading commit data...
shell.nix Loading commit data...
stack.yaml Loading commit data...
start Loading commit data...
weeder.toml Loading commit data...