Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 159
    • Issues 159
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 7
    • Merge Requests 7
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Merge Requests
  • !408

Merged
Opened May 05, 2025 by Alfredo Di Napoli@AlfredoDiNapoli
  • Report abuse
Report abuse

Port (almost all) DB operations in GGTX to use the transaction API

Fixes #466.

This refactoring is gargantuan (no pun intended), unfortunately.

This gigantic commit ports the existing DB operations in GGTX to use the transactional API, meaning that we can now compose DB operations and they will all run in the same Postgres transaction using the same connection, which will eliminate those class of bugs where concurrent DB access might result in an inconsistent state.

On top of that, we simplify some parts of the API, for which a summary is given below:

  1. The NodeStoryEnv management has been greatly simplified; in the new API we don't need an external connection pool to be passed and we don't have to pass IO actions, we can just pass DB operations, therefore we can greatly simplify the API to just pass mostly pure values;

  2. Due to the fact that our DBTx monad can't do arbitrary IO (which is a good thing) we cannot fire Central Exchange notifications immediately. Rather that happens now is that we collect the CEMessage to be sent and we fire them in the relevant concrete monad after we finished with the DB transaction. This means that in principle there would be a small delay between the DB operation taking place and the notification firing but in practice the latency should be negligible and bear in mind this is typically what we want: if we have a long DB Tx that triggers an error in the middle we don't want to be sending out CE messages prematurely if the overall operation didn't succeed!

  3. There are still a few places in the codebase where we couldn't make things fully compositional with regards to the DBTx API, because we had Servant handlers which had DB operations mixed with other IO effectful computations (or other things like the notification from the MonadJobStatus). For now we are splitting these functions by manually running the partial DB operations, and while this is not ideal it can be fixed in subsequent merge requests.

  4. The WorkerEnv doesn't use IOException as its MonadError anymore, as for consistency we can just use BackendInternalError by adding a InternalWorkerError data constructor accepting the IOException triggered by the Worker monad.

More testing is needed, with particular attention to performance (regression) but this should hopefully offer a decent baseline.

Edited May 12, 2025 by Alfredo Di Napoli

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b adinapoli/issue-466 origin/adinapoli/issue-466

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/dev
git merge --no-ff adinapoli/issue-466

Step 4. Push the result of the merge to GitLab

git push origin dev

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

  • Discussion 2
  • Commits 6
  • Pipelines 5
  • Changes 119
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
1
Labels
Ready to merge
Assign labels
  • View project labels
Reference: gargantext/haskell-gargantext!408

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.