Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 159
    • Issues 159
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 6
    • Merge Requests 6
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Merge Requests
  • !109

Merged
Opened Oct 18, 2022 by Alp Mestanogullari@amestanogullari
  • Report abuse
Report abuse

introduce and use a flexible job queue system

This MR introduces a new job queue system, implemented by the Gargantext.Utils.Jobs.* modules.

  • Queue implements the actual queue(s). We have one queue per "job type" and will draw the next job to run from the highest priority job type's queue.
  • Map implements the job ID => job data mapping.
  • State gathers the two in a single data type
  • Monad offers some interface and operations that can work in any monad that has a job environment of sorts
  • API offers an HTTP interface to all of this that is API-compatible with servant-job's async stuffs.

This patch changes all uses of servant-job's serveJobsAPI to the new queue system's serveJobsAPI, which is almost identical: it just takes as additional argument the job type.

This system also comes with a modest but useful-starting-point test suite that looks at basic properties.

PS: I'm also removing gargantext-client, it's just too painfully slow to even load in ghci, let alone compile. I'll resurrect it if we need it some day, but let's not weight ourselves down until then.

@anoe Could you give a shot at using this branch to see how it goes? Feel free to go crazy and spawn a bunch of stuffs.

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b flexible-job-queue origin/flexible-job-queue

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/dev
git merge --no-ff flexible-job-queue

Step 4. Push the result of the merge to GitLab

git push origin dev

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

  • Discussion 0
  • Commits 1
  • Pipelines 1
  • Changes 38
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
0
Labels
None
Assign labels
  • View project labels
Reference: gargantext/haskell-gargantext!109

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.