Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 159
    • Issues 159
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 7
    • Merge Requests 7
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Merge Requests
  • !382

Merged
Opened Jan 20, 2025 by Alfredo Di Napoli@AlfredoDiNapoli
  • Report abuse
Report abuse

Replace performance-critical parts of the algorithm with `massiv` (and ditch `accelerate-llvm`)

This is an initial work in progress for #291 (closed).

I have started adding the necessary testing scaffolding to eventually refactor our linear algebra code and, most importantly, drop the accelerate-llvm dependency.

I have done an audit of the codebase and it's quite a lot of work to get rid of accelerate altogether as it's used in a lot of places, however, there is only a single module that uses the llvm backend, which is the Gargantext.Core.Methods.Similarities.Accelerate.Distributional module, which contains the critical function distributional, which is used in the Graph computation algorithm.

The plan of attack is that I have now sufficient infrastructure to start benchmarking distributional. Unfortunately it seems there are no viable backend in accelerate we could use apart from the accellerate-llvm one, so what I will explore is using a different library altogether to implement the distributional function.

A good candidate to explore is massiv, which seems to be a new library which supports multicore and parallelism and it's full Haskell, afaik.

@anoe While I was working on the tests, I have found a bug in the implementation of distributional, which could lead to matrixes populated by NaN. See commit 7a24253fc9eb5668d5752b562e384805ea3d8a5d for more details on the bug.

Edited Feb 27, 2025 by Alfredo Di Napoli

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b adinapoli/issue-291 origin/adinapoli/issue-291

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/dev
git merge --no-ff adinapoli/issue-291

Step 4. Push the result of the merge to GitLab

git push origin dev

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

  • Discussion 27
  • Commits 23
  • Pipelines 17
  • Changes 17
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
1
Labels
Ready to merge
Assign labels
  • View project labels
Reference: gargantext/haskell-gargantext!382

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.