Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 160
    • Issues 160
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 8
    • Merge Requests 8
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Merge Requests
  • !382

Merged
Opened 4 months ago by Alfredo Di Napoli@AlfredoDiNapoli
  • Report abuse
Report abuse

Replace performance-critical parts of the algorithm with `massiv` (and ditch `accelerate-llvm`)

This is an initial work in progress for #291 (closed).

I have started adding the necessary testing scaffolding to eventually refactor our linear algebra code and, most importantly, drop the accelerate-llvm dependency.

I have done an audit of the codebase and it's quite a lot of work to get rid of accelerate altogether as it's used in a lot of places, however, there is only a single module that uses the llvm backend, which is the Gargantext.Core.Methods.Similarities.Accelerate.Distributional module, which contains the critical function distributional, which is used in the Graph computation algorithm.

The plan of attack is that I have now sufficient infrastructure to start benchmarking distributional. Unfortunately it seems there are no viable backend in accelerate we could use apart from the accellerate-llvm one, so what I will explore is using a different library altogether to implement the distributional function.

A good candidate to explore is massiv, which seems to be a new library which supports multicore and parallelism and it's full Haskell, afaik.

@anoe While I was working on the tests, I have found a bug in the implementation of distributional, which could lead to matrixes populated by NaN. See commit 7a24253fc9eb5668d5752b562e384805ea3d8a5d for more details on the bug.

Edited 3 months ago by Alfredo Di Napoli

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b adinapoli/issue-291 origin/adinapoli/issue-291

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/dev
git merge --no-ff adinapoli/issue-291

Step 4. Push the result of the merge to GitLab

git push origin dev

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

Request to merge adinapoli/issue-291 into dev
  • Email patches
  • Plain diff
Pipeline #7376 passed for 2c7541cd on adinapoli/issue-291

          Merged by Alfredo Di Napoli 3 months ago

          The changes were merged into dev with 3192b0f5

          Mentions #291 (closed)

          Pipeline #7378 passed for 3192b0f5 on dev
                  • Discussion 27
                  • Commits 23
                  • Pipelines 17
                  • Changes 17
                  • Loading...
                  • You're only seeing other activity in the feed. To add a comment, switch to one of the following options.
                  Please register or sign in to reply
                  Assignee
                  Assign to
                  None
                  Milestone
                  None
                  Assign milestone
                  None
                  Time tracking
                  No estimate or time spent
                  1
                  Labels
                  Ready to merge
                  Assign labels
                  • View project labels
                  Lock merge request
                  Unlocked
                  participants
                  Reference: gargantext/haskell-gargantext!382

                  Revert this merge request

                  This will create a new commit in order to revert the existing changes.

                  Switch branch
                  Cancel
                  A new branch will be created in your fork and a new merge request will be started.

                  Cherry-pick this merge request

                  Switch branch
                  Cancel
                  A new branch will be created in your fork and a new merge request will be started.