WIP: Replace performance-critical parts of the algorithm with `massiv` (and ditch `accelerate-llvm`)
This is an initial work in progress for #291.
I have started adding the necessary testing scaffolding to eventually refactor our linear algebra code and, most importantly, drop the accelerate-llvm
dependency.
I have done an audit of the codebase and it's quite a lot of work to get rid of accelerate
altogether as it's used in a lot of places, however, there is only a single module that uses the llvm
backend, which is the Gargantext.Core.Methods.Similarities.Accelerate.Distributional
module, which contains the critical function distributional
, which is used in the Graph computation algorithm.
The plan of attack is that I have now sufficient infrastructure to start benchmarking distributional
. Unfortunately it seems there are no viable backend in accelerate
we could use apart from the accellerate-llvm
one, so what I will explore is using a different library altogether to implement the distributional
function.
A good candidate to explore is massiv, which seems to be a new library which supports multicore and parallelism and it's full Haskell, afaik.
@anoe While I was working on the tests, I have found a bug in the implementation of distributional
, which could lead to matrixes populated by NaN. See commit 7a24253fc9eb5668d5752b562e384805ea3d8a5d
for more details on the bug.