Skip to content

haskell-gargantext
haskell-gargantext
Closed
Opened by Przemyslaw Kaminski@cgenie
Report abuse

Resolve "Uniform ngrams creation"

Related to #224

I am trying to understand better how the ngrams algorithms work.

  1. In analyzing G.D.A.Flow -> reIndex, I simplified things:
    1. We don't need to select whole contexts with hyperdata, it's enough just to select context id with hyperdata
    2. ngramsByDoc was a list, generated from a list of docs. I split that into a pure function being mapped over docs.
    3. I added unit tests in T.Ngrams.Count
    4. I added TermsWeight, this newtype is clearer than Int that we had before
  2. I refactored WithQuery to have only datafield. This is described in #441
Edited by Przemyslaw Kaminski