Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 175
    • Issues 175
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 9
    • Merge Requests 9
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Issues
  • #166

Closed
Open
Opened Jan 10, 2023 by Przemyslaw Kaminski@cgenie
  • Report abuse
  • New issue
Report abuse New issue

Fix flow so that corenlp is called together with API fetch

Currently our code looks like this:

https://gitlab.iscpif.fr/gargantext/haskell-gargantext/blob/79490deb7d2e2459d79620488e06b61d5c2bd43a/src/Gargantext/Database/Action/Flow.hs#L279

We call flowCorpusUser which makes the CoreNLP query after all docs are imported. However, the external APIs are fragile and with longer queries they fail often. It is bad that ngrams are empty after such import. We could rewrite this so that CoreNLP is called alongside of API fetch, in conduit. Later we could parallelize that even further so that the API fetch and CoreNLP are called in parallel.

Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
0
Labels
None
Assign labels
  • View project labels
Reference: gargantext/haskell-gargantext#166