Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
  • Sign in
haskell-gargantext
haskell-gargantext
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 154
    • Issues 154
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 7
    • Merge Requests 7
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • gargantext
  • haskell-gargantexthaskell-gargantext
  • Merge Requests
  • !226

Merged
Opened Nov 21, 2023 by Alfredo Di Napoli@AlfredoDiNapoli
  • Report abuse
Report abuse

Unify CSV & JSON uploads for Ngrams

Hopefully fixes #287 (closed).

This MR modifies the implementation of the /api/v1.0/lists/:id/csv/add/form/async endpoint to internally parse the input CSV into an NgramList type so that we can call the postAsyncJSON function, effectively reusing code and implementing form uploads with a single function rather than two distinct ones.

@anoe This should eventually pass CI and the tests. There are important aspects to consider, none of which is related to this MR, but this is the way the CSV endpoint worked even before:

  1. There is a loss of information between a CSV import/export and a JSON import/exports. In a CSV (or, I should say, TSV) file, all we can represent are terms, which means we cannot export or import things like authors, docs etc. like we can do with JSON;

  2. In a standard JSON import/export, we can store the size into an NgramsRepoElement. However, when we import/export using TSVs, we lose this information, which means that all the imported elements will have a size of 1.

At this point let me know if you hit the original bug where the number of imported entries from your TSV file didn't match the total rows of the file and we will have a proper look.

Edited Nov 27, 2023 by Alfredo Di Napoli

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b adinapoli/issue-287 origin/adinapoli/issue-287

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/dev
git merge --no-ff adinapoli/issue-287

Step 4. Push the result of the merge to GitLab

git push origin dev

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

  • Discussion 0
  • Commits 6
  • Pipelines 5
  • Changes 13
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
1
Labels
Ready to merge
Assign labels
  • View project labels
Reference: gargantext/haskell-gargantext!226

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.