Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
153
Issues
153
List
Board
Labels
Milestones
Merge Requests
9
Merge Requests
9
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
5e8bc9f2
Commit
5e8bc9f2
authored
Oct 12, 2021
by
Przemyslaw Kaminski
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[upload zip] this works now
parent
4b09273b
Pipeline
#1964
failed with stage
in 9 minutes and 57 seconds
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
4 additions
and
10 deletions
+4
-10
New.hs
src/Gargantext/API/Node/Corpus/New.hs
+2
-7
Parsers.hs
src/Gargantext/Core/Text/Corpus/Parsers.hs
+2
-3
No files found.
src/Gargantext/API/Node/Corpus/New.hs
View file @
5e8bc9f2
...
...
@@ -28,8 +28,6 @@ import Data.Swagger
import
Data.Text
(
Text
)
import
qualified
Data.Text
as
T
import
GHC.Generics
(
Generic
)
import
qualified
Prelude
as
Prelude
import
Protolude
(
readFile
)
import
Servant
import
Servant.Job.Utils
(
jsonOptions
)
-- import Servant.Multipart
...
...
@@ -270,6 +268,8 @@ addToCorpusWithForm user cid (NewWithForm ft d l _n) logStatus jobLog = do
eDocs
<-
liftBase
$
parse
data
'
case
eDocs
of
Right
docs'
->
do
-- TODO Add progress (jobStatus) update for docs - this is a
-- long action
let
docs
=
splitEvery
500
$
take
1000000
docs'
printDebug
"Parsing corpus finished : "
cid
...
...
@@ -303,11 +303,6 @@ addToCorpusWithForm user cid (NewWithForm ft d l _n) logStatus jobLog = do
jobLog3
=
jobLogSuccess
jobLog2
jobLogE
=
jobLogFailTotal
jobLog
parseCsvGargV3Path
::
[
Char
]
->
IO
(
Either
Prelude
.
String
[
HyperdataDocument
])
parseCsvGargV3Path
fp
=
do
contents
<-
readFile
fp
Parser
.
parseFormat
Parser
.
CsvGargV3
$
cs
contents
{-
addToCorpusWithFile :: FlowCmdM env err m
=> CorpusId
...
...
src/Gargantext/Core/Text/Corpus/Parsers.hs
View file @
5e8bc9f2
...
...
@@ -100,12 +100,11 @@ parseFormat WOS bs = do
parseFormat
ZIP
bs
=
do
path
<-
emptySystemTempFile
"parsed.zip"
DB
.
writeFile
path
bs
parsedZip
<-
withArchive
path
$
do
withArchive
path
$
do
files
<-
DM
.
keys
<$>
getEntries
filesContents
<-
mapM
getEntry
files
ddocs
<-
liftIO
$
mapM
(
parseFormat
CsvGargV3
)
filesContents
pure
$
sequence
ddocs
pure
$
Left
$
"Not implemented for ZIP, parsedZip"
<>
show
parsedZip
pure
$
concat
<$>
sequence
ddocs
parseFormat
_
_
=
undefined
-- | Parse file into documents
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment