Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
153
Issues
153
List
Board
Labels
Milestones
Merge Requests
11
Merge Requests
11
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
91b97fbd
Commit
91b97fbd
authored
2 years ago
by
Alexandre Delanoë
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[CLEAN] readFile homonyms
parent
96f02a40
Pipeline
#3127
passed with stage
in 92 minutes and 4 seconds
Changes
6
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
11 additions
and
11 deletions
+11
-11
CleanCsvCorpus.hs
bin/gargantext-cli/CleanCsvCorpus.hs
+1
-1
Main.hs
bin/gargantext-cli/Main.hs
+2
-2
Main.hs
bin/gargantext-phylo/Main.hs
+1
-1
File.hs
src/Gargantext/API/Node/File.hs
+1
-1
CSV.hs
src/Gargantext/Core/Text/Corpus/Parsers/CSV.hs
+4
-4
GargDB.hs
src/Gargantext/Database/GargDB.hs
+2
-2
No files found.
bin/gargantext-cli/CleanCsvCorpus.hs
View file @
91b97fbd
...
@@ -40,7 +40,7 @@ main = do
...
@@ -40,7 +40,7 @@ main = do
--let q = ["water", "scarcity", "morocco", "shortage","flood"]
--let q = ["water", "scarcity", "morocco", "shortage","flood"]
let
q
=
[
"gratuit"
,
"gratuité"
,
"culture"
,
"culturel"
]
let
q
=
[
"gratuit"
,
"gratuité"
,
"culture"
,
"culturel"
]
eDocs
<-
CSV
.
readFile
rPath
eDocs
<-
CSV
.
read
CSV
File
rPath
case
eDocs
of
case
eDocs
of
Right
(
h
,
csvDocs
)
->
do
Right
(
h
,
csvDocs
)
->
do
putStrLn
$
"Number of documents before:"
<>
show
(
V
.
length
csvDocs
)
putStrLn
$
"Number of documents before:"
<>
show
(
V
.
length
csvDocs
)
...
...
This diff is collapsed.
Click to expand it.
bin/gargantext-cli/Main.hs
View file @
91b97fbd
...
@@ -42,7 +42,7 @@ import Gargantext.Core.Types
...
@@ -42,7 +42,7 @@ import Gargantext.Core.Types
import
Gargantext.Core.Text.Terms
import
Gargantext.Core.Text.Terms
import
Gargantext.Core.Text.Context
import
Gargantext.Core.Text.Context
import
Gargantext.Core.Text.Terms.WithList
import
Gargantext.Core.Text.Terms.WithList
import
Gargantext.Core.Text.Corpus.Parsers.CSV
(
readFile
,
csv_title
,
csv_abstract
,
csv_publication_year
,
unIntOrDec
,
fromMIntOrDec
,
defaultYear
)
import
Gargantext.Core.Text.Corpus.Parsers.CSV
(
read
CSV
File
,
csv_title
,
csv_abstract
,
csv_publication_year
,
unIntOrDec
,
fromMIntOrDec
,
defaultYear
)
import
Gargantext.Core.Text.List.Formats.CSV
(
csvMapTermList
)
import
Gargantext.Core.Text.List.Formats.CSV
(
csvMapTermList
)
import
Gargantext.Core.Text.Terms
(
terms
)
import
Gargantext.Core.Text.Terms
(
terms
)
import
Gargantext.Core.Text.Metrics.Count
(
coocOnContexts
,
Coocs
)
import
Gargantext.Core.Text.Metrics.Count
(
coocOnContexts
,
Coocs
)
...
@@ -86,7 +86,7 @@ main = do
...
@@ -86,7 +86,7 @@ main = do
[
corpusFile
,
termListFile
,
outputFile
]
<-
getArgs
[
corpusFile
,
termListFile
,
outputFile
]
<-
getArgs
--corpus :: IO (DM.IntMap [[Text]])
--corpus :: IO (DM.IntMap [[Text]])
eCorpusFile
<-
readFile
corpusFile
eCorpusFile
<-
read
CSV
File
corpusFile
case
eCorpusFile
of
case
eCorpusFile
of
Right
cf
->
do
Right
cf
->
do
let
corpus
=
DM
.
fromListWith
(
<>
)
let
corpus
=
DM
.
fromListWith
(
<>
)
...
...
This diff is collapsed.
Click to expand it.
bin/gargantext-phylo/Main.hs
View file @
91b97fbd
...
@@ -109,7 +109,7 @@ csvToDocs parser patterns time path =
...
@@ -109,7 +109,7 @@ csvToDocs parser patterns time path =
(
termsInText
patterns
$
(
csv_title
row
)
<>
" "
<>
(
csv_abstract
row
))
(
termsInText
patterns
$
(
csv_title
row
)
<>
" "
<>
(
csv_abstract
row
))
Nothing
Nothing
[]
[]
)
<$>
snd
<$>
either
(
\
err
->
panic
$
cs
$
"CSV error"
<>
(
show
err
))
identity
<$>
Csv
.
readFile
path
)
<$>
snd
<$>
either
(
\
err
->
panic
$
cs
$
"CSV error"
<>
(
show
err
))
identity
<$>
Csv
.
read
CSV
File
path
Csv'
limit
->
Vector
.
toList
Csv'
limit
->
Vector
.
toList
<$>
Vector
.
take
limit
<$>
Vector
.
take
limit
<$>
Vector
.
map
(
\
row
->
Document
(
toPhyloDate
(
csv'_publication_year
row
)
(
csv'_publication_month
row
)
(
csv'_publication_day
row
)
time
)
<$>
Vector
.
map
(
\
row
->
Document
(
toPhyloDate
(
csv'_publication_year
row
)
(
csv'_publication_month
row
)
(
csv'_publication_day
row
)
time
)
...
...
This diff is collapsed.
Click to expand it.
src/Gargantext/API/Node/File.hs
View file @
91b97fbd
...
@@ -76,7 +76,7 @@ fileDownload uId nId = do
...
@@ -76,7 +76,7 @@ fileDownload uId nId = do
let
(
HyperdataFile
{
_hff_name
=
name'
let
(
HyperdataFile
{
_hff_name
=
name'
,
_hff_path
=
path
})
=
node
^.
node_hyperdata
,
_hff_path
=
path
})
=
node
^.
node_hyperdata
Contents
c
<-
GargDB
.
readFile
$
unpack
path
Contents
c
<-
GargDB
.
read
Garg
File
$
unpack
path
let
(
mMime
,
_
)
=
DMT
.
guessType
DMT
.
defaultmtd
False
$
unpack
name'
let
(
mMime
,
_
)
=
DMT
.
guessType
DMT
.
defaultmtd
False
$
unpack
name'
mime
=
case
mMime
of
mime
=
case
mMime
of
...
...
This diff is collapsed.
Click to expand it.
src/Gargantext/Core/Text/Corpus/Parsers/CSV.hs
View file @
91b97fbd
...
@@ -234,7 +234,7 @@ delimiter Comma = fromIntegral $ ord ','
...
@@ -234,7 +234,7 @@ delimiter Comma = fromIntegral $ ord ','
------------------------------------------------------------------------
------------------------------------------------------------------------
readCsvOn'
::
[
CsvDoc
->
Text
]
->
FilePath
->
IO
(
Either
Prelude
.
String
[
Text
])
readCsvOn'
::
[
CsvDoc
->
Text
]
->
FilePath
->
IO
(
Either
Prelude
.
String
[
Text
])
readCsvOn'
fields
fp
=
do
readCsvOn'
fields
fp
=
do
r
<-
readFile
fp
r
<-
read
CSV
File
fp
pure
$
(
V
.
toList
pure
$
(
V
.
toList
.
V
.
map
(
\
l
->
intercalate
(
pack
" "
)
$
map
(
\
field
->
field
l
)
fields
)
.
V
.
map
(
\
l
->
intercalate
(
pack
" "
)
$
map
(
\
field
->
field
l
)
fields
)
.
snd
)
<$>
r
.
snd
)
<$>
r
...
@@ -267,8 +267,8 @@ readByteStringStrict d ff = (readByteStringLazy d ff) . BL.fromStrict
...
@@ -267,8 +267,8 @@ readByteStringStrict d ff = (readByteStringLazy d ff) . BL.fromStrict
------------------------------------------------------------------------
------------------------------------------------------------------------
-- | TODO use readFileLazy
-- | TODO use readFileLazy
readFile
::
FilePath
->
IO
(
Either
Prelude
.
String
(
Header
,
Vector
CsvDoc
))
read
CSV
File
::
FilePath
->
IO
(
Either
Prelude
.
String
(
Header
,
Vector
CsvDoc
))
readFile
fp
=
do
read
CSV
File
fp
=
do
result
<-
fmap
(
readCsvLazyBS
Comma
)
$
BL
.
readFile
fp
result
<-
fmap
(
readCsvLazyBS
Comma
)
$
BL
.
readFile
fp
case
result
of
case
result
of
Left
_err
->
fmap
(
readCsvLazyBS
Tab
)
$
BL
.
readFile
fp
Left
_err
->
fmap
(
readCsvLazyBS
Tab
)
$
BL
.
readFile
fp
...
@@ -448,7 +448,7 @@ parseHal' bs = (V.toList . V.map csvHal2doc . snd) <$> readCsvHalLazyBS bs
...
@@ -448,7 +448,7 @@ parseHal' bs = (V.toList . V.map csvHal2doc . snd) <$> readCsvHalLazyBS bs
------------------------------------------------------------------------
------------------------------------------------------------------------
parseCsv
::
FilePath
->
IO
(
Either
Prelude
.
String
[
HyperdataDocument
])
parseCsv
::
FilePath
->
IO
(
Either
Prelude
.
String
[
HyperdataDocument
])
parseCsv
fp
=
fmap
(
V
.
toList
.
V
.
map
csv2doc
.
snd
)
<$>
readFile
fp
parseCsv
fp
=
fmap
(
V
.
toList
.
V
.
map
csv2doc
.
snd
)
<$>
read
CSV
File
fp
{-
{-
parseCsv' :: BL.ByteString -> Either Prelude.String [HyperdataDocument]
parseCsv' :: BL.ByteString -> Either Prelude.String [HyperdataDocument]
...
...
This diff is collapsed.
Click to expand it.
src/Gargantext/Database/GargDB.hs
View file @
91b97fbd
...
@@ -140,13 +140,13 @@ writeFile a = do
...
@@ -140,13 +140,13 @@ writeFile a = do
---
---
-- | Example to read a file with Type
-- | Example to read a file with Type
readFile
::
(
MonadReader
env
m
read
Garg
File
::
(
MonadReader
env
m
,
HasConfig
env
,
HasConfig
env
,
MonadBase
IO
m
,
MonadBase
IO
m
,
ReadFile
a
,
ReadFile
a
)
)
=>
FilePath
->
m
a
=>
FilePath
->
m
a
readFile
fp
=
do
read
Garg
File
fp
=
do
dataPath
<-
view
$
hasConfig
.
gc_datafilepath
dataPath
<-
view
$
hasConfig
.
gc_datafilepath
liftBase
$
readFile'
$
toFilePath
dataPath
fp
liftBase
$
readFile'
$
toFilePath
dataPath
fp
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment