Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
145
Issues
145
List
Board
Labels
Milestones
Merge Requests
6
Merge Requests
6
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
2e194ab3
Commit
2e194ab3
authored
May 25, 2021
by
Alexandre Delanoë
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[FIX] quick fix to enable WOS parser
parent
dc10d9c3
Pipeline
#1466
failed with stage
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
6 additions
and
6 deletions
+6
-6
Parsers.hs
src/Gargantext/Core/Text/Corpus/Parsers.hs
+5
-5
Learn.hs
src/Gargantext/Core/Text/Learn.hs
+1
-1
No files found.
src/Gargantext/Core/Text/Corpus/Parsers.hs
View file @
2e194ab3
...
...
@@ -41,7 +41,7 @@ import Gargantext.Database.Admin.Types.Hyperdata (HyperdataDocument(..))
import
Gargantext.Prelude
import
Gargantext.Core.Text.Corpus.Parsers.CSV
(
parseHal
,
parseHal'
,
parseCsv
,
parseCsv'
)
import
Gargantext.Core.Text.Corpus.Parsers.RIS.Presse
(
presseEnrich
)
import
Gargantext.Core.Text.Learn
(
detectLangDefault
)
--
import Gargantext.Core.Text.Learn (detectLangDefault)
import
System.FilePath
(
FilePath
(),
takeExtension
)
import
qualified
Data.ByteString
as
DB
import
qualified
Data.ByteString.Char8
as
DBC
...
...
@@ -103,11 +103,11 @@ parseFile ff p = join $ mapM (toDoc ff) <$> snd <$> enrichWith ff
toDoc
::
FileFormat
->
[(
Text
,
Text
)]
->
IO
HyperdataDocument
-- TODO use language for RIS
toDoc
ff
d
=
do
let
abstract
=
lookup
"abstract"
d
let
lang
=
maybe
EN
identity
(
join
$
detectLangDefault
<$>
(
fmap
(
DT
.
take
50
)
abstract
))
--
let abstract = lookup "abstract" d
let
lang
=
EN
--
maybe EN identity (join $ detectLangDefault <$> (fmap (DT.take 50) abstract))
let
dateToParse
=
DT
.
replace
"-"
" "
<$>
lookup
"PY"
d
<>
Just
" "
<>
lookup
"publication_date"
d
(
utcTime
,
(
pub_year
,
pub_month
,
pub_day
))
<-
Date
.
dateSplit
lang
dateToParse
pure
$
HyperdataDocument
(
Just
$
DT
.
pack
$
show
ff
)
...
...
src/Gargantext/Core/Text/Learn.hs
View file @
2e194ab3
...
...
@@ -112,7 +112,7 @@ detectLangDefault = detectCat 99 eventLang
textSample
::
Lang
->
String
textSample
EN
=
EN
.
textSample
textSample
FR
=
FR
.
textSample
textSample
_
=
panic
"
textSample:
not impl yet"
textSample
_
=
panic
"
[G.C.T.L:detectLangDefault]
not impl yet"
--textSample DE = DE.textSample
--textSample SP = SP.textSample
--textSample CH = CH.textSample
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment