Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
153
Issues
153
List
Board
Labels
Milestones
Merge Requests
9
Merge Requests
9
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
d97e2510
Commit
d97e2510
authored
Apr 29, 2021
by
Alexandre Delanoë
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[FIX] testing (WIP)
parent
9f5df649
Pipeline
#1455
failed with stage
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
14 additions
and
6 deletions
+14
-6
List.hs
src/Gargantext/API/Ngrams/List.hs
+14
-6
No files found.
src/Gargantext/API/Ngrams/List.hs
View file @
d97e2510
...
...
@@ -39,7 +39,7 @@ import Gargantext.Database.Action.Metrics.NgramsByNode (getOccByNgramsOnlyFast')
import
Gargantext.Database.Admin.Types.Hyperdata.Document
import
Gargantext.Database.Admin.Types.Node
import
Gargantext.Database.Query.Table.Ngrams
(
insertNgrams
)
import
Gargantext.Database.Query.Table.Node
(
getDocumentsWithParentId
)
import
Gargantext.Database.Query.Table.Node
Node
(
selectDocNodes
)
import
Gargantext.Database.Schema.Ngrams
import
Gargantext.Database.Schema.Node
import
Gargantext.Database.Types
(
Indexed
(
..
))
...
...
@@ -114,12 +114,16 @@ reIndexWith cId lId nt lts = do
<$>
HashMap
.
toList
<$>
getTermsWith
identity
[
lId
]
nt
lts
--
printDebug "ts" ts
--
printDebug "ts" ts
-- Taking the ngrams with 0 occurrences only (orphans)
orphans
<-
HashMap
.
keys
<$>
HashMap
.
filter
(
==
0
)
<$>
getOccByNgramsOnlyFast'
cId
lId
nt
ts
occs
<-
getOccByNgramsOnlyFast'
cId
lId
nt
ts
let
orphans
=
List
.
concat
$
map
(
\
t
->
case
HashMap
.
lookup
t
occs
of
Nothing
->
[
t
]
Just
n
->
if
n
==
1
then
[
t
]
else
[ ]
)
ts
-- Getting the Id of orphan ngrams
mapTextNgramsId
<-
insertNgrams
(
map
(
text2ngrams
.
unNgramsTerm
)
orphans
)
...
...
@@ -127,7 +131,9 @@ reIndexWith cId lId nt lts = do
printDebug
"orphans"
orphans
-- Get all documents of the corpus
docs
<-
getDocumentsWithParentId
cId
docs
<-
selectDocNodes
cId
printDebug
"docs length"
(
List
.
length
docs
)
-- Checking Text documents where orphans match
-- TODO Tests here
...
...
@@ -143,6 +149,8 @@ reIndexWith cId lId nt lts = do
(
List
.
cycle
[
Map
.
fromList
$
[(
nt
,
Map
.
singleton
(
doc
^.
node_id
)
1
)]])
)
docs
printDebug
"ngramsByDoc"
ngramsByDoc
-- Saving the indexation in database
_
<-
insertDocNgrams
lId
(
HashMap
.
fromList
$
catMaybes
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment