Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
150
Issues
150
List
Board
Labels
Milestones
Merge Requests
5
Merge Requests
5
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
461124e8
Commit
461124e8
authored
Dec 20, 2022
by
Przemyslaw Kaminski
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[flow] fix hardcoded list size in build terms
parent
e71c52ec
Pipeline
#3488
passed with stage
in 92 minutes and 24 seconds
Changes
5
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
8 additions
and
11 deletions
+8
-11
Flow.hs
src/Gargantext/API/Flow.hs
+0
-1
Hal.hs
src/Gargantext/Core/Text/Corpus/API/Hal.hs
+0
-1
List.hs
src/Gargantext/Core/Text/List.hs
+6
-6
Flow.hs
src/Gargantext/Database/Action/Flow.hs
+1
-1
Add.hs
src/Gargantext/Database/Query/Table/Node/Document/Add.hs
+1
-2
No files found.
src/Gargantext/API/Flow.hs
View file @
461124e8
...
@@ -29,4 +29,3 @@ data OutputFlow
...
@@ -29,4 +29,3 @@ data OutputFlow
flow
::
Flow
->
OutputFlow
flow
::
Flow
->
OutputFlow
flow
=
undefined
flow
=
undefined
src/Gargantext/Core/Text/Corpus/API/Hal.hs
View file @
461124e8
...
@@ -63,4 +63,3 @@ toDoc' la (HAL.Corpus i t ab d s aus affs struct_id) = do
...
@@ -63,4 +63,3 @@ toDoc' la (HAL.Corpus i t ab d s aus affs struct_id) = do
,
_hd_publication_minute
=
Nothing
,
_hd_publication_minute
=
Nothing
,
_hd_publication_second
=
Nothing
,
_hd_publication_second
=
Nothing
,
_hd_language_iso2
=
Just
$
(
pack
.
show
)
la
}
,
_hd_language_iso2
=
Just
$
(
pack
.
show
)
la
}
src/Gargantext/Core/Text/List.hs
View file @
461124e8
...
@@ -159,7 +159,7 @@ buildNgramsTermsList :: ( HasNodeError err
...
@@ -159,7 +159,7 @@ buildNgramsTermsList :: ( HasNodeError err
->
GroupParams
->
GroupParams
->
(
NgramsType
,
MapListSize
)
->
(
NgramsType
,
MapListSize
)
->
m
(
Map
NgramsType
[
NgramsElement
])
->
m
(
Map
NgramsType
[
NgramsElement
])
buildNgramsTermsList
user
uCid
mCid
mfslw
groupParams
(
nt
,
_
mapListSize
)
=
do
buildNgramsTermsList
user
uCid
mCid
mfslw
groupParams
(
nt
,
MapListSize
mapListSize
)
=
do
-- Filter 0 With Double
-- Filter 0 With Double
-- Computing global speGen score
-- Computing global speGen score
...
@@ -179,13 +179,13 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
...
@@ -179,13 +179,13 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
)
)
printDebug
"[buildNgramsTermsList: Flow Social List / end]"
nt
printDebug
"[buildNgramsTermsList: Flow Social List / end]"
nt
let
!
ngramsKeys
=
HashSet
.
fromList
$
List
.
take
1000
$
HashSet
.
toList
$
HashMap
.
keysSet
allTerms
let
!
ngramsKeys
=
HashSet
.
fromList
$
List
.
take
mapListSize
$
HashSet
.
toList
$
HashMap
.
keysSet
allTerms
printDebug
"[buildNgramsTermsList: ngramsKeys]"
(
HashSet
.
size
ngramsKeys
)
printDebug
"[buildNgramsTermsList: ngramsKeys]"
(
HashSet
.
size
ngramsKeys
)
!
groupParams'
<-
getGroupParams
groupParams
(
HashSet
.
map
(
text2ngrams
.
unNgramsTerm
)
ngramsKeys
)
!
groupParams'
<-
getGroupParams
groupParams
(
HashSet
.
map
(
text2ngrams
.
unNgramsTerm
)
ngramsKeys
)
printDebug
"[buildNgramsTermsList: groupParams']"
(
""
::
Text
)
printDebug
"[buildNgramsTermsList: groupParams']"
(
""
::
Text
)
let
let
!
socialLists_Stemmed
=
addScoreStem
groupParams'
ngramsKeys
socialLists
!
socialLists_Stemmed
=
addScoreStem
groupParams'
ngramsKeys
socialLists
...
@@ -217,7 +217,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
...
@@ -217,7 +217,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
-------------------------
-------------------------
-- Filter 1 With Set NodeId and SpeGen
-- Filter 1 With Set NodeId and SpeGen
!
selectedTerms
=
Set
.
toList
$
hasTerms
(
groupedMonoHead
<>
groupedMultHead
)
!
selectedTerms
=
Set
.
toList
$
hasTerms
(
groupedMonoHead
<>
groupedMultHead
)
printDebug
"[buildNgramsTermsList: selectedTerms]"
selectedTerms
printDebug
"[buildNgramsTermsList: selectedTerms]"
selectedTerms
-- TODO remove (and remove HasNodeError instance)
-- TODO remove (and remove HasNodeError instance)
...
@@ -283,7 +283,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
...
@@ -283,7 +283,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
-- use % of list if to big, or Int if to small
-- use % of list if to big, or Int if to small
!
mapSize
=
1000
::
Double
!
mapSize
=
1000
::
Double
!
canSize
=
mapSize
*
2
::
Double
!
canSize
=
mapSize
*
2
::
Double
!
inclSize
=
0.4
::
Double
!
inclSize
=
0.4
::
Double
!
exclSize
=
1
-
inclSize
!
exclSize
=
1
-
inclSize
...
@@ -318,7 +318,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
...
@@ -318,7 +318,7 @@ buildNgramsTermsList user uCid mCid mfslw groupParams (nt, _mapListSize)= do
<>
mapMultScoredExclHead
<>
mapMultScoredExclHead
-- An original way to filter to start with
-- An original way to filter to start with
!
cands
=
setListType
(
Just
CandidateTerm
)
!
cands
=
setListType
(
Just
CandidateTerm
)
$
canMonoScoredIncHead
$
canMonoScoredIncHead
<>
canMonoScoredExclHead
<>
canMonoScoredExclHead
<>
canMulScoredInclHead
<>
canMulScoredInclHead
...
...
src/Gargantext/Database/Action/Flow.hs
View file @
461124e8
...
@@ -325,7 +325,7 @@ flowCorpusUser l user corpusName ctype ids mfslw = do
...
@@ -325,7 +325,7 @@ flowCorpusUser l user corpusName ctype ids mfslw = do
--let gp = (GroupParams l 2 3 (StopSize 3))
--let gp = (GroupParams l 2 3 (StopSize 3))
-- Here the PosTagAlgo should be chosen according to the Lang
-- Here the PosTagAlgo should be chosen according to the Lang
_
<-
case
mfslw
of
_
<-
case
mfslw
of
(
Just
(
NoList
_
))
->
do
(
Just
(
NoList
_
))
->
do
printDebug
"Do not build list"
mfslw
printDebug
"Do not build list"
mfslw
pure
()
pure
()
...
...
src/Gargantext/Database/Query/Table/Node/Document/Add.hs
View file @
461124e8
...
@@ -8,7 +8,7 @@ Stability : experimental
...
@@ -8,7 +8,7 @@ Stability : experimental
Portability : POSIX
Portability : POSIX
Add Documents/Contact to a Corpus/Annuaire.
Add Documents/Contact to a Corpus/Annuaire.
-}
-}
------------------------------------------------------------------------
------------------------------------------------------------------------
{-# LANGUAGE DeriveDataTypeable #-}
{-# LANGUAGE DeriveDataTypeable #-}
...
@@ -78,4 +78,3 @@ instance ToRow InputData where
...
@@ -78,4 +78,3 @@ instance ToRow InputData where
,
toField
(
0
::
Int
)
,
toField
(
0
::
Int
)
,
toField
(
1
::
Int
)
,
toField
(
1
::
Int
)
]
]
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment