Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
195
Issues
195
List
Board
Labels
Milestones
Merge Requests
12
Merge Requests
12
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
d35f64fd
Commit
d35f64fd
authored
May 26, 2025
by
Alfredo Di Napoli
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Generalise index in DocumentIdWithNgrams
parent
f280b7e1
Pipeline
#7619
passed with stages
in 53 minutes and 17 seconds
Changes
2
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
11 additions
and
11 deletions
+11
-11
Types.hs
src/Gargantext/Database/Action/Flow/Types.hs
+2
-2
Utils.hs
src/Gargantext/Database/Action/Flow/Utils.hs
+9
-9
No files found.
src/Gargantext/Database/Action/Flow/Types.hs
View file @
d35f64fd
...
@@ -66,9 +66,9 @@ type FlowInsertDB a = ( AddUniqId a
...
@@ -66,9 +66,9 @@ type FlowInsertDB a = ( AddUniqId a
data
DocumentIdWithNgrams
a
b
=
data
DocumentIdWithNgrams
ix
a
b
=
DocumentIdWithNgrams
DocumentIdWithNgrams
{
documentWithId
::
Indexed
NodeId
a
{
documentWithId
::
Indexed
ix
a
,
documentNgrams
::
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
)
,
documentNgrams
::
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
)
}
deriving
(
Show
)
}
deriving
(
Show
)
...
...
src/Gargantext/Database/Action/Flow/Utils.hs
View file @
d35f64fd
...
@@ -87,8 +87,8 @@ docNgrams lang ts doc =
...
@@ -87,8 +87,8 @@ docNgrams lang ts doc =
documentIdWithNgrams
::
Monad
m
documentIdWithNgrams
::
Monad
m
=>
(
a
->
m
(
HashMap
.
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
))
)
=>
(
a
->
m
(
HashMap
.
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
))
)
->
Indexed
NodeId
a
->
Indexed
ix
a
->
m
(
DocumentIdWithNgrams
a
b
)
->
m
(
DocumentIdWithNgrams
ix
a
b
)
documentIdWithNgrams
f
=
toDocumentIdWithNgrams
documentIdWithNgrams
f
=
toDocumentIdWithNgrams
where
where
toDocumentIdWithNgrams
d
=
do
toDocumentIdWithNgrams
d
=
do
...
@@ -97,18 +97,18 @@ documentIdWithNgrams f = toDocumentIdWithNgrams
...
@@ -97,18 +97,18 @@ documentIdWithNgrams f = toDocumentIdWithNgrams
mapDocumentIdWithNgrams
::
Monad
m
mapDocumentIdWithNgrams
::
Monad
m
=>
(
a
->
m
(
HashMap
.
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
))
)
=>
(
a
->
m
(
HashMap
.
HashMap
b
(
Map
NgramsType
TermsWeight
,
TermsCount
))
)
->
[
Indexed
NodeId
a
]
->
[
Indexed
ix
a
]
->
m
[
DocumentIdWithNgrams
a
b
]
->
m
[
DocumentIdWithNgrams
ix
a
b
]
mapDocumentIdWithNgrams
f
=
mapM
(
documentIdWithNgrams
f
)
mapDocumentIdWithNgrams
f
=
mapM
(
documentIdWithNgrams
f
)
-- | Creates a NodeIdNgrams map out of the input 'DocumentIdWithNgrams' list.
-- | Creates a NodeIdNgrams map out of the input 'DocumentIdWithNgrams' list.
-- TODO check optimization
-- TODO check optimization
mkNodeIdNgramsMap
::
(
Ord
b
,
Hashable
b
)
mkNodeIdNgramsMap
::
forall
ix
a
b
.
(
Ord
b
,
Hashable
b
,
Ord
ix
)
=>
[
DocumentIdWithNgrams
a
b
]
=>
[
DocumentIdWithNgrams
ix
a
b
]
->
HashMap
.
HashMap
b
->
HashMap
.
HashMap
b
(
Map
NgramsType
(
Map
NgramsType
(
Map
NodeId
(
TermsWeight
,
TermsCount
))
(
Map
ix
(
TermsWeight
,
TermsCount
))
)
)
mkNodeIdNgramsMap
=
HashMap
.
unionsWith
(
DM
.
unionWith
(
DM
.
unionWith
addTuples
))
.
fmap
f
mkNodeIdNgramsMap
=
HashMap
.
unionsWith
(
DM
.
unionWith
(
DM
.
unionWith
addTuples
))
.
fmap
f
where
where
...
@@ -116,8 +116,8 @@ mkNodeIdNgramsMap = HashMap.unionsWith (DM.unionWith (DM.unionWith addTuples)) .
...
@@ -116,8 +116,8 @@ mkNodeIdNgramsMap = HashMap.unionsWith (DM.unionWith (DM.unionWith addTuples)) .
-- same ngrams term has different ngrams types, the 'TermsCount'
-- same ngrams term has different ngrams types, the 'TermsCount'
-- for it (which is the number of times the terms appears in a
-- for it (which is the number of times the terms appears in a
-- document) is copied over to all its types.
-- document) is copied over to all its types.
f
::
DocumentIdWithNgrams
a
b
f
::
DocumentIdWithNgrams
ix
a
b
->
HashMap
.
HashMap
b
(
Map
NgramsType
(
Map
NodeId
(
TermsWeight
,
TermsCount
)))
->
HashMap
.
HashMap
b
(
Map
NgramsType
(
Map
ix
(
TermsWeight
,
TermsCount
)))
f
d
=
fmap
(
\
(
ngramsTypeMap
,
cnt
)
->
fmap
(
\
w
->
DM
.
singleton
nId
(
w
,
cnt
))
ngramsTypeMap
)
$
documentNgrams
d
f
d
=
fmap
(
\
(
ngramsTypeMap
,
cnt
)
->
fmap
(
\
w
->
DM
.
singleton
nId
(
w
,
cnt
))
ngramsTypeMap
)
$
documentNgrams
d
where
where
nId
=
_index
$
documentWithId
d
nId
=
_index
$
documentWithId
d
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment