Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Grégoire Locqueville
haskell-gargantext
Commits
e26da1cb
Commit
e26da1cb
authored
Jul 24, 2024
by
Grégoire Locqueville
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Now automatically builds term list!
parent
4efcb92f
Pipeline
#6463
failed with stages
Changes
1
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
23 additions
and
32 deletions
+23
-32
REPL.hs
src/REPL.hs
+23
-32
No files found.
src/REPL.hs
View file @
e26da1cb
...
...
@@ -19,10 +19,13 @@ import Gargantext.Core.Text.List (buildNgramsLists)
import
Gargantext.Core.NLP
(
HasNLPServer
)
import
Gargantext.Database.Query.Tree.Error
(
HasTreeError
)
import
Gargantext.Core.Text.List.Group.WithStem
(
GroupParams
(
..
))
import
Gargantext.Database.Action.Flow
(
getOrMkRootWithCorpus
,
saveDocNgramsWith
)
import
Gargantext.Database.Action.Flow
(
getOrMkRootWithCorpus
)
import
Gargantext.Database.Query.Tree.Root
(
MkCorpusUser
(
..
))
import
Gargantext.API.Admin.Types
(
HasSettings
)
import
Gargantext.Database.Admin.Types.Hyperdata.Corpus
(
HyperdataCorpus
)
import
Gargantext.Database.Action.Flow.List
(
flowList_DbRepo
)
import
Gargantext.Core.Types
(
HasValidationError
)
import
Gargantext.Core.Text.List.Social
(
FlowSocialListWith
(
..
),
FlowSocialListPriority
(
..
))
execText
::
IO
(
Maybe
CorpusId
)
...
...
@@ -56,6 +59,7 @@ testSubcorpusFunction username parentId queryText reuseParentList =
-- either copied from the parent corpus or recomputed based on the subcorpus docs.
makeSubcorpusFromQuery
::
forall
env
err
m
.
(
DbCmd'
env
err
m
,
HasValidationError
err
,
HasNodeError
err
,
HasNodeStory
env
err
m
,
HasNLPServer
env
...
...
@@ -65,8 +69,7 @@ makeSubcorpusFromQuery ::
=>
User
-- ^ The corpus owner
->
CorpusId
-- ^ ID of the parent corpus
->
Q
.
Query
-- ^ The query to determine the subset of documents that will appear in the subcorpus
->
Bool
-- ^ Whether to reuse parent term list (True) or compute a new one
-- based only on the documents in the subcorpus (False)
->
Bool
-- ^ Whether to reuse parent term list (True) or compute a new one based only on the documents in the subcorpus (False)
->
m
CorpusId
-- ^ The child corpus ID
makeSubcorpusFromQuery
user
parentId
query
reuseParentList
=
do
userId
<-
getUserId
user
...
...
@@ -74,41 +77,29 @@ makeSubcorpusFromQuery user parentId query reuseParentList = do
(
_
,
_
,
masterCorpusId
)
<-
getOrMkRootWithCorpus
MkCorpusUserMaster
(
Nothing
::
Maybe
HyperdataCorpus
)
-- Get ahold of all documents that match the query
facetDocs
<-
searchInCorpus
parentId
False
query
Nothing
Nothing
Nothing
-- Create subcorpus node with all the documents
-- Create
a
subcorpus node with all the documents
_
<-
Document
.
add
subcorpusId
$
nodeId2ContextId
.
facetDoc_id
<$>
facetDocs
-- Create nodes for docs and terms as children of the subcorpus
void
$
insertDefaultNode
NodeTexts
subcorpusId
userId
void
$
insertDefaultNode
NodeList
subcorpusId
userId
listId
<-
insertDefaultNode
NodeList
subcorpusId
userId
-- Either simply copy parent terms... (TODO)
if
reuseParentList
then
return
()
-- ... or rebuild a term list from scratch
else
do
void
$
buildNgramsLists
user
subcorpusId
masterCorpusId
Nothing
GroupIdentity
-- Map NgramsType [NgramsElement]
-- saveDocNgramsWith :: (DbCmd' env err m)
-- => ListId
-- -> HashMap.HashMap ExtractedNgrams (Map NgramsType (Map NodeId (Int, TermsCount)))
-- -> m ()
saveDocNgramsWith
lId
mapNgramsDocs'
ngrams
<-
buildNgramsLists
user
subcorpusId
masterCorpusId
(
Just
(
FlowSocialListWithPriority
MySelfFirst
)
::
Maybe
FlowSocialListWith
)
GroupIdentity
-- Save computed list
_
<-
flowList_DbRepo
listId
ngrams
return
()
-- comment Save ??
return
subcorpusId
-- TODO
-- Permettre de (mettre les 2 en option) :
-- [
] 1a. relancer le buildNgramsLists (cf. module Flow)
-- [
X
] 1a. relancer le buildNgramsLists (cf. module Flow)
-- [ ] 1b. ou copier la liste
-- [ ] 2. générer un graphe
-- Problèmes :
-- * Permission error quand j'essaie d'avoir l'ID du master corpus
-- * Comment save une ngramslist ?
-- * Je ne sais pas ce que sont les arguments de type
-- Maybe FlowSocialListWith
-- GroupParams
-- de buildNgramsList
-- * Les types de :
-- - le résultat de buildNgramsLists
-- - les arguments de saveDocNgramsWith
-- n'ont rien à voir l'un avec l'autre
-- * Pour 1b. ... par où commencer, où prendre modèle ?
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment