Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
148
Issues
148
List
Board
Labels
Milestones
Merge Requests
7
Merge Requests
7
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
c3464a38
Commit
c3464a38
authored
Nov 29, 2023
by
Alexandre Delanoë
Browse files
Options
Browse Files
Download
Plain Diff
Merge remote-tracking branch 'origin/dev-epo-integration' into dev
parents
113fffba
3bbb49a9
Changes
13
Hide whitespace changes
Inline
Side-by-side
Showing
13 changed files
with
239 additions
and
20 deletions
+239
-20
cabal.project
cabal.project
+12
-2
gargantext.cabal
gargantext.cabal
+6
-3
Types.hs
src/Gargantext/API/Admin/Orchestrator/Types.hs
+1
-0
GraphQL.hs
src/Gargantext/API/GraphQL.hs
+4
-0
User.hs
src/Gargantext/API/GraphQL/User.hs
+26
-0
New.hs
src/Gargantext/API/Node/Corpus/New.hs
+11
-2
Types.hs
src/Gargantext/API/Node/Corpus/Types.hs
+2
-0
API.hs
src/Gargantext/Core/Text/Corpus/API.hs
+8
-2
EPO.hs
src/Gargantext/Core/Text/Corpus/API/EPO.hs
+128
-0
OpenAlex.hs
src/Gargantext/Core/Text/Corpus/API/OpenAlex.hs
+3
-3
Flow.hs
src/Gargantext/Database/Action/Flow.hs
+9
-5
User.hs
src/Gargantext/Database/Admin/Types/Hyperdata/User.hs
+11
-2
User.hs
src/Gargantext/Database/Query/Table/User.hs
+18
-1
No files found.
cabal.project
View file @
c3464a38
...
@@ -81,6 +81,11 @@ source-repository-package
...
@@ -81,6 +81,11 @@ source-repository-package
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
arxiv
-
api
.
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
arxiv
-
api
.
git
tag
:
eb130c71fa17adaceed6ff66beefbccb13df51ba
tag
:
eb130c71fa17adaceed6ff66beefbccb13df51ba
source
-
repository
-
package
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
epo
-
proxy
-
api
.
git
tag
:
9225
d046083853200b9045c8d71161e6a234fc5c
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
hal
.
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
hal
.
git
...
@@ -91,6 +96,11 @@ source-repository-package
...
@@ -91,6 +96,11 @@ source-repository-package
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
isidore
.
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
isidore
.
git
tag
:
3
db385e767d2100d8abe900833c6e7de3ac55e1b
tag
:
3
db385e767d2100d8abe900833c6e7de3ac55e1b
source
-
repository
-
package
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
iso639
.
git
tag
:
1
a9e23e210a02da3b846d7cdc666541e2148334d
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
istex
.
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
istex
.
git
...
@@ -99,7 +109,7 @@ source-repository-package
...
@@ -99,7 +109,7 @@ source-repository-package
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
openalex
.
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
crawlers
/
openalex
.
git
tag
:
1
cf872fb3bd0e3e44af31247833c4b6bb7d0dca5
tag
:
a46abe16700449a1f863deb89380b05d89ad9723
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
...
@@ -114,7 +124,7 @@ source-repository-package
...
@@ -114,7 +124,7 @@ source-repository-package
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
haskell
-
gargantext
-
prelude
location
:
https
://
gitlab
.
iscpif
.
fr
/
gargantext
/
haskell
-
gargantext
-
prelude
tag
:
40135
d166830690b1101180b79e9fd3663284b2b
tag
:
9
b33568da92f92bfa89c8bd8f05b07b70fd021f9
source
-
repository
-
package
source
-
repository
-
package
type
:
git
type
:
git
...
...
gargantext.cabal
View file @
c3464a38
...
@@ -79,6 +79,7 @@ library
...
@@ -79,6 +79,7 @@ library
Gargantext.Core.Text.Context
Gargantext.Core.Text.Context
Gargantext.Core.Text.Corpus.API
Gargantext.Core.Text.Corpus.API
Gargantext.Core.Text.Corpus.API.Arxiv
Gargantext.Core.Text.Corpus.API.Arxiv
Gargantext.Core.Text.Corpus.API.EPO
Gargantext.Core.Text.Corpus.API.Pubmed
Gargantext.Core.Text.Corpus.API.Pubmed
Gargantext.Core.Text.Corpus.API.OpenAlex
Gargantext.Core.Text.Corpus.API.OpenAlex
Gargantext.Core.Text.Corpus.Query
Gargantext.Core.Text.Corpus.Query
...
@@ -436,6 +437,7 @@ library
...
@@ -436,6 +437,7 @@ library
, duckling ^>= 0.2.0.0
, duckling ^>= 0.2.0.0
, ekg-core ^>= 0.1.1.7
, ekg-core ^>= 0.1.1.7
, ekg-json ^>= 0.1.0.7
, ekg-json ^>= 0.1.0.7
, epo-api-client
, exceptions ^>= 0.10.4
, exceptions ^>= 0.10.4
, extra ^>= 1.7.9
, extra ^>= 1.7.9
, fast-logger ^>= 3.0.5
, fast-logger ^>= 3.0.5
...
@@ -466,7 +468,7 @@ library
...
@@ -466,7 +468,7 @@ library
, ihaskell ^>= 0.10.2.2
, ihaskell ^>= 0.10.2.2
, ini ^>= 0.4.1
, ini ^>= 0.4.1
, insert-ordered-containers ^>= 0.2.5.1
, insert-ordered-containers ^>= 0.2.5.1
, iso639
^>= 0.1.0.3
, iso639
, jose ^>= 0.8.4
, jose ^>= 0.8.4
, json-stream ^>= 0.4.2.4
, json-stream ^>= 0.4.2.4
, lens ^>= 4.19.2
, lens ^>= 4.19.2
...
@@ -499,7 +501,7 @@ library
...
@@ -499,7 +501,7 @@ library
, process ^>= 1.6.13.2
, process ^>= 1.6.13.2
, product-profunctors ^>= 0.11.0.3
, product-profunctors ^>= 0.11.0.3
, profunctors ^>= 5.6.2
, profunctors ^>= 5.6.2
, protolude ^>= 0.3.
0
, protolude ^>= 0.3.
3
, pureMD5 ^>= 2.1.4
, pureMD5 ^>= 2.1.4
, quickcheck-instances ^>= 0.3.25.2
, quickcheck-instances ^>= 0.3.25.2
, rake ^>= 0.0.1
, rake ^>= 0.0.1
...
@@ -522,6 +524,7 @@ library
...
@@ -522,6 +524,7 @@ library
, servant-blaze ^>= 0.9.1
, servant-blaze ^>= 0.9.1
, servant-cassava ^>= 0.10.1
, servant-cassava ^>= 0.10.1
, servant-client ^>= 0.18.3
, servant-client ^>= 0.18.3
, servant-client-core ^>= 0.18.3
, servant-ekg ^>= 0.3.1
, servant-ekg ^>= 0.3.1
, servant-flatten ^>= 0.2
, servant-flatten ^>= 0.2
, servant-job >= 0.2.0.0
, servant-job >= 0.2.0.0
...
@@ -672,7 +675,7 @@ executable gargantext-cli
...
@@ -672,7 +675,7 @@ executable gargantext-cli
, gargantext-prelude
, gargantext-prelude
, ini ^>= 0.4.1
, ini ^>= 0.4.1
, optparse-generic ^>= 1.4.7
, optparse-generic ^>= 1.4.7
, protolude ^>= 0.3.
0
, protolude ^>= 0.3.
3
, split ^>= 0.2.3.4
, split ^>= 0.2.3.4
, text ^>= 1.2.4.1
, text ^>= 1.2.4.1
, unordered-containers ^>= 0.2.16.0
, unordered-containers ^>= 0.2.16.0
...
...
src/Gargantext/API/Admin/Orchestrator/Types.hs
View file @
c3464a38
...
@@ -36,6 +36,7 @@ data ExternalAPIs = OpenAlex
...
@@ -36,6 +36,7 @@ data ExternalAPIs = OpenAlex
|
HAL
|
HAL
|
IsTex
|
IsTex
|
Isidore
|
Isidore
|
EPO
deriving
(
Show
,
Eq
,
Generic
,
Enum
,
Bounded
)
deriving
(
Show
,
Eq
,
Generic
,
Enum
,
Bounded
)
...
...
src/Gargantext/API/GraphQL.hs
View file @
c3464a38
...
@@ -82,6 +82,8 @@ data Mutation m
...
@@ -82,6 +82,8 @@ data Mutation m
=
Mutation
=
Mutation
{
update_user_info
::
GQLUserInfo
.
UserInfoMArgs
->
m
Int
{
update_user_info
::
GQLUserInfo
.
UserInfoMArgs
->
m
Int
,
update_user_pubmed_api_key
::
GQLUser
.
UserPubmedAPIKeyMArgs
->
m
Int
,
update_user_pubmed_api_key
::
GQLUser
.
UserPubmedAPIKeyMArgs
->
m
Int
,
update_user_epo_api_user
::
GQLUser
.
UserEPOAPIUserMArgs
->
m
Int
,
update_user_epo_api_token
::
GQLUser
.
UserEPOAPITokenMArgs
->
m
Int
,
delete_team_membership
::
GQLTeam
.
TeamDeleteMArgs
->
m
[
Int
]
,
delete_team_membership
::
GQLTeam
.
TeamDeleteMArgs
->
m
[
Int
]
,
update_node_context_category
::
GQLCTX
.
NodeContextCategoryMArgs
->
m
[
Int
]
,
update_node_context_category
::
GQLCTX
.
NodeContextCategoryMArgs
->
m
[
Int
]
}
deriving
(
Generic
,
GQLType
)
}
deriving
(
Generic
,
GQLType
)
...
@@ -127,6 +129,8 @@ rootResolver authenticatedUser policyManager =
...
@@ -127,6 +129,8 @@ rootResolver authenticatedUser policyManager =
,
tree_branch
=
GQLTree
.
resolveBreadcrumb
}
,
tree_branch
=
GQLTree
.
resolveBreadcrumb
}
,
mutationResolver
=
Mutation
{
update_user_info
=
GQLUserInfo
.
updateUserInfo
,
mutationResolver
=
Mutation
{
update_user_info
=
GQLUserInfo
.
updateUserInfo
,
update_user_pubmed_api_key
=
GQLUser
.
updateUserPubmedAPIKey
,
update_user_pubmed_api_key
=
GQLUser
.
updateUserPubmedAPIKey
,
update_user_epo_api_user
=
GQLUser
.
updateUserEPOAPIUser
,
update_user_epo_api_token
=
GQLUser
.
updateUserEPOAPIToken
,
delete_team_membership
=
GQLTeam
.
deleteTeamMembership
,
delete_team_membership
=
GQLTeam
.
deleteTeamMembership
,
update_node_context_category
=
GQLCTX
.
updateNodeContextCategory
}
,
update_node_context_category
=
GQLCTX
.
updateNodeContextCategory
}
,
subscriptionResolver
=
Undefined
}
,
subscriptionResolver
=
Undefined
}
...
...
src/Gargantext/API/GraphQL/User.hs
View file @
c3464a38
...
@@ -47,6 +47,18 @@ data UserPubmedAPIKeyMArgs
...
@@ -47,6 +47,18 @@ data UserPubmedAPIKeyMArgs
,
api_key
::
Text
}
,
api_key
::
Text
}
deriving
(
Generic
,
GQLType
)
deriving
(
Generic
,
GQLType
)
data
UserEPOAPIUserMArgs
=
UserEPOAPIUserMArgs
{
user_id
::
Int
,
api_user
::
Text
}
deriving
(
Generic
,
GQLType
)
data
UserEPOAPITokenMArgs
=
UserEPOAPITokenMArgs
{
user_id
::
Int
,
api_token
::
Text
}
deriving
(
Generic
,
GQLType
)
-- | Function to resolve user from a query.
-- | Function to resolve user from a query.
resolveUsers
resolveUsers
::
(
CmdCommon
env
)
::
(
CmdCommon
env
)
...
@@ -82,3 +94,17 @@ updateUserPubmedAPIKey UserPubmedAPIKeyMArgs { user_id, api_key } = do
...
@@ -82,3 +94,17 @@ updateUserPubmedAPIKey UserPubmedAPIKeyMArgs { user_id, api_key } = do
_
<-
lift
$
DBUser
.
updateUserPubmedAPIKey
(
Individu
.
RootId
$
UnsafeMkNodeId
user_id
)
api_key
_
<-
lift
$
DBUser
.
updateUserPubmedAPIKey
(
Individu
.
RootId
$
UnsafeMkNodeId
user_id
)
api_key
pure
1
pure
1
updateUserEPOAPIUser
::
(
CmdCommon
env
,
HasSettings
env
)
=>
UserEPOAPIUserMArgs
->
GqlM'
e
env
Int
updateUserEPOAPIUser
UserEPOAPIUserMArgs
{
user_id
,
api_user
}
=
do
_
<-
lift
$
DBUser
.
updateUserEPOAPIUser
(
Individu
.
RootId
$
UnsafeMkNodeId
user_id
)
api_user
pure
1
updateUserEPOAPIToken
::
(
CmdCommon
env
,
HasSettings
env
)
=>
UserEPOAPITokenMArgs
->
GqlM'
e
env
Int
updateUserEPOAPIToken
UserEPOAPITokenMArgs
{
user_id
,
api_token
}
=
do
_
<-
lift
$
DBUser
.
updateUserEPOAPIToken
(
Individu
.
RootId
$
UnsafeMkNodeId
user_id
)
api_token
pure
1
src/Gargantext/API/Node/Corpus/New.hs
View file @
c3464a38
...
@@ -31,6 +31,7 @@ import Data.Conduit.Internal (zipSources)
...
@@ -31,6 +31,7 @@ import Data.Conduit.Internal (zipSources)
import
Data.Swagger
import
Data.Swagger
import
Data.Text
qualified
as
T
import
Data.Text
qualified
as
T
import
Data.Text.Encoding
qualified
as
TE
import
Data.Text.Encoding
qualified
as
TE
import
EPO.API.Client.Types
qualified
as
EPO
import
Gargantext.API.Admin.Orchestrator.Types
(
JobLog
(
..
),
AsyncJobs
)
import
Gargantext.API.Admin.Orchestrator.Types
(
JobLog
(
..
),
AsyncJobs
)
import
Gargantext.API.Admin.Types
(
HasSettings
)
import
Gargantext.API.Admin.Types
(
HasSettings
)
import
Gargantext.API.Ngrams
(
commitStatePatch
,
Versioned
(
..
))
import
Gargantext.API.Ngrams
(
commitStatePatch
,
Versioned
(
..
))
...
@@ -144,6 +145,8 @@ data WithQuery = WithQuery
...
@@ -144,6 +145,8 @@ data WithQuery = WithQuery
,
_wq_node_id
::
!
Int
,
_wq_node_id
::
!
Int
,
_wq_flowListWith
::
!
FlowSocialListWith
,
_wq_flowListWith
::
!
FlowSocialListWith
,
_wq_pubmedAPIKey
::
!
(
Maybe
Text
)
,
_wq_pubmedAPIKey
::
!
(
Maybe
Text
)
,
_wq_epoAPIUser
::
!
(
Maybe
Text
)
,
_wq_epoAPIToken
::
!
(
Maybe
Text
)
}
}
deriving
(
Show
,
Eq
,
Generic
)
deriving
(
Show
,
Eq
,
Generic
)
...
@@ -163,6 +166,8 @@ instance Arbitrary WithQuery where
...
@@ -163,6 +166,8 @@ instance Arbitrary WithQuery where
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
------------------------------------------------------------------------
------------------------------------------------------------------------
...
@@ -204,12 +209,16 @@ addToCorpusWithQuery user cid (WithQuery { _wq_query = q
...
@@ -204,12 +209,16 @@ addToCorpusWithQuery user cid (WithQuery { _wq_query = q
,
_wq_datafield
=
datafield
,
_wq_datafield
=
datafield
,
_wq_lang
=
l
,
_wq_lang
=
l
,
_wq_flowListWith
=
flw
,
_wq_flowListWith
=
flw
,
_wq_pubmedAPIKey
=
mPubmedAPIKey
})
maybeLimit
jobHandle
=
do
,
_wq_pubmedAPIKey
=
mPubmedAPIKey
,
..
})
maybeLimit
jobHandle
=
do
-- TODO ...
-- TODO ...
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"(cid, dbs) "
<>
show
(
cid
,
dbs
)
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"(cid, dbs) "
<>
show
(
cid
,
dbs
)
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"datafield "
<>
show
datafield
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"datafield "
<>
show
datafield
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"flowListWith "
<>
show
flw
$
(
logLocM
)
DEBUG
$
T
.
pack
$
"flowListWith "
<>
show
flw
let
mEPOAuthKey
=
EPO
.
AuthKey
<$>
(
EPO
.
User
<$>
_wq_epoAPIUser
)
<*>
(
EPO
.
Token
<$>
_wq_epoAPIToken
)
addLanguageToCorpus
cid
l
addLanguageToCorpus
cid
l
case
datafield
of
case
datafield
of
...
@@ -233,7 +242,7 @@ addToCorpusWithQuery user cid (WithQuery { _wq_query = q
...
@@ -233,7 +242,7 @@ addToCorpusWithQuery user cid (WithQuery { _wq_query = q
let
db
=
database2origin
dbs
let
db
=
database2origin
dbs
-- mPubmedAPIKey <- getUserPubmedAPIKey user
-- mPubmedAPIKey <- getUserPubmedAPIKey user
-- printDebug "[addToCorpusWithQuery] mPubmedAPIKey" mPubmedAPIKey
-- printDebug "[addToCorpusWithQuery] mPubmedAPIKey" mPubmedAPIKey
eTxt
<-
getDataText
db
(
Multi
l
)
q
mPubmedAPIKey
maybeLimit
eTxt
<-
getDataText
db
(
Multi
l
)
q
mPubmedAPIKey
m
EPOAuthKey
m
aybeLimit
-- printDebug "[G.A.N.C.New] lTxts" lTxts
-- printDebug "[G.A.N.C.New] lTxts" lTxts
case
eTxt
of
case
eTxt
of
...
...
src/Gargantext/API/Node/Corpus/Types.hs
View file @
c3464a38
...
@@ -32,6 +32,7 @@ data Database = Empty
...
@@ -32,6 +32,7 @@ data Database = Empty
|
HAL
|
HAL
|
IsTex
|
IsTex
|
Isidore
|
Isidore
|
EPO
deriving
(
Eq
,
Show
,
Generic
,
Enum
,
Bounded
)
deriving
(
Eq
,
Show
,
Generic
,
Enum
,
Bounded
)
instance
Arbitrary
Database
where
instance
Arbitrary
Database
where
...
@@ -49,6 +50,7 @@ database2origin Arxiv = ExternalOrigin Types.Arxiv
...
@@ -49,6 +50,7 @@ database2origin Arxiv = ExternalOrigin Types.Arxiv
database2origin
HAL
=
ExternalOrigin
Types
.
HAL
database2origin
HAL
=
ExternalOrigin
Types
.
HAL
database2origin
IsTex
=
ExternalOrigin
Types
.
IsTex
database2origin
IsTex
=
ExternalOrigin
Types
.
IsTex
database2origin
Isidore
=
ExternalOrigin
Types
.
Isidore
database2origin
Isidore
=
ExternalOrigin
Types
.
Isidore
database2origin
EPO
=
ExternalOrigin
Types
.
EPO
------------------------------------------------------------------------
------------------------------------------------------------------------
data
Datafield
=
Gargantext
data
Datafield
=
Gargantext
...
...
src/Gargantext/Core/Text/Corpus/API.hs
View file @
c3464a38
...
@@ -21,9 +21,11 @@ module Gargantext.Core.Text.Corpus.API
...
@@ -21,9 +21,11 @@ module Gargantext.Core.Text.Corpus.API
import
Conduit
import
Conduit
import
Control.Monad.Except
import
Control.Monad.Except
import
Data.Text
qualified
as
T
import
Data.Text
qualified
as
T
import
EPO.API.Client.Types
qualified
as
EPO
import
Gargantext.API.Admin.Orchestrator.Types
(
ExternalAPIs
(
..
),
externalAPIs
)
import
Gargantext.API.Admin.Orchestrator.Types
(
ExternalAPIs
(
..
),
externalAPIs
)
import
Gargantext.Core
(
Lang
(
..
),
toISO639
)
import
Gargantext.Core
(
Lang
(
..
),
toISO639
,
toISO639EN
)
import
Gargantext.Core.Text.Corpus.API.Arxiv
qualified
as
Arxiv
import
Gargantext.Core.Text.Corpus.API.Arxiv
qualified
as
Arxiv
import
Gargantext.Core.Text.Corpus.API.EPO
qualified
as
EPO
import
Gargantext.Core.Text.Corpus.API.Hal
qualified
as
HAL
import
Gargantext.Core.Text.Corpus.API.Hal
qualified
as
HAL
import
Gargantext.Core.Text.Corpus.API.Isidore
qualified
as
ISIDORE
import
Gargantext.Core.Text.Corpus.API.Isidore
qualified
as
ISIDORE
import
Gargantext.Core.Text.Corpus.API.Istex
qualified
as
ISTEX
import
Gargantext.Core.Text.Corpus.API.Istex
qualified
as
ISTEX
...
@@ -47,10 +49,12 @@ get :: ExternalAPIs
...
@@ -47,10 +49,12 @@ get :: ExternalAPIs
->
Lang
->
Lang
->
Corpus
.
RawQuery
->
Corpus
.
RawQuery
->
Maybe
PUBMED
.
APIKey
->
Maybe
PUBMED
.
APIKey
->
Maybe
EPO
.
AuthKey
->
Text
->
Maybe
Corpus
.
Limit
->
Maybe
Corpus
.
Limit
-- -> IO [HyperdataDocument]
-- -> IO [HyperdataDocument]
->
IO
(
Either
GetCorpusError
(
Maybe
Integer
,
ConduitT
()
HyperdataDocument
IO
()
))
->
IO
(
Either
GetCorpusError
(
Maybe
Integer
,
ConduitT
()
HyperdataDocument
IO
()
))
get
externalAPI
la
q
mPubmedAPIKey
limit
=
do
get
externalAPI
la
q
mPubmedAPIKey
epoAuthKey
epoAPIUrl
limit
=
do
-- For PUBMED, HAL, IsTex, Isidore and OpenAlex, we want to send the query as-it.
-- For PUBMED, HAL, IsTex, Isidore and OpenAlex, we want to send the query as-it.
-- For Arxiv we parse the query into a structured boolean query we submit over.
-- For Arxiv we parse the query into a structured boolean query we submit over.
case
externalAPI
of
case
externalAPI
of
...
@@ -69,5 +73,7 @@ get externalAPI la q mPubmedAPIKey limit = do
...
@@ -69,5 +73,7 @@ get externalAPI la q mPubmedAPIKey limit = do
Isidore
->
do
Isidore
->
do
docs
<-
ISIDORE
.
get
la
(
Corpus
.
getLimit
<$>
limit
)
(
Just
$
Corpus
.
getRawQuery
q
)
Nothing
docs
<-
ISIDORE
.
get
la
(
Corpus
.
getLimit
<$>
limit
)
(
Just
$
Corpus
.
getRawQuery
q
)
Nothing
pure
$
Right
(
Just
$
fromIntegral
$
length
docs
,
yieldMany
docs
)
pure
$
Right
(
Just
$
fromIntegral
$
length
docs
,
yieldMany
docs
)
EPO
->
do
first
ExternalAPIError
<$>
EPO
.
get
epoAuthKey
epoAPIUrl
q
(
toISO639EN
la
)
limit
where
where
parse_query
=
first
(
InvalidInputQuery
q
.
T
.
pack
)
$
Corpus
.
parseQuery
q
parse_query
=
first
(
InvalidInputQuery
q
.
T
.
pack
)
$
Corpus
.
parseQuery
q
src/Gargantext/Core/Text/Corpus/API/EPO.hs
0 → 100644
View file @
c3464a38
{-|
Module : Gargantext.Core.Text.Corpus.API.EPO
Description : EPO (patents) API interface
Copyright : (c) CNRS, 2023
License : AGPL + CECILL v3
Maintainer : team@gargantext.org
Stability : experimental
Portability : POSIX
-}
module
Gargantext.Core.Text.Corpus.API.EPO
where
import
Conduit
import
Data.LanguageCodes
(
ISO639_1
)
import
Data.Map.Strict
qualified
as
Map
import
Data.Text
qualified
as
T
import
EPO.API.Client.Types
qualified
as
EPO
import
EPO.API.Client.Implementation
qualified
as
EPO
import
Gargantext.Core
(
iso639ToText
)
import
Gargantext.Core.Text.Corpus.Query
qualified
as
Corpus
import
Gargantext.Database.Admin.Types.Hyperdata
(
HyperdataDocument
(
..
))
import
Network.URI
(
parseURI
)
import
Protolude
import
Servant.Client.Core
(
ClientError
(
ConnectionError
))
get
::
Maybe
EPO
.
AuthKey
->
Text
->
Corpus
.
RawQuery
->
ISO639_1
->
Maybe
Corpus
.
Limit
->
IO
(
Either
ClientError
(
Maybe
Integer
,
ConduitT
()
HyperdataDocument
IO
()
))
get
Nothing
_
_
_
_
=
do
-- throwIO $ EPO.OtherError "AuthKey is required"
pure
$
Left
$
ConnectionError
$
toException
$
ErrorCall
"AuthKey is required"
get
(
Just
authKey
)
epoAPIUrl
q
lang
mLimit
=
do
let
limit
=
Corpus
.
getLimit
<$>
mLimit
case
parseURI
(
T
.
unpack
epoAPIUrl
)
of
Nothing
->
pure
$
Left
$
ConnectionError
$
toException
$
ErrorCall
"Cannot parse API URL"
Just
apiUrl
->
do
eRes
<-
EPO
.
searchEPOAPIC
apiUrl
authKey
Nothing
limit
(
Corpus
.
getRawQuery
q
)
pure
$
(
\
(
total
,
itemsC
)
->
(
Just
total
,
itemsC
.|
mapC
(
toDoc
lang
)))
<$>
eRes
-- EPO.Paginated { .. } <- EPO.searchEPOAPI apiUrl authKey 1 20 (Corpus.getRawQuery q)
-- pure $ Right ( Just $ fromIntegral total, yieldMany items .| mapC (toDoc lang) )
toDoc
::
ISO639_1
->
EPO
.
HyperdataDocument
->
HyperdataDocument
toDoc
lang
(
EPO
.
HyperdataDocument
{
..
})
=
HyperdataDocument
{
_hd_bdd
=
Just
"EPO"
,
_hd_doi
=
Nothing
,
_hd_url
=
Nothing
,
_hd_uniqId
=
id
,
_hd_uniqIdBdd
=
id
,
_hd_page
=
Nothing
,
_hd_title
=
Map
.
lookup
lang
titles
,
_hd_authors
=
authors_
,
_hd_institutes
=
Nothing
,
_hd_source
=
Nothing
,
_hd_abstract
=
Map
.
lookup
lang
abstracts
,
_hd_publication_date
=
publication_date
,
_hd_publication_year
=
publication_year
,
_hd_publication_month
=
publication_month
,
_hd_publication_day
=
publication_day
,
_hd_publication_hour
=
Nothing
,
_hd_publication_minute
=
Nothing
,
_hd_publication_second
=
Nothing
,
_hd_language_iso2
=
Just
$
iso639ToText
lang
}
where
authors_
=
if
authors
==
[]
then
Nothing
else
Just
(
T
.
intercalate
", "
authors
)
-- EPO.withAuthKey authKey $ \token -> do
-- let range = EPO.Range { rBegin = 1, rEnd = limit }
-- (len, docsC) <- EPO.searchPublishedDataWithFetchC token (Just $ Corpus.getRawQuery q) (Just range)
-- pure (len, docsC .|
-- takeC limit .|
-- mapC (toDoc lang))
-- toDoc :: ISO639_1 -> EPO.ExchangeDocument -> HyperdataDocument
-- toDoc lang (EPO.ExchangeDocument { bibliographicData = EPO.BibliographicData { .. }
-- , abstracts } ) =
-- HyperdataDocument { _hd_bdd = Just "EPO"
-- , _hd_doi = Nothing
-- , _hd_url = Nothing
-- , _hd_uniqId = EPO.documentIdToText <$> (head documentIds)
-- , _hd_uniqIdBdd = EPO.documentIdToText <$> (head documentIds)
-- , _hd_page = Nothing
-- , _hd_title = Map.lookup lang inventionTitlesMap
-- , _hd_authors = authors parties
-- , _hd_institutes = Nothing
-- , _hd_source = Nothing
-- , _hd_abstract = Map.lookup lang abstractMap
-- , _hd_publication_date = T.pack <$> showGregorian <$> publicationDate
-- , _hd_publication_year = year
-- , _hd_publication_month = month
-- , _hd_publication_day = day
-- , _hd_publication_hour = Nothing
-- , _hd_publication_minute = Nothing
-- , _hd_publication_second = Nothing
-- , _hd_language_iso2 = Just $ iso639ToText lang }
-- where
-- authors :: EPO.Parties -> Maybe Text
-- authors (EPO.Parties { inventors = Nothing }) = Nothing
-- authors (EPO.Parties { inventors = Just EPO.Inventors { inventors } }) =
-- Just $ T.intercalate ", " (getInventorName <$> inventors)
-- getInventorName :: EPO.Inventor -> Text
-- getInventorName (EPO.Inventor { inventorName = EPO.InventorName { name } }) = name
-- abstractMap :: Map.Map ISO639_1 Text
-- abstractMap = Map.fromList [(l, text) | (EPO.Abstract { lang = l, text }) <- abstracts]
-- EPO.PublicationReferenceDet { documentIds } = publicationReference
-- dates :: [Day]
-- dates = catMaybes (EPO.date <$> documentIds)
-- publicationDate :: Maybe Day
-- publicationDate = head dates
-- (year, month, day) = case publicationDate of
-- Nothing -> (Nothing, Nothing, Nothing)
-- Just pd -> let (y, m, d) = toGregorian pd in
-- (Just $ fromIntegral y, Just m, Just d)
-- inventionTitlesMap :: Map ISO639_1 Text
-- inventionTitlesMap = Map.fromList [(l, text) | EPO.InventionTitle { lang = l, text } <- inventionTitles]
src/Gargantext/Core/Text/Corpus/API/OpenAlex.hs
View file @
c3464a38
...
@@ -45,10 +45,10 @@ toDoc (OA.Work { .. } ) =
...
@@ -45,10 +45,10 @@ toDoc (OA.Work { .. } ) =
,
_hd_institutes
=
institutes
authorships
,
_hd_institutes
=
institutes
authorships
,
_hd_source
=
source
,
_hd_source
=
source
,
_hd_abstract
=
Just
abstract_reconstructed
,
_hd_abstract
=
Just
abstract_reconstructed
,
_hd_publication_date
=
Just
$
show
publication_date
,
_hd_publication_date
=
Just
$
OA
.
showDate
publication_date
,
_hd_publication_year
=
Just
$
publication_year
,
_hd_publication_year
=
Just
$
publication_year
,
_hd_publication_month
=
Nothing
-- TODO
,
_hd_publication_month
=
Just
$
OA
.
dateMonth
publication_date
,
_hd_publication_day
=
Nothing
-- TODO
,
_hd_publication_day
=
Just
$
OA
.
dateDay
publication_date
,
_hd_publication_hour
=
Nothing
-- TODO
,
_hd_publication_hour
=
Nothing
-- TODO
,
_hd_publication_minute
=
Nothing
-- TODO
,
_hd_publication_minute
=
Nothing
-- TODO
,
_hd_publication_second
=
Nothing
-- TODO
,
_hd_publication_second
=
Nothing
-- TODO
...
...
src/Gargantext/Database/Action/Flow.hs
View file @
c3464a38
...
@@ -69,6 +69,7 @@ import Data.Proxy
...
@@ -69,6 +69,7 @@ import Data.Proxy
import
Data.Set
qualified
as
Set
import
Data.Set
qualified
as
Set
import
Data.Swagger
import
Data.Swagger
import
Data.Text
qualified
as
T
import
Data.Text
qualified
as
T
import
EPO.API.Client.Types
qualified
as
EPO
import
Gargantext.API.Ngrams.Tools
(
getTermsWith
)
import
Gargantext.API.Ngrams.Tools
(
getTermsWith
)
import
Gargantext.API.Ngrams.Types
qualified
as
NT
import
Gargantext.API.Ngrams.Types
qualified
as
NT
import
Gargantext.Core
(
Lang
(
..
),
PosTagAlgo
(
..
),
NLPServerConfig
)
import
Gargantext.Core
(
Lang
(
..
),
PosTagAlgo
(
..
),
NLPServerConfig
)
...
@@ -101,7 +102,7 @@ import Gargantext.Database.Action.Search (searchDocInDatabase)
...
@@ -101,7 +102,7 @@ import Gargantext.Database.Action.Search (searchDocInDatabase)
import
Gargantext.Database.Admin.Config
(
userMaster
,
corpusMasterName
)
import
Gargantext.Database.Admin.Config
(
userMaster
,
corpusMasterName
)
import
Gargantext.Database.Admin.Types.Hyperdata
import
Gargantext.Database.Admin.Types.Hyperdata
import
Gargantext.Database.Admin.Types.Node
hiding
(
DEBUG
)
-- (HyperdataDocument(..), NodeType(..), NodeId, UserId, ListId, CorpusId, RootId, MasterCorpusId, MasterUserId)
import
Gargantext.Database.Admin.Types.Node
hiding
(
DEBUG
)
-- (HyperdataDocument(..), NodeType(..), NodeId, UserId, ListId, CorpusId, RootId, MasterCorpusId, MasterUserId)
import
Gargantext.Database.Prelude
(
DbCmd
'
,
DBCmd
)
import
Gargantext.Database.Prelude
(
DbCmd
'
,
DBCmd
,
hasConfig
)
import
Gargantext.Database.Query.Table.ContextNodeNgrams2
import
Gargantext.Database.Query.Table.ContextNodeNgrams2
import
Gargantext.Database.Query.Table.Ngrams
import
Gargantext.Database.Query.Table.Ngrams
import
Gargantext.Database.Query.Table.Node
import
Gargantext.Database.Query.Table.Node
...
@@ -116,6 +117,7 @@ import Gargantext.Database.Schema.Node (NodePoly(..), node_id)
...
@@ -116,6 +117,7 @@ import Gargantext.Database.Schema.Node (NodePoly(..), node_id)
import
Gargantext.Database.Schema.Node
(
node_hyperdata
)
import
Gargantext.Database.Schema.Node
(
node_hyperdata
)
import
Gargantext.Database.Types
import
Gargantext.Database.Types
import
Gargantext.Prelude
hiding
(
to
)
import
Gargantext.Prelude
hiding
(
to
)
import
Gargantext.Prelude.Config
(
GargConfig
(
..
))
import
Gargantext.Prelude.Crypto.Hash
(
Hash
)
import
Gargantext.Prelude.Crypto.Hash
(
Hash
)
import
Gargantext.System.Logging
import
Gargantext.System.Logging
import
Gargantext.Utils.Jobs
(
JobHandle
,
MonadJobStatus
(
..
))
import
Gargantext.Utils.Jobs
(
JobHandle
,
MonadJobStatus
(
..
))
...
@@ -160,12 +162,14 @@ getDataText :: (HasNodeError err)
...
@@ -160,12 +162,14 @@ getDataText :: (HasNodeError err)
->
TermType
Lang
->
TermType
Lang
->
API
.
RawQuery
->
API
.
RawQuery
->
Maybe
PUBMED
.
APIKey
->
Maybe
PUBMED
.
APIKey
->
Maybe
EPO
.
AuthKey
->
Maybe
API
.
Limit
->
Maybe
API
.
Limit
->
DBCmd
err
(
Either
API
.
GetCorpusError
DataText
)
->
DBCmd
err
(
Either
API
.
GetCorpusError
DataText
)
getDataText
(
ExternalOrigin
api
)
la
q
mPubmedAPIKey
li
=
do
getDataText
(
ExternalOrigin
api
)
la
q
mPubmedAPIKey
mAuthKey
li
=
do
eRes
<-
liftBase
$
API
.
get
api
(
_tt_lang
la
)
q
mPubmedAPIKey
li
cfg
<-
view
hasConfig
eRes
<-
liftBase
$
API
.
get
api
(
_tt_lang
la
)
q
mPubmedAPIKey
mAuthKey
(
_gc_epo_api_url
cfg
)
li
pure
$
DataNew
<$>
eRes
pure
$
DataNew
<$>
eRes
getDataText
(
InternalOrigin
_
)
_la
q
_
_li
=
do
getDataText
(
InternalOrigin
_
)
_la
q
_
_
_
li
=
do
(
_masterUserId
,
_masterRootId
,
cId
)
<-
getOrMk_RootWithCorpus
(
_masterUserId
,
_masterRootId
,
cId
)
<-
getOrMk_RootWithCorpus
(
UserName
userMaster
)
(
UserName
userMaster
)
(
Left
""
)
(
Left
""
)
...
@@ -180,7 +184,7 @@ getDataText_Debug :: (HasNodeError err)
...
@@ -180,7 +184,7 @@ getDataText_Debug :: (HasNodeError err)
->
Maybe
API
.
Limit
->
Maybe
API
.
Limit
->
DBCmd
err
()
->
DBCmd
err
()
getDataText_Debug
a
l
q
li
=
do
getDataText_Debug
a
l
q
li
=
do
result
<-
getDataText
a
l
q
Nothing
li
result
<-
getDataText
a
l
q
Nothing
Nothing
li
case
result
of
case
result
of
Left
err
->
liftBase
$
putText
$
show
err
Left
err
->
liftBase
$
putText
$
show
err
Right
res
->
liftBase
$
printDataText
res
Right
res
->
liftBase
$
printDataText
res
...
...
src/Gargantext/Database/Admin/Types/Hyperdata/User.hs
View file @
c3464a38
...
@@ -40,6 +40,8 @@ data HyperdataUser =
...
@@ -40,6 +40,8 @@ data HyperdataUser =
,
_hu_shared
::
!
(
Maybe
HyperdataContact
)
,
_hu_shared
::
!
(
Maybe
HyperdataContact
)
,
_hu_public
::
!
(
Maybe
HyperdataPublic
)
,
_hu_public
::
!
(
Maybe
HyperdataPublic
)
,
_hu_pubmed_api_key
::
!
(
Maybe
PUBMED
.
APIKey
)
,
_hu_pubmed_api_key
::
!
(
Maybe
PUBMED
.
APIKey
)
,
_hu_epo_api_user
::
!
(
Maybe
Text
)
,
_hu_epo_api_token
::
!
(
Maybe
Text
)
}
deriving
(
Eq
,
Show
,
Generic
)
}
deriving
(
Eq
,
Show
,
Generic
)
instance
GQLType
HyperdataUser
where
instance
GQLType
HyperdataUser
where
...
@@ -71,7 +73,9 @@ defaultHyperdataUser =
...
@@ -71,7 +73,9 @@ defaultHyperdataUser =
{
_hu_private
=
Just
defaultHyperdataPrivate
{
_hu_private
=
Just
defaultHyperdataPrivate
,
_hu_shared
=
Just
defaultHyperdataContact
,
_hu_shared
=
Just
defaultHyperdataContact
,
_hu_public
=
Just
defaultHyperdataPublic
,
_hu_public
=
Just
defaultHyperdataPublic
,
_hu_pubmed_api_key
=
Nothing
}
,
_hu_pubmed_api_key
=
Nothing
,
_hu_epo_api_user
=
Nothing
,
_hu_epo_api_token
=
Nothing
}
defaultHyperdataPublic
::
HyperdataPublic
defaultHyperdataPublic
::
HyperdataPublic
defaultHyperdataPublic
=
HyperdataPublic
"pseudo"
[
1
..
10
]
defaultHyperdataPublic
=
HyperdataPublic
"pseudo"
[
1
..
10
]
...
@@ -100,7 +104,12 @@ $(deriveJSON (unPrefix "_hpu_") ''HyperdataPublic)
...
@@ -100,7 +104,12 @@ $(deriveJSON (unPrefix "_hpu_") ''HyperdataPublic)
-- | Arbitrary instances
-- | Arbitrary instances
instance
Arbitrary
HyperdataUser
where
instance
Arbitrary
HyperdataUser
where
arbitrary
=
HyperdataUser
<$>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
arbitrary
=
HyperdataUser
<$>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
<*>
arbitrary
instance
Arbitrary
HyperdataPrivate
where
instance
Arbitrary
HyperdataPrivate
where
arbitrary
=
pure
defaultHyperdataPrivate
arbitrary
=
pure
defaultHyperdataPrivate
...
...
src/Gargantext/Database/Query/Table/User.hs
View file @
c3464a38
...
@@ -34,6 +34,8 @@ module Gargantext.Database.Query.Table.User
...
@@ -34,6 +34,8 @@ module Gargantext.Database.Query.Table.User
,
updateUserForgotPasswordUUID
,
updateUserForgotPasswordUUID
,
getUserPubmedAPIKey
,
getUserPubmedAPIKey
,
updateUserPubmedAPIKey
,
updateUserPubmedAPIKey
,
updateUserEPOAPIUser
,
updateUserEPOAPIToken
,
getUser
,
getUser
,
insertNewUsers
,
insertNewUsers
,
selectUsersLightWith
,
selectUsersLightWith
...
@@ -56,7 +58,7 @@ import Data.UUID qualified as UUID
...
@@ -56,7 +58,7 @@ import Data.UUID qualified as UUID
import
Gargantext.Core
(
HasDBid
)
import
Gargantext.Core
(
HasDBid
)
import
Gargantext.Core.Types.Individu
import
Gargantext.Core.Types.Individu
import
Gargantext.Database.Admin.Config
(
nodeTypeId
)
import
Gargantext.Database.Admin.Config
(
nodeTypeId
)
import
Gargantext.Database.Admin.Types.Hyperdata
(
HyperdataUser
(
..
),
hu_pubmed_api_key
)
import
Gargantext.Database.Admin.Types.Hyperdata
(
HyperdataUser
(
..
),
hu_pubmed_api_key
,
hu_epo_api_user
,
hu_epo_api_token
)
import
Gargantext.Database.Admin.Types.Node
(
NodeType
(
NodeUser
),
Node
,
NodeId
(
..
),
pgNodeId
)
import
Gargantext.Database.Admin.Types.Node
(
NodeType
(
NodeUser
),
Node
,
NodeId
(
..
),
pgNodeId
)
import
Gargantext.Database.Admin.Types.Node
(
UserId
(
..
))
import
Gargantext.Database.Admin.Types.Node
(
UserId
(
..
))
import
Gargantext.Database.Prelude
import
Gargantext.Database.Prelude
...
@@ -275,6 +277,21 @@ updateUserPubmedAPIKey (RootId uId) apiKey = do
...
@@ -275,6 +277,21 @@ updateUserPubmedAPIKey (RootId uId) apiKey = do
_
<-
updateNodeWithType
uId
NodeUser
(
Proxy
::
Proxy
HyperdataUser
)
(
\
h
->
h
&
hu_pubmed_api_key
?~
apiKey
)
_
<-
updateNodeWithType
uId
NodeUser
(
Proxy
::
Proxy
HyperdataUser
)
(
\
h
->
h
&
hu_pubmed_api_key
?~
apiKey
)
pure
1
pure
1
updateUserPubmedAPIKey
_
_
=
undefined
updateUserPubmedAPIKey
_
_
=
undefined
updateUserEPOAPIUser
::
(
HasDBid
NodeType
,
HasNodeError
err
)
=>
User
->
Text
->
DBCmd
err
Int64
updateUserEPOAPIUser
(
RootId
uId
)
apiUser
=
do
_
<-
updateNodeWithType
uId
NodeUser
(
Proxy
::
Proxy
HyperdataUser
)
(
\
h
->
h
&
hu_epo_api_user
?~
apiUser
)
pure
1
updateUserEPOAPIUser
_
_
=
undefined
updateUserEPOAPIToken
::
(
HasDBid
NodeType
,
HasNodeError
err
)
=>
User
->
Text
->
DBCmd
err
Int64
updateUserEPOAPIToken
(
RootId
uId
)
apiToken
=
do
_
<-
updateNodeWithType
uId
NodeUser
(
Proxy
::
Proxy
HyperdataUser
)
(
\
h
->
h
&
hu_epo_api_token
?~
apiToken
)
pure
1
updateUserEPOAPIToken
_
_
=
undefined
------------------------------------------------------------------
------------------------------------------------------------------
-- | Select User with some parameters
-- | Select User with some parameters
-- Not optimized version
-- Not optimized version
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment