Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
157
Issues
157
List
Board
Labels
Milestones
Merge Requests
9
Merge Requests
9
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
0e384787
Commit
0e384787
authored
Dec 07, 2021
by
Przemyslaw Kaminski
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[searx] parser works now
One issue is that it creates new list/docs/board/graph nodes.
parent
fc4a45c7
Pipeline
#2235
failed with stage
in 10 minutes and 25 seconds
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
5 deletions
+8
-5
Searx.hs
src/Gargantext/API/Node/Corpus/Searx.hs
+7
-4
Insert.hs
src/Gargantext/Database/Query/Table/Node/Document/Insert.hs
+1
-1
No files found.
src/Gargantext/API/Node/Corpus/Searx.hs
View file @
0e384787
...
...
@@ -142,7 +142,7 @@ triggerSearxSearch user cId q l logStatus = do
,
_fsp_query
=
q
,
_fsp_url
=
surl
}
printDebug
"[triggerSearxSearch] res"
res
--
printDebug "[triggerSearxSearch] res" res
case
res
of
Left
_
->
pure
()
...
...
@@ -152,7 +152,10 @@ triggerSearxSearch user cId q l logStatus = do
-- docs :: [Either Text HyperdataDocument]
let
docs'
=
catMaybes
$
rightToMaybe
<$>
docs
Prelude
.
mapM_
(
\
(
HyperdataDocument
{
_hd_title
,
_hd_publication_year
,
_hd_publication_date
})
->
do
printDebug
"[triggerSearxSearch] doc time"
$
(
show
_hd_title
)
<>
" :: "
<>
(
show
_hd_publication_year
)
<>
" :: "
<>
(
show
_hd_publication_date
)
printDebug
"[triggerSearxSearch] doc time"
$
"[title] "
<>
(
show
_hd_title
)
<>
" :: [publication_year] "
<>
(
show
_hd_publication_year
)
<>
" :: [publication_date] "
<>
(
show
_hd_publication_date
)
)
docs'
_
<-
flowDataText
user
(
DataNew
[
docs'
])
(
Multi
EN
)
cId
Nothing
logStatus
pure
()
...
...
@@ -161,7 +164,7 @@ triggerSearxSearch user cId q l logStatus = do
hyperdataDocumentFromSearxResult
::
SearxResult
->
Either
T
.
Text
HyperdataDocument
hyperdataDocumentFromSearxResult
(
SearxResult
{
_sr_content
,
_sr_engine
,
_sr_pubdate
,
_sr_title
})
=
do
let
mDate
=
parseTimeM
False
defaultTimeLocale
"%Y-%m-%d %H:%M:%S"
(
T
.
unpack
_sr_pubdate
)
::
Maybe
Day
let
mDate
=
parseTimeM
False
defaultTimeLocale
"%Y-%m-%d %H:%M:%S
+0000
"
(
T
.
unpack
_sr_pubdate
)
::
Maybe
Day
let
mGregorian
=
toGregorian
<$>
mDate
Right
HyperdataDocument
{
_hd_bdd
=
Just
"Searx"
,
_hd_doi
=
Nothing
...
...
@@ -169,7 +172,7 @@ hyperdataDocumentFromSearxResult (SearxResult { _sr_content, _sr_engine, _sr_pub
,
_hd_uniqId
=
Nothing
,
_hd_uniqIdBdd
=
Nothing
,
_hd_page
=
Nothing
,
_hd_title
=
Just
_sr_title
,
_hd_title
=
Just
$
(
"["
<>
_sr_pubdate
<>
"] "
)
<>
_sr_title
,
_hd_authors
=
Nothing
,
_hd_institutes
=
Nothing
,
_hd_source
=
Just
_sr_engine
...
...
src/Gargantext/Database/Query/Table/Node/Document/Insert.hs
View file @
0e384787
...
...
@@ -279,7 +279,7 @@ instance ToNode HyperdataDocument where
where
n
=
maybe
"No Title"
(
DT
.
take
255
)
(
_hd_title
h
)
date
=
jour
y
m
d
y
=
maybe
0
fromIntegral
$
_hd_publication_year
h
y
=
maybe
1
fromIntegral
$
_hd_publication_year
h
m
=
fromMaybe
1
$
_hd_publication_month
h
d
=
fromMaybe
1
$
_hd_publication_day
h
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment