Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
haskell-gargantext
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
195
Issues
195
List
Board
Labels
Milestones
Merge Requests
12
Merge Requests
12
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
gargantext
haskell-gargantext
Commits
fb0d8413
Verified
Commit
fb0d8413
authored
Mar 04, 2025
by
Przemyslaw Kaminski
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[temp file] simplify worker cleanup, add comments to large object functions
parent
0e65a7b3
Pipeline
#7402
passed with stages
in 122 minutes and 47 seconds
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
12 additions
and
4 deletions
+12
-4
Worker.hs
src/Gargantext/Core/Worker.hs
+1
-4
Prelude.hs
src/Gargantext/Database/Prelude.hs
+11
-0
No files found.
src/Gargantext/Core/Worker.hs
View file @
fb0d8413
...
@@ -235,10 +235,7 @@ performAction env _state bm = do
...
@@ -235,10 +235,7 @@ performAction env _state bm = do
-- TODO CES.filnally
-- TODO CES.filnally
$
(
logLocM
)
DEBUG
"[performAction] add to corpus with temporary file"
$
(
logLocM
)
DEBUG
"[performAction] add to corpus with temporary file"
CES
.
finally
(
addToCorpusWithTempFile
_actf_user
_actf_cid
_actf_args
jh
)
CES
.
finally
(
addToCorpusWithTempFile
_actf_user
_actf_cid
_actf_args
jh
)
(
do
(
removeLargeObject
$
_wtf_file_oid
_actf_args
)
let
oId
=
_wtf_file_oid
_actf_args
$
(
logLocM
)
DEBUG
$
"[performAction] removing large object: "
<>
show
oId
removeLargeObject
oId
)
-- | Perform external API search query and index documents in corpus
-- | Perform external API search query and index documents in corpus
AddCorpusWithQuery
{
..
}
->
runWorkerMonad
env
$
do
AddCorpusWithQuery
{
..
}
->
runWorkerMonad
env
$
do
...
...
src/Gargantext/Database/Prelude.hs
View file @
fb0d8413
...
@@ -278,6 +278,12 @@ createDBIfNotExists connStr dbName = do
...
@@ -278,6 +278,12 @@ createDBIfNotExists connStr dbName = do
-- PostgreSQL Large Object functionality
-- PostgreSQL Large Object functionality
-- https://www.postgresql.org/docs/17/largeobjects.html
-- https://www.postgresql.org/docs/17/largeobjects.html
-- NOTE: During development of this feature, I had problems (in tests)
-- with a hanging transaction. After debugging, it turned out this
-- was, for some reason, conflicting with our `logLocM` (though I'm no
-- sure why). Please be careful when adding debug info to large
-- objects and if you do, make sure the tests run.
createLargeObject
::
BS
.
ByteString
->
DBCmd
err
PSQL
.
Oid
createLargeObject
::
BS
.
ByteString
->
DBCmd
err
PSQL
.
Oid
createLargeObject
bs
=
mkCmd
$
\
c
->
PGS
.
withTransaction
c
$
do
createLargeObject
bs
=
mkCmd
$
\
c
->
PGS
.
withTransaction
c
$
do
oId
<-
PSQL
.
loCreat
c
oId
<-
PSQL
.
loCreat
c
...
@@ -286,6 +292,8 @@ createLargeObject bs = mkCmd $ \c -> PGS.withTransaction c $ do
...
@@ -286,6 +292,8 @@ createLargeObject bs = mkCmd $ \c -> PGS.withTransaction c $ do
PSQL
.
loClose
c
loFd
PSQL
.
loClose
c
loFd
pure
oId
pure
oId
-- | Read a large object directly, given an oid. We read it in a
-- single transaction, looping by given chunk size
readLargeObject
::
PSQL
.
Oid
->
DBCmd
err
BS
.
ByteString
readLargeObject
::
PSQL
.
Oid
->
DBCmd
err
BS
.
ByteString
readLargeObject
oId
=
mkCmd
$
\
c
->
PGS
.
withTransaction
c
$
do
readLargeObject
oId
=
mkCmd
$
\
c
->
PGS
.
withTransaction
c
$
do
loFd
<-
PSQL
.
loOpen
c
oId
PSQL
.
ReadMode
loFd
<-
PSQL
.
loOpen
c
oId
PSQL
.
ReadMode
...
@@ -303,6 +311,9 @@ readLargeObject oId = mkCmd $ \c -> PGS.withTransaction c $ do
...
@@ -303,6 +311,9 @@ readLargeObject oId = mkCmd $ \c -> PGS.withTransaction c $ do
PSQL
.
loClose
c
loFd
PSQL
.
loClose
c
loFd
pure
s
pure
s
-- | Read large object by exporting it to a temporary file, then
-- reading that file. The difference from 'readLargeObject' is that we
-- have only 1 call inside a transaction
readLargeObjectViaTempFile
::
(
CES
.
MonadMask
m
,
IsDBCmd
env
err
m
)
readLargeObjectViaTempFile
::
(
CES
.
MonadMask
m
,
IsDBCmd
env
err
m
)
=>
PSQL
.
Oid
->
m
BS
.
ByteString
=>
PSQL
.
Oid
->
m
BS
.
ByteString
readLargeObjectViaTempFile
oId
=
do
readLargeObjectViaTempFile
oId
=
do
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment