• Przemyslaw Kaminski's avatar
    [worker] fix an unfortunate coincidence of various async issues · 406cd89e
    Przemyslaw Kaminski authored
    This described in this comment:
    #477 (comment 14458)
    
    I repaste, for history:
    
    - job timeout was 30 seconds only and this is a big zip file, so the job timed out in worker
    - however, this was recently added https://gitlab.iscpif.fr/gargantext/haskell-gargantext/blame/dev/src/Gargantext/Database/Action/Flow.hs#L490 and the timeout wasn't caught and the worker continued happily
    - the job finished normally (most probably)
    - the job was restarted, because default strategy for timeouts is to restart a job
    - for sending files, we use postgres large objects because it keeps our JSONs small
    - when the job finishes, it clears definitely the large object so that we don't leave large, unused blob data
    - however, that job was restarted and there was no more a large object to work on
    - you got some sql error, but that wasn't the root cause
    
    Solution is:
    - don't catch any exception, but be careful and handle `Timeout` or `KillWorkerSafely`
    - increase job timeout for file upload
    - change timeout strategy for file upload to `TSDelete`, i.e. don't retry that job anymore
    406cd89e
Name
Last commit
Last update
.gitlab/issue_templates Loading commit data...
bench Loading commit data...
bench-data/phylo Loading commit data...
bin Loading commit data...
devops Loading commit data...
docs Loading commit data...
ekg-assets Loading commit data...
nix Loading commit data...
src Loading commit data...
src-doctest Loading commit data...
test Loading commit data...
test-data Loading commit data...
.clippy.dhall Loading commit data...
.envrc Loading commit data...
.ghci Loading commit data...
.gitignore Loading commit data...
.gitlab-ci.yml Loading commit data...
.mailmap Loading commit data...
CHANGELOG.md Loading commit data...
CODE_OF_CONDUCT.md Loading commit data...
DEVELOPER_GUIDELINES.md Loading commit data...
LICENSE Loading commit data...
README.md Loading commit data...
cabal.project Loading commit data...
cabal.project.freeze Loading commit data...
cabal.project.local_toCopy Loading commit data...
gargantext-settings.toml_toModify Loading commit data...
gargantext.cabal Loading commit data...
gargantext.ini_toModify Loading commit data...
hie.yaml Loading commit data...
run Loading commit data...
shell.nix Loading commit data...
stack.yaml Loading commit data...
start Loading commit data...
weeder.toml Loading commit data...