[README] info about worker config

parent 611fcb17
Pipeline #6976 passed with stages
in 55 minutes and 9 seconds
......@@ -402,13 +402,8 @@ allows for scaling.
To run the worker, follow these steps:
- start a PostgreSQL DB, usually the one with Gargantext DB is enough
- `"simple"` worker definition is in `gargantext-settings.toml`
- run worker: `cabal v2-run gargantext-cli -- worker --name simple`
When running the worker for the first time (or sending a job), best
attempt is made to ensure the DB exists (if not, we will try to create
it) and the `pgmq` schema is initialized. This allows for hassle-free
maintenance and easier onboarding.
- `"default"` worker definition is in `gargantext-settings.toml_toModify`
- run worker: `cabal v2-run gargantext-cli -- worker --name default`
The project that we base our worker is
[haskell-bee](https://gitlab.iscpif.fr/gargantext/haskell-bee/). It's
......@@ -416,6 +411,45 @@ a more generic framework for managing asynchronous workers, supporting
different brokers. Here, we decided to use `pgmq` because we already
have PostgreSQL deployed.
## Configuration
Edit your `gargantext-settings.toml` file and add this section:
```toml
[worker]
# After this number of seconds, the job will be available again.
# You can set timeout for each job individually and this is the
# preferred method over using defaultVt.
default_visibility_timeout = 1
# default delay before job is visible to the worker
default_delay = 0
[worker.database]
host = "127.0.0.1"
port = 5432
name = "pgmq"
user = "gargantua"
pass = PASSWORD_TO_CHANGE
[[worker.definitions]]
name = "default"
queue = "default"
```
When running the worker for the first time (or sending a job), best
attempt is made to ensure the DB exists (if not, we will try to create
it) and the `pgmq` schema is initialized. This allows for hassle-free
maintenance and easier onboarding.
The way this is done is that we use the gargantext DB credentials as a
pivot to create the worker DB. This works of course if both DBs are on
the same server (`host` and `port` for both definitions must agree)
and the garg DB user can create new DBs. If they're on different
server, it is just assumed that the `worker.database` from above
config exists. See `G.C.W.Broker` -> `initBrokerWithDBCreate` for more
details.
## Design
Thanks to the fact that we already use `Servant.Jobs` (which executes
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment