@anoe asked me to review this library, originally written by @cgenie in the context of gargantext. I will try to focus on the general design aspect of it, to see if there is any area where we could streamline the design.

Some initial instructions / directives I was given, to kickstart the review:

• There is a HasBroker class with type families, which needs to be implemented to have a useable broker.
• There are some integration tests written already;
• There exist two implementations: a pgmq (postgres-based) and a Redis one;
• Another key component is the worker itself, i.e. really a process that connects to given broker at given queue and processes incoming messages. It also has tests;
• One can spawn a worker with run, which can be thought an infinite loop: fetch message, process it, remove it from queue, repeat;
• Some complexity lies in the error handling;
• The are "strategies" implemented: ArchiveStrategy for example is telling what to do when a job finished successfully, ErrorStrategy tells worker what to do when job ends with error, TimeoutStrategy tells worker what to do when job times out;
• Each job has some metadata attached to it, which describes strategies and other things to the worker, see this job wrapper;
• In order to run the tests, one can start pgmq via podman run --rm -p 5433:5432 -e POSTGRES_PASSWORD=postgres cgenie/pgmq:16-1.3.3.1 and redis via podman run --rm -it -p 6379:6379 redis:latest;
• Jobs don't currently store the result of their computation, but in principle they could.
• There are some known quirks, for example in Redis one wants to poll for next message and it works like this but for pgmq, they do have polling, but cgenie reported that in tests that it doesn't play nice with ghc threads so he implemented a crude version of his own.