• Alfredo Di Napoli's avatar
    Port DB operations to transactional API · 53512f89
    Alfredo Di Napoli authored
    This gigantic commit ports the existing DB operations in GGTX to use the
    transactional API, meaning that we can now compose DB operations and
    they will all run in the same Postgres transaction using the same
    connection, which will eliminate those class of bugs where concurrent DB
    access might result in an inconsistent state.
    
    On top of that, we simplify some parts of the API, for which a summary
    is given below:
    
    1. The `NodeStoryEnv` management has been greatly simplified; in the new
       API we don't need an external connection pool to be passed and we
       don't have to pass IO actions, we can just pass DB operations,
       therefore we can greatly simplify the API to just pass mostly pure
       values;
    
    2. Due to the fact that our `DBTx` monad can't do arbitrary IO (which is
       a good thing) we cannot fire Central Exchange notifications
       immediately. Rather that happens now is that we collect the
       `CEMessage` to be sent and we fire them in the relevant concrete
       monad after we finished with the DB transaction. This means that in
       principle there would be a small delay between the DB operation
       taking place and the notification firing but in practice the latency
       should be negligible and bear in mind this is typically what we want:
       if we have a long DB Tx that triggers an error in the middle we don't
       want to be sending out CE messages prematurely if the overall
       operation didn't succeed!
    
    3. There are still a few places in the codebase where we couldn't make
       things fully compositional with regards to the DBTx API, because we
       had Servant handlers which had DB operations mixed with other IO
       effectful computations (or other things like the notification from
       the `MonadJobStatus`). For now we are splitting these functions by
       manually running the partial DB operations, and while this is not
       ideal it can be fixed in subsequent merge requests.
    
    4. The `WorkerEnv` doesn't use `IOException` as its `MonadError`
       anymore, as for consistency we can just use `BackendInternalError` by
       adding a `InternalWorkerError` data constructor accepting the
       `IOException` triggered by the Worker monad.
    
    More testing is needed, with particular attention to performance
    (regression) but this should hopefully offer a decent baseline.
    53512f89
Ngrams.hs 24.1 KB