1. 22 May, 2025 8 commits
  2. 21 May, 2025 1 commit
  3. 19 May, 2025 1 commit
  4. 12 May, 2025 5 commits
    • Alfredo Di Napoli's avatar
      Fix bug in selectCountDocs · 6ff05ee1
      Alfredo Di Napoli authored
      The refactored DB API now has a separate building block to create an Opaleye query that
      counts the number of returned results; we do that via `countRows`, exactly like the previous version.
      
      However, I have discovered a small footgun in the Opaleye API -- if you have
      two `Select` statements both calling countRows in a chain, that will always yield a value of 1,
      because the inner `countRows` will give you the actual number of results by returning
      a single row with an integer inside (i.e. the count).
      
      However, the subsequent (outer) call to `countRows` will return the number of rows
      of the previous step .. which is always going to be one!
      
      The bug was that I had left somewhere the spurious `countRows` in the query which
      would return the number of documents needed for the TFICF field, triggering the bug
      (because then we had `it` ALWAYS equal to 1.0).
      
      In the new API, while we cannot prevent the bug at the type level we can
      easily do an audit by grepping for `countRows`, making sure we have exactly one instance,
      i.e. inside `mkOpaCountQuery`.
      6ff05ee1
    • Alfredo Di Napoli's avatar
      47ed29c5
    • Alfredo Di Napoli's avatar
      Make CLI compile again · b0568522
      Alfredo Di Napoli authored
      b0568522
    • Alfredo Di Napoli's avatar
      Tests compile again · d4116e48
      Alfredo Di Napoli authored
      d4116e48
    • Alfredo Di Napoli's avatar
      Port DB operations to transactional API · 53512f89
      Alfredo Di Napoli authored
      This gigantic commit ports the existing DB operations in GGTX to use the
      transactional API, meaning that we can now compose DB operations and
      they will all run in the same Postgres transaction using the same
      connection, which will eliminate those class of bugs where concurrent DB
      access might result in an inconsistent state.
      
      On top of that, we simplify some parts of the API, for which a summary
      is given below:
      
      1. The `NodeStoryEnv` management has been greatly simplified; in the new
         API we don't need an external connection pool to be passed and we
         don't have to pass IO actions, we can just pass DB operations,
         therefore we can greatly simplify the API to just pass mostly pure
         values;
      
      2. Due to the fact that our `DBTx` monad can't do arbitrary IO (which is
         a good thing) we cannot fire Central Exchange notifications
         immediately. Rather that happens now is that we collect the
         `CEMessage` to be sent and we fire them in the relevant concrete
         monad after we finished with the DB transaction. This means that in
         principle there would be a small delay between the DB operation
         taking place and the notification firing but in practice the latency
         should be negligible and bear in mind this is typically what we want:
         if we have a long DB Tx that triggers an error in the middle we don't
         want to be sending out CE messages prematurely if the overall
         operation didn't succeed!
      
      3. There are still a few places in the codebase where we couldn't make
         things fully compositional with regards to the DBTx API, because we
         had Servant handlers which had DB operations mixed with other IO
         effectful computations (or other things like the notification from
         the `MonadJobStatus`). For now we are splitting these functions by
         manually running the partial DB operations, and while this is not
         ideal it can be fixed in subsequent merge requests.
      
      4. The `WorkerEnv` doesn't use `IOException` as its `MonadError`
         anymore, as for consistency we can just use `BackendInternalError` by
         adding a `InternalWorkerError` data constructor accepting the
         `IOException` triggered by the Worker monad.
      
      More testing is needed, with particular attention to performance
      (regression) but this should hopefully offer a decent baseline.
      53512f89
  5. 05 May, 2025 1 commit
  6. 28 Apr, 2025 5 commits
  7. 24 Apr, 2025 3 commits
  8. 22 Apr, 2025 1 commit
  9. 14 Apr, 2025 3 commits
  10. 11 Apr, 2025 1 commit
  11. 09 Apr, 2025 2 commits
  12. 07 Apr, 2025 8 commits
  13. 05 Apr, 2025 1 commit