- 22 May, 2025 1 commit
-
-
Przemyslaw Kaminski authored
This specifies where purescript-gargantext/dist directory is. This way we don't hardcode it in our code and don't force that purescript-gargantext is strictly under haskell-gargantext.
-
- 21 May, 2025 1 commit
-
-
Alfredo Di Napoli authored
Port (almost all) DB operations in GGTX to use the transaction API Closes #466 See merge request !408
-
- 19 May, 2025 1 commit
-
-
Alfredo Di Napoli authored
-
- 12 May, 2025 5 commits
-
-
Alfredo Di Napoli authored
The refactored DB API now has a separate building block to create an Opaleye query that counts the number of returned results; we do that via `countRows`, exactly like the previous version. However, I have discovered a small footgun in the Opaleye API -- if you have two `Select` statements both calling countRows in a chain, that will always yield a value of 1, because the inner `countRows` will give you the actual number of results by returning a single row with an integer inside (i.e. the count). However, the subsequent (outer) call to `countRows` will return the number of rows of the previous step .. which is always going to be one! The bug was that I had left somewhere the spurious `countRows` in the query which would return the number of documents needed for the TFICF field, triggering the bug (because then we had `it` ALWAYS equal to 1.0). In the new API, while we cannot prevent the bug at the type level we can easily do an audit by grepping for `countRows`, making sure we have exactly one instance, i.e. inside `mkOpaCountQuery`.
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
This gigantic commit ports the existing DB operations in GGTX to use the transactional API, meaning that we can now compose DB operations and they will all run in the same Postgres transaction using the same connection, which will eliminate those class of bugs where concurrent DB access might result in an inconsistent state. On top of that, we simplify some parts of the API, for which a summary is given below: 1. The `NodeStoryEnv` management has been greatly simplified; in the new API we don't need an external connection pool to be passed and we don't have to pass IO actions, we can just pass DB operations, therefore we can greatly simplify the API to just pass mostly pure values; 2. Due to the fact that our `DBTx` monad can't do arbitrary IO (which is a good thing) we cannot fire Central Exchange notifications immediately. Rather that happens now is that we collect the `CEMessage` to be sent and we fire them in the relevant concrete monad after we finished with the DB transaction. This means that in principle there would be a small delay between the DB operation taking place and the notification firing but in practice the latency should be negligible and bear in mind this is typically what we want: if we have a long DB Tx that triggers an error in the middle we don't want to be sending out CE messages prematurely if the overall operation didn't succeed! 3. There are still a few places in the codebase where we couldn't make things fully compositional with regards to the DBTx API, because we had Servant handlers which had DB operations mixed with other IO effectful computations (or other things like the notification from the `MonadJobStatus`). For now we are splitting these functions by manually running the partial DB operations, and while this is not ideal it can be fixed in subsequent merge requests. 4. The `WorkerEnv` doesn't use `IOException` as its `MonadError` anymore, as for consistency we can just use `BackendInternalError` by adding a `InternalWorkerError` data constructor accepting the `IOException` triggered by the Worker monad. More testing is needed, with particular attention to performance (regression) but this should hopefully offer a decent baseline.
-
- 05 May, 2025 1 commit
-
-
Alfredo Di Napoli authored
Transaction API for DB Transactions See merge request !406
-
- 28 Apr, 2025 5 commits
-
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
.. before we start adding the transactional tests (and testing the rollback).
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
- 24 Apr, 2025 3 commits
-
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
- 22 Apr, 2025 1 commit
-
-
Alfredo Di Napoli authored
Use Async.race in serverCLI start-all to fix initialisation bug See merge request !404
-
- 14 Apr, 2025 3 commits
-
-
Alfredo Di Napoli authored
It's a simple DB query, it shouldn't need any extra effect.
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
This commit uses the `race` function from the async library to fix a bug where exceptions raised from the server async wouldn't be caught by the top-level code. The bug was stemming from the fact that `runAllWorkers` is **blocking**, despite its usage of `forConcurrently_`. Therefore we were never actually running the `wait` function below, but rather we were hanging waiting on the result of the first function. As a result the server could die but the workers could keep the main thread alive, causing the bug we just saw as part of #463.
-
- 11 Apr, 2025 1 commit
-
-
Alfredo Di Napoli authored
Proper incremental TSV parser See merge request !402
-
- 09 Apr, 2025 2 commits
-
-
Przemyslaw Kaminski authored
README: add info about system postgresql configuration See merge request !403
-
Przemyslaw Kaminski authored
-
- 07 Apr, 2025 8 commits
-
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
This commit introduces/improves the `parseTvsWithDiagnostics` function to parse the input TSV incrementally, collecting errors as we go, and eventually reporting them upstream via the newly added `emitTsvParseWarning` function.
-
Alfredo Di Napoli authored
-
Alfredo Di Napoli authored
This will pave the way for a more thorough refactoring to be able to emit diagnostics after parsing.
-
Alfredo Di Napoli authored
-
- 05 Apr, 2025 1 commit
-
-
Przemyslaw Kaminski authored
Resolve "Import/export in SQLite format" See merge request !393
-
- 04 Apr, 2025 6 commits
-
-
Przemyslaw Kaminski authored
Also, no need for `liftBase` in `getCorpusSQLite`.
-
Przemyslaw Kaminski authored
-
Przemyslaw Kaminski authored
Resolve "Implement temporary file storage" See merge request !392
-
Przemyslaw Kaminski authored
-
Przemyslaw Kaminski authored
-
Przemyslaw Kaminski authored
-
- 27 Mar, 2025 1 commit
-
-
Alexandre Delanoë authored
-