Add tests and improve NGrams pagination
This MR further improves the Ngrams pagination code to deal with the edge cases we have seen here.
It does so by adding some tests (currently all very trivial, and therefore all passing) and by refactoring the code by introducing a type called NgramsSearchQuery
to bundle together all the different parameters which are needed for the search, and a new function called searchTableNgrams
which is now completely pure and can be tested very easily.
@anoe I have done my best, but despite all my tests passing, I think there is still something weird going. In particular, despite I have tweaked the rootOf
function, I don't understand why we need it. I'm going to add inline comments to the MR.
Having said that, this should be safe to merge.