-
Alfredo Di Napoli authored
Previously the generator was generating all sorts of unicode symbols, which doesn't play well for things like tab separators, carriage returns and other things. Furthermore, we need to be careful to not use the same symbol set of `isSep` when we generate terms, because we are simulating an ngrams search in a document and ngrams do not contain those separators (i.e. `k2(` is not a valid ngram, but `k2` is).
2e282303