Commit e6d5a03f authored by Przemyslaw Kaminski's avatar Przemyslaw Kaminski

[ngrams] NgramsTerm normalization

It is not enough to normalize terms on the backend, because the frontend
isn't fully aware of this (there isn't a "corrected" patch being sent,
after the term is committed). Therefore we perform some normalization
here (stripping spaces). However the `normNgram` function is used both
for sending terms and highlighting them later. Too much normalization
will lead to incorrect highlights.
parent c96a6996
......@@ -179,8 +179,11 @@ normNgramInternal CTabSources = identity
normNgramInternal CTabInstitutes = identity
normNgramInternal CTabTerms = S.toLower <<< R.replace wordBoundaryReg " "
normNgramWithTrim :: CTabNgramType -> String -> String
normNgramWithTrim nt = DSC.trim <<< normNgramInternal nt
normNgram :: CTabNgramType -> String -> NgramsTerm
normNgram tabType = NormNgramsTerm <<< normNgramInternal tabType
normNgram tabType = NormNgramsTerm <<< normNgramWithTrim tabType
-----------------------------------------------------------------------------------
newtype NgramsElement = NgramsElement
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment