Skip to content

haskell-gargantext
haskell-gargantext
Unverified Commit 75af58a2 authored by Nicolas Pouillard's avatar Nicolas Pouillard
Browse files
Options

Eleve fix toToken'

parent 12a74cfb
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
src/Gargantext/Text/Eleve.hs
View file @ 75af58a2
...@@ -362,7 +362,7 @@ data Order = Backward | Forward ...@@ -362,7 +362,7 @@ data Order = Backward | Forward
toToken' :: Order -> Int -> [[Text]] -> [[Token]] toToken' :: Order -> Int -> [[Text]] -> [[Token]]
toToken' o n input = L.concat toToken' o n input = L.concat
$ ( filter (/= [Terminal Stop]) $ ( filter (/= [Terminal (term o)])
. chunkAlongEleve (n + 2) . chunkAlongEleve (n + 2)
. (order o) . (order o)
) )
...@@ -371,6 +371,8 @@ toToken' o n input = L.concat ...@@ -371,6 +371,8 @@ toToken' o n input = L.concat
where where
order Forward = identity order Forward = identity
order Backward = reverse order Backward = reverse
term Forward = Stop
term Backward = Start
--------------------------------------------- ---------------------------------------------
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment