Commit 4ee9c0da authored by delanoe's avatar delanoe

Merge branch 'refactoring-rom' into refactoring-alex

parents e2094c28 bdbd4bdb
......@@ -27,7 +27,8 @@ def parse(corpus):
hyperdata = hyperdata,
)
session.add(document)
observed_languages[hyperdata["language_iso2"]] += 1
if "language_iso2" in hyperdata:
observed_languages[hyperdata["language_iso2"]] += 1
if documents_count % BATCH_PARSING_SIZE == 0:
corpus.status('parsing', progress=documents_count)
corpus.save_hyperdata()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment