Commit 8718b320 authored by Romain Loth's avatar Romain Loth

fix language count when undefined by upstream fileparser

parent b3108107
......@@ -27,7 +27,8 @@ def parse(corpus):
hyperdata = hyperdata,
)
session.add(document)
observed_languages[hyperdata["language_iso2"]] += 1
if "language_iso2" in hyperdata:
observed_languages[hyperdata["language_iso2"]] += 1
if documents_count % BATCH_PARSING_SIZE == 0:
corpus.status('parsing', progress=documents_count)
corpus.save_hyperdata()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment