Issue #473 fixed the code for insertMasterDocs such that we can now generate the ngrams before we try to persist the documents into the database. However, the ngrams extractions for such documents might fail, so could the insertion of a single document.

In the current state of play, what will happen for the former is that we would skip ngrams extraction for that single document, defaulting to empty (which is defensible behaviour) but in case of the latter the whole transaction would rollback, meaning that a single failure in inserting a document would result in the whole batch to be skipped.

Even though inserting documents should be a relatively safe operation, it would be better if we could improve our error handling strategy such that errors are reported upstream without aborting the whole lot. We could reuse the "warning" API we added a while ago to report documents which failed insertion.