Error in corpus upload / construction are not reflected in the overall JobStatus
While working on the tests for #385 (closed), I have noticed something that is worth sharing. Observe the following result from /poll?limit=1
:
As you can see, there was an error, but that wasn't propagated in the final status
(or error code), which was happily returning a IsFinished
and a 200 OK.
While the frontend can report errors to the user just fine by looking at the events
payload, from a REST standpoint this is slightly weird; I can defend the choice of returning 200 OK (because after we polled the job status just fine, so is not like there was an error at that stage), but I think we should treat the job as failed and mark the job using the IsFailure
constructor for its status. Due to this, I was scratching my head on why my tests passed, but there were no imported documents: I have later discovered importing was failing but this was completely masked by that seemingly-successful JobLog
.
In particular, I think we might want to modify the implementation of markCompleted
to check for errors -- if we have any, we shouldn't report IsFinished
as its status.
This will help in testing as well, because the pollUntilFinished
helper does already take into account the fact a job is marked as IsFailure
, and can mark the relevant test as failed.