AI expert ticketed by an AI system.

 From this article. 

https://www.jpost.com/business-and-innovation/tech-and-start-ups/article-787534

One way to double check a person that is in a double check loop like this to backstop an ai system is to send extra messages through where you know the answer, and see if the human classifies the message correctly. If you added one extra message every 10 you could track their accuracy over time.  Maybe they should only do a 2 hour shift a day on this kind of classification.  But we don't know unless we can track an error rate over time.