Training AI for When Humans Will Use It
Costly AI can be used to make a prediction. You can try to verify that prediction at some cost. You can then try to guess the true state. If you guess right, you earn 1. If you guess wrong, you lose L. If you do nothing, you get 0. An AI is a prediction machine that makes some prediction with probability q, and makes a correct prediction with probability a≤q. What will the human do given an AI prediction? And how should we therefore train the AI?
What will the human do?
What determines the AI balance of coverage and accuracy? It is chosen by model makers. Consider a training frontier which tells you all of the sets of coverage and accuracy you could in principle train an AI to achieve in some domain.
Now let's combine the human decision problem and the training frontier to see how to optimally train an AI.