One topic I am currently thinking about is calculating policies for Markov Decions Processes when we are uncertain about the reward function. I find this to be an interesting question because we have good tools for allowing domain experts to specify the dynamics of a decision process, yet specifying the reward function is not always easy or intuitive.
Here's the idea: let a domain expert specify some kind of an incomplete reward function and use the minimax regret criterion to measure how much one could regret choosing a particular policy given the unknown reward function. We can further use the minimax regret to compute questions to ask a designer to quickly eliminate the uncertainty in the reward function.
