For some predictions that AI comes up with, the results will seem counter intuitive. These include the many things that humans are bad at — saving money, eating well, or making other health decisions. For example, a counter intuitive suggestion might be that to lose weight, you should skip breakfast and eat late at night before going to bed. This goes against our common wisdom, hence the counter part.
Then there are things that AI might suggest that have no reference point for us to judge at all. They might include design of certain structures, suggestions on public policy, or load balancing of power systems.The suggestions might seem out of this world. How do we judge these?
In this respect, AI will act to us like an alien we’ve encountered. This alien will have advanced knowledge and technology and will offer it to us. However, we won’t be sure if this will be helpful or harmful to us. We need to think of ways of testing out these suggestions in a controlled manner. It’s important for our survival.