The Wimmelbook Check In
A test I’ve come to rely on for evaluating the state of generative AI for images is asking different services to create a Wimmelbook style illustration. When you ask generative AI to create many images of humans at once, you can see very quickly identify artifacts and discontinuities. You have lots of hands that you can observe for extra digits, or weird poses.
Perhaps another experiment to gauge progress is to ask these services to create images of Mandelbrot fractals. This is where Dall E 3 fell down. It did produce a stunning image but it wasn’t really what I had in mind:
Gemini wouldn’t even try. After some coaxing, I got it to produce code to plot out a fractal and it was fairly seamless to run this in Colab, albeit not instantly obvious that this was possible. It also required correcting for errors in the first versions of its code.
While a Turning test looked at consciousness, we might consider mini tests that are specifically designed to gauge the advancement of generative AIs.
