Trust But Verify
A few days ago, I tasked ChatGPT to provide me with some code that could help me find the probability of collecting the remaining items in a set of collectibles, solving probabilities for the Coupon Problem. I ended up working with Python in Google Colab to run a Monte Carlo Simulation. It was the first time I had run such a simulation since my first year in engineering. At that time, I had used C++ and DOS.
At first I blindly following what the generative AI had given me. Hey, it works! No errors when running code for the first time! Plugging in numbers worked. Yay!
However, as I started to look at the probabilities it was generating, the likelihoods seemed high. Hmm… the chance of receiving at one collectible was 1 out of 24. As I collect more, nothing changes with the fact that it’s 1 out of 24. However, in the code, the 24 never factored in. That was odd!
I asked ChatGPT what gave and sure enough, it apologized. Oh, sorry! I actually completely messed up. Here is the real deal. What?!
If I had continued to blindly follow what the service had given me, I’d have made some seriously wrong assumptions on completing the set. Instead, it was having done the hard work of having to retake statistics in my second year of engineering because I had done so poorly and was on probation. It might have been the other courses I took later on about business statistic, or my interest in Freakonomics and other economics courses. It was some exposure to methods of calculating statistics that made me question the result.
We won’t be able to offload all of our thinking to the machines or we’ll have no idea if they’re wrong or not. Trust and learn enough to verify,
