Between 2020 and 2023 something changed. Machines stopped only analysing the world and started producing new text, pictures and code that looked as though a person had made them. It was exciting, and it also came with three lessons that are still useful today. Each one starts with a story.

Key takeaways

GPT-3's Guardian op-ed was stitched from several drafts, so the headline overstated what the model did alone.
OpenAI's claim that GPT-4 scored around the top 10% on a bar exam was later re-examined and found to be far lower.
The puffer-jacket Pope showed that seeing is no longer believing.

The generative big bang

2020: a robot writes an op-ed

The Guardian published an article titled "A robot wrote this entire article. Are you scared yet, human?" It was produced by GPT-3, which has 175 billion parameters, a rough measure of the model's size. The story is more complicated than the headline, though. TechTalks wrote a careful piece pointing out that the published article was edited together from several machine-generated drafts, and that the framing exaggerated what the model could do alone.

Lesson one: read how a demo was produced, not only what it shows. A polished result can hide a lot of human work in the background.

2021 and 2022: text becomes images

DALL-E and Midjourney turned prompts like "an armchair shaped like an avocado" into pictures. In 2022 a Midjourney-assisted piece took first prize in a digital art category at the Colorado State Fair, which started a long argument about what counts as art and who deserves the credit. Photographers, illustrators and designers suddenly had to ask how their work compared with something produced in seconds.

30 November 2022: ChatGPT

ChatGPT launched as a free chat box that wrote essays, code and poems. According to a UBS analyst note reported by Reuters and Euronews, it reached an estimated 100 million users in about two months, the fastest-growing consumer app on record at that point. You can read the whole growth story in From Zero to a Billion People.

Video: Andrej Karpathy: Intro to Large Language Models (a one-hour talk for a general audience)

If you want to understand what is actually happening inside a chat model, this talk is one of the best starting points. It explains training, fine-tuning and the main security risks in plain language.

March 2023: GPT-4 and the bar exam

OpenAI said GPT-4 passed a simulated bar exam and scored around the top 10% of test takers. That claim travelled everywhere. Later, MIT research re-checked the result and, as LiveScience summarised it, found the model did not do nearly as well as claimed. The re-analysis put it below the 70th percentile overall.

GPT-4 bar exam: claim versus re-check

Lesson two: verify the hype, even when it comes from the company that built the model. Companies have good reasons to show their products in the best light. Independent replication is how science, and good journalism, catches the gap.

March 2023: the Pope in a puffer jacket

A Midjourney image of Pope Francis in a white designer puffer coat went viral on Twitter. NBC News described it as one of the first wide-scale examples of misinformation from AI-generated images. Many people believed it. The picture was funny and harmless, but it showed how easily a convincing fake could travel across the internet in a weekend.

Lesson three: seeing stopped being believing. The same tools can produce a fake receipt, a fake screenshot or a fake politician. Later posts in this series cover what happens when it gets harder to tell what is real.

What these three lessons mean for you

Read the method. A demo is a story, not a measurement. Ask how many tries it took and who edited the output.
Check the claim. If an exam score or a benchmark matters to your decision, find the independent replication.
Doubt the picture. For anything that matters, look for the original source, a second angle, or a credible outlet that confirmed it.

A simple habit that works: before you share or act on something striking, spend sixty seconds searching for the original source. If you cannot find one, treat it as unconfirmed.

AI tools are strong at producing convincing output. Whether that output is correct, fair and safe to use is still your job. In the next post I look at how fast the pace of change became once the big bang had happened: two years where every month brought a headline.

Keep reading

See the complete AI Era series

Written by Faisal Adnan

Business Intelligence and AI mentor. I teach Data Analytics, Power BI, Microsoft Fabric, SQL, Python and applied AI, and write about what is changing and what it means for your career.

Then Machines Started to Create: GPT-3, ChatGPT, GPT-4 and the Pope in a Puffer Jacket