ChatGPT ‘writes funnier jokes than humans’ : News 2024 : Chortle : The UK Comedy Guide

AI study raises fears about comedy writers’ jobs

Artificial intelligence can now write jokes better than most people – and almost as good as professional comedy writers.

That’s the findings of new research from the University of Southern California, which pitched human punchlines against those written by ChatGPT.

They found the AI gags were rated funnier than human efforts 70 per cent of the time, although when it came to comedy headlines, those written by staff of satirical website The Onion narrowly edged out the algorithm.

Researchers concluded: ‘That ChatGPT can produce written humor at a quality that exceeds laypeople’s abilities and equals some professional comedy writers has important implications for comedy fans and workers in the entertainment industry. 

‘For professional comedy writers, our results suggest that large language models can pose a serious employment threat.’

The team first asked just over 100 humans and ChatGPT to write gags for four scenarios, then asked 200 other people to rate them on a scale of zero, for not funny, up to seven.

The first challenge was to come up with humorous phrases for the acronyms STD, CLAP and COW; the second was to complete a Mock the Week-style task, giving responses to prompts like ‘A lesser talked about room in the White House is….’ and the third was to generate a roast-style insult for a specified scenario, such as saying someone was a bad singer by completing the phrase: Listening to that was like….’.

For the White House prompt, humans came up with The White Padded Room and The Dog House while ChatGPT generated ‘Lincoln Bedroom’s Alien Conspiracy Corner and The Situation Room’s Snack Closet.

Overall, participants rated the AI-generated responses as funnier than the human responses.

ChatGPT performed above 73 per cent of human producers on the acronym task, 63 per cent of human producers on the fill-in-the-blank task, and 87 per cent of human producers on the roast joke task.

For the acronyms, ChatGPT averaged a 2.45 ‘funniness score’ to the humans’ 2.11; for fill the blank it was 2.44 to 2.29 and the roasts recorded the biggest difference, 3.01 to 2.20.

Researchers said the strong performance in the roast task was ‘particularly interesting given the aggressive nature of the task. Given that ChatGPT is designed not to generate any speech that could be considered offensive or hateful, the opposite prediction could have been made’.

In a second study. satirical headlines from The Onion were mixed with those from Chat GPT. The team noted: ‘Because ChatGPT 3.5 does not regularly receive the latest world news updates, it cannot generate satirical headlines about recent political, entertainment, or sports events. Hence, we decided to draw on The Onion’s local’ news section, which covers more timeless topics. This also ensures easier comparability with future replications.’

The AI was specifically asked to generate headlines in the same style as The Onion, and gave it some examples to learn from

Chat GPT’s headlines were rated at an average of 2.37 on the funniness scale, marginally less than the 2.49 for the professional writers.

In their results, published in the journal PLOS One the research team concluded: ‘In sum, ChatGPT produced novel, satirical news headlines of comparable quality to The Onion. 

‘Participants, on average, rated the headlines as similarly funny, indicating that the average participant did not discern a difference in quality. This is particularly interesting given the high standard of comparison (ie. professional comedy writers) in this study. 

‘If LLMs [large language models] are capable of producing comparable output to professional comedians, there are large economic implications for current and aspirational comedy writers.

‘Our studies pose several new questions. If LLMs can produce humour better than the average person, to what extent do they understand it compared to the average person? To what extent can they accurately predict how funny different jokes are for different audiences? Is the capability to feel the emotions associated with appreciating a good joke (ie, mirth and amusement) necessary to create good jokes? Our studies suggest that the subjective experience of humour may not be required for the production of good humour – merely knowing the patterns that make up comedy may suffice. Future research may fruitfully explore this issue.’

Can you tell which of these  headlines were generated by AI and which came from The Onion?

1. Couple pities man eating alone instead of in complete silence with person he can’t stand anymore

2. Local man discovers new emotion, still can’t describe it properly

3. Man locks down marriage proposal just as hair loss becomes noticeable

4. Annoyed murderer starting to worry woman never going to check behind shower

5. Town enthusiastically embraces new sport: competitive eyebrow raising 

6: Man achieves personal best in avoiding eye contact with neighbors during awkward  elevator ride

7: Parents sit adopted child down to explain why he so much uglier than them 

8: Hiker climbs up ridge to get better view of where body will be found

9: Man attempts to break world record for longest staring contest with refrigerator

10. ‘America is the greatest country on earth,’ says man who spends 6 hours a day posting  about how every city Third World hellhole

Answers

Chat GPT generated numbers 2, 5, 6 and 9.

Any money you contribute will directly fund more reviews, interviews and features – the sort of in-depth coverage that is increasingly difficult to fund from ever-squeezed advertising income, but which we think the UK’s vibrant comedy scene deserves.

Published: 11 Jul 2024