OpenAI has announced that it has developed an AI model called ‘ CriticGPT ‘ that detects errors in ChatGPT. CriticGPT is based on GPT-4, just like ChatGPT.
Finding GPT-4’s mistakes with GPT-4 | OpenAI
https://openai.com/index/finding-gpt4s-mistakes-with-gpt-4/
Chat AI such as ChatGPT allows you to generate code and create long sentences with few operations. However, the code and sentences generated by chat AI often contain errors, and there are reports of ‘using the code generated by ChatGPT as is, resulting in bugs and actual harm.’
OpenAI has developed a new AI model called ‘CriticGPT’ that detects and corrects errors in ChatGPT. CriticGPT is a model developed based on GPT-4, and its code error detection and correction capabilities have been enhanced by learning ‘code that includes manual mistakes’ and ‘sentences that correct code mistakes’.
Below is an example of how to use CriticGPT. CriticGPT points out that the startswith method is not appropriate for this purpose and offers an alternative to the code generated by ChatGPT.
The graph below compares the completeness of code criticism by humans (green), CriticGPT (orange), and humans and CriticGPT (pink). We can see that CriticGPT’s criticism is more complete than human criticism.
Below is a graph comparing the percentage of code criticism that contains false information (
hallucination ). We can see that the percentage of hallucinations is lowest when humans use CriticGPT.
OpenAI said, ‘We need better tools to tune increasingly complex AI systems,’ and intends to continue developing tools to tune AI outputs such as CriticGPT.
Source: A new GPT-4-based model called ‘CriticGPT’ will be developed to find mistakes made by
