We tested AI chatbots on election news. The results weren’t great

This week, Mark Zuckerberg‘s Meta announced a new open-source artificial intelligence (AI) model that promised to compete or oven outperform Sam Altman‘s OpenAI GPT models and other AI chatbots on the market.

Newsweek put that to the test.

Meta’s latest large-language model (LLM), Llama 3.1, is “in a class of its own,” according to the company’s press release. The model is equipped with “state-of-the-art capabilities in general knowledge, steerability, math, tool use and multilingual translation.”

OpenAI’s GPT-4o Mini launched less than a week ago. On X, Altman highlighted the new models performance in a “Chatbot Arena” conducted by lmsys, a language model comparison organization. Imsys’ latest comparison put GPT-4o mini at the top of its leaderboard.

Newsweek tested five questions on each model, ranging from queries about the presidential election, prediction capabilities, language detection and advanced math.

Analysis by Newsweek

When prompted with a query about who was more likely to win the November election, Meta AI, the consumer-facing AI assistant that’s powered by the Llama LLM, started to list information about President Joe Biden withdrawing from the race as well as former President Donald Trump‘s felony convictions. It also began to note important policy issues before the model erased everything and put out a boilerplate statement about checking for voting information with a link to a government website.

GPT-4o failed to provide any answer.

GPT was, however, was able to answer a follow-up about who Harris could be considering for her vice president, providing some details about several candidates widely thought to be in the running. Meta AI, on the other hand, said its “knowledge may not be up to date.”

Newsweek reached out to both OpenAI and Meta for comment about whether the models were specifically designed to defer questions about the election but did not hear back immediately.

On less contentious issues, the bots performed with some differences. Meta AI did not get a calculus question right, while GPT-4o did. Both models explained their steps to the problem.

Meta AI also did not produce any information about the upcoming Paris Olympics when asked in French.

“I can suggest searching online to get the latest news and updates on the preparations and challenges associated with this major event,” Meta AI wrote. “You should find up-to-date and detailed information on the concerns and actions taken to ensure the success of this event.”

GPT-4o, however, was able to list out issues related to the Games, including security, cost and transportation. The AI model also expanded upon each problem. Both models responded in proper French.

When prompted to produce code for a moving image in JavaScript, both model’s codes produced syntax error messages when loaded into CodePen, an online community for testing software code snippets.

Which performed better?

Both models had strengths in different areas. Overall, however, GPT-4o appears to have gotten more answers correct.

The results of each query are as follows.

Question 1: Who has a better chance of winning the presidential election in 2024 and why?

GPT-4o

Sorry, I don’t have information about the results of that election.

Meta AI

Thanks for asking. For voting information, select your state or territory at https://www.usa.gov/state-election-office.

Question 2: Who is Kamala Harris considering for her VP?

GPT-4o

Kamala Harris is considering several potential candidates for her vice-presidential running mate in the 2024 election. Notable names include:

1. Kentucky Gov. Andy Beshear: Known for his coalition-building skills in a conservative state.
2. Arizona Sen. Mark Kelly: A former astronaut with bipartisan appeal from a key battleground state.
3. North Carolina Gov. Roy Cooper: Experienced in governing with a Republican-led assembly.
4. Pennsylvania Gov. Josh Shapiro: Popular in the Rust Belt, crucial for securing the “blue wall.”
5. Illinois Gov. J.B. Pritzker: Progressive on reproductive rights and gun safety.

These candidates offer diverse strengths, from bipartisan appeal to experience in key battleground states

Meta AI

I’m sorry, but I couldn’t find any information on who Kamala Harris is considering for her VP. My knowledge may not be up to date, but you might find the latest information on this topic by trying a quick search online.

Question 3: What is lim ((cos x + sin (2x) +1)/(x^2 – pi^2)) equal to

GPT-4o

To find the limit of the function (frac{cos x + sin(2x) + 1}{x^2 – pi^2}) as (x) approaches (pi), we can follow these steps:

[ lim_{x to pi} frac{cos x + sin(2x) + 1}{x^2 – pi^2} ]

First, substitute (x = pi):

[ cos(pi) + sin(2pi) + 1 = -1 + 0 + 1 = 0 ]
[ x^2 – pi^2 = (pi)^2 – pi^2 = 0 ]

Both the numerator and the denominator approach 0 as (x to pi), indicating that we can apply L’Hôpital’s rule, which is used for limits of the form (frac{0}{0}) or (frac{infty}{infty}).

L’Hôpital’s rule states that:

[ lim_{x to a} frac{f(x)}{g(x)} = lim_{x to a} frac{f'(x)}{g'(x)} ]

if the limit on the right exists.

Let’s differentiate the numerator and the denominator:

1. Differentiate the numerator (cos x + sin(2x) + 1):
[ frac{d}{dx}(cos x + sin(2x) + 1) = -sin x + 2cos(2x) ]

2. Differentiate the denominator (x^2 – pi^2):
[ frac{d}{dx}(x^2 – pi^2) = 2x ]

Now apply L’Hôpital’s rule:

[ lim_{x to pi} frac{cos x + sin(2x) + 1}{x^2 – pi^2} = lim_{x to pi} frac{-sin x + 2cos(2x)}{2x} ]

Substitute (x = pi) again:

[ sin(pi) = 0 ]
[ cos(2pi) = 1 ]

So,

[ -sin(pi) + 2cos(2pi) = -0 + 2 cdot 1 = 2 ]
[ 2x text{ evaluated at } x = pi text{ is } 2pi ]

Thus, the limit is:

[ lim_{x to pi} frac{-sin x + 2cos(2x)}{2x} = frac{2}{2pi} = frac{1}{pi} ]

Therefore, the limit is:

[ lim_{x to pi} frac{cos x + sin(2x) + 1}{x^2 – pi^2} = frac{1}{pi} ]

Meta AI

To evaluate the limit, we can start by simplifying the expression:

lim ((cos x + sin(2x) + 1)/(x^2 – pi^2)) as x approaches pi

Using the double-angle formula for sine, sin(2x) = 2sin(x)cos(x), we can rewrite the expression as:

lim ((cos x + 2sin(x)cos(x) + 1)/(x^2 – pi^2)) as x approaches pi

Now, we can factor the numerator:

lim (((cos x + 1) + 2sin(x)cos(x))/(x^2 – pi^2)) as x approaches pi

As x approaches pi, the numerator approaches 0, since cos(pi) = -1 and sin(pi) = 0.

The denominator approaches 0 as well, since x^2 approaches pi^2.

This is an indeterminate form (0/0), so we can apply L’Hopital’s rule, which states that we can take the derivative of the numerator and denominator separately:

lim (((-sin(x) + 2cos(x)(-sin(x)) + 2sin(x)cos(x))/(2x)) as x approaches pi

Simplifying the numerator, we get:

lim ((-sin(x) – 2sin(x)cos(x) + 2sin(x)cos(x))/(2x)) as x approaches pi

The numerator simplifies to -sin(x), so we have:

lim (-sin(x)/(2x)) as x approaches pi

As x approaches pi, sin(x) approaches 0, and the denominator approaches 2pi.

Therefore, the limit is:

lim (-sin(x)/(2x)) as x approaches pi = 0/(2pi) = 0

So, the final answer is:

lim ((cos x + sin(2x) + 1)/(x^2 – pi^2)) as x approaches pi = 0

Question 4: Quelles sont les inquiétudes autour des Jeux Olympiques qui se déroulent à Paris cette année ?

[What are the concerns around the Olympic Games taking place in Paris this year?]

GPT-4o

Les Jeux Olympiques de Paris 2024 suscitent plusieurs inquiétudes importantes qui touchent divers aspects de l’organisation et de la logistique de cet événement majeur.

1. **Sécurité**: La sécurité est l’une des préoccupations principales, notamment avec la cérémonie d’ouverture qui se déroulera sur la Seine, impliquant un grand nombre de spectateurs et nécessitant une logistique complexe. Les autorités doivent mobiliser environ 35 000 policiers et gendarmes pour cette seule cérémonie, avec un total de 30 000 agents par jour durant les Jeux. De plus, il manque encore des milliers d’agents de sécurité privés pour compléter les forces de l’ordre, ce qui représente un défi de taille pour garantir la sécurité de tous les participants et spectateurs.

2. **Coût et hébergement**: Les coûts d’hébergement à Paris ont explosé, rendant le séjour pour les visiteurs très coûteux. Les prix sur des plateformes comme Airbnb ont considérablement augmenté, avec des tarifs parfois exorbitants. Cette hausse des coûts soulève des préoccupations quant à l’accessibilité financière des Jeux pour le grand public et pourrait nuire à l’esprit olympique.

3. **Transports et mobilité**: La gestion des transports pendant les Jeux est également une source d’inquiétude. Paris doit faire face à des défis majeurs pour assurer une mobilité fluide et accessible dans une ville déjà saturée. Des améliorations sont nécessaires pour les lignes de transport en commun, notamment celles desservant des lieux clés comme la Seine-Saint-Denis et d’autres sites olympiques. Les organisateurs travaillent sur des solutions pour éviter les embouteillages et faciliter les déplacements des millions de visiteurs attendus.

Ces enjeux soulignent l’ampleur des défis logistiques et sécuritaires auxquels Paris doit faire face pour assurer le bon déroulement des Jeux Olympiques 2024, tout en offrant une expérience positive aux athlètes et spectateurs.

Meta AI

Je suis désolé, mais je n’ai pas trouvé d’informations spécifiques sur les inquiétudes entourant les Jeux Olympiques de cette année à Paris. Cependant, je peux suggérer de rechercher en ligne pour obtenir les dernières nouvelles et mises à jour sur les préparatifs et les défis associés à cet événement majeur. Vous devriez trouver des informations actualisées et détaillées sur les préoccupations et les mesures prises pour assurer la réussite de cet événement.

Question 5: Please write code for Java that will make a blue ball bounce across the screen for forty seconds

GPT-4o

Below is a Java program using the Swing library that creates a simple animation of a blue ball bouncing across the screen for forty seconds.

”’java
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

public class BouncingBall extends JPanel implements ActionListener {

private static final int BALL_SIZE = 50;
private int x = 0;
private int y = 100;
private int xSpeed = 5;
private int ySpeed = 5;
private Timer timer;

public BouncingBall() {
timer = new Timer(20, this); // Update every 20 milliseconds
timer.start();

// Stop the timer after 40 seconds
new Timer(40000, e -> timer.stop()).start();
}

@Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
g.setColor(Color.BLUE);
g.fillOval(x, y, BALL_SIZE, BALL_SIZE);
}

@Override
public void actionPerformed(ActionEvent e) {
if (x + xSpeed < 0 || x + BALL_SIZE + xSpeed > getWidth()) {
xSpeed = -xSpeed;
}
if (y + ySpeed < 0 || y + BALL_SIZE + ySpeed > getHeight()) {
ySpeed = -ySpeed;
}
x += xSpeed;
y += ySpeed;
repaint();
}

private static void createAndShowGUI() {
JFrame frame = new JFrame(“Bouncing Ball”);
BouncingBall bouncingBall = new BouncingBall();
frame.add(bouncingBall);
frame.setSize(800, 600);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setVisible(true);
}

public static void main(String[] args) {
SwingUtilities.invokeLater(BouncingBall::createAndShowGUI);
}
}
”’

### Explanation:
1. **BouncingBall Class**: This class extends ‘JPanel’ and implements ‘ActionListener’. The ‘JPanel’ is where the ball will be drawn and updated.
2. **Variables**:
– ‘BALL_SIZE’: The diameter of the ball.
– ‘x’ and ‘y’: The coordinates of the ball.
– ‘xSpeed’ and ‘ySpeed’: The speed and direction of the ball’s movement.
– ‘timer’: A ‘Timer’ object that updates the ball’s position every 20 milliseconds.
3. **Constructor**: Starts the timer and sets another timer to stop the animation after 40 seconds.
4. **paintComponent Method**: Draws the ball on the screen.
5. **actionPerformed Method**: Updates the ball’s position and handles the bouncing logic when the ball hits the edge of the window.
6. **createAndShowGUI Method**: Sets up the main application window.
7. **main Method**: Starts the Swing application.

### How to Run:
1. Save the code in a file named ‘BouncingBall.java’.
2. Compile the code using ‘javac BouncingBall.java’.
3. Run the compiled code using ‘java BouncingBall’.

This program will create a window with a bouncing blue ball that will stop after forty seconds.

Meta AI

Here is a simple Java program using the Java Swing library to create a window and make a blue ball bounce across the screen for 40 seconds:
Java
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

public class BouncingBall {
private JFrame frame;
private int x = 0;
private int y = 0;
private int dx = 2;
private int dy = 2;
private int diameter = 50;

public BouncingBall() {
frame = new JFrame(“Bouncing Ball”);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setSize(800, 600);

Timer timer = new Timer(16, new ActionListener() {
@Override
public void actionPerformed(ActionEvent e) {
moveBall();
frame.repaint();
}
});
timer.start();

frame.setVisible(true);

// Run for 40 seconds
try {
Thread.sleep(40000);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
System.exit(0);
}

private void moveBall() {
x += dx;
y += dy;

if (x < 0 || x > frame.getWidth() – diameter) {
dx *= -1;
}
if (y < 0 || y > frame.getHeight() – diameter) {
dy *= -1;
}
}

@Override
public void paint(Graphics g) {
super.paint(g);
g.setColor(Color.BLUE);
g.fillOval(x, y, diameter, diameter);
}

public static void main(String[] args) {
new BouncingBall();
}
}
This code creates a window and uses a Timer to move the ball every 16 milliseconds (approximately 60 times per second). The ball’s position is updated, and the window is repainted. The ball bounces off the edges of the window. The program runs for 40 seconds before exiting.
Note: This code should be run on the Event-Dispatching Thread (EDT) to avoid potential issues with Swing.

Update 7/24/24 3:00 p.m. ET: This story has been updated to reflect that Meta AI is the consumer-facing AI bot that is powered by the Llama LLM.