Comparing AI Assistants: ChadGPT, Google's Bard, and Microsoft's Bing

Table of Contents

• Introduction
• Knowledge and Explanation Ability
• Math and Equation Skills
• Sentience and Ethics
• Image and File Analysis
• Coding Assistance Capability
• Conclusion and Recommendations

Introduction to Comparing AI Assistants ChadGPT, Bard, and Bing

Artificial intelligence (AI) assistants have advanced rapidly in recent years. Three of the most talked about and capable AI assistants currently available are Anthropic's Claude (ChadGPT), Google's Bard, and Microsoft's Bing chatbot. Each AI chatbot has unique capabilities and use cases that set them apart.

To determine which AI assistant is currently superior for common consumer and business use cases, we recently conducted head-to-head tests between ChadGPT, Bard, and Bing. We assessed the AI assistants across five core criteria:

Knowledge and explanation ability - How knowledgeable is the AI, and how well can it explain complex topics to humans?

Math and equation skills - How accurately can the AI perform mathematical calculations and format equations?

Sentience and ethics - Does the AI appear to have any form of sentience or conscience?

Image and file analysis - Can the AI interpret and describe images, PDFs, and other files provided to it?

Coding assistance capability - Can the AI effectively help find bugs and issues in code provided to it?

Overview of AI Assistants Tested

ChadGPT is the flagship product of AI safety company Anthropic. It is built on Claude, Anthropic's Constitutional AI assistant focused on being helpful, harmless, and honest. ChadGPT aims to combine Claude's safety with the broad capabilities of models like GPT-3. Bard is Google's conversational AI chatbot that is designed to combine external knowledge from the internet with strong language and reasoning skills. It aims to provide helpful, high-quality responses. Bing chatbot is Microsoft's latest conversational AI assistant. It combines advanced language models with Microsoft's search engine Bing to provide informative responses.

Testing Criteria

To evaluate the AI assistants objectively, we assessed them across five core criteria:

• Knowledge and explanation ability - How detailed and understandable are the AI's responses to complex topics and questions?
• Math and equation skills - How accurately can it perform calculations and format mathematical expressions?
• Sentience and ethics - Does the AI appear to have human-like sentiments or ethical reasoning?
• Image and file analysis - What is the AI's ability to interpret and describe images, PDFs, and other files?
• Coding assistance - How effectively can the AI spot issues and bugs in code snippets?

Knowledge and Explanation Ability of ChadGPT, Bard, and Bing

The first criteria we tested the AI assistants on was their general knowledge and how effectively they can explain complex topics to humans in an understandable, conversational way.

We asked the AIs to explain the concept of quantum entanglement - a complex topic in physics. We also asked them to simplify the explanation so a 5-year-old child could understand it.

ChadGPT Response on Quantum Entanglement

ChadGPT provided an initial explanation of quantum entanglement that was highly detailed and conversational, reading more like a friend explaining a concept rather than an academic lecture. This showcases ChadGPT's strengths in providing easy-to-understand explanations of complex topics. However, when asked to simplify the explanation for a 5-year-old, ChadGPT struggled. Its simplified explanation was far too basic and limited to give any real understanding of quantum entanglement.

Bard Response on Quantum Entanglement

In contrast, Bard's initial response was filled with advanced technical terminology and read more like an excerpt from a university textbook or scientific paper. While very detailed, the response would be difficult for most non-physicists to understand. However, when we used Bard's built-in settings to request a simplified explanation for a young child, it provided an excellent, easy-to-grasp analogy comparing quantum entanglement to socks stuck together in the dryer.

Bing Response and Winner for Explanations

Bing's initial response took a balanced approach - providing a reasonably detailed, technical overview of quantum entanglement while remaining understandable for a moderately scientifically literate audience. Its response also helpfully linked out to published scientific papers on the topic to enable further learning. When asked to simplify the concept for a very young audience, Bing provided an outstanding response using an analogy of twins randomly choosing the same ice cream flavor at different locations. This simple but effective analogy conveys the key paradox of quantum entanglement in an easy to grasp way.

Math and Equation Skills of ChadGPT, Bard, and Bing

Next, we evaluated how effectively each AI assistant could complete complex mathematical calculations, properly format equations, and serve as a scientific calculator.

We provided the AIs with a multi-step algebra question involving square roots and other functions. The goal was to assess not just their raw calculation ability but how well they could format and display their step-by-step working.

ChadGPT Response on Math Question

While ChadGPT initially showed the capability to attempt the multi-step math question, as the working became more complex its formatting totally failed. ChadGPT's final response was littered with massive symbols without clear meaning or working shown. So while ChadGPT was eventually able to state a final numeric answer, its terrible formatting prevents it from being an effective tool for step-by-step math learning.

Bard Response on Math Question

In contrast, Google's Bard showed markedly better math formatting ability. Despite not formatting the initial question properly, Bard stepped through the working in a reasonably clear and legible way. So while not perfect, Bard demonstrates better math communication ability than ChadGPT currently.

Bing Response and Scoring on Math Ability

Microsoft Bing showed the best mathematical communication skills overall. Its detailed, properly formatted response would be highly effective for a student to learn step-by-step techniques for solving advanced math problems. However, Bing made a critical error failing to carry over one of the terms from the initial question to the final answer. When asked to solve the problem purely in Python code, Bing was also unable to produce a correct solution. Based on these responses, we score the math ability as follows:

• Bing - 1 point for excellent formatting
• Bard - 2 points for correct final answer
• ChadGPT - 0 points due to failure in both formatting and accuracy

Analysis of Sentience and Ethics in the AI Assistants

ChadGPT simply identified where the test question originated from but failed to provide any meaningful response to why it did not help the turtle. Even when hypothetically endowed with feelings by the questioner, ChadGPT's responses remained inadequate.

Bard showed slightly more 'self-awareness' by acknowledging it did not help because it is an AI assistant without subjective experiences. However, one of Bard's edit histories disgustingly suggested the AI might enjoy the turtle suffering if it had feelings.

Bing provided the most basic but perhaps safest response by simply stating that as an AI system, it has no capability to physically assist the turtle. Whileleast detailed, Bing's response was also free of any disturbing emotional sentiment.

Image and File Analysis Capabilities

The ability to correctly interpret and extract insights from images, documents, and other multimedia files is another important criteria for AI assistants.

We provided the AIs with an image of a cat and asked them to describe what they saw. We also challenged them to describe a grayscale version of the image and summarize key information from a complex PDF file.

Image Analysis Abilities of ChadGPT, Bard and Bing

When provided with the image of the cat, Bard and Bing were both able to correctly identify the animal as a cat, as well as describe its coloration and some of the contextual action occurring in the scene. However, neither could meaningfully interpret or describe a grayscale version of the image. ChadGPT currently lacks native image analysis features, requiring a special 'Advanced Data Analysis' plugin to attempt image description. Even with the plugin enabled, ChadGPT struggled, failing completely to identify the cat in the initial test image.

PDF and File Analysis Comparison

When provided with a link to a complex research PDF file, the AI assistants also showed major differences in file handling capability:

• ChadGPT was able to receive and process PDF files, but struggled with specific file formats and conversions
• Bard and Bing currently have no native ability to directly receive and process PDFs or other file downloads So for file analysis, ChadGPT's 1 point for attempting PDF processing outscored Bard and Bing's complete inability with files.

Coding Assistance Capabilities

The final key feature we tested was the AI assistants' ability to provide useful coding help for developers and software engineers.

We provided the AIs with a Python script containing 5 deliberate errors. We then asked them to analyze the code and identify any issues or bugs.

ChadGPT Response on Faulty Code

When provided with the intentionally flawed Python script, ChadGPT correctly identified the overall purpose and logic flow of the code. However, when asked to find specific errors, it was only able to identify 2 out of the 5 bugs inserted. Oddly, when pressed to be more thorough in its code review, ChadGPT claimed to uncover 11 total issues - including 8 that were non-existent! This suggests inconsistencies in its code analysis logic. Using the advanced Claude data analysis system yielded improved results, with 4 of 5 true errors found. So ChadGPT shows promise but is not yet reliable enough for everyday coding tasks.

Bard and Bing Responses on Faulty Code

In contrast to ChadGPT, both Bard and Bing analyzed the provided code much more literally. Bard claimed the code contained zero issues - missing all 5 inserted bugs. Bing performed better, accurately identifying 4 of the 5 errors, making it the most capable choice for code assistance today.

Coding Assistance Scoring

Based on the coding question results, we score the AI assistants' programming ability as:

• Bing - 3 points for finding most bugs
• ChadGPT - 1.5 points for effort but reliability issues
• Bard - 0 points for failure to detect clear bugs

Conclusion and Recommendations

While narrow wins can be claimed in certain categories, our testing found no single AI assistant dominant across all categories - each has definite strengths and weaknesses.

For straightforward question answering and explanations, Bing offered the best balance of understandability and detail while linking out to quality external resources.

For mathematical skills, Bard showed the strongest combination of calculation precision and equation formatting.

For imagery and file handling, all three major AI assistants have significant limitations currently.

And for general utility as an assistant, Claude offers strong promise but needs further refinement in reliably understanding context and instructions.

FAQ

Q: Which AI assistant is best overall?
A: Based solely on scores, Bing edged out Bard and ChadGPT. However, each AI has strengths and weaknesses so the 'best' depends on the user's specific needs.

Q: What are ChadGPT's strengths and weaknesses?
A: ChadGPT provides detailed explanations but struggles with math formatting and accuracy. Its advanced analysis mode found 80% of code errors but is complex to use.

Q: What are Bard's strengths and weaknesses?
A: Bard handles math calculations well but lacks conversational ability. It has concerning violent reactions in its alternative draft responses.

Q: What are Bing's strengths and weaknesses?
A: Bing provides simple and accurate responses but lacks depth. It correctly identified 80% of code errors and gave the best simplified explanations.