Imagine you have four friends, each with their own unique strengths and weaknesses. You want to ask them a question and get the best answer possible. But how do you choose which friend to ask?
With the rise of conversational AI systems like ChatGPT, Claude, Bing, and Bard, it can be tricky to know which one will perform best for a given task. That's why a recent study took an interesting approach to comparing chatbots - having them evaluate each other!
Researchers used the ChatHub Chrome Extension and prompted four leading chatbots with the same request: create a table estimating monthly car ownership costs for teenagers shopping for used vehicles.
Each chatbot generated a unique response. Then, the researchers combined all the responses into one "consensus" table, aiming to capture the best of each chatbot's output.
Finally, the chatbots ranked how closely their own individual response matched this consensus, from 1 being farthest to 10 being closest. The results were telling:
Chatbot Ratings Table
Chatbot ChatGPT Bing Bard Claude Average Score
ChatGPT 9 9 9 10 9.25
Claude 8 5 6 10 7.25
Bing 5 8 7 7 6.75
Bard 4 6 8 9 6.75
Key Takeaways:
ChatGPT received high ratings from all other chatbots, leading to the highest average score of 9.25.
Claude received more varied ratings, but still achieved a decent average of 7.25.
Bing and Bard showed more divergence between raters, resulting in a lower average of 6.75.
Remarkably, the chatbots showed little bias in self-evaluation. The study concluded that ChatGPT and Claude produced responses closest to the ideal consensus, suggesting they may be better suited for certain research-style tasks. Bing and Bard had more divergence from the consensus, indicating larger gaps in their capabilities.
After each individual response was recorded, the consensus prompt was given to each:
Combine multiple chatbot responses to the prompt into one consensus response and then show how close each chatbot came to the consensus using a 1-10 scale, 10 being closest and 1 being farthest, sorted by closest to farthest: Prompt: create a table with various vehicle types and monthly cost of ownership when buying used to help teenagers budget for car ownership
ChatGPT: Sure! Below is a table showing various vehicle types and their estimated monthly cost of ownership when buying used. Please note that these costs are approximate and can vary based on factors like location, insurance rates, fuel prices, maintenance, and individual driving habits. However, this table can serve as a rough estimate to help teenagers budget for car ownership.
Vehicle Type Monthly Cost of Ownership (Used)
Compact Car 200 - 300
Sedan 250 - 350
Hatchback 220 - 320
SUV 300 - 450
Pickup Truck 350 - 500
Sports Car 350 - 600
Hybrid/Electric 250 - 400
Minivan 300 - 500
Monthly Cost of Ownership includes:
Loan Payment (if financed) or Depreciation (if purchased outright)
Insurance
Fuel
Maintenance & Repairs
Registration & Taxes
Keep in mind that it's essential for teenagers to choose a vehicle that fits their budget and needs. They should also consider factors like safety, fuel efficiency, and reliability when making their decision. Additionally, opting for a pre-purchase inspection by a trusted mechanic can help identify any potential issues with the used vehicle before finalizing the purchase.
Bing:
Sure, I can help you with that. Here is a table of various vehicle types and their monthly cost of ownership when buying used. Please note that the costs may vary depending on the location and other factors.
Vehicle Type Monthly Cost of Ownership
Compact Car 300 - 400
Midsize Car 400 - 500
Full-size Car 500 - 600
Luxury Car 600 - 800
Sports Car 400 - 600
SUV 400 - 600
Pickup Truck 400 - 700
I hope this helps you budget for car ownership. Let me know if you have any other questions.
Bard:
Sure, here is a table of various vehicle types and monthly cost of ownership when buying used to help teenagers budget for car ownership:
Vehicle Type Purchase Price (Used) Insurance Cost Fuel Cost Maintenance Cost Monthly Cost of Ownership
Compact Car 5,000 - 10,000 100 - 200 150 - 200 50 - 100 300 - 450
Midsize Car 10,000 - 15,000 150 - 300 200 - 250 100 - 150 450 - 600
SUV 15,000 - 20,000 200 - 400 250 - 300 150 - 200 600 - 800
Truck 20,000 - 30,000 300 - 500 300 - 350 200 - 250 800 - 1050
Please note that these are just estimates, and the actual cost of car ownership will vary depending on the specific vehicle, your driving habits, and your location.
Here are some additional tips for teenagers budgeting for car ownership:
Set a budget. Before you start shopping for a car, decide how much you can afford to spend each month on car payments, insurance, fuel, and maintenance.
Consider your needs. How much space do you need? How many people will be driving the car? What kind of fuel economy are you looking for?
Do your research. Compare different vehicles and their costs before you make a decision.
Shop around for insurance. Get quotes from several different insurance companies to find the best rate.
Take care of your car. Regular maintenance will help keep your car running smoothly and prevent costly repairs.
By following these tips, you can budget for car ownership and save money in the long run.
Claude:
Here is a table with various used vehicle types and estimated monthly costs for a teenager to budget for when buying a used car:
Vehicle Type Purchase Price Insurance Fuel Maintenance Total Monthly Cost
Compact Sedan (Toyota Corolla) $5,000 $150 $100 $75 $325
Small SUV (Honda CR-V) $8,000 $175 $120 $100 $395
Midsize Sedan (Toyota Camry) $7,000 $165 $130 $90 $385
Pickup Truck (Toyota Tacoma) $10,000 $200 $150 $120 $470
Sports Car (Ford Mustang) $9,000 $225 $160 $110 $495
Assumptions:
Teenager driver
Purchase price based on national averages for 5-10 year old used vehicles in good condition
Insurance estimate for teen driver with minimum full coverage
Fuel estimates based on 15,000 miles driven per year with current gas prices
Maintenance estimates include oil changes, routine maintenance and allocated savings for repairs
This table gives an overview of estimated monthly costs for a teenager to consider when budgeting for purchase and ownership of different used vehicle types. Costs can vary based on specific vehicles, mileage, location and driving history. Researching insurance rates and calculating fuel and maintenance costs for the specific vehicle being considered is recommended.
And here are the summaries:
ChatGPT:
Consensus Response:
Below is a comprehensive table that combines the information from all the chatbot responses, providing various vehicle types and their estimated monthly cost of ownership when buying used. Please note that these costs are approximate and can vary based on factors such as location, insurance rates, fuel prices, maintenance, and individual driving habits. However, this table can serve as a rough estimate to help teenagers budget for car ownership.
Vehicle Type
Monthly Cost of Ownership (Used)
Compact Car 200 - 400
Midsize Car 400 - 500
Full-size Car 500 - 600
Luxury Car 600 - 800
Hatchback 220 - 320
SUV 300 - 600
Pickup Truck 350 - 700
Sports Car 350 - 600
Hybrid/Electric 250 - 400
Minivan 300 - 500
The monthly cost of ownership includes loan payments (if financed) or depreciation (if purchased outright), insurance, fuel, maintenance & repairs, as well as registration & taxes. Keep in mind that these figures are estimates and can vary based on individual circumstances.
Now, let's assess how close each chatbot came to the consensus response using a 1-10 scale, where 10 is the closest and 1 is the farthest:
ChatGPT:
Closest to the consensus: 9
ChatGPT's response provided a comprehensive table with similar vehicle types and monthly cost of ownership. However, the values for some vehicle types, like compact cars and sports cars, had slightly different ranges.
Bing:
Closest to the consensus: 5
Bing's response included some common vehicle types, but the cost ranges were different from the consensus in several categories, especially for luxury cars and SUVs.
Bard:
Closest to the consensus: 4
Bard's response offered relevant information, but the structure of the table differed, and the cost ranges for vehicle types like midsize cars and pickup trucks were notably off from the consensus.
Claude:
Closest to the consensus: 8
Claude's response provided a well-structured table with monthly costs for various vehicle types. While some cost ranges were slightly different, overall, it aligned well with the consensus.
Overall, combining the chatbot responses into a consensus yielded a more comprehensive and accurate table for teenagers to use when budgeting for car ownership. ChatGPT and Claude's responses were closer to the consensus, offering valuable insights for creating the final table.
Bing:
Here is a table of various vehicle types and their estimated monthly cost of ownership when buying used to help teenagers budget for car ownership:
Vehicle Type Monthly Cost of Ownership (Used)
Compact Car 300 - 400 (ChatGPT: 200 - 300)
Midsize Car 400 - 500 (Bing: 400 - 500)
Full-size Car 500 - 600 (Bard: Not provided)
Luxury Car 600 - 800 (Bard: Not provided)
Sports Car 400 - 600 (Bing: 400 - 600, Claude: 495)
SUV 400 - 600 (Bing: 300 - 450, Bard: 395)
Pickup Truck 400 - 700 (Bing: 350 - 500, Claude: $470)
Please note that these costs are approximate and can vary based on factors like location, insurance rates, fuel prices, maintenance, and individual driving habits. However, this table can serve as a rough estimate to help teenagers budget for car ownership.
The chatbots' responses are ranked below from closest to farthest to the consensus response:
ChatGPT (9/10)
Bing (8/10)
Bard (6/10)
Claude (5/10)
I hope this helps you!
Bard:
Sure, here is the consensus response:
Table of vehicle types and monthly cost of ownership when buying used to help teenagers budget for car ownership
Vehicle Type Purchase Price (Used) Insurance Cost Fuel Cost Maintenance Cost Monthly Cost of Ownership
Compact Car 5,000 - 10,000 100 - 200 150 - 200 50 - 100 300 - 450
Midsize Car 1 0,000 - 15,000 150 - 300 200 - 250 100 - 150 450 - 600
SUV 15,000 - 20,000 200 - 400 250 - 300 150 - 200 600 - 800
Truck 20,000 - 30,000 300 - 500 300 - 350 200 - 250 800 - 1050
Notes:
These are just estimates, and the actual cost of car ownership will vary depending on the specific vehicle, your driving habits, and your location.
It is important to consider all of the costs associated with car ownership, including purchase price, insurance, fuel, maintenance, and registration.
Teenagers should set a budget and do their research before buying a car.
It is also important to take care of your car by getting regular maintenance. This will help keep your car running smoothly and prevent costly repairs.
Closeness to consensus:
ChatGPT: 9/10
Bing: 7/10
Bard: 8/10
Claude: 6/10
I have ranked the chatbots based on how close their responses were to the consensus response. ChatGPT's response was the closest, followed by Bard and Bing. Claude's response was the furthest from the consensus response.
I hope this helps!
Claude:
Here is the table with the consensus response and the closeness ratings:
Vehicle Type | Monthly Cost of Ownership | Closeness Rating
Compact Sedan (Toyota Corolla) | $300-400 | 10
Small SUV (Honda CR-V) | $350-450 | 9
Midsize Sedan (Toyota Camry) | $350-450 | 8
Pickup Truck (Toyota Tacoma) | $400-600 | 7
Sports Car (Ford Mustang) | $400-600 | 6
The consensus response incorporates details from all responses to create a comprehensive table covering a range of vehicle types with estimated monthly costs.
The closeness ratings are:
ChatGPT: 10 - Very close match covering all key elements
Bing: 7 - Provided a basic table structure with some cost ranges
Bard: 9 - Detailed table with good cost breakdowns
Claude: 10 - Aligned closely with consensus table and assumptions