OpenAI recently released o1 and o1 pro in their 12 Days of OpenAI – Live updates, offering unlimited access through a $200 ChatGPT Pro subscription. With much speculation surrounding their capabilities, I wondered – Is this premium subscription worth the investment? To answer this, I pitted these two AI models against each other in six challenging tasks. This article explores their strengths, weaknesses, and overall performance. By the end, you’ll have a clear understanding of whether the $200 ChatGPT Pro subscription is the right choice for you or not!

👁 o1 vs o1 pro

Challenge 1. Solving a Zebra Problem

As the first test of o1 vs o1 pro, I am taking a zebra problem – hard level from this website. Let’s see which one cracks it better!

Prompt:

Solve this zebra problem:

👁 Image

o1 Resonse:

👁 Image

Putting this response as the solution, it turns out to be incorrect:

👁 Image

o1 pro Response:

👁 Image

Putting this response as the solution, it turns out to be correct:

👁 Image

Observation:

o1 pro took much more time than o1 to respond. o1 fails to solve the problem, whereas o1 pro succeds!

Verdict:

o1 ❌ | o1 pro ✅

Challenge 2: Find Difference

Prompt:

Find 3 differences in the two images:

👁 Image

o1 Response:

Item in hand: In the left image, she’s holding a spoon. In the right image, she’s holding an ice cream cone instead.
Hair accessory: In the left image, her hair is plain. In the right image, she is wearing a small pink bow in her hair.
Background object: In the left image, there is a green sewing machine on the shelf behind her. In the right image, the sewing machine is replaced by a simple green box (or frame).

Observation:

Only the second difference is correct which is the hair accessory missing, rest 2 are incorrect.

o1 pro Response:

The girl’s hairstyle accessory: In the right image, she has a pink bow in her hair, which is not present in the left image.
What she’s holding in her hand: In the left image, she’s holding a spoon. In the right image, she’s holding an ice cream cone instead.
The color of the middle scoop in the ice cream bowl: In the left image, the middle scoop is pink, while in the right image, that scoop has turned white.

Observation:

Only the first difference is correct. Rest 2 are incorrect.

Both o1 and o1 pro were not able to respond correctlty. However, o1 was faster in generating the response.

👁 Image

Verdict:

o1 ❌ | o1 pro ❌

Challenge 3: Solve a Math Problem

In this challenge, I will be giving a computing Indefinite Integrals problem to the models. Let’s see which one is able to solve it!

Prompt:

Solve this math problem:

👁 Image

o1 and o1 pro Response (Same):

👁 Image

Observation:

Both the models provided the correct answer but the o1 was much faster than o1 pro in finding the solution.

Verdict:

o1 ✅ | o1 pro ✅

Challenge 4: Analyzing Article

Prompt:

Read the article – https://www.analyticsvidhya.com/blog/2024/07/building-agentic-rag-systems-with-langgraph/ to understand the process of building a vector database for Wikipedia data. Summarize the key steps in a concise manner.

o1 Response:

👁 Image

o1 pro Response:

👁 Image

Observation:

The “o1 pro response” is closer to the actual implementation in the article. Here’s why:

The article provides a much more detailed, step-by-step implementation involving:

Using specific libraries like LangChain and OpenAI embeddings
Loading Wikipedia data from a specific archive
Using Chroma as the vector database
Implementing advanced RAG components like:
- Query rephrasing
- Document relevance grading
- Web search integration
- A complex LangGraph workflow

The o1 pro response captures more nuance by mentioning:

Specific embedding models (sentence-transformers)
Vector database options
Metadata storage
Testing retrieval
Integration with a RAG pipeline

By contrast, the initial “o1 response” is more generic and lacks the technical depth demonstrated in the article. So the o1 pro response is significantly closer to the article’s actual implementation.

Verdict:

o1 ❌ | o1 pro ✅

Challenge 5: Image Creation

Prompt:

Create an image of a cat.

o1 Response:

👁 Image

o 1 pro Response:

👁 Image

Observation:

Both o1 and o1 pro were not able to generated images indicating both the o1 versions do not support image generation. However, on giving the same prompt to GPT 4o, I got the response:

👁 Image

Hence, it is safe to say that only GPT 4o is beating both o1 and o1 pro in image generation!

Verdict:

o1 ❌ | o1 pro ❌

Challenge 6: Creating a Logical Flow Chart

Prompt:

Create a comprehensive flow chart illustrating the Reflection Pattern in Agentic AI.

o1 Response:

👁 Image

o1 pro Response:

👁 Image

Both provided incomplete flow chats, so I decided to update my prompt. Here’s my updated prompt:

New Prompt:

These are the steps involved in reflection patter –

Generate Initial Output
Self-Review/Critique the Output
Identify Errors, Gaps, or Improvement Areas
Develop Improvement Suggestions
Revise/Refine the Output
Repeat Steps 2-5 Until Satisfactory Result is Achieved
Create a new flow chart now

o1 Response:

👁 Image

o1 pro Response:

👁 Image

Observation:

Even though the content in both the responses is the same, o1 is definetly winning by providing an actual flow chart, whereas o1 pro only provided the correct content.

Verdict:

o1 ✅ | o1 pro ❌

Result Chart: o1 vs o1 pro

Challenge	Verdict
Zebra Problem	o1 pro succeeded, but was slower
Find Differences	Both models performed poorly
Math Problem	Both solved correctly, o1 was faster
Analyzing Article	o1 pro provided more depth
Image Creation	Neither could generate images (GPT 4o could)
Creating a Logical Flow Chart	o1 won by creating an actual flow chart

o1 pro seems to have a slight edge in terms of problem-solving depth and accuracy, particularly in complex tasks like solving the zebra problem and analyzing technical articles. However, o1 tends to be faster and performs well in simpler tasks.The verdict appears to be that o1 pro is marginally better, especially for more complex or technical challenges that require deeper understanding.

Also Read: Is the New o1 Model Better than GPT-4o?

End Note

While o1 pro shows promise in complex problem-solving, it’s important to consider your specific needs and budget. For basic to intermediate tasks, GPT-4o or other more affordable alternatives might suffice. If complex problem-solving is a priority and you’re willing to invest, o1 pro could be a valuable tool.

However, given that OpenAI is continually refining these models, it might be wise to wait for further updates before making a definitive decision. OpenAI is likely to add more benefits to the $200 ChatGPT Pro plan in the future.

What are your thoughts on this? Let me know in the comment section below.

Stay tuned to Analytics Vidhya Blog for more such awesome updates!

👁 Nitika Sharma

Nitika Sharma

Hello, I am Nitika, a tech-savvy Content Creator and Marketer. Creativity and learning new things come naturally to me. I have expertise in creating result-driven content strategies. I am well versed in SEO Management, Keyword Operations, Web Content Writing, Communication, Content Strategy, Editing, and Writing.

Beginner Generative AI Generative AI Application LLMs