Elon Musk’s response to ChatGPT was to update it to make it better at math, coding, and more. Musk’s xAI has rolled out Grok-1.5 to early testers, with “improved functionality and reasoning capabilities” and the ability to handle longer contexts. The company claims it is now comparable to GPT-4, Gemini Pro 1.5 and Claude 3 Opus in multiple areas.
According to xAI’s data, Grok-1.5 appears to be a big improvement over Grok-1. On the MATH benchmark, it surged to 50.6%, more than double the previous score. In GSM8K (mathematical word problems) and HumanEval (coding), its accuracy also climbed to 90% and 74.1% respectively from the previous 62.9% and 63.2%. These numbers are not far off from Gemini Pro 1.5, GPT-4, and Claude 3 Opus—in fact, the HumanEval encoding score beat all competitors except the Claude 3 Opus.
It can also handle long contexts of up to 128K tokens within its context window, meaning it can incorporate data from more sources to understand the situation. “This increases Grok’s memory capacity by 16 times over previous context lengths, enabling the utilization of information in longer documents,” the company said.
However, xAI didn’t detail Grok’s progress in other areas where it may still be lagging behind (academic performance, multimodality, etc.). Grok-1.5’s status may not last long. According to OpenAI, ChatGPT 5 will be released sometime this summer, promising a set of features that “make it feel like you’re communicating with a human rather than a machine.”
Currently, Grok is only available to users at the Premium+ level on X (formerly known as Twitter), although Musk recently promised to open it up to regular Premium users of X. The company also recently open-sourced its Grok chatbot after Musk sued OpenAI and Sam Altman for allegedly abandoning its nonprofit mission.
3 Comments
Pingback: Elon Musk’s updated Grok AI claims to be better at coding, math – Tech Empire Solutions
Pingback: Elon Musk’s updated Grok AI claims to be better at coding, math – Paxton Willson
Pingback: Elon Musk’s updated Grok AI claims to be better at coding, math – Mary Ashley