The rise of AI 'reasoning' models is making benchmarking more expensive

The rise of AI ‘reasoning’ models is making benchmarking more expensive

AI labs like OpenAI claim that their so-called “reasoning” AI models, which can “think” through problems step by step, are more capable than their non-reasoning counterparts in specific domains, such as physics. But while this generally appears to be the case, reasoning models are also much more expensive to benchmark, making it difficult to independently verify these claims. According to data from Artificial Analysis, a third-party AI testing o…

The rise of AI 'reasoning' models is making benchmarking more expensive

The rise of AI 'reasoning' models is making benchmarking more expensive, data from Artificial Analysis shows.

AI reasoning models cost more to benchmark, making it harder to independently verify claims; Artificial Analysis says evaluating OpenAI's o1 costs $2,767.05

Top news and commentary for technology's leaders, from all around the web.

What are the Reasoning Models, After All? And Where to Use them?

If you are a tech enthusiast and follow the LLM world, you must have heard the term “Reasoning Models”! Especially in relation to the recent buzzing news about DeepSeek-R1 (where R stands for Reasoning). So, what are these Reasoning Models and what can they do what existing models like ChatGPT 4o, Claude Sonnet or Google Gemini can’t do? In September 2024, OpenAI unveiled its first reasoning model o1, marking a significant breakthrough in the ev…

DeepSeek V3.1 Update Brings Improved AI Coding and Reasoning

DeepSeek V3.1 represents a notable step forward in artificial intelligence, particularly in the realms of coding and reasoning. With its enhanced token generation, improved reasoning capabilities, and flexible accessibility options, it is designed to address the evolving needs of developers and researchers. While it offers significant advantages, it also has certain limitations that merit attention. […] The post DeepSeek V3.1 Update Brings Impro…