In recent years, the AI field has been captivated by the success of large language models (LLM). Initially designed for natural language processing, these models have evolved into powerful reasoning tools capable of solving complex problems with the human thought process. However, despite their exceptional reasoning abilities, LLM has significant defects, including high calculation costs and slow implementation speeds, which makes them impractical for use in the real world in the environments limited by resources, such as mobile devices or edge processing. This led to the growing interest in developing smaller, more efficient models that can offer similar reasoning, while minimizing costs and demand for resources. In this article, he examines the increase in these small reasoning models, their potential, challenges and implications for the future of AI.
Change of perspective
In most of the latest AI history, the field followed the principle of “scaling provisions”, which suggests that the model's performance is improved in relation to data, computing power and the increase in the size of the model. Although this approach brought powerful models, it also caused significant compromises, including high infrastructure costs, environmental impact and delays problems. Not all applications require full possibilities of mass models with hundreds of billions of parameters. In many practical cases-as assistants on the device, healthcare and education-matches, marriage can achieve similar results if they can effectively reasonably reason.
Understanding reasoning in artificial intelligence
The reasoning in AI refers to the model's ability to follow logical chains, understand the cause and effect, deduce implications, plan steps in the process and identify contradictions. In the case of language models, this often means not only downloading information, but also manipulating and inference to information through a structured approach step by step. This level of reasoning is usually achieved by refining LLM to perform multi -stage reasoning before obtaining a response. These methods require significant calculation resources and may be slow and expensive implementation, increasing the concerns about their availability and environmental impact.
Understanding small models of reasoning
Fashionable reasoning models are aimed at repeating the possibility of reasoning of large models, but with greater efficiency in terms of computing power, memory consumption and delays. These models often use the technique called knowledge distillationWhere a smaller model (“student”) learns from a larger, pre -trained model (“teacher”). The distillation process consists in training a smaller model on data generated by a larger one in order to transfer the ability to reason. The student model is then adapted to improve its performance. In some cases, learning to strengthen with specialized functions of a specific prize for the domain is used to further increase the model's ability to perform specific reasoning for the task.
Growth and progress of small models of reasoning
A significant milestone in the development of small reasoning models appeared with the release of Deepseek-R1. Despite the training in the relatively modest cluster of older GPU, Deepseek-R1 reached performance comparable to larger models, such as O1 Openai on comparative tests such as MMLU and GSM-8K. This achievement led to the re -examination of the traditional scaling approach, which assumed that larger models were better by nature.
The success of the Deepseek-R1 can be attributed to an innovative training process that combined a large-scale reinforcement learning without relying on supervised tuning in the early stages. This innovation led to creation Deepseek-R1-ZeroA model that showed impressive reasoning skills compared to large models of reasoning. Further improvements, such as the use of cold data, increased the consistency and performing the tasks of the model, especially in areas such as mathematics and code.
In addition, distillation techniques proved to be crucial in the development of smaller, more efficient models from larger ones. For example, Deepseek has released distilled versions of his models, size from 1.5 billion to 70 billion parameters. Using these models, scientists trained a relatively much smaller model Deepseek-R1-Distill-Qwen-32B which he surpassed O1-mini openai on various comparative tests. These models can now be implemented with standard equipment, which makes them a more profitable option for a wide range of application.
Can small models match the reasoning at the GPT level
To assess whether small reasoning models (SRM) can match the power of large models (LRM), such as GPT, it is important to assess their performance on standard comparative tests. For example, the Deepseek-R1 model conquered about 0.844 on Mml testcomparable to larger models, such as O1. On GSM-8K Data set that focuses on mathematics of class schools, the DEEPSEEK-R1 distilled model achieved The highest level performance, exceeding both O1 and O1-Mini.
In coding tasks, such as these Livecodebench AND CodeDistilled Deepseek-R1 models made Like O1-Mini and GPT-4O, showing strong programming reasoning. However, larger models still have edge In tasks requiring a broader understanding or service of a long context of windows, because smaller models are more specific to the task.
Despite their strengths, small models can struggle with extended reasoning tasks or in the face of data outside distribution. For example, in chess simulations in LLM Deepseek-R1 he made more mistakes than larger models, which suggests limiting his ability to maintain concentration and accuracy for a long time.
Compromises and practical implications
The compromises between the size of the model and performance are crucial when comparing SRMS with LRM at GPT level. Smaller models require less memory and computing power, thanks to which they are ideal for edge devices, mobile applications or situations where offline inference is necessary. This performance causes lower operating costs, with models such as Deepseek-R1 cheaper Start than larger models, such as O1.
However, these performance increases are associated with some compromises. Smaller models are usually refined to specific tasks that can limit their versatility compared to larger models. For example, while Deepseek-R1 is distinguished by mathematics and coding, it lack Multimodal possibilities, such as the ability to interpret images with which larger models, such as GPT-4O, can support.
Despite these restrictions, the practical applications of small reasoning models are huge. In healthcare, they can supply diagnostic tools that analyze medical data on standard hospital servers. In education, they can be used to develop personalized tutoring systems, providing students with opinions step by step. In scientific research, they can help in analyzing data and testing hypotheses in fields such as mathematics and physics. The nature of models, such as Deepseek-R1, also promotes cooperation and democratizes access to AI, enabling smaller organizations to use advanced technologies.
Lower line
The evolution of language models into smaller reasoning models is a significant progress in AI. Although these models may not yet fully match the wide possibilities of large language models, they offer key advantages in terms of performance, profitability and availability. Stronging the balance between the strength of reasoning and the efficiency of resources, smaller models are to play a key role in various applications, thanks to which AI is more practical and balanced for use in the real world.
