Because artificial intelligence (AI) is widely used in areas such as healthcare and self -propelled cars, the question of how much we can trust it becomes more critical. She drew attention to one method called chain reasoning (COT). It helps AI to break down complex problems, showing how it will reach the final answer. This not only improves performance, but also checks how he thinks, what is important for the trust and security of AI systems.
But the latest tests From anthropic questions, whether COT really reflects what is happening in the model. In this article appears how Cot works, which he found anthropic and what all this means for building reliable artificial intelligence.
Understanding chain reasoning
Chain reasoning is a way to encourage artificial intelligence to solve problems step by step. Instead of just giving the final answer, the model explains every step along the way. This method was introduced in 2022 and since then it helped improve the results in tasks such as mathematics, logic and reasoning.
Models such as O1 Openai i O3IN Gemini 2.5IN Deepseek R1AND SONET CLAUDE 3.7 Use this method. One of the reasons why the cot is popular is that AI's reasoning is more visible. This is useful when the cost of errors is high, for example in medical tools or independent systems.
Still, despite the fact that COT helps in transparency, it does not always reflect what this model really thinks. In some cases, explanations may look logical, but they are not based on the actual steps that the model used to make decisions.
Can we trust the chain of reflection
Anthropic tested whether COT explanations really reflect how AI models make decisions. This quality is called “loyalty”. They studied four models, including Claude 3.5, Claude 3.7 Sonet, Deepseek R1 and Deepseek V1. Among these models, Claude 3.7 and Deepseek R1 were trained using COT techniques, while others did not.
They gave models various hints. Some of these hints included tips that were to affect the model in an uneven way. Then they checked whether AI used these tips in her reasoning.
The results raised concerns. The models only confessed to using tips less than 20 percent of the time. Even models trained in use COT explained faithful explanations in just 25 to 33 percent of cases.
When the instructions included unethical actions, such as cheating the reward system, models rarely recognized them. It happened even though they relied on these tips to make decisions.
Model training using the reinforcement learning more made a slight improvement. But it still didn't help much when the behavior was unethical.
Scientists also noticed that when the explanations were not real, they were often longer and more complicated. This may mean that the models tried to hide what they really did.
They also discovered that the more complex the task, the less faithful the explanations became. This suggests that COT may not work well in case of difficult problems. It can hide what the model really does, especially in the case of sensitive or risky decisions.
What does this mean for trust
The study emphasizes a significant gap between how transparent cots appear and how honest it really is. In critical areas such as medicine or transport, this is a serious risk. If AI gives a logically looking explanation, but hides unethical actions, people may wrongly trust the result.
Cot is helpful in case of problems requiring logical reasoning in a few steps. But it may not be useful in detecting rare or risky errors. It also does not stop the model from giving misleading or ambiguous answers.
Studies show that COT itself is not enough for AI decision trust. You also need other tools and checks to make sure that AI behaves in a safe and honest way.
Strong and limits of the thoughts chain
Despite these challenges, COT offers many advantages. It helps AI to solve complex problems, dividing them into parts. For example, when there is a large language model Raised Thanks to Cot, he showed the accuracy of the highest level of problems with mathematical words, using this reasoning step by step. COT also makes it easier for programmers and users to follow what the model does. This is useful in areas such as robotics, natural language processing or education.
However, Cot is not deprived of its flaws. Smaller models try to generate reasoning step by step, while large models need more memory and power to use it well. These restrictions make it difficult to use COT in tools such as chatbots or systems in real time.
COT performance also depends on how the hints are written. Poor hints can lead to bad or misleading steps. In some cases, models generate long explanations that do not help and do not increase this process. In addition, errors early in reasoning can move to the final response. And in specialized fields of COT it may not work well, unless the model is trained in this area.
When we add an anthropic arrangements, it becomes clear that Cot is useful, but not enough in itself. This is one part of more effort to build artificial intelligence that people can trust.
Key arrangements and ahead
These studies indicate several lessons. First of all, COT should not be the only method we use to check AI behavior. In critical areas, we need more controls, such as looking at the internal activity of the model or the use of external tools to test decisions.
We must also accept that only because the model gives a clear explanation does not mean that he is telling the truth. An explanation can be a cover, not a real reason.
To deal with this, scientists suggest a combination of COT with other approaches. They include better training methods, supervised learning and reviews of people.
Anthropic also recommends a deeper look at the model of the model. For example, checking activation patterns or hidden layers can show if the model hides something.
Most importantly, the fact that models can hide unethical behavior shows why strong testing and ethical principles are needed in AI development.
Building trust in artificial intelligence is not only good performance. It is also about making sure that the models are fair, safe and open to control.
Lower line
Chain reasoning helped improve the way AI solves complex problems and explains its answers. But research shows that these explanations are not always true, especially when ethical problems are involved.
Cot has limits such as high costs, large models are needed and dependence on good hints. It cannot guarantee that artificial intelligence will work in a safe or honest way.
To build artificial intelligence, which we can really rely on, we must combine COT with other methods, including human supervision and internal control. Research must also continue to improve the credibility of these models.
