Despite all their impressive possibilities, large language models (LLM) often have no problem when they receive difficult tasks requiring complex reasoning skills.
While LLM of the accounting company may stand out in the summary of financial reports, the same model may unexpectedly fail if the task of predicting market trends or identifying dishonest transactions.
To make LLM more adapting, the scientists have examined the myth how you can strategically implement a certain training technique to increase the model's performance on unknown, difficult problems.
They show that training during a test time, a method including temporarily updating some internal activities of the model during implementation, can lead to six times improvement in accuracy. Scientists have developed a framework for implementing training strategies during a test time, which uses examples of a new task to maximize these profits.
Their work can improve the flexibility of the model, enabling the available LLM to adapt to complex tasks requiring planning or abstraction. This can lead to LLM, which would be more accurate in many applications that require logical deduction, from medical diagnostics to supply chain management.
“Real learning is what we have done here with time training-it is something that these models cannot do after sending them. They cannot acquire new skills or be better in the task. But we showed that if you push the model for learning, you will see that there may be a huge improvement in efficiency,” says Ekin Akyrek PHD '25, the main author of the study.
Aksürek is attached to paper by doctoral students Mehul Damani, Linl Qiu, Han Guo and Jyothish Pari; License Adam Zweiger; and older authors Yoon Kim, an assistant professor of electrical and computer science (EECS) and a member of the IT laboratory and artificial intelligence (CSAIL); and Jacob Andreas, associate professor at EECS and CSAIL member. Research will be presented at an international conference on machine learning.
Employing hard domains
LLM users often try to improve the performance of their model in a new task using a technique called learning in context. They feed the model a few examples of the new task, because the hints of the text that direct the model outputs.
But learning in the context does not always work on problems that require logic and reasoning.
MIT scientists have examined how training during test time can be used in combination with context in the context to increase the efficiency of these difficult tasks. Time training includes the update of some model parameters-internal resources, which it uses to predict-to use a small amount of new data specific to a given task.
Scientists have examined how the training during the test time interacts with learning in context. They studied design choices that will maximize performance improvements that can be persuaded from the general purpose of LLM.
“We find that training during test time is a much stronger form of learning. Although simply providing examples can modestly increase accuracy, actually updating the model with these examples can lead to much better performance, especially in difficult domains,” says Damani.
Learning in the context requires a small set of examples of tasks, including problems and their solutions. Scientists use these examples to create a set of data specific for the task needed to train test time.
To extend the size of this set of data, they create new input data, slightly changing problems and solutions in examples, such as horizontally shifting input data. It turns out that the model training at the results of this new set of data leads to the best performance.
In addition, scientists update only a small number of model parameters using a low ranking adaptation technique, which improves the performance of the test time training process.
“This is important because our method must be efficient if it is to be implemented in the real world. We think that you can get a huge improvement in accuracy due to a very small amount of parameters training,” says Aleyrek.
Developing new skills
Improving this process is crucial because time training is used on the basis of instances, which means that the user would have to do it for each task. The model updates are only temporary, and the model returns to its original form after the forecast.
The model, which usually takes less than a minute to answer the question, can take five or 10 minutes to answer the training during the test time, adds Aksürek.
“We would not like to do this for all user queries, but it is useful if you have a very difficult task that you want to solve well. There may also be tasks that are too difficult for LLM to solve without this method,” he says.
Scientists tested their approach to two sets of comparative data of extremely complex problems, such as IQ puzzles. This increased accuracy up to six times than techniques that only use learning in context.
Tasks covering structural patterns or those that used completely unknown data types showed the greatest improvement in performance.
“In the case of simpler tasks, learning in the context can be fine. But the parameter update itself can develop a new skill in the model,” says Damani.
In the future, scientists want to use these observations on the development of models that are constantly learning.
The long -term goal is LLM, which, taking into account the inquiry, can automatically determine whether he has to use test time training to update parameters, or can solve the task by learning in context, and then implement the best training strategy during the test without the need for human intervention.
These works are partly supported by MIT-IBM Watson Ai Lab and National Science Foundation.
