Travel agencies help to provide comprehensive logistics-as transport, accommodation, meals and accommodation-for businessmen, holidays and all. For those who want to make their own findings, large language models (LLM) seem to be a strong tool for use in this task because of their ability to interact iterantly through natural language, ensuring reasonable reasoning, collecting information and calling other tools that will help in a given task. However, recent works have shown that the most modern LLM struggles with complex logistics and mathematical reasoning, as well as problems with many restrictions, such as travel planning, in which it was found that they provide profitable 4 percent or less time solutions, even with additional tools and interfaces of application programming (API).
Then the research team MIT and MIT-IBM Watson Ai Lab changed this problem to see if they can increase the success rate of LLM solutions for complex problems. “We believe that many of these planning problems are naturally a problem of combinatorial optimization,” in which you need to satisfy a few restrictions in a way that can be certified, says Chuchu, a professor in the Aeronautics and MIT (Aeroastro) Aeronautics and Astronautics Department and the laboratory of information and decision systems (LIDS). She is also a researcher at the MIT-IBM Watson AI laboratory. Her team uses machine learning, control theory and formal methods of developing safe and verifiable control systems for robotics, autonomous systems, controllers and human interactions.
Paying attention to the transmitted nature of their work in the field of travel planning, the group tried to create a user -friendly framework that can act as AI travel broker to help develop realistic, logical and full travel plans. To achieve this, scientists combined a common LLM with algorithms and a full solution to satisfaction. Solvers are mathematical tools that strictly check whether the criteria can be met and how, but require complex computer programming for use. This makes them natural LLM companions in such problems in which users want to help in planning in a timely manner, without the need to program knowledge or research on travel options. In addition, if the user's restriction cannot be met, the new technique can identify and express where the problem lies, and proposes alternative means to a user who can then choose their acceptance, rejection or modification until the formulation of the correct plan, if it exists.
“Different complexities of travel planning are something that everyone will have to deal with at some point. There are different needs, requirements, restrictions and real information that you can gather,” says the fan. “Our idea is not to ask LLM to propose a travel plan. Instead, LLM acts here as a translator to translate this natural language a problem with which Solver can handle (and then provide it to the user),” says the fan.
Co -author A paper At work with the fan is Yang Zhang from MIT-IBM Watson Ai Lab, Aeroastro Yilun Hao graduate and a graduate of Yongchao Chen from Mit Lids and Harvard University. These works have recently been presented at the conference of the chapter of the Nations of the American Association of Computing Linguistics.
Solver breaking
Mathematics is usually specific to the domain. For example, in natural language processing, LLM performs regressions to predict the next token, or “Word” in a series for analysis or creating a document. It works well to generalize a variety of human cartridges. However, LLM themselves would not work in the case of formal verification applications, such as in the field of aviation or cyber security, in which connections and restrictions on circuits must be complete and proven, otherwise gaps and gaps may slip and cause critical security problems. Here, Solvers Excel, but they need a fixed formatting of input data and the fight against dissatisfied asking. Hybrid technique, however, is the opportunity to develop solutions to complex problems, such as travel planning, intuitively for everyday people.
“Solver is really a key here, because when we develop these algorithms, we know exactly how the problem is solved as an optimization problem,” says the fan. In particular, the research group used a solution called SATISTIC MODULO (SMT) Theories, which determines whether the formula can be met. “Thanks to this specific Solver, this is not just optimization. It is reasoning over many different algorithms to understand whether the planning problem is possible or not to be solved. This is quite a significant thing in planning travel. This is not a very traditional problem of mathematical optimization, because people come with all these limitations, restrictions, restrictions, restrictions,” notes the fan.
Translation in action
The “Travel Agency” works in four steps that can be repeated if necessary. Scientists used GPT-4, Claude-3 or Mistral-Large as LLM methods. First of all, LLM analyzes the travel plan required by the user to plan the planning steps, noting preferences regarding budget, hotels, transport, destinations, attractions, restaurants and the duration of travel within days, as well as all other prescriptions for the user. These steps are then converted into a performed Python code (with a natural language annotation for each limit), which is called API interfaces, such as Citysearch, FlightSearch, etc. In order to collect data, and SMT Solver to start taking the steps specified in the problems of restriction satisfaction. If you can find a sound and complete solution, Solver displays the LLM result, which then provides the user with a coherent travel plan.
If one or more restrictions cannot be met, the frame begins to look for an alternative. Solver comes out a code identifying contradictory restrictions (with the appropriate annotation), which LLM provides the user with potential remedies. The user can then decide how to continue until the solution (or maximum number of iterations) is achieved.
Generalized and solid planning
Scientists tested their method using the above-mentioned LLM in relation to other base lines: in itself GPT-4, OpenAI O1-Preview, GPT-4 with information collection tool and a search algorithm that optimizes the total cost. By using the Travelplanner data set, which includes data to real plans, the team looked at many performance indicators: how often the method can provide a solution if the solution met with common criteria, such as not visiting two cities in one day, the ability of the method to meet one or more restrictions, and the final pass indicator indicates that it can meet all restrictions. The new technique generally reached over 90 percent of the pass speed, compared to 10 percent or lower for basic lines. The team also examined the addition of the JSON team at the question stage, which further facilitated the method of providing solutions with a pass rates of 84.4-98.9 percent.
The MIT-IBM team is additional challenges for their method. They looked at how important every element of their solution was – such as the removal of human feedback or Solver – and how it affected the adaptation of the plan to dissatisfied queries within 10 or 20 iteration using a new set of data that he created called Unsatchristmas, which includes invisible restrictions and a modified version of planner Travelplanner. On average, the frames of the MIT-IBM group reached 78.6 and 85 percent of success, which increases to 81.6 and 91.7 percent with additional rounds of plan modification. Scientists analyzed how well they coped well with new, invisible restrictions and paraphrased tips on step and prostate. In both cases it worked very well, especially with 86.7 percent pass for the paraphraction process.
Finally, MIT-IBM scientists applied their frames to other domains with tasks such as the selection of blocks, allocation of tasks, a problem with the traveler and warehouse. Here, the method must choose numbered, colorful blocks and maximize its result; Optimize the task of robot tasks for various scenarios; Plan trips minimizing the distance; and completion of robot tasks and optimization.
“I think it is a very strong and innovative frame that can save a lot of time for people, as well as a very innovative LLM and Solver connection,” says Hao.
These works were partly financed by the Office of Naval Research and MIT-IBM Watson Ai Lab.
