Nvidia introduced the Nemotron-4 340B model familyPackage of powerful models with open access designed to improve the synthetic generation of data and training of large language models (LLM). This release includes three separate models: Nemotron-4 340B base, Nemotron-4 340B instructor and the NEMOTRON-4 340B prize. These models promise to significantly increase the possibilities of artificial intelligence in a wide range of industries, including healthcare, finance, production and retail.
The basic innovation of Nemotron-4 340b consists in its ability to generate high quality synthetic data, which is a key element of training effective LLM. High-quality training data is often expensive and difficult to obtain, but thanks to Nemotron-4 340B programmers can create solid large-scale data sets. The basic model NEMOTRON-4 340B The base has been trained on a huge body of 9 trillion tokens and can be additionally adapted to the reserved data. The NEMOTRON-4 340B instruction model generates various synthetic data that imitates scenarios in the real world, while the NEMOTRON-4 340B award model ensures the quality of this data by assessing the answers based on usefulness, correctness, consistency, complexity and verbality.
The distinguished feature of NEMOTRON-4 340B is its sophisticated equalization process, which uses both the optimization of direct preferences (DPO) and the optimization of rewarded preferences (RPO) to tune the models. DPO optimizes the model's answers, maximizing the gap in rewards between the preferred and unrelated answers, while the RPO decreases, taking into account the differences in the awards between the answers. This double approach ensures that models not only produce high -quality results, but also maintain balance in various indicators of the assessment.
NVIDIA used a stage supervised refinement process (SFT) to increase the possibilities of the model. The first stage, Code SFT, focuses on improving coding and reasoning ability using synthetic coding data generated using the genetic instructions-the metody simulating evolutionary processes to create high-quality samples. The later general stage of SFT consists in training from a variety of data set to ensure that the model copes well in a wide range of tasks, while maintaining its proficiency in coding.
The NEMOTRON-4 340B models use the iterative process of leveling weak to the engine, which constantly improves models through subsequent data generation cycles and refinement. Starting from the initial even model, each iteration produces higher quality data and more sophisticated models, creating a self -controlling cycle of improvement. This iterative process uses both strong basic models and high -quality data sets to increase the overall efficiency of instructional models.
Practical applications of NEMOTRON-4 340B models are huge. By generating synthetic data and matching the refining model, these tools can significantly improve the accuracy and reliability of AI systems in various fields. Programmers can easily access these models through NVIDIA NGCIN Huggingand the upcoming platform ai.nvidia.com.
