Scientists from Mit Computer Science and Artificial Intelligence Laboratory (CSAIL) and Google Research introduced “Alchemist”, A model that offers unprecedented precision in controlling the properties of material in images. This innovative tool concerns a significant challenge, which is facing users of the models of the text to the image: Achieving detailed and exact material properties.
Alchemist It enables users to modify four key attributes of both real and photos generated by AI:
- Roughness
- Metallicity
- Albedo
- Transparency
Alchemist takes any photo as input data and allows users to customize any property on a continuous scale from -1 to 1, creating a new visual. The magic behind him lies in its Denoising diffusion model, in particular stable diffusion 1.5. This image model is known for its photorealistic results and editing. Unlike previous diffusion systems, which focused on higher level changes (such as the change of objects or the change in the depth of the image), the alchemist honors the attributes of low levels. Its unique zip -based interface exceeds other methods, enabling precise regulations of the material properties.
Alchemist design capabilities promise significant progress in various fields:
- Video game design: Alchemist can be used to modify video games models, adapt them to different environments or improve their realism.
- Visual effects (VFX): By adjusting the properties of the material, the alchemist can expand the possibilities of artificial intelligence in visual effects, thanks to which the scenes are more convincing and addictive.
- Training data robots: By exposing robots to a wider range of textures, they can better understand and manipulate various objects in scenarios in the real world. In addition, alchemist capabilities in the image classification can help determine where neural networks try to recognize material changes, thus improving the accuracy of these systems.
In comparative research, the alchemist exceeded similar models, carefully editing only a specific subject of interest. For example, when it is a task that the dolphin fully transparent without changing the ocean background, Alchemist was the only model that achieved it. Users' research has shown a preference for an alchemist, with many believe that his results are more photorealistic than those of its counterparts.
To overcome the impracticity of collecting real data, scientists trained alchemist in the scope of a synthetic data set. This set of data included random editing of attributes of materials of 1200 materials used for 100 unique 3D objects in Blender, a popular computer graphic tool.
Despite his progress, Alchemist has some restrictions, especially in the exact application of lighting, which can lead to physically incredible results. For example, with maximum transparency settings, the hand partly inside the cereal box may appear as a clear container without visible fingers.
The research team aims to extend the possibilities of alchemist. Future work can focus on improving 3D resources for graphics at the stage level and the application of material properties from images, potentially combining visual and mechanical features.
Watch our video on YouTube to get a short demonstration of the Alchemist's magic in action.
