Because artificial intelligence systems (AI) are becoming more and more complex, understanding their internal actions is crucial for safety, honesty and transparency. Scientists from Mit's Computer Science and Artificial Intelligence Laboratory (CSAIL) have introduced an innovative solution called “Maia” (multimodal agent of an automated interpretation)A system that automates the interpretation of neural networks.
Maia was designed to solve the problem of understanding large and complex AI models. It automates the process of interpretation of computer vision models that assess the different properties of images. Maia uses a skeleton with a vision language in combination with the Library of Interpretation Tools, enabling it to conduct experiments in other AI systems.
According to Tamar Rott Shaham, a co -author of the research article, their goal was to create an AI researcher who can autonomously conduct interpretation experiments. Since existing methods only mean or visualize data in a one -time process, Maia can, however, generate hypotheses, design experiments to test them and improve their understanding by iterative analysis.
Maii's capabilities were shown in three key tasks:
- Labeling of components: Maia identifies individual elements in vision models and describes the visual concepts that activate them.
- Cleansing the model: By removing irrelevant features from image classifiers, Maia increases its resistance in innovative situations.
- Detection of prejudices: Maia hunts hidden prejudices, helping to discover potential problems of honesty in AI products.
One of the noteworthy features of Maia is his ability to describe the concepts detected by individual neurons in the vision model. For example, the user may ask Maia to determine what specific neuron detects. Maia downloads “examples of data” from Imagenet, which maximally activate neuron, hypothesize the causes of neuron activity and designs experiments to test these hypotheses. By generating and editing synthetic images, Maia can isolate specific causes of neuron activity, as did the scientific experiment.
Maia's explanations are evaluated using synthetic systems with known behavior and new automatic protocols for true neurons in trained AI systems. The method directed by CSAIL exceeded the initial methods of describing neurons in various models of view, often corresponding to the quality of descriptions written by people.
The field of interpretation is evolving with the development of the “Black Box” machine learning models. Current methods are often limited on a scale or precision. Scientists tried to build a flexible, scalable system to answer various interpretative questions. Detection of prejudices in image classifiers was a critical area of focus. For example, Maia identified the prejudice in the classifier, which incorrectly classified the paintings of black labradors, while thoroughly classifying yellow retrievers.
Despite the strong pages, Maia's performance is limited by the quality of its external tools. With the increase in models of image synthesis and other tools, Maia will improve. Scientists also implemented a text tool for alleviating prejudices confirming and overwork problems.
Looking to the future, scientists plan to apply similar experiments to perception by people. Traditionally, testing human visual perception was intense. In the case of Maia, this process can be scale, potentially enabling comparisons between man and artificial visual perception.
Understanding the neuron networks is difficult due to their complexity. Maia helps to fill this gap, automatically analyzing neurons and reporting the arrangements in a digestible way. Increasing these methods can be of key importance for understanding and supervising AI systems.
Maia's contribution went beyond the academic community. Because AI becomes an integral part of various fields, the interpretation of his behavior is necessary. Maia connects the gap between complexity and transparency, thanks to which AI systems are more responsible. By equipping AI researchers with tools that keep up with the system scaling, we can better understand and solve the challenges related to new AI models.
For more information, research is published on Server Arxiv Preprint.
