Mapping of the wrong use of generative AI

New research is analyzing the improper use of multimodal generative artificial intelligence today to help build safer and more responsible technologies

Generative models artificial intelligence (AI), which can produce image, text, audio, video and others, allow a new era of creativity and commercial possibilities. However, as the ability increases, these potential for their improper use, including manipulation, fraud, intimidation or harassment.

As part Our commitment To develop and use AI, we published new paperin cooperation with Jigsaw AND Google.orgAnalyzing how generative AI technologies are abused today. Google teams use these and other studies to develop better security for our AI generative technologies, including security initiatives.

Together, we collected and analyzed almost 200 media reports, intercepting public incidents of improper use, published between January 2023 and March 2024. On the basis of these reports, we defined and categorized the common tactics of improper use of generative artificial intelligence and found new patterns in the use of technology.

Explaining the current threats and tactics used in various types of generative artificial intelligence results, our work can help in shaping AI management and running companies such as Google and other building AI technology in developing more comprehensive assessments of security and relief strategy.

Emphasizing the main categories of improper use

While AI generative tools are unique and convincing measures to increase creativity, the ability to create on request, realistic content can be used in the wrong way by malicious actors.

Analyzing media reports, we identified two main categories of generative tactics of improper use of AI: The use of AI generative capabilities and compromise of generative AI systems. Examples of the technologies used included the creation of realistic representations of human similarities in order to impersonate public figures; While the instances of threatened technologies included “jailbreaking” in order to delete the model's security and use opposite data to cause a failure.

Cases of exploitation-making malicious entities using easily accessible, AI generative tools at the consumer level, often in a way that did not require advanced technical skills-the most widespread data in our set. For example, we checked the loud case of February 2024, in which an international company Apparently he lost 200 million HK $ (Approx. $ 26 million) after an employee was cheated to make a financial transfer during an online meeting. In this case, every other “person” at the meeting, including the company's financial director, was in fact a convincing, generated cheater.

Some of the most visible tactics that we observed, such as impersonation under fraud and synthetic figures, are preliminary inventing generative artificial intelligence and have long been used to influence the information ecosystem and manipulation of others. But wider access to AI generative tools can change costs and incentives for manipulation of information, giving these eternal tactics new strength and potential, especially those who previously lacked technical sophistication to take into account such tactics.

Identification of the strategy and combination of improper use

Faling evidence and manipulating human similarities at the root of the most common tactics in real cases of improper use. In the periods we analyze, most cases of improper use of artificial intelligence have been arranged to influence public opinion, enabling fraud or dishonest activities or generate profit.

Observing how bad actors combine their generative tactics of improper use of artificial intelligence in pursuit of different goals, we identified specific combinations of improper use and marked these combinations as strategies.

The emerging forms of improper use of AI, which are not explicitly malicious, still arouse ethical concerns. For example, new forms of political scope blur the boundaries between authenticity and fraud, such as District officials suddenly speak a variety of language friendly languages without transparent disclosure that they use generative artificial intelligence and Activists using the votes generated by AI of deceased victims to beg for a reform of weapons.

Although the study provides new information on the emerging forms of improper use, it is worth noting that this set of data is a limited sample of media reports. Media reports can prioritize sensational incidents, which in turn can distort the set of data towards specific types of improper use. Detection or reporting cases of incorrect use can also be more difficult for those involved because AI generative systems are so new. The data set also does not make a direct comparison between the wrong use of AI generative systems and traditional tactics of creating and manipulating content, such as editing images or configuring “content farm” to create large amounts of text, video, gifs, images and others. So far, anecdotal evidence suggests that traditional tactics of content manipulation remain more common.

Overtaking potential inappropriate

Our paper It emphasizes the possibilities of designing initiatives that protect society, such as the development of broad generative reading and writing campaigns, developing better interventions to protect society against bad entities or Adjudicating people and equipping them To detect and overthrow manipulation strategies used in the generative improper use of artificial intelligence.

These research helps our teams better secure our products by informing about the development of security initiatives. On YouTube Mytube Now they require creators to share when their work is significantly changed or generated synthetically and seems realistic. Similarly, we updated our electoral advertising policies to require advertisers to disclose when their election advertising includes materials that have been digitally changed or generated.

When we continue to broaden our understanding of malicious applications of generative artificial intelligence and we make further technical progress, we know that it is more important than ever to make sure that our work is not happening in the silo. We recently joined to Content on the origin of the coalition and authenticity (C2pa) as a member of the control committee that helps to develop a technical standard and drive the reception of content certificates that are manipulation resistant metadata, which shows how the content was introduced and edited in time.

At the same time, we also conduct research that develops existing efforts to the red team, including Improving the best practices of testing the safety of large language models (LLM)and developing pioneering tools to facilitate the content generated by AI, such as Synthid, which is integrated with a growing range of products.

In recent years, Jigsaw has conducted research with the creators of disinformation To understand the tools and tactics they use, Prepared Pleuking Videos present people attempts to manipulate them and It has been shown that precursing campaigns can improve large -scale disinformation resistance. This work is part of the wider portfolio of Jigsaw information interventions to help people protect online.

Through proactively dealing with potential improper uses, we can support the responsible and ethical use of generative artificial intelligence, while minimizing its risk. We hope that these observations on the most common tactics and strategies of improper use will help researchers, decision makers, trust and safety teams in the industry to build safer, more responsible technologies and develop better means to combat improper use.