By Eric Horvitz, Chief Scientific Officer of Microsoft

Chief Scientific Officer, Microsoft

February 9, 2024

No single solution exists to confront the complex challenge of deceptive online content, whether that content is handcrafted or AI-generated. Thwarting motivated adversaries will take ongoing vigilance and a multifaceted approach in technical and sociotechnical realms. One important technology is digital media provenance, which has been in the news this week: OpenAI, Meta, and Google announced that they will join Microsoft, Adobe, BBC, Sony, Truepic, Witness, and many other organizations in embracing Content Credentials, C2PA’s technical standard for digital media provenance.

Media provenance methods were developed several years ago with the explicit goal of helping people to grapple with the rise of AI-generated content. A good way to think about the workings of media provenance is to imagine a digital world where each piece of content is verified with a seal, akin to a digital version of a ‘wax seal’ on the back of an envelope—a symbol of its authenticity and unaltered state of its contents. Media provenance, leveraging cryptographic methods, offers just that. The goal is to ensure that the information you consume, like a news report or an image, is not just authentic but also originates from a trusted source and remains unchanged from its production to your screen. Microsoft currently supports media provenance in Microsoft Designer and Designer in Copilot .

The development of effective media provenance tools depends on delivering a widely embraced standard. Three years ago, two outstanding efforts that focused on implementing media provenance, Project Origin and the Content Authenticity Initiative, came together to create a standards organization named the Coalition for Content Provenance and Authenticity (C2PA). The organization now includes nearly 100 companies, and supports a broader ecosystem of over 2,000 organizations, spanning camera manufacturers, content producers, major technology companies, and NGOs. The wide acceptance and continuing collaboration across many groups is an important step forward in bringing consistent content integrity protocols to the public.

Media provenance can be combined with other approaches, such as steganographic watermarking, to add another layer of protection. This invisible watermarking approach is based on hiding information in human-imperceptible locations of the content. That information is later retrieved by use of a watermark detector, providing a way to share information about content that is invisible to the end user. Watermark technologies can be resistant to a range of alterations, including taking screenshots of content, and can be used to provide pointers to publicly available provenance information. However, attackers can manipulate an image until a detector can no longer find the mark, which is why using watermarking in conjunction with media provenance is promising.

While media provenance and watermarking technologies offer significant strides in authenticating content, they are not impervious to manipulation. This current reality calls for a multifaceted approach, combining these technologies with a spectrum of other approaches.  Directions include developing techniques for recognizing and mitigating attempts to create deceptive or offensive content at the point of creation, (including the use of AI methods themselves), ongoing vigilance and fast-paced responses to malevolent uses, and sensitive, balanced regulations that disincentivize deceptive uses and help to establish norms and expectations.

The battle against online deception will be dynamic and ever-evolving, requiring more than just a single technological breakthrough. It necessitates continuous collaboration and innovation across both technical and sociotechnical fronts. Essential to this endeavor is the active involvement of a diverse array of stakeholders, including technology firms, content creators, academic researchers, civil society organizations, and policymakers. By uniting these varied perspectives and expertise, we fortify our ability to stay ahead in the struggle against online deception.