Artists May Have a New Weapon in the Fight Against A.I. Generators—a Data ‘Poisoning’ Tool

The tool has been created by the same team behind Glaze.

An image generated by Stable Diffusion. Photo courtesy of Stability A.I.

In the midst of Britain’s industrial revolution, the Luddites protested the impact of rapid technological adoption on working people by sabotaging factory machinery. Weaving frames were smashed, engines jammed, and in extreme instances whole factories burned.

Today, artificial intelligence poses a similar threat to the labor of creatives. Tech companies, perhaps a digital grandchild of Victorian industrialists, build machine-learning models by scraping the work of artists without consent or compensation. A resistance movement is emerging, though its members are armed with coding smarts rather than clubs.

To empower artists in the fight against image-generating A.I. models, a group of computer science researchers at the University of Chicago are releasing (at an as-yet unspecified date) a new potentially disruptive tool.

It’s called Nightshade and it uses techniques such as targeted adversarial perturbations to allow users to invisibly alter their images before uploading them to the internet. If such images are scraped into an A.I. training set, it can cause the model to mis-learn image objects and scenery, an effect called “poisoning.”

The stated goal of the tool, as proposed in a paper submitted for peer review to a computer security conference, is to encourage the companies behind A.I. models to negotiate with content owners.

“We show that a moderate number of Nightshade attacks can destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images,” the researchers wrote. “We propose the use of Nightshade and similar tools as a last defense for content creators against web scrapers.”

The work builds off Glaze, a tool released earlier this year that allows creatives to cloak images by altering pixels and therefore prevent their artistic style from being stolen by A.I. models. The hope is to integrate Nightshade into Glaze.

Nightshade is also more proactive than its forebear. An artist that uploads a poisoned image can manipulate a model into learning, for instance, that a dog is a cat, a handbag is a toaster, and a hat is a cake. Its open source means anyone can play with Nightshade and create new versions. The greater the number of versions and the more numerous the poisoned images, the greater the potential damage becomes to A.I. image models.

The University of Chicago team tested Nightshade on Stable Diffusion. After plugging in 50 poisoned images of dogs, the model began spitting out strange, contorted animals, some with excess limbs and cartoon faces. After 300 poisoned images, Stable Diffusion began issuing cats. The tool also touches associated words, what researchers call “bleed through,” such that canine, wolf, and puppy can also conjure felines. Crucially, undoing the effects of poisoning is difficult and requires each infected image to be removed.

The tool arrives at a moment when A.I. giants including Meta, Google, OpenAI, and Stability AI are being confronted with lawsuits from artists claiming their copyrighted material and intellectual property have been stolen.

Ignoring the inherent challenges of individuals taking on ultra-wealthy companies, the pushback may have come too late given scraped data may have been irreversibly baked into models. In this light, Nightshade offers an alternate and clandestine means of encouraging tech companies to respect the copyright and intellectual property of artists.


More Trending Stories:  

The World’s Most Popular Painter Sent His Followers After Me Because He Didn’t Like a Review of His Work. Here’s What I Learned 

Archaeologists Excavating the Tomb of Egypt’s First Female Pharaoh Found Hundreds of Jars Still Holding Remnants of Wine 

The Second Paris+ Started With a Bang. Could Art Basel’s New Venture Unseat Its Flagship Fair One Day? 

Is There a UFO in That Renaissance Painting? See 7 Historical Artworks That (Possibly) Depict Close Encounters With the Third Kind 

A Glasgow Museum Reveals a $3.7 Million Rodin Sculpture Has Been Missing From Its Collection for Nearly 75 Years 

What I Buy and Why: Art Entrepreneur Hélène Nguyen-Ban on Her Original ‘Art Crush’ and Owning a Half-Ton Book by Anselm Kiefer 

Christie’s 20th/21st Century Evening Sale Notches Steady Results, a Feat in the Current Tepid Art Market 

Four ‘Excellently Preserved’ Ancient Roman Swords Have Been Found in the Judean Desert 

An Early Edition of an ‘Unhinged’ Christopher Columbus Letter Outlining What He Discovered in America Could Fetch $1.5 Million at Auction 

Follow Artnet News on Facebook:

Want to stay ahead of the art world? Subscribe to our newsletter to get the breaking news, eye-opening interviews, and incisive critical takes that drive the conversation forward.