These “AI models” (meaning the free and open Stable Diffusion in particular) consist of different parts. The important parts here are the VAE and the actual “image maker” (U-Net).
A VAE (Variational AutoEncoder) is a kind of AI that can be used to compress data. In image generators, a VAE is used to compress the images. The actual image AI only works on the smaller, compressed image (the latent representation), which means it takes a less powerful computer (and uses less energy). It’s that which makes it possible to run Stable Diffusion at home.
This attack targets the VAE. The image is altered so that the latent representation is that of a very different image, but still roughly the same to humans. Say, you take images of a cat and of a dog. You put both of them through the VAE to get the latent representation. Now you alter the image of the cat until its latent representation is similar to that of the dog. You alter it only in small ways and use methods to check that it still looks similar for humans. So, what the actual image maker AI “sees” is very different from the image the human sees.
Obviously, this only works if you have access to the VAE used by the image generator. So, it only works against open source AI; basically only Stable Diffusion at this point. Companies that use a closed source VAE cannot be attacked in this way.
I guess it makes sense if your ideology is that information must be owned and everything should make money for someone. I guess some people see cyberpunk dystopia as a desirable future. I wonder if it bothers them that all the tools they used are free (EG the method to check if images are similar to humans).
It doesn’t seem to be a very effective attack but it may have some long-term PR effect. Training an AI costs a fair amount of money. People who give that away for free probably still have some ulterior motive, such as being liked. If instead you get the full hate of a few anarcho-capitalists that threaten digital vandalism, you may be deterred. Well, my two cents.
So, it only works against open source AI; basically only Stable Diffusion at this point.
I very much doubt it even works against the multitude of VAEs out there. There’s not just the ones derived from StabilitiyAI’s models but ones right now simply intended to be faster (at a loss of quality): TAESD can also encode and has a completely different architecture thus is completely unlikely to be fooled by the same attack vector. That failing, you can use a simple affine transformation to convert between latent and rgb space (that’s what “latent2rgb” is) and compare outputs to know whether the big VAE model got fooled into generating something unrelated. That thing just doesn’t have any attack surface, there’s several magnitudes too few weights in there.
Which means that there’s an undefeatable way to detect that the VAE was defeated. Which means it’s only a matter of processing power until Nightshade is defeated, no human input needed. They’ll of course again train and try to fool the now hardened VAE, starting another round, ultimately achieving nothing but making the VAE harder and harder to defeat.
It’s like with Russia: They’ve already lost the war but they haven’t noticed, yet – though I wouldn’t be too sure that Nightshade devs themselves aren’t aware of that: What they’re doing is a powerful way to grift a lot of money from artists without a technical bone in their body.
Yeah. Not that it’s the fault of artists that capitalism exists in its current form. Their art is the fruit of their labor, and therefore, means should be taken to ensure that their labor is properly compensated. And I’m a marxist anarchist, no part of me agrees with any part of the capitalist system. But artists are effectively workers, and we enjoy the fruits of their labor. They are rarely fairly compensated for their work. In this particular instance, under the system we live in, artists rights should be prioritized over
I’m all for janky (getting less janky as time goes on) AI images, but I don’t understand why it’s so hard to ask artists permission first to use their data. We already maintain public domain image databases, and loads of artists have in the past allowed their art to be used freely for any purpose. How hard is it to gather a database of art who’s creators have agreed to let it be used for AI? All the time we’ve (the collective we) been arguing over thise could’ve been spent implementing a system to create such a database.
You should check out this article by Kit Walsh, a senior staff attorney at the EFF. The EFF is a digital rights group who recently won a historic case: border guards now need a warrant to search your phone. It should help clear some things up for you.
Fair enough, and I can’t claim to be a fan of copyright law or how it’s used. Maybe what I’m moreso talking about is a standard of ethics? Or some laws governing the usage of image and text generating AI specifically as opposed to copyright law. Like just straight up a law making it mandatory for AI to provide a list of all the data it used, as well as proof of the source of that data having consented to it’s use in training the AI.
Or some laws governing the usage of image and text generating AI specifically as opposed to copyright law.
What you are talking about is an expansion of copyright law. Copyright includes more than just the right to make copies. It also includes the right to authorize derivatives, such as translations of texts, movies based on comics, or games based on movies. Fan art is also a derivative and relies on fair use for its legality (assuming it is legal).
If one were to create an “AI training right”, then the natural place to put it, would be with the other rights covered by copyright. Of course, one could lay down such a right outside the copyright statute, and write that it is not part of copyright law.
In any case, it would be intellectual property. The person, who can allow or deny AI training on some work, would own that right as intellectual property.
Yeah, I’m not too concerned with janky AI generators having to ask before training a model on someone’s art. Sucks for them I guess.
I don’t agree with copyright. I’m an anarchist. I’m openly in favor of piracy, derivative, whatever else a human being might do with something. I don’t agree with judicial systems, let alone market economies or even currency as a concept. And that’s all fine and dandy, but there are people alive right now under capitalism. Unlike piracy, which pretty much exclusively takes from corporations like the overwhelming majority of things that are pirated are produced by corporate studios and studio funded artists, this one very specific thing takes the most specifically from artists the overwhelming majority of whom are already very poorly compensated many of them literally barely get by at all. AI models should have to ask them to copy and repurpose their works.
That’s my only statement. You can assume I effectively don’t agree with any other thing. I’m not here to have a long winded nuanced debate about a legal system I don’t agree with and am not supporting in literally any capacity. I’m pointing at pixiv the website and saying “hey can you guys like actually ask before you start using these people’s shit to make AI that is purposefully built to make sure that they are run out of jobs”
Unless you’re going to somehow explain why artists aren’t worth existing or something then don’t even bother answering. I’m genuinely not interested in what you have to say and am tired of repeating myself in this thread.
I just thought you should know where you stand on the issue. It will make it easier to communicate. Just say that you want to expand copyright to cover AI training and boom. Clear statement. No long winded, nuanced debate needed.
Don’t actually know where the hostility comes from. Are you mistaking me for someone else?
I dont want copyright to be expanded, I dont want laws governing intellectual property at all. I’ve described what I think is right pretty fully. I don’t need you to tell me where I stand.
You can read my other comments if you want to engage with it any further. I’m not mistaking you for someone else. I’m just tired of people rehashing the same endless points. Arguing with AI bros is tireless, pointlessly futile. It consistently devolves into innane nonsense. I’m fully on board with doing away with copyright as a concept entirely. My request is that artificial image and text generation be regulated in a way that is ethical with respect to small content creators who should have a say in what software their art is used to generate. That’s it fam I’m out
There’s nothing wrong with being able to use others’ copyrighted material without permission though. For analysis, criticism, research, satire, parody and artistic expression like literature, art, and music. In the US, fair use balances the interests of copyright holders with the public’s right to access and use information. There are rights people can maintain over their work, and the rights they do not maintain have always been to the benefit of self-expression and discussion.
It would be awful for everyone if IP holders could take down any review, finding, reverse engineering, or indexes they didn’t like. That would be the dream of every corporation, bully, troll, or wannabe autocrat. It really shouldn’t be legislated.
I’m not talking about IP holders, and I do not agree with copyright law. I’m not having a broad discussion on copyright here. I’m only saying, and not saying anything more, that people who sit down and make a painting and share it with their friends and communities online should be asked before it is scanned to train a model. That’s it.
How’re we supposed to have things like reviews, research findings, reverse engineering, or indexes if you have to ask first? The scams you could pull if you could attack anyone caught reviewing you. These rights exist to protect us from the monopolies on expression that would increase disparities and divisions, manipulate discourse, and in the end, fundamentally alter how we interact online with each other for the worse.
I’m just gonna ask you to read my above comment again. What I’m suggesting is:
“Before you scrape and analyze art with the specific purpose of making an AI art generator model, you must ask permission from the original creating artist.”
I read that. That’s what I’ve been responding to the whole time. This is a way to analyze and reverse engineer images so you can make your own original works. In the US, the first major case that established reverse engineering as fair use was Sega Enterprises Ltd. v. Accolade, Inc in 1992, and then affirmed in Sony Computer Entertainment, Inc. v. Connectix Corporation in 2000. Do you think SONY or SEGA would have allowed anyone to reverse engineer their stuff if they asked nice? Artists have already said they would deny anyone.
It’s not about the data, people having a way to make quality art themselves is an attack on their status, and when asked about generators that didn’t use their art, they came out overwhelmingly against with the same condescending and reductive takes they’ve been using this whole time.
Explanation of how this works.
These “AI models” (meaning the free and open Stable Diffusion in particular) consist of different parts. The important parts here are the VAE and the actual “image maker” (U-Net).
A VAE (Variational AutoEncoder) is a kind of AI that can be used to compress data. In image generators, a VAE is used to compress the images. The actual image AI only works on the smaller, compressed image (the latent representation), which means it takes a less powerful computer (and uses less energy). It’s that which makes it possible to run Stable Diffusion at home.
This attack targets the VAE. The image is altered so that the latent representation is that of a very different image, but still roughly the same to humans. Say, you take images of a cat and of a dog. You put both of them through the VAE to get the latent representation. Now you alter the image of the cat until its latent representation is similar to that of the dog. You alter it only in small ways and use methods to check that it still looks similar for humans. So, what the actual image maker AI “sees” is very different from the image the human sees.
Obviously, this only works if you have access to the VAE used by the image generator. So, it only works against open source AI; basically only Stable Diffusion at this point. Companies that use a closed source VAE cannot be attacked in this way.
I guess it makes sense if your ideology is that information must be owned and everything should make money for someone. I guess some people see cyberpunk dystopia as a desirable future. I wonder if it bothers them that all the tools they used are free (EG the method to check if images are similar to humans).
It doesn’t seem to be a very effective attack but it may have some long-term PR effect. Training an AI costs a fair amount of money. People who give that away for free probably still have some ulterior motive, such as being liked. If instead you get the full hate of a few anarcho-capitalists that threaten digital vandalism, you may be deterred. Well, my two cents.
I very much doubt it even works against the multitude of VAEs out there. There’s not just the ones derived from StabilitiyAI’s models but ones right now simply intended to be faster (at a loss of quality): TAESD can also encode and has a completely different architecture thus is completely unlikely to be fooled by the same attack vector. That failing, you can use a simple affine transformation to convert between latent and rgb space (that’s what “latent2rgb” is) and compare outputs to know whether the big VAE model got fooled into generating something unrelated. That thing just doesn’t have any attack surface, there’s several magnitudes too few weights in there.
Which means that there’s an undefeatable way to detect that the VAE was defeated. Which means it’s only a matter of processing power until Nightshade is defeated, no human input needed. They’ll of course again train and try to fool the now hardened VAE, starting another round, ultimately achieving nothing but making the VAE harder and harder to defeat.
It’s like with Russia: They’ve already lost the war but they haven’t noticed, yet – though I wouldn’t be too sure that Nightshade devs themselves aren’t aware of that: What they’re doing is a powerful way to grift a lot of money from artists without a technical bone in their body.
Yeah. Not that it’s the fault of artists that capitalism exists in its current form. Their art is the fruit of their labor, and therefore, means should be taken to ensure that their labor is properly compensated. And I’m a marxist anarchist, no part of me agrees with any part of the capitalist system. But artists are effectively workers, and we enjoy the fruits of their labor. They are rarely fairly compensated for their work. In this particular instance, under the system we live in, artists rights should be prioritized over
I’m all for janky (getting less janky as time goes on) AI images, but I don’t understand why it’s so hard to ask artists permission first to use their data. We already maintain public domain image databases, and loads of artists have in the past allowed their art to be used freely for any purpose. How hard is it to gather a database of art who’s creators have agreed to let it be used for AI? All the time we’ve (the collective we) been arguing over thise could’ve been spent implementing a system to create such a database.
You should check out this article by Kit Walsh, a senior staff attorney at the EFF. The EFF is a digital rights group who recently won a historic case: border guards now need a warrant to search your phone. It should help clear some things up for you.
Fair enough, and I can’t claim to be a fan of copyright law or how it’s used. Maybe what I’m moreso talking about is a standard of ethics? Or some laws governing the usage of image and text generating AI specifically as opposed to copyright law. Like just straight up a law making it mandatory for AI to provide a list of all the data it used, as well as proof of the source of that data having consented to it’s use in training the AI.
What you are talking about is an expansion of copyright law. Copyright includes more than just the right to make copies. It also includes the right to authorize derivatives, such as translations of texts, movies based on comics, or games based on movies. Fan art is also a derivative and relies on fair use for its legality (assuming it is legal).
If one were to create an “AI training right”, then the natural place to put it, would be with the other rights covered by copyright. Of course, one could lay down such a right outside the copyright statute, and write that it is not part of copyright law.
In any case, it would be intellectual property. The person, who can allow or deny AI training on some work, would own that right as intellectual property.
Yeah, I’m not too concerned with janky AI generators having to ask before training a model on someone’s art. Sucks for them I guess.
I don’t agree with copyright. I’m an anarchist. I’m openly in favor of piracy, derivative, whatever else a human being might do with something. I don’t agree with judicial systems, let alone market economies or even currency as a concept. And that’s all fine and dandy, but there are people alive right now under capitalism. Unlike piracy, which pretty much exclusively takes from corporations like the overwhelming majority of things that are pirated are produced by corporate studios and studio funded artists, this one very specific thing takes the most specifically from artists the overwhelming majority of whom are already very poorly compensated many of them literally barely get by at all. AI models should have to ask them to copy and repurpose their works.
That’s my only statement. You can assume I effectively don’t agree with any other thing. I’m not here to have a long winded nuanced debate about a legal system I don’t agree with and am not supporting in literally any capacity. I’m pointing at pixiv the website and saying “hey can you guys like actually ask before you start using these people’s shit to make AI that is purposefully built to make sure that they are run out of jobs”
Unless you’re going to somehow explain why artists aren’t worth existing or something then don’t even bother answering. I’m genuinely not interested in what you have to say and am tired of repeating myself in this thread.
I just thought you should know where you stand on the issue. It will make it easier to communicate. Just say that you want to expand copyright to cover AI training and boom. Clear statement. No long winded, nuanced debate needed.
Don’t actually know where the hostility comes from. Are you mistaking me for someone else?
I dont want copyright to be expanded, I dont want laws governing intellectual property at all. I’ve described what I think is right pretty fully. I don’t need you to tell me where I stand.
You can read my other comments if you want to engage with it any further. I’m not mistaking you for someone else. I’m just tired of people rehashing the same endless points. Arguing with AI bros is tireless, pointlessly futile. It consistently devolves into innane nonsense. I’m fully on board with doing away with copyright as a concept entirely. My request is that artificial image and text generation be regulated in a way that is ethical with respect to small content creators who should have a say in what software their art is used to generate. That’s it fam I’m out
There’s nothing wrong with being able to use others’ copyrighted material without permission though. For analysis, criticism, research, satire, parody and artistic expression like literature, art, and music. In the US, fair use balances the interests of copyright holders with the public’s right to access and use information. There are rights people can maintain over their work, and the rights they do not maintain have always been to the benefit of self-expression and discussion.
It would be awful for everyone if IP holders could take down any review, finding, reverse engineering, or indexes they didn’t like. That would be the dream of every corporation, bully, troll, or wannabe autocrat. It really shouldn’t be legislated.
I’m not talking about IP holders, and I do not agree with copyright law. I’m not having a broad discussion on copyright here. I’m only saying, and not saying anything more, that people who sit down and make a painting and share it with their friends and communities online should be asked before it is scanned to train a model. That’s it.
How’re we supposed to have things like reviews, research findings, reverse engineering, or indexes if you have to ask first? The scams you could pull if you could attack anyone caught reviewing you. These rights exist to protect us from the monopolies on expression that would increase disparities and divisions, manipulate discourse, and in the end, fundamentally alter how we interact online with each other for the worse.
I’m just gonna ask you to read my above comment again. What I’m suggesting is:
“Before you scrape and analyze art with the specific purpose of making an AI art generator model, you must ask permission from the original creating artist.”
I read that. That’s what I’ve been responding to the whole time. This is a way to analyze and reverse engineer images so you can make your own original works. In the US, the first major case that established reverse engineering as fair use was Sega Enterprises Ltd. v. Accolade, Inc in 1992, and then affirmed in Sony Computer Entertainment, Inc. v. Connectix Corporation in 2000. Do you think SONY or SEGA would have allowed anyone to reverse engineer their stuff if they asked nice? Artists have already said they would deny anyone.
It’s not about the data, people having a way to make quality art themselves is an attack on their status, and when asked about generators that didn’t use their art, they came out overwhelmingly against with the same condescending and reductive takes they’ve been using this whole time.
o7 General Effort
Is dalle3 and midjourney using other methods?