The use of AI for backgrounds

I'll do a status update next week but I need to rip off this bandaid. I've been agonizing for weeks about this.

I’ve decided to use AI art for background art (again?). I want to stress this is for BACKGROUND art only!

All Patreon funds, profits from Obsession, and a lot of my pocket money will be going into CGs and other human-generated assets.

For those who don’t care, you can skip the rest. (Or scroll to the end for a laugh at bad AI art.) Nothing else important here. Thanks for stopping by.

For those who do care, I ask that you please hear me out. I want to explain the reason I’ve made this choice. I’ve done a lot of reading and put a lot of thought into this.

I included this choice in a post some months ago, got a bit of pushback, and then had a CG artist volunteer to help with the 70+ backgrounds I need for Remembrance. So I reconsidered, dropped the idea, and deleted that post content.

Unfortunately, that artist withdrew for personal reasons quickly. (It’s a lot of work to do 70+ backgrounds for very little payment, so I was hesitant to even accept the offer to start with, but they insisted despite my raised concerns.)

After the artist withdrew, I looked around to try to figure out another solution that was both affordable and a good enough quality. And… there just isn’t one. The quality suffers when the price goes down for obvious and understandable reasons. But AI art… it’s $40 for unlimited high quality custom commercial art.

I don’t want anyone who has an objection to feel like their money was stolen or supporting something they disagree with or feel like they were tricked, so I’m letting everyone know now.

But I do want to outline the reasons I’m pursing this again, because I do understand the objections raised. I’m not writing all this out to make ragebait or start arguments, I just want to explain my perspective and why I’ve decided to pursue this.

• I don’t want my funds to go to AI!

It won't!

To start with, it was $40 and much of the AI generation work is already done. This is my money I spent, I'm not including it in my expense tracker. But that aside…

I mentioned this in my previous post and I’ll commit to it again. I have EVERY intention of spending 100% of the Patreon money, the profits from Obsession, AND a lot of my pocket money on top of that into Remembrance. The change is WHERE those funds are being spent.

Background art honestly just… isn’t important outside of some key pieces.

Obsession was proof of that. I used a LOT of stock images and freeware vector art when I ran out of funds. I hated it. But… no one really cared. There were zero comments/reviews that mentioned the BG art. Why waste money on something most people don’t care much about?

On the flip side, what everyone DOES care about is CGs. And the quality of CGs was something I got comments about. I hated that too, but it was all I could afford at the end.

I plan to spend an equal amount of money on Remembrance as I did on Obsession. I kept track of every penny I spent.

In Obsession, I spent about $4k on background art. And only got about ~50% of the pieces I wanted.

When I first started Obsession, I wanted 1 CG art PER ENDING. Every route had 10+ endings. CGs are generally around $100 a piece. So $100 x 10 endings x 6 routes = 6k. I wanted to do more CGs with Victor but I had to limit it a lot due to funds and I already didn't love the art. I would have loved to have CG art in the side content too, but that wasn’t an option.

I spent $3k on sprite art. Probably the best use of my money, the sprite art is great. I spent other money on some failed art attempts, the trailer for the game, and other miscellaneous things. That’s $13k minimum I would have spent if I’d kept to the original plan, $17k if I’d pursued all the background art. I was already really pushing it (and my relationship!) and was wayyyyyyy outside of my initial budget with the $10k I spent.

So… I had to cut CGs. That’s the reason why Sebastian’s route has a dozen CGs and every other route has 5. Seb’s route was the original plan.

Using AI art, I can and will take that $4k on background art and spend it on CGs.

I’ve barely written the intro into Remembrance and already have 2 CGs being worked on and am planning a couple more just for the common route. Remembrance will have a lot more CGs than Obsession due to this choice.

ART IS EXPENSIVE. Good art is even more expensive.

Since I have the profit funds from Obsession, I’ve also added additional costs like voice acting, more sprites like Gabriella, Josh, and David, and an editor.

• It’s unethical! It’s theft!

Sort of? But not exactly…

AI art programs do not collect and store endless amounts of images to ‘copy’ when it generates its art. (That would take a massive amount of servers and cost billions and performance times would be awful.)

What AI art does it is scans a large quantity of existing images (yes, some of them with copyright), to aggregate the pixel data of patterns.

Imagine a stool. 99% of images of stools you see, regardless if they are art or photos or any particular style, are the same. A stool is typically wooden, has 4 pillars as legs and a circular top. There’s only so many ways you can draw a stool before it stops becoming a stool. AI art scans a billion images, somehow (by magic I don’t understand yet) associates those patterns with the word ‘stool’, and saves that pattern I just described (in data of pixels).

The same is true with art styles. It aggregates all that data to identify those patterns.

Some people may disagree with this opinion, but in my opinion… that’s the exact same way a human learns language and art.

We all know what an ‘anime’ art style is because we’ve seen countless anime works. If we’d never seen any of them, we’d have no idea. But you wouldn’t say Quinn is ‘stolen’ art because he is in an existing art style that I did not invent. I invented his character, his personality, and his collection of physical features (hair color, eyes, clothes).

The one argument I do hear against this creative aspect of the tool that I understand is that AI can’t (currently?) invent new art styles like humans can. It can only repeat what it knows. And yeah, that’s true. But again… it’s also true with most humans. Most of us don’t invent our own unique art styles, we adapt to art styles we’ve seen. As I’ve done with my game’s anime art style.

And you and I know about those art styles because we’ve seen countless (copywritten!) works.

It’s my opinion that if it’s the way humans learn, it makes sense for a machine to learn the same way, and therefore its not theft. It’s not using the images directly, it’s using aggregated data patterns.

• Artists didn't give consent!

There’s also the argument that these artists didn’t give their permission for their work to feed the learning algorithms of the machines. Yes, that’s true. But again I come back to the human method of learning. I know what anime art is because I've seen copywritten works of anime.

By putting their work on the internet, they consented to millions of human eyes seeing the work for free, and with the understand that work could be used as reference material and learning material by humans without payment or individual permission for personal use (they wouldn’t even know it’s happening).

The vast majority of artists google images for reference material. Not to copy but to learn and correct their mistakes. This is advice given on any YouTube video about learning how to draw. It's a mantra. Always use a reference.

But all that aside...

If the AI developers had respected that argument from the start, it doesn’t change anything but time. It would have slowed down the learning and development of AI, but not prevented it. We’d be right back to where we are today sooner or later.

• The cat’s out of the bag.

There’s no putting pandora back in the box at this point. AI is here, and it’s here to stay, and no amount of protesting or boycotting is going to permanently stop it. Only delay it.

And from my research… AI has actually been around a lot longer than most of us realize. We have all probably consumed some AI content without knowing it already. It just exploded in PUBLIC popularity recently.

On the extreme, even if Europe/US made it illegal, it’s not going to stop AI. They would move offshore, same as VPNs or piracy. But I doubt it would be made illegal despite any moral arguments because it’s simply too profitable of a tool for businesses. And (at least in the US) capitalism and money trumps everything else for better and worse.

AI now exists not just for 2D art, but for writing, for voice, for 3D models, and probably more I don’t even know about yet.

And the fact of the matter is AI art is good enough that you won’t often know it when you’re looking at it. From my experiments with AI art, it’s absolutely not perfect, but the quality it produces is amazing and impressive. You currently have to stop and LOOK for the flaws to identify if its AI generated. And those programs are only going to improve.

As a random example of its use in life today...

Teachers in school are already facing this problem every day now. Some students are using AI to write their papers and complete their projects. Teachers are desperately looking for scanners that will tell them if something is AI generated… and those scanners are unreliable. It’s also dangerous territory. Do you want to fail a student who put in a ton of effort and wrote a high quality paper if you’re not completely confident it’s AI?

Students are easily getting away with using it (the smart ones anyway who review the work and fix its mistakes and awkwardness). The existence of AI is going to force teachers to change the way they teach and grade. But on the flip side, AI can probably (and probably already does) help teachers with that, too. Teachers spend so much of their personal time on their work… reducing time with something like an AI lesson plans or quiz generation could be a good thing.

A lot has to change for us all to adapt. AI brings a big social change, too.

• AI is evil! It’s ruining everything! Taking our jobs!

All technology is both good and evil.

Librarians lost jobs when card catalogues became computerized. Newspaper printing companies lost jobs when the internet existed. Manufacturing lost jobs with the advent of machines. Cashiers lost jobs with self-checkout lanes. People are overworked with excessive performance expectations due to tech.

On the flip side, the internet is pretty awesome. My game exists because of the internet. Those machines lowered costs. Self-checkout is great for those with autism or social anxiety. The internet helped advance social change (arguably for both good and bad). It’s so easy to learn new things with the internet. Youtube taught me how to do laundry, no joke. (My parents sucked.)

We could spend days outlining all the positives and negatives of every technology. AI is no different.

Yes, AI really does suck for artists. I completely agree and I feel for them. I had an interest in doing art myself, but I lack the skill and talent to do so. (Some of you saw my initial sprites. The horror!) Artists are the ones who are going to be hit the hardest by this tech. And art was already a hard if not impossible career path.

But they are being hit the hardest because art is expensive and time-consuming and produces variable results based on the artist. (Look at the wide range of quality and styles in Obsession. Finding multiple artists on a budget with the same style is impossible.)

AI addresses all of those problems. It’s quick and cheap and consistent. And if it doesn’t give me what I want, I can try again and have completely different results within a literal minute. Changes and retries takes weeks of back-and-forth communication with an artist.

AI does open the door for a lot of new things, too. And I think visual novels are a great example of that.

For Obsession, I spent ~10k, most of that out of pocket. (It was a big problem with my partner at the time!) Few people have that kind of personal cash to invest in a risky passion project.

AI means a single person can create an entire game quite cheap. And removing financial roadblocks and decreasing risk for game developers (and other projects) is a positive. It means more games for us. It means more really wild and risky games like Obsession can be developed. Games no publisher would touch.

I’m not arguing against the negatives. I agree there are a lot of negatives. I’m only pointing out that there are also some positives.

• Liar! You’ll use this for CGs & stuff too!

I won’t say I’ll never use it for CGs in my life, but I will commit to not use it for CGs (or anything other than backgrounds) with Remembrance. And the proof of that is in the sneak-peek art previews of in progress work and my PayPal receipts.

But there are also a few practical reasons you can trust I won’t use it for CGs.

A) AI art sucks in character consistency.

Reproducing the same character with the same features in the same clothes in different backgrounds and environments and poses is just not currently possible.

This is something the tools are working to improve. And probably will one day. Midjourney, for example, recently introduced some features around this. But even with those features, it still sucks. Check out the image at the end of this post. Why on earth does Victor have a hair tie…? Why does he look feminine…? These were not a part of the prompts.

These images were generated with the raw sprites for Quinn and Victor provided. I literally gave the AI the exact art to use and it cannot reproduce those characters. Victor is close, but it’s not good enough. Quinn is way off.

B) AI sucks in details.

It’s great in broad, large, generic images. Like a picture of a bedroom or building.

It’s horrible at specific unique details, like the various objects that exist in a kitchen or grocery store. Kitchen and bathroom faucets come out looking warped and jagged. Coffee machines and stove tops get blurry and shaped wrong.

The smaller and more important the details are, the worse the image.

Trying to get an AI to generate something as specific and unique as Quinn’s split-color hairstyle seems impossible right now. Much less to have it done repeatedly and identically across numerous images.

And when it comes to CGs (especially for some scenes in Remembrance) those details are really, really important. Some critical to the plot.

Eventually, AI will probably get there. But it’s going to take time.

C) As I’ve said, I do enjoy working with artists and I don’t want the additional workload. BGs is a huge time sink I am not enjoying.

D) One person suggested I used AI in Obsession for the writing. AI wasn’t even a thing I knew about during its development (omgwtf). WRITING IS THE PART I LIKE! So please don’t even think that. Writing is the entire reason I made this game. I wanted to be a professional writer my entire life. If I didn’t enjoy writing this game wouldn’t exist. It wouldn’t be a passion project. It’d be work. A job. No thanks. (But yes… I realize AI can replace me as a writer, too. AI can write fiction. I’m very aware of that.)

• You’re just being lazy!

Lazy…. oh my god no.

As mentioned previously, AI sucks at details. It does not produce flawless work. I have spent literal days generating images, tweaking large failures and regenerating, and then fixing these smaller details in photoshop.

Using AI art is !A LOT OF WORK FOR ME! OMG SO MUCH WORK! It’s so boring and frustrating and tedious and I need 70 of these.

Using an artist means I can write a paragraph or two, wait, exchange some change requests, wait, then boom, I have completed art. I can focus on other stuff, like writing and programming.

And I’d really, really, would prefer to be writing right now.

I’d prefer to use real artists. I really would. It takes a huge workload off my shoulders. But the costs are so high.

• Didn’t you say it’s faster…? You’re contradicting yourself!

I’m talking about the total timescale. An background artist takes typically in my experience about 1.5 weeks per image, assuming there’s no changes needed. Which there always are.

The timespan for me to generate a useable image is a few minutes. I could literally say f-it right here, say good enough, and be done. But as I’ve mentioned, the problems in the details stand out if you look for them.

And once you see the patterns about where AI often fails, it gets really obvious. And annoying. It's distracting.

The time to fix all the AI problems in the details of that usable image can be hours per image. It’s still faster. It’s just MY time instead of someone else’s.

Fixing the issues in an image can be 2-6 hours depending on the level of detail. I’m also not a photoshop expert or an artist. I can’t just… redraw the failed parts myself. I suck at drawing, it’s obvious. And the AI re-generation of those failed parts often makes things worse, too.

So I have to search for all the failures like a hidden object game and fix them by hand with photoshop. It’s extremely time consuming and tedious per image for me. But I can do multiple images in one (long) day.

But also speed isn’t the primary reason for this decision, it’s just a bonus.

• But… you didn’t use AI for Obsession?

I’m incredibly grateful Obsession was a success despite its problems, but the art was a problem and hurt the overall game a lot. And that was really frustrating to have to settle for low quality or replacement art.

The lack of CGs was a huge negative that received complaints.

The styles of all of the CG art were inconsistent and the quality variable, clearly decreasing over time due to affordability.

The backgrounds were widely variable in styles and often didn’t fit the scenes. Some scenes I literally had to change to fit the background. Some scenes just… don’t have backgrounds.

Julian’s a great example of absent backgrounds. The hallway to Julian’s warehouse basement. Julian’s torture room. The warehouse room that lead to his hallway with the secret entrance. All of those are backgrounds I needed and were super important in the story, couldn’t afford, and couldn’t find replacement freeware art to match the scenes.

Trying to find freeware backgrounds that fit a scene is also a nightmare, even more time consuming than fixing the AI art. At least with AI art, I won’t spend hours and fail to get anything for my effort.

-

And as I’ve said before, I really would prefer to use artists for many reasons. I’d like to support artists over corporations, I’d like to take this workload and headache off my shoulders, I’d like to spend my time on other aspects of the game. I honestly have enjoyed working with all the artists on the game. I like that the project is collaborative and that other people get invested into it. It’s really exciting to have other people invested when they get interested in the overall game.

If I had unlimited resources, I’d absolutely choose artists.

But I think this is the best move for the game… and at the end of the day, it’s my game that I’m investing my time and money into, my passion project, my ‘vision’ if you will be cliché. I want to make the best game I can and it’s my choice in the end.

When I started Particles… I didn’t expect to make a single penny from it.

But of course, it’s your choice whether or not to support the game. And if anyone chooses to leave due to this choice, I understand and I want to say thank you for all the support you’ve already given. Thank you for playing the game. Thank you for helping my dream become reality.

-

So, that’s my side of the debate and thanks for listening to my rambling ted-talk.

Here's the funny character AI experiments (that I will not be using, this was for laughs only):

Victor and Quinn (they're supposed to be yelling at each other):

Alexander........