What Would Concept Artists Want From An AI Tool?
By Neil Blevins
Created On: Sept 8th 2022
Updated On: Jan 29th 2023

A lot of the current work I've seen in AI Art Generation has been about replacing artists. But what about tools that help an artist to work faster? If the AI field could be anything a concept artist wanted it to be, what would that look like? This discussion will explore some of the things I personally would like to see from an AI tool that helps our work, as opposed to disrupting it entirely.

You have two choices with this lesson, watch me discuss the issue in the video below, or read the full text.



Automating Your Art Process

When the software 3dsmax 2 came out in the late 90s, it introduced its first scripting language, and I used it to write tools. Some people though I did this because I liked programming, and it may shock you to know I don't particularly like programming, and am not a very good programmer. But the reason I made those tools was because something I dislike even more is doing boring repetitive tasks, and writing tools gave me a way to remove the "boring" parts from my artistic workflow. I didn't want to place every single tree by hand, I wanted to define what the forest looks like, and then the software fills in the details, which I can then later tweak. And if someone hasn't already written that software, then I was going to.

So that said, I am not new to the idea of letting software assist me in handling the parts of my art process that I don't find fun. As my friend Apurva put it, "Machine Assisted Human Expression". And AI has a lot of potential to help in that area.

As I said earlier, most of the current work I've seen in AI Art Generation has been about replacing artists. This is bad in 2 ways.

First off, obviously, the monetary issue. A lot of people's careers are based on producing artwork for all sorts of products. Yes, there will always need to be some human artists, but if the number of jobs for a human artist decreases, a lot of people are going to be unable to pay their rent, unable to afford college for their kids, etc. Yes, artists will need to adapt their skillset, but if there's just less work to do, no amount of adaptation is going to replace that lost income.

The second issue is the larger philosophical one. If a computer can make artwork that looks good, why should I bother making art? I certainly agree that the thing we humans bring to the table is the why, it's the idea behind the art, it's the emotion, it's the message. And those skills are stuff that can only be replaced by an AI if the AI becomes sentient, which I think is still a very long ways away. But if an AI can make artwork that's good enough, if it replaces meaning with using the right combination of color and contrast to give the audience a dopamine hit, if quality is replaced with literally infinite quantity, if the audience no longer cares about meaning or expression just as long as the rods and cones in their eyes are excited, then we're going to see very few artists feel the need to produce anything, at least digitally. I've seen many people say "AI isn't making creative work, it's just remixing everything that's already been done." The thought that the audience cares a lot about originality is a wonderful fiction we tell ourselves, but at least in the film and videogame industry, a lot of work is about making something similar to what has already been done before. And these AI art generators seem really good at doing that.

But lets put these issues aside right now and imagine a world where these companies producing AIs wanted to use the algorithms to enhance human art instead of replace it. What would those pieces of software look like? If you could design your own personal AI tool, what would that tool do? If you could steer the direction of this technology, what direction would you steer it? Some people have said AI Art Generation is no different than the introduction of Photoshop to the industry. But right now 99% of the artists contribution to an AI image isn't that different from doing a google image search in a large dynamically created database. What if the AI software really was more like photoshop, a tool that gave the artist far more control over the images than just writing a line of text. This is what I hope to approach in this talk.

Tasks I'd Like To See An AI Perform

So let's go big picture. What parts of the art making process would I want the AI to help with? And note many of these tasks are things I'm already doing or experimenting with in AI, but there are extra options and tools that could make these tasks smoother.

1) Generate Mood Boards

This task is already covered for the most part with the current AI tools. When making a mood board either for personal use or you're an art director and want to make a design brief for one of your artists, you frequently go onto google images to find a bunch pictures that give the general mood of what you're looking for in the final piece of concept art.



It's a no brainer that if you can't find what you're looking for on Google Images, you can ask an AI to generate some art for the same purpose, not to be used directly in the concept art, but to be reference.



2) Sketch to Realistic

Whether it's a good thing or not, a lot of clients want their concept art to look realistic as opposed to sketchy. And getting something to look realistic takes a lot of time, while the rough painting can be made quite quick, and for many people is the most fun part of the process. Here's an example of one of my paintings, the rough painting on the left took me a few hours to paint in photoshop. And detailed painting on the right which includes photomanip and some 3d took an extra 3 extra days.



While I know some concept artists and illustrators love to add all the little details, for many they'd love a way to do the rough sketch and then have the AI fill in the rest. An example of software that already does this is Nvidia's Canvas, where you paint a simple sketch of different types of landscape elements like sky, trees and mountain and then it makes the results look real. Right now though this software is limited to landscapes.



Here's another image showing a rough sketch on the left and the disco diffusion AI doing its magic on the right.



So the pieces to go from sketch to realistic are there, but I think a little more work will be needed to produce a generalized tool that gets the end result I'm after in all cases.

3) Make variations of my image

After coming up with an initial design, it's quite common to make variations in order to explore different details or shapes. Here's an example of many variations of a single spacecraft.



Both midjourney AI and Dall-e have methods for making variations of an existing image. Here's an example of the same painting of mine using both software.



As I mentioned in previous videos, Dall-e seems good at making similar variations, Midjourney seems good at making very different variations, would love to have a single piece of software that's good at both with a simple sliding scale.



4) Making Photobash Elements

Most concept artists use some combination of photo elements, 3d and hand painting in Photoshop or another paint app to arrive at their final concept, a technique I like calling the digital collage. Adding AI elements in the same way we can add photo elements to a concept is another great way to incorporate AI into your images.



While we can do this using current Ai Art Generation tools, having options to create just elements, maybe with an automatic alpha channel, would be super helpful. Instead of taking a photo of a tree and trying to extract it from the background to then place it in the distance of a concept painting, it would be far quicker to ask an AI for a photoreal tree, have it generated already with an alpha, and then give it a lighting direction to match the lighting you want in your final painting (no more relighting photographs!)



5) Doing Orthos

Another task that many people don't like is making orthographic drawings of a finished concept. So another task an ai might be good at is taking a single concept image and producing orthographic views, side, top, bottom, etc of the design that the artist has made. It could compare the design you did a front view of, look through its dataset of similar designs, and guess what that object might look like from other angles.



A couple of notes about this.

First off, the current AI art generators are not great at producing images from a specific camera angle. So if you ask for a character face from a 3/4 view, you're very likely to not get a 3/4 view. So to get an AI to produce orthos, we'd need a more reliable way to specify camera angles. This might actually be pretty simple to do if someone fed data into an AI that included a lot of these keywords, from what I've seen, a lot of the current software have not prioritized these sorts of images.

Second, we'd need a more reliable way to specify if you want a background or no background on your subject. This again might be just about a better dataset.

And third, it is important to mention that traditionally, a lot of entry level concept art positions in the film and videogame industry were all about making orthos. So even if its not the most fun job to do sometimes, having it all automated with AI may drastically reduce the number of entry level jobs in the field. And if people don't get those jobs, they don't get training, and then there's no one to replace to more senior level people when they decide to move on. Of course if AI replaces most of us at every level of a company, then I suppose this phenomena won't matter because there won't be senior jobs to move into.

6) A simple way to construct a 3d model from the 2d concepts.

If we have AI that can make orthos, then its likely we could have an AI that could use the same technique to produce at least simple 3d models from a 2d image. I did this manually for a test a few months back, but a completely automated way would certainly speed up the process, and be helpful at producing rough models to pass to the next department in your pipeline.



Other Wishlist Items

These last things aren't exactly tasks I'd like an AI to do, but more changes to the AI Art Generator ecosystem to facilitate their usage in producing art for the concept field.

7) Training the AI on a dataset that contains more concept art (artists have to OPT IN to get their work included in the dataset, artists shouldn't be forced to OPT OUT)

8) The ability to train the AI model on your own images, which can be kept locally and private. Basically the ability to create your own digital apprentice. I've seen a few examples of this already in the testing phase, but hasn't officially rolled out yet.

9) Provide datasets that are guaranteed to not include copyrighted material. And providing better licensing that gives more rights to the final images to the concept artist or studio using the AI. Right now these AI images live in a very legal grey area, and as such you basically can't use them for large companies. Any videogame, film or book company is going to need copyright over their imagery, so a lot of work need to be done on the legal side before it's appropriate to use AI art in this capacity.

Conclusion

So in closing, these are some of the things I'd like to see from an AI art assistant. In short, I'd love AI to help me do all the art making parts that I find boring. Of course, everyone has different elements of the art making process they find boring or exciting, by no means are all artists the same. So what are some of your wishlist items? The area of concept art is very broad, and the kind I do is a small fraction of it, so I'd love to hear from concept artists working in other fields. If you could have an AI that helped speed up your workflow instead of replace it, what would you wish for?

This site is ©2023 by Neil Blevins, All rights are reserved.
NeilBlevins.com BloggerTwitterFacebookLinkedIn TumblrArtStationKickstarterGumroad YouTubeIMDB