What Would Concept Artists
Want From An AI Tool?
By Neil Blevins
Created On: Sept 8th 2022
A lot of the current work I've seen in AI Art Generation has been
about replacing artists. But what about tools that help an artist to
work faster? If the AI field could be anything a concept artist wanted
it to be, what would that look like? This discussion will explore some
of the things I personally would like to see from an AI tool that helps
our work, as opposed to disrupting it entirely.
You have two choices with this lesson, watch me discuss the issue in
the video below, or read the full text.
Automating Your Art Process
When the software 3dsmax 2 came out in the late 90s, it
introduced its first scripting language, and I used it to write tools.
Some people though I did this because I liked programming, and it may
shock you to know I don't particularly like programming, and am not a
very good programmer. But the reason I made those tools was because
something I dislike even more is doing boring repetitive tasks, and
writing tools gave me a way to remove the "boring" parts from my
artistic workflow. I didn't want to place every single tree by hand, I
wanted to define what the forest looks like, and then the software
fills in the details, which I can then later tweak. And if someone
hasn't already written that software, then I was going to.
So that said, I am not new to the idea of letting software assist me in
handling the parts of my art process that I don't find fun. As my
friend Apurva put it, "Machine Assisted Human Expression". And AI has a
lot of potential to help in that area.
As I said earlier, most of the current work I've seen in AI Art
Generation has been about replacing artists. This is bad in 2 ways.
First off, obviously, the monetary issue. A lot of people's careers are
based on producing artwork for all sorts of products. Yes, there will
always need to be some human artists, but if the number of jobs for a
human artist decreases, a lot of people are going to be unable to pay
their rent, unable to afford college for their kids, etc. Yes, artists
will need to adapt their skillset, but if there's just less work to do,
no amount of adaptation is going to replace that lost income.
The second issue is the larger philosophical one. If a computer can
make artwork that looks good, why should I bother making art? I
certainly agree that the thing we humans bring to the table is the why,
it's the idea behind the art, it's the emotion, it's the message. And
those skills are stuff that can only be replaced by an AI if the AI
becomes sentient, which I think is still a very long ways away. But if
an AI can make artwork that's good enough, if it replaces meaning with
using the right combination of color and contrast to give the audience
a dopamine hit, if quality is replaced with literally infinite
quantity, if the audience no longer cares about meaning or expression
just as long as the rods and cones in their eyes are excited, then
we're going to see very few artists feel the need to produce anything,
at least digitally. I've seen many people say "AI isn't making creative
work, it's just remixing everything that's already been done." The
thought that the audience cares a lot about originality is a wonderful
fiction we tell ourselves, but at least in the film and videogame
industry, a lot of work is about making something similar to what has
already been done before. And these AI art generators seem really good
at doing that.
But lets put these issues aside right now and imagine a world where
these companies producing AIs wanted to use the algorithms to enhance
human art instead of replace it. What would those pieces of software
look like? If you could design your own personal AI tool, what would
that tool do? If you could steer the direction of this technology, what
direction would you steer it? Some people have said AI Art Generation
is no different than the introduction of Photoshop to the industry. But
right now 99% of the artists contribution to an AI image isn't that
different from doing a google image search in a large dynamically
created database. What if the AI software really was more like
photoshop, a tool that gave the artist far more control over the images
than just writing a line of text. This is what I hope to approach in
this talk.
Tasks I'd Like To See An AI
Perform
So let's go big picture. What parts of the art making process would I
want the AI to help with? And note many of these tasks are things I'm
already doing or experimenting with in AI, but there are extra options
and tools that could make these tasks smoother.
1) Generate Mood Boards
This task is already covered for the most part with the current AI
tools. When making a mood board either for personal use or you're an
art director and want to make a design brief for one of your artists,
you frequently go onto google images to find a bunch pictures that give
the general mood of what you're looking for in the final piece of
concept art.
It's a no brainer that if you can't find what you're looking for on
Google Images, you can ask an AI to generate some art for the same
purpose, not to be used directly in the concept art, but to be
reference.
2) Sketch to Realistic
Whether it's a good thing or not, a lot of clients want their concept
art to look realistic as opposed to sketchy. And getting something to
look realistic takes a lot of time, while the rough painting can be
made quite quick, and for many people is the most fun part of the
process. Here's an example of one of my paintings, the rough painting
on the left took me a few hours to paint in photoshop. And detailed
painting on the right which includes photomanip and some 3d took an
extra 3 extra days.
While I know some concept artists and illustrators love to add all the
little details, for many they'd love a way to do the rough sketch and
then have the AI fill in the rest. An example of software that already
does this is Nvidia's Canvas, where you paint a simple sketch of
different types of landscape elements like sky, trees and mountain and
then it makes the results look real. Right now though this software is
limited to landscapes.
Here's another image showing a rough sketch on the left and the disco
diffusion AI doing its magic on the right.
So the pieces to go from sketch to realistic are there, but I think a
little more work will be needed to produce a generalized tool that gets
the end result I'm after in all cases.
3) Make variations of my image
After coming up with an initial design, it's quite common to make
variations in order to explore different details or shapes. Here's an
example of many variations of a single spacecraft.
Both midjourney AI and Dall-e have methods for making variations of an
existing image. Here's an example of the same painting of mine using
both software.
As I mentioned in previous videos, Dall-e seems good at making similar
variations, Midjourney seems good at making very different variations,
would love to have a single piece of software that's good at both with
a simple sliding scale.
4) Making Photobash Elements
Most concept artists use some combination of photo elements, 3d and
hand painting in Photoshop or another paint app to arrive at their
final concept, a technique I like calling the digital collage. Adding
AI elements in the same way we can add photo elements to a concept is
another great way to incorporate AI into your images.
While we can do this using current Ai Art Generation tools, having
options to create just elements, maybe with an automatic alpha channel,
would be super helpful. Instead of taking a photo of a tree and trying
to extract it from the background to then place it in the distance of a
concept painting, it would be far quicker to ask an AI for a photoreal
tree, have it generated already with an alpha, and then give it a
lighting direction to match the lighting you want in your final
painting (no more relighting photographs!)
5) Doing Orthos
Another task that many people don't like is making orthographic
drawings of a finished concept. So another task an ai might be good at
is taking a single concept image and producing orthographic views,
side, top, bottom, etc of the design that the artist has made. It could
compare the design you did a front view of, look through its dataset of
similar designs, and guess what that object might look like from other
angles.
A couple of notes about this.
First off, the current AI art generators are not great at producing
images from a specific camera angle. So if you ask for a character face
from a 3/4 view, you're very likely to not get a 3/4 view. So to get an
AI to produce orthos, we'd need a more reliable way to specify camera
angles. This might actually be pretty simple to do if someone fed data
into an AI that included a lot of these keywords, from what I've seen,
a lot of the current software have not prioritized these sorts of
images.
Second, we'd need a more reliable way to specify if you want a
background or no background on your subject. This again might be just
about a better dataset.
And third, it is important to mention that traditionally, a lot of
entry level concept art positions in the film and videogame industry
were all about making orthos. So even if its not the most fun job to do
sometimes, having it all automated with AI may drastically reduce the
number of entry level jobs in the field. And if people don't get those
jobs, they don't get training, and then there's no one to replace to
more senior level people when they decide to move on. Of course if AI
replaces most of us at every level of a company, then I suppose this
phenomena won't matter because there won't be senior jobs to move into.
6) A simple way to construct a 3d model from the 2d concepts.
If we have AI that can make orthos, then its likely we could have an AI
that could use the same technique to produce at least simple 3d models
from a 2d image. I did this manually for a test a few months back, but
a completely automated way would certainly speed up the process, and be
helpful at producing rough models to pass to the next department in
your pipeline.
Other Wishlist Items
These last things aren't exactly tasks I'd like an AI to
do, but more changes to the AI Art Generator ecosystem to facilitate
their usage in producing art for the concept field.
7) Training the AI on a dataset that contains more concept art (artists
have to OPT IN to get their work included in the dataset, artists
shouldn't be forced to OPT OUT)
8) The ability to train the AI model on your own images, which can be
kept locally and private. Basically the ability to create your own
digital apprentice. I've seen a few examples of this already in the
testing phase, but hasn't officially rolled out yet.
9) Provide datasets that are guaranteed to not include copyrighted
material. And providing better licensing that gives more rights to the
final images to the concept artist or studio using the AI. Right now
these AI images live in a very legal grey area, and as such you
basically can't use them for large companies. Any videogame, film or
book company is going to need copyright over their imagery, so a lot of
work need to be done on the legal side before it's appropriate to use
AI art in this capacity.
Conclusion
So in closing, these are some of the things I'd like to see from
an AI art assistant. In short, I'd love AI to help me do all the art
making parts that I find boring. Of course, everyone has different
elements of the art making process they find boring or exciting, by no
means are all artists the same. So what are some of your wishlist
items? The area of concept art is very broad, and the kind I do is a
small fraction of it, so I'd love to hear from concept artists working
in other fields. If you could have an AI that helped speed up your
workflow instead of replace it, what would you wish for?
