Paint 3D
Sketches In Tilt Brush By Neil Blevins Created On: July 2nd 2016 Software: Tilt Brush
So a good friend of mine let me test out Google's new Tilt Brush
software, a 3d
painting program on the HTC Vive virtual reality device. Like a 2D
paint program, you paint with various brushes. However, you are
painting in 3 dimensional
space. What's more, you are inside the 3 dimensional space using your
vr goggles, rather than seeing the 3d space through a viewport on a 2d
screen. So unlike working in 3d in say 3dsmax on your computer, you
can walk around your model with the ease of walking around a real life
object.
This lesson isn't really a tutorial on how to use Tilt Brush, it's
more a discussion of the process I went through, and some discussion of
my views on the future of the technology.
Painting In Virtual Reality
Once you have the vr goggles on and pick up the two hand
controllers, you are now in a empty room, except for a faint horizon
(the default environment for painting, but you can choose others). Your
right hand controller is the paint brush, and the left is an artist
palette, much like an old school artist palette where you mix your
paint. This palette though has many of the features one would expect
from a 2d app like photoshop, paint brushes, color picker, eraser, etc.
I painted the space battle scene above using primarily 4 brushes...
Flat Marker: A fully opaque marker was used for the ships, with
controls for size, and also using the straight line function where you
specify a start point and an end point, and it draws a brush stroke
between those two points. That way you don't get wobbly strokes for the
hard surfaces of the spaceships.
Glow: Similar to the marker, but it glows, used for the gun shots
and the engines.
Highlighter: Doesn't get shadowed and is partly transparent,
used it for the engine exhaust and the explosion.
Smoke: Paints opacity mapped cards that look like smoke, for the
explosion.
I also used the following tools...
Eraser: You get a sphere, and any brush stroke that intersects
with that sphere gets deleted
Straight Line: Makes every stroke a straight line
Symmetry: For the first spaceship, I turned on symmetry mode so
that I could paint one side and it would automatically paint the other
side
Brush Picker: You select a stroke you've already made and it
copies the color and brush type, and makes that your active brush
Viewing The Painting In Virtual
Reality
Once I finished the 3d painting, I took a number of screenshots, where
I
walked around the battle and used a camera feature to take snapshots
just like I would do if I were using a real camera and walking around a
practical model. Here's the results:
This was a great way to choose exciting camera angles on my scene, far
quicker and more intuitive than doing so in a standard 3d application.
I then recorded myself walking around the 3d painting, seeing it from
all
sorts of angles (even inside). Because the object feels very real, I
noticed myself trying to avoid bumping into the object, even though I
could walk right through it without any issues,
since it's not real.
Then I save the painting in the Tilt Brush format. Later, if I load the
painting back up, it loads by painting it step by step, which is a
great way to see the painting progression.
Exporting The Painting
Another feature is the ability to export the 3d painting
to an fbx file.
While you are using paint brushes to "paint" the painting, what you're
actually doing is constructing 3d splines that have width, like ribbons
in space. So using the fbx export, you can export the geometry of the
painting to a full
featured 3d application like 3dsmax or maya. Here's a render of the
scene
in 3dsmax using a shader to see the polygonal edges. Note it exports
tris and not quads.
Click this image to see it larger...
You can shade these objects just like in Tilt Brush, so it wouldn't be
that difficult to replicate your Tilt Brush scene inside your 3d app of
choice. Or replace the meshes with more detailed meshes, using the Tilt
Brush geometry as templates, now that you've worked out the overall
composition.
The Future
I see so many applications for this sort of software in the future.
Certainly one thing is to have fun painting, but I also think it has
many applications to the kind of work many of us computer graphics folk
do every day in the entertainment or design field, especially as a
collaborative tool. Here's an example.
Say I'm working on an animated film. I am a modeler, and we need a room
for a scene we're going to have in our film. I invite the director of
photography (the person in charge of the camera) and the director (in
charge of the story) to a room. All 3 of us put on vr
goggles, and go into the same virtual room. We all have the ability to
paint with hand controllers. So I start, I make a 3d sketch of the room
at a scale of say 1 foot by 2 foot, as though it was a dollhouse in
front of me. I sketch the floor and 4
walls, and some quick furniture like chair, a table and a floor lamp.
None of this needs to be super
detailed, all I need is to communicate scale, proportion and position
of the room and interior objects.
I finish my doodle, then the director says "Don't forget, we need a
doorway since our main character will be walking into the room. The
director carves a doorway into my wall. I say "How high does the door
need to be?" I import into the scene a representation of one of our
characters from the
film, and move the character beside the door. The director says
"Since our character is taller than a standard human, we'll probably
need the doorway to be a bit bigger than normal. He carves a larger
doorway to accommodate the character. The director says "I think we
should have him looking out a window once he enters", so I carve a
window into the
side of the room. "What will be out the window" I ask. The director
says "This takes place in a city". I import a city model, and place it
outside the room. "We'd better check that we see the city properly
though the window?"
The Director Of Photography takes the scene and scales it up to be life
sized, and looks through his virtual camera attached to his hand. He
changes lenses to whatever lens he think would work best in the scene.
I move the character towards the window. The Director Of Photography
says "That window should work, I'm seeing the buildings outside,
and the character is nicely placed in the foreground. Can we move that
chair
out of the way though? It's sticking in the left of the shot!". I walk
over to the chair and move it out of the way. "Great". Now we shrink
the scene back down to dollhouse size. "I think we're done" says the
director. I export the result of our painting to fbx. I then go
back to my desk, import the model into maya, it's rough, but it has
everything I
need to now start modeling the real chairs, the real room, etc using
more traditional 3d software.
Now you can certainly do these interactive collaborative sessions right
now using regular 3d software, but its a lot harder, everyone is
crowed around a small monitor, only one person has control of the
keyboard and mouse so there's lots of hand waving and "can you just
move this here, no a little more, no a little more". Having everyone in
the same space and giving everyone the ability to modify the scene
would allow for far better communication, which would lead to a much
better workflow than what we currently have.
Wishlist
Here's a short wishlist of stuff I'd love to see added to the software,
much of which would allow the workflow I outlined above
Ability to scale and shrink the world interactively
Ability to move brush strokes. You could have a resizable sphere
of influence attached to your hand controller, and any brushstroke
inside that sphere can be grabbed and
moved to a new location
Multiple users in the same virtual space painting together
Quad fbx export
Conclusion
This is only the tip of the iceberg, I'm sure we'll see newer versions
of tilt brush, and other 3d painting and sculpting apps appear on the
vive and other vr devices in the near future. I'm super excited,
hopefully you're excited too!
