|
|
|| Adventure Links || Archives || Articles || Independent Developers || Interviews || JA Forum ||
|| JA Staff/Contacts || The JAVE || Letters || Reviews || Search || Upcoming Releases || Walkthroughs ||
|| What's New / Home || Play Games! ||
| Over 1 Million Visitors a Month! |   |
|
Articles
|
Brief
Glimpse into the Future of 3D Game Graphics |
On September 19, 2002 I
had the privilege (and good luck) of attending "ATI Mojo Day" conference
in downtown San Francisco. This event was aimed in particular at 3D
game developers, with a number of presentations explaining bleeding
edge 3D graphics technologies and showing them off on ATI's latest
and greatest graphics card, Radeon 9700 Pro. Developers from companies
like Blizzard, Bioware, Ubi Soft or Interplay were seen at the event.
I am not a game developer myself but I do have a little bit of
background in 3D graphics so I could at least follow what the presenters
were talking about. And I was quite impressed - both by the mostly
very competent speakers and the extraordinarily powerful hardware.
It is not much an exaggeration when I say that I've seen the future
of 3D game graphics and taken a sneak peek at some of the things that
we'll be seeing in 3D computer games in the following months and years.
This article is by its very nature rather technical. I will attempt
to explain some of the latest buzzwords such as pixel shaders, vertex
shaders, tessellation, bump mapping and so on - and I will try to
provide examples of how these technologies can be employed to create
better looking games (not better games - no technology can do that).
It is probably best if I follow the history of 3D games and explain
the technologies as they began to be used over the years.
I am not sure what the first 3D game was and when it appeared
but I'm certain it was well before 1990, and probably before 1985.
I have seen 3D games on 8-bit micros such as the famous Commodore
64. The first 3D games were usually flight sims or space shooters
- it is just difficult to make a flight sim that's not 3D.
One of the games I remember was Elite: the graphics were not
exactly impressive by today's standards, all objects were just wireframe
models. But they were 3D, they could move, rotate and so on. That
was about all the 8-bit machines could do.
In late 1980's the Amigas, Atari STs and PCs entered the gaming scene.
These machines were 16-bit and had a lot more computing power (still
almost nothing by today's standards). Instead of wireframe models
the games now used objects composed of flat polygons, giving
them a solid look.The polygons were usually in their simplest form,
that is triangles. A 3D object would consist of tens or maybe hundreds
of triangles. Usually tens. Each polygon had a color associated with
it and it was often shaded. In every 3D scene there is one
or more sources of light (if you want to see anything that is) and
the polygons change their shade depending on their position relative
to the light source(s).
There are several methods of shading. I will explain them because
they have become important again in today's 3D graphics.
Most basic is flat shading. The entire polygon is uniformly
colored and the color depends on the polygon's orientation, or in
tech speak, on the orientation of its surface normal relative to the
directional vector of the light source(s). Flat shaded models don't
look terribly good and they are very angular. Creating flat shaded
objects that have a rounded look is next to impossible (and somewhat
useless). But flat shaded 3D graphics has one major advantage: it
is fast. Drawing a uniformly colored polygon is something even a 286
can do reasonably well and without any hardware support. Quite a few
games made in early 1990's used flat shading, one of my personal favorites
was Chuck Yeager's Air Combat. These games weren't stunningly
beautiful but they ran with decent speed on modest hardware and gave
the player pretty good 3D feel.
The next step is so called Gouraud shading. Here a polygon
no longer has uniform color. The light intensity is calculated separately
for each vertex of the polygon and linearly interpolated across the
surface of the polygon. With this method it is possible to create
3D models that actually look round. The computational intensity is
quite a bit higher but still doable purely in software, given a 486
or Pentium class PC.
The most advanced method is called Phong shading. Here the
light intensity is calculated separately for every pixel, allowing
for near perfectly round objects but requiring a lot of computational
horsepower. I am not aware of any game that actually used Phong shading
in software, and it is rarely used even with hardware support because
Phong shading in most cases does not give significantly better results
than Gouraud shading but tends to be noticeably slower.
The differences in shading models are perhaps best illustrated by
the following pictures which all show the exact same sphere model
rendered with different lighting models:
The sphere on the left uses flat shading. It doesn't look very spherical
because the human eye is uncannily good at picking up the differences
in shade. Notice how each polygon is uniformly colored and "flat".
The middle sphere employs Gouraud (or "smooth") shading and you can
clearly see that it looks far better - almost real. On the right is
a sphere that shows Phong shading: even better than Gouraud shading.
It is worth noticing that no matter how good the shading is, you can
still discern the angly edges of the sphere. Note: the rightmost sphere
is not rendered using true Phong shading, just approximated with Gouraud
shading and finer mesh. But it serves well for the purpose of illustration.
The Big Thing in mid-1990s was texture mapping. Texture mapping
is a method of applying a "skin" on the flat polygons. The texture
is a essentially a color 2D bitmap. Textures are in some instances
called decals and that is very descriptive: you take a flat surface
(polygon) and slap a picture (texture) onto it. Texture mapping is
relatively computationally intensive as it requires a lookup into
the texture map for every pixel of the output image. It can be combined
with shading to provide lighting effects.
Quake or Descent are good examples of 3D games that
used texture mapping purely in software (Doom or Duke Nukem
weren't really 3D). Doing texture mapping in software has the obvious
advantage of not requiring any special hardware. The disadvantage
is that it is extremely difficult to achieve good performance at high
resolution: assuming that the effort for calculating one pixel is
constant (and it more or less is), a game running at 640x480 resolution
needs to display four times as many pixels as 320x240 resolution and
will consequently run four times slower.
A related technology that I need to mention here is z-buffering.
When drawing a complex 3D scene, there will usually be a lot of overlap
between the polygons as they are viewed by the camera (or eye). It
is necessary to determine which pixels will be closer and obscure
those behind them. It turns out that this is a rather difficult problem
and the technique commonly used today is a brute force approach. Parallel
to the framebuffer (which holds the final 2D image) exists the z-bufer
of identical dimensions. Every pixel on the screen has a depth information
associated with it. When processing each polygon and pixel, the renderer
calculates its depth (or z value, ie. distance from the camera on
the z axis) and compares it to the value in the z-buffer. If the current
pixel is closer to the camera, it will replace the previous value.
If not, it will be discarded. This method usually provides quite good
results visually but potentially has a large amount of overdraw, ie.
processing pixels that will not contribute anything to the final image.
Now we're somewhere in 1995-1996. Enter 3Dfx Voodoo Graphics, the
3D accelerator that took the gaming world by storm. The Voodoo was
an interesting beast. It was completely separate from the regular
graphics card (that everyone already had to have) and used a passthrough
cable. As a consequence it could only run programs in fullscreen modes.
But it ran them really fast! The 3Dfx Voodoo chip was ridiculously
basic when compared to today's hardware. All it could do was draw
texture mapped polygons. But it could also do bilinear filtering-
a method of "smoothing" texture mapped polygons, removing the jaggy
look of software rendered games. Filtering is not a complex concept
but it is just too CPU and memory intensive for purely software rendering
engines.
The first generation 3D accelerators offloaded the most computationally
intensive part - texture mapping - from the main CPU. The CPU however
still had to do some 3D work - all the 3D transforms of the scene
caused by your Lara Croft character running around (or just looking
around). But the games looked much better than they ever did before
(thanks to filtering) and they ran fairly fast at decent resolutions
like 640x480 or 800x600. I still remember how impressed I was by Tomb
Raider on the old Voodoo.
Mainstream games haven't progressed all that much since. They run
at higher resolutions, use more detailed textures and more detailed
models (significantly higher polygon counts) but are essentially based
on the same technology. They could (and sometimes do) run on the old
Voodoo, just slow.
The next jump came with the nVidia GeForce chip which integrated hardware
T&L, or Transforms and Lighting. Now the entire scene with
all the polygons was stored on the graphics card and the graphics
chip, newly called GPU (Graphics Processing Unit), did all the heavy
math. Hardware T&L itself does not make games look any better
(just faster) but it is a prerequisite for the advancements that followed.
Which came with the nVidia GeForce III chip in the form of vertex
shaders and pixel shaders. After what I've seen at the
ATI Mojo Day I'm convinced that these technologies will to some (large?)
extent replace texture mapping. Texture mapping has always been a
cheap crutch in many ways. Look at the bark of a tree, look at a brick
wall, look at the skin on your hand. They do not have pictures painted
on them. They have varying color and uneven surface. The shaders can
to some extent simulate that, much better than 2D textures.
The shaders are essentially little programs that can be run on the
graphics chip, now called VPU (Visual Processing Unit). The VPU is
not a general purpose unit. That means it can't do just anything.
But what it can do it does really, really well. And it is very fast
compared to the CPU, in large part thanks to the ability to process
several pixels in parallel and also because graphics cards have extremely
fast memory.
Vertex shaders, as the name implies, process vertices (each triangle
has three of them - what a surprise!). That means they can alter the
geometry and lighting of a scene. If you've seen something like the
Nature demo, that's vertex shaders in action. They have the power
to animate each leaf on the tree and each blade of the grass. Or they
can create those cool waves on the water. Vertex shaders are most
useful when it is necessary to animate lots of similar objects, something
that would be extremely difficult to do fast enough with "traditional"
methods. They also can simulate complex lighting effects - hence the
name, shaders.
Pixel shaders, not too surprisingly, process pixels. For every pixel
on the screen you can run a little program - imagine that! Pixel shaders
are great for simulating various materials without the need for textures,
or providing additional effects for textured objects. I will mention
some applications of pixel shaders later.
There are interesting performance tradeoffs between pixel shaders
and texture maps. Nowadays most games are limited by memory bandwidth
because they process massive amounts of textures. Pixel shaders on
the other hand consume almost no memory bandwidth, just VPU time.
But since the VPU is waiting for the memory anyway, you can get the
cool effects essentially for free!
Another technology I need to mention is bump mapping aka normal
mapping. Here a bump map (which is a sort of 2D texture) is applied
to each polygon, in addition to whatever texture etc. it might already
have. Normal mapping is closely related to Phong shading in that the
light intensity is calculated for each pixel. But where Phong shading
simply interpolated the surface normals, bump mapping determines them
from the normal map. This allows the creation of "bumps" - a surface
that looks 3D even though it's really just a big ol' flat polygon.
You could achieve the same effect by splitting the model into zillions
of small polygons but the performance hit would be prohibitive.
The last technology I'll talk about is tessellation. This technique
has been used in 3D graphics for ages. It is a method of refining
rounded surfaces of 3D models. It essentially adds more polygons to
a model through some predetermined algorithm. It can transform blocky
models into well rounded ones and can be done purely in hardware with
practically no loss in speed. ATI calls this technology Truform.
Enough theory (bored to death yet?). Now I'll give practical
examples of how the technologies with those intimidating names that
I described above can actually be made to do something useful (if
"useful" is the right word). All the following illustrations were
"borrowed" from materials handed out by ATI to the Mojo Day attendees.
Depth sprites. Games need explosions, right? The problem is
that it is extremely difficult to create a good 3D explosion. Most
games use 2D sprites that are drawn in the 3D world. The downside
is that the sprites often intersect with the 3D environment and it
becomes painfully obvious how flat they are. Depth sprites can fix
that. In addition to the actual sprite image there is a depth map.
Combine with a simple pixel shader and voila, you have a 3D sprite
with real depth! This picture illustrates the difference (z perturbation
is just a fancy name for the effect):
Depth of field. Real cameras (and eyes too) have depth of field:
they are focused at one plane that has particular distance and everything
else is more or less fuzzy. 3D games have somewhat different internal
workings (simulating an infinitely small lens) and as a result the
entire scene is in focus. With a pixel shader it is possible to simulate
depth of field and achieve more photorealistic scenes by selectively
blurring parts of the image.
Procedural wood. This has to be one of the coolest demos I've
seen at the ATI Mojo Day. It is a combination of vertex and pixel
shaders and an alternative to texture mapping. Imagine you are creating
a 3D game and want to have some wooden objects in it. Traditionally
you would create the object and slap a wood texture on it. With today's
technology you can grow your own tree instead! Allow me to explain.
Certain materials, such as wood or marble, can be fairly accurately
simulated with a few equations. Imagine a tree trunk as a bunch of
concentric cylinders alternating between darker and lighter color.
If you add some noise (ie. artificial irregularities), you will end
up with a decent approximation of wood. You can use this virtual tree
trunk and "chisel" your 3D object out of it. It can look just as good
as a texture and you can create models which would be extremely difficult
to create textures for (see picture). Moreover you can easily vary
the parameters of the fake wood (in real time if you wish to), altering
its color etc. Similar approach can be used with other materials.
Isn't this wooden elephant cool?
Ocean water. Yet another vertex and pixel shader application.
With these technologies you can easily simulate waves including appropriate
lighting effects. The nice thing is that again you can alter many
parameters at runtime - speed, size or direction of the waves, color
of the water and so on. Note that it is also possible to create "water
effects" using bump mapping, which achieves similar results through
a different method.
Tone mapping. Another application of pixel shaders. Through
pixel shaders you could easily adjust the lighting of a scene, simulating
overexposure or underexposure. An obvious application is overexposure
when leaving a dark dungeon and entering the sunny outside world,
with gradual adjustment back to normal level. And vice versa for going
back to the dungeon.
Fur. Another amazing application of vertex and pixel shaders
is fur. Creating fur that looks even remotely close to reality with
"traditional" methods is probably impossible. But with vertex and
pixel shaders you can grow your own! I will not go into the details
but it is possible to display pretty good looking fur in real time.
Some parameters of fur are pre-generated (ATI has a Fur Generator
tool), others can be altered dynamically at run-time.
Normal maps. I have already mentioned these before. Another
application besides bump maps is this: during game development the
artists can create a very complex high polygon model of, say, a car.
Such model looks great but can't really be used in the game because
it'd make it too slow. An interesting technique is creating a normal
map from the high-res model and applying it to a simple low polygon
model used in the finished game. That way there is almost no performance
loss and the object looks nearly indistinguishable from the real McCoy.
The illustration shows just how much difference normal maps can make
- the right half of the image uses normal maps, the left half doesn't.
Otherwise they're identical.
Non-Photorealistic Rendering. This must be the most unexpected
application of vertex and pixel shaders I've seen at the ATI Mojo
Day. I love cartoons and especially anime. I have always maintained
that some games look much better as cartoons than they would in 3D
(think Broken Sword). Imagine my surprise when I saw that you
can turn a 3D scene into a cartoon! The technology behind this is
fairly complex and I won't pretend to understand all of it. But the
illustration shows that you can arrive at a scene that looks like
a hand drawn cartoon when you start with a 3D model. It is likely
that something similar will be used in animated movies and I would
be curious to see if any future game will employ this technique.
So what will all this wonderful new technology do for games, and adventure
games in particular? At first, very little: extremely few people own
the hardware capable of these feats. I am one of the lucky few but
that's only because every attendee of the ATI Mojo Day received a
free Radeon 9700 Pro. Game developers need to reach the widest possible
market and hence usually don't require the latest and the greatest.
The only major game I know of that will require vertex and pixel shaders
is Doom 3. That's not even out yet and I have serious doubts
that the game will be any good, although I'm sure it'll look absolutely
gorgeous.
But in a year or two we'll certainly start seeing more games that
make good use of the shaders and other advanced technologies. I wonder
if that will bring more variety into gaming again. Right now I have
a depressing feeling that all games look terrifyingly similar. They're
almost impossible to tell apart because they all try to be realistic
- and reality looks the same everywhere.
It will naturally take even longer for these technologies to make
appearance in adventure games. Adventures were never on the bleeding
edge of graphics technology, probably because their primary purpose
usually is to tell a story and not just look good. And the target
audience is exactly the kind of people who do not have the
irresistible urge to waste money on the latest and fastest hardware
(I'm sure there are exceptions). So it'll take time, but we will
see this technology in real games one day. The technology is there,
now we "only" need human ingenuity and creativity to unlock it.
In conclusion: look forward to some really cool looking games in the
future. Whether they will be great games as well remains to be seen.
There is always hope.
Oh, by the way, I give the ATI Mojo Day in San Francisco an A+.