GeForce GTX260+ vs Radeon 4850 Part 2: Image Quality

Introduction

After comparing the performance of the Radeon 4850 to the GeForce GTX260+ in part one of this article, some interesting findings were made. More specifically, it was found that the 4850 was administering the smackdown on the 260+ in certain situations with 8xMSAA, and/or when running adaptive anti-aliasing against the 260+’s transparency anti-aliasing.

Clearly this is puzzling since the 4850 has inferior specs, especially with respect to memory resources, where it has almost half the bandwidth and capacity of the 260+. So part two of this article will compare image quality of the two cards in order to try to provide some kind of explanation.

Since still screenshots seldom do the issue justice (as opposed to actual movement during gaming), all of the pictures have been substantially zoomed for your convenience.

For the best comparison, I’d suggest loading the images into individual browser tabs and quickly switching between them. Also, make sure that your browser’s image scaling options are off.

8x Multi-Sampling

Let’s start with 8xMSAA. More reviewers are starting to use this setting and this is good to see as I’m a strong proponent of it. In addition to providing a visible benefit over 4xAA, it also represents a major advancement in anti-aliasing in consumer space since the Radeon 9700 Pro debuted sparse grid 6xAA back in 2002.

Starting with the theory, here are the sample grids for both cards running 8xMSAA. Note that 8xMSAA corresponds to ATi’s 8x Box and nVidia’s 8xQ setting.

We’re interested in the blue dots here, which represent the locations of the multi-samples in the pixel’s area. Both cards have eight of them arranged in a sparse grid, hence theoretically both cards should have similar image quality, though one could argue that nVidia’s samples are a bit more evenly distributed. Regardless, the workload performed by each vendor should be identical.

Theoretical grids are all well and good, but what does it look like in actual games?

For the screenshots I’ve picked a scene from Doom 3 (a title where the 4850 handily beat the 260+ with 8xAA), and I’ve zoomed to a section that contains angled lines and shadowing.

Pay attention to the three areas labeled with the white arrows in the 4xAA picture. Compared to the reference 4xAA image taken on the 260+, you can clearly see that 8xAA is smoother on both cards in all three sections. You could even argue that the 4850 is a bit better than the 260+, especially with the smoother shadow edge in section (3).

From the screenshots, we can conclude that 4850 is not only applying 8xMSAA when requested, but it also looks a little better than the 260+ in this particular instance. So then, the issue of slow 8xMSAA performance is squarely a problem with nVidia and is not related to ATi doing something inferior or wrong; quite the opposite actually in this case.

It’s also worth mentioning that based on my Catalyst 9.1 review, the 4850 is now basically tied with the 260+ at 1920×1440 with 8xMSAA in Quake Wars due to a driver performance gain. Again, this simply shouldn’t be happening based on the cards’ relative specs.

nVidia needs to fix whatever is causing this abysmal 8xMSAA performance in OpenGL.

Transparency/Adaptive Anti-Aliasing Introduction

The transparency/adaptive class of anti-aliasing tackles objects called alpha textures that are not affected by regular multi-sampling.

This usually includes transparent content such as chain-link fences, grass, and individual tree leaves; situations where the developer has chosen to use a texture rather than rendering the content with geometry. Using textures in these situations is faster but since multi-sampling only affects polygon edges, they do not receive AA.

nVidia’s transparency super-sampling (TrAA) and ATi’s quality adaptive anti-aliasing (AAA) are functionally equivalent in that both use sample patterns identical to the base multi-sampling pattern currently in operation, and the samples are super-samples.

The scheme works by resubmitting the texture in question multiple times to the pipeline and shifting the sample position, later averaging the samples into one. Thus the performance hit will be proportional to the amount of alpha textures in the scene. The odd grating or fence won’t impact performance much, but large open areas with masses of vegetation can cause a significant performance impact, even more-so than regular super-sampling.

nVidia’s super-sampling setting is equivalent to ATi’s quality setting, providing both vendors are set to the same multi-sampling level.

TrAA/AAA (Call of Duty 2)

First we’ll start with Call of Duty 2, one of the titles where the 260+ was beaten by the 4850 with TrAA/AAA. The images are game screenshots of a tree relatively close to the player.

As a matter of interest, here are the reference images with AAA/TrAA disabled (so alpha textures are not receiving AA):

Straight away we can see some differences: the 4850 does a better job preserving some of the fine detail in the branches, especially the very thin stalks at the end. Looking at the leaves, both vendors have spots that have more detail than the other, so they’re probably even overall.

Now let’s enable TrAA/AAA (super-sampling) on both vendors:

Compared to the reference images above, both vendors are clearly applying anti-aliasing to the branches and the leaves, which squashes my initial suspicion that ATi wasn’t actually applying AAA when requested in this title. But that’s where the similarities end. The 4850 seems to gain even more detail in the branches, especially the very fine sticks at the ends of the tree. The 4850 also “fattens” up the leaves, something the 260+ doesn’t do.

However the leaf edges are clearly smoother on the nVidia card, as are the edges of the larger branches. The 260+’s image is also blurrier overall, especially the interior of the leaves, indicating that nVidia is applying more anti-aliasing than ATi is. As a result, nVidia’s leaves will shimmer less during in-game movement because they’ll receive more anti-aliasing.

TrAA/AAA (Serious Sam 2)

Serious Sam 2 was also another title where the 4850 administered a beating to the GTX260+. My second suspicion was that ATi was reducing AAA’s workload at long range, so the next set of images are of palm trees and tree leaves at a very long range relative to the player.

Here’s an image of the 260+ with TrAA off, along with TrAA/AAA (super-sampling) enabled on each vendor:

When compared to the reference image (off), both vendors clearly anti-alias the foliage, squashing my second theory that ATi isn’t applying AAA at long range.

Firstly, checking the dark green tree leaves to the left, we see the 4850 smoothes the edges and also fattens up the image again, but the interior has less blurring than the 260+. nVidia’s edges are clearly smoother though and the interior of the leaves are also blurrier, again likely suggesting that nVidia is doing more work to the tree.

Moving onto the palm tree leaves to the right, both vendors’ edges look better depending on where you look, so it’s probably a wash there. nVidia’s leaves are again a bit more blurred which means they’re doing more work to the interior of the image.

TrAA/AAA Summary

Despite my initial suspicions, the 4850 is applying AAA when requested and it doesn’t seem to falter at a long range either.

As for the implementation, given that ATi’s images show less interior blurring, one possible explanation is that their hardware is somehow detecting texture edges and applying the most anti-aliasing there, but avoiding working too much inside the edges.

Given their edge detect CFAA modes can already detect polygon edges, it’s not much of a stretch to suspect they could do something similar with textures. This would explain the large performance rift between the vendors, and why the 4850 sometimes shows absolutely miniscule performance hits when enabling AAA.

It also looks like ATi somehow “builds up” the edges in order to anti-alias them, giving them a fatter appearance, but strictly speaking this deviates from the original image because it’s made bigger.

So from a workload and accuracy perspective, nVidia appears to be doing more work and generating an image truer to the original than ATi is. nVidia’s alpha textures are more likely to sit still during in-game movement because they’ll receive more anti-aliasing.

Subjectively some may prefer ATi’s sharper look, the fact that ATi’s image is “fuller”, and also because there is more detail in very fine elements like twigs. Needless to say, the much lower performance hit is also quite desirable.

Another major point of note is that ATi’s AAA functions in OpenGL while nVidia’s TrAA doesn’t. That means to get anti-aliased alpha textures in OpenGL games on nVidia’s cards, your only option is to use combined mode super-sampling modes. Unfortunately these modes are too slow for new games, and in old games they don’t anti-alias alpha textures as good as AAA does.

nVidia needs to implement TrAA in OpenGL as they are behind ATi in this area.

Combined Mode Anti-Aliasing (xS)

As noted in part 1 of this article, the combined AA modes are available exclusively to nVidia cards, and their notable advantage is the ability to offer full screen super-sampling. Aside from stacking with anisotropic filtering, they deliver excellent image quality as they anti-alias both regular textures and shaders, something regular multi-sampling can’t do. TrAA/AAA is restricted to alpha textures, so regular textures require super-sampling in order to improve their look.

Here are the sample patterns for 8xS and 16xS, the two combined modes I’ll be covering today:

You can see eight and sixteen blue dots but more importantly, remember the single green dot that both 8xMSAA modes had earlier? That’s because multi-sampling only takes one texture (and shader) sample. Not so with super-sampling, and you can see two and four green dots with 8xS and 16xS respectively, lined up in an ordered grid. These samples will anti-alias everything, including all textures and shaders.

I’ve picked Doom 3 again and focused on a metallic floor. It exhibits texture aliasing and also shader aliasing (generated from the lighting shader). A strong anisotropic filter will help, but to really improve image quality in this situation, you need super-sampling.

You can clearly see both 8xS and 16xS impacting the pattern on the metallic floor, making the lines and lighting look smoother and more consistent. Note the extra improvement with the edges of the thick vertical lines that 16xS brings over 8xS. During in-game movement the floor will shimmer less when using xS modes compared to regular MSAA/CSAA/CFAA (edge) modes, because none of the latter modes have any effect on the texture whatsoever.

Since only nVidia offers full screen super-sampling on single card configurations, they are uncontested in this area. Older titles that are fast enough to use them have superb image quality. I really think ATi should offer them, at least unofficially. I can’t imagine it would be extraordinarily difficult to implement ordered grid super-sampling in the driver, and many ATi users have been asking for it for years.

Super-sampling can cause marginal blurring to 3D scenes in rare situations, but most of the time it’s only visible under a very heavy zoom from my testing. As a result, the readily visible in-game improvements more than outweigh this potential problem.

Pure Edge Anti-Aliasing

As I explained earlier, super-sampling can cause marginal blurring and some are opposed to it on this basis, and they demand stronger pure edge AA instead. So this next section will test AA modes that do just that.

16xQ is nVidia’s strongest pure edge AA mode, combining 8xMSAA with 8xCSAA. The other 16xAA mode (4xMSAA + 12 CSAA) is generally inferior to 8xMSAA, so I won’t bother looking at it today.

ATi’s enhanced edge modes are 12xAA (4xMSAA + edge detect) and 24xAA (8xMSAA + edge detect). These CFAA modes are like narrow/wide tent in that they use adjacent pixels for additional samples, but the shader detects polygon edges and only anti-aliases those situations, hence avoiding the internal blurring that narrow/wide tent cause. Edge detect modes don’t render the scene higher than the output, hence saving memory bandwidth and storage compared to traditional AA methods.

Here are some zoomed images of a crane in Half-Life 2:

The red arrows in the 12x image illustrate the three sections of interest.

16xQ is slightly smoother than 12xAA along most of the crane’s main support beams, areas like (1).

16xQ and 24xAA are tremendously close, and the differences at this level of edge AA probably become academic in this particular scene. Nevertheless, 24xAA has a slight advantage with the power line (2) and the thin rails (3).

It’s worth noting that while 16xQ can approach 16xMSAA in terms of quality, it can lose effectiveness in some situations due to its coverage sampling component not being completely robust. Some examples include polygons sharing the same edges, or stencil shadows. In those situations the image quality of 16xQ will be closer to 8xMSAA, but 24xAA will be free from such limitations and should deliver more consistent results.

Anisotropic Filtering

Lastly, I’ll cover AF. Usually it’s almost impossible to demonstrate differences in anisotropic filtering with screenshots, but I found a good example by accident when I was checking some Doom 3 screenshots.

You can see the interior horizontal lines inside the beam are much rougher on the 4850 than the 260+, and this is an area that won’t be impacted by regular AA because it’s texture aliasing. Also the curved lines to the right of the yellow light are smoother on the 260+ than on the 4850.

Because nVidia’s image looks blurrier, many would assume it is worse, but this is incorrect. During in-game movement you’ll find the lines in question crawl and wiggle more on the 4850 than on the 260+, so image quality will be visibly worse on ATi’s card.

nVidia provides visibly superior anisotropic filtering, and ATi needs to lift their game in this area.

nVidia’s Advantages

• Best anisotropic filtering in consumer space.
• TrAA is more accurate and performs more work to alpha textures.
• Full scene super-sampling AA modes.

ATi’s Advantages

• 24xAA is the best pure edge AA mode in consumer space.
• AAA functions in OpenGL.
• Very fast and efficient 8xMSAA relative to hardware specs.

Conclusion

Factoring everything that has been covered in this article, I tend to prefer nVidia’s image quality overall as I’m a big fan of full scene super-sampling. Visible gains from edge AA tend to rapidly diminish after 4xAA, so to see large improvements during gaming you have to start anti-aliasing other parts of the scene, not just edges. That and after gaming with nVidia’s superior AF since November 2006, I can immediately spot the regression when I game on ATi hardware.

Since you might have different criteria for evaluating image quality, I’d encourage you to pick the vendor strongest in the areas you care the most about.

Please join us in our Forums

Follow us on Twitter

For the latest updates from ABT, please join our RSS News Feed

«1. Introduction2. 8xMSAA3. TrAA/AAA (Introduction)4. TrAA/AAA (Call of Duty 2)5. TrAA/AAA (Serious Sam 2)6. TrAA/AAA (Summary)7. Combined Mode AA (xS)8. Edge AA9. Anisotropic Filtering10. Conclusion11. View All»