Exploring “Frame time” measurement – Part 1 – Is Fraps a good tool?

Metro 2033

Let’s first look at Metro 2033.  Everything is maxed out at 1920×1080 except for DOF which provides very little in the way of visuals but takes a large performance hit.  We are going to examine one representative run and compare the frame time chart as reported by the engine and made into a chart as compared to a Fraps run of the same run and compared with the frame time chart as reported by Fraps.

First up is the fps chart that the game engine reports:

Now here is an Excel fps chart of the very same run as charted by Fraps.  Thanks to Grstanford of ABT forum for making the calculations for all of the following charts!

We note that in both cases, the results are very similar.  What about Frame Times?  Here is a comparison of how the Fraps and the Metro 2033 game engine displays  frame times in Excel:

On the left are the frame times as reported by the game engine opened in an Excel spreadsheet.  In the center is the same run as reported by Fraps; and on the right is the Metro 2033 game engine’s individual frame times after being recalculated.

Here are the Excel charts that were produced – first the Metro 2033 frame times as reported by the game engine:

Now the same run as reported by Fraps:

As you can see, Fraps reports very similarly to Metro 2033 in both frame rate and frame times.  The Fraps run would be a second or two different from the official benchmark due to starting Fraps benching immediately before or after the bench actually begins and stopping it slightly before or after it ends.  When watching this benchmark, one can actually see the stutters.  It appears to have been optimized for performance and higher fps, not smoothness.

Let’s now look at STALKER, Call of Pripyat

STALKER, Call of Pripyat

We use maxed out settings but only used 2xAA as we want to average over 60 fps.

Here are our fps averages as reported by the SunShafts benchmark:

As with Metro 2033, we are going to examine one run and compare the frame time chart as reported by the engine and made into a chart as compared to a Fraps run of the same run and the frame time chart as reported by Fraps.  First up is the chart that the game engine reports:

Now the same run as reported by Fraps:

And again, the runs are so similar as to convey approximately the same information.  It fits very well in with the experience of watching this benchmark – a small chug at the beginning, then it settles down until the scene changes and lighting comes into play – smoothness alternates with slight jitter until we reach the last scene that settles down nicely.  There is a better balance of good performance with overall smoothness compared with Metro 2033 which jitters more noticeably.

These frame time charts are all well and good in supporting frame rates, but who is going to want to spend the time to make the calculations necessary to create these individual frame time charts?  Fortunately there is an easy solution.

Enter Frafs Bench Viewer

Lindsay Bigelow, an established member (raffriff) of FrapsForum.com, developed an uncomplicated tool for simply dragging and dropping Fraps frame time csv files into a program, named Frafs Bench Viewer, that quickly makes a chart.   We would like to support his efforts by also using his tool and encouraging our readers to also make these simple charts and perhaps share them on ABT forum.

Frafs Bench Viewer can be downloaded at SourceForge.com or from Softpedia.com as freeware.  All you do is drag and drop a Fraps frametimes.csv file into the program and it automatically creates customizable graphs!

Here is a graph produced by the Frafs Bench Viewer of the same Call of Pripyat benchmark as the last graph above:

It may look different from the frame time Excel graph that we are repeating again here  . . .

  . . . until we compress it:

The Frafs Benchmark Viewer is a very useful and easy-to-use tool to display frame times and we encourage our readers to submit their own graphs and experiences on ABT forum.

Let’s look next at Far Cry 2.

Far Cry 2

Far Cry 2 benchmark also reports the frame times but it rounds them off quite differently so that the spikes are not as apparant as they are with Fraps.  Here is the game engine frame times (in brown) superimposed over the Fraps frame times (in blue):

In all three cases, Fraps frame times follows the three game engine frame times closely enough to be useful in conveying the experience of “smoothness” and some of the spikes in the chart can be noticed by a careful observer – or as we shall show in this continuing “smoothness” series – by a high-speed digital camera.

Let’s head to our conclusion about Fraps frame time measurements.