The Ultimate Guide on Benchmarking


benchmarking-stopwatch2If you have done any research on computer components, you have likely come across a number of benchmark tests used for comparison of various pieces of equipment or overall computer builds. Looking at all the numbers and the testing procedures behind benchmarking may seem daunting for the uninitiated, but it is less complex than you might imagine. This guide will walk you through the first steps of benchmarking like a pro.

What Exactly Is Benchmarking?

Benchmarking means to evaluate or test a system in order to gain an idea of its performance level. A benchmark test will put the computer through a series of tasks, sometimes predefined, and record its performance statistics. Commonly tested statistics include the time it takes for the CPU to finish a specific task or how well a computer runs a game.

Why Is Benchmarking Useful?

If you were to judge from PC hardware reviews, you would think that benchmarking is used only as a tool to compare components to each other. While this is definitely a primary purpose, it can do so much more. For example, if you want to see how your computer handles a game at various configurations, it is necessary to use benchmarks to notice changes in performance that are too small to perceive without the aid of software. The same goes if you are tweaking the BIOS or installing new components. Of course, benchmarking also comes into play for bragging rights over whose computer is the best.

“Synthetic” Versus “Real world” Benchmarking

There are two main categories of tools used by people wishing to benchmark their computers. The first is known as synthetic tests. These are specifically designed to stress the performance of your rig in certain ways. The second is referred to as real world testing. Both methods have their place in a benchmarking toolkit.
Some system components simply do not undergo enough stress in normal operations to adequately test their limits. System RAM is notably capable of going well above what is necessary, considering many motherboards support 32 and 64 GB of RAM when 16 GB can handle a modern game and a slew of background tasks without even coming close to being tapped. This is where synthetic testing comes into play.
Unless you have a specific desire to see how your components max out, then you are most likely interested in the real world performance of your computer. For gamers, this means running through the game and recording the results. Some applications come with built-in benchmarking for this reason, and additional software is available as add-ons for programs like video creation tools.

Benchmarking Tools

Of course, if you want to benchmark, you need the tools to do so. We have compiled a list of some of the best benchmarking tools available. Considering this guide is intended for those just getting into the world of performing their own benchmark tests, we focused on either free software or ones with decent free versions in order to allow you to get started quickly without any overhead. Because going into the details of each tool and how to use it would make this article incredibly lengthy, most of the tools will only be mentioned with a link. For further explanation on their usage and to download them, follow the links.

General Testing Tools

These tools provide benchmarking software designed for more than one type of computer component. All of these tools provide the ability to test almost every aspect of your system by combining the results of multiple types of tests.


SiSoft’s Sandra
FutureMark’s PCMark 7
PassMark’s Performance Test
NovaBench

Component Specific

Some pieces of your system are responsible for specific tasks and processes; therefore, getting a good idea of their performance requires a targeted approach. The tools below are divided into the components on which they focus.

CPU

CPU-Z
wPrime
Prime95
SuperPI – This tool is not optimized for multithreading, but it is still useful

GPU

It is worth noting that the best tests for measuring GPU performance are typically paid for in some way, be they games, rendering software, or dedicated benchmark tools.
3D Mark
Unigine Heaven 3.0

Memory

Memtest86+
MaxxMem
Streambench


Storage

AS-SSD
ATTO

Just showing off my system before going into games (as I am a PC gaming freak)

my gaming system

 

Games and FRAPS

Using games to benchmark computers is about as “real world” as it gets when it comes to high stress benchmarks that test the limits of your computer. Because it is one of the more common interests of those looking for information on benchmarking, we will spend a little more time discussing gaming benchmarks using FRAPS.

A number of modern games come with built-in benchmarking software due to the relevance of gaming to the PC hardware community. Notably, Crysis 3  and Metro 2033 have served as the most recent mountains to try and climb with home computers.

FRAPS is a multifunctional piece of software that is handy for gaming in general, but you can look at the additional features like video recording later. We are interested in it for the fact that it provides a simple frame rate analysis tool that is perfect for testing any games which do not come with benchmarking software included.
When you install and startup FRAPS, getting it to display the frame rate is simple enough. All you have to do is run both FRAPS and the game, and voila! You will see the frame rate in yellow, and you can move it around the screen as you like. This little feature is useful for less stringent or more haphazard testing such as troubleshooting a specific system change or game activity, but do not rely on it for precise measurements as it is typically difficult to pay attention to both the game and the readout.

The FRAPS program has a simple menu for configuring where your benchmarking tests are stored and what information is included. You will at least want FPS and the minimum, maximums, and averages recorded, and there is no harm in using the frame time for further analysis. You can setup a hotkey for enabling the test, defaulted to F11, and you can determine the length of the test. Once you are ready, find a repeatable portion of the game to test. For example, in a single player game, you might create a save point and run through a minute or two of the game, then repeat. Correlating any spikes in performance with the gameplay may indicate specific performance issues.

Frame Rate and Frame Time

To get the most out of the numbers FRAPS gives you, it is important to understand their context and meaning. Frame rate is the number of frames displayed over a length of time, and it is most commonly given in terms of Frames Per Second (FPS). For most gamers, 30 FPS is considered the bottom line for what is considered adequately playable, and 60 FPS is the usual target rate. You may have heard that the human eye can only perceive so many frames per second, but the truth is that there is a noticeable difference in the fluidity and clarity of the image even beyond what scientists would call the average human capability to perceive.
Frame time is linked to the frame rate, though it indicates the actual time it takes for your computer to render each frame. For a quick comparison, an average frame time of 33 ms per frame translates into an average frame rate of 30 FPS. There is much debate as to whether frame time or FPS is the more accurate depiction of performance, but both have their merits in analysis.

The Testing Procedure

Just because you have the tools and know what you want to test does not mean you are ready to accurately run your benchmarks. The following steps should be taken every time you intend to run a test. Yes, in some cases, they may make the procedure more aggravating and time-consuming overall, but if you want the most accurate results, then they should be followed.

Install All Relevant Driver and Software Updates

Lacking the proper software updates may skew your test results due to a number of reasons, more than likely resulting in a decrease in performance over optimal conditions. Your operating system, the benchmarking tools you are using, and hardware drivers should all be on their latest updates.

Record System Information

Whether you are testing a specific component, tweaking a BIOS setting, or just want to see how your PC holds up, you should record every bit of information you can about your system. If you end up changing something between tests, then you should make note of the change. No difference should be seen as insignificant, and every piece of data may prove invaluable.

Restart Your Computer

While it may seem like an almost too simple step, it is an important one. As a computer runs, it builds up system heat and memory leaks. Restarting will remove the memory leaks, and giving your computer a minute or two to rest in between testing cycles will help to ensure each test starts from as close to identical conditions as possible.

Disable Any Extra Processes

For the most accurate results, you should eliminate as many variables that are not part of your testing procedure as possible. Most operating systems will have a number of secondary processes that are enabled during startup, such as your security software, messenger systems, media tools, and so forth. If you do want to see how your system performs while these tasks are running, this is perfectly acceptable, but the active processes should be identical for every test for proper comparison.

Run Benchmark Tests Individually

Benchmarking utilities typically take a decent amount of time to accomplish, and the individual tests can interfere with each other by dividing up system resources even if they were designed to stress different components. The only time it may be acceptable to run multiple tests is when you are performing something called a “burn-in,” a specially designed stress test that is intended to detect any possible hardware defects and to ensure the system is stable at maximum strain. Otherwise, run each test separately for the most accurate results.

Repeat, Repeat, Repeat, Repeat

No matter which test you run, it is imperative that you repeat the test multiple times. Just like in scientific testing, getting a result once means almost nothing. A random, unforeseen variable may suddenly drop or increase your PC’s performance. Ideally, you want to run enough tests to consistently get the same results for three to five tests in a row. If you are doing system or hardware comparisons, you should get verifiably repeated results for each variation of the test.

Putting the Data Together

Once you have configured your system, ran your benchmark tests, and compiled the data, it is then time to make sense of the horde of numbers you have collected. Having a large list or spreadsheet of numbers is useful, but displaying the data in graphs and charts can more easily indicate anything notable the information contains and is a large reason why computer hardware sites use these graphs in their articles. Creating these visuals yourself is a relatively simple process.

Making Graphs Like a Pro

Now to make those nice looking graphs you have to find the fps file that was created after recording the frames from an in game benchmark. The file is the fps one which contains the frame rate per second of the benchmark. All you have to do copy and paste the data (excel), highlight the fps and press insert – line and then customize the graph to your colour and style preference.
Voila your now a benchmark pro!!!

benchmark graph

About the Author

Nikolas Nikolaou

a pc gaming enthusiast with love for benchmarking and reviewing the latest in technology releases. You can read more about his passion at HardwarePal.

There are 10 comments

Comments are closed.