What Exactly Is Benchmarking?
Benchmarking means to evaluate or test a system in order to gain an idea of its performance level. A benchmark test will put the computer through a series of tasks, sometimes predefined, and record its performance statistics. Commonly tested statistics include the time it takes for the CPU to finish a specific task or how well a computer runs a game.
Why Is Benchmarking Useful?
If you were to judge from PC hardware reviews, you would think that benchmarking is used only as a tool to compare components to each other. While this is definitely a primary purpose, it can do so much more. For example, if you want to see how your computer handles a game at various configurations, it is necessary to use benchmarks to notice changes in performance that are too small to perceive without the aid of software. The same goes if you are tweaking the BIOS or installing new components. Of course, benchmarking also comes into play for bragging rights over whose computer is the best.
“Synthetic” Versus “Real world” Benchmarking
There are two main categories of tools used by people wishing to benchmark their computers. The first is known as synthetic tests. These are specifically designed to stress the performance of your rig in certain ways. The second is referred to as real world testing. Both methods have their place in a benchmarking toolkit.
Some system components simply do not undergo enough stress in normal operations to adequately test their limits. System RAM is notably capable of going well above what is necessary, considering many motherboards support 32 and 64 GB of RAM when 16 GB can handle a modern game and a slew of background tasks without even coming close to being tapped. This is where synthetic testing comes into play.
Unless you have a specific desire to see how your components max out, then you are most likely interested in the real world performance of your computer. For gamers, this means running through the game and recording the results. Some applications come with built-in benchmarking for this reason, and additional software is available as add-ons for programs like video creation tools.
Benchmarking Tools
Of course, if you want to benchmark, you need the tools to do so. We have compiled a list of some of the best benchmarking tools available. Considering this guide is intended for those just getting into the world of performing their own benchmark tests, we focused on either free software or ones with decent free versions in order to allow you to get started quickly without any overhead. Because going into the details of each tool and how to use it would make this article incredibly lengthy, most of the tools will only be mentioned with a link. For further explanation on their usage and to download them, follow the links.
General Testing Tools
These tools provide benchmarking software designed for more than one type of computer component. All of these tools provide the ability to test almost every aspect of your system by combining the results of multiple types of tests.
SiSoft’s Sandra
FutureMark’s PCMark 7
PassMark’s Performance Test
NovaBench
Component Specific
Some pieces of your system are responsible for specific tasks and processes; therefore, getting a good idea of their performance requires a targeted approach. The tools below are divided into the components on which they focus.
CPU
CPU-Z
wPrime
Prime95
SuperPI – This tool is not optimized for multithreading, but it is still useful
GPU
It is worth noting that the best tests for measuring GPU performance are typically paid for in some way, be they games, rendering software, or dedicated benchmark tools.
3D Mark
Unigine Heaven 3.0
Memory
Memtest86+
MaxxMem
Streambench
Storage
Just showing off my system before going into games (as I am a PC gaming freak)
Games and FRAPS
Using games to benchmark computers is about as “real world” as it gets when it comes to high stress benchmarks that test the limits of your computer. Because it is one of the more common interests of those looking for information on benchmarking, we will spend a little more time discussing gaming benchmarks using FRAPS.
A number of modern games come with built-in benchmarking software due to the relevance of gaming to the PC hardware community. Notably, Crysis 3 and Metro 2033 have served as the most recent mountains to try and climb with home computers.
FRAPS is a multifunctional piece of software that is handy for gaming in general, but you can look at the additional features like video recording later. We are interested in it for the fact that it provides a simple frame rate analysis tool that is perfect for testing any games which do not come with benchmarking software included.
When you install and startup FRAPS, getting it to display the frame rate is simple enough. All you have to do is run both FRAPS and the game, and voila! You will see the frame rate in yellow, and you can move it around the screen as you like. This little feature is useful for less stringent or more haphazard testing such as troubleshooting a specific system change or game activity, but do not rely on it for precise measurements as it is typically difficult to pay attention to both the game and the readout.
The FRAPS program has a simple menu for configuring where your benchmarking tests are stored and what information is included. You will at least want FPS and the minimum, maximums, and averages recorded, and there is no harm in using the frame time for further analysis. You can setup a hotkey for enabling the test, defaulted to F11, and you can determine the length of the test. Once you are ready, find a repeatable portion of the game to test. For example, in a single player game, you might create a save point and run through a minute or two of the game, then repeat. Correlating any spikes in performance with the gameplay may indicate specific performance issues.
Frame Rate and Frame Time
To get the most out of the numbers FRAPS gives you, it is important to understand their context and meaning. Frame rate is the number of frames displayed over a length of time, and it is most commonly given in terms of Frames Per Second (FPS). For most gamers, 30 FPS is considered the bottom line for what is considered adequately playable, and 60 FPS is the usual target rate. You may have heard that the human eye can only perceive so many frames per second, but the truth is that there is a noticeable difference in the fluidity and clarity of the image even beyond what scientists would call the average human capability to perceive.
Frame time is linked to the frame rate, though it indicates the actual time it takes for your computer to render each frame. For a quick comparison, an average frame time of 33 ms per frame translates into an average frame rate of 30 FPS. There is much debate as to whether frame time or FPS is the more accurate depiction of performance, but both have their merits in analysis.
The Testing Procedure
Just because you have the tools and know what you want to test does not mean you are ready to accurately run your benchmarks. The following steps should be taken every time you intend to run a test. Yes, in some cases, they may make the procedure more aggravating and time-consuming overall, but if you want the most accurate results, then they should be followed.
Install All Relevant Driver and Software Updates
Lacking the proper software updates may skew your test results due to a number of reasons, more than likely resulting in a decrease in performance over optimal conditions. Your operating system, the benchmarking tools you are using, and hardware drivers should all be on their latest updates.
Record System Information
Whether you are testing a specific component, tweaking a BIOS setting, or just want to see how your PC holds up, you should record every bit of information you can about your system. If you end up changing something between tests, then you should make note of the change. No difference should be seen as insignificant, and every piece of data may prove invaluable.
Restart Your Computer
While it may seem like an almost too simple step, it is an important one. As a computer runs, it builds up system heat and memory leaks. Restarting will remove the memory leaks, and giving your computer a minute or two to rest in between testing cycles will help to ensure each test starts from as close to identical conditions as possible.
Disable Any Extra Processes
For the most accurate results, you should eliminate as many variables that are not part of your testing procedure as possible. Most operating systems will have a number of secondary processes that are enabled during startup, such as your security software, messenger systems, media tools, and so forth. If you do want to see how your system performs while these tasks are running, this is perfectly acceptable, but the active processes should be identical for every test for proper comparison.
Run Benchmark Tests Individually
Benchmarking utilities typically take a decent amount of time to accomplish, and the individual tests can interfere with each other by dividing up system resources even if they were designed to stress different components. The only time it may be acceptable to run multiple tests is when you are performing something called a “burn-in,” a specially designed stress test that is intended to detect any possible hardware defects and to ensure the system is stable at maximum strain. Otherwise, run each test separately for the most accurate results.
Repeat, Repeat, Repeat, Repeat
No matter which test you run, it is imperative that you repeat the test multiple times. Just like in scientific testing, getting a result once means almost nothing. A random, unforeseen variable may suddenly drop or increase your PC’s performance. Ideally, you want to run enough tests to consistently get the same results for three to five tests in a row. If you are doing system or hardware comparisons, you should get verifiably repeated results for each variation of the test.
Putting the Data Together
Once you have configured your system, ran your benchmark tests, and compiled the data, it is then time to make sense of the horde of numbers you have collected. Having a large list or spreadsheet of numbers is useful, but displaying the data in graphs and charts can more easily indicate anything notable the information contains and is a large reason why computer hardware sites use these graphs in their articles. Creating these visuals yourself is a relatively simple process.
Making Graphs Like a Pro
Now to make those nice looking graphs you have to find the fps file that was created after recording the frames from an in game benchmark. The file is the fps one which contains the frame rate per second of the benchmark. All you have to do copy and paste the data (excel), highlight the fps and press insert – line and then customize the graph to your colour and style preference.
Voila your now a benchmark pro!!!
Thank you for the very in-depth reference for Benchmarking, Nikolas Nikolaou,
I have previously used some of these programs when upgrading hardware, but I did not know that some of your referenced recommendations even existed.
I cheated few weeks ago, when I found a great price on a new-ish Radeon graphics card:
I used the Win8 built-in rudimentary tool (called “Windows Experience Index”).
Although I was expecting a much higher increase in the Graphics/Gaming performance in the upgrade, my Win8Pro told me that FX only got improved by two-tenths of a point. But since I have never been a gamer and my system is still using the older Asus Maximus II Formula (ROG) motherboard, I have nothing to complain about. I continually upgrade the subsystems of my rig (SSD/USB3/FX) in the hopes that my MoBo will die soon to justify upgrading it, but this is the biggest problem with buying rugged hardware. 😉
A lot of users seem to think the same about the windows experience index , the only way to really “feel” your new hardware is to actually testing it real time. Using an older processor/mobo/ram combo will sometimes hold your newer graphics card back , because it can’t perform at its peek power. The Maximus II Formula was once a very solid board for gamers but as you know technology moves fast. You should see a 10%-20% increase in games if you move to a pci-e 3.0 motherboard (if your gpu is pci-e 3.0 ready).
I’m sorry, this was billed as a review of different computer brands to see how yours stacks up. It is nothing of the kind. What are you THINKING – we’re going to walk into five computer stores and Walmart with you’re stupid heavy duty software and some gaming software in hand? And on TOP of it, there’s some stupid orange social networking bar down the left side of the screen with no x button to close it! Stupid idiots! Your stupid software has frozen your minds.
Dora,
I’m not sure where you deduct that from the above quote. The article clearly mentions “components”, “pieces of equipment”, and “computer builds” in the first paragraph.
At no point does Nikolas mention testing anything in store. The article is obviously meant to show the reader how to judge the performance of computer components, but since the whole computer must be running to do so, benchmarking also factors in the components which aren’t the target of the test.
Actually, there are four arrows at the bottom that minimize the social sharing sidebar.
At least it wasn’t the alcohol!
You’re obviously frustrated, or having a really bad day, but may I suggest you take a lighter tone in the future? The old saying “you catch more flies with honey than with vinegar” comes to mind.
Dora has a right to complain. Social media share bars are useless and garish. Not to mention the article only tells you how to use benchmarking to test hardware you own. While ON pc pitstop it claims to show you how to test new pc’s before purchasing.
Hi James – Nobody here at DCT would ever deny anyone’s right to complain. However, there is a big difference between “complaining” and flat out rudeness.
That’s your ‘opinion’ James, one which you have every right to express. However, that does not necessarily make it so. I’m sure many Facebook and Twitter users would disagree… not to mention the millions of sites which offer their services absolutely free, such as DCT.
I’ve been all through the PC Pitstop item and I cannot locate any mention of “testing PCs before purchasing” whatsoever. If you could copy and paste the exact reference, that would be most helpful.
Cheers… Jim
I’d recommend everyone skip this article. There are too many errors and it is more confusing than helpful.
A benchmarking tools shows the speed of which your PC completes a task – whether real time or synthetic.
CPUZ is not a CPU benchmarking tool, it shows settings and actuals for speeds and voltages of several components (motherboard, CPU, memory, and video).
SuperPI is a memory benchmarking tool, not a CPU benchmarking tool. It tests how fast the memory can calculate a formula.
Prime95 and wPrime are CPU stress tools, not benchmark tools. Neither of these programs test speed of your computer, they simply push it to the max and hold it there for a set period of time – which assists in identifying faulty components and drivers. Unless you’re having blue screens or lockups or overclocking your system, there’s no need to ever stress test your system.
Most popular CPU benchmarking tools are included in video benchmarking tools, like Futuremark’s 3D Mark.
Memory leaks haven’t been an issue since Windows XP SP2. If your system benchmarks faster after a reboot, it’s due to something other than a memory leak – like not stopping the same processes – or different ambient temperatures. I’ve yet to see a memory leak identified in Windows Vista, 7, or 8.
The author doesn’t seem to know the difference between stress testing and benchmarking. They’re 2 different things and done for different reasons. If you want to know how your PC compares to others, benchmark. For stability issues and testing overclock settings, stress test.
David , I am sorry the article wasn’t fit for you. Explaining each and every benchmark tool and assistant tools on the market would be somewhat difficult as there are a few hundred if not more. Searching through google or youtube for the functionality of each tool would be a lot easier and the developers of each tool have sections on their websites which explain each and every detail to their software. This was intended for a guide on getting people starting on benchmarks and to understand their functionality. Cpu-Z as you have said is not a benchmark tool it’s I like to say an “assistant tool” to help you monitor some settings (example: when overclocking and benchmarking for stability). SuperPi as seen in their site’s about page – “Super PI is a single threaded benchmark ideal for testing pure, single threaded x86 floating point performance and while most of the computing market has shifted towards multithreaded applications and more modern instruction sets, Super PI still remains quite indicative of CPU capability in specific applications such as computer gaming” – it is a cpu memory and system tool. This was actually quite hard to write as It was intended for novices . Thanks though for your opinion and I welcome you if you would like to write an article on my website.
Nikolas, I just reread my previous post and it comes across as rude. Sorry, as that wasn’t my intention. I give you a lot of credit for trying, as making this topic easy for novices to understand is quite a challenge.
You probably know this, but I’ll state it for anyone who doesn’t; SuperPI is somewhat CPU dependent – because memory speeds are tied directly to CPU speeds via the bus divider (ratio). So as the CPU bus speed is increased, so is the memory speed. Ideally you want 1:1 ratio for testing both CPU and memory. Systems with faster CPUs tend to have high performance memory as well (high performance usually has a higher MHz limit and can take more voltage/heat). Having been on overclocking forums for over a decade, I’ve yet to see SuperPI used to benchmark anything other than memory. I’m not saying it hasn’t been done, but I haven’t seen it. I think their website is a bit misleading in that regard.
I tend to use the automobile analogy for computer components to my customers (everyone knows basically how an automobile works).
Think of the CPU as the engine. The bigger it is, the faster the computer goes. (This gets complicated when people try to cross reference different CPU families, such as Pentium 4 and i3. You can only compare CPUs of the same family. IE: An Intel Pentium 4 3.0 GHz CPU is NOT faster than an Intel i3 2.2GHz CPU, but it is faster than a Intel Pentium 4 2.2GHz CPU.)
Think of the motherboard as the transmission. It ties the engine to the wheels and dictates what the gear ratio will be.
Think of the memory as the ignition. It temporarily stores information from what your computer is doing, and resets after restarting the computer. The more memory, the more you can do at once without lag. The faster the memory, the faster it can retrieve information.
Think of the hard drive as the gas tank. The bigger the drive, the farther you can go (the more information it can store). The hard drive continues from where it was last, upon restarting the computer.
Think of the video as the windows. The bigger/faster the video (card or onboard), the better you can see and the less blind spots you have.
Hope this helps. 🙂
Nikolas, I too am a gamer and enjoyed reading your article on benchmarking.
I benchmark from time to time, but I find that generally it just leaves me with a desire to upgrade my kit, which of course, I can’t afford lol.
I’ve got Crysis 3 installed and it’s simply stunning. Probably the best looking game ever to grace any PC in my humble opinion. Could you please give some pointers as to how the benchmarking in that game actually works please?
Cheers
Marc