Subject: Re: But how do you know a review is biased or a marketing gimmick?
Date: 11/1/2001 11:06:35 PM (GMT-7)
IP Address: 126.96.36.199
In Reply to: But how do you know a review is biased or a marketing gimmick? posted by
a reviwer cannot compare memory the same way one would compare, for example, video cards or microprocessors.
Memory is simply a utility device. You pick a particular bus speed, assign settings, and it then does exactly what you tell it to do, regardless of what is actually sitting in the slot. To simplify that, think in terms of light bulbs -- lots of companies sell them, but I cannot tell from simply plugging each one in for a few minutes and turning it on whether one bulb is better than another because each one is doing the exact same thing.
Something I can test for is, say, quality -- if a particular brand of light bulb has a higher tendency to be DOA, or if another bulb brand is made of lower quality materials and tends to burn out sooner, that is something I can prove with adequate testing.
The TwinMOS memory in that article operated faster because, as they stated, they were running it at more agressive timings. But that's like running my light bulbs on a dimmer switch and then saying one bulb was brighter because I had the dimmer switch set higher for that bulb. Well duh
Now, they DO indicate in one brief sentence that the TwinMOS part was able to operate at 146MHz bus with the most aggressive timings whereas the Crucial parts were not -- but this doesn't say a much because they never felt bothered to indicate how they did that testing. What's more, voltage settings can come into play -- some DDR parts like higher voltage and some like lower, if both are run at the same setting then it is possible to "break" one DIMM's ability to clock high while the other one operates at full potential. This can skew the results of the stability testing.
In other words, the information given was wholly inadequate to confirm their claims.
Then they ran back-to-back benchmarks, another mistake -- those benchmarks are only showing the obvious, the light bulb with the dimmer switch on the higher setting is glowing brighter. Also impacting the results is the fact that the system with 256MB part will use the swap file less than the system with the 128MB part, so the side-by-side Crucial comparisons are only demonstrating another obvious fact: The system with more memory will perform better than the system with less.
In terms of evaluating hardware review sites:
-- Anything published at LostCircuits is usually outstanding.
-- Anything published at Ace's Hardware is usually also outstanding.
-- PenstarSys is generally excellent.
-- Most of RealWorldTech's articles are excellent.
-- Most of ArsTechnica's technical articles are excellent (although they haven't been publishing as many of late).
-- AnandTech's general methodology and benchmarking suites are usually good, but the results are sometimes inconsistent.
-- FiringSquad likewise has good general methodology and benchmarking suites; I cannot comment on the results they get b/c I don't read their stuff as often.
-- Tom's Hardware Guide publishes whatever suits Tom's editorial needs at the moment. Usually you can pick up some good general information there, but never trust his results until you have compared them to a lot of other sites.
-- [H]ard|OCP has published some strange stuff in the past but recent articles have been good, bordering on excellent.
-- JCNews is usually pretty good.
-- GamePC usually publishes whatever makes its newest hardware addition look good, trust their results ONLY after a lot of double-checking.
-- Van's Hardware is usually alright, my personal opinion is that Van has a particular way of looking at things but unfortunately fails to recognize that fact -- if he did recognize it, I think his site would be excellent.
-- AMDZone does alright but...it IS Chris Tom and you have to understand that he is a Crusader of The Cause His reviews are usually alright but I wouldn't call them thorough.
-- PC World exists specifically to make money. You may pick up some good general info there but never trust the results at first glance.
-- Overclockers.com is inhabited by Ed Stroligo, take everything he says with a grain of salt -- or a salt lick, as necessary.
-- xbit-labs usually means well, and they do a decent job of polishing their reviews. Sometimes I like what I see there, occassionally I don't. Try to compare their results to what other sites are finding.
The list could go on,but the general rule of thumb is to read a lot of different reviews and compare the results, and always come back to places like LC and Ace's when you want to know for sure