Forums - PC Discussion - Digital Foundry is the Performance Review Top Tier But....

Their new reviews of the Intel 10th Gen are not good from a scientific perspective. 
When you're comparing the difference between 2 different variables you want to keep everything is equal as possible.

You wouldn't test the sound quality between Spotify and Youtube by using different headphones for each platform. So obviously with CPU reviews it's very important to compare using the same GPU, RAM amount and speed, etc etc.

I assume Digital Foundry did this, but as they explain in the review they used a 240mm Storm Castle AIO Water Cooler on the Intel platform. And they went with the stock cooler bundled with the AMD CPU.

I genuinely do not see the logic behind this, we know that Ryzen's power management clocks the CPU higher as the temps are lower. It's CPU reviewing 101 to use the same(roughly the same) cooler and not entirely different cooling mediums.

(Also I'm not sure why they're still testing Crysis 3, The Witcher 3 and older titles when people are buying these very expensive CPU's to play modern games).

I love these guys, but that's a serious miss-step. 

Last edited by ArchangelMadzz - on 25 May 2020

PS4(PS5 Soon)and PC gaming

There's only 2 races: White and 'Political Agenda'
2 Genders: Male and 'Political Agenda'
2 Hairstyles for female characters: Long and 'Political Agenda'
2 Sexualities: Straight and 'Political Agenda'

Around the Network

Yea I haven't watched DF for PC tech reviews for a very long time. I like Gamers Nexus cause they do very in depth tech reviews and they do it very well.

I will say that one of the problems with Ryzen is that it doesn't overclock very well vs Intel which does. But that does mean that Intel's power draw is pretty crazy, specially with 14nm vs 7nm on Ryzen and needs very beefy cooling compared to Ryzen. I do think they still should have used the same cooler though and made the setups as close to each other as possible. I guess DF's problem is that they don't do as many PC videos as say Gamers Nexus so they don't invest much when it comes to their PC testing setups.

Last edited by Captain_Yuri - on 25 May 2020

             

                               Anime: Haruhi                                                                                      Anime: Love Live
                              Nsfw Anime Thread                                                                             Join our Anime Threads!
                             Sfw Anime Thread                                                                                VGC Tutorial Thread

ArchangelMadzz said:


(Also I'm not sure why they're still testing Crysis 3, The Witcher 3 and older titles when people are buying these very expensive CPU's to play modern games).

I love these guys, but that's a serious miss-step. 

Probably sponsored by Intel or such.
This type happends basically everywhere, intel owns the media.

You have to question why pc guys are benchmarking these things at 1080p or 720p as well.

You should be giveing people a honest representation of real world performance.
Now one buys a Geforce 2080ti to game at 720p or 1080p.

The only reason they dont show the 1440p or 2160p (4k) results, is because the differnce in CPU performance in gameing go away at that point.
Ei. the graphics card becomes the bottleneck.

Naturally the reviewers cant just say "their both fine, they can both feed the gpu all it needs, for gameing".
So they instead show you "non real world performance" (ei. games running a way no one with that GPU would use it), so they can sell people something they don't even need.  Here is a new CPU, its from Intel! buy it!


Truth of the matter is.... for most people your better off just buying a cheaper CPU, and spending most of your budget on the GPU.
The 10900k is for people that are willing to pay 500$ extra for like 5-10 fps more at 1440p/4k, for games that are already above playable levels in fps.

Intel motherboards are stupid expensive too, and the 10900k needs a water cooling unit? cool....
Compaired to to a cheap mobo amd ryzen + cpu running its stock cooler.

Its not quite apples to apples.

Last edited by JRPGfan - on 25 May 2020

Ceteris Paribus is something that should be respected for most analysis as you really should isolate variable per variable first, then sure you can look for interactions between them.
But as said by others perhaps the purpose of that comparison wasn't to be a fair and honest one.



duduspace11 "Well, since we are estimating costs, Pokemon Red/Blue did cost Nintendo about $50m to make back in 1996"

http://gamrconnect.vgchartz.com/post.php?id=8808363

Mr Puggsly: "Hehe, I said good profit. You said big profit. Frankly, not losing money is what I meant by good. Don't get hung up on semantics"

http://gamrconnect.vgchartz.com/post.php?id=9008994

I never watch DF for any hardware related review, their knowledge is severely limited. Anandatech for architectural deep dives, Guru 3D for thermals of specific cards.



Around the Network

DF loves gaming, and Intel, the 'gaming heavyweight champion of the world', looks to be at least trying now even though they don't have a lot to work with considering the present arch and node. I think DF is mostly trying to throw them a bone here.

If DF's word is all you need to buy in then you're lost anyway or have more money than you know what to do with. It doesn't take much more research and comparing to realize what the better option likely is at this point in time. Plus, it's not like Intel won't bother trying to compete with new arch's and nodes in the future. There's almost certainly many more rounds to come and DF isn't exactly ready to count the 'boxer in blue shorts' out just yet.

Last edited by EricHiggin - on 25 May 2020

ArchangelMadzz said:

Their new reviews of the Intel 10th Gen are not good from a scientific perspective. 
When you're comparing the difference between 2 different variables you want to keep everything is equal as possible.

You wouldn't test the sound quality between Spotify and Youtube by using different headphones for each platform. So obviously with CPU reviews it's very important to compare using the same GPU, RAM amount and speed, etc etc.

I assume Digital Foundry did this, but as they explain in the review they used a 240mm Storm Castle AIO Water Cooler on the Intel platform. And they went with the stock cooler bundled with the AMD CPU.

I genuinely do not see the logic behind this, we know that Ryzen's power management clocks the CPU higher as the temps are lower. It's CPU reviewing 101 to use the same(roughly the same) cooler and not entirely different cooling mediums.

(Also I'm not sure why they're still testing Crysis 3, The Witcher 3 and older titles when people are buying these very expensive CPU's to play modern games).

I love these guys, but that's a serious miss-step. 

When it comes to hardware reviews, it's best to stick with Anandtech.

Crysis 3 is a logical choice due to it's game engine that is used in other titles, even more modern ones, it gives us an idea of how that engine will typically run on certain sets of hardware.

And when gathering a heap of games for a benchmark suite, it is good to get an equal mix of games which typically favor Intel and AMD, that way you remove hardware bias.

Witcher 3 is still a pretty technically impressive title to benchmark with, got no issue with it being used... It's good to retain a few older games in a benchmark suite so you can see the performance progressions generationally with hardware, because lets face it... The individual who is upgrading to a Core i5 10600K isn't typically going to be a Ryzen 5 or i5 9600K, it will be from much older hardware like the 5600K, 4600K etc'.


As for the cooler, I can see what Digital Foundry has done here... And I share your concerns.
The issue is AMD bundles a cooler, Intel doesn't with the i5 10600K and i9 10900K, so digital foundry probably just grabbed "whatever" and threw it onto the Intel platform.

They should have done the same for the AMD.

In saying that, they could have ran the AMD numbers a long time ago, so they may not have re-ran the actual numbers again for this particular benchmark/review.

Captain_Yuri said:

Yea I haven't watched DF for PC tech reviews for a very long time. I like Gamers Nexus cause they do very in depth tech reviews and they do it very well.

I will say that one of the problems with Ryzen is that it doesn't overclock very well vs Intel which does. But that does mean that Intel's power draw is pretty crazy, specially with 14nm vs 7nm on Ryzen and needs very beefy cooling compared to Ryzen. I do think they still should have used the same cooler though and made the setups as close to each other as possible. I guess DF's problem is that they don't do as many PC videos as say Gamers Nexus so they don't invest much when it comes to their PC testing setups.

I don't mind gamers nexus's actual content.
I just can't stand the presenters voice, hair and how he waffles on. But that's just me personally... So will prefer to wait on Anandtech or Linus.

JRPGfan said:


You have to question why pc guys are benchmarking these things at 1080p or 720p as well.

You should be giveing people a honest representation of real world performance.
Now one buys a Geforce 2080ti to game at 720p or 1080p.

The only reason they dont show the 1440p or 2160p (4k) results, is because the differnce in CPU performance in gameing go away at that point.
Ei. the graphics card becomes the bottleneck.

That is exactly why you don't run CPU benchmarks at a high resolution... It turns the benchmark into a GPU benchmark, not a CPU one.

You want to showcase the CPU differences as much as possible.








--::{PC Gaming Master Race}::--

Pemalite said:
ArchangelMadzz said:

Their new reviews of the Intel 10th Gen are not good from a scientific perspective. 
When you're comparing the difference between 2 different variables you want to keep everything is equal as possible.

You wouldn't test the sound quality between Spotify and Youtube by using different headphones for each platform. So obviously with CPU reviews it's very important to compare using the same GPU, RAM amount and speed, etc etc.

I assume Digital Foundry did this, but as they explain in the review they used a 240mm Storm Castle AIO Water Cooler on the Intel platform. And they went with the stock cooler bundled with the AMD CPU.

I genuinely do not see the logic behind this, we know that Ryzen's power management clocks the CPU higher as the temps are lower. It's CPU reviewing 101 to use the same(roughly the same) cooler and not entirely different cooling mediums.

(Also I'm not sure why they're still testing Crysis 3, The Witcher 3 and older titles when people are buying these very expensive CPU's to play modern games).

I love these guys, but that's a serious miss-step. 

When it comes to hardware reviews, it's best to stick with Anandtech.

Crysis 3 is a logical choice due to it's game engine that is used in other titles, even more modern ones, it gives us an idea of how that engine will typically run on certain sets of hardware.

And when gathering a heap of games for a benchmark suite, it is good to get an equal mix of games which typically favor Intel and AMD, that way you remove hardware bias.

Witcher 3 is still a pretty technically impressive title to benchmark with, got no issue with it being used... It's good to retain a few older games in a benchmark suite so you can see the performance progressions generationally with hardware, because lets face it... The individual who is upgrading to a Core i5 10600K isn't typically going to be a Ryzen 5 or i5 9600K, it will be from much older hardware like the 5600K, 4600K etc'.


As for the cooler, I can see what Digital Foundry has done here... And I share your concerns.
The issue is AMD bundles a cooler, Intel doesn't with the i5 10600K and i9 10900K, so digital foundry probably just grabbed "whatever" and threw it onto the Intel platform.

They should have done the same for the AMD.

In saying that, they could have ran the AMD numbers a long time ago, so they may not have re-ran the actual numbers again for this particular benchmark/review.

Captain_Yuri said:

Yea I haven't watched DF for PC tech reviews for a very long time. I like Gamers Nexus cause they do very in depth tech reviews and they do it very well.

I will say that one of the problems with Ryzen is that it doesn't overclock very well vs Intel which does. But that does mean that Intel's power draw is pretty crazy, specially with 14nm vs 7nm on Ryzen and needs very beefy cooling compared to Ryzen. I do think they still should have used the same cooler though and made the setups as close to each other as possible. I guess DF's problem is that they don't do as many PC videos as say Gamers Nexus so they don't invest much when it comes to their PC testing setups.

I don't mind gamers nexus's actual content.
I just can't stand the presenters voice, hair and how he waffles on. But that's just me personally... So will prefer to wait on Anandtech or Linus.

JRPGfan said:


You have to question why pc guys are benchmarking these things at 1080p or 720p as well.

You should be giveing people a honest representation of real world performance.
Now one buys a Geforce 2080ti to game at 720p or 1080p.

The only reason they dont show the 1440p or 2160p (4k) results, is because the differnce in CPU performance in gameing go away at that point.
Ei. the graphics card becomes the bottleneck.

That is exactly why you don't run CPU benchmarks at a high resolution... It turns the benchmark into a GPU benchmark, not a CPU one.

You want to showcase the CPU differences as much as possible.






Gamers Nexus is very good. But he does waffle on a lot in his graphs where it's very clear to see what's happening there. 

Running these games at higher resolutions does begin to just benchmark the GPU, I play games at 1440p because I'd rather have the higher framerate than higher resolution but it's known that you should spend more on your GPU than your CPU. 

I see your point about the game choice, and sure including them makes sense, it's just when I was buying my 3900X I was looking at performance in recent AAA games because that's the kind of games I was going to be playing. (Also at blender render and video export times cause I do that too)



PS4(PS5 Soon)and PC gaming

There's only 2 races: White and 'Political Agenda'
2 Genders: Male and 'Political Agenda'
2 Hairstyles for female characters: Long and 'Political Agenda'
2 Sexualities: Straight and 'Political Agenda'

There are plenty of reviews that do use the same cooler and they still find that the 10900k is very slightly better in games, ignoring power draw and heat. It's still a better chip than the 9900K(S). I will probably get it unless Zen 3 comes this year and eclipses it.

People who buy that chip are out for performance, anything else doesn't really matter.



If you demand respect or gratitude for your volunteer work, you're doing volunteering wrong.

Yes, that's the basic, when you are making a comparision about specific parts of a build the rest should be exactly the same for all of the test subjects whenever possible.
I also agree that someone buying those kinds of chips is hardly expecting to game only in 1080p, the numbers can be there but higher resolutions should also be included (didn't watch the videos, so if there is also a 1440p/4K comparision, its fine).
That being said, basically every profit based outlet will be sponsored by some the brands of products they review, which means they have way more reasons to be as parity friendly as possible to avoid looking like they are biased.
Well, there are several other outlets to check, so if DF is particularly not looking good enough, one can always check another one.