Coffee Lake Thread

[SteelCrysis]

https://www.extremetech.com/computing/27...nce-claims

The company’s explanations for its memory configurations, however, do not pass elementary scrutiny. We are told that the RAM configurations used were chosen “to have complete parity across all systems.”

This is a lie.

XMP was enabled on the Intel Z390, disabled on the X299-Deluxe, and enabled on the Prime Z370-A used to test both the Core i7-8086K and the Core i7-8700K. For Threadripper, the D.O.C.P. Standard setting was used (Asus has called this “XMP by another name”). The Asus Prime X470 Pro motherboard used to test the 2700X didn’t enable XMP or DOCP at all. In other words, out of five different motherboards, three had XMP or an equivalent enabled and two did not. RAM clocks were apparently set according to maximum manufacturer specifications. But choosing to use manufacturer-specified memory clocks and leaving XMP disabled and enabled at turns means the memory subsystems of these platforms were not configured for maximum parity.
...
If Principled wanted to configure for maximum parity, it should have tested with XMP fully enabled or disabled across all products and all systems using the same RAM clock. If it wanted to test in a manufacturer’s approved configuration, it should have stated it was doing so and explained why XMP / DOCP was enabled or disabled in each specific circumstance relative to the best practices communicated by AMD and Intel. What we got was neither.

The situation gets even stranger once you see that even some of PT’s Intel performance claims are odd. Looking through the whitepaper, I realized Principled Technologies had used an Asus Prime Z370-A motherboard running UEFI 1406 with a GTX 1080 Ti — which happens to correspond exactly to the system I used for our recent review of the RTX 2080 and 2080 Ti. They were even kind enough to test many of the same games.

This made me curious. Could I validate the Intel results they reported for the Core i7-8086K? For the most part, yes — with some notable and odd exceptions.
...
Finally, in the Final Fantasy XV benchmark, the score of 10,858 reported by PT was 7-8 percent higher than our own results in both configurations. There was no clear advantage or disadvantage to enabling or disabling Performance Mode in the tests we could verify, but we were unable to confirm benchmark results in three out of the 13 tests we investigated. That’s not a great finding for a project when we already know the AMD CPU was blatantly misconfigured. I don’t know why I can’t replicate PT’s results in every test — but I know I’ve done similar validation of benchmark results by using Reviewer Guides provided by Intel and AMD on many occasions. Something is wrong here.
...
Companies are typically extremely careful when they talk about their competition because it can open them up to legal liability if they’re found to be making false claims. In many cases, companies prefer to simply talk about “the competition” even when they literally compete with one other company. Engaging a third-party to write a whitepaper is one way of offering additional information that isn’t seen as coming directly from Intel itself. But make no mistake — Principled Technologies is a company writing a document for a client, which means the client has oversight and input into both the test conditions and the final product. If Intel doesn’t like a setting, game, or performance test, that test isn’t in this document.

Neither Intel nor Principled Technologies have responded to our emails. Could this be the result of an honest mistake or oversight? Of course. It wouldn’t be the first time someone flipped a wrong switch and then spent 10 hours cheerfully benchmarking the wrong hardware configuration, believe me. But there are problems, plural, with this whitepaper, its data, Intel’s failure to respond to it, and the fact that Principled Technologies is still trying to slide out of from acknowledging that it published verifiably false results with a lame reference to retesting AMD CPUs that it should have known were misconfigured in the first place. The company continues advancing a false narrative about its memory configuration and how it supposedly established memory configuration parity between testbeds when it did nothing of the sort.

At a time when Intel needs to be building a narrative around the Core i9-9900K as its first mainstream eight-core desktop processor, a demonstration of forward progress, and a distraction from its own security concerns and manufacturing delays, we’re instead having an extended conversation about whether the chip giant attempted to lie to consumers by proxy. It’s not a good look.