PassMark Software

Announcement

Collapse
No announcement yet.

New algorithm for determining overclocking in charts

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • New algorithm for determining overclocking in charts

    After our study of the 'Distribution of benchmark results for Intel i5 CPUs' we decided that we could improve the accuracy of the charts by better filtering of the overclocked machines.

    What we were doing up until today (2/Feb/2012) was including a CPU in the standard benchmark chart if the measured base clock speed of the CPU was within 15% of the factory specified speed. Anything over 15% got moved to the overclocking chart.

    While it could be argued that the 15% level wasn't strict enough, this worked pretty well in the past. A 3.0Ghz CPU found to be running at less than 3.3Ghz at base speed got put into the standard chart, while anything at 3.3Ghz and above was classified as overclocked. But with the introduction of the Turbo Boost feature in Intel i5 and i7 CPUs new ways of overclocking are now available. It is now possible (and even common) to overclock just the turbo speed of the CPU, leaving the base speed unchanged, or only slightly changed.

    The result was that CPUs that had overclocked turbo multipliers were slipping through the overclocking detection algorithm. The main noticeable consequence of this was that the 'K' series CPUs (e.g. Core i5-2500K, i7-2600K, i7-2700K, etc..) were ranking higher than they should have been in the standard charts.

    From today we are adding new data into the database on factory and measured turbo clock speeds and tweaking the algorithm to trigger the overclocking flag based on either overclocking of the base speed, or overclocking of the turbo speed. We are also going to change the criteria for what is overclocked from 15% of the clock speed to 90Mhz difference from the factory speeds. This will mean even lightly overclocked CPUs are now going to moved into the overclocked chart. Hopefully this won't result in too many false positives.

    The result will be a bunch of CPUs with high benchmark results being moved from the standard chart to the overclocking chart.

    We think the 'K' series CPUs will still rank better than their almost identical non K counterparts for other reasons however. But this should narrow the gap and improve the accuracy of the charts.

  • #2
    This change has just gone live. You might need to force a refresh in your browser to see the new results.

    Here are some examples of the impact. The old values were what was in the charts yesterday. The new values is what we have today (3/Feb/2012).

    CPU - Old / New CPUMark Rating
    Intel 2600K - 10074 / 9106
    Intel 2600 - 8964 / 8912

    Intel 2500K - 7991 / 6739
    Intel 2500 - 6649 / 6614

    AMD FX-8150 - 8369 / 8226
    AMD FX-6100 - 5714 / 5556
    AMD 1090T - 6069 / 5977

    So there was a gap of 1110 in the CPUmark score between the 2600K and 2600. After this change in overclocking detection the gap is now 194 (2%). Which is much more reasonable considering these are effectively the same CPU in their stock state.

    Comment


    • #3
      Distribution of turbo clock speeds for Intel i5 2500K CPU

      Also here is an interesting break down of what we have measured for the turbo speed of an i5-2500K.

      Note that for the i5-2500K, the factory turbo speed is 3.7Ghz. Which makes our cut off point 3.79Ghz. The red dotted line show the break point where we have classified a CPU as being overclocked based on its turbo clock speed.



      Based on this criteria 49% of baselines people have sent in for the i5-2500K are overclocked.

      By comparison, for the non 'K' chip, the i5-2500, only 19% were found to be overclocked.

      Comment


      • #4
        The other change today was a change to the information displayed on the overclocking chart.
        http://www.cpubenchmark.net/overclocked_cpus.html

        Previously we displayed the average percentage increase in the clock speed for each type of CPU.

        This calculation became a lot harder to do as both the base clock speed (without turbo active) and the turbo clock speed can be separately overclocked. Also many of the older CPUs don't support turbo mode. There was also a problem that we don't have the turbo mode value for even all of the new CPUs, as the older releases of PerformanceTest never collected this value.

        The simplest solution in the end was to stop reporting clock speed increases, and instead report performance increases.

        So now for each CPU in the Overclocked chart you can mouse over the row in the chart to see the average percentage increase in performance you get by overclocking that model of CPU.

        Here are some examples from popular CPUs from the OC Chart.

        Intel Core i7-2700K @ 3.50GHz, 18% performance increase
        Intel Core i7-2600K @ 3.40GHz, 24% performance increase

        AMD FX-8150 Eight-Core, 18% increase
        AMD FX-8120 Eight-Core, 25% increase

        AMD Phenom II X6 1100T, 16% increase
        AMD Phenom II X6 1090T, 18% increase
        AMD Phenom II X6 1075T, 23% increase
        AMD Phenom II X6 1055T, 24% increase

        As a general rule if you start with a CPU with a low clock speed, you can overclock it more than a CPU with a higher initial clock speed.

        So if you are an overlocker looking for the best value, then it makes sense to buy the cheaper CPU, lower clocked, CPUs and overclock them harder.

        This is really just statistical proof of what overclocker have known for years. Cheap CPUs have more overclocking headroom.

        Comment


        • #5
          A new patch release of PerformanceTest V7 was made public today, build 1026.
          http://www.passmark.com/products/pt_history.htm

          Changes include better turbo mode clock speed detection on the newer AMD chips. Which should in turn allow better detection of overclocking and thus more charts accurate benchmark charts (over time).

          Comment


          • #6
            Non K-version SB processors

            Just one thing is puzzling about this - non K-version (desktop) and non XM-version (mobile) processors are now appearing in the overclocked chart. As I understand it, these cannot have their turbo multipliers changed, and are difficult to overclock in any other way. I must have missed something...

            Comment


            • #7
              Non K chips can still be overclocked. But only to a much more limited extent.

              Comment


              • #8
                ?

                I'm curious to know more about how Passmark creates it's benchmark score. For one, how does integrated graphics affect CPU score or does the CPU score include any integrated graphics benchmarks at all?

                Another thing I noticed while searching around is that the CPU list includes only an overall score that some might view as 'synthetic'. Honestly, I see no other way of obtaining an accurate benchmark of a CPU then to test all of it's capabilities, but since not all applications use all of these capabilities would it not be better to list benchmarks of individual features?

                I believe that showing results of individual tests might help individuals choose the best upgrade path for them. Benchmarks of things such as standard x86, 64-bit instructions, MMX, SSE, SSE2, ..., AVX, integrated GPU, etc. can give much more relevant information to the user instead of having to rely on an overall score where the calculation of such a score could be flawed. It would also help identify where overclocking is least/most beneficial and be a good tie breaker for closely matched devices.

                Does this make any sense?

                Comment


                • #9
                  > how does integrated graphics affect CPU score.

                  It doesn't significant effect the CPU result.

                  > Does this make any sense?

                  No. Not for 99% of users.
                  Even people with university degrees in computer science are going to struggle to describe what MMX and AVX are. Then to know which applications make use of these CPU features is nearly impossible.

                  I interview a lot of programmers applying for jobs. Most can't describe the real difference between 32bit and 64bit. So the general population has no chance.

                  So the breakdown you are suggesting would not be more relevant.

                  What would be more relevant would be a benchmark of World of Warcraft for people who play World of Warcraft. And a benchmark of Photoshop for Photoshop users. etc...
                  But making it more relevant to some users makes it much less relevant to others.

                  Thus the need for a simple broad based CPU benchmark that has some relevance to every computer user.

                  Comment


                  • #10
                    No. Not for 99% of users.
                    Even people with university degrees in computer science are going to struggle to describe what MMX and AVX are. Then to know which applications make use of these CPU features is nearly impossible.
                    I don't doubt that they'd struggle, but I do think you are underestimating them with MMX. I don't have any degree in computer science, but I could explain the technologies.

                    I interview a lot of programmers applying for jobs. Most can't describe the real difference between 32bit and 64bit. So the general population has no chance.
                    Wow... Maybe they didn't start programming when they were 12 like real programmers?
                    I don't doubt that many programmers don't know what's really under the hood, but I think you are underestimating a lot of other people here.

                    So the breakdown you are suggesting would not be more relevant.
                    So I'm basically too geek to be relevant. Sorry no offense, but it all just sounds like a bunch of marketing fud to me. I guess I'm just that 1% that looks at numbers like everybody else looks at shiny objects. I would rather see factual scores of something that's real though rather then a score calculated with magic (although I do love the magic )... But you're right, it would only be useful to that 1% and it is a lot of work and a lot of data to collect.

                    Comment


                    • #11
                      You can actually see the individual numbers for the CPU tests by downloading the baselines from within PerformanceTest. The breakdown isn't as detailed as you are suggesting above, but it is more detailed that just a single CPUMark figure.

                      but I think you are underestimating a lot of other people here.
                      I disagree. Only assembly language programmers are going to really understand MMX, SSEx, etc.. And even with an understanding of what it is you still can't tell what real applications use which technology.

                      If we take your argument to the logical conclusion then we would end up with 1000s of benchmarks. e.g. one for 32bit addition of single precision floats, one for 32bit addition 8 bit integers, one for 32bit addition of a deferenced address holding a single word that is in cache, etc... It really has to be brought up to a higher level.

                      You might also be interested in this short study on the speed and breakdown of CPU instructions.

                      Comment


                      • #12
                        You can actually see the individual numbers for the CPU tests by downloading the baselines from within PerformanceTest. The breakdown isn't as detailed as you are suggesting above, but it is more detailed that just a single CPUMark figure.
                        This is cool. I never really explored it.

                        ... And even with an understanding of what it is you still can't tell what real applications use which technology.
                        Probably only 1%...

                        e.g. one for 32bit addition of single precision floats, one for 32bit addition 8 bit integers, one for 32bit addition of a deferenced address holding a single word that is in cache, etc... It really has to be brought up to a higher level.
                        Yeah, that makes sense. And I guess anyone really in need of those results would know how to write the code to profile it themselves (or could figure it out anyway).

                        You might also be interested in this short study on the speed and breakdown of CPU instructions.
                        Now this was interesting. I knew that divisions were slow, but I had no idea that they were that much slower. I got a lot more reading to do.

                        Comment


                        • #13
                          Way higher scores...

                          Originally posted by passmark View Post
                          This change has just gone live. You might need to force a refresh in your browser to see the new results.

                          Here are some examples of the impact. The old values were what was in the charts yesterday. The new values is what we have today (3/Feb/2012).

                          CPU - Old / New CPUMark Rating
                          Intel 2600K - 10074 / 9106
                          Intel 2600 - 8964 / 8912

                          Intel 2500K - 7991 / 6739
                          Intel 2500 - 6649 / 6614

                          AMD FX-8150 - 8369 / 8226
                          AMD FX-6100 - 5714 / 5556
                          AMD 1090T - 6069 / 5977

                          So there was a gap of 1110 in the CPUmark score between the 2600K and 2600. After this change in overclocking detection the gap is now 194 (2%). Which is much more reasonable considering these are effectively the same CPU in their stock state.
                          Hi.

                          What could explain the current differences between the i5-2500K scores and the i5-2550K scores?

                          They have basically the same specs:... but the scores are very different:
                          Intel Core i5-2500K @ 3.30GHz: 6,745
                          Intel Core i5-2550K @ 3.40GHz: 8,653
                          Thanks for your time.

                          Comment


                          • #14
                            It has been fixed up. Results are now (1/March/2012),

                            Intel Core i5-2500K @ 3.30GHz: 6,745
                            Intel Core i5-2550K @ 3.40GHz: 7,167

                            The i5-2550K was new, so we didn't have the max factory turbo speed set in the database. It is there now. So the split between normally clocked and overclocked should now be done properly for the 2550K.

                            Comment


                            • #15
                              Makes sense...

                              Originally posted by passmark View Post
                              It has been fixed up. Results are now (1/March/2012),

                              Intel Core i5-2500K @ 3.30GHz: 6,745
                              Intel Core i5-2550K @ 3.40GHz: 7,167

                              The i5-2550K was new, so we didn't have the max factory turbo speed set in the database. It is there now. So the split between normally clocked and overclocked should now be done properly for the 2550K.
                              Thanks for the info.

                              Regards.

                              Comment

                              Working...
                              X