Announcement

Collapse
No announcement yet.

PerformanceTest V10 Beta Release

Collapse
This is a sticky topic.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • PerformanceTest V10 Beta Release

    PerformanceTest V10 is now available for beta testing.

    Current Version: V10 Beta 3
    Date: 26/Febraury/2020
    Download link: PerformanceTest V10 Beta

    Latest changes: See changes in beta 3.

    Please contact us to report any issues found in this build.

    Baseline submissions: Currently baselines can be submitted but the returned link to display them will not work (they are being stored in a different database to what is live on our current site).

    What's new

    Score rebalancing
    • Due to the large amount of changes made to the 2D/3D/CPU/Disk tests all the calculated mark values have been rebalanced and scaled to be similar (but not exactly the same) to that of PerformanceTest 9.
    • Individual test scores have not been scaled so a direct comparison cannot be made in many cases between version 9 and version 10.

    Windows Support
    • No longer supporting Windows version older than Vista.

    Compiler updates
    • We switched from using Visual Studio 2013 (for V9) to Visual Studio 2017 for V10 Alpha. Likely we will move to Visual Studio 2019 before the first beta release. 2019 is being tested at the moment. Newer compiler versions typically bring improved code optimisation and use of newer CPU instructions.

    CPU Tests
    • Enabled compiler optimisations for the CPU tests that had previously been disabled. This has resulted in better performance on newer CPUs when compared to older ones.
    • Compression Test, Replace the existing algorithm with a new version that uses the Cypto++ Gzip library. Previous versions of PerformanceTest were using a Adaptive encoding algorithm, which gave good compression rates on text, but was't in common use. Zip is the defacto standard for real world data compression.
    • Encryption test, removed Salsa and TwoFish from the sub tests that are run and replaced them with an ECDSA (Elliptic Curve Digital Signature Algorithm) sub test. Previously PerformanceTest V9 had 4 sub-tests. These were, TwoFish, AES, Salsa20 & SHA256. Now we have three sub-tests. AES, ECDSA & SHA256. These are all algorithms that are heavily used in the real world.
    • Extended Instructions (SSE), added an AVX512 test (when available). AVX512 are a new set of CPU instructions that are in newer CPUs.
    • Extended Instructions (SSE), fixed a pointer math issue in the test that was referencing incorrect memory locations. Accessing the correct data helped with data alignment and improved test throughput.
    • Extended Instructions (SSE), made some changes to the SSE/AVX/FMA tests for how the results are retrieved and stored for next calculation loop (using the _mm_storeu_ps and mm256_storeu_ps CPU instructions).
    • Extended Instructions (SSE), removed custom aligned vector/matrix class and switched to standard vector/matrix class. Changed how matrix data is loaded before test (using the _mm256_broadcast_ps CPU instruction).
    • Integer Math, made some changes to add better support for out of order execution. This makes the algorithm less linear and gives modern CPUs the change to get more calculations done in parallel.
    • Physics test. Updated Bullet physics engine to version 2.88. Previously were were using 2.83.
    • Added a Cross-platform mark to the CPU test, made up of the Integer, Floating point, sorting and prime test scores. This will be calculated when loading a V9 baseline in V10 if the required scores are available. This cross platform score is not used when generating the overall CPU mark as it based on previously run tests. We envisage that we'll use these results in the future for cross platform comparisons (x86 PCs vs ARM based mobile devices).

    2D Tests
    • Added a new “PDF Render” test that will open a PDF and scroll through the available pages
    • Changed default test size to 1920 x 1080 this tend to place more load on the video card than was previously the case. So frame rates are lower across the board compared to V9. Tests will scale down in size to 1024x768 and a penalty will be applied and will not run at lower resolutions than this.
    • Direct 2D, increased amount of objects displayed during test.
    • Windows Interface test, increased size of dialog and number of controls on dialog.

    3D Tests
    • DX9, changed default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution.
    • DX9, increased default Anti Aliasing level to 8, changed skybox and number of objects in scene. This was done with the aim of making the test less CPU bound.
    • DX10, increased default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution. This adds load to the video card.
    • DX10, increased number of islands and meteors during test
    • DX10, fixed a bug when enumerating display modes for the DX10 test where no compatible card would be found if there was a large amount (>500) of display modes
    • GPU Compute , replaced the Bittonic sort test with an NBody Gravity test. Nobody knew what a Bittonic sort was or how it applied to the real world. NBody simulations on the other hand are a fairly common scientific application and they make a good visual impact.
    • GPU Compute, increased default size for sub tests to 1920x1080
    • GPU Compute, OpenCL test, made some changes to particle size and variables used for calculations so more is happening during the test.
    • Started allowing ”Microsoft RemoteFX Graphics Device” for 3D support on some VMs

    Disk tests
    • Changed name of “Random Seek” test to “32KQD20” to better represent the test (using 32K block size with a queue depth of 20)
    • Added a new “4KQD1” test (using 4k block size with a queue depth of 1)
    • Increased test files size to 400MB for write test, 800MB for read test (traditional hard drive). If drive is an SSD then it is 1GB for write test and 2GB for read test.
    • Removed the CD test. Surely nobody was using this anymore?

    Advanced Physics Test
    • Added option to allow resolution to be selected
    • Updated Bullet physics engine to version 2.88. Previously were were using 2.83.

    Advanced Network Test
    • Added threading and the new Windows RIO sockets option. RIO is Registered Input/Output (RIO) API Extensions. These changes were made to achieve higher throughput & lower latency, particularly on 10Gb+ networks. You should now be able to really push the limits of your networking gear with these changes.

    Advanced Disk Test
    • Made some changes to try and stop crashes during the advanced disk IOPS test when the hard drive is failing/responding abnormally
    • Added temperature collection to test results. Can now choose to display the temperature or the latency heat map when displaying a graph. This has become important as some SSDs throttle down their speed under high temperatures.

    NEW Advanced CPU Test
    • Added an advanced CPU test, this allows individual CPU tests to be run from 1 thread up to a specified number of threads and then the results graphed.

    NEW Advanced Internet Speed test
    • Added an implementation of M-Labs (https://www.measurementlab.net) internet speed test that will connect to their servers, performance a 10 second upload test, a 10 second download test and then display the results from that test.

    Localisation
    • Added a section in the installer to allow selection of a supported language, on install first launch PT will now choose that language by default.

    Reporting
    • Added percentile options to exported text, formatted text and HTML reports.

    Baseline Management
    • Added a way of tracking submitted baselines and displaying them in the advanced baseline management dialog under the “Uploaded” tab

    Scripting
    • Changed "Result Date" to be local time instead of UTC for consistency (windows install time was already local time)
    • Added HIDEBASELINES option to only add current computer results to exported results files
    • HTML report output, fixed a bug where the Unicode BOM was not being correctly written to the start of the file if the file was opened in append mode and didn't already exist

    Misc
    • Chart display, added percentage difference to baselines when a score the "this computer" is available. Also added option in the preferences to enable/disable.
    • Fixed an incorrect error message in the advanced drive performance test when a selected disk did not have enough free space
    • Fixed a bug when saving results to an image, the scrollbar width was being applied and causing some result to be hidden.
    • Added a gray rectangle to custom list view header so the boundary that can be used to resize the columns is highlighted
    Last edited by Tim (PassMark); 02-26-2020, 04:13 AM.

  • #2
    Here is an example of what the new Advanced CPU test in PerformanceTest 10 looks like.
    It gives a great idea of how well the system scales as you increase the number of active CPU cores.

    You can select the test to run, the test duration, how many times to repeat each test & and the maximum number of threads.

    CPU thread scaling options

    This gives a result that looks like this graph below. This was on a i7-4770 CPU, which has 4 cores (8 with hyperthreading active).
    You can see that the CPU scaled in a nearly linear fashion until 4 threads and then performance dropped off. Meaning that either hyperthreading was detrimental, or we hit a bottleneck elsewhere in the system (e.g. RAM bandwidth)


    CPU Test thread scaling graph primes

    Here's a 2nd example. In this example hyperthreading gave a big boost and the test doesn't have much of a dependency on RAM bandwidth.
    however once we get to 8 threads, there are no more CPU cores available and performance flat lines.

    CPU Thread scaling integer operations

    Comment


    • #3
      I'm really interested in the scripting and outputs, is there a way I can get around the license requirement to run the scripts and exports? It's not accepting my v9 license.

      Comment


      • #4
        Rofl, If you contact us we can supply you with a PT10 key if you have purchased PT9 in the last 12 months.

        Comment


        • #5
          I'm getting 2-D PDF Rendering test failures - Unexpected Error Running PDF Render Test, Number -1

          Comment


          • #6

            Changes in Alpha 2

            Memory Tests
            •Database Operations, changed to use an in memory sqlite3 database. Will now use 4 threads to generate a database approximately 125MB in size (500MB total) and measure the insert and query speed.
            •Increased amount of memory that non-cached tests use to 512MB of RAM (up from 256MB)
            •Threaded Memory, changed to run a test for 4, 8 and 16 threads and use the best result as the test score

            2D Tests
            • Complex vectors, changed so that rendering loop resets sooner rather than most of the drawing happening off screen towards the end of the test
            • Simple vectors, fixed incorrect window tittle during test
            • Fonts and Text test, fixed missing icon

            Advanced Memory Test
            • Added "Threaded" test option to advanced memory test, currently will run the threded memory test starting with 1 thread up to (Cores * Threads per core) threads. Each test loop is repeated 3 times and the best result stored.

            Scripting
            • Added SETRERUNRESULT to toggle re-run config setting. Choose between keeping BEST and NEW result when re running tests.

            Advanced Visualised Physics
            • Added message check on exit to stop "not responding" when closing while using a large number of objects
            • Fixed missing icon

            GPU Compute
            • Mandelbrot test, increased number of iterations 10x to slow down the test execution

            Misc
            • System information, Changed ram details in baseline system info to display in GB
            • Internet speed test, updated due to some protocol changes that were breaking the results retrieval
            • Fixed bug where cross platform mark was using compression score instead of string sort score
            • Now displaying "Baseline #X" instead of just "#X" on system information window for loaded baselines
            • Fixed some tooltip strings being incorrect for menu items
            • Report exports, added BIOS version and hard drive size as a separate field to the exported report system information
            • User Interface, Made back/prev/next buttons in 3D component info view slightly lighter so they stand out more
            • No longer displaying PNP ID on Video card system info display

            Comment


            • #7
              Changes in Alpha 3

              Advanced CPU Test
              • Made stop button more responsive while test is running
              • Added HTML report export function for advanced CPU test
              • Added CPU_RUNTEST and ACPU_SETEXPORT commands for scripting the advanced CPU test

              2D Tests
              • Fixed window titles.

              Advanced Memory Test
              • Now opens graph automatically at end of testing.

              Temperature collection
              • Fixed a bug preventing the CPU temperature from being updated on the main window.

              Memory Tests
              • Memory Test - Threaded, changed to run a range of threads up the maximum available (eg 2, 4, 8, 16 or 3, 6, 12 depending on available core count) and use highest score
              • Memory Test - Database operations, slight changes to how test operating when dropping/creating a new table to increase load
              • Memory Test - Database operations,changed to run a range of threads up the maximum available (eg 2, 4, 8, 16 or 3, 6, 12 depending on available core count) and use highest score
              • Memory Test - Threaded and Database operations, amount of RAM used will depend on number of threads, <= 16 threads 512 MB, <=64 threads 1 GB, > 64 threads 2 GB
              • Memory Test - Threaded and Database operations, added support for processor groups and thread affinity
              • Memory Test - Database operations, started using recommended Compile-time Options to improve performance from https://sqlite.org/compile.html (except -DSQLITE_THREADSAFE=0)

              Comment


              • #8
                JohnG,

                Can you try the PDF rendering test again in Alpha 3. It might be fixed.

                Comment


                • #9
                  Confirmed that PDF Rendering does work in Alpha 3.

                  Comment


                  • #10
                    Also confirmed - was getting PDF 2D failure in a2, no longer receiving error in a3.

                    Comment


                    • #11
                      Correction - resolved for onboard graphics, not resolved for discrete gfx? Have to do more testing.
                      Attached Files

                      Comment


                      • #12
                        The error from the screenshot points to a missing DLL, likely the system doesn't have the Microsoft Visual C++ 2015 Redistributable package installed (https://www.microsoft.com/en-us/down....aspx?id=52685).

                        What version of Windows are you using?

                        Comment


                        • #13
                          W10 1909
                          There were also later errors for another driver for the NBody test.
                          Attached Files

                          Comment


                          • #14
                            Thanks, I believe we are able to make some compiler option changes to stop the errors occurring and not require installing of the Microsoft redistributable package.
                            We should be putting out a new alpha build this week with these changes.

                            Comment


                            • #15
                              -It seems that I am still getting the PDF error when executing from script, or maybe it's intermittent, not enough testing completed to be conclusive.
                              "Unexpected error running PDF Render test. Error number -2147024894"

                              -When opening script "MyScript" is pre-populated, and you have to manually enter * <enter> to find other files and does not seem to filter ONLY *.ptscript files. <annoying not broken>
                              -Scripting multi-runs still stops and requires manually clicking OK at the end of a single run to continue
                              +Loading x86 VisC++ redist <not x64> seemed to resolve driver issue on discrete gfx system


                              <working on new scripting commands and outputs now>

                              Comment

                              Working...
                              X