SYSmark 25
See the FAQ posted here on Windows Defender and the related Tamper Protection setting.
Does Windows Tamper Protection affect the SYSmark performance score?
Tamper Protection can limit the ability of the SYSmark/MobileMark System Configuration Tool to modify some system settings. Some of those settings, like disabling Windows Defender, can affect the SYSmark/MobileMark performance score.
Error CR_Premiere_03_OpenProject_Transcode error (22008) may occur on systems that do not have updated audio drivers installed.
When testing with SYSmark 25, BAPCo recommends that all system drivers be installed and updated to the latest available version.
When testing with SYSmark 25, some users may encounter a System Compatibility Report dialog in Adobe Premiere Pro. This warning appears when the graphics driver installed on the system under test does not fully support the capabilities of Adobe Premiere Pro in SYSmark 25. To avoid this warning, please update to the latest publicly available graphics driver for the system under test.
SYSmark 25 can be uninstalled from Add/Remove programs in Windows 10. However, the uninstaller is not able to remove the DRM signatures placed on disk by some of the applications distributed with SYSmark 25.
BAPCo recommends that users make a back up of the system image prior to installation of SYSmark 25. After testing is complete, restore the back up, or install a clean image to the target drive to avoid issues with application DRM.
SYSmark 25 installations will expire 30 days after the first run of the benchmark, as per the licensing agreement between the application vendors and BAPCo.
Users should be sure that system time and date are correct before installing SYSmark 25. Do not attempt to change the system time and date once SYSmark 25 is installed, as it may trigger application DRM and immediately expire the installation.
Although SYSmark 25 has been engineered to disable network connections while the test is running, this error may occur on some configurations when the system under test has been connected to a wireless network. If this error occurs, and the system under test happens to be connected to a wireless network, please disconnect from the wireless network and attempt to run again.
With the release of Windows 10 19H2, Microsoft made a change to the mechanism used for Windows 10 version reporting.
Current generation BAPCo products have been updated (SYSmark 2018 fix will be included in the upcoming patch 3) to report the OS version correctly. However, customers with down-level versions of BAPCo benchmarks may find that in some cases, Windows 10 19H2 will be reported as Windows 10 19H1.
BAPCo encourages customers to always update to the latest patch for their product. Please see the product page on the BAPCo website for the latest updates for your benchmark product.
Microsoft recently released an update to Windows 10’s Microsoft Defender application which makes it impossible to permanently disable the Microsoft Defender Antivirus protection service. This change appears in Windows 10 build 20H1 (19041.450), and is part of KB4052623. As per the Microsoft documentation, this setting is no longer necessary, as Defender will automatically disable itself if third party AntiVirus software is installed on the system.
Customers running BAPCo benchmarks on Windows 10 build 20H1 (19041.450) may notice a negative impact to benchmark scores as a consequence of this update.
BAPCo is also investigating a known issue with Windows 10 build 20H1 (19041.450), in which the PDF results file may incorrectly report the state of Microsoft Defender as disabled.
Windows 10 build 20H1 (19041.450) introduced a change that prevents the user from permanently disabling Microsoft Defender. The inability to disable Microsoft Defender may negatively impact scores when running BAPCo benchmarks. Microsoft has provided documentation for users that may wish to roll back their instance of Microsoft Defender to a previous version.
This FAQ applies to MobileMark 25, SYSmark 25, MobileMark 2018, and SYSmark 2018.
Upon completion of a benchmark run, a folder called ‘Results’ is created on the desktop, and the output from the run will deposited into a subfolder that matches the project name. For example, if the project is ‘My Project’ the output folder will be on the desktop under ‘../Results/My Project/’.
This error may occur on the SUT when testing with MobileMark 2018, MobileMark 25, SYSmark 30 if the system does not have an audio driver installed and an audio device such as headphones, or speakers connected to the SUT. Note that Bluetooth devices may not be sufficient as the SUT may place them in a low power mode or disconnect them completely in order conserve energy.
To avoid this error, please install an audio device driver and connect an audio device to the SUT prior to installing and testing with BAPCo benchmarks.
Windows Defender status can produce an impact to some BAPCo benchmarks. BAPCo benchmarks will automatically attempt to disable Windows Defender (if possible). Microsoft Windows 11 Operating System prohibits all 3rd party attempts to disable Windows Defender.
In the course of testing with BAPCo benchmarks, users sometimes experience a timeout error. Timeout errors are usually attributed to the following causes:
- The SUT’s battery charge is depleted, and the SUT is throttling performance to extend battery life.
- The SUT is a lower performing configuration, and therefore may take longer to perform some benchmark tasks
If timeout errors occur, users may wish to adjust the BENCHMARK_TIMEOUT_MULT environment variable to extend the timeout before a benchmark error is registered. Please refer to the attached screenshot for guidance on modifying the BENCHMARK_TIMEOUT_MULT variable. After modifying the variable, the SUT must be rebooted for the new setting to take effect.
Update: The 11 July 2023 Windows 11 Cumulative update has resolved this issue. BAPCo recommends users testing with Windows 11 apply the KB update prior to installing and testing BAPCo benchmarks. See details from Microsoft.
Windows 11 build 1848, launching a browser other than Microsoft Edge may prompt the user to choose a default browser.
To avoid this prompt causing issues during benchmark runs, follow this procedure to workaround the issue.
- Install Windows 11 image
- Apply Windows 11 updates, if desired
- Be sure all drivers are installed and updated
- Disconnect from the Internet
- Install your BAPCo benchmark product
- Reboot
- Select the taskbar search box and type ‘Default Apps’
- In ‘Default Apps’ control panel search box, type ‘Google Chrome’
- Click the button to set Chrome as the default browser
- Reboot
- Run benchmark.
BAPCo
