Yes. Some applications packaged with the benchmark suite, such as Microsoft® Office® 2010 SP1, will expire 30 days after installation. Once the applications have expired, the disk containing the MobileMark® 2014 installation must be re-imaged with a new copy of the operating system, and a new instance of MobileMark® 2014 must be installed.
No. MobileMark® 2014 must be installed on and run from the C:\ drive and the user account installing the benchmark must have administrator rights on the target system. In addition, MobileMark® 2014 tests must be executed from the same account that was used to install the benchmark.
Test systems with MobileMark® 2014 installed should not be connected to the Internet. Some applications installed as part of MobileMark® 2014 may be deactivated and cause the benchmark to fail if an Internet connection is detected. These applications include:
- Adobe® Acrobat® Pro X
- Adobe® Photoshop® CS5 Extended
- Adobe® Premiere® Pro CS5
Yes. MobileMark® 2014 can be installed from external media such as USB disk or memory keys. Follow the steps below to install from external media. At least 20GB of free space is required.
- Create a new folder on the target USB media. Example folder name “MM2014”
- Create a folder for the contents of DVD1 inside the “MM2014” folder created in step 1. Example folder name “MM2014\Disc1”
- Create a folder for the contents of DVD2 inside the “MM2014” folder created in step 1. Example folder name “MM2014\Disc2”
- Copy the contents of DVD1 into “MM2014\Disc1” folder
- Copy the contents of DVD2 into “MM2014\Disc2” folder
- Copy the individual files Autorun.inf, BAPCo.ico, and MobileMark2014_Setup.exe from DVD1 into folder “MM2014\Disc1”
- Copy files individual files Autorun.inf, BAPCo.ico, from DVD2 into folder “MM2014\Disc2”
- Folder “MM2014\Disc1” should contain folders: Apps, Data, Help, and Setup, and individual files: Autorun.inf, BAPCo.ico, and MobileMark2012_Setup.exe
- Folder “MM2014\Disc2” should contain folders: Apps2, Data2, and Setup2, and individual files: Autorun.inf, BAPCo.ico
- Install the benchmark by launching MobileMark2014_Setup.exe from folder “MM2014\Disc1”
Microsoft Windows operating system attempts to boost performance by caching and pre-fetching data whenever it can. Scores may be lower on the first iteration as MobileMark® is running while Windows caching and pre-fetching is in progress.
The Conditioning Run provides the opportunity for the system to take advantage of built in OS performance optimizations that may help improve scores. If selected, the Conditioning Run iteration will be executed at the beginning of a project. Typically, the Conditioning Run will be executed on AC power. When one iteration of the selected scenario has been successfully completed, the MobileMark 2014 will prompt the user to unplug the system. Once the system is unplugged, MobileMark 2014 will continue to execute the selected scenario until the system battery is completely discharged. Results from the Conditioning Run are not included in the project performance or battery life scoring.
In cases where the Conditioning Run is executed on DC (battery power), the selected iteration will be run one time. Upon completion of the Conditioning Run iteration, MobileMark 2014 will pause the test to check if the system is plugged in or running on battery power. If the system is on battery power, MobileMark 2014 will continue to execute the chosen scenario until the system’s remaining battery charge is exhausted. However, the system will only report battery life and performance qualification scores if the remaining charge in the battery is sufficient to complete one iteration of the scenario. Results from the Conditioning Run will not be included in either the performance qualification or battery life scores.
MobileMark® 2014 supports installation on US English, Brazilian Portuguese, Chinese (Simplified), French, German, Japanese, Spanish, and Russian. The operating system’s Regional Language settings must be set to US English. Additional languages may be supported in future updates. Check the MobileMark® 2014 FAQ online for the latest information.
Installation time for MobileMark® 2014 takes approximately 1 hour. Installation time may vary based on the target system configuration.
MobileMark® 2014 run time will vary according to factors such as battery design capacity, battery charge level at the start of the test, test system configuration, and the selected scenario.
Be sure that the system under test is set to the correct date and time to avoid the ‘Clock error detected’ message.
Applications within BAPCo benchmarks are not compiled by BAPCo, but by the original application vendors. All other components within BAPCo benchmarks are compiled using Microsoft Visual Studio 2008 or Microsoft Visual Studio 2010, or Microsoft Visual Studio 2012 (with default options).
This issue typically manifests itself on certain machines with special drive hidden partitions installed by OEMs on their branded machines. One can work around this issue by deleting the special hidden partition installed on the drive(s) and re-running MobileMark® 2014.
MobileMark 2014 will support Windows 7 ‘N’ versions provided that the test image has Windows Media Player version 12 installed.
Internet Explorer 9 may crash when playing Flash content on systems equipped with NVIDIA Optimus graphics. Workaround this issue by going to the NVIDIA graphics control panel and set “Preferred graphics processor” to “Integrated graphics” for IE9.
Windows 8/8.1 is designed to prompt the user for activation every 3 hours. The activation prompt may interfere with MobileMark 2014 tests. There are three ways to prevent this interference:
- Activate the Windows 8 installation using a retail license key.
- Activate the Windows 8 installation using a Key Management Server. Note, this option is only available for Windows 8 Enterprise edition
- Enable Windows 8 Test Mode
- Launch the “regedit” application
- Navigate to the key “HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control.”
- Create a new key with name “7503491f-4a39-4f84-b231-8aca3e203b94”
MobileMark 2014 should be run with DPI scaling set to 100%. Setting the DPI scaling value to something other than 100% may cause the test to fail. The Configuration tool now includes an option to automate setting the DPI scaling value to 100%.
MobileMark 2014 documentation is located on the MobileMark 2014 product page here.
Please note: The below “Error 1010” Hotfix does not apply to versions 1.5.x.x
Download Hotfix for error 1010
1) MobileMark 2014 patch 2 or SYSmark 2014 patch 2 must be installed to use this hotfix.
2) Download and extract the above zip file to the test system where error 1010 occurred.
3) If any SYSmark 2014 or MobileMark 2014 windows remain open, close them before attempting to apply the hotfix.
4) Right click the ‘Hotfix_for_Error_1010.exe’ file that you extracted from the zip and choose ‘Run as Administrator’.
5) The package will launch a command prompt window.
6) Follow the prompts to apply the hotfix.
7) Upon successful completion, press any key to close the command window. You may now attempt to run SM14 or MM14, and should no longer see error 1010.
Note: This fix applies only SM14 and MM14 patch 2. If the hotfix package cannot detect the SM14 or MM14 installation, it will exit.
Note: SYSmark 2014 v1.5 and MobileMark 2014 v1.5 with patch 1 installed no longer need this hotfix. Please download patch 1 from the product page.
The November update to Windows 10 may cause an error with Adobe Acrobat when running MobileMark 2014 ver 1.5 or SYSmark 2014 ver 1.5. Please download the following .zip archive Hotfix_for_default_PDF_reader.
- Extract the zip file to a directory
- Copy the self-extracting executable from the .zip to the test system
- Be sure all SYSmark/MobileMark windows and applications are closed
- Right click the executable and choose ‘Run as administrator’
- Follow the prompts to apply the hotfix.
- Once the hotfix completes, run the benchmark test.
For testing on Non-Engish OS, SYSmark 2018 and MobileMark 2018 installers will configure the system correctly for the supported languages, provided the operating system is installed in a supported language, i.e., German.
SYSmark and MobileMark support seven Non-English languages. Download the .pdf for the target language and follow the instructions to configure the operating system.
- BAPCo Product Configuration Guide for Non English OS (English)
- BAPCo Product Configuration Guide for Non English OS (Portuguese)
- BAPCo Product Configuration Guide for Non English OS (Chinese)
- BAPCo Product Configuration Guide for Non English OS (French)
- BAPCo Product Configuration Guide for Non English OS (German)
- BAPCo Product Configuration Guide for Non English OS (Japanese)
- BAPCo Product Configuration Guide for Non English OS (Russian)
- BAPCo Product Configuration Guide for Non English OS (Spanish)
When using BAPCo benchmarks to evaluate performance or battery life, one should install the latest production drivers from the hardware manufacturer. Drivers included with the operating system may not be device specific or newer drivers may be available from the device manufacturer. Using incorrect drivers could result in significantly reduced performance.
This update is now included as part of MobileMark 2014 v1.5 HotFix 3. Please see the MobileMark 2014 v1.5 FAQ entry for HotFix 3
Note: BAPCo has issued a HotFix which automates this workaround. Please see this FAQ entry for more information.
When testing Redstone 3 (build 1709) with MobileMark 2014 v1.5, some users may experience a smart screen warning that blocks the test from completing and reporting a result. This issue is due to changes that Microsoft has made in the way Redstone 3 warns users when the remaining battery charge is low. This warning cannot be disabled as per Microsoft design.
BAPCo continues to work with Microsoft to resolve this issue. In the meantime, if you experience this issue when testing on Redstone 3, please refer to the workaround detailed in this document MM14.smart.screen.workaround.for.RS3
The following HotFix has been issued to correct issues when testing on Redstone 3 . HotFix 3 is a cumulative update.
1) Copy hotfix package to the to target system
2) Unzip the archive to extract the hotfix installer
3) Be sure that any instances of MobileMark 2014 v1.5 patch 1 are closed before attempting to apply the hotfix package
4) Right click the extracted .exe and run as administrator
MobileMark 2014 patch 1 must be installed to apply this hot fix. The update will not apply to older versions of MobileMark 2014.
After the hotfix is applied, launch MobileMark 2014. The new version should be reported as 220.127.116.11.
Systems with Intel Dynamic Power and Thermal Framework (DPTF) driver version (8.4.11000.6436) may fail to execute and display Error 1: Errorstate::Enter: unknown error code (1) in the results browser.
You may encounter this issue when testing with older versions of SYSmark 2014 SE or MobileMark v1.5. If so, please visit either the SYSmark 2014 SE product page or the MobileMark 2014 v1.5 product page to download the latest patch and hotfix for your product. Update your benchmark according the patch or hotfix instructions and reboot the system under test. You should now be able to run SYSmark 2014 SE or MobileMark 2014 v1.5 successfully.
If you are testing SYSmark 2014 v1.5, please follow these steps to work around the issue:
1) Please visit the SYSmark 2014 v1.5 product page to download and apply any patches or hotfixes to your installation.
2) Download the Dptf_Sysmark_2014_v1.5_fix package and copy to the system under test
3) Extract the contents of the zip file
4) Right click the .reg file and choose ‘Merge’
5) Reboot the system under test
6) You should now be able to run SYSmark 2014 v1.5 successfully.
On some systems, MobileMark 2014 installation hangs when attempting to enable the direct play service and becomes unresponsive, even though the benchmark installation has completed. User experiencing this issue may work around the problem by rebooting the system to clear the installation hang. When the desktop appears, launch MobileMark 2014 and start a test.
As BAPCo benchmarks such as SYSmark and MobileMark include all the applications required to run the benchmark(s) users must be certain that any previous installation of those same applications are completely removed from the OEM preload prior to attempting to install a BAPCo product.
Microsoft provides a tool for removing Office installations. Please use this tool prior to installing any BAPCo benchmark on systems where Office is preinstalled.
Note, there may be other applications, ie, Adobe Creative Suite that will need to be removed as well. This FAQ item covers MS Office only.
If Windows 10 detects the installation of an application that is not compatible with the memory integrity feature, Windows 10 will disable the feature in order to prevent system instability. Users may notice that memory detection has been disabled Windows 10 if they install older BAPCo benchmarks such as SYSmark 2014, SYSmark 2014 SE, or MobileMark 2014.
BAPCo recommends updating to the latest generation of SYSmark 2018 and MobileMark 2018 for users who wish to test with memory integrity enabled.
Starting with Windows 10 May 2019 Update (version 1903), the SYSmark performance score can be over 10% higher when Windows Defender is turned off. To address this, BAPCo is making two changes to SYSmark in upcoming patches. First, we will document the status of Windows Defender in the results PDF file. Second, we will change the default behavior of the SYSmark System Configuration Tool to not disable Windows Defender. You can still optionally disable it if you want.
Tamper Protection can limit the ability of the SYSmark System Configuration Tool to modify some system settings. Some of those settings, like disabling Windows Defender, can affect the SYSmark performance score.