Revision control
++ QMLBenchrunner
++ ==============
++ QMLBenchrunner is a simple script and python application to programmatically
++ clone a Qt Git repository, run associated QML Benchmark tests, and post
++ results to the qt testresults timeseries database.
++
++
++ Prerequisites
++ -------------
++ -Qt library
++ -initialized and configured Qt modules
++ -Qmlbench repository
++ - Python3
++ - 'requests' module
++
++ Windows-specific Prerequisites:
++ * ActivePerl - Download and install from http://www.activestate.com/Products/activeperl/index.mhtml
++ * GPerf - Download and install from http://gnuwin32.sourceforge.net/downlinks/gperf.php
++ * (Qt 5.x) Flex & Bison for windows - Included with qmlbenchrunner, or downloadable from
++ If Win Flex-Bison is downloaded, you must rename the executables to "flex.exe" and "bison.exe"
++ and specify the location of your flex-bison executables using the powershell parameter specified
++ in the parameters section below.
++ * VS Build Tools 2019 (VS 2017 is supported for Qt 5.x) - https://visualstudio.microsoft.com/downloads/
++ * JOM - A version is included with qmlbenchrunner. If a new version is required, it can be found at
++ * (Optional) IncrediBuild - https://www.incredibuild.com/
++
++
++ Usage
++ -----
++ QMLBenchrunner will build the required Qt Git repositories as well as qmlbench into the local machine.
++
++ Always set environment variables TIMESCALEDBUSER and TIMESCALEDBPASSWORD in the console before calling the build_and_test
++ script. Results will still be saved to disk in "results.json" even if unable to write to the database.
++
++ Optional environment variables:
++
++ TIMESCALEDBUSER=username #Set if writing to a database.
++ TIMESCALEDBPASSWORD=password #Set if writing to a database.
++ BADTESTS=/path/to/bad/test #Specify a space-separated list of known bad tests or directories
++ that are too difficult for the client to complete and would cause a crash.
++ EXTRA_CONFIGURE_ARGS="string of additional Configure arguments"
++
++ ### Linux: ###
++ Some machines may require XAUTHORITY to be specified. If problems are encountered, set XAUTHORITY
++ and DISPLAY in the console before running this script.
++
++ **build_and_test.sh arguments are strictly positional. Do not skip arguments that are required.**
++
++ Args:
++ QtVersion (required) | MachineName (required) | BuildCores (required) |
++
++ Example:
++ export TIMESCALEDBUSER=dbuser1
++ export TIMESCALEDBPASSWORD=dbuser1password
++ export XAUTHORITY=/home/user1/.Xauthority
++ export DISPLAY=:0
++ qmlbenchrunner/build_and_test.sh 5.15 $NODE_NAME 8
++
++ ### Windows: ###
++ Qmlbenchrunner should be executed from Powershell for best compatibility. Because QMLBench is a
++ graphical application, if qmlbenchrunner is being executed via a jenkins slave, the slave must use
++ the java web start method. Running jenkins as a Windows service will not display QMLBench on the
++ real user desktop.
++
++ Args:
++ QtVersion (Required)
++ MachineName (Required)
++ BuildCores (Optional)
++ Annotate (Optional)
++ QtDeclarativeVersion (Optional)
++ FlexBisonDir = (Optional, defaults to "[current working directory]\flex_bison\" if omitted)
++
++ Example:
++ $env:TIMESCALEDBUSER=dbuser1
++ $env:TIMESCALEDBPASSWORD=dbuser1password
++ $env:BADTESTS=""
++ .\qmlbenchrunner\build_and_test_windows.ps1 -QtVersion 5.15 -BuildCores 7 -MachineName
++ $ENV:NODE_NAME -FlexBisonDir C:\flex_bison -Annotate
++
++ ### macOS: ###
++ See instructions for Linux. Some trouble has been observed with Jenkins macOS clients not setting
++ PATH correctly. If you experience issues when submitting results to a database, make sure that
++ Python3 is set in the PATH environment variable used by Jenkins scripts.
++
++ Example:
++ export PATH+=:/Library/Frameworks/Python.framework/Versions/3.6/bin/
++ qmlbenchrunner/build_and_test.sh dev $NODE_NAME 6 annotate
++
++ ### Embedded ###
++ The build_and_test_embedded.sh script is intended to be used with a Jenkins host, but can be used
++ on it' own. The script file will need to be modified based on your specific environment as detailed
++ below.
++
++ Terminology for this section:
++ * Host - The machine this script will run on.
++ * Client - The target embedded device that will execute QMLBench.
++
++ Assumptions:
++ 1) The host is configured for cross-compilation to a given target device.
++ 2) An official Boot2Qt SDK is installed on the host.
++ 3) The target embedded device (client) is accessible via SSH over a local network by IP and the
++ host has connected to it at least once.
++ 4) The host is accessible via ssh over a local network by IP and the host's ssh configuration is configured to accept connections from the client.
++
++ Required script alterations:
++ 1) Update the sysrootDir with your Boot2Qt version number.
++ 2) Update the user and IP of the host in order to pass the results.json file back from the
++ client.
++
++ Qt5 Required host configuration (customize for your target device):
++
++ DEVMKSPEC=linux-imx6-g++
++ DEVNAME=apalis-imx6
++ CROSSCOMPILE=x86_64-pokysdk-linux/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-
++ ARMSYSROOT=cortexa9hf-neon-poky-linux-gnueabi
++ DEVIP=IP of the target device to execute the tests on.
++
++ Qt6 Required host configuration:
++ CMAKE_GENERATOR="Unix Makefiles"
++ LB_TOOLCHAIN: Full path of the Boot2Qt sdk toolchain to use when cross-compiling.
++ TOOLCHAIN_FILE: Full path of the cmake toolchain file to use when cross-compiling.
++ DEVIP: IP of the target device to execute the tests on.
++
++ Running the script:
++
++ **build_and_test_embedded_[qt5|qt6].sh arguments are strictly positional. Do not skip arguments that are
++ required.**
++
++ Args:
++ QtVersion (required) | MachineName (required) | BuildCores (required for qt5, omit in qt6) |
++ Annotate (required, set to "False" if not desired) |
++ QtDeclarativeVersion (optional, leave missing if same as main QtVersion)
++
++ qmlbenchrunner/build_and_test_embedded.sh 5.15 $NODE_NAME 8 annotate
++
++ Required client configuration:
++ 1) Verify that at least 1GB of free space exists on the client.
++
++ NB! This script will wipe out /opt/qt on the device. Verify that this directory does not
++ contain anything you want to keep.