Commit Graph

62 Commits

Author SHA1 Message Date
Simon Hausmann
2c4b06a13d Centralize Mesa LLVMPipe installation on Windows
Install the DLL for general availability on all Windows systems. Due to
its distinct name it will not be picked up by anyone unless Qt
explicitly selects it.

This gives us the ability to choose the GLES backend to use in Qt.

Change-Id: Ie36f202ead7080a8f7690734d305bd27f0c9f56b
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-20 04:44:49 +00:00
Simon Hausmann
5bc43f0cc3 Fix Postgresql and Mysql environment variables
The correct paths to the headers are

    c:\utils\postgresql[x86]\pgsql\include

not

    c:\utils\postgresql[x86]\pgsql\pgsql\include

Same for libs.

Regression introduced by commit ce47238ed9
but only noticed in the 5.9 branch.

Change-Id: Ib16072c0cc1c52f743f5500bc9732c8b04e962c7
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-12 04:43:24 +00:00
Simon Hausmann
0358b6d28c Remove QNX Windows builds
We can't get those to work with the faster KVM based VMs, so disable
these builds for the time being.

Task-number: QTQAINFRA-1208
Change-Id: I74c19374d57e3f84d8f982b95aade4a779623228
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-07 18:10:08 +00:00
Simon Hausmann
303d617329 Disable tests on macOS 10.10
We cannot run graphical tests reliably on 10.10 anymore with the new
virtualization without vmware. As outlined in the referenced task, we
occasionally get the Window Server locking up. So for the time being,
this change disables testing on 10.10.

Task-number: QTQAINFRA-1198
Change-Id: I020d4a298562554838cd7e898df52438db536d7a
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2017-06-07 04:16:32 +00:00
Simon Hausmann
35057fe4b2 Clean up windows cleanup script
* There's no need to have a numbered prefix in the common directory
* Instead of deleting directories trees worth several gigabytes of data,
  just rename the directory. The underlying storage system is
  copy-on-write and so deleting the files produces more data than just
  moving the directory out of the way. Deleting the files does not
  actually free up any physical disk space.

Change-Id: I6de02d56f3485651726ea11183ce7571d5b61520
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:40 +00:00
Simon Hausmann
038d263081 Speed up Windows installation of cmake, python, opengl, mysys and Android
We're better off copying off the internal server using CIFS than using
PowerShell's incredibly slow Invoke-WebRequest.

Change-Id: Ic39cf08be70335e9309575d543155bffaceb0db0
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:34 +00:00
Simon Hausmann
999b5f5dd7 Speed up Visual Studio update installation
We have a prepared package on the internal server that has everything
necessary included. This way we do not have to download the actual
update content from the internet.

Change-Id: Ie4cb3116a4f91a47df5be1c2b2e473d7ddec9a78
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Aapo Keskimolo <aapo.keskimolo@qt.io>
2017-06-06 20:15:31 +00:00
Simon Hausmann
ce47238ed9 Improve Postgresql installation on Windows
Similar to the mysql change (parent commit) it's faster to rename any
older artifacts and install only what we need. This also generalizes the
zip archive extraction code for "dev" packages.

Change-Id: I3ad1f23efaed80fab3e0778a3e8c46317138f1ad
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:28 +00:00
Simon Hausmann
982659b081 Improve mysql installation
* Instead of deleting the files from a previous installation, which will
  not actually release any disk space but just make the qcow2 file
  larger, let's just rename the directory the old files are in.
  That is a significantly faster operation.
* When extracting the zip archive, only extract what we need for Qt:
  Libraries, binaries and headers. We can skip the installation of
  several thousand documentation files for example, which takes a long
  time.
* Instead of downloading the zip archive via PowerShell's
  Invoke-WebRequest we can get the data much faster using CopyItem off
  the CIFS share. As a bonus this is also more robust against download
  failures.

Change-Id: I9128bd70e5f4a4ebcdbf046765e76b63bad033a5
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:09 +00:00
Heikki Halmet
a4a7f71f37 MSVC 2017 updated and missing component for winrt arm
MSVC 2017 and Build Tools updated to version 15.1. Also missing
component 'Visual C++ compilers and libraries for ARM' installed
for winrt arm

Also script added to storage msvc version numbers to versions.txt

Change-Id: I015e15d5d613b178c1266a2aecee5c3270780486
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-05-03 09:51:17 +00:00
Simon Hausmann
2ea45a59a7 Improve Android SDK/NDK installation on macOS
* Remove unused installation on macOS 10.10 as we only build packages
  on 10.11.

* Replace the use of the ~1GB downloads of the zip files with a direct
  extraction via NFS. This is more resilient to flakyness in the network
  connection.

Change-Id: I1bfe5bbf00a6614233b517801daf130069094ec9
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-04-24 19:19:00 +00:00
Heikki Halmet
370fdfcefd Provisioning: MSVC 2017 Build Tools
MSVC 2017 build tools are pre-provisioned to windows 10 templates

Change-Id: I98d857101ad209acd6eea9f821e367e514359d8c
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-24 05:17:04 +00:00
Heikki Halmet
e48d653150 Provisioning: Change tier1 templates for Windows 10
These tier 1 templates includes changes needed by 5.9, but we can use
these same tier 1 templates for 5.6 also

Change-Id: I3ad0d75ab89f7b6502b758fb94f984cc6e0b7f16
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-13 12:31:58 +00:00
Tony Sarajärvi
b8211774d9 Fix provision help script DownloadURL.sh
The script neglected the result of the sha1 after download.
Even with a failure it exited nicely.

Change-Id: I6c0e25c2b764a0ad54acbe317eddf395a56d7f1c
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-04-10 17:27:24 +00:00
Heikki Halmet
e00cb3ac59 Provisioning: Bug fix patch for msvc 2015 update 3
This bug fix patch is needed for QtWebEngine

Change-Id: Id2c624461fbe27d9fd12a149281cae324bf8b14e
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2017-03-31 06:42:09 +00:00
Simo Fält
8b59bbc2a4 Coin provisioning: Use arg instead of autodetecting architecture
When provisioning windows machine, it seems that detecting architecture
by checking [System.Environment]::Is64BitProcess doesn't work properly.
Instead pass 32 or 64 as an argument when calling powershell script.

Change-Id: I890ad141b8ef194275848d0e656431753463cd2b
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-03-30 15:14:04 +00:00
Heikki Halmet
80b739b1e8 Provisioning: Move Visual Studio installation to pre-provisioning
Visual Studios will be pre-provisioned to tier1/vanilla os when visual
studio update is needed. Even if using pre-configured offline
installer with specific workload and components the installation is
still taking too long. This way we'll reduce provisioning time
significantly

Change-Id: I1f831aef648716dc640ce9cbfb4e63aa54e7afd0
Reviewed-by: Akseli Salovaara <akseli.salovaara@qt.io>
2017-03-22 09:31:06 +00:00
Heikki Halmet
460cfd01b3 Provision: Storage icu version to versions.txt
~/versions.txt will be used to storage sw version information

Change-Id: Ia84e0fc55f5bbcf4afd902ce601556defcc565a0
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-28 07:23:53 +00:00
Akseli Salovaara
880e750629 Provisioning: Update opengl32.dll x86 on Windows 10 x64
OpenGL32 update is needed for autotests to pass.
At least tst_qglbuffer vertexBuffer will fail without an update.
This update is already in place on Windows x86 and x64 CI hosts but
applied only to System32 -folders.

Change-Id: I223ffc53ecba77030385f29463c419acdc12e1a3
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:13 +00:00
Akseli Salovaara
f1f9b94b00 Provisioning: Install postgresql x86 and x64 libraries to Windows 10
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: Ied654efb2f33ab1c643f38a1ea50a127aeb7af7e
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:10 +00:00
Akseli Salovaara
fb7581847e Provisioning: Install OpenSSL x86 and x64 libraries to Windows 10 x64
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: Idada7fc41202298c6f1b2919f0f4343b717e890a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:08 +00:00
Akseli Salovaara
da4c2e21b1 Provisioning: Install MySQL x86 and x64 libraries to Windows10 x64
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: I8eea51a0bfd05bc72d1d004649211c7bc5901d97
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:06 +00:00
Heikki Halmet
ad97fd589f Provisioning: Visual Studio 2015 with update 3
Common install script for Visual Studio installation and
updates. Latest Visual Studio 2015 update is required
for releases

Change-Id: I5b41625ca7f161b4ba2d417797f8839ef175cdbc
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-07 19:25:11 +00:00
Konstantin Tokarev
25160f24b6 Upgrade CMake to 3.6.2
QtWebKit requires CMake >= 2.8.12, while only 2.8.11 is available on
Windows 10 machines.

Task-number: QTQAINFRA-996
Change-Id: I734be4cc36f64dd5aeb243b0fbe4395c75c523b3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit 77eeee64a4)
2017-02-07 19:25:07 +00:00
Konstantin Tokarev
2835a4d873 Upgrade CMake to 3.6.2 on Windows 7 builder
Task-number: QTQAINFRA-996
Change-Id: I84867a5b630c4cf1e712047c58c6cad9e5e43800
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit daa8653121)
2017-02-07 19:25:03 +00:00
Simo Fält
f65479ee0c Upgrade CMake to 3.6.2 on Windows 8
Pyside requires CMake 3.0 or higher

Change-Id: Ic3f7eb4e89cd022364b1437fa8ff4476c86c3e47
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-07 19:24:59 +00:00
Heikki Halmet
a36977963a Provision: Storage android sdk and ndk version information
Sdk and ndk version information will be storaged to versions.txt.
Versions.txt will be used for all provision scripts and later on it's
data will be printed to provision log.

Also script name changed to match both android sdk and ndk.

Change-Id: I3366e596f539099c3b587466d90905d0889398fe
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-03 05:47:05 +00:00
Heikki Halmet
6afffd6985 Provision: Script to print sw versions to provision log
This script is the last script that is run by provisioning. It will
print all storaged sw information to provision log

Change-Id: I2c76d5f7cccf9c116cb660ffcb67d3bece473da8
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-03 05:46:57 +00:00
Joerg Bornemann
32b5e271a8 Update jom to 1.1.2
The most important changes that affect the CI are:
  - Fixed bogus path in error message for failed targets.
  - Fixed error "JobClient destroyed while still acquiring."
    that could be yielded for failed builds.
  - Fixed /k option (QTCREATORBUG-17131).
  - Added the special .NOTPARALLEL target that disables parallel
    build for the containing makefile. (This one is needed to
    build QtWebEngine in debug-and-release properly.)
  - Introduced the JOMFLAGS environment variable.
    This is a MAKEFLAGS pendant. JOMFLAGS is useful to set up
    separate flags for nmake and jom.

Change-Id: Icb2462d06b000a3fe29e1e9290b294cb1e900409
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-03 05:46:51 +00:00
Heikki Halmet
0c8536fc0a Provision: Script for linux to remove preinstalled software
Script to remove tier1's preinstalled software. Script itself includes
all information about the software that needs to be removed from tier2
before provision starts to run installations. When this script is run
for certain platform that platform needs to provision all removed
software back to tier2 template. So user needs to make sure that all
needed provision script are available when calling this removal-script.
Goal is to have vanilla os when provision don't need to remove any
software before installations. When this goal is reached this script
and script calls can be removed from provision.

Change-Id: I048909800807fabb1dc599e768a90e7457fbc2cf
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-03 05:46:33 +00:00
Heikki Halmet
d4681ee690 Android sdk and ndk to provisioning for Windows 7
Installs latest Android sdk and ndk.
It also runs update for SDK API level 18, latest SDK- and platform-
tools and build-tools version 23.0.3.

Task-number: QTBUG-58369
Change-Id: Ib42f5149243fc5d262f6542e26726d1c8cbb2e8c
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-01-28 03:57:41 +00:00
Heikki Halmet
3276f075b9 Provision: Set bash_profile to read .bashrc
The method (vmware tools) of launching a process inside a virtual
machine results in the launch of a process that does not originate
from a login shell. Therefore the CI system explicitly starts the
agent with "bash -l -c" to enforce the invocation as login shell
and thus execute settings from /etc/profile, ~/.bash_profile, etc.

This brings the behavior of the system closer to what we see as
developers.

However the shell is a non-interactive shell, therefore ~/.bashrc
is not read by default.

On Linux systems such as RHEL, ~/.bash_profile is configured to still
read ~/.bashrc.

For consistency we apply the same logic to the macOS configurations.

Change-Id: I417ecd7866bc4ae1b50beb14c19b456588b4050a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-01-24 19:19:27 +00:00
Heikki Halmet
eeea4e5c3a Add Android NDK/SDK to provision for macOS 10.10 and 10.11
Install latest Android sdk and ndk.
It also runs update for SDK API level 18

Task-number: QTBUG-58370
Change-Id: I1bf8cb42dd3e753be27fa9fcd2c1ac8ccbdb87c3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-01-24 19:19:24 +00:00
Simon Hausmann
3035f20934 Add CI build configurations
These files were previously stored in the CI source tree, but they do
belong here. They describe the different configurations under which we
want to build Qt and its submodules, in terms of underlying virtual
machine template, host OS, target OS and CI related features that may
also map to Qt configuration features.

Task-number: QTQAINFRA-1074
Change-Id: I7ce48e3b15da757166eeac7531478d3746c8aa7f
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2017-01-11 13:57:45 +00:00
Heikki Halmet
2cf8327999 Provision: Cmake 3.6.2
Red Hat 6.6 needs cmake for GammaRay
OpenSUSE 42.1 and Ubuntu 14.04 needs cmake for pyside

Change-Id: I64b25be70057b86be7bfc96fb07446f7727bbefe
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-01-11 10:05:59 +00:00
Tony Sarajärvi
8266919b6e Add provision helper file to install compressed packages
Change-Id: I6a7e1af7c91bd11a0cc6379f8bb7762acf42eae5
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-01-11 10:05:50 +00:00
Tony Sarajarvi
2059ce3d2d Add DownloadURL.sh provisioning helper script
Change-Id: I64bef8470555e33772b72847aa76e0e9b4720696
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-01-11 10:05:40 +00:00
Tony Sarajarvi
da48107b08 Add provisioning try-catch helper script for bash
Change-Id: Ia61e503ed5315546421fdeb2f4b8b1be37dab3e9
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-01-11 10:05:21 +00:00
Simon Hausmann
5f40fd649e Improve build performance on Windows
Disable the windows search indexing service, which has a tendency to start
indexing source code archives that we've just extracted. We are not interested
in this service.

Change-Id: I1d3f5178a035bc951499d993641a9315db9712ca
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2017-01-09 13:37:16 +00:00
Simo Fält
167e4fc36f Provisioning: Disable Windows Defender
This will reduct the timeouts coin has had recently with win machines.

Change-Id: I32118f226194a3ea5e3c85d24103ec8341925d3c
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-01-06 12:15:36 +00:00
Simon Hausmann
8a60455e61 Remove shutdown "helpers" for macOS
It turns out that they don't quite work the way we expected them to, and
instead we Coin is going to shut down the VMs directly at the right
time.

Change-Id: I9f7a6f2f898002a25544fc513a0c2414fcd5c263
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2016-12-21 11:32:28 +00:00
Simo Fält
ff1ad70a19 Provision: Install pyside python dependencies to Windows
To test successfully pyside dependencies with Coin, we need
python virtual env.

Change-Id: I59c620b7da9c78d2b5c55e233d75597b747a8e84
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2016-12-19 21:39:35 +00:00
Simo Fält
fc6717cf23 Provision: Install pyside python dependencies to Linux
To test successfully pyside dependencies with Coin, we need
python-dev packages and virtual env.

Change-Id: I29a5eed769798da4e4799eb5ccdff1cd032070d5
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2016-12-19 21:39:26 +00:00
Heikki Halmet
e86ba9e1f7 Provision: Android openssl 1.0.2j for Windows 7
We are currently using too old openssl for Android builds in Windows 7.
Openssl version need to be at least 1.0.1.

This script will configure openssl 1.0.2j for Android using tools
from msys.

Change-Id: I9ab9a3b4f9f3953cf99bf15fee631f5a7961b42d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2016-12-16 14:44:55 +00:00
Simon Hausmann
16d05b088e Improve disabling of screensaver on macOS
Setting the idleTime on currentHost works well, until the hardware UUID changes
and thus the host ID for the setting. With kvm that happens quite frequently,
with vmware less but also occasionally (deep clone for example). By disabling
the screensaver on start-up we can cover all cases quite easily.

Change-Id: Ibdcf404430656f9caa47386ca3c732366ddfc8c7
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2016-12-08 08:18:56 +00:00
Jędrzej Nowacki
0ce4f02c5f Suppress shutdown indicator in Ubuntu
We want to quit machines without being asked, if we are
really sure.

Change-Id: Ic7156f5b0fe236c0270d8c2e4fcc322a82776caf
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-12-07 13:48:24 +00:00
Simon Hausmann
be37958406 Disable the screensaver on macOS definitely and permanently
Change-Id: I8856eebfe91b9e1cc2cc318a1e7ea80348f1486e
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2016-12-07 13:47:48 +00:00
Simon Hausmann
3718450891 Fix paths for provisioning on macOS
The directory names were missing the architecture, otherwise the content was
unused.

Change-Id: I0316da5c4b1352012f27ce330c3319a8aedf01ea
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2016-12-07 13:47:46 +00:00
Frederik Gladhorn
d0552ebf2d Fix macOS provisioning not shutting down cleanly
After provisioning we want the machine to be sensibly shut down since we
will use the disk image later. VSphere doesn't manage to shut down 10.8
VMs. Help out by calling shutdown.
For the other OS X/macOS versions, this is also desirable since
controlling the power state of these machines is hard with different
virtualization solutions (we're looking at OpenNebula/KVM).

Change-Id: Ieabc14f9c8c1af9421a0ef91dd670f4790a84461
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2016-12-07 13:47:42 +00:00
Frederik Gladhorn
a7c6550fff Add basic Ubuntu 16.04 provisioning
Change-Id: Iaae5bb295bfd48cc5bc036baef9b0fa899d393ef
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2016-11-25 11:05:49 +00:00