Commit Graph

83 Commits

Author SHA1 Message Date
Simo Fält
92d1bdcb43 Provisioning: Fix shasum path
Commit c48355ebb5 changed
the path of shasum, which was not intended. While the
DownloadURL.sh utility is used by rhel in addition to
macOS we can't rely on /usr/bin/shasum but we have to
use the one found from PATH.

Change-Id: I67bb37dcb72ba3d8d1f279646e44228b27137ec9
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-08 04:34:41 +00:00
Simo Fält
c48355ebb5 Provisioning: Fix DownloadURL utility's shasum check
MacOS 10.10 has older shasum tool, which does not support oneliner
verification.

Change-Id: If00b846d5772aa1991960675d022249d63e7bd77
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-05 11:36:43 +00:00
Tony Sarajärvi
a7b1b00350 Provisioning: Hide powershell progress bar for performance
Change-Id: I74abdcc2e7d2dccf4f85cd64fe5fb4b56e54d613
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-09-04 12:12:17 +00:00
Joni Jäntti
dbf737a5c0 Fix Add-Path helper function and start using it in scripts
This function was broken and only added the latest addition to the
actual PATH.

Change-Id: Ida352c7b605c1b9748bb98e4f51ce68a765c3ce3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-04 12:11:17 +00:00
Akseli Salovaara
4e1f4665c5 Build Windows MSVC2013 & MSVC2015 32bit binaries on 64bit host
From Qt 5.6.3 onwards building QtWebEngine Windows 32bit binaries
requires Windows 64bit machine. Linking requires more RAM than
Windows 32bit can address. Change MSVC2013 & MSVC2015
integration host machines accordingly.

Change-Id: I5b88536507dcf59efd124d4e8c3ead79772021fb
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2017-09-04 07:10:14 +00:00
Aapo Keskimolo
42b502f7bc Add UploadConfigureExe feature
We used to solve the requirement for uploading configure.exe from
product version, but we can add this as a feature and remove the
dependency to product version.

Change-Id: If671e7193771453cd5cd41e881875c44ff0992fb
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-09-02 11:32:38 +00:00
Tony Sarajärvi
3ca7658781 Provision vanilla Windows 8.1 x86 and use it
Replaces existing Windows 8 x86 builds with
a vanilla tier 1 image that gets all needed
software provisioned except for VS2013 that is
pre-installed.
Also modifies existing provisioning scripts
under common to support this 32bit OS.

Task-number: QTQAINFRA-1383
Change-Id: I495cb2d78b4d2d8b2abb9af1254fff0a78a570d9
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-09-01 13:02:12 +00:00
Joni Jäntti
05dd83c73a provisioning: windows: ICU
Fixed ICU script to actually install ICU instead of just setting the
environment variables. It is required to build qtwebkit.

Change-Id: I8881f9c11599d99b4d400595c9b929cf6f37088a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-09-01 08:05:57 +00:00
Joni Jäntti
30660bf987 Make CI use a new provisioned Win 10 template
Added all the provisioning scripts that install required
programs and make changes to certain settings. These are needed to
build qt5. This replaces the old Windows 10 template.

Task-number: QTQAINFRA-1263
Change-Id: Ieeb2ae3c53841644996d99ca86228df6f825a63a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2017-09-01 08:05:54 +00:00
Simo Fält
a99d3432f7 Install python3 on all platforms
Python3 is required in 5.6 by pyside-setup.

Task-number: QTAUTO-345
Change-Id: If11ca425fa08cce8a518710ded0a6ec58159343d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit a5f3f8ed48)
(cherry picked from commit a765838889)
(cherry picked from commit 1859f9b26c)
2017-08-30 19:57:54 +00:00
Simon Hausmann
1bd2b2acfa Fix code duplication for cmake installation on Linux
Share the code for installing cmake on Linux.

Change-Id: I535f2696151ea4db26dc7194a1fe7016799ed0e2
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-30 17:16:59 +00:00
Joni Jäntti
c7a9bb92e1 Fix adding to path lines for Windows provisioning scripts
Currently the path addition command will forget the previous addition
when adding a new string. This hasn't been found out because python
was the only script using this command. Now we also need to set the
path for ruby, so the python string is left out.

Change-Id: I39721d78111550189e52a9f095030033f9660c0a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
2017-08-30 17:16:56 +00:00
Tony Sarajärvi
835ef953f0 Provisioning: Fine tune cmake provisioning script
Add version printing and move version strings to variables for
easier reuse.

Change-Id: I5372872bf455609dfb7cc7bbcf7a110cd71933fb
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-08-30 17:16:52 +00:00
Tony Sarajärvi
f05289d622 Provisioning: Add $PATH properly in .bashrc
If we, in a provisioning script, add $PATH to another
file, that $PATH gets expanded there and now. We need it
written as $PATH to the target file, so that it gets expanded
when the other file is run. Not at the moment of running this
provisioning script.

Change-Id: Id963f0f8b1cc74b02d35c1a91eb974ed816839ab
Reviewed-by: Simo Fält <simo.falt@qt.io>
2017-08-30 17:16:48 +00:00
Heikki Halmet
5df417aeb9 Provisioning: Disable system updates
This change will disable system updates from App Store to prevent
unnecessary network load

Change-Id: I7915ec6ffce681a850abef59d58f953effd6deb0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-16 06:03:50 +00:00
Joni Jäntti
bd3ddeddd2 Make CI use a new win 8.1 provisioning instead of win 8
Added all the provisioning scripts that install required
programs and make changes to certain settings. These are needed to
build qt5. This replaces the old Windows 8 template.

Made all the Windows platforms use the scripts under
provisioning/common/ instead of duplicate files.

Task-number: QTQAINFRA-1357
Change-Id: Ic3f0b50079b212437fff711379f243753ab84872
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-11 16:49:10 +00:00
Tony Sarajärvi
043e54c7c6 Update provisioning scripts with new intra.qt.io address
Change-Id: I9b4c5e9d0fe8e0e84596ad9086f563d11e97930b
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-08 20:05:08 +00:00
Simon Hausmann
49ccf716dc Remove unused provisioning scripts
RHEL 7.2, Ubuntu 15.04, 16.04 and OpenSuSE 42.1 are not used in the Qt
5.6 branch. Let's remove the scripts to avoid confusion.

Change-Id: I3829ea320ad77ebd25c9cf5bebfeed89f37f44dc
Reviewed-by: Simo Fält <simo.falt@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2017-08-04 19:22:40 +00:00
Frederik Gladhorn
d1d36cbbf1 Add Documentation feature to one Ubuntu 14.04 build
When creating packages, we build the docs on one configuration.
Previously that was done by magic inside the continuous integration
system. Make it a bit less magic by having a flag for it.

Change-Id: Iee5be7037023929b319beffaacb1592dc87d6078
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-04 16:12:56 +00:00
Simo Fält
c9d5341307 Provisioning: Fix virtualenv in macOS 10.11
There are multiple Python installations in our macOS 10.11 Tier1
template. Most of them are pretty old and not even working properly.
One of the affected Python tool is virtualenv, which atm is creating
Python 2.6 env by default even if Python 2.7 is the default Python.

Change-Id: I8bc0900976a9dc79637539802345175c85b7b433
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-07-19 05:54:01 +00:00
Jędrzej Nowacki
eca392d3f3 Use updated Ubuntu 14.04 in CI
The system packages in the base template were
updated via apt-get on 12.07.2017.

We hoped that it will solve mysterious problem of
a network interface being randomly disabled. Sadly
we had no luck, but getting all security updates
also make sense.

Change-Id: I34d7a56a2c5530376f74bb35faf461e2e18da14a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-07-18 16:59:07 +00:00
Simon Hausmann
2c4b06a13d Centralize Mesa LLVMPipe installation on Windows
Install the DLL for general availability on all Windows systems. Due to
its distinct name it will not be picked up by anyone unless Qt
explicitly selects it.

This gives us the ability to choose the GLES backend to use in Qt.

Change-Id: Ie36f202ead7080a8f7690734d305bd27f0c9f56b
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-20 04:44:49 +00:00
Simon Hausmann
5bc43f0cc3 Fix Postgresql and Mysql environment variables
The correct paths to the headers are

    c:\utils\postgresql[x86]\pgsql\include

not

    c:\utils\postgresql[x86]\pgsql\pgsql\include

Same for libs.

Regression introduced by commit ce47238ed9
but only noticed in the 5.9 branch.

Change-Id: Ib16072c0cc1c52f743f5500bc9732c8b04e962c7
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-12 04:43:24 +00:00
Simon Hausmann
0358b6d28c Remove QNX Windows builds
We can't get those to work with the faster KVM based VMs, so disable
these builds for the time being.

Task-number: QTQAINFRA-1208
Change-Id: I74c19374d57e3f84d8f982b95aade4a779623228
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-07 18:10:08 +00:00
Simon Hausmann
303d617329 Disable tests on macOS 10.10
We cannot run graphical tests reliably on 10.10 anymore with the new
virtualization without vmware. As outlined in the referenced task, we
occasionally get the Window Server locking up. So for the time being,
this change disables testing on 10.10.

Task-number: QTQAINFRA-1198
Change-Id: I020d4a298562554838cd7e898df52438db536d7a
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2017-06-07 04:16:32 +00:00
Simon Hausmann
35057fe4b2 Clean up windows cleanup script
* There's no need to have a numbered prefix in the common directory
* Instead of deleting directories trees worth several gigabytes of data,
  just rename the directory. The underlying storage system is
  copy-on-write and so deleting the files produces more data than just
  moving the directory out of the way. Deleting the files does not
  actually free up any physical disk space.

Change-Id: I6de02d56f3485651726ea11183ce7571d5b61520
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:40 +00:00
Simon Hausmann
038d263081 Speed up Windows installation of cmake, python, opengl, mysys and Android
We're better off copying off the internal server using CIFS than using
PowerShell's incredibly slow Invoke-WebRequest.

Change-Id: Ic39cf08be70335e9309575d543155bffaceb0db0
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:34 +00:00
Simon Hausmann
999b5f5dd7 Speed up Visual Studio update installation
We have a prepared package on the internal server that has everything
necessary included. This way we do not have to download the actual
update content from the internet.

Change-Id: Ie4cb3116a4f91a47df5be1c2b2e473d7ddec9a78
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Aapo Keskimolo <aapo.keskimolo@qt.io>
2017-06-06 20:15:31 +00:00
Simon Hausmann
ce47238ed9 Improve Postgresql installation on Windows
Similar to the mysql change (parent commit) it's faster to rename any
older artifacts and install only what we need. This also generalizes the
zip archive extraction code for "dev" packages.

Change-Id: I3ad1f23efaed80fab3e0778a3e8c46317138f1ad
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:28 +00:00
Simon Hausmann
982659b081 Improve mysql installation
* Instead of deleting the files from a previous installation, which will
  not actually release any disk space but just make the qcow2 file
  larger, let's just rename the directory the old files are in.
  That is a significantly faster operation.
* When extracting the zip archive, only extract what we need for Qt:
  Libraries, binaries and headers. We can skip the installation of
  several thousand documentation files for example, which takes a long
  time.
* Instead of downloading the zip archive via PowerShell's
  Invoke-WebRequest we can get the data much faster using CopyItem off
  the CIFS share. As a bonus this is also more robust against download
  failures.

Change-Id: I9128bd70e5f4a4ebcdbf046765e76b63bad033a5
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:09 +00:00
Heikki Halmet
a4a7f71f37 MSVC 2017 updated and missing component for winrt arm
MSVC 2017 and Build Tools updated to version 15.1. Also missing
component 'Visual C++ compilers and libraries for ARM' installed
for winrt arm

Also script added to storage msvc version numbers to versions.txt

Change-Id: I015e15d5d613b178c1266a2aecee5c3270780486
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-05-03 09:51:17 +00:00
Simon Hausmann
2ea45a59a7 Improve Android SDK/NDK installation on macOS
* Remove unused installation on macOS 10.10 as we only build packages
  on 10.11.

* Replace the use of the ~1GB downloads of the zip files with a direct
  extraction via NFS. This is more resilient to flakyness in the network
  connection.

Change-Id: I1bfe5bbf00a6614233b517801daf130069094ec9
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-04-24 19:19:00 +00:00
Heikki Halmet
370fdfcefd Provisioning: MSVC 2017 Build Tools
MSVC 2017 build tools are pre-provisioned to windows 10 templates

Change-Id: I98d857101ad209acd6eea9f821e367e514359d8c
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-24 05:17:04 +00:00
Heikki Halmet
e48d653150 Provisioning: Change tier1 templates for Windows 10
These tier 1 templates includes changes needed by 5.9, but we can use
these same tier 1 templates for 5.6 also

Change-Id: I3ad0d75ab89f7b6502b758fb94f984cc6e0b7f16
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-13 12:31:58 +00:00
Tony Sarajärvi
b8211774d9 Fix provision help script DownloadURL.sh
The script neglected the result of the sha1 after download.
Even with a failure it exited nicely.

Change-Id: I6c0e25c2b764a0ad54acbe317eddf395a56d7f1c
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-04-10 17:27:24 +00:00
Heikki Halmet
e00cb3ac59 Provisioning: Bug fix patch for msvc 2015 update 3
This bug fix patch is needed for QtWebEngine

Change-Id: Id2c624461fbe27d9fd12a149281cae324bf8b14e
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2017-03-31 06:42:09 +00:00
Simo Fält
8b59bbc2a4 Coin provisioning: Use arg instead of autodetecting architecture
When provisioning windows machine, it seems that detecting architecture
by checking [System.Environment]::Is64BitProcess doesn't work properly.
Instead pass 32 or 64 as an argument when calling powershell script.

Change-Id: I890ad141b8ef194275848d0e656431753463cd2b
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-03-30 15:14:04 +00:00
Heikki Halmet
80b739b1e8 Provisioning: Move Visual Studio installation to pre-provisioning
Visual Studios will be pre-provisioned to tier1/vanilla os when visual
studio update is needed. Even if using pre-configured offline
installer with specific workload and components the installation is
still taking too long. This way we'll reduce provisioning time
significantly

Change-Id: I1f831aef648716dc640ce9cbfb4e63aa54e7afd0
Reviewed-by: Akseli Salovaara <akseli.salovaara@qt.io>
2017-03-22 09:31:06 +00:00
Heikki Halmet
460cfd01b3 Provision: Storage icu version to versions.txt
~/versions.txt will be used to storage sw version information

Change-Id: Ia84e0fc55f5bbcf4afd902ce601556defcc565a0
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-28 07:23:53 +00:00
Akseli Salovaara
880e750629 Provisioning: Update opengl32.dll x86 on Windows 10 x64
OpenGL32 update is needed for autotests to pass.
At least tst_qglbuffer vertexBuffer will fail without an update.
This update is already in place on Windows x86 and x64 CI hosts but
applied only to System32 -folders.

Change-Id: I223ffc53ecba77030385f29463c419acdc12e1a3
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:13 +00:00
Akseli Salovaara
f1f9b94b00 Provisioning: Install postgresql x86 and x64 libraries to Windows 10
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: Ied654efb2f33ab1c643f38a1ea50a127aeb7af7e
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:10 +00:00
Akseli Salovaara
fb7581847e Provisioning: Install OpenSSL x86 and x64 libraries to Windows 10 x64
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: Idada7fc41202298c6f1b2919f0f4343b717e890a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:08 +00:00
Akseli Salovaara
da4c2e21b1 Provisioning: Install MySQL x86 and x64 libraries to Windows10 x64
Install both x86 and x64 libraries in order to enable Windows x86
builds on Windows x64 hosts. Environment variables are used by Coin.

Change-Id: I8eea51a0bfd05bc72d1d004649211c7bc5901d97
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-10 06:14:06 +00:00
Heikki Halmet
ad97fd589f Provisioning: Visual Studio 2015 with update 3
Common install script for Visual Studio installation and
updates. Latest Visual Studio 2015 update is required
for releases

Change-Id: I5b41625ca7f161b4ba2d417797f8839ef175cdbc
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-07 19:25:11 +00:00
Konstantin Tokarev
25160f24b6 Upgrade CMake to 3.6.2
QtWebKit requires CMake >= 2.8.12, while only 2.8.11 is available on
Windows 10 machines.

Task-number: QTQAINFRA-996
Change-Id: I734be4cc36f64dd5aeb243b0fbe4395c75c523b3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit 77eeee64a4)
2017-02-07 19:25:07 +00:00
Konstantin Tokarev
2835a4d873 Upgrade CMake to 3.6.2 on Windows 7 builder
Task-number: QTQAINFRA-996
Change-Id: I84867a5b630c4cf1e712047c58c6cad9e5e43800
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit daa8653121)
2017-02-07 19:25:03 +00:00
Simo Fält
f65479ee0c Upgrade CMake to 3.6.2 on Windows 8
Pyside requires CMake 3.0 or higher

Change-Id: Ic3f7eb4e89cd022364b1437fa8ff4476c86c3e47
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-07 19:24:59 +00:00
Heikki Halmet
a36977963a Provision: Storage android sdk and ndk version information
Sdk and ndk version information will be storaged to versions.txt.
Versions.txt will be used for all provision scripts and later on it's
data will be printed to provision log.

Also script name changed to match both android sdk and ndk.

Change-Id: I3366e596f539099c3b587466d90905d0889398fe
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-03 05:47:05 +00:00
Heikki Halmet
6afffd6985 Provision: Script to print sw versions to provision log
This script is the last script that is run by provisioning. It will
print all storaged sw information to provision log

Change-Id: I2c76d5f7cccf9c116cb660ffcb67d3bece473da8
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-02-03 05:46:57 +00:00
Joerg Bornemann
32b5e271a8 Update jom to 1.1.2
The most important changes that affect the CI are:
  - Fixed bogus path in error message for failed targets.
  - Fixed error "JobClient destroyed while still acquiring."
    that could be yielded for failed builds.
  - Fixed /k option (QTCREATORBUG-17131).
  - Added the special .NOTPARALLEL target that disables parallel
    build for the containing makefile. (This one is needed to
    build QtWebEngine in debug-and-release properly.)
  - Introduced the JOMFLAGS environment variable.
    This is a MAKEFLAGS pendant. JOMFLAGS is useful to set up
    separate flags for nmake and jom.

Change-Id: Icb2462d06b000a3fe29e1e9290b294cb1e900409
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-02-03 05:46:51 +00:00