Commit Graph

99 Commits

Author SHA1 Message Date
Frederik Gladhorn
24f6a2f28a Provisioning on windows: Fix deletion of utils
We want to be able to re-provision machiens without errors.
The directory doesn't exist on the machine any more.

Change-Id: I2b5b92037492e51aeb4d041d62f371b8546cb0a3
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-02-05 14:13:08 +00:00
Frederik Gladhorn
324a16f67c Fix provisioning: NotePad++ path on 64 bit windows
We install the 64 bit version of np++ which is installed in the regular
program files folder.

Change-Id: I702210eaf7dfae75bbfcfd9cc98d8ed96d775f61
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
(cherry picked from commit b86b3a757b)
2018-02-02 06:04:21 +00:00
Frederik Gladhorn
9348e3ca55 Fix provisioning: Wait for msvc-2013 update 5 to finish
Installing Update 5 for Visual Studio 2013 Update 5 (KB2829760)...
Removing C:\Windows\Temp\vs12-kb2829760.exe ...
Caught System.UnauthorizedAccessException
Access to the path 'C:\Windows\Temp\vs12-kb2829760.exe' is denied.

Change-Id: I9fe136f0361ee21e38563dd17bf76bf4fd53747f
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
(cherry picked from commit b1198d3402)
2018-02-02 06:04:17 +00:00
Simo Fält
ee20f27b46 Partially revert "Use vanilla OS X 10.11"
This partially reverts beecc6e86e
but also amends the script, which is not working with
vanilla template. We need the virtualenv fix for pyside.

Change-Id: I12c5f7a7399bd92729e81a1d2a35bceb990261ae
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-01-29 09:06:17 +00:00
Tony Sarajärvi
9c36723454 Use vanilla Windows 10 x86_64
Provisioning scripts taken from 5.9 @ commit
1540a87590

Change-Id: Iac96271bb2b08196463376b1327722c78f92d18a
Reviewed-by: Matti Paaso <matti.paaso@qt.io>
2018-01-24 16:32:35 +00:00
Heikki Halmet
b2efef4875 Split Common folder based on operating system families
This change will split common folder to four different folders:
Linux, Unix, Windows and MacOS. Each of these folders includes scripts
which are common for their operating system familes

Task-number: QTQAINFRA-1451
Change-Id: I0668885da43222663a54521ce85653da697425be
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-01-24 16:32:25 +00:00
Tony Sarajärvi
8c657080f6 Temporarily remove iOS due to code signing issues
Task-number: QTBUG-64477
Change-Id: Ib068425cea85cd82be8192061daa92df01f5233b
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2018-01-22 12:11:59 +00:00
Jan Arve Sæther
bc2b175b72 Provisioning: Disable NTP synchronization for Windows
During CI, it has been observed that some processes didn't leave their
wait state. (So they were "hanging")
During analysis of the event log on that CI node, we found out that at the
same time those processes entered this state, the NTP client set a new time
and indeed the local time of the CI node changed (it was
changed 2 hours forward in our case).
For the record, the processes we observed this with was jom.exe and
cc1plus.exe (the latter is the g++ compiler in the mingw toolchain)).

We therefore have a strong suspicion that the NTP time change was causing
the CI hang, so we simply disable it.
The virtual machine should anyway get the correct time from its VM host
when its started.

Change-Id: I1204063399e2a4741b7b26b79858759c1bc5eab2
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
(cherry picked from commit 3653630d22)
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-01-22 12:11:57 +00:00
Heikki Halmet
d3faf2f0df Provisioning: Install signing tools to osx 10.11
Install needed certificates and signing tools which are
needed when signing release packages

Task-number: QTQAINFRA-1617
Task-number: QTBUG-64477
Change-Id: Ifaefd0e93baa396479380df0b8b99b8b2f1e2017
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-01-22 12:11:54 +00:00
Tony Sarajärvi
beecc6e86e Use vanilla OS X 10.11
The old version takes up 60GB of disk space which we need to deploy
to all hosts building Qt. As this adds up, should use the same
tier1 images as all the other branches also use.

Task-number: QTQAINFRA-1629
Task-number: QTBUG-64729
Change-Id: I132b65c381a097333577f3132dc61b87da1c78da
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-01-22 12:11:50 +00:00
Liang Qi
e7472f4308 Merge remote-tracking branch 'origin/5.6.3' into 5.6
Conflicts:
	.gitmodules

Change-Id: I3f5310cfb9b839e622ab606fd674a58d50515c0c
2017-09-18 09:42:25 +02:00
Heikki Halmet
12fd0d5b9b Ensure 'clean' android Installation
Remove old android installation and make sure new 'clean' android
folder is created.

Change-Id: I4c561797791e08976b07a194c2c9ed2f5aa302bb
Reviewed-by: Simo Fält <simo.falt@qt.io>
2017-09-15 10:38:24 +00:00
Tony Sarajärvi
b0a9259c6e Provision and build QNX660 on Win 10 x64
Changes the mingw from 4.9 to 5.3.

Task-number: QTQAINFRA-1208
Change-Id: I360b9806c35e597a4b9fd0d57c10eebfb8643905
Reviewed-by: Sami Nurmenniemi <sami.nurmenniemi@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2017-09-14 12:08:39 +00:00
Tony Sarajärvi
2a5f54d07e provisioning: install MinGW 530 and 630 for Win10 x64
MinGW is an alternative option for Visual Studio. It's
used to create desktop builds and for cross compiling
Android and QNX. At this point it is still unclear which
version is going to be used for what, but as both are
supported, let's think ahead and provide them both.
This also allows us to debug with the other, if the
other is causing problems.

Task-number: QTQAINFRA-1289
Change-Id: I2a922669c07d49370e608c093be80481c8613dc7
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2017-09-14 12:08:36 +00:00
Tony Sarajärvi
892833cda9 Provisioning: Invoke installer with Start-Process
By launching an installer with Start-Process and -Wait,
we make sure no handles are left holding on to the
installer when we try to delete it at cleanup.

Change-Id: Ic9b700ea1ce9e70089ba4cb7e91d9765ca976333
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2017-09-14 12:08:32 +00:00
Tony Sarajärvi
a9cf936a63 Provision: Move Python3 scripts from win8 to win8.1 folder
Task-number: QTAUTO-345
Task-number: QTBUG-63133
Change-Id: I739e18a7fcb1abc72092f91662e506cfffcecb0c
Reviewed-by: Dominik Holland <dominik.holland@pelagicore.com>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2017-09-14 12:08:28 +00:00
Simo Fält
92d1bdcb43 Provisioning: Fix shasum path
Commit c48355ebb5 changed
the path of shasum, which was not intended. While the
DownloadURL.sh utility is used by rhel in addition to
macOS we can't rely on /usr/bin/shasum but we have to
use the one found from PATH.

Change-Id: I67bb37dcb72ba3d8d1f279646e44228b27137ec9
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-08 04:34:41 +00:00
Simo Fält
c48355ebb5 Provisioning: Fix DownloadURL utility's shasum check
MacOS 10.10 has older shasum tool, which does not support oneliner
verification.

Change-Id: If00b846d5772aa1991960675d022249d63e7bd77
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-05 11:36:43 +00:00
Tony Sarajärvi
a7b1b00350 Provisioning: Hide powershell progress bar for performance
Change-Id: I74abdcc2e7d2dccf4f85cd64fe5fb4b56e54d613
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-09-04 12:12:17 +00:00
Joni Jäntti
dbf737a5c0 Fix Add-Path helper function and start using it in scripts
This function was broken and only added the latest addition to the
actual PATH.

Change-Id: Ida352c7b605c1b9748bb98e4f51ce68a765c3ce3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-09-04 12:11:17 +00:00
Akseli Salovaara
4e1f4665c5 Build Windows MSVC2013 & MSVC2015 32bit binaries on 64bit host
From Qt 5.6.3 onwards building QtWebEngine Windows 32bit binaries
requires Windows 64bit machine. Linking requires more RAM than
Windows 32bit can address. Change MSVC2013 & MSVC2015
integration host machines accordingly.

Change-Id: I5b88536507dcf59efd124d4e8c3ead79772021fb
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2017-09-04 07:10:14 +00:00
Aapo Keskimolo
42b502f7bc Add UploadConfigureExe feature
We used to solve the requirement for uploading configure.exe from
product version, but we can add this as a feature and remove the
dependency to product version.

Change-Id: If671e7193771453cd5cd41e881875c44ff0992fb
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-09-02 11:32:38 +00:00
Tony Sarajärvi
3ca7658781 Provision vanilla Windows 8.1 x86 and use it
Replaces existing Windows 8 x86 builds with
a vanilla tier 1 image that gets all needed
software provisioned except for VS2013 that is
pre-installed.
Also modifies existing provisioning scripts
under common to support this 32bit OS.

Task-number: QTQAINFRA-1383
Change-Id: I495cb2d78b4d2d8b2abb9af1254fff0a78a570d9
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-09-01 13:02:12 +00:00
Joni Jäntti
05dd83c73a provisioning: windows: ICU
Fixed ICU script to actually install ICU instead of just setting the
environment variables. It is required to build qtwebkit.

Change-Id: I8881f9c11599d99b4d400595c9b929cf6f37088a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-09-01 08:05:57 +00:00
Joni Jäntti
30660bf987 Make CI use a new provisioned Win 10 template
Added all the provisioning scripts that install required
programs and make changes to certain settings. These are needed to
build qt5. This replaces the old Windows 10 template.

Task-number: QTQAINFRA-1263
Change-Id: Ieeb2ae3c53841644996d99ca86228df6f825a63a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2017-09-01 08:05:54 +00:00
Simo Fält
a99d3432f7 Install python3 on all platforms
Python3 is required in 5.6 by pyside-setup.

Task-number: QTAUTO-345
Change-Id: If11ca425fa08cce8a518710ded0a6ec58159343d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
(cherry picked from commit a5f3f8ed48)
(cherry picked from commit a765838889)
(cherry picked from commit 1859f9b26c)
2017-08-30 19:57:54 +00:00
Simon Hausmann
1bd2b2acfa Fix code duplication for cmake installation on Linux
Share the code for installing cmake on Linux.

Change-Id: I535f2696151ea4db26dc7194a1fe7016799ed0e2
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-30 17:16:59 +00:00
Joni Jäntti
c7a9bb92e1 Fix adding to path lines for Windows provisioning scripts
Currently the path addition command will forget the previous addition
when adding a new string. This hasn't been found out because python
was the only script using this command. Now we also need to set the
path for ruby, so the python string is left out.

Change-Id: I39721d78111550189e52a9f095030033f9660c0a
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
2017-08-30 17:16:56 +00:00
Tony Sarajärvi
835ef953f0 Provisioning: Fine tune cmake provisioning script
Add version printing and move version strings to variables for
easier reuse.

Change-Id: I5372872bf455609dfb7cc7bbcf7a110cd71933fb
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-08-30 17:16:52 +00:00
Tony Sarajärvi
f05289d622 Provisioning: Add $PATH properly in .bashrc
If we, in a provisioning script, add $PATH to another
file, that $PATH gets expanded there and now. We need it
written as $PATH to the target file, so that it gets expanded
when the other file is run. Not at the moment of running this
provisioning script.

Change-Id: Id963f0f8b1cc74b02d35c1a91eb974ed816839ab
Reviewed-by: Simo Fält <simo.falt@qt.io>
2017-08-30 17:16:48 +00:00
Heikki Halmet
5df417aeb9 Provisioning: Disable system updates
This change will disable system updates from App Store to prevent
unnecessary network load

Change-Id: I7915ec6ffce681a850abef59d58f953effd6deb0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-16 06:03:50 +00:00
Joni Jäntti
bd3ddeddd2 Make CI use a new win 8.1 provisioning instead of win 8
Added all the provisioning scripts that install required
programs and make changes to certain settings. These are needed to
build qt5. This replaces the old Windows 8 template.

Made all the Windows platforms use the scripts under
provisioning/common/ instead of duplicate files.

Task-number: QTQAINFRA-1357
Change-Id: Ic3f0b50079b212437fff711379f243753ab84872
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-11 16:49:10 +00:00
Tony Sarajärvi
043e54c7c6 Update provisioning scripts with new intra.qt.io address
Change-Id: I9b4c5e9d0fe8e0e84596ad9086f563d11e97930b
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-08 20:05:08 +00:00
Simon Hausmann
49ccf716dc Remove unused provisioning scripts
RHEL 7.2, Ubuntu 15.04, 16.04 and OpenSuSE 42.1 are not used in the Qt
5.6 branch. Let's remove the scripts to avoid confusion.

Change-Id: I3829ea320ad77ebd25c9cf5bebfeed89f37f44dc
Reviewed-by: Simo Fält <simo.falt@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2017-08-04 19:22:40 +00:00
Frederik Gladhorn
d1d36cbbf1 Add Documentation feature to one Ubuntu 14.04 build
When creating packages, we build the docs on one configuration.
Previously that was done by magic inside the continuous integration
system. Make it a bit less magic by having a flag for it.

Change-Id: Iee5be7037023929b319beffaacb1592dc87d6078
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-08-04 16:12:56 +00:00
Simo Fält
c9d5341307 Provisioning: Fix virtualenv in macOS 10.11
There are multiple Python installations in our macOS 10.11 Tier1
template. Most of them are pretty old and not even working properly.
One of the affected Python tool is virtualenv, which atm is creating
Python 2.6 env by default even if Python 2.7 is the default Python.

Change-Id: I8bc0900976a9dc79637539802345175c85b7b433
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-07-19 05:54:01 +00:00
Jędrzej Nowacki
eca392d3f3 Use updated Ubuntu 14.04 in CI
The system packages in the base template were
updated via apt-get on 12.07.2017.

We hoped that it will solve mysterious problem of
a network interface being randomly disabled. Sadly
we had no luck, but getting all security updates
also make sense.

Change-Id: I34d7a56a2c5530376f74bb35faf461e2e18da14a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-07-18 16:59:07 +00:00
Simon Hausmann
2c4b06a13d Centralize Mesa LLVMPipe installation on Windows
Install the DLL for general availability on all Windows systems. Due to
its distinct name it will not be picked up by anyone unless Qt
explicitly selects it.

This gives us the ability to choose the GLES backend to use in Qt.

Change-Id: Ie36f202ead7080a8f7690734d305bd27f0c9f56b
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-20 04:44:49 +00:00
Simon Hausmann
5bc43f0cc3 Fix Postgresql and Mysql environment variables
The correct paths to the headers are

    c:\utils\postgresql[x86]\pgsql\include

not

    c:\utils\postgresql[x86]\pgsql\pgsql\include

Same for libs.

Regression introduced by commit ce47238ed9
but only noticed in the 5.9 branch.

Change-Id: Ib16072c0cc1c52f743f5500bc9732c8b04e962c7
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-12 04:43:24 +00:00
Simon Hausmann
0358b6d28c Remove QNX Windows builds
We can't get those to work with the faster KVM based VMs, so disable
these builds for the time being.

Task-number: QTQAINFRA-1208
Change-Id: I74c19374d57e3f84d8f982b95aade4a779623228
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-06-07 18:10:08 +00:00
Simon Hausmann
303d617329 Disable tests on macOS 10.10
We cannot run graphical tests reliably on 10.10 anymore with the new
virtualization without vmware. As outlined in the referenced task, we
occasionally get the Window Server locking up. So for the time being,
this change disables testing on 10.10.

Task-number: QTQAINFRA-1198
Change-Id: I020d4a298562554838cd7e898df52438db536d7a
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2017-06-07 04:16:32 +00:00
Simon Hausmann
35057fe4b2 Clean up windows cleanup script
* There's no need to have a numbered prefix in the common directory
* Instead of deleting directories trees worth several gigabytes of data,
  just rename the directory. The underlying storage system is
  copy-on-write and so deleting the files produces more data than just
  moving the directory out of the way. Deleting the files does not
  actually free up any physical disk space.

Change-Id: I6de02d56f3485651726ea11183ce7571d5b61520
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:40 +00:00
Simon Hausmann
038d263081 Speed up Windows installation of cmake, python, opengl, mysys and Android
We're better off copying off the internal server using CIFS than using
PowerShell's incredibly slow Invoke-WebRequest.

Change-Id: Ic39cf08be70335e9309575d543155bffaceb0db0
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2017-06-06 20:15:34 +00:00
Simon Hausmann
999b5f5dd7 Speed up Visual Studio update installation
We have a prepared package on the internal server that has everything
necessary included. This way we do not have to download the actual
update content from the internet.

Change-Id: Ie4cb3116a4f91a47df5be1c2b2e473d7ddec9a78
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Aapo Keskimolo <aapo.keskimolo@qt.io>
2017-06-06 20:15:31 +00:00
Simon Hausmann
ce47238ed9 Improve Postgresql installation on Windows
Similar to the mysql change (parent commit) it's faster to rename any
older artifacts and install only what we need. This also generalizes the
zip archive extraction code for "dev" packages.

Change-Id: I3ad1f23efaed80fab3e0778a3e8c46317138f1ad
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:28 +00:00
Simon Hausmann
982659b081 Improve mysql installation
* Instead of deleting the files from a previous installation, which will
  not actually release any disk space but just make the qcow2 file
  larger, let's just rename the directory the old files are in.
  That is a significantly faster operation.
* When extracting the zip archive, only extract what we need for Qt:
  Libraries, binaries and headers. We can skip the installation of
  several thousand documentation files for example, which takes a long
  time.
* Instead of downloading the zip archive via PowerShell's
  Invoke-WebRequest we can get the data much faster using CopyItem off
  the CIFS share. As a bonus this is also more robust against download
  failures.

Change-Id: I9128bd70e5f4a4ebcdbf046765e76b63bad033a5
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-06 20:15:09 +00:00
Heikki Halmet
a4a7f71f37 MSVC 2017 updated and missing component for winrt arm
MSVC 2017 and Build Tools updated to version 15.1. Also missing
component 'Visual C++ compilers and libraries for ARM' installed
for winrt arm

Also script added to storage msvc version numbers to versions.txt

Change-Id: I015e15d5d613b178c1266a2aecee5c3270780486
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-05-03 09:51:17 +00:00
Simon Hausmann
2ea45a59a7 Improve Android SDK/NDK installation on macOS
* Remove unused installation on macOS 10.10 as we only build packages
  on 10.11.

* Replace the use of the ~1GB downloads of the zip files with a direct
  extraction via NFS. This is more resilient to flakyness in the network
  connection.

Change-Id: I1bfe5bbf00a6614233b517801daf130069094ec9
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2017-04-24 19:19:00 +00:00
Heikki Halmet
370fdfcefd Provisioning: MSVC 2017 Build Tools
MSVC 2017 build tools are pre-provisioned to windows 10 templates

Change-Id: I98d857101ad209acd6eea9f821e367e514359d8c
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-24 05:17:04 +00:00
Heikki Halmet
e48d653150 Provisioning: Change tier1 templates for Windows 10
These tier 1 templates includes changes needed by 5.9, but we can use
these same tier 1 templates for 5.6 also

Change-Id: I3ad0d75ab89f7b6502b758fb94f984cc6e0b7f16
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-04-13 12:31:58 +00:00