Commit Graph

595 Commits

Author SHA1 Message Date
Heikki Halmet
da2bd45ba6 Provisioning: Update mysql to Windows
Currently this is the last version that support x86 for at least Windows.

Task-number: QTQAINFRA-1996
Change-Id: Idd55694837afad2188697b4d5cc009a127ddb760
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
2019-02-18 07:43:26 +00:00
Juha Karjalainen
ce91433364 Fix failing to disable windows update service
Sometimes when trying to stop windows update service it fails.
Will now retry disabling wuauserv

Task-number: QTQAINFRA-2632
Change-Id: I5b6bf40aa15bb1f5225dc51e2f7212a01a13e9a6
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-02-14 15:22:56 +00:00
Juha Karjalainen
80deae9b74 Fix provisioning script dxsdk.ps1
Due to typo in dxsdk.ps1 provisioning script, it did not download from
cached location. Also checksum Check was missing.

Task-number: QTQAINFRA-2614
Change-Id: I321549721b256b85513fa1e4ff57ab1302e28f43
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-02-12 09:10:50 +00:00
Ryan Chu
5cfa5637ba Provision docker test servers based on Ubuntu 18.04
Comparing to the previous release, Ubuntu 18.04 includes dante-server
1.4.2 by default. Therefore, it doesn't rely on the Third-party PPA.

The SHA-1 of the provisioned images in docker cache:
qt-test-server-apache2_18.04:69478e75b362d57691a53700434a7034d471459e
qt-test-server-squid_18.04:cd5a5e2ff5d23c9a7adef1bd82847e027c090e5b
qt-test-server-vsftpd_18.04:a40425c0494f8f4e8ba27ac9fb5cc02aaedd62d8
qt-test-server-ftp-proxy_18.04:5f78e23a0579a579fcae7f3e5704bb5eec0e2f6d
qt-test-server-danted_18.04:d590b915cda9d524620d18a0398fdfa4d5876f97

Change-Id: Iffb4abe144a957465a2daab657c6259f0b75e800
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-02-02 12:23:22 +00:00
Ryan Chu
b4b2152dd3 Start the docker machine and update with new TLS certificates
After the system reboots, the docker machine is created, but in a
stopped state. To handle the case that a developer may rerun the
provisioning scripts for the missed docker images, make sure the
docker machine is both started and running with new TLS certificates.

Change-Id: Ibdb58152cac860132a77d6d094f9bca20a5e95e8
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2019-02-02 12:23:18 +00:00
Ryan Chu
177df80f09 Docker Provisioning: Install docker test servers on Windows 10 (x86_64)
Build and cache docker images to VirtualBox virtual machine.
1. Download and install the docker tool kits and Apple Bonjour
2. Deploy boot2docker image into VirtualBox via docker-machine commands
3. Use Git Bash to run the shared provisioning scripts
4. Print CPU features to verify that CI has enabled VT-X/AMD-v support
5. Disable Hyper-V from Windows 10 Pro/Enterprise editions

Change-Id: I5520e4a28376aba14f880c077bd7b083be971f0a
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2019-02-02 12:23:15 +00:00
Ryan Chu
ab3e34827a Display the provisioned version of docker tool kits
For debug purpose, print the installed version of docker,
docker-compose, and docker-engine. Because 'docker info' is related to
the docker-engine, postpone calling the command until docker virtual
machine has been created.

Change-Id: I4c869b7122b831b7f22c9c3846ac9cbe47c3b355
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Liang Qi <liang.qi@qt.io>
2019-02-02 12:23:11 +00:00
Ryan Chu
b97e448ba4 Move and merge the provisioning scripts of docker to a shared folder
To reuse the provisioning scripts for all platforms, they are moved to a
shared folder (coin/common/shared/testserver). The scripts of macOS and
linux are combined and controlled by an input parameter (macos/linux).

Change-Id: Ibf9a438d1825d01d707c2fc8fbcf2bc70a8b4902
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-02-02 12:23:07 +00:00
Juha Karjalainen
afbd9305e3 Fix provisioning failure when unsetting proxy with no proxy
When unset-proxy.ps1 script is run when there is no proxy set
unset script fails leading provisioning to fail. Now will check
if  proxy is set before removing registery keys for it.

Task-number: QTQAINFRA-2311
Change-Id: Ic50bcebc634bb90385ca817dac9bed24e9e897ec
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-01-29 15:24:38 +00:00
Heikki Halmet
8ba5474da7 Provisioning: Install Google's Protocol Buffers
Protobuf is needed for Automotive Suite

Task-number: QTQAINFRA-2314
Change-Id: I12b4248f1de5aea88f573297a47b791b5b6296fb
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
2019-01-29 12:10:57 +00:00
Heikki Halmet
54cdba9e3d Provisioning: Remove Unneeded Android variables
Change-Id: I7dd008aef6465fdddd07e227b48e9b6361c13f81
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-01-29 12:10:53 +00:00
Liang Qi
85eb7b7a03 Merge remote-tracking branch 'origin/5.12' into dev
Conflicts:
	coin/platform_configs/default.txt

Change-Id: Ie21baa51738ce0761fb45fba3819b1894cc808b6
2019-01-23 08:02:11 +01:00
Tony Sarajärvi
47d14b365e Accept license in Android SDK provisioning
A change in Android SDK prompts us now for
accepting license.

Task-number: QTQAINFRA-2591
Change-Id: I1ff05d84ccda743649c6d38784c6170e08f50eb1
Reviewed-by: Liang Qi <liang.qi@qt.io>
2019-01-23 01:29:37 +00:00
Juha Karjalainen
3332223130 Provisioning: breakpad for qt3dstudio
Breakpad https://chromium.googlesource.com/breakpad/breakpad/
sources are needed for qt3studio. Also Breakpad need sources for
https://chromium.googlesource.com/linux-syscall-support.

Task-number: QT3DS-2650
Change-Id: I13c7534e1c6e97a1e345dad3c2c82eeb6428313d
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-01-17 09:45:12 +00:00
Simo Fält
6d2929d1aa Provisioning: Move Android toolchain and config to win10
Installing Android toolchains and ssl headers to win10 template
and moving Android windows platform configs to win10 from win7.

Task-number: QTQAINFRA-1266
Change-Id: I9b3e3a378451e0bdfc7dc5c7b75ee035faf8994c
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
(cherry picked from commit 5a040ca46a)
2019-01-17 09:45:05 +00:00
Heikki Halmet
ea0825abe8 Provisioning: Remove update notifier
Update notifier is showing when vm is running.
It can affect to test results and that's why it need to be removed

Task-number: QTQAINFRA-2453
Change-Id: Icb27ba998ef9a51aaf8f40274c182b0c3dcd496a
Reviewed-by: Sami Nurmenniemi <sami.nurmenniemi@qt.io>
Reviewed-by: Juha Karjalainen <juha.karjalainen@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2019-01-17 09:45:02 +00:00
Maurice Kalinowski
209d09d530 Provisioning: Update Unified Automation OpcUA backend
Adding in the x64 bit installers to deployment. Furthermore, we can use
the msvc2015 installer to create msvc2017 binaries with Qt.
After the licensing change, we will be able to generate packages for all
msvc based installers.

Change-Id: I96db345b535a2518c758175137c808b15130ff41
Reviewed-by: Rainer Keller <Rainer.Keller@qt.io>
2019-01-16 07:30:37 +00:00
Tony Sarajärvi
89e0159bbc Swap openSUSE 42.3 to 15
Task-number: QTQAINFRA-2067
Change-Id: I71f79181704d6ce8faac0019f6d7a5016b0ecb5e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Gatis Paeglis <gatis.paeglis@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-01-10 17:39:51 +00:00
Juha Karjalainen
cfa7877afe Fix provisioning ODBC environmetal variables had : at end
CPLUS_INCLUDE_PATH and LIBRARY_PATH environmental variables had
 --> : <-- at end of variable. As this adds current dir to search
path this is undesirable and causes bugs.

Change-Id: Ia36c3f3f791f5d506eea15aba6be6d665049a922
Reviewed-by: Juha Karjalainen <juha.karjalainen@qt.io>
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
2019-01-10 12:32:57 +00:00
Maurice Kalinowski
56c43a03cd Provisioning: Update Qt OpcUA backends
- Unified Automation SDK needs to be updated to 1.6x to reflect latest
API changes
- Open62541 is now inside 3rdparty of the Qt OpcUA module. Hence, no
need to provision it.

Change-Id: I96a3764053ca684fa71f9d49e998ebe0ce45a6ae
Reviewed-by: Jannis Völker <jannis.voelker@basyskom.com>
Reviewed-by: Frank Meerkoetter <frank.meerkoetter@basyskom.com>
Reviewed-by: Rainer Keller <Rainer.Keller@qt.io>
2019-01-09 09:57:35 +00:00
Juha Karjalainen
7028ee3184 Fix broken provisioning script DownloadURL.sh
DownloadURL returns 0 even when alt download fails.

Task-number: QTQAINFRA-2501
Change-Id: Id2fb1e0b4ff12a48b6adb27291f55fc129296233
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2019-01-09 09:14:02 +00:00
Tony Sarajärvi
6b64780b61 Fix .ci.local to .intra.qt.io in docker provisioning
Change-Id: I9de241e70a5dd15568619da6e4a19d54e76e5a4c
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-01-07 15:01:31 +00:00
Heikki Halmet
72d0cfc6dc Provisioning: Add Squish Coco for RTA
Task-number: QTQAINFRA-2393
Change-Id: I391a2670a46835988da6fd1d9121172c706c7a21
Reviewed-by: Johanna Äijälä <johanna.aijala@qt.io>
2019-01-04 14:09:59 +00:00
Juha Karjalainen
e28da040ee Provisioning: Unify duplicated openssl provisioning script
MacOS(10.12 and 10.13) had two identical provisioning scripts for
openssl. Now unified under common.

Change-Id: If469a56d431e72b291a810fa9c1b701873fe382d
Reviewed-by: Aapo Keskimolo <aapo.keskimolo@qt.io>
2019-01-03 11:46:20 +00:00
Joni Jäntti
2306cb69c8 Provisioning: Windows: Disable automatic defragmentation
Automatic disk defragmentation is enabled by default on Windows
machines. It should be disabled to save some time.

Task-number: QTQAINFRA-2217
Change-Id: I3f024208bd35c9e305569e6b2b562f5a8b16f6f1
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-01-02 12:23:01 +00:00
Liang Qi
38e2fc6b80 Merge remote-tracking branch 'origin/5.12' into dev
Conflicts:
	.gitmodules
	coin/provisioning/common/linux/docker_testserver.sh
	coin/provisioning/qtci-linux-Ubuntu-18.04-x86_64/02-apt.sh

Done-With: Ryan Chu <ryan.chu@qt.io>
Change-Id: I9bc3afcb50783a9f0f7718731d237de4be287cae
2019-01-02 10:46:49 +01:00
Tony Sarajärvi
2dd72ec5a6 Fix .ci.local to .intra.qt.io in provisioning scripts
Change-Id: I113095b337c3592757878fb919a0a18a60a8ce7e
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-12-31 13:02:00 +00:00
Juha Karjalainen
55a990322b Provisioning: Install ODBC for macOS 10.12 10.13
This script installs libiodbc from brew and adds it to
CPLUS_INCLUDE_PATH and LIBRARY_PATH thus enabling configure to
find it and thus enabling odbc support for qt build on macOS.

Task-number: QTBUG-58956
Change-Id: I7f865800b580a87afcc480534da57ee546c74406
Reviewed-by: Antti Kokko <antti.kokko@qt.io>
2018-12-31 10:45:04 +00:00
Liang Qi
0e82de327b Merge remote-tracking branch 'origin/5.12.0' into 5.12
Change-Id: Ifb4277657a6f85e6a6d0ee62e54360b2cb1c0514
2018-12-22 23:21:54 +01:00
Juha Karjalainen
cbdd122e18 Provisioning: Update install android sdk to version 28
QT needs at least buildSdk 24 after commit
70057dfc0e53b7996acd5ef2c0f9c40aa3f66062. Updating provisioned sdk
version to 28. Please note minSDK <= targetSDK <= build/compileSDK.

Task-number: QTQAINFRA-2412
Change-Id: I0bf67cf42dfc6f707bdeae973a60c6601d2e73c3
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-12-18 08:20:41 +00:00
Heikki Halmet
deacc72f73 Provisioning: Hide squish installation output from provisioning log
This will prevent the output of unwanted information

Change-Id: I547e91ef344ec51f799b26f254139b03d20a315c
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-12-17 07:16:39 +00:00
Juha Karjalainen
e21401751a Provisioning: Change how executables are waited to finish
This patch only affects Windows.
Python3 provisioning took over 30-35 minutes. As both
32 and 64 bit versions are provisioned it took over 1h
this change lowers time taken to 5 minutes.

start-process with -wait flag waits for given process and its
sub-processes to exit.  Wait-process insteads just waits for
process to exit.

Change-Id: Ib9a323dcf9e2d07318a0d259a72ca4c4d0a4ed80
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-12-12 13:49:42 +00:00
Ryan Chu
959d9bdb67 Docker Provisioning: Install Avahi daemon on Ubuntu
Avahi is a free Zeroconf implementation and provides mDNS service
discovery. It enables Docker containers to publish their services on a
local network. For the host systems supporting mDNS technology, they
will automatically discover the services when running network tests.

However, some of the Linux distributions don't have avahi-daemon
installation by default (e.g. Ubuntu Server 18.04). To not surprise the
user, the provisioning script will help to install avahi-daemon package.

Change-Id: Ibb4c278c552c149b15da8cf7918f7f71ffa3ebec
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-10 14:11:01 +00:00
Ryan Chu
a695130478 Assert the provisioning process in macOS if VMX is not enabled
The Docker-machine requires VT-x/AMD-v to build up boot2docker virtual
machine. The VMX flag should be enabled when deploying the VM into
Virtualbox. In the CI framework, it relies on Coin to expose the VMX
flag to the CPU feature. To well handle the case that the VMX could be
removed due to some CI updates, the provisioning script will assert the
process with the related debug message if VMX is not found.

Change-Id: I9f3f123d33629df1655a5b55d776d5361878d1f5
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-10 14:10:58 +00:00
Ryan Chu
477fc6a73f Docker Provisioning: Remove the obsolete tag of test servers from Ubuntu
The SHA-1 tag of a server image is generated by the shared function
(sha1tree) in testserver_util.sh. The sha1tree function sorts the files
by the their SHA-1 and use the accumulated results as the TAG. So that,
we can assign the same SHA-1 value to a server on all the platforms.

Change-Id: If87ac72f7a9c5cdebef04977480d27b26a701b73
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-10 14:10:54 +00:00
Ryan Chu
ce3791a214 Docker Provisioning: Remove the fixed version of package dependencies
Free the dependencies of the specific Ubuntu packages. Ensure that test
server is using the latest version of the Ubuntu packages to test
network changes.

The SHA-1 of the provisioned images in docker cache:
qt-test-server-apache2:537fe302f61851d1663f41495230d8e3554a4a13
qt-test-server-squid:9c32f41b19aca3d778733c4d8fb0ecc5955e893c
qt-test-server-vsftpd:f3a9c8d793a77cc007c0e4e481bec01f9e3eeb7e
qt-test-server-ftp-proxy:d7de8b28392d173db512a558ccc84ead8bece2ae
qt-test-server-danted:35607f9b790524cf9690c7d12a9a401696b7b6b5

Change-Id: I4e6c8cb235f1f798274eb52858e06d8755ae626f
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-10 14:10:51 +00:00
Tony Sarajärvi
4de6c12b17 Don't set http_proxy in its own subshell
The subshell actually meant, that the http_proxy environment
variable was never seen outside that check. Thus proxy was never
used where this was checked.

Change-Id: I069f43f13dffca98d8b60a47c2e105209018c84d
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2018-11-29 20:20:26 +00:00
Ryan Chu
3ae624cd6e Docker Provisioning: Remove the fixed version of package dependencies
Free the dependencies of the specific Ubuntu packages. Ensure that test
server is using the latest version of the Ubuntu packages to test
network changes. Otherwise, all the docker files need to be manually
updated when upgrading to the next Ubuntu version (e.g. 18.04).

For debugging purpose, the installed packaged will be listed in the
provisioning logs.

Change-Id: I38d119d11f33cfd60dc34fcd57ec0fe1615e98f2
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2018-11-22 07:57:53 +00:00
Liang Qi
0c839f87fb Merge remote-tracking branch 'origin/5.11' into 5.12
Conflicts:
	coin/provisioning/common/unix/mqtt_broker.sh

Change-Id: I990dbf66b7120f2f1bcf202f0a2547a5ad9881f2
2018-11-20 19:37:06 +01:00
Heikki Halmet
84aaa89cf4 Provisioning: Fix path for fbx files
Currently Qt3d Studio build don't recognize fbx path because of space
in folder name. This change will fix it by taken use the packages which
don't have space in the folder name.

Task-number: QT3DS-2440
Change-Id: I502bb04489a03a5a81e164f8a86f3200f4ec3910
Reviewed-by: Miikka Heikkinen <miikka.heikkinen@qt.io>
2018-11-13 05:51:53 +00:00
Heikki Halmet
1248a6787d Provisioning: Add scp and ssh location in the PATH for Windows
RTA needs to have these on the PATH

Change-Id: I59694737e9284bc31eefcc4ff89219665c2aa430
Reviewed-by: Matti Paaso <matti.paaso@qt.io>
2018-11-07 20:07:36 +00:00
Heikki Halmet
7974f26b20 Provisioning: Disable notifications and remove apport
These can cause flakiness during tests

Task-number: QTQAINFRA-2307
Change-Id: Ib8b850e3e1fabe3070bbd27785d043984e085cb7
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-11-07 07:23:49 +00:00
Simo Fält
5a040ca46a Provisioning: Move Android toolchain and config to win10
Installing Android toolchains and ssl headers to win10 template
and moving Android windows platform configs to win10 from win7.

Task-number: QTQAINFRA-1266
Change-Id: I9b3e3a378451e0bdfc7dc5c7b75ee035faf8994c
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-11-06 19:30:23 +00:00
Tony Sarajärvi
b082d7d544 Replace .ci.local addresses with .intra.qt.io in provisioning scripts
Change-Id: I5e5bc084faabb28ffdf0437871445ecb8970bfb3
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-11-01 07:16:57 +00:00
Heikki Halmet
1f07276692 Provisioning: Remove environment variables from androind-openssl
We shouldn't need those anymore.

Task-number: QTQAINFRA-1266
Change-Id: Ie2161eb1d05cc1af26cbbb1a46525a9a8d7c3a30
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-10-31 15:47:01 +00:00
Ryan Chu
ea76ace825 Docker Provisioning: Install Docker-based test servers on macOS
Build and cache docker images based on the configurations in testserver.
1. Move the shared Docker files to common/shared/testserver
2. Deploy boot2docker image into VirtualBox via docker-machine commands.
3. Build and tag the docker images by the SHA-1 of server context.

Change-Id: Ic5a588e5cc5e753ec4c4e2e1bc79917bbd03f95d
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-10-30 08:15:43 +00:00
Juha Karjalainen
3b4ca5d6c0 Provisioning: Update openssl to version 1.0.2p
Task-number: QTQAINFRA-2266
Change-Id: I3f09d9169f17bcd83dcfd72547440bca487fd9db
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-10-26 07:23:14 +00:00
Tony Sarajärvi
bc5e5baf55 Update Android NDK and SDK on Windows envs
Task-number: QTBUG-47672
Change-Id: I3e6526b296f15927b90ab3310d91f8505b0228b3
Reviewed-by: Liang Qi <liang.qi@qt.io>
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2018-10-20 05:15:55 +00:00
Tony Sarajärvi
97efaacb2b Update Android NDK and SDK on Linux envs
Task-number: QTBUG-47672
Task-number: QTQAINFRA-2298
Done-with: Liang Qi <liang.qi@qt.io>
Change-Id: I42c87d8453423530173b826c09dc6eb4bf9ad5b8
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-10-20 05:15:40 +00:00
Juha Karjalainen
0ac22704fa Provisioning: Disable powershell quickedit
Powershell quickedit causes powershell window freeze until quickedit
mode is exited by pressing enter. For some reason quickedit mode get's
activated during build. This causes time out as agent does not get
any output from processes.

Task-number: QTQAINFRA-2263
Change-Id: I2878cc7a39bae504f826c5622cc5817fd227d7f0
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-10-13 08:41:42 +00:00