Commit Graph

93 Commits

Author SHA1 Message Date
Heikki Halmet
0afdcac0c4 Provisioning: Upgrade OpenSSL version to 1.1.1k
Change-Id: I539d6da1953a40190fbfc8ff80f8e8332d797b1f
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
(cherry picked from commit 0e9f489192)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2021-05-31 06:10:29 +00:00
Johanna Äijälä
7231d7ccf7 Provisioning: Fix Squish for Qt6 on macOS
Add correct Squish package to the 'squish_for_qt6' location.

Change-Id: Ib997092a5d6350d9b7298d7cc8baf5a2afaca702
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
(cherry picked from commit 03821e82a5)
Reviewed-by: Johanna Äijälä <johanna.aijala@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2021-04-21 09:01:35 +03:00
Toni Saario
d1601aa0f9 Add conan to path
Change-Id: I90e028409f606a703fd9b567a221f3f946f11d61
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
(cherry picked from commit 32579bc3c4)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2021-04-12 07:21:53 +00:00
Toni Saario
88b3474bcf Enable conan revisions mode
Change-Id: I0fb9d6943cf0e6e54f631314a67348f703c31e41
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
(cherry picked from commit 3802e21afe)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2021-04-12 07:21:47 +00:00
Johanna Äijälä
eb0962abdd Provisioning: update Squish packges
Still using unreleased custom snapshot packages from Froglogic.
macOS: use custom build Qt5.15 package with separate Squish server
adjusted for Qt6.

Change-Id: I8605204118015feb347268c335ef731ff99e3c68
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
(cherry picked from commit 02d4c34b7d)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2021-03-28 15:29:03 +00:00
Cristian Adam
4e3b53f228 Update QDoc libclang to 11.0
Change-Id: I1cc74a1efb702d32d99749ec681e37345579c5b7
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
(cherry picked from commit d3406fedd1)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2021-02-05 21:12:54 +00:00
Johanna Äijälä
d86303a561 Provisioning: Update Squish for Qt6
These are snapshot packages from Froglogic, not available for macOS yet,
thus it will still use Qt5.15 based Squish. Only Linux and Win packages
are updated now.
Along with the Squish for Qt6 we need the Squish for Qt5 to be able to
run Creator tests (since Creator is still Qt5 based).

Change-Id: Iec0294e1de036db004633ee6b95e1e68a6dd23a5
Pick-to: 6.0
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2021-02-02 07:38:13 +02:00
Heikki Halmet
e67fa087db Provisioning: Install Conan package manager
Conan package manager is need for addtional libraries

Task-number: QTQAINFRA-4190
Change-Id: I5cb07a137505f15d536989c6d5127f607735a17e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ville-Pekka Karhu <ville-pekka.karhu@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
2021-01-26 14:15:15 +02:00
Heikki Halmet
f6b0b21ba2 Add Big Sur macOS 11.0 to CI
Only one target is added because current HW capasity is minor.

Task-number: QTQAINFRA-3967
Change-Id: Iae2ac0e02b38b45a219a78e046fb21338ff9cd97
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2021-01-22 10:35:11 +02:00
Heikki Halmet
8c32c1dcd7 Provisioning: Add libclang packages to cache
Both static and dynamic libclang packages are missing from our local
cache. Dynamic and static packages need to be in different folders
because package names are identicals

Pick-to: 6.0
Pick-to: 5.15
Change-Id: I0d4c0c776f1c5f318417cae6c16d683b31a10710
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2021-01-21 08:47:49 +02:00
Simo Fält
b792fd56db Provisioning: Install libClang 10 with dynamic libs
As Qt uses only static archives of libclang, we have to install
dynamic version as an additional package. Libclang is required
for example QtForPython.

Pick-to: 5.15
Pick-to: 6.0
Change-Id: Ic9d70398113defe3561f261aee369b423c18686a
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-12-23 08:02:32 +02:00
Heikki Halmet
b34fa5f9cf Remove RedHat 7.6 from CI
CentOS 8.1 is now in CI without Insignificant flags so
Rhel 7.6 can be removed.

Task-number: QTQAINFRA-2760
Change-Id: Id797a6ea5bf9c4d64929b4d6e183d0c7146309ca
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2020-11-16 07:38:21 +02:00
Maurice Kalinowski
9a82a87954 Update MQTT test broker
Latest additions of CentOS cause troubles due to
- Updated SSL to set up tls servers
- Updated Python version, which removed API calls

For the latter, we only need to update the broker. TLS test servers
are not used in our setup. However, qtmqtt.git requires on further
update to load a different configuration.

Task-number: QTBUG-88189
Change-Id: I2d6e559a21dff41deca94aeec8c5db6fdf72a0ac
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Karsten Heimrich <karsten.heimrich@qt.io>
2020-11-11 07:59:22 +01:00
Heikki Halmet
f1e911e344 Provisioning: Replace SLES-15 with SLES-15 SP2
Task-number: QTQAINFRA-3921
Change-Id: I325a4b8f272dda80168aed1389e647bbd525b10f
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-10-12 19:28:28 +03:00
Cristian Adam
ca02358af2 Provision RedHat libclang packages only for RedHat
While bulding on CentOS with Red Hat libclang packages we end up with
linker errors due to differences in C++ ABI used.

Ubuntu 18.04 package works fine with CentOS 8.1.

Task-number: QTBUG-86968
Change-Id: Idf0435a06046fb791b8ca1086e6465521ba6006a
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2020-10-11 14:10:42 +02:00
Johanna Äijälä
91c267a677 Provisioning: update Squish to 6.6.1 based on Qt5.15.0
Add Squish for Web installation, update Squish license.

Pick-to:5.15

Change-Id: Id418830e82718afc20ac7390400342a6b0401a64
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-09-03 10:39:41 +03:00
Tony Sarajärvi
7a5069e979 Update QDoc libclang to 10.0
Change-Id: Ic4899f188c92502c6f6808dcc05922d4980cadfd
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2020-08-25 21:39:00 +02:00
Jędrzej Nowacki
015c86534c Enable use of sccache in the CI
The sccache may significantly reduce build times.
We need to evaluate real cases to see how much is to gain.

This patch enables sccache only on cmake, exluding one msvc target
with debug flag -Zi, which creates separate debug symbols.
Related sccache issue:
  https://github.com/mozilla/sccache/issues/242

Task-number: COIN-130
Done-with: Aapo Keskimolo <aapo.keskimolo@qt.io>
Change-Id: Ic1f599c5194bcdee9528a548dabead1097821cd1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-08-19 12:59:16 +03:00
Heikki Halmet
2ac3dd2d6f Provisioning: Update OpenSSL to version 1.1.1g
Pick-to: 5.15
Pick-to: 5.12
Task-number: QTQAINFRA-3819
Change-Id: Ie7d0f3a0c73d123c3f3536e6b2dd2eb8608eb9cd
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-06-23 06:36:38 +00:00
Heikki Halmet
2c0816aa99 Provisioning: Change mounting folder for Squish installation
Sometimes some process is locking the mounted folder (/tmp) so that
it can't be unmounted untill the process has been ended.
Let's create squish folder under tmp and use that for mounting
instead

Task-number: QTQAINFRA-3709
Change-Id: I49ab8fd90af37a670e45764a176b62d7a732cc6c
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
(cherry picked from commit c671c2352e)
Reviewed-by: Antti Kokko <antti.kokko@qt.io>
2020-06-10 07:24:41 +03:00
Heikki Halmet
4ca819a713 Provisioning: Fetch pre-installed squish from local cache
Squish installation is flaky so the installation phase has been
moved from provisioning to Jenkins. Jenkins job will install the
squish to target VM and copy it to local cache. This change only
fetch the installed squish packages from local cache and
uncompress those to VMs.

Task-number: QTQAINFRA-3510
Task-number: QTQAINFRA-3107
Change-Id: I862da3250120dc6a43d9c04d973a25a691ca66a1
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
(cherry picked from commit b0286d6cd6)
Reviewed-by: Toni Saario <toni.saario@qt.io>
2020-06-09 06:09:45 +00:00
Morten Johan Sørvig
922cca96bf Provisioning: Bump Emscripten version to 1.39.8
(cherry picked from commit 10e843566e)

The upstream URLs have changed:
 - emscripten packages now at github.com/emscripten-core/emscripten/archive/
 - binaries (llvm/binaryen/node) now at storage.googleapis.com/webassembly/emscripten-releases-builds/

Add support for unpacking tbz2 files. Keep caching as
before, update .emscripten config file with new paths.

Task-number: QTQAINFRA-3577
Change-Id: I239aaf89bc199e5f6267519861f9b2ab54f10b0f
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Liang Qi <liang.qi@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-04 08:15:13 +02:00
Liang Qi
08e63423e2 Merge remote-tracking branch 'origin/5.15' into dev
Conflicts:
	.gitmodules
	coin/platform_configs/default.yaml
	coin/platform_configs/flashing-wizard.yaml
	coin/platform_configs/qt5.yaml
	coin/platform_configs/qtbase.yaml
	coin/platform_configs/qtconnectivity.yaml
	coin/platform_configs/tqtc-qt-config-gui.yaml
	coin/provisioning/qtci-linux-SLES-15-x86_64/02-zypperpackages.sh

Change-Id: I7af8a52d8b4452211982d86105dafacdd3d845c0
2020-02-28 13:49:10 +01:00
Dimitrios Apostolou
150f6277e0 Update telegraf version
Change-Id: I1a023e4f1de860a03a135f3b10f2c99d5309dd8b
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:58 +01:00
Dimitrios Apostolou
2693e482b6 Extract archive directly to destination filesystem
Save some seconds or even minutes during provisioning, by avoiding copying
many files across filesystems, from /tmp to /usr/local. We now install
directly to /usr/local.

Change-Id: I3a8f545f1400995095193c4eb337f9dd8534ef72
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:43 +01:00
Dimitrios Apostolou
b018b841c1 Do not complain about existing empty files
Sometimes we pre-create the target filename using mktemp. In that case of
course the hash will not match, so avoid comparing.

Change-Id: Id0feb0178d659e03f5ceb000f738167cdc28a3ec
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:39 +01:00
Dimitrios Apostolou
25fcda6ff3 Alias expansion is not working reliably in non-interactive shells
It happened that old bash version on macOS the alias was not being expanded
for non-interactive shells running under "bash" but was being expanded fine
when under "sh". Using functions is the recommended way.

Here we define a function conditionally.

Change-Id: I607b265cb9e88b2b529ed170121e5a0833003acd
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:37 +01:00
Dimitrios Apostolou
fe4a724a3e Use internal mirror repositories for Ubuntu docker containers
Depending on different external URLs makes our provisioning scripts flaky,
because they can fail if any one of the external servers goes down.
In this patch

* the docker images qt_ubuntu_16.04 and qt_ubuntu_18.04 are being built first,
  they apt-get from the internal ubuntu mirror

* all the existing docker images have been rebased to these two images, so
  they inherit this behavior

* images that fetch files from the internet have been modified to not do the
  fetching themselves; insted URLs are fetched in advance using "DownloadURL"
  which accepts internal mirrored URLs, and the downloaded files are ADDed to
  the images with a local copy

* previously existing git clone commands have been modified to fetch
  internally mirrored tarballs

Task-number: QTQAINFRA-3343
Change-Id: Ic4640a4f7776e38623edba90f0eba7cb68631223
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-13 19:44:45 +01:00
Simon Hausmann
68c8693e5b Merge remote-tracking branch 'origin/wip/cmake' into dev
This merges merely the provisioning changes needed to perform cmake
builds in the CI. The platform configuration changes will be applied
incremental as we merge support in the individual modules.

 Conflicts:
	coin/provisioning/qtci-linux-Ubuntu-18.04-x86_64/02-apt.sh

Added ninja to
coin/provisioning/qtci-linux-SLES-15-x86_64/02-zypperpackages.sh
just to force re-provisioning.

Removed module_config.yaml because that breaks license check in
qmake builds.

Change-Id: Ic074c0c881404f4361e61f66130c0cb07f5e8b84
2019-12-13 14:58:08 +01:00
Liang Qi
63c515a6cb Merge remote-tracking branch 'origin/dev' into wip/cmake
Also update the Windows config to use MSVC2019 instead of 2017
to circumvent the issue that vcpkg is building libraries with the
latest compiler it finds (MSVC 2019), but Qt is building with 2017.

 Conflicts:
	coin/platform_configs/default.yaml
	coin/platform_configs/qt5.yaml
	coin/platform_configs/qtsaferenderer.yaml

Change-Id: I88deb114e171046b50fc617ab6bc18b707d61486
2019-11-28 19:44:30 +01:00
Simo Fält
9cf8af5248 Provisioning: Remove sudo when installing pip packages for Qt user
There is no need to install as sudo for qt user. Using sudo will
change the cache dir's permissions preventing us from
installing anything with pip as normal user.

Change-Id: Id45bf3805ce4d772a6c82e429d4e1948289f076e
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2019-11-25 13:15:51 +00:00
Johanna Äijälä
08ded9e2f8 Provisioning: update Squish
Update with custom build of Squish to be able to launch Qt5.14
applications in RTA tests.

Change-Id: Ic420656852ae885b057de937baabf01135526764
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-11-21 11:26:11 +03:00
Heikki Halmet
601a9d39b2 Provisioning: Install python modules needed by packaging
Needed by packaging scripts

Task-number: COIN-214
Change-Id: I3ac719800c3afd183996379f6c23fbe3f8bcf7cb
Reviewed-by: Iikka Eklund <iikka.eklund@qt.io>
2019-11-20 08:07:58 +02:00
Dimitrios Apostolou
6cc2215b8c DownloadURL last argument (targetFile) is now optional
Change-Id: I0864bcb052cddcc9753c056758b6e65fd561d928
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-11-14 08:46:14 +01:00
Simon Hausmann
c85543f88d Merge remote-tracking branch 'origin/dev' into wip/cmake
Conflicts:
	coin/platform_configs/default.yaml
	coin/platform_configs/qt5.yaml
	coin/platform_configs/qtbase.yaml
	coin/platform_configs/qtdeclarative.yaml
	coin/platform_configs/qtsaferenderer.yaml
	coin/platform_configs/qtsdk.yaml
	coin/provisioning/common/linux/cmake_linux.sh
	coin/provisioning/common/windows/cmake.ps1

Change-Id: I37bb513e3be19e4de4003679f8528677e57c09ee
2019-11-11 10:52:16 +01:00
Heikki Halmet
52fe759eba Provisioning: Upgrade OpenSSL version to 1.1.1d
Plus removing unused file install_openssl_111a.sh

Task-number: QTQAINFRA-3224
Change-Id: If5f9ee12e118e0329eb27f8d3ca9829011874382
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-11-02 19:41:33 +03:00
Dimitrios Apostolou
518e114dc7 Fix for DownloadURL to try alternative URL
Change-Id: I7e9a77815ef8f4c258dda44bda6fdd4eb3736ef4
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-15 12:41:04 +02:00
Liang Qi
9e3b3c838c platform_configs: add a RHEL 7.6 build
A preparing for having android build later.

* install gcc 7.x
* install git 2.x
* install a few other dependencies

Change-Id: I18f05bdfb78c92b7f05f67609613b5271b400f91
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Qt CMake Build Bot
2019-10-10 05:34:27 +00:00
Dimitrios Apostolou
869a9b3ae7 Fix shell function is_script_executed
It used to return error when tested inside a login shell, because $0 is
"-bash" and `basename` was trying to interpret -b as an option.

Change-Id: I5faa2dc01e50c7d5bcfc0ab1fcfd910b2538afc2
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-08 08:26:34 +02:00
Dimitrios Apostolou
d97d1af1ee Fix provisioning script for RHEL 6
It seems RHEL 6.6 does not provide /etc/os-release.

Change-Id: I01ba2bb17b89c01f9991944a11eeaa9c2eba6f20
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-08 08:26:34 +02:00
Dimitrios Apostolou
4e3a0f79fb DownloadURL now works with either wget or curl
The reason is that Ubuntu does not have curl at the very beginning of
provisioning scripts, and also does not have the repositories configured
in order to install it.

Change-Id: Ia485c1672f894bc800f199b742d238b2945938e5
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-09-27 08:42:49 +02:00
Dimitrios Apostolou
fbe3a8d84e Install telegraf on all provisioned VMs
Install binaries as early as possible in the provisioning process, and
start them, in order to monitor provisioning too.

To achieve this, some OS auto-detection logic is introduced in shell
script common.sourced.sh. The script can be sourced and used in all kind
of other generic scripts.

ioping is also installed from custom-compiled binaries in order to
monitor the disk latency of the VMs. Reason we don't use ioping from the
repositories, is that the units differ between ioping versions (new
ioping reports latencies in nanoseconds, old in microseconds).

Fixes: QTQAINFRA-3092
Change-Id: I9d9afb791955725d4bd0b32dae97dfc0bd4a76a3
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-09-27 08:42:31 +02:00
Dimitrios Apostolou
c0fce7a3b3 Refactor DownloadURL.sh
Auto-detects and verifies all kinds of hashes, not only SHA1.
Hash verification is now an autonomous function.
If the file is found at the destination, skip redownloading it if the
hash is OK.
Runs on POSIX sh, not requiring bash.
Retain compatibility with previous version.

Change-Id: I9930336030aa776f49dc460fe766f3e831700255
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:31 +02:00
Dimitrios Apostolou
c7f87acb37 Do not silence standard error
Provisioning was failing and the logs did not provide any useful
information because of the silenced squish install command. So I went
through all the scripts and I re-enabled stderr logging where I found it
silenced (with the exception of where it was purposeful or harmless).

Change-Id: I5fa0b0ba4362970b214c150d399b6720b1780e1e
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:25 +02:00
Dimitrios Apostolou
bdc5dae67b Speed up provisioning on Linux
Change-Id: I6e4e5621ed0b063215476ef51d61ef1f38198d0c
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:11 +02:00
Simo Fält
d19f60d745 Provisioning: Append the emscripten to path instead of prepend
Having it first in path seems to cause some confusion when detecting
libc++ in darwin.

Change-Id: I841e7f169e4771d3818cf156d9d78fc242e9995c
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-08-12 21:01:55 +00:00
Juha Karjalainen
38e416be66 Add webAssembly build for windows and macos
Provision emsdk and its dependecies to windows and macos.

Task-number: QTBUG-63917
Task-number: QTQAINFRA-2835
Done-with: Juha Karjalainen <juha.karjalainen@qt.io>
Change-Id: I24ea9cbc4f7cc95e6952765099d5e46247470011
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-07-31 09:44:53 +03:00
Heikki Halmet
644f8b7006 Fix openssl 1.1 library configurations
Change-Id: Idcb919e5f388b509e4215cf5515c8716490d5872
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>

(cherry picked from commit de41591477)
Change-Id: Ifbb5ecc4613319265a3378b60959fffc7301ae46
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-06-13 22:47:35 +03:00
Juha Karjalainen
3e838f2a34 Provisioning: Change openssl version to 1.1.1b
Use newest LTS version of openssl as support for 1.0.2 series
will end at end of 2019.

For windows 7 x86 openssl 1.0.2 will be used until all need fixed
are made to submodule tests

Task-number: QTQAINFRA-2327
Change-Id: I930d239cd139b6202ebdfb5cdc8068ff2ad0b8a3
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-05-22 15:34:04 +03:00
Liang Qi
78afd2d4b6 Merge remote-tracking branch 'origin/5.12.2' into 5.12
Change-Id: I8795e84dd750cbb2b55a03c861b911f57a1e2626
2019-03-21 14:26:56 +01:00