Commit Graph

57 Commits

Author SHA1 Message Date
Heikki Halmet
52fe759eba Provisioning: Upgrade OpenSSL version to 1.1.1d
Plus removing unused file install_openssl_111a.sh

Task-number: QTQAINFRA-3224
Change-Id: If5f9ee12e118e0329eb27f8d3ca9829011874382
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-11-02 19:41:33 +03:00
Dimitrios Apostolou
518e114dc7 Fix for DownloadURL to try alternative URL
Change-Id: I7e9a77815ef8f4c258dda44bda6fdd4eb3736ef4
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-15 12:41:04 +02:00
Dimitrios Apostolou
869a9b3ae7 Fix shell function is_script_executed
It used to return error when tested inside a login shell, because $0 is
"-bash" and `basename` was trying to interpret -b as an option.

Change-Id: I5faa2dc01e50c7d5bcfc0ab1fcfd910b2538afc2
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-08 08:26:34 +02:00
Dimitrios Apostolou
d97d1af1ee Fix provisioning script for RHEL 6
It seems RHEL 6.6 does not provide /etc/os-release.

Change-Id: I01ba2bb17b89c01f9991944a11eeaa9c2eba6f20
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-10-08 08:26:34 +02:00
Dimitrios Apostolou
4e3a0f79fb DownloadURL now works with either wget or curl
The reason is that Ubuntu does not have curl at the very beginning of
provisioning scripts, and also does not have the repositories configured
in order to install it.

Change-Id: Ia485c1672f894bc800f199b742d238b2945938e5
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-09-27 08:42:49 +02:00
Dimitrios Apostolou
fbe3a8d84e Install telegraf on all provisioned VMs
Install binaries as early as possible in the provisioning process, and
start them, in order to monitor provisioning too.

To achieve this, some OS auto-detection logic is introduced in shell
script common.sourced.sh. The script can be sourced and used in all kind
of other generic scripts.

ioping is also installed from custom-compiled binaries in order to
monitor the disk latency of the VMs. Reason we don't use ioping from the
repositories, is that the units differ between ioping versions (new
ioping reports latencies in nanoseconds, old in microseconds).

Fixes: QTQAINFRA-3092
Change-Id: I9d9afb791955725d4bd0b32dae97dfc0bd4a76a3
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-09-27 08:42:31 +02:00
Dimitrios Apostolou
c0fce7a3b3 Refactor DownloadURL.sh
Auto-detects and verifies all kinds of hashes, not only SHA1.
Hash verification is now an autonomous function.
If the file is found at the destination, skip redownloading it if the
hash is OK.
Runs on POSIX sh, not requiring bash.
Retain compatibility with previous version.

Change-Id: I9930336030aa776f49dc460fe766f3e831700255
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:31 +02:00
Dimitrios Apostolou
c7f87acb37 Do not silence standard error
Provisioning was failing and the logs did not provide any useful
information because of the silenced squish install command. So I went
through all the scripts and I re-enabled stderr logging where I found it
silenced (with the exception of where it was purposeful or harmless).

Change-Id: I5fa0b0ba4362970b214c150d399b6720b1780e1e
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:25 +02:00
Dimitrios Apostolou
bdc5dae67b Speed up provisioning on Linux
Change-Id: I6e4e5621ed0b063215476ef51d61ef1f38198d0c
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-09-26 20:12:11 +02:00
Simo Fält
d19f60d745 Provisioning: Append the emscripten to path instead of prepend
Having it first in path seems to cause some confusion when detecting
libc++ in darwin.

Change-Id: I841e7f169e4771d3818cf156d9d78fc242e9995c
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-08-12 21:01:55 +00:00
Juha Karjalainen
38e416be66 Add webAssembly build for windows and macos
Provision emsdk and its dependecies to windows and macos.

Task-number: QTBUG-63917
Task-number: QTQAINFRA-2835
Done-with: Juha Karjalainen <juha.karjalainen@qt.io>
Change-Id: I24ea9cbc4f7cc95e6952765099d5e46247470011
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-07-31 09:44:53 +03:00
Heikki Halmet
644f8b7006 Fix openssl 1.1 library configurations
Change-Id: Idcb919e5f388b509e4215cf5515c8716490d5872
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>

(cherry picked from commit de41591477)
Change-Id: Ifbb5ecc4613319265a3378b60959fffc7301ae46
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-06-13 22:47:35 +03:00
Juha Karjalainen
3e838f2a34 Provisioning: Change openssl version to 1.1.1b
Use newest LTS version of openssl as support for 1.0.2 series
will end at end of 2019.

For windows 7 x86 openssl 1.0.2 will be used until all need fixed
are made to submodule tests

Task-number: QTQAINFRA-2327
Change-Id: I930d239cd139b6202ebdfb5cdc8068ff2ad0b8a3
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-05-22 15:34:04 +03:00
Liang Qi
78afd2d4b6 Merge remote-tracking branch 'origin/5.12.2' into 5.12
Change-Id: I8795e84dd750cbb2b55a03c861b911f57a1e2626
2019-03-21 14:26:56 +01:00
Johanna Aijala
88b35a4921 Update Squish to 6.4.3
Froglogic has not yet released Squish for Qt5.13 but we can use Squish
packages build with Qt5.12.

Remove msvc12 Squish, not needed.

Change-Id: I1de1bb67a5577fa7f7233c69c3a58bedd15fd16b
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-03-07 06:18:26 +00:00
Juha Karjalainen
9c60d8b190 Fix: Set correct path to environmental variable
Env variable BREAKPAD_SOURCE_DIR had /src at end which was
not needed.

Change-Id: I4d126c3b7c3b2da03dc2a0ba075b0d4ddad92eeb
Reviewed-by: Miikka Heikkinen <miikka.heikkinen@qt.io>
2019-02-18 13:17:52 +00:00
Heikki Halmet
8ba5474da7 Provisioning: Install Google's Protocol Buffers
Protobuf is needed for Automotive Suite

Task-number: QTQAINFRA-2314
Change-Id: I12b4248f1de5aea88f573297a47b791b5b6296fb
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
2019-01-29 12:10:57 +00:00
Juha Karjalainen
3332223130 Provisioning: breakpad for qt3dstudio
Breakpad https://chromium.googlesource.com/breakpad/breakpad/
sources are needed for qt3studio. Also Breakpad need sources for
https://chromium.googlesource.com/linux-syscall-support.

Task-number: QT3DS-2650
Change-Id: I13c7534e1c6e97a1e345dad3c2c82eeb6428313d
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2019-01-17 09:45:12 +00:00
Juha Karjalainen
7028ee3184 Fix broken provisioning script DownloadURL.sh
DownloadURL returns 0 even when alt download fails.

Task-number: QTQAINFRA-2501
Change-Id: Id2fb1e0b4ff12a48b6adb27291f55fc129296233
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2019-01-09 09:14:02 +00:00
Heikki Halmet
deacc72f73 Provisioning: Hide squish installation output from provisioning log
This will prevent the output of unwanted information

Change-Id: I547e91ef344ec51f799b26f254139b03d20a315c
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-12-17 07:16:39 +00:00
Tony Sarajärvi
4de6c12b17 Don't set http_proxy in its own subshell
The subshell actually meant, that the http_proxy environment
variable was never seen outside that check. Thus proxy was never
used where this was checked.

Change-Id: I069f43f13dffca98d8b60a47c2e105209018c84d
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2018-11-29 20:20:26 +00:00
Liang Qi
0c839f87fb Merge remote-tracking branch 'origin/5.11' into 5.12
Conflicts:
	coin/provisioning/common/unix/mqtt_broker.sh

Change-Id: I990dbf66b7120f2f1bcf202f0a2547a5ad9881f2
2018-11-20 19:37:06 +01:00
Tony Sarajärvi
b082d7d544 Replace .ci.local addresses with .intra.qt.io in provisioning scripts
Change-Id: I5e5bc084faabb28ffdf0437871445ecb8970bfb3
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-11-01 07:16:57 +00:00
Heikki Halmet
0e0d99d617 Provisioning: Support for Webassembly
This change will install specified packages of Emscripten and set up
the build environment for making binary artifacts of WebAssembly

Task-number: QTQAINFRA-1972
Change-Id: Ia5df6c208ea8f29ca8948f0273f321a59fc694a2
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
2018-09-15 10:06:47 +00:00
Johanna Aijala
1cad3084a9 Use Squish 6.3.2 Qt5.10 on Qt5.12 RTA tests
Update to latest Squish versions that Froglogic is providing. May need
to update to Squish 6.4 later, but now we use 6.3.2.

Change-Id: I89e1f20b58cdd2e948c8021a5fdc357bfc201db1
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-09-11 13:32:27 +00:00
Joni Jäntti
36dbfa3b97 Provisioning: Add chmod +x to Squish .run file
This file needs execution rights before it can be run in provisioning.

Change-Id: Id49de89e0260b18a71cbcd4f4dc2b5a28e0a9890
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
Reviewed-by: Johanna Äijälä <johanna.aijala@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-09-11 13:32:23 +00:00
Tony Sarajärvi
7d0c343fc6 Cleanup of RHEL 6 codes from provisioning scripts
RHEL 6 has been removed in 5.12+ verifications, so no need
to maintain those changes.

Change-Id: Ie65120bf8ec7a7d0aaa71b7c28e95283263e62eb
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-09-04 08:29:01 +00:00
Tony Sarajärvi
413fcafc14 Change 'test suite' in license headers of prov scripts
Change-Id: I9687bcb15b26287ea6eeb2098079137d6e3fc288
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-09-04 08:28:58 +00:00
Maurice Kalinowski
21b3591580 Provisioning: Update MQTT broker
Required to get shared subscription testing working, namely to fix
https://github.com/eclipse/paho.mqtt.testing/issues/56

Change-Id: Ib8657cde0d8e33135bbc2ebcb9558f8e6b13e40f
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
2018-08-17 16:01:51 +00:00
Maurice Kalinowski
53f4bb2d92 Provisioning: Update MQTT test broker
Updates required for latest MQTT5 features and fixes.

Change-Id: I63e967efafbe1b22800e338b6c431febe3389a45
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-08-04 12:12:06 +00:00
Heikki Halmet
74fd490884 Provisioning: Add force option when unmounting squish
Sometimes unmount of squish fail because resource busy even if
the installation is already succeeded. To avoid this let's add force
option to make sure unmounting works every time.

Task-number: QTQAINFRA-1964
Change-Id: Icacd01341d09577d9880e3ed208cb6ea18a7ac4f
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-06-29 13:37:34 +00:00
Heikki Halmet
4a7366f960 Provisioning: Refactor libclang installation
Libclang must use cached url (ci-files01-hki) to reduce the download time
Libclang release_60 versions were rebuild which requires sha1 changes

Task-number: QTQAINFRA-1966
Task-number: QTQAINFRA-2100
Change-Id: Iddf47a2a3a5bc38dbed684320120164bb40c028c
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-06-29 05:49:22 +00:00
Maurice Kalinowski
7551b13276 Provisioning: Update MQTT test broker
Change-Id: Ie4e9ff5c13eb5f882b70e4e948836ae78496dd40
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2018-06-25 06:36:50 +00:00
Liang Qi
a2b5c37d7a Merge remote-tracking branch 'origin/5.11.0' into 5.11
Change-Id: Ifa6c63ef62e339b90e571b13c73addaa17d7af4e
2018-05-18 11:55:23 +02:00
Simon Hausmann
d75411d6e5 Fix linking of qdoc against an external libclang (part 1)
This is the qt5 part of the fix for the task. For the special version of
libclang we hacked together that allows static linkage, set an
environment variable that the mighty qdoc build system can utilize to
activate the release packaging specific linkage.

Change-Id: Ie340955214585329fe8fd3bc790e3cd6c6be42f9
Task-number: QTBUG-68178
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-05-14 12:30:59 +00:00
Liang Qi
b1f1bf5560 Merge remote-tracking branch 'origin/5.11.0' into 5.11
Change-Id: Ia252225aa1339ac448dd726b149e08739c02dfc5
2018-05-09 22:12:22 +02:00
Tony Sarajärvi
9bf79e6bab Fix shellcheck complaints in provisioning scripts
Change-Id: I38306c6d2ff360dc12f8c33b289bbc4ae4935aca
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-04-29 13:20:29 +00:00
Jędrzej Nowacki
c5ddbdb86b Update clang version to 6.0
The version also contains statically compiled libraries so we can
ship qdoc without shipping libclang.

The change temporarly removes CI qdoc builds from RHEL6 and MinGW,
these two configurations are not used for packaging, just for CI
testing. We should restore MinGW later.

Task-number: QTBUG-66015
Task-number: QTBUG-67764
Change-Id: Ie37b4be6a639e81d9ef0815682c6d60981cbeaae
Reviewed-by: Jesus Fernandez <Jesus.Fernandez@qt.io>
Reviewed-by: Fredrik de Vibe <fredrik.devibe@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-04-28 11:59:06 +00:00
Heikki Halmet
83f2a25f17 Provisioning: Add test for squish installation
Squish is needed by RTA which is not run during normal qt5 build. Other
words squish installation is not tested during integration. That's why
we need separate test for it.
This change will fetch test and run test suite which is made to check
that squish installation works

Task-number: QTQAINFRA-1733
Change-Id: I9284fd52d87767ae04022c3a6996e0bb972315dd
Reviewed-by: Johanna Äijälä <johanna.aijala@qt.io>
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Matti Paaso <matti.paaso@qt.io>
2018-04-15 16:41:18 +00:00
Tony Sarajärvi
8375a33d4a Fix squish provisioning script typo
Change-Id: If33847a552fe40128cef577b99adf3152de3a9e2
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2018-04-15 16:40:53 +00:00
Heikki Halmet
c79d285ffa Provisioning: Remove sha1 file after shasum check
Change-Id: Iee8e66d98e5ebd2dee18baf8e1752a86065417d1
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-04-15 05:55:32 +00:00
Tony Sarajärvi
1b5f8cec03 Remove try_catch.sh helper script
Change-Id: I3981920bb8085ffbad89c31e0fd628a6e11146b6
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-03-19 11:57:06 +00:00
Tony Sarajärvi
90261c1df5 Remove try_catch codes from unix scripts
Change-Id: Iabadbf28c65132ae614048cb98e92f57e3786056
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-03-19 11:56:58 +00:00
Heikki Halmet
23b4bcf6c5 Provisioning: Move http_proxy.txt under shared folder
Change-Id: Id5b2f07a92eadfb21fbfdb3fb65621d6322dc8c4
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2018-03-15 15:07:27 +00:00
Jędrzej Nowacki
8a47814c91 Simplify DownloadURL
The script was over-engineered. The new version provides the same
functionality while removing uninteresting postmortem messages.

In addition try_catch.sh has a bug that injects set +e to a sourcing
script which was propagating through DownloadURL.sh to almost all
provisioning scripts. So it is good to avoid using it.

Change-Id: If191a0dcf52a29c4bec580a254e8e58a00130f6d
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2018-03-14 05:12:41 +00:00
Rainer Keller
04d1ee1df4 Provisioning: Create symbolic for shasum tool on macOS for compatibility
On macOS the sha1 tool is named 'shasum' while on all other unix systems
it is called 'sha1sum'. In order to make all unix provisioning scripts
run on macOS without special case handling a symbolic link is created.

Change-Id: Ib1365cbb0dcb31e68a22959ce939fb450ec7e3f7
Reviewed-by: Joni Jäntti <joni.jantti@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2018-03-10 13:31:34 +00:00
Tony Sarajärvi
54d8396d68 Set proxy on Windows VMs while provisioning
Task-number: QTQAINFRA-1669
Change-Id: Ia6b5b668051642a3729d7c7c83c69d8797045847
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-02-23 21:00:11 +00:00
Liang Qi
4cd14f4625 Merge remote-tracking branch 'origin/5.9' into 5.10
Change-Id: I3e4ae2508d632919db77af0f365eb25ace1a69e0
2018-02-03 11:08:26 +01:00
Heikki Halmet
4eaa017023 Provisioning: Disable Network Time Protocol
NTP need to be disabled from all tier2 templates

Task-number: QTQAINFRA-1608
Change-Id: I89e019c23b7a194efffa011200a8c870e3a65d66
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2018-02-02 07:14:40 +00:00
Maurice Kalinowski
f1422dc64b Provisioning: Switch download sources for mqtt paho broker
Internal resources are preferred and should be used first.

Change-Id: Ifd2a54858c9fb6b82c3dc268566c67238aad262e
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2018-02-02 06:22:53 +00:00