Commit Graph

1036 Commits

Author SHA1 Message Date
Konstantin Tokarev
4c880b3aff Provisioning: install dwz 0.13 to RHEL 7.6
It is used for compression of QtWebKit debug info, saving
about 1Gb (~40%) for non-compressed debug file of
libQt5WebKit.so, and ~50% of 7z-compressed debug package.

Version 0.12 provided in RedHat devtoolset exposes a bug
when applied to QtWebKit debug info:

https://sourceware.org/bugzilla/show_bug.cgi?id=24251

Change-Id: I0395aadfb7babbbc150f367f8b5dd296dc6d2064
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-03-05 04:12:22 +03:00
Konstantin Tokarev
7ba8c15d79 Update platform_configs for QtWebKit
* Add 64-bit MinGW config
* Don't use DisableTests for 64-bit MinGW, as it's necessary for at
least for the purpose of JS engine smoke check
* Sync configure flags with qt5.yaml, but not enable forced debug info
which causes build issues for both 32-bit and 64-bit configs

Change-Id: Iad5ff1459f8b39f17223da75b3319ab0cfc033d8
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-03-05 04:12:11 +03:00
Konstantin Tokarev
b7b6e31d47 Provisioning: Install QtWebKit dependencies on Ubuntu 18.04
Change-Id: Ib9a142ed6641436669c31ae8489ab9c89576e77a
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-03-05 04:12:01 +03:00
Konstantin Tokarev
b263f0a2d1 Provisioning: Install Conan packages for VS2019
Change-Id: Ibbee96ea7ad9e054e1521c897206c9863acc9c08
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-03-05 04:11:50 +03:00
Konstantin Tokarev
b73e61a458 Provisioning: update Conan and Conan packages to latest versions
Task-number: QTQAINFRA-3556
Change-Id: If9c600a6bb3aae893924f9372c8b248d3e58f887
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-22 12:01:44 +03:00
Heikki Halmet
fdb88edba0 Provisioning: Make sure that Perl's Data:Dumper is installed
Data:dumper is critical for release script build.pl. Currently data:dumper
is installed by devtoolset as a dependency for autoconf. But, it seems,
that in earlier version of devtoolset the autoconf wasn't installed by
devtoolset which caused the failure with build.pl.

Change-Id: I487a17ca0eb37c5984eb2f195e1024ed019b80d2
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-02-15 09:03:00 +00:00
Dimitrios Apostolou
d757c6d0ec Try to catch all errors when sdkmanager installs
Task-number: QTQAINFRA-3531
Change-Id: I008c31a824287c60e67a665d0c2a3aebec863929
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:39:06 +01:00
Dimitrios Apostolou
0d208f77ec Update python3 version on SLES-15
We update to the latest 3.7.x maintenance release for security/stability.

Change-Id: Ic2c251ab8e0db2c5e622f685e54e5cdd715723fc
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:39:04 +01:00
Dimitrios Apostolou
5d431ecc97 Refactor python3 installation in SLES-15 provisioning
+ Use DownloadURL to download from local cache
+ Verify download SHA
+ Silence the tarball extraction
+ Install some generic packages in 02-zypper.sh
+ Use some other generic facilities from common.sourced.sh
+ Other minor changes

Task-number: QTQAINFRA-3528
Change-Id: Ic085d43381e220a0a4a52ef6d5bb0cf8a9bf7032
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:39:02 +01:00
Dimitrios Apostolou
b2d3df3553 Exclude monitoring of some filesystems on macos
For a short time after boot, macos sometimes mounts a filesystem of type
"msdos" at the unique path /Volumes/firmwaresyncd.XXXXXX.

Telegraf [inputs.disk] plug-in stores records to InfluxDB, with the unique
mount path under "path" tag. The uniqueness of the path blows up the
cardinality of the database.

So exclude "msdos" filesystems from being monitored by telegraf.  Additionally
exclude "cd9660" filesystems that appear always as 100% full.

Change-Id: I4cdd3fca57b3db6157df168cc8a5314e64528b2d
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:39:01 +01:00
Dimitrios Apostolou
150f6277e0 Update telegraf version
Change-Id: I1a023e4f1de860a03a135f3b10f2c99d5309dd8b
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:58 +01:00
Dimitrios Apostolou
d84b366d55 Silence the sdkmanager progress bar on macos
The exact same method is already in use on all Linux provisions,
it was just copied to macos.

Change-Id: I873f6efd99b147c5364e0dda6c26c7dad7dee480
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:57 +01:00
Dimitrios Apostolou
27e42b7a3c Silence sdkmanager on Windows
sdkmanager is already silenced on the other OSes, as it prints thousands
of lines of progress bars in the logfiles.

Change-Id: I4a2aa422a83cfe5fc6a3a7ba7be69e1e9c01acc0
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:54 +01:00
Dimitrios Apostolou
47b74fb0ab Print filesystem information at the end of each provisioning
Change-Id: I549a917126954fae62dcdebb5c45a4f6aeb07551
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:52 +01:00
Dimitrios Apostolou
de2b280ad8 Windows now report versions.txt and df info at the end of provisioning
Change-Id: Ic2b299c04c91eb498822aca110cea774fdb7cf17
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:50 +01:00
Dimitrios Apostolou
5dda5f374f Mask PackageKit on SUSE
Fixes a flaky conflict between zypper and packagekitd.

packagekit is a *static* unit, that means it is missing an [Install] section,
so it can't be enabled or disabled. It is started as a dependency by other
units, or by dbus activation.

We *mask* the unit to make it impossible to start in any way.

For that we had replace a few occurrences of "pkcon" with the native package
manager.

Finally the package "libpython3_4m1_0" is not being installed any longer
because it never was: this package does not exist. It's just that pkcon does
not fail on invalid package names.

Fixes: QTQAINFRA-3523
Change-Id: I6ff580afb05aa508042c444b2ce777c3a9443dfe
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:47 +01:00
Dimitrios Apostolou
db23c64876 08-pythondev build should not execute external script
The important bits of the script were copied and modified.

NOTE that we no longer build openssl from source, the distro (SLES15) provides
a version that works well.

Task-number: QTQAINFRA-3528
Change-Id: Iad63d75055a7efc3e104789e4a515b8511cf0880
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:45 +01:00
Dimitrios Apostolou
2693e482b6 Extract archive directly to destination filesystem
Save some seconds or even minutes during provisioning, by avoiding copying
many files across filesystems, from /tmp to /usr/local. We now install
directly to /usr/local.

Change-Id: I3a8f545f1400995095193c4eb337f9dd8534ef72
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:43 +01:00
Dimitrios Apostolou
fc2d0b5150 Create more meaningful temporary file names
Change-Id: I4bbc94993e51502ea08109a4c99f3e212349ab24
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:41 +01:00
Dimitrios Apostolou
b018b841c1 Do not complain about existing empty files
Sometimes we pre-create the target filename using mktemp. In that case of
course the hash will not match, so avoid comparing.

Change-Id: Id0feb0178d659e03f5ceb000f738167cdc28a3ec
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:39 +01:00
Dimitrios Apostolou
25fcda6ff3 Alias expansion is not working reliably in non-interactive shells
It happened that old bash version on macOS the alias was not being expanded
for non-interactive shells running under "bash" but was being expanded fine
when under "sh". Using functions is the recommended way.

Here we define a function conditionally.

Change-Id: I607b265cb9e88b2b529ed170121e5a0833003acd
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:37 +01:00
Dimitrios Apostolou
c0033df4b9 Execute a specific version of homebrew install script
Also first try to download it from local mirror.

This install script appears to be doing a lot of git-cloning and downloading
itself though, so it is still far from the ideal solution: to only download
from locally cached endpoints.

Task-number: QTQAINFRA-3134
Change-Id: I32131045d6e62f302ec666b4ac055ca5ee9662d5
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-15 07:38:35 +01:00
Dominik Holland
7b4b2f86a0 Provisioning: Update jinja2 to 2.10.3
This makes sure we don't run into CVE-2019-10906 when using
jinja2 in QtIvis Autogenerator.

Change-Id: If51bf54bd26858c6b5faa3dd6a6627f2979b862d
Reviewed-by: Robert Griebl <robert.griebl@qt.io>
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2020-02-15 07:38:32 +01:00
Heikki Halmet
fd7bfcc0c5 Provisioning: Update signing tools for macOS
Update expired certificates

Task-number: QTQAINFRA-3540
Change-Id: I21e718901cb3f5c901273c0c122430f5caaea3d0
Reviewed-by: Eike Ziller <eike.ziller@qt.io>
Reviewed-by: Tim Jenssen <tim.jenssen@qt.io>
2020-02-14 05:08:31 +02:00
Dimitrios Apostolou
eee60c0824 Fix openssl build in SLES-15
Upstream changes of binutils or elfutils broke the build. Error was:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/libc_nonshared.a(elf-init.oS): unable to initialize decompress status for section .debug_info
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/libc_nonshared.a: error adding symbols: file format not recognized
collect2: error: ld returned 1 exit status

Task-number: QTQAINFRA-3527.
Change-Id: I50b677598b4415a281abbf0ddb2db39613646b36
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-02-05 22:25:34 +01:00
Tuomas Heimonen
eb3d2ed68b Provisioning: Remove unistd.h from postreSQL from 64 bit Windows
PostreSQL 9.6 version creates emtpy unistd.h file which will cause
failure in Qt Safe Renderer' MinGW build. By removing it the system
should use the one from the MinGW environment.

Task-number: COIN-488
Change-Id: I76d3f34dd9726dc4f0aac06c410f306dffc8f365
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-02-02 10:57:42 +02:00
Jani Heikkinen
89fceff6a4 Add '-qt-zlib' as a configure argument for windows builds
This is needed to use the bundled zlib in windows builds. In Qt 5.14.0
windows packages were using bundled one but in Qt 5.14.1 not anymore. So
this change is to restore same behavior than in Qt 5.14.0.

Linux and macOS packages seems to use system zlib also in 5.14.0; that's
why no changes needed there

Task-number: QTBUG-81724
Change-Id: Ife531d96c8438491e5d7b1f97f5338c4627be7e8
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-01-31 05:43:18 +00:00
Heikki Halmet
e7cf4b21f1 Provisioning: Remove unistd.h from postreSQL
PostreSQL 9.6 version creates emtpy unistd.h file which will cause
failure in Qt Safe Renderer' MinGW build. By removing it the system
should use the one from the MinGW environment.

Task-number: COIN-488
Change-Id: Id7f2f5024998cc5fbeb61e144bcd87f6e09b2bb1
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
2020-01-28 17:07:16 +02:00
Liang Qi
eee24c15d7 Merge remote-tracking branch 'origin/5.14.1' into 5.14
Change-Id: Ie43351d7e64b121137e9a923c79a0f26ae7e086a
2020-01-28 13:14:41 +01:00
Simon Hausmann
0363e5f7de Run tests on macOS 10.14
Also bump qtbase to pull in the latest fixes for 10.14 testing.

Task-number: QTQAINFRA-3108
Change-Id: I7a198c74a4f7e537d95053bfad25a905029aa2db
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-01-27 15:15:40 +01:00
Heikki Halmet
d42e5bda88 Provisioning: Install needed packages for Python build
For Conan python needs to be built from sources.

Task-number: QTBUG-81014
Change-Id: I8d14d65ce2f1c57a7fa370b93aa61bc156d617ca
Reviewed-by: Iikka Eklund <iikka.eklund@qt.io>
2020-01-23 08:59:12 +02:00
Samuli Piippo
154ae1ebef Remove default hostname from ubuntu
Removing the default hostname makes system create unique hostname
based on the IP address. This is needed to make NFS operations
reliable.

Task-number: QTQAINFRA-3257
Change-Id: I9e4dc4950f5d1a1b7f85372674f79d1701d1eab6
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-20 20:04:48 +00:00
Kai Koehne
a0e5b4b466 MinGW: Do not ship host tools with debug information
Follow the approach for Linux (see 269387), and do set the -s ln flag to
make sure the linker does not include debug information in host tools.

From ld documentation:
  -s: Omit all symbol information from the output file.

Fixes: QTBUG-81301
Change-Id: I21f828adec4a93855331d8b833cfa708b73baafd
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-01-17 12:32:09 +00:00
Heikki Halmet
5d027444b5 Provisioning: Update postgresql
Apparently libpq version 9.1.9 is no longer supported.
Let's use version 9.6 to keep maximum compatibility and align
the version with MacOS.

Task-number: QTBUG-79555
Change-Id: Iea98a1eb821f103e3a610ed602746407bfe5ea97
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-14 14:35:06 +02:00
Dimitrios Apostolou
fe4a724a3e Use internal mirror repositories for Ubuntu docker containers
Depending on different external URLs makes our provisioning scripts flaky,
because they can fail if any one of the external servers goes down.
In this patch

* the docker images qt_ubuntu_16.04 and qt_ubuntu_18.04 are being built first,
  they apt-get from the internal ubuntu mirror

* all the existing docker images have been rebased to these two images, so
  they inherit this behavior

* images that fetch files from the internet have been modified to not do the
  fetching themselves; insted URLs are fetched in advance using "DownloadURL"
  which accepts internal mirrored URLs, and the downloaded files are ADDed to
  the images with a local copy

* previously existing git clone commands have been modified to fetch
  internally mirrored tarballs

Task-number: QTQAINFRA-3343
Change-Id: Ic4640a4f7776e38623edba90f0eba7cb68631223
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-13 19:44:45 +01:00
Dimitrios Apostolou
302b228c7a Do not try to double-install docker-ce-cli
That happened because docker-ce* was installing both docker-ce and
docker-ce-cli, so apt was printing a warning.

Change-Id: I0cad82243c8c3d5c0b9f26fc7b24673268259aa4
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2020-01-13 19:44:45 +01:00
Dimitrios Apostolou
4d7f6c711c Remove unused script
Change-Id: Ic3c57966c78fd8158a2467956fd31eaa4374412d
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-10 16:06:07 +02:00
Heikki Halmet
9e834b7651 Provisioning: Update Xcode to 11.3 for macOS 10.14
Task-number: QTQAINFRA-3189
Change-Id: If16062254b33579b14f141d1db4aca2ad1e23afb
Reviewed-by: Liang Qi <liang.qi@qt.io>
2020-01-04 19:48:32 +00:00
Liang Qi
f09dbf65e1 Enable a no-gui build on Ubuntu 18.04 for 5.14
Task-number: QTQAINFRA-3406
Change-Id: Idb3e828f851fe00e76b0f3db1badb336062cd20a
Reviewed-by: Liang Qi <liang.qi@qt.io>
2020-01-04 18:03:26 +01:00
Assam Boudjelthia
51417379f7 Install QEMU from .deb package from internal cache
QEMU cloning and building with each provisioning takes
time, thus installing it from a prebuilt installable package
can save lots of time in provisioning.

Task-number: QTQAINFRA-3396
Change-Id: I3e9b0fedeecb6ddbae3997742827552ce9419399
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-01-03 17:33:24 +02:00
Tony Sarajärvi
11ded96d6e Remove SLES 12 provisioning scripts, as the platforms is already removed
Change-Id: I52ded9be93b65b74ac4d42434f82a44831e3ec48
Reviewed-by: Liang Qi <liang.qi@qt.io>
2019-12-10 17:17:51 +00:00
Heikki Halmet
a7c5f204bc Provisioning: New Tier1 image for Windows 10 x86_64
Change will upgrade Visual studios to the latest ones. Also MSVC 2019
build tools are installed for RTA.

MSVC 2019 version 16.3.10
MSVC 2017 version 15.9.17
MSVC 2019 build tools version 16.3.10

Task-number: QTQAINFRA-3389
Change-Id: Ic21645f96f8d4e6abc38951bd6bab519b4864418
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-12-10 09:48:42 +02:00
Dimitrios Apostolou
e4f9ac5607 Tag the docker images as "latest"
We will use this tag in qtbase to refer to the image to run, because
with the current sha1tree() tag, every minor change in the docker images
requires synchronized changes between repositories.

Task-number: QTBUG-79867
Change-Id: If94eb6dd8b97526b589e58064837bc6554fed79c
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-12-05 09:13:39 +01:00
Simo Fält
9cf8af5248 Provisioning: Remove sudo when installing pip packages for Qt user
There is no need to install as sudo for qt user. Using sudo will
change the cache dir's permissions preventing us from
installing anything with pip as normal user.

Change-Id: Id45bf3805ce4d772a6c82e429d4e1948289f076e
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2019-11-25 13:15:51 +00:00
Johanna Äijälä
08ded9e2f8 Provisioning: update Squish
Update with custom build of Squish to be able to launch Qt5.14
applications in RTA tests.

Change-Id: Ic420656852ae885b057de937baabf01135526764
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2019-11-21 11:26:11 +03:00
Heikki Halmet
dcfef7226f Provisioning: Change JDK script to use 'Remove' function from helpers
Task-number: QTQAINFRA-3369
Change-Id: I2e873eb50d9129c48d92370af80fe121039d172e
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-11-20 08:08:10 +02:00
Heikki Halmet
0335e9c4e5 Provisioning: Remove shell scripts under windows
Change-Id: I3d7a949e95bab39adc349c25d267b7fc28e676cf
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2019-11-20 09:08:04 +03:00
Heikki Halmet
601a9d39b2 Provisioning: Install python modules needed by packaging
Needed by packaging scripts

Task-number: COIN-214
Change-Id: I3ac719800c3afd183996379f6c23fbe3f8bcf7cb
Reviewed-by: Iikka Eklund <iikka.eklund@qt.io>
2019-11-20 08:07:58 +02:00
Liang Qi
6d7ae7ebe1 Merge remote-tracking branch 'origin/5.13' into 5.14
Conflicts:
	coin/provisioning/common/linux/docker.sh

Change-Id: Ic5c3fcaa856137de46c6f3830ca75cedb084927d
2019-11-18 19:21:53 +01:00
Heikki Halmet
d8eaed9b09 Provisioning: Update Android NDK version to r20 for macOS 10.14
Task-number: QTQAINFRA-3337
Change-Id: Ib14e3cc484198bb23bb611262b205144f8a8ac70
Reviewed-by: Johanna Äijälä <johanna.aijala@qt.io>
2019-11-18 14:23:08 +02:00