We run "git pull" against "$cask_repo"; clarify that we are
expecting not to have any of our own modifications and running "git
pull" to merely update, by passing "--ff-only" on the command line.
Also, the "brew cask install" command line triggers an error message
that says:
Error: Calling brew cask install is disabled! Use brew install
[--cask] instead.
In addition, "brew install caskroom/cask/perforce" step triggers an
error that says:
Error: caskroom/cask was moved. Tap homebrew/cask instead.
Attempt to see if blindly following the suggestion in these error
messages gets us into a better shape.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a later patch, we will run Documentation job in GitHub Actions.
The job will run without elevated permission.
Run `gem` with `sudo` to elevate permission in order to be able to
install to system location.
This will also keep this installation in-line with other installation in
our Linux system for CI.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[Danh: reword commit message]
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a later patch, we will support GitHub Action.
Explicitly install all of our build dependencies on Linux.
Since GitHub Action's Linux VM hasn't installed our build dependencies.
And there're no harm to reinstall them (in Travis)
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The most recent Azure Pipelines macOS agents enable what Apple calls
"System Integrity Protection". This makes `p4d -V` hang: there is some
sort of GUI dialog waiting for the user to acknowledge that the copied
binaries are legit and may be executed, but on build agents, there is no
user who could acknowledge that.
Let's ask Homebrew specifically to _not_ quarantine the Perforce
binaries.
Helped-by: Aleksandr Chebotov
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
CI jobs for macOS has been made less chatty when updating perforce
package used during testing.
* jc/azure-ci-osx-fix-fix:
ci(osx): update homebrew-cask repository with less noise
Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed
(but not linked) in the image we use [1]. Alas, since the last update
of this image a few days ago this is not the case anymore, and now it
contains GCC 9 (installed and linked) instead of GCC 8. The results
are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command
[2].
Let's move on to use GCC 9, with hopefully better error reporting and
improved -Wfoo flags and what not. On Travis CI this has the benefit
that we can spare a few seconds while installing dependencies, because
it already comes pre-installed, at least for now. The Azure Pipelines
OSX image doesn't include GCC, so we have to install it ourselves
anyway, and then we might as well install the newer version.
In a vain attempt I tried to future-proof this a bit:
- Install 'gcc@9' specifically, so we'll still get what we want even
after GCC 10 comes out, and the "plain" 'gcc' package starts to
refer to 'gcc@10'.
- Run both 'brew install gcc@9' and 'brew link gcc@9'. If 'gcc@9'
is already installed and linked, then both commands are noop and
exit with success. But as we saw in the past, sometimes the image
contains the expected GCC package installed but not linked, so
maybe it will happen again in the future as well. In that case
'brew install' is still a noop, and instructs the user to run
'brew link' instead, so that's what we'll do. And if 'gcc@9' is
not installed, then 'brew install' will install it, and the
subsequent 'brew link' becomes a noop.
An additional benefit of this patch is that from now on we won't
unnecessarily install GCC and its dependencies in the 'osx-clang' jobs
on Azure Pipelines.
[1] 7d4733c501 (ci: fix GCC install in the Travis CI GCC OSX job,
2019-10-24)
[2] https://travis-ci.org/git/git/jobs/615442297#L333
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The OSX CI build procedure updates the homebrew-cask repository
before attempting to install perforce again, after seeing an
installation failure. This involves a "git pull" that by default
computes and outputs diffstat, which would only grow as the time
goes by and the repository cast in stone in the CI build image
becomes more and more stale relative to the upstream repository in
the outside world.
Suppress the diffstat to both save cycles to generate it, and strain
on the eyeballs to skip it.
Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few days ago Travis CI updated their existing OSX images, including
the Homebrew database in the xcode10.1 OSX image that we use. Since
then installing dependencies in the 'osx-gcc' job fails when it tries
to link gcc@8:
+ brew link gcc@8
Error: No such keg: /usr/local/Cellar/gcc@8
GCC8 is still installed but not linked to '/usr/local' in the updated
image, as it was before this update, but now we have to link it by
running 'brew link gcc'. So let's do that then, and fall back to
linking gcc@8 if it doesn't, just to be sure.
Our builds on Azure Pipelines are unaffected by this issue. The OSX
image over there doesn't contain the gcc@8 package, so we have to
'brew install' it, which already takes care of linking it to
'/usr/local'. After that the 'brew link gcc' command added by this
patch fails, but the ||-chained fallback 'brew link gcc@8' command
succeeds with an "already linked" warning.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The Azure Pipelines builds are failing for macOS due to a change in the
location of the perforce cask. The command outputs the following error:
+ brew install caskroom/cask/perforce
Error: caskroom/cask was moved. Tap homebrew/cask-cask instead.
So let's try to call `brew cask install perforce` first (which is what
that error message suggests, in a most round-about way).
Prior to 672f51cb we used to install the 'perforce' package with 'brew
install perforce' (note: no 'cask' in there). The justification for
672f51cb was that the command 'brew install perforce' simply stopped
working, after Homebrew folks decided that it's better to move the
'perforce' package to a "cask". Their justification for this move was
that 'brew install perforce' "can fail due to a checksum mismatch ...",
and casks can be installed without checksum verification. And indeed,
both 'brew cask install perforce' and 'brew install
caskroom/cask/perforce' printed something along the lines of:
==> No checksum defined for Cask perforce, skipping verification
It is unclear why 672f51cb used 'brew install caskroom/cask/perforce'
instead of 'brew cask install perforce'. It appears (by running both
commands on old Travis CI macOS images) that both commands worked all
the same already back then.
In any case, as the error message at the top of this commit message
shows, 'brew install caskroom/cask/perforce' has stopped working
recently, but 'brew cask install perforce' still does, so let's use
that.
CI servers are typically fresh virtual machines, but not always. To
accommodate for that, let's try harder if `brew cask install perforce`
fails, by specifically pulling the latest `master` of the
`homebrew-cask` repository.
This will still fail, of course, when `homebrew-cask` falls behind
Perforce's release schedule. But once it is updated, we can now simply
re-run the failed jobs and they will pick up that update.
As for updating `homebrew-cask`: the beginnings of automating this in
https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary
will be finished once the next Perforce upgrade comes around.
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Dev support.
* dl/honor-cflags-in-hdr-check:
ci: run `hdr-check` as part of the `Static Analysis` job
Makefile: emulate compile in $(HCO) target better
pack-bitmap.h: remove magic number
promisor-remote.h: include missing header
apply.h: include missing header
Our documentation toolchain has traditionally been built around DocBook
4.5. This version of DocBook is the last DTD-based version of DocBook.
In 2009, DocBook 5 was introduced using namespaces and its syntax is
expressed in RELAX NG, which is more expressive and allows a wider
variety of syntax forms.
Asciidoctor, one of the alternatives for building our documentation,
moved support for DocBook 4.5 out of core in its recent 2.0 release and
now only supports DocBook 5 in the main release. The DocBoook 4.5
converter is still available as a separate component, but this is not
available in most distro packages. This would not be a problem but for
the fact that we use xmlto, which is still stuck in the DocBook 4.5 era.
xmlto performs DTD validation as part of the build process. This is not
problematic for DocBook 4.5, which has a valid DTD, but it clearly
cannot work for DocBook 5, since no DTD can adequately express its full
syntax. In addition, even if xmlto did support RELAX NG validation,
that wouldn't be sufficient because it uses the libxml2-based xmllint to
do so, which has known problems with validating interleaves in RELAX NG.
Fortunately, there's an easy way forward: ask Asciidoctor to use its
DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has
supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported
skipping validation for probably longer than that.
We also need to teach xmlto how to use the namespaced DocBook XSLT
stylesheets instead of the non-namespaced ones it usually uses.
Normally these stylesheets are interchangeable, but the non-namespaced
ones have a bug that causes them not to strip whitespace automatically
from certain elements when namespaces are in use. This results in
additional whitespace at the beginning of list elements, which is
jarring and unsightly.
We can do this by passing a custom stylesheet with the -x option that
simply imports the namespaced stylesheets via a URL. Any system with
support for XML catalogs will automatically look this URL up and
reference a local copy instead without us having to know where this
local copy is located. We know that anyone using xmlto will already
have catalogs set up properly since the DocBook 4.5 DTD used during
validation is also looked up via catalogs. All major Linux
distributions distribute the necessary stylesheets and have built-in
catalog support, and Homebrew does as well, albeit with a requirement to
set an environment variable to enable catalog support.
On the off chance that someone lacks support for catalogs, it is
possible for xmlto (via xmllint) to download the stylesheets from the
URLs in question, although this will likely perform poorly enough to
attract attention. People still have the option of using the prebuilt
documentation that we ship, so happily this should not be an impediment.
Finally, we need to filter out some messages from other stylesheets that
occur when invoking dblatex in the CI job. This tool strips namespaces
much like the unnamespaced DocBook stylesheets and prints similar
messages. If we permit these messages to be printed to standard error,
our documentation CI job will fail because we check standard error for
unexpected output. Due to dblatex's reliance on Python 2, we may need
to revisit its use in the future, in which case this problem may go
away, but this can be delayed until a future patch.
The final message we filter is due to libxslt on modern Debian and
Ubuntu. The patch which they use to implement reproducible ID
generation also prints messages about the ID generation. While this
doesn't affect our current CI images since they use Ubuntu 16.04 which
lacks this patch, if we upgrade to Ubuntu 18.04 or a modern Debian,
these messages will appear and, like the above messages, cause a CI
failure.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lately Homebrew learned to automagically clean up information about
outdated packages during other 'brew' commands, which might be useful
for the avarage user, but is a waste of time in CI build jobs, because
the next build jobs will start from the exact same image containing
the same outdated packages anyway.
Export HOMEBREW_NO_INSTALL_CLEANUP=1 to disable this auto cleanup feature,
shaving off about 20-30s from the time needed to install dependencies
in our macOS build jobs on Travis CI.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lately our GCC macOS build job on Travis CI has been erroring out
while installing dependencies with:
+brew link gcc@8
Error: No such keg: /usr/local/Cellar/gcc@8
The command "ci/install-dependencies.sh" failed and exited with 1 during .
Now, while gcc@8 is still pre-installed (but not linked) and would be
perfectly usable in the Travis CI macOS image we use [1], it's at
version 8.2. However, when installing dependencies we first
explicitly run 'brew update', which spends over two minutes to update
itself and information about the available packages, and it learns
about GCC 8.3. After that point gcc@8 exclusively refers to v8.3,
and, unfortunately, 'brew' is just too dumb to be able to do anything
with the still installed 8.2 package, and the subsequent 'brew link
gcc@8' fails. (Even 'brew uninstall gcc@8' fails with the same
error!)
Don't run 'brew update' to keep the already installed GCC 8.2 'brew
link'-able. Note that in addition we have to 'export
HOMEBREW_NO_AUTO_UPDATE=1' first, because 'brew' is so very helpful
that it would implicitly run update for us on the next 'brew install
<pkg>' otherwise.
Disabling 'brew update' has additional benefits:
- It shaves off 2-3mins from the ~4mins currently spent on
installing dependencies, and the macOS build jobs have always been
prone to exceeding the time limit on Travis CI.
- Our builds won't suddenly break because of the occasional Homebrew
breakages [2].
The drawback is that we'll be stuck with slightly older versions of
the packages that we install via Homebrew (Git-LFS 2.5.2 and Perforce
2018.1; they are currently at 2.7.2 and 2019.1, respectively). We
might want to reconsider this decision as time goes on and/or switch
to a more recent macOS image as they become available.
[1] 2000ac9fbf (travis-ci: switch to Xcode 10.1 macOS image,
2019-01-17)
[2] See e.g. a1ccaedd62 (travis-ci: make the OSX build jobs' 'brew
update' more quiet, 2019-02-02) or
https://public-inbox.org/git/20180907032002.23366-1-szeder.dev@gmail.com/T/#+u
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To run tests for Git SVN, our scripts for CI used to install the
git-svn package (in the hope that it would bring in the right
dependencies). This has been updated to install the more direct
dependency, namely, libsvn-perl.
* sg/ci-libsvn-perl:
ci: install 'libsvn-perl' instead of 'git-svn'
Since e7e9f5e7a1 (travis-ci: enable Git SVN tests t91xx on Linux,
2016-05-19) some of our Travis CI build jobs install the 'git-svn'
package, because it was a convenient way to install its dependencies,
which are necessary to run our 'git-svn' tests (we don't actually need
the 'git-svn' package itself). However, from those dependencies,
namely the 'libsvn-perl', 'libyaml-perl', and 'libterm-readkey-perl'
packages, only 'libsvn-perl' is necessary to run those tests, the
others arent, not even to fulfill some prereqs.
So update 'ci/install-dependencies.sh' to install only 'libsvn-perl'
instead of 'git-svn' and its additional dependencies.
Note that this change has more important implications than merely not
installing three unnecessary packages, as it keeps our builds working
with Travis CI's Xenial images. In our '.travis.yml' we never
explicitly specified which Linux image we want to use to run our Linux
build jobs, and so far they have been run on the default Ubuntu 14.04
Trusty image. However, 14.04 just reached its EOL, and Travis CI has
already began the transition to use 16.04 Xenial as the default Linux
build environment [1]. Alas, our Linux Clang and GCC build jobs can't
simply 'apt-get install git-svn' in the current Xenial images [2],
like they did in the Trusty images, and, consequently, fail.
Installing only 'libsvn-perl' avoids this issue, while the 'git svn'
tests are still run as they should.
[1] https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment
[2] 'apt-get install git-svn' in the Xenial image fails with:
The following packages have unmet dependencies:
git-svn : Depends: git (< 1:2.7.4-.)
E: Unable to correct problems, you have held broken packages.
The reason is that both the Trusty and Xenial images contain the
'git' package installed from 'ppa:git-core/ppa', so it's
considerably newer than the 'git' package in the corresponding
standard Ubuntu package repositories. The difference is that the
Trusty image still contains these third-party apt repositories, so
the 'git-svn' package was installed from the same PPA, and its
version matched the version of the already installed 'git'
package. In the Xenial image, however, these third-party
apt-repositories are removed (to reduce the risk of unrelated
interference and faster 'apt-get update') [3], and the version of
the 'git-svn' package coming from the standard Ubuntu package
repositories doesn't match the much more recent version of the
'git' package installed from the PPA, resulting in this dependecy
error.
Adding back the 'ppa:git-core/ppa' package repository would solve
this dependency issue as well, but since the troublesome package
happens to be unnecessary, not installing it in the first place is
better.
[3] https://docs.travis-ci.com/user/reference/xenial/#third-party-apt-repositories-removed
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The recent release of Asciidoctor v2.0.0 broke our documentation
build job on Travis CI, where we 'gem install asciidoctor', which
always brings us the latest and (supposedly) greatest. Alas, we are
not ready for that just yet, because it removed support for DocBook
4.5, and we have been requiring that particular DocBook version to
build 'user-manual.xml' with Asciidoctor, resulting in:
ASCIIDOC user-manual.xml
asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted.
Use --trace for backtrace
make[1]: *** [user-manual.xml] Error 1
Unfortunately, we can't simply switch to DocBook 5 right away, as
doing so leads to validation errors from 'xmlto', and working around
those leads to yet another errors... [1]
So let's stick with Asciidoctor v1.5.8 (latest stable release before
v2.0.0) in our documentation build job on Travis CI for now, until we
figure out how to deal with the fallout from Asciidoctor v2.0.0.
[1] https://public-inbox.org/git/20190324162131.GL4047@pobox.com/
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When our '.travis.yml' was split into several 'ci/*' scripts [1], the
installation of the 'asciidoctor' gem somehow ended up in
'ci/test-documentation.sh'.
Install it in 'ci/install-dependencies.sh', where we install other
dependencies of the Documentation build job as well (asciidoc,
xmlto).
[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts,
2017-09-10)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The way the OSX build jobs updates its build environment used the
"--quiet" option to "brew update" command, but it wasn't all that
quiet to be useful. The use of the option has been replaced with
an explicit redirection to the /dev/null (which incidentally would
have worked around a breakage by recent updates to homebrew, which
has fixed itself already).
* sg/travis-osx-brew-breakage-workaround:
travis-ci: make the OSX build jobs' 'brew update' more quiet
Prepare to run test suite on Azure Pipeline.
* js/vsts-ci: (22 commits)
test-date: drop unused parameter to getnanos()
ci: parallelize testing on Windows
ci: speed up Windows phase
tests: optionally skip bin-wrappers/
t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
tests: add t/helper/ to the PATH with --with-dashes
mingw: try to work around issues with the test cleanup
tests: include detailed trace logs with --write-junit-xml upon failure
tests: avoid calling Perl just to determine file sizes
README: add a build badge (status of the Azure Pipelines build)
mingw: be more generous when wrapping up the setitimer() emulation
ci: use git-sdk-64-minimal build artifact
ci: add a Windows job to the Azure Pipelines definition
Add a build definition for Azure DevOps
ci/lib.sh: add support for Azure Pipelines
tests: optionally write results as JUnit-style .xml
test-date: add a subcommand to measure times in shell scripts
ci: use a junction on Windows instead of a symlink
ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
ci/lib.sh: encapsulate Travis-specific things
...
Before installing the necessary dependencies, our OSX build jobs run
'brew update --quiet'. This is problematic for two reasons:
- This '--quiet' flag apparently broke overnight, resulting in
errored builds:
+brew update --quiet
==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
######################################################################## 100.0%
==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
Usage: brew update_report [--preinstall]
The Ruby implementation of brew update. Never called manually.
--preinstall Run in 'auto-update' mode (faster, less
output).
-f, --force Override warnings and enable potentially
unsafe operations.
-d, --debug Display any debugging information.
-v, --verbose Make some output more verbose.
-h, --help Show this message.
Error: invalid option: --quiet
The command "ci/install-dependencies.sh" failed and exited with 1 during .
I belive that this breakage will be noticed and fixed soon-ish, so
we could probably just wait a bit for this issue to solve itself,
but:
- 'brew update --quiet' wasn't really quiet in the first place, as
it listed over about 2000 lines worth of available packages that
we absolutely don't care about, see e.g. one of the latest
'master' builds:
https://travis-ci.org/git/git/jobs/486134962#L113
So drop this '--quiet' option and redirect 'brew update's standard
output to /dev/null to make it really quiet, thereby making the OSX
builds work again despite the above mentioned breakage.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.
Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The name is hard-coded to reflect that we use Travis CI for continuous
testing.
In the next commits, we will extend this to be able use Azure DevOps,
too.
So let's adjust the name to make it more generic.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This
CC variable can be overridden from the command line, i.e. 'make
CC=gcc-X.Y' will build with that particular GCC version, but not from
the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
'cc' happens to be on the platform.
Our build jobs on Travis CI are badly affected by this. In the build
matrix we have dedicated build jobs to build Git with GCC and Clang
both on Linux and macOS from the very beginning (522354d70f (Add
Travis CI support, 2015-11-27)). Alas, this never really worked as
supposed to, because Travis CI specifies the compiler for those build
jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
projects built with './configure && make'). Consequently, our
'linux-clang' build job has always used GCC, because that's where 'cc'
points at in Travis CI's Linux images, while the 'osx-gcc' build job
has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
attempt to build with a more modern compiler, but to no avail.
Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
will run the "right" compiler. The Xcode 10.1 macOS image on Travis
CI already contains the gcc@8 package from Homebrew, but we have to
'brew link' it first to be able to use it.
So with this patch our build jobs will build Git with the following
compiler versions:
linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0
GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ever since we started using Travis CI, we specified the list of
packages to install in '.travis.yml' via the APT addon. While running
our builds on Travis CI's container-based infrastructure we didn't
have another choice, because that environment didn't support 'sudo',
and thus we didn't have permission to install packages ourselves. With
the switch to the VM-based infrastructure in the previous patch we do
get a working 'sudo', so we can install packages by running 'sudo
apt-get -y install ...' as well.
Let's make use of this and install necessary packages in
'ci/install-dependencies.sh', so all the dependencies (i.e. both
packages and "non-packages" (P4 and Git-LFS)) are handled in the same
file. Install gcc-8 only in the 'linux-gcc' build job; so far it has
been unnecessarily installed in the 'linux-clang' build job as well.
Print the versions of P4 and Git-LFS conditionally, i.e. only when
they have been installed; with this change even the static analysis
and documentation build jobs start using 'ci/install-dependencies.sh'
to install packages, and neither of these two build jobs depend on and
thus install those.
This change will presumably be beneficial for the upcoming Azure
Pipelines integration [1]: preliminary versions of that patch series
run a couple of 'apt-get' commands to install the necessary packages
before running 'ci/install-dependencies.sh', but with this patch it
will be sufficient to run only 'ci/install-dependencies.sh'.
[1] https://public-inbox.org/git/1a22efe849d6da79f2c639c62a1483361a130238.1539598316.git.gitgitgadget@gmail.com/
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10) converted '.travis.yml's default 'before_install'
scriptlet to the 'ci/install-dependencies.sh' script, and while doing
so moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang
Linux build jobs to that script. This is wrong for two reasons:
- The purpose of that script is, as its name suggests, to install
dependencies, not to set any environment variables influencing
which tests should be run (though, arguably, this was already an
issue with the original 'before_install' scriptlet).
- Setting the variable has no effect anymore, because that script is
run in a separate shell process, and the variable won't be visible
in any of the other scripts, notably in 'ci/run-tests.sh'
responsible for, well, running the tests.
Luckily, this didn't have a negative effect on our Travis CI build
jobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to
"auto" and a functioning web server was installed in those Linux build
jobs, so the httpd tests were run anyway.
Apparently the httpd tests run just fine without GIT_TEST_HTTPD being
set, therefore we could simply remove this environment variable.
However, if a bug were to creep in to change the Travis CI build
environment to run the tests as root or to not install Apache, then
the httpd tests would be skipped and the build job would still
succeed. We would only notice if someone actually were to look
through the build job's trace log; but who would look at the trace log
of a successful build job?!
Since httpd tests are important, we do want to run them and we want to
be loudly reminded if they can't be run. Therefore, move setting
GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs
to 'ci/lib-travisci.sh' to ensure that the build job fails when the
httpd tests can't be run. (We could set it in 'ci/run-tests.sh' just
as well, but it's better to keep all environment variables in one
place in 'ci/lib-travisci.sh'.)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the Travis CI commands are in the '.travis.yml'. The yml format
does not support functions and therefore code duplication is necessary
to run commands across all builds.
To fix this, add a library for common CI functions. Move all Travis CI
code into dedicated scripts and make them call the library first.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
