2017-09-10 16:44:28 +02:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
#
|
|
|
|
# Install dependencies required to build and test Git on Linux and macOS
|
|
|
|
#
|
|
|
|
|
2019-01-28 00:26:50 +01:00
|
|
|
. ${0%/*}/lib.sh
|
2017-09-10 16:44:28 +02:00
|
|
|
|
2017-09-11 03:18:29 +02:00
|
|
|
P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
|
|
|
|
LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
|
|
|
|
|
travis-ci: introduce a $jobname variable for 'ci/*' scripts
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-12 00:34:44 +01:00
|
|
|
case "$jobname" in
|
|
|
|
linux-clang|linux-gcc)
|
2018-11-01 12:47:14 +01:00
|
|
|
sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test"
|
|
|
|
sudo apt-get -q update
|
ci: install 'libsvn-perl' instead of 'git-svn'
Since e7e9f5e7a1 (travis-ci: enable Git SVN tests t91xx on Linux,
2016-05-19) some of our Travis CI build jobs install the 'git-svn'
package, because it was a convenient way to install its dependencies,
which are necessary to run our 'git-svn' tests (we don't actually need
the 'git-svn' package itself). However, from those dependencies,
namely the 'libsvn-perl', 'libyaml-perl', and 'libterm-readkey-perl'
packages, only 'libsvn-perl' is necessary to run those tests, the
others arent, not even to fulfill some prereqs.
So update 'ci/install-dependencies.sh' to install only 'libsvn-perl'
instead of 'git-svn' and its additional dependencies.
Note that this change has more important implications than merely not
installing three unnecessary packages, as it keeps our builds working
with Travis CI's Xenial images. In our '.travis.yml' we never
explicitly specified which Linux image we want to use to run our Linux
build jobs, and so far they have been run on the default Ubuntu 14.04
Trusty image. However, 14.04 just reached its EOL, and Travis CI has
already began the transition to use 16.04 Xenial as the default Linux
build environment [1]. Alas, our Linux Clang and GCC build jobs can't
simply 'apt-get install git-svn' in the current Xenial images [2],
like they did in the Trusty images, and, consequently, fail.
Installing only 'libsvn-perl' avoids this issue, while the 'git svn'
tests are still run as they should.
[1] https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment
[2] 'apt-get install git-svn' in the Xenial image fails with:
The following packages have unmet dependencies:
git-svn : Depends: git (< 1:2.7.4-.)
E: Unable to correct problems, you have held broken packages.
The reason is that both the Trusty and Xenial images contain the
'git' package installed from 'ppa:git-core/ppa', so it's
considerably newer than the 'git' package in the corresponding
standard Ubuntu package repositories. The difference is that the
Trusty image still contains these third-party apt repositories, so
the 'git-svn' package was installed from the same PPA, and its
version matched the version of the already installed 'git'
package. In the Xenial image, however, these third-party
apt-repositories are removed (to reduce the risk of unrelated
interference and faster 'apt-get update') [3], and the version of
the 'git-svn' package coming from the standard Ubuntu package
repositories doesn't match the much more recent version of the
'git' package installed from the PPA, resulting in this dependecy
error.
Adding back the 'ppa:git-core/ppa' package repository would solve
this dependency issue as well, but since the troublesome package
happens to be unnecessary, not installing it in the first place is
better.
[3] https://docs.travis-ci.com/user/reference/xenial/#third-party-apt-repositories-removed
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-30 14:37:24 +02:00
|
|
|
sudo apt-get -q -y install language-pack-is libsvn-perl apache2
|
2018-11-01 12:47:14 +01:00
|
|
|
case "$jobname" in
|
|
|
|
linux-gcc)
|
|
|
|
sudo apt-get -q -y install gcc-8
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
|
travis-ci: fix running P4 and Git LFS tests in Linux build jobs
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01 12:55:35 +01:00
|
|
|
mkdir --parents "$P4_PATH"
|
|
|
|
pushd "$P4_PATH"
|
2017-09-11 03:18:29 +02:00
|
|
|
wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
|
|
|
|
wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
|
|
|
|
chmod u+x p4d
|
|
|
|
chmod u+x p4
|
|
|
|
popd
|
travis-ci: fix running P4 and Git LFS tests in Linux build jobs
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01 12:55:35 +01:00
|
|
|
mkdir --parents "$GIT_LFS_PATH"
|
|
|
|
pushd "$GIT_LFS_PATH"
|
2017-09-11 03:18:29 +02:00
|
|
|
wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
|
|
|
|
tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
|
|
|
|
cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
|
|
|
|
popd
|
2017-09-10 16:44:28 +02:00
|
|
|
;;
|
travis-ci: introduce a $jobname variable for 'ci/*' scripts
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-12 00:34:44 +01:00
|
|
|
osx-clang|osx-gcc)
|
2019-07-03 12:47:48 +02:00
|
|
|
export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
|
2017-09-11 03:18:29 +02:00
|
|
|
# Uncomment this if you want to run perf tests:
|
|
|
|
# brew install gnu-time
|
ci/lib.sh: encapsulate Travis-specific things
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.
Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 00:26:51 +01:00
|
|
|
test -z "$BREW_INSTALL_PACKAGES" ||
|
|
|
|
brew install $BREW_INSTALL_PACKAGES
|
2017-09-11 03:18:29 +02:00
|
|
|
brew link --force gettext
|
|
|
|
brew install caskroom/cask/perforce
|
travis-ci: build with the right compiler
Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This
CC variable can be overridden from the command line, i.e. 'make
CC=gcc-X.Y' will build with that particular GCC version, but not from
the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
'cc' happens to be on the platform.
Our build jobs on Travis CI are badly affected by this. In the build
matrix we have dedicated build jobs to build Git with GCC and Clang
both on Linux and macOS from the very beginning (522354d70f (Add
Travis CI support, 2015-11-27)). Alas, this never really worked as
supposed to, because Travis CI specifies the compiler for those build
jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
projects built with './configure && make'). Consequently, our
'linux-clang' build job has always used GCC, because that's where 'cc'
points at in Travis CI's Linux images, while the 'osx-gcc' build job
has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
attempt to build with a more modern compiler, but to no avail.
Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
will run the "right" compiler. The Xcode 10.1 macOS image on Travis
CI already contains the gcc@8 package from Homebrew, but we have to
'brew link' it first to be able to use it.
So with this patch our build jobs will build Git with the following
compiler versions:
linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0
GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 02:29:13 +01:00
|
|
|
case "$jobname" in
|
|
|
|
osx-gcc)
|
|
|
|
brew link gcc@8
|
|
|
|
;;
|
|
|
|
esac
|
2017-09-10 16:44:28 +02:00
|
|
|
;;
|
2018-11-01 12:47:14 +01:00
|
|
|
StaticAnalysis)
|
|
|
|
sudo apt-get -q update
|
|
|
|
sudo apt-get -q -y install coccinelle
|
|
|
|
;;
|
|
|
|
Documentation)
|
|
|
|
sudo apt-get -q update
|
|
|
|
sudo apt-get -q -y install asciidoc xmlto
|
2019-03-29 13:35:18 +01:00
|
|
|
|
|
|
|
test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
|
ci: stick with Asciidoctor v1.5.8 for now
The recent release of Asciidoctor v2.0.0 broke our documentation
build job on Travis CI, where we 'gem install asciidoctor', which
always brings us the latest and (supposedly) greatest. Alas, we are
not ready for that just yet, because it removed support for DocBook
4.5, and we have been requiring that particular DocBook version to
build 'user-manual.xml' with Asciidoctor, resulting in:
ASCIIDOC user-manual.xml
asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted.
Use --trace for backtrace
make[1]: *** [user-manual.xml] Error 1
Unfortunately, we can't simply switch to DocBook 5 right away, as
doing so leads to validation errors from 'xmlto', and working around
those leads to yet another errors... [1]
So let's stick with Asciidoctor v1.5.8 (latest stable release before
v2.0.0) in our documentation build job on Travis CI for now, until we
figure out how to deal with the fallout from Asciidoctor v2.0.0.
[1] https://public-inbox.org/git/20190324162131.GL4047@pobox.com/
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-29 20:52:46 +01:00
|
|
|
gem install --version 1.5.8 asciidoctor
|
2018-11-01 12:47:14 +01:00
|
|
|
;;
|
2017-09-10 16:44:28 +02:00
|
|
|
esac
|
|
|
|
|
2018-11-01 12:47:14 +01:00
|
|
|
if type p4d >/dev/null && type p4 >/dev/null
|
|
|
|
then
|
|
|
|
echo "$(tput setaf 6)Perforce Server Version$(tput sgr0)"
|
|
|
|
p4d -V | grep Rev.
|
|
|
|
echo "$(tput setaf 6)Perforce Client Version$(tput sgr0)"
|
|
|
|
p4 -V | grep Rev.
|
|
|
|
fi
|
|
|
|
if type git-lfs >/dev/null
|
|
|
|
then
|
|
|
|
echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)"
|
|
|
|
git-lfs version
|
|
|
|
fi
|