2017-09-10 16:44:28 +02:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
#
|
|
|
|
# Install dependencies required to build and test Git on Linux and macOS
|
|
|
|
#
|
|
|
|
|
2019-01-28 00:26:50 +01:00
|
|
|
. ${0%/*}/lib.sh
|
2017-09-10 16:44:28 +02:00
|
|
|
|
2017-09-11 03:18:29 +02:00
|
|
|
P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
|
|
|
|
LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
|
2020-04-10 19:18:07 +02:00
|
|
|
UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
|
|
|
|
tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
|
|
|
|
libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
|
2017-09-11 03:18:29 +02:00
|
|
|
|
travis-ci: introduce a $jobname variable for 'ci/*' scripts
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-12 00:34:44 +01:00
|
|
|
case "$jobname" in
|
|
|
|
linux-clang|linux-gcc)
|
2018-11-01 12:47:14 +01:00
|
|
|
sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test"
|
|
|
|
sudo apt-get -q update
|
2020-04-10 19:18:07 +02:00
|
|
|
sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
|
|
|
|
$UBUNTU_COMMON_PKGS
|
2018-11-01 12:47:14 +01:00
|
|
|
case "$jobname" in
|
|
|
|
linux-gcc)
|
|
|
|
sudo apt-get -q -y install gcc-8
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
|
travis-ci: fix running P4 and Git LFS tests in Linux build jobs
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01 12:55:35 +01:00
|
|
|
mkdir --parents "$P4_PATH"
|
|
|
|
pushd "$P4_PATH"
|
2017-09-11 03:18:29 +02:00
|
|
|
wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
|
|
|
|
wget --quiet "$P4WHENCE/bin.linux26x86_64/p4"
|
|
|
|
chmod u+x p4d
|
|
|
|
chmod u+x p4
|
|
|
|
popd
|
travis-ci: fix running P4 and Git LFS tests in Linux build jobs
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01 12:55:35 +01:00
|
|
|
mkdir --parents "$GIT_LFS_PATH"
|
|
|
|
pushd "$GIT_LFS_PATH"
|
2017-09-11 03:18:29 +02:00
|
|
|
wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
|
|
|
|
tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
|
|
|
|
cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
|
|
|
|
popd
|
2017-09-10 16:44:28 +02:00
|
|
|
;;
|
travis-ci: introduce a $jobname variable for 'ci/*' scripts
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-12 00:34:44 +01:00
|
|
|
osx-clang|osx-gcc)
|
2019-07-03 12:47:48 +02:00
|
|
|
export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
|
2017-09-11 03:18:29 +02:00
|
|
|
# Uncomment this if you want to run perf tests:
|
|
|
|
# brew install gnu-time
|
ci/lib.sh: encapsulate Travis-specific things
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.
Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 00:26:51 +01:00
|
|
|
test -z "$BREW_INSTALL_PACKAGES" ||
|
|
|
|
brew install $BREW_INSTALL_PACKAGES
|
2017-09-11 03:18:29 +02:00
|
|
|
brew link --force gettext
|
2021-01-15 03:51:02 +01:00
|
|
|
brew install --cask --no-quarantine perforce || {
|
ci(osx): use new location of the `perforce` cask
The Azure Pipelines builds are failing for macOS due to a change in the
location of the perforce cask. The command outputs the following error:
+ brew install caskroom/cask/perforce
Error: caskroom/cask was moved. Tap homebrew/cask-cask instead.
So let's try to call `brew cask install perforce` first (which is what
that error message suggests, in a most round-about way).
Prior to 672f51cb we used to install the 'perforce' package with 'brew
install perforce' (note: no 'cask' in there). The justification for
672f51cb was that the command 'brew install perforce' simply stopped
working, after Homebrew folks decided that it's better to move the
'perforce' package to a "cask". Their justification for this move was
that 'brew install perforce' "can fail due to a checksum mismatch ...",
and casks can be installed without checksum verification. And indeed,
both 'brew cask install perforce' and 'brew install
caskroom/cask/perforce' printed something along the lines of:
==> No checksum defined for Cask perforce, skipping verification
It is unclear why 672f51cb used 'brew install caskroom/cask/perforce'
instead of 'brew cask install perforce'. It appears (by running both
commands on old Travis CI macOS images) that both commands worked all
the same already back then.
In any case, as the error message at the top of this commit message
shows, 'brew install caskroom/cask/perforce' has stopped working
recently, but 'brew cask install perforce' still does, so let's use
that.
CI servers are typically fresh virtual machines, but not always. To
accommodate for that, let's try harder if `brew cask install perforce`
fails, by specifically pulling the latest `master` of the
`homebrew-cask` repository.
This will still fail, of course, when `homebrew-cask` falls behind
Perforce's release schedule. But once it is updated, we can now simply
re-run the failed jobs and they will pick up that update.
As for updating `homebrew-cask`: the beginnings of automating this in
https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary
will be finished once the next Perforce upgrade comes around.
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23 02:19:38 +02:00
|
|
|
# Update the definitions and try again
|
2019-11-20 02:18:39 +01:00
|
|
|
cask_repo="$(brew --repository)"/Library/Taps/homebrew/homebrew-cask &&
|
2021-01-15 03:51:02 +01:00
|
|
|
git -C "$cask_repo" pull --no-stat --ff-only &&
|
|
|
|
brew install --cask --no-quarantine perforce
|
ci(osx): use new location of the `perforce` cask
The Azure Pipelines builds are failing for macOS due to a change in the
location of the perforce cask. The command outputs the following error:
+ brew install caskroom/cask/perforce
Error: caskroom/cask was moved. Tap homebrew/cask-cask instead.
So let's try to call `brew cask install perforce` first (which is what
that error message suggests, in a most round-about way).
Prior to 672f51cb we used to install the 'perforce' package with 'brew
install perforce' (note: no 'cask' in there). The justification for
672f51cb was that the command 'brew install perforce' simply stopped
working, after Homebrew folks decided that it's better to move the
'perforce' package to a "cask". Their justification for this move was
that 'brew install perforce' "can fail due to a checksum mismatch ...",
and casks can be installed without checksum verification. And indeed,
both 'brew cask install perforce' and 'brew install
caskroom/cask/perforce' printed something along the lines of:
==> No checksum defined for Cask perforce, skipping verification
It is unclear why 672f51cb used 'brew install caskroom/cask/perforce'
instead of 'brew cask install perforce'. It appears (by running both
commands on old Travis CI macOS images) that both commands worked all
the same already back then.
In any case, as the error message at the top of this commit message
shows, 'brew install caskroom/cask/perforce' has stopped working
recently, but 'brew cask install perforce' still does, so let's use
that.
CI servers are typically fresh virtual machines, but not always. To
accommodate for that, let's try harder if `brew cask install perforce`
fails, by specifically pulling the latest `master` of the
`homebrew-cask` repository.
This will still fail, of course, when `homebrew-cask` falls behind
Perforce's release schedule. But once it is updated, we can now simply
re-run the failed jobs and they will pick up that update.
As for updating `homebrew-cask`: the beginnings of automating this in
https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary
will be finished once the next Perforce upgrade comes around.
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23 02:19:38 +02:00
|
|
|
} ||
|
2021-01-15 03:51:02 +01:00
|
|
|
brew install homebrew/cask/perforce
|
travis-ci: build with the right compiler
Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This
CC variable can be overridden from the command line, i.e. 'make
CC=gcc-X.Y' will build with that particular GCC version, but not from
the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
'cc' happens to be on the platform.
Our build jobs on Travis CI are badly affected by this. In the build
matrix we have dedicated build jobs to build Git with GCC and Clang
both on Linux and macOS from the very beginning (522354d70f (Add
Travis CI support, 2015-11-27)). Alas, this never really worked as
supposed to, because Travis CI specifies the compiler for those build
jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
projects built with './configure && make'). Consequently, our
'linux-clang' build job has always used GCC, because that's where 'cc'
points at in Travis CI's Linux images, while the 'osx-gcc' build job
has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
attempt to build with a more modern compiler, but to no avail.
Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
will run the "right" compiler. The Xcode 10.1 macOS image on Travis
CI already contains the gcc@8 package from Homebrew, but we have to
'brew link' it first to be able to use it.
So with this patch our build jobs will build Git with the following
compiler versions:
linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0
GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 02:29:13 +01:00
|
|
|
case "$jobname" in
|
|
|
|
osx-gcc)
|
ci: build Git with GCC 9 in the 'osx-gcc' build job
Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed
(but not linked) in the image we use [1]. Alas, since the last update
of this image a few days ago this is not the case anymore, and now it
contains GCC 9 (installed and linked) instead of GCC 8. The results
are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command
[2].
Let's move on to use GCC 9, with hopefully better error reporting and
improved -Wfoo flags and what not. On Travis CI this has the benefit
that we can spare a few seconds while installing dependencies, because
it already comes pre-installed, at least for now. The Azure Pipelines
OSX image doesn't include GCC, so we have to install it ourselves
anyway, and then we might as well install the newer version.
In a vain attempt I tried to future-proof this a bit:
- Install 'gcc@9' specifically, so we'll still get what we want even
after GCC 10 comes out, and the "plain" 'gcc' package starts to
refer to 'gcc@10'.
- Run both 'brew install gcc@9' and 'brew link gcc@9'. If 'gcc@9'
is already installed and linked, then both commands are noop and
exit with success. But as we saw in the past, sometimes the image
contains the expected GCC package installed but not linked, so
maybe it will happen again in the future as well. In that case
'brew install' is still a noop, and instructs the user to run
'brew link' instead, so that's what we'll do. And if 'gcc@9' is
not installed, then 'brew install' will install it, and the
subsequent 'brew link' becomes a noop.
An additional benefit of this patch is that from now on we won't
unnecessarily install GCC and its dependencies in the 'osx-clang' jobs
on Azure Pipelines.
[1] 7d4733c501 (ci: fix GCC install in the Travis CI GCC OSX job,
2019-10-24)
[2] https://travis-ci.org/git/git/jobs/615442297#L333
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-27 17:24:16 +01:00
|
|
|
brew install gcc@9
|
|
|
|
# Just in case the image is updated to contain gcc@9
|
|
|
|
# pre-installed but not linked.
|
|
|
|
brew link gcc@9
|
travis-ci: build with the right compiler
Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This
CC variable can be overridden from the command line, i.e. 'make
CC=gcc-X.Y' will build with that particular GCC version, but not from
the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
'cc' happens to be on the platform.
Our build jobs on Travis CI are badly affected by this. In the build
matrix we have dedicated build jobs to build Git with GCC and Clang
both on Linux and macOS from the very beginning (522354d70f (Add
Travis CI support, 2015-11-27)). Alas, this never really worked as
supposed to, because Travis CI specifies the compiler for those build
jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
projects built with './configure && make'). Consequently, our
'linux-clang' build job has always used GCC, because that's where 'cc'
points at in Travis CI's Linux images, while the 'osx-gcc' build job
has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
attempt to build with a more modern compiler, but to no avail.
Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
will run the "right" compiler. The Xcode 10.1 macOS image on Travis
CI already contains the gcc@8 package from Homebrew, but we have to
'brew link' it first to be able to use it.
So with this patch our build jobs will build Git with the following
compiler versions:
linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0
GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 02:29:13 +01:00
|
|
|
;;
|
|
|
|
esac
|
2017-09-10 16:44:28 +02:00
|
|
|
;;
|
2018-11-01 12:47:14 +01:00
|
|
|
StaticAnalysis)
|
|
|
|
sudo apt-get -q update
|
2019-10-01 13:16:26 +02:00
|
|
|
sudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \
|
2020-04-10 19:18:07 +02:00
|
|
|
libexpat-dev gettext make
|
2018-11-01 12:47:14 +01:00
|
|
|
;;
|
|
|
|
Documentation)
|
|
|
|
sudo apt-get -q update
|
2020-04-10 19:18:07 +02:00
|
|
|
sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make
|
2019-03-29 13:35:18 +01:00
|
|
|
|
|
|
|
test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
|
2020-04-10 19:18:08 +02:00
|
|
|
sudo gem install --version 1.5.8 asciidoctor
|
2018-11-01 12:47:14 +01:00
|
|
|
;;
|
2021-01-20 19:27:57 +01:00
|
|
|
linux-gcc-default|linux-gcc-4.8)
|
2020-04-10 19:18:07 +02:00
|
|
|
sudo apt-get -q update
|
|
|
|
sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
|
|
|
|
;;
|
2017-09-10 16:44:28 +02:00
|
|
|
esac
|
|
|
|
|
2018-11-01 12:47:14 +01:00
|
|
|
if type p4d >/dev/null && type p4 >/dev/null
|
|
|
|
then
|
|
|
|
echo "$(tput setaf 6)Perforce Server Version$(tput sgr0)"
|
|
|
|
p4d -V | grep Rev.
|
|
|
|
echo "$(tput setaf 6)Perforce Client Version$(tput sgr0)"
|
|
|
|
p4 -V | grep Rev.
|
|
|
|
fi
|
|
|
|
if type git-lfs >/dev/null
|
|
|
|
then
|
|
|
|
echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)"
|
|
|
|
git-lfs version
|
|
|
|
fi
|