undef/git-commit-vandalism
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2cdc292b31
git-commit-vandalism/.github/workflows/main.yml

318 lines
10 KiB
YAML
Raw Normal View History

ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
name: CI/PR
on: [push, pull_request]
env:
DEVELOPER: 1
jobs:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
ci-config:
ci: fix inconsistent indentation While YAML allows different indentation styles as long as each block is consistent, it is rather unusual to mix different indentations in a single file. Adjust to use two-space indentation everywhere. Signed-off-by: Adrian Moennich <adrian@planetcoding.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-21 11:27:26 +02:00
runs-on: ubuntu-latest
outputs:
ci: skip GitHub workflow runs for already-tested commits/trees When pushing a commit that has already passed a CI or PR build successfully, it makes sense to save some energy and time and skip the new build. Let's teach our GitHub workflow to do that. For good measure, we also compare the tree ID, which is what we actually test (the commit ID might have changed due to a reworded commit message, which should not affect the outcome of the run). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 17:29:34 +02:00
enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}
ci: fix inconsistent indentation While YAML allows different indentation styles as long as each block is consistent, it is rather unusual to mix different indentations in a single file. Adjust to use two-space indentation everywhere. Signed-off-by: Adrian Moennich <adrian@planetcoding.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-21 11:27:26 +02:00
steps:
- name: try to clone ci-config branch
run: |
git -c protocol.version=2 clone \
--no-tags \
--single-branch \
-b ci-config \
--depth 1 \
--no-checkout \
--filter=blob:none \
https://github.com/${{ github.repository }} \
config-repo &&
ci: fix indentation of the `ci-config` job The section added in e76eec35540f (ci: allow per-branch config for GitHub Actions, 2020-05-07) contains a `&&`-chain that connects several commands. The first command is actually so long that it stretches over multiple lines, and as per usual, the continuation lines are indented one more level than the first. However, the subsequent commands in the `&&`-chain were also indented one more level than the first command, which was almost certainly unintended. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-01 22:19:26 +02:00
cd config-repo &&
ci: avoid ugly "failure" in the `ci-config` job In the common case where users have _not_ pushed a `ci-config` branch to configure which branches should be included in the GitHub workflow runs, there is a big fat ugly annotation about a failure in the run's log: X Check failure on line 1 in .github @github-actions github-actions / ci-config .github#L1 Process completed with exit code 128. The reason is that the `ci-config` job tries to clone that `ci-config` branch, and even if it is configured to continue on error, the annotation is displayed, and it is distracting. Let's just handle this on the shell script level, so that the job's step is not marked as a failure. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-01 22:19:27 +02:00
git checkout HEAD -- ci/config || : ignore
ci: fix inconsistent indentation While YAML allows different indentation styles as long as each block is consistent, it is rather unusual to mix different indentations in a single file. Adjust to use two-space indentation everywhere. Signed-off-by: Adrian Moennich <adrian@planetcoding.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-21 11:27:26 +02:00
- id: check-ref
name: check whether CI is enabled for ref
run: |
enabled=yes
if test -x config-repo/ci/config/allow-ref &&
! config-repo/ci/config/allow-ref '${{ github.ref }}'
then
enabled=no
fi
echo "::set-output name=enabled::$enabled"
ci: skip GitHub workflow runs for already-tested commits/trees When pushing a commit that has already passed a CI or PR build successfully, it makes sense to save some energy and time and skip the new build. Let's teach our GitHub workflow to do that. For good measure, we also compare the tree ID, which is what we actually test (the commit ID might have changed due to a reworded commit message, which should not affect the outcome of the run). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 17:29:34 +02:00
- name: skip if the commit or tree was already tested
id: skip-if-redundant
uses: actions/github-script@v3
if: steps.check-ref.outputs.enabled == 'yes'
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
ci: make the "skip-if-redundant" check more defensive In 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08), we added a check that determines whether there is already a workflow run for the given commit (or at least tree), and if found, skips the current run. We just worked around an issue with this check where older runs might unexpectedly miss the `head_commit` attribute. Let's be even more defensive by catching all kinds of exceptions, logging them as warnings, and continue the run without skipping it (after all, if the check fails, we _want_ to continue with the run). This commit is best viewed with the diff option `-w` because it increases the indentation level of the GitHub Action script by two spaces, surrounding it by a `try ... catch` construct. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-11 23:27:11 +02:00
try {
// Figure out workflow ID, commit and tree
const { data: run } = await github.actions.getWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
});
const workflow_id = run.workflow_id;
const head_sha = run.head_sha;
const tree_id = run.head_commit.tree_id;
ci: skip GitHub workflow runs for already-tested commits/trees When pushing a commit that has already passed a CI or PR build successfully, it makes sense to save some energy and time and skip the new build. Let's teach our GitHub workflow to do that. For good measure, we also compare the tree ID, which is what we actually test (the commit ID might have changed due to a reworded commit message, which should not affect the outcome of the run). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 17:29:34 +02:00
ci: make the "skip-if-redundant" check more defensive In 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08), we added a check that determines whether there is already a workflow run for the given commit (or at least tree), and if found, skips the current run. We just worked around an issue with this check where older runs might unexpectedly miss the `head_commit` attribute. Let's be even more defensive by catching all kinds of exceptions, logging them as warnings, and continue the run without skipping it (after all, if the check fails, we _want_ to continue with the run). This commit is best viewed with the diff option `-w` because it increases the indentation level of the GitHub Action script by two spaces, surrounding it by a `try ... catch` construct. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-11 23:27:11 +02:00
// See whether there is a successful run for that commit or tree
const { data: runs } = await github.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 500,
status: 'success',
workflow_id,
});
for (const run of runs.workflow_runs) {
if (head_sha === run.head_sha) {
core.warning(`Successful run for the commit ${head_sha}: ${run.html_url}`);
core.setOutput('enabled', ' but skip');
break;
}
if (run.head_commit && tree_id === run.head_commit.tree_id) {
core.warning(`Successful run for the tree ${tree_id}: ${run.html_url}`);
core.setOutput('enabled', ' but skip');
break;
}
ci: skip GitHub workflow runs for already-tested commits/trees When pushing a commit that has already passed a CI or PR build successfully, it makes sense to save some energy and time and skip the new build. Let's teach our GitHub workflow to do that. For good measure, we also compare the tree ID, which is what we actually test (the commit ID might have changed due to a reworded commit message, which should not affect the outcome of the run). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 17:29:34 +02:00
}
ci: make the "skip-if-redundant" check more defensive In 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08), we added a check that determines whether there is already a workflow run for the given commit (or at least tree), and if found, skips the current run. We just worked around an issue with this check where older runs might unexpectedly miss the `head_commit` attribute. Let's be even more defensive by catching all kinds of exceptions, logging them as warnings, and continue the run without skipping it (after all, if the check fails, we _want_ to continue with the run). This commit is best viewed with the diff option `-w` because it increases the indentation level of the GitHub Action script by two spaces, surrounding it by a `try ... catch` construct. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-11 23:27:11 +02:00
} catch (e) {
core.warning(e);
ci: skip GitHub workflow runs for already-tested commits/trees When pushing a commit that has already passed a CI or PR build successfully, it makes sense to save some energy and time and skip the new build. Let's teach our GitHub workflow to do that. For good measure, we also compare the tree ID, which is what we actually test (the commit ID might have changed due to a reworded commit message, which should not affect the outcome of the run). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 17:29:34 +02:00
}
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
windows-build:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
runs-on: windows-latest
steps:
ci: accelerate the checkout By upgrading from v1 to v2 of `actions/checkout`, we avoid fetching all the tags and the complete history: v2 only fetches one revision by default. This should make things a lot faster. Note that `actions/checkout@v2` seems to be incompatible with running in containers: https://github.com/actions/checkout/issues/151. Therefore, we stick with v1 there. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:14 +02:00
- uses: actions/checkout@v2
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
- uses: git-for-windows/setup-git-for-windows-sdk@v1
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- name: build
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
HOME: ${{runner.workspace}}
NO_PERL: 1
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
run: ci/make-test-artifacts.sh artifacts
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: zip up tracked files
run: git archive -o artifacts/tracked.tar.gz HEAD
- name: upload tracked files and build artifacts
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/upload-artifact@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
with:
name: windows-artifacts
path: artifacts
windows-test:
runs-on: windows-latest
needs: [windows-build]
strategy:
ci: do not cancel all jobs of a matrix if one fails The CI/PR GitHub Actions workflow uses the 'matrix' strategy for the "windows-test", "vs-test", "regular" and "dockerized" jobs. The default behaviour of GitHub Actions is to cancel all in-progress jobs in a matrix if one of the job of the matrix fails [1]. This is not ideal as a failure early in a job, like during installation of the build/test dependencies on a specific platform, leads to the cancellation of all other jobs in the matrix. Set the 'fail-fast' variable to 'false' for all four matrix jobs in the workflow. [1] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28 05:06:08 +01:00
fail-fast: false
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
matrix:
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
steps:
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: download tracked files and build artifacts
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/download-artifact@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
with:
name: windows-artifacts
path: ${{github.workspace}}
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: extract tracked files and build artifacts
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
shell: bash
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
- uses: git-for-windows/setup-git-for-windows-sdk@v1
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- name: test
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
run: ci/run-test-slice.sh ${{matrix.nr}} 10
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- name: ci/print-test-failures.sh
if: failure()
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
run: ci/print-test-failures.sh
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/upload-artifact@v2
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
vs-build:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
NO_PERL: 1
GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
runs-on: windows-latest
steps:
ci: accelerate the checkout By upgrading from v1 to v2 of `actions/checkout`, we avoid fetching all the tags and the complete history: v2 only fetches one revision by default. This should make things a lot faster. Note that `actions/checkout@v2` seems to be incompatible with running in containers: https://github.com/actions/checkout/issues/151. Therefore, we stick with v1 there. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:14 +02:00
- uses: actions/checkout@v2
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
- uses: git-for-windows/setup-git-for-windows-sdk@v1
cmake(install): include vcpkg dlls Our CMake configuration generates not only build definitions, but also install definitions: After building Git using `msbuild git.sln`, the built artifacts can be installed via `msbuild INSTALL.vcxproj`. To specify _where_ the files should be installed, the `-DCMAKE_INSTALL_PREFIX=<path>` option can be used when running CMake. However, this process would really only install the files that were just built. On Windows, we need more than that: We also need the `.dll` files of the dependencies (such as libcurl). The `vcpkg` ecosystem, which we use to obtain those dependencies, can be asked to install said `.dll` files really easily, so let's do that. This requires more than just the built `vcpkg` artifacts in the CI build definition; We now clone the `vcpkg` repository so that the relevant CMake scripts are available, in particular the ones related to defining the toolchain. Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-29 14:41:45 +02:00
- name: initialize vcpkg
uses: actions/checkout@v2
with:
repository: 'microsoft/vcpkg'
path: 'compat/vcbuild/vcpkg'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- name: download vcpkg artifacts
shell: powershell
run: |
$urlbase = "https://dev.azure.com/git/git/_apis/build/builds"
$id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=9&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
$downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[0].resource.downloadUrl
(New-Object Net.WebClient).DownloadFile($downloadUrl, "compat.zip")
Expand-Archive compat.zip -DestinationPath . -Force
Remove-Item compat.zip
- name: add msbuild to PATH
GitHub workflow: automatically follow minor updates of setup-msbuild It is the custom to follow minor updates of GitHub Actions automatically, by using the suffix `@v1`. Actions' maintainers will then update that `v1` ref to point to the newest. However, for `microsoft/setup-msbuild`, 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11) uses a very specific `@v1.0.0` suffix. In this instance, that is a problem: should `setup-msbuild` release a new version that intends to fix a critical bug, we won't know it, and we won't use it. Such a scenario is not theoretical. It is happening right now: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands Let's simplify our setup, allowing us to benefit from automatically using the newest v1.x. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-07 10:17:49 +02:00
uses: microsoft/setup-msbuild@v1
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
- name: copy dlls to root
ci (vs-build): use `cmd` to copy the DLLs, not `powershell` We use a `.bat` script to copy the DLLs in the `vs-build` job, and those type of scripts are native to CMD, not to PowerShell. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:12 +02:00
shell: cmd
run: compat\vcbuild\vcpkg_copy_dlls.bat release
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
- name: generate Visual Studio solution
shell: bash
run: |
cmake `pwd`/contrib/buildsystems/ -DCMAKE_PREFIX_PATH=`pwd`/compat/vcbuild/vcpkg/installed/x64-windows \
ci (vs-build): build with NO_GETTEXT We already build Git for Windows with `NO_GETTEXT` when compiling with GCC. Let's do the same with Visual C, too. Note that we do not technically _need_ to pass `NO_GETTEXT` explicitly in that `make artifacts-tar` invocation because we do this while `MSVC` is set (which will set `uname_S := Windows`, which in turn will set `NO_GETTEXT = YesPlease`). But it is definitely nicer to be explicit here. Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Helped-by: Matthias Aßhauer <mha1993@live.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:13 +02:00
-DNO_GETTEXT=YesPlease -DPERL_TESTS=OFF -DPYTHON_TESTS=OFF -DCURL_NO_CURL_CMAKE=ON
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- name: MSBuild
run: msbuild git.sln -property:Configuration=Release -property:Platform=x64 -maxCpuCount:4 -property:PlatformToolset=v142
- name: bundle artifact tar
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
MSVC: 1
VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
run: |
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
mkdir -p artifacts &&
ci (vs-build): build with NO_GETTEXT We already build Git for Windows with `NO_GETTEXT` when compiling with GCC. Let's do the same with Visual C, too. Note that we do not technically _need_ to pass `NO_GETTEXT` explicitly in that `make artifacts-tar` invocation because we do this while `MSVC` is set (which will set `uname_S := Windows`, which in turn will set `NO_GETTEXT = YesPlease`). But it is definitely nicer to be explicit here. Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Helped-by: Matthias Aßhauer <mha1993@live.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:13 +02:00
eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)"
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: zip up tracked files
run: git archive -o artifacts/tracked.tar.gz HEAD
- name: upload tracked files and build artifacts
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/upload-artifact@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
with:
name: vs-artifacts
path: artifacts
vs-test:
runs-on: windows-latest
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
needs: vs-build
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
strategy:
ci: do not cancel all jobs of a matrix if one fails The CI/PR GitHub Actions workflow uses the 'matrix' strategy for the "windows-test", "vs-test", "regular" and "dockerized" jobs. The default behaviour of GitHub Actions is to cancel all in-progress jobs in a matrix if one of the job of the matrix fails [1]. This is not ideal as a failure early in a job, like during installation of the build/test dependencies on a specific platform, leads to the cancellation of all other jobs in the matrix. Set the 'fail-fast' variable to 'false' for all four matrix jobs in the workflow. [1] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28 05:06:08 +01:00
fail-fast: false
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
matrix:
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
steps:
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
- uses: git-for-windows/setup-git-for-windows-sdk@v1
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: download tracked files and build artifacts
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/download-artifact@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
with:
name: vs-artifacts
path: ${{github.workspace}}
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
- name: extract tracked files and build artifacts
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
shell: bash
ci (windows): transfer also the Git-tracked files to the test jobs Git's test suite is excruciatingly slow on Windows, mainly due to the fact that it executes a lot of shell script code, and that's simply not native to Windows. To help with that, we established the pattern where the artifacts are first built in one job, and then multiple test jobs run in parallel using the artifacts built in the first job. We take pains in transferring only the build outputs, and letting `actions/checkout` fill in the rest of the files. One major downside of that strategy is that the test jobs might fail to check out the intended revision (e.g. because the branch has been updated while the build was running, as is frequently the case with the `seen` branch). Let's transfer also the files tracked by Git, and skip the checkout step in the test jobs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:11 +02:00
run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
- name: test
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
NO_SVN_TESTS: 1
GIT_TEST_SKIP_REBASE_P: 1
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
run: ci/run-test-slice.sh ${{matrix.nr}} 10
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
- name: ci/print-test-failures.sh
if: failure()
ci: use the new GitHub Action to download git-sdk-64-minimal In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is no suitable one installed by default on Windows. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, unfortunately enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) to stop PowerShell from doing unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. Caching it (unfortunately only on a per-branch basis) speeds up the download step, and makes it much more robust at the same time by virtue of accessing a cache location that is closer in the network topology. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:11 +02:00
shell: bash
run: ci/print-test-failures.sh
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/upload-artifact@v2
ci: modification of main.yml to use cmake for vs-build job Teach .github/workflows/main.yml to use CMake for VS builds. Modified the vs-test step to match windows-test step. This speeds up the vs-test. Calling git-cmd from powershell and then calling git-bash to perform the tests slows things down(factor of about 6). So git-bash is directly called from powershell to perform the tests using prove. NOTE: Since GitHub keeps the same directory for each job (with respect to path) absolute paths are used in the bin-wrapper scripts. GitHub has switched to CMake 3.17.1 which changed the behaviour of FindCURL module. An extra definition (-DCURL_NO_CURL_CMAKE=ON) has been added to revert to the old behaviour. In the configuration phase CMake looks for the required libraries for building git (eg zlib,libiconv). So we extract the libraries before we configure. To check for ICONV_OMITS_BOM libiconv.dll needs to be in the working directory of script or path. So we copy the dlls before we configure. Signed-off-by: Sibi Siddharthan <sibisiddharthan.github@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-26 18:11:38 +02:00
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
regular:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
strategy:
ci: do not cancel all jobs of a matrix if one fails The CI/PR GitHub Actions workflow uses the 'matrix' strategy for the "windows-test", "vs-test", "regular" and "dockerized" jobs. The default behaviour of GitHub Actions is to cancel all in-progress jobs in a matrix if one of the job of the matrix fails [1]. This is not ideal as a failure early in a job, like during installation of the build/test dependencies on a specific platform, leads to the cancellation of all other jobs in the matrix. Set the 'fail-fast' variable to 'false' for all four matrix jobs in the workflow. [1] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28 05:06:08 +01:00
fail-fast: false
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
matrix:
vector:
- jobname: linux-clang
cc: clang
pool: ubuntu-latest
- jobname: linux-gcc
cc: gcc
pool: ubuntu-latest
- jobname: osx-clang
cc: clang
pool: macos-latest
- jobname: osx-gcc
cc: gcc
pool: macos-latest
ci: remove GETTEXT_POISON jobs A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option. We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job in the sense that it's the job that was otherwise run with the default compiler, no other GIT_TEST_* options etc. So let's keep it under the name "linux-gcc-default". This means we can remove the initial "make test" from the "linux-gcc" job (it does another one after setting a bunch of GIT_TEST_* variables). I'm not doing that because it would conflict with the in-flight 334afbc76fb (tests: mark tests relying on the current default for `init.defaultBranch`, 2020-11-18) (currently on the "seen" branch, so the SHA-1 will almost definitely change). It's going to use that "make test" again for different reasons, so let's preserve it for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-20 19:27:57 +01:00
- jobname: linux-gcc-default
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
cc: gcc
pool: ubuntu-latest
env:
CC: ${{matrix.vector.cc}}
jobname: ${{matrix.vector.jobname}}
runs-on: ${{matrix.vector.pool}}
steps:
ci: accelerate the checkout By upgrading from v1 to v2 of `actions/checkout`, we avoid fetching all the tags and the complete history: v2 only fetches one revision by default. This should make things a lot faster. Note that `actions/checkout@v2` seems to be incompatible with running in containers: https://github.com/actions/checkout/issues/151. Therefore, we stick with v1 there. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:14 +02:00
- uses: actions/checkout@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- run: ci/install-dependencies.sh
- run: ci/run-build-and-tests.sh
- run: ci/print-test-failures.sh
if: failure()
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
ci: upgrade to using actions/{up,down}load-artifacts v2 The GitHub Actions to upload/download workflow artifacts saw a major upgrade since Git's GitHub workflow was established. Let's use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-23 17:24:13 +02:00
uses: actions/upload-artifact@v2
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
dockerized:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
strategy:
ci: do not cancel all jobs of a matrix if one fails The CI/PR GitHub Actions workflow uses the 'matrix' strategy for the "windows-test", "vs-test", "regular" and "dockerized" jobs. The default behaviour of GitHub Actions is to cancel all in-progress jobs in a matrix if one of the job of the matrix fails [1]. This is not ideal as a failure early in a job, like during installation of the build/test dependencies on a specific platform, leads to the cancellation of all other jobs in the matrix. Set the 'fail-fast' variable to 'false' for all four matrix jobs in the workflow. [1] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28 05:06:08 +01:00
fail-fast: false
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
matrix:
vector:
- jobname: linux-musl
image: alpine
- jobname: Linux32
image: daald/ubuntu32:xenial
env:
jobname: ${{matrix.vector.jobname}}
runs-on: ubuntu-latest
container: ${{matrix.vector.image}}
steps:
- uses: actions/checkout@v1
- run: ci/install-docker-dependencies.sh
- run: ci/run-build-and-tests.sh
- run: ci/print-test-failures.sh
if: failure()
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
ci: use upload-artifacts v1 for dockerized jobs e9f79acb28 (ci: upgrade to using actions/{up,down}load-artifacts v2, 2021-06-23) changed all calls to that action from v1 to v2, but there is still an open bug[1] that affects all nodejs actions and prevents its use in 32-bit linux (as used by the Linux32 container) move all dockerized jobs to use v1 that was built in C# and therefore doesn't have this problem, which will otherwise manifest with confusing messages like: /usr/bin/docker exec 0285adacc4536b7cd962079c46f85fa05a71e66d7905b5e4b9b1a0e8b305722a sh -c "cat /etc/*release | grep ^ID" OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: no such file or directory: unknown [1] https://github.com/actions/runner/issues/1011 Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-13 11:13:49 +02:00
uses: actions/upload-artifact@v1
ci: let GitHub Actions upload failed tests' directories Arguably, CI builds' most important task is to not only identify regressions, but to make it as easy as possible to investigate what went wrong. In that light, we will want to provide users with a way to inspect the tests' output as well as the corresponding directories. This commit adds build steps that are only executed when tests failed, uploading the relevant information as build artifacts. These artifacts can then be downloaded by interested parties to diagnose the failures more efficiently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:14 +02:00
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
static-analysis:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
jobname: StaticAnalysis
.github/workflows/main.yml: run static-analysis on bionic GitHub Actions is transitioning workflow steps that run on 'ubuntu-latest' from 18.04 to 20.04 [1]. This works fine in all steps except the static-analysis one, since Coccinelle isn't available on Ubuntu focal (it is only available in the universe suite). Until Coccinelle can be installed from 20.04's main suite, pin the static-analysis build to run on 18.04, where it can be installed by default. [1]: https://github.com/actions/virtual-environments/issues/1816 Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-08 22:22:34 +01:00
runs-on: ubuntu-18.04
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
steps:
ci: accelerate the checkout By upgrading from v1 to v2 of `actions/checkout`, we avoid fetching all the tags and the complete history: v2 only fetches one revision by default. This should make things a lot faster. Note that `actions/checkout@v2` seems to be incompatible with running in containers: https://github.com/actions/checkout/issues/151. Therefore, we stick with v1 there. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:14 +02:00
- uses: actions/checkout@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- run: ci/install-dependencies.sh
- run: ci/run-static-analysis.sh
ci: run `make sparse` as part of the GitHub workflow Occasionally we receive reviews after patches were integrated, where `sparse` (https://sparse.docs.kernel.org/en/latest/ has more information on that project) identified problems such as file-local variables or functions being declared as global. By running `sparse` as part of our Continuous Integration, we can catch such things much earlier. Even better: developers who activated GitHub Actions on their forks can catch such issues before even sending their patches to the Git mailing list. This addresses https://github.com/gitgitgadget/git/issues/345 Note: Not even Ubuntu 20.04 ships with a new enough version of `sparse` to accommodate Git's needs. The symptom looks like this: add-interactive.c:537:51: error: Using plain integer as NULL pointer To counter that, we download and install the custom-built `sparse` package from the Azure Pipeline that we specifically created to address this issue. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-14 13:50:33 +02:00
sparse:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
env:
jobname: sparse
runs-on: ubuntu-20.04
steps:
- name: Download a current `sparse` package
# Ubuntu's `sparse` version is too old for us
uses: git-for-windows/get-azure-pipelines-artifact@v0
with:
repository: git/git
definitionId: 10
artifact: sparse-20.04
- name: Install the current `sparse` package
run: sudo dpkg -i sparse-20.04/sparse_*.deb
- uses: actions/checkout@v2
ci/install-dependencies: handle "sparse" job package installs This just matches the style/location of the package installation for other jobs. There should be no functional change. I did flip the order of the options and command-name ("-y update" instead of "update -y") for consistency with other lines in the same file. Note also that we have to reorder the dependency install with the "checkout" action, so that we actually have the "ci" scripts available. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-26 20:22:27 +02:00
- name: Install other dependencies
run: ci/install-dependencies.sh
ci: run `make sparse` as part of the GitHub workflow Occasionally we receive reviews after patches were integrated, where `sparse` (https://sparse.docs.kernel.org/en/latest/ has more information on that project) identified problems such as file-local variables or functions being declared as global. By running `sparse` as part of our Continuous Integration, we can catch such things much earlier. Even better: developers who activated GitHub Actions on their forks can catch such issues before even sending their patches to the Git mailing list. This addresses https://github.com/gitgitgadget/git/issues/345 Note: Not even Ubuntu 20.04 ships with a new enough version of `sparse` to accommodate Git's needs. The symptom looks like this: add-interactive.c:537:51: error: Using plain integer as NULL pointer To counter that, we download and install the custom-built `sparse` package from the Azure Pipeline that we specifically created to address this issue. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-14 13:50:33 +02:00
- run: make sparse
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
documentation:
ci: allow per-branch config for GitHub Actions Depending on the workflows of individual developers, it can either be convenient or annoying that our GitHub Actions CI jobs are run on every branch. As an example of annoying: if you carry many half-finished work-in-progress branches and rebase them frequently against master, you'd get tons of failure reports that aren't interesting (not to mention the wasted CPU). This commit adds a new job which checks a special branch within the repository for CI config, and then runs a shell script it finds there to decide whether to skip the rest of the tests. The default will continue to run tests for all refs if that branch or script is missing. There have been a few alternatives discussed: One option is to carry information in the commit itself about whether it should be tested, either in the tree itself (changing the workflow YAML file) or in the commit message (a "[skip ci]" flag or similar). But these are frustrating and error-prone to use: - you have to manually apply them to each branch that you want to mark - it's easy for them to leak into other workflows, like emailing patches We could likewise try to get some information from the branch name. But that leads to debates about whether the default should be "off" or "on", and overriding still ends up somewhat awkward. If we default to "on", you have to remember to name your branches appropriately to skip CI. And if "off", you end up having to contort your branch names or duplicate your pushes with an extra refspec. By comparison, this commit's solution lets you specify your config once and forget about it, and all of the data is off in its own ref, where it can be changed by individual forks without touching the main tree. There were a few design decisions that came out of on-list discussion. I'll summarize here: - we could use GitHub's API to retrieve the config ref, rather than a real checkout (and then just operate on it via some javascript). We still have to spin up a VM and contact GitHub over the network from it either way, so it ends up not being much faster. I opted to go with shell to keep things similar to our other tools (and really could implement allow-refs in any language you want). This also makes it easy to test your script locally, and to modify it within the context of a normal git.git tree. - we could keep the well-known refname out of refs/heads/ to avoid cluttering the branch namespace. But that makes it awkward to manipulate. By contrast, you can just "git checkout ci-config" to make changes. - we could assume the ci-config ref has nothing in it except config (i.e., a branch unrelated to the rest of git.git). But dealing with orphan branches is awkward. Instead, we'll do our best to efficiently check out only the ci/config directory using a shallow partial clone, which allows your ci-config branch to be just a normal branch, with your config changes on top. - we could provide a simpler interface, like a static list of ref patterns. But we can't get out of spinning up a whole VM anyway, so we might as well use that feature to make the config as flexible as possible. If we add more config, we should be able to reuse our partial-clone to set more outputs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 18:20:11 +02:00
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
env:
jobname: Documentation
runs-on: ubuntu-latest
steps:
ci: accelerate the checkout By upgrading from v1 to v2 of `actions/checkout`, we avoid fetching all the tags and the complete history: v2 only fetches one revision by default. This should make things a lot faster. Note that `actions/checkout@v2` seems to be incompatible with running in containers: https://github.com/actions/checkout/issues/151. Therefore, we stick with v1 there. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-05 00:55:14 +02:00
- uses: actions/checkout@v2
ci: configure GitHub Actions for CI/PR This patch adds CI builds via GitHub Actions. While the underlying technology is at least _very_ similar to that of Azure Pipelines, GitHub Actions are much easier to set up than Azure Pipelines: - no need to install a GitHub App, - no need to set up an Azure DevOps account, - all you need to do is push to your fork on GitHub. Therefore, it makes a lot of sense for us to have a working GitHub Actions setup. While copy/editing `azure-pipelines.yml` into `.github/workflows/main.yml`, we also use the opportunity to accelerate the step that sets up a minimal subset of Git for Windows' SDK in the Windows-build job: - we now download a `.tar.xz` stored in Azure Blobs and extract it simultaneously by calling `curl` and piping the result to `tar`, - decompressing via `xz`, - all three utilities are installed together with Git for Windows At the same time, we also make use of the matrix build feature, which reduces the amount of repeated text by quite a bit. Also, we do away with the parts that try to mount a file share on which `prove` can store data between runs. It is just too complicated to set up, and most times the tree changes anyway, so there is little return on investment there. Initial-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 19:18:09 +02:00
- run: ci/install-dependencies.sh
- run: ci/test-documentation.sh
Reference in New Issue Copy Permalink