As part of its importing process, git-p4 sends a `checkpoint` followed
immediately by `progress` to fast-import to force synchronization.
Due to buffering, it is possible for the `progress` command to not be
flushed before git-p4 begins to wait for the corresponding response.
This causes the script to freeze completely, and is consistently
observable at least on python-3.6.9.
Make sure this command sequence is completely flushed before waiting.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Reviewed-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
p4 does not appear to understand marshal format version 3 and above.
Version 2 was the latest supported by python-2.7.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Reviewed-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Opening .gitp4-usercache.txt in text mode makes python 3 happy without
explicitly adding encoding and decoding.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Reviewed-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
P4 allows essentially arbitrary encoding for path data while we would
perfer to be dealing only with unicode strings. Since path data need to
survive round-trip back to p4, this patch implements the general policy
that we store path data as-is, but decode them to unicode before doing
any non-trivial processing.
A new `decode_path()` method is provided that generally does the correct
conversion, taking into account `git-p4.pathEncoding` configuration.
For python2.7, path strings will be left as-is if it only contains ASCII
characters.
For python3, decoding is always done so that we have str objects.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Reviewed-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Under python3, calls to write() on the stream to `git fast-import` must
be encoded. This patch wraps the IO object such that this encoding is
done transparently.
Conversely, any text data read from subprocesses must also be decoded
before running through the rest of the pipeline.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Reviewed-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The marshalled dict in the response given on STDOUT by p4 uses `str` for
keys and string values. When run using python3, these values are
deserialized as `bytes`, leading to a whole host of problems as the rest
of the code assumes `str` is used throughout.
This patch changes the deserialization behaviour such that, as much as
possible, text output from p4 is decoded to native unicode strings.
Exceptions are made for the field `data` as it is usually arbitrary
binary data. `depotFile[0-9]*`, `path`, and `clientFile` are also exempt
as they contain path strings not encoded with UTF-8, and must survive
round-trip back to p4.
Conversely, text data being piped to p4 must always be encoded when
running under python3.
encode_text_stream() and decode_text_stream() were added to make these
transformations more convenient.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that python2.7 is the minimum required version and we no longer use
the basestring type, it is not necessary to use type aliasing to ensure
python3 compatibility.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python 3 handles strings differently than Python 2.7. Since Python 2
is reaching it's end of life, a series of changes are being submitted to
enable python 3.5 and following support. The current code fails basic
tests under python 3.5.
Some codepaths can represent a command line the program
internally prepares to execute either as a single string
(i.e. each token properly quoted, concatenated with $IFS) or
as a list of argv[] elements, and there are 9 places where
we say "if X is isinstance(_, basestring), then do this
thing to handle X as a command line in a single string; if
not, X is a command line in a list form".
This does not work well with Python 3, as there is no
basestring (everything is Unicode now), and even with Python
2, it was not an ideal way to tell the two cases apart,
because an internally formed command line could have been in
a single Unicode string.
Flip the check to say "if X is not a list, then handle X as
a command line in a single string; otherwise treat it as a
command line in a list form".
This will get rid of references to 'basestring', to migrate
the code ready for Python 3.
Thanks-to: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python2.6 and earlier have been end-of-life'd for many years now, and
we actually already use 2.7-only features in the code. Make the version
check reflect current realities.
This also removes the need to explicitly define CalledProcessError if
it's not available.
Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Miscellaneous small UX improvements on "git-p4".
* bk/p4-misc-usability:
git-p4: show detailed help when parsing options fail
git-p4: yes/no prompts should sanitize user text
When a user provides invalid parameters to git-p4, the program
reports the failure but does not provide the correct command syntax.
Add an exception handler to the command-line argument parser to display
the command's specific command line parameter syntax when an exception
is thrown. Rethrow the exception so the current behavior is retained.
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When prompting the user interactively for direction, the tests are
not forgiving of user input format.
For example, the first query asks for a yes/no response. If the user
enters the full word "yes" or "no" or enters a capital "Y" the test
will fail.
Create a new function, prompt(prompt_text) where
* prompt_text is the text prompt for the user
* returns a single character where valid return values are
found by inspecting prompt_text for single characters
surrounded by square brackets
This new function must prompt the user for input and sanitize it by
converting the response to a lower case string, trimming leading and
trailing spaces, and checking if the first character is in the list
of choices. If it is, return the first letter.
Change the current references to raw_input() to use this new function.
Since the method requires the returned text to be one of the available
choices, remove the loop from the calling code that handles response
verification.
Thanks-to: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Ben Keene <seraphire@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git lfs" allows users to specify the custom storage location with
the configuration variable `lfs.storage`, but when interacting with
GitLFS pointers, "git p4" always uses the hardcoded default that is
the `.git/lfs/` directory, without paying attention to the
configuration.
Use the value configured in `lfs.storage`, if exists, as all the
"git" operations do, for consistency.
Signed-off-by: r.burenkov <panzercheg@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Avoid double-open exceptions on Windows platform when
calculating for lfs compressed size threshold
(git-p4.largeFileCompressedThreshold) comparisons.
Take new approach using the NamedTemporaryFile()
file-like object as input to the ZipFile() which
auto-deletes after implicit close leaving with scope.
Original code had double-open exception on Windows
platform because file still open from NamedTemporaryFile()
using generated filename instead of object.
Thanks to Andrey for patiently suggesting several
iterations on this change for avoiding exceptions!
Also print error details after resulting IOError to make
debugging cause of exception less mysterious when it has
nothing to do with "git version recent enough."
Signed-off-by: Philip.McGraw <Philip.McGraw@bentley.com>
Reviewed-by: Andrey Mazo <ahippo@yandex.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" update.
* am/p4-branches-excludes:
git-p4: respect excluded paths when detecting branches
git-p4: add failing test for "git-p4: respect excluded paths when detecting branches"
git-p4: don't exclude other files with same prefix
git-p4: add failing test for "don't exclude other files with same prefix"
git-p4: don't groom exclude path list on every commit
git-p4: match branches case insensitively if configured
git-p4: add failing test for "git-p4: match branches case insensitively if configured"
git-p4: detect/prevent infinite loop in gitCommitByP4Change()
The command line to invoke a "git cat-file" command from inside
"git p4" was not properly quoted to protect a caret and running a
broken command on Windows, which has been corrected.
* mm/p4-unshelve-windows-fix:
p4 unshelve: fix "Not a valid object name HEAD0" on Windows
git p4 unshelve was failing with these errors:
fatal: Not a valid object name HEAD0
Command failed: git cat-file commit HEAD^0
(git version 2.21.0.windows.1, python 2.7.16)
The pOpen call used by git-p4 to invoke the git command can take either a
string or an array as a first argument. The array form is preferred
because platform-specific escaping of special characters will be
handled automatically.(https://docs.python.org/2/library/subprocess.html)
The extractLogMessageFromGitCommit method was, however, using the string
form and so the caret (^) character in the HEAD^0 argument was not being
escaped on Windows. The caret happens to be the escape character, which
is why the git command was receiving HEAD0.
The behaviour can be confirmed by typing ECHO HEAD^0 at the command-
prompt, which emits HEAD0.
The solution is simply to use the array format of passing the command to
fOpen, which is recommended and used in other parts of this code anyway.
Signed-off-by: Mike Mueller <mike.mueller@moodys.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When unshelving a changelist, git-p4 tries to work out the appropriate
parent commit in a given branch (default: HEAD). To do this, it looks
at the state of any pre-existing files in the target Perforce branch,
omitting files added in the shelved changelist. Currently, only files
added (or move targets) are classed as new. However, files integrated
from other branches (i.e. a 'branch' action) also need to be considered
as added, for this purpose.
Signed-off-by: Simon Williams <simon@no-dns-yet.org.uk>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, excluded paths are only handled in the following cases:
* no branch detection;
* branch detection with using clientspec.
However, excluded paths are not respected in case of
branch detection without using clientspec.
Fix this by consulting the list of excluded paths
when splitting files across branches.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure not to exclude files unintentionally
if exclude paths are specified without a trailing /.
I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file".
Do this by ensuring that paths without a trailing "/" are only matched completely.
Also, abort path search on the first match as a micro-optimization.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, `cloneExclude` array is being groomed (by removing trailing "...")
on every changeset.
(since `extractFilesFromCommit()` is called on every imported changeset)
As a micro-optimization, do it once while parsing arguments.
Also, prepend "/" and remove trailing "..." at the same time.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 knows how to handle case insensitivity in file paths
if core.ignorecase is set.
However, when determining a branch for a file,
it still does a case-sensitive prefix match.
This may result in some file changes to be lost on import.
For example, given the following commits
1. add //depot/main/file1
2. add //depot/DirA/file2
3. add //depot/dira/file3
4. add //depot/DirA/file4
and "branchList = main:DirA" branch mapping,
commit 3 will be lost.
So, do branch search case insensitively if running with core.ignorecase set.
Teach splitFilesIntoBranches() to use the p4PathStartsWith() function
for path prefix matches instead of always case-sensitive match.
Signed-off-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" failed to update a shelved change when there were moved
files, which has been corrected.
* ld/git-p4-shelve-update-fix:
git-p4: handle update of moved/copied files when updating a shelve
git-p4: add failing test for shelved CL update involving move/copy
Perforce requires a complete list of files being operated on. If
git is updating an existing shelved changelist, then any files
which are moved or copied were not being added to this list.
Signed-off-by: Luke Diamand <luke@diamand.org>
Acked-by: Andrey Mazo <amazo@checkvideo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a perforce setup where login is not required, communication fails
because p4_check_access does not understand the response from the p4
client. Fixed by detecting and ignoring the "info" response.
Signed-off-by: Peter Osterlund <peterosterlund2@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous git-p4 unshelve support would check for changes
in Perforce to the files being unshelved since the original
shelve, and would complain if any were found.
This was to ensure that the user wouldn't end up with both the
shelved change delta, and some deltas from other changes in their
git commit.
e.g. given fileA:
the
quick
brown
fox
change1: s/the/The/ <- p4 shelve this change
change2: s/fox/Fox/ <- p4 submit this change
git p4 unshelve 1 <- FAIL
This change teaches the P4Unshelve class to always create a parent
commit which matches the P4 tree (for the files being unshelved) at
the point prior to the P4 shelve being created (which is reported
in the p4 description for a shelved changelist).
That then means git-p4 can always create a git commit matching the
P4 shelve that was originally created, without any extra deltas.
The user might still need to use the --origin option though - there
is no way for git-p4 to work out the versions of all of the other
*unchanged* files in the shelve, since this information is not recorded
by Perforce.
Additionally this fixes handling of shelved 'move' operations.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The branch detection code looks for branches under refs/remotes/p4/...
and can end up getting confused if there are unshelved changes in
there as well. This happens in the function p4BranchesInGit().
Instead, put the unshelved changes into refs/remotes/p4-unshelved/<N>.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If deleting or moving a file, sometimes P4 doesn't report the file size.
The code handles this just fine but some logging crashes. Stop this
happening.
There was some earlier discussion on the list about this:
https://public-inbox.org/git/xmqq1sqpp1vv.fsf@gitster.mtv.corp.google.com/
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `p4-pre-submit` hook is executed before git-p4 submits code.
If the hook exits with non-zero value, submit process not start.
Signed-off-by: Chen Bin <chenbin.sh@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace calls to print ... with the function form, print(...), to
allow use with python3 as well as python2.x.
Converted using 2to3 (and some hand-editing).
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Python3, basestring no longer exists, so use this workaround.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Backticks around a variable are a deprecated alias for repr().
This has been removed in python3, so just use the string
representation instead, which is equivalent.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Python3 does not have the dict.has_key() function, so replace all
such calls with "k in dict". This will still work with python2.6
and python2.7.
Converted using 2to3 (plus some hand-editing)
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The <> string inequality operator (which doesn't seem to be even
documented) no longer exists in python3. Replace with !=.
This still works with python2.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" updates.
* ld/git-p4-updates:
git-p4: auto-size the block
git-p4: narrow the scope of exceptions caught when parsing an int
git-p4: raise exceptions from p4CmdList based on error from p4 server
git-p4: better error reporting when p4 fails
git-p4: add option to disable syncing of p4/master with p4
git-p4: disable-rebase: allow setting this via configuration
git-p4: add options --commit and --disable-rebase
git-p4 originally would fetch changes in one query. On large repos this
could fail because of the limits that Perforce imposes on the number of
items returned and the number of queries in the database.
To fix this, git-p4 learned to query changes in blocks of 512 changes,
However, this can be very slow - if you have a few million changes,
with each chunk taking about a second, it can be an hour or so.
Although it's possible to tune this value manually with the
"--changes-block-size" option, it's far from obvious to ordinary users
that this is what needs doing.
This change alters the block size dynamically by looking for the
specific error messages returned from the Perforce server, and reducing
the block size if the error is seen, either to the limit reported by the
server, or to half the current block size.
That means we can start out with a very large block size, and then let
it automatically drop down to a value that works without error, while
still failing correctly if some other error occurs.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current code traps all exceptions around some code which parses an
integer, and then talks to Perforce.
That can result in errors from Perforce being ignored. Change the code
to only catch the integer conversion exceptions.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This change lays some groundwork for better handling of rowcount errors
from the server, where it fails to send us results because we requested
too many.
It adds an option to p4CmdList() to return errors as a Python exception.
The exceptions are derived from P4Exception (something went wrong),
P4ServerException (the server sent us an error code) and
P4RequestSizeException (we requested too many rows/results from the
server database).
This makes the code that handles the errors a bit easier.
The default behavior is unchanged; the new code is enabled with a flag.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently when p4 fails to run, git-p4 just crashes with an obscure
error message.
For example, if the P4 ticket has expired, you get:
Error: Cannot locate perforce checkout of <path> in client view
This change checks whether git-p4 can talk to the Perforce server when
the first P4 operation is attempted, and tries to print a meaningful
error message if it fails.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add an option to the git-p4 submit command to disable syncing
with Perforce.
This is useful for the case where a git-p4 mirror has been setup
on a server somewhere, running from (e.g.) cron, and developers
then clone from this. Having the local cloned copy also sync
from Perforce just isn't useful.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This just lets you set the --disable-rebase option with the
git configuration options git-p4.disableRebase. If you're
using this option, you probably want to set it all the time
for a given repo.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On a daily work with multiple local git branches, the usual way to
submit only a specified commit was to cherry-pick the commit on
master then run git-p4 submit. It can be very annoying to switch
between local branches and master, only to submit one commit. The
proposed new way is to select directly the commit you want to
submit.
Add option --commit to command 'git-p4 submit' in order to submit
only specified commit(s) in p4.
On a daily work developping software with big compilation time, one
may not want to rebase on his local git tree, in order to avoid long
recompilation.
Add option --disable-rebase to command 'git-p4 submit' in order to
disable rebase after submission.
Thanks-to: Cedric Borgese <cedric.borgese@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Romain Merland <merlorom@yahoo.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This can be used to "unshelve" a shelved P4 commit into
a git commit.
For example:
$ git p4 unshelve 12345
The resulting commit ends up in the branch:
refs/remotes/p4/unshelved/12345
If that branch already exists, it is renamed - for example
the above branch would be saved as p4/unshelved/12345.1.
git-p4 checks that the shelved changelist is based on files
which are at the same Perforce revision as the origin branch
being used for the unshelve (HEAD by default). If they are not,
it will refuse to unshelve. This is to ensure that the unshelved
change does not contain other changes mixed-in.
The reference branch can be changed manually with the "--origin"
option.
The change adds a new Unshelve command class. This just runs the
existing P4Sync code tweaked to handle a shelved changelist.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This was the only occurrence of "commitish" in the tree, but as the
log will reveal we've had others in the past. Fixes up code added in
00ad6e3182 ("git-p4: work with a detached head", 2015-11-21).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
--update-shelve can now be specified multiple times on the
command-line, to update multiple shelved changelists in a single
submit.
This then means that a git patch series can be mirrored to a
sequence of shelved changelists, and (relatively easily) kept in
sync as changes are made in git.
Note that Perforce does not really support overlapping shelved
changelists where one change touches the files modified by
another. Trying to do this will result in merge conflicts.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Follow the Oxford style, which says to use "up-to-date" before the noun,
but "up to date" after it. Don't change plumbing (specifically
send-pack.c, but transport.c (git push) also has the same string).
This was produced by grepping for "up-to-date" and "up to date". It
turned out we only had to edit in one direction, removing the hyphens.
Fix a typo in Documentation/git-diff-index.txt while we're there.
Reported-by: Jeffrey Manian <jeffrey.manian@gmail.com>
Reported-by: STEVEN WHITE <stevencharleswhitevoices@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function p4CmdList accepts a new argument: skip_info. When set to
True it ignores any 'code':'info' entry (skip_info=False by default).
That allows us to fix some of the tests in t9831-git-p4-triggers.sh
known to be broken with verobse p4 triggers
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The option -G of p4 (python marshal output) gives more context about the
data being output. That's useful when using the command "change -o" as
we can distinguish between warning/error line and real change description.
This fixes the case where a p4 trigger for "p4 change" is set and the command git-p4 submit is run.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4 was using "git name-rev" to find out the current branch.
That is not safe, since if multiple branches or tags point at
the same revision, the result obtained might not be what is
expected.
Instead use "git symbolic-ref".
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The existing read_pipe() function returns an empty string on
error, but also returns an empty string if the command returns
an empty string.
This leads to ugly constructions trying to detect error cases.
Add read_pipe_text() which just returns None on error.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "git p4" imports changelist that removes paths, it failed to
convert pathnames when the p4 used encoding different from the one
used on the Git side. This has been corrected.
* ls/p4-path-encoding:
git-p4: fix git-p4.pathEncoding for removed files
In a9e38359e3 we taught git-p4 a way to re-encode path names from what
was used in Perforce to UTF-8. This path re-encoding worked properly for
"added" paths. "Removed" paths were not re-encoded and therefore
different from the "added" paths. Consequently, these files were not
removed in a git-p4 cloned Git repository because the path names did not
match.
Fix this by moving the re-encoding to a place that affects "added" and
"removed" paths. Add a test to demonstrate the issue.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running git-p4 on Windows, with multiple git-p4.mapUser entries in
git config - no user mappings are applied to the generated repository.
Reproduction Steps:
1. Add multiple git-p4.mapUser entries to git config on a Windows
machine
2. Attempt to clone a p4 repository
None of the user mappings will be applied.
This issue is actually caused by gitConfigList, using split(os.linesep)
to convert the output of git config --get-all into a list. On Windows,
os.linesep is equal to '\r\n' - however git.exe returns configuration
with a line seperator of '\n'.
This leads to the list returned by gitConfigList containing only one
element - which contains the full output of git config --get-all in
string form, which causes problems for the code introduced to
getUserMapFromPerforceServer in 10d08a149d ("git-p4: map a P4 user to
Git author name and email address", 2016-03-01)
This issue should be caught by the test introduced in 10d08a1, however
would require running on Windows to reproduce.
Using splitlines solves this issue, by splitting config on all
typical delimiters ('\n', '\r\n' etc.)
Signed-off-by: George Vanburgh <gvanburgh@bloomberg.net>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A recent updates to "git p4" was not usable for older p4 but it
could be made to work with minimum changes. Do so.
* ls/p4-retry-thrice:
git-p4: do not pass '-r 0' to p4 commands
"git p4" that tracks multile p4 paths imported a single changelist
that touches files in these multiple paths as one commit, followed
by many empty commits. This has been fixed.
* gv/p4-multi-path-commit-fix:
git-p4: fix multi-path changelist empty commits
git-p4 crashes when used with a very old p4 client version
that does not support the '-r <number>' option in its commands.
Allow making git-p4 work with old p4 clients by setting git-p4.retries to 0.
Alternatively git-p4.retries could be made opt-in.
But since only very old, barely maintained p4 versions don't support
the '-r' option, the setting-retries-to-0 workaround would do.
The "-r retries" option is present in Perforce 2012.2 Command Reference,
but absent from Perforce 2012.1 Command Reference.
Signed-off-by: Igor Kushnir <igorkuo@gmail.com>
Acked-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" that tracks multile p4 paths imported a single changelist
that touches files in these multiple paths as one commit, followed
by many empty commits. This has been fixed.
* gv/p4-multi-path-commit-fix:
git-p4: fix multi-path changelist empty commits
The `git lfs track` command generates a .gitattributes file with diff
and merge properties [1]. Set the same .gitattributes format for files
tracked with GitLFS in git-p4.
[1] https://github.com/git-lfs/git-lfs/blob/v1.5.3/commands/command_track.go#L121
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" didn't interact with the internal of .git directory
correctly in the modern "git-worktree"-enabled world.
* ld/p4-worktree:
git-p4: support git worktrees
When importing from multiple perforce paths - we may attempt to
import a changelist that contains files from two (or more) of these
depot paths. Currently, this results in multiple git commits - one
containing the changes, and the other(s) as empty commit(s). This
behavior was introduced in commit 1f90a64891 ("git-p4: reduce number
of server queries for fetches", 2015-12-19).
Reproduction Steps:
1. Have a git repo cloned from a perforce repo using multiple
depot paths (e.g. //depot/foo and //depot/bar).
2. Submit a single change to the perforce repo that makes changes
in both //depot/foo and //depot/bar.
3. Run "git p4 sync" to sync the change from #2.
Change is synced as multiple commits, one for each depot path that
was affected.
Using a set, instead of a list inside p4ChangesForPaths() ensures
that each changelist is unique to the returned list, and therefore
only a single commit is generated for each changelist.
Reported-by: James Farwell <jfarwell@vmware.com>
Signed-off-by: George Vanburgh <gvanburgh@bloomberg.net>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When submitting to P4, if git-p4 came across a symlinked
directory, then during the generation of the submit diff, it would
try to open it as a normal file and fail.
Spot symlinks (of any type) and output a description of the symlink
instead.
Add a test case.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" LFS support was broken when LFS stores an empty blob.
* ls/p4-empty-file-on-lfs:
git-p4: fix empty file processing for large file system backend GitLFS
git-p4 would attempt to find the git directory using
its own specific code, which did not know about git
worktrees.
Rework it to use "git rev-parse --git-dir" instead.
Add test cases for worktree usage and specifying
git directory via --git-dir and $GIT_DIR.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If git-p4 tried to store an empty file in GitLFS then it crashed while
parsing the pointer file:
oid = re.search(r'^oid \w+:(\w+)', pointerFile, re.MULTILINE).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
This happens because GitLFS does not create a pointer file for an empty
file. Teach git-p4 this behavior to fix the problem and add a test case.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
P4 commands can fail due to random network issues. P4 users can counter
these issues by using a retry flag supported by all p4 commands [1].
Add an integer Git config value `git-p4.retries` to define the number of
retries for all p4 invocations. If the config is not defined then set
the default retry count to 3.
[1] https://www.perforce.com/perforce/doc.current/manuals/cmdref/global.options.html
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adds new option "--update-shelve CHANGELIST" which updates
an existing shelved changelist.
The original changelist must have been created by the current user.
This allows workflow something like:
hack hack hack
git commit
git p4 submit --shelve
$mail interested parties about shelved changelist
make corrections
git commit --amend
git p4 submit --update-shelve $CHANGELIST
$mail interested parties about shelved changelist
etc
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a --shelve command line argument which invokes p4 shelve instead
of submitting changes. After shelving the changes are reverted from the
p4 workspace.
Signed-off-by: Vinicius Kursancew <viniciusalexandre@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" used a location outside $GIT_DIR/refs/ to place its
temporary branches, which has been moved to refs/git-p4-tmp/.
* ls/p4-tmp-refs:
git-p4: place temporary refs used for branch import under refs/git-p4-tmp
A bug, which caused "git p4" while running under verbose mode to
report paths that are omitted due to branch prefix incorrectly, has
been fixed; the command said "Ignoring file outside of prefix" for
paths that are _inside_.
* ao/p4-has-branch-prefix-fix:
git-p4: correct hasBranchPrefix verbose output
Git-P4 used to place temporary refs under "git-p4-tmp". Since 3da1f37
Git checks that all refs are placed under "refs". Instruct Git-P4 to
place temporary refs under "refs/git-p4-tmp". There are no backwards
compatibility considerations as these refs are transient.
Use "git show-ref --verify" to check the (non-)existience of the refs
instead of file checks assuming the file-based ref backend.
All refs under "refs" are shared across all worktrees. This is not
desired for temporary Git-P4 refs and will be adressed in a later patch.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Vitor Antunes <vitor.hda@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A bug, which caused "git p4" while running under verbose mode to
report paths that are omitted due to branch prefix incorrectly, has
been fixed; the command said "Ignoring file outside of prefix" for
paths that are _inside_.
* ao/p4-has-branch-prefix-fix:
git-p4: correct hasBranchPrefix verbose output
The logic here was inverted, you got a message saying the file is
ignored for each file that is not ignored.
Signed-off-by: Andrew Oakley <aoakley@roku.com>
Acked-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recent update to Git LFS broke "git p4" by changing the output from
its "lfs pointer" subcommand.
* ls/p4-lfs:
git-p4: fix Git LFS pointer parsing
travis-ci: express Linux/OS X dependency versions more clearly
travis-ci: update Git-LFS and P4 to the latest version
Git LFS 1.2.0 removed a preamble from the output of the 'git lfs pointer'
command [1] which broke the parsing of this output. Adjust the parser
to support the old and the new format.
Please note that this patch slightly changes the second return parameter
from a list of LF terminated strings to a single string that contains
a number of LF characters.
[1] da2935d9a7
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Sebastian Schuberth <sschuberth@gmail.com>
Helped-by: Ben Woosley <ben.woosley@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When migrating from Perforce to git the information about P4 jobs
associated with P4 changelists is lost.
Having these jobs listed on messages of related git commits enables smooth
migration for projects that take advantage of e.g. JIRA integration
(which uses jobs on Perforce side and parses commit messages on git side).
The jobs are added to the message in the same format as is expected when
migrating in the reverse direction.
Signed-off-by: Jan Durovec <jan.durovec@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Map a P4 user to a specific name and email address in Git with the
"git-p4.mapUser" config. The config value must be a string adhering
to the format "p4user = First Lastname <email@address.com>".
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After changing the type of a file in the git repository, it is not possible to
"git p4 publish" the commit to perforce. This is due to the fact that the git
"T" status is not handled in git-p4.py. This can typically occur when replacing
an existing file with a symbolic link.
The "T" modifier is now supported in git-p4.py. When a file type has changed,
inform perforce with the "p4 edit -f auto" command.
Signed-off-by: Romain Picard <romain.picard@oakbits.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" when interacting with multiple depots at the same time
used to incorrectly drop changes.
* sh/p4-multi-depot:
git-p4: reduce number of server queries for fetches
git-p4: support multiple depot paths in p4 submit
git-p4: failing test case for skipping changes with multiple depots
When fetching changes from a depot using a full client spec, there
is no need to perform as many queries as there are top-level paths
in the client spec. Instead we query all changes in chronological
order, also getting rid of the need to sort the results and remove
duplicates.
Signed-off-by: Sam Hocevar <sam@hocevar.net>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When submitting from a repository that was cloned using a client spec,
use the full list of paths when ruling out files that are outside the
view. This fixes a bug where only files pertaining to the first path
would be included in the p4 submit.
Signed-off-by: Sam Hocevar <sam@hocevar.net>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" used to import Perforce CLs that touch only paths outside
the client spec as empty commits. It has been corrected to ignore
them instead, with a new configuration git-p4.keepEmptyCommits as a
backward compatibility knob.
* ls/p4-keep-empty-commits:
git-p4: add option to keep empty commits
A changelist that contains only excluded files due to a client spec was
imported as an empty commit. Fix that issue by ignoring these commits.
Add option "git-p4.keepEmptyCommits" to make the previous behavior
available.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Helped-by: Pete Harlan
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
http: treat config options sslCAPath and sslCAInfo as paths
Documentation/diff: give --word-diff-regex=. example
filter-branch: deal with object name vs. pathname ambiguity in tree-filter
check-ignore: correct documentation about output
git-p4: clean up after p4 submit failure
git-p4: work with a detached head
git-p4: add option to system() to return subshell status
git-p4: add failing test for submit from detached head
remote-http(s): support SOCKS proxies
t5813: avoid creating urls that break on cygwin
Escape Git's exec path in contrib/rerere-train.sh script
allow hooks to ignore their standard input stream
rebase-i-exec: Allow space in SHELL_PATH
Documentation: make environment variable formatting more consistent
Just like the working tree is cleaned up when the user cancelled
submission in P4Submit.applyCommit(), clean up the mess if "p4
submit" fails.
* eg/p4-submit-catch-failure:
git-p4: clean up after p4 submit failure
Make git-p4 work on a detached head.
* ld/p4-detached-head:
git-p4: work with a detached head
git-p4: add option to system() to return subshell status
git-p4: add failing test for submit from detached head
When "p4 submit" command fails in P4Submit.applyCommit, the
workspace is left with the changes. We already have code to revert
the changes to the workspace when the user decides to cancel
submission by aborting the editor that edits the change description,
and we should treat the "p4 submit" failure the same way.
Clean the workspace if p4_write_pipe raised SystemExit, so that the
user don't have to do it themselves.
Signed-off-by: GIRARD Etienne <egirard@murex.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
When submitting, git-p4 finds the current branch in
order to know if it is allowed to submit (configuration
"git-p4.allowSubmit").
On a detached head, detecting the branch would fail, and
git-p4 would report a cryptic error.
This change teaches git-p4 to recognise a detached head and
submit successfully.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
Add an optional parameter ignore_error to the git-p4 system()
function. If used, it will return the subshell exit status
rather than throwing an exception.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Jeff King <peff@peff.net>
Work around "git p4" failing when the P4 depot records the contents
in UTF-16 without UTF-16 BOM.
* ls/p4-translation-failure:
git-p4: handle "Translation of file content failed"
git-p4: add test case for "Translation of file content failed" error
Correct "git p4 --detect-labels" so that it does not fail to create
a tag that points at a commit that is also being imported.
* ld/p4-import-labels:
git-p4: fix P4 label import for unprocessed commits
git-p4: do not terminate creating tag for unknown commit
git-p4: failing test for ignoring invalid p4 labels
