This was originally needed before we used the delta fetcher and
had a less-clean follow-parent implementation that could leave
holes in the history.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
It looks better (like [remote "origin"]) instead of whatever
refname came up first in our directory traversal. Of course
--remote= overrides this.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Having 'fetch' entries in the config file created from
--follow-parent is wasteful because it can cause *future* of
invocations to follow revisions we were never interested in
in the first place.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Using buffered IO for reading 40-41 bytes at a time isn't very
efficient. Buffering writes for a short duration is alright
since we close() right away and buffers will be flushed.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Prefill .rev_db to the maximum revision we tried to fetch;
and take advantage of that so we can avoid using get_log()
on ranges we've already seen (and have deemed uninteresting).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Defer any signals that cause termination while they are
updating; and put the update-ref call as close to the rename()
as possible. Also, make things extra-safe (but slower) for
people using --no-metadata since they can't rely on .rev_db
being rebuilt if it's clobbered (well, I'm calling update-ref
with the -m flag for reflogs, we don't yet have a way to rebuild
.rev_db from reflogs.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Passing very large strings as arguments is bad for memory usage
as it never seems to get freed in Perl. The .rev_db format is
already not optimized for projects with sparse history.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
get_log with explicit paths is the safest way to get revisions
that change a particular path we're interested in.
Unfortunately that means we still have to run get_log multiple
times for each path we're interested in, and even more if
a path gets deleted.
The first argument of get_log() is an array reference, but we
shouldn't use more than one element in that array ref because
the non-existence of _one_ of those paths for a particular range
would cause an error for all paths in that range, so yes, we
need multiple get_log calls to be on the safe side...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
--svn-remote allows the default remote name to be overridden (useful
for tracking multiple SVN repositories).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
We no longer delete the top-level directory even if it got
deleted from the upstream repository. In gs_do_update; we
double-check that the path we're tracking exists at both
endpoints before proceeding. We have also added additional
protection against fetching revisions out-of-order.
To simplify our internal interfaces, I've disabled passing the
'recursive' flag to the gs_do_{switch,update} wrapper functions
since we always want it in git-svn. We also pass the
entire Git::SVN object rather than just the path because it
helped me debug.
When printing progress, the refname is printed out to make
it less confusing when multi-fetch is running.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Since refs/remotes/* are not automatically cloned, we expect the
user to be capable of copying those references themselves
anyways.
Also removed the documentation for --ignore-nodate while we're
at it; it has also been made automatic.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
We were still skipping path information from get_log if we are
tracking /r9270/drunk/subversion/bindings/..., but got something
like this in the log:
A /r9270/drunk (from /r9270/trunk:14)
Signed-off-by: Eric Wong <normalperson@yhbt.net>
I can't seem to figure out what I or the SVN libraries are doing
wrong, but it appears to be related to reparent and probably
some global structure that gets reset if multiple SVN
connections are being used.
So now, in order to use do_switch; we'll open a new connection
to the repository with the complete URL; but we can't seem to
ever use an existing Ra object after another one has been
created...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
We don't need our own error handler for other operations. Also
add a message about the successfully do_switch or do_update in
follow-parent for debugging do_switch failures with svn:// and
svn+ssh:// connections.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Historically, git-svn did not always use Digest::MD5 because
it did not use the SVN::Delta::Editor interfaces. Nowadays
it does, and the requires make strace more noisy.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Using path names as refnames breaks horribly if a user is
tracking one large, toplevel directory, and a lower-level
directory is followed from another project is a parent
of another ref, as it will cause refnames such as:
'refs/remotes/trunk/path/to/stuff', which will conflict
with a refname of 'refs/remotes/trunk'.
Now we just append @$revno to the end of it the current
refname. And if we have followed back to a grandparent, then
we'll strip any existing '@$parent_revno' strings before
appending our own '@$revno' string to it.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
The do_update or do_switch functions in SVN only allow for a
single path component; so 'path/to/deep/dir' would be
interpreted as 'path'.
SVN 1.4.x has a reparent function that can let us change the
session to use a higher-level root of the repository, so we can
use that for do_switch (which still doesn't seem to work in SVN
1.4.3 (a fix was attempted, but they missed the rest of the
typemap changes needed in trunk...)).
On the do_update side, we can use set_path on higher level
directories and set them to a newer revision so they don't get
updated. We can't do this with do_switch, either, because the
relative path we're tracking can change (directory moving into
a child of itself).
Because of these changes, we need to double check that our Fetch
editor is correctly performing stripping on any prefixed paths
from update, otherwise we'll just die() because that would be
a bug.
Added a test case which helped me notice and fix problems with
do_switch, too.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Also, this should allow for the tracking of new, but empty
directories where we would want to see the log message.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Since single fetching is a special case of multi-fetch,
share code with it and the fetch loop into Git::SVN::Ra
since it uses a single Ra connection and multiple
Git::SVN objects.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Before, we needed a separate svn_ra instance to run
our check_path calls once the editor was active; but
we can avoid that by running all the check_path calls
before our editor is active.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
I broke this part with the URL minimization; since
git-svn will now try to connect to the root of
the repository and will end up writing files
there if it can...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
svn_log_changed_path_t structs were being used out of scope
outside of svn_ra_get_log (because I wanted to eventually be
able to use git-svn with only a single connection to the
repository). So now we dup them into a hash.
This was fixed while making --follow-parent fetches more
efficient. I've moved parsing of the command-line --revision
argument outside of the Git::SVN module so Git::SVN::fetch() can
be used in more places (such as find_parent_branch).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
git-svn has never been able to handle deleted branches very well
because svn_ra_get_log() is all-or-nothing, meaning that if the
max revision passed to it does not contain the path we're
tracking, we miss all the revisions in the repository.
Branches fetched using --follow-parent still do this
sub-optimally (will be fixed soon). --follow-parent will soon
become the default, so we will assume that when using get_log();
We will also avoid tracking revprops for revisions with no
path-related changes since otherwise we just end up pulling
logs to paths we don't care about.
Also added a test for this to t9104-git-svn-follow-parent.sh and
correctly commit the log message in the preceeding test (which
conflicted with a filename).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
They simply aren't interesting to track, and this will allow
us to avoid get_log().
Since r0 is covered by this, we need to update the tests to not
rely on r0 (which is always empty).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Introducing Git::IndexInfo. This module will probably be useful
outside of git-svn, so I'm not putting it in the Git::SVN
namespace.
This will allow me to more easily avoid the use of get_log() in
the future and simply run do_update in incrementing ranges.
get_log() should be avoided because there are cases where
moved/deleted directories do not track correctly (until
--follow-parent is run on a new branch).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This means that tracking the path of:
/another-larger/trunk/thunk/bump/thud inside a repository
would follow:
/larger-parent/trunk/thunk/bump/thud
even if the svn log output looks like this:
--------------------------------------------
Changed paths:
A /another-larger (from /larger-parent:5)
--------------------------------------------
Note: the usage of get_log() in git-svn still makes a
an assumption that shouldn't be made with regard to
revisions existing for a particular path.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This allows connections to be used more efficiently and not require
users to run 'git-svn migrate --minimize' for new repositories.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Tests always ran 'git init' before we ran so that repo-config
would always have something to read. However that does not work
in real-world situations where the user expects 'git svn init'
to work without running 'git init' first.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
It's not really useful anymore now that we have a better
--follow-parent for the valid cases. Any other use
of it is not valid.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Having multiple fetch refspecs pointing to the same local ref
would be a very bad thing. Start avoiding the use of fatal() or
exit() inside the modules so we can libify more easily.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Bugs fixed:
* We didn't allow manually (not using git-svn) init-ed
remotes/fetch refspecs to be used before. It works now
because that's what I did in this test. git-svn init should
offer more control in the future.
* correctly strip paths in the delta editor when using
do_switch().
* Make the -i / GIT_SVN_ID option work correctly when doing
fetch on a multi-ref svn-remote
Signed-off-by: Eric Wong <normalperson@yhbt.net>
--minimize will update the git-svn configuration to attempt to
connect to the repository root (instead of directly to the
path(s) we are tracking) in order to allow more efficient reuse
of connections (for multi-fetch and follow-parent).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Note: this can cause problems with Perl's reference counting GC,
so I'm disabling Git::SVN::Ra::DESTROY. If we notice more
problems down the line, we can disable this enhancement.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Of course, we handle metadata migrations from previous versions
and we have added unit tests.
The new .git/config remotes resemble non-SVN remotes. Below
is an example with comments:
[svn-remote "git-svn"]
; like non-svn remotes, we have one URL per-remote
url = http://foo.bar.org/svn
; 'fetch' keys are done in the same way as non-svn
; remotes, too. With the left-hand-side of the ':'
; being the remote (SVN) repository path relative to the
; above 'url' key; and the right-hand-side being a
; remote ref in git (refs/remotes/*).
; An empty left-hand-side means that it will fetch
; the entire contents of the 'url' key.
; old-style (migrated from previous versions of git-svn)
; are like this:
fetch = :refs/remotes/git-svn
; this is created by a current version of git-svn
; using the multi-init command with an explicit
; url (specified above). This allows multi-init
; to reuse SVN::Ra connections.
fetch = trunk:refs/remotes/trunk
fetch = branches/a:refs/remotes/a
fetch = branches/b:refs/remotes/b
fetch = tags/0.1:refs/remotes/tags/0.1
fetch = tags/0.2:refs/remotes/tags/0.2
fetch = tags/0.3:refs/remotes/tags/0.3
[svn-remote "alt"]
; this is another old-style remote migrated over
; to the new config format
url = http://foo.bar.org/alt
fetch = :refs/remotes/alt
Signed-off-by: Eric Wong <normalperson@yhbt.net>
It's becoming a maintenance burden. I've never found it
particularly useful myself, nor have I heard much feedback about
it; so I'm assuming it's just as useless to everyone else.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Also, convert all usage of 'log_msg' to 'log_entry' for
consistency's sake
SVN::Git::Editor::apply_diff now drives the rest of the
editor.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
More cleanup to separate out functionality and make things
nicer to hack on.
While we're at it, centralize loading of the authors into
one place and correctly handle '(no author)' cases in
when showing logs after-the-fact; and not just at commit
time.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
If a user specified a seperate URL and --tags/--branches as
a sepearte URL, allow the Ra object (and therefore the connection)
to be reused.
We'll get rid of libsvn_ls_fullurl() since it was only used
in one place.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
I will be using functions from Git.pm in more modules, so I
want to avoid re-importing the long argument list everywhere
it's used.
Also removed an unused command-line switch
(--no-ignore-externals) and some variables.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Force some svn_ra functions to use a temporary pool via wrapper
This cleans up the code a bit by removing explicit instances of
pool allocation and deallocation and providing wrapper functions
that make use of temporary pools.
I've also added an explicit pool usage when creating the commit
editor for commit-diff where get_commit_editor can be called
multiple times with the same pool previously.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
We don't append a space after '--bool', so we can't expect
a regular expression to match the space.
Semi-noticed by Junio C Hamano :)
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This mainly quiets down warnings when running git svn log.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Not sure why it was there in the first place, we always do our
work relative to the URL we're connected to; even if that URL is
the root of the repository, so the leading slash is pointless...
Lets be consistent when printing things for the user to see.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
People that redirect STDOUT output should always see STDERR
prompts interactively.
STDERR should always be flushed without buffering, so
they should always show up. If that is unset, we still
explicitly flush by calling STDERR->flush.
The svn command-line client prompts to STDERR, too.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
While 'init-db' still is and probably will always remain a valid git
command for obvious backward compatibility reasons, it would be a good
idea to move shipped tools and docs to using 'init' instead.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some users apparently create local heads with the same basename
as the remote branch they're tracking.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Also, document --{trunk,branches,tags} options while we're
documenting multi-init options.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Since fetch reforks itself at most every 1000 revisions, we
need to update the counter in the parent process to have a
working count if we set our repack interval to be > ~1000
revisions. multi-fetch has always done this correctly
because of an extra process; now fetch uses the extra process;
as well.
While we're at it, only compile the $sha1 regex that checks for
repacking once.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It now requires at least one of the (trunk|branch|tags) arguments
(either from the command-line or in .git/config). Also we make
sure that anything that is passed as a URL ('help') in David's
case is actually a URL.
Thanks to David Kågedal for reporting this issue.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This could be useful in finding new problems and helping users
debug.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We have less code to worry about now. As a bonus, --revision
can be used to reliably skip parts of history whenever fetch is
run, not just the first time. I'm not sure why anybody would
want to skip history in the middle, however...
For people (nearly everyone at the moment) without the
do_switch() function in their Perl SVN library, the entire tree
must be refetched if --follow-parent is used and a parent is
found. Future versions of SVN will have a working do_switch()
function accessible via Perl.
Accessing repositories on the local machine (especially file://
ones) is also slightly slower as a result; but I suspect most
git-svn users will be using it to access remote repositories.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a followup to dd31da2fdc.
Regardless of whether we commit an alternate head, we always
diff-tree based on the current HEAD, and rebase against our
remote reference as necessary.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
dcommit does commits and fetches, so all options used for those
should work, too, including --authors-file.
Reported missing by Nicolas Vilz.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
'set-tree' probably accurately describes what the command
formerly known as 'commit' does.
I'm not entirely sure that 'dcommit' should be renamed to 'commit'
just yet... Perhaps 'push' or 'push-changes'?
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Using the command-line client was great for prototyping and
getting something working quickly. Eventually I found time
to study the library documentation and add support for using
the libraries which are much faster and more flexible when
it comes to supporting new features.
Note that we require version 1.1 of the SVN libraries, whereas
we supported the command-line svn client down to version 1.0.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Thanks to Git.pm, I've been able to greatly reduce the amount
of extra work that needs to be done to manage input/output
pipes in Perl.
chomp usage has also been greatly reduced, too.
All tests (including full-svn-test) still pass, but this has
not been tested extensively in the real-world.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The list concensus is to group color related configuration under
"color.*" so let's be consistent.
Inspired by Andy Parkins's patch to do the same for diff/log
family. With fixes from Eric Wong.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The low-level parts of the SVN library return NULL/undef for
author-less revisions, whereas "(no author)" is a (svn) client
convention.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We now use git-rev-parse universally to read refs, instead
of our own file_to_s function (which I plan on removing).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If I wanted to print $@, I'd pass $@ to fatal(). This looks like
a stupid typo on my part.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Previously dcommit would unconditionally commit all patches
up-to and including the current HEAD. Now if an optional
command-line argument is specified, it will only commit
up to the specified revision.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The changes are now tracked in
$GIT_DIR/svn/$GIT_SVN_ID/untracked.log
Information in the untracked.log include:
* the addition and removal of empty directories
(changes of these will also warn the user)
* file and directory property changes, including (but not
limited to) svk:merge and svn:externals
* revision properties (revprops) are also tracked
* users will be warned of 'absent' file and directories
(if users are forbidden access)
Fields in entries are separated by spaces; "unsafe" characters
are URI-encoded so that each entry takes exactly one line.
There is currently no automated parser for dealing with the data
in untracked.log, but it should be possible to write one to
create empty directories on checkout and manage
externals/subprojects.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
do_switch works with the SVN Perl bindings after r22312 in the
Subversion trunk. Since no released version of SVN currently
supports it; we'll just autodetect it and enable its usage
when a user has a recent-enough version of SVN.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This may be an issue with repositories imported with commit
27a1a8014b or later, but before
commit dad73c0bb9.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Long-running fetches run inside children to avoid memory leaks.
When we refork, the connection in the parent can be idle for a
long time; attempting to reuse it in the next child can result
in timeouts.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
SVN is not entirely consistent in returning log information and
sometimes returns file information when adding subdirectories,
and sometimes it does not (only returning information about the
directory that was added). This caused git-svn to occasionally
add a file to the list of files to be fetched twice. Now we
change the data structure to be hash to avoid repeated fetches.
As of now (in master), this only affects repositories fetched
without deltas enabled (file://, and when manually overriden
with GIT_SVN_DELTA_FETCH=0); so this bug mainly affects users of
1.4.4.1 and maint.
Thanks to Florian Weimer for reporting this bug.
[jc: backported for maint]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
After the bugfix to connect to repositories where the user has
limited read permissions, multi-init was broken due to our
SVN::Ra connection being limited to working in a subdirectory;
so we now create a new Ra connection for init-ing branches
and another for tags
Along with that fix, allow the user to use the command-line
option flags for multi-init (--revision being the most notable;
but also --no-auth-cache, --config-dir, --username (for passing
to SVN), and --shared/--template for passing to git-init-db
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* match LESS environment settings to those in pager.c
* parse diff.color and pager.color settings in the
config file, and pass --color to git-log
* --color and --pager= settings are supported
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There was nothing printed in the code originally because I left
out a pair of parentheses. Nevertheless, the affected code has
been replaced with a more efficient version that respects the -q
flag as well as requiring less bandwidth.
We save some bandwidth by not requesting changed paths
information when calling get_log() since we're using the delta
fetcher.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
finish_report does seem to return a useful value indicating success
or failure, so we'll just set a flag when close_edit is called
(it is not called on failures, nor is abort_edit) and check
the flag before proceeding.
Thanks to Pazu for pointing this out.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This should drastically reduce bandwidth used for network
transfers. This is not enabled for file:// repositories by
default because of the increased CPU usage and I/O needed.
GIT_SVN_DELTA_FETCH may be set to a true value to enable or
false (0) to disable delta transfers regardless of the
repository type.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows users to use HTTP proxy information (among other settings)
from ~/.subversion/servers and ~/.subversion/config
--config-dir (as before) may be passed to git-svn to override the
default choice of '~/.subversion' for the configuration directory.
Thanks to tko on #git for pointing this out.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I broke this again in 747fa12cef.
Thanks to merlyn for pointing this out to me on IRC.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
