Commit Graph

275 Commits

Author SHA1 Message Date
Junio C Hamano
486c8e8c6a connect.c: stop conflating ssh command names and overrides
dd33e07766 ("connect: Add the envvar GIT_SSH_VARIANT and ssh.variant
config", 2017-02-01) attempted to add support for configuration and
environment variable to override the different handling of
port_option and needs_batch settings suitable for variants of the
ssh implementation that was autodetected by looking at the ssh
command name.  Because it piggybacked on the code that turns command
name to specific override (e.g. "plink.exe" and "plink" means
port_option needs to be set to 'P' instead of the default 'p'), yet
it defined a separate namespace for these overrides (e.g. "putty"
can be usable to signal that port_option needs to be 'P'), however,
it made the auto-detection based on the command name less robust
(e.g. the code now accepts "putty" as a SSH command name and applies
the same override).

Separate the code that interprets the override that was read from
the configuration & environment from the original code that handles
the command names, as they are in separate namespaces, to fix this
confusion.

This incidentally also makes it easier for future enhancement of the
override syntax (e.g. "port_option=p,needs_batch=1" may want to be
accepted as a more explicit syntax) without affecting the code for
auto-detection based on the command name.

While at it, update the return type of the handle_ssh_variant()
helper function to void; the caller does not use it, and the
function does not return any meaningful value.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-10 13:47:24 -08:00
Segev Finer
dd33e07766 connect: Add the envvar GIT_SSH_VARIANT and ssh.variant config
This environment variable and configuration value allow to
override the autodetection of plink/tortoiseplink in case that
Git gets it wrong.

[jes: wrapped overly-long lines, factored out and changed
get_ssh_variant() to handle_ssh_variant() to accomodate the
change from the putty/tortoiseplink variables to
port_option/needs_batch, adjusted the documentation, free()d
value obtained from the config.]

Signed-off-by: Segev Finer <segev208@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01 10:57:53 -08:00
Johannes Schindelin
e2824e47e7 git_connect(): factor out SSH variant handling
We handle plink and tortoiseplink as OpenSSH replacements, by passing
the correct command-line options when detecting that they are used.

To let users override that auto-detection (in case Git gets it wrong),
we need to introduce new code to that end.

In preparation for this code, let's factor out the SSH variant handling
into its own function, handle_ssh_variant().

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01 10:57:00 -08:00
Junio C Hamano
6a4f3a9edc connect: rename tortoiseplink and putty variables
One of these two may have originally been named after "what exact
SSH implementation do we have?" so that we can tweak the command
line options for that exact implementation.  But "putty=1" no longer
means "We are using the plink SSH implementation that comes with
PuTTY" these days.  It is set when we guess that either PuTTY plink
or Tortoiseplink is in use.

Rename them after what effect is desired.  The current 'putty'
option is about using "-P <port>" when OpenSSH would use "-p <port>",
so rename it to 'port_option' whose value is either 'p' or 'P".  The
other one is about passing an extra command line option "-batch",
so rename it to 'needs_batch'.

[jes: wrapped overly-long line]

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-26 11:09:17 -08:00
Segev Finer
e9d9a8a4d2 connect: handle putty/plink also in GIT_SSH_COMMAND
Git for Windows has special support for the popular SSH client PuTTY:
when using PuTTY's non-interactive version ("plink.exe"), we use the -P
option to specify the port rather than OpenSSH's -p option. TortoiseGit
ships with its own, forked version of plink.exe, that adds support for
the -batch option, and for good measure we special-case that, too.

However, this special-casing of PuTTY only covers the case where the
user overrides the SSH command via the environment variable GIT_SSH
(which allows specifying the name of the executable), not
GIT_SSH_COMMAND (which allows specifying a full command, including
additional command-line options).

When users want to pass any additional arguments to (Tortoise-)Plink,
such as setting a private key, they are required to either use a shell
script named plink or tortoiseplink or duplicate the logic that is
already in Git for passing the correct style of command line arguments,
which can be difficult, error prone and annoying to get right.

This patch simply reuses the existing logic and expands it to cover
GIT_SSH_COMMAND, too.

Note: it may look a little heavy-handed to duplicate the entire
command-line and then split it, only to extract the name of the
executable. However, this is not a performance-critical code path, and
the code is much more readable this way.

Signed-off-by: Segev Finer <segev208@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-25 13:47:22 -08:00
Junio C Hamano
dbaa6bdce2 Merge branch 'ls/filter-process'
The smudge/clean filter API expect an external process is spawned
to filter the contents for each path that has a filter defined.  A
new type of "process" filter API has been added to allow the first
request to run the filter for a path to spawn a single process, and
all filtering need is served by this single process for multiple
paths, reducing the process creation overhead.

* ls/filter-process:
  contrib/long-running-filter: add long running filter example
  convert: add filter.<driver>.process option
  convert: prepare filter.<driver>.process option
  convert: make apply_filter() adhere to standard Git error handling
  pkt-line: add functions to read/write flush terminated packet streams
  pkt-line: add packet_write_gently()
  pkt-line: add packet_flush_gently()
  pkt-line: add packet_write_fmt_gently()
  pkt-line: extract set_packet_header()
  pkt-line: rename packet_write() to packet_write_fmt()
  run-command: add clean_on_exit_handler
  run-command: move check_pipe() from write_or_die to run_command
  convert: modernize tests
  convert: quote filter names in error messages
2016-10-31 13:15:21 -07:00
Lars Schneider
81c634e94f pkt-line: rename packet_write() to packet_write_fmt()
packet_write() should be called packet_write_fmt() because it is a
printf-like function that takes a format string as first parameter.

packet_write_fmt() should be used for text strings only. Arbitrary
binary data should use a new packet_write() function that is introduced
in a subsequent patch.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-17 11:36:50 -07:00
Junio C Hamano
8969feac7e Merge branch 'va/i18n-more'
Even more i18n.

* va/i18n-more:
  i18n: stash: mark messages for translation
  i18n: notes-merge: mark die messages for translation
  i18n: ident: mark hint for translation
  i18n: i18n: diff: mark die messages for translation
  i18n: connect: mark die messages for translation
  i18n: commit: mark message for translation
2016-09-26 16:09:18 -07:00
Junio C Hamano
07d872434d Merge branch 'jt/accept-capability-advertisement-when-fetching-from-void'
JGit can show a fake ref "capabilities^{}" to "git fetch" when it
does not advertise any refs, but "git fetch" was not prepared to
see such an advertisement.  When the other side disconnects without
giving any ref advertisement, we used to say "there may not be a
repository at that URL", but we may have seen other advertisement
like "shallow" and ".have" in which case we definitely know that a
repository is there.  The code to detect this case has also been
updated.

* jt/accept-capability-advertisement-when-fetching-from-void:
  connect: advertized capability is not a ref
  connect: tighten check for unexpected early hang up
  tests: move test_lazy_prereq JGIT to test-lib.sh
2016-09-21 15:15:18 -07:00
Vasco Almeida
f2b93b388c i18n: connect: mark die messages for translation
Mark messages passed to die() in die_initial_contact().

Update test to reflect changes.

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-19 10:55:36 -07:00
Jonathan Tan
eb398797cd connect: advertized capability is not a ref
When cloning an empty repository served by standard git, "git clone" produces
the following reassuring message:

	$ git clone git://localhost/tmp/empty
	Cloning into 'empty'...
	warning: You appear to have cloned an empty repository.
	Checking connectivity... done.

Meanwhile when cloning an empty repository served by JGit, the output is more
haphazard:

	$ git clone git://localhost/tmp/empty
	Cloning into 'empty'...
	Checking connectivity... done.
	warning: remote HEAD refers to nonexistent ref, unable to checkout.

This is a common command to run immediately after creating a remote repository
as preparation for adding content to populate it and pushing. The warning is
confusing and needlessly worrying.

The cause is that, since v3.1.0.201309270735-rc1~22 (Advertise capabilities
with no refs in upload service., 2013-08-08), JGit's ref advertisement includes
a ref named capabilities^{} to advertise its capabilities on, while git's ref
advertisement is empty in this case. This allows the client to learn about the
server's capabilities and is needed, for example, for fetch-by-sha1 to work
when no refs are advertised.

This also affects "ls-remote". For example, against an empty repository served
by JGit:

	$ git ls-remote git://localhost/tmp/empty
	0000000000000000000000000000000000000000        capabilities^{}

Git advertises the same capabilities^{} ref in its ref advertisement for push
but since it never did so for fetch, the client didn't need to handle this
case.  Handle it.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-09 13:40:36 -07:00
Jonathan Nieder
55e4f9365a connect: tighten check for unexpected early hang up
A server hanging up immediately to mark access being denied does not
send any .have refs, shallow lines, or anything else before hanging
up.  If the server has sent anything, then the hangup is unexpected.

That is, if the server hangs up after a shallow line but before sending
any refs, then git should tell me so:

	fatal: The remote end hung up upon initial contact

instead of suggesting an access control problem:

	fatal: Could not read from remote repository.
	Please make sure you have the correct access rights
	and the repository exists.

Noticed while examining this code.  This case isn't likely to come up
in practice but tightening the check makes the code easier to read and
manipulate.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-09 13:37:53 -07:00
Nguyễn Thái Ngọc Duy
3c8ede3ff3 connect: read $GIT_SSH_COMMAND from config file
Similar to $GIT_ASKPASS or $GIT_PROXY_COMMAND, we also read from
config file first then fall back to $GIT_SSH_COMMAND.

This is useful for selecting different private keys targetting the
same host (e.g. github)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 14:04:09 -07:00
Junio C Hamano
9e689802e3 Merge branch 'cn/deprecate-ssh-git-url'
The two alternative ways to spell "ssh://" transport have been
deprecated for a long time.  The last mention of them has finally
removed from the documentation.

* cn/deprecate-ssh-git-url:
  Disown ssh+git and git+ssh
2016-03-16 13:16:40 -07:00
Carlos Martín Nieto
07c7782cc8 Disown ssh+git and git+ssh
Some people argue that these were silly from the beginning (see
http://thread.gmane.org/gmane.comp.version-control.git/285590/focus=285601
for example), but we have to support them for compatibility.

That doesn't mean we have to show them in the documentation.  These
were already left out of the main list, but a reference in the main
manpage was left, so remove that.

Also add a note to discourage their use if anybody goes looking for them
in the source code.

Signed-off-by: Carlos Martín Nieto <cmn@dwim.me>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-09 13:54:38 -08:00
Eric Wong
c915f11eb4 connect & http: support -4 and -6 switches for remote operations
Sometimes it is necessary to force IPv4-only or IPv6-only operation
on networks where name lookups may return a non-routable address and
stall remote operations.

The ssh(1) command has an equivalent switches which we may pass when
we run them.  There may be old ssh(1) implementations out there
which do not support these switches; they should report the
appropriate error in that case.

rsync support is untouched for now since it is deprecated and
scheduled to be removed.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Reviewed-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-12 11:34:14 -08:00
brian m. carlson
e96b16cc2a get_remote_heads: convert to struct object_id
Replace an unsigned char array with struct object_id and express several
hard-coded constants in terms of GIT_SHA1_HEXSZ.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00
brian m. carlson
f4e54d02b8 Convert struct ref to use object_id.
Use struct object_id in three fields in struct ref and convert all the
necessary places that use it.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00
Junio C Hamano
78891795df Merge branch 'jk/war-on-sprintf'
Many allocations that is manually counted (correctly) that are
followed by strcpy/sprintf have been replaced with a less error
prone constructs such as xstrfmt.

Macintosh-specific breakage was noticed and corrected in this
reroll.

* jk/war-on-sprintf: (70 commits)
  name-rev: use strip_suffix to avoid magic numbers
  use strbuf_complete to conditionally append slash
  fsck: use for_each_loose_file_in_objdir
  Makefile: drop D_INO_IN_DIRENT build knob
  fsck: drop inode-sorting code
  convert strncpy to memcpy
  notes: document length of fanout path with a constant
  color: add color_set helper for copying raw colors
  prefer memcpy to strcpy
  help: clean up kfmclient munging
  receive-pack: simplify keep_arg computation
  avoid sprintf and strcpy with flex arrays
  use alloc_ref rather than hand-allocating "struct ref"
  color: add overflow checks for parsing colors
  drop strcpy in favor of raw sha1_to_hex
  use sha1_to_hex_r() instead of strcpy
  daemon: use cld->env_array when re-spawning
  stat_tracking_info: convert to argv_array
  http-push: use an argv_array for setup_revisions
  fetch-pack: use argv_array for index-pack / unpack-objects
  ...
2015-10-20 15:24:01 -07:00
Junio C Hamano
d10a7f7535 Merge branch 'tk/typofix-connect-unknown-proto-error'
* tk/typofix-connect-unknown-proto-error:
  connect: fix typo in result string of prot_name()
2015-10-14 14:30:17 -07:00
Junio C Hamano
590f6e4235 Sync with 2.6.1 2015-10-05 13:20:08 -07:00
Junio C Hamano
6343e2f6f2 Sync with 2.3.10 2015-09-28 15:28:31 -07:00
Jeff King
5096d4909f convert trivial sprintf / strcpy calls to xsnprintf
We sometimes sprintf into fixed-size buffers when we know
that the buffer is large enough to fit the input (either
because it's a constant, or because it's numeric input that
is bounded in size). Likewise with strcpy of constant
strings.

However, these sites make it hard to audit sprintf and
strcpy calls for buffer overflows, as a reader has to
cross-reference the size of the array with the input. Let's
use xsnprintf instead, which communicates to a reader that
we don't expect this to overflow (and catches the mistake in
case we do).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25 10:18:18 -07:00
Tobias Klauser
83e6bda3fa connect: fix typo in result string of prot_name()
Replace 'unkown' with 'unknown'.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25 10:01:07 -07:00
Jeff King
a5adaced2e transport: add a protocol-whitelist environment variable
If we are cloning an untrusted remote repository into a
sandbox, we may also want to fetch remote submodules in
order to get the complete view as intended by the other
side. However, that opens us up to attacks where a malicious
user gets us to clone something they would not otherwise
have access to (this is not necessarily a problem by itself,
but we may then act on the cloned contents in a way that
exposes them to the attacker).

Ideally such a setup would sandbox git entirely away from
high-value items, but this is not always practical or easy
to set up (e.g., OS network controls may block multiple
protocols, and we would want to enable some but not others).

We can help this case by providing a way to restrict
particular protocols. We use a whitelist in the environment.
This is more annoying to set up than a blacklist, but
defaults to safety if the set of protocols git supports
grows). If no whitelist is specified, we continue to default
to allowing all protocols (this is an "unsafe" default, but
since the minority of users will want this sandboxing
effect, it is the only sensible one).

A note on the tests: ideally these would all be in a single
test file, but the git-daemon and httpd test infrastructure
is an all-or-nothing proposition rather than a test-by-test
prerequisite. By putting them all together, we would be
unable to test the file-local code on machines without
apache.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-23 11:35:48 -07:00
Jeff King
a48b409f9c git_connect: clarify conn->use_shell flag
When executing user-specified programs, we generally always
want to use a shell, for flexibility and consistency. One
big exception is executing $GIT_SSH, which for historical
reasons must not use a shell.

Once upon a time the logic in git_connect looked like:

  if (protocol == PROTO_SSH) {
	  ... setup ssh ...
  } else {
	  ... setup local connection ...
	  conn->use_shell = 1;
  }

But over time the PROTO_SSH block has grown, and the "local"
block has shrunk so that it contains only conn->use_shell;
it's easy to miss at the end of the large block.  Moreover,
PROTO_SSH now also sometimes sets use_shell, when the new
GIT_SSH_COMMAND is used.

Let's just set conn->use_shell when we're setting up the "conn"
struct, and unset it (with a comment) in the historical GIT_SSH
case.  This will make the flow easier to follow.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08 15:33:21 -07:00
Jeff King
aab4043851 git_connect: clear GIT_* environment for ssh
When we "switch" to another local repository to run the server
side of a fetch or push, we must clear the variables in
local_repo_env so that our local $GIT_DIR, etc, do not
pollute the upload-pack or receive-pack that is executing in
the "remote" repository.

We have never done so for ssh connections. For the most
part, nobody has noticed because ssh will not pass unknown
environment variables by default. However, it is not out of
the question for a user to configure ssh to pass along GIT_*
variables using SendEnv/AcceptEnv.

We can demonstrate the problem by using "git -c" on a local
command and seeing its impact on a remote repository.  This
config ends up in $GIT_CONFIG_PARAMETERS. In the local case,
the config has no impact, but in the ssh transport, it does
(our test script has a fake ssh that passes through all
environment variables; this isn't normal, but does simulate
one possible setup).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-04 15:50:33 -07:00
Junio C Hamano
2d8bb4685c Merge branch 'bc/connect-plink' into maint
The connection initiation code for "ssh" transport tried to absorb
differences between the stock "ssh" and Putty-supplied "plink" and
its derivatives, but the logic to tell that we are using "plink"
variants were too loose and falsely triggered when "plink" appeared
anywhere in the path (e.g. "/home/me/bin/uplink/ssh").

* bc/connect-plink:
  connect: improve check for plink to reduce false positives
  t5601: fix quotation error leading to skipped tests
  connect: simplify SSH connection code path
2015-06-05 12:00:11 -07:00
Junio C Hamano
7a4f891329 Merge branch 'bc/connect-plink'
The connection initiation code for "ssh" transport tried to absorb
differences between the stock "ssh" and Putty-supplied "plink" and
its derivatives, but the logic to tell that we are using "plink"
variants were too loose and falsely triggered when "plink" appeared
anywhere in the path (e.g. "/home/me/bin/uplink/ssh").

* bc/connect-plink:
  connect: improve check for plink to reduce false positives
  t5601: fix quotation error leading to skipped tests
  connect: simplify SSH connection code path
2015-05-19 13:17:55 -07:00
brian m. carlson
baaf233755 connect: improve check for plink to reduce false positives
The git_connect function has code to handle plink and tortoiseplink
specially, as they require different command line arguments from
OpenSSH (-P instead of -p for ports; tortoiseplink additionally requires
-batch).  However, the match was done by checking for "plink" anywhere
in the string, which led to a GIT_SSH value containing "uplink" being
treated as an invocation of putty's plink.

Improve the check by looking for "plink" or "tortoiseplink" (or those
names suffixed with ".exe") only in the final component of the path.
This has the downside that a program such as "plink-0.63" would no
longer be recognized, but the increased robustness is likely worth it.
Add tests to cover these cases to avoid regressions.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-28 15:23:12 -07:00
brian m. carlson
37ee646e72 connect: simplify SSH connection code path
The code path used in git_connect pushed the majority of the SSH
connection code into an else block, even though the if block returns.
Simplify the code by eliminating the else block, as it is unneeded.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-28 15:23:12 -07:00
Junio C Hamano
ad34ad614d Merge branch 'tb/connect-ipv6-parse-fix' into maint
An earlier update to the parser that disects a URL broke an
address, followed by a colon, followed by an empty string (instead
of the port number), e.g. ssh://example.com:/path/to/repo.

* tb/connect-ipv6-parse-fix:
  connect.c: ignore extra colon after hostname
2015-04-27 12:23:54 -07:00
Junio C Hamano
9718c7c0c2 Merge branch 'tb/connect-ipv6-parse-fix'
An earlier update to the parser that disects an address broke an
address, followed by a colon, followed by an empty string (instead
of the port number).

* tb/connect-ipv6-parse-fix:
  connect.c: ignore extra colon after hostname
2015-04-20 15:28:33 -07:00
Torsten Bögershausen
6b6c5f7a2f connect.c: ignore extra colon after hostname
Ignore an extra ':' at the end of the hostname in URL's like
"ssh://example.com:/path/to/repo"

The colon is meant to separate a port number from the hostname.
If the port is empty, the colon should be ignored, see RFC 3986.

It had been working for URLs with ssh:// scheme, but was unintentionally
broken in 86ceb3, "allow ssh://user@[2001:db8::1]/repo.git"

Reported-by: Reid Woodbury Jr. <reidw@rawsound.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-08 21:00:53 -07:00
Junio C Hamano
8c2ea51254 Merge branch 'tb/connect-ipv6-parse-fix' into maint
We did not parse username followed by literal IPv6 address in SSH
transport URLs, e.g. ssh://user@[2001:db8::1]:22/repo.git
correctly.

* tb/connect-ipv6-parse-fix:
  t5500: show user name and host in diag-url
  t5601: add more test cases for IPV6
  connect.c: allow ssh://user@[2001:db8::1]/repo.git
2015-03-23 11:23:13 -07:00
Junio C Hamano
c722ba4814 Merge branch 'jk/daemon-interpolate' into maint
The "interpolated-path" option of "git daemon" inserted any string
client declared on the "host=" capability request without checking.
Sanitize and limit %H and %CH to a saner and a valid DNS name.

* jk/daemon-interpolate:
  daemon: sanitize incoming virtual hostname
  t5570: test git-daemon's --interpolated-path option
  git_connect: let user override virtual-host we send to daemon
2015-03-13 22:55:59 -07:00
Stefan Beller
04f20c04c6 connect.c: do not leak "conn" after showing diagnosis
When git_connect() is called to see how the URL is parsed for
debugging purposes with CONNECT_DIAG_URL set, the variable conn is
leaked.  At this point in the codeflow, it only has its memory and
no other resource is associated with it, so it is sufficient to
clean it up by just freeing it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-10 15:36:03 -07:00
Junio C Hamano
f56a5f4fed Merge branch 'rs/simple-cleanups' into maint
Code cleanups.

* rs/simple-cleanups:
  sha1_name: use strlcpy() to copy strings
  pretty: use starts_with() to check for a prefix
  for-each-ref: use skip_prefix() to avoid duplicate string comparison
  connect: use strcmp() for string comparison
2015-03-06 14:57:57 -08:00
Junio C Hamano
42da484006 Merge branch 'tb/connect-ipv6-parse-fix'
We did not parse username followed by literal IPv6 address in SSH
transport URLs, e.g. ssh://user@[2001:db8::1]:22/repo.git
correctly.

* tb/connect-ipv6-parse-fix:
  t5500: show user name and host in diag-url
  t5601: add more test cases for IPV6
  connect.c: allow ssh://user@[2001:db8::1]/repo.git
2015-03-05 12:45:44 -08:00
Junio C Hamano
8a6444d50e Merge branch 'rs/simple-cleanups'
Code cleanups.

* rs/simple-cleanups:
  sha1_name: use strlcpy() to copy strings
  pretty: use starts_with() to check for a prefix
  for-each-ref: use skip_prefix() to avoid duplicate string comparison
  connect: use strcmp() for string comparison
2015-03-05 12:45:42 -08:00
Junio C Hamano
4c3dbbf722 Merge branch 'jk/daemon-interpolate'
The "interpolated-path" option of "git daemon" inserted any string
client declared on the "host=" capability request without checking.
Sanitize and limit %H and %CH to a saner and a valid DNS name.

* jk/daemon-interpolate:
  daemon: sanitize incoming virtual hostname
  t5570: test git-daemon's --interpolated-path option
  git_connect: let user override virtual-host we send to daemon
2015-03-03 14:37:06 -08:00
Torsten Bögershausen
3f55ccab8e t5500: show user name and host in diag-url
The URL for ssh may have include a username before the hostname,
like ssh://user@host/repo.
When literal IPV6 addresses are used together with a username,
the substring "user@[::1]" must be converted into "user@::1".

Make that conversion visible for the user, and write userandhost
in the diagnostics

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-22 12:04:01 -08:00
Torsten Bögershausen
86ceb337ec connect.c: allow ssh://user@[2001:db8::1]/repo.git
The ssh:// syntax was added in 2386d658 (Add first cut at "git
protocol" connect logic., 2005-07-13), it accepted
ssh://user@2001:db8::1/repo.git, which is now legacy.

Over the years the parser was improved to support [] and port numbers,
but the combination of ssh://user@[2001:db8::1]:222/repo.git did
never work.

The only only way to use a user name, a literall IPV6 address and a port
number was ssh://[user@2001:db8::1]:222/repo.git

(Thanks to Christian Taube <lists@hcf.yourweb.de> for reporting this long
standing issue)

New users would use ssh://user@[2001:db8::1]:222/repo.git,
so change the parser to handle it correctly.

Support the old legacy URLs as well, to be backwards compatible,
and avoid regressions for users which upgrade an existing installation
to a later Git version.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-22 12:02:59 -08:00
René Scharfe
2ae7f90f26 connect: use strcmp() for string comparison
Get rid of magic string length constants and simply compare the strings
using strcmp().  This makes the intent of the code a bit clearer.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-22 12:01:37 -08:00
Jeff King
94bc83c593 git_connect: let user override virtual-host we send to daemon
When we connect to a git-daemon at a given host and port, we
actually send the string "localhost:9418" to the other side,
which allows it to do virtual-hosting lookups. For testing
and debugging, we'd like to be able to send arbitrary
strings, rather than the hostname we actually connected to.

Using "insteadOf" config does not work for this purpose, as
the hostname determination happens at a very low level,
right before we feed the hostname to our lookup routines.
You could use /etc/hosts or similar to get around this, but
we cannot do that portably from our test suite.

Instead, this patch provides an environment variable that
can be used to send an arbitrary string.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-17 13:15:07 -08:00
Junio C Hamano
a7ddaa8eac Merge branch 'mh/simplify-repack-without-refs'
"git remote update --prune" to drop many refs has been optimized.

* mh/simplify-repack-without-refs:
  sort_string_list(): rename to string_list_sort()
  prune_remote(): iterate using for_each_string_list_item()
  prune_remote(): rename local variable
  repack_without_refs(): make the refnames argument a string_list
  prune_remote(): sort delete_refs_list references en masse
  prune_remote(): initialize both delete_refs lists in a single loop
  prune_remote(): exit early if there are no stale references
2014-12-22 12:26:50 -08:00
Michael Haggerty
3383e19984 sort_string_list(): rename to string_list_sort()
The new name is more consistent with the names of other
string_list-related functions.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-11-25 10:11:34 -08:00
Thomas Quinot
39942766ab git_connect: set ssh shell command in GIT_SSH_COMMAND
It may be impractical to install a wrapper script for GIT_SSH
when additional parameters need to be passed. Provide an alternative
way of specifying a shell command to be run, including command line
arguments, by means of the GIT_SSH_COMMAND environment variable,
which behaves like GIT_SSH but is passed to the shell.

The special circuitry to modify parameters in the case of using
PuTTY's plink/tortoiseplink is activated only when using GIT_SSH;
in the case of using GIT_SSH_COMMAND, it is deliberately left up to
the user to make any required parameters adaptation before calling
the underlying ssh implementation.

Signed-off-by: Thomas Quinot <thomas@quinot.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-11-10 08:55:10 -08:00
Junio C Hamano
4fc72d9106 Merge branch 'rs/more-uses-of-skip-prefix'
Code clean-up.

* rs/more-uses-of-skip-prefix:
  pack-write: simplify index_pack_lockfile using skip_prefix() and xstrfmt()
  connect: simplify check_ref() using skip_prefix() and starts_with()
2014-09-19 11:38:35 -07:00
René Scharfe
be0b3f822b connect: simplify check_ref() using skip_prefix() and starts_with()
Both callers of check_ref() pass in NUL-terminated strings for name.
Remove the len parameter and then use skip_prefix() and starts_with()
instead of memcmp() to check if it starts with certain strings.  This
gets rid of several magic string length constants and a strlen() call.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-02 10:36:42 -07:00
René Scharfe
483bbd4e4c run-command: introduce child_process_init()
Add a helper function for initializing those struct child_process
variables for which the macro CHILD_PROCESS_INIT can't be used.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-20 09:54:58 -07:00
René Scharfe
d318027932 run-command: introduce CHILD_PROCESS_INIT
Most struct child_process variables are cleared using memset first after
declaration.  Provide a macro, CHILD_PROCESS_INIT, that can be used to
initialize them statically instead.  That's shorter, doesn't require a
function call and is slightly more readable (especially given that we
already have STRBUF_INIT, ARGV_ARRAY_INIT etc.).

Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-20 09:53:37 -07:00
Junio C Hamano
9ab0882255 Merge branch 'maint'
* maint:
  use xmemdupz() to allocate copies of strings given by start and length
  use xcalloc() to allocate zero-initialized memory
2014-07-21 12:35:39 -07:00
René Scharfe
5c0b13f85a use xmemdupz() to allocate copies of strings given by start and length
Use xmemdupz() to allocate the memory, copy the data and make sure to
NUL-terminate the result, all in one step.  The resulting code is
shorter, doesn't contain the constants 1 and '\0', and avoids
duplicating function parameters.

For blame, the last copied byte (o->file.ptr[o->file.size]) is always
set to NUL by fake_working_tree_commit() or read_sha1_file(), so no
information is lost by the conversion to using xmemdupz().

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-21 10:37:02 -07:00
Jeff King
ae021d8791 use skip_prefix to avoid magic numbers
It's a common idiom to match a prefix and then skip past it
with a magic number, like:

  if (starts_with(foo, "bar"))
	  foo += 3;

This is easy to get wrong, since you have to count the
prefix string yourself, and there's no compiler check if the
string changes.  We can use skip_prefix to avoid the magic
numbers here.

Note that some of these conversions could be much shorter.
For example:

  if (starts_with(arg, "--foo=")) {
	  bar = arg + 6;
	  continue;
  }

could become:

  if (skip_prefix(arg, "--foo=", &bar))
	  continue;

However, I have left it as:

  if (skip_prefix(arg, "--foo=", &v)) {
	  bar = v;
	  continue;
  }

to visually match nearby cases which need to actually
process the string. Like:

  if (skip_prefix(arg, "--foo=", &v)) {
	  bar = atoi(v);
	  continue;
  }

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-20 10:44:45 -07:00
Jeff King
1823bea10f git_connect: use argv_array
This avoids magic numbers when we allocate fixed-size argv
arrays, and makes it more obvious that we are not
overflowing.

It is also the first step to fixing a memory leak. When
git_connect returns a child_process struct, the argv array
in the struct is dynamically allocated, but the individual
strings are not (they are either owned elsewhere, or are
freed). Later, in finish_connect, we free the array but
leave the strings alone.

This works for the child_process created by git_connect, but
if we use transport_take_over, we may also end up with a
child_process created by transport-helper's get_helper.
In that case, the strings are freshly allocated, and we
would want to free them.  However, we have no idea in
finish_connect which type we have.

By consistently using run-command's internal argv-array, we
do not have to worry about this issue at all; finish_command
takes care of it for us, and we can drop our manual free
entirely.

Note that this actually makes the get_helper leak slightly
worse; now we are leaking both the strings and the array.
But when we adjust it in a future patch, that leak will go
away entirely.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-15 09:49:10 -07:00
Junio C Hamano
34a2e88ae2 Merge branch 'nd/indent-fix-connect-c'
* nd/indent-fix-connect-c:
  connect.c: SP after "}", not TAB
2014-03-25 11:07:06 -07:00
Nguyễn Thái Ngọc Duy
c049b61d42 connect.c: SP after "}", not TAB
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-13 10:54:21 -07:00
Junio C Hamano
92251b1b5b Merge branch 'nd/shallow-clone'
Fetching from a shallow-cloned repository used to be forbidden,
primarily because the codepaths involved were not carefully vetted
and we did not bother supporting such usage. This attempts to allow
object transfer out of a shallow-cloned repository in a controlled
way (i.e. the receiver become a shallow repository with truncated
history).

* nd/shallow-clone: (31 commits)
  t5537: fix incorrect expectation in test case 10
  shallow: remove unused code
  send-pack.c: mark a file-local function static
  git-clone.txt: remove shallow clone limitations
  prune: clean .git/shallow after pruning objects
  clone: use git protocol for cloning shallow repo locally
  send-pack: support pushing from a shallow clone via http
  receive-pack: support pushing to a shallow clone via http
  smart-http: support shallow fetch/clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  send-pack: support pushing to a shallow clone
  receive-pack: allow pushes that update .git/shallow
  connected.c: add new variant that runs with --shallow-file
  add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  receive/send-pack: support pushing from a shallow clone
  receive-pack: reorder some code in unpack()
  fetch: add --update-shallow to accept refs that update .git/shallow
  upload-pack: make sure deepening preserves shallow roots
  fetch: support fetching from a shallow repository
  clone: support remote shallow repository
  ...
2014-01-17 12:21:20 -08:00
Junio C Hamano
1945e8ac85 Merge branch 'tb/clone-ssh-with-colon-for-port'
Be more careful when parsing remote repository URL given in the
scp-style host:path notation.

* tb/clone-ssh-with-colon-for-port:
  git_connect(): use common return point
  connect.c: refactor url parsing
  git_connect(): refactor the port handling for ssh
  git fetch: support host:/~repo
  t5500: add test cases for diag-url
  git fetch-pack: add --diag-url
  git_connect: factor out discovery of the protocol and its parts
  git_connect: remove artificial limit of a remote command
  t5601: add tests for ssh
  t5601: remove clear_ssh, refactor setup_ssh_wrapper
2013-12-17 12:03:32 -08:00
Nguyễn Thái Ngọc Duy
b06dcd7d68 connect.c: teach get_remote_heads to parse "shallow" lines
No callers pass a non-empty pointer as shallow_points at this
stage. As a result, all clients still refuse to talk to shallow
repository on the other end.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10 16:14:16 -08:00
Nguyễn Thái Ngọc Duy
13eb4626c4 remote.h: replace struct extra_have_objects with struct sha1_array
The latter can do everything the former can and is used in many more
places.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10 16:14:15 -08:00
Torsten Bögershausen
a2036d7e00 git_connect(): use common return point
Use only one return point from git_connect(), doing the

    free();
    return conn;

only at one place in the code.

There may be a little confusion what the variable "host" is for.  At
some places it is only the host part, at other places it may include
the port number, so change host into hostandport here.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:48 -08:00
Torsten Bögershausen
c59ab2e52a connect.c: refactor url parsing
Make the function is_local() in transport.c public, rename it into
url_is_local_not_ssh() and use it in both transport.c and connect.c

Use a protocol "local" for URLs for the local file system.

One note about using file:// under Windows:

The (absolute) path on Unix like system typically starts with "/".
When the host is empty, it can be omitted, so that a shell scriptlet
url=file://$pwd
will give a URL like "file:///home/user/repo".

Windows does not have the same concept of a root directory located in "/".
When parsing the URL allow "file://C:/user/repo"
(even if RFC1738 indicates that "file:///C:/user/repo" should be used).

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:48 -08:00
Torsten Bögershausen
83b0587527 git_connect(): refactor the port handling for ssh
Use get_host_and_port() even for ssh.
Remove the variable port git_connect(), and simplify parse_connect_url()
Use only one return point in git_connect(), doing the free() and return conn.

t5601 had 2 corner test cases which now pass.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:47 -08:00
Torsten Bögershausen
6a59974869 git fetch: support host:/~repo
The documentation (in urls.txt) says that

    "ssh://host:/~repo",
    "host:/~repo" or
    "host:~repo"

specify the repository "repo" in the home directory at "host".

This has not been working for "host:/~repo".

Before commit 356bec "Support [address] in URLs", the comparison
"url != hostname" could be used to determine if the URL had a scheme
or not: "ssh://host/host" != "host".

However, after 356bec "[::1]" was converted into "::1", yielding
url != hostname as well.  To fix this regression, don't use
"if (url != hostname)", but look at the separator instead.

Rename the variable "c" into "separator" to make it easier to read.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:47 -08:00
Torsten Bögershausen
5610b7c0c6 git fetch-pack: add --diag-url
The main purpose is to trace the URL parser called by git_connect() in
connect.c

The main features of the parser can be listed as this:

- parse out host and path for URLs with a scheme (git:// file:// ssh://)
- parse host names embedded by [] correctly
- extract the port number, if present
- separate URLs like "file" (which are local)
  from URLs like "host:repo" which should use ssh

Add the new parameter "--diag-url" to "git fetch-pack", which prints
the value for protocol, host and path to stderr and exits.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:47 -08:00
Johannes Sixt
cabc3c12e4 git_connect: factor out discovery of the protocol and its parts
git_connect has grown large due to the many different protocols syntaxes
that are supported. Move the part of the function that parses the URL to
connect to into a separate function for readability.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:47 -08:00
Johannes Sixt
d98d109979 git_connect: remove artificial limit of a remote command
Since day one, function git_connect() had a limit on the command line of
the command that is invoked to make a connection. 7a33bcbe converted the
code that constructs the command to strbuf. This would have been the
right time to remove the limit, but it did not happen. Remove it now.

git_connect() uses start_command() to invoke the command; consequently,
the limits of the system still apply, but are diagnosed only at execve()
time. But these limits are more lenient than the 1K that git_connect()
imposed.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09 14:54:47 -08:00
Christian Couder
5955654823 replace {pre,suf}fixcmp() with {starts,ends}_with()
Leaving only the function definitions and declarations so that any
new topic in flight can still make use of the old functions, replace
existing uses of the prefixcmp() and suffixcmp() with new API
functions.

The change can be recreated by mechanically applying this:

    $ git grep -l -e prefixcmp -e suffixcmp -- \*.c |
      grep -v strbuf\\.c |
      xargs perl -pi -e '
        s|!prefixcmp\(|starts_with\(|g;
        s|prefixcmp\(|!starts_with\(|g;
        s|!suffixcmp\(|ends_with\(|g;
        s|suffixcmp\(|!ends_with\(|g;
      '

on the result of preparatory changes in this series.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-05 14:13:21 -08:00
Junio C Hamano
9907d1359c Merge branch 'jc/upload-pack-send-symref'
One long-standing flaw in the pack transfer protocol used by "git
clone" was that there was no way to tell the other end which branch
"HEAD" points at, and the receiving end needed to guess.  A new
capability has been defined in the pack protocol to convey this
information so that cloning from a repository with more than one
branches pointing at the same commit where the HEAD is at now
reliably sets the initial branch in the resulting repository.

* jc/upload-pack-send-symref:
  t5570: Update for clone-progress-to-stderr branch
  t5570: Update for symref capability
  clone: test the new HEAD detection logic
  connect: annotate refs with their symref information in get_remote_head()
  connect.c: make parse_feature_value() static
  upload-pack: send non-HEAD symbolic refs
  upload-pack: send symbolic ref information as capability
  upload-pack.c: do not pass confusing cb_data to mark_our_ref()
  t5505: fix "set-head --auto with ambiguous HEAD" test
2013-10-30 12:10:06 -07:00
Jonathan Nieder
cabb411fcf Merge branch 'nd/clone-local-with-colon'
* nd/clone-local-with-colon:
  clone: tighten "local paths with colons" check a bit
2013-10-14 11:06:57 -07:00
Nguyễn Thái Ngọc Duy
8d3d28f5db clone: tighten "local paths with colons" check a bit
commit 6000334 (clone: allow cloning local paths with colons in them -
2013-05-04) made it possible to specify a path that has colons in it
without file://, e.g. ../foo:bar/somewhere. But the check was a bit
sloppy.

Consider the url '[foo]:bar'. The '[]' unwrapping code will turn the
string to 'foo\0:bar'. In effect this new string is the same as
'foo/:bar' in the check "path < strchrnul(host, '/')", which mistakes
it for a local path (with '/' before the first ':') when it's actually
not.

So disable the check for '/' before ':' when the URL has been mangled
by '[]' unwrapping.

[jn: with tests from Jeff King]

Noticed-by: Morten Stenshorne <mstensho@opera.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-09-27 14:47:49 -07:00
Junio C Hamano
a45b5f0552 connect: annotate refs with their symref information in get_remote_head()
By doing this, clients of upload-pack can now reliably tell what ref
a symbolic ref points at; the updated test in t5505 used to expect
failure due to the ambiguity and made sure we give diagnostics, but
we no longer need to be so pessimistic. Make sure we correctly learn
which branch HEAD points at from the other side instead.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-17 21:58:46 -07:00
Junio C Hamano
5d54cffc36 connect.c: make parse_feature_value() static
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-17 21:52:06 -07:00
Junio C Hamano
47a5918536 cache.h: move remote/connect API out of it
The definition of "struct ref" in "cache.h", a header file so
central to the system, always confused me.  This structure is not
about the local ref used by sha1-name API to name local objects.

It is what refspecs are expanded into, after finding out what refs
the other side has, to define what refs are updated after object
transfer succeeds to what values.  It belongs to "remote.h" together
with "struct refspec".

While we are at it, also move the types and functions related to the
Git transport connection to a new header file connect.h

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-08 14:34:24 -07:00
Junio C Hamano
41aaccdcf9 Merge branch 'nd/clone-local-with-colon'
"git clone foo/bar:baz" cannot be a request to clone from a remote
over git-over-ssh specified in the scp style.  Detect this case and
clone from a local repository at "foo/bar:baz".

* nd/clone-local-with-colon:
  clone: allow cloning local paths with colons in them
2013-06-02 15:52:22 -07:00
Nguyễn Thái Ngọc Duy
60003340cd clone: allow cloning local paths with colons in them
Usually "foo:bar" is interpreted as an ssh url. This patch allows to
clone from such paths by putting at least one slash before the colon
(i.e. /path/to/foo:bar or just ./foo:bar).

file://foo:bar should also work, but local optimizations are off in
that case, which may be unwanted. While at there, warn the users about
--local being ignored in this case.

Reported-by: William Giokas <1007380@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-07 08:32:14 -07:00
Jeff King
85edf4f58b teach get_remote_heads to read from a memory buffer
Now that we can read packet data from memory as easily as a
descriptor, get_remote_heads can take either one as a
source. This will allow further refactoring in remote-curl.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-24 00:17:38 -08:00
Jeff King
4981fe750b pkt-line: share buffer/descriptor reading implementation
The packet_read function reads from a descriptor. The
packet_get_line function is similar, but reads from an
in-memory buffer, and uses a completely separate
implementation. This patch teaches the generic packet_read
function to accept either source, and we can do away with
packet_get_line's implementation.

There are two other differences to account for between the
old and new functions. The first is that we used to read
into a strbuf, but now read into a fixed size buffer. The
only two callers are fine with that, and in fact it
simplifies their code, since they can use the same
static-buffer interface as the rest of the packet_read_line
callers (and we provide a similar convenience wrapper for
reading from a buffer rather than a descriptor).

This is technically an externally-visible behavior change in
that we used to accept arbitrary sized packets up to 65532
bytes, and now cap out at LARGE_PACKET_MAX, 65520. In
practice this doesn't matter, as we use it only for parsing
smart-http headers (of which there is exactly one defined,
and it is small and fixed-size). And any extension headers
would be breaking the protocol to go over LARGE_PACKET_MAX
anyway.

The other difference is that packet_get_line would return
on error rather than dying. However, both callers of
packet_get_line are actually improved by dying.

The first caller does its own error checking, but we can
drop that; as a result, we'll actually get more specific
reporting about protocol breakage when packet_read dies
internally. The only downside is that packet_read will not
print the smart-http URL that failed, but that's not a big
deal; anybody not debugging can already see the remote's URL
already, and anybody debugging would want to run with
GIT_CURL_VERBOSE anyway to see way more information.

The second caller, which is just trying to skip past any
extra smart-http headers (of which there are none defined,
but which we allow to keep room for future expansion), did
not error check at all. As a result, it would treat an error
just like a flush packet. The resulting mess would generally
cause an error later in get_remote_heads, but now we get
error reporting much closer to the source of the problem.

Brown-paper-bag-fixes-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-24 00:14:15 -08:00
Jeff King
74543a0423 pkt-line: provide a LARGE_PACKET_MAX static buffer
Most of the callers of packet_read_line just read into a
static 1000-byte buffer (callers which handle arbitrary
binary data already use LARGE_PACKET_MAX). This works fine
in practice, because:

  1. The only variable-sized data in these lines is a ref
     name, and refs tend to be a lot shorter than 1000
     characters.

  2. When sending ref lines, git-core always limits itself
     to 1000 byte packets.

However, the only limit given in the protocol specification
in Documentation/technical/protocol-common.txt is
LARGE_PACKET_MAX; the 1000 byte limit is mentioned only in
pack-protocol.txt, and then only describing what we write,
not as a specific limit for readers.

This patch lets us bump the 1000-byte limit to
LARGE_PACKET_MAX. Even though git-core will never write a
packet where this makes a difference, there are two good
reasons to do this:

  1. Other git implementations may have followed
     protocol-common.txt and used a larger maximum size. We
     don't bump into it in practice because it would involve
     very long ref names.

  2. We may want to increase the 1000-byte limit one day.
     Since packets are transferred before any capabilities,
     it's difficult to do this in a backwards-compatible
     way. But if we bump the size of buffer the readers can
     handle, eventually older versions of git will be
     obsolete enough that we can justify bumping the
     writers, as well. We don't have plans to do this
     anytime soon, but there is no reason not to start the
     clock ticking now.

Just bumping all of the reading bufs to LARGE_PACKET_MAX
would waste memory. Instead, since most readers just read
into a temporary buffer anyway, let's provide a single
static buffer that all callers can use. We can further wrap
this detail away by having the packet_read_line wrapper just
use the buffer transparently and return a pointer to the
static storage.  That covers most of the cases, and the
remaining ones already read into their own LARGE_PACKET_MAX
buffers.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-20 13:42:22 -08:00
Jeff King
819b929d33 pkt-line: teach packet_read_line to chomp newlines
The packets sent during ref negotiation are all terminated
by newline; even though the code to chomp these newlines is
short, we end up doing it in a lot of places.

This patch teaches packet_read_line to auto-chomp the
trailing newline; this lets us get rid of a lot of inline
chomping code.

As a result, some call-sites which are not reading
line-oriented data (e.g., when reading chunks of packfiles
alongside sideband) transition away from packet_read_line to
the generic packet_read interface. This patch converts all
of the existing callsites.

Since the function signature of packet_read_line does not
change (but its behavior does), there is a possibility of
new callsites being introduced in later commits, silently
introducing an incompatibility.  However, since a later
patch in this series will change the signature, such a
commit would have to be merged directly into this commit,
not to the tip of the series; we can therefore ignore the
issue.

This is an internal cleanup and should produce no change of
behavior in the normal case. However, there is one corner
case to note. Callers of packet_read_line have never been
able to tell the difference between a flush packet ("0000")
and an empty packet ("0004"), as both cause packet_read_line
to return a length of 0. Readers treat them identically,
even though Documentation/technical/protocol-common.txt says
we must not; it also says that implementations should not
send an empty pkt-line.

By stripping out the newline before the result gets to the
caller, we will now treat the newline-only packet ("0005\n")
the same as an empty packet, which in turn gets treated like
a flush packet. In practice this doesn't matter, as neither
empty nor newline-only packets are part of git's protocols
(at least not for the line-oriented bits, and readers who
are not expecting line-oriented packets will be calling
packet_read directly, anyway). But even if we do decide to
care about the distinction later, it is orthogonal to this
patch.  The right place to tighten would be to stop treating
empty packets as flush packets, and this change does not
make doing so any harder.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-20 13:42:21 -08:00
Jeff King
0380942902 pkt-line: provide a generic reading function with options
Originally we had a single function for reading packetized
data: packet_read_line. Commit 46284dd grew a more "gentle"
form, packet_read, that returns an error instead of dying
upon reading a truncated input stream. However, it is not
clear from the names which should be called, or what the
difference is.

Let's instead make packet_read be a generic public interface
that can take option flags, and update the single callsite
that uses it. This is less code, more clear, and paves the
way for introducing more options into the generic interface
later. The function signature is changed, so there should be
no hidden conflicts with topics in flight.

While we're at it, we'll document how error conditions are
handled based on the options, and rename the confusing
"return_line_fail" option to "gentle_on_eof".  While we are
cleaning up the names, we can drop the "return_line_fail"
checks in packet_read_internal entirely.  They look like
this:

  ret = safe_read(..., return_line_fail);
  if (return_line_fail && ret < 0)
	  ...

The check for return_line_fail is a no-op; safe_read will
only ever return an error value if return_line_fail was true
in the first place.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-20 13:42:21 -08:00
Junio C Hamano
97349a2a74 Merge branch 'jc/capabilities'
Some capabilities were asked by fetch-pack even when upload-pack did
not advertise that they are available.  Fix fetch-pack not to do so.

* jc/capabilities:
  fetch-pack: mention server version with verbose output
  parse_feature_request: make it easier to see feature values
  fetch-pack: do not ask for unadvertised capabilities
  do not send client agent unless server does first
  send-pack: fix capability-sending logic
  include agent identifier in capability string
2012-08-29 14:50:07 -07:00
Jeff King
9442710801 parse_feature_request: make it easier to see feature values
We already take care to parse key/value capabilities like
"foo=bar", but the code does not provide a good way of
actually finding out what is on the right-hand side of the
"=".

A server using "parse_feature_request" could accomplish this
with some extra parsing. You must skip past the "key"
portion manually, check for "=" versus NUL or space, and
then find the length by searching for the next space (or
NUL).  But clients can't even do that, since the
"server_supports" interface does not even return the
pointer.

Instead, let's have our parser share more information by
providing a pointer to the value and its length. The
"parse_feature_value" function returns a pointer to the
feature's value portion, along with the length of the value.
If the feature is missing, NULL is returned. If it does not
have an "=", then a zero-length value is returned.

Similarly, "server_feature_value" behaves in the same way,
but always checks the static server_feature_list variable.

We can then implement "server_supports" in terms of
"server_feature_value". We cannot implement the original
"parse_feature_request" in terms of our new function,
because it returned a pointer to the beginning of the
feature. However, no callers actually cared about the value
of the returned pointer, so we can simplify it to a boolean
just as we do for "server_supports".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-13 21:52:36 -07:00
Junio C Hamano
348c44e78e Merge branch 'hv/remote-end-hung-up'
When we get disconnected while expecting a response from the remote
side because authentication failed, we issued an error message "The
remote side hung up unexpectedly."

Give hint that it may be a permission problem in the message when we
can reasonably suspect it.

* hv/remote-end-hung-up:
  remove the impression of unexpectedness when access is denied
2012-07-04 23:40:12 -07:00
Heiko Voigt
46284dd152 remove the impression of unexpectedness when access is denied
If a server accessed through ssh is denying access git will currently
issue the message

	"fatal: The remote end hung up unexpectedly"

as the last line. This sounds as if something really ugly just happened.
Since this is a quite typical situation in which users regularly get
we do not say that if it happens at the beginning when reading the
remote heads.

If its in the very first beginning of reading the remote heads it is
very likely an authentication error or a missing repository.

If it happens later during reading the remote heads we still indicate
that it happened during this initial contact phase.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-19 13:37:02 -07:00
René Scharfe
7acf438215 git: Wrong parsing of ssh urls with IPv6 literals ignores port
If we encounter an address part shaped like "[HOST]:PORT", we skip the opening
bracket and replace the closing one with a NUL.  The variable host then points
to HOST and we've cut off the PORT part.  Thus, when we go looking for it using
host a bit later, we can't find it.  Start at end instead, which either points
to the colon, if present, or is equal to host.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-13 13:10:55 -07:00
Junio C Hamano
f47182c852 server_supports(): parse feature list more carefully
We have been carefully choosing feature names used in the protocol
extensions so that the vocabulary does not contain a word that is a
substring of another word, so it is not a real problem, but we have
recently added "quiet" feature word, which would mean we cannot later
add some other word with "quiet" (e.g. "quiet-push"), which is awkward.

Let's make sure that we can eventually be able to do so by teaching the
clients and servers that feature words consist of non whitespace
letters. This parser also allows us to later add features with parameters
e.g. "feature=1.5" (parameter values need to be quoted for whitespaces,
but we will worry about the detauls when we do introduce them).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-08 14:26:28 -08:00
Junio C Hamano
ded408fd20 Merge branch 'jk/git-prompt'
* jk/git-prompt:
  contrib: add credential helper for OS X Keychain
  Makefile: OS X has /dev/tty
  Makefile: linux has /dev/tty
  credential: use git_prompt instead of git_getpass
  prompt: use git_terminal_prompt
  add generic terminal prompt function
  refactor git_getpass into generic prompt function
  move git_getpass to its own source file
  imap-send: don't check return value of git_getpass
  imap-send: avoid buffer overflow

Conflicts:
	Makefile
2011-12-22 11:27:23 -08:00
Junio C Hamano
85878dd0c9 Merge branch 'ew/keepalive'
* ew/keepalive:
  enable SO_KEEPALIVE for connected TCP sockets
2011-12-19 16:06:32 -08:00
Jeff King
bab8d28e77 connect.c: drop path_match function
This function was used for comparing local and remote ref
names during fetch (which makes it a candidate for "most
confusingly named function of the year").

It no longer has any callers, so let's get rid of it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13 10:18:12 -08:00
Jeff King
afe7c5ff1f drop "match" parameter from get_remote_heads
The get_remote_heads function reads the list of remote refs
during git protocol session. It dates all the way back to
def88e9 (Commit first cut at "git-fetch-pack", 2005-07-04).
At that time, the idea was to come up with a list of refs we
were interested in, and then filter the list as we got it
from the remote side.

Later, 1baaae5 (Make maximal use of the remote refs,
2005-10-28) stopped filtering at the get_remote_heads layer,
letting us use the non-matching refs to find common history.

As a result, all callers now simply pass an empty match
list (and any future callers will want to do the same). So
let's drop these now-useless parameters.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13 10:08:24 -08:00
Jeff King
d3c58b83ae move git_getpass to its own source file
This is currently in connect.c, but really has nothing to
do with the git protocol itself. Let's make a new source
file all about prompting the user, which will make it
cleaner to refactor.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 16:09:38 -08:00
Eric Wong
e47a8583a2 enable SO_KEEPALIVE for connected TCP sockets
Sockets may never receive notification of some link errors,
causing "git fetch" or similar processes to hang forever.
Enabling keepalive messages allows hung processes to error out
after a few minutes/hours depending on the keepalive settings of
the system.

This is a problem noticed when running non-interactive
cronjobs to mirror repositories using "git fetch".

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-05 21:06:53 -08:00
Michael Haggerty
8d9c50105f Change check_ref_format() to take a flags argument
Change check_ref_format() to take a flags argument that indicates what
is acceptable in the reference name (analogous to "git
check-ref-format"'s "--allow-onelevel" and "--refspec-pattern").  This
is more convenient for callers and also fixes a failure in the test
suite (and likely elsewhere in the code) by enabling "onelevel" and
"refspec-pattern" to be allowed independently of each other.

Also rename check_ref_format() to check_refname_format() to make it
obvious that it deals with refnames rather than references themselves.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-05 13:45:29 -07:00
Junio C Hamano
908bb1a9b7 Merge branch 'dz/connect-error-report' into maint
* dz/connect-error-report:
  Do not log unless all connect() attempts fail
2011-09-11 21:53:47 -07:00
Erik Faye-Lund
7203a2d1ca connect: only log if all attempts failed (ipv4)
In 63a995b (Do not log unless all connect() attempts fail), a
mechanism to only log connection errors if all attempts failed
was introduced for the IPv6 code-path, but not for the IPv4 one.

Introduce a matching mechanism so IPv4-users also benefit from
this noise-reduction.

Move the call to socket after filling in sa, to make it more
apparent that errno can't change in between.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-01 10:49:45 -07:00
Junio C Hamano
9353e09afa Merge branch 'maint' into ef/ipv4-connect-error-report
* maint:
  connect: correctly number ipv6 network adapter
2011-08-01 10:49:40 -07:00
Erik Faye-Lund
e08afecd7f connect: correctly number ipv6 network adapter
In ba50532, the variable 'cnt' was added to both the IPv6 and the
IPv4 version of git_tcp_connect_sock, intended to identify which
network adapter the connection failed on. But in the IPv6 version,
the variable was never increased, leaving it constantly at zero.

This behaviour isn't very useful, so let's fix it by increasing
the variable at every loop-iteration.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-01 10:47:36 -07:00