Commit Graph

928 Commits

Author SHA1 Message Date
brian m. carlson
0f2dc722dd refs: convert resolve_refdup and refs_resolve_refdup to struct object_id
All of the callers already pass the hash member of struct object_id, so
update them to pass a pointer to the struct directly,

This transformation was done with an update to declaration and
definition and the following semantic patch:

@@
expression E1, E2, E3, E4;
@@
- resolve_refdup(E1, E2, E3.hash, E4)
+ resolve_refdup(E1, E2, &E3, E4)

@@
expression E1, E2, E3, E4;
@@
- resolve_refdup(E1, E2, E3->hash, E4)
+ resolve_refdup(E1, E2, E3, E4)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
brian m. carlson
89f3bbdd3b refs: update ref transactions to use struct object_id
Update the ref transaction code to use struct object_id.  Remove one
NULL pointer check which was previously inserted around a dereference;
since we now pass a pointer to struct object_id directly through, the
code we're calling handles this for us.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
brian m. carlson
6ee18216d8 refs: prevent accidental NULL dereference in write_pseudoref
Several of the refs functions take NULL to indicate that the ref is not
to be updated.  If refs_update_ref were called with a NULL new object
ID, we could pass that NULL pointer to write_pseudoref, which would then
segfault when it dereferenced it.  Instead, simply return successfully,
since if we don't want to update the pseudoref, there's nothing to do.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
brian m. carlson
ae077771b0 refs: convert update_ref and refs_update_ref to use struct object_id
Convert update_ref, refs_update_ref, and write_pseudoref to use struct
object_id.  Update the existing callers as well.  Remove update_ref_oid,
as it is no longer needed.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
brian m. carlson
2616a5e508 refs: convert delete_ref and refs_delete_ref to struct object_id
Convert delete_ref and refs_delete_ref to take a pointer to struct
object_id.  Update the documentation accordingly, including referring to
null_oid in lowercase, as it is not a #define constant.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
Jeff King
a1c1d8170d refs_resolve_ref_unsafe: handle d/f conflicts for writes
If our call to refs_read_raw_ref() fails, we check errno to
see if the ref is simply missing, or if we encountered a
more serious error. If it's just missing, then in "write"
mode (i.e., when RESOLVE_REFS_READING is not set), this is
perfectly fine.

However, checking for ENOENT isn't sufficient to catch all
missing-ref cases. In the filesystem backend, we may also
see EISDIR when we try to resolve "a" and "a/b" exists.
Likewise, we may see ENOTDIR if we try to resolve "a/b" and
"a" exists. In both of those cases, we know that our
resolved ref doesn't exist, but we return an error (rather
than reporting the refname and returning a null sha1).

This has been broken for a long time, but nobody really
noticed because the next step after resolving without the
READING flag is usually to lock the ref and write it. But in
both of those cases, the write will fail with the same
errno due to the directory/file conflict.

There are two cases where we can notice this, though:

  1. If we try to write "a" and there's a leftover directory
     already at "a", even though there is no ref "a/b". The
     actual write is smart enough to move the empty "a" out
     of the way.

     This is reasonably rare, if only because the writing
     code has to do an independent resolution before trying
     its write (because the actual update_ref() code handles
     this case fine). The notes-merge code does this, and
     before the fix in the prior commit t3308 erroneously
     expected this case to fail.

  2. When resolving symbolic refs, we typically do not use
     the READING flag because we want to resolve even
     symrefs that point to unborn refs. Even if those unborn
     refs could not actually be written because of d/f
     conflicts with existing refs.

     You can see this by asking "git symbolic-ref" to report
     the target of a symref pointing past a d/f conflict.

We can fix the problem by recognizing the other "missing"
errnos and treating them like ENOENT. This should be safe to
do even for callers who are then going to actually write the
ref, because the actual writing process will fail if the d/f
conflict is a real one (and t1404 checks these cases).

Arguably this should be the responsibility of the
files-backend to normalize all "missing ref" errors into
ENOENT (since something like EISDIR may not be meaningful at
all to a database backend). However other callers of
refs_read_raw_ref() may actually care about the distinction;
putting this into resolve_ref() is the minimal fix for now.

The new tests in t1401 use git-symbolic-ref, which is the
most direct way to check the resolution by itself.
Interestingly we actually had a test that setup this case
already, but we only used it to verify that the funny state
could be overwritten, not that it could be resolved.

We also add a new test in t3200, as "branch -m" was the
original motivation for looking into this. What happens is
this:

  0. HEAD is pointing to branch "a"

  1. The user asks to rename "a" to "a/b".

  2. We create "a/b" and delete "a".

  3. We then try to update any worktree HEADs that point to
     the renamed ref (including the main repo HEAD). To do
     that, we have to resolve each HEAD. But now our HEAD is
     pointing at "a", and we get EISDIR due to the loose
     "a/b". As a result, we think there is no HEAD, and we
     do not update it. It now points to the bogus "a".

Interestingly this case used to work, but only accidentally.
Before 31824d180d (branch: fix branch renaming not updating
HEADs correctly, 2017-08-24), we'd update any HEAD which we
couldn't resolve. That was wrong, but it papered over the
fact that we were incorrectly failing to resolve HEAD.

So while the bug demonstrated by the git-symbolic-ref is
quite old, the regression to "branch -m" is recent.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-07 10:32:13 +09:00
Junio C Hamano
1a2e1a76ec Merge branch 'mh/mmap-packed-refs'
Operations that do not touch (majority of) packed refs have been
optimized by making accesses to packed-refs file lazy; we no longer
pre-parse everything, and an access to a single ref in the
packed-refs does not touch majority of irrelevant refs, either.

* mh/mmap-packed-refs: (21 commits)
  packed-backend.c: rename a bunch of things and update comments
  mmapped_ref_iterator: inline into `packed_ref_iterator`
  ref_cache: remove support for storing peeled values
  packed_ref_store: get rid of the `ref_cache` entirely
  ref_store: implement `refs_peel_ref()` generically
  packed_read_raw_ref(): read the reference from the mmapped buffer
  packed_ref_iterator_begin(): iterate using `mmapped_ref_iterator`
  read_packed_refs(): ensure that references are ordered when read
  packed_ref_cache: keep the `packed-refs` file mmapped if possible
  packed-backend.c: reorder some definitions
  mmapped_ref_iterator_advance(): no peeled value for broken refs
  mmapped_ref_iterator: add iterator over a packed-refs file
  packed_ref_cache: remember the file-wide peeling state
  read_packed_refs(): read references with minimal copying
  read_packed_refs(): make parsing of the header line more robust
  read_packed_refs(): only check for a header at the top of the file
  read_packed_refs(): use mmap to read the `packed-refs` file
  die_unterminated_line(), die_invalid_line(): new functions
  packed_ref_cache: add a backlink to the associated `packed_ref_store`
  prefix_ref_iterator: break when we leave the prefix
  ...
2017-10-03 15:42:50 +09:00
Junio C Hamano
3b48045c6c Merge branch 'sd/branch-copy'
"git branch" learned "-c/-C" to create a new branch by copying an
existing one.

* sd/branch-copy:
  branch: fix "copy" to never touch HEAD
  branch: add a --copy (-c) option to go with --move (-m)
  branch: add test for -m renaming multiple config sections
  config: create a function to format section headers
2017-10-03 15:42:48 +09:00
Junio C Hamano
73ecdc606e Merge branch 'rs/resolve-ref-optional-result'
Code clean-up.

* rs/resolve-ref-optional-result:
  refs: pass NULL to resolve_ref_unsafe() if hash is not needed
  refs: pass NULL to refs_resolve_ref_unsafe() if hash is not needed
  refs: make sha1 output parameter of refs_resolve_ref_unsafe() optional
2017-09-28 14:47:56 +09:00
Michael Haggerty
ba1c052fa6 ref_store: implement refs_peel_ref() generically
We're about to stop storing packed refs in a `ref_cache`. That means
that the only way we have left to optimize `peel_ref()` is by checking
whether the reference being peeled is the one currently being iterated
over (in `current_ref_iter`), and if so, using `ref_iterator_peel()`.
But this can be done generically; it doesn't have to be implemented
per-backend.

So implement `refs_peel_ref()` in `refs.c` and remove the `peel_ref()`
method from the refs API.

This removes the last callers of a couple of functions, so delete
them. More cleanup to come...

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-25 18:02:46 +09:00
Junio C Hamano
d019010559 Merge branch 'tg/refs-allowed-flags'
API error-proofing which happens to also squelch warnings from GCC.

* tg/refs-allowed-flags:
  refs: strip out not allowed flags from ref_transaction_update
2017-09-25 15:24:07 +09:00
Junio C Hamano
ceb7a01aac Merge branch 'jn/per-repo-object-store-fixes'
Step #0 of a planned & larger series to make the in-core object
store per in-core repository object.

* jn/per-repo-object-store-fixes:
  replace-objects: evaluate replacement refs without using the object store
  push, fetch: error out for submodule entries not pointing to commits
  pack: make packed_git_mru global a value instead of a pointer
2017-09-25 15:24:07 +09:00
Junio C Hamano
c50424a6f0 Merge branch 'jk/write-in-full-fix'
Many codepaths did not diagnose write failures correctly when disks
go full, due to their misuse of write_in_full() helper function,
which have been corrected.

* jk/write-in-full-fix:
  read_pack_header: handle signed/unsigned comparison in read result
  config: flip return value of store_write_*()
  notes-merge: use ssize_t for write_in_full() return value
  pkt-line: check write_in_full() errors against "< 0"
  convert less-trivial versions of "write_in_full() != len"
  avoid "write_in_full(fd, buf, len) != len" pattern
  get-tar-commit-id: check write_in_full() return against 0
  config: avoid "write_in_full(fd, buf, len) < len" pattern
2017-09-25 15:24:06 +09:00
René Scharfe
744c040b19 refs: pass NULL to resolve_ref_unsafe() if hash is not needed
This allows us to get rid of some write-only variables, among them seven
SHA1 buffers.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 10:18:21 +09:00
René Scharfe
54fad6614f refs: make sha1 output parameter of refs_resolve_ref_unsafe() optional
Allow callers of refs_resolve_ref_unsafe() to pass NULL if they don't
need the resolved hash value.  We already allow the same for the flags
parameter.  This new leniency is inherited by the various wrappers like
resolve_ref_unsafe().

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 10:18:18 +09:00
Junio C Hamano
d811ba1897 Merge branch 'rs/strbuf-leakfix'
Many leaks of strbuf have been fixed.

* rs/strbuf-leakfix: (34 commits)
  wt-status: release strbuf after use in wt_longstatus_print_tracking()
  wt-status: release strbuf after use in read_rebase_todolist()
  vcs-svn: release strbuf after use in end_revision()
  utf8: release strbuf on error return in strbuf_utf8_replace()
  userdiff: release strbuf after use in userdiff_get_textconv()
  transport-helper: release strbuf after use in process_connect_service()
  sequencer: release strbuf after use in save_head()
  shortlog: release strbuf after use in insert_one_record()
  sha1_file: release strbuf on error return in index_path()
  send-pack: release strbuf on error return in send_pack()
  remote: release strbuf after use in set_url()
  remote: release strbuf after use in migrate_file()
  remote: release strbuf after use in read_remote_branches()
  refs: release strbuf on error return in write_pseudoref()
  notes: release strbuf after use in notes_copy_from_stdin()
  merge: release strbuf after use in write_merge_heads()
  merge: release strbuf after use in save_state()
  mailinfo: release strbuf on error return in handle_boundary()
  mailinfo: release strbuf after use in handle_from()
  help: release strbuf on error return in exec_woman_emacs()
  ...
2017-09-19 10:47:57 +09:00
Junio C Hamano
8a044c7f1d Merge branch 'nd/prune-in-worktree'
"git gc" and friends when multiple worktrees are used off of a
single repository did not consider the index and per-worktree refs
of other worktrees as the root for reachability traversal, making
objects that are in use only in other worktrees to be subject to
garbage collection.

* nd/prune-in-worktree:
  refs.c: reindent get_submodule_ref_store()
  refs.c: remove fallback-to-main-store code get_submodule_ref_store()
  rev-list: expose and document --single-worktree
  revision.c: --reflog add HEAD reflog from all worktrees
  files-backend: make reflog iterator go through per-worktree reflog
  revision.c: --all adds HEAD from all worktrees
  refs: remove dead for_each_*_submodule()
  refs.c: move for_each_remote_ref_submodule() to submodule.c
  revision.c: use refs_for_each*() instead of for_each_*_submodule()
  refs: add refs_head_ref()
  refs: move submodule slash stripping code to get_submodule_ref_store
  refs.c: refactor get_submodule_ref_store(), share common free block
  revision.c: --indexed-objects add objects from all worktrees
  revision.c: refactor add_index_objects_to_pending()
  refs.c: use is_dir_sep() in resolve_gitlink_ref()
  revision.h: new flag in struct rev_info wrt. worktree-related refs
2017-09-19 10:47:53 +09:00
Michael Haggerty
8738a8a4df ref_iterator: keep track of whether the iterator output is ordered
References are iterated over in order by refname, but reflogs are not.
Some consumers of reference iteration care about the difference. Teach
each `ref_iterator` to keep track of whether its output is ordered.

`overlay_ref_iterator` is one of the picky consumers. Add a sanity
check in `overlay_ref_iterator_begin()` to verify that its inputs are
ordered.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-14 15:19:07 +09:00
Jeff King
06f46f237a avoid "write_in_full(fd, buf, len) != len" pattern
The return value of write_in_full() is either "-1", or the
requested number of bytes[1]. If we make a partial write
before seeing an error, we still return -1, not a partial
value. This goes back to f6aa66cb95 (write_in_full: really
write in full or return error on disk full., 2007-01-11).

So checking anything except "was the return value negative"
is pointless. And there are a couple of reasons not to do
so:

  1. It can do a funny signed/unsigned comparison. If your
     "len" is signed (e.g., a size_t) then the compiler will
     promote the "-1" to its unsigned variant.

     This works out for "!= len" (unless you really were
     trying to write the maximum size_t bytes), but is a
     bug if you check "< len" (an example of which was fixed
     recently in config.c).

     We should avoid promoting the mental model that you
     need to check the length at all, so that new sites are
     not tempted to copy us.

  2. Checking for a negative value is shorter to type,
     especially when the length is an expression.

  3. Linus says so. In d34cf19b89 (Clean up write_in_full()
     users, 2007-01-11), right after the write_in_full()
     semantics were changed, he wrote:

       I really wish every "write_in_full()" user would just
       check against "<0" now, but this fixes the nasty and
       stupid ones.

     Appeals to authority aside, this makes it clear that
     writing it this way does not have an intentional
     benefit. It's a historical curiosity that we never
     bothered to clean up (and which was undoubtedly
     cargo-culted into new sites).

So let's convert these obviously-correct cases (this
includes write_str_in_full(), which is just a wrapper for
write_in_full()).

[1] A careful reader may notice there is one way that
    write_in_full() can return a different value. If we ask
    write() to write N bytes and get a return value that is
    _larger_ than N, we could return a larger total. But
    besides the fact that this would imply a totally broken
    version of write(), it would already invoke undefined
    behavior. Our internal remaining counter is an unsigned
    size_t, which means that subtracting too many byte will
    wrap it around to a very large number. So we'll instantly
    begin reading off the end of the buffer, trying to write
    gigabytes (or petabytes) of data.

Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-14 15:17:59 +09:00
Stefan Beller
006f3f28af replace-objects: evaluate replacement refs without using the object store
Pass DO_FOR_EACH_INCLUDE_BROKEN when iterating over replacement refs
so that the iteration does not require opening the named objects from
the object store. This avoids a dependency cycle between object access
and replace ref iteration.

Moreover the ref subsystem has not been migrated yet to access the
object store via passed in repository objects.  As a result, without
this patch, iterating over replace refs in a repository other than
the_repository it produces errors:

   error: refs/replace/3afabef75c627b894cccc3bcae86837abc7c32fe does not point to a valid object!

Noticed while adapting the object store (and in particular its
evaluation of replace refs) to handle arbitrary repositories.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-14 15:05:54 +09:00
Thomas Gummerer
c788c54cde refs: strip out not allowed flags from ref_transaction_update
Callers are only allowed to pass certain flags into
ref_transaction_update, other flags are internal to it.  To prevent
mistakes from the callers, strip the internal only flags out before
continuing.

This was noticed because of a compiler warning gcc 7.1.1 issued about
passing a NULL parameter as second parameter to memcpy (through
hashcpy):

In file included from refs.c:5:0:
refs.c: In function ‘ref_transaction_verify’:
cache.h:948:2: error: argument 2 null where non-null expected [-Werror=nonnull]
  memcpy(sha_dst, sha_src, GIT_SHA1_RAWSZ);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from git-compat-util.h:165:0,
                 from cache.h:4,
                 from refs.c:5:
/usr/include/string.h:43:14: note: in a call to function ‘memcpy’ declared here
 extern void *memcpy (void *__restrict __dest, const void *__restrict __src,
              ^~~~~~

The call to hascpy in ref_transaction_add_update is protected by the
passed in flags, but as we only add flags there, gcc notices
REF_HAVE_NEW or REF_HAVE_OLD flags could be passed in from the outside,
which would potentially result in passing in NULL as second parameter to
memcpy.

Fix both the compiler warning, and make the interface safer for its
users by stripping the internal flags out.

Suggested-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-14 14:46:05 +09:00
Rene Scharfe
aeb014f6ae refs: release strbuf on error return in write_pseudoref()
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-07 08:49:28 +09:00
Junio C Hamano
f2dd90fc1c Merge branch 'mh/ref-lock-entry'
The code to acquire a lock on a reference (e.g. while accepting a
push from a client) used to immediately fail when the reference is
already locked---now it waits for a very short while and retries,
which can make it succeed if the lock holder was holding it during
a read-only operation.

* mh/ref-lock-entry:
  refs: retry acquiring reference locks for 100ms
2017-08-26 22:55:09 -07:00
Nguyễn Thái Ngọc Duy
873ea90d61 refs.c: reindent get_submodule_ref_store()
With the new "if (!submodule) return NULL;" code added in the previous
commit, we don't need to check if submodule is not NULL anymore.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:59:04 -07:00
Nguyễn Thái Ngọc Duy
82a150f27a refs.c: remove fallback-to-main-store code get_submodule_ref_store()
At this state, there are three get_submodule_ref_store() callers:

 - for_each_remote_ref_submodule()
 - handle_revision_pseudo_opt()
 - resolve_gitlink_ref()

The first two deal explicitly with submodules (and we should never fall
back to the main ref store as a result). They are only called from
submodule.c:

 - find_first_merges()
 - submodule_needs_pushing()
 - push_submodule()

The last one, as its name implies, deals only with submodules too, and
the "submodule" (path) argument must be a non-NULL, non-empty string.

So, this "if NULL or empty string" code block should never ever
trigger. And it's wrong to fall back to the main ref store
anyway. Delete it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:59:02 -07:00
Nguyễn Thái Ngọc Duy
419221c106 refs: remove dead for_each_*_submodule()
These are used in revision.c. After the last patch they are replaced
with the refs_ version. Delete them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:56:28 -07:00
Nguyễn Thái Ngọc Duy
2e2d4040bd refs.c: move for_each_remote_ref_submodule() to submodule.c
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:56:10 -07:00
Nguyễn Thái Ngọc Duy
073cf63c52 revision.c: use refs_for_each*() instead of for_each_*_submodule()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:52:33 -07:00
Nguyễn Thái Ngọc Duy
62f0b399e0 refs: add refs_head_ref()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:47:31 -07:00
Nguyễn Thái Ngọc Duy
29babbeeb3 refs: move submodule slash stripping code to get_submodule_ref_store
This is a better place that will benefit all submodule callers instead
of just resolve_gitlink_ref()

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:46:50 -07:00
Nguyễn Thái Ngọc Duy
2c616c172d refs.c: refactor get_submodule_ref_store(), share common free block
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:46:02 -07:00
Nguyễn Thái Ngọc Duy
ee394bd376 refs.c: use is_dir_sep() in resolve_gitlink_ref()
The "submodule" argument in this function is a path, which can have
either '/' or '\\' as a separator. Use is_dir_sep() to support both.

Noticed-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:42:24 -07:00
Junio C Hamano
d0dffcacf3 Merge branch 'cc/ref-is-hidden-microcleanup' into maint
Code cleanup.

* cc/ref-is-hidden-microcleanup:
  refs: use skip_prefix() in ref_is_hidden()
2017-08-23 14:33:50 -07:00
Michael Haggerty
4ff0f01cb7 refs: retry acquiring reference locks for 100ms
The philosophy of reference locking has been, "if another process is
changing a reference, then whatever I'm trying to do to it will
probably fail anyway because my old-SHA-1 value is probably no longer
current". But this argument falls down if the other process has locked
the reference to do something that doesn't actually change the value
of the reference, such as `pack-refs` or `reflog expire`. There
actually *is* a decent chance that a planned reference update will
still be able to go through after the other process has released the
lock.

So when trying to lock an individual reference (e.g., when creating
"refs/heads/master.lock"), if it is already locked, then retry the
lock acquisition for approximately 100 ms before giving up. This
should eliminate some unnecessary lock conflicts without wasting a lot
of time.

Add a configuration setting, `core.filesRefLockTimeout`, to allow this
setting to be tweaked.

Note: the function `get_files_ref_lock_timeout_ms()` cannot be private
to the files backend because it is also used by `write_pseudoref()`
and `delete_pseudoref()`, which are defined in `refs.c` so that they
can be used by other reference backends.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23 10:37:21 -07:00
Junio C Hamano
44c2339e55 Merge branch 'mh/packed-ref-store'
The "ref-store" code reorganization continues.

* mh/packed-ref-store: (32 commits)
  files-backend: cheapen refname_available check when locking refs
  packed_ref_store: handle a packed-refs file that is a symlink
  read_packed_refs(): die if `packed-refs` contains bogus data
  t3210: add some tests of bogus packed-refs file contents
  repack_without_refs(): don't lock or unlock the packed refs
  commit_packed_refs(): remove call to `packed_refs_unlock()`
  clear_packed_ref_cache(): don't protest if the lock is held
  packed_refs_unlock(), packed_refs_is_locked(): new functions
  packed_refs_lock(): report errors via a `struct strbuf *err`
  packed_refs_lock(): function renamed from lock_packed_refs()
  commit_packed_refs(): use a staging file separate from the lockfile
  commit_packed_refs(): report errors rather than dying
  packed_ref_store: make class into a subclass of `ref_store`
  packed-backend: new module for handling packed references
  packed_read_raw_ref(): new function, replacing `resolve_packed_ref()`
  packed_ref_store: support iteration
  packed_peel_ref(): new function, extracted from `files_peel_ref()`
  repack_without_refs(): take a `packed_ref_store *` parameter
  get_packed_ref(): take a `packed_ref_store *` parameter
  rollback_packed_refs(): take a `packed_ref_store *` parameter
  ...
2017-08-22 10:29:16 -07:00
Junio C Hamano
afb456a383 Merge branch 'cc/ref-is-hidden-microcleanup'
Code cleanup.

* cc/ref-is-hidden-microcleanup:
  refs: use skip_prefix() in ref_is_hidden()
2017-08-11 13:27:03 -07:00
Junio C Hamano
df422678a8 Merge branch 'bc/object-id'
Conversion from uchar[20] to struct object_id continues.

* bc/object-id:
  sha1_name: convert uses of 40 to GIT_SHA1_HEXSZ
  sha1_name: convert GET_SHA1* flags to GET_OID*
  sha1_name: convert get_sha1* to get_oid*
  Convert remaining callers of get_sha1 to get_oid.
  builtin/unpack-file: convert to struct object_id
  bisect: convert bisect_checkout to struct object_id
  builtin/update_ref: convert to struct object_id
  sequencer: convert to struct object_id
  remote: convert struct push_cas to struct object_id
  submodule: convert submodule config lookup to use object_id
  builtin/merge-tree: convert remaining caller of get_sha1 to object_id
  builtin/fsck: convert remaining caller of get_sha1 to object_id
2017-08-11 13:26:55 -07:00
Christian Couder
7a40a95eb4 refs: use skip_prefix() in ref_is_hidden()
This is shorter, makes the logic a bit easier to follow, and is
perhaps a bit faster too.

The logic is to make the final decision only when "subject" is there,
its early part matches "match", and the match is at the slash
boundary (or the whole thing).

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-24 14:48:59 -07:00
brian m. carlson
321c89bf5f sha1_name: convert GET_SHA1* flags to GET_OID*
Convert the flags for get_oid_with_context and friends to use "OID"
instead of "SHA1" in their names.

This transform was made by running the following one-liner on the
affected files:

  perl -pi -e 's/GET_SHA1/GET_OID/g'

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17 13:54:51 -07:00
Junio C Hamano
91f6922544 Merge branch 'sb/hashmap-customize-comparison'
Update the hashmap API so that data to customize the behaviour of
the comparison function can be specified at the time a hashmap is
initialized.

* sb/hashmap-customize-comparison:
  hashmap: migrate documentation from Documentation/technical into header
  patch-ids.c: use hashmap correctly
  hashmap.h: compare function has access to a data field
2017-07-13 16:14:54 -07:00
Junio C Hamano
0c6435a4d6 Merge branch 'ab/wildmatch'
Minor code cleanup.

* ab/wildmatch:
  wildmatch: remove unused wildopts parameter
2017-07-10 13:42:51 -07:00
Stefan Beller
7663cdc86c hashmap.h: compare function has access to a data field
When using the hashmap a common need is to have access to caller provided
data in the compare function. A couple of times we abuse the keydata field
to pass in the data needed. This happens for example in patch-ids.c.

This patch changes the function signature of the compare function
to have one more void pointer available. The pointer given for each
invocation of the compare function must be defined in the init function
of the hashmap and is just passed through.

Documentation of this new feature is deferred to a later patch.
This is a rather mechanical conversion, just adding the new pass-through
parameter.  However while at it improve the naming of the fields of all
compare functions used by hashmaps by ensuring unused parameters are
prefixed with 'unused_' and naming the parameters what they are (instead
of 'unused' make it 'unused_keydata').

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-30 12:49:28 -07:00
Junio C Hamano
5c83d850d0 Merge branch 'mh/packed-ref-store-prep'
Bugfix for a topic that is (only) in 'master'.

* mh/packed-ref-store-prep:
  for_each_bisect_ref(): don't trim refnames
  lock_packed_refs(): fix cache validity check
2017-06-26 14:09:29 -07:00
Junio C Hamano
f31d23a399 Merge branch 'bw/config-h'
Fix configuration codepath to pay proper attention to commondir
that is used in multi-worktree situation, and isolate config API
into its own header file.

* bw/config-h:
  config: don't implicitly use gitdir or commondir
  config: respect commondir
  setup: teach discover_git_directory to respect the commondir
  config: don't include config.h by default
  config: remove git_config_iter
  config: create config.h
2017-06-24 14:28:41 -07:00
Ævar Arnfjörð Bjarmason
55d3426929 wildmatch: remove unused wildopts parameter
Remove the unused wildopts placeholder struct from being passed to all
wildmatch() invocations, or rather remove all the boilerplate NULL
parameters.

This parameter was added back in commit 9b3497cab9 ("wildmatch: rename
constants and update prototype", 2013-01-01) as a placeholder for
future use. Over 4 years later nothing has made use of it, let's just
remove it. It can be added in the future if we find some reason to
start using such a parameter.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 18:27:07 -07:00
Michael Haggerty
67be7c5a59 packed-backend: new module for handling packed references
Now that the interface between `files_ref_store` and
`packed_ref_store` is relatively narrow, move the latter into a new
module, "refs/packed-backend.h" and "refs/packed-backend.c". It still
doesn't quite implement the `ref_store` interface, but it will soon.

This commit moves code around and adjusts its visibility, but doesn't
change anything.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 13:27:32 -07:00
Michael Haggerty
03df567fbf for_each_bisect_ref(): don't trim refnames
`for_each_bisect_ref()` is called by `for_each_bad_bisect_ref()` with
a term "bad". This used to make it call `for_each_ref_in_submodule()`
with a prefix "refs/bisect/bad". But the latter is the name of the
reference that is being sought, so the empty string was being passed
to the callback as the trimmed refname. Moreover, this questionable
practice was turned into an error by

    b9c8e7f2fb prefix_ref_iterator: don't trim too much, 2017-05-22

It makes more sense (and agrees better with the documentation of
`--bisect`) for the callers to receive the full reference names. So

* Add a new function, `for_each_fullref_in_submodule()`, to the refs
  API. This plugs a gap in the existing functionality, analogous to
  `for_each_fullref_in()` but accepting a `submodule` argument.

* Change `for_each_bad_bisect_ref()` to call the new function rather
  than `for_each_ref_in_submodule()`.

* Add a test.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-18 22:13:42 -07:00
Sahil Dua
52d59cc645 branch: add a --copy (-c) option to go with --move (-m)
Add the ability to --copy a branch and its reflog and configuration,
this uses the same underlying machinery as the --move (-m) option
except the reflog and configuration is copied instead of being moved.

This is useful for e.g. copying a topic branch to a new version,
e.g. work to work-2 after submitting the work topic to the list, while
preserving all the tracking info and other configuration that goes
with the branch, and unlike --move keeping the other already-submitted
branch around for reference.

Like --move, when the source branch is the currently checked out
branch the HEAD is moved to the destination branch. In the case of
--move we don't really have a choice (other than remaining on a
detached HEAD) and in order to keep the functionality consistent, we
are doing it in similar way for --copy too.

The most common usage of this feature is expected to be moving to a
new topic branch which is a copy of the current one, in that case
moving to the target branch is what the user wants, and doesn't
unexpectedly behave differently than --move would.

One outstanding caveat of this implementation is that:

    git checkout maint &&
    git checkout master &&
    git branch -c topic &&
    git checkout -

Will check out 'maint' instead of 'master'. This is because the @{-N}
feature (or its -1 shorthand "-") relies on HEAD reflogs created by
the checkout command, so in this case we'll checkout maint instead of
master, as the user might expect. What to do about that is left to a
future change.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Sahil Dua <sahildua2305@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-18 21:47:59 -07:00
Brandon Williams
b2141fc1d2 config: don't include config.h by default
Stop including config.h by default in cache.h.  Instead only include
config.h in those files which require use of the config system.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15 12:56:22 -07:00
Michael Haggerty
0a0865b8f1 refs_ref_iterator_begin(): handle GIT_REF_PARANOIA
Instead of handling `GIT_REF_PARANOIA` in
`files_ref_iterator_begin()`, handle it in
`refs_ref_iterator_begin()`, where it will cover all reference stores.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:56 +09:00
Michael Haggerty
8556f8d613 ref_update_reject_duplicates(): add a sanity check
It's pretty cheap to make sure that the caller didn't pass us an
unsorted list by accident, so do so.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:55 +09:00
Michael Haggerty
a552e50e5a ref_update_reject_duplicates(): use size_t rather than int
Eliminate a theoretical risk of integer overflow if the two types have
different sizes.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:55 +09:00
Michael Haggerty
2ced105cb1 ref_update_reject_duplicates(): expose function to whole refs module
It will soon have some other users.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:55 +09:00
Michael Haggerty
30173b8851 ref_transaction_prepare(): new optional step for reference updates
In the future, compound reference stores will sometimes need to modify
references in two different reference stores at the same time, meaning
that a single logical reference transaction might have to be
implemented as two internal sub-transactions. They won't want to call
`ref_transaction_commit()` for the two sub-transactions one after the
other, because that wouldn't be atomic (the first commit could succeed
and the second one fail). Instead, they will want to prepare both
sub-transactions (i.e., obtain any necessary locks and do any
pre-checks), and only if both prepare steps succeed, then commit both
sub-transactions.

Start preparing for that day by adding a new, optional
`ref_transaction_prepare()` step to the reference transaction
sequence, which obtains the locks and does any prechecks, reporting
any errors that occur. Also add a `ref_transaction_abort()` function
that can be used to abort a sub-transaction even if it has already
been prepared.

That is on the side of the public-facing API. On the side of the
`ref_store` VTABLE, get rid of `transaction_commit` and instead add
methods `transaction_prepare`, `transaction_finish`, and
`transaction_abort`. A `ref_transaction_commit()` now basically calls
methods `transaction_prepare` then `transaction_finish`.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:55 +09:00
Michael Haggerty
8d4240d3c8 ref_transaction_commit(): check for valid transaction->state
Move the check that `transaction->state` is valid from
`files_transaction_commit()` to `ref_transaction_commit()`, where
other future reference backends can benefit from it as well.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:55 +09:00
Michael Haggerty
64da41993a ref_store: take a msg parameter when deleting references
Just because the files backend can't retain reflogs for deleted
references is no reason that they shouldn't be supported by the
virtual method interface. Also, `delete_ref()` and `refs_delete_ref()`
have already gained `msg` parameters. Now let's add them to
`delete_refs()` and `refs_delete_refs()`.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:53 +09:00
Michael Haggerty
43a2dfde76 refs: use size_t indexes when iterating over ref transaction updates
Eliminate any chance of integer overflow on platforms where the two
types have different sizes.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:53 +09:00
Michael Haggerty
c759971816 refs_ref_iterator_begin(): don't check prefixes redundantly
The backend already correctly restricts its output to references whose
names start with the prefix. By passing the prefix again to
`prefix_ref_iterator`, we were forcing that iterator to do redundant
prefix comparisons. So set it to the empty string.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-23 14:29:53 +09:00
Junio C Hamano
ca7b2ab07d Merge branch 'bc/object-id'
* bc/object-id: (53 commits)
  object: convert parse_object* to take struct object_id
  tree: convert parse_tree_indirect to struct object_id
  sequencer: convert do_recursive_merge to struct object_id
  diff-lib: convert do_diff_cache to struct object_id
  builtin/ls-tree: convert to struct object_id
  merge: convert checkout_fast_forward to struct object_id
  sequencer: convert fast_forward_to to struct object_id
  builtin/ls-files: convert overlay_tree_on_cache to object_id
  builtin/read-tree: convert to struct object_id
  sha1_name: convert internals of peel_onion to object_id
  upload-pack: convert remaining parse_object callers to object_id
  revision: convert remaining parse_object callers to object_id
  revision: rename add_pending_sha1 to add_pending_oid
  http-push: convert process_ls_object and descendants to object_id
  refs/files-backend: convert many internals to struct object_id
  refs: convert struct ref_update to use struct object_id
  ref-filter: convert some static functions to struct object_id
  Convert struct ref_array_item to struct object_id
  Convert the verify_pack callback to struct object_id
  Convert lookup_tag to struct object_id
  ...
2017-05-23 14:29:19 +09:00
Junio C Hamano
b15667bbdc Merge branch 'js/larger-timestamps'
Some platforms have ulong that is smaller than time_t, and our
historical use of ulong for timestamp would mean they cannot
represent some timestamp that the platform allows.  Invent a
separate and dedicated timestamp_t (so that we can distingiuish
timestamps and a vanilla ulongs, which along is already a good
move), and then declare uintmax_t is the type to be used as the
timestamp_t.

* js/larger-timestamps:
  archive-tar: fix a sparse 'constant too large' warning
  use uintmax_t for timestamps
  date.c: abort if the system time cannot handle one of our timestamps
  timestamp_t: a new data type for timestamps
  PRItime: introduce a new "printf format" for timestamps
  parse_timestamp(): specify explicitly where we parse timestamps
  t0006 & t5000: skip "far in the future" test when time_t is too limited
  t0006 & t5000: prepare for 64-bit timestamps
  ref-filter: avoid using `unsigned long` for catch-all data type
2017-05-16 11:51:59 +09:00
Junio C Hamano
4b44b7b1df Merge branch 'nd/worktree-kill-parse-ref'
"git gc" did not interact well with "git worktree"-managed
per-worktree refs.

* nd/worktree-kill-parse-ref:
  refs: kill set_worktree_head_symref()
  worktree.c: kill parse_ref() in favor of refs_resolve_ref_unsafe()
  refs: introduce get_worktree_ref_store()
  refs: add REFS_STORE_ALL_CAPS
  refs.c: make submodule ref store hashmap generic
  environment.c: fix potential segfault by get_git_common_dir()
2017-05-16 11:51:51 +09:00
brian m. carlson
984912989d refs: convert struct ref_update to use struct object_id
Convert struct ref_array_item to use struct object_id by changing the
definition and applying the following semantic patch, plus the standard
object_id transforms:

@@
struct ref_update E1;
@@
- E1.new_sha1
+ E1.new_oid.hash

@@
struct ref_update *E1;
@@
- E1->new_sha1
+ E1->new_oid.hash

@@
struct ref_update E1;
@@
- E1.old_sha1
+ E1.old_oid.hash

@@
struct ref_update *E1;
@@
- E1->old_sha1
+ E1->old_oid.hash

This transformation allows us to convert write_ref_to_lockfile, which is
required to convert parse_object.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 15:12:58 +09:00
Johannes Schindelin
dddbad728c timestamp_t: a new data type for timestamps
Git's source code assumes that unsigned long is at least as precise as
time_t. Which is incorrect, and causes a lot of problems, in particular
where unsigned long is only 32-bit (notably on Windows, even in 64-bit
versions).

So let's just use a more appropriate data type instead. In preparation
for this, we introduce the new `timestamp_t` data type.

By necessity, this is a very, very large patch, as it has to replace all
timestamps' data type in one go.

As we will use a data type that is not necessarily identical to `time_t`,
we need to be very careful to use `time_t` whenever we interact with the
system functions, and `timestamp_t` everywhere else.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27 13:07:39 +09:00
Junio C Hamano
77b34eaa07 Merge branch 'mh/separate-ref-cache'
The internals of the refs API around the cached refs has been
streamlined.

* mh/separate-ref-cache:
  do_for_each_entry_in_dir(): delete function
  files_pack_refs(): use reference iteration
  commit_packed_refs(): use reference iteration
  cache_ref_iterator_begin(): make function smarter
  get_loose_ref_cache(): new function
  get_loose_ref_dir(): function renamed from get_loose_refs()
  do_for_each_entry_in_dir(): eliminate `offset` argument
  refs: handle "refs/bisect/" in `loose_fill_ref_dir()`
  ref-cache: use a callback function to fill the cache
  refs: record the ref_store in ref_cache, not ref_dir
  ref-cache: introduce a new type, ref_cache
  refs: split `ref_cache` code into separate files
  ref-cache: rename `remove_entry()` to `remove_entry_from_dir()`
  ref-cache: rename `find_ref()` to `find_ref_entry()`
  ref-cache: rename `add_ref()` to `add_ref_entry()`
  refs_verify_refname_available(): use function in more places
  refs_verify_refname_available(): implement once for all backends
  refs_ref_iterator_begin(): new function
  refs_read_raw_ref(): new function
  get_ref_dir(): don't call read_loose_refs() for "refs/bisect"
2017-04-26 15:39:13 +09:00
Nguyễn Thái Ngọc Duy
17eff96b83 refs: introduce get_worktree_ref_store()
files-backend at this point is still aware of the per-repo/worktree
separation in refs, so it can handle a linked worktree.

Some refs operations are known not working when current files-backend is
used in a linked worktree (e.g. reflog). Tests will be written when
refs_* functions start to be called with worktree backend to verify that
they work as expected.

Note: accessing a worktree of a submodule remains unaddressed. Perhaps
after get_worktrees() can access submodule (or rather a new function
get_submodule_worktrees(), that lists worktrees of a submodule), we can
update this function to work with submodules as well.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-24 21:28:55 -07:00
Nguyễn Thái Ngọc Duy
0d8a814d8a refs: add REFS_STORE_ALL_CAPS
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-24 21:28:55 -07:00
Junio C Hamano
9f1384f711 Merge branch 'jk/quarantine-received-objects'
Add finishing touches to a recent topic.

* jk/quarantine-received-objects:
  refs: reject ref updates while GIT_QUARANTINE_PATH is set
  receive-pack: document user-visible quarantine effects
  receive-pack: drop tmp_objdir_env from run_update_hook
2017-04-23 22:07:52 -07:00
Junio C Hamano
5ab8f2261f Merge branch 'nd/files-backend-git-dir'
The "submodule" specific field in the ref_store structure is
replaced with a more generic "gitdir" that can later be used also
when dealing with ref_store that represents the set of refs visible
from the other worktrees.

* nd/files-backend-git-dir: (28 commits)
  refs.h: add a note about sorting order of for_each_ref_*
  t1406: new tests for submodule ref store
  t1405: some basic tests on main ref store
  t/helper: add test-ref-store to test ref-store functions
  refs: delete pack_refs() in favor of refs_pack_refs()
  files-backend: avoid ref api targeting main ref store
  refs: new transaction related ref-store api
  refs: add new ref-store api
  refs: rename get_ref_store() to get_submodule_ref_store() and make it public
  files-backend: replace submodule_allowed check in files_downcast()
  refs: move submodule code out of files-backend.c
  path.c: move some code out of strbuf_git_path_submodule()
  refs.c: make get_main_ref_store() public and use it
  refs.c: kill register_ref_store(), add register_submodule_ref_store()
  refs.c: flatten get_ref_store() a bit
  refs: rename lookup_ref_store() to lookup_submodule_ref_store()
  refs.c: introduce get_main_ref_store()
  files-backend: remove the use of git_path()
  files-backend: add and use files_ref_path()
  files-backend: add and use files_reflog_path()
  ...
2017-04-19 21:37:19 -07:00
Michael Haggerty
b05855b5bc refs_verify_refname_available(): implement once for all backends
It turns out that we can now implement
`refs_verify_refname_available()` based on the other virtual
functions, so there is no need for it to be defined at the backend
level. Instead, define it once in `refs.c` and remove the
`files_backend` definition.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-16 21:32:45 -07:00
Nguyễn Thái Ngọc Duy
0c064d907b refs.c: make submodule ref store hashmap generic
This removes the "submodule" from submodule_hash_entry and other
function names. The goal is to reuse the same code and data structure
for other ref store types. The first one is worktree ref stores.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-16 18:24:29 -07:00
Jeff King
d8f4481c4f refs: reject ref updates while GIT_QUARANTINE_PATH is set
As documented in git-receive-pack(1), updating a ref from
within the pre-receive hook is dangerous and can corrupt
your repo. This patch forbids ref updates entirely during
the hook to make it harder for adventurous hook writers to
shoot themselves in the foot.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-16 18:19:18 -07:00
Michael Haggerty
e121b9cb5f refs_ref_iterator_begin(): new function
Extract a new function from `do_for_each_ref()`. It will be useful
elsewhere.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:54:31 -07:00
Michael Haggerty
470be51862 refs_read_raw_ref(): new function
Extract a new function from `refs_resolve_ref_unsafe()`. It will be
useful elsewhere.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:54:31 -07:00
Nguyễn Thái Ngọc Duy
7c744782ab refs: delete pack_refs() in favor of refs_pack_refs()
It only has one caller, not worth keeping just for convenience.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:53:25 -07:00
Nguyễn Thái Ngọc Duy
c0fe4e8ba3 refs: new transaction related ref-store api
The transaction struct now takes a ref store at creation and will
operate on that ref store alone.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:53:25 -07:00
Nguyễn Thái Ngọc Duy
7d2df051d0 refs: add new ref-store api
This is not meant to cover all existing API. It adds enough to test ref
stores with the new test program test-ref-store, coming soon and to be
used by files-backend.c.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:53:25 -07:00
Nguyễn Thái Ngọc Duy
18d0002d6d refs: rename get_ref_store() to get_submodule_ref_store() and make it public
This function is intended to replace *_submodule() refs API. It provides
a ref store for a specific submodule, which can be operated on by a new
set of refs API.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:53:25 -07:00
Nguyễn Thái Ngọc Duy
9e7ec634a1 files-backend: replace submodule_allowed check in files_downcast()
files-backend.c is unlearning submodules. Instead of having a specific
check for submodules to see what operation is allowed, files backend
now takes a set of flags at init. Each operation will check if the
required flags is present before performing.

For now we have four flags: read, write and odb access. Main ref store
has all flags, obviously, while submodule stores are read-only and have
access to odb (*).

The "main" flag stays because many functions in the backend calls
frontend ones without a ref store, so these functions always target the
main ref store. Ideally the flag should be gone after ref-store-aware
api is in place and used by backends.

(*) Submodule code needs for_each_ref. Try take REF_STORE_ODB flag
out. At least t3404 would fail. The "have access to odb" in submodule is
a bit hacky since we don't know from he whether add_submodule_odb() has
been called.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-14 03:53:18 -07:00
Jeff King
6cd4a8982d avoid using mksnpath for refs
Like the previous commit, we'd like to avoid the assumption
that refs fit into PATH_MAX-sized buffers. These callsites
have an extra twist, though: they write the refnames using
mksnpath. This does two things beyond a regular snprintf:

  1. It quietly writes "/bad-path/" when truncation occurs.
     This saves the caller having to check the error code,
     but if you aren't actually feeding the result to a
     system call (and we aren't here), it's questionable.

  2. It calls cleanup_path(), which removes leading
     instances of "./".  That's questionable when dealing
     with refnames, as we could silently canonicalize a
     syntactically bogus refname into a valid one.

Let's convert each case to use a strbuf. This is preferable
to xstrfmt() because we can reuse the same buffer as we
loop.

Signed-off-by: Jeff King <peff@peff.net>
2017-03-30 14:59:50 -07:00
Junio C Hamano
f494890292 Merge branch 'sg/skip-prefix-in-prettify-refname'
Code cleanup.

* sg/skip-prefix-in-prettify-refname:
  refs.c: use skip_prefix() in prettify_refname()
2017-03-28 14:05:59 -07:00
Junio C Hamano
41534b626e Merge branch 'jk/interpret-branch-name' into maint
"git branch @" created refs/heads/@ as a branch, and in general the
code that handled @{-1} and @{upstream} was a bit too loose in
disambiguating.

* jk/interpret-branch-name:
  checkout: restrict @-expansions when finding branch
  strbuf_check_ref_format(): expand only local branches
  branch: restrict @-expansions when deleting
  t3204: test git-branch @-expansion corner cases
  interpret_branch_name: allow callers to restrict expansions
  strbuf_branchname: add docstring
  strbuf_branchname: drop return value
  interpret_branch_name: move docstring to header file
  interpret_branch_name(): handle auto-namelen for @{-1}
2017-03-28 13:52:22 -07:00
Junio C Hamano
c772d1bcdc Merge branch 'jk/parse-config-key-cleanup' into maint
The "parse_config_key()" API function has been cleaned up.

* jk/parse-config-key-cleanup:
  parse_hide_refs_config: tell parse_config_key we don't want a subsection
  parse_config_key: allow matching single-level config
  parse_config_key: use skip_prefix instead of starts_with
  refs: parse_hide_refs_config to use parse_config_key
2017-03-28 13:52:19 -07:00
Nguyễn Thái Ngọc Duy
5d0bc90e5d refs: move submodule code out of files-backend.c
files-backend is now initialized with a $GIT_DIR. Converting a submodule
path to where real submodule gitdir is located is done in get_ref_store().

This gives a slight performance improvement for submodules since we
don't convert submodule path to gitdir at every backend call like
before. We pay that once at ref-store creation.

More cleanup in files_downcast() and files_assert_main_repository()
follows shortly. It's separate to keep noises from this patch.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:40 -07:00
Nguyễn Thái Ngọc Duy
077be78d7f refs.c: make get_main_ref_store() public and use it
get_ref_store() will soon be renamed to get_submodule_ref_store().
Together with future get_worktree_ref_store(), the three functions
provide an appropriate ref store for different operation modes. New APIs
will be added to operate directly on ref stores.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:39 -07:00
Nguyễn Thái Ngọc Duy
378dc9103a refs.c: kill register_ref_store(), add register_submodule_ref_store()
This is the last function in this code (besides public API) that takes
submodule argument and handles both main/submodule cases. Break it down,
move main store registration in get_main_ref_store() and keep the rest
in register_submodule_ref_store().

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:39 -07:00
Nguyễn Thái Ngọc Duy
126c9e0576 refs.c: flatten get_ref_store() a bit
This helps the future changes in this code. And because get_ref_store()
is destined to become get_submodule_ref_store(), the "get main store"
code path will be removed eventually. After this the patch to delete
that code will be cleaner.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:39 -07:00
Nguyễn Thái Ngọc Duy
9476c6ed3d refs: rename lookup_ref_store() to lookup_submodule_ref_store()
With get_main_ref_store() being used inside get_ref_store(),
lookup_ref_store() is only used for submodule code path. Rename to
reflect that and delete dead code.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:39 -07:00
Nguyễn Thái Ngọc Duy
24c8407e0a refs.c: introduce get_main_ref_store()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:39 -07:00
SZEDER Gábor
3e5b36c637 refs.c: use skip_prefix() in prettify_refname()
This eliminates three magic numbers.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-23 11:22:01 -07:00
Junio C Hamano
e1fae93019 Merge branch 'bc/object-id'
"uchar [40]" to "struct object_id" conversion continues.

* bc/object-id:
  wt-status: convert to struct object_id
  builtin/merge-base: convert to struct object_id
  Convert object iteration callbacks to struct object_id
  sha1_file: introduce an nth_packed_object_oid function
  refs: simplify parsing of reflog entries
  refs: convert each_reflog_ent_fn to struct object_id
  reflog-walk: convert struct reflog_info to struct object_id
  builtin/replace: convert to struct object_id
  Convert remaining callers of resolve_refdup to object_id
  builtin/merge: convert to struct object_id
  builtin/clone: convert to struct object_id
  builtin/branch: convert to struct object_id
  builtin/grep: convert to struct object_id
  builtin/fmt-merge-message: convert to struct object_id
  builtin/fast-export: convert to struct object_id
  builtin/describe: convert to struct object_id
  builtin/diff-tree: convert to struct object_id
  builtin/commit: convert to struct object_id
  hex: introduce parse_oid_hex
2017-03-17 13:50:25 -07:00
Junio C Hamano
c809496c97 Merge branch 'jk/interpret-branch-name'
"git branch @" created refs/heads/@ as a branch, and in general the
code that handled @{-1} and @{upstream} was a bit too loose in
disambiguating.

* jk/interpret-branch-name:
  checkout: restrict @-expansions when finding branch
  strbuf_check_ref_format(): expand only local branches
  branch: restrict @-expansions when deleting
  t3204: test git-branch @-expansion corner cases
  interpret_branch_name: allow callers to restrict expansions
  strbuf_branchname: add docstring
  strbuf_branchname: drop return value
  interpret_branch_name: move docstring to header file
  interpret_branch_name(): handle auto-namelen for @{-1}
2017-03-14 15:23:18 -07:00
Junio C Hamano
963792ed27 Merge branch 'jk/parse-config-key-cleanup'
The "parse_config_key()" API function has been cleaned up.

* jk/parse-config-key-cleanup:
  parse_hide_refs_config: tell parse_config_key we don't want a subsection
  parse_config_key: allow matching single-level config
  parse_config_key: use skip_prefix instead of starts_with
2017-03-10 13:24:22 -08:00
Junio C Hamano
3406129900 Merge branch 'sb/parse-hide-refs-config-cleanup'
Code clean-up.

* sb/parse-hide-refs-config-cleanup:
  refs: parse_hide_refs_config to use parse_config_key
2017-03-10 13:24:21 -08:00
Jeff King
0e9f62dab9 interpret_branch_name: allow callers to restrict expansions
The interpret_branch_name() function converts names like
@{-1} and @{upstream} into branch names. The expanded ref
names are not fully qualified, and may be outside of the
refs/heads/ namespace (e.g., "@" expands to "HEAD", and
"@{upstream}" is likely to be in "refs/remotes/").

This is OK for callers like dwim_ref() which are primarily
interested in resolving the resulting name, no matter where
it is. But callers like "git branch" treat the result as a
branch name in refs/heads/.  When we expand to a ref outside
that namespace, the results are very confusing (e.g., "git
branch @" tries to create refs/heads/HEAD, which is
nonsense).

Callers can't know from the returned string how the
expansion happened (e.g., did the user really ask for a
branch named "HEAD", or did we do a bogus expansion?). One
fix would be to return some out-parameters describing the
types of expansion that occurred. This has the benefit that
the caller can generate precise error messages ("I
understood @{upstream} to mean origin/master, but that is a
remote tracking branch, so you cannot create it as a local
name").

However, out-parameters make the function interface somewhat
cumbersome. Instead, let's do the opposite: let the caller
tell us which elements to expand. That's easier to pass in,
and none of the callers give more precise error messages
than "@{upstream} isn't a valid branch name" anyway (which
should be sufficient).

The strbuf_branchname() function needs a similar parameter,
as most of the callers access interpret_branch_name()
through it.

We can break the callers down into two groups:

  1. Callers that are happy with any kind of ref in the
     result. We pass "0" here, so they continue to work
     without restrictions. This includes merge_name(),
     the reflog handling in add_pending_object_with_path(),
     and substitute_branch_name(). This last is what powers
     dwim_ref().

  2. Callers that have funny corner cases (mostly in
     git-branch and git-checkout). These need to make use of
     the new parameter, but I've left them as "0" in this
     patch, and will address them individually in follow-on
     patches.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-02 11:05:04 -08:00
Junio C Hamano
c13c783c9d Merge branch 'km/delete-ref-reflog-message'
"git update-ref -d" and other operations to delete references did
not leave any entry in HEAD's reflog when the reference being
deleted was the current branch.  This is not a problem in practice
because you do not want to delete the branch you are currently on,
but caused renaming of the current branch to something else not to
be logged in a useful way.

* km/delete-ref-reflog-message:
  branch: record creation of renamed branch in HEAD's log
  rename_ref: replace empty message in HEAD's log
  update-ref: pass reflog message to delete_ref()
  delete_ref: accept a reflog message argument
2017-02-27 13:57:18 -08:00
Junio C Hamano
466be3e1de Merge branch 'mh/submodule-hash'
Code and design clean-up for the refs API.

* mh/submodule-hash:
  read_loose_refs(): read refs using resolve_ref_recursively()
  files_ref_store::submodule: use NULL for the main repository
  base_ref_store_init(): remove submodule argument
  refs: push the submodule attribute down
  refs: store submodule ref stores in a hashmap
  register_ref_store(): new function
  refs: remove some unnecessary handling of submodule == ""
  refs: make some ref_store lookup functions private
  refs: reorder some function definitions
2017-02-27 13:57:15 -08:00
Jeff King
ad8c7cdadd parse_hide_refs_config: tell parse_config_key we don't want a subsection
This lets us avoid declaring some otherwise useless
variables.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-24 13:22:12 -08:00
Stefan Beller
49624d1e51 refs: parse_hide_refs_config to use parse_config_key
parse_config_key was introduced in 1b86bbb0ad (config: add helper
function for parsing key names, 2013-01-22), the NEEDSWORK that is removed
in this patch was introduced at daebaa7813 (upload/receive-pack: allow
hiding ref hierarchies, 2013-01-18), which is only a couple days apart,
so presumably the code replaced in this patch was only introduced due
to not wanting to wait on the proper helper function being available.

Make the condition easier to read by using parse_config_key.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-24 12:49:29 -08:00
brian m. carlson
9461d27240 refs: convert each_reflog_ent_fn to struct object_id
Make each_reflog_ent_fn take two struct object_id pointers instead of
two pointers to unsigned char.  Convert the various callbacks to use
struct object_id as well.  Also, rename fsck_handle_reflog_sha1 to
fsck_handle_reflog_oid.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-22 10:12:15 -08:00
Kyle Meyer
755b49ae96 delete_ref: accept a reflog message argument
When the current branch is renamed with 'git branch -m/-M' or deleted
with 'git update-ref -m<msg> -d', the event is recorded in HEAD's log
with an empty message.  In preparation for adding a more meaningful
message to HEAD's log in these cases, update delete_ref() to take a
message argument and pass it along to ref_transaction_delete().
Modify all callers to pass NULL for the new message argument; no
change in behavior is intended.

Note that this is relevant for HEAD's log but not for the deleted
ref's log, which is currently deleted along with the ref.  Even if it
were not, an entry for the deletion wouldn't be present in the deleted
ref's log.  files_transaction_commit() writes to the log if
REF_NEEDS_COMMIT or REF_LOG_ONLY are set, but lock_ref_for_update()
doesn't set REF_NEEDS_COMMIT for the deleted ref because REF_DELETING
is set.  In contrast, the update for HEAD has REF_LOG_ONLY set by
split_head_update(), resulting in the deletion being logged.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-20 22:04:47 -08:00