Commit Graph

521 Commits

Author SHA1 Message Date
Junio C Hamano
e05336bdda Merge branch 'bp/fsmonitor'
We learned to talk to watchman to speed up "git status" and other
operations that need to see which paths have been modified.

* bp/fsmonitor:
  fsmonitor: preserve utf8 filenames in fsmonitor-watchman log
  fsmonitor: read entirety of watchman output
  fsmonitor: MINGW support for watchman integration
  fsmonitor: add a performance test
  fsmonitor: add a sample integration script for Watchman
  fsmonitor: add test cases for fsmonitor extension
  split-index: disable the fsmonitor extension when running the split index test
  fsmonitor: add a test tool to dump the index extension
  update-index: add fsmonitor support to update-index
  ls-files: Add support in ls-files to display the fsmonitor valid bit
  fsmonitor: add documentation for the fsmonitor extension.
  fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files.
  update-index: add a new --force-write-index option
  preload-index: add override to enable testing preload-index
  bswap: add 64 bit endianness helper get_be64
2017-11-21 14:07:50 +09:00
Jameson Miller
1b2bc3912a status: document options to show matching ignored files
Signed-off-by: Jameson Miller <jamill@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-31 11:54:21 +09:00
Junio C Hamano
4af0500a51 Merge branch 'hn/string-list-doc'
Docfix.

* hn/string-list-doc:
  api-argv-array.txt: remove broken link to string-list API
2017-10-11 14:52:22 +09:00
Junio C Hamano
95649bc6f5 Merge branch 'jr/hash-migration-plan-doc'
Lay out plans for weaning us off of SHA-1.

* jr/hash-migration-plan-doc:
  technical doc: add a design doc for hash function transition
2017-10-11 14:52:22 +09:00
Todd Zullinger
204f6d6987 api-argv-array.txt: remove broken link to string-list API
In 4f665f2cf3 (string-list.h: move documentation from Documentation/api/
into header, 2017-09-26) the string-list API documentation was moved to
string-list.h.  The argv-array API documentation may follow a similar
course in the future.  Until then, prevent the broken link from making
it to the end-user documentation.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-06 21:16:03 +09:00
Ben Peart
780494b1f5 fsmonitor: add documentation for the fsmonitor extension.
This includes the core.fsmonitor setting, the fsmonitor integration hook,
and the fsmonitor index extension.

Also add documentation for the new fsmonitor options to ls-files and
update-index.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01 17:23:05 +09:00
Jonathan Nieder
752414ae43 technical doc: add a design doc for hash function transition
This document describes what a transition to a new hash function for
Git would look like.  Add it to Documentation/technical/ as the plan
of record so that future changes can be recorded as patches.

Also-by: Brandon Williams <bmwill@google.com>
Also-by: Jonathan Tan <jonathantanmy@google.com>
Also-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-28 19:37:52 +09:00
Han-Wen Nienhuys
4f665f2cf3 string-list.h: move documentation from Documentation/api/ into header
This mirrors commit 'bdfdaa497 ("strbuf.h: integrate api-strbuf.txt
documentation, 2015-01-16") which did the same for strbuf.h:

* API documentation uses /** */ to set it apart from other comments.

* Function names were stripped from the comments.

* Ordering of the header was adjusted to follow the one from the text
  file.

* Edited some existing comments from string-list.h for consistency.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-27 09:14:34 +09:00
Junio C Hamano
c39da2c08e Merge branch 'ma/remove-config-maybe-bool'
Finishing touches to a recent topic.

* ma/remove-config-maybe-bool:
  config: remove git_config_maybe_bool
2017-09-19 10:47:55 +09:00
Junio C Hamano
8a044c7f1d Merge branch 'nd/prune-in-worktree'
"git gc" and friends when multiple worktrees are used off of a
single repository did not consider the index and per-worktree refs
of other worktrees as the root for reachability traversal, making
objects that are in use only in other worktrees to be subject to
garbage collection.

* nd/prune-in-worktree:
  refs.c: reindent get_submodule_ref_store()
  refs.c: remove fallback-to-main-store code get_submodule_ref_store()
  rev-list: expose and document --single-worktree
  revision.c: --reflog add HEAD reflog from all worktrees
  files-backend: make reflog iterator go through per-worktree reflog
  revision.c: --all adds HEAD from all worktrees
  refs: remove dead for_each_*_submodule()
  refs.c: move for_each_remote_ref_submodule() to submodule.c
  revision.c: use refs_for_each*() instead of for_each_*_submodule()
  refs: add refs_head_ref()
  refs: move submodule slash stripping code to get_submodule_ref_store
  refs.c: refactor get_submodule_ref_store(), share common free block
  revision.c: --indexed-objects add objects from all worktrees
  revision.c: refactor add_index_objects_to_pending()
  refs.c: use is_dir_sep() in resolve_gitlink_ref()
  revision.h: new flag in struct rev_info wrt. worktree-related refs
2017-09-19 10:47:53 +09:00
Junio C Hamano
8e36002add Merge branch 'ma/up-to-date'
Message and doc updates.

* ma/up-to-date:
  treewide: correct several "up-to-date" to "up to date"
  Documentation/user-manual: update outdated example output
2017-09-10 17:08:22 +09:00
Martin Ågren
d389028695 config: remove git_config_maybe_bool
The function was deprecated in commit 89576613 ("treewide: deprecate
git_config_maybe_bool, use git_parse_maybe_bool", 2017-08-07) and has no
users.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-07 09:48:20 +09:00
Nguyễn Thái Ngọc Duy
419221c106 refs: remove dead for_each_*_submodule()
These are used in revision.c. After the last patch they are replaced
with the refs_ version. Delete them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:56:28 -07:00
Junio C Hamano
11bd95604a Merge branch 'rs/object-id'
Conversion from uchar[20] to struct object_id continues.

* rs/object-id:
  tree-walk: convert fill_tree_descriptor() to object_id
2017-08-24 10:20:02 -07:00
Martin Ågren
7560f547e6 treewide: correct several "up-to-date" to "up to date"
Follow the Oxford style, which says to use "up-to-date" before the noun,
but "up to date" after it. Don't change plumbing (specifically
send-pack.c, but transport.c (git push) also has the same string).

This was produced by grepping for "up-to-date" and "up to date". It
turned out we only had to edit in one direction, removing the hyphens.

Fix a typo in Documentation/git-diff-index.txt while we're there.

Reported-by: Jeffrey Manian <jeffrey.manian@gmail.com>
Reported-by: STEVEN WHITE <stevencharleswhitevoices@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23 12:17:22 -07:00
Junio C Hamano
bdfcdefd2f Merge branch 'ma/parse-maybe-bool'
Code clean-up.

* ma/parse-maybe-bool:
  parse_decoration_style: drop unused argument `var`
  treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool
  config: make git_{config,parse}_maybe_bool equivalent
  config: introduce git_parse_maybe_bool_text
  t5334: document that git push --signed=1 does not work
  Doc/git-{push,send-pack}: correct --sign= to --signed=
2017-08-22 10:29:03 -07:00
René Scharfe
5c377d3d59 tree-walk: convert fill_tree_descriptor() to object_id
All callers of fill_tree_descriptor() have been converted to object_id
already, so convert that function as well.  As a nice side-effect we get
rid of NULL checks in tree-diff.c, as fill_tree_descriptor() already
does them for us.

Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-14 12:38:54 -07:00
Junio C Hamano
297872f0c2 Merge branch 'ma/pager-per-subcommand-action'
The "tag.pager" configuration variable was useless for those who
actually create tag objects, as it interfered with the use of an
editor.  A new mechanism has been introduced for commands to enable
pager depending on what operation is being carried out to fix this,
and then "git tag -l" is made to run pager by default.

* ma/pager-per-subcommand-action:
  git.c: ignore pager.* when launching builtin as dashed external
  tag: change default of `pager.tag` to "on"
  tag: respect `pager.tag` in list-mode only
  t7006: add tests for how git tag paginates
  git.c: provide setup_auto_pager()
  git.c: let builtins opt for handling `pager.foo` themselves
  builtin.h: take over documentation from api-builtin.txt
2017-08-11 13:27:07 -07:00
Martin Ågren
8957661378 treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool
The only difference between these is that the former takes an argument
`name` which it ignores completely. Still, the callers are quite careful
to provide reasonable values for it.

Once in-flight topics have landed, we should be able to remove
git_config_maybe_bool. In the meantime, document it as deprecated in the
technical documentation. While at it, document git_parse_maybe_bool.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-07 13:29:22 -07:00
Martin Ågren
ec14d4ecb5 builtin.h: take over documentation from api-builtin.txt
Delete Documentation/technical/api-builtin.txt and move its content
into builtin.h. Format it as a comment. Remove a '+' which was needed
when the information was formatted for AsciiDoc. Similarly, change
"::" to ":".

Document SUPPORT_SUPER_PREFIX, thereby bringing the documentation up to
date with the available flags.

While at it, correct '3 more things to do' to '4 more things to do'.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03 11:08:10 -07:00
Jonathan Tan
7e2e1bbb24 Documentation: migrate sub-process docs to header
Move the documentation for the sub-process API from a separate txt file
to its header file.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-26 12:56:40 -07:00
Stefan Beller
1ecbf31d02 hashmap: migrate documentation from Documentation/technical into header
While at it, clarify the use of `key`, `keydata`, `entry_or_key` as well
as documenting the new data pointer for the compare function.

Rework the example.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-30 13:11:59 -07:00
Junio C Hamano
f4fd99bf6e Merge branch 'sl/clean-d-ignored-fix'
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored"  did not list ignored and untracked files
without "-uall".  These have been corrected.

* sl/clean-d-ignored-fix:
  clean: teach clean -d to preserve ignored paths
  dir: expose cmp_name() and check_contains()
  dir: hide untracked contents of untracked dirs
  dir: recurse into untracked dirs for ignored files
  t7061: status --ignored should search untracked dirs
  t7300: clean -d should skip dirs with ignored files
2017-06-02 15:06:05 +09:00
Junio C Hamano
ae7785de0e Merge branch 'bp/sub-process-convert-filter'
Code from "conversion using external process" codepath has been
extracted to a separate sub-process.[ch] module.

* bp/sub-process-convert-filter:
  convert: update subprocess_read_status() to not die on EOF
  sub-process: move sub-process functions into separate files
  convert: rename reusable sub-process functions
  convert: update generic functions to only use generic data structures
  convert: separate generic structures and variables from the filter specific ones
  convert: split start_multi_file_filter() into two separate functions
  pkt-line: annotate packet_writel with LAST_ARG_MUST_BE_NULL
  convert: move packet_write_line() into pkt-line as packet_writel()
  pkt-line: add packet_read_line_gently()
  pkt-line: fix packet_read_line() to handle len < 0 errors
  convert: remove erroneous tests for errno == EPIPE
2017-05-30 11:16:42 +09:00
Junio C Hamano
3c980083bc Merge branch 'jt/push-options-doc'
The receive-pack program now makes sure that the push certificate
records the same set of push options used for pushing.

* jt/push-options-doc:
  receive-pack: verify push options in cert
  docs: correct receive.advertisePushOptions default
2017-05-23 13:46:07 +09:00
Samuel Lijin
fb89888849 dir: hide untracked contents of untracked dirs
When we taught read_directory_recursive() to recurse into untracked
directories in search of ignored files given DIR_SHOW_IGNORED_TOO, that
had the side effect of teaching it to collect the untracked contents of
untracked directories. It doesn't always make sense to return these,
though (we do need them for `clean -d`), so we introduce a flag
(DIR_KEEP_UNTRACKED_CONTENTS) to control whether or not read_directory()
strips dir->entries of the untracked contents of untracked dirs.

We also introduce check_contains() to check if one dir_entry corresponds
to a path which contains the path corresponding to another dir_entry.

This also fixes known breakages in t7061, since status --ignored now
searches untracked directories for ignored files.

Signed-off-by: Samuel Lijin <sxlijin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-22 12:14:09 +09:00
Junio C Hamano
b15667bbdc Merge branch 'js/larger-timestamps'
Some platforms have ulong that is smaller than time_t, and our
historical use of ulong for timestamp would mean they cannot
represent some timestamp that the platform allows.  Invent a
separate and dedicated timestamp_t (so that we can distingiuish
timestamps and a vanilla ulongs, which along is already a good
move), and then declare uintmax_t is the type to be used as the
timestamp_t.

* js/larger-timestamps:
  archive-tar: fix a sparse 'constant too large' warning
  use uintmax_t for timestamps
  date.c: abort if the system time cannot handle one of our timestamps
  timestamp_t: a new data type for timestamps
  PRItime: introduce a new "printf format" for timestamps
  parse_timestamp(): specify explicitly where we parse timestamps
  t0006 & t5000: skip "far in the future" test when time_t is too limited
  t0006 & t5000: prepare for 64-bit timestamps
  ref-filter: avoid using `unsigned long` for catch-all data type
2017-05-16 11:51:59 +09:00
Ben Peart
99605d62e8 sub-process: move sub-process functions into separate files
Move the sub-proces functions into sub-process.h/c.  Add documentation
for the new module in Documentation/technical/api-sub-process.txt

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-15 13:01:57 +09:00
Jonathan Tan
cbaf82cc6b receive-pack: verify push options in cert
In commit f6a4e61 ("push: accept push options", 2016-07-14), send-pack
was taught to include push options both within the signed cert (if the
push is a signed push) and outside the signed cert; however,
receive-pack ignores push options within the cert, only handling push
options outside the cert.

Teach receive-pack, in the case that push options are provided for a
signed push, to verify that the push options both within the cert and
outside the cert are consistent.

This sets in stone the requirement that send-pack redundantly send its
push options in 2 places, but I think that this is better than the
alternatives. Sending push options only within the cert is
backwards-incompatible with existing Git servers (which read push
options only from outside the cert), and sending push options only
outside the cert means that the push options are not signed for.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-10 13:18:28 +09:00
Johannes Schindelin
dddbad728c timestamp_t: a new data type for timestamps
Git's source code assumes that unsigned long is at least as precise as
time_t. Which is incorrect, and causes a lot of problems, in particular
where unsigned long is only 32-bit (notably on Windows, even in 64-bit
versions).

So let's just use a more appropriate data type instead. In preparation
for this, we introduce the new `timestamp_t` data type.

By necessity, this is a very, very large patch, as it has to replace all
timestamps' data type in one go.

As we will use a data type that is not necessarily identical to `time_t`,
we need to be very careful to use `time_t` whenever we interact with the
system functions, and `timestamp_t` everywhere else.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27 13:07:39 +09:00
Junio C Hamano
d2617eb984 Merge branch 'jt/fetch-pack-error-reporting'
"git fetch-pack" was not prepared to accept ERR packet that the
upload-pack can send with a human-readable error message.  It
showed the packet contents with ERR prefix, so there was no data
loss, but it was redundant to say "ERR" in an error message.

* jt/fetch-pack-error-reporting:
  fetch-pack: show clearer error message upon ERR
2017-04-23 22:07:53 -07:00
Junio C Hamano
b1081e4004 Merge branch 'bc/object-id'
Conversion from unsigned char [40] to struct object_id continues.

* bc/object-id:
  Documentation: update and rename api-sha1-array.txt
  Rename sha1_array to oid_array
  Convert sha1_array_for_each_unique and for_each_abbrev to object_id
  Convert sha1_array_lookup to take struct object_id
  Convert remaining callers of sha1_array_lookup to object_id
  Make sha1_array_append take a struct object_id *
  sha1-array: convert internal storage for struct sha1_array to object_id
  builtin/pull: convert to struct object_id
  submodule: convert check_for_new_submodule_commits to object_id
  sha1_name: convert disambiguate_hint_fn to take object_id
  sha1_name: convert struct disambiguate_state to object_id
  test-sha1-array: convert most code to struct object_id
  parse-options-cb: convert sha1_array_append caller to struct object_id
  fsck: convert init_skiplist to struct object_id
  builtin/receive-pack: convert portions to struct object_id
  builtin/pull: convert portions to struct object_id
  builtin/diff: convert to struct object_id
  Convert GIT_SHA1_RAWSZ used for allocation to GIT_MAX_RAWSZ
  Convert GIT_SHA1_HEXSZ used for allocation to GIT_MAX_HEXSZ
  Define new hash-size constants for allocating memory
2017-04-19 21:37:13 -07:00
Jonathan Tan
8e2c7bef03 fetch-pack: show clearer error message upon ERR
Currently, fetch-pack prints a confusing error message ("expected
ACK/NAK") when the server it's communicating with sends a pkt-line
starting with "ERR".  Replace it with a less confusing error message.

Also update the documentation describing the fetch-pack/upload-pack
protocol (pack-protocol.txt) to indicate that "ERR" can be sent in the
place of "ACK" or "NAK". In practice, this has been done for quite some
time by other Git implementations (e.g. JGit sends "want $id not valid")
and by Git itself (since commit bdb31ea: "upload-pack: report "not our
ref" to client", 2017-02-23) whenever a "want" line references an object
that it does not have. (This is uncommon, but can happen if a repository
is garbage-collected during a negotiation.)

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-17 18:51:28 -07:00
brian m. carlson
e239dabb14 Documentation: update and rename api-sha1-array.txt
Since the structure and functions have changed names, update the code
examples and the documentation.  Rename the file to match the new name
of the API.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-31 08:33:56 -07:00
Junio C Hamano
0330344e0f Merge branch 'jh/memihash-opt'
The name-hash used for detecting paths that are different only in
cases (which matter on case insensitive filesystems) has been
optimized to take advantage of multi-threading when it makes sense.

* jh/memihash-opt:
  name-hash: add test-lazy-init-name-hash to .gitignore
  name-hash: add perf test for lazy_init_name_hash
  name-hash: add test-lazy-init-name-hash
  name-hash: perf improvement for lazy_init_name_hash
  hashmap: document memihash_cont, hashmap_disallow_rehash api
  hashmap: add disallow_rehash setting
  hashmap: allow memihash computation to be continued
  name-hash: specify initial size for istate.dir_hash table
2017-03-28 14:06:00 -07:00
Jeff Hostetler
4f4458aafa hashmap: document memihash_cont, hashmap_disallow_rehash api
Document memihash_cont() and hashmap_disallow_rehash()
in Documentation/technical/api-hashmap.txt.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-23 10:40:07 -07:00
Junio C Hamano
a04855bae8 Merge branch 'bw/attr'
The gitattributes machinery is being taught to work better in a
multi-threaded environment.

* bw/attr: (27 commits)
  attr: reformat git_attr_set_direction() function
  attr: push the bare repo check into read_attr()
  attr: store attribute stack in attr_check structure
  attr: tighten const correctness with git_attr and match_attr
  attr: remove maybe-real, maybe-macro from git_attr
  attr: eliminate global check_all_attr array
  attr: use hashmap for attribute dictionary
  attr: change validity check for attribute names to use positive logic
  attr: pass struct attr_check to collect_some_attrs
  attr: retire git_check_attrs() API
  attr: convert git_check_attrs() callers to use the new API
  attr: convert git_all_attrs() to use "struct attr_check"
  attr: (re)introduce git_check_attr() and struct attr_check
  attr: rename function and struct related to checking attributes
  attr.c: outline the future plans by heavily commenting
  Documentation: fix a typo
  attr.c: add push_stack() helper
  attr: support quoting pathname patterns in C style
  attr.c: plug small leak in parse_attr_line()
  attr.c: tighten constness around "git_attr" structure
  ...
2017-02-27 13:57:14 -08:00
Junio C Hamano
1b324988ac Merge branch 'jk/describe-omit-some-refs'
"git describe" and "git name-rev" have been taught to take more
than one refname patterns to restrict the set of refs to base their
naming output on, and also learned to take negative patterns to
name refs not to be used for naming via their "--exclude" option.

* jk/describe-omit-some-refs:
  describe: teach describe negative pattern matches
  describe: teach --match to accept multiple patterns
  name-rev: add support to exclude refs by pattern match
  name-rev: extend --refs to accept multiple patterns
  doc: add documentation for OPT_STRING_LIST
2017-02-27 13:57:11 -08:00
Junio C Hamano
1295c21524 attr: retire git_check_attrs() API
Since nobody uses the old API, make it file-scope static, and update
the documentation to describe the new API.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01 13:46:52 -08:00
Junio C Hamano
feaad0eec7 Merge branch 'sb/in-core-index-doc'
Documentation and in-code comments updates.

* sb/in-core-index-doc:
  documentation: retire unfinished documentation
  cache.h: document add_[file_]to_index
  cache.h: document remove_index_entry_at
  cache.h: document index_name_pos
2017-01-31 13:14:59 -08:00
Junio C Hamano
c54ba283fa Merge branch 'jk/clear-delta-base-cache-fix'
A crashing bug introduced in v2.11 timeframe has been found (it is
triggerable only in fast-import) and fixed.

* jk/clear-delta-base-cache-fix:
  clear_delta_base_cache(): don't modify hashmap while iterating
2017-01-31 13:14:59 -08:00
Jacob Keller
4a68748893 doc: add documentation for OPT_STRING_LIST
Commit c8ba163916 ("parse-options: add OPT_STRING_LIST helper",
2011-06-09) added the OPT_STRING_LIST as a way to accumulate a repeated
list of strings. However, this was not documented in the
api-parse-options documentation. Add documentation now so that future
developers may learn of its existence.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19 13:10:09 -08:00
Stefan Beller
830c912a0e documentation: retire unfinished documentation
When looking for documentation for a specific function, you may be tempted
to run

  git -C Documentation grep index_name_pos

only to find the file technical/api-in-core-index.txt, which doesn't
help for understanding the given function. It would be better to not find
these functions in the documentation, such that people directly dive into
the code instead.

In the previous patches we have documented
* index_name_pos()
* remove_index_entry_at()
* add_[file_]to_index()
in cache.h

We already have documentation for:
* add_index_entry()
* read_index()

Which leaves us with a TODO for:
* cache -> the_index macros
* refresh_index()
* discard_index()
* ie_match_stat() and ie_modified(); how they are different and when to
  use which.
* write_index() that was renamed to write_locked_index
* cache_tree_invalidate_path()
* cache_tree_update()

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19 12:18:43 -08:00
Jeff King
abd5a00268 clear_delta_base_cache(): don't modify hashmap while iterating
On Thu, Jan 19, 2017 at 03:03:46PM +0100, Ulrich Spörlein wrote:

> > I suspect the patch below may fix things for you. It works around it by
> > walking over the lru list (either is fine, as they both contain all
> > entries, and since we're clearing everything, we don't care about the
> > order).
>
> Confirmed. With the patch applied, I can import the whole 55G in one go
> without any crashes or aborts. Thanks much!

Thanks. Here it is rolled up with a commit message.

-- >8 --
Subject: clear_delta_base_cache(): don't modify hashmap while iterating

Removing entries while iterating causes fast-import to
access an already-freed `struct packed_git`, leading to
various confusing errors.

What happens is that clear_delta_base_cache() drops the
whole contents of the cache by iterating over the hashmap,
calling release_delta_base_cache() on each entry. That
function removes the item from the hashmap. The hashmap code
may then shrink the table, but the hashmap_iter struct
retains an offset from the old table.

As a result, the next call to hashmap_iter_next() may claim
that the iteration is done, even though some items haven't
been visited.

The only caller of clear_delta_base_cache() is fast-import,
which wants to clear the cache because it is discarding the
packed_git struct for its temporary pack. So by failing to
remove all of the entries, we still have references to the
freed packed_git.

To make things even more confusing, this doesn't seem to
trigger with the test suite, because it depends on
complexities like the size of the hash table, which entries
got cleared, whether we try to access them before they're
evicted from the cache, etc.

So I've been able to identify the problem with large
imports like freebsd's svn import, or a fast-export of
linux.git. But nothing that would be reasonable to run as
part of the normal test suite.

We can fix this easily by iterating over the lru linked list
instead of the hashmap. They both contain the same entries,
and we can use the "safe" variant of the list iterator,
which exists for exactly this case.

Let's also add a warning to the hashmap API documentation to
reduce the chances of getting bit by this again.

Reported-by: Ulrich Spörlein <uqs@freebsd.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19 11:17:20 -08:00
Junio C Hamano
fe9ec8bdf6 Merge branch 'bw/pathspec-cleanup'
Code clean-up in the pathspec API.

* bw/pathspec-cleanup:
  pathspec: rename prefix_pathspec to init_pathspec_item
  pathspec: small readability changes
  pathspec: create strip submodule slash helpers
  pathspec: create parse_element_magic helper
  pathspec: create parse_long_magic function
  pathspec: create parse_short_magic function
  pathspec: factor global magic into its own function
  pathspec: simpler logic to prefix original pathspec elements
  pathspec: always show mnemonic and name in unsupported_magic
  pathspec: remove unused variable from unsupported_magic
  pathspec: copy and free owned memory
  pathspec: remove the deprecated get_pathspec function
  ls-tree: convert show_recursive to use the pathspec struct interface
  dir: convert fill_directory to use the pathspec struct interface
  dir: remove struct path_simplify
  mv: remove use of deprecated 'get_pathspec()'
2017-01-18 15:12:15 -08:00
Brandon Williams
34305f7753 pathspec: remove the deprecated get_pathspec function
Now that all callers of the old 'get_pathspec' interface have been
migrated to use the new pathspec struct interface it can be removed
from the codebase.

Since there are no more users of the '_raw' field in the pathspec struct
it can also be removed.  This patch also removes the old functionality
of modifying the const char **argv array that was passed into
parse_pathspec.  Instead the constructed 'match' string (which is a
pathspec element with the prefix prepended) is only stored in its
corresponding pathspec_item entry.

Signed-off-by: Brandon Williams <bmwill@google.com>
Reviewed-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-08 18:04:17 -08:00
Stefan Beller
f2627d9b19 submodule-config: clarify parsing of null_sha1 element
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reviewed-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22 14:43:04 -08:00
Stefan Beller
73c293bb6c submodule-config: rename commit_sha1 to treeish_name
It is also possible to pass in any treeish name to lookup a submodule
config. Make it clear by naming the variables accordingly. Looking up
a submodule config by tree hash will come in handy in a later patch.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22 14:43:03 -08:00
Junio C Hamano
a460ea4a3c Merge branch 'nd/shallow-deepen'
The existing "git fetch --depth=<n>" option was hard to use
correctly when making the history of an existing shallow clone
deeper.  A new option, "--deepen=<n>", has been added to make this
easier to use.  "git clone" also learned "--shallow-since=<date>"
and "--shallow-exclude=<tag>" options to make it easier to specify
"I am interested only in the recent N months worth of history" and
"Give me only the history since that version".

* nd/shallow-deepen: (27 commits)
  fetch, upload-pack: --deepen=N extends shallow boundary by N commits
  upload-pack: add get_reachable_list()
  upload-pack: split check_unreachable() in two, prep for get_reachable_list()
  t5500, t5539: tests for shallow depth excluding a ref
  clone: define shallow clone boundary with --shallow-exclude
  fetch: define shallow boundary with --shallow-exclude
  upload-pack: support define shallow boundary by excluding revisions
  refs: add expand_ref()
  t5500, t5539: tests for shallow depth since a specific date
  clone: define shallow clone boundary based on time with --shallow-since
  fetch: define shallow boundary with --shallow-since
  upload-pack: add deepen-since to cut shallow repos based on time
  shallow.c: implement a generic shallow boundary finder based on rev-list
  fetch-pack: use a separate flag for fetch in deepening mode
  fetch-pack.c: mark strings for translating
  fetch-pack: use a common function for verbose printing
  fetch-pack: use skip_prefix() instead of starts_with()
  upload-pack: move rev-list code out of check_non_tip()
  upload-pack: make check_non_tip() clean things up on error
  upload-pack: tighten number parsing at "deepen" lines
  ...
2016-10-10 14:03:50 -07:00
Jeff King
16ddcd403b sha1_array: let callbacks interrupt iteration
The callbacks for iterating a sha1_array must have a void
return.  This is unlike our usual for_each semantics, where
a callback may interrupt iteration and have its value
propagated. Let's switch it to the usual form, which will
enable its use in more places (e.g., where we are replacing
an existing iteration with a different data structure).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-26 11:46:41 -07:00