git-commit-vandalism/name-hash.c

191 lines
4.7 KiB
C
Raw Normal View History

/*
* name-hash.c
*
* Hashing names in the index state
*
* Copyright (C) 2008 Linus Torvalds
*/
#define NO_THE_INDEX_COMPATIBILITY_MACROS
#include "cache.h"
/*
* This removes bit 5 if bit 6 is set.
*
* That will make US-ASCII characters hash to their upper-case
* equivalent. We could easily do this one whole word at a time,
* but that's for future worries.
*/
static inline unsigned char icase_hash(unsigned char c)
{
return c & ~((c & 0x40) >> 1);
}
static unsigned int hash_name(const char *name, int namelen)
{
unsigned int hash = 0x123;
do {
unsigned char c = *name++;
c = icase_hash(c);
hash = hash*101 + c;
} while (--namelen);
return hash;
}
static void hash_index_entry_directories(struct index_state *istate, struct cache_entry *ce)
{
/*
* Throw each directory component in the hash for quick lookup
* during a git status. Directory components are stored with their
* closing slash. Despite submodules being a directory, they never
* reach this point, because they are stored without a closing slash
* in the cache.
*
* Note that the cache_entry stored with the directory does not
* represent the directory itself. It is a pointer to an existing
* filename, and its only purpose is to represent existence of the
* directory in the cache. It is very possible multiple directory
* hash entries may point to the same cache_entry.
*/
unsigned int hash;
void **pos;
const char *ptr = ce->name;
while (*ptr) {
while (*ptr && *ptr != '/')
++ptr;
if (*ptr == '/') {
++ptr;
hash = hash_name(ce->name, ptr - ce->name);
fix phantom untracked files when core.ignorecase is set When core.ignorecase is turned on and there are stale index entries, "git commit" can sometimes report directories as untracked, even though they contain tracked files. You can see an example of this with: # make a case-insensitive repo git init repo && cd repo && git config core.ignorecase true && # with some tracked files in a subdir mkdir subdir && > subdir/one && > subdir/two && git add . && git commit -m base && # now make the index entries stale touch subdir/* && # and then ask commit to update those entries and show # us the status template git commit -a which will report "subdir/" as untracked, even though it clearly contains two tracked files. What is happening in the commit program is this: 1. We load the index, and for each entry, insert it into the index's name_hash. In addition, if ignorecase is turned on, we make an entry in the name_hash for the directory (e.g., "contrib/"), which uses the following code from 5102c61's hash_index_entry_directories: hash = hash_name(ce->name, ptr - ce->name); if (!lookup_hash(hash, &istate->name_hash)) { pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } } Note that we only add the directory entry if there is not already an entry. 2. We run add_files_to_cache, which gets updated information for each cache entry. It helpfully inserts this information into the cache, which calls replace_index_entry. This in turn calls remove_name_hash() on the old entry, and add_name_hash() on the new one. But remove_name_hash doesn't actually remove from the hash, it only marks it as "no longer interesting" (from cache.h): /* * We don't actually *remove* it, we can just mark it invalid so that * we won't find it in lookups. * * Not only would we have to search the lists (simple enough), but * we'd also have to rehash other hash buckets in case this makes the * hash bucket empty (common). So it's much better to just mark * it. */ static inline void remove_name_hash(struct cache_entry *ce) { ce->ce_flags |= CE_UNHASHED; } This is OK in the specific-file case, since the entries in the hash form a linked list, and we can just skip the "not here anymore" entries during lookup. But for the directory hash entry, we will _not_ write a new entry, because there is already one there: the old one that is actually no longer interesting! 3. While traversing the directories, we end up in the directory_exists_in_index_icase function to see if a directory is interesting. This in turn checks index_name_exists, which will look up the directory in the index's name_hash. We see the old, deleted record, and assume there is nothing interesting. The directory gets marked as untracked, even though there are index entries in it. The problem is in the code I showed above: hash = hash_name(ce->name, ptr - ce->name); if (!lookup_hash(hash, &istate->name_hash)) { pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } } Having a single cache entry that represents the directory is not enough; that entry may go away if the index is changed. It may be tempting to say that the problem is in our removal method; if we removed the entry entirely instead of simply marking it as "not here anymore", then we would know we need to insert a new entry. But that only covers this particular case of remove-replace. In the more general case, consider something like this: 1. We add "foo/bar" and "foo/baz" to the index. Each gets their own entry in name_hash, plus we make a "foo/" entry that points to "foo/bar". 2. We remove the "foo/bar" entry from the index, and from the name_hash. 3. We ask if "foo/" exists, and see no entry, even though "foo/baz" exists. So we need that directory entry to have the list of _all_ cache entries that indicate that the directory is tracked. So that implies making a linked list as we do for other entries, like: hash = hash_name(ce->name, ptr - ce->name); pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } But that's not right either. In fact, it shows a second bug in the current code, which is that the "ce->next" pointer is supposed to be linking entries for a specific filename entry, but here we are overwriting it for the directory entry. So the same cache entry ends up in two linked lists, but they share the same "next" pointer. As it turns out, this second bug can't be triggered in the current code. The "if (pos)" conditional is totally dead code; pos will only be non-NULL if there was an existing hash entry, and we already checked that there wasn't one through our call to lookup_hash. But fixing the first bug means taking out that call to lookup_hash, which is going to activate the buggy dead code, and we'll end up splicing the two linked lists together. So we need to have a separate next pointer for the list in the directory bucket, and we need to traverse that list in index_name_exists when we are looking up a directory. This bloats "struct cache_entry" by a few bytes. Which is annoying, because it's only necessary when core.ignorecase is enabled. There's not an easy way around it, short of separating out the "next" pointers from cache_entry entirely (i.e., having a separate "cache_entry_list" struct that gets stored in the name_hash). In practice, it probably doesn't matter; we have thousands of cache entries, compared to the millions of objects (where adding 4 bytes to the struct actually does impact performance). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-06 18:06:09 +02:00
pos = insert_hash(hash, ce, &istate->name_hash);
if (pos) {
ce->dir_next = *pos;
*pos = ce;
}
}
}
}
static void hash_index_entry(struct index_state *istate, struct cache_entry *ce)
{
void **pos;
unsigned int hash;
if (ce->ce_flags & CE_HASHED)
return;
ce->ce_flags |= CE_HASHED;
ce->next = ce->dir_next = NULL;
hash = hash_name(ce->name, ce_namelen(ce));
pos = insert_hash(hash, ce, &istate->name_hash);
if (pos) {
ce->next = *pos;
*pos = ce;
}
if (ignore_case)
hash_index_entry_directories(istate, ce);
}
static void lazy_init_name_hash(struct index_state *istate)
{
int nr;
if (istate->name_hash_initialized)
return;
for (nr = 0; nr < istate->cache_nr; nr++)
hash_index_entry(istate, istate->cache[nr]);
istate->name_hash_initialized = 1;
}
void add_name_hash(struct index_state *istate, struct cache_entry *ce)
{
ce->ce_flags &= ~CE_UNHASHED;
if (istate->name_hash_initialized)
hash_index_entry(istate, ce);
}
static int slow_same_name(const char *name1, int len1, const char *name2, int len2)
{
if (len1 != len2)
return 0;
while (len1) {
unsigned char c1 = *name1++;
unsigned char c2 = *name2++;
len1--;
if (c1 != c2) {
c1 = toupper(c1);
c2 = toupper(c2);
if (c1 != c2)
return 0;
}
}
return 1;
}
static int same_name(const struct cache_entry *ce, const char *name, int namelen, int icase)
{
int len = ce_namelen(ce);
/*
* Always do exact compare, even if we want a case-ignoring comparison;
* we do the quick exact one first, because it will be the common case.
*/
if (len == namelen && !cache_name_compare(name, namelen, ce->name, len))
return 1;
if (!icase)
return 0;
/*
* If the entry we're comparing is a filename (no trailing slash), then compare
* the lengths exactly.
*/
if (name[namelen - 1] != '/')
return slow_same_name(name, namelen, ce->name, len);
/*
* For a directory, we point to an arbitrary cache_entry filename. Just
* make sure the directory portion matches.
*/
return slow_same_name(name, namelen, ce->name, namelen < len ? namelen : len);
}
struct cache_entry *index_name_exists(struct index_state *istate, const char *name, int namelen, int icase)
{
unsigned int hash = hash_name(name, namelen);
struct cache_entry *ce;
lazy_init_name_hash(istate);
ce = lookup_hash(hash, &istate->name_hash);
while (ce) {
if (!(ce->ce_flags & CE_UNHASHED)) {
if (same_name(ce, name, namelen, icase))
return ce;
}
fix phantom untracked files when core.ignorecase is set When core.ignorecase is turned on and there are stale index entries, "git commit" can sometimes report directories as untracked, even though they contain tracked files. You can see an example of this with: # make a case-insensitive repo git init repo && cd repo && git config core.ignorecase true && # with some tracked files in a subdir mkdir subdir && > subdir/one && > subdir/two && git add . && git commit -m base && # now make the index entries stale touch subdir/* && # and then ask commit to update those entries and show # us the status template git commit -a which will report "subdir/" as untracked, even though it clearly contains two tracked files. What is happening in the commit program is this: 1. We load the index, and for each entry, insert it into the index's name_hash. In addition, if ignorecase is turned on, we make an entry in the name_hash for the directory (e.g., "contrib/"), which uses the following code from 5102c61's hash_index_entry_directories: hash = hash_name(ce->name, ptr - ce->name); if (!lookup_hash(hash, &istate->name_hash)) { pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } } Note that we only add the directory entry if there is not already an entry. 2. We run add_files_to_cache, which gets updated information for each cache entry. It helpfully inserts this information into the cache, which calls replace_index_entry. This in turn calls remove_name_hash() on the old entry, and add_name_hash() on the new one. But remove_name_hash doesn't actually remove from the hash, it only marks it as "no longer interesting" (from cache.h): /* * We don't actually *remove* it, we can just mark it invalid so that * we won't find it in lookups. * * Not only would we have to search the lists (simple enough), but * we'd also have to rehash other hash buckets in case this makes the * hash bucket empty (common). So it's much better to just mark * it. */ static inline void remove_name_hash(struct cache_entry *ce) { ce->ce_flags |= CE_UNHASHED; } This is OK in the specific-file case, since the entries in the hash form a linked list, and we can just skip the "not here anymore" entries during lookup. But for the directory hash entry, we will _not_ write a new entry, because there is already one there: the old one that is actually no longer interesting! 3. While traversing the directories, we end up in the directory_exists_in_index_icase function to see if a directory is interesting. This in turn checks index_name_exists, which will look up the directory in the index's name_hash. We see the old, deleted record, and assume there is nothing interesting. The directory gets marked as untracked, even though there are index entries in it. The problem is in the code I showed above: hash = hash_name(ce->name, ptr - ce->name); if (!lookup_hash(hash, &istate->name_hash)) { pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } } Having a single cache entry that represents the directory is not enough; that entry may go away if the index is changed. It may be tempting to say that the problem is in our removal method; if we removed the entry entirely instead of simply marking it as "not here anymore", then we would know we need to insert a new entry. But that only covers this particular case of remove-replace. In the more general case, consider something like this: 1. We add "foo/bar" and "foo/baz" to the index. Each gets their own entry in name_hash, plus we make a "foo/" entry that points to "foo/bar". 2. We remove the "foo/bar" entry from the index, and from the name_hash. 3. We ask if "foo/" exists, and see no entry, even though "foo/baz" exists. So we need that directory entry to have the list of _all_ cache entries that indicate that the directory is tracked. So that implies making a linked list as we do for other entries, like: hash = hash_name(ce->name, ptr - ce->name); pos = insert_hash(hash, &istate->name_hash); if (pos) { ce->next = *pos; *pos = ce; } But that's not right either. In fact, it shows a second bug in the current code, which is that the "ce->next" pointer is supposed to be linking entries for a specific filename entry, but here we are overwriting it for the directory entry. So the same cache entry ends up in two linked lists, but they share the same "next" pointer. As it turns out, this second bug can't be triggered in the current code. The "if (pos)" conditional is totally dead code; pos will only be non-NULL if there was an existing hash entry, and we already checked that there wasn't one through our call to lookup_hash. But fixing the first bug means taking out that call to lookup_hash, which is going to activate the buggy dead code, and we'll end up splicing the two linked lists together. So we need to have a separate next pointer for the list in the directory bucket, and we need to traverse that list in index_name_exists when we are looking up a directory. This bloats "struct cache_entry" by a few bytes. Which is annoying, because it's only necessary when core.ignorecase is enabled. There's not an easy way around it, short of separating out the "next" pointers from cache_entry entirely (i.e., having a separate "cache_entry_list" struct that gets stored in the name_hash). In practice, it probably doesn't matter; we have thousands of cache entries, compared to the millions of objects (where adding 4 bytes to the struct actually does impact performance). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-06 18:06:09 +02:00
if (icase && name[namelen - 1] == '/')
ce = ce->dir_next;
else
ce = ce->next;
}
/*
* Might be a submodule. Despite submodules being directories,
* they are stored in the name hash without a closing slash.
* When ignore_case is 1, directories are stored in the name hash
* with their closing slash.
*
* The side effect of this storage technique is we have need to
* remove the slash from name and perform the lookup again without
* the slash. If a match is made, S_ISGITLINK(ce->mode) will be
* true.
*/
if (icase && name[namelen - 1] == '/') {
ce = index_name_exists(istate, name, namelen - 1, icase);
if (ce && S_ISGITLINK(ce->ce_mode))
return ce;
}
return NULL;
}