* jn/gitweb-search-optim:
gitweb: Faster project search
gitweb: Option for filling only specified info in fill_project_list_info
gitweb: Refactor checking if part of project info need filling
Before searching by some field the information we search for must be
filled in, but we do not have to fill other fields that are not
involved in the search.
To be able to request filling only specified fields,
fill_project_list_info() was enhanced in previous commit to take
additional parameters which specify part of projects info to fill.
This way we can limit doing expensive calculations (like running
git-for-each-ref to get 'age' / "Last changed" info) to doing those
only for projects which we will show as search results.
This commit actually uses this interface, changing gitweb code from
the following behavior
fill all project info on all projects
search projects
to behaving like this pseudocode
fill search fields on all projects
search projects
fill all project info on search results
With this commit the number of git commands used to generate search
results is 2*<matched projects> + 1, and depends on number of matched
projects rather than number of all projects (all repositories).
Note: this is 'git for-each-ref' to find last activity, and 'git config'
for each project, and 'git --version' once.
Example performance improvements, for search that selects 2
repositories out of 12 in total:
* Before (warm cache):
"This page took 0.867151 seconds and 27 git commands to generate."
* After (warm cache):
"This page took 0.673643 seconds and 5 git commands to generate."
Now imagine that they are 5 repositories out of 5000, and cold or
trashed cache case.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Enhance fill_project_list_info() subroutine to accept optional
parameters that specify which fields in project information needs to
be filled. If none are specified then fill_project_list_info()
behaves as it used to, and ensure that all project info is filled.
This is in preparation of future lazy filling of project info in
project search and pagination of sorted list of projects.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extract the check if given keys (given parts) of project info needs to
be filled into project_info_needs_filling() subroutine. It is for now
a thin wrapper around "!exists $project_info->{$key}".
Note that !defined was replaced by more correct !exists.
While at it uniquify treating of all project info, adding checks for
'age' field before running git_get_last_activity(), and also checking
for all keys filled in code protected by conditional, and not only
one.
The code now looks like this
foreach my $project (@$project_list) {
if (given keys need to be filled) {
fill given keys
}
...
}
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit ff7f218 (gitweb: Fix file links in "grep" search, 2012-01-05),
added $file_href variable, to reduce duplication and have the fix
applied in single place.
Unfortunately it made variable defined inside the loop, not taking into
account the fact that $file_href was set only if file changed.
Therefore for files with multiple matches $file_href was undefined for
second and subsequent matches.
Fix this bug by moving $file_href declaration outside loop.
Adds tests for almost all forms of sarch in gitweb, which were missing
from testuite. Note that it only tests if there are no warnings, and
it doesn't check that gitweb finds what it should find.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a repository whose HEAD points to an unborn branch with no commits,
"heads" view and "summary" view (which shows what is shown in "heads"
view) compared the object names of commits at the tip of branches with the
output from "git rev-parse HEAD", which caused comparison of a string with
undef and resulted in a warning in the server log.
This can happen if non-bare repository (with default 'master' branch)
is updated not via committing but by other means like push to it, or
Gerrit. It can happen also just after running "git checkout --orphan
<new branch>" but before creating any new commit on this branch.
Rewrite the comparison so that it also works when $head points at nothing;
in such a case, no branch can be "the current branch", add a test for it.
While at it, rename local variable $head to $head_at, as it points to
current commit rather than current branch name (HEAD contents).
The code still incorrectly shows all branches that point at the same
commit as what HEAD points as "the current branch", even when HEAD is
detached. Fixing this bug is outside the scope of this patch.
Reported-by: Rajesh Boyapati
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 19d2d23 (gitweb: add project_filter to limit project list
to a subdirectory, 2012-01-30) added also support for displaying
$project_filter, if present, in page title.
Unfortunately it forgot to treat $project_filter as path, and escape
it using esc_path(), like it is done for $filename.
Also, it was not obvious that "$site_name - $project_filter" is about
project filtering: use "$site_name - projects in '$project_filter'".
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bl/gitweb-project-filter:
gitweb: Make project search respect project_filter
gitweb: improve usability of projects search form
gitweb: place links to parent directories in page header
gitweb: show active project_filter in project_list page header
gitweb: limit links to alternate forms of project_list to active project_filter
gitweb: add project_filter to limit project list to a subdirectory
gitweb: prepare git_get_projects_list for use outside 'forks'.
gitweb: move hard coded .git suffix out of git_get_projects_list
Gitweb forgot to turn query parameters into UTF-8. This results in a bug
that one cannot search for a string with characters outside US-ASCII. For
example searching for "Michał Kiedrowicz" (containing letter 'ł' - LATIN
SMALL LETTER L WITH STROKE, with Unicode codepoint U+0142, represented
with 0xc5 0x82 bytes in UTF-8 and percent-encoded as %C5%82) result in the
following incorrect data in search field
MichaÅ\202 Kiedrowicz
This is caused by CGI by default treating '0xc5 0x82' bytes as two
characters in Perl legacy encoding latin-1 (iso-8859-1), because 's'
query parameter is not processed explicitly as UTF-8 encoded string.
The solution used here follows "Using Unicode in a Perl CGI script"
article on http://www.lemoda.net/cgi/perl-unicode/index.html:
use CGI;
use Encode 'decode_utf8;
my $value = params('input');
$value = decode_utf8($value);
Decoding UTF-8 is done when filling %input_params hash and $path_info
variable; the former requires to move from explicit $cgi->param(<label>)
to $input_params{<name>} in a few places, which is a good idea anyway.
Also add -override=>1 parameter to $cgi->textfield() invocation in search
form. Otherwise CGI would use values from query string if it is present,
filling value from $cgi->param... without decode_utf8(). As we are using
value of appropriate parameter anyway, -override=>1 doesn't change the
situation but makes gitweb fill search field correctly.
We could simply use the '-utf8' pragma (via "use CGI '-utf8';") to solve
this, but according to CGI.pm documentation, it may cause problems with
POST requests containing binary files, and it requires CGI 3.31 (I think),
released with perl v5.8.9.
Reported-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Jakub Narębski <jnareb@gmail.com>
Tested-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make gitweb search within filtered projects (i.e. projects shown), and
change "List all projects" to "List all projects in '$project_filter/'"
if project_filter is used.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor generating project search form into git_project_search_form().
Make text field wider and add on mouse over explanation (via "title"
attribute), add an option to use regular expressions, and replace
'Search:' label with [Search] button.
Also add "List all projects" link to make it easier to go back from search
result to list of all projects (note that an empty search term is
disallowed).
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change html page headers to not only link the project root and the
currently selected project but also the directories in between using
project_filter. (Allowing to jump to a list of all projects within
that intermediate directory directly and making the project_filter
feature visible to users).
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the page header of a project_list view with a project_filter
given show breadcrumbs in the page headers showing which directory
it is currently limited to and also containing links to the parent
directories.
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If project_list action is given a project_filter argument, pass that to
TXT and OPML formats.
This way [OPML] and [TXT] links provide the same list of projects as
the projects_list page they are linked from.
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit changes the project listing views (project_list,
project_index and opml) to limit the output to only projects in a
subdirectory if the new optional parameter ?pf=directory name is
used.
The implementation of the filter reuses the implementation used for
the 'forks' action (i.e. listing all projects within that directory
from the projects list file (GITWEB_LIST) or only projects in the
given subdirectory of the project root directory without a projects
list file).
Reusing $project instead of adding a new parameter would have been
nicer from a UI point-of-view (including PATH_INFO support) but
would complicate the $project validating code that is currently
being used to ensure nothing is exported that should not be viewable.
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use of the filter option of git_get_projects_list is currently limited
to forks. It currently assumes the project belonging to the filter
directory was already validated to be visible in the project list.
To make it more generic add an optional argument to denote visibility
verification is still needed.
If there is a projects list file (GITWEB_LIST) only projects from
this list are returned anyway, so no more checks needed.
If there is no projects list file and the caller requests strict
checking (GITWEB_STRICT_EXPORT), do not jump directly to the
given directory but instead do a normal search and filter the
results instead.
The only effect of GITWEB_STRICT_EXPORT without GITWEB_LIST is to make
sure no project can be viewed without also be found starting from
project root. git_get_projects_list without this patch does not enforce
this but all callers only call it with a filter already checked this
way. With this parameter a caller can request this check if the filter
cannot be checked this way.
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use of the filter option of git_get_projects_list is currently
limited to forks. It hard codes removal of ".git" suffixes from
the filter.
To make it more generic move the .git suffix removal to the callers.
Signed-off-by: Bernhard R. Link <brlink@debian.org>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When gitweb URL does not provide action explicitly, e.g.
http://git.example.org/repo.git/branch
dispatch() tries to guess action (view to be used) based on remaining
parameters. Among others it is based on the type of requested object,
which gave problems when asking for non-existent branch or file (for
example misspelt name).
Now undefined $action from dispatch() should not result in problems.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Run "git grep" in "grep" search with '-z' option, to be able to parse
response also for files with filename containing ':' character. The
':' character is otherwise (without '-z') used to separate filename
from line number and from matched line.
Note that this does not protect files with filename containing
embedded newline. This would be hard but doable for text files, and
harder or even currently impossible with binary files: git does not
quote filename in
"Binary file <foo> matches"
message, but new `--break` and/or `--header` options to git-grep could
help here.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were two bugs in generating file links (links to "blob" view),
one hidden by the other. The correct way of generating file link is
href(action=>"blob", hash_base=>$co{'id'},
file_name=>$file);
It was $co{'hash'} (this key does not exist, and therefore this is
undef), and 'hash' instead of 'hash_base'.
To have this fix applied in single place, this commit also reduces
code duplication by saving file link (which is used for line links) in
$file_href.
Reported-by: Thomas Perl <th.perl@gmail.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current code is removing the trailing "/", but computing the string
length on the previous value, i.e. with the trailing "/". Later in the
code, we do
my $path = substr($File::Find::name, $pfxlen + 1);
And the "$pfxlen + 1" is supposed to mean "the length of the prefix, plus
1 for the / separating the prefix and the path", but with an incorrect
$pfxlen, this basically eats the first character of the path, and yields
"404 - No projects found".
While we're there, also fix $pfxdepth to use $dir, although a change of 1
in the depth shouldn't really matter.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/maint-gitweb-utf8-fix:
gitweb: Fix fallback mode of to_utf8 subroutine
gitweb: Output valid utf8 in git_blame_common('data')
gitweb: esc_html() site name for title in OPML
gitweb: Call to_utf8() on input string in chop_and_escape_str()
e5d3de5 (gitweb: use Perl built-in utf8 function for UTF-8 decoding.,
2007-12-04) was meant to make gitweb faster by using Perl's internals
(see subsection "Messing with Perl's Internals" in Encode(3pm) manpage)
Simple benchmark confirms that (old = 00f429a, new = this version):
old new
old -- -65%
new 189% --
Unfortunately it made fallback mode of to_utf8 do not work... except
for default value 'latin1' of $fallback_encoding ('latin1' is Perl
native encoding), which is why it was not noticed for such long time.
utf8::valid(STRING) is an internal function that tests whether STRING
is in a _consistent state_ regarding UTF-8. It returns true is
well-formed UTF-8 and has the UTF-8 flag on _*or*_ if string is held
as bytes (both these states are 'consistent'). For gitweb the second
option was true, as output from git commands is opened without ':utf8'
layer.
What made it work at all for STRING in 'latin1' encoding is the fact
that utf8:decode(STRING) turns on UTF-8 flag only if source string is
valid UTF-8 and contains multi-byte UTF-8 characters... and that if
string doesn't have UTF-8 flag set it is treated as in native Perl
encoding, i.e. 'latin1' / 'iso-8859-1' (unless native encoding it is
EBCDIC ;-)). It was ':utf8' layer that actually converted 'latin1'
(no UTF-8 flag == native == 'latin1) to 'utf8'.
Let's make use of the fact that utf8:decode(STRING) returns false if
STRING is invalid as UTF-8 to check whether to enable fallback mode.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Otherwise when javascript-actions are enabled gitweb shown broken
author names in the tooltips on blame pages ('blame_incremental'
view).
Signed-off-by: Jürgen Kreileder <jk@blackdown.de>
Acked-by: Jakub Narębski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This escapes the site name in OPML (XML uses the same escaping rules
as HTML). Also fixes encoding issues because esc_html() uses
to_utf8().
Signed-off-by: Jürgen Kreileder <jk@blackdown.de>
Acked-by: Jakub Narębski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
a) To fix the comparison with the chopped string,
otherwise we compare bytes with characters, as
chop_str() must run to_utf8() for correct operation
b) To give the title attribute correct encoding;
we need to mark strings as UTF-8 before outpur
Signed-off-by: Jürgen Kreileder <jk@blackdown.de>
Acked-by: Jakub Narębski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add to the lower part of navigation bar (the action specific part)
links allowing to switch between 'inline' (ordinary) diff and
'side by side' style diff.
It is not shown for combined / compact combined diff.
Signed-off-by: Kato Kazuyoshi <kato.kazuyoshi@gmail.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use href(-replay->1,...) in (sub)navigation links (like changing style
of view, or going to parent commit) so that extra options are
preserved.
This is needed so clicking on such (sub)navigation link would preserve
style of diff; for example when using "side-by-side" diff style then
going to parent commit would now also use this style.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use separate background colors for pure removal, pure addition and
change for side-by-side diff. This makes reading such diff easier,
allowing to easily distinguish empty lines in diff from vertical
whitespace used to align chunk blocks.
Note that if lines in diff were numbered, the absence of line numbers
[for one side] would help in distinguishing empty lines from vertical
align.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commits adds to support for showing "side-by-side" style diff.
Currently you have to hand-craft the URL; navigation for selecting
diff style is to be added in the next commit.
The diff output in unified format from "git diff-tree" is reorganized to
side-by-side style chunk by chunk with format_sidebyside_diff_chunk().
This reorganization requires knowledge about diff line classification,
so format_diff_line() was renamed to process_diff_line(), and changed to
return tuple (list) consisting of class of diff line and of
HTML-formatted (but not wrapped in <div class="diff ...">...</div>) diff
line. Wrapping is now done by caller, i.e. git_patchset_body().
Gitweb uses float+margin CSS-based layout for "side by side" diff.
You can specify style of diff with "ds" ('diff_style') query
parameter. Currently supported values are 'inline' and 'sidebyside';
the default is 'inline'.
Another solution would be to use "opt" ('extra_options') for that...
though current use of it in gitweb seems to suggest that "opt" is more
about passing extra options to underlying git commands, and "git diff"
doesn't support '--side-by-side' like GNU diff does, (yet?).
Signed-off-by: Kato Kazuyoshi <kato.kazuyoshi@gmail.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor main parts of HTML-formatting for diff chunk headers
(formatting means here adding links and syntax hightlighting) into
separate subroutines:
* format_unidiff_chunk_header for ordinary diff,
* format_cc_diff_chunk_header for combined diff
(more than one parent)
This makes format_diff_line() subroutine easier to follow.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simplify classification of diff line body in format_diff_line(),
replacing two long if-elsif chains (one for ordinary diff and one for
combined diff of a merge commit) with a single regexp match. Refactor
this code into diff_line_class() function.
While at it:
* Fix an artifact in that $diff_class included leading space to be
able to compose classes like this "class=\"diff$diff_class\"', even
when $diff_class was an empty string. This made code unnecessary
ugly: $diff_class is now just class name or an empty string.
* Introduce "ctx" class for context lines ($diff_class was set to ""
in this case before this commit).
Idea and initial code by Junio C Hamano, polish and testing by Jakub
Narebski. Inspired by patch adding side-by-side diff by Kato Kazuyoshi,
which required $diff_class to be name of class without extra space.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 9a86dd5 (gitweb: Split JavaScript for maintability, combining on
build, 2011-04-28), static/gitweb.js has been a build product that should
be cleaned upon "make clean".
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes a condition in filter_forks_from_projects_list that failed if
process directory was different from project root: in such case, the subroutine
was a no-op and forks were not detected.
Signed-off-by: Julien Muchembled <jm@jmuchemb.eu>
Tested-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/gitweb-manpages:
gitweb: Add gitweb manpages to 'gitweb' package in git.spec
Documentation: Add gitweb config variables to git-config(1)
Documentation: Link to gitweb(1) and gitweb.conf(5) in other manpages
gitweb: Add gitweb(1) manpage for gitweb itself
gitweb: Add gitweb.conf(5) manpage for gitweb configuration files
Most of what is in gitweb.txt it has been pulled directly from the
README and INSTALL files of gitweb.
Current version is somewhat based on structure of SVN::Web manpage
(one of web interfaces for Subversion).
gitweb.conf(5) i.e. gitweb configuration manpage now refers to
appropriate sections in gitweb(1). gitweb/README now refers to
gitweb/INSTALL and gitweb(1) manpage. gitweb/INSTALL now refers to
gitweb.conf(5) and gitweb(1).
Inspired-by: Drew Northup <drew.northup@maine.edu>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Much of what is in gitweb.conf.txt has been pulled directly from the
README file of gitweb. The manpage was supplemented with description
of missing gitweb config variables, and with description of gitweb's
%features.
There remains a bit of redundancy, which should be reduced if
possible... but I think some of duplication of information is
inevitable.
[jn: Improved, extended, removed duplicate info from README]
Signed-off-by: Drew Northup <drew.northup@maine.edu>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The fixLinks() function adds 'js=1' to each link that does not already
have 'js' query parameter specified. This is used to signal to gitweb
that the browser can actually do javascript when these links are used.
There are two problems with the existing code:
1. URIs with fragment and 'js' query parameter, like e.g.
...foo?js=0#l199
were not recognized as having 'js' query parameter already.
2. The 'js' query parameter, in the form of either '?js=1' or ';js=1'
was appended at the end of URI, even if it included a fragment
(had a hash part). This lead to the incorrect links like this
...foo#l199?js=1
instead of adding query parameter as last part of query, but
before the fragment part, i.e.
...foo?js=1#l199
Signed-off-by: Peter Stuge <peter@stuge.se>
Acked-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
