We can just feed our URL straight to git-credential and it
will parse it for us, saving us some code.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When writing a credential to git-credential, we omit fields
that do not have a true value. This will skip empty or
undefined fields (which we want), but will also accidentally
skip usernames or passwords which happen to have a non-true
value (e.g., "0"). Be more careful by checking for non-zero
length.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Traditionnally, pages named Foo:Bar are page 'Bar' in namespace 'Foo'.
However, it is also possible to call a page Foo:Bar if 'Foo' is not a
namespace. In this case, the actual name of the page is 'Foo:Bar', in the
main namespace. Since we can't tell with only the filename, query the
wiki for a namespace 'Foo' in these cases, but deal with the case where
no such namespace is found.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some wiki, including https://git.wiki.kernel.org/ have invalid revision
numbers (i.e. the actual revision numbers are non-contiguous). Don't die
when encountering one.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Initial phases of push and pull with git-remote-mediawiki can be long on
a large wiki. Let the user know what's going on.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When notes are created to record a push, it normally doesn't exist yet.
However, when a push is interrupted and then restarted, it may happen
that a commit already has notes attached, and we want to reflect the newly
created remote revision, hence use 'git notes add -f' to override the
existing one
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The algorithm to find a path from the local revision to the remote one
was calling "git rev-list" and parsing its output N times. Run rev-list
only once, and fill a hashtable with the result to optimize the body of
the loop.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is possible to use git-remote-mediawiki on a tree with both .mw files
and other files. Before git-remote-mediawiki learnt how to export
mediafiles, such mixed trees allowed the user to maintain both the wiki
and other files for the same project in the same repository. With the
newly added support for exporting mediafiles, pushing such mixed trees
would upload unrelated files as mediafiles, which may not be desired.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* mm/mediawiki-tests:
git-remote-mediawiki: be more defensive when requests fail
git-remote-mediawiki: more efficient 'pull' in the best case
git-remote-mediawiki: extract revision-importing loop to a function
git-remote-mediawiki: refactor loop over revision ids
git-remote-mediawiki: change return type of get_mw_pages
git-remote-mediawiki (t9363): test 'File:' import and export
git-remote-mediawiki: support for uploading file in test environment
git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8 characters
git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push
git-remote-mediawiki (t9360): test git-remote-mediawiki clone
git-remote-mediawiki: test environment of git-remote-mediawiki
git-remote-mediawiki: scripts to install, delete and clear a MediaWiki
"mediawiki" remote helper (in contrib/) learned to handle file
attachments.
* mm/mediawiki-file-attachments:
git-remote-mediawiki: improve support for non-English Wikis
git-remote-mediawiki: import "File:" attachments
git-remote-mediawiki: split get_mw_pages into smaller functions
git-remote-mediawiki: send "File:" attachments to a remote wiki
git-remote-mediawiki: don't "use encoding 'utf8';"
git-remote-mediawiki: don't compute the diff when getting commit message
Expose the credential API to scripted Porcelain writers.
* mm/credential-plumbing:
git-remote-mediawiki: update comments to reflect credential support
git-remote-mediawiki: add credential support
git credential fill: output the whole 'struct credential'
add 'git credential' plumbing command
The only way to fetch new revisions from a wiki before this patch was to
query each page for new revisions. This is good when tracking a small set
of pages on a large wiki, but very inefficient when tracking many pages
on a wiki with little activity.
Implement a new strategy that queries the wiki for its last global
revision, queries each new revision, and filter out pages that are not
tracked.
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Without changing the behavior, we turn the foreach loop on an array of
revisions into a loop on an array of integer. It will be easier to
implement other strategies as they will only need to produce an array of
integer instead of a more complex data-structure.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous version was returning the list of pages to be fetched, but
we are going to need an efficient membership test (i.e. is the page
$title tracked), hence exposing a hash will be more convenient.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Pavel Volek <Pavel.Volek@ensimag.imag.fr>
Signed-off-by: NGUYEN Kim Thuat <Kim-Thuat.Nguyen@ensimag.imag.fr>
Signed-off-by: ROUCHER IGLESIAS Javier <roucherj@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This will be used for testing git-remote-mediawiki's import feature on a
wiki containing media files.
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Non-ascii encoding create many particular cases when used in page
content, name, and edit/commit message. Test these cases.
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch provides a set of tests for the pull and push fonctionnality
of git-remote-mediawiki. The actual tests are kept in a separate function
to allow further tests to re-run the same set of commands with different
push and pull strategies.
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order to test git-remote-mediawiki, a set of functions is needed to
manage a MediaWiki: edit a page, remove a page, fetch a page, fetch all
pages on a given wiki.
A few helper function are also provided to check the content of
directories.
In addition, this patch provides Makefiles to execute tests.
See the README file for more details.
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
install_wiki.sh allows the user to install a MediaWiki instance in a
single shell command. Like "git instaweb", it configures and launches
lighttpd without requiring root priviledges. To simplify database
management, it uses SQLite, which doesn't require a running daemon, and
allows reseting the database by simply replacing a single file. This
allows install_wiki to also defines a function wiki_reset which clear all
content of the previously created wiki, which will be very useful to run
several indepenant tests on the same wiki.
Note those functionnalities are made to be used from the user command
line in the directory git/contrib/mw-to-git/t/
Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr>
Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr>
Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr>
Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr>
Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mediafiles can live in namespaces with names different from Image
and File. While at it, rework the code to make it simpler and easier
to read.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Split a rather heavy-ish "git completion" script out to create a
separate "git prompting" script, to help lazy-autoloading of the
completion part while making prompting part always available.
Add the symmetrical feature to the "File:" export support in the previous
patch. Download files from the wiki as needed, and feed them into the
fast-import stream. Import both the file itself, and the corresponding
description page.
Signed-off-by: Pavel Volek <Pavel.Volek@ensimag.imag.fr>
Signed-off-by: NGUYEN Kim Thuat <Kim-Thuat.Nguyen@ensimag.imag.fr>
Signed-off-by: ROUCHER IGLESIAS Javier <roucherj@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current version of the git-remote-mediawiki supports only import and
export of plain wiki pages. This patch adds the functionality to export
file attachments (i.e. the content of the File: MediaWiki namespace),
which are also exposed by MediaWiki API.
This requires a recent version of MediaWiki::API (Version 0.37 works.
Version 0.34 doesn't).
Signed-off-by: Pavel Volek <Pavel.Volek@ensimag.imag.fr>
Signed-off-by: NGUYEN Kim Thuat <Kim-Thuat.Nguyen@ensimag.imag.fr>
Signed-off-by: ROUCHER IGLESIAS Javier <roucherj@ensimag.imag.fr>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The use of this statement is generally discouraged, and is too intrusive
for us: it forces the HTTP requests made by the API to contain only valid
UTF-8 characters. This would break the upload of binary files.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While we're there, simplify the code a bit: since log --format=%s anyway
shows the subject line as a single line, no need to split to take the
first line.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous version implemented the possibility to log in a wiki, but
the username and password had to be provided as configuration variables.
We add the possibility to use the Git credential system to prompt
the password.
The support if implemented with generic functions that mimic the C API,
designed to be usable from other contexts in the future (i.e. they may
migrate to Git.pm if someone is interested).
While we're there, do a bit of refactoring in mw_connect_maybe.
Based on patch by: Javier Roucher Iglesias <Javier.Roucher-Iglesias@ensimag.imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
eLisp fixes for a contrib/ script.
* lm/git-blame-el:
git-blame.el: Do not use bare 0 to mean (point-min)
git-blame.el: Use with-current-buffer where appropriate
git-blame.el: Do not use goto-line in lisp code
The __gitdir() helper function finds out the path of the git
repository by running 'git rev-parse --git-dir'. However, it has a
shortcut first to avoid the overhead of running a git command in a
subshell when the current directory is at the top of the work tree,
i.e. when it contains a '.git' subdirectory.
If the 'GIT_DIR' environment variable is set then it specifies the
path to the git repository, and the autodetection of the '.git'
directory is not necessary. However, $GIT_DIR is only taken into
acocunt by 'git rev-parse --git-dir', and the check for the '.git'
subdirectory is performed first, so it wins over the path given in
$GIT_DIR.
There are several completion (helper) functions that depend on
__gitdir(), and when the above case triggers the completion script
will do weird things, like offering refs, aliases, or stashes from a
different repository, or displaying wrong or broken prompt, etc.
So check first whether $GIT_DIR is set, and only proceed with checking
the '.git' directory in the current directory if it isn't. 'git
rev-parse' would also check whether the path in $GIT_DIR is a proper
'.git' directory, i.e. 'HEAD', 'refs/', and 'objects/' are present and
accessible, but we don't have to be that thorough for the bash prompt.
And we've lived with an equally permissive check for '.git' in the
current working directory for years anyway.
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In git-blame-filter and git-blame-create-overlay we want to save
(along with the values of point and mark) the current-buffer in scope
when calling the functions. The idiom
(save-excursion
(set-buffer buf)
...)
will correctly restore the correct buffer, but will not save the
values of point and mark in buf (only in the buffer current when the
save-excursion call is executed). The intention of these functions is
to save the current buffer from the calling scope and the values of
point and mark in the buffer they are modifying. The correct idiom
for this is
(with-current-buffer buf
(save-excursion
...))
Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
Signed-off-by: Lawrence Mitchell <wence@gmx.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
goto-line is a user-level command, instead use the lisp-level
construct recommended in Emacs documentation.
Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
Signed-off-by: Lawrence Mitchell <wence@gmx.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 7f02f3d7 (completion: rename internal helpers _git and _gitk,
2012-05-19) renamed said functions to _main_git() and _main_gitk(),
respectively. By convention the name of our git-completion-specific
functions start with '_git' or '__git' prefix, so rename those
functions once again to put them back into our "namespace". Use the
two underscore prefix, because _git_main() could be mistaken for the
completion function of the (not yet existing) 'git main' command.
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using mapcar here is a waste of memory because the mapped result
is not used.
Noticed by emacs ("Warning: `mapcar' called for effect").
[jn: split from a larger patch, with new description]
Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The __gitdir function is duplicated between completion and prompt
scripts, and these definitions should not diverge; otherwise one of
them can be subtly broken depending on the order the user's shell
dot-sources them.
Leave a note to people who may want to touch one copy to make sure
they update the other one in sync. Hopefully this line would also
appear in the context of the patch to allow reviewers to notice a
patch that attempts to update only one of them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A remote helper that acts as a proxy and caches ssl session for the
https:// transport is added to the contrib/ area.
By Colby Ranger
* cr/persistent-https:
Add persistent-https to contrib
Git over HTTPS has a high request startup latency, since the SSL
negotiation can take up to a second. In order to reduce this latency,
connections should be left open to the Git server across requests
(or invocations of the git commandline).
Reduce SSL startup latency by running a daemon job that keeps
connections open to a Git server. The daemon job
(git-remote-persistent-https--proxy) is started on the first request
through the client binary (git-remote-persistent-https) and remains
running for 24 hours after the last request, or until a new daemon
binary is placed in the PATH. The client determines the daemon's
HTTP address by communicating over a UNIX socket with the daemon.
From there, the rest of the Git protocol work is delegated to the
"git-remote-http" binary, with the environment's http_proxy set to
the daemon.
Accessing /pub/scm/linux/kernel/git/torvalds/linux repository hosted
at kernel.googlesource.com with "git ls-remote" over https:// and
persistent-https:// 5 times shows that the first request takes about
the same time (0.193s vs 0.208s---there is a slight set-up cost for
the local proxy); as expected, the other four requests are much
faster (~0.18s vs ~0.08s).
Incidentally, this also has the benefit of HTTP keep-alive working
across Git command invocations. Its common for servers to use a 5
minute keep-alive on an HTTP 1.1 connection. Git-over-HTTP commonly
uses Transfer-Encoding: chunked on replies, so keep-alive will
generally just work, even though a pack stream's length isn't known
in advance. Because the helper is an external process holding that
connection open, we also benefit from being able to reuse an
existing TCP connection to the server. The same "git ls-remote"
test against http:// vs persistent-https:// URL shows that the
former takes ~0.09s while the first request for the latter is about
0.134s with set-up cost, and subsequent requests are ~0.065s,
shaving around one RTT to the server.
Signed-off-by: Colby Ranger <cranger@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
