Commit Graph

31 Commits

Author SHA1 Message Date
Junio C Hamano
ac2158649d Merge branch 'ab/trace2-squelch-gcc-warning'
Workaround compiler warnings.

* ab/trace2-squelch-gcc-warning:
  trace2: refactor to avoid gcc warning under -O3
2021-06-14 13:33:28 +09:00
Ævar Arnfjörð Bjarmason
4e0a64a713 trace2: refactor to avoid gcc warning under -O3
Refactor tr2_dst_try_uds_connect() to avoid a gcc warning[1] that
appears under -O3 (but not -O2). This makes the build pass under
DEVELOPER=1 without needing a DEVOPTS=no-error.

This can be reproduced with GCC Debian 8.3.0-6, but not e.g. with
clang 7.0.1-8+deb10u2. We've had this warning since
ee4512ed48 (trace2: create new combined trace facility, 2019-02-22).

As noted in [2] this warning happens because the compiler doesn't
assume that errno must be non-zero after a failed syscall.

Let's work around by using the well-established "saved_errno" pattern,
along with returning -1 ourselves instead of "errno". The caller can
thus rely on our "errno" on failure.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61846 for a related
bug report against GCC.

1.

    trace2/tr2_dst.c: In function ‘tr2_dst_get_trace_fd.part.5’:
    trace2/tr2_dst.c:296:10: warning: ‘fd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
      dst->fd = fd;
      ~~~~~~~~^~~~
    trace2/tr2_dst.c:229:6: note: ‘fd’ was declared here
      int fd;
          ^~
2. https://lore.kernel.org/git/20200404142131.GA679473@coredump.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21 06:56:11 +09:00
Josh Steadmon
3d3adaad91 trace2: teach Git to log environment variables
Via trace2, Git can already log interesting config parameters (see the
trace2_cmd_list_config() function). However, this can grant an
incomplete picture because many config parameters also allow overrides
via environment variables.

To allow for more complete logs, we add a new trace2_cmd_list_env_vars()
function and supporting implementation, modeled after the pre-existing
config param logging implementation.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Acked-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-23 13:14:53 -07:00
Junio C Hamano
d3096d2ba6 Merge branch 'en/doc-typofix'
Docfix.

* en/doc-typofix:
  Fix spelling errors in no-longer-updated-from-upstream modules
  multimail: fix a few simple spelling errors
  sha1dc: fix trivial comment spelling error
  Fix spelling errors in test commands
  Fix spelling errors in messages shown to users
  Fix spelling errors in names of tests
  Fix spelling errors in comments of testcases
  Fix spelling errors in code comments
  Fix spelling errors in documentation outside of Documentation/
  Documentation: fix a bunch of typos, both old and new
2019-12-01 09:04:35 -08:00
Elijah Newren
15beaaa3d1 Fix spelling errors in code comments
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10 16:00:54 +09:00
René Scharfe
5c34d2f03e trace2: add dots directly to strbuf in perf_fmt_prepare()
The initialization function of the Trace2 performance format target sets
aside a stash of dots for indenting output.  Get rid of it and use
strbuf_addchars() to provide dots on demand instead.  This shortens the
code, gets rid of a small heap allocation and is a bit more efficient.

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Acked-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-06 11:45:18 +09:00
Josh Steadmon
87db61a436 trace2: write discard message to sentinel files
Add a new "discard" event type for trace2 event destinations. When the
trace2 file count check creates a sentinel file, it will include the
normal trace2 output in the sentinel, along with this new discard
event.

Writing this message into the sentinel file is useful for tracking how
often the file count check triggers in practice.

Bump up the event format version since we've added a new event type.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-05 17:53:51 +09:00
Josh Steadmon
83e57b04e6 trace2: discard new traces if target directory has too many files
trace2 can write files into a target directory. With heavy usage, this
directory can fill up with files, causing difficulty for
trace-processing systems.

This patch adds a config option (trace2.maxFiles) to set a maximum
number of files that trace2 will write to a target directory. The
following behavior is enabled when the maxFiles is set to a positive
integer:
  When trace2 would write a file to a target directory, first check
  whether or not the traces should be discarded. Traces should be
  discarded if:
    * there is a sentinel file declaring that there are too many files
    * OR, the number of files exceeds trace2.maxFiles.
  In the latter case, we create a sentinel file named git-trace2-discard
  to speed up future checks.

The assumption is that a separate trace-processing system is dealing
with the generated traces; once it processes and removes the sentinel
file, it should be safe to generate new trace files again.

The default value for trace2.maxFiles is zero, which disables the file
count check.

The config can also be overridden with a new environment variable:
GIT_TRACE2_MAX_FILES.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-05 17:53:51 +09:00
Junio C Hamano
93fc8760e7 Merge branch 'jh/trace2-pretty-output'
Output from trace2 subsystem is formatted more prettily now.

* jh/trace2-pretty-output:
  trace2: cleanup whitespace in perf format
  trace2: cleanup whitespace in normal format
  quote: add sq_append_quote_argv_pretty()
  trace2: trim trailing whitespace in normal format error message
  trace2: remove dead code in maybe_add_string_va()
  trace2: trim whitespace in region messages in perf target format
  trace2: cleanup column alignment in perf target format
2019-09-30 13:19:31 +09:00
Junio C Hamano
a2e22bb3ad Merge branch 'rs/trace2-dst-warning'
Code cleanup.

* rs/trace2-dst-warning:
  trace2: use warning() directly in tr2_dst_malformed_warning()
2019-09-30 13:19:24 +09:00
René Scharfe
1fd881d404 trace2: use warning() directly in tr2_dst_malformed_warning()
Let warning() format the message instead of using an intermediate strbuf
for that.  This is shorter, easier to read and avoids an allocation.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:34:23 -07:00
Jeff Hostetler
742ed63345 trace2: cleanup whitespace in perf format
Make use of new sq_append_quote_argv_pretty() to normalize
how we handle leading whitespace in perf format messages.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-09 10:48:02 -07:00
Jeff Hostetler
e34430556c trace2: cleanup whitespace in normal format
Make use of new sq_append_quote_argv_pretty() to normalize
how we handle leading whitespace in normal format messages.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-09 10:48:02 -07:00
Jeff Hostetler
ad43e37839 trace2: trim trailing whitespace in normal format error message
Avoid creating unnecessary trailing whitespace in normal
target format error messages when the message is omitted.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-08 10:39:44 -07:00
Jeff Hostetler
04f10d332f trace2: remove dead code in maybe_add_string_va()
Remove an unnecessary "if" block in maybe_add_string_va().

Commit "ad006fe419e trace2: NULL is not allowed for va_list"
changed "if (fmt && *fmt && ap)" to just "if (fmt && *fmt)"
because it isn't safe to treat 'ap' as a pointer.  This made
the "if" block following it unnecessary.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-08 10:39:44 -07:00
Jeff Hostetler
da4589ce7e trace2: trim whitespace in region messages in perf target format
Avoid unecessary trailing whitespace in "region_enter" and "region_leave"
messages in perf target format.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-08 10:39:44 -07:00
Jeff Hostetler
371df1bea9 trace2: cleanup column alignment in perf target format
Truncate/elide very long "filename:linenumber" field.
Truncate region and data "category" field if necessary.
Adjust overall column widths.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-01 14:36:42 -07:00
Junio C Hamano
463dca6476 Merge branch 'sg/trace2-rename'
Rename environment variables that are used to control the "trace2"
mechanism to a more readable name.

* sg/trace2-rename:
  trace2: document the supported values of GIT_TRACE2* env variables
  trace2: rename environment variables to GIT_TRACE2*
2019-05-30 10:50:45 -07:00
Junio C Hamano
90f2d88e61 Merge branch 'jh/trace2'
* jh/trace2:
  trace2: fix tracing when NO_PTHREADS is defined
2019-05-30 10:50:45 -07:00
Jeff Hostetler
5fdae9d3be trace2: fix tracing when NO_PTHREADS is defined
Teach trace2 TLS code to not rely on pthread_getspecific() when NO_PTHREADS
is defined.  Instead, always assume the context data of the main thread.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-28 10:52:34 -07:00
SZEDER Gábor
e4b75d6a1d trace2: rename environment variables to GIT_TRACE2*
For an environment variable that is supposed to be set by users, the
GIT_TR2* env vars are just too unclear, inconsistent, and ugly.

Most of the established GIT_* environment variables don't use
abbreviations, and in case of the few that do (GIT_DIR,
GIT_COMMON_DIR, GIT_DIFF_OPTS) it's quite obvious what the
abbreviations (DIR and OPTS) stand for.  But what does TR stand for?
Track, traditional, trailer, transaction, transfer, transformation,
transition, translation, transplant, transport, traversal, tree,
trigger, truncate, trust, or ...?!

The trace2 facility, as the '2' suffix in its name suggests, is
supposed to eventually supercede Git's original trace facility.  It's
reasonable to expect that the corresponding environment variables
follow suit, and after the original GIT_TRACE variables they are
called GIT_TRACE2; there is no such thing is 'GIT_TR'.

All trace2-specific config variables are, very sensibly, in the
'trace2' section, not in 'tr2'.

OTOH, we don't gain anything at all by omitting the last three
characters of "trace" from the names of these environment variables.

So let's rename all GIT_TR2* environment variables to GIT_TRACE2*,
before they make their way into a stable release.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-28 10:20:34 -07:00
Junio C Hamano
5b2d1c0c6e Merge branch 'jh/trace2-sid-fix'
Polishing of the new trace2 facility continues.  The system-level
configuration can specify site-wide trace2 settings, which can be
overridden with per-user configuration and environment variables.

* jh/trace2-sid-fix:
  trace2: fixup access problem on /etc/gitconfig in read_very_early_config
  trace2: update docs to describe system/global config settings
  trace2: make SIDs more unique
  trace2: clarify UTC datetime formatting
  trace2: report peak memory usage of the process
  trace2: use system/global config for default trace2 settings
  config: add read_very_early_config()
  trace2: find exec-dir before trace2 initialization
  trace2: add absolute elapsed time to start event
  trace2: refactor setting process starting time
  config: initialize opts structure in repo_read_config()
2019-05-13 23:50:31 +09:00
Junio C Hamano
1b40314661 Merge branch 'js/trace2-to-directory'
The trace2 tracing facility learned to auto-generate a filename
when told to log to a directory.

* js/trace2-to-directory:
  trace2: write to directory targets
2019-05-09 00:37:24 +09:00
Jeff Hostetler
08881b9a4c trace2: make SIDs more unique
Update SID component construction to use the current UTC datetime
and a portion of the SHA1 of the hostname.

Use an simplified date/time format to make it easier to use the
SID component as a logfile filename.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:07 +09:00
Jeff Hostetler
bad229aef2 trace2: clarify UTC datetime formatting
Update tr2_tbuf_utc_datetime to generate extended UTC format.
Update tr2_tgt_event target to use extended format in 'time' columns.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:07 +09:00
Jeff Hostetler
bce9db6de9 trace2: use system/global config for default trace2 settings
Teach git to read the system and global config files for
default Trace2 settings.  This allows system-wide Trace2 settings to
be installed and inherited to make it easier to manage a collection of
systems.

The original GIT_TR2* environment variables are loaded afterwards and
can be used to override the system settings.

Only the system and global config files are used.  Repo and worktree
local config files are ignored.  Likewise, the "-c" command line
arguments are also ignored.  These limits are for performance reasons.

(1) For users not using Trace2, there should be minimal overhead to
detect that Trace2 is not enabled.  In particular, Trace2 should not
allocate lots of otherwise unused data strucutres.

(2) For accurate performance measurements, Trace2 should be initialized
as early in the git process as possible, and before most of the normal
git process initialization (which involves discovering the .git directory
and reading a hierarchy of config files).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Jeff Hostetler
39f4317744 trace2: add absolute elapsed time to start event
Add elapsed process time to "start" event to measure
the performance of early process startup.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Jeff Hostetler
a089724958 trace2: refactor setting process starting time
Create trace2_initialize_clock() and call from main() to capture
process start time in isolation and before other sub-systems are
ready.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Josh Steadmon
a4d3a283db trace2: write to directory targets
When the value of a trace2 environment variable is an absolute path
referring to an existing directory, write output to files (one per
process) underneath the given directory. Files will be named according
to the final component of the trace2 SID, followed by a counter to avoid
potential collisions.

This makes it more convenient to collect traces for every git invocation
by unconditionally setting the relevant trace2 envvar to a constant
directory name.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-22 14:27:02 +09:00
Torsten Bögershausen
ad006fe419 trace2: NULL is not allowed for va_list
Some compilers don't allow NULL to be passed for a va_list,
and e.g. "gcc (Raspbian 6.3.0-18+rpi1+deb9u1) 6.3.0 20170516"
errors out like this:
 trace2/tr2_tgt_event.c:193:18:
   error: invalid operands to binary &&
   (have ‘int’ and ‘va_list {aka __va_list}’)
    if (fmt && *fmt && ap) {
                       ^^
I couldn't find any hints that va_list and pointers can be mixed,
and no hints that they can't either. Morten Welinder comments:

"C99, Section 7.15, simply says that va_list "is an object type suitable for
holding information needed by the macros va_start, va_end, and
va_copy". So clearly not guaranteed to be mixable with pointers...

The portable solution is to use "va_list" everywhere in the callchain.
As a consequence, both trace2_region_enter_fl() and trace2_region_leave_fl()
now take a variable argument list.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Acked-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20 09:48:53 +09:00
Jeff Hostetler
ee4512ed48 trace2: create new combined trace facility
Create a new unified tracing facility for git.  The eventual intent is to
replace the current trace_printf* and trace_performance* routines with a
unified set of git_trace2* routines.

In addition to the usual printf-style API, trace2 provides higer-level
event verbs with fixed-fields allowing structured data to be written.
This makes post-processing and analysis easier for external tools.

Trace2 defines 3 output targets.  These are set using the environment
variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT".  These may be
set to "1" or to an absolute pathname (just like the current GIT_TRACE).

* GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command
  summary data.

* GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE.
  It extends the output with columns for the command process, thread,
  repo, absolute and relative elapsed times.  It reports events for
  child process start/stop, thread start/stop, and per-thread function
  nesting.

* GIT_TR2_EVENT is a new structured format. It writes event data as a
  series of JSON records.

Calls to trace2 functions log to any of the 3 output targets enabled
without the need to call different trace_printf* or trace_performance*
routines.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:27:59 -08:00