Remove --debug option and use a formatter instead
Originally created by @anonym on #9491 (Redmine)
Now that #9424 (closed) is solved, we could do something pretty interesting. We
could remove the --debug
option and replace it with a formatter that
includes debug information. It’d be nice to call it simply debug
, but
it already exists (and is useful when developing new formatters,
apparently), but we could override it with our one, or pick a new name,
like pretty_debug
. So, this formatter would be the pretty one, but
also have all the debug info we print to STDERR, and the info we let
some subprocesses (sikuli, guestfs) print to STDERR when --debug
is
enabled.
To make a formatter that we can print to with e.g. a debug_log()
function is pretty straight-forward:
def debug_log(message)
return if not($config["DEBUG"])
$debug_log_fns.each { |fn| fn.call(message) }
end
require 'cucumber/formatter/pretty'
module ExtraFormatters
class PrettyDebug < Cucumber::Formatter::Pretty
def initialize(*args)
super(*args)
$debug_log_fns ||= []
$debug_log_fns << self.method(:debug_log)
end
def debug_log(msg)
@io.puts(msg)
end
end
end
module Cucumber
module Cli
class Options
BUILTIN_FORMATS['debug'] = ['ExtraFormatters::PrettyDebug', 'Prints the feature with debuggin information - in colours.']
end
end
end
The hard part is to connect the STDOUT/STDERR of subprocesses so it writes to the formatters destination (which can be a file or the terminal/stdout). I only have some vague ideas based on crazy stuff that the JUnit formatter does with some sort of stream interceptor. Not sure at all if that will work out though.
Now, why would we want to do this? Well, the output when --debug
is
enabled is quite hard to follow given the amount of debugging info we
print (especially for Tor leak tests) so I generally run without it.
However, when there’s an issue I often regret not having the debug info,
since some issues do not reappear every time due to our robustness
issues. But if we had this we could run cucumber with --format pretty --format debug --out /tmp/debug.log
so that we get the pretty non-debug
written to the terminal that is easy to follow, and the full debug info
(including the pretty output, for the all-important context) written to
a file that we can investigate if there’s some unexpected error.
Awesome! Currently we can do the opposite (pretty + debug to the
terminal, pretty only to file) with --debug -- --format pretty --format pretty --out /tmp/pretty.log
which isn’t as nice, but I actually do
this quite often.
Note if we have this we can also kill run_test_suite
’s --log-to-file
option.
Feature Branch: test/9491-debug-cucumber-formatter