Once Bro has been deployed in an environment and monitoring live traffic, it will, in its default configuration, begin to produce human-readable ASCII logs. Each log file, produced by Bro’s Logging Framework, is populated with organized, mostly connection-oriented data. As the standard log files are simple ASCII data, working with the data contained in them can be done from a command line terminal once you have been familiarized with the types of data that can be found in each file. In the following, we work through the logs general structure and then examine some standard ways of working with them.
Generally, all of Bro’s log files are produced by a corresponding
script that defines their individual structure. However, as each log
file flows through the Logging Framework, they share a set of
structural similarities. Without breaking into the scripting aspect of
Bro here, a bird’s eye view of how the log files are produced
progresses as follows. The script’s author defines the kinds of data,
such as the originating IP address or the duration of a connection,
which will make up the fields (i.e., columns) of the log file. The
author then decides what network activity should generate a single log
file entry (i.e., one line). For example, this could be a connection
having been completed or an HTTP GET
request being issued by an
originator. When these behaviors are observed during operation, the
data is passed to the Logging Framework which adds the entry
to the appropriate log file.
As the fields of the log entries can be further customized by the
user, the Logging Framework makes use of a header block to ensure that
it remains self-describing. This header entry can be see by running
the Unix utility head
and outputting the first lines of the file:
1 | # bro -r wikipedia.trace
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | #separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#open 2018-03-08-00-43-41
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents
#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string]
1300475168.724007 Cx3Bgp1UPFu1e3OR8h 141.142.220.118 48649 208.80.152.118 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.855330 CwevtS2YVW733ZOP33 141.142.220.118 49997 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.855305 C4Erbx2KQiOTEIpHmk 141.142.220.118 49996 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.859163 ChUkQsy5dfX85LPxj 141.142.220.118 49998 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.892936 CuPxvFMDIME55REic 141.142.220.118 50000 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.892913 CAmvS51j6gFXGXU3oe 141.142.220.118 49999 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
1300475168.895267 ClQ3PB2k6Ltu6rGHQ7 141.142.220.118 50001 208.80.152.3 80 tcp - - - - OTH - - 0 C 0 0 0 0 (empty)
[...]
|
As you can see, the header consists of lines prefixed by #
and
includes information such as what separators are being used for
various types of data, what an empty field looks like and what an
unset field looks like. In this example, the default TAB separator is
being used as the delimiter between fields (\x09
is the tab
character in hex). It also lists the comma as the separator for set
data, the string (empty)
as the indicator for an empty field and
the -
character as the indicator for a field that hasn’t been set.
The timestamp for when the file was created is included under
#open
. The header then goes on to detail the fields being listed
in the file and the data types of those fields, in #fields
and
#types
, respectively. These two entries are often the two most
significant points of interest as they detail not only the field names
but the data types used. When navigating through the different log
files with tools like sed
, awk
, or grep
, having the field
definitions readily available saves the user some mental leg work. The
field names are also a key resource for using the bro-cut utility included with Bro, see below.
Next to the header follows the main content. In this example we see 7
connections with their key properties, such as originator and
responder IP addresses (note how Bro transparently handles both IPv4 and
IPv6), transport-layer ports, application-layer services ( - the
service
field is filled in as Bro determines a specific protocol to
be in use, independent of the connection’s ports), payload size, and
more. See Conn::Info
for a description of all fields.
In addition to conn.log
, Bro generates many further logs by
default, including:
dpd.log
dns.log
ftp.log
files.log
http.log
known_certs.log
smtp.log
ssl.log
weird.log
As you can see, some log files are specific to a particular protocol, while others aggregate information across different types of activity. For a complete list of log files and a description of its purpose, see Log Files.
bro-cut
The bro-cut
utility can be used in place of other tools to build
terminal commands that remain flexible and accurate independent of
possible changes to the log file itself. It accomplishes this by parsing
the header in each file and allowing the user to refer to the specific
columnar data available (in contrast to tools like awk
that
require the user to refer to fields referenced by their position).
For example, the following command extracts just the given columns
from a conn.log
:
1 2 3 4 5 6 7 8 9 10 11 12 | # cat conn.log | bro-cut id.orig_h id.orig_p id.resp_h duration
141.142.220.118 48649 208.80.152.118 -
141.142.220.118 49997 208.80.152.3 -
141.142.220.118 49996 208.80.152.3 -
141.142.220.118 49998 208.80.152.3 -
141.142.220.118 50000 208.80.152.3 -
141.142.220.118 49999 208.80.152.3 -
141.142.220.118 50001 208.80.152.3 -
141.142.220.118 35642 208.80.152.2 -
141.142.220.202 5353 224.0.0.251 -
fe80::217:f2ff:fed7:cf65 5353 ff02::fb -
[...]
|
The corresponding awk
command will look like this:
1 2 3 4 5 6 7 8 9 10 11 12 | # awk '/^[^#]/ {print $3, $4, $5, $6, $9}' conn.log
141.142.220.118 48649 208.80.152.118 80 -
141.142.220.118 49997 208.80.152.3 80 -
141.142.220.118 49996 208.80.152.3 80 -
141.142.220.118 49998 208.80.152.3 80 -
141.142.220.118 50000 208.80.152.3 80 -
141.142.220.118 49999 208.80.152.3 80 -
141.142.220.118 50001 208.80.152.3 80 -
141.142.220.118 35642 208.80.152.2 80 -
141.142.220.202 5353 224.0.0.251 5353 -
fe80::217:f2ff:fed7:cf65 5353 ff02::fb 5353 -
[...]
|
While the output is similar, the advantages to using bro-cut over
awk
lay in that, while awk
is flexible and powerful, bro-cut
was specifically designed to work with Bro’s log files. Firstly, the
bro-cut
output includes only the log file entries, while the
awk
solution needs to skip the header manually. Secondly, since
bro-cut
uses the field descriptors to identify and extract data,
it allows for flexibility independent of the format and contents of
the log file. It’s not uncommon for a Bro configuration to add extra
fields to various log files as required by the environment. In this
case, the fields in the awk
command would have to be altered to
compensate for the new position whereas the bro-cut
output would
not change.
Note
The sequence of field names given to bro-cut
determines the
output order, which means you can also use bro-cut
to reorder
fields. That can be helpful when piping into, e.g., sort
.
As you may have noticed, the command for bro-cut
uses the output
redirection through the cat
command and |
operator. Whereas
tools like awk
allow you to indicate the log file as a command
line option, bro-cut only takes input through redirection such as
|
and <
. There are a couple of ways to direct log file data
into bro-cut
, each dependent upon the type of log file you’re
processing. A caveat of its use, however, is that all of the
header lines must be present.
Note
bro-cut
provides an option -c
to include a corresponding
format header into the output, which allows to chain multiple
bro-cut
instances or perform further post-processing that
evaluates the header information.
In its default setup, Bro will rotate log files on an hourly basis,
moving the current log file into a directory with format
YYYY-MM-DD
and gzip compressing the file with a file format that
includes the log file type and time range of the file. In the case of
processing a compressed log file you simply adjust your command line
tools to use the complementary z*
versions of commands such as cat
(zcat
) or grep
(zgrep
).
bro-cut
accepts the flag -d
to convert the epoch time values
in the log files to human-readable format. The following command
includes the human readable time stamp, the unique identifier, the
HTTP Host
, and HTTP URI
as extracted from the http.log
file:
ERROR executing test 'doc.sphinx.using_bro' (part 4)
% 'btest-rst-cmd -n 5 "bro-cut -d ts uid host uri < http.log"' failed unexpectedly (exit code 1)
% cat .stderr
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP and UDP checksums, most likely from NIC checksum offloading. By default, packets with invalid checksums are discarded by Bro unless using the -C command-line option or toggling the 'ignore_checksums' variable. Alternatively, disable checksum offloading by the network adapter to ensure Bro analyzes the actual checksums that are transmitted.
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired.
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
Often times log files from multiple sources are stored in UTC time to
allow easy correlation. Converting the timestamp from a log file to
UTC can be accomplished with the -u
option:
ERROR executing test 'doc.sphinx.using_bro' (part 5)
% 'btest-rst-cmd -n 5 "bro-cut -u ts uid host uri < http.log"' failed unexpectedly (exit code 1)
% cat .stderr
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP and UDP checksums, most likely from NIC checksum offloading. By default, packets with invalid checksums are discarded by Bro unless using the -C command-line option or toggling the 'ignore_checksums' variable. Alternatively, disable checksum offloading by the network adapter to ensure Bro analyzes the actual checksums that are transmitted.
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired.
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
The default time format when using the -d
or -u
is the
strftime
format string %Y-%m-%dT%H:%M:%S%z
which results in a
string with year, month, day of month, followed by hour, minutes,
seconds and the timezone offset. The default format can be altered by
using the -D
and -U
flags, using the standard strftime
syntax. For example, to format the timestamp in the US-typical “Middle
Endian” you could use a format string of: %d-%m-%YT%H:%M:%S%z
ERROR executing test 'doc.sphinx.using_bro' (part 6)
% 'btest-rst-cmd -n 5 "bro-cut -D %d-%m-%YT%H:%M:%S%z ts uid host uri < http.log"' failed unexpectedly (exit code 1)
% cat .stderr
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP and UDP checksums, most likely from NIC checksum offloading. By default, packets with invalid checksums are discarded by Bro unless using the -C command-line option or toggling the 'ignore_checksums' variable. Alternatively, disable checksum offloading by the network adapter to ensure Bro analyzes the actual checksums that are transmitted.
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired.
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
See man strfime
for more options for the format string.
While Bro can do signature-based analysis, its primary focus is on behavioral detection which alters the practice of log review from “reactionary review” to a process a little more akin to a hunting trip. A common progression of review includes correlating a session across multiple log files. As a connection is processed by Bro, a unique identifier is assigned to each session. This unique identifier is generally included in any log file entry associated with that connection and can be used to cross-reference different log files.
A simple example would be to cross-reference a UID seen in a
conn.log
file. Here, we’re looking for the connection with the
largest number of bytes from the responder by redirecting the output
for cat conn.log
into bro-cut to extract the UID and the
resp_bytes, then sorting that output by the resp_bytes field.
1 2 3 4 5 6 | # cat conn.log | bro-cut uid resp_bytes | sort -nrk2 | head -5
CzZNJF09F592w9c8 -
CxwIIF2z9M2e620Cye -
Cx3Bgp1UPFu1e3OR8h -
CwgpmF27aRpCOZ7n8a -
CwevtS2YVW733ZOP33 -
|
Taking the UID of the first of the top responses, we can now
crossreference that with the UIDs in the http.log
file.
ERROR executing test 'doc.sphinx.using_bro' (part 8)
% 'btest-rst-cmd "cat http.log | bro-cut uid id.resp_h method status_code host uri | grep UM0KZ3MLUfNB0cl11"' failed unexpectedly (exit code 1)
% cat .stderr
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP and UDP checksums, most likely from NIC checksum offloading. By default, packets with invalid checksums are discarded by Bro unless using the -C command-line option or toggling the 'ignore_checksums' variable. Alternatively, disable checksum offloading by the network adapter to ensure Bro analyzes the actual checksums that are transmitted.
1300475173.475401 warning in /builddir/build/BUILD/bro-2.5.3/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired.
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
/builddir/build/BUILD/bro-2.5.3/testing/btest/../../aux/btest/sphinx/btest-rst-cmd: line 81: http.log: No such file or directory
DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
If you cannot make the switch now, please follow instructions at https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
cat: http.log: No such file or directory
As you can see there are two HTTP GET
requests within the
session that Bro identified and logged. Given that HTTP is a stream
protocol, it can have multiple GET
/POST
/etc requests in a
stream and Bro is able to extract and track that information for you,
giving you an in-depth and structured view into HTTP traffic on your
network.