CallFlow Sequence Diagram Generator
===================================
This is a collection of awk and shell scripts that will take a capture
file that can be read by ethereal and produce a callflow sequence
diagram. The scripts have been primarily tested with SIP call flows,
but should work for other network traffic as well.
Examples of a generated callflow can be found here and here
Building the callflow package
=============================
Prerequisites:
batik-1.5beta3
java (jre 1.3 or greater)
tethereal
1. Obtain batik-1.5beta3 from
http://xml.apache.org/batik/dist/archives/batik-1.5beta3.zip
2. Set the Makefile variable BATIK_LOCATION to the location
of the Batik directory.
3. Type make
Using callflow
==============
With callflow in your path, just type
callflow capture-file
| -t text-input-file
| -o capture-file
| -d text-input-file
| -s text-input-file
This will produce at least a file called callflow.svg and a
directory called frames. If you have java installed you will
also get a callflow.jpg file and an index.html file.
You can view callflow.svg with the Adobe SVG plugin, or you can
view index.html with any web browser. Both the SVG file and the
html file contain links into the frames directory so that you can
look at the contents of the full packet frame. All the frames have
been processed to remove the IP headers, which usually aren't
interesting.
Filtering the diagram
=====================
You can create a file in the current directory called filter, or a
file in your home directory called .filter which contains an ethereal
DISPLAY filter (as opposed to a capture filter). You can learn
more about these from the ethereal man page. A useful filter for
SIP traffic is 'sip'.
Ordering the diagram
====================
If you know what nodes in your network are logically connected, you
can have the diagram ordered in a meaningful way. Just create a file
called order in your current directory, or .order in your home
directory with the order of your network nodes, one per line. You
must use the exact name or IP address that appears in your capture
file in the form: <host/ip>:<port>, or use a regular expression that
resolves to that name. Using a regular expression can be useful
to collapse several nodes in to one logical node.
You can generate an order file with:
$ callflow -o capture-file > order
The format of the order file is as follows:
<node-regex-pattern> [optional-label]
Network nodes that are in your filtered capture file but not in your
order file will appear on the right of the diagram, and nodes that are
in your order file but not in the filtered capture will not be present
in the diagram.
Labeling the diagram
====================
First, you need to create on order file, as above. Anything that
appears after the node name or IP will be used as the label in
the diagram. Otherwise, the node name or IP will be used as the
label.
Forcing a node into the diagram
===============================
If you would like a node that has no traffic to appear in the diagram,
put the string "!f!" as part of its label in the order file.
The string "!f!" will be edited out of the label. This can
be useful to show the position of a firewall in a trace, or to
illustrate a proxy that does not receive traffic.
Make sure that the forced node does not resolve to a regex pattern that
another node will match! (ie. don't use ".")
Example order file
==================
10.250.250.101:(1917|5060) Pingtel
10.250.250.201:5060 VOCAL-UAMS
10.250.250.201:5070 VOCAL-RS
10.250.250.201:5065 VOCAL-GWMS
firewall !f!Firewall
10.250.250.104:5060 PSTN-GW
Titling the diagram
===================
You can create a file in the current directory called title, or a
file in your home directory called .title which contains text to be
displayed as a title to the diagram. If neither of these files exist,
then a title text will be generated based upon basename of the capture
file name.
Tweaking ~./ethereal/preferences for frames/Frame*.html
=======================================================
You can change the level of detail provided in the "Internet Protocol",
"User Datagram Protocol", and "Transmission Control Protocol" sections of
the detailed frames/Frame*.html pages by tweaking the following entries
in your ~/.ethereal/preferences:
ip.ip_summary_in_tree
tcp.tcp_summary_in_tree
udp.udp_summary_in_tree
If any of these items are set to TRUE, then only the summary line will
appear in frames/Frame*.html. Otherwise, the gory details will be
displayed.
Creating diagrams manually
==========================
You can manually create a diagram by creating two text files, foo and
foo.long.
When you run callflow, you can take the output files callflow.short and
callflow.long as a starting point. Just rename callflow.short so it doesn't
have the .short extension.
The first file contains the trace information, one
transaction per line, in the following format:
<frame #> <src node> <src port> <ignored> <dest node> <dest port> <description>
For example:
1 Alice 1000 -> Bob 2000 Alice sends Bob an Invite
2 Bob 2000 -> Alice 1000 Bob sends Alice a 200
etc.
If this file is called foo, you can create another file called foo.long
with longer descriptions of each transaction, in the following format:
Frame #
Descriptive information
goes here
Frame #
Description of another frame
For example:
Frame 1
Via: SIP/2.0/UDP 192.168.1.112:5060
From: "C7960 (x1201)" <sip:1201@192.168.1.42>;tag=aab70900293102348a-7008
To: <sip:1020@192.168.1.42>
Frame 2
To: <sip:1020@192.168.1.42:5060>;tag=53d20696
From: "C7960 (x1201)"<sip:1201@192.168.1.42:5060>;tag=aab70900293102348a-7008
Call-ID: aab70900-2fe3102-3ae-2b27@192.168.1.112
etc.
To process your text files, type the following:
callflow -t foo
Adding Comments
===============
You can add a comments to your diagram by adding a line that start with "# " to
short text-file. If in the middle of the line there is a " ! " token, then the
comment becomes a hyper-link with what precedes the " ! " is the text that is
displayed, and what comes after is the link to be followed when a user clicks on
the text.
For example:
# Click here for original ethereal capture file ! mycapture.cap
# This is a comment
1 Alice 1000 -> Bob 2000 Alice sends Bob an Invite
2 Bob 2000 -> Alice 1000 Bob sends Alice a 200
etc.
It is important that there be no blank lines in the short text-file.
Removing Duplicate Frames
=========================
You can remove duplicate frames from the short text-file with:
callflow -d foo
Duplicates are determined by examining each line in foo, and comparing
frames/Frame<#>.html with all previously seen frames. The non-duplicates version
of foo will be called "callflow.short".
Adding "Session ID" to Sequence Lines
=====================================
You can have callflow add a "Session ID" to every Frame that contains a pattern.
For instances, all SIP packets contains a header "Call-Id:" (case insensitive)
followed by an identifier that indicates which "SIP dialog" this packet belongs to
To have callflow create to the left of every sequence line that matches "Call-Id:"
a "Session ID", create a file at either ~/.session or ./session with the contents:
Call-Id:
and run callflow with:
callflow -s <text-file>
Sample session
==============
Create a filter file:
$ cat > filter << "EOF"
> sip
> EOF
Create a title file:
$ cat > title << "EOF"
> This is a sample capture
> EOF
Create an order file:
$ callflow -o mycapture.cap > order
(edit order file as needed)
Generate initial diagram:
$ callflow mycapture.cap
$ cp callflow.short callflow
We will now use "callflow" and "callflow.long" for further processing:
Remove retransmitted SIP packets:
$ callflow -d callflow
$ cp callflow.short callflow
Add "Session ID" to Sequence Lines:
$ cat > session << "EOF"
> Call-Id:
> EOF
$ callflow -d callflow
$ cp callflow.short callflow
Add any desired comments to callflow, then regenerate diagram (repeat as needed):
$ callflow -t callflow
How it works
============
Well, this is kind of ugly:
1. untar callflow shell script contents into .callflow directory
2. run .callflow/callflow.sh
a. check for order and filter files
b. run tethereal twice to get short and long output files
c. use long2html.awk to convert long output file into frame*.html files
d. get all the unique nodes from the short output file
e. order them and make sure all nodes are present
f. create the first few lines of the main awk script using makevars.awk
g. run the main awk script (callflow.awk) on the short output file:
I. create SVG file header and html <map> header
II. create labels in SVG file
III. for each line, create an arrow in the SVG file and a
<area> element for the image map
IV. write out the SVG and map file footers
h. check if java is installed, and if so generate index.html
and callflow.jpg using batik
i. clean up all the /tmp files
3. clean up .callflow
TO DO
=====
It would be much nicer to take the tethereal filtered output and
build an intermediate XML representation of it. Then use XSLT
to transform it into the SVG file. This way you could write up
illustrative sequence diagrams more easily and make use of more
tools that can manipulate XML.
Build sequence diagrams directly into ethereal.
Write port numbers at the ends of each arrow in an unobtrusive fashion.
Put packet contents right into diagram, available as dhtml popups, with
similar "drill-down" functionality as ethereal.
Automatically find a "best order" for the nodes, possibly based on
total arrow length minimization.
Specify output filenames and locations on the command line.
Add CSS support.