Overview   Package  Class  Tree  Deprecated  Index  Help 
 PREV PACKAGE   NEXT PACKAGE FRAMES    NO FRAMES    

Package org.jnetstream.capture

A high performance network packet capture framework.

See:
          Description

Interface Summary
Capture<T extends CapturePacket> Main capture session interface.
CaptureDevice Describes the capture device that captured the packet.
CaptureDeviceCounters  
CaptureDeviceState  
CaptureListener A listener that can be registered with a Capture object, to be notified of capture events.
CapturePacket A captured network packet.
CapturePacketInput<T extends CapturePacket> CapturePacketInput interface reads CapturePacket objects from underlying storage or stream which is defined by the class implementing this interface.
CapturePacketOutput<T extends Packet> Allows writting of CapturePackets.
Captures.Factory A factory interface for interfacing with implementation of jNetStream API.
Captures.LiveCaptureFactory Factory classes which create live capture sessions and transmit packets.
Captures.LocalFactory Factory interface for local sessions.
Captures.RemoteFactory Factory interface for remote sessions.
DeserializedPacket A packet that was sent accross a stream.
FastIterator Special iterator that allows very low level iteration over large buffers of aquired packet data.
FileCapture<T extends FilePacket> File capture extends the basic Capture interface and adds several capabilites which can only be done with files, as opposed to live captures for example.
FilePacket A file based packet.
FormatType.Detail Detailed information about a specific format.
InputCapture<T extends CapturePacket> Formatted InputStream based capture session.
InputCapture.FormatTypeOtherFactory Factory interface used by InputCapture factories to load NPL based files who's format is FormatType.Other.
InputIterator<T> Iterator which iterates over the elements within the stream.
LiveCapture A network packet cature on a live network interface.
LiveCaptureDevice  
LiveIterator IO based iterator over a live capture session.
LivePacket A packet that was captured on a live network interface.
LivePacketFactory Factory for creating new Live Packets
NetTransmitter Transmitter which transmits or send low level packets using the link layer of the open network interface.
OutputCapture A capture session that allows content to be sent to a formatted stream.
PacketIndexer<I extends CapturePacket> All getter methods, that return a packet return subclass of CapturePacket while all mutable methods, methods that take packets, simply require any type of packet.
PacketIterator<I extends CapturePacket> All getter methods, that return a packet return subclass of CapturePacket while all mutable methods, methods that take packets, simply require any type of packet.
PacketModifier<T extends CapturePacket> ElementModifier interface allows a capture session that is mutable (see Capture.isMutable()) to add, insert, swap and remove packets from the underlying packet dataset of the capture session.
RemoteSession A remote session which allows remote capture of live packets or capture file manipulation.
RemoteSession.RemoteSessionFactory Remote session handler is an object that is called by the CaptureServer to handle an incomming RemoteSession connection.
 

Class Summary
Captures Static factory methods for creating and accessing capture sessions.
Messages  
PacketInputStream A PacketInputStream deserializes Packet objects into DeserializedPacket objects previously written using a PacketOutputStream.
PacketOutputStream A PacketOutputStream writes CapturePacket objects to an OutputStream.
RemoteServer Allows remote capture sessions to be started from non-local system.
TestCaptureSyntax  
 

Enum Summary
CaptureDevice.TimestampResolution Specified the maximum supported timestamp resolution of the capture device.
CapturedProperty Enum constants that describe certain properties and thus information a particular type of record may contain.
CaptureType Defines the type of capture this is.
FileMode Defines a file mode for either open or new files created.
FormatType Some standard supplied file types with some convenience methods.
LiveCapture.Option Options which can be set on open captures.
 

Exception Summary
CaptureException  
CaptureOpenException Any errors while opening a capture session.
FileFormatException Thrown when Invalid format has been encountered in the capture file.
RemoteAuthenticationException Indicates the remote authentication failed.
RemoteProtocolException If a protocol error occures between CaptureSender and CaptureReceiver this exception will contain specific information about the failure.
 

Package org.jnetstream.capture Description

A high performance network packet capture framework. This is a high level packet capture framework that abstracts all of the complexity of live network captures, reading captured packets from capture files (trace files), streaming packet data accross java.io streams or enabling remote capture sesssion accross the network. The org.jnetstream.capture package allows easy processing and capture of network packets. Live packets can be captured on a local network interface or remotely.

The main class you will be dealing with is the Capture interface. You aquire a reference to a Capture instance using factory methods in the Captures class for local sessions and RemoteSession for, you guessed it, remote sessions on a remote server. (To get a RemoteSession call the static method RemoteServer.openSession(URI) .) The factory methods return the most specific subclass of LiveCapture and FileCapture, all of which are subclasses of the base Capture interface. So remember that LiveCapture and FileCapture both extend Capture. You will see in the examples below the use of Capture and sometimes the more specific subclass depending if the method involved in the example need something out of the more specific subclass such as LiveCapture.interfaceSnapshot() method (see section "CaptureInterface state changes" below.)

CapturePacket is a high level object that holds a buffer with packet data as captured from a live network interface, a file or remote sender. It also contains some additional information needed for proper processing of the buffer. Also contains some meta data such as capture timestamp, a reference to the CaptureDevice which holds detailed descriptors about the network interface that the packet was captured on.

Getting started, aquiring a Capture instance

The Captures class provides the following static factory methods which instantiate capture sessions:

And many many more variations of the above.

Reading from a Capture instance

Next you will want get the packets from a capture session. The Capture interaface extends the IOIterator interface which uses the familiar hasNext() and next() method calls. So lets just open a live capture on all network interfaces a system has and print the packet properties to stdout: 
Capture capture = Captures.openLive(); // Opens up all network interfaces, except loopback ints
while (capture.hasNext()) {
        CapturePacket packet = capture.next();
        System.out.println("packet properties=" + packet.toString());
}
Note: Note that you also need to catch some exceptions that are declared, but not included in the example.

Capturing packets from live network interface and storing them in a capture file

Another common usage is to capture packets and store them in a file.

So we first setup a source of packets which is a live network capture from a specific network interface. 

List<CaptureDevice> ints = Captures.listInterfaces();
if (ints.isEmpty()) {
        return; // No interfaces on this system
}
CaptureInterface netint = ints.get(0); // Just grab the first network interface
Capture capture = Captures.openLive(netint);
So now we have a source of packets which is the first network interface returned all the interfaces. Next we want to create a new file and append all the captured packets to that file. We will create the file in PCAP format. 
Captures.newFile(new File("mycapture.pcap", SuppliedFileTypes.PCAP, capture);
The above line creates a new file called "mycapture.pcap" in PCAP format (2nd parameter). At this point all the needed initialization is done and a new file created with the right file header. The the last argument is the source of packets, our capture instance. The above method iterates through all the way to the end using Capture#hasNext() and Capture#next() method calls until hasNext eventually returns false. Those packets are written in the PCAP format into the capture file.

Filtering

Filtering of packets being captured or read from a file is another common operation. Filtering involves creating a Filter object and using one of the factory methods that takes a filter. Most methods have varioutions of a call that does take a filter. Lets modify our above example and use a filter to only capture IP packets. 
List<CaptureDevice> ints = Captures.listInterfaces();
if (ints.isEmpty()) {
        return; // No interfaces on this system
}
CaptureDevice netint = ints.get(0); // Just grab the first network interface
Filter filter = new BpfFilter(new PcapExpression("ip"));
Capture capture = Captures.openLive(filter, netint);

The filter is a BPF filter (Berkley Packet Filter) that efficiently filters incomming packets being captured and only allows packets that match the filter criteria to be returned. The filter is actually applied to operating system kernel (if OS supports this option) and the packet capture driver efficiently filters packets without any extraneous copies of packet contents. Lastly the filter is simply supplied as the first parameter to the openLive method.

An alternate way to set a filter is to set the filter on the CaptureDevice. Such as in this case: 

List<CaptureDevice> ints = Captures.listInterfaces();
if (ints.isEmpty()) {
        return; // No interfaces on this system
}
CaptureDevice netint = ints.get(0); // Just grab the first network interface
Filter filter = new BpfFilter(new PcapExpression("ip"));
netint.setFilter(filter);
Capture capture = Captures.openLive(netint);

This sets the filter only on this interface and not any other interfaces if they were specified. You can also change the filter after the capture has begun and even some packets returned. The only way to change a filter after the CaptureInterface has been applied, is to set it on the capture itself and not on the device instance directly. 

List<CaptureDevice> ints = Captures.listInterfaces();
if (ints.isEmpty()) {
        return; // No interfaces on this system
}
CaptureDevice netint = ints.get(0); // Just grab the first network interface
Filter ipFilter = new BpfFilter(new PcapExpression("ip"));
netint.setFilter(ipFilter);
Capture capture = Captures.openLive(netint);

// Get 10 IP packets
for (int i = 0; i < 10 && capture.hasNext(); i ++) {
  System.out.println("CaptureInstance=" + capture.next().getCaptureInterface());
}

// After netint has been applied it becomes immutable and we can no longer call
// on its CaptureDevice.setFilter method. Use CaptureInterface.isMutable() to check for this.

Filter ipxFilter = new BpfFilter(new JpcapExpression("ipx"));
CaptureInterface ni = capture.setFilter(filter, netint); // Sets a new filter and creates a new instance of CaptureInterface

// Get 10 IPX packets
for (int i = 0; i < 10 && capture.hasNext(); i ++) {
  System.out.println("CaptureInstance=" + capture.next().getCaptureInterface());
}

capture.close();
The call to capture.setFilter(filter, netint) does not modify the netint instance, this instance is immutable after the call to Captures.openLive(netint). What happens is a new instance of CaptureInterface is created based on the old one, and the new filter set in the new instance. The call returns the new CaptureInstance reference, notice its no longer returned as CaptureDevice as its immediately applied to captures and becomes immutable. The reason for this strict mutable/immutable control is that any perviously captured packets will still reference the same CaptureInterface reference at the time they were captured. New packets returned after the new filter was applied to filter on IPX packets, will return references to new CaptureInterface that has the new filter applied.

Appending captured packets to an existing file

To append packets to an existing file you need to first get your source of packets setup, open up the capture file and use the methods in MutableFileCapture or simply call a utility method in Captures.append() to do the copying for you. This time lets use a NAP file and do the copying ourselves: 

Capture capture = Captures.openLive(); // Opens up all network interfaces, except loopback
MutableCaptureFile file = CaptureFiles.openFile(new File("mycapture.nap");
file.last(); // advance the iterator to past the last packet in the file

while (capture.hasNext()) {
        CapturePacket packet = capture.next();
        file.add(packet);
}
First we create a new capture which opens up all the network interfaces for live capture. Second we open a NAP file. Third we advance the file position to the end of the file by calling MutableFileCapture#last which returns the last packet and advances the position past it. We want to append our packets to the end not insert them at the beginning although for an empty file beginning is the last element. Lastly we go into a loop that reads one packet at a time and using the mutable file adds or appends the packets to the file.

We could have done steps 3 and 4 with a utility method. Here is the same example using the utility method: 

Capture capture = Captures.openLive(); // Opens up all network interfaces
CaptureFile file = CaptureFiles.openFile(new File("mycapture.nap"));
CaptureFiles.append(file, capture);
Or this compact notation: 
CaptureFiles.catFile(new File("mycapture.nap"), Captures.openLive());

CaptureInterface state changes

It is certainly possible for a network interface on which a capture is taking place to change state such as different set of LiveCapture.Option has been set by a separate instance of LiveCapture, someone had alterned the IP addresses assigned to the interface at runtime, a new filter applied, etc etc... All the while our capture session is still in progress. There is a simple way to record these changes by taking a snapshot of the interface config. After a snapshot is taken any packets returned will reference new CaptureInterface objects that reflect the exact config of the interface, while any previously returned packets will continue to reference the old CaptureInterface objects which contain the state before the snapshot took place. To take a snapshot simply call the LiveCapture.interfaceSnapshot method. 

LiveCapture capture = Captures.openLive();
for (int i = 0; i < 10000 && capture.hasNext(); i ++) {
  LivePacket packet = capture.next();
  // Do something with the packet
}

// Some even caused one of the interface to change state, i.e. new IP address alias added to the interface
capture.interfaceSnapshot();

// Now capture 10000 more packets which reference new CaptureInterface objects
for (int i = 0; i < 10000 && capture.hasNext(); i ++) {
  LivePacket packet = capture.next();
  // Do something with the packet
}

capture.close();
The call to interfaceSnapshot() creates new CaptureInterface instances on all currently open interfaces. The new CaptureInterfaces are initialized to the latest state of the network interface, reflecting the fact that new IP address alias has been added to one network interfaces. You could more specifically target the exact interface instead of taking a snapshot of all interface by using the variation of the method interfaceSnapshot(CaptureInterface).

Note that using the Capture.setFilter method to set a new filter automatically takes a snapshot to reflect the changed filter. It is not required to take a second snapshot by explicitely calling another interfaceSnapshot.

Static factory vs. abstract factory and working with RemoteSession objects

All the above examples have used the static factory methods of Captures class. There is another way by using the abstract factory methods of Captures.LocalFactory and Captures.RemoteFactory interface methods both of which extend the common Captures.Factory, super interface. The static Captures methods are delegate methods that simply call on the default Captures.LocalFactory instance. You can use the Captures.getLocal() method to aquire a reference to the main factory that does all the work. To get a reference to the RemoteFactory reference you call on RemoteServer.openSession method which returns RemoteSession which is simply a subclass of the RemoteFactory interface. 
List<CaptureDevice> ints;
LiveCapture live;
FileCapture file;

Captures.LocalFactory lf = Captures.getLocal();
ints = lf.listInterfaces(); // List of all interfaces on local system
live = lf.openLive();                        // Open all interfaces on local system for capture
file = lf.openFile(new File("myfile.pcap")); // Open file on local system for reading and writting
live.close();
file.close();

Captures.RemoteFactory rf = RemoteServer.openSession(new URI("192.168.1.1"));
ints = rf.listInterfaces();                              // List of all interfaces on remote system
live = rf.openLive();                                    // Open all interfaces on remote system for capture
file = rf.openFile(new File("/tmp/myfile.pcap"));        // Open file on remote system for reading and writting

live.close();
file.close();
Or even more abstracted since all these above methods are actually in the super interface Captures.Factory: 
List<CaptureDevice> ints;
LiveCapture live;
FileCapture file;
Captures.Factory factory;

factory = Captures.getLocal();
ints = factory.listCaptureDevices();                        // List of all interfaces on local system
live = factory.openLive();                              // Open all interfaces on local system for capture
file = factory.openFile(new File("myfile.pcap"));       // Open file on local system for reading and writting
live.close();
file.close();

RemoteSession session = RemoteServer.openSession(new URI("192.168.1.1"));
factory = session;                                      // RemoteSession extends RemoteFactory     
ints = factory.listCaptureDevices();                        // List of all interfaces on remote system
live = factory.openLive();                              // Open all interfaces on remote system for capture
file = factory.openFile(new File("/tmp/myfile.pcap"));  // Open file on remote system for reading and writting

live.close();
file.close();
session.close(); // We needed to keep RemoteSession reference only so that we could explicitly close it
                 // because RemoteFactory itself is not Closeable.
You get the point. There is not much difference in invoking factory methods for local or remote sessions. There are some restrictions and they are reflected in methods included in the LocalFactory and RemoteFactory interfaces. Certain methods that can only be invoked locally are not included in the RemoteFactory and vise versa. The above example assumes that you have an instance of RemoteServer running on the remote system at IP address 192.168.1.1. It should be clear at this point that the static Captures methods are only defined for all LocalFactory abstract methods. RemoteSession itself adds a few additional methods related to a remote session besides the methods extended from RemoteFactory.

Creating a new file and inserting customized records

Another thing you may wish to do, is to actually control all the parameters of a capture file. So lets do an example of where we create a Pcap file with a custom block record (file header).

There are actually several ways to create an empty file in jNetStream capture framework.

Once we have a Pcap capture file we can proceed to tweak its contents. By default the capture file is created with a PcapBlockRecord already in place with default values, there is no way to create a file without these defaults. But its easy to clear out the file to its absolute minimum. 
PcapFile pcap = FileType.Pcap.newFile("capture.pcap"):
PcapModifier modifier = pcap.getPcapIterator();
modifier.removeAllRecords();
At this point the file size is 0 bytes, all records have been completely removed. We can then proceed to add our custom block record with our new values: 
PcapFile pcap = FileType.Pcap.newFile("capture.pcap"):
PcapModifier modifier = pcap.getPcapIterator();
modifier.removeAllRecords();

PcapBlockRecord block = modifier.addBlockRecord(
        new byte[] {0x11, 0x22, 0x33, 0x44}, // Magic Number
        2, 4,                                // Major and minor verions
        PcapDLT.EN10,                        // DLT
        0,                                   // Accuracy
        0,                                   // Timezone
        1024                                 // Snaplen
);
We're convieniently returned a reference to the newly created block record. We can then proceed to add packet records which is the only other type of record a Pcap file can have using PcapModifier.addPacketRecord() method. Notice that we added a bogus magic number, which the API allowed us to do. This creates an invalid pcap file that no program could read in, but if that's what the user wanted to do, its allowed.

The other file formats, especially NAP file formats are a lot more involved with many different types of records possible and their Modifiers are much more comprehensive. 

NapFile nap = FileType.Nap.newFile("capture.nap");
NapModifier modifier = nap.getNapIterator();
/* 
 * Now, using the NapModifier, we can add block, property, capture device, packet records.
 * NapModifier allows addition of over a dozen different types of NAP records.
 */
You get the point.

Efficiency

Factory methods may be much more efficient at copying packets around as more efficient algorithms may be used to accomplish the task then simply iterating one packet at a time. For example if you were appending PCAP file as sources to a destination file that was also a PCAP file, then raw bulk buffers of source file data may be copied at once, without the need to even do the minimal decoding that is done when packets are iterated over. Or if the destination file was a PCAP file and the source LiveCapture was also based on libpcap implementation (as it is, part of the first release) the much more efficient offline capture would be initiated which uses kernel drivers to dump packets directly into the destination capture file; a much more efficient short cut indeed. So the capture framework will pick the most efficient algorithm available for the circumstances at hand.

The capture framework uses java.nio package extensively to allocate large direct buffers, map file contents into memory using native OS shortcuts and then, decodes only the smallest amount of data needed to accomplish the task. The objects returned such as CapturePacket and its subclassed variations reference into these buffers and don't typically allocate a lot of memory themselves. This means these objects can be created efficiently. Multiple packets are typically prefetched into a cache in larger chunks and share a single large buffer. Some buffers map file contents directly into buffer's memory ensuring that the file contents are only read into memory once; by the kernel.

Also use the methods that take entire collections instead of accomplishing the same task by using methods that take a single element and you doing the iteration. This is especially true when modifying files or inserting/deleting packets from files. The implementation algorithms are tuned for executing operations as efficiently as possible given all the information supplied. So inserting or deleting a bunch of packets at once in a file is much more efficient then inserting or deleting a single packet at a time. The same is true for all methods that accept collections or arrays.

Future enhancements may provide additional efficiencies as technologies develop and this project matures. The API is designed to hide the implementation details well and allows a lot of leaverage for the implementors to do their magic behind the scenes.

Overview   Package  Class  Tree  Deprecated  Index  Help 
 PREV PACKAGE   NEXT PACKAGE FRAMES    NO FRAMES