10Gbps+ stream to disk capture appliance

hard to implement with commodity hardware. Take a look at this paper:

http://www.net.t-labs.tu-berlin.de/papers/SWF-PCCH10GEE-07.pdf

Challenges are: disk speed and bus speed. The paper is from 2007 and some things have changed. PCI Express now offers sufficient bandwidth (via several lanes), but if you need to capture a fully utilized 10 gig link (full duplex), you still have to write approx. 2500 +/- MByte/s to disk at a sustained bit rate. That's not an easy task and probably requires special hardware. With a "software" sniffer/analyzer you have to copy data twice. Once from the network adapter to the RAM/CPU and then back to the disk subsystem.

Of course, if you don't need the full payload, or you don't want to capture all conversations, you can reduce the required bandwidth by any rate. The trick at 10 gig is good pre-filtering during the capturing stage. Otherwise you get in trouble with I/O bandwith and much more with loads of data to walk through during the analyzing stage.

There are commercial solutions available, that claim to be able to capture at full 10 Gig speed (and even 40 Gig). I have no idea how they do it. I never checked and I guess there is a lot of marketing involved ;-). They will face the same problems, bus speed, I/O subsystem speed, unless they use some special "hardware magic", like copying data directly from an intelligent network adapter to the disk subsystem. Some of them do what was suggested in the paper: split the 10 gig channel into several 1 Gig channels and capture those with several capturing units. Search google for: 10 gig capture.

Whatever you are going to build, you don't want to capture with Wireshark or tshark itself, as both are analyzing tools and not barebone capture tools. They build internal state in memory and that will be ways to much overhead for a live 10 gig capture. Use either dumpcap or tcpdump (or any other high speed capture tool like gulp, netsniff-ng or ringmap) to do the raw capture job and then you 'might' be able to use wireshark to analyze that huge pile of data.

The following blog post might be interesting for you:

http://www.lovemytool.com/blog/2012/02/when-the-shark-bites-by-mike-canney.html#more

Regards
Kurt