Managing Multi-GBs in Wireshark

This summer I wrote a short paper on a process that I developed to support some analysis of 54GBs of pcap data captured using Wireshark.

Prior to the capture (lesson learned), it was never investigated whether or not Wireshark could support analysis of this quantity of data. The short answer is NO. Depending on your computer’s resources, when attempting to open any significantly sized pcap file (500MB) you will likely experience the dreadful out of memory error.

This obstacle created an opportunity. I have captured what I learned using Wireshark to process this large pcap files here. If you find this process remotely useful or if you have any questions, feel free to contact me.

"A Network-Usage Baseline, within this paper, is defined as a profile of the characteristics of a communication network within a particular window of usage. For example, these characteristics can include utilization, network applications, number of users, etc. These baselines can be used for forecasting and planning, as well as optimization and troubleshooting. This paper is used to document a methodology for creating network-usage baselines using Wireshark, its packaged toolsets (e.g. dumpcap.exe, tshark.exe), an advanced text editing tool and a common spreadsheet application. Notwithstanding Wireshark’s many capabilities, there are limitations when attempting to analyze extremely large capture files (GB+). The task of creating a baseline on modern, multi-user networks normally requires the manipulation and analysis of many very large, multi-GB data captures. This methodology was developed based on roughly 72 hours of captured data, at three distinct intersections in the network architecture. The resulting data equated to 52GBs of libpcap capture files. All of these files were captured using the tool dumpcap.exe, which accompanies the Wireshark installed. During the capturing, each packet was intentionally “snapped” at 50 bytes to minimize the size of the resulting capture files. To the author’s current knowledge, this “snapping” at 50 bytes does not impact the statistics being generated. The example charts at the end of this paper support this understanding..."

In the Beginning…

After many failed attempts, I am attempting to use this mode of communication once more. Fundamentally it is my belief that I am a terrible writer. This is due, in part, to many years of vehemently avoiding the art form. I have always despised writing. To overcome this obstacle, I decided earlier this year that it was time to write a book.

A book - why take on such a daunting task? If the task isn’t difficult then it probably isn’t very rewarding, I’ve never written a book and lastly, I have so much to share.

Now, after a couple of months working off-and-on on the book, the research is slowing me down and not granting me the necessary practice. So I will now use this blog to practice the art and hopefully, as a good citizen of the Internet, to share something useful.

I'm Sleepy

Things have been good, but this lack of sleep thing is killing.  Goal for the next few weeks, attempt to go to sleep earlier and get up at the same time.

Graduation Has Arrived

Well, I have finally graduated with my MBA.  It has taken about 5 1/2 years.  Not much to say except, about time!!!