For incident handling, forensics or troubleshooting purposes, packet sniffing is often used to understand the information exchange between two hosts. HTTP traffic packets are often sniffed so that the full header and body can be revealed easily, especially on the server side. On the client side, most commonly used technique is to use a proxy tool like Webscarab to intercept the request. While making new exercise for my DEV422, I noticed some interesting behavior of how different programs handles the gzip compressed HTTP content.
Normally, if the server sends gzipped content to the client, it sends a special HTTP header indicating to the browser the content should be decompressed before rendering on screen, this is the header normally sent
Content-Encoding: gzip
When there is a sniffer or interception proxy in the middle, the condition is complicated, as these man-in-the-middle need to be able to decode to read the actual content.
Let examine how Webscarab handles gzipped content, here is an interception on response coming from server.
Webscarab shows a "X-Content-Encoding: gzip" indicating the gzip process had already been performed. The browser does not have to take any action as the content is already decompressed by Webscarab.
Wireshark, on the other hand is a simple sniffer, it doesn't alter anything the user is able to view. Let's see what Wireshark can see while sniffing packets on the wire.
Unlike Webscarab, Wireshark does not alter the headers and only shows what is actually transferred. At the bottom of the above screen cap, Wireshark is able to decode the gzipped data and show the HTTP data.
Everything is good so far, Webscarab and Wireshark essentially handled gzip data transparently by default.
Now, what if I follow TCP stream within Wireshark? This is a very common function used by a lot of users to view the full conversation between two hosts.
Wireshark Follow TCP Stream function does not decode gzip compression. The content is not decoded while displayed on screen.
While handling packet sniffer or using a man-in-the-middle manipulating HTTP responses, make sure you are aware that some compression encoding could be at play if things are not displaying properly on screen.
