Pixel carpet data visualization

Researching new tools to visualize large amounts of network traffic data.
Network security analysts are constantly looking for new tools to discover malicious activity in traffic data that automated systems can't detect. We helped co-create ideas that harness human visual pattern recognition to find anomalies in large data sets.

Finding needles of attacks in a haystack of network traffic

The goal of the EU wide research project SASER was to lay the foundation for more reliable, efficient, and secure communication networks (the  revelations about the NSA infiltration might have played a role in the decision to work with substantial effort inside the EU on advanced networks). We got the opportunity to take part in this large research activity on computer network security and data visualization.

We joined a team of researchers at the Interaction Design Lab at FH Potsdam to investigate data visualization, visual analysis tools, and dashboards as a support for computer security engineers. While watching traffic and server activity, they often have to sift through loads of data to filter out the suspicious traces of attacks and other malicious activity. Data visualization would certainly help them explore data, especially bringing patterns to the light that they would have not expected (and therefore wouldn’t have looked for with their data mining tools).


New tools for security experts through co-creation

We compiled the key aspects of our visualization into a paper and how we co-developed a demonstrator with users from the field, which we call Pixel Carpet (the demostrator is quite demanding due to the large data set). It got accepted to the IEEE VAST conference in Paris, one of the leading conferences in the area of data visualizaion! (yeah!)

It builds on the observation that security engineers know their data and the requirements of their work very well. However, they might not be acquainted with advanced visualization techniques. Visualization researchers, on the other hand, know methods to visualize and analyze data but usually lack insight into the specific requirements of computer network security. The paper revolves around two main contributions:

1. results and learnings from a co-creative approach of jointly developing visualizations

2. a pixel oriented visualization technique that graphically represents multi-dimensional data sets (such as computer log files), reflecting ideas from the collaboration

Landstorfer, Herrmann, Stange, Dörk, Wettach (2014): Weaving a Carpet from Log Entries: a Network Security Visualization Built with Co-Creation. in Visual Analytics Science and Technology (VAST), 2014 IEEE Conference on, 2014

More details, also on associated projects, on the project website complexdatavisualized.com.

Pixel Carpet in a 30s teaser video