Monday, 14 October 2013

The Economics of False Positives

DeepSEC 2013 – The Economics of False Positives - By Gavin 'jac0byterebel' Ewan

It seems fitting that the last blog post I wrote for my own site was also a prelude to DeepSEC, albeit 12 months ago! I am equal parts excited and terrified at the prospect of delivering my new talk 'The Economics of False Positives' at DeepSEC in Vienna this year. Not only am I giving this talk for the first time at DeepSEC, but it also represents a marked departure from my usual bread and butter, social engineering.

The talk draws inspiration from two points of my life, the distant past and the present day. In the distant past while studying at university most of you that know me are well aware of my study of psychology, but few are aware of the fact my degree also comprised a large part of economics. While the psychology side served well for serving up social engineering talks and research (indeed, very much ANTI-psychology), I have had no need to call upon my economics background while working within infosec. Until now.

My good friend, founder of Alba 13 Research Labs and renowned speaker, Arron 'Finux' Finnon, founded not only Alba 13, but also the Open Source Network Intrusion Framework (OSNIF) project. I could write an entire blog post on the merits of OSNIF, and indeed am an ardent supporter of the project, but as a type-cast social engineer, really had nothing much to offer other than moral support. As previously stated, until now.

The OSNIF project, amongst other things, has composed a top 5 of areas that the powers that be in the field of NIDS/NIPS agree need to be addressed in any effective IDS/IPS test. None of the areas have weight over another (so no jockeying for position ala the OWASP Top 10), but one has drawn my attention right from the start, that is the area of false positives. Flogged to death from a technical perspective, false positives are an area of concern in many technical fields, both in and out of computer security. While a quality checker of loafs accidentally throwing out a bad loaf that turned out to be good is not really a big deal, telling someone they have cancer and then having to retract the statement cannot be covered with an 'Oops, sorry'. Sadly, the attitude to false positives within computer security seems to bear closer to the former than the latter in terms of PERCEIVED impact.

In actual fact, false positives have a tangible business cost. My talk will address this at both a micro and macro level. At the micro level I shall explore how a business can drill costs down to the level of the individual false positive. At the macro level I will demonstrate that taking the figures from above we can provide a model to determine the 'optimum level' of false positives, carefully balancing the need to reduce false positives with the very real effect of having no true positives either! Thus the talk will also move into another of the OSNIF top 5, Detection Rates, and why the two go hand in hand.

I look forward to seeing you all there!

Gavin 'jac0byerebel' Ewan

You can find out more about Gavin's talk here:

If you want to learn more about testing NIDS/NIPS effectiveness then come join us at DeepSEC in Vienna this year, where Arron Finnon and Gavin Ewan of Alba13 Research Labs will be showing what goes into making an effective and useful NIDS/NIPS assessment. Also we'll be showing how enterprise threats can be managed and how you can build better detection. Find out more details here