1. Events
  2. 2016
  3. April
  4. On the Metrics for Benchmarking Vulnerability Detection Tool




On the Metrics for Benchmarking Vulnerability Detection Tool



Staff, Students

Speaker: Marco Vieira

Abstract: Research and practice show that the effectiveness of vulnerability detection tools depends on the concrete use scenario. Benchmarking can be used for selecting the most appropriate tool, helping assessing and comparing alternative solutions, but its effectiveness largely depends on the adequacy of the metrics. In this seminar we will discuss the problem of selecting the metrics to be used in a benchmark for software vulnerability detection tools. Based on the characteristics of a good metric for the vulnerability detection domain, a set of metrics is analyzed in the context of specific scenarios to select the most adequate one and a MCDA algorithm together with experts’ judgment is applied to validate the conclusions. Results show that although some of the metrics traditionally used like precision and recall are adequate in some scenarios, others require alternative metrics that are seldom used in the benchmarking area.

Short Bio: Marco Vieira is an Associate Professor at the University of Coimbra (UC) and is the Director of the Center for Informatics and Systems of the University of Coimbra (CISUC). Marco’s research focuses mainly in the area of dependable and secure systems, namely in benchmarking and experimental assessment of security and dependability attributes. His research interests also include failure prediction, evaluation and improvement of software robustness, fault injection, database systems and software development processes. Marco Vieira has published more than 150 papers in international conferences and journals with peer reviewing process, and has served on the program committee of the major conferences on the dependability area. Currently, he is the coordinator of the DEVASSES (FP7 IRSES) and EUBrasilCloudFORUM (H2020 CSA) projects and the Principal Investigator at the University of Coimbra for the CECRIS (FP7 IAPP) and EUBra-BIGSEA (H2020 RIA) projects.

Share this event

When & where

4.00pm - 5.00pmThursday 28th April 2016

AG01 College Building City, University of London St John Street London EC1V 4PB United Kingdom

Contact Details

Dr Vladimir Stankovic


Centre for Software Reliability

School of Mathematics, Computer Science & Engineering


College Building
City, University of London
Northampton Square
United Kingdom
+44 (0)20 7040 3079

Contact the organiser