Search
photo of Bev Littlewood

Professor Bev Littlewood

Emeritus Professor

School of Science & Technology Department of Computer Science

Contact details

Address

Professor Bev Littlewood A304F, College Building [A]
City, University of London
Northampton Square
London EC1V 0HB
United Kingdom

About

Overview

Bev Littlewood has degrees in mathematics and statistics, and a PhD in statistics and computer science; he is a Chartered Engineer, and a Chartered Statistician. He has worked for more than 30 years on problems associated with the dependability of software-based systems, and has published many papers in international journals and conference proceedings and has edited several books.

In 1983 he founded the Centre for Software Reliability (CSR) at City University, London, and was its Director from then until his semi-retirement in 2003. During this period CSR attracted many millions of pounds of research funding from various European and UK national agencies such as EPSRC, and industry, and gained an international reputation for the quality of its research. He has successfully managed almost 40 research projects as Principal Investigator. He is currently Professor of Software Engineering at City University.

His technical contributions have largely focused on the application of probabilistic and statistical techniques in software systems engineering. He has published over a hundred scientific papers on topics that include:

- Software reliability growth modeling
- Modelling of software design diversity for fault tolerance
- Exploring the limits of what can be claimed rigorously for system dependability
- Application of Bayesian probability and statistics to systems dependability
- Software testing
- Combining reliability and correctness claims in safety arguments

He has a long term interest in the question of how far these approaches can take us in providing assurance that systems are fit for purpose - in particular, whether certain critical systems are sufficiently safe to be allowed to operate. He has encouraged - and taken part in - debate about such issues in both technical and non-technical forums. This has involved several TV and radio appearances in documentary and news programmes discussing general issues of software reliability and safety. For example, he presented, and was co-author, of an edition of the BBC2 TV programme, 'Antenna', on safety-critical software.

From 1990 to 2005 he was a member of the UK Nuclear Safety Advisory Committee (NuSAC), in which role he played a part in the extensive discussions, and controversy, concerning the first use of a software-based protection system for a UK power reactor. He chaired the NuSAC Working Group on the use of software in critical applications, which reported to the Health and Safety Commission in 1998. Its report was published as 'The Use of Computers in Safety-critical Applications' (www.hse.gov.uk/nuclear/computers.pdf).

His professional activities related to his scientific work are numerous. He is a member of the UK Computing Research Committee (UKCRC), and is a past member of the EPSRC College for Computing. He is currently serving his third term as an Associate Editor of IEEE Trans on Software Engineering, the premier journal in SE. He is on the editorial boards of several other international journals. He is a member of IFIP Working Group 10.4 on Reliable Computing and Fault Tolerance. He was a member of the British Computer Society's Safety Critical Systems Task Force until its recent untimely demise. He was the only non-US member of the Panel on Statistical Methods in Software Engineering (1992-4), convened by the US National Research Council. He was a member of the HSC Working Group that wrote the influential 'Tolerability of Risk from Nuclear Power Stations', being responsible for the section on the contribution of critical computing systems to risk. From 2007 until 2009 he was a member of the IEEE John von Neumann Award Committee. He is currently a member of the IEEE Computer Society's Harlan D Mills Award Committee.

He has been a consultant to many organizations and companies in Europe and the US. As consultant to Harlan Mills' IBM Fellow Department in the 1980s, BL gave advice on the statistical underpinnings of the well-known Cleanroom approach to software development and evaluation.

BL has been a member of the programme committees of many international conferences, including several ICSEs. He has given keynote addresses and invited talks at numerous conferences.

In 2007 he was the recipient of the IEEE Computer Society's prestigious Harlan D Mills Award, the citation of which reads: "For leading research on the application of rigorous probabilistic and statistical techniques in software engineering, particularly in systems dependability".

Qualifications

  • CITP (Chartered IT Professional) IT, British Computer Society, Jan 2008
  • CE (Chartered Engineer) Computing, British Computer Society, 1994
  • CS (Chartered Statistician) Statistics, Royal Statistical Society, 1993
  • PhD Statistics & Computer Science, City, University of London, United Kingdom, Jan 1978
  • MSc Statistics, University of London, United Kingdom, 1966
  • ARCS (Associate of the Royal College of Science) Mathematics, Imperial College London, 1963
  • BSc Mathematics, Imperial College London, United Kingdom, Jan 1963

Employment

  • Emeritus Professor of Software Engineering, City, University of London, 2018 – present
  • Professor of Software Engineering (50% research-only appointment), City, University of London, 2011 – 2018
  • Visiting Associate Professor, George Washington University, 1980 – 1981
  • Professor of Software Engineering since 1985, City, University of London, Jan 1966 – Jan 2011

Memberships of professional organisations

  • Member, UK Computing Research Committee, 2003 – present
  • Member, Conference of Professors & Heads of Computing, 2000 – present
  • Member, British Computer Society, 1985 – present
  • Member, IEEE Computer Society, 1984 – present
  • Fellow, Royal Statistical Society, 1966 – present

Research

Research interests

- Reliability and safety modeling of software-based systems
- Software fault tolerance
- Software testing
- Dependability cases for software-based systems

Publications

Publications by category

Books (2)

  • Commission, G.B.H.A.S., Staff, H.A.S.E. and Littlewood, B. (1998). Use of Computers in Safety-Critical Applications, Final Report to Nuclear Safety Advisory Committee (NuSAC) of the Study Group on the Safety of Operational Computer Systems. ISBN 978-0-7176-1620-6.
  • Fenton, N.E. and Littlewood, B. (1991). Software reliability and metrics. Garmisch-Partenkirchen: Kluwer Academic Pub. ISBN 978-1-85166-675-1.

Chapters (11)

  • Littlewood, B. and Strigini, L. (2000). Software reliability and dependability: a roadmap. In Finkelstein, A. (Ed.), 22nd International Conference on on Software Engineering, Future of Software Engineering Track, ICSE 2000, Limerick Ireland, June 4-11, 2000. (pp. 175–188). ACM. ISBN 1-58113-253-0.
  • Littlewood, B. and Wright, D. (1997). Mathematics of dependable systems 2. In Stavridou, V. and Applications, I.F.M.A.I. (Eds.), Mathematics of dependable systems 2 (pp. 243–258). Oxford University Press, USA.
  • Littlewood, B. (1996). Evaluation of software dependability. In Wand, I.C. and Milner, R. (Eds.), Computing Tomorrow: Future Research Directions in Computer Science Cambridge Univ Pr. ISBN 978-0-521-46085-9.
  • Littlewood, B. and Brocklehurst, S. (1996). Techniques for prediction analysis and recalibration. In Lyu, M.R. (Ed.), Handbook of software reliability engineering (pp. 119–166). New York: McGraw-Hill. ISBN 978-0-07-039400-1.
  • Strigini, L. and Littlewood, B. (1995). Validation of Ultra-High Dependability for Software-Based Systems. In Randell, B., Laprie, J.C., Kopetz, H. and Littlewood, B. (Eds.), Predictably Dependable Computing Systems (ESPRIT Basic Research Series) (pp. 473–493). Springer. ISBN 978-3-540-59334-8.
  • Littlewood, B. (1994). Software Reliability Modelling. In McDermid, J. (Ed.), Software engineer's reference book Butterworth-Heinemann Ltd. ISBN 978-0-7506-0813-8.
  • Littlewood, B. (1991). Limits to evaluation of software dependability. In Fenton, N.E. and Littlewood, B. (Eds.), Software reliability and metrics (pp. 81–110). Elsevier. ISBN 978-1-85166-675-1.
  • Littlewood, B. (1991). Reliability. In Fenton, N.E. (Ed.), Software Metrics: A Rigorous Approach (pp. 230–249). London: Chapman and Hall. ISBN 978-0-412-40440-5.
  • Littlewood, B. (1990). Modelling growth in software reliability. In Rook, P. (Ed.), Software Reliability Handbook Elsevier. ISBN 978-1-85166-400-9.
  • Littlewood, B., Chan, P.Y. and Abdel Ghaly, A.A. (1986). Tools for the analysis of the accuracy of software reliability predictions. In Skwirzynski, J.K. (Ed.), NATO advanced Study Institute, Software System Design Methods (pp. 229–336). Springer-Verlag.
  • Littlewood, B. (1985). Software Reliability Prediction. In Anderson, T. (Ed.), Resilient computing systems (pp. 144–162). Wiley-Interscience.

Conference papers and proceedings (73)

  • Zhao, X., Littlewood, B., Povyakalo, A. and Wright, D. (2015). Conservative claims about the probability of perfection of software-based systems. 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE) 2-5 November. doi:10.1109/issre.2015.7381807
  • Rushby, J., Littlewood, B. and Strigini, L. (2014). Evaluating the Assessment of Software Fault-Freeness. AESSCS 2014 Workshop, "Planning the Unplanned Experiment: Assessing the Efficacy of Standards for Safety Critical Software At the European Dependable Computing Conference (EDCC) 13 May, Newcastle upon Tyne.
  • Littlewood, B. and Burns, A. (2010). Reasoning About the Reliability of Multi-version, Diverse Real-Time Systems. Real Time Systems Symposium 30 Nov 2010 – 3 Dec 2010, San Diego, CA.
  • Littlewood, B. and Burns, A. (2010). Reasoning About the Reliability of Multi-version, Diverse Real-Time Systems. 31st IEEE Real-Time Systems Symposium 30 Nov 2010 – 3 Dec 2010, San Diego. doi:10.1109/RTSS.2010.43
  • Littlewood, B., Popov, P.T., Strigini, L. and Shryane, N. (2008). Modelling the Effects of Combining Diverse Software Fault Detection Techniques.
  • Bloomfield, R.E., Littlewood, B. and Wright, D. (2007). Confidence: Its role in dependability cases for risk assessment. doi:10.1109/DSN.2007.29
  • Bishop, P., Gashi, I., Littlewood, B. and Wright, D. (2007). Reliability modeling of a 1-out-of-2 system: Research with diverse Off-the-shelf SQL database servers.
  • Bryans, J.W., Littlewood, B., Ryan, P.Y.A. and Strigini, L. (2006). E-voting: Dependability requirements and design for dependability. 1st International Conference on Availability, Reliability and Security 20-22 April, Vienna Univ Technol, Vienna, AUSTRIA.
  • Littlewood, B. (2005). Dependability assessment of software-based systems: state of the art.
  • Popov, P.T. and Littlewood, B. (2004). The Effect of Testing on Reliability of Fault-Tolerant Software. doi:10.1109/DSN.2004.1311896
  • Littlewood, B. and Strigini, L. (2004). Redundancy and Diversity in Security.
  • Littlewood, B., Bloomfield, R., Popov, P., Povyakalo, A. and Strigini, L. (2004). The impact of ‘difficulty’ variation on the probability of coincident failure of diverse systems. International Conference on Control and Instrumentation in Nuclear Installations Liverpool.
  • Littlewood, B. and Bloomfield, R. (2004). On the use of diverse arguments to increase confidence in dependability claims. International Conference on Control and Instrumentation in Nuclear Installations.
  • Littlewood, B. (2003). Editor SAFECOMP2003. International Conference on Computer Safety, Reliability and Security September, Edinburgh.
  • Bloomfield, R. and Littlewood, B. (2003). Multi-legged arguments: the impact of diversity upon confidence in dependability arguments. doi:10.1109/DSN.2003.1209913
  • (2003). Computer Safety, Reliability, and Security, 22nd International Conference, SAFECOMP 2003, Edinburgh, UK, September 23-26, 2003, Proceedings.
  • Strigini, L., Bosio, D., Littlewood, B. and Newby, M.J. (2002). Advantages of open source processes for reliability: clarifying the issues. 25-26 February, Newcastle upon Tyne.
  • Littlewood, B. (2002). Complexity is the enemy of dependability - Can diversity provide a defence? doi:10.1109/ICECCS.2002.1181492
  • Littlewood, B. (2002). On Diversity, and the Elusiveness of Independence.
  • Oussalah, M., Littlewood, B., Strigini, L. and Newby, M. (2001). Combination of qualitative and quantitative sources of knowledge for risk assessment.
  • Littlewood, B., Popov, P. and Strigini, L. (2001). Design diversity: an update from research on reliability modelling.
  • Littlewood, B., Popov, P. and Strigini, L. (2001). Modeling software design diversity - A review. doi:10.1145/384192.384195
  • Littlewood, B. and Strigini, L. (2000). Software reliability and dependability: a roadmap. 22nd International Conference on Software Engineering 4-11 June, Limerick, Ireland.
  • Littlewood, B. and Strigini, L. (2000). Fault tolerance via diversity against design faults (tutorial session): design principles and reliability assessment.
  • Littlewood, B. and Strigini, L. (2000). Software reliability (tutorial session): basic concepts and assessment methods.
  • Littlewood, B. (2000). The Problems of Assessing Software Reliability .... when you really need to depend on it.
  • Popov, P.T., Strigini, L. and Littlewood, B. (2000). Choosing Between Fault-Tolerance and Increased V&V for Improving Reliability.
  • Littlewood, B., Popov, P.T. and Strigini, L. (2000). Assessment of the Reliability of Fault-Tolerant Software: A Bayesian Approach.
  • Littlewood, B., Strigini, L., Wright, D., Fenton, N., Neil, M. and Courtois, P.-.J. (2000). Bayesian Belief Networks for safety assessment of computer-based systems.
  • Littlewood, B., Popov, P. and Strigini, L. (2000). N-version design versus one good version. Proc International Conference on Dependable Systems and Networks (Fast Abstracts) New York.
  • Littlewood, B. (1999). Two Heads Are Better Than One: Using Diversity to Make Software Dependable.
  • Frankl, P., Hamlet, D., Littlewood, B. and Strigini, L. (1997). Choosing a testing method to deliver reliability. 19th International Conference on Software Engineering (ICSE 97) - Pulling Together 17-23 May, BOSTON, MA.
  • Barghout, M., Littlewood, B. and AbdelGhaly, A. (1997). A non-parametric approach to software reliability prediction. doi:10.1109/ISSRE.1997.630885
  • Littlewood, B., Combelles, A. and Sorel, V. (1996). Difficulties in evaluating the dependability of safety-critical systems: investigation on the combination of evidence. International Conference on Probabilistic Safety Assessment and Management, (ESREL ’96 - PSAM III) 24-26 June, Crete.
  • Littlewood, B., Neil, M. and Fenton, N. (1996). Applying Bayesian belief networks in systems dependability assessment. Safety Critical Systems Symposium February, Leeds.
  • Littlewood, B., Wright, D. and Meyer, F. (1995). Dependability of modular software in a multiuser operational environment. International Symposium on Software Reliability Engineering (ISSRE ’95) November, Toulouse.
  • LITTLEWOOD, B. and WRIGHT, D. (1995). Stopping rules for the operational testing of safety-critical software. doi:10.1109/FTCS.1995.466955
  • Littlewood, B. and Wright, D. (1995). A Bayesian model that combines disparate evidence for the quantitative assessment of system dependability. 14th International Conference on Computer Safety (SafeComp’95).
  • Littlewood, B. (1994). Learning to live with uncertainty in our software. 2nd Intl Software Metrics Symposium 24-26 October, London.
  • LITTLEWOOD, B. (1994). WHY WE SHOULD LEARN NOT TO DEPEND TOO MUCH UPON SOFTWARE.
  • BROCKLEHURST, S., OLOVSSON, T., LITTLEWOOD, B. and JONSSON, E. (1994). ON MEASUREMENT OF OPERATIONAL SECURITY.
  • Littlewood, B. (1994). Measurement of operational security. IFIP Conference on Distributed Computing for Critical Applications (DCCA-4) January, San Diego.
  • Littlewood, B. (1993). The need for evidence from disparate sources to evaluate software safety. Directions in Safety-Critical Computing 9-11 February, Bristol.
  • Littlewood, B. (1992). Measurement-based modelling issues - the problem of ultra-high dependability. Experimental Software Engineering Issues: Critical Assessment and Future Directions International Workshop September, Dagstuhl Castle, Germany.
  • Littlewood, B., Brocklehurst, S., Kanoun, K., Laprie, J.C., Metge, S., Mellor, P. … Tanner, A. (1991). Reliability analyses of workstation failure data. ESPRIT ‘91 25-29 November, Brussels.
  • Littlewood, B. (1991). Software reliability modelling: achievements and limitations. IEEE CompEuro 91 13-16 May, Bologna.
  • Littlewood, B. and Fenton, N. (1991). Evaluating software engineering standards and methods. Eurometrics 91 March, Issy-les-Molinieaux.
  • Littlewood, B. and Thomas, M. (1991). Reasons why safety-critical avionics software cannot be adequately validated. Safety and Reliability Society Spring Conference,.
  • Littlewood, B. (1990). Software reliability modelling: some achievements and limitations. ICL/Newcastle University International Seminar on the Teaching of Computer Science September, University of Newcastle-upon-Tyne.
  • Littlewood, B. (1989). Hardware and software dependability evaluation: software dependability. 11th Information Processing World Congress (IFIP) 28 Aug 1989 – 1 Sep 1989, San Francisco.
  • Littlewood, B., Brocklehurst, S., Chan, P.Y. and Snell, J. (1988). A general adaptive approach to software reliability prediction. UK IT '88.
  • Littlewood, B. and Miller, D.R. (1988). A conceptual model for software diversity. UK IT '88.
  • Littlewood, B. and Miller, D.R. (1987). A conceptual model of the effect of diverse methodologies on coincident failures in multi-version software. 3rd International Fault-Tolerant Computing Systems (IFTCS) September, Bremerhaven.
  • Littlewood, B. and Miller, D.R. (1987). Modelling multi-version software. 4th CSR Conference: Measurement for Software Control and Assurance September, Bristol.
  • Littlewood, B., Brocklehurst, S., Chan, P.Y. and Snell, J. (1987). Adaptive software reliability modelling. 4th CSR Conference: Measurement for Software Control and Assurance September, Bristol.
  • Littlewood, B. (1987). Proceedings of 2nd CSR Workshop, Software Reliability: Achievement and Assessment. 2nd CSR Workshop, Software Reliability: Achievement and Assessment September.
  • Littlewood, B. (1987). Proceedings of 4th CSR Conference: Measurement for Software Control and Assurance. 4th CSR Conference: Measurement for Software Control and Assurance September, Bristol.
  • Littlewood, B. (1987). How good are software reliability predictions? 2nd CSR Workshop, Software Reliability: Achievement and Assessment.
  • Littlewood, B. and Keiller, P.A. (1984). Adaptive software reliability modelling. IEEE 14th Annual International Symposium Fault-Tolerant Computing (FTCS 1984) Orlando.
  • Littlewood, B., Keiller, P.A. and Sofer, A. (1984). Predictive quality of software reliability models. 4th Jerusalem Conference on Information Technology (JCIT).
  • Littlewood, B., Keiller, P.A., Miller, D.R. and Sofer, A. (1983). On the quality of software reliability prediction. 1982 NATO Advanced Study Institute on Electronic Systems Effectiveness and Life Cycle Costing.
  • Littlewood, B., Keiller, P.A., Miller, D.R. and Sofer, A. (1983). Comparison of software reliability predictions. IEEE 13th Annual International Symposium Fault-Tolerant Computing (1983 FTCS) Milan.
  • Littlewood, B. (1981). A critique of the Jelinski-Moranda model for software reliability. Annual Reliability and Maintainability Symposium 27-29 June, Philadephia.
  • Littlewood, B. (1980). A Bayesian differential debugging model for software reliability. COMPSAC 80 27-31 October, Chicago, IL.
  • Littlewood, B. (1980). What makes a reliable program - few bugs, or a small failure rate. 1980 National Computer Conference (NCC 80) 19-22 May, Arlington, VA.
  • Littlewood, B. (1979). A Bayesian differential debugging model for software reliability. Workshop on Quantitative Software Models 9-11 October, Kiamesha Lake, New York.
  • Littlewood, B. (1978). Validation of a software reliability model. 2nd Software Life Cycle Management Workshop 20-22 August, Atlanta, GA.
  • Littlewood, B. (1978). How to measure software reliability, and how not to ... 3rd International Conference on Software Engineering 10-12 May, Atlanta, GA.
  • Littlewood, B. (1978). Computer software reliability measurement. Thirteenth Annual Conference in Statistics, Computer Science, and Operations Research, ISSR March, Cairo University, Egypt.
  • Littlewood, B. (1977). Software reliability measurement: some criticisms and suggestions. Conference on Software Life Cycle Management 22-23 August, Airlie, VA.
  • Littlewood, B. (1976). A semi-Markov model for software reliability with failure costs. MRI Symposium on Computer Software Engineering 20-22 April, New York.
  • Littlewood, B. (1975). A reliability model for Markov structured software. International Conference on Reliable Software 21-23 April, Los Angeles, California.
  • Littlewood, B. and Verrall, J.L. (1973). A Bayesian reliability growth model for computer software. 1973 IEEE Symposium on Computer Software Reliability 30 Apr 1973 – 2 May 1973, New York.

Exhibition

  • Littlewood, B. and Haines, T. (1990). Computer Safety. 24 October, BBC Television, Channel 2 (Edition of Antenna).

Journal articles (67)

Reports (7)

Other

  • Littlewood, B. and CSR, (1970). Software Reliability and Metrics Program.