Abstract
Constrained random simulation based verification (CRV) becomes an important means of verifying the functional correctness of the increasingly complex hardware designs. Effective coverage metric still lacks for assessing the adequacy of these processes. In contrast to other coverage metrics, the syntax-based Mutation Analysis (MA) defines a systematic correlation between the coverage results and the test’s ability to reveal design errors. However, it always suffers from extremely high computation cost. In this paper we present an efficient integration of mutation analysis into CRV flows, not only as a coverage gauge for simulation adequacy but also, a step further, to direct a dynamic adjustment of the test probability distribution. We consider the distinct cost model of this MA-based random simulation flow and try to optimize the coverage process. From the probabilistic analysis of the simulation cost, a heuristics for steering the test generation is derived. The automated flow is implemented by the SystemC Verification Library and by CertitudeTM for mutation analysis. Results from the experiment with an IEEE floating point arithmetic design show the efficiency of our approach.
Chapter PDF
Similar content being viewed by others
References
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Computer 11(4) (April 1978)
DeMillo, R.A., Offutt, A.J.: Constraint-Based Automatic Test Data Generation. IEEE Transactions on Software Engineering 17(9) (September 1991)
Howden, W.E.: Weak Mutation Testing and Completeness of Test Sets. IEEE Transactions on Software Engineering 8(4) (July 1982)
Offutt, A.J.: The coupling effect: fact or fiction. ACM SIGSOFT Software Engineering Notes 14(8) (December 1989)
Offutt, A.J., Seaman, E.J.: Using Symbolic Execution to Aid Automatic Test Data Generation. In: Proceedings of the fifth Annual Conference on Computer Assurance, COMPASS 1990, Gaithersburg, MD, USA (June 1990)
Untch, R.H., Offutt, A.J., Harrold, M.J.: Mutation Analysis Using Mutant Schemata. ACM SIGSOFT Software Engineering Notes 18(3) (July 1993)
Hampton, M., Petithomme, S.: Leveraging a Commercial Mutation Analysis Tool For Research. In: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques- MUTATION. TAICPART-MUTATION 2007 (September 2007)
Sengupta, S.: Defect-based Tests: A Key Enabler for Successful Migration to Structural Test. Intel Technology Journal (1999)
Al Hayek, G., Robach, C.: From Specification Validation to Hardware Testing: A Unified Method. In: Proceedings of the IEEE International Test Conference on Test and Design Validity, ITC 1996, Washington, DC, USA (October 1996)
Ghosh, S., Chakraborty, T.J.: On Behavior Fault Modeling for Digital Designs. Journal of Electronic Testing: Theory and Applications 2(2) (June 1991)
Ferrandi, F., Fummi, F., Sciuto, D.: Implicit Test Generation for Behavioral VHDL Models. In: Proceedings of the 1998 IEEE International Test Conference, ITC 1998, Washington, DC, USA (October 1998)
Fin, A., Fummi, F.: A VHDL Error Simulator for Functional Test Generation. In: Proceedings of the 2000 conference on Design, Automation and Test in Europe, DATE 2000, Paris, France (March 2000)
International Technology Roadmap for Semiconductors (ITRS). ITRS 2009 Edition, http://www.itrs.net/Links/2009ITRS/Home2009.htm
SpringSoft. Functional Qualification Tool CertitudeTM, http://www.springsoft.com/products/functional-qualification/certitude
Corno, F., Sonza Reorda, M., Squillero, G., Manzone, A., Pincetti, A.: Automatic Test Bench Generation for Validation of RT-Level Descriptions: An Industrial Experience. In: Proceedings of the 2000 Conference on Design, Automation and Test in Europe, DATE 2000, Paris, France (March 2000)
Tasiran, S., Fallah, F., Chinnery, D., Weber, S., Keutzer, K.: Coverage-Directed Generation of Biased Random Inputs for Functional Validation of Sequential Circuits. In: Proceedings of the International Workshop on Logic and Synthesis (June 2001)
Fine, S., Ziv, A.: Coverage directed test generation for functional verification using bayesian networks. In: Proceedings of the 40th annual Design Automation Conference, DAC 2003, Anaheim, CA, USA (2003)
Open SystemC Initiative Verification Working Group, SystemC Verification Library Standard, release 1.0p2 (2006), http://www.systemc.org/downloads/standards
Bombieri, N., Fummi, F., Pravadelli, G., Hampton, M., Letombe, F.: Functional Qualification of TLM Verification. In: Proc. of the 2009 ACM/IEEE Design, Automation and Test in Europe, DATE 2009, Nice, France (April 2009)
Lachish, O., Marcus, E., Ur, S., Ziv, A.: Hole Analysis for Functional Coverage Data. In: Proceedings of the 39th Conference on Design Automation, pp. 807–812 (June 2002)
Asaf, S., Marcus, E., Ziv, A.: Defining coverage views to improve functional coverage analysis. In: Proceedings of the 41st Conference on Design Automation, pp. 41–44 (June 2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 IFIP
About this paper
Cite this paper
Xie, T., Mueller, W., Letombe, F. (2010). Efficient Mutation-Analysis Coverage for Constrained Random Verification. In: Hinchey, M., et al. Distributed, Parallel and Biologically Inspired Systems. DIPES BICC 2010 2010. IFIP Advances in Information and Communication Technology, vol 329. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15234-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-15234-4_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15233-7
Online ISBN: 978-3-642-15234-4
eBook Packages: Computer ScienceComputer Science (R0)