<%BANNER%>

Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-08-31.

Permanent Link: http://ufdc.ufl.edu/UFE0041877/00001

Material Information

Title: Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-08-31.
Physical Description: Book
Language: english
Creator: Chen, Mingsong
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Statement of Responsibility: by Mingsong Chen.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Mishra, Prabhat.
Electronic Access: INACCESSIBLE UNTIL 2012-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041877:00001

Permanent Link: http://ufdc.ufl.edu/UFE0041877/00001

Material Information

Title: Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-08-31.
Physical Description: Book
Language: english
Creator: Chen, Mingsong
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Statement of Responsibility: by Mingsong Chen.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Mishra, Prabhat.
Electronic Access: INACCESSIBLE UNTIL 2012-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041877:00001


This item has the following downloads:


Full Text

PAGE 1

EFFICIENTAPPROACHESFORFUNCTIONALVALIDATIONOFSOCDESI GNS USINGHIGH-LEVELSPECIFICATIONS By MINGSONGCHEN ADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF DOCTOROFPHILOSOPHY UNIVERSITYOFFLORIDA 2010 1

PAGE 2

c r 2010MingsongChen 2

PAGE 3

Tomyparentsfortheirloveandencouragement 3

PAGE 4

ACKNOWLEDGMENTS Althoughfouryearspassedinatwinkling,allthevividsnap shotsaredeeplyengraved inmymemory.IalwaysthinkthatIwasluckytobeaGatorinUF, notonlybecauseI witnessedfourNationalChampionships,butalsoIachieved anothermilestoneinmylife here.IneedtoconfessthatthejourneytogetaPh.D.ischall enging.Itisimpossibleto imaginecompletingitwithoutthepreciousadviceandhelpf romotherpeople. Firstofall,IreallyappreciatewhatmysupervisorDr.Prab hatMishradidforme. Hisexpertiseandinsightshelpedmetoquicklycapturether esearchdirectionandmade thisdissertationcometrue.ThroughoutmyPh.D.study,heg avemeenduringsupport, guidanceandencouragementwhichhelpedmetoovercomevari ousproblems.Thereisno doubtthathisattitudeonresearchhasdeeplyaectedmeand willbehelpfulinmyfuture career.FinallyIunderstoodwhyhewasalwaysurgingmetoma keprogress.Hiseorts mademyCVlooksstrongerwhichisbenecialtome. IwouldalsoliketothankmyPh.D.committeemembers:Prof.S artajSahni,Porf. Jih-KwonPeir,Prof.TaoLiandProf.RaymondIssa.Theirval uablesuggestionsat dierentstagesofmyresearchwereconstructiveandthough t-provoking.Theircriticisms enhancedthequalityofmyresearch.Colleaguesandfriends areanimportantpartin mygraduatelife.Iamverygratefulforthefriendshipofall themembersinmyresearch group-KanadBasu,HadiHajimiri,Heon-MoKoo,ChetanMurth y,KartikShrivastava, XiaokeQinandWeixunWang.Ireallyenjoyedtheharmoniousa tmosphereofourlaband theexperienceofcollaboratingwiththem. Lastbutnotleast,Isincerelythankmyparents,whouncondi tionallygavemethe loveandencouragement.Withouttheirsupport,Iwon'treac hthisfar.Idedicatethis dissertationtothem. ThisworkwaspartiallysupportedbygrantsfromIntelCorpo rationandNSF CAREERaward0746261. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................. 4 LISTOFTABLES ..................................... 9 LISTOFFIGURES .................................... 11 ABSTRACT ........................................ 13 CHAPTER 1INTRODUCTION .................................. 14 1.1SoCDesignFlow ................................ 15 1.2FunctionalValidationofSoCDesigns .................... 16 1.2.1OverviewofFunctionalValidationMethods ............. 17 1.2.2PotentialImprovementOpportunities ................. 18 1.2.3Challenges ................................ 20 1.3DissertationContributions ........................... 21 2FORMALMODELINGOFSOCSPECIFICATIONS ............... 23 2.1SpecicationusingSystemCTLMs ...................... 24 2.1.1FormalModelingofSystemCTLMs ................. 25 2.1.2TransformationfromSystemCTLMtoSMV ............. 27 2.1.2.1StructureExtraction ..................... 28 2.1.2.2BehaviorExtraction ..................... 30 2.1.3APrototypeToolForTLMtoSMVTranslation ........... 32 2.2SpecicationusingUMLActivityDiagrams ................. 32 2.2.1Notations ................................ 33 2.2.2FormalModelingofUMLActivityDiagrams ............. 36 2.2.3TransformationfromUMLActivityDiagramstoSMV ........ 40 2.2.3.1StaticInformationExtraction ................ 40 2.2.3.2DynamicInformationExtraction .............. 42 2.2.4APrototypeToolForUMLtoSMVTranslation ........... 44 2.3CaseStudy ................................... 45 2.3.1Example1:ARouter .......................... 45 2.3.2Example2:AMIPSProcessor ..................... 46 2.3.3Example3:AnAlphaProcessor .................... 47 2.3.4Example4:AControlSystem ..................... 48 2.3.5Example5:AStockExchangeSystem ................ 48 2.4Summary .................................... 49 5

PAGE 6

3COVERAGE-DRIVENAUTOMATICGENERATIONOFDIRECTEDTESTS 50 3.1Coverage-DrivenPropertyGeneration ..................... 51 3.1.1FaultModels .............................. 52 3.1.1.1GenericFaultModelsforGraphBasedModels ....... 52 3.1.1.2FaultModelsforSystemCTLMSpecications ...... 53 3.1.1.3FaultModelsforUMLActivityDiagrams ......... 54 3.1.2FunctionalCoverageBasedonFaultModels ............. 55 3.2TestGenerationusingModelCheckingTechniques .............. 56 3.2.1TestGenerationusingUnboundedModelChecking ......... 56 3.2.1.1UnboundedModelChecking ................. 56 3.2.1.2TestGenerationAlgorithm ................. 57 3.2.2TestGenerationusingBoundedModelChecking .......... 57 3.2.2.1SAT-BasedBoundedModelChecking ........... 57 3.2.2.2TestGenerationAlgorithm .................. 58 3.2.2.3DeterminationofBound ................... 59 3.3CaseStudies .................................. 60 3.3.1AControlSystem ............................ 61 3.3.2AStockExchangeSystem(OSES) ................... 62 3.4Summary .................................... 63 4PROPERTYCLUSTERINGFOREFFICIENTTESTGENERATION ..... 64 4.1RelatedWork .................................. 65 4.2Background:SATSolverImplementation ................... 66 4.2.1DPLLAlgorithm ............................ 67 4.2.2ConrictClauseBasedLearning .................... 67 4.3PropertyClustering ............................... 70 4.3.1SimilaritybasedonStructuralOverlap ................ 72 4.3.2SimilaritybasedonTextualOverlap .................. 73 4.3.3SimilaritybasedonInruence ...................... 74 4.3.4SimilaritybasedonCNFIntersection ................. 76 4.3.5DeterminationofBaseProperty .................... 76 4.4EcientTestGenerationusingLearningTechniques ............. 77 4.4.1ConrictClauseForwardingTechniques ................ 77 4.4.2NameSubstitutionforComputationofIntersections ......... 80 4.4.3IdenticationandReuseofCommonConrictClauses ........ 81 4.5CaseStudies ................................... 83 4.5.1AVLIWMIPSProcessor ........................ 84 4.5.1.1Structure-basedClustering .................. 84 4.5.1.2ClusteringbasedonTextualSimilarity ........... 87 4.5.1.3Inruence-basedClustering .................. 88 4.5.1.4Intersection-basedClustering ................ 89 4.5.1.5ComparisonofClusteringTechnqiues ............ 91 4.5.2AStockExchangeSystem ....................... 92 4.6Summary .................................... 95 6

PAGE 7

5DECISIONORDERINGBASEDINTRA-ANDINTER-PROPERTYLEARNI NG 96 5.1RelatedWork .................................. 97 5.2DecisionOrderingBasedLearnings ...................... 97 5.2.1Overview ................................. 98 5.2.2BitValueOrdering ........................... 99 5.2.3VariableOrdering ............................ 101 5.2.4ConrictClausebasedDecisionOrdering(Hybrid) .......... 102 5.3TestGenerationusingDecisionOrdering ................... 103 5.3.1TestGenerationforaSingleProperty ................. 104 5.3.1.1HeuristicImplementation .................. 105 5.3.1.2TestGeneration ........................ 106 5.3.2TestGenerationforaClusterofSimilarProperties .......... 107 5.3.2.1HeuristicImplementation .................. 108 5.3.2.2TestGeneration ........................ 110 5.4CaseStudy ................................... 111 5.4.1Intra-PropertyLearning ........................ 111 5.4.2Inter-PropertyLearning ......................... 115 5.4.2.1AMIPSProcessor ...................... 115 5.4.2.2AStockExchangeSystem .................. 118 5.5Summary .................................... 119 6EFFICIENTPROPERTYDECOMPOSITIONTECHNIQUES ......... 120 6.1Learning-OrientedPropertyDecomposition .................. 122 6.1.1PotentialLearningsforComplexProperties .............. 122 6.1.2SpatialPropertyDecomposition .................... 124 6.1.3TemporalPropertyDecomposition ................... 127 6.2DecisionOrderingBasedLearningTechniques ................ 130 6.3TestGenerationusingOurMethods ...................... 132 6.4AnIllustrativeExample ............................ 133 6.4.1SpatialDecomposition ......................... 133 6.4.2TemporalDecomposition ........................ 135 6.5Experiments ................................... 135 6.5.1AVLIWMIPSProcessor ........................ 136 6.5.2AStockExchangeSystem ....................... 138 6.6Summary .................................... 139 7REUSEOFVALIDATIONEFFORTFORASSERTION-BASEDEQUIVALEN CE 140 7.1RelatedWork .................................. 142 7.2AFrameworkforCheckingTLM-to-RTLFunctionalEquival ence ...... 144 7.2.1AutomaticTransactionLevelValidation ................ 144 7.2.1.1GenerationofTLMAssertions ................ 145 7.2.1.2GenerationofTLMTests .................. 146 7.2.2RenementofTLMAssertionsandTests ............... 147 7

PAGE 8

7.2.2.1SymbolMapping ....................... 148 7.2.2.2AssertionRenementRules ................. 148 7.2.2.3TestRenementRules .................... 150 7.2.3APrototypeToolforTLM-to-RTLValidationRenement ...... 151 7.2.3.1TLM2SMV .......................... 152 7.2.3.2TLMTestGeneration .................... 153 7.2.3.3TLM2RTL .......................... 153 7.2.4Assertion-BasedFunctionalEquivalence ............... 154 7.2.4.1Assertion-BasedFunctionalCoverage ........... 154 7.2.4.2AssertionOrdering ...................... 155 7.2.4.3AssertionBasedFunctionalEquivalence .......... 157 7.3CaseStudy ................................... 159 7.3.1ARouterExample ........................... 159 7.3.2APipelinedProcessorExample .................... 164 7.4Summary .................................... 165 8CONCLUSIONSANDFUTUREWORK ..................... 166 8.1Conclusions ................................... 166 8.2FutureResearchDirections ........................... 167 REFERENCES ....................................... 169 BIOGRAPHICALSKETCH ................................ 176 8

PAGE 9

LISTOFTABLES Table page 1-1Acomparisonforfouroptimizations ........................ 20 2-1BreakdownofatokeninFigure2-8 ........................ 36 2-2ConditionontherowedgesinFigure2-8 ...................... 36 3-1Comparisonoftwomethods ............................. 61 3-2Implementationlevelcoverageofthecontrolsystem ................ 61 3-3Comparisonofthreemethods ............................ 62 3-4ImplementationlevelcoverageofOSES ....................... 63 4-1Vericationresultsforastructure-basedcluster .................. 85 4-2Structure-basedclusteringresultsforMIPSprocessor ............... 86 4-3Vericationresultsforatextualcluster ....................... 87 4-4TextualclusteringresultsforMIPSprocessor ................... 88 4-5Vericationresultsforaninruence-basedcluster .................. 89 4-6Inruence-basedclusteringresultsforMIPSprocessor ............... 90 4-7Vericationresultsforanintersection-basedcluster ................ 91 4-8Intersection-basedclusteringresultsforMIPSproces sor .............. 91 4-9PropertyclusteringandvericationforMIPSprocessor .............. 92 4-10Structure-basedclusteringresultsforOSES .................... 93 4-11TextualclusteringresultsforOSES ......................... 93 4-12Inruence-basedclusteringresultsforOSES ..................... 94 4-13Intersection-basedclusteringresultsforOSES ................... 94 4-14PropertyclusteringandvericationforOSES ................... 94 5-1Testgenerationresultsusingintralearnings .................... 113 5-2TestgenerationresultforMIPSprocessor ..................... 117 5-3Testgenerationresultforstockexchangesystem .................. 118 6-1TestgenerationresultforMIPSprocessor ..................... 136 9

PAGE 10

6-2TestgenerationresultforOSES ........................... 138 7-1Assertionrenementfortherouterexample .................... 161 7-2RTLcoveragefortherouterexample ........................ 163 7-3AssertionsrenementfortheAlphaAXPprocessor ................ 164 7-4RTLcoveragefortheAlphaAXPprocessor .................... 164 10

PAGE 11

LISTOFFIGURES Figure page 1-1SoCdesignandvalidationrow ........................... 15 1-2Comparisonoffunctionalvalidationbetweenspecicat ionandimplementation 18 1-3Top-downvalidationofSoCarchitectures ...................... 21 2-1MappingfromaSystemCstructuretocorrespondinggraph model ........ 26 2-2Anexampleofdatatypetransformation ...................... 28 2-3AnexampleofSystemCTLMmodule ....................... 29 2-4AnexampleofSMVmodule ............................. 30 2-5AnexampleofTLMprocess ............................. 31 2-6AnexampleofSMVprocess ............................. 32 2-7UMLactivitynodes ................................. 33 2-8TheUMLactivitydiagramofanATM ....................... 35 2-9Thegeneratedskeletonafterstructureextraction ................. 42 2-10Translationrulesforstateanddatatransitions ................... 43 2-11TheTLMstructureoftherouter .......................... 45 2-12GraphmodelofaVLIWMIPSprocessor ...................... 46 2-13TLMoftheAlphaAXPprocessor .......................... 47 2-14Theactivitydiagramforacontrolsystem ..................... 48 2-15Theactivitydiagramforastockexchangesystem ................. 49 3-1Testgenerationusingmodelchecking ........................ 50 3-2Faultmodelexamples ................................ 55 4-1Ourtestgenerationmethodology .......................... 64 4-2Conrictanalysisusinganimplicationgraph .................... 68 4-3Anexampleofnamesubstitution .......................... 81 4-4Anexampleofconrictclausereuse ......................... 84 5-1TwoexamplesofSATsearch ............................. 99 11

PAGE 12

5-2Ascenariowherebit-valueorderingworks ..................... 100 5-3Ascenariowherebitvalueorderingfails ...................... 101 5-4Anexampleofbit-valueandvariableordering ................... 101 5-5Anexampleofconrictclausesbasedvariableordering ............... 102 5-6Learningtechniquesforasingleproperty ...................... 106 5-7Statisticsfortwoproperties ............................. 108 5-8Conrictstatisticsusingvariousintra-propertylearn ings .............. 114 5-9Implicationstatisticsusingvariousintra-learnings ................. 115 5-10ConrictstatisticsforMIPSprocessor ........................ 116 5-11ImplicationstatisticsforMIPSprocessor ...................... 118 6-1Twopropertydecompositiontechniques ...................... 120 6-2Ourtestgenerationframework ........................... 121 6-3TheCOIofadesignblock .............................. 123 6-4Afunctionalscenariowiththreetransactions .................... 124 6-5ADAGofeventrelation ............................... 128 6-6Learningstatisticsappliedondecisiontrees .................... 131 6-7EventimplicationgraphforpropertyP ....................... 135 6-8PropertycheckingresultforMIPSprocessor .................... 137 7-1Ourequivalencecheckingframework ........................ 144 7-2Thestructureofourprototypetool ......................... 152 7-3Anexampleofassertionequivalence ......................... 159 7-4ThepacketformatoftherouterinTLMandRTL ................. 160 7-5TheI/Ointerfaceoftherouterexample ...................... 161 7-6AnexampleofTLM-to-RTLrenement ...................... 162 12

PAGE 13

AbstractofDissertationPresentedtotheGraduateSchool oftheUniversityofFloridainPartialFulllmentofthe RequirementsfortheDegreeofDoctorofPhilosophy EFFICIENTAPPROACHESFORFUNCTIONALVALIDATIONOFSOCDESI GNS USINGHIGH-LEVELSPECIFICATIONS By MingsongChen August2010 Chair:PrabhatMishraMajor:ComputerEngineering Increasingcomplexitycoupledwithtime-to-marketpressu recreateacriticalneed toraisetheabstractionlevelforSystem-on-Chip(SoC)des igns.Functionalvalidation iswidelyacknowledgedasamajorbottleneckduetolackofau tomatedtechniquesand limitedreuseofvalidationeortsbetweenabstractionlev els.Simulationisthemostwidely usedformofvalidationusingrandomorconstrained-random tests.Directedtestsare verypromisingforsimulationsinceonlyfewerdirectedtes tsarerequiredcomparedto billionsofrandomteststoachieveacoveragegoal.Current ly,directedtestgenerationis performedmanuallywhichistime-consuminganderror-pron e.Thisdissertationpresents anoveltop-downmethodologyforautomaticallygenerating directedtestsfromhigh-level specicationsandreusethemacrossdierentabstractionl evels.Theobjectiveistoreduce theoverallfunctionalvalidationeort.Myresearchhasfo urmajorcontributions:i)it proposesamethodthatcanextractformalmodelsfromhigh-l evelSoCspecications;ii)it presentsanapproachthatcanautomaticallyderivepropert iesbasedonfaultmodels;iii)it proposesecientclustering,learninganddecompositiont echniquestoreducethedirected testgenerationtime;andiv)itprovidesvalidationrenem entapproachestoenablereuse ofthesystem-levelvalidationeortsforlow-levelimplem entationvalidationaswellasto checktheconsistencybetweendierentabstractionlayers .Ourexperimentalresultsusing bothsoftwareandhardwarebenchmarksdemonstratethatthe proposedapproachescan signicantlyreducetheoverallvalidationeort. 13

PAGE 14

CHAPTER1 INTRODUCTION Functionalvalidation 1 iswidelyacknowledgedasamajorbottleneckinSystem-on-C hip (SoC)designmethodology{upto70%oftheoveralldesigntim eandresourcesarespent onfunctionalvalidation.Inspiteofsuchextensiveeorts ,majorityoftheSoCdesigns failattheveryrsttime(siliconfailures)primarilyduet ofunctionalerrors[ 79 ].The functionalvalidationcomplexityisexpectedtoincreasef urtherduetothecombined eectsofincreasingdesigncomplexityandrecentparadigm shiftfromsingleprocessorSoC designstoheterogeneousmultiprocessorarchitectures[ 90 ]. TraditionalSoCvalidationadoptsacombinationofsimulat ion-basedapproachesand formalmethods.RandomtestingiswidelyusedforSoCsimula tion.Ingeneral,random testscannotguaranteethecoverageanditmayexercisethes amefunctionalscenariofor severaltimesbecauseofrandomness.Thusdirectedtestsar eabetteralternativesince onlyasmallnumberoftestsarerequiredtoachieveafunctio nalcoveragegoalcompared torandomorconstrained-randomtests.However,duetolack ofautomatedtoolsto generatedirectedtests,humaninterventionisnecessaryd uringthetestgeneration.All thesescenarioscanleadtotime-consuminganderror-prone validation.Myresearch targetstoreducetheoverallfunctionalvalidationeortb yautomatingvariousstepsinthe thevalidationrowaswellasbydevelopingecientlearning andreusetechniques. Therestofthechapterisorganizedasfollows.Section 1.1 presentstheSoCdesign row.Section 1.2 surveystheexistingSoCfunctionalvalidationmethods.Fi nally, Section 1.3 presentsthecontributionsofthisdissertation. 1 Theterm\validation"generallyreferstosimulation-base dapproaches,while \verication"isusedforbothsimulation-basedandformal methods.Thisdissertation focusesondirectedtestgenerationforsimulation,soitus esthetermvalidation. 14

PAGE 15

1.1SoCDesignFlow SoCintegratesallcomponentsofacomputerintoasingleint egratedcircuit(chip).It consistsofbothhardware(suchasprocessor,memoryandper ipherals)andsoftware(such asapplicationprograms).SoCmayperformavarietyofcompu tationsincludingdigital, analogandmixed-signalfunctions.Thusitiswidelyusedin theeldofembeddedand hybridsystems. SoCisbecomingincreasinglycomplexsincenewapplication srequiremorefeatures. Asaresult,extensivesystem-levelsimulationsarerequir edtomaketherightarchitectural trade-os.Toecientlyandquicklymakethedecisiononthe setrade-os,design architectsincreasinglyleveragesystem-levelspecicat ionsinsteadofimplementations toperformsuchanalysis. Specification Validation Implementation Validation Specification C/JAVA Implementation (TLM/UML) Hardware Software HW/SW Partitioning VHDL/Verilog Figure1-1.SoCdesignandvalidationrow Figure 1-1 presentsaSoCdesignandvalidationrow.Varioushardwarea ndsoftware modelingparadigmsareusedforSoCspecications.Twoofth emostwidelyused specicationsareTransactionLevelModeling(TLM)[ 16 78 ]andUniedModeling Language(UML)[ 69 ].Theyestablishastandardtoenablefastsimulationspeed andeasy modelinteroperabilityforhardware/softwareco-design. Generally,TLMispromisingfor 15

PAGE 16

hardwaremodelingandUMLfocusesonsoftwaremodeling.TLM mainlyallowsmodeling ofcommunicationbetweendierenthardwarecomponentsofa systemanddataprocessing ineachcomponent.UMLcancapturebothstructuralandbehav ioralinformationof asoftwaresystem.Validatedspecicationcanbeusedasago ldenreferencemodelfor validationofsoftwareandhardwareimplementations.Alth oughspecicationscancapture mostimportantfunctionalscenarios(systembehaviors),s omeimplementationdetailscan bestillmissing.Forexample,TLMprovidestwokindsofmode lingstyles:loosely-timed modelscanbeusedtomodelthesystembehaviorwithlesstimi nginformationand approximately-timedmodelscanenabletiminganalysisofs ystembehavior.Although TLMispromisingforsystem-levelmodelingandsimulation, itisstillhardtoaccurately describethehardwarebehaviorbecauseitlacksmanydetail edinformationsuchastiming details.SoRegisterTransferLevel(RTL)isneededtomodel theimplementation-level behaviorafterthesystem-levelsimulation.InFigure 1-1 ,thehardwarepartwillbe implementedusingaRTLlanguagesuchasVHDLorVerilog,and thesoftwarewillbe implementedusingaprogramminglanguagesuchasCorJAVA.S ignicantamountof validationworkisneededtocheckthespeciedfunctionals cenariosaswellastocheckthe consistencybetweenthespecicationandimplementation. 1.2FunctionalValidationofSoCDesigns Specicationvalidationisextremelyimportanttoensuret hatthespecieddesignis correctandcanbeusedasagoldenreferencemodelfortheimp lementation.Accordingto [ 79 ],therearetwokeycontributorstotheSoCfailures(silico nrespin):specicationerrors andimplementationerrors.Asexpected,82%ofthedesignsw ithrespinsresultingfrom functionalrawshadimplementationerrors.Interestingly ,almost47%ofthedesignswith respinsresultingfromfunctionalrawshadalsoincorrecto rincompletespecications[ 79 ]. Therefore,itisnecessarytovalidatespecicationsbefor evalidatingtheimplementation. Thissectionrstsurveysexistingfunctionalvalidationm ethods,andthendescribes severalimprovementopportunitiestoreducetheoverallfu nctionalvalidationeort. 16

PAGE 17

1.2.1OverviewofFunctionalValidationMethods SimulationisthemostwidelyusedSoCvalidationmethod.Co mparedtorandom testingmethodswhichusebillionsofrandomandpseudo-ran domtestsinthetraditional designrow,directedtestsareverypromisinginreducingth eoverallvalidationeortsince asignicantlysmallernumberofdirectedtestscanachieve thesamecoveragegoal[ 61 ]. However,amajorproblemincurrentdirectedtestgeneratio napproachisthatitismostly performedbyhumanintervention.Hand-writtentestsentai llaboriousandtimeconsuming eortofvericationengineerswhohavedeepknowledgeofth edesignunderverication. Duetothemanualdevelopment,itisinfeasibletogeneratea lldirectedteststoachievea comprehensivecoveragegoalinashorttime.Automaticdire ctedtestgenerationbasedon acomprehensivefunctionalcoveragemetricisanalternati vetoaddressthisproblem. Modelchecking[ 21 ]isoneofthemostwidelyusedformalmethodsforautomatedt est generationtovalidatesoftware/hardwaredesigns[ 5 ].Inthecontextoftestgeneration,a designspecicationisdescribedusingaformalmodel.Ther equiredfunctionalscenarios aredescribedintheformoftemporallogicformulas.Whench eckingafalseproperty usingamodelchecker,onecounterexampleisreportedtofal sifytheproperty.Because thiscounterexampleisasequenceofvariableassignments, itcanbeusedasadirectedtest tovalidatethefunctionalscenarioofthespecication.Ho wever,modelcheckingbased techniquesdonotscalewellforlargedesignsduetothe\sta tespaceexplosion" 2 Simulationbasedmethodsarefastbutcannotguaranteethec onvergenceoffunctional coverage.Modelcheckingbasedmethodscanautomaticallyg eneratedirectedtestsbut cannotdealwithlargedesigns.Currently,mostSoCvalidat ionapproachesuseahybrid methodwhichincorporatesbothtechniques.Thehybridmeth odrstperformsthe 2 Thenumberofstatesgeneratedforverifyingapropertyishu geandcannotbe handledduetothememorycapacityofcomputers. 17

PAGE 18

randomsimulationtogetasmuchfunctionalcoverageasposs ible.Thentheuncovered functionalscenariosandcornercasesareactivatedusingt hedirectedtests. 1.2.2PotentialImprovementOpportunities Sincesystem-levelspecicationistreatedasthegoldenre ferencemodelinthe SoCdesignrow,alogicerrorinthesystem-levelspecicati oncertainlywillcause themalfunctionintheimplementation.Becauseimplementa tionsaremorecomplex thansystem-levelspecications,ndinganerrorinimplem entationswillbemore time-consuming.Soitisnecessarytoguaranteethatsystem -levelspecicationvalidation cancoverasmanyfunctionalscenariosaspossible.Inaddit ion,thedierencesbetween specicationandimplementationlimitthedegreeofvalida tionreuse.Intheabsence ofsignicantreuseofvalidationeortsbetweendierenta bstractionlevels,theoverall functionalvalidationeortwillincreasesincedesigners havetoverifythespecicationas wellasitsimplementation. Validation ComplexityFunctional ScenariosF Totalb) Functional Scenarios a) Validation Complexity F Imp T Spec T Imp F Spec Figure1-2.Comparisonoffunctionalvalidationbetweensp ecicationandimplementation Figure 1-2 comparesspecicationandimplementationlevels.Assumet hata design D hasatotal F Total numberoffunctionalscenariosthatneedtobechecked. Forspecications,thereare F Spec numberoffunctionalscenariosthatneedtobechecked, andeachspecicationleveltestgenerationneedanaverage timeof T Spec .Inaddition to F Spec functionalscenarios,thereare F Imp functionalscenariosneedtobecheckedin implementations,andeachimplementationtestneedsanave ragetimeof T Imp .Figure 1-2 18

PAGE 19

a)indicatesthatwhencheckingafunctionalscenario,impl ementationvalidationismore dicultthanspecicationvalidation.Figure 1-2 b)showsthatspecicationscovermajority oftheoverallsystemfunctionalscenarios(e.g.,70%),and implementationsinheritall suchscenarioswithitsownnewadditionalfunctionalscena rios(e.g.,30%)duetothe introductionofimplementationdetails.Inthisdissertat ion,thecomplexityofvalidating afunctionalscenarioisequivalenttogeneratingandapply ingadirectedtest.Sotest generationandcorrespondingsimulationtimeisusedtoind icatethefunctionalscenario validationeort. Inordertoachievea100%functionalcoverageaswellastomi nimizetheoverall specicationandimplementationtestgenerationtime,iti snecessarytondamethodto optimizetheEquation( 1{1 ). Minimize : F Spec T Spec + f F Spec + F Imp g T Imp Subjectto : 8>>>>><>>>>>: F Spec + F Imp = F Total T Spec <F Imp (1{1) Fordirectedtestgeneration,therearefourfeasibleoptio ns.Table 1-1 comparesthese approaches. Nooptimization: Specicationleveltestgenerationandimplementationlev eltest generationareindependent,andineachleveltherearenoop timizations. Specicationleveloptimization: Specicationleveltestgenerationand implementationleveltestgenerationareindependent.The overallspecication testgenerationtimecanbereducedbycertainoptimization methods. Reusebetweenspecicationandimplementation: Nooptimizationfor specicationandimplementationleveltestgeneration,bu tthespecicationtests canbereusedforimplementationlevelvalidation. Specicationleveloptimization+reusebetweenspecicat ionandimplementation: Optimizationsreducetheoverallspecicationleveltestg enerationtime, andthespecicationleveltestscanbereusedforimplement ationlevelvalidation. 19

PAGE 20

Assumethatinsystemvalidationwecanndaspecicationle veltestgeneration optimizationthatcanproduce times( > 1)speedup,andwecanobtainanother times( > 1)speedupduetovalidationreuse.Accordingtothecompari sonshownin Table 1-1 ,thelastoptioncanachievethebestpossibleperformance. Thegoalofthis dissertationistodevelopecienttechniquestoreducethe overallvalidationeortusing thefourth(last)option. Table1-1.Acomparisonforfouroptimizations OptimizationTime None F Spec T Spec + F Total T Imp Specicationlevel F Spec T Spec = + F Total T Imp Reuse F Spec T Spec + F Spec T Imp = + F Imp T Imp Specicationlevel+Reuse F Spec T Spec = + F Spec T Imp = + F Imp T Imp 1.2.3Challenges Eachofthecomponents(suchasIPcores,processorsandmemo ries)inaSoC designcanbeveriedusingexistingvalidationapproaches .However,thevalidationof theoverallsystemisextremelycomplexduetoexponentiall ylargenumberofpossible interactionsthatareextremelyhardtomodel,analyzeandv alidate.Althoughthe potentialimprovementsproposedintheprevioussectionse emspromising,therearefour fundamentalproblemsinautomatedgenerationofdirectedt estsforSoCarchitectures. TherstchallengeistodecidespecicationmodelsforSoCa rchitecturesandhow toverifythespecicationtoensurethatitcanbeusedasago ldenreferencemodel. Thenextchallengeistoidentifyacomprehensivefunctiona lcoveragemetrictoenable coverage-drivengenerationofpropertiesandassociatedh igh-leveltests.Thethirdand mostimportantchallengeishowtosignicantlyreducethet estgenerationcomplexity toavoidstatespaceexplosionproblem.Finally,duetosign icantdierencesbetween specicationsandimplementations,amajorchallengeisho wtoecientlyreusethe specication-levelpropertiesandtestsforvalidationof SoCimplementations. 20

PAGE 21

1.3DissertationContributions Myresearchemploysatop-downvalidationmethodologyusin gacombinationof simulationbasedapproachesandformalmethodstoaddresst hefourchallengesmentioned inSection 1.2.3 .Theobjectiveofmyresearchistodeveloptools,technique sand methodologiestoenableautomaticgenerationofdirectedf unctionalteststodrastically reducetheoverallvericationeortaswellastoimproveth equalityofSoCdesigns. Property Generation Validation Specification Properties TLM Tests RTL Tests Coverage Model (fault models) SOC Architecture (System-level Models) Refinement Test Property RefinementGeneration Test Clustering Property Clusters Property System-level Tests RTL Assertions SOC Design(Implementation) Figure1-3.Top-downvalidationofSoCarchitectures Figure 1-3 outlinestheproposedvalidationmethodologyforSoCarchi tecturesusing system-levelspecication.Itconsistsoffourmajorcontr ibutionsasfollows: FormalmodelingofSoCdesigns: SincemostexistingSoCspecicationsare notformalenoughtoenableautomatedtestgeneration,this dissertationproposes anapproachforautomaticspecicationanalysis.Itcanext ractformalmodelsfrom semi-formalhardwareandsoftwarespecications. Coverage-drivenpropertygeneration: Functionalcoverageplaysanimportant roletodeterminetheadequacyoffunctionalvalidation.Th isdissertationdenes variousfaultmodelsforSoCspecications.Basedonthesef aultmodels,wecan automaticallyderivepropertiestovalidatethespeciedf unctionalscenarios. Ecientdirectedtestgeneration: Toreducetheoveralltestgenerationtime forthesamedesignwithalargesetofproperties,thisdisse rtationproposesvarious clusteringmethodswhichcanclusterthesimilarpropertie stogethertosharethe learningsduringtestgeneration.Theproposedframeworki nvestigatestwokinds oflearningsbasedonconrictclauseforwardingaswellasde cisionordering.Such learningscanbeusedtoavoidrepeatedvalidationeortsbe tweensimilarproperties. Forcomplexpropertieswithoutlearningopportunity,this dissertationproposestwo decompositiontechniquesthatcanactivelyachievethelea rningtoreduceitstest generationtime. Automatedrenementofvalidationeorts: Thisdissertationdevelopsa prototypetoolwhichcanautomaticallyconvertTLMlevelte stsandpropertiesinto 21

PAGE 22

RTLtestsandassertionstoenableimplementationlevelval idation.Basedonthis validationeortreuse,thisdissertationproposesametho dologywhichcancheckthe assertion-basedfunctionalequivalencebetweenspecica tionsandimplementations. Therestofthisdissertationisorganizedasfollows.Chapt er 2 describeshowto extractformalmodelsfromsystemlevelspecicationsofSo Cdesigns.Chapter 3 describes howtogeneratepropertiesbasedonourproposedfaultmodel s.Chapter 4 to 6 discuss howtoecientlygenerateteststoenablefunctionalvalida tion.Chapter 4 describes howtodividethepropertiesintoseveralgroupssuchthatea chgroupcontainssimilar propertiesthatcanbenetfromeachotherduringtestgener ation.Chapter 5 presentsthe decisionorderingbasedlearningtechniqueswhichcandras ticallyreducetheoveralltest generationtime.Chapter 6 proposesvariousdecompositiontechniquestoactivelynd thelearningsforacomplexproperty.Chapter 7 presentsthemethodologyforautomated propertyandtestrenements.Italsodescribeshowtoutili zethevalidationrenementfor functionalequivalencechecking.Finally,Chapter 8 concludesthedissertationandoutlines severalfutureresearchdirections. 22

PAGE 23

CHAPTER2 FORMALMODELINGOFSOCSPECIFICATIONS ModelingplaysacentralroleindesignautomationofSoCarc hitectures.Itis necessarytodevelopaspecicationlanguagethatcanmodel complexsystemsatahigher levelofabstractionandalsoenableautomaticanalysisand generationofecientreference models.Thelanguageshouldbepowerfulenoughtocapturehi gh-leveldescriptionofa widevarietyofSoCarchitecturesaswellasshouldbesimple enoughtoallowcorrelation oftheinformationbetweenthespecicationandthearchite cture/systemmanual. Asasystemlevelspecication,SystemCTLM[ 78 ]establishesastandardtoenable fastsimulationspeedandeasymodelinteroperabilityforh ardware/softwareco-design. Itmainlyfocusesonthecommunicationbetweendierentfun ctionalcomponentsofa systemanddataprocessingineachcomponent.AlthoughUMLi sbeingusedasadefacto softwaremodelingtool,UMLProleforSoC[ 68 ]isproposedasanextensionofUML 2.XtoenableSoChardwaremodeling.Itcanbeusedtocapture thesystembehavior forbothSoCsoftwareandhardwarecomponents[ 19 65 77 ].However,bothSystemC TLMandUMLdiagramsarenotformalenoughforautomatictest generationusingmodel checkingtechniques[ 5 ].Consequently,theambiguity,incompleteness,andcontr adictionin specicationscanleadtodierentinterpretations.There foreitisnecessarytoformalize thesemanticsofSoCspecications. ThischapterintroducestwowidelyusedSoCspecications: SystemCTLMsfor hardwaremodeling,andUMLactivitydiagramsforsoftwarem odeling.Next,itdescribes howtoautomaticallyextracttheformalmodelsfromspecic ationstoenablesubsequent validationsteps.Therestofthechapterisorganizedasfol lows.Section 2.1 introducesthe formalmodelingofSystemCTLMs.Section 2.2 proposestheformalmodelingtechniques ofUMLactivitydiagrams.Section 2.3 presentsthecasestudiesusingbothSystemCTLM designsandUMLactivitydiagrams.Finally,Section 2.4 summarizesthechapter. 23

PAGE 24

2.1SpecicationusingSystemCTLMs AsaframeworkbuiltonC++,SystemC[ 70 ]deliberatelymimicsthehardware descriptionlanguagessuchasVHDLandVerilog.Withaneven t-drivensimulation kernel,SystemCcanbeusedtosimulatethebehaviorofconcu rrentprocesseswhichcan communicatewitheachotherusingprocedurecallsorotherm echanismsoeredbythe SystemClibrary.Generally,SystemCisoftenassociatedwi thTransaction-LevelModeling (TLM)[ 16 78 ],becauseSystemCTLMprovidesawrappertofacilitatethep rocessof communicationmodeling.SinceSystemCTLMprovidesarapid prototypingplatformfor thearchitectureexplorationandhardware/softwareinteg ration[ 30 ],itiswidelyusedto enableearlyexplorationforbothhardwareandsoftwaredes igns.Itcanreducetheoverall designandvalidationeortofcomplexSoCarchitectures. Toenableautomatedanalysis,variousresearchershavetri edtoextractformal representationsfromSystemCTLMspecications.Abdietal .[ 2 ]introduced ModelAlgebra ,aformalismforrepresentingSoCdesignsatsystemlevel.T heworkbyKroeninget al.[ 48 ]formalizedthesemanticsofSystemCbymeansoflabeledKri pkestructures.Moy etal.[ 64 ]providedacompilerfront-endthatcanextractarchitectu reandsynchronization informationfromSystemCTLMdesignsusingHPIOM.Karlsson etal.[ 41 ]translated SystemCmodelsintoaPetri-NetbasedrepresentationPRES+ .Thismodelcanbeused formodelcheckingofpropertiesexpressedinatimedtempor allogic.Habibietal.[ 34 ] proposedamethodthatadoptstheformalmodelAsmL.Astatem achinegeneratedfrom AsmLcanbeveried,andthencanbetranslatedtobothSystem Ccodeandproperties forlowlevelvalidation.Allthesemodelingtechniquesfoc usontheformalmodelingof SystemCspecications.However,noneoftheminvestigatet heautomatedtestgeneration fortransactionvalidation.Thissectiondiscusseshowtoe xtracttheformalmodelsfrom SystemCTLMspecicationstoenableautomatedtestgenerat ion. 24

PAGE 25

2.1.1FormalModelingofSystemCTLMs Asahighlevelspecication,SystemCTLMemphasizesthefun ctionalityofthedata transfersinsteadofactualimplementation.ASystemCTLMd esigninterconnectsasetof processescommunicatingwitheachotherusingtransaction datatoken(i.e.,C++objects). Theinitialprocessstartsacommunication,andthetargetp rocesspassivelyresponds tothecommunication.Similartotheproducer/consumermod els,eachprocessdoesthe followingtasks:consumingdata,processingdataandprodu cingdata. SinceSystemCisbasedonC++,itsupportsvariousprogrammi ngconstructs(e.g., template,inheritance,etc.).Althoughtheconceptofsome TLMcomponents(signals, ports,etc.)iseasy,theirC++implementationdetailsarer eallycomplex.Therefore, directlytranslatingtheirbehaviorstoenableautomatedv alidationisdicult.Inour framework,weabstractsuchSystemCcomponentsandhidethe implementationdetails usingthepre-denedSMVconstructs.Furthermore,theunde rlyingcomplexSystemC scheduleraggravatesthemodelingcomplexity.ForSystemC TLM,tomimictheparallel executionofprocesses,theSystemCscheduleractivatesth e ready-to-run processesina \non-deterministic"way.However,sinceSMVisparallelin essence,itisnotnecessaryto modeltheSystemCschedulerexplicitly. ForTLM,twomostimportantfactorsarethetransactiondata tokenandthe transactionrow.SotheextractedformalmodelofTLMspeci cationsshouldrerect bothinformation.Inourtestgenerationframework,itisre quiredthattheextracted modelscannotonlyguidethegenerationofSMVspecication ,butalsocanbeusedto automaticallyderivethepropertiesforTLMtestgeneratio n.Denition1givestheformal modelofSystemCTLMdesigns. Denition1. The formalmodel ofaSystemCTLMdesignisaneight-tuple( ,P,T, A,E,M,I,F)where isasetoftransactiondatatokens. P = f p 1 ;p 2 ;:::;p m g isasetofplaces. 25

PAGE 26

T = f t 1 ;t 2 ;:::;t n g isasetoftransitions. A f P T g[f T P g isasetofarcsbetweenplacesandtransitions. E = f e 1 ;e 2 ;:::;e k g isasetofarcexpressions.ThemappingExpression( a i )= e i ( a i 2 A 1 i k )givestheenablecondition e i for a i .Atokencanpassarc a i only when e i istrue. M :2 P T 2 P isafunctionthatdescribestheinternaloperationsoninpu t transactiondataandoutputtransactiondataofatransitio n. I 2 2 P speciestheinitialstate. F 2 P speciesthenalstates. Inourframework,weusethegraphmodelasanimmediateformt ocapturethe executionaswellasinterconnectionofprocesses. M1M2 M3M4 M6 M5 t1 t2 t3 t4 t5t6 a) Interconnection of modules b) Graph model of the module interconnections Figure2-1.MappingfromaSystemCstructuretocorrespondi nggraphmodel Figure 2-1 a)showsaninterconnectionofsixmodules.Eacharrowindic atesa portbindingbetweentwomodules.Figure 2-1 b)showsthegraphrepresentationofits correspondingformalmodel.Intheformalmodel,eachcircl eiscalleda place thatisused toindicatetheinputoroutputbuerofamodule.Itcantempo rarilyholdthetransaction dataforlaterprocessing.Theverticalbarsare transitions whichareusedtoindicate moduleswhichcontainprocessestomanipulateinputandout puttransactiondatatokens. Theplaceswithoutincomingarcsare initialplaces whichstartatransition.Theplaces 26

PAGE 27

withoutoutgoingarcsare targetplaces .Atransactiondatatokenrowsfromtheinitial placestothetargetplacesandtokenvaluesmaychangeintra nsitionswhennecessary. Theinternallogicofatransitiondeterminestherowofthet ransaction. 2.1.2TransformationfromSystemCTLMtoSMV Modelcheckingtechniquesareverypromisingfordirectedt estgenerationinhardware andsoftwaredomains[ 5 9 71 ].Inourframework,weadoptSMV[ 56 ]astheformal specicationtodescribeboththestructureandbehaviorin formationofSystemCTLMs becauseofthefollowingreasons.First,theunderlyingsem anticsofSMVissimilarto thesemanticsofSystemCscheduler.SowecanmimicmostTLM' sbehaviorsusingSMV withoutmodelingcomplexschedulerbehavior.Second,SMVa ndTLMhavethesimilar structurehierarchy.Eachprocessingunitencapsulatedby aTLMmodulecorrespondsto aSMVmodule.Theinterconnections(e.g.channels,portsan dsockets)betweenTLM modulescanbeabstractedbyusingmoduleparametersinSMV. Third,likeSystemC, SMVprovidesarichsetofprogramminglanguageconstructss uchas if-then-else caseswitch and forloop statements.Fourth,SMVmainmoduleconnects,similartoSy stemC, eachcomponentofthesystem.Finally,SMVsupportsvarious kindsofdatatypesand dataoperations.Especiallyuserscandenetheirowndatat ype.AlloftheseSMV featuresfacilitatethetranslationfromTLMstoSMVspeci cation.Itisimportantto notethat,duetotheexpressivenessoftheSMVlanguage,cur rentlyourframeworkjust supportslooselytimedmodeling.Weareplanningtousethet imedautomatachecker (suchasUPPAAL[ 8 ])inourframeworktoenablethetimingvericationoftrans actions. AsanintermediateformforTLMtoSMVtranslation,thegraph modelprovides bothstructureandbehaviorinformation.Suchinformation needtobecollectedfora translationtoaSMVrepresentationtoenableautomateddir ectedtestgeneration.The structureinformationincludesthedatatypedenitionand connectivitybetweenmodules. Itcorrespondstothedescriptionoftransactiondatatoken aswellasinterconnection oftransitionsandplacesinthegraphmodel.Thebehaviorin formationcontainstoken 27

PAGE 28

processingandtokenrouting.Intheformalmodel,itrepres entstheinternalprocessing ofatransition.Thissectiondiscusseshowtoextractboths tructuralandbehavioral informationandtransformittoaSMVspecication.Weuseth eexampleshownin Section 2.3.1 toillustratehowtoextracttheformalmodelfromarouterex ample. 2.1.2.1StructureExtraction InTLM,thecontentofatransactiondatatokenindicatesthe transactionrowand theoutputofeachcomponent.Soitconsistsofthekeypartof TLMtests.Generallya transactiontokenconsistsofseveralattributeswithdie renttypes.Becausedatatype determinesthesizeofthespeciedvariablewhichinturna ectsthemodelchecking performance,itisnecessarytogureoutthedatatypeofato ken.BesidesallnativeC++ types,SystemCdenesasetofdatatypeclasseswithinthena mespace sc dt torepresent valueswithapplication-specicwordlengthsapplicablet odigitalhardware.SMValso supportsvariousdatatypessuchasarray,Boolean,integer ,structandsoon.Suchdata typedenitionsfacilitatethemappingofdatatypesbetwee nSystemCTLMandSMV specication.Duringthetransformation,thewordlengths ofuser-denedtypeneedtobe considered.Figure 2-2 givesanexampleoftherouter packet intheformofSystemCTLM andSMVrespectively.Forexample, sc uint< 2 > has2bitsandwillbetransformedtoa range0 :: 3inSMV. classpacket{typedefpacketstruct{public: sc_uint<2>to_chan;to_chan:0..3;sc_uint<6>payload_sz;payload_sz:0..63;sc_uint<8>payload[4];payload:array3..0of0..255;sc_uint<8>parity;parity:0..255; };} a)packetinSystemCTLMb)packetinSMV Figure2-2.Anexampleofdatatypetransformation Derivedfromthebaseclass sc module ,TLMmodulesarethemainprocessingunits forthetransactiondata.Generallyeach sc module containsthedenitionsofprocesses whosetypesare SC METHOD or SC THREAD .Modulescommunicatewitheachother 28

PAGE 29

bysendingandreceivingtransactiondatatokensviaoutput andinputports.SystemC providesacommunicationwrapperforthesystemcomponents (modules).InSystemC, thereexistsvariousbindingmechanism(e.g.porttoexport binding,exporttoexport bindingandporttochannelbinding)toestablishinterconn ectionbetweenmodules. Usuallyeachbindingcorrespondstoachannelsuchasarstin-rst-out(FIFO)channel totemporarilyholdtransactiontokens. classrouter:publicsc_module{ public: sc_export>packet_in;sc_export>packet_out0;sc_export>packet_out1;sc_export>packet_out2;router(sc_module_namemodule_name);voidroute(); private: tlm_fifochan0,chan1,chan2,input_;packettmp_packet; }; Figure2-3.AnexampleofSystemCTLMmodule Figure 2-3 showstheTLMmodulestructureofarouter.Theclass sc export canbe usedasaporttocommunicatewithothermodules.Becausethe interfacetypeofport packet in is tlm put if ,itisaninputport.Incontrast, packet outx (x=0,1,2) havetheinterface tlm fifo get if ,sotheyareoutputports.Duringtherouter communication,eachconnectionbetweenaportandanexport usesaFIFOchannelto temporarilyholdapacket. StructurallysimilartoSystemCTLMs,SMVspecicationisa lsomodularized andhierarchicallyorganized.Sotheextractionofstructu reinformationneedstomap theTLMconstructsintotherightplaceoftheSMVspecicati on.Figure 2-4 shows theSMVmoduleskeletoncorrespondingtoexampleinFigure 2-3 afterthestructure extraction.InSMV,amoduleusestheparametersastheinput andoutputportstoboth communicatewithothermodulesandcongurethesystemstat usdenedinthe main module.IntheexampleofFigure 2-4 ,theSMVmodulehasoneinputportandthree 29

PAGE 30

outputports.Thetypeoftheinputandoutputportsispacket .Allthedeclarationsof membervariablesexceptfortheFIFOchannelsaredeclaredi ntheSMVspecication. BecauseaFIFOchanneltogetherwithitsportpairsareabstr actedasaSMVparameter, itisnotnecessarytocreateavariableinSMVexplicitly.Ba sedoncontextduringthe elaboration,someofthedeclaredvariableswillbeinitial ized.InSMVspecication,each outputportsandlocalvariablesneedtobeinitialized.For example, packet out 0isa parameterwhichreferstoanoutputport,soitwillbeinitia lizedwithavalue\0".Inour framework,itisrequiredthatallsuchmoduleconnectionss houldbedenedinthemodule sc top modulerouter(packet_in,packet_out0,packet_out1,pack et_out2){ inputpacket_in:packet;outputpacket_out0,packet_out1,packet_out2:packet;tmp_packet:packet;init(packet_out0):=0;init(packet_out1):=0;init(packet_out2):=0;init(tmp_packet):=0;...... } Figure2-4.AnexampleofSMVmodule 2.1.2.2BehaviorExtraction TLMbehaviordescribestherun-timeinformationofTLMincl udingtransaction creation,transactionmanipulationandmodulecommunicat ion.Transactioncreation initializesatransactionbycreatingadatatoken(i.e.aC+ +object)withpropervalues. Transactionexecutiondescribesthetransactionrowamong themodules.Amoduleis acontainerwhichhasaclusterofrelevantprocesses.Suchp rocesseswillhandlethe incomingtransactiontokensanddecidewheretosendthemac cordingtothespecied conditions.Thusdierentvalueofatokenwillleadtodier enttransactionrows.In ourcurrentprototyperelease,therearetwokindofprocess communicationsupportedin transactionrows:1)directprocedurecallfromoneprocess toanotherprocess,and2) channel-basedeventstriggeredbytheprocedurecall.Fore xample,intheblockingmode, 30

PAGE 31

aprocesscanfetchatransactiondatatokenfromthespecie dinputportonlywhenthe correspondingchannelisnotempty.Otherwise,theoperati on\get"willbeblockeduntil thereisaneventtriggeredbythe\put"operationbyotherpr ocesses. router::router(sc_module_namemname):sc_module(mname ){ packet_in(input_);packet_out0(chan0);packet_out1(chan1);packet_out2(chan2);SC_METHOD(route);sensitive<)0) chan0.nb_put(tmp_packet); elseif(tmp_packet.to_chan==(sc_uint<2>)1) chan1.nb_put(tmp_packet); elsechan2.nb_put(tmp_packet); } Figure2-5.AnexampleofTLMprocess Figure 2-5 givesthemoduleprocess route oftherouterexample.Theprocessreceives apacketfromthedriverviachannel input ,andthenitdecideswheretosenddatabased onthepacketheaderinformation to chan TLMmodelingprovidessomesynchronizationmechanismfort hecommunications betweenmodules.AsshowninFigure 2-5 ,theroutercanfetchthedatafromtheFIFO queue input onlywhenthedriverputapackageandtheFIFOchannelevent ok to get is triggered.Thusthesynchronizationbetweentwomodulesis implicitlyachieved. SMVsupportsmanyconstructssimilartothecommonprogramm inglanguage suchas if-then-else switch-case and forloop .Sotheseconstructsfacilitatethebehavior modelingofprocessesfromTLMtoSMVspecication.Figure 2-6 isthetranslatedSMV specicationoftheTLMexamplepresentedinFigure 2-5 .Duringthetranslationfrom TLMtoSMV,weabstractachannelasanimplicitbuerbetween twoports.SoaSMV modulewillgettheinputdatafromitsinputports.Thereisn omappingofthechannelin transformedSMVspecication.Forexample,the tmp packet isassignedthevalueofthe packet in insteadofthevalueof input shownintheTLMexampleinFigure 2-5 31

PAGE 32

modulerouter(packet_in,packet_out0,packet_out1,pack et_out2){ ......next(tmp_packet):=packet_in;if(tmp_packet.to_chan=0){ next(packet_out0):=tmp_packet;next(packet_out1):=0;next(packet_out2):=0; }elseif(tmp_packet.to_chan=1){ next(packet_out0):=0;next(packet_out1):=tmp_packet;next(packet_out2):=0; }else{ next(packet_out0):=0;next(packet_out1):=0;next(packet_out2):=tmp_packet; } } Figure2-6.AnexampleofSMVprocess 2.1.3APrototypeToolForTLMtoSMVTranslation Wedevelopedaprototypetool TLM 2 SMV whichcantransformSystemCTLM specicationstocorrespondingSMVmodelsforautomateddi rectedtestgeneration.The detailsoftheimplementationaredescribedinSection 7.2.3.1 2.2SpecicationusingUMLActivityDiagrams Formalvericationcanbeusedtoverifythecorrectnessofs pecications,soitcanbe usedtoguaranteethequalityofUMLmodels[ 84 ].UMLactivitydiagramadoptsPetri-net semanticswhichispromisingtodescribetheconcurrentbeh avior[ 18 51 88 ].Thereare severalapproachesthatusemodelcheckingtechniquestove rifyUMLactivitydiagrams. Eshuis[ 75 ]presentedatranslationprocedurefromUMLactivitydiagr amstotheinput languageofNUSMV[ 20 ].However,thetranslationisusedtoverifytheconsistenc y betweenUMLactivitydiagramsandclassdiagrams.Itfocuse soncheckingtheconsistency betweentwodierentmodels.GuelandMammar[ 31 ]providedaformaldenitionfor timedactivitydiagrams.Theyoutlinedthetranslationfro mthesemanticspecications intoPROMELA-aninputlanguageoftheSPINmodelchecker.Da setal.[ 22 ]proposed amethodtodealwithtimingvericationofUMLactivitydiag rams.Alltheseverication 32

PAGE 33

workprimarilyfocusoncheckingtheconsistencyorcorrect nessofthemodelitselfinstead ofgeneratingdirectedtestcases. Inthischapter,weadoptUML2.1.2[ 69 ]asourspecication.Toreducethe complexityofthetestingwork,werestrictourtestingtarg etandinvestigateasubset ofactivitydiagrams.Thesubsetmainlycontainsactionnod es,controlnodes,objectnodes andcontrolanddatarow.Especiallyfortheobjectnode,wea ssumethatitcanholdat mostoneobjectatatimeanditdoesnotsupport competition and datastore .Thissection rstgivesthenotationsusedinUMLactivitydiagrams.Then itpresentstheformal denitionsoftheUMLactivitydiagrams.Finally,itdescri besthetranslationfromUML activitydiagramstoSMVformalmodels.2.2.1Notations UMLactivitydiagramisusedtocoordinatetheexecutionofa ctions.Anactiontakes asetofinputsandconvertsthemintocorrespondingoutputs .Anactivity(behavior) consistsofasetofactionsandrowedges.Theactionsarecon nectedbyobjectrowedges toshowhowobjecttokensrowthroughandconnectedbycontro lrowedgestoindicate theexecutionorder. ... ... Decision/MergeFork Join Activity Final Flow Final Initial Control Nodes: Label Object Node: Action Node: Label Figure2-7.UMLactivitynodes UMLactivitydiagramsadoptthesemanticslikePetri-net[ 72 ].Itisatypeofdirected graphicalrepresentation.Tokenswhichindicatecontrolo rdatavaluesrowalongtheedges fromthesourcenodetothesinknodesdrivenbytheactionsan dconditions.Anactivity 33

PAGE 34

diagramhastwokindsofmodelingelements:activitynodesa ndactivityedges.More specially,therearethreekindsofnodesinactivitydiagra ms: ActionNode :Actionnodesconsumeallinputdata/controltokenswhenth eyare ready,generatenewtokensandsendthemtooutputactivitye dges. ObjectNode :Objectnodesprovideandacceptdatatokens,andmayactas buers,collectingdatatokensastheywaittomovedownstre am. ControlNode :Controlnodesroutetokensthroughthegraph.Thecontroln odes includeconstructstochoosebetweenalternativerows(dec ision/merge),tosplitor mergetherowforconcurrentprocessing(fork/join). Figure 2-7 showsthebasicconstructsofactivitynodes.Anactionnode isdenoted byroundcorneredboxes.Itrepresentsanexecutionofopera tionsoninputtokens,and generatednewtokenswillbedeliveredtoanoutgoingedges. Anobjectnodedenoted usingrectangleboxesisusedtotemporarilyholdthedatato kenswaitingtobeprocessed ordelivered.Forsimplicity,weassumethatobjectnodesdo notsupport competition and datastore fortestcasegeneration.Arowinanactivitystartsfromthe initialnode. Whenatokenarrivesatarownalnode,itwillbedestroyed.T herownalnodehas nooutgoingedges,sothereisnodownstreameect.Whennoto kensexistinanactivity diagram,theactivitywillbeterminated.Theactivitynal nodesaresimilartorownal nodes,exceptthatwhenatokenreachesoneactivitynalnod e,theentirerowwillbe terminated.Decisionnodesandmergenodesusethesameshap eofdiamond.Decision nodeschooseoneoftheoutgoingrowsaccordingtothevalueo fBooleanexpressions labeledontheoutgoingedge.Mergenodesselectonlyoneofi ncomingrowstodeliverto thenextactivitynode.Forksorjoinsareshownbymultiplea rrowsleavingorenteringthe synchronizationbar,respectively,todescribetheconcur rentbehaviorofasystem.When atokenarrivesataforknode,itwillbeduplicatedacrossth eoutgoingedges.Joinnodes synchronizemultiplerows.Thetokensmustbeavailableone veryincomingedgeinorder tobepassedtooutgoingedges. 34

PAGE 35

Activitynodesareconnectedbyactivityedgesalongwhicht okensmayrowunder somecondition.Activityedgesincludecontrolanddatarow edgesasfollows: ControlFlowEdge :Controlrowedgesindicatetheexecutionsequenceofactio ns. ObjectFlowEdge :Objectrowedgesindicatetherelationofdatatoken transmissions.Itprovidestheinputstoactions. Inourmethod,wesimplifythesyntaxandsemanticsofUMLact ivitydiagrams.We combinethecontrolanddatatokentogetherasanewkindofto kenwhichcontainsboth controlanddatainformation.Suchtokencanrowthroughact ivityedges.Inotherwords, wedonotdistinguishcontrolrowedgesandobjectrowedgesi nourframework. [incorrect] [resolved] Ask for amount Prepare to print receipt end start a c syn_2 syn_1 g content t1 t2 t3 t4 t9 t11 t10 t6 [amount available] t7 t8 [amount not available] Generate receipt e d f b t5 [correct] [not resolved] Finish transaction and print receipt Verify access code Dispense cash Handle incorrect access code Figure2-8.TheUMLactivitydiagramofanATM Figure 2-8 showsanexamplewhichusesmostoftheelementsshowninFigu re 2-7 Itdescribesthefunctionalityofwithdrawingmoneyfroman AutomatedTellerMachine 35

PAGE 36

(ATM)[ 26 ].Auserneedstoentertheaccesscoderst.Incaseoffailur e,theusercan inputtheaccesscodeagain.Theoperationwillabortifacce sscodeiswronginboth cases.Iftheinputaccesscodeisright,theusercanenterth eamountofmoneyhewants towithdraw.Atthesametime,theprinterwillbereadytopri ntareceipt.Oncethe ATMdecideswhetherthereisenoughmoneytheusercanwithdr aw,itprovidesthecash andgeneratestheinformationforthistransaction.Finall y,theprinterprintsthereceipt andthetransactioniscomplete. ThetokenforthisexamplecontainstheATMtransactioninfo rmationsuchasthe inputaccesscodeandinputcashamount,thecontextinforma tionsuchastheavailable cashamountandcorrectaccesscode.Ingeneral,atokenrere ctsallthedatainformation requiredforthisactivity.Table 2-1 showsthecompositionofatokenoftheATMactivity diagram.Itconsistsof5variableswhichwillbeusedtomake thedecisionsillustratedin Table 2-2 Table2-1.BreakdownofatokeninFigure 2-8 VariableTypeDescription access code stringuser'saccesscode access code input stringuseraccesscodeinput access code resolve stringuseraccesscodeinputcorrection amount input integerusercashamountinput amount available integercashamountavailable Table2-2.ConditionontherowedgesinFigure 2-8 ActivityEdgeConditionDescription t2incorrect access code != access code input t3correct access code = access code input t4resolved access code = access code resolve t5notresolved access code != access code resolve t7amountavailable amount input< = amount available t8amountnotavailable amount input>amount available 2.2.2FormalModelingofUMLActivityDiagrams Withoutformalism,itishardtodescribeandmodeltheactiv itydiagramsaccurately. UMLactivitydiagramitselfisasemi-formalspecicationt hatcannotbedirectlymapped 36

PAGE 37

toamodelcheckerinput(e.g.,SMVmodels).WeusePetri-net asanintermediateformal modelbetweenactivitydiagramsandSMVmodel,becausetheP etri-netformalismcan capturethemajorfunctionalscenariosaswellasguidethet ranslation. Denition 2 describestherelationbetweentheactivitynodesandrowed geswitha Petri-netsemantics.Itdoesnotmodelthefullfeaturesofa ctivitydiagramsandformally depictsthestaticabstractedstructureofactivitydiagra mswhichcanbeusedtodescribe thescenariosthatneedtobetested. Denition2. Anactivitydiagramisadirectedgraphdescribedusingeigh t-tuple(A,T,F, C,V,A, a I a F )where A = f a 1 ;a 2 ;:::;a m g isasetofactionnodes. T = f t 1 ;t 2 ;:::;t n g isasetofcompletiontransitions. F f A T g[f T A g isasetofrowedgesbetweenactivitynodesandcompletion transitions. C = f c 1 ;c 2 ;:::;c n g isanitesetofguardconditions.Here, c i ( 1 i n )isa predicate(expression)basedontheinputvariables.There isamappingfrom f i 2 F to c i ,referredas Cond ( f i )= c i Let V bethesetofallpossibleassignmentsforinputvariables V 1 ;V 2 ;:::;V k where k isapositiveinteger. M : A V V isamappingthatdescribesthevaluechangeoftheinputvari ables insideanactivitynode. a I 2 A istheinitialnode,and a F 2 A isthenalnode.Thereisonlyonecompletion transition t 2 T and c 2 C suchthat ( a I ;t ) 2 F ,andforany t 0 2 T ( t 0 ;a I ) = 2 F and ( a F ;t 0 ) = 2 F Inourformalization,anodecanbeanactionnode,aninitial nodeoranalnode. Weusethe completiontransition and rowedge tomodelthebehaviorofthecontrol nodes.Inthegraph,thenodesareconnectedbyrowedgesasso ciatedwithacompletion transition.Becauseactivitydiagramsallowtokenstoexis tintherowsconcurrently, 37

PAGE 38

thecompletiontransitioncanbeusedtosynchronizethetok enrows.Ifacompletion transitionhasmultipleincomingrowedges,itwilldothejo inoperation.Ifthereare multipleoutgoingrowedges,thenitwilldotheforkoperati on.Foreachrowedge,there maybeaconditionwhichcanguidethetokentraverse.Thegra phhasoneinitialnode thatindicatesthestartofcontrolanddatarows.Activityd iagramshavetwokindsof nalnodes:rownalnodesandactivitynalnodes.Wecombin ethemtogetherandusea joinoperationtogetanewactivitynalnode.Sointhedeni tionthereisonlyonenal node. Whenanalyzingdynamicbehaviorsofanactivitydiagram,we needtousethe states (asetofactionsexecutingconcurrently)tomodelthestatu sofasystem.Currentstate (denotedby CS )ofanactivitydiagramindicatestheactionswhicharebein gactivated. Denition3. Let D beanactivitydiagram.Thecurrentstate CS of D isasubsetof A Foranytransition t 2 T t denotesthepresetof t ,then t = f a j ( a;t ) 2 F g t denotesthepostsetof t ,then t = f a j ( t;a ) 2 F g enabled ( CS ) denotesthesetofcompletiontransitionsthatareassociat edwiththe outgoingrowedgesof CS ,then enabled ( CS ) = f t j t CS g rable(CS)denotesthesetoftransitionsthatcanberedfr om CS ,thenrable(CS)= f t j t 2 enabled(CS) V t areallcompleted V 9 n 2 A Cond (( t;n )) issatised V ( CS t ) \ t = ;g .Aftersome t isred,thenewcurrentstate CS 0 = fire ( CS;t )= ( CS t ) [ t Thecurrentstateofanactivitydiagramindicateswhichact ivitynodesareholding thetokens.Forexample,when f d;f g isthecurrentstateoftheactivitydiagramin Figure 2-8 ,twotokensareintheactivitynodes d and f individually.Atthistime,only thetransitionassociatedwith t 9 israble.Ifitisred,thenthenextstateis f e;f g Becauseoftheinherentconcurrency,severaltransitionsc anberedatthesametime. Foranactivitydiagram,alltherabletransitionsinastat eforma concurrenttransition 38

PAGE 39

Denition4. Let D beanactivitydiagram.Forastate CS of D ,aconcurrenttransition isasetofcompletiontransitions t 1 ;t 2 ;:::;t n 2 firable ( CS ) where 1. 8 i;j (1 i
PAGE 40

Denition6. Let D beanactivitydiagram.Aninteractionoftheactivitydiagr amisa setofactivitynodes(actions)thatcanbeactivatedsimult aneously.A\k-interaction"isa setthatcontainskactivitynodes. Inordertodetectwhetheraconcurrentstateofanactivityd iagramisreachable orcanbeactivated,weusetheterm interaction 1 todescribethescenariothataset ofactionscanbeactivatedsimultaneously.Forexample,in theFigure 2-8 f d;f g isan exampleof\2-interaction"intheATM.2.2.3TransformationfromUMLActivityDiagramstoSMV Ourtechniquecanextractboththecontrolanddatarowsbypa rsingaUMLactivity diagram.Thetranslationconsistsoftwoparts:staticinfo rmationextractionanddynamic informationextraction.Staticinformationextractionan alyzesthestructureofanactivity diagramandthengeneratesaskeletonoftheSMVinput.Thedy namicinformation extractionanalyzesthedynamicbehaviorofthesystembyfo cusingoncontrolanddata rowanalysis(i.e.thestatechangeofactivities,datamani pulationinactivitiesandthe conditionofthetransitions).2.2.3.1StaticInformationExtraction Thisstepcollectsboththeinputdatamanipulatedbytheact ivitiesandthe predicatesusedasguardconditionsofthetransitions.For exampleinFigure 2-8 ,there areveinputdatavariablesthatdeterminethedataandcont rolrows: access code access code input access code resolve amount input ,and amount available .Becausethere maybeanumberofpossiblevaluesforavariable,duringmode lcheckingitwillcausethe statespaceexplosion.Inourapproach,weadoptthemodelch eckerSMVwhichdoesnot supportcomplexdatatypes(e.g.,roat,doubleandetc.).Fo reachvariable,itisrequired 1 UnliketheinteractioninUMLInteractionoverviewdiagram ,theinteractionhere meansthatseveralactionsareactivedatthesametime. 40

PAGE 41

thatthevaluerangeshouldbespeciedexplicitly.Toavoid statespaceexplosion,weuse thefollowingmethodstoreducethecomplexityofdatatypes : Scaling: Scalingisusedtoproportionallyreducethevaluerangeofa variable. Reduction: Reductionisusedtoreducethecardinalityofpossiblevalu esfora variable. Sinceitishardtoimplementtheabovetechniquesautomatic ally,beforetheSMV translation,thevariabletypeinformationistunedmanual lyforactivitydiagrams. Inourtranslation,weassigneachactivitywitha statevariable whichhasthree possiblestatevalues: unvisited (0), visiting (1)and visited (2). Unvisited indicatesthat notokenhaspassedthroughthisactivitynode. Visiting indicatescurrentlytheactivity isholdingoneormoretokens. visited indicatesthatsometokenhaspassedthrough thisactivitynodeandcurrentlythereisnotokeninthisact ivitynode.Theextraction procedureinstantiatestheactivitystatevariablesandas signssuitablevaluestothem. Duringinitialization,theinitialactivitynodeisassign ed visiting thatmeansthereisa tokenreadyattheinitialstate.Othernodesareinitialize dto unvisited .Also,weassign eachrowedgeastatevariablewhichhastwopossiblevalues: fired (1)and unfired (0). Fired meanssometokenshaverowedfromtheincomingactivitynode stoitsoutgoing activitynodes. Unfired meansnotokenhaspassedthroughthisactivityedge.Initia lly wesetthemwithvalue0. Figure 2-9 showsthegeneratedskeletonofFigure 2-8 inSMVformat[ 20 56 ].There are3modulesinthisskeleton.Themodule state denesthetokeninformation(described inTable 2-1 )aswellasthestatevariableforactivitynodesandrowedge s.Forexample, verify access code isastatevariableforanactionwiththreestates.Initiall yitis assignedthestate unvisited (0).Module ATM givesastaticskeletonwithoutdynamic behaviorinformation.Inthisphase,wejustcollectvariab leswithoutanyprocessing.The missingstatetransitiondetailswillbedescribedinSecti on 2.2.3.2 .Themodule main createsthemoduleinstancesandelaboratesthemtogether. Forexample, st isaninstance 41

PAGE 42

ofstatemoduleand atm isaninstanceofATMmodule.Webindthe st and atm together, because atm willhandlethestatechangesofvariablesin st MODULEstate VAR access_code:{A1,B1,C1};access_code_input:{A1,B1,C1,D1};start:0..2;syn_1:0..2;verify_access_code:0..2;t2_cond:0..1;t3_cond:0..1;...... ASSIGN init(start):=1;init(syn_1):=0;init(verify_access_code):=0;...... MODULEATM(st) ASSIGN next(st.start):=next(st.t2_cond):=......next(st.prepare_print_receipt):=......next(st.dispense_cash):=next(st.t7_cond):=...... MODULEmain(){ st:state;atm:ATM(st);p_print:prepare_print(st);check:check_amount(st); } Figure2-9.Thegeneratedskeletonafterstructureextract ion 2.2.3.2DynamicInformationExtraction Afterstaticinformationextraction,weneedtoextractbot hdatamanipulationsand transitionsofstatevariables,becausetheywilldetermin ethedataandcontrolrows. Inourmethod,wedeneasetofrulesthatspecifythestatetr ansitionforeach activitynodeandthevaluechangesofeachdata.Figure 2-10 showsthedetailsof therules.Intheserules,weusethepresetandpostsetnotat ions.Intheserules,the assignmentandconstrainttoasetmeanstheassignmentandc onstrainttoeachelement 42

PAGE 43

Rule1 :Ifnisaninitialnode init ( n ):=1; next ( n ):=2; Rule2 :Ifnisanalnode,andtherearekincomingtransitions t 1 ;t 2 :::t k : init ( n ):=0; next ( n ):= case (( t 1 =1& cond ( t 1 )) j ( t 2 =1& cond ( t 2 )) j ::: j ( t k =1& cond ( t k ))):2; 1: n ; esac ; Rule3 :Ifnisanactivitynode(notjoinorfork),andtherearekinc oming transitions t 1 ;t 2 :::t k init ( n ):=0; next ( n ):= case n =1:2; ( t 1 =1& cond ( t 1 )) j ( t 2 =1& cond ( t 2 )) j ::: j ( t k =1& cond ( t k ))):1; 1: n ; esac ; Rule4 :Ifnisaforknode,andthecorrespondingtransitionis t init ( n ):=0; next ( n ):= case n =1& t > 0:2; t =1:1; 1: n ; esac ; Rule5 :Ifnisajoinnodeoftransition t ,and a 1 ;a 2 :::a k arekelementsof t init ( n ):=0; next ( n ):= case n =1:2; n =0&( a 1 + a 2 + ::: + a k =2 k ):1; n =2&( a 1 + a 2 + ::: + a k < 2 k ):0; 1: n ; esac ; Rule6 :If t isatransitionwhichcorrespondstotherowedges. init ( t ):=0; next ( t ):= case cond ( t )& t =1:0; cond ( t )& t =1:1; 1: t ; esac ;DynamicInformation Rule7 :If v isavariablewhosenewvalueischangedbyexpression exp i in theactivity act i (1 i n ). next ( v ):= case act 1 =1: exp 1 ; act 2 =1: exp 2 ; ::::::act n =1: exp n ; 1: v ; esac ; Figure2-10.Translationrulesforstateanddatatransitio ns 43

PAGE 44

intheset.Forexample,if t = f a 1 ;a 2 ;:::a k g ,then t =1means a 1 =1& a 2 = 1& ::: & a k =1and cond ( t )means cond (( a 1 ;t ))& cond (( a 2 ;t ))& ::: & cond (( a k ;t )). Rule1speciesthetranslationrulefortheinitialnode.Th etokenwillberstput attheinitialstateandthenodeismarkedas visited inthenextstep.Rule2species thetranslationruleforthenalnode.Atrst,thestateis unvisited ,whenoneofthe incomingedgesisactivated,itsstatewillbecome visited .Rule3denesthestatechanges ofanactivity.Initially,thestateofanactivityis unvisited .Iftheincomingedgeis activated,thestatewillbecome visiting inthenextstep.Ifthecurrentstateis visiting thestatewillchangeto visited inthenextstep.Rule4presentsthestatetransitionof theforknodes.Whentheincomingedgeisactivated,thefork nodewillmaintainthe visiting statusuntilalltheoutgoingedgesarevisitingorvisited. Rule5givesthestate transitionofjoinnodes.Thejoinnodeisusedtosynchroniz ethetokenrows.Whenall theincomingrowsareready,thetransitioncorrespondingt othejoinnodecanbered.In thisrule,ifwewanttorethetransition,weneedtowaitunt ilalltheactivitynodesin thepresetofthetransitionarevisited.Rule6showshowtom anipulatethestatechange ofthetransitionwhenitisred.Rule7presentsthetransla tionforvaluechangeofthe variables.Ifanactivityperformssomeoperationonthevar iable,wecanmodifythevalue ofthevariableonlywhentheactivitystateis visiting 2.2.4APrototypeToolForUMLtoSMVTranslation BasedontheframeworkproposedinSection 2.2.3 ,wedevelopedaprototypetool whichcanautomatetheprocessoftestcasegeneration.Thet ooltakesthreeinputs:type denitionofthedatawhichisusedintheactivitydiagram,t hecontextinformationwhich settheparametersfortheexecutionofanactivitydiagram( e.g.whentotriggertheinitial nodeandsoon),andUMLactivitydiagrams.TheUMLactivityd iagramsarestoredin theformatofXMLMetadataInterchange(XMI)les.Thetoolc anparsetheXMIlesto getthestaticanddynamicinformationforformalmodeltran slation.Combinedwiththe 44

PAGE 45

contextinformationanddatatypeinformation,aformalmod elcanbegeneratedusingthe proposedmappingrules. 2.3CaseStudy Thissectionpresentsverepresentativehigh-levelspeci cationsforSoCdesigns. First,itdescribesthreeTLMspecications:router,MIPSp rocessorandAlphaAXP processor.Next,itpresentstwoUMLactivitydiagrams:aco ntrolsystemandaonline stockexchangesystem(OSES).2.3.1Example1:ARouter Figure 2-11 showstheTLMstructureofarouterdesign.Therouterconsis tsofve modules:onemaster,onerouterandthreeslaves.TheSystem Cprogramconsistsof4 classes(oneclassforpacketdenition,oneclassforthedr iver,oneclassfortherouterand oneclassfortheslave),8functions,and143linesofcode.T hemainfunctionoftherouter istoanalyzeanddistributethepacketsreceivedfromthema stertotargetslaves. get_data get_data get_data Master put_data FIFO FIFO FIFO FIFO Slave 0 Slave 2 route Router Slave 1 Figure2-11.TheTLMstructureoftherouter Atthebeginningofatransaction,themastermodulecreates apacket.Then,the driversendsthepackettotherouterforpackagedistributi on.Therouterhasoneinput portandthreeoutputports.EachportisconnectedtoaFIFOb uer(channel)which temporarilystorespackets.Therouterhasoneprocess route whichisimplementedasa SC METHOD .The route rstcollectsapacketfromthechannelconnectedtothedriv er, decodestheheaderofthepackettogetthetargetaddressofa slave,andthensendsthe 45

PAGE 46

packettothechannelconnectedtothetargetslave.Finally ,theslavemodulesreadthe packetswhendataisavailableintherespectiveFIFOs.Thet ransactiondata(i.e.packet) rowsfromthemastertoitstargetslaveviatherouter.Thero wiscontrolledbythe address to chan inthepacketheader.ByusingourproposedapproachinSecti on 2.1.2 theautomaticallygeneratedSMVmodelcontainsfourmodule sand145linesofcode. Instruction Flow Data Transfer Fetch FADD2FADD3FADD4 WriteBack MEM MUL2 MUL1FADD1 Decode IALU MUL7 DIV RegFile Memory Figure2-12.GraphmodelofaVLIWMIPSprocessor 2.3.2Example2:AMIPSProcessor Figure 2-12 showsasimpliedversionofasingle-issueMIPS[ 35 ]architecture.It hasvepipelinestages:fetch,decode,execute,memory(ME M),andwriteback.The execute stagehasfourparallelexecutionpaths:integerALU,7stag emultiplier(MUL1 -MUL7),fourstageroating-pointadder(FADD1-FADD4),and multi-cycledivider (DIV).Theovalboxesrepresentunitsanddashedboxesrepre sentstorages.Thesolidlines 46

PAGE 47

representinstruction-transferpathsanddottedlinesrep resentdata-transferpaths.After TLM-to-SMVtransformation,theSMVmodelhas1134linesofc ode. 2.3.3Example3:AnAlphaProcessor Figure 2-13 showsasimpliedTLMspecicationstructureoftheAlphaAX P processor.Itconsistsofvestages:Fetch(IF),Decode(ID ),Execute(EX),Memory (MEM)andWriteback(WB).IFmodulefetchesinstructionsfr omtheinstructionmemory. IDmoduledecodesinstructionsandfetchestheoperanddata ifnecessary.EXmodule doesALUoperationsaswellasassertswhetherthecondition alorunconditionalbranch happens.Memorymodulereadsandwritesdatafrom(to)theda tamemory.Writeback modulestorestheresultinspeciedregisters.Thecommuni cationbetweentwomodules usestheportbindingassociatedwithablockingFIFOchanne lwithonlyoneslot.For example,thereisabindingfromthe port ofIFmoduletothe export ofIDmodule,and theexportofIDmodulebindstoablocking FIFOchannel forholdinganincoming instruction.Soeachtime,theIFmodulecanonlyissueonein structiontoIDmodule; otherwiseitwillbeblocked.ThewholeTLMdesigncontains6 classes,11functionsand 797linesofcode.AftertheTLM-to-SMVtransformation,the generatedSMVmodelhas6 modulesand821linesofcode. EXMEMWB IF BranchDataMem ID RegFile Figure2-13.TLMoftheAlphaAXPprocessor 47

PAGE 48

2.3.4Example4:AControlSystem AsshowninFigure 2-14 ,theUMLactivitydiagramrepresentationofthecontrol systemconsistsof17activities,23transitionsand6keypa ths.Ithasaglobalinteger variable i whichdeterminestokenrows.ThegeneratedSMVleshave365 linesofcode. a c kl m o n g:i=i+5 p:i=i-10 d b h e f i j q i>=50 i<50 i<20 i>=20 i<80 i>=80 i<80 i<60 i>=60 i>=10 i>=80 i<10 Figure2-14.Theactivitydiagramforacontrolsystem 2.3.5Example5:AStockExchangeSystem Thepurposeoftheon-linestockexchangesystem(OSES)isto processthree scenarios:accept,checkandexecutethecustomer'sorders (marketorderandlimit order).ThesystemusestheUMLactivitydiagramasitsbehav iorspecication.Figure 2-15 showsthespecicationofthestocksystem.Ithas27activit ies,29transitionsand18 keypaths.ThegeneratedSMVmodelhas756linesofcode. 48

PAGE 49

StockBroker VerigyOrderForm DisplayOrderErrorInfo getNewOrder trade_FAILURE tradeMarkderOrderSaletradeMarketOrderBuy tradeLimitOrderSaletradeLimitOrderBuy getOrderResult updateOrderDB_FAILURE updateOrderDB_SUCESS updateStockDB_SUCESS t10 t8 t9 t0 t2 t1 t4 t5 t6 t23 t29 t15 t16 t11 t12 t18 t14 t13 t17 t7 checkLimitOrderPrice t3 t22 trade_NOMATCHupdateOrderDB_NOMATCH t27 t26 t28 t24 t21 t20 t19 t25 trade_PARTEXE updateStrockHolderDB_SUCESS addOrderFormList endOrderProcess updateOrderHashMap updateStockHolderDB_PARTEXE updateStrockDB_PARTEXE updateOrderDB_PARTEXE trade_SUCESS settleTrade Figure2-15.Theactivitydiagramforastockexchangesyste m 2.4Summary Thischapterintroducedtwohighleveldesignspecication sforSoCdesigns: SystemCTLMtomodelthehardwarebahaviorandUMLactivityd iagramstodescribe theconcurrentsoftwarebehavior.Themaincontributionof thischapteristodevise mechanismstoextractbothstaticanddynamicinformationf romspecicationsand thenconvertthemtoformalSMVmodelstoenableautomatican alysisanddirectedtest generation. 49

PAGE 50

CHAPTER3 COVERAGE-DRIVENAUTOMATICGENERATIONOFDIRECTEDTESTS Figure 3-1 presentsourmethodologyforspecicationdriventestgene rationusing modelcheckingtechniques.First,adesignisdescribedusi ngaspecicationlanguagethat cancapturebothstructureandbehaviorofSoCsystems.Next ,thedesignspecicationis translatedtoaformalmodel(describedinChapter 2 ),andthepropertiesintheformof CTLorLTLformulasaregeneratedbasedonthefunctional faultmodels (seeSection 3.1 ). Finally,thepropertiesareappliedontheformalmodelusin gamodelcheckertogenerate requiredtests(counterexamples).Themodelcheckerexhau stivelysearchesallreachable statesofthemodeltocheckifanystateviolatesthepropert y.Ifitndsaviolation,itwill produceacounterexample.Thecounterexamplecontainsase quenceofinputassignments fromaninitialstatetoastatewherethespeciedpropertyf ails.Ifweassumethat thedesigniscorrectandthepropertyisafalseproperty,th emodelcheckerwillalways generateavalidcounterexampleunlessitencountersstate spaceexplosionproblem.The generatedtestscanbeusedforvalidatingbothspecicatio nsandimplementations. Fault Models Test cases Formal Model Properties Model Checker Counterexamples Validation Validation Specification Implementation SoC Specification (SMV) Figure3-1.Testgenerationusingmodelchecking Therearethreemajorchallengesinimplementingthistestg enerationmethodologyin practice:i)automaticextractionofformalmodelsfromSoC specications,ii)development 50

PAGE 51

ofecientfunctionalfaultmodelsandassociatedcoverage -drivenpropertygeneration,and iii)howtoaddressstatespaceexplosionproblem.Wehavedi scussedtheformalmodel generationinChapter 2 .Chapter 4 to 6 willpresentnovelapproachesforaddressing statespaceexplosionproblem.Inthefollowingsections,w ewillfocusontheautomatic generationofpropertiesandcorrespondingdirectedtests .Therestofthischapteris organizedasfollows.Section 3.1 presentsthepropertygenerationusingvariousfault models.Section 3.2 describesthetestgenerationmethodsusingbothunbounded model checkingandboundedmodelchecking.Section 3.3 demonstratestwocasestudiesbasedon UMLactivitydiagrams.Finally,Section 3.4 summarizesthechapter. 3.1Coverage-DrivenPropertyGeneration Formodelcheckingbasedtesting,atestisderivedfromthec ounterexampleof afalsesafetyproperty.Asafetypropertyinthetemporallo gicform F(p) asserts thataspeciedscenariocannothappen(i.e.,property p cannotbetrue).Otherwise, acounterexamplewhichexplainsthereasonoftheerrorwill bereportedbyamodel checker.Inotherwords,suchcounterexamplecanthenbeuse dasatesttovalidatethe speciedscenario.Inourmethod,thequalityofthegenerat edtestsisdeterminedbythe correspondingproperties.Duringthepropertygeneration ,itisrequiredtoguaranteethat thegeneratedpropertiescansucientlyvalidatethesyste m. Thecoveragemetrics[ 32 ]playanimportantroleintestingtoindicatethetesting adequacy.Testgenerationusingmodelcheckingtechniques requiresthattheautomatically generatedpropertiescancoverasmanydesiredscenariosin thedesignaspossible.Inour approach,propertiesarederivedfroma faultmodel whichrepresentsacompletesetof specicerrors.Each fault inthefaultmodelindicatesapotential\designerror"whic h canbedescribedbyatemporallogicproperty.Thetestgener atedfromsuchaproperty canbeappliedonthedesigntocheckthespecicscenario(ne gationofthefault).For example,whenvalidatingadesiredscenariodescribedbyaL TLformula p ,weusethe negation p asafault.Bycheckingtheproperty F ( p ),wecanderiveatesttocheck 51

PAGE 52

thescenariowhereproperty p holds.Sinceinthisdissertationwefocusononlysafety propertygenerationforabovefaultmodels,majorityofthe propertieswillbeintheform of F(p)orG( p).However,otherformsofsafetypropertiesarealsopossi bleand allowedinourframework.3.1.1FaultModels Faultmodel[ 28 ]playsanimportantroleindirectedtestgeneration.Eachf aultmodel representsakindof\falsefunctionalscenarios".Theeci encyofdirectedtestsisdirectly relatedtothegeneratedpropertieswhichinturnarerelate dtotheassociatedfaultmodel. Thefollowingthreesubsectionspresentthegenericfaultm odelsforgraphmodelaswell asitstwovariants:faultmodelsforSystemCTLMdesignsand faultmodelsforUML activitydiagrams.Itisimportanttonotethatthesefaultm odelsarebynomeansthe \golden"modelratheritisarepresentativemodelwhichcan berenedormodiedfor improvedvericationmethodology.3.1.1.1GenericFaultModelsforGraphBasedModels Forasimplegraphmodel,thereisonlynodeandedgeinformat ion.Byinvestigating thestatusofthenodesandedgeswecaninfervarioussystemb ehaviors.Therearefour widelyusedfaultmodelsforgraphmodelsasfollows. NodeFault: Eachnodeisfaulty.Forexample,anodecannotbeactivated. EdgeFault: Eachedgeisfaulty.Forexample,therespectivenodescanno tbe activatedinthatorder. PathFault: Eachexecutionpathisfaulty.Forexample,theassociatedn odesand edgesareeitherfaultyortheirbehaviorcannotbecomposed correctlytoactivatethe path. InteractionFault: Eachinteractionisfaulty.Forexample,aninteractioninv olving asetofnodescannotbeactivatedsimultaneously. Wegenerateonepropertyforeachfaultinafaultmodel.Soth etransformationfrom thefaultmodeltothepropertiesintheformoftemporallogi cisaone-to-onemapping. 52

PAGE 53

Becauseafaultisalreadyanegationofthesystemrequiredb ehavior,itcanbedirectly usedtoderiveapropertyfortestgeneration. Let'sconsiderFigure 2-12 inChapter 2 asanexampleofagraphmodel.The followingexampleshowsfourproperties(oneforeachfault type)forthegraphmodel. Property1:ThenodeFetchcannotbeactivated. LTLformula:~F(fetch_active=1) Property2:TheedgebetweennodeMUL4andMUL5cannotbeacti vated. LTLformula:~F(mul4_active=1->X(mul5_active=1)) Property3:ThepathofFADDcannotbeactivated. LTLformula:~F(fetch_active=1&decode_active=1&fadd1_ active=1 &fadd2_active=1&fadd3_active=1&fadd4_active=1&mem_a ctive=1 &writeback_active=1) Property4:DIV,FADD4andMUL7cannotbeactivatedatthesam etime. LTLformula:~F(div_active=1&fadd4_active=1&mul7_acti ve=1) Dependingonthedesign,thegeneratedpropertiesmayleadt oredundanttests. Therefore,propertycompactioncanbeemployedtoreduceth enumberofproperties withoutaectingthecoveragegoal[ 45 ]. 3.1.1.2FaultModelsforSystemCTLMSpecications InTLM,transactiondata,transactionrowandeventsarethr eemostimportant factors.Theyrerectboththestructureandbehaviorinform ationofsystemlevelhardware designs.InadditiontothefaultmodelspresentedinSectio n 3.1.1.1 ,inourframeworkwe havedenedanotherthreefaultmodelsbasedontransaction sasfollows. Transactiondatafaultmodel investigatesthecontentofthevariablesrelevantto thetransaction.Foreachvariable,itisassumedthataspec icvaluecan/cannotbe assignedinsomescenario. Transactionrowfaultmodel investigatesthecontrolsalongthepathwherethe transactionrows.Foreachbranchconditionalongthetrans actionpath,itisassumed thatitcan/cannotbeactivatedinsomescenario. Transactioneventfaultmodel investigatestheeventoccurrencewithina transaction.Foreachevent,itisassumedthatitcan/canno tbeactivated. 53

PAGE 54

Transactiondatafaultmodeldealswiththepossiblevaluea ssignmentforeachpart ofthetransactiondata.However,duringpropertygenerati on,duetothelargesizeof valuespace,tryingallpossiblevaluesofadataistime-con sumingandimpractical.By checkingeachbitofavariable(databitfault)separately, thedatacontentcoveragecan bepartiallyguaranteed.Transactionrowfaultmodeldeals withthecontrolsalongwith thetransactionrow.Toensuretransactionrowcoverage,on ecancoverbranchconditions whichexistin if-then-else and switch-case statements.Thegoalistocheckallpossible transactionrows.Transactioneventindicatestheexecuti onstageofatransactionorthe interactionbetweenprocesses.Theactivationandtheorde roftransactioneventsisan importantissue.Section 7.2.1.1 givesanexampleforeachtypeofTLMtransactionfaults. 3.1.1.3FaultModelsforUMLActivityDiagrams Intraditionalsoftwaretesting,thedenitionoftestinga dequacyisgivenin[ 32 ] asameasurementfunction.ThecaseofUMLactivitydiagrams isdierentbecauseit isintheformofmodelinsteadofcode.Especiallythecovera geofactivitydiagramis morecomplexbecauseoftheconcurrency.Wecreatefourfaul tmodelsforUMLactivity diagrams(AD)whicharesimilartothegenericfaultmodelsp resentedinSection 3.1.1.1 asfollows. ActivityFaultModel .Foreachactivityof AD ,themodelassumesthatsuch activityisnotreachable. TransitionFaultModel .Foreachtransitionof AD ,themodelassumesthatsuch transitioncannotbered. KeyPathFaultModel .Foreachkeypathof AD ,thereisnocorresponding executable path InteractionFaultModel .Foreachinteractionof AD ,theactivitiesassociatedwith theinteractioncannotbeactivatedatthesametime. Fromthesefourdierentmodels,wecangeneratevariouspro pertiestovalidate activitydiagrams.Theactivityfaultmodelcanbeusedtoch eckthereachabilityofeach activity.Soitcanbeusedtocheckwhetherthereexistsinn iteloopsinthesystem.The 54

PAGE 55

transitionfaultmodelcanbeusedtochecktheexecutionord eroftheactivities.Itcan alsobeusedtocheckwhethertheconditionguardofthetrans itioncanbesatised.We alsoneedtocheckallthedynamicbehaviorsofthesystem,so keypathfaultmodelis preferableinthiscase.Theinteractionfaultmodelcanbeu sedtocheckwhetherseveral activitiescanbeactivatedsimultaneously.Ingeneral,if alltheinteractionshaveonlyone activity,theinteractionfaultmodelisthesameastheacti vityfaultmodel. Thefollowingexampleshowsfourproperties(oneforeachfa ulttype)fortheUML activitydiagramshowninFigure 2-8 Property1:Theactivitydispense_cashisnotreachable. LTLformula:~F(st.dispense_cash=2) Property2:Thetransitionwithcondition[amountavailabl e]cannotbefired. LTLformula:~F(st.t7_cond=1) Property3:Thekeypath4cannotbecovered. LTLformula:~F(st.start=2&st.verify_access_code=2&st .handle_access_code=2 &st.ask_for_amount=2&st.prepare_print_receipt=2&st. dispense_cash=2 &st.generate_receipt_content=2&st.finish_transactio n_print_receipt=2 &st.end=2&st.t2_cond=1&st.t4_cond=1&st.t7_cond=1) Property4:Theactivitiesdispense_cashandprepare_to_p rint_receiptcannot beactivatedsimultaneously. LTLformula:~F(st.dispense_cash=1&st.prepare_to_prin t_receipt=1) Figure3-2.Faultmodelexamples 3.1.2FunctionalCoverageBasedonFaultModels The functionalcoverage ofasystemleveldesignisdenedbasedontheoverallfaults ofafaultmodelandthefaultsactivatedbythederivedtests Denition7. Foradesign D ,wearegivenitsfaultmodel F andatestsuite T F isa completesetofsametypefaults.Eachfaultindicatesthene gationofarequiredfunctional behaviorof D T isasetofdirectedtestswhichisderivedfrom F .Byapplying T on D thefunctionalcoverage D F using T canbecalculatedas: D F = # ofexercisedFtypefunctionalscenarios j F j 55

PAGE 56

3.2TestGenerationusingModelCheckingTechniques Modelchecking[ 21 56 ]isaformalmethodthatcanenumerateallthepossiblestate tocheckwhetheranitestatesystem M satisesaproperty p intheformoftemporal logic(e.g.LTLorCTL[ 21 ]),i.e., M j = p .Whenthepropertyfailsatsomestate,itwill reportacounterexampletofalsifythespeciedproperty p .Let'sconsideratestgeneration exampleforapipelinedprocessor.Toactivateafaultinthe stallfunctionalityofadecode unit(i.e.,thedecodeunitcanneverbestalled),thesystem willgeneratetheproperty \ F(dec stall=1)" .Takingthepropertyandtheprocessormodelasinputs,them odel checkerwillgenerateacounterexampletostallthedecodeu nitwhichcanbeusedasa testtoactivatethestallfunctionalityofthedecodeunit. Thecounterexamplecontainsa sequenceofinstructionsfromaninitialstatetoastatewhe rethepropertyfails.Inthis section,webrieryintroducetwokindsoftestgenerationme thodsbasedondierentmodel checkingtechniques.3.2.1TestGenerationusingUnboundedModelChecking Thissectionintroducesthepreliminaryknowledgeoftheun boundedmodelchecking andgivesageneralalgorithmfortestgeneration.3.2.1.1UnboundedModelChecking SymbolicModelVerier(SMV[ 21 ])isawidelyusedmodelchecker.Bytakingmodel ofthedesignandtemporallogicpropertiesasinputs,SMVca ndeterminewhetherthe designsatisestheproperty.Duringtheverication,SMVa bstractsthegivenmodelinto aformalKripkestructurewhichconsistsofasetofstates,a setoftransitionsbetween states,andafunctionthatlabelseachstatewithasetofpro pertiesthataretruein thisstate.ThenSMVdoesthestatespacesearchonthisKripk estructure.Themodel checkingalgorithmstopsbecause:i)itencountersafalses tatefortheproperty,then thecounterexamplewhichleadstothisstatewillbegenerat ed,orii)allthestateshave beenexploredandnoerrorisdetected.Generally,theimple mentationofthestatesearch 56

PAGE 57

adoptsthedatastructurebasedonBDDs.However,theyareno tscalabletohandlelarge practicalsystemsinpractice.3.2.1.2TestGenerationAlgorithm Algorithm 1 outlinesthegeneraltestgenerationapproachusingunboun dedmodel checking(UBMC)[ 47 59 60 ].ThealgorithmtakesaSMVmodel M andasetoffalse properties P (basedoncoverage)asinputsandgeneratesatestsuiteextr actedfrom counterexamples.Foreachproperty P i ,onetestisgenerated.Thealgorithmiterates untilallthepropertiesarechecked.Foreachiteration,on epropertyishandledandthe correspondingtestwillbegenerated.Inthisdissertation ,wefocusonthegenerationof safetypropertieswhichassertthatthespeciedscenarios cannothappen. Algorithm1 :TestGenerationusingUBMC Input :i)SMVModel, M ,andii)Asetoffalseproperties P Output :Testsuite TestSuite= ; for eachproperty P i intheset P do test i =ModelChecking( P i M ); TestSuite=TestSuite [ test i ; endreturn TestSuite; 3.2.2TestGenerationusingBoundedModelChecking ThissectionintroducesthepreliminaryknowledgeoftheSA T-basedBoundedModel Checking(BMC).Italsodescribeshowtopre-determinetheb oundsofproperties.Finally aBMCbasedtestgenerationalgorithmispresented.3.2.2.1SAT-BasedBoundedModelChecking Forcomplexdesignsandproperties,BDDsbasedmethodsusua llycausethestate spaceexplosionproblem.Asanalternative,Booleansatis ability(SAT)basedapproaches haveemerged,especiallyfortheboundedmodelchecking(BM C).SAT-basedBMC [ 11 ]isapromisingmethodwhichcanprovewhetherthereisacoun terexampleforthe 57

PAGE 58

propertywithinagivenbound.Givenamodel M ,asafetyproperty p ,andabound k ,SAT-basedBMCwillunfoldthemodelktimesandencodeitusi ngthetheBoolean formulaEquation( 3{1 ). BMC ( M;p;k )= I ( s 0 ) ^ k 1 ^ i =0 T ( s i ;s i +1 ) ^ k i =0 : p ( s i )(3{1) Here, I ( s 0 )meanstheinitialstateofthesystem, T ( s i ;s i +1 )describesthestate transitionfromstate s i tostate s i +1 ,and p ( s i )testswhetherproperty p holdsonstate s i ThenthisformulawillbetransformedtoaConjunctiveNorma lForm(CNF)andchecked byaSATsolver.Ifthereisasatisableassignment,theprop ertyisfalseandasatisable variableassignmentwillbereported,i.e., M j = = k p .Otherwise,itimpliesthattheproperty istruewithinthespeciedtimesteps.Inotherwords,there isnocounterexamplewith lengthkforthisproperty,written M j = k p .TestgenerationusingBMCissimilarto modelcheckingbasedapproachexceptthatitneedstodeterm inetheboundforeach property.SAT-basedBMCtakesmodel M ,negatedproperty p i ,and bound i asinputsand generatesacounterexample(test).3.2.2.2TestGenerationAlgorithm Algorithm 2 describesthewidelyusedtestgenerationprocedureusingB MC[ 46 62 ]. ThisalgorithmtakesthemodelMgeneratedfromadesignmode landpropertiesasinputs andgeneratestestsuiteextractedfromthecounterexample s.Foreachproperty P i ,one testisgenerated.Thealgorithmiteratesuntilalltheprop ertiesarecovered.Ineach iteration,thebound k i ofeachproperty P i isdecided.SAT-basedBMCtakesmodel M negatedproperty P i ,andbound k i asinputsandgeneratesacounterexample(test). Duringthetestcasegeneration,bounddeterminationplays animportantrole.Ifit canbeknownapriori,SAT-basedBMCcanbemoreeectivethan BDDbasedmodel checkingtechniques.However,anyincorrectbounddetermi nationwillincreasetestcase generationtimeaswellasmemoryrequirement.Therefore,t hetechniquesofdeciding propertyboundsdeterminetheeciencyoftestcasegenerat ionusingSAT-basedBMC.In 58

PAGE 59

ourmethod,becausethepropertyisderivedfrommodels,the boundcanbederivedfrom thestructureofthemodels. Algorithm2 :TestGenerationusingBMC Input :i)DesignModel, M andii)Asetoffalseproperties P (basedonfaultmodels) Output :Testsuite TestSuite= ; for eachproperty P i intheset P do bound i =DetermineBound( M P i ); test i =BoundedModelChecking( P i M bound i ); TestSuite=TestSuite [ test i ; endreturn TestSuite; 3.2.2.3DeterminationofBound Biereetal.[ 10 ]describedseveralwaystodeterminethebound.If M j = = k p forall k withinthebound,then M j = =p .Howeverthereisnodeterministicwayofcomputingthe boundofthesystem.Infact,determiningtheminimalboundf orapropertyisashardas themodelcheckingitself.Soboundedmodelcheckingisprom isingonlywhenthebound canbepre-determinedandisshallow. Accordingtothedenitionofthediameterin[ 10 ],theboundforeachnodeerror instanceisdecidedbythetemporaldistancebetweentheroo tnodeandthenodeunder verication.Forexample,inUMLactivitydiagrams,thebou ndforthekeypatherroris determinedbytheactivitiesandtransitionsalongthepath .InFigure 2-8 ,thelengthof thekeypath 4 = f start g f t 1 g !f a g f t 2 g !f b g f t 4 g !f c g f t 6 g !f dummy;f g f t 7 g !f d;f g f t 9 g f e;f g f t 10 g !f g g f t 11 g !f end g is9.Thepropertyderivedforthiskeypathisshowninthe Figure 3-2 .Inourtranslationrules,anactivitystatetransitionnee dsonestepdelay.Fork nodeneedsonestepdelay,andjoinnodeneedstwostepsdelay .Onestepdelayatthe startnodeisalsorequired.Theboundsizewillbe9+1+2+1=1 3.Theboundofthe activityerrorortransitionerrorisdeterminedbythedela yofactivitiesandtransitions 59

PAGE 60

onavalidshortestpathfromthe start nodetotheactivityortransitionwhichneedtobe veriedintheUMLactivitydiagram.Forexample,whenwewan ttochecktheactivity errormodelinstance\ prepare to print receipt cannotbeactivated",thesystemwill generatetheproperty F ( st:prepare print receipt =2).Theshortestpathfromstart tosuchanactivityis = f start g f t 1 g !f a g f t 3 g !f c g f t 6 g !f dummy;f g .Inasimilarway, theboundforthispropertyis4+1+1=6.Sometimesinthesyst em,thereisacounterthat actslikeaclockwhichcountstheexecutionsteps.Suchvari ableinapropertywillaect theboundoftheproperty.Forexample,becauseoftheintrod uctionofacounter,the property F ( clk =10& st:prepare print receipt =2)hasaboundof10insteadof6. Dierentpropertiesbasedondierentfaultmodelshavedi erentmethodstocompute thebounds.Assumethatthereisnocountervariables,thede terminationoftheboundof agraphbasedmodelcanusethefollowingrules: Nodeoredgebasedfaults .Extractallthepathswithoutloopsfromtheinitial nodetothetargetnodeoredge.Calculatetheboundforeache xtractedpathand choosetheshortestoneasthepropertybound. Pathbasedfaults .Calculatetheboundsforthepathbasedonthedelayofnodes andedgesonthepath. Interactionfaults .Calculatetheboundforeachelement(nodeoredge)inthe interaction.Choosethelargestboundasthepropertybound Ifapropertycontainsacountervariable.Thenboundofthep ropertyisthelarger oneofthecountervalueandtheboundcalculatedusingtheab overules.Therefore,the complexityofbounddeterminationispolynomialtothenode sinthegraph-basedmodels. Ingeneral,itismoreecienttouseBMCforshallowcountere xampleswhenthebound canbepre-determined. 3.3CaseStudies Inthissection,wedemonstratetwocasestudiesforUMLacti vitydiagrams:acontrol systemandaon-linestockexchangesystem.Wedonotgivethe casestudyforTLM 60

PAGE 61

designssinceSection 7.3 willgivethedetailsofautomateddirectedtestgeneration for TLMdesigns.Wecomparedourmodelcheckingbasedapproachw iththerandomtest basedmethod[ 51 ],whichisthebestknownresultinthecategoryoftestgener ationfor UMLactivitydiagrams.Theexperimentalresultsindicatet hatourmethodcandrastically reducetheoverallvalidationeortbyproducingfewertest s.Furthermore,forUML activitydiagrams,thegeneratedhigh-leveltestcanbedir ectlyappliedonthelow-level implementations(e.g.Javacode).Thereforeitcanbeusedt ochecktheconsistency betweenUMLactivitydiagramsanditslow-levelimplementa tions.WeusedCadenceSMV modelchecker[ 56 ]inourstudy.Alltheexperimentswereconductedusing2.0G HzIntel Core2DuoCPUwith1GBRAM.3.3.1AControlSystem Therstcasestudyisasmallcontrolsystem.Thiscasestudy isbasedonthe examplepresentedinSection 2.3.4 Table3-1.Comparisonoftwomethods Coverage(%)Time Methodactivitytransitionpath(second) random309085501.33random509593672.35random100100100835.13random1501001001008.83OurApproach(UMC)1001001000.91 Table 3-1 showsthecomparisonbetweenourapproachandtherandomtes tbased method[ 51 ].Forgeneratingtestswithhighestcoverage,therandomme thodrequires8.83 secondstorun150randomtests,howeverourapproachusingu nboundedmodelchecking method(UMC)justneeds0.91seconds.Inthiscasestudy,UMC approachimprovesthe testgenerationtimebyanorderofmagnitude. Table3-2.Implementationlevelcoverageofthecontrolsys tem PackageClassMethodBlockLine 100%100%90%88%93% 61

PAGE 62

WeappliedthegeneratedteststotheJavaimplementationof thecontrolsystem. Table 3-2 showsthecoverageoftheJavacode.Thegeneratedtestsobta ined100% package aswellas class coverage.However,the method block and line coverageare around90%.OuranalysisshowedthattheJavaimplementatio nhavemany\try" and\catch"blockstohandleexceptionswhereasthespecic ationdoesnothaveany informationontheexceptionscenarios.Asaresult,thegen eratedtestsdidnotactivate anyoftheexceptionblockswhichresultedinlowcoverageof methods,blocksaswellas lines.Clearly,thisisanissueofincompletespecication .Basedonthisobservation,we addedexceptioninformationatthespecicationlevelandg eneratedtestswhichledtothe requiredcoverageinallthecategoriesoftheimplementati on. 3.3.2AStockExchangeSystem(OSES) Thestockexchangesystemisbasedontheexamplepresentedi nSection 2.3.5 .Ituses theUMLactivitydiagramasitsbehaviorspecication.Thes ystemisimplementedin JAVAandconsistsof7packages,39classes,372methodsand2 510lines. Table3-3.Comparisonofthreemethods Coverage(%)Time Methodactivitytransitionpath(minute) random80096838919.06random100096869424.26random150010010010030.25 OurApproach(UMC)1001001003.47 OurApproach(BMC)1001001000.15 InTable 3-3 ,therstthreerowsdepicttheresultsbyusing800,1000,15 00random testsrespectively.Theresultbyourmethodisshowninthel asttworows.Inthecaseof random 800,twokeypathsaremissingduetotherandomness.Sotheco veragemetrics arenot100%.Ifweincreasethenumberoftherandomteststo1 000,onekeypathisstill missing.Basedonourobservation,intherandommethod,iti shardtodeterminewhat isanappropriateupperboundforthenumberofrequiredrand omtests.Asaresult,it ishardtoobtain100%specicationcoverageusingtherando mtests.Theresultofthe UMCshowsthatwecangetanorderofmagnitudeimprovementco mparedtotherandom 62

PAGE 63

method.BecausetheboundsofthepropertiesofOSESsystema reshallowandcanbe pre-determined,weappliedSAT-basedBMCinthissituation .TheresultshowsthatBMC methodcanbeanorderofmagnitudefasterthanUMCmethod.Cl early,BMCapproach reducesthevalidationeortbytwohundredtimescomparedt othebestknownresult[ 51 ] inthiscategory. Table3-4.ImplementationlevelcoverageofOSES PackageClassMethodBlockLine 100%100%58%55%51% Table 3-4 presentsthecoverageoftheimplementationbyapplyingthe generated tests.Thecoverageofmethod,blockandlinearenotsucien tbecausetheactivity diagramdoesnotconsiderallthescenariosofthesystem,su chastheregistrationofthe customersandsoon.Inthiscase,weneededtoaddthemissing detailsinthespecication toobtaintherequiredcoverage. 3.4Summary Inthischapter,wepresentedaframeworktoautomaticallyg eneratedirectedtests fromSoCspecications.Ourexperimentalresultsdemonstr atedthatthegeneratedtests canproducetherequiredfunctionalcoverageandalsocanma keasignicantreduction invalidationeortforspecicationsaswellasimplementa tions.Modelcheckingbased testgenerationispromisingforautomatedtestgeneration butitcanleadtostatespace explosioninthepresenceofcomplexdesignsandproperties .Sointhefollowingchapters, wewillpresentvariousoptimizationtechniquestoreducet heoveralltestgeneration complexity. 63

PAGE 64

CHAPTER4 PROPERTYCLUSTERINGFOREFFICIENTTESTGENERATION Althoughmodelcheckingtechniquesarepromisingforautom ateddirectedtest generation,itiscostlyforcomplicateddesignsduetothes tatespaceexplosionproblem. Especiallyforacomplexdesign,therewillbealargenumber ofpropertiestobevalidated. Whenvalidatingaspecicsystemcomponent,itiscommontha tseveralpropertieshavea largeoverlaponsub-functionalities.Validatingtheprop ertiesindividuallywillbeawaste oftimeduetotherepeatedvalidationeortsonthesamefunc tionalscenarios.Potentially theseredundancycanbeavoidedandconsequentlytheoveral ltestgenerationtimecanbe signicantlyreduced. Fault Models Properties Test cases Validation Specification Validation Implementation Property Generation Test Generation Clustering Property Counterexamples Formal Model (Graph Model) Design Specification Figure4-1.Ourtestgenerationmethodology 64

PAGE 65

Thetargetofpropertyclusteringistoreducetheoverallte stgenerationtime byexploitingthesimilaritiesamongproperties.Figure 4-1 showsthetestgeneration frameworkusingourpropertyclusteringapproach.Theprop osedmethodologyhasthree importantsteps:coverage-drivenpropertygeneration,cl usteringofsimilarproperties, andtestgenerationusinglearningtechniques.Itisimport anttonotethateachofthese threestepscanbeindependent.Forexample,ourmethoduses thecoverageofourfault modelstoderiveproperties.Theothertwostepswillproduc ebenecialresultseven ifotherfaultmodelsareusedtogenerateproperties.Desig nerscanevenaddvarious propertiesmanuallytothesetofgeneratedpropertieswith outaectingtheusefulnessof ourapproach. Thischaptermakestwoprimarycontributions:i)itpropose snovelmethodsto clustersimilarproperties;andii)itutilizestheconrict clausebasedlearningtoreduce theoveralltestgenerationtimeforaclusterofsimilarpro perties.Therestofthischapter isorganizedasfollows.Section 4.1 presentsrelatedworkonecientmodelchecking techniques.Section 4.2 introducestheimplementationdetailsofstate-of-the-ar tSAT solvers.Section 4.3 proposesourpropertyclusteringapproaches.Section 4.4 presents howtoecientlygeneratetestsusingpropertyclusteringa ndconrictclauseforwarding technques.Section 4.5 demonstratescasestudiesonbothhardwareandsoftwaredes igns. Finally,Section 4.6 summarizesthechapter. 4.1RelatedWork DuetothescalabilityissuesofconventionalBinaryDecisi onDiagram(BDD)based methods,SAT-basedBMCisproposedasacomplementarysolut ionforlargedesigns. Manystudiesinbothsoftwareandhardwaredomains[ 4 ]showthatBMChasbetter capacityandproductivityoverunboundedmodelcheckingfo rrealdesigns.Currently, varioustechniquesbasedonconrictclauseforwardingandv ariableordering[ 63 ]are proposedtofurtherimprovetheeciencyofBMCbasedtestge neration. 65

PAGE 66

Asapromisinglearningbasedapproach,incrementalSAT[ 40 67 89 92 ]triesto leveragethesimilaritybetweentheelementsofasequenceo fSATinstances{mostdoso byre-utilizinglearnedknowledgebasedonconrictclauses .Whenmanycloselyrelated instancesneedtobesolved,cachingsolutions[ 43 ]andincrementaltranslation[ 7 ]canalso beeective.IfaSATinstanceisobtainedfromanotherbyaug mentingsomeclauses(as in[ 38 ]),allconrictclausesoftherstcanbeforwardedtothesec ond.Therefore,when clausesareonlyaddedthroughasequenceofinstances,ther eisnoneedtoscreenconrict clausestodeterminewhichonescanbeforwarded.This,onth eotherhand,isnecessary whenarbitraryclausesarebothaddedordeletedtocreatean ewinstance.Acommon approachforsuchageneralcaseistohaveincrementalSATso lverskeeptrackofwhether aconrictclausedependsonsomeremovedclauses.Majorityo ftheexistingapproaches exploitincrementalsatisabilitytoimprovethetestgene rationtimeinvolvingonlyone propertywithdierentbounds.Thereareveryfewapproache ssuchas[ 17 ]whereboth staticanddynamiclearningareusedacrosstestgeneration instancesforpath-delayfault modelbydynamicallyexcludingtheuntestablepathduringt estgeneration.Sincethe learningisemployedacrossalltestscenarioswithouteci entclusteringmethods,the improvementintestgenerationtimeissmall(6%onaverage) andhasawidevariation (-7%to27%)ondierentISCAScircuits. Tothebestofourknowledge,ourapproachistherstattempt toclustersimilartest generationinstancesinvolvingmultiplepropertiesandut ilizesharedknowledgeacross similarinstancesinthecontextofdirectedtestgeneratio n. 4.2Background:SATSolverImplementation ThissectionintroducesthepreliminaryknowledgeofSATso lverimplementation.In thecontextofdirectedtestgeneration,wedescribehowSAT -basedBMCcanbeusedto improvetestgenerationtimebyemployinglearningtechniq ues. 66

PAGE 67

4.2.1DPLLAlgorithm MostmodernSATsolverssuchasGRASP[ 39 ]andCha[ 63 ]adoptsthe DavisPutnam-Logemann-Loveland (DPLL)algorithm[ 52 53 ]. Algorithm3 :DPLLsearchprocedureofzCha while TRUE do run periodic functions (); if decide next branch () then while deduce()==CONFLICT do blevel= analyze conflicts (); if blevel < 0 then returnUNSAT; end end else returnSAT; end end Algorithm 3 showstheDPLLimplementationinzCha.Itcontainsthreepa rts: PeriodicfunctionupdatestheSATcongurationtriggeredb ysomespeciedevents, suchasupdatingthescoresofliteralsafteracertainnumbe rofbacktracks. BooleanConstraintPropagation(BCP)isimplementedin deduce .Itguresoutall possibleimplicationsbypreviousdecisionassignment. Conrictanalysisdoesaproperbacktrackwhenencountering aconrict.Itanalyzes thereasonfortheconrictandmakeitasaconrictclausetoav oidthesameconrict infutureprocessing. Studiesin[ 63 ]showthatmodernSATsolversspendapproximately80%oftim eto carryoutBCP.Inaddition,duringtheconrictanalysis,lon gdistancebacktrackswill increasetheburdenofSATsolvers.4.2.2ConrictClauseBasedLearning AsshowninAlgorithm 3 ,SATsolversusetheconrictanalysistechniquetotracethe reasonforaconrict.Theconrictanalysiscontainstwopart :conrict-drivenback-tracking 67

PAGE 68

andconrict-drivenlearning.Conrict-drivenbacktrackin genablesthenon-chronological backtrackinguptotheclosestdecisionwhichcausedthecon rict.Conrict-drivenlearning learnssomeknowledgeandsavetheminconrictclausesandad dsthemtotheoriginal clauses,inordertoavoidthesameconrictinthefuture.Bot htechniquescandrastically boosttheperformanceoftheSATsolvers. Thekerneloftheconrictanalysistechniqueistheimplicat iongraph[ 39 91 ].The graphkeepsthecurrentstateandtheimplicationhistoryof thesearchduringtheSAT solvingbyrecordingthedependenceofthevariableassignm ents.Theimplicationgraphis adirectedacyclicgraphwhereeachvertexrepresentsanass ignmenttoavariableandeach edgeimpliesthatallthein-edgesimplicatetheassignment ofthevertex. Implicate Cut Conflicting Vertex Implication Vertex Decision Vertex Clauses: Conflict Clauses: Conflict Reason: ( X8, X8') C4: X3' + X4' C6: X1' + X5 + X6 + X7' x4 @ 4 x1 @ 3 x2 @ 4 x8 @ 4 x7 @ 2 CUT 1 x6' @ 1 x5' @ 4 x3' @ 4 x8' @ 4 C3: X1' + X4 + X5 C5: X2' + X3 + X8 C1: X2 + X4' + X6 C2: X3 + X7' + X8' Figure4-2.Conrictanalysisusinganimplicationgraph Figure 4-2 showsasmallexampleofconrictanalysisusinganimplicati ongraph.As shownattheleftofthegure,thereareveoriginalclauses C1-C5 .Therightpartis ascenarioofimplicationgraphfor C1-C5 .Inthisexample, x 4@4meansvariable x 4is assignedvalue1atdecisionlevel4.Thenodehasacorrespon dingclause (x1'+x4+x5) wecallittheantecedentclauseof x 4,i.e.,theassignments x 1=1and x 5=0imply 68

PAGE 69

x 4=1.Onlytheimplicationvertex(non-decisionvertex)has anantecedentclause.A conricthappenswhentherearetwonodesintheimplicationg raphthathavedierent valueassignmentsforthesamevariable.Forexample,theim plicationsinthegraphlead totheambiguousassignmenttovariable X 8( X 8=0and X 8=1).Whenencounteringa conrict,conrictanalysiswilltracebackalongtheimplica tionrelationstondthereason fortheconrictandencodethereasonusingaconrictclause. Aconrictclausecanbe foundbyabipartitionoftheimplicationgraph.Thesidecon tainingtheconrictingvertex calledconrictside,andtheothersideiscalledreasonside whichcanbeusedtoformthe conrictclause.InFigure 4-2 CUT 1isacutthatdividestheimplicationgraphintotwo parts.Theconrictanalysisstopsat CUT 1.Theleftpartof CUT 1intheimplication graphisthereasonside,andtherightpartistheconrictsid e.Fromthereasonside,we cangetthe conrictcluaseC6=(X1+X5'+X6'+X7) .Thatmeans,theassignmentof variables X 1=1, X 5=0, X 6=0and X 7=1willalwaysleadtoaconrictbecauseof theclauses C1-C5 .Lemma 1 indicatesthatthegeneratedconrictclausesduringtheSAT searchcanbeaddedtooriginalclausesetasanassignmentco nstraint.Thereforewecan addtheclause C 6totheoriginalclausesettoavoidthesameconrictinthefu ture. Lemma1. GivenasetofCNFclauses S 1 and isaconrictclausederivedduringthe conrictanalysis,then S 1 issatisablei S 1 V issatisable. Proof. Because S 1 V isasupersetof S 1,soif S 1 V issatisablethen S 1issatisable. Accordingtothedenitionoftheconrictclause,theassign mentsthatmaketheclause falsewillmaketheclauseset S 1false.If S 1issatisable,thenthereexistsavariable assignmentthatmakes S 1true.Thisassignmentshouldmake true.Sotheassignments willmake S 1 V true. FortwoSATinstances,ifoneinstanceisasubsetoftheother SATinstance, accordingtoTheorem 1 ,theconrictclausesgeneratedfromthesmallerSATinstanc e canbeforwardedtothelargerSATinstance.Inotherwords,t helocallearningcanbe 69

PAGE 70

forwardedasaknowledgeforglobalsearching.Usuallythea veragecostoflocallylearned conrictclausesismuchcheaperthanthegloballylearnedco nrictclauses. Theorem1. GiventwoCNFclausesets S 1 and S 2 ,where S 1 S 2 ,and isaconrict clausederivedfromtheclausesin S 1 ,written S 1 ` ,then S 2 issatisablei S 2 V is satisable. Proof. Since S 2 V isasupersetof S 2,if S 2 V issatisablethen S 2issatisable. Because S 1 ` and S 1 S 2,then isalsoaconrictclauseof S 2.AccordingtoLemma 1 S 2issatisablei S 2 V issatisable. AccordingtotheEquation( 3{1 ),similarpropertiessharealargepartofthe CNFclauses.Regardlessoftheconeofinruence,theequatio nsharesthesystempart (transitionrelation T ( s i ;s i +1 ))andthepartofpropertytesting(i.e., p ( s i )).Sharinga largepartofCNFclausesindicatesthatwhencheckingther stproperty,thelearned knowledge(conrictclauses)canbeforwardedtothesecondp ropertywithoutaectingthe truthassignmentoftheCNFclausesofthesecondproperty. Theorem2. AssumethatwehavetwosetsofCNFclauses S 1 and S 2 ,andlet = S 1 T S 2 bethecommonclausessharedbyboth S 1 and S 2 isaconrictclausederived onlybytheclausesin ,written ` .Then S 2 issatisablei S 2 V issatisable. Proof. Because S 2 V isasupersetof S 2,so S 2 V issatisablethen S 2issatisable. Because ` and S 2,then S 2 ` .AccordingtoLemma 1 S 2issatisablei S 2 V issatisable. 4.3PropertyClustering Givenasetofproperties,aclusteringmethoddeterminesho wtodividetheproperties intoseveralgroupssuchthateachgroupcontainssimilarpr opertiesthatcanbenetfrom eachotherduringtestgeneration.Thesimilaritycanbestr ucturalorbehavioralbutthe assumptionisthatthereisasignicantoverlapbetweenthe counterexamplegeneration tracesinvolvingasetofsimilarproperties. 70

PAGE 71

Algorithm4 :PropertyClustering Input :i)Asetofproperties, P ii)Similaritystrategy CS ,andthreshold W th Output :Clustersconsistingofsimilarproperties PropertyClusters = ; 1. Constructagraph, G whereeachnodeisaproperty; for eachpairofnodes( n i n j )in G do Weight w j i =ComputeSimilarity CS ( n i n j ); if( w i W th )Createanedgebetween n i and n j withweight w j i ; end2. k =1;/*rstcluster*/ while G isnotempty do Base k =Nodewithhighestoveralledgeweight; Cluster k =allthenodesconnectedto Base k ; G = G Cluster k ; PropertyClusters = PropertyClusters [ Cluster k ; k = k +1; endreturn PropertyClusters; Algorithm 4 outlinesthemajorstepsinpropertyclustering.Therstst epconstructs apropertygraph 1 wherethepropertiesarenodesandedgesrepresentsimilari ty.Anedge isaddedbetweentwoproperties(nodes)whentheyaresimila r.Eachedge e j includes weightinformation( w j ,0 w 1)toquantifythesimilarity.Anedgewithweight0 or1isnotpossiblesinceanweightof0meansnosimilarity,a ndanweightof1implies same(identical)property.Tocomputetheweightinformati onforeachedgewepropose fourmethods{structural,textual,inruenceandCNFinters ectionbasedsimilarity.Each methodwilluseasimilaritythresholdforclustering.Inot herwords,therewillbenoedge 1 Inthischapter,weusethreedierenttypesofgraphsforthr eedierentpurposes. Thegraphmodelofthedesign(or designgraph inshort)isusedtomodelthedesign.The implicationgraph isusedtostorethedependenceofvariableassignmentsthat isusedfor conrictanalysis.The propertygraph modelsthesimilaritybetweenpropertiesandusedfor clustering. 71

PAGE 72

betweentwopropertieswhentheweightvalueisbelowcertai nthreshold.Thesecondstep determinestheclustersbasedonthebaseproperty.Thebase propertyistheproperty (node)withhighestweight(summationofweightsofalledge sconnectedtothatnode). Theclusterisformedbyaddingalltheadjacentnodeswithth ebaseproperty.Allthe nodesselectedforaclusteraredeletedfromthepropertygr aphforthenextiteration.The remainderofthissectiondescribesfourdierentwaysofco mputingsimilaritybetween properties.4.3.1SimilaritybasedonStructuralOverlap Asimpleandnaturalwaytoclusterpropertiesistoexploitt hestructuralinformation ofthedesignmodelanditsproperties.Theintuitionisthat twosimilarpropertieswill sharesimilarvariableassignments(globalandlocalvaria bles 2 )inthecounterexamples. Infact,aconrictclauseisaconstraintontheassignmentof thevariables.Therefore, propertieswithsimilarstructuralinformationwillshare alotofconrictclauses. Asmentionedearlier,inthecontextofdirectedtestgenera tion,propertiesare generatedbasedonfunctionalcoverageofthedesign.These propertiestrytocover dierentpartsofthedesign(e.g.,allcomputationnodes,v ariousinteractions,etc.). Therefore,wecanclusterthepropertiesthattrytocoveras pecicfunctionalityor interactions.Forexample,inanSoCenvironment,theprope rtiescanbeclusteredbased onwhethertheyarerelatedtoverifyingtheprocessor,copr ocessor,FPGA,memory,bus synchronization,orcontrollers.Eachclustercanbefurth errenedbasedonstructural detailsofeachcomponent.Forexample,theprocessorrelat edpropertiescanbefurther dividedbasedonwhichexecutionpaththeyactivatesuchasA LUpipeline,load-store pipelineetc. 2 Inagraphmodel,alocalvariableisdenedlocallyinanodew hereasthescopeofa globalvariableisvalidacrossnodes. 72

PAGE 73

InthepipelinedprocessorexampleinFigure 2-12 ,therearefourexecutionpipelines: IALU MUL FADD and DIV .Thecorrespondingpathsareasfollows. 1 = FET DEC IALU MEM WB 2 = FET DEC MUL 1 MUL 7 MEM WB 3 = FET DEC FADD 1 FADD 4 MEM WB 4 = FET DEC DIV MEM WB Considertwoproperties p 1= F ( fadd 3 active =1)andproperty p 2= F ( fadd 4 active =1).Theysharethesamepath 3 ,andtheboundof p 1isjustone smallerthan p 2.Sowecanclusterthemtogether.Alsofortheinteractionp roperty p 3= F ( fadd 4 active =1& mul 3 active =1)and p 4= F ( fadd 3 active = 1& mul 4 active =1),thetwointeractionsarerelatedtothesamesetofpaths 2 and 3 andhavesimilarbounds.Therefore,clusteringthemtogeth erisagoodchoice. 4.3.2SimilaritybasedonTextualOverlap Anothersimplewaytoquantifysimilarityistomeasurethet extualdierences betweentwoproperties.Forexample,thesimilaritybetwee n F ( a & b & c )and F ( b & c & d )is67%sincetheyshareacommonsub-expressionconsisting oftwo variables b and c Inthissection,wefocusonboundedmodelcheckingofinvari ants(safetyproperties) suchasthepropertyintheform F(p).Informally, BMC ( M;p;k )istruemeansfrom cycle0tocyclek,thepropertywillbefalse.Sotheinvarian tcannotalwaysbetrueand onecounterexamplewillbereported.Becausethepart I ( s 0 ) ^ V k 1 i =0 T ( s i ;s i +1 )comes fromthedesign,sofordierentpropertiesthispartissame .Thepart W ki =0 : p ( s i )usually determinesthedierenceamongtheproperties. Thenegativeformatofeachliteralintheconrictclauseisa falseassignmentforthe logicformula BMC ( M;p;k ).Infact,theconrictclausecanberegardedasaconstraint forthevariableassignment.Let P and Q betwopropertiesofthemodel,theproperties P P ^ Q and P Q canbeexpandedasfollows: 73

PAGE 74

BMC 1 ( M;P;k )= I ( s 0 ) ^ V k 1 i =0 T ( s i ;s i +1 ) ^ W ki =0 : P ( s i ) BMC 2 ( M;P ^ Q;k )= I ( s 0 ) ^ V k 1 i =0 T ( s i ;s i +1 ) ^ W ki =0 : ( P ^ Q )( s i ) = I ( s 0 ) ^ V k 1 i =0 T ( s i ;s i +1 ) ^ W ki =0 ( : P ( s i ) _: Q ( s i )) BMC 3 ( M;P Q;k )= I ( s 0 ) ^ V k 1 i =0 T ( s i ;s i +1 ) ^ W ki =0 ( : P ( s i ) ^: Q ( s i )) IntheexpandedCNFsabove,weassumethatthesamevariablei nrespective expansionhasthesamemeaning.Let A beapartialassignmentoftheCNFvariables thatcanmakethewholeCNFfalse,then A j = =BMC 1 implies A j = =BMC 3 A j = =BMC 2 implies A j = =BMC 1 ,and A j = =BMC 2 implies A j = =BMC 3 .Inotherwords,theconrict clausesof BMC 1 canbeforwardedto BMC 3 ,andconrictclauseof BMC 2 canbe forwardedtoboth BMC 1 and BMC 3 InmostexistingBMCtools,thevariablesinthegeneratedCN Fledonothave specicmeaning.Theconrictclausesofthestrongerproper tycannotbedirectly forwardedtosomeweakerproperties.Forexample,someconr ictclausesofproperty P ^ Q cannotbeforwardedtocheckproperty P Q .However,whenpropertieshave therelationofimplication,andtheirtextualsimilarityi shigh,clusteringthemtogether willhaveapositiveeect.Iftwopropertiesareinthesamef ormatandhaveasignicant (morethan50%)textualoverlap,thetwopropertiescanbene tfromeachother. Textualclusteringisveryfastbutitmaynotbeveryaccurat e.Forexample,the properties F ( a )and F ( c )havenooverlap,however,itispossiblethatbothvariable s areverycloselyrelatedinthedesignmodel(suchasactivat esthesamepath),and thereforetheyaregoodcandidateforclustering.Unfortun ately,intheabsenceofsuch structuralinformation,puretextualclusteringmaynotge neratesignicantsavingsintest generationtime.Textualclusteringisbenecialwheninfo rmationregardingthedesignor originalfaultmodelarenotavailableand/orwhenthereare toomanyproperties. 4.3.3SimilaritybasedonInruence Anassignmenttoaglobalvariabledeterminesthestatetran sitionofvarious componentsinthedesign(graph)model.Forexample,intheM IPSmodel,whenthe 74

PAGE 75

instructionbuercontainsonlydivisioninstruction,onl ythecomponentsin DIV path willbeactivated.However,itistimeconsumingtoanalyzea lltheglobalandlocal variablesofthemodelsinceitneedtoconsiderthestatetra nsitionofeachcomponent. Basedonthegraphmodelstructure,wecandeterminevarious cause-eectrelations. Forexample,thestatechangeof MUL 6willbeoneclockcyclelaterthan MUL 5.That meanstheexecutionof MUL 5hasaninruenceontheexecutionof MUL 6.Theinruence nodesindirectlyrerecttheassignmentoftheglobalvariab les,sincetheassignmentof globalvariablesisrelevanttothevariableassignmentint hecounterexample. Priortoclustering,weneedtogureouttheinruencenodese tforeachnodeinthe graphmodel.Wecancomputetheinruencenodesetforeachnod eusingDepthFirst Search( DFS )algorithm.Ifthereisapathstartingfromthestartnodeto thecurrent node,thenallthenodesonthispathareinruencenodesforth ecurrentnode. DFS can exploreallthepaths(exceptthepathswithloops)fromthes tartnodetothecurrent node.Forexampletheinruencenodesetsfor MUL 2, FADD 3and WB areasfollows: Influence ( MUL 2)= f FET;DEC;MUL 1 ;MUL 2 g Influence ( FADD 3)= f FET;DEC;FADD 1 ;FADD 2 ;FADD 3 g Influence ( WB )= f n j nisanodeintheMIPSgraphmodel. g Apropertycorrespondstoseveralnodes(modules)inthegra phmodel.Sothe inruencenodesetofapropertyistheunionoftheinruenceof allrelevantnodes.When comparingthesimilarityoftwoproperties,weneedtocompu tetheintersectionof inruencesets.Forexample,theinruencesetofproperty F ( mul 2 active =1& fadd 3 active =1)is S 1 = f FET DEC MUL 1, MUL 2, FADD 1, FADD 2, FADD 3 g andtheinruencesetfor F ( mul 3 active =1& fadd 3 active =1)is S 2 = f FET DEC MUL 1, MUL 2, MUL 3, FADD 1, FADD 2, FADD 3 g .Thetwosetssharealarge intersection.Forset S 1 ,thesimilaritywith S 2 is7 = 7=100%.Forset S 2 thesimilarity with S 1 is7 = 8=87 : 5%.Basedonourexperience,whentheoverlapofinruenceset sare 75

PAGE 76

largerthan70%,forwardingconrictclausesisbenecial.I nthisexample, S 1 and S 2 can beclusteredtogether.4.3.4SimilaritybasedonCNFIntersection Oneobvious,butcostly,waytodeterminepropertysimilari tyforclusteringisto computeintersectionsofCNFclausesbetweenproperties.W ecanclusterpropertiesthat havearelativelylargenumberofclausesintheintersectio n.Basedonourexperience, athresholdof0.9isbenecial.Inotherwords,whentwoprop ertiesshareatleast90% commonclauses,itisbenecialtoforwardconrictclausesb etweentwoinstances. Thismethodisverytimeconsumingbecauseitrequires O ( n 2 )intersectionsfor n properties.When n islarge,thismethodisnotfeasible,becausethecalculati onof intersectionofirrelevantpropertiesmaywastemoretimet hanactualSATsolutiontime. Moreover,incertainscenarios,forwardingconrictclause smaynotimprovetheoverall testgenerationtimeforacluster,sinceitmaychangevaria bleorderingandsearching heuristics.CNFbasedclusteringisagoodchoicewhenthenu mberofpropertiesissmall orwhenothermethodsfailtondbenecialclusters.4.3.5DeterminationofBaseProperty Determinationofbasepropertyinaclusteriscrucialforte stgenerationusing learningtechniques.Thebasepropertyissolvedrstandit sconrictclausesaresharedby theremainingpropertiesinthecluster.Although,anyprop ertyintheclustercanbeused asthebasepropertyforthatcluster,ourstudieshaveshown thatcertainpropertiesserve betterasbasepropertyandtherebygeneratebetteroverall savingsforthecluster.We needtoconsidertwoimportantfactorswhilechoosingabase propertyforacluster.First, thebasepropertyshouldbeabletogeneratealargenumberof conrictclauses.Inother words,aweakbasepropertymayndthesatisableassignmen tquicklywithoutmaking mistakes(generatingconrictclauses).Inthisscenario,t heremainingpropertieshave nothingtolearnfromthebaseproperty.Moreover,theSATch eckingtimeforthebase propertyshouldberelativelysmall.Thiswillensurethatt heoverallgainismaximized 76

PAGE 77

byreducingthesolutiontimeofthepropertieswhichtakesl ongertimetosolve.None oftheserequirementscanbedeterminedwithoutactuallyso lvingthem.Basedonour experience,wehaveobservedthatthefollowingheuristics workswellmostofthetime. Chooseapropertythathassignicantvariableand/orsub-e xpressionoverlapwith otherpropertiesinthecluster. Ifboundforeachpropertyisknown,choosethepropertywhos eboundisclosestto theremainingproperties. Computeintersectionofeverypairofpropertiesintheclus ter,andchoosetheone thatsharesthemostwiththeremainingproperties. 4.4EcientTestGenerationusingLearningTechniques IncrementalSAT-basedBMC[ 54 ]isverypromisingtoreducethetestgeneration complexity.However,existingapproachesarerestrictedf oratestgenerationscenario consistingofonedesignandonlyoneproperty(withvarying bounds).Manyproperties generatedfromthedesignspecicationsharealotofsimila rinformation.Iftheshared informationcanbeexploitedandre-utilizedacrossthesim ilarproperties,manyrepeated vericationeortscanbeavoided.Inotherwords,theknowl edgelearnedduringthe executionofonepropertycanbenetothersimilarproperti es.Therefore,knowledge sharingcanreducecomplexityandimprovetheoverallveri cationeort.Althougheach testgenerationinstancerequiresadierentproperty,sev eralpropertiesrelatedtotesting specicfunctionalitiesaresimilarorhaveasignicantov erlap.Reuseoflearnedknowledge (e.g.,constraints)derivedfromsuchoverlapcanavoidthe repeatedstatespacesearch.In thissection,wediscusstwokindsoflearningtechniqueswh ichcandrasticallyreducethe overalltestgenerationtimeinaclusterofsimilarpropert ies. 4.4.1ConrictClauseForwardingTechniques Thebasicideaistolearnfromsolvingonepropertyandshare learning(through conrictclauses)forsolvingthesimilarpropertiesinthec luster.Whilesolvingthe rstproperty(baseproperty),theSATsolvermayhavetaken manywrongdecisions (leadtoconricts)andthereforeneedslongtimetondacoun terexample.Forwarding 77

PAGE 78

conrictclausesensuresthatthesewrongdecisionsareavoi dedwhilesolvingthesimilar properties.Animportantquestioniswhetherallthewrongd ecisionsoftherstproperty arerelevanttoalltheotherpropertiesintheclusters?Sin cethepropertiesaresimilarbut notthesame,someofthedecisionsarenotrelevant.Inourap proach,wedeterminethe commonCNFclausesbycomputingtheintersectionofclauses andusethisintersection informationtoexactlyidentifytheconrictclausesthatar erelevanttosolvingthe respectiveproperties. Algorithm5 :TestGenerationusingLearningTechniques Input :i)DesignmodelDandii)Clustersofsimilarproperties Output :Tests for eachcluster, i ,ofproperties do GenerateCNFforthebaseproperty P i 1 CNF i 1 ; for j isfrom2tothe size i ofcluster i do /* P i j isthe j th propertyinthe i th cluster*/; 1. GenerateCNF, CNF i j = BMC ( D;P i j ;bound ij ); 2. Performnamesubstitutionon CNF i j ; 3. INT i j =ComputeIntersection( CNF i 1 CNF i j ); 4. Marktheclausesof CNF i 1 using INT i j ; end/*Generateacounterexampleandrecordconrictclauses*/;5. ( ConflictClauses i test i1 )=SAT( CNF i 1 ); Tests = f test i1 g ; for j isfrom2tothe size i ofcluster i do /*Findrelevantonesfor P i j fromconrictclauses*/; 6. CC i j =Filter( ConflictClauses i j ); endfor j isfrom2tothe size i ofcluster i do 7. test ij =SAT( CNF i j S CC i j ); Tests = Tests [ test ij ; end endreturn Tests ; 78

PAGE 79

Algorithm 5 describesourtestgenerationmethodology.Itacceptsalis tofclusters whereeachclusterconsistsofasetofsimilarproperties.S inceonepropertyisusedto generateatest,thenumberofinputpropertiesisexactlyth esameasthenumberof outputtests.TherststepgeneratestheCNFclausesforall thepropertiesineachcluster usingthedesignandrespectivebounds.Thesecondstepperf ormsnamesubstitutionto maximizeknowledgesharing.Thethirdstepcomputestheint ersectionofCNFclauses betweenthebasepropertyandalltheremainingpropertiesi nthecluster.Therstthree stepscanbeomitted,ifCNFintersectionbasedclusteringi semployed.Thefourthstep markstheclausesinthebasepropertytoindicatewhetherap articularclauseisalso intheclausesetofanotherpropertyinthecluster.Thenext stepusesaSATsolverto generatetheconrictclausesandthecounterexampleforthe baseproperty.Basedonthe intersectioninformationwiththebaseproperty,thesetof conrictclausesislteredto identifytherelevantonesforsolvingtheremainingproper tiesinstep6.Thenalstep usestherelevantconrictclausestosolvetheremainingpro pertiesusingourapproach. Thealgorithmreportsallthegeneratedcounterexamples. WeuseasimpleexampletoillustratehowAlgorithm 5 works.Letusassumethat wearegeneratingtestsusing n propertiesforadesign.Theinputisalistof m ( m n )clustersbasedonpropertysimilarities.Eachclustercan havedierentnumberof properties.Intheworstcase,eachclustercanhaveonlyone propertywhichwillbe veriednormally.However,thisscenarioisrareinpractic esinceatypicaldesignuses thousandsofpropertiesfordirectedtestgenerationandma jorityofthemsharesignicant partsofthedesignfunctionality.Foreaseofillustration ,letusassumethatthereisa clusterwiththreesimilarproperties, f P 1 P 2 P 3 g .Letusfurtherassumethatthesecond stepselects P 1 asthebaseproperty.Thefourthstepcomputesintersection ofCNFclauses of P 1 with P 2 ,and P 1 with P 3 .Thisinformationisusedtolterconrictclauses(generat ed whilesolving P 1 )relevantfor P 2 and P 3 instep6.Thelaststepaddstherelevantconrict clauseswhilesolvingtherespectivepropertiestoreducet hetestgenerationtime. 79

PAGE 80

Thefollowingsubsectionsdescribetwoimportanttechniqu esinourapproach,name substitutionforcomputationofintersections,andidenti cationofrelevantconrictclauses. 4.4.2NameSubstitutionforComputationofIntersections NamesubstitutionisanimportantpreprocessingstepinAlg orithm 4 .Currently,few BMCtoolssupportthenamemappingfromthevariablesoftheC NFclausesandthe namesinthemodeloftheunrolleddesign.Asaresult,thevar iablesoftheCNFclauses oftwodierentpropertiesmaynothaveanynamecorresponde nce.Inotherwords,the samevariableintwopropertiesmayhavedierentnameinthe irrespectiveCNFclauses. Therefore,withoutnamesubstitution(mapping),itwillmi sstheoverlapinformation. Asaresult,thecomputedintersectionwillbesmallandwill adverselyaectthesharing oflearnedconrictclauses.Weobservedthattheimprovemen tintestgenerationtime withoutusingnamesubstitutionisnegligiblysmallduetov erysmallnumberofclauses beingforwardedasaresultofsmallnumberofclausesinthei ntersection.Sincethe propertiesaresimilarandthedesignisexactlythesame,th esizeoftheintersectionis verylargewhenournamesubstitutionmethodisemployed. OurframeworkuseszChaSATsolver[ 74 ]whichacceptstheinputintheDIMACS format.ThegeneratedDIMACSleforeachpropertyprovides thenamemappingfrom theCNFvariabletotheunrolleddesign.Forexample,\c8=V1 var[6]"showsthat thevariable8isusedintheCNFletorefertothe7 th bitofvariable var inthedesign specicationattimestep1.Thiscanalsobewrittenas,8= >var [6] 1. GiventwoDIMACSles f 1and f 2fortwoproperties P 1 and P 2 respectively,the namesubstitutionisaprocedurethatchangesthenamesofcl ausevariablesof f 2using thenamemappingdenedin f 1.Figure 4-3 showsanexamplefornamesubstitution. Beforethenamesubstitution,theintersection( f 1 \ f 2)isempty.However,aftername substitution,therearetwocommonclausesintheintersect ion( f 1 \ f 2 0 ).Thecomplexity ofbothnamesubstitutionandcomputationofintersectioni slinear(usinghashtable) tothesizeoftheDIMACSleoftheproperties.Therefore,th etimerequiredforname 80

PAGE 81

substitutionandintersectioncomputationisnegligiblec omparedtotheSATsolvingtime forcomplexproperties. Itisimportanttonotethatthesamevariableatdierenttim estepscanbeassigned adierentnumber.Therefore,thenamemapping(substituti on)methodneedstoconsider thesamevariableatdierenttimestepsintheCNFclausesof thesamepropertyaswell asintheCNFclausesforthedierentpropertiesinthesamec luster.Moreover,thename mappingroutineneedstoremapsomeofthevariablesintheCN Fclauses.Forexamplein Figure 4-3 ,whenthevariable4inle f 2isreplacedwiththevariable1(in f 2 0 ),thename mappingroutineneedstoremaptheoriginalvariable1inle f 2 0 toadierentvariable. c 2 => b_1 c 1 => a_1c 3 => a_2p cnf 3 3-1 2 0 DIMACS f1DIMACS f2 1 4 0 DIMACS f2'c 2 => b_1c 3 => a_2p cnf 6 4 p cnf 6 4 c 1 => a_1 c 6 => a_2 c 5 => b_1 ......c 4 => a_1 ...... 5 -4 05 6 0 3 2 0 1 3 0 2 -3 05 -6 0 4 1 0 2 -1 02 3 0 Figure4-3.Anexampleofnamesubstitution 4.4.3IdenticationandReuseofCommonConrictClauses Ourimplementationofrelevantconrictclausedeterminati onismotivatedbythework of[ 67 ]whichprovedthatfortwosetsofCNFclauses C 1 and C 2 ,andtheirintersection useofconrictclausesgeneratedfrom whenchecking C 1 willnotaectthesatisability oftheCNFclauses C 2 S .Therefore,theconrictclausesgeneratedfromtheinterse ction whencheckingthebasepropertycanbesharedbyotherproper tiesinthecluster. Strichman[ 67 ]suggestedanisolationprocedurethatcanisolatetheconr ictclauses whicharededucedsolelyfromtheintersectionoftwoCNFcla usesets.Wehavemodied theisolationproceduretoimprovetheeciencyoftestgene rationforaclusterof properties.WehavemodiedzCha[ 74 ]SATsolveranduseditinourframework. ThezChaprovidesutilitiesforimplementingincremental satisability.Foreachclause, 81

PAGE 82

ituses32bitstostoreagroupidtoidentifythegroupwheret hisclausebelongs.Useof groupidallowsustogeneratetheconrictclausesfordiere ntpropertieswhenchecking thebaseproperty.Ifthe i th bitoftheclause'sgroupidis1,itimpliesthattheclauseis sharedbytheCNFclausesofproperty P i .Iftheclauseofthebasepropertyisnotshared byanyproperty,theeldwillbe0. Assumethatthereare k +1propertiesinaclusterwith C i asthesetofCNFclauses fortheproperty P i .Moreover,assumethat P 0 isthebaseproperty.Inotherwords,there are k +1setsofclauseswith C 0 asthebaseset,and C 1 ;C 2 ;:::;C k are k similarsetswith C 0 .Weusethefollowingstepstocalculatetheconrictclauses for C 1 ;C 2 ;:::;C k when solving C 0 Duringpreprocessing,foreachclause cl in C 1 ,ifthisclausealsoexistsin C i (2 i k ),thenmarkthe i th bitof cl 'sgroupidas1. Whenoneconrictclauseisencounteredduringthecheckingo fthebaseproperty, collectallthegroupidsoftheclausesintheconrictside.T hegroupidoftheconrict clauseislogical\AND"ofallthesegroupids. Foreachconrictclause,ifthe i th bitofthegroupidis1,thenthisconrictclausecan besharedby C i Inourapproach,eachconrictsideclausehasagroupidwhich ismarkedduringthe preprocessingstepormarkedduringtheconrictanalysisif itisaconrictclause.The procedureofgroupiddeterminationofaconrictclauseisde scribedinAlgorithm 6 Thisalgorithmtracesbackfromtheconrictingassignmentt oacutsuchas first UniqueImplicationPoint (UIP)[ 91 ]inzCha.Theconrictsidewillcontainallthe implicationsofthevariableassignmentsofthereasonside .ForUIP,theyareimplication variableassignmentsinthesamedecisionlevelastheconri ctingvariableassignmentwhich ledtotheconrict.Thegroupidoftheconrictclauseisthelo gical\AND"valueofallthe groupidsoftheconrictsideclauses.Thisalgorithmcangua ranteethatifthe i th bitof thegroupidoftheconrictclauseis1,thenthisconrictclau secanbeforwardedtothe i th CNFclauseset. 82

PAGE 83

Algorithm6 :DeterminationofconrictclauseanditsgroupID Input :i)Conrictingnode N Output :Conrictclausewithitsgroupid Visited = f N g ; ConflictAssign = fg ; groupId =groupidof N 0 s antecedentclause; while theset Visited isnotempty do 1. v =RemoveOneElement( Visited ); 2. clause =AntecedentOf( v ); groupID = groupID \AND" groupidof clause ; if v isontheconrictside then 3. Putallthenodesof clause inimplicationgraphexcept v totheset Visited ; else 4. ConflictAssign = ConflictAssign [f v g ; end end5. ConflictClause =Logicaldisjunctionofnegatedassignmentsofallelement sin ConflictAssign ; return ConflictClause and groupId ; Figure 4-4 illustrateshowthiscomputationisdone.Theimplicationg raphbelongs toabasepropertyofacluster.Eachclauseinthisgraphisma rkedwiththegroupid information.Hereweusefourbitstoexpressthegroupid.Fo rexample,thegroupidof clause( x 3 0 + x 4 0 )is\1010".ItmeansthatthisclauseexistsbothinCNFclaus eset2and CNFclauseset4.Thegroupidoftheconrictclauseisthelogi cal\AND"ofallconrict sideclauses,andtheresultis0010.Thatmeans,thisconric tclausecanbeforwardedto clauseset C 2 .Therefore,theuseofthisconrictclauseinsolving P 2 willreducetheSAT solving(testgeneration)time. 4.5CaseStudies Wehaveappliedourtestgenerationmethodologyforvalidat ionofvarioussoftware andhardwaredesigns.Inthissection,wepresenttwocasest udies:theVLIWimplementation oftheMIPSarchitecture,andastockexchangesystem.Bothe xperimentswereperformed 83

PAGE 84

-x5 @ 4 x1 @ 3 x4 @ 4 -x6 @ 1 x2 @ 4 x8 @ 4 -x8 @ 4 x7 @ 2 -x3 @ 4 CUT 1 (X1' + x5 + x6 + x7') Conflict ClauseConflict Side Clauses (x1' + x4 +x5) (x3' + x4') (x2 + x4' +x6) (x3 + x7' +x8') (x2' + x3 + x8) Implicate Cut Conflicting Vertex Implication Vertex Decision Vertex Clauses 4 3 2 1 0 1 1 1 1 0 1 01 1 1 11 0 1 0 1 1 1 0 Group id Figure4-4.Anexampleofconrictclausereuse onaLinuxPCusing2.0GHzCore2DuoCPUwith1GBRAM.Inourexp eriments,we usedtheNuSMV[ 27 ]asourBMCtooltogeneratetheCNFclauses(intheDIMACS format)forthedesignandproperties.Wedevelopedthetool PropertyCluster which acceptsthegraphmodel,thecoveragecriteriaandtheclust eringstrategiesasinputs. Thistoolgeneratestherequiredproperties(usingdieren tcoveragecriteriapresentedin Section 3.1 )andclustersthemusingtheclusteringstrategiespropose dinSection 4.3 .We alsomodiedzCha[ 74 ]tointegrateourtechniquesincludingnamesubstitution, clause intersection,andconstraintsharingbasedtestgeneratio ndescribedinSection 4.4.1 4.5.1AVLIWMIPSProcessor Weappliedourmethodologyonthesingle-issueMIPSpresent edinSection 2.3.2 .The PropertyCluster generated171propertiesusingthenodecoverage,2-intera ctioncoverage, andthepathcoveragecriteria.Inthissectionwerstprese ntresultsforeachclustering technique,andthenpresentasummarytocomparethecluster ingtechniques. 4.5.1.1Structure-basedClustering ThegraphmodelofMIPSprocessorhasfourparallelpipeline paths.Eachofthem sharesfourunits(fetch,decode,memoryandwriteback),an ddiersonlyintheexecution units.Thestructuralsimilarityisestablishedbasedonth epaththatasetofproperties 84

PAGE 85

activates.Forexample,thefollowing7propertiesisgroup edinaclusterbecauseallof themrefertothedivisionpath. p 13= F ( fet active !=1& div active !=1) p 28= F ( dec active !=1& div active !=1) p 133= F ( div active !=1& mem active !=1) p 134= F ( div active !=1& wb active !=1) p 150= F ( div active !=1)) p 165= F ( fet active !=1& dec active !=1& div active !=1) p 170= F ( fet active !=1& dec active !=1& div active !=1& mem active != 1& wb active !=1) Table 4-1 presentsthevericationdetailsfortheabovecluster.Thi sclusterhas7 propertieswhere p 13isthebaseproperty.Thesecondcolumnshowstheproperty type (nodecoverage,edgecoverage,interactioncoverageetc.) .Thethirdcolumnindicatesthe boundforthatproperty.Thefourthcolumnshowsthenumbero fCNFclauses(size)for thatproperty.Thefthcolumnpresentsthenumberofconric tclausesforwardedfromthe baseproperty.Thenextcolumnpresentsthetestgeneration time(original,inseconds) usingunmodiedzCha.Theseventhcolumnpresentsthetest geneationtimeusingour approach.Thenewtimeislargerforthebasepropertysincei tincludestheintersection calculationtimewithotherpropertiesinthecluster.Thes peedupiscomputedusingthe formula(OriginalTime/NewTime).Theoverallspeedupfort hisclusteris4 : 18x. Table4-1.Vericationresultsforastructure-basedclust er Prop.TypeBoundSizeForwardOrig.(s)New(s)Speedup p 13Inter.8461122-15.6121.990.71 p 28Edge7395566325768.310.1651.94 p 133Edge73955643257611.990.1866.60 p 134Inter.7395564325769.070.1947.74 p 150Node6330002217484.700.1629.38 p 165Path84611323512122.870.2784.70 p 170Path84611423512124.450.2694.04 Avg: -7.29414299-13.863.324.18 85

PAGE 86

Table4-2.Structure-basedclusteringresultsforMIPSpro cessor ClusterSizeBaseOriginalImprovedTimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 1101.2168.2632.015.911.78281.8483.2637.436.021.8831715.90193.212.4915.446.1841718.31173.203.8114.475.235715.6181.401.226.384.18672.03120.3840.055.712.56742.1515.945.792.621.71818.568.568.560.001.0091730.92582.8059.4417.575.69 10172.30149.7550.7412.832.3111710.54140.3130.776.783.1412179.40669.83164.3417.393.55131121.21365.7944.112.264.9914410.6246.583.843.543.18151415.84142.784.0011.475.0716132.65263.93149.1911.921.63 Avg: 10.6910.57194.1239.869.393.42 Table 4-2 providestheoverallvericationdetailsoftheclustersge neratedusingthe structuralsimilarity.Thetotal171propertiesaregroupe dinto16clustersshowninthe rstcolumn.TheexamplepresentedinTable 4-1 istheexpansionofthefthclusterin Table 4-2 (row5).Thesecondcolumnpresentsthesizeofthatclusteri ntermsofnumber ofproperties.Thebasetimeistheexecutiontimeofthebase property.Theoriginal timeistherunningtimeoftheremainingproperties(except thebaseproperty)without usinganyknowledgesharingtechniques.Sinceintersectio ncalculationisnecessarybefore executingthebaseproperty,weshowtheimprovedtime(oura pproach)intwoparts:new vericationtime,andoverhead(intersectioncalculation time).Thelastcolumnshows thespeedupusingtheformula(Basetime+Originaltime)/(B asetime+Improved time).Inthistable,wecanseethattheoverheadhasalinear relationwiththenumberof propertiesinthecluster.Usingstructuralclustering,we canachieveaspeedupof3 : 42x 3 3 Clusteringtimeusingstructuralsimilarityisnegligible andnotshowninthetable. 86

PAGE 87

4.5.1.2ClusteringbasedonTextualSimilarity Sincethepropertiesaregeneratedbasedonfaultmodels,th eyusesimilarformatand thereforehelpfulforclusteringbasedontextualsimilari ty.Inthiscase,weassumethat 50%isareasonablethresholdfortextualsimilarity.Forex ample,thefollowingproperties aretextuallysimilar.Inthiscase, p 49isthebaseproperty,andother6propertieshas50% similaritywithit.Sotheycanbeclusteredtogether. p 49= F ( m 1 active !=1& m 6 active !=1) p 50= F ( m 1 active !=1& m 7 active !=1) p 61= F ( m 2 active !=1& m 6 active !=1) p 72= F ( m 3 active !=1& m 6 active !=1) p 82= F ( m 4 active !=1& m 6 active !=1) p 91= F ( m 5 active !=1& m 6 active !=1) p 100= F ( m 6 active !=1& m 7 active !=1) Table4-3.Vericationresultsforatextualcluster Prop.TypeBoundSizeForwardOrig.(s)New(s)Speedup p 49Inter.10592239-59.5468.810.87 p 50Inter.116578067882681.095.8851.94 p 61Inter.105922397882660.720.31195.87 p 72Inter.105922397882662.370.31201.19 p 82Inter.105922397882661.910.31199.71 p 91Edge105922397882667.960.31219.23 p 100Edge116578067882684.176.0813.84 Avg: -10.29610972-68.2511.725.82 Table 4-3 showsthevericationdetailsforaclusterconsistingofab ove7properties. ThenumbersinthetableareinthesameformatasTable 4-1 .Duetoknowledgesharing, thespeedupforthisclusteris5 : 82x. Table 4-4 showsthetestgenerationdetailsforall32clustersusingt extualsimilarity. Table 4-3 istheexpansionofthe22 nd clusterofTable 4-4 (row22).Inthiscase,our approachisabletoobtaintheoverallspeedupof3 : 72. 87

PAGE 88

Table4-4.TextualclusteringresultsforMIPSprocessor ClusterSizeBaseOriginalImprovedTimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 110.110.110.1101.00210.120.120.1201.00310.350.350.3501.00410.350.350.3501.00531.284.622.571.531.10652.7515.636.023.341.52785.5672.6115.236.552.8681111.30183.4426.3110.574.0491117.72249.1940.5712.033.80 101030.58456.9748.4412.385.331110.300.300.300.001.001231.284.652.001.571.221352.6917.787.823.401.471485.0077.0421.916.622.4515114.7100.1934.179.162.181631.554.771.221.621.441752.7318.174.283.422.001821.211.841.420.970.85191715.67269.536.1816.457.3920137.74127.904.4911.245.782142.047.781.132.381.7722759.54418.2213.229.275.8223710.3469.919.165.823.1724329.0761.340.323.392.7625495.77288.450.615.663.7726621.63104.190.855.984.422744.0229.974.243.053.0028210.4610.500.151.721.7029518.6481.710.835.084.0930521.0778.806.615.223.0431322.2544.910.463.052.6132128.7828.7828.7801.00 Avg: 5.3413.6488.449.074.743.72 4.5.1.3Inruence-basedClustering Thefollowing7propertiesaregroupedusinginruence-base dclusteringwith p 111 asthebaseproperty.Wesetthethresholdofthesimilaritya s70%.Forinstance,the inruencenodesof p 111are f FET DEC MUL 1, MUL 2, MUL 3, MUL 4, MUL 5, MUL 6, MUL 7, FADD 1, FADD 2, FADD 3, FADD 4 g ,andtheinruenceof p 108is f FET DEC MUL 1, MUL 2, MUL 3, MUL 4, MUL 5, MUL 6, MUL 7, FADD 1 g .The similaritybetween p 108and p 111is10 = 13=77%. p 111= F ( m 7 active !=1& f 4 active !=1) 88

PAGE 89

p 104= F ( m 6 active !=1& f 4 active !=1) p 110= F ( m 7 active !=1& f 3 active !=1) p 103= F ( m 6 active !=1& f 3 active !=1) p 109= F ( m 7 active !=1& f 2 active !=1) p 102= F ( m 6 active !=1& f 2 active !=1) p 108= F ( m 7 active !=1& f 1 active !=1) Table 4-5 showsthevericationresultsforaninruence-basedcluste rconsistingofthe above7properties.Inthiscase,theoverallspeedupusingo urapproachis4 : 52x. Table4-5.Vericationresultsforaninruence-basedclust er Prop.TypeBoundSizeForwardOrig.(s)New(s)Speedup p 111Inter.10592239-54.8063.400.87 p 104Inter.95266876677325.980.22118.09 p 110Inter.105922397097554.260.25217.04 p 103Inter.95266876677325.830.22117.41 p 109Inter.105922397097549.160.25196.64 p 102Inter.95266876677333.270.22151.23 p 108Inter.105922397097549.740.26191.31 Avg: -9.57564145-41.869.264.52 Table 4-6 showsthevericationresultsusinginruence-basedcluste ringforall27 clusters.Thedetailsoftherstcluster(row1)isshowninT able 4-5 .Theoverallspeedup usingourapproachis4 : 30x. 4.5.1.4Intersection-basedClustering Intersection-basedclusteringisintuitiveandeasiertoi mplementsinceitdoesnot requireanypriorknowledgeaboutthestructureofthegraph modelortheformatof theproperties.Itonlyusesthemappingofthevariablesfor namesubstitutionandthe intersectionbetweentheCNFs.Duetouseofdatastructure hashmap ,theintersection timeislineartothesizeoftheCNFle.Thefollowingproper tiesaregroupedasacluster usingathresholdfortheintersectionas90%. p 50= F ( m 1 active !=1& m 7 active !=1) p 62= F ( m 2 active !=1& m 7 active !=1) 89

PAGE 90

Table4-6.Inruence-basedclusteringresultsforMIPSproc essor ClusterSizeBaseOriginalImprovedTimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 1754.80238.241.428.604.5221555.31874.0738.3819.188.23360.0772.3083.015.180.8241121.22173.934.8110.445.3551725.94570.7748.3619.226.386710.4962.394.895.923.427148.98188.1822.3912.644.48869.4119.760.864.451.9891711.76192.7520.4414.624.37 1074.0644.3310.765.292.411184.3949.227.265.913.0512424.2949.000.903.922.5213615.5473.460.725.744.051452.198.992.252.861.531562.1812.601.423.442.1016712.9884.548.656.453.4717619.4963.141.015.593.171824.581.830.111.271.081912.312.312.310.001.0020910.57107.5016.858.143.322121.540.350.080.740.8022318.2426.830.432.902.092310.350.350.350.001.002410.300.300.300.001.002511.211.211.210.001.002610.120.120.120.001.002710.120.120.120.001.00 Avg: 6.3311.94108.110.355.654.30 p 73= F ( m 3 active !=1& m 7 active !=1) p 83= F ( m 4 active !=1& m 7 active !=1) p 92= F ( m 5 active !=1& m 7 active !=1) p 100= F ( m 6 active !=1& m 7 active !=1) Table 4-7 presentsthevericationdetailsfortheaboveclusterusin g p 50asthebase property.Thespeedupforthisclusteris5 : 96x. Table 4-8 presentstheintersectionclusteringvericationforallt he171properties. Thedetailsofthe9 th clusterareshowninTable 4-7 .Theoverallspeedupusingour approachis5 : 90x. 90

PAGE 91

Table4-7.Vericationresultsforanintersection-basedc luster Prop.TypeBoundSizeForwardOrig.(s)New(s)Speedup p 50Inter.11657806-80.9189.410.90 p 62Inter.116578069154895.870.58165.29 p 73Inter.116578069154895.750.46208.15 p 83Inter.116578069154896.290.59163.20 p 92Inter.116578069154896.830.59164.12 p 100Inter.116578069154883.990.59142.36 Avg: -11657806-91.6115.375.96 Table4-8.Intersection-basedclusteringresultsforMIPS processor ClusterSizeBaseOriginalImprovedTimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 141.224.080.271.751.642131.8228.441.317.482.8531715.68266.612.7616.997.974177.72147.751.8014.516.475173.6566.502.0011.963.9861426.19383.102.2815.919.2271360.61691.412.6816.589.428178.51172.233.1014.207.009680.91468.732.818.505.96 101720.57323.982.7316.718.61111213.01120.282.1710.265.251244.7415.290.412.882.491320.110.110.040.300.491430.350.650.160.890.71151318.91249.842.4013.297.7716130.6330.6330.630117129.5429.5429.5401 Avg: 1019.07176.425.128.955.90 4.5.1.5ComparisonofClusteringTechnqiues Table 4-9 comparesthefourclusteringtechnqiues.Therstrowshows ourproposed clusteringmethods.Thesecondrowindicatesthenumberofc lustersusingtherespective clusteringmethods,andthethirdrowshowsthecorrespondi ngclusteringtime(in seconds).Thefourthrowpresentsthetestgenerationtimef orthebaseproperty.Similar totheprevioustables,theoriginaltimereferstotraditio nal(noclustering)verication timeforallthepropertiesexcludingthebaseproperty.The sixthrowpresentsthe vericationtimeforallthepropertiesexceptthebaseprop ertyusingtherespective clusteringmethod.Thespeedupiscomputedusingtheformul a(Basetime+Original time)/(Clusteringtime+Basetime+Improvedtime).Forthe rstthreeclustering 91

PAGE 92

methods,theclusteringisveryfastandtheassociatedcost (time)isnegligible.However, fortheintersection-basedclustering,theintersectiont imeislongercomparedtoother threemethodsandisnotnegligible.Therefore,forinterse ction-basedclustering,we providespeedupvaluesforbothscenarios{withoutconside ringclusteringtime(therst number)aswellaswithclusteringtime(thenumberinparent hesis). Table4-9.PropertyclusteringandvericationforMIPSpro cessor Methods StructureTextualInfluenceIntersection ClusterNo.16322717Clust.Time0.240.060.22187.90BaseTime169.09436.60322.44324.18Orig.Time3105.982830.132918.562999.16Impr.Time788.09442.53431.92239.28 Speedup3.423.724.335.90(4.42) Itisimportanttonotethatintersection-basedclustering ismostbenecialfor reducingoveralltestgenerationtime.However,thecluste ringoverheadismuchmorethan otherstrategies.Whenalargenumberofcomplexproperties areinvolved,theintersection overheadmaybecomeprohibitivelylarge.Insuchcases,inr uence-basedclusteringismost benecial.Interestingly,textualclusteringconsumesle astamountofclusteringtimebut generatesbetterresultsthanstructurebasedclustering. Whendetailedinformationabout thedesignisnotavailable,textualclusteringismostbene cial. 4.5.2AStockExchangeSystem Thissectionpresentsthetestgenerationresultsoftheonlinestockexchangesystem (OSES)(describedinSection 2.3.5 ).Thespecicationisusedtogenerate51properties basedonthefaultmodel.Weappliedtheclusteringmethodsd iscussedinSection 4.3 on allthepropertiestogeneratethetests. Table 4-10 presentsthetestgenerationresultsusingstructure-base dclusteringforall the51properties.Theoverallspeedupusingourapproachis 2 : 26x. Table 4-11 presentsthetestgenerationresultsusingtextualcluster ingforallthe51 properties.Theoverallspeedupusingourapproachis2 : 33x. 92

PAGE 93

Table4-10.Structure-basedclusteringresultsforOSES ClusterSizeBaseOriginalImprovedtimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 124.483.720.630.971.35246.1445.513.131.922.44321.762.030.600.971.144459.56160.9915.161.902.88529.3411.0919.580.980.686410.74123.795.971.957.21720.400.320.250.970.448496.44150.4531.111.911.91926.627.400.711.131.66 10410.0882.6148.022.261.541123.364.691.221.131.41124101.16154.6238.482.221.8013229.5536.52.901.141.97144106.51168.302.242.241.951520.210.2019.341.140.0216495.91588.49120.002.263.1417218.9115.531.160.821.651810.880.880.880.001.00 Avg: 2.8331.2386.5119.511.442.26 Table4-11.TextualclusteringresultsforOSES ClusterSizeBaseOriginalImprovedtimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 110.680.680.680.001.002215.5518.867.730.811.43394.33196.5960.884.262.894860.25135.3736.833.801.945133.5733.5733.570.001.006611.62246.232.052.8615.60796.44469.61130.685.013.358810.61155.8295.904.501.50970.21760.38390.693.911.93 Avg: 5.6715.87224.1284.332.792.33 Table 4-12 presentsthetestgenerationresultsusinginruence-based clusteringforall the51properties.Theoverallspeedupusingourapproachis 2 : 44x. Table 4-13 presentsthetestgenerationresultsusingintersection-b asedclusteringfor allthe51properties.Theoverallspeedupusingourapproac his2 : 84xwithoutconsidering clusteringoverhead.Ifclusteringoverheadisconsidered theoverallspeedupis2 : 69x. Table 4-14 summarizestheresultsusingfourclusteringmethodswhere 2{3times improvementisachieved.Itisimportanttonotethattheres ultsforOSESareconsistent withtheresultsforMIPSinTable 4-9 .AsTable 4-14 shows,intersection-basedclustering 93

PAGE 94

Table4-12.Inruence-basedclusteringresultsforOSES ClusterSizeBaseOriginalImprovedtimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 1522.97147.8450.482.752.242810.10369.97120.274.402.823336.6259.6538.781.691.264510.66135.9811.372.376.01540.324.003.281.900.786193.4893.4893.4801.007728.89629.39132.413.893.988212.879.850.370.981.589614.23302.63115.312.832.40 10734.66261.8069.813.342.7511215.8718.987.630.811.431210.750.750.7501.00 Avg: 4.2523.12169.5053.652.082.44 Table4-13.Intersection-basedclusteringresultsforOSE S ClusterSizeBaseOriginalImprovedtimeSpeedup Index(#Prop)Time(s)Time(s)Verify(s)Overhd(s) 174.8453.9116.643.312.372310.9394.796.21.465.69327.1356.725.810.984.594235.3268.9624.970.981.70535.0620.6022.561.450.886784.18243.6022.783.302.97786.54393.75147.454.532.53863.3798.4642.393.322.079329.4568.7119.071.741.95 103107.27457.5239.591.693.801140.20247.4662.832.243.7912218.7415.351.170.821.641310.70.70.701.00 Avg: 3.9224.13140.0431.701.992.84 ismostbenecialforreducingoveralltestgenerationtime .However,whenclustering overheadisprohibitivelylarge,inruence-basedclusteri ngisbenecial.Similarly,when detailedinformationaboutthedesignisnotavailable,tex tualclusteringisthebestchoice. Table4-14.PropertyclusteringandvericationforOSES Methods StructureTextualInfluenceIntersection ClusterNo.1891213Clust.Time0.050.010.0542.77BaseTime562.05142.81277.42313.73Orig.Time1557.112017.112034.051820.53Impr.Time377.15784.16668.72437.98 Speedup2.262.332.442.84(2.69) 94

PAGE 95

Ontwocasestudies(MIPSandOSES)ourapproachdemonstrate d3{5times improvementinoveralltestgenerationtimeusingecienti ntegrationofproperty clusteringandconrictclauseforwardingbasedlearningte chniques. 4.6Summary Directedtestvectorscanreduceoverallvalidationeorts incefewertestscanobtain thesamecoveragegoalcomparedtotherandomtests.Theappl icabilityoftheexisting approachesfordirectedtestgenerationislimitedduetoca pacityrestrictionsofthe automatedtools.Thischapteraddressedthetestgeneratio ncomplexitybyclustering similarpropertiesandexploitingthecommonalitiesbetwe enthem.Toenableknowledge sharingacrossmultipleproperties,wehavedevelopedanum berofconceptuallysimple, butextremelyeective,techniquesincludingnamesubstit utionandselectiveforwarding oflearnedconrictclauses.Ourexperimentalresultsusing bothhardwareandsoftware designsdemonstratedanaverageoffourtimesspeedupindir ectedtestgenerationtime. 95

PAGE 96

CHAPTER5 DECISIONORDERINGBASEDINTRA-ANDINTER-PROPERTYLEARNIN G Theprimarygoalofecienttestgenerationishowtoquickly getsatisable assignmentsforSATinstances.Variousheuristicmethodsa ndtools[ 39 63 ]areproposed toimprovetheSATsearchingtime. Decisionordering [ 55 ]playsanimportantroleduring thesearchbecausedierentdecisionorderingimpliesdie rentdecisiontreeaswellas dierentsearchpathwhichstronglyaectthesearchtime.E xistingdecisionordering methodsfocusonexploitingtheusefulinformationofgener alSATproblemwitha singleSATinstance.Mostofthemarebasedonthestatistics ofSATinstanceswithout consideringanyotherlearninginformation.Fortestgener ation,adesignmayhavevarious propertiesandgenerallymodelcheckingtechniqueswillch eckeachofthemindividually. Foragivendesign,similarpropertiesdescribecorrelated functionalscenarios.Therefore therespectivecounterexamplesareexpectedtohaveasigni cantoverlapwhichcanbe usedforsharinglearning.Furthermore,evenforasingleSA Tinstance,theresultofthe localsearchcanalsobenettheglobalsearch.Themethodpr oposedinthischapter exploitsthelearningfromdecisionorderinginthecontext oftestgenerationinvolvingone ormorepropertiesofadesign.Thischaptermakesthreecont ributions:i)investigatesthe decisionorderingbasedlearningforasingleSATinstance; ii)appliesthedecisionordering basedlearningbetweensimilarSATinstances;andiii)expl oitstherelationbetweenthe decisionorderingandconrictclauseforwardingbasedmeth ods. Therestofthechapterisorganizedasfollows.Section 5.1 presentsrelatedworkon decisionorderingbasedheuristics.Section 5.2 describesourlearningtechniquesbasedon decisionordering.Section 5.3 proposesthetestgenerationmethodologyusingecient decisionorderingtechniques.Section 5.4 presentstheexperimentalresults.Finally, Section 5.5 summarizesthechapter. 96

PAGE 97

5.1RelatedWork Dierentvariableorderingwillleadtodierentsearchtre es,thereforebranching heuristicscanimprovetheSATsearchingperformancesigni cantly[ 55 ].Asapopular SATsolver,zChausestheVariableStateIndependentDecay ingSum(VSIDS)heuristic [ 63 ].Thisheuristiccontainstwoparts:i)thestaticpartcoll ectsthestatisticsofthe ConjunctiveNormalForm(CNF)literalspriortoSATsolving andsetstheinitialdecision ordering,andii)duringtheSATsolving,thedynamicpartpe riodicallyupdatesthe prioritybasedonconrictclauses.Althoughtheabovegener al-purposeheuristicsare promisingforpropositionalformulas,theyneglectsomeun iqueinformationofBMC. In[ 81 ],StrichmanexploitedthecharacteristicsoftheBMCformu lasforavarietyof optimizationsincludingdecisionordering.Whenthebound isunknown,SAT-basedBMC needstoincreasetheunrollingdepthone-by-oneuntilndi ngacounterexample.Wanget al.[ 87 ]analyzedthecorrelationamongdierentSATinstancesofa property.Theyused the unsatisablecore ofpreviouslycheckedSATinstancestoguidethevariableor dering forthecurrentSATinstance. Tothebestofourknowledge,alltheexistingapproachesexp loitvariableorderingto improvetheSATsolvingtimeinvolvingonlyoneproperty(on eSATinstanceorseveral correlatedSATinstanceswithdierentbounds).Ourapproa chistherstattempttouse bothdecisionorderingandconrictclausestoreducetheBMC basedtestgenerationtime forasingleSATinstanceaswellasforaclusterofsimilarSA Tinstances.Thecomparison betweenvariouslearningtechniquesisprovidedinSection 5.4 5.2DecisionOrderingBasedLearnings DecisionorderingplaysanimportantroleduringtheSATsea rch.Itindicateswhich variablewillbeselectedrstandwhichvalue(trueorfalse )willberstassignedto thisvariable.SimilartoBDDbasedmethods[ 15 ],variableorderingdeterminesthe performanceoftheSATsolvingtime.IntheVSDISheuristics implementationofzCha, eachliteral l isassociatedwitha zcha score(l) whichisusedfordecisionorderingat 97

PAGE 98

decide next branch() (seeAlgorithm 3 inChapter 4 ).Initiallythescoreisequaltothe literalcountincorrespondingCNFle.DuringtheSATsolvi ng,thescorewillbeupdated inperiodicfunctionafteracertainnumbersofbacktracks. Thecalculationofthenew literalscoreisasfollows: chaff score ( l )= chaff score ( l ) = 2+ lits in new confs ( l )(5{1) where lits in new confs(l) isthenumberofnewlyaddedconrictclauseswhichcontain literal l sincelastupdate. Similarpropertiesusuallyhavesimilarcounterexamplesw hichindicatesthatthey mayhavesimilarBooleanconstraintsduringthetestgenera tion.Consequentlythe generatedSATinstancesshouldhavealargeoverlapinCNFcl ausesandcanbeclustered tosharethelearning.Thissectionpresentsourdecisionor deringheuristicwhichwillbe incorporatedinthetestgenerationapproachesinSection 5.3 5.2.1Overview AsdiscussedinSection 4.2.1 ,themosttimeconsumingpartsareBCPandlong distancebacktracking.Theyareindicatedbyimplicationn umberandconrictclause numberwhichrepresentthesuccessfuldecisionratioandba cktracknumberrespectively. Ideally,asearchmethodcangetasatisableassignmentbym akingtheassignmentfor eachvariableonlyonce.However,generallyitisimpossibl etoachievesuchscenario.For aclusterofsimilarpropertiesandpre-determinedbounds, theobjectiveofourmethod istoreducethenumberofimplicationsandconrictclauseso funcheckedpropertiesby incorporatingthelearneddecisionorderingknowledgefro mpreviouslycheckedproperties. Assumingthatwehavetwosimilarproperties,bothproperti eswillhavealarge overlaponCNFclausesandcounterexampleassignments.Fig ure 5-1 showsthepartial viewsofsearchtreesandsearchpathsofthetwoproperties. Thesearchpathsareformed accordingtothedecisionordering(shownontopofthesearc htrees).Foreachvariable v intheordering,therearetwoliterals( v means v=1 and v' means v=0 ).Asshownin 98

PAGE 99

Figure 5-1 a,thereare6conrictsencountered.Thesearchstopsafter ndingasatisable assignment a =1, b =0, c =0, d =1inthisscenario.InFigure 5-1 b,thesearchwillbe successfulonlywhen a =0, b =0, c =0, d =1afterencountering14conricts.Therefore thesearchofthesecondexamplewillbemoretime-consuming becauseofmorebacktracks. 10 0 1 1 0 1 0 1 0 0 1 1 0 1 0 10 1 0 1 0 1 0 1 0 1 0 b Ordering: a, a', b, b', c, c', d, d'Ordering: a, a', b, b', c, c', d, d' Search Path Variables: a, b, c, d X: Conflict b) Partial view of the second example a) Partial view of the first example : Success X X XX XX 0 0 00 0 0 0 0 1 1 1 1 111 1 11 1 1 1 1 1 1 0 00 0 0 00 0 X X X X X X X X X XXXX X a b c c c c d d d d d d dd d d d d dd dd a b b c c c c Figure5-1.TwoexamplesofSATsearch Becauseofthelargeoverlapintheassignmentofcounterexa mples,theresultof previouslycheckedpropertiescanbeusedasalearningforu ncheckedproperties.For example,inFigure 5-1 ,theresultofrstexamplestronglyindicatestheassignme ntofthe secondexamplebecauseofthesatisableassignmentinters ection b =0, c =0, d =1. Ifthesecondexampleusesthedecisionorderingbasedonthe variableassignmentsin therstexample,thesearchingtimeofthesecondexampleca nbedrasticallyreducedas showninFigure 5-4 5.2.2BitValueOrdering Similarpropertiesgenerallyhavealargeintersectiononb othcorrespondingCNF clausesandcounterexampleassignments.Thisindicatesth atthesatisableassignmentof checkedSATinstancescontainrichdecisionorderingknowl edgeforuncheckedsatisable SATinstance.InSATsearch,incorrectvalueselectionfore achvariablewillcauseconricts whichwillresultinbacktrackstoremovethereasonoftheco nricts.Agooddecision 99

PAGE 100

orderingcanmostlyavoidsuchfaultyassignments.Unlikep runingthesearchtreeusing conrictclauseforwarding[ 58 ],bitvalueorderingchangesthe searchpath .Bysettingthe bitpriority (choose0or1rst)foreachvariableusingtheknowledgeofp reviousproperty checking,thelengthofthesearchpathcanbereduced. 1 1 1 1 0 1 a 0 0 1 0 0 1 1 1 1 0 1 b b 0 0 1 0 0 Ordering: a, a', b, b', c, c', d, d' a=1, b=0, c=0, d=1 0 1 0 0 0 1 Learned assignment: Ordering: a, a', b', b, c', c, d, d' a) Without bit-value ordering b) With bit-value ordering 11 11 1 1 1 1 1 1 11 11 1 1 0 0 0 0 0 0 0 0 0 0 0 00 0 0 000 0 X X X X X X X X X X X XX X X X X X XX X X cc c c b b d d d dd d d d a d c c c c d dd d d d d Figure5-2.Ascenariowherebit-valueorderingworks Figure 5-2 showsanexamplewherebit-valueorderingworks.Asshownin Figure 5-1 a, wecangetasatisableassignment a =1, b =0, c =0and d =1.Thisassignmentcan beusedtochangethebit-valueorderingofthesecondexampl e.Thatmeans,whennode b isencountered,thesearchchooses b =0rstinitssearchpath.Thesamerulealso appliesonothernodes.ApplyingsuchheuristicinFigure 5-2 b,thereareonly8conricts encounteredcomparedto14conrictsinFigure 5-2 a.Inaddition,thesearchpathisalso shortened.Therefore,thesearchingtimeisreduced. Itisimportanttonotethatthebit-valueorderingitselfis notalwayshelpfulfor theSATsearching.ForexampleinFigure 5-3 a =1, b =1, c =0, d =1istheonly satisableassignmentinthegivenscenario.Thesearching inFigure 5-3 awithoutbitvalue orderingisfasterthanthesearchinginFigure 5-3 bbecauseoflessconricts.Ifthelearning assignmentinFigure 5-3 wasa=0,b=1,c=0andd=1,thesearchingperformancewill bemuchworsethanthesearchinFigure 5-3 b.Clearly,inthesearchtree,thehighlevel 100

PAGE 101

1 1 1 1 0 1 a 0 0 1 0 0 1 1 1 1 0 1 b b 0 0 1 0 0 Ordering: a, a', b, b', c, c', d, d' a=1, b=0, c=0, d=1 0 1 0 0 0 1 Learned assignment: Ordering: a, a', b', b, c', c, d, d' a) Without bit-value ordering b) With bit-value ordering 11 11 1 1 1 1 1 1 11 11 1 1 0 0 0 0 0 0 0 0 0 0 0 00 0 0 000 0 X X X X X X X cc c c b b d d d dd d d d a d c c c c d dd d d d d Figure5-3.Ascenariowherebitvalueorderingfails variables(e.g.,node a )stronglyaecttheperformanceofthesearchingiftheyare not consistentwithlearnedbit-valueordering.5.2.3VariableOrdering Althoughbit-valueorderingispromisingingeneral,there arestillalotofconricts encounteredduringthesearch.Accordingtotheexamplesho wninFigure 5-3 ,ifhighlevel nodes(e.g.,node a )makethewrongdecision,thesearchpathwillbelengthened dueto thelongdistancebacktrack.Toreducethesearchingtime,i tisnecessarytorestrictthe conrictdetectionandreasoninginasmallarea. 1 1 1 1 0 1 a 0 0 1 0 0 1 1 1 1 0 1 c c 0 0 1 0 0 Ordering: a, a', b, b', c, c', d, d' a=1, b=0, c=0, d=1 0 1 0 0 0 1 Learned assignment: Ordering: b, b', c', c, a, a', d, d' 11 11 1 1 1 1 1 1 11 11 1 1 0 0 0 0 0 0 0 0 0 0 0 00 0 0 000 0 X X X X X X XX X X X X X X cc c c b b d d d dd d d d b d a a a d dd d d d d a X X a) Without any learningb) With bit-value and variable ordering Figure5-4.Anexampleofbit-valueandvariableordering 101

PAGE 102

Ecientcombinationofvariableorderingandbit-valueord eringisverypromising.As showninFigure 5-4 b,thesearchtimeisbetterthanthatinFigure 5-4 aduetoashorter searchpathandlessconricts.Thereasonofthisimprovemen tisthatweenhancethe priorityofvariables b and c .Since a isthevariablewithdierentvaluesbetweenthetwo satisableassignmentsshowninFigure 5-1 ,loweringdownthepriorityofsuchvariables (oneswithdierentvaluesbetweentwoCNFs)canecientlya voidthelongdistance backtrack.Generally,beforeSATsolving,itishardtogur eoutthedierencebetween twosatisableCNFvariableassignments.However,basedon thevalueassignment statisticsofthecheckedproperties,thevariableorderin gcanbeconstructed.Fora variablewiththelowerassignmentvaluevariation,whichi ndicateshighchanceofsame value,wewillenhanceitsprioritybyincreasingthescoreo fitstwoliterals. 1 1 1 0 1 0 0 1 0 0 0 1 1 111 0 0 0 0 0 a c c c d d d d d X X b b 1 1 1 1 1 0 1 a 0 0 1 0 0 0 1 0 11 11 1 1 1 1 0 0 0 00 0 0 000 0 X X X X X X XX cc c c b b d d d dd d d d a=1, b=0, c=0, d=1 Learned assignment: Ordering: a, a', b', b, c', c, d, d' Ordering: a, a', b', b, c', c, d, d' b) Bit-value ordering + Conflict Clauses a) With bit-value ordering Figure5-5.Anexampleofconrictclausesbasedvariableord ering 5.2.4ConrictClausebasedDecisionOrdering(Hybrid) Conrictclauseispromisingtoavoidrepeatedconrictsduri ngtheSATsearching. Thereforeitcanbeusedasalearningduringthetestgenerat ion(describedinChapter 4 ). Inessence,conrictclauseforwardingcanbeusedtopruneth edecisiontreeandcanbe utilizedasacomplementaryapproachforthedecisionorder ingtechniquesproposedin Section 5.2.2 andSection 5.2.3 .FortwosimilarSATinstances,iftheconrictclausesofthe 102

PAGE 103

checkedSATinstancecanbeforwardedtotheuncheckedone,i twillreducetheconricts, thusfurthershortenthesearchpath. Figure 5-5 ashowsapplicationofbit-valueorderingontheexamplesho wnin Figure 5-1 b.Thereare8conrictsduringtheSATsearchinthiscase.Let 'sassume theconrictclausesgeneratedfromFigure 5-1 acanbeforwardedtotheCNFclausesof Figure 5-1 b.Thegenerated6conrictclausesareasfollows: ( a 0 b 0 c 0 d 0 ) ( a 0 b 0 c 0 d ) ( a 0 b 0_ c d 0 ) ( a 0 b 0 c d ) 9>>>>>>>=>>>>>>>; ) ( a 0 b 0 ) ( a 0 b c 0 d 0 ) ( a 0 b c 0 d ) 9>=>; ) ( a 0 b c 0 ) (5{2) Equation 5{2 showstheresolutionoftheforwardedconrictclauses.Base don theresult,wecanprunethesearchtreeasshowninFigure 5-5 b.Itindicatesthat thereareonly2conrictsbyapplyingthebitvalueorderingo ntheprunedsearchtree. Thereforethetestgenerationtimecanbesignicantlyredu ced.Fortheexampleshown inFigure 5-4 b,theconrictclauseforwardingisnotbenecialsincethes earchdoesnot traversetheprunedpartofthedecisiontree.Generally,th econrictclauseforwardingcan furtherimprovetheperformanceofthedecisionorderingba sedmethods. 5.3TestGenerationusingDecisionOrdering Formodelcheckingbasedtestgeneration,eachpropertyisa negationofadesired systembehavior.Consequentlyeachpropertycanproduceac ounterexample.Since ourmethodadoptsSAT-basedBMC,weassumethattheboundcan bepre-determined andthegeneratedSATinstancesaresatisable.Thegoaloft hetestgenerationfor thepropertywithaknownboundistogureoutasatisableas signmentforthisSAT instance. 103

PAGE 104

Toreducetheoveralltestgenerationeort,thissectionut ilizestheheuristics proposedinSection 5.2 asalearning.Section 5.3.1 appliesthelearningbasedonthe decisionorderingfortestgenerationofasingleproperty. InSection 5.3.2 ,wepresentan algorithmwhichshareslearningfromthedecisionordering amongaclusterofsimilar properties.5.3.1TestGenerationforaSingleProperty Whencheckingabasepropertyusingpropertyclusteringtec hniques,orwhen checkingonlyasingleproperty,currentmethodssolvetheS ATinstancealonesincethere isnosourceoflearning.Thereforeitistime-consumingand itcanbeamajorbottleneck oftheclusteringbasedtestgeneration. Duringtestgeneration,iftheboundofapropertyisincreas edbyone,thecomplexity willbedrasticallyincreased.Basedontheobservationof[ 81 ],thereasonoftime-consuming searchisduetothelongdistancebacktracking.Sincelarge setofclausesthatbelongto dierentdistantcyclesarebeingsatisedindependently( locally),[ 81 ]foundthatthereare threetypicalscenarioswhichcancausetheconricts: Distantcyclesarebeingsatisedindependentlyuntilthey collideeachotherwith assignmentconrict. Somecycleassignmentcollideswiththeconstraintsimpose dbytheinitialstate. Somecycleassignmentcollideswiththeconstraintsimpose dbythenegationofthe speciedproperty. Theresolutionofsuchconrictsneedstocancellargenumber ofvariableassignment betweentheconrictingcycles.EspeciallyfortheSATinsta ncewithlargebound,thecost ofnon-chronologicalbacktrackingisstillhugesincelarg eboundindicateshugenumberof clausesandvariables. Toalleviatelongdistancebacktrackingsduringtestgener ation,learningisrequiredto guidetheSATsearch.Conrictclauseisapromisinglearning thatcanprunethedecision tree.However,inaSATinstancewithlargebound,thecostof derivingaconrictclauseis 104

PAGE 105

costlyduetolargeinterleavingofirrelevantvariablesdu ringtheSATsearch.Furthermore, largesetofCNFclausesislikelytogeneratealargenumbero fconrictclauseswhich canaectthesearchperformance.Thereforeifwecangetcon rictclausesfromasmaller SATinstance,thentheaveragecostofconrictclausegenera tionwillbereduced.Asan alternative,decisionorderingcanbeusedaslearning.Sin cetheSATinstanceisassumed tobesatisable,eachsegment 1 oftheCNFclausesshouldbesatisable.Thesearching timeforasegmentismuchshorterthantheoriginalSATinsta nce.Althoughasegment cannotrerecttheglobalviewofthesystem,ifthesatisabl eassignmentofthesegmentis consistenttothepartialvariableassignmentoftheorigin alSATinstance,itwillbehelpful toreducetheoveralltestgenerationtimeoftheoriginalSA Tinstance. 5.3.1.1HeuristicImplementation Thebasicideaofourheuristicfortestgenerationinvolvin gasinglepropertyis tousethelearningsfromasmallpartoftheSATinstancetogu idethesearchofthe wholeSATinstance.BydividingtheSATinstanceintotwoseg ments,wecangetthe rstsegmentwhichcontainstheinitialstateconstraintsa ndthesecondsegmentwhich containspropertyconstraints.Aftercheckinganyoneofth em,wecangetthepartial variableassignmentswhichcanbeusedasdecisionordering learning,andwecangetthe conrictclauseswhichcanbeforwardedtotheoriginalprope rtyaccordingtoTheorem 2 in Chapter 4 Figure 5-6 demonstratesanexampleofusingsuchlearnings.InFigure 5-6 b,werst checkonepartoftheSATinstanceandgetthecorrespondingl earnings.Thenduringthe checkingofwholeSATinstance,undertheguidanceofthelea rnedknowledge,theoverall searchpathisshortenedcomparedtoFigure 5-6 a. 1 ACNFSATinstancecanbeviewedasaunionofasetofsegmentsw hereeach segmentconsistsofasetofCNFclauses. 105

PAGE 106

1st search 2nd search 1st search trace Search trace 2nd search trace a) A search without any learnings b) A search with two kinds of learnings Figure5-6.Learningtechniquesforasingleproperty Ourdecisionorderingheuristicsimplementationusesanar ray var [ sz ]( sz isthe largestvariablenumberforCNFs)toindicatethesatisabl eassignmentresultofthe rstsearch.Eachelementofthearray var [ i ](0 >>>><>>>>>: max ( v i )( var [ i ]==1& l i = v i ) or ( var [ i ]==0& l i = v 0 i ) chaff score ( v i ) otherwise (5{3) 5.3.1.2TestGeneration Algorithm 7 describesourtestgenerationprocedureforasingleproper tyusing learningsfromsomepartoftheSATinstancecorrespondingt otheoriginalproperty.Step 1initializesalltheelementsof var with-1.Step2generatestheCNFclausesforthe propertyp.Afterdividingthe CNF intotwopartsinstep3,step4solvestheclausesin anyonepartandderivesthelearningintheformofdecisiono rderingandconrictclauses. Step5updatesthe var .Finally,step6usesthelearningtoguidethetestgenerati onof theoriginalproperty. 106

PAGE 107

Algorithm7 :TestGenerationforaSingleProperty Input :i)Formalmodelofthedesign, D ii)Property p withbound b Output :Atest t for p withgeneratedconrictclauses 1. Initialize var ; 2. CNF = BMC ( D;p;b ); 3. Divide CNF into CNF 1 and CNF 2 ; 4. ( assign conf clauses 1)=SAT( CNF 1 or CNF 2 var ,NULL); 5. Update var using assign ; 6. ( t conf clauses 2)=SAT( CNF var conf clauses 1); return( t; conf clauses1+conf clauses2); Itisimportanttonotethatourheuristicforasingleproper tyisbasedonthe assumptionthatthedecisionorderingknowledgelearnedfr omtherstsearchhasalarge overlapwithasatisableassignmentofthesecondsearch.A lthoughtheforwardedconrict clausescanprunethedecisionspace,itisstillpossibleth attherstsearchmaymislead thesecondsearchwhichwillaggravatetheoverallsearchin gtime.Sincewehalvethe SATinstanceandeachpartcanbecheckedindividually,fort estgeneration,weusethe followingthreestrategiesinparallel: DirectlysolvetheoriginalSATinstance. Solvetherstpartandusethelearningstosolvetheorigina linstance. Solvethesecondpartandusethelearningstosolvetheorigi nalinstance. Onceoneoftheabovemethodsndsasatisableassignment,t heremainingtwo processeswillbeterminated.Therefore,wecanguaranteet heworstcaseofthetest generationtimeisthesameasdirectlysolvingtheoriginal SATinstance. 5.3.2TestGenerationforaClusterofSimilarProperties Forsimilarproperties,thereexistsalargeoverlapbetwee ncorrespondingcounterexamples. Thereforethesatisableassignmentsofcheckedpropertie scanbeusedasalearningfor 107

PAGE 108

otherpropertiesinthecluster.Someofthederivedconrict clausescanalsobeforwarded aslearning.Thissub-sectionwilldiscusshowtoextractth ebit-valueorderingand variableorderingbasedlearningsfromthecheckedpropert iesindetails.Alsowewill describeanalgorithmtoutilizethelearningbasedondecis ionorderingfortestgeneration ofaclusterofsimilarproperties.5.3.2.1HeuristicImplementation Inourheuristicimplementation,wepredictthedecisionor deringbasedonthe statisticscollectedfromthecheckedproperties.Let varStat [ sz ][2]( sz isthelargest variablenumberforCNFs)bea2-dimensionalarraytokeepth ecountofvariable assignments.Initially, varStat [ i ][0]= varStat [ i ][1]=0(0
PAGE 109

thatvariables a and b aremorelikelytobe0, c ismorelikelytobe1and d canbe assignedanyvalue.Furthermore, varStat impliesthattheassignmentsforvariable a b and c aremoreconsistentthantheassignmentforvariable d .Thusthescoreofvariable a b and c willbeincreased.Inotherwords,theywillbesearchedrst asdescribedin Section 5.2.3 Assuming l i isaliteralof v i ,weusethefollowingequationtopredictthebitvalue assignmentof v i whenchecking p j +1 potential ( l i )= 8>>>>><>>>>>: 1( varStat [ i ][1] >varStat [ i ][0]& l i = v i ) or ( varStat [ i ][1] <>: max ( v i ) ratio ( i ) pontential ( l i )=1 score ( l i ) ratio ( i ) otherwise (5{6) 109

PAGE 110

5.3.2.2TestGeneration Algorithm8 :TestGenerationforAPropertyCluster Input :i)Formalmodelofthedesign, D ii)Propertycluster, P ,withsatisablebounds Output : Test-suite 1. Initialize varStat ; 2. Selectthebaseproperty p 1 andgenerateCNF, CNF 1 ; for i isfrom2tothesizeofclusterP do 3. GenerateCNF, CNF i = BMC ( D;p i ;bound i ); 4. INT i = ComputerIntersection ( CNF 1 ;CNF i ); 5. Marktheclauseof CNF 1 using INT i ; end6. ( test 1 conf clause )=Algorithm 7 ( D p 1 bound 1 ); Test-suite = f test 1 g ; for i isfrom2tothesizeofclusterP do 7. Update varStat using test i 1 ; /*Figureoutthelearnedconrictclausesfrom p 1 */; 8. CC i = Filter ( conf clause;i ); 9. ( test i ,)=SAT( CNF i vatStat CC i ); Test-suite = Test-suite [ test i ; endreturn Test-suite ; Algorithm 8 describesourtestgenerationmethodology.Theinputsofth ealgorithm areaformalmodelofthedesignandaclusterofsimilarprope rties.Therststep initializes varStat whichisusedtokeepstatisticsofthevariableassignments .Step2 generatestheCNFclausesforthebaseproperty p 1 .Step3generatestheCNFclausesfor otherproperties.Afterguringouttheintersectionbetwe enthebasepropertywithother propertiesinstep4,step5markstheclausesofbasepropert y(themarkingisusedfor conrictclauseidenticationinstep8).Step6solvestheba sepropertyusingAlgorithm 7 110

PAGE 111

andgeneratesatestaswellasthe conf clause whichcanbeusedaslearningsforthe testgenerationoftheremainingpropertiesintheclusteru singsteps7-9.Aftersolving eachproperty,weneedtoupdatethe varStat instep7.Step8ndstheproperconrict clauseswhichcanbeforwardedtothecurrentproperty.Step 9solvesthecurrentproperty usingthelearningsbasedonconrictclausesanddecisionor dering.Finally,thealgorithm reportsallthegeneratedcounterexamples(tests).Itisim portanttonotethatthis algorithmcombinesboththeconrictclauseanddecisionord eringbasedlearnings.If onlydecisionorderinglearningisused,steps4,5,8should beomitted.Similarly,ifonly conrictclauseforwardingisapplied,thenstep7shouldbeo mitted. 5.4CaseStudy Thissectionpresentscasestudiesforecienttestgenerat ionusingourdecision orderingaswellasconrictclausebasedheuristics.Sectio n 5.4.1 presentsthecasestudies usingintra-propertylearningsforcheckingindividualSA Tinstances.Thebenchmarks collectedareallpre-generatedsatisableSATinstances. Byusinginter-propertylearning foraclusterofsimilarSATinstances,Section 5.4.2 presentstwocasestudies:aVLIW implementationoftheMIPSarchitecture(describedinSect ion 2.3.2 )andthestock exchangesystem(describedinSection 2.3.5 ).WeusedNuSMV[ 27 ]togeneratetheCNF clauses(inDIMACSformat).WemodiedtheSATsolverzCha[ 74 ]toincorporate ourproposeddecisionorderingheuristicontopofVSDIS.Th eexperimentalresultsare obtainedonaLinuxPCusing2.4GHzCore2DuoCPUwith2GBRAM.5.4.1Intra-PropertyLearning Thebenchmarksarecollectedfrom[ 83 ]and[ 85 ].In[ 83 ],thereare13SATinstances giveninthebenchmarksetwhicharealltakenfromrealindus trialhardwaredesigns (contributionofIBMresearchandGalileo).Wechosefourco mplexinstancesfromthem, becausemostSATinstancesprovidedin[ 83 ]takeshorttimeduringfalsication.Apart fromthesefourbenchmarks,wealsochosethebenchmarksoft wocomplexdesignsfrom 111

PAGE 112

[ 85 ]asfollows.Sincewearefocusingontestgeneration,theco llectedSATinstancesare allsatisable. VLIW-SAT-4.0 ,buggyVLIWprocessorswithinstructionqueuesand9-stage pipelines;theprocessorssupportadvancedloads,predica tedexecution,branch prediction,andexceptions. PIPE-SAT-1.1 ,buggyvariantsofthepipebenchmarksaspresentedin[ 86 ]. Fortheintra-propertylearning,wedivideeachSATinstanc eintotwosegments withthesamenumberofclauses.Table 5-1 showsthetestgenerationdetailsusing variousintra-propertylearningtechniques.Therstcolu mnshowsthenamesoftheSAT instances.ThesecondandthirdcolumnsindicatetheCNFsiz einformationincluding thevariablenumberandclausenumber.Thefourthcolumnind icatesthecheckingtime bydirectlyusingzChawithoutanyotherlearninginformat ion.Thefthcolumnshows thecheckingtimeusingintra-propertylearningbasedonco nrictclauseforwarding, andthesixthcolumnshowsthetestgenerationtimeusingour decisionorderingbased heuristics.Theseventhcolumnpresentstheresultwhichin corporatesbothconrictclause forwardinganddecisionorderingtechniquesasdescribedi nSection 5.2.4 .Sincewerun dierentmethodsondierentcomputerswiththesamesettin gs,whenonemachinegets thesatisableassignment,alltheremainingSATsearcheso ntheothermachineswillbe terminated.ThereforetheSATsearchingtimeistheminimum searchingtimeamong thesetechniques.Basedonsuchminimumtime,thelastcolum nindicatesthemaximum speedupusingthefollowingformula: speedup = MIN ( zChaff;ConflictClause;DecisionOrdering;Hybrid ) zChaff (5{7) where zChaff ConflictClause DecisionOrdering and Hybrid indicatetheresultsof columns4-7inTable 5-1 ,respectively. Itisimportanttonotethattheexecutiontimeincolumns5-7 whichadoptthe intra-propertylearningtechniquesincludesthelearning timefromdivided/segmented 112

PAGE 113

Table5-1.Testgenerationresultsusingintralearnings SATCNFSizezCha[ 74 ]ConritClauseDecisionOrderingHybridMax Instance#Variable#ClauseTime(s)Time(s)Time(s)Time(s )Speedup bmc-galileo-8580742948210.990.430.74 0.41 2.30 bmc-galileo-9636243269991.740.990.94 0.56 3.11 Bmc-ibm-10590563237007.98 3.96 8.237.862.02 Bmc-ibm-11321091500276.984.58 1.8 6.973.88 VLIW-1521188133784611366.781070.42074.19 489.15 2.79 VLIW-252115813378532198.12 77.45 221.17298.162.56 VLIW-352104613376161145.46151.8555.66 52.93 2.75 VLIW-4520721133481171126.13295.15599.22 94.4 11.93 VLIW-552077013380350879.24757.09703.1 167.78 5.24 VLIW-652119213378781211.50 51.49 544.26317.264.11 VLIW-752114713378010 87.61 189.41357.63400.741.00 VLIW-8521179133786171227.75952.13443.38 377.74 3.25 VLIW-952118713378624962.82 107.99 1523.441590.548.92 VLIW-10521182133786251769.14 915.73 930.21595.051.93 PIPE-113891746787561327.92752.52279.82 278.17 4.77 PIPE-213891846787181710.661703.37403.97 403.97 4.23 PIPE-31389174678757825.78394.07 365.04 969.332.26 PIPE-413856346750401080.1032.57408.13 14.06 76.82 PIPE-51389184678760626.9566.75603.45 114.57 5.47 PIPE-61387954671352 0.43 0.65117.44117.231.00 PIPE-713891846787601734.26987.881359.35 534.72 3.24 PIPE-81387114688614113.072.060.65 0.65 173.95 PIPE-913891646760076062.276065.7 355.56 355.6217.05 PIPE-1013891846787601430.291074.18 277.98 978.935.15 113

PAGE 114

CNFs.Thistableshowsthatourmethodcandrasticallyreduc ethetestgenerationtime (upto174times).Wecanobservethatinmajorityofthecases ,theconrictclauses forwardingbasedintra-propertylearningcanimprovethet estgenerationtimecompared tozCha.However,decisionorderingmethodandhybridmeth odarenotalwayshelpful. Thisisbecausethedecisionorderingbasedmethodmayleadt hesearchinawrongway withmoreconricts. Figure 5-8 and 5-9 showthestatisticsofconrictsandimplicationsforthecol lected benchmarksusingvariousintra-propertylearningmethods .Wenormalizedthegenerated conrictclausesforeachlearningmethodusingthetotalcon rictclausesandimplications generatedbythefourdierentmethodsshowninTable 5-1 .Theverticalaxisofthe stackedgraphsshowsthenormalizedpercentageofconrictc lausesandimplications respectively.Wecanndthattheresultofthepercentageof conrictclausesand implicationsisconsistent.Inotherwords,lessconrictsw illresultinlessimplications. Furthermore,theseguresalsoareconsistenttothetestge nerationperformanceshown inTable 5-1 .Itindicatesthat,byusingtheproposedintra-propertyle arningmethodsin parallel,wecandrasticallyreducetheconrictsaswellasi mplicationsduringtheSAT searching.Consequentlywecansavethetestgenerationtim e. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Conf li c t Clau se P erce n tage z C ha ff C onflic t C lause De c isi on Or de ring Hy br id Figure5-8.Conrictstatisticsusingvariousintra-proper tylearnings 114

PAGE 115

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Im p li c at ion P erce n tage z C ha ff C onflic t C lause De c isi on Or de ring Hy br id Figure5-9.Implicationstatisticsusingvariousintra-le arnings 5.4.2Inter-PropertyLearning5.4.2.1AMIPSProcessor TheMIPSprocessordesignisbasedontheexampledescribedi nSection 2.3.2 .We appliedourmethodologytogeneratetherequireddirectedt estsforfourpipelinepathsin theexecutestage(ALU,FADD,MULandDIV). Duetothesimilarity,weclusterthepropertiesofeachpath togethertosharethe learning.Thereare16propertiesdividedinto4clusters.E achclusterhasabaseproperty. Table 5-2 showstheresults.Therstcolumnindicatesthecomponentu ndertest.The secondcolumnshowsthepropertiesusedfortestgeneration .Thethirdcolumngives thetestgenerationtimeusingzChadirectly.Thefourthco lumnshowstheresultby forwardingconrictclausesamongproperties.Ithasthrees ub-columns.Sincetheconrict clausesforwardingbasedmethodneedstoexplorethecommon clauses,weneedtogure outtheintersectionbetweenSATinstances.Thereforethe rstsub-columngivesthe intersectiontime.Thesecondsub-columngivesthecheckin gtimeunderthelearning ofconrictclauses.Thethirdsub-columngivesthespeedupo verzCha(speedup= zChaffTime IntersectionTime + CheckingTime ).Thefthcolumngivesthetestgenerationresultusing decisionorderingbasedlearnings.Ithastwosub-columns: i)testgenerationtime,andii) 115

PAGE 116

speedupoverzCha.Thelastcolumnshowstheresultwhichus esbothconrictclauses anddecisionorderingbasedlearnings. Forthebasepropertyofeachcluster,weadopttheintra-pro pertylearningtechniques. Sincethebasepropertyisamajorbottleneckoftheclusteri ngbasedmethodsdescribed inChapter 4 ,thetestgenerationtimereductionforthebasepropertyca ndrastically increasetheoverallperformance.InTable 5-2 ,wealsogivethesummaryforeach propertycluster.Wefoundthatthehybridmethodneedsless timeduringthetest generation.However,sincetheconrictclauseforwardingn eedstoconsidertheSAT instanceintersection,theoverallperformanceofhybridm ethodisworsethanthedecision orderingbasedmethod.Ingeneral,thedecisionorderingba sedmethodcanachievethe bestperformance.Inthiscasestudyoffourclusterswithfo urpropertyineachcluster,we canachieve4-6timesimprovement. 1 E +0 1 E +1 1 E +2 1 E +3 1 E +4 1 E +5 1 E +6 p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 Co nflict Cla is e Num ber P ro pert ies zCh af f C on f lict C lau s e Var iab le Or d er in g Hy b r id Figure5-10.ConrictstatisticsforMIPSprocessor DuringtheSATsearching,thenumberofconrictclausesandn umberofimplications stronglyindicatethesearchingtime.Figure 5-10 illustratestheconrictclausegeneration foreachpropertyduringthesearchusingdierentmethods. Figure 5-11 showsthe correspondingimplicationnumbers.Itcanbeseenthat,byu singourmethod,the numberofconrictclausesandimplicationscanbereduceddr asticallybyseveral orders-of-magnitude,whichresultsinsignicantimprove mentintestgenerationtime. 116

PAGE 117

Table5-2.TestgenerationresultforMIPSprocessor MIPSProp.zCha[ 74 ]ConrictClauseDecisionOrderingHybrid Component(Tests)Time(s)Inter.(s)Time(s)SpeedupTime( s)SpeedupInter.(s)Time(s)Speedup p 1 1 19.78011.91.6613.081.5109.362.11 ALU p 2 16.552.490.874.930.13127.312.490.117.84 Unit p 3 15.412.081.823.950.15102.732.080.116.45 p 4 16.212.660.545.070.1890.062.660.125.69 Summary all67.9522.363.0413.54 5.02 16.714.07 p 5 1 15.21016.140.9416.090.9508.341.82 DIV p 6 19.832.771.844.300.12165.252.770.119.40 Unit p 7 13.742.790.983.640.4928.042.790.155.56 p 8 13.242.840.913.530.1494.572.840.184.66 Summary all62.0228.272.1916.843.6815.76 3.94 p 9 1 16.01018.000.8911.591.3809.331.72 FADD p 10 15.382.612.602.950.1696.132.610.125.01 Unit p 11 15.632.081.804.030.12130.252.080.126.65 p 12 18.372.880.924.830.12153.082.880.127.09 Summary all65.3930.892.1211.99 5.45 17.343.77 p 13 1 50.90038.91.3131.881.60026.181.94 MUL p 14 51.273.3513.143.110.29176.793.350.6615.40 Unit p 15 47.853.1415.062.630.22217.503.140.6112.24 p 16 53.442.8914.593.060.25213.762.890.1516.75 Summary all203.4691.072.2332.64 6.23 36.615.56 1 Baseproperty117

PAGE 118

1 E +0 1 E +1 1 E +2 1 E +3 1 E +4 1 E +5 1 E +6 1 E +7 1 E +8 p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 I m pli ca t io n Num ber P ro pert ies zCh af f C on f lict C lau s e Var iab le Or d er in g Hy b r id Figure5-11.ImplicationstatisticsforMIPSprocessor Itisimportanttonotethatthehybridmethodcanachievelea stnumberofconrictsand implications,whichjustiesourdiscussioninSection 5.2.4 5.4.2.2AStockExchangeSystem TheformalNuSMVdescriptionoftheon-linestockexchanges ystem(OSES) isderivedfromitsUMLactivitydiagramspecicationdescr ibedinSection 2.3.5 in Chapter 2 .ApathintheUMLactivitydiagramindicatesastocktransac tionrow.There areatotalof49propertiesgeneratedbasedonpathcoverage criteria.Accordingtotheir similarity,wegroupthemintonineclusters. Table5-3.Testgenerationresultforstockexchangesystem ClusterSizezChaConrictClauseDecisionOrderingHybrid Max [ 74 ](s)(s)(s)(s)speedup C 1 313.6311.938.5510.711.59 C 2 426.3535.373.997.756.60 C 3 8463.54183.0641.2450.4311.24 C 4 43.365.011.495.562.26 C 5 466.5940.476.3811.3010.44 C 6 8343.88270.4812.2823.3328.00 C 7 217.816.737.036.202.87 C 8 8666.61343.9451.8071.1912.87 C 9 8208.50101.9134.9934.835.99 Average-201.14110.9918.6424.5910.79 Table 5-3 showsthetestgenerationresultsinvolvingallthe9cluste rs.Therst columnindicatestheclusters.Thesecondcolumnindicates thesizeofeachcluster 118

PAGE 119

(numberofproperties).Thethirdcolumnpresentsthetestg enerationtime(including baseproperty)usingzCha.Thefourthcolumngivestheresu ltusingconrictclausebased inter-andintra-propertylearnings.Thefthcolumnprese ntstheresultusingdecision orderingbasedinter-andintra-propertylearnings.Thesi xthcolumnindicatesthetest generationtimeusingbothlearnings(i.e.,hybridmethod) .Thelastcolumnindicates themaximumspeedupusingourheuristicmethods.Inthiscas estudy,ourapproachcan produceanaverageof10.79timesoverallimprovementintes tgenerationtimecompared tozCha.Itisimportanttonotethatthedecisionorderingm ethodcanachievethebest performance,whichisconsistentwiththeresultobtainedi nSection 5.4.2.1 5.5Summary ToaddressthecomplexityoftestgenerationusingSAT-base dBMC,thischapter presentedanovelmethodologywhichexplorestheintra-pro pertylearningswithinaSAT instanceandinter-propertylearningsbetweensimilarSAT instances.Alltheselearnings arebasedondecisionorderingheuristicsaswellasconrict clauseforwardingtechniques. Tothebestofourknowledge,ourworkistherstattempttosh arethedecisionordering learningsondierentpartsofaSATinstanceaswellasacros smultipleproperties. Byexploitingthecommonalitiesduringthesearchofsatis ableassignments,thetest generationtimeofasinglepropertyaswellasasetofsimila rpropertiescanbereduced. Theexperimentalresultsusingbothhardwareandsoftwared esignsdemonstratedthe eectivenessofourmethod.Ourstudiesshowthathybridlea rningismoreprotablefor solvingoneSATinstance,whereasdecision-orderingbased learningismorebenecialfor solvingasetofsimilarSATinstances. 119

PAGE 120

CHAPTER6 EFFICIENTPROPERTYDECOMPOSITIONTECHNIQUES Checkingtherst(base)propertyisamajorbottleneckduri ngthetestgeneration usingclusteringandlearningtechniques,sincethebasepr opertycannotactivelyobtain learningsfromotherstoimproveitstestgenerationtime.E speciallywhencheckinga largedesignwithcomplexproperties(i.e.,propertieswit hlargeconeofinruenceordeep bounds),BMCbasedmethodsareverycostlysincelargeSATin stancesindicatelongSAT searchtime. p1pm ...... p2p1 p2 pn ...... t1t2 ...... tm Composition T learnings P BMC T' a) Test-oriented decomposition b) Our Learning-oriented decomposition + P P Figure6-1.Twopropertydecompositiontechniques Toaddressthisproblem,Kooetal.proposedapropertydecom positiontechnique [ 47 ]asshowninFigure 6-1 a.Thebasicideaistodecomposeacomplexpropertyinto severalsimplesub-properties,andthencomposethetestsc orrespondingtosub-properties toderiveatestfortheoriginalproperty.Sincethetestgen erationtimeofsub-properties istypicallyseveralordersofmagnitudesmallerthantheor iginalproperty,thestate spaceexplosionproblemcanbeavoidedinmanyscenarios.Ho wever,thecompositionof testsofsub-propertiesisamajorbottleneckinthismethod sinceitishardtoautomate. Theinevitablehumaninterventionandexpertknowledgeisr equiredduringthetest composition.Inmanycases,itmaynotbepossibletoobtaint herequiredcounterexample bycomposingpartial(local)counterexamples.Asanaltern ative,inthischapter,we 120

PAGE 121

proposealearning-orienteddecompositiontechniqueshow ninFigure 6-1 bwhichcan befullyautomated.Unlikethetest-orientedmethodin[ 47 ],ourapproachisbasedon thelearnedknowledge(i.e.,decisionordering)duringthe testgenerationofdecomposed protablesub-properties.Suchlearningscanbeusedtodra sticallyacceleratetheoriginal propertyfalsication.Thereforetheoveralltestgenerat ioneortcanbesignicantly reduced.Ourmethodmakesthreeimportantcontributions:i )itproposesamethod thatcanspatiallyortemporallydecomposeacomplexproper tyintoseveralsimplebut protablesub-properties;ii)itproposesanapproachthat canderivelearningsfromthe decomposedsub-properties;andiii)itproposesamethodth atcanguidethecomplex propertycheckingusingderivedlearnings. Formal Model Generation Formal Model Learnings Test Generation SpecificationImplementation Validation Validation Test cases (SMV Input) Design Specification Decomposition Property Complex Checking Properties Figure6-2.Ourtestgenerationframework Figure 6-2 showsourtestgenerationframework.Theinputstothisfram eworkare thedesignspecicationandrequiredproperties.Toreduce thiscomplexity,therearethree importantsteps(threeshadedboxesinthegure).First,we proposetwonovelproperty decompositiontechniqueswhichcansignicantlyreduceth ecomplexityduringproperty 121

PAGE 122

falsication.Next,bycheckingtheselectedprotablesub -properties,wecancollect usefullearningsfortheoriginalpropertychecking.Final ly,thelearnedknowledgecanbe utilizedasadecisionorderingheuristictoavoidtheunnec essaryconrictsduringthetest generation.Therefore,thetestgenerationtimecanbedras ticallyreduced. Therestofthechapterisorganizedasfollows.Section 6.1 proposestwonovel propertydecompositionmethodsbasedonlearningtechniqu es.Section 6.2 presentsthe decisionorderingbasedlearningtechniquesfororiginalp ropertychecking.Section 6.3 describeshowtousethelearnedknowledgefromthedecompos edpropertiesfortest generation.Section 6.4 showsanexampleusingourdecompositiontechniques.Secti on 6.5 presentscasestudiesusingbothhardwareandsoftwaredesi gns.Finally,Section 6.6 summarizesthechapter. 6.1Learning-OrientedPropertyDecomposition Thissectionrstdiscussesthepotentiallearningsofthep ropertiesfortestgeneration. Next,weproposeourspatialandtemporaldecompositiontec hniques. 6.1.1PotentialLearningsforComplexProperties DuringtestgenerationusingBMCbasedmethods,therearetw okindsofcomplex propertieswhichareoftenencountered:i)propertieswhic hdescribecomplexscenarios involvingmultiplecomponentsofthedesign;andii)proper tieswhichindicateeventswith longdelay.BothcaseswillresultinlargeSATinstancesbec auseofthecorresponding largeConeofInruence(COI)andlargebounds.Thereforeiti snecessarytoexplore learningstoreducethecomplexityduringthetestgenerati on. Foracomplexsystemlevelpropertywhichdescribesinterac tionsbetweendierent components,itcanbepartitionedintomultiplecomponentl evelsub-formulas.Asan exampleshowninFigure 6-3 ,asystemlevelproperty P canbebrokeninto3component levelsub-properties P 1 P 2 and P 3 withdierentCOI.Whencheckingasub-property suchas P 1 withasmallCOI,itusuallyneedsmuchlesstimeandspacetha nthatof checkingthecomplexproperty P .Theknowledgelearnedduringchecking P 1 canbeused 122

PAGE 123

fortestgenerationoftheproperty P .InSection 6.1.2 ,weproposeaspatialproperty decompositionmethodtoexploresuchlearnings. Cone 2Cone 3 P1P2 P3 V1 V2 V3 V4 V5 Vn. . .Design Block Cone 1 P Figure6-3.TheCOIofadesignblock Transactions arewidelyusedtodescribeSoCsystemlevelbehaviors.Atra nsaction isasequenceofstronglyrelevant events .AsanexampleshowninFigure 6-4 ,thereare 3transactions,andeachtransactionhastwoevents.Weclas sifytherelationbetween theseeventsintwocategories.The causeeect relation(markedby ) )denesthe relationofintra-transactionevents.Forexample,intran saction T 1,if e 1happens,then e 2 shouldhappeninfuture.The happenbefore relation(markedby )speciestherelation ofinter-transactionevents.Itindicateswhicheventshap penbeforeotherevents.For example, e 4 e 5means e 4happensbefore e 5. Duringthetestgenerationfortransactions,wespecifyane gatedsafetypropertyto indicatetheoccurrenceofevent e intheformof F ( e ).Generally,ifaneventhappens withalongdelay,BMCwillunrollthedesignmanytimeswhich willdrasticallyincrease thecheckingcomplexity.Accordingtothedenition,the\ ) "relationcanbeusedto derivehelpfullearnings.ForexampleinFigure 6-4 ,letproperty P 1 = F ( e 1 )and property P 2 = F ( e 2 ).Since e 1 ) e 2 implies F ( e 1 ) F ( e 2 ),i.e., P 1 P 2 ,it showsthatthe P 1'scounterexamplewillbehelpfulforderiving P 2'scounterexample. Suchinformationcanbeusedasalearning.The\ "relationalsocanbeusedtoindicate 123

PAGE 124

thelearninginformation.Assuming e 4 e 5 ,thecounterexampleof F ( e 4 )isshorter thanthecounterexampleof F ( e 5 ).However,byourobservation,counterexampleof F ( e 4 )mayhavealargeoverlapofvariableassignmentswiththeco unterexampleof F ( e 5 ).Thereforethelearningfrom F ( e 4 )canbenetthetestgenerationof F ( e 5 ). InSection 6.1.3 ,weproposeatemporalpropertydecompositionmethodtoexp loresuch learnings. e1e2 e6 T1 e3 T2 e4 e5 T3 Figure6-4.Afunctionalscenariowiththreetransactions 6.1.2SpatialPropertyDecomposition Acomplexfalsesafetypropertycanbedecomposedintoaseto fsub-propertieswith equivalentsemantics.Ifthepartialcounterexamplesgene ratedbythesub-propertiescan berenedtoguidethecomplexpropertyfalsication,theor iginalpropertyis spatially decomposable Denition8. Let P beafalsesafetyproperty.Pisspatiallydecomposableinth eform p 1 ^ p 2 ^ ::: ^ p n orintheform p 1 p 2 ::: p n ifallthefollowingconditionsaresatised. Ifthedecomposedpropertiesareintheform p 1 ^ p 2 ^ ::: ^ p n ,thenatleastoneproperty p i ( 1 i n )hasacounterexample.Inthiscase,theboundof P istheminimum boundof p i whichhasacounterexample. Ifthedecomposedpropertiesareintheform p 1 p 2 ::: p n ,theneachproperty p i ( 1 i n )hasacounterexample.Inthiscase,theboundof P isthemaximumbound ofalldecomposedproperties. 124

PAGE 125

Thecounterexamplesgeneratedfromproperties p i ( 1 i n )canguidethetest generationforproperty P AccordingtoDenition 13 ,thefollowingrulescanbeusedforcomplexproperty decomposition. X ( p q ) X ( p ) ^ X ( q ) X ( p ^ q ) X ( p ) X ( q ) F ( p q ) F ( p ) ^ F ( q ) (6{1) Thefalsepropertyintheformof F ( p ^ q )and F ( p q )cannotbe directlydecomposedintoconjunctiveordisjunctiveform. However,byintroducinga synchronizationclock clk ,theycanbespatiallydecomposed.Itisimportanttonoteth at thevalueofthe clk indicatestheboundofthefalseproperty.TheEquation( 6{2 )shows thatthecounterexampleof F ( p ^ q ^ clk = k )canberenedbythecounterexamplesof F ( p ^ clk = k )and F ( q ^ clk = k ). F ( p ^ q ^ clk = k ) F ( p ^ clk = k ) F ( q ^ clk = k ) where F ( p ^ q ^ clk = k )isfalse : (6{2) Forafalsepropertyintheform F ( p q ), p describesthepre-condition,and q indicates thepost-condition.Whentheproperty G ( p )holds, F ( p q )willbevacuously true,andthecheckingof F ( p q )willreportacounterexamplewithoutsatisfying theprecondition p .Thiscounterexamplemaynotmatchtheoriginalintention. In Equation( 6{3 ),weonlyconsiderthecasewherethepre-condition p issatised. F ( p q ^ clk = k ) F ( p ^ clk = k ) F ( q ^ clk = k ) where F ( p q ^ clk = k )and F ( p ^ clk = k )arefalse : (6{3) BasedontherulespresentedinEquation( 6{1 )-Equation( 6{3 ),asystemlevelproperty canbedecomposedintheformof p 1 ^ p 2 ^ ::: ^ p n or p 1 p 2 ::: p n .Infact,ifthe complexpropertycanbedecomposedintheform p 1 ^ p 2 ^ ::: ^ p n ,itisnotnecessaryto 125

PAGE 126

usethelearninginformation.Wejustneedtosorttheproper ty p i (1
PAGE 127

i th cluster.IftheCOIofsuchclusterissmallerthan k n of P s COI,step4willgenerate anewrenedproperty newP forthe i th cluster.Step5adds newP to SD props .The renedproperty newP forlearningrepresentsaclusterofsub-propertiesasshow ninstep 3.Finallythisalgorithmwillreturnasetofrenedsub-pro pertiesforderivinglearnings (describedinSection 6.2 ).SincetheCOIofarenedpropertyin SD props issmall,its testgenerationtimewillbemuchsmallerthanthatoftheori ginalcomplexproperty.Itis importanttonotethatthisalgorithmmayreturnanemptyset whichmeanstheproperty cannotbespatiallydecomposed. Algorithm9 :SpatialDecomposition Input :i)Thedesignmodel, D ii)Aproperty P intheform p 1 p 2 ::: p n Output :Asetofrenedsub-propertiesforlearning, SD props 1. SD props = fg ; 2. ( cluster 1 ;:::;cluster m )= clustering ( P;modular=functional ); for i isfrom1to m do 3. cluster i = f prop 1 ;:::;prop k g ; if COI ( cluster i ) k n COI ( P ) then 4. generatearenedproperty newP forthe cluster i ; 5. SD props = SD props S newP ; end endreturn SD props ; 6.1.3TemporalPropertyDecomposition Temporalpropertydecomposition triestoeclipsetheboundeect.Thebasicidea oftemporaldecompositionistodeducealongboundproperty fromasequenceof shortboundproperties.Forexample, P 1, P 2, P 3and P 4( P 4= P )areproperties indicatingfourdierentstagesofproperty P .Theboundofthemare K 1, K 2, K 3and K 4,respectively,and K 1
PAGE 128

thatcanbeusedwhenchecking P 2.Similarly,duringthepropertychecking, P 3can benetfrom P 2and P 4canbenetfrom P 3.Thereforetheknowledgelearnedfromlower boundpropertiescanbereusedbythelargerboundproperty. Suchlearningcanavoid someunnecessaryrandomSATsearchingandcanquicklyobtai nthecounterexamplefor property P Denition9. Let P beafalsesafetyproperty,andPistemporallydecomposable ifallthe followingconditionsaresatised. P canbedividedintofalseproperties p 1 p 2 ::: and p n ( P = p n )withincreasing bounds. p i p i +1 (1 i k n 1) ,whichindicatesthecounterexamplegeneratedfrom properties p i canguidethetestgenerationforproperty p i +1 Ifthecounterexamplesoflowerboundpropertycanbeusedto reasonabout P ,the property P is temporallydecomposable .Intemporaldecomposition,ndingtheimplication relation(\ ")betweenpropertiesisakeyprocess.Inourframework,wec onstructsuch implicationrelationbyexploringtheorderbetweenevents ,i.e.\ ) "or\ ". e3 e5 e6 e4 e1 e2 e7 e8 e9 1 3 5 5 1 2 2 2 1 Event Happen before Cause effect Figure6-5.ADAGofeventrelation Whencheckingalargeboundpropertyforatransaction,ther emaybemanyevents alongthepathtothetargetevents.Checkingalltheseevent stoobtainlearningsis time-consuming.Forexample,assumingthatwewanttocheck theproperty F ( e 9 ), 128

PAGE 129

therelationbetweeneventsisdescribedusingadirectedac yclicgraph(DAG)shownin Figure 6-5 .Eachnodeindicatesanevent,andeachdirectededgeindica testherelation of\ ) "or\ ",andeachedgeisassociatedwiththedelaybetweenevents. InthisDAG, thereare8eventsthathappenbefore e 9.However,itisnotnecessarytocheckallofthem. SincethebranchnodesofaDAGcontainthecriticalvariable assignmentinformation, inourdecompositionmethod,weonlyconsidertheeventswhi chdeterminethebranches alongthepathfrominitialstate e 1tothetargetstate e 9. Algorithm 10 describeshowtoobtainasequenceofpropertiesbasedontem poral decomposition.ItacceptsaneventDAGwiththeinitialandt argeteventsasinputs.Step 1usesDijkstra'salgorithm[ 24 ]tondashortest path .Step2initializesthesequence TD props withapropertyfortheinitialevent.Step3and4selectthe branchevents and appendtheircorrespondpropertiestothe TD props .Finallythealgorithmreportsthe propertysequenceforderivinglearnings.Byusingthisalg orithm,( F ( e 1), F ( e 3), F ( e 7))isapropertysequencefromthetemporaldecompositioni nFigure 6-5 Algorithm10 :TemporalDecomposition Input :i)AneventDAG, D ii)Initialevent src ,targetevent dest Output :Apropertysequence TD props 1. path =Dijkstra(D, src dest )tondtheshortestdelaypath; 2. TD props =(propertyfor src ); for i isfrom2to len (numberofeventsin path ) do 3. ( e i 1 ;e i )=( i 1) th edgeof path ; if out degree ( e i 1 ) + in degree ( e i ) > 2 then 4. Appendthepropertyfor e i to TD events ; end endreturn TD props ; 129

PAGE 130

6.2DecisionOrderingBasedLearningTechniques SATbasedmodelcheckingencodesapropertycheckingproble mintoaSATinstance (aBooleanformula).Acounterexampleofthepropertyisasa tisablevariableassignment forthisformula.Althoughthevariableassignmentofcount erexamplesderivedfromthe decomposedsub-propertiesmaynotsatisfytheSATinstance ofthecomplexproperty, ithasalargeoverlapwiththecomplexpropertyonthevariab leassignment.Such informationcanbeusedasalearningtobiasthedecisionord eringwhencheckingthe complexproperty. DuringtheSATsearch,decisionorderingplaysanimportant roletoquicklynd asatisableassignment.Thelearningapproachinthischap terismotivatedbythe workproposedinChapter 5 .ItisbasedonVariableStateIndependentDecayingSum (VSIDS)method[ 63 ].Amajordierenceisthatourmethodincorporatesthestat isticsof decomposedproperties.Sincedierentsub-propertieshav edierentbounds,weconsider suchinformationinourheuristics. Let bounds beanarraywhichstorestheboundof k sub-properties.Becausein spatialmethodthedecomposedsub-propertiesmaybeindepe ndent,thelearningbetween sub-propertiesisnotsignicant.Soweset bound [ i ]=1(1 i k ).Howeverfor temporaldecomposition,the vstat informationoflowerboundpropertiescanfurther benetthelargerboundpropertychecking.Moreoverthelar gerboundsub-propertyis closertothenalpropertiesthansmallerboundsub-proper ties.Therefore,fortemporal decompositionbasedmethod,thesub-propertiesissorteda ccordingtotheincreasing bound and bound [ i ]indicatestheboundof i th property.Let vstat [ sz ][2]( sz isthevariable numberofthecomplexproperty)bea2-dimensionalarraytor ecordthestatisticsof variableassignments.Initially, vstat [ i ][0]= vstat [ i ][1]=0(0
PAGE 131

Assuming l i isaliteralof v i ( v i hastwoliterals, v i and v i '),weuse score ( l i )to indicateitsdecisionordering.Initially, score ( l i )isequaltotheliteralcountof l i .However, atthebeginningofSATsearchingandperiodicscoredecayin g,theliteralscorewillbe recalculated.Let bias = MAX ( vstat ( v i ) ;vstat ( v 0 i ))+1 MIN ( vstat ( v i ) ;vstat ( v 0 i ))+1 indicatethevariableassignmentvariance. score ( l i )= 8>>>>><>>>>>: max ( v i ) bias ( vstat [ i ][1] >vstat [ i ][0]& l i = v i ) or ( vstat [ i ][1]
PAGE 132

Figure 6-6 showsanexampleoftemporaldecompositionusingourheuris tic.The complexproperty P isdecomposedintothreeproperties p 1 p 2 and p 3 (= P )withbound1, 2and3respectivelyandweassumethatwealwayscheckthevar iablesintheorderof a b c .Initially,whenchecking p 1 ,thereisnolearninginformation.However,afterchecking p 1 ,wecanpredictthedecisionorderingfor p 2 basedonthecollected vstat information from p 1 .Alsowecanpredictthedecisionorderingof p 3 (= P )fromthe vstat of p 1 and p 2 Whenchecking P ,thecontentof vstat indicatesthatvariables a ismorelikelytobe0, b and c aremorelikelytobe1. 6.3TestGenerationusingOurMethods Inthischapter,weassumethattheboundofcomplexproperty anddecomposed propertiescanbepre-determined.Determinationofboundi shardingeneral.However,for directedtestgeneration,theboundcanbedeterminedbyexp loitingthestructureofthe design.AnexampleofbounddeterminationispresentedinSe ction 6.4 Algorithm11 :TestGenerationbasedonPropertyDecomposition Input :i)Formalmodelofthedesign, D ii)Decomposedproperties props andsatisable bounds iii)Thecomplexproperty P ,withthesatisablebound bound p Output :Atest test P for P 1. CNFs = BMC ( D;props;bounds ); 2. ( CNF 1 ;:::;CNF n )=sort CNFs usingincreasinglesize; 3. Initialize vstat ; for i isfrom1tothe n do 4. test i =SAT( CNF i vstat ); 5. Update ( vstat;test i ;bounds [ i ]); end6. Generate CNF = BMC ( D;P;bound P ); 7. test P =SAT( CNF vstat ); return test P ; 132

PAGE 133

Algorithm 11 describesourtestgenerationmethodology.Theinputsofth ealgorithm areaformalmodelofthedesign,asetofdecomposedproperti es props andtheir satisablebounds bounds ,andthecomplexproperty P withitssatisablebound bound p Step1generatesCNFlesintheDIMACSformat[ 74 ]foreachdecomposedpropertyin props .Step2sortstheCNFsbytheirDIMACSlesize.Step3initial izes vstat whichis usedtokeepstatisticsofthevariableassignmentsfordeco mposedsub-properties.Then foreachdecomposedsub-property,wecollectitscounterex ampleassignmentsfromstep4 tostep5.Foreachiteration,weneedtoupdate vstat statistics.Instep6andstep7,the complexproperty P ischeckedusingthedecisionorderingderivedfromthedeco mposed sub-properties.Finally,thealgorithmreportsatestfort hecomplexproperty P 6.4AnIllustrativeExample Thissectionpresentsanexampleofhowtousedecomposition methodsonadesign illustratedinSection 2.3.2 .Assumethatwewanttocheckacomplexscenariothatthe units MUL 5and FADD 4willbeactiveatthesametime.Wegeneratetheproperty P whichisanegationofthedesiredbehaviorasfollows.There mainderofthissectionwill solveitusingspatialandtemporaldecompositionmethods. /*OriginalcomplexpropertyP*/P:~F(mul5_active=1&fadd3_active=1) 6.4.1SpatialDecomposition IntheMIPSdesign,eachfunctionalunithasadelayofoneclo ckcycle.Totrigger thefunctionalunit MUL 5,weneedatleast7clockcycles(thereare7unitsalongthe path Fectch Decode ::: MUL5).Similarly,totriggerthefunctionalunit FADD 3, weneedatleast5clockcycles.Plusoneclockcycleforiniti alization,weneed8clock cyclesfortriggeringthisinteraction.Thustheboundofth ispropertyis8.According toEquation( 6{2 )andAlgorithm 9 ,property P canbespatiallydecomposedintotwo sub-propertiesasfollows,assumingtheCOIof P 1and P 2arebothsmallerthanhalfof COIof P 133

PAGE 134

/*ModifiedoriginalcomplexpropertyP'*/P':~F(mul5_active=1&fadd3_active=1&clk=8)/*Spatiallydecomposedproperties*/P1:~F(mul5_active=1&clk=8)P2:~F(fadd3_active=1&clk=8) Whenchecking P 1and P 2individually,wecangetthefollowingtwocounterexample s. CounterexamplesforP1andP2 CyclesP1'sInstructionsP2'sInstructions 1NOPNOP2MULR2,R2,R0NOP3NOPNOP4NOPFADDR1,R1,R05NOPNOP6NOPNOP7NOPNOP8NOPNOP However,accordingtoAlgorithm 11 ,thetestgenerationfor P 2isunderthe guidanceof P 1'sresult.Thus,thecounterexampleof P 2guidedby P 1contains P 1's partialbehavior(seeclockcycle2below).Sothescoreofli teralswhichhaverepetitive occurrencesisenhanced. CounterexampleforP2guidedbyP1CyclesP2'sInstructions 1NOP2MULR2,R2,R03NOP4FADDR1,R1,R05NOP6NOP7NOP8NOP 134

PAGE 135

Thestatisticssavedin vstat indicatesanassignmentwhichhasalargeoverlapofthe assignmentswiththerealcounterexamplethatcanactivate property P .Thusitcanbe usedasthedecisionorderinglearningtoguidetheproperty checkingof P 6.4.2TemporalDecomposition Fortemporaldecomposition,weneedtogureouttheeventim plicationrelationrst. Becausewewanttochecktheproperty F ( mul 5 active =1& fadd 3 active =1),the targeteventis mul 5 active =1& fadd 3 active =1.Figure 6-7 showstheimplicationfor thisevent.Thereare7eventsinthisgraph,and e 7 isthetargetevent. DIV MUL3 MUL3 IALU e4 e2 e3e5 e1 e6 FETCH MUL1 MUL2 DECODE MUL3 FADD1 MUL4 FADD2 MUL5 FADD3 e7 1 1 1 11 1 1 1 1 Figure6-7.EventimplicationgraphforpropertyP Assuming e 1istheinitialevent,from e 1to e 7,thereisonlyonepath e 1 e 2 e 4 e 6 e 7 .Alongthispaththereisabranchnode e 2 .AccordingtotheAlgorithm2, weneedtochecktwoevents e 1 and e 4 usingfollowingproperties.Byusingourlearning technique,duringthetestgeneration, P e4 canbenetfrom P e1 ,and P canbenetfrom P e4 /*Spatiallydecomposedproperties*/P_e1:~F(fetch_active=1&mul1_active=1)P_e4:~F(mul3_active=1&fadd1_active=1) 6.5Experiments Thissectionpresetstwocasestudies:theVLIWimplementat ionoftheMIPS architecture(describedinSection 4.5.1 )andthestockexchangesystem(describedin 135

PAGE 136

Section 4.5.2 ).Inourframework,weusedNuSMV[ 27 ]togeneratetheCNFclauses(in DIMACSformat)andintegratedourproposedmethodsinthezC ha[ 74 ]SATsolver.The experimentalresultsareobtainedonaLinuxPCusing2.0GHz Core2DuoCPUwith1 GBRAM.6.5.1AVLIWMIPSProcessor Thissectionpresentstheexperimentalresultusingave-s tagepipelinedMIPS processordesign.Thedetailsofthedesignareillustrated inSection 2.3.2 andSection 6.4 Sincethegeneratedpropertiesareinvariouscomplexforma ts,itisdiculttogure outtheimplicationbetweenevents.Thereforeinthiscases tudy,weonlyinvestigatethe spatialdecompositionbasedlearnings. Table6-1.TestgenerationresultforMIPSprocessor PropertyzCha[ 74 ]ClusterRenementSpatialSpeedup (Tests)(sec)##(sec) zCha vs Spa. Propertyformat: F ( p q ) p 1 119.96330.033999 p 2 56.22220.031874 p 3 2.32220.01232 Propertyformat: F ( p ^ q ) p 4 43.963218.882.33 p 5 15.24216.572.32 p 6 9.28214.422.10 Propertyformat: F ( p q ) p 7 13.59214.163.27 p 8 68.332113.165.19 p 9 160.513230.315.30 WeselectninecomplexpropertiesfromtheMIPSdesign.Tabl e 6-1 showsthetest generationresultsusingourspatialdecompositionmethod .Therstcolumnindicatesthe selectedproperties.Thesecondcolumngivesthetestgener ationtimeusingzCha.The thirdandfourthcolumnspresentthenumberofsub-property clustersandthenumberof renedsub-propertiesforderivinglearnings.Thelasttwo columnsshowtestgeneration timeusinglearningsandtheimprovementofourspatialdeco mpositionbasedmethod overthemethodusingzCha.Wecluster9propertiesinto3gr oups(3propertiesineach 136

PAGE 137

group),andeachgrouphasaspecicpropertyformat.Forexa mple,therstgroupcan bedecomposedas p 1 ^ p 2 ^ p n .Thusthetestgenerationcanbedonewhenndinga counterexamplefromalowerboundsub-propertywithoutany learnings.Forthesecond andthirdgroups,thepropertiescanbedecomposedinthefor mof p 1 p 2 p n Eachsub-propertyareofthesamebound.Thereforeweneedto clusterthesub-property accordingtothesimilarityrulespresentedinSection 6.1.2 .Comparedtothemethod withoutanylearnings(column2),ourspatialdecompositio nbasedlearningmethodcan drasticallyreducethetestgenerationtime. 0 1 2 3 4 5 6 p4 p5 p6 p7 p8 p9 Spe e dup P roper t i e s T im e Co nflict Cla us es I m pli ca t io ns Figure6-8.PropertycheckingresultforMIPSprocessor DuringtheSAT-basedBMCfalsication,conrictclausenumb erandimplication numberarekeyfactorswhichdeterminethetestgenerationp erformance.Decision orderinglearnedfromdecomposedpropertiescaneciently avoidtheconrictswhen checkingthecomplexproperty.Figure 6-8 showstheresultofproperties p 4 p 9 presented inTable 6-1 .Itillustratestheperformanceimprovement(spatialmeth odoverzCha) usingtime,implicationnumberandconrictclausenumber.I tcanbeseenthat,byusing spatialmethod,thenumberofconrictclausesandimplicati onscanbereduceddrastically by2-4times,whichconsistentlyresultsinsignicantimpr ovementintestgenerationtime (2-5times). 137

PAGE 138

6.5.2AStockExchangeSystem Theon-linestockexchangesystem(OSES)isasoftware(desc ribedinSection 2.3.5 ) whichmainlydealswithstockordertransactions.Wegenera te18complexpropertiesto checkthestocktransactions.IntheUMLactivitydiagram,e achtransactionisindicated byapathwhichisasequenceofactivities(events).Thetest generationforatransaction usingonlyonecomplexpropertyistimeconsuming.Sowetemp orallydecomposedthe transactionintoseveralstageswhichspecifythebranchac tivitiesalongthepath,andfor eachstagewecreateasub-property. Amongthe18complexproperties,tenofthemaretime-consum ing(morethan10 secondswithoutusingourmethod).Table 6-2 showsthetestgenerationresultsforthese tenpropertiesusingtemporaldecomposition.Therstcolu mnindicatestheproperty.The secondcolumnindicatesthetestgenerationtimeusingzCha withoutanydecomposition andlearningtechniques.Thethirdcolumnpresentstheboun dofthecomplexproperty. Thefourthcolumnindicatesthenumberoftemporalsub-prop ertiesdecomposedalong thestocktransactionrow.Thelasttwocolumnsindicatethe testgenerationtime(using temporaldecomposition)anditsspeedupoverzCha.Inthis casestudy,ourapproachcan producearound3-60timesimprovementcomparedtothemetho dusingzCha. Table6-2.TestgenerationresultforOSES PropertyzCha[ 74 ]BoundDecomposedTemporalSpeedup (Tests)(sec)#(sec) zCha vs Temp. p 1 25.99830.7833.32 p 2 48.991042.6918.21 p 3 39.671153.4511.50 p 4 247.2611522.4611.01 p 5 160.7311515.6810.25 p 6 97.541141.5662.53 p 7 31.3910412.312.55 p 8 161.7411412.6212.82 p 9 142.9110417.578.13 p 10 33.771041.7619.19 138

PAGE 139

6.6Summary Toaddressthetestgenerationcomplexityofasinglecomple xpropertyusing SAT-basedBMC,thischapterpresentedanovelmethodwhichc ombinestheproperty decompositionandlearningtechniques.Bydecomposingaco mplexpropertyspatially andtemporally,wecangetasetofsub-propertieswhosecoun terexamplescanbeused topredictthedecisionorderingforthecomplexproperty.B ecauseofthelearningfrom thesimplesub-propertiestothecomplexproperty,theover alltestgenerationeortcan bereduced.Thecasestudiesdemonstratedtheeectiveness ofourmethodusingboth hardwareandsoftwaredesignsthatgeneratedsignicantsa vings(2-60times)intest generationtime. 139

PAGE 140

CHAPTER7 REUSEOFVALIDATIONEFFORTFORASSERTION-BASEDEQUIVALENC E Forsoftwaredesigns,thedierencebetweenspecicationl eveltestsandimplementation leveltestsissmall.Generally,thespecicationleveltes tscanbeautomaticallyreusedand appliedonsoftwareimplementations.Consequently,theco nsistencybetweendierent softwaredesignscanbechecked.However,duetothesignic antdierenceintiming andotherdetails,maintainingthefunctionalequivalence betweendierenthardware abstractionlayersisamajorchallengeduringtheSoCdesig n.Inthischapter,weare focusingoncheckingthefunctionalconsistencybetweendi erenthardwareabstractions. Inthischapter,wefocusonreusingvalidationeortbetwee nTLMandRTLmodels. SincethereisnomatureautomaticTLMtoRTLrenementtoola ndmanual conversioniserror-prone,variousapproachesarepropose dtoguidetheTLMtoRTL conversion.Simulationisawidelyusedmethodforfunction alvalidation.Byusing atransactor[ 6 ]betweenTLMandRTLdesignsforcommunication,thepreviou sly generatedTLMtestscanbeexercisedontherenedRTLimplem entationstocheckthe functionalcorrectness.However,duetosubstantialdier encesbetweenTLMandRTL models,traditionalsimulationmethodscannotguaranteet hefunctionalequivalence. Forblack-boxmethods[ 66 ],simulationcannotguaranteethebugpropagationtothe outputs.Similarly,forwhite-boxsimulationmethods[ 66 ],thecodecoverage[ 32 ]and togglecoveragecannotfullyindicatetherequiredfunctio nalcoverage.Thisisduetothe lackoffunctionalobservationmechanismsfortraditional simulationbasedmethods. Assertionbasedvalidation(ABV)[ 23 29 ]hasbeensuccessfullyappliedinSoC validationtoensurethefunctionalcorrectness.Itnotonl yincreasesthedesignobservability basedonsimulationusingad-hoctests,butalsotakesadvan tageofmoreemergingformal methodsforimprovingtheoverallvericationqualityandr esults.Asafunctional observationpoint,anassertioncanbeinstrumentedintoTL MorRTLdesignsto monitorthespeciedfunctionalscenario.Currently,ther earetwomostpopularassertion 140

PAGE 141

languages:PropertySpecicationLanguage(PSL)[ 3 ]andSystem-VerilogAssertion(SVA) [ 37 ].PSLisplatform-independentandcanbeusedinmulti-laye rdesigns.SVAissimilar toPSL,neverthelessitisonlycustomizedforSystem-Veril ogdesigns.Inthecontextof ABV,a property isdenedasalogicconstraintdescriptionbuiltonBoolean expressions, sequencesandtemporaloperators,while assertion isdenedasadirectivetoprovethe correctnessoftheproperty.Forsimplicity,inthischapte rweusethetermassertionto indicatebothassertionandproperty. Inthischapter,weproposeamethodologytoguaranteethefu nctionalequivalence betweenTLMandRTLmodelsbasedontheobservabilityofasse rtions.Thebasicidea isthatintheTLMspecication,ifatestcanexerciseaspeci edfunctionalscenario monitoredbysomeassertion,thenintheRTLimplementation ,thecounterpartof theTLMtestcanalsoactivatethecounterpartoftheTLMasse rtion.Duringthe TLM-to-RTLfunctionalequivalencechecking,weneedtoadd ressesthefollowingfour issues: HowtodetermineasetofTLMassertionsforobservingallthe functional scenarios? Weproposedseveralfaultmodelswhichrequirethatallthes pecied faultsshouldbecoveredbythegeneratedassertions. HowtoactivateagivenTLMassertion? Weadoptedthemodelchecking falsicationtechniquetoderivetestsforactivatingTLMa ssertions.Foreach assertion,wegenerateonetesttoactivateit. HowtoreuseTLMvalidationeort? Wedevelopedthevalidationrenement ruleswhichcanconvertTLMassertionsandteststotheirRTL counterparts. HowtousethecorrelationbetweenTLMandRTLassertionsfor equivalencechecking? WeproposedamethodtoverifytheTLM-to-RTLequivalence basedonthecriteriaofassertioncoverageandassertionor dering. Ourproposedapproachaddressestheabovechallengesandma kestwomajor contributions:i)developsaprototypetoolforautomaticT LM-to-RTLtestandassertion renement,andii)proposesamethodthatusestheassertion observabilityforchecking thefunctionalequivalencebetweenTLMandRTLmodels.Beca useourworkisbasedon 141

PAGE 142

thereuseofTLMvalidationeort,thereisnoextracost(exc ludesdeningtherenement rules)sinceitneedstobevalidatedanyway.Furthermore,o urmethodisfullyautomated andcanbeeasilyscaledforlargedesigns. Therestofthischapterisorganizedasfollows.Section 7.1 presentsrelatedworkon validationreuseandequivalencecheckingbetweenTLMandR TLmodels.Section 7.2 proposesourequivalencecheckingframeworkbasedonvalid ationreuse.Section 7.3 presentstheexperimentalresults.Finally,Section 7.4 summarizesthechapter. 7.1RelatedWork TLMispromisingtoenableearlydesignspaceexplorationan dhardware/softwarecosimulation.Hsiungetal.[ 36 ]adoptedSystemCTLMmodelstoenablerapidexploration ofdierentrecongurabledesignalternatives.In[ 44 ],Kogeletal.presentedaSystemC basedmethodologywhichprovidessucientperformance,re xibilityandcosteciencyas requiredbydemandingapplications.Shinetal.[ 80 ]proposedamethodtoautomatically generateTLMmodelsfromvirtualarchitecturemodelswhich canachievesignicant productivitygains. Asahybridmethodbasedonbothsimulationandformalveric ation,ABVis acknowledgedasapromisingapproachforfunctionalvalida tioninRTLlevel[ 1 ].However, ABVisstillachallengingdomaininsystemleveldesign.Toa ddresstheissueswhen incorporatingPSLwithinSystemCenvironments,Lahbibeta l.[ 49 ]proposedan automatedsolutionwhichcanembedPSLassertionsinaSyste mCdesign.Basedon staticcodeanalysisandgeneticalgorithms,Habibietal.[ 33 ]presentedanecient methodtooptimizetestgenerationinordertoincreasethea ssertioncoverage.Eckeret al.[ 25 ]proposedatransactionlevelassertionframeworkusingan ewspecializedlanguage. In[ 73 ],Pierredescribedanecientandtractablesolutionforve rifyingthePSLbased propertiesofTLMdesignduringthesimulation.However,mo stresearchesarefocusedon implementingPSLassertionsinSystemCframework,andnone ofthemuseassertionsfor checkingtheTLM-to-RTLfunctionalequivalence. 142

PAGE 143

Reusingthevalidationeortbetweenabstractionlevelsca nreduceoverallvalidation eort.Assertionscanbetreatedasconstraintsofsystemsp ecications.Thereforethe assertionreusecanpartiallyguaranteetheconsistencybe tweendierentabstractionlevels. In[ 42 ],KasuyaandTesfayepresentedamechanismtoconstructand reusetemporal assertionsinvariousTLMabstractionlevels.Asanalterna tive,testreusecannotonly reducethetestgenerationandsimulationtime,butalsoena bletheco-simulationbetween dierentabstractionlevels.In[ 12 ],Bombierietal.proposedatransactor-baseddynamic vericationmethod.Byusingtransactors,theTLMtestbenc hescanbereusedduring theTLM-RTLco-simulation.In[ 14 ],Bombierietal.presentedaformaldenitionof functionalequivalencebasedoneventsorderwithouttimin ginformation.However, theydidnotprovideanyimplementationdetailsforcheckin gtheproposedfunctional equivalence.Similartoourwork,theresearchofincrement alABVmethodologydescribed in[ 13 ]usesvariouskindsofassertionstocheckthecorrectnesso fTLM-to-RTLrenement. However,sincetheirworkisbasedontransactors,itisrequ iredthatRTLimplementations shouldbereadybeforetheco-simulation.Also,theirworkd oesnotprovideanymethods abouthowtoactivatealltheinstrumentedassertions.Ther eforeitisdiculttoguarantee thatthesimulationcanachievetherequiredassertioncove ragequickly.Furthermore,their methodappliesassertionsonTLMspecicationsonly.Itjus tmonitorsprimaryinputand outputsignalswithoutinvestigatingRTLimplementationd etails. Tothebestofourknowledge,whencheckingrenementconsis tencyandcorrectness, existingapproachesfocusontest/assertioncoveragewith outconsideringmoredetailssuch asthecorrelationbetweenTLMandRTLassertions.Ourappro achistherstattemptto reusethevalidationeorttoenableassertion-basedequiv alencecheckingbetweenTLM andRTLmodels. Figure 7-1 showstheframeworkofourmethodology.FirstbyanalyzingT LM specications,theTLMassertionsandtestscanbeautomati callyderivedaccordingto speciedfaultmodels.Nexttherenementprocesstranslat estheTLMassertionsand 143

PAGE 144

testsforRTLvalidationusingourproposedmappingrules.T herenedassertionswill beinstrumentedinRTLimplementations.Therenedtestswi llbeappliedontheRTL implementationsandtheoutputofthetestsandtheactivate dassertionswillbemonitored byaRTLassertionchecker.Finally,bycomparingsimulatio ntracesrecordedbyTLMand RTLassertioncheckers,theequivalencecheckerreportsth eresults. ack req VERT clk Assertion Checker Assertion Checker Assertions Assertions RTL Refinement Test Mapping Rules Rules Assertion Mapping Equivalence Checker TLM Validation RTL Validation RTL TLM Assertions Assertions Tests TLMRTL Tests TLM Figure7-1.Ourequivalencecheckingframework 7.2AFrameworkforCheckingTLM-to-RTLFunctionalEquival ence Ourmethodologyhasthreeimportantsteps:i)automaticval idationofTLM specications(i.e.,TLMassertion/testgeneration),ii) validationeortrenement, andiii)assertionbasedequivalencechecking.Thefollowi ngsubsectionsdiscusseachof thesestepsindetail.7.2.1AutomaticTransactionLevelValidation SystemCTLMemphasizesthefunctionalityofthedatatransf ersinsteadofactual implementation.EssentiallyaSystemCTLMdesignintercon nectsasetofprocesses usingtransactions(i.e.,C++functioncalls)forcommunic ation.Eachprocessdoes thefollowingtasks:receivingdata,processingdataandse ndingdata.Duetovarious complexconstructsinC++,extractingallsuchbehaviortoe nableautomatedanalysisand 144

PAGE 145

validationisdicult.Furthermore,theunderlyingcomple xSystemCscheduleraggravates themodelingcomplexity.Infact,investigatingthegenera lfeaturesofSystemCTLMis notnecessaryforfunctionalvalidationofTLMmodels.ForT LM,themostimportant factorsarethetransactiondata,thetransactionrowandth etransactioneventorder.So duringourassertion/testgenerationprocess,thesefacto rsneedtobeconsidered.Other elementscanbeselectivelyabstracted.7.2.1.1GenerationofTLMAssertions Assertionsareusedtospecifytherequiredfunctionalbeha viorsofasystem.To investigatetheequivalencebetweenTLMandRTLmodels,wen eedtoexploreasmany assertionsaspossible.Inourmethod,wedeneasetof faultmodels toachieveacomplete setofassertions.Eachfaultindicatesarequired\designb ehavior"whichmaybeviolated duringthesystemdesign.Forexample,whenvalidatingades iredscenariodescribedby a sequence p( sequence isaPSLtermwhichindicatesasequentialexpression),weus e thefollowingPSLstatementpairstodetectwhetherthesequ encepwillhappennally. The Prop 1 1assertsthatthesequence p must\ eventually !"holdstronglyduringthe simulation,and Prop 1 2isusedtorecordtheassertioncoverageduringthesimulat ionby usingvericationdirective\cover". Prop1_1:asserteventually!p;Prop1_2:cover(p); WeconsiderthethreeTLMfaultmodelswhicharedescribedin Section 3.1.1.2 Transactiondatafaultmodeldealswiththepossiblevaluea ssignmentforeachpartofthe transactiondata.However,forpropertygeneration,dueto thelargesizeofvaluespace, tryingallpossiblevaluesofadataisinfeasible.Bychecki ngeachbitofavariable(data bitfault)separately,thedatacontentcoveragecanbepart iallyguaranteed.Thefollowing isanexampleofadatafault. 145

PAGE 146

//Thesecondbitof"packet.parity"canbe1.asserteventually!(packet.parity==2);cover(packet.parity==2); Transactionrowfaultmodelhandlesthecontrolsalongatra nsactionrow.Toensure transactionrowcoverage,onecancoverbranchconditionsw hichexistin if-then-else or switch-case statements.Thegoalistocheckallpossibletransactionro ws.Thefollowingis anexampleofatransactionrowfault. //Theconditionpacket.to_chan=1canbetrue.asserteventually!(packet.to_chan==1);cover(packet.to_chan==1); Transactioneventindicatestheexecutionstageofatransa ctionortheinteraction betweenprocesses.Therefore,duringtheequivalencechec king,theorderofevents shouldbeinvestigated.Inourmethod,weconsidervariouse ventsintwocategories:1) eventsofprocedurecalls,suchas read and write ,and put and get operations;and2) synchronizationevents,suchas wait and notify operations.Thefollowingisanexampleof areadprocedurecall. //Theeventa=A.read()canbeactivated.asserteventually!{A==a};cover{A==a}; Itisimportanttonotethatanassertionthatisgeneratedfr omtheabovethreefault modelsactivateaspecicfunctionalscenario.Inourmetho ditjustactslikeafunctional checkpointtomonitortheoccurrenceofaspeciceventinst eadofdescribingacomplex scenario.Theorderoftheassertionactivationsplaysanim portantroleandwillbe handledwhenverifyingfunctionalequivalencedescribedi nSection 7.2.4 7.2.1.2GenerationofTLMTests Ourequivalencecheckingapproachisbasedonsimulation,s oweneedtogenerate teststocoveralltheassertionsderivedusingthemethodpr oposedinSection 7.2.1.1 146

PAGE 147

Conventionalmethodsusemillionsofrandom/constrainedrandomtests,however,itis diculttoexercisealltheassertionsinareasonabletime. Asanalternative,directed testsarepromisingsincetheyexploitthestructuralinfor mationandcanconvergeto100% assertioncoveragequickly.However,mostdirectedtestge nerationmethodsneedhuman interventionwhichiserror-proneandcostly.Inourframew ork,wedevelopedatoolwhich canenableautomaticdirectedtestgeneration.Itisimport anttonotethatforrandomtest basedmethods,wemayrequirealargesetoftestsforeachass ertion.However,whenusing directedmethods,wejustneedtoderiveonetestforeachass ertion.Chapter 3 givesthe detailsforTLMtestgeneration.7.2.2RenementofTLMAssertionsandTests WhenTLMassertionsandtestsareready,weneedtorenethem toRTLcounterparts forreuse.Amajorchallengeinthetranslationishowtobrid geabstractiongapbetween TLMandRTLmodels.Asweknow,TLMdesignissignicantlydi erentfromitsRTL implementationininput/outputportdenition,internals tructureandtiminginformation. ThusforTLM-to-RTLvalidationrenement,itisnecessaryt oprovidesuchmissing informationwhichisalsoneededduringthemanualorautoma ticTLM-to-RTLsynthesis. InourframeworkshowninFigure 7-1 ,the ValidationEortReuseTool (VERT)isa majorcomponentwhichenablesTLM-to-RTLrenementbyspec ifyingrules.Theinputs ofVERTareTLMassertions/testsaswellasaValidationRen ementSpecication(VRS) whichcontainstherulestoguidethevalidationrenement. GenerallyaVRScontains threepartsasfollows. SymbolMapping speciesthenameandtypemappingbetweenTLMvariablesand RTLsignals. AssertionRenementRules specifypatternsandtiminginformationforRTL assertions. TestRenementRules specifytheinterfaceprotocolsandtiminginformationfor RTLinputstimulus. Thefollowingsubsectionsdescribeeachpartindetails. 147

PAGE 148

7.2.2.1SymbolMapping Inourprototypetool,weuseSystemCfortransactionlevelm odelingandVerilogfor RTLmodeling.Duetothenamingconventioninconsistencybe tweenTLMspecications andRTLimplementations,duringthevalidationrenement, itisnecessarytohavea symboltablewhichspeciesthenamemappings.Eachitemint hesymboltabledenes thecorrespondencebetweenTLMvariablesandRTLvariables .Generallyitprovidesthe followinginformation:i)namemapping,ii)datetypemappi ng,andiii)bitmapping.The followingisanexampleofsymbolmapping. SYMBOL_MAPPING bit[7:0]parity=packet.parity;bit[7:0]header={packet.payload_sz[7:2],packet.to_ch an[1:0]}; bit[7:0]payload[0..packet.payload_sz-1]=packet.payl oad[0..packet.payload_sz-1]; END_SYMBOL_MAPPING Foreachsymbolmappingitem,thelefthandsideistheRTLdat adeclaration,and therighthandsideisthebitmappingdetailsfromTLMdatato RTLdata.TheVRS allowstheusertospecifytheRTLdatausingtheconcatenati onofseveralTLMdata. AlsoitsupportsthemappingfromanarrayofTLMdatatoanarr ayofRTLdata.For example, parity isaRTLdatawith8bits.ItreferstotheTLMvariable packet.parity TheheaderisaRTLdatawhosemostsignicantsixbitscorres pondstotheTLMdata payload sz andtheleastsignicanttwobitscorrespondtotheTLMdata to chan .The RTLdata payload isanarraywherethewidthofeachelementis8bits.The( i +1) th element payload [ i ]correspondstothe( i +1) th elementoftheTLMdata packet:payload [ i ]. 7.2.2.2AssertionRenementRules Accordingtothedenitionin[ 3 ],aPSLorSVAassertionconsistsoffourlayers: BooleanLayer denestheBooleanexpressionsofsignalswhichareevaluat edina singleevaluationcycle. 148

PAGE 149

TemporalLayer describesassertionsinvolvingcomplextemporalrelation sbetween Booleanexpressions.Temporalassertionsareevaluatedov eraseriesofevaluation cycles. Vericationlayer speciesthedirectivestovericationtoolstohandlethet emporal assertions. Modelinglayer isusedtomodelthebehaviorofdesigninputs. OurTLM-to-RTLassertionrenementonlyconsiderstherst threelayerssince thefourthlayerisnotrelevantinourframework.Aspresent edinSection 7.2.1.1 ,the generatedTLMassertionsareinthesimplesyntaxlike\asse rteventually!p".Most ofthemaretemporalassertionsinvolvingtransactiondata onlywithoutanyclockand controlsignalinformation.However,RTLassertionsgener allyhavesuchlowerlevel details.Therefore,duringtheassertionrenement,wenee dtoconsiderclockexpression andcontrolsignals.Ifallsuchinformationisprovided,th eassertionrenementcanbe donebyinsertingthetiming(i.e.,clockexpression)andco ntrolinformationaswellasby substitutingsymbols. SYMBOL_MAPPING bit[1:0]data_o_fsm=tmp_packet.to_chan;...... END_SYMBOL_MAPPINGASSERTION_SPEC `set_clock(posedgeclock);......`control tmp_packet.to_chan@$rose(write_enb[%tmp_packet.to_chan]); ...... END_ASSERTION_SPEC Intheaboveassertionrenementrules, tmp packet.to chan isaTLMvariablethat denotesthetargetslaveaddressofthepacket.Fromthesymb olmapping,wecangure 149

PAGE 150

outthecorrespondingRTLinternalsignalis data o fsm whichisa2-bitregister.In the ASSERTION SPEC block,thedirective 8 set clock setstheclockexpressionforthe renedassertions.BecauseinRTLdierentvalueofcontrol signalsmayspecifydierent meaningtoinputdatasignals,weusethedirective 8 control tosettheRTLcontrolsignals duringtheTLMdatarenement.Therstparameterof 8 control isaTLMvariable thatappearsintheTLMassertion.Thesecondparameteristh ecorrespondingRTL controlsignalexpressionfortheTLMvariable.Inthisexam ple,onlywhentheRTLsignal write enb[%tmp packet.to chan] asserts,theRTLsignal data o fsm canindicatethetarget slaveaddress.Here% tmp packet:to chan denotesthevalueof tmp packet.to chan TLMassertion:cover(tmp_packet.to_chan==1);RTLassertion:coverproperty(@(posedgeclock)($rose(write_enb[1])&&data_o_fsm[1:0]==2'd1)); Theaboveexampleshowstheusageoftheassertionrenement rules.TheTLM assertionwantstocheckwhetherthepacketcanbedelivered totheslave1.Wecannd thattheRTLassertionincludestheclockexpression.TheVE RTsubstitutestheTLM variable tmp packet:to chan foritsRTLsignal data o fsm accompaniedbyitscontrol signal$ rose ( write enb [1]). 7.2.2.3TestRenementRules Fromthesymbolmappings,wecangetthesizeinformationofe achRTLsignalas wellasthebitcorrespondencebetweenTLMdataandRTLdata. However,theRTL teststimulusisatimedsequenceofdatasignalinputscontr olledbycontrolsignals. Thereforeitisrequiredthatthetestrenementrulesneedt obeprogrammable.Similar toVerilogtestbench,VRSsupportsbasicprogrammingconst ructslike if-then-else and for-loop statements,sub-functionsandsoon.Inessence,thetestre nementrulesconsists ofasequenceofstatements.Thesestatementsarewellorgan izedtodescribethetiming 150

PAGE 151

sequenceofRTLdatainputs.Basedonthesymbolmappingsand thecompiledtest renementrules,the VERT willproduceoneRTLtestforoneTLMtest. TEST_SPECrouter(packet) ......main: begin initialize();reset();#5PKT_VALID=1'b1;DATA=header;for(inti=0;i
PAGE 152

TLM2SMV Specification SystemC TLM Fault Model Specification SMV Properties Model Checker (SMV) TLM-Test-Gen TLM Tests Simulator RTL Implementation AutomaticManual Coverage Analysis Formal Model Generation ProcessingProcessing TLM2RTL Validation Refinement Specification TLM RTL TLM Assertions Figure7-2.Thestructureofourprototypetool 7.2.3.1TLM2SMV ImplementedbasedontheC++parserElsa[ 57 ], TLM2SMV canautomatically translatetheSystemCTLMtoaSMVspecicationandderivepr opertiesbasedonthe faultmodels.Duetothecomplexdatatypedenitionandcomp lexconstructsdenedin SystemCTLMlibraryles,directtranslationtoSMVwillcau sethestatespaceexplosion. Soinourtool,wesimplifysuchdenitionandpredenethemf orSMVtransformation. Forexample,werestrictthequeuesizeforTLMFIFOchannels .InSystemC,aninteger is32-bit(with2 32 states ).However,wereduceitssizeto8bits(with2 8 states )duringthe SMVtransformation. BeforetheTLMtoSMVtranslation,preprocessingprocedure of TLM2SMV will dothefollowingthreetasks:i)eliminatetheheaderlesan dthecomments,ii)addthe necessarypredeneconstructs,iii)convertthedatatypei fnecessary.Then TLM2SMV willtransformtheTLMspecication.AsdescribedinSectio n 2.1.2 TLM2SMV will extractbothstaticanddynamicinformation.Atthemeantim e,italsocollectsthe 152

PAGE 153

informationsuchastransactionrelevantdata,andbranchc onditionsfortheproperty generation.Finallybasedonthecollectedinformation,we cangetbothaformal specicationinSMVandpropertiesderivedbyspeciedfaul tmodels.ByusingCadence SMVverier[ 56 ],wecangetasetofcounterexamples.TheTLMtestsareextra ctedfrom thesecounterexamples.7.2.3.2TLMTestGeneration Whenaspeciedsafetypropertyisfalse,SMVmodelcheckerw illgeneratea counterexampletofalsifyit.ThegeneratedTLMcounterexa mpleisintheformofa sequenceofstateassignments.Thissequencestartsfromr ststate(initialstate)andends attheerrorstatewhichviolatestheproperty.IftheConeof Inruence(COI)isenabled duringthepropertychecking,eachstatewillonlycontaint hevariableswhicharerelevant tothespeciedproperty.Thegeneratedcounterexampleisr enedtoproducetheTLM test.7.2.3.3TLM2RTL BecauseSystemCTLMfocusesonthesystemlevelmodeling,th egeneratedTLM tests/assertionslacktheimplementationlevelknowledge .SothegeneratedTLM tests/assertionsaredierentfromRTLtests/assertionsa ndcannotbedirectlyused tovalidateRTLimplementation.Forexample,mostlooselyt imedTLMmodelsare tooabstractandassumethatatransactionhappenedinoneor asequenceoffunction calls.However,aRTLdesignhasmuchmoredetailsanditneed sthedetailedtiming informationforeachsignal.Inourframework,theusershou ldprovideaVRSwhich providesthemappingrulesfortheTLMtoRTLtest/assertion translation.Withthe generatedTLMtests/assertionsandtheVRSasinputs,the TLM2RTL cantranslate theTLMtests/assertionstoRTLtests/assertions.Finally ,thecoverageoftheTLM implementationwillbereportedwhensimulatingthegenera tedRTLtests/assertionson RTLdesigns. 153

PAGE 154

7.2.4Assertion-BasedFunctionalEquivalence Aftertheassertionandtestrenement,weneedtoperformth esimulationonboth TLMandRTLdesignstochecktheassertion-basedfunctional equivalence.Asshown inFigure 7-1 ,thereisaequivalencecheckerwhichmonitorsboththeTLMa ndRTL simulationresultandreportstheequivalenceresultbased onitscomparison. TLMandRTLindicatedierentabstractionlevelsofthesyst em.Thetraditional simulationbasedmethodcanonlyguaranteethecorrectness byenumeratinginputtests andcomparingtheprimaryoutputresults.However,thesign icantdierenceofinternal structureofTLMandRTLdesignsisofteneclipsed.Therefor enorelationontheinternal structurecanbeassumedduringthesimulation.Asafunctio nalconstraint,theassertion canbeusedasacheckpointduringthesimulation.Basedonth einherentobservability, theexerciseofsuchcheckpointsenablesrevealingtheinte rnalfunctionalbehaviors. 7.2.4.1Assertion-BasedFunctionalCoverage Duringsimulation,anassertioniscoveredmeansthatthesp ecicfunctionalscenario isactivated.Thereforethecoverageoftheassertionsindi catestheadequacyofthe functionalvalidation.Let T beaTLMdesignand R beaRTLdesignof T .Wegenerate asetofTLMassertions T assertion accordingtothespeciedfaultmodelsof T ,andwe obtainasetofTLMtests T test toactivatesuchassertions.ByusingVRS,wecanrene the T test toaRTLtestset R test ,andrene T asserion asasubsetoftheRTLassertion set R asserion .Whenrunningthe T test and R test on T and R individually,wecangetthe assertioncoveragedenedasfollows. Denition10. GivenaTLMspecication T anditsRTLimplementation R ,byapplying T test on T and R test on R ,theassertioncoveragecanbecalculatedas: T coverage = # ofexercisedTLMassertions j T asserion j R coverage = # ofexercisedRTLassertions j R asserion j 154

PAGE 155

Inourframework,therearetwokindsofassertions:i)TLMas sertionswhichare automaticallygeneratedfromtheTLMspecications,andii )RTLassertionswhichare renedfromtheTLMassertions.TheRTLprogrammerscanalso provideadditional assertionsbasedonotherfaultmodelsandcornercasescena rios.Therefore100%TLM assertioncoveragemaynotindicate100%RTLassertioncove rageincaseadditionalRTL assertionsareintroduced.7.2.4.2AssertionOrdering TheassertionorderingplaysanimportantroleinTLM-to-RT Lequivalencechecking. ForaTLMorRTLdesignwhichisinstrumentedwithalargenumb erofassertions,during thesimulation,atestmayexerciseasequenceofassertions .Anassertionindicatesa functionalcheckpoint.Suchsimulationresultofatestlea dstoan assertiontrace which revealsthetemporalorderofcheckedfunctionsinasystemb ehavior.ForaTLMtestand itsrenedRTLversion,whenapplyingthemontheTLMandRTLd esignsindividually, itisrequiredthattheTLMfunctionsandRTLfunctionshappe nconsistently.Inother words,theTLMassertionsandcorrespondingRTLassertions shouldhappenintheir tracesinthesameorder. Itisdiculttodeterminetheorderoftheassertionsduring thesimulationofa test.Sincetheassertionsbelongtodierentparallelproc essesintheTLMspecications andRTLimplementations,eveninthesameassertiontraceth eassertionsmaynot beactivatedlinearly.Thatmeansseveralassertionsmaybe exercisedsimultaneously. Inaddition,duetotheexistenceofloopstructureinadesig n,anassertionmaybe exercisedseveraltimesinadesign.Thiswillfurtherincre asethedicultyinassertion matchingbetweenTLMtracesandRTLtraces.Inspiredbythea lgorithmproposedby Lamport[ 50 ],inourframework,eachassertionactivationinatraceisa ssociatedwitha \timestamp"toindicatethe happensbefore (markedby )relation.Weusethetimed assertionintheformof( a;t )todenotethattheassertion a happensatclockcycle t 155

PAGE 156

Denition11. Giventwotimedassertions ( a;t 1) and ( b;t 2) inanassertiontrace.The relationsbetweenthemareasfollows. ( a;t 1) happensbefore ( b;t 2) i t 1
PAGE 157

schedulercallsthe update ()functiontohandlethependingprocessesregisteredbyus ing the request update ()function.Eachtinydeltacycleconsistsofthesetwostep swithout advancingthesimulationtime.Thereforethedeltacycleca nbeutilizedfororderingthe assertions.Forassertionordering,weneedtouseaglobalv ariableasacounterofdelta cycles.Thiscountercanbeusedasthetimestampforasserti ons.Iftwoassertionshappen inthesamedeltacycle,thentheyareconcurrent.Otherwise thereisa\happenbefore" relationbetweenthem.Let'stakethefollowingsimpleprog ramasanexample. //process1//process2while(true){while(true){ a=FIFO.read();FIFO.write(random()); }} Assumingthesizeofthe FIFO channelis1.Therealactionsequenceoftheabove codecanbe\( write; 1),( read; 1),( write; 2),( read; 2), ::: ".Foreachdeltacycle,only onewriteandreadpaircanhappen.Sowecanndtheorder( write; 1) ( write; 2)and ( read; 1) ( write; 2). 7.2.4.3AssertionBasedFunctionalEquivalence Inourframework,wedenefunctionalequivalencebasedont heassumptionthatif aTLMtestcantriggeraTLMassertion,thenitsRTLcounterpa rtwillalsotriggerthe correspondingRTLassertion.Itisimportanttonotethatin thischapterwedonotintent tointroducenewmeaningoftheclassical equivalencechecking .Ourmethodstillrelieson thesameconcept-iftwodesignsareequivalent,whengiving thesameinputtests,they willproducethesameoutputs.Ourgoalistoincreasethecon denceofTLM-to-RTL functionalequivalencecheckingunderthemonitoringofas sertions. TherenementprocessisdescribedbytwofunctionsAR forassertionrenement and TR fortestrenementasfollows. AR : T assertion R assertion TR : T test R test 157

PAGE 158

Wealsodenethefunctions M TLM and M RTL toindicatetherelationbetweentestsand assertions,i.e.,whattheassertionsareactivatedduring thesimulationofagiventest. M TLM : T test 2 T assertion M RTL : R test 2 R assertion M TLM indicateswhichTLMassertionsarecoveredbyagivenTLMtes t. M RTL indicateswhichRTLassertionsarecoveredbyagivenRTLtes t.Basedontheabove denitions,thedenitionofTLM-to-RTLequivalenceisgiv enasfollows. Denition12. GivenaTLMspecication T anditsRTLimplementation R T and R areassertionequivalenti T test canachieve100%TLMassertioncoverageand 8 t 2 T test :M RTL ( TR ( t )) f AR ( a 1 ) ;AR ( a 2 ) ;:::;AR ( a n ) g whereM TLM ( t )= f a 1 ;a 2 ;:::;a n g : Theassertionequivalenceonlydenetheassertioncoverag eforeachtest.Infact, thereisatemporalrelationbetweenassertions.Iftheasse rtionequivalenceconsidersthe eventorder,wecallit stronglyassertionequivalent Denition13. GivenaTLMspecication T anditsRTLimplementation R T and R arestronglyassertionequivalenti T and R areassertionequivalent;and 8 t 2 T test ,theTLMassertionscoveredby t andtheRTLassertionscoveredby TR ( t ) areactivatedinthesameorder. Figure 7-3 illustratesanexampleofassertionequivalence.Assuming theTLM specicationandtheRTLimplementationareassertionequi valentand t isaTLMtest and t 0 = TR ( t ),wecanget M TLM (t)= f a1,a2,a3 g and AR ( M TLM ( t )) = f b1,b2,b3 g which isasubsetof M RTL ( t 0 ).However,theassertionactiviationorderisnotconsiste nt( a 2 happensbefore a 1,but b 1happensbefore b 2).Therefore,inthiscase,theTLMdesignand RTLdesignareassertionequivalentbutnotstronglyassert ionequivalent. 158

PAGE 159

RTL TLM t t' a1 a2 a3 b3 b1 b2 Figure7-3.Anexampleofassertionequivalence 7.3CaseStudy Thissectionpresentstwocasestudies:aroutersystemanda simpliedversion ofthepipelinedAlphaAXPprocessor[ 76 ].Weusetheprototypetool(describedin Section 7.2.3 )toautomaticallygeneratetheTLMassertionsandtestsasw ellasrened RTLassertionsandtests.Theexperimentalresultsareobta inedonan3GHzAMD Opteronserverwith16GRAMusingLinuxoperationsystem.7.3.1ARouterExample Figure 2-11 showsthestructureoftheTLMspecicationoftherouterexa mple.The mainfunctionoftherouteristoparsetheincomingpacketsa nddistributethemtotarget slaves.TheTLMandRTLpacketformatsareshowninFigure 7-4 .Thepacketconsistsof threeparts:header,payloadandparity.Theheaderhas8bit s,bit0andbit1areusedas theaddressofoutputport(i.e.,targetslaveaddress).The other6bitsindicatethesizeof thepayload.Sothemaximumpayloadsizeis63.Thelastbyteo fthepacketistheparity ofbothheaderandpayload.InTLMdesign,themastermodulec reatesapacketrst. Then,themastersendsthepackettotherouterforpackagedi stribution.Therouterhas oneinputportandthreeoutputports.Eachportisconnected toaFIFObuer(channel) whichtemporarilystorespackets.Therouterhasoneproces s route whichisimplemented 159

PAGE 160

asa SC METHOD .Triggeredbytheincomingpackets,therouteprocessrstc ollectsa packetfromthechannelconnectedtothemaster,nextdecode stheheaderofthepacket todeterminethetargetslaveaddress,andthensendsthepac kettothechannelconnected tothetargetslave.Finally,theslavemodulesreadthepack etswhendataisavailable intherespectiveFIFOs.Thetransactiondata(i.e.,packet )rowsfromthemastertoits targetslaveviatherouter.Thetransactionrowiscontroll edbythevariable to chan in thepacketheader. // Packet description in TLMclass Packet {}; public: sc_unit<2> to_chan; sc_unit<6> payload_sz; sc_unit<8> parity; sc_unit<8> payload[63]; a) TLM Packet 7 6 5 4 3 2 1 0 length = N 1 byte chan......N-byte Payload 1 byte data[N] data[1]parity b) RTL Packet Figure7-4.ThepacketformatoftherouterinTLMandRTL IntheTLMspecication,theI/Oportoftherouterwilldeliv eronewholepacket atatime.However,inRTLimplementation,duringeachclock cycleonlyabytecan betransferredthroughtheI/Oports.Figure 7-5 showstheRTLI/Ointerfaceofthe routerexample.Duringthevalidationrenement,weneedto specifysuchmappingrules betweentheTLMandRTLdesignsusingaVRS.Section 7.2.2.3 showsthepartialVRS detailsoftherouterexample.Forinstance,inthesymbolma ppingpart, packet:to chan inTLMcorrespondstotheRTLdata header [0:1]and packet:payload sz corresponds to header [2:7].ThearrayofTLMdata packet:payload willbemappedtoRTLdata 160

PAGE 161

payload ,andtheTLMvariable packet:parity correspondstoRTLvariable parity .All suchRTLpacketdatawillbeappliedtoinputsignal DATA [7:0] Router DATA[7:0]ERR PKT_VALIDCLKRST CHAN0[7:0]CHAN1[7:0] ENB0 VLD0ENB1 VLD1 CHAN2[7:0] ENB2 VLD2 Figure7-5.TheI/Ointerfaceoftherouterexample InSection 7.2.2.3 ,wehaveshownthepartialVRStospecifytestandassertion renementrules.ByusingourtoolVERT,95TLMassertionswe regeneratedaccording totheproposedfaultmodels.Foreachassertion,wederived apropertyanduseditas aninputofamodelchecker.Themodelcheckergeneratedonec ounterexample(test) toexerciseeachassertion.Soweobtained95TLMassertions and95TLMtestsfrom theTLMdesign.Table 7-1 givesthedetails.Therstrowdenesthefaulttypes.The secondrowshowsthenumberofTLMassertionswithdierentf aulttype.Thethirdrow indicatesthenumberofgeneratedTLMtests.Thelastrowgiv esthetestgenerationtime (inminutes)usingtheSMVmodelchecker. Table7-1.Assertionrenementfortherouterexample FaultTypeDataFaultsFlowFaultsEventFaultsTotal NumbersofTLMAssertions 884395 NumbersofTLMTests 884395 TestGenerationTime(min.) 73.702.6031.50107.8 DuringtheTLMspecicationparsing,wedidnotconsiderthe FIFOchannel informationbecauseitisdenedinthestandardSystemClib rary.Thereforethereis 161

PAGE 162

noassertionfortheFIFOchannelsintherouterexample.How ever,toimprovethe RTLcodecoverage,wemanuallycreated4TLMtests(2testsfo rFIFOoverrow,1test forresetcheckand1testforasynchronousread).Finallywe got99TLMtestsand99 RTLtestsforvalidationpurposes.Itisimportanttonoteth atthegenerationofTLM assertions/testsandrenementareindependent.Inotherw ords,TLMassertionsandtests cancomefrommultiplesources.WeusetheVRStodescribethe bothassertionandtest renementrulesfortherouterexample.Undertheguidanceo ftheVRS,ourtoolVERT cantranslatetheTLMtestsandTLMPSLassertionstothecorr espondingRTLtestsas wellasRTLassertionsintheformofSVA. RST = 0;ENB0 = 0;ENB1 = 0;ENB2 = 0;PKT_VALID = 0;#20 RST = 0; #5 RST = 1; #5 PKT_VALID = 1'b1; p -> payload_sz = 4; p -> to_chan = 1; Initialization Reset Sequence #10 ENB1 = 1; Read p -> payload[0] = 128;p -> payload[1] = 0;p -> payload[2] = 0;p -> payload[3] = 0; #10 DATA = 8'b00000000; DATA = 8'b10000100; p >parity = 132; #10 DATA = 8'b10000000;#10 PKT_VALID = 1b'0; #10 DATA = 8'b00000000; DATA = 8'b00010001; Compose #10 DATA = 8'b00000000; //RTL Test ($rose(write_enb[1]) && date_o_fsm[1:0]==2'd1)); RTL assertion: cover property (@(posedge clock) //TLM Test TLM assertion: cover (tmp_packet.to_chan = 1); Figure7-6.AnexampleofTLM-to-RTLrenement 162

PAGE 163

Figure 7-6 showsanexampleofTLM-to-RTLrenement.Thegoalofthisex ampleis toexercisethescenariothatthepacketcanbesenttoslave1 .ByusingtheTLMtest,we canactivatetheTLMassertion.Similarly,theRTLtestcana ctivatetheRTLassertion. WeappliedtheTLMandRTLtestsontheTLMandRTLdesignsinde pendently. FortheTLMdesign,wecanget100%coverageonbothcodeandas sertions.Forthe RTLdesign,wemeasuredvariouscoveragemetrics 1 usingSynopsysVCScmView[ 82 ]. Table 7-2 showsthecoverageobtainedusingthegeneratedtests.Duet osomeunreachable codeandmissing\else"statementsinRTLimplementation,i tisnotpossibletoobtain 100%coverageinallthecategories.Itisimportanttonotet hatthedirectedtestscan onlygive94.7%assertioncoverageontherenedassertions .Weinvestigatedtheassertions whicharenotcovered.Thereasonisthatthegeneratedasser tionsandteststryto activatethescenario to chan =3whichisusedasanerrorstateinTLM.SinceRTL implementationdidnotconsiderthiscase,i.e.,sendingap ackettoslave3,wemodied theRTLimplementationandnallywecanget100%assertionc overage. Table7-2.RTLcoveragefortherouterexample SourceConditionFSMTogglePathAssertion 99.5%76.6%100%76.6%73.6%94.7% Whenapplyingatestduringthevalidation,severalTLMorRT Lassertionsmaybe exercised.TochecktheequivalencebetweenTLMandRTL,our prototypetoolrecorded thesimulationorderforassertionactivation.Suchinform ationisusedtocheckthe equivalencebetweentheTLMandRTLdesign.Ourresultshows thattheTLMand RTLdesignsoftherouterexampleareassertionequivalent, Forstronglyequivalence checking,weonlyusedtheassertionsderivedfromthetrans actionrowandeventfaults. Bymatchingthetimedassertionsontheassertiontraceofea chtest,itshowsthatthe TLMandRTLdesignsoftherouterexampleisalsostronglyass ertionequivalent. 1 TheassertioncoveragecannotbeobtainedbyVCScmView. 163

PAGE 164

7.3.2APipelinedProcessorExample InFigure 2-13 ofSection 2.3.3 ,wegivetheTLMspecicationstructureoftheAlpha AXPprocessor.AsshowninTable 7-3 ,wegenerated212TLMassertionsusingvarious faultmodelsfortheprocessormodel.ByusingSMV,wegenera ted212TLMtests(117 testsfordatafaults,86testsforrowfaultsand9testsfore ventfaults)toexerciseallsuch assertions. Table7-3.AssertionsrenementfortheAlphaAXPprocessor FaultTypeDataFaultsFlowFaultsEventFaultsTotal NumbersofTLMAssertions 117869212 NumbersofTLMTests 117869212 TestGenerationTime(min.) 369.0010.830.03379.86 Weappliedallthegeneratedtestsundertheobservationofo urtoolVERT.According totheresultsprovidedbyVCScmView,weobtainedtheRTLimp lementationcoverage reportshowninTable 7-4 .Wefoundthatthesourceandconditioncoveragecannot beimprovedfurtherbecausealltheuncoveredcodearedueto unreachable MISSING ELSE anddefault CASEITEM statementsthatdonotexistintheRTLimplementation. Weusedalltheassertionforassertionequivalencecheckin g,andtheresultshowsthat assertionequivalencecanbeachieved.Forstronglyequiva lencechecking,wedidnot includetheassertionsderivedfromthetransactiondatafa ultmodel.Bycomparingthe assertionactivationsequence,theequivalencecheckersh owsthatwecanachieveastrong assertionequivalencebyusingthegenerateddirectedtest s. Table7-4.RTLcoveragefortheAlphaAXPprocessor SourceConditionFSMTogglePathAssertion 98.9%97.0%NA70.2%86.3%100% 164

PAGE 165

7.4Summary RaisingtheabstractionlevelinSoCdesignrowcansignica ntlyreducetheoverall designeortbutintroducestwochallenges:i)howtoguaran teefunctionalequivalence betweensystemleveldesignsandlowlevelimplementations ,andii)howtoreuse validationeortbetweendierentabstractionlevels.Toa ddressbothproblems,this chapterproposedamethodologywhichreusesTLMvalidation eorttoenableRTL validationaswellasassertion-basedfunctionalequivale ncecheckingbetweenTLMand RTLmodels.ByextractingformalmodelsfromTLMspecicati ons,wecangeneratea setofassertionsandcorrespondingteststovalidateallth especiedTLM\faults".Then theassertionsandtestscanbetranslatedtotheirRTLcount erpartsusingourproposed VRS.Duringthesimulation,theTLM-to-RTLfunctionalequi valencecanbeveriedbased ontheassertioncoverageandassertionordering.Theexper imentalresultsusingseveral industrialdesignsdemonstratedtheeectivenessandbene tsofourapproach. 165

PAGE 166

CHAPTER8 CONCLUSIONSANDFUTUREWORK SystemCTLMsandUMLactivitydiagramsarewidelyusedtoena bleearly explorationforbothhardwareandsoftwaredesigns.Itcanr educetheoveralldesignand validationeortofcomplexSoCarchitectures.SoCvalidat ionisamajorbottleneckdue tolackofecientautomatedtechniquescoupledwithlimite dreuseofvalidationeorts betweenabstractionlevels.Thisdissertationpresenteda noveltop-downmethodologyfor automaticallygeneratingtestsfromsystem-levelspecic ationsforfunctionalvalidation atdierentabstractionlevels.Thischapterconcludesthe dissertationandoutlinesfuture researchdirections. 8.1Conclusions ExistingSoCvalidationtechniqueswidelyemployacombina tionofsimulationbased techniquesandformalmethods.Simulationbasedvalidatio nusesrandomordirectedtest vectorstocheckthecorrectnessofthedesign.Certainheur isticsareusedtogenerate directedrandomtests.However,duetothebottom-upnature andlocalizedviewofthese heuristics,thegeneratedtestsmaynotyieldagoodcoverag e.Simulationusingdirected testsispromisingforfunctionalvalidation,sincerunnin gtimecanbesignicantlyreduced withfewertestswhilethecoveragerequirementcanstillbe achieved. Amajorchallengetoenabledirectedtestgenerationistoau tomaticallyextracta formalrepresentationfromsystemlevelspecicationsand developanecientcoverage metricthatallowscoverage-drivendirectedtestgenerati on.Chapter 2 and 3 described amodelcheckingbasedframeworkfordirectedtestgenerati on.Thisapproachcan automaticallyextractformalmodelsfromthehighlevelspe cications(includingSystemC TLMsandUMLactivitydiagramsasdescribedinChapter 2 )aswellascangenerate properties(assertions)tocoveralltheerrorsforthegive nfaultmodels(describedin Chapter 3 ). 166

PAGE 167

Mostautomaticdirectedtestgenerationmethods,especial lyformodelcheckingbased techniques,areimpededbythecapacityrestrictionsofcor respondingtools.Toaddress thecomplexityoftestgenerationusingSAT-basedBMC,this dissertationpresentedthree ecienttechniquestoreducetheoveralltestgenerationti me: Propertyclustering exploitedvarioussimilaritiesbetweenpropertiesinaclu ster (describedinChapter 4 )tosharelearnings. Ecientdecisionordering enablesbenecialknowledgesharing(describedin Chapter 5 )betweenpropertiestoavoidrepeatedvalidationeort. Decompositiontechniques triedtoscaledownthepropertycheckingprobleminto severalsub-problems(describedinChapter 6 ).Thelearningfromthedecomposed sub-problemsisbenecialtothetestgenerationoftheorig inalcomplexproperty. Byexploitingthecommonalitiesbetweenproperties,thete stgenerationtimeofaset ofsimilarpropertiescanbesignicantlyreduced. Furthermore,thisdissertationpresentedapromisingmeth odologythatcancheck theTLM-to-RTLfunctionalequivalencebyreusingtheTLMle velvalidationeort.The renedassertionsaswellastestscannotonlycheckthecons istencybetweendierent abstractionlevels,butalsocanbeusedforvalidatingthes ystembehaviorofRTLdesigns. Sinceourmethodcanbeautomated,completereuseofTLMtest swillleadtoadrastic reductioninRTLvalidation. Inconclusion,thisdissertationpresentedanecientfram eworkthatcanautomatically generatetestsfromhigh-levelSoCspecicationsandenabl echeckingdesignerrorsin dierentstagesoftheSoCdesign.Duetodrasticreductioni noverallvalidationeort,this researchwillleadtocost-eectiveandhigh-qualitysyste ms. 8.2FutureResearchDirections Automatedcoverage-driventestgenerationandrenementf orvalidationofSoCis achallengingproblem.Theworkpresentedinthisdissertat ioncanbeextendedinthe followingdirections: 167

PAGE 168

Thecoverage-drivenpropertygenerationwillgenerateala rgesetofproperties,and manyofthemmayactivatethesamescenarios.Consequently, thereexistsalotof redundancyinthederivedtests.Therefore,propertycompa ctioncanbeemployed beforetheautomatedtestgenerationtoreducetherequired numberofproperties.To furtherreducethenumberofdirectedtests,existingtestc ompactiontechniquescan beused. Findawaytoecientlygeneratetestsfordierentdesignsb utusingthesame propertyset.Forexample,spiralmodeliswidelyusedasaso ftwaredevelopment process.Thedesignisoftenslightlymodiedaccordington ewrequirements.Thus weneedtore-generatethenewtestsforthepropertiesofthe previousdesign. Becausemostofthefunctionalityremainsthesame,properl earningtechniquescan beusedtogeneratethenewtests. Currentlymostassertionbasedvalidationmethodsarebase donsimulationforboth TLMandRTLdesigns.Generallyforalargedesign,therewill bethousandsof assertionsthatneedtobecheckedatthesametime.Checking themindependently willstronglyaectthesimulationperformance.Inthewors tcase,activatingone assertionneedsonetest.Thereforeitisnecessarytodesig namethodologythatcan investigatethedependencebetweenassertionsandgenerat easmallsetoftestsforthe simulationbutstillcanachievethesameassertioncoverag e. ItisnecessarytodevelopaframeworkthatcandebugtheRTLl evelfunctionalerrors usingitsTLMspecication.Thiscanhelpdesignerstoquick lyndtheerrorandx it. Post-silicondebuggingisanimportantstageduringSoCdes ign.However,inthe post-siliconstage,allthedebuggingtasksarefocusedats ignallevel.Itisvery diculttodetectandcheckhighlevelfunctionalscenarios .Thereforeitisnecessary torenethehighlevelvalidationeortintogate-levelimp lementation. Thisdissertationdemonstratedthelearningtechniques(i .e.,conrictclauseforwarding anddecisionordering)arepromisingforsystemleveltestg eneration.Itcanbe extendedtootherdomains,suchascircuit-levelvalidatio n.Byincorporatingour learningtechniques,webelievethattheperformanceofcur rentSAT-basedautomatic testpatterngeneration(ATPG)approachescanbedrastical lyimproved. 168

PAGE 169

REFERENCES [1] Y.Abarbanel,I.Beer,L.Gluhovsky,S.Keidar,andY.Wolfst hal.FoCs-Automatic GenerationofSimulationCheckersfromFormalSpecicatio ns.In Proceedingsof ComputerAidedVerication(CAV) ,pages414{427,2000. [2] S.AbdiandD.Gajski.Aformalismforfunctionalitypreserv ingsystemlevel transformations.In ProceedingsofAsiaandSouthPacicDesignAutomation Conference(ASPDAC) ,pages139{144,2005. [3] Accellera.PropertySpecicationLanguage.[updatedMay2 008;citedFebruary2010]. Availableathttp://www.eda.org/ieee-1850/. [4] N.Amla,X.Du,A.Kuehlmann,R.Kurshan,andK.McMillan.Ana nalysisof SAT-basedmodelcheckingtechniquesinanindustrialenvir onment.In Proceedingsof ConferenceonCorrectHardwareDesignandVericationMeth ods(CHARME) ,pages 254{268.Springer,2005. [5] P.Ammann,P.Black,andW.Majurski.Usingmodelcheckingto generatetestsfrom specications.In ProceedingsofInternationalConferenceonFormalEnginee ring Methods(ICFEM) ,pages46{54,1998. [6] F.BalarinandR.Passerone.FunctionalVericationMethod ologybasedonFormal InterfaceSpecicationandTransactorGeneration.In ProceedingsofDesign,Automation,andTestinEurope(DATE) ,pages1013{1018,2006. [7] M.BenedettiandS.Bernardini.Incrementalcompilation-t o-SATprocedures.In ProceedingsofInternationalConferenceonTheoryandAppl icationsofSatisability Testing(SAT) ,2004. [8] J.Bengtsson,K.G.Larsen,F.Larsson,P.Pettersson,andY. Wang.Uppaal-atool suiteforautomaticvericationofreal-timesystems.In ProceedingsofHybridSystems (HSCC) ,pages232{243,1995. [9] D.Beyer,A.Chlipala,T.Henzinger,R.Jhala,andR.Majumda r.Generatingtests fromcounterexamples.In Proceedingsofthe26thIEEEInternationalConferenceon SoftwareEngineering(ICSE) ,pages326{335,LosAlamitos,CAUSA,2004. [10] A.Biere,A.Cimatti,E.Clarke,andY.Zhu.SymbolicModelCh eckingwithout BDDs.In ProceedingsofInternationalConferenceonToolsandAlgor ithmsforThe ConstructionAndAnalysisofSystems ,pages193{207,1999. [11] A.Biere,A.Cimatti,E.M.Clarke,andY.Zhu.Symbolicmodel checkingwithout BDDs.In ToolsandAlgorithmsfortheAnalysisandConstructionofSy stems (TACAS) ,volume1579of LNCS ,pages193{207.Springer,1999. [12] N.Bombieri,F.Fummi,andG.Pravadelli.Ontheevaluationo ftransactor-based vericationforreusingtlmassertionsandtestbenchesatr tl.In ProceedingsofDesign, Automation,andTestinEurope(DATE) ,pages1{6,2006. 169

PAGE 170

[13] N.Bombieri,F.Fummi,andG.Pravadelli.Incrementalabvfo rfunctionalvalidation oftl-to-rtldesignrenement.In ProceedingsofDesignAutomationandTestin Europe(DATE) ,pages882{887,2007. [14] N.Bombieri,F.Fummi,G.Pravadelli,andJ.Marques-Silva. TowardsEquivalence CheckingBetweenTLMandRTLModels.In ProceedingsofInternationalConference onFormalMethodsandModelsforCo-Design(MEMOCODE) ,pages113{122,2007. [15] R.Bryant.Graph-BasedAlgorithmsforBooleanFunctionMan ipulation. IEEE Trans.Computers ,C-35(8):677{691,August1986. [16] L.CaiandD.Gajski.TransactionLevelModeling:AnOvervie w.In Proceedings ofInternationalConferenceonHardware/SoftwareCodesig nandSystemSynthesis (CODES+ISSS) ,pages19{24,2003. [17] K.ChandrasekarandM.S.Hsiao.Integrationoflearningtec hniquesintoincremental satisabilityforecientpath-delayfaulttestgeneratio n.In ProceedingsofDesign AutomationandTestinEurope(DATE) ,pages1002{1007,2005. [18] M.Chen,X.Qiu,andX.Li.Automatictestcasegenerationfor umlactivity diagrams.In ProceedingsofInternationalWorkshoponAutomationonSof twareTest pages2{8,2006. [19] A.Chureau,Y.Savaria,andE.M.Aboulhamid.Theroleofmode l-leveltransactors andumlinfunctionalprototypingofsystems-on-chip:Asof tware-radioapplication. In ProceedingsofDesignAutomationandTestinEurope(DATE) ,pages698{703, 2005. [20] A.Cimatti,E.M.Clarke,F.Giunchiglia,andM.Roveri.NUSM V:Anewsymbolic modelverier.In Proc.ofIntl.ConferenceonComputerAidedVerication(CA V) volume1633of LNCS ,pages495{499.Springer,1999. [21] E.M.Clarke,O.Grumberg,andD.A.Peled. ModelChecking .TheMITpress,2000. [22] D.Das,R.Kumar,andP.P.Chakrabarti.Timingvericationo fumlactivity diagrambasedcodeblocklevelmodelsforrealtimemultipro cessorsystem-on-chip applications.In ProceedingsofAsia-PacicSoftwareEngineeringConferen ce (APSEC) ,pages199{208,2006. [23] K.DattaandP.P.Das.AssertionBasedVericationUsingHDV L.In Proceedingsof theInternationalConferenceonVLSIDesign(VLSID) ,page319,2004. [24] E.W.Dijkstra.Anoteontwoproblemsinconnexionwithgraph s. Numerische Mathematik ,1:269{271,1959. [25] W.Ecker,V.Esen,T.Teininger,M.Velten,andM.Hull.Inter activePresentation: ImplementationofATransactionLevelAssertionFramework inSystemC.In ProceedingsofDesign,Automation,andTestinEurope(DATE ) ,pages894{899, 2007. 170

PAGE 171

[26] M.Ericsson.Activitydiagrams:whattheyareandhowtouset hem. TheRational Edge ,2004. [27] FBK-irstandCMU.NUSMV.[updatedAugust2006;citedAugust 2008]. Available athttp://nusmv.irst.itc.it/. [28] F.Ferrandi,F.Fummi,L.Gerli,andD.Sciuto.Symbolicfunc tionalvectorgeneration forVHDLspecications.In Design,AutomationandTestinEurope(DATE) ,pages 442{446,1999. [29] H.D.Foster,A.C.Krolnik,andD.Lacey. Assertion-BasedDesign,2ndEdition KluwerAcademicPublishers,Boston,MA,2004. [30] F.Ghenassia. TransactionLevelModelingwithSystemC .Springer,2005. [31] N.GuelandA.Mammar.Aformalsemanticsoftimedactivityd iagramsandits promelatranslation.In ProceedingsofAsia-PacicSoftwareEngineeringConferen ce (APSEC) ,pages283{290,2005. [32] H.ZhuandP.HallandJ.May.SoftwareUnitTestCoverageandA dequacy. ACM ComputingSurveys ,29(4):366{427,1997. [33] A.HabibiandS.Tahar.TowardsAnEcientAssertionBasedVe ricationof SystemCDesigns.In ProceedingsofInternationalHighLevelDesignValidation and TestWorkshop(HLDVT) ,pages19{22,2004. [34] A.HabibiandS.Tahar.DesignandVericationofSystemCTra nsaction-Level Models. IEEETransactionsonVeryLargeScaleIntegrationSystems( TVLSI) 14(1):57{68,2006. [35] J.HennessyandD.Patterson. ComputerArchitecture:AQuantitativeApproach MorganKaufmann,Sanfrancisco,CA,2003. [36] P.Hsiung,C.Lin,andC.Liao.Perfecto:ASystemC-basedDes ign-SpaceExploration FrameworkforDynamicallyRecongurableArchitectures. ACMTransactionson RecongurableTechnologyandSystems(TRETS) ,1(3),2008. [37] IEEEP1800WorkingGroup.SystemVerilogAssertion.[updat edSeptember2008; citedMarch2010]. Availableathttp://www.eda.org/sv-ac/. [38] J.Hooker.Solvingtheincrementalsatisabilityproblem. JournalofLogicProgramming ,15(12):177{186,1993. [39] J.Marques-SilvaandK.Sakallah.Grasp:Asearchalgorithm forpropositional satisability. IEEETransactionsonComputers ,48(5):506{521,1999. [40] H.JinandF.Somenzi.Anincrementalalgorithmtochecksati sabilityforbounded modelchecking.In BMC ,pages51{65,2004. 171

PAGE 172

[41] D.Karlsson,P.Eles,andZ.Peng.Formalvericationofsyst emcdesignsusinga petri-netbasedrepresentation.In ProceedingsofDesign,Automation,andTestin Europe(DATE) ,pages1228{1233,2006. [42] A.KasuyaandT.Tesfaye.VericationMethodologiesinaTLM -to-RTLDesignFlow. In ProceedingsofDesignAutomationConference(DAC) ,pages199{204,2007. [43] J.Kim,J.Whittemore,J.Marques-Silva,andK.Sakallah.On solvingstack-based incrementalsatisabilityproblems.In ProceedingsofInternationalConferenceon ComputerDesign(ICCD) ,pages379{382,2000. [44] T.Kogel,M.Doerper,T.Kempf,A.Wieferink,R.Leupers,and H.Meyr. VirtualArchitectureMapping:ASystemCbasedMethodology forArchitectural ExplorationofSystem-on-Chips. InternationalJournalofEmbeddedSystems(IJES) 3(3):150{159,2008. [45] H.KooandP.Mishra.Specication-basedcompactionofdire ctedtestsforfunctional validationofpipelinedprocessors.In InternationalSymposiumonHardware/Software CodesignandSystemSynthesis(CODES+ISSS) ,pages137{142,2008. [46] H.-M.KooandP.Mishra.Testgenerationusing(SAT)-basedb oundedmodel checkingforvalidationofpipelinedprocessors.In Proc.ofACMGreatLakes SymposiumonVLSI(GSLVLSI) ,pages362{365,2006. [47] H.-M.KooandP.Mishra.Functionaltestgenerationusingde signandproperty decompositiontechniques. ACMTransactionsonEmbeddedComputingSystems (TECS) ,8(4),2009. [48] D.KroeningandN.Sharygina.Formalvericationofsystemc byautomatic hardware/softwarepartitioning.In ProceedingsofInternationalConferenceon FormalMethodsandModelsforCo-Design(MEMOCODE) ,pages101{110,2005. [49] M.Lahbib,R.Kamdem,M.Benalycherif,andR.Tourki.AnAuto maticABV MethodologyEnablingPSLAssertionsacrossSLDFlowforSOC sModeledin SystemC. ComputersandElectricalEngineering ,31(4):282{302,2005. [50] L.Lamport.Time,Clocks,andtheOrderingofEventsinaDist ributedSystem. CommunicationofACM ,21(7):558{565,1978. [51] M.ChenandX.QiuandW.XuandL.WangandJ.ZhaoandX.Li.UMLA ctivity DiagramBasedAutomaticTestCaseGenerationforJavaProgr ams. TheComputer Journal ,52(5):545{556,2009. [52] M.DavisandH.Putnam.Acomputingprocedureforquanticat iontheory. Journal ofACM ,7(3):201{215,1960. [53] M.Davis,G.LogemannandD.Loveland.Amachineprogramfort heorem-proving. CommunicationofACM ,5(7):394{397,1962. 172

PAGE 173

[54] M.PrasadandA.BiereandA.Gupta.Asurveyofrecentadvance sinSAT-based formalverication. InternationalJournalonSoftwareToolsforTechnologyTra nsfer (STTT) ,7(2):156{173,2005. [55] J.P.Marques-SilvaandK.A.Sakallah.Theimpactofbranchi ngheuristicsin propositionalsatisability.In Proceedingsofthe9thPortugueseConferenceon ArticialIntelligence ,pages62{74,1999. [56] K.L.McMillan.SMVModelChecker,CadenceBerkeleyLaborat ory.[updatedJune 2006;citedAugust2008]. Availableathttp://www.kenmcmil.com/. [57] S.McPeak.Elsa.[updatedAugust2005;citedAugust2008]. Availableat http://www.eecs.berkeley.edu/~smcpeak [58] P.MishraandM.Chen.Ecienttechniquesfordirectedtestg enerationusing incrementalsatisability.In ProceedingsofInternationalConferenceofVLSIDesign pages65{70,2009. [59] P.MishraandN.Dutt.Graph-basedfunctionaltestprogramg enerationforpipelined processors.In Proc.ofDesignAutomationandTestinEurope(DATE) ,pages 182{187,2004. [60] P.MishraandN.Dutt.Functionalcoveragedriventestgener ationforvalidationof pipelinedprocessors.In Proc.ofDesignAutomationandTestinEurope(DATE) pages678{683,2005. [61] P.MishraandN.Dutt. FunctionalVericationofProgrammableEmbeddedArchitec tures:ATop-DownApproach .Springer,2005. [62] P.Mishra,H.-M.Koo,andZ.Huang.Language-drivenvalidat ionofpipelined processorsusingsatisabilitysolvers.In IEEEInternationalWorkshoponMicroprocessorTestandVerication(MTV)) ,pages119{126,2005. [63] M.W.Moskewicz,C.F.Madigan,Y.Zhao,L.Zhang,andS.Malik .Cha: EngineeringanecientSATsolver.In Proceedingsofthe38thDesignAutomationConference(DAC) ,pages530{535,2001. [64] M.Moy,F.Maraninchi,andL.Maillet-Contoz.Lussy:Atoolb oxfortheanalysis ofsystems-on-a-chipatthetransactionallevel.In ProceedingsoftheInternational ConferenceonApplicationofConcurrencytoSystemDesign ,pages26{35,2005. [65] W.Mueller,A.Rosti,S.Bocchio,E.Riccobene,P.Scandurra ,W.Dehaene,and Y.Vanderperren.Umlforesldesign:basicprinciples,tool s,andapplications.In ProceedingsofInternationalConferenceonComputer-Aide dDesign(ICCAD) ,pages 73{80,2006. [66] G.J.Myers,C.Sandler,T.Badgett,andT.M.Thomas. TheArtofSoftware Testing,2ndEdition .JohnWiley&Sons,Hoboken,NewJersey,2004. 173

PAGE 174

[67] O.Strichman.PruningtechniquesfortheSAT-basedbounded modelchecking problem. ProceedingsofCorrectHardwareDesignandVericationMet hods (CHARME),ser.LNCS,T.MargariaandT.Melham,Ed.Springer -Verlag 2144:58{70,2001. [68] ObjectManagementGroup.UMLProleforSystemonaChip(SoC),v1.0.1.[updatedAugust2006;citedAugust2008]. Availableat http://www.omg.org/technology/documents/formal/pro le soc.htm. [69] ObjectManagementGroup.UMLSuperstructureV2.1.2.[upda tedNovember2007; citedAugust2008]. http://www.omg.org/docs/formal/07-11-02.pdf. [70] OpenSystemCInitiative(OSCI).Systemc.[updatedaugust2 006;citedaugust2008]. Availableathttp://www.systemc.org. [71] P.MishraandN.Dutt.Specication-drivenDirectedTestGe nerationforValidation ofPipelinedProcessors. ACMTransactionsonDesignAutomationofElectronic Systems(TODAES) ,13(3):1{36,2008. [72] J.Peterson. PetriNetsTheoryandtheModelingofSystems .Prentice-Hall,N.J., 1981. [73] L.PierreandL.Ferro.ATractableandFastMethodforMonito ringSystemCTLM Specications. IEEETransactionsonComputers ,57(10):1346{1356,2008. [74] PrincetonUniveristy.zCha.[updatedNovember2004;cite dAugust2007]. Available athttp://www.princeton.edu/~cha/zcha.html. [75] R.Eshuis.SymbolicModelCheckingofUMLActivityDiagrams ACMTransactions onSoftwareEngineeringandMethodology ,15(1):1{38,2006. [76] R.L.Sites.AlphaAXPArchitecture. DigitalTechnicalJournal ,4(4),1992. [77] E.Riccobene,P.Scandurra,A.Rosti,andS.Bocchio.Auml2. 0proleforsystemc: towardhigh-levelsocdesign.In ProceedingsoftheACMInternationalconferenceon Embeddedsoftware ,pages138{141,2005. [78] A.Rose,S.Swan,J.Pierce,andJ.Fernandez. TransactionLevelModelingin SystemC .OSCITLMWorkingGroup,2005. [79] R.SchuttenandT.Fitzpatrick.Designforvericationmeth odologyallowssilicon success. EETIMES ,(16500856),2003. [80] D.Shin,A.Gerstlauer,J.Peng,R.Domer,andD.Gajski.Aut omaticGeneration ofTransactionLevelModelsforRapidDesignSpaceExplorat ion.In Proceedings ofInternationalConferenceonHardware/SoftwareCodesig nandSystemSynthesis (CODES+ISSS) ,pages64{69,2006. 174

PAGE 175

[81] O.Shtrichman.TuningSATcheckersforboundedmodelchecki ng.In Proceedings oftheTheInternationalConferenceonComputerAidedVeri cation(CAV) ,pages 480{494,2000. [82] SYNOPSYS.VCSVericationLibrary.[updatedAugust2007;c itedAugust2007]. Availableathttp://www.synopsys.com. [83] TheSatisabilityLibrary.SATBenchmarkProblems.[updat edSeptember2003;cited March2010]. http://www.satlib.org/Benchmarks/SAT/BMC/descriptio n.html. [84] B.Unhelkar. VericationandValidationforQualityofUML2.0Models .JohnWiley &Sons,2005. [85] M.Velev.BooleanSatisability(SAT)benchmarks.[update dNovember2006;cited March2010]. http://www.miroslav-velev.com/sat benchmarks.html. [86] M.Velev.Automaticabstractionofequationsinalogicofeq uality.In Proceedingsof ProceedingsofAnalyticTableauxandRelatedMethods(TABL EAUX) ,pages196{213, 2003. [87] C.Wang,H.Jin,G.D.Hachtel,andF.Somenzi.ReningtheSAT decisionordering forboundedmodelchecking.In ProceedingsofDesignAutomationConference (DAC) ,pages535{538,2004. [88] L.Wang,J.Yuan,X.Yu,J.Hu,X.Li,andG.Zheng.Generatingt estcasesfromuml activitydiagrambasedongray-boxmethod.In ProceedingsofAsia-PacicSoftware EngineeringConference(APSEC) ,pages284{291,2004. [89] J.Whittemore,J.Kim,andK.Sakallah.SATIRE:Anewincreme ntalsatisability engine.In ProceedingsofDesignAutomationConference(DAC) ,pages542{545,2001. [90] W.Wolf,A.A.Jerraya,andG.Martin.PDFMultiprocessorSys tem-on-Chip (MPSoC)Technology. IEEETransactionsonInComputer-AidedDesignofIntegrate d CircuitsandSystems(TCAD) ,27(10):1701{1713,2008. [91] L.Zhang,C.Madigan,M.H.Moskewicz,andS.Malik.Ecientc onrictdriven learninginabooleansatisabilitysolver.In ProceedingsofInternationalConference onComputer-AidedDesign(ICCAD) ,pages279{285,2001. [92] L.Zhang,M.Prasad,andM.Hsiao.Incrementaldeductive&in ductivereasoningfor SAT-basedboundedmodelchecking.In ProceedingsofInternationalConferenceon Computer-AidedDesign(ICCAD) ,pages502{509,2004. 175

PAGE 176

BIOGRAPHICALSKETCH MingsongChenreceivedhisB.S.andM.E.degreesfromtheDep artmentofComputer ScienceandTechnologyofNanjingUniversityinChinain200 3and2006respectively. Hisresearchfocusesondesignautomationofembeddedsyste ms,functionalvericationof System-on-Chiparchitectures,modelcheckingtechniques andsoftwareengineering. In2002,Mr.ChenjoinedtheSoftwareEngineeringGroupofNa njingUnverisity asaresearchassitant.Hisresearchwasfocusedonmodelche ckingofreal-timesystems andautomatictestgenerationforUMLactivitydiagrams.Un derthesupervisionof Prof.XuandongLiandJianhuaZhao,hereceivedhismaster's degreewiththesis titled\DynamicOptimizationTechniquesforStateSpacein TimedAutomataduring ReachabilityAnalysis".Since2006,hepursuedhisPh.D.de greeinCISEdepartment ofUniversityofFlorida.HejoinedtheCISEEmbeddedSystem sGroupin2007under thesupervisionofProf.PrabhatMihsra.Heparticipatedth eresearchprojecttitled \SOCValidationusingSystemCTransactionLevelModels"wh ichwasfundedbyIntel CorporationandU.S.NationalScienceFoundation.Duringh isPh.D.study,oneofhis paperspresentedin InternationalConferenceonVLSIDesign2009 wasnominatedfor bestpaperaward.Hewasalsoarecipientof DACYoungStudentSupportProgramAward in2008. Mr.ChencurrentlyservesasareviewerofseveralACMandIEE Econferences includingDAC,DATE,CODES+ISSS,ASP-DAC,GLSVLSI,VLSIDe sign,andISVLSI. HeisastudentmemberofIEEE. 176