<%BANNER%>

Java Racefinder

Permanent Link: http://ufdc.ufl.edu/UFE0042527/00001

Material Information

Title: Java Racefinder Precise Data Race Detector in a Relaxed Memory Model
Physical Description: 1 online resource (150 p.)
Language: english
Creator: Kim, Kyung
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: checking, counterexample, data, heuristic, memory, model, race
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Widespread use of multicore computers makes multithreaded programs ubiquitous, and concurrent programming as a main software paradigm. As difficult as sequential programming is, concurrent programming is even harder, and the correctness of it is more difficult to prove. Most approaches to reasoning about multithreaded programs, including model checking, make the implicit assumption that the system being considered is sequentially consistent. However, this is not a valid assumption for most current multiprocessor/core systems and this fact is exposed to the programmer for many concurrent programming languages in the form of a relaxed memory model. For example, the Java Memory Model only promises sequentially consistent behaviors for programs that are free from data races, making the ability to detect and eliminate data races essential for sound reasoning about Java programs. Toward this end, we introduce a new summary function that captures the information necessary for precise data race detection along with an efficient representation of the function that allows data race detection by model checking. We also introduce novel search heuristics specialized for data race detection that lead to shorter counterexample paths than standard search strategies. The ideas have been implemented in Java RaceFinder (\Tool), an extension to the model checker Java PathFinder (JPF). In contrast to many data race detection tools that can only deal with a restricted set of concurrent programming idioms, such as lock-based synchronization, \tool correctly handles programs that contain memory model-relevant features, including volatile fields, final fields, compareAndSwap, and static initialization in addition to both intrinsic and extrinsic locks. As a result, \Tool\ is powerful enough to be effectively used with wait-free and lock-free data structures. In addition to precise data race detection, the tool provides specific advice for eliminating data races from a program by analyzing the counterexample trace provided by the model checker and the acquiring history recorded by our tool. Case studies of widely used multithreaded programs and concurrent libraries have proved the usefulness of our tool and the effectiveness of our advice to correct found races. Finally, we provided an extended framework to guarantee the race-free use of a library using a library programmer's precondition annotations and the modular verification of their correctness using an assume-guarantee reasoning. This has been applied to various concurrent libraries and has verified the correct preconditions successfully. Once a concurrent program is proven to be free from a data race, standard model checking techniques can be soundly used to check other properties of interest. Our approach successfully addresses the relaxed memory model issue in model checking through precise data race detection.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Kyung Kim.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Sanders, Beverly A.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042527:00001

Permanent Link: http://ufdc.ufl.edu/UFE0042527/00001

Material Information

Title: Java Racefinder Precise Data Race Detector in a Relaxed Memory Model
Physical Description: 1 online resource (150 p.)
Language: english
Creator: Kim, Kyung
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: checking, counterexample, data, heuristic, memory, model, race
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Widespread use of multicore computers makes multithreaded programs ubiquitous, and concurrent programming as a main software paradigm. As difficult as sequential programming is, concurrent programming is even harder, and the correctness of it is more difficult to prove. Most approaches to reasoning about multithreaded programs, including model checking, make the implicit assumption that the system being considered is sequentially consistent. However, this is not a valid assumption for most current multiprocessor/core systems and this fact is exposed to the programmer for many concurrent programming languages in the form of a relaxed memory model. For example, the Java Memory Model only promises sequentially consistent behaviors for programs that are free from data races, making the ability to detect and eliminate data races essential for sound reasoning about Java programs. Toward this end, we introduce a new summary function that captures the information necessary for precise data race detection along with an efficient representation of the function that allows data race detection by model checking. We also introduce novel search heuristics specialized for data race detection that lead to shorter counterexample paths than standard search strategies. The ideas have been implemented in Java RaceFinder (\Tool), an extension to the model checker Java PathFinder (JPF). In contrast to many data race detection tools that can only deal with a restricted set of concurrent programming idioms, such as lock-based synchronization, \tool correctly handles programs that contain memory model-relevant features, including volatile fields, final fields, compareAndSwap, and static initialization in addition to both intrinsic and extrinsic locks. As a result, \Tool\ is powerful enough to be effectively used with wait-free and lock-free data structures. In addition to precise data race detection, the tool provides specific advice for eliminating data races from a program by analyzing the counterexample trace provided by the model checker and the acquiring history recorded by our tool. Case studies of widely used multithreaded programs and concurrent libraries have proved the usefulness of our tool and the effectiveness of our advice to correct found races. Finally, we provided an extended framework to guarantee the race-free use of a library using a library programmer's precondition annotations and the modular verification of their correctness using an assume-guarantee reasoning. This has been applied to various concurrent libraries and has verified the correct preconditions successfully. Once a concurrent program is proven to be free from a data race, standard model checking techniques can be soundly used to check other properties of interest. Our approach successfully addresses the relaxed memory model issue in model checking through precise data race detection.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Kyung Kim.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Sanders, Beverly A.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042527:00001


This item has the following downloads:


Full Text

PAGE 1

JAVARACEFINDER:PRECISEDATARACEDETECTORINARELAXEDMEMORYMODELByKYUNGHEEKIMADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2010

PAGE 2

c2010KyungHeeKim 2

PAGE 3

Tomyparentsfortheirloveandsacrice 3

PAGE 4

ACKNOWLEDGMENTS IwouldliketobeginwithexpressingmydeepestgratitudetoProfessorBeverlyA.Sanders,myadvisor,forherendlessadvice,guidance,andsupportthroughoutmygraduatestudies.Shehasbeenarolemodelandamentorforme.ShehasalwaysbeenreadywithananswerwhenIquestioned,andencouragedmewheneverIwaslostinthelongpathtomynalgoal.Sheinspiredmewithherpassion,knowledge,andpatience.IwouldalsoliketothankProfessorTubaYavuz-Kahveciforhergreatideas,advice,andsupport.ShehasrespondedtomewiththebestadvicewheneverIaskedaquestionandsharedherexperiencewheneverIwasafraidtogoforward.ThreeofushavebeenagreatresearchteamandIamveryproudofbeingpartofit.Mygratefulthanksalsogotomycommitteemembers,Dr.StephenM.Thebaut,Dr.Jih-KwonPeir,andDr.DouglasCenzer,fortheirvaluableinsightsandcomments.IwouldalsoliketothankDr.EricG.MercerfromBrighamYoungUniversityandDr.NehaRungtafromNASAAmesResearchCenterfortheirguidanceandadvice.Theirhelpmadetheresearchonthemodularextensionpossible.IamindebtedtoNASAJavaPathFinderteamincludingMr.PeterMehlitzfortheirsupportandkindcomments.WithoutJavaPathFinder,thisresearchwouldhavebeenimpossible.IwouldliketoexpressmydeepestrespectandappreciationtoProfessorChulE.Kim,myundergraduateadvisor,forhisencouragementandadvices.Lastbutnotleast,Iwanttothankmyfamilyfortheirloveandsupport.Wordsarenotenoughtoexpressmygratitudeforwhattheyhavegiventome.Myparentstaughtmetobeagoodperson.Mychildren,ChrisandHan,mademewanttobecomeabetterperson.Myhusband,YoungSang,wasthereformeandtrieswithmetodoourbest.IloveyouChris,Han,andspeciallyYoungSang. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 11 CHAPTER 1MOTIVATION ..................................... 13 2RELATEDWORK .................................. 20 2.1DataRaceDetection .............................. 20 2.2ModelChecking ................................ 23 3FOUNDATION .................................... 27 3.1TheJavaMemoryModel ........................... 27 3.1.1ModelCheckerProperties ....................... 32 3.1.2AWeakenedConditionforSequentialConsistency ......... 33 3.2ModelCheckingandJavaPathFinder .................... 36 4JAVARACEFINDER ................................. 40 4.1TheSummaryFunctionh ........................... 40 4.2DataRaceSpecicSearchHeuristic ..................... 45 4.2.1Heuristics ................................ 46 4.2.2Algorithm ................................ 49 4.3Implementation ................................. 51 4.3.1Representationofh ........................... 54 4.3.2TheListenerImplementation ..................... 56 4.3.3PruningtheSearchSpace ....................... 57 4.3.4UsingtheModelJavaInterface .................... 58 4.3.5ProblemswithStateBacktracking ................... 59 4.3.6UntrackedVariables .......................... 60 4.3.7LazyRepresentationofArrayElements ............... 64 4.3.8ThreadlocalOptimization ........................ 66 4.3.9BenignRace .............................. 67 5EXTENSION ..................................... 70 5.1EliminatingDataRaceUsingCounterexample ............... 70 5.1.1Analysis ................................. 71 5.1.2Algorithms ................................ 73 5

PAGE 6

5.1.3TheoreticalResults ........................... 84 5.1.4Implementation ............................. 86 5.2ModularVericationoftheCorrectnessofRace-FreePreconditions .... 89 5.2.1Analysis ................................. 93 5.2.2Algorithm ................................ 95 5.2.3TheoreticalResults ........................... 101 6CASESTUDIES ................................... 105 6.1SimpleExamples ................................ 105 6.2Herily-ShavitExamples ............................ 106 6.2.1Examples ................................ 108 6.2.1.1ConcurrentHashSets .................... 108 6.2.1.2Queue-basedSpinLocks .................. 112 6.2.2ExperimentalResults .......................... 115 6.2.2.1RaceDetectioninJRF ................... 115 6.2.2.2RaceAnalysisinJRF-E ................... 115 6.3AminoConcurrentBasicBlocks ........................ 117 6.3.1Examples ................................ 119 6.3.2ExperimentalResults .......................... 123 6.3.2.1RaceDetectioninJRF ................... 123 6.3.2.2RaceAnalysisinJRF-E ................... 124 6.4GoogleConcurrentPackage ......................... 125 6.4.1ConcurrentAdtExperimentFramework ................ 125 6.4.2GoogleConcurrentDataStructuresWorkshopBarrier ....... 126 6.4.3ExperimentalResults .......................... 128 6.4.3.1RaceDetectioninJRF ................... 128 6.4.3.2RaceAnalysisinJRF-E ................... 129 6.5JavaGrandeForumTestSuite ........................ 130 6.5.1ExperimentalResults .......................... 133 6.5.1.1RaceDetectioninJRF ................... 133 6.5.1.2RaceAnalysisinJRF-E ................... 133 7DISCUSSION ..................................... 135 7.1Performance .................................. 135 7.1.1ThreadlocalOptimization ........................ 135 7.1.2HeuristicSearch ............................ 135 7.1.3ModularExtension ........................... 139 7.2Overhead .................................... 141 8CONCLUSION .................................... 142 REFERENCES ....................................... 145 BIOGRAPHICALSKETCH ................................ 150 6

PAGE 7

LISTOFTABLES Table page 6-1Experimentalresultsfor[ 1 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. 116 6-2ExecutionresultofPreciseRaceDetectorforHerlihy-Shavittestsfrom 6-1 ... 117 6-3Experimentalresultsfor[ 1 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. ......................................... 118 6-4JRF-EsuggestionsfromcounterexampleandacquiringhistoryanalysisforHerlihy-Shavitexampleswitharace ........................ 119 6-5Experimentalresultsfor[ 2 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. 123 6-6ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-5 ..... 124 6-7Experimentalresultsfor[ 2 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. ......................................... 124 6-8JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 2 ]exampleswitharace ............................... 125 6-9Experimentalresultsfor[ 3 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. 128 6-10ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-9 ..... 129 6-11Experimentalresultsfor[ 3 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. ......................................... 129 6-12JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 3 ]exampleswitharace ............................... 130 6-13Experimentalresultsfor[ 4 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. 133 6-14ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-13 ..... 134 6-15Experimentalresultsfor[ 4 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. ......................................... 134 6-16JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 4 ]exampleswitharace ............................... 134 7

PAGE 8

LISTOFFIGURES Figure page 3-1Operationalmodelofdirectvsindirectannotationinmodelchecking ...... 38 4-1Denitionofhn+1 ................................... 42 4-2OneiterationofPeterson'sAlgorithm ........................ 48 4-3ModelcheckingofPeterson'salgorithmusingdifferentsearchstrategies .... 48 4-4Heuristicsearchalgorithm.StatesareprioritizesbasedontheirheuristicvalueascomputedinFigure 4-5 andthesearchdepth. ................. 49 4-5Algorithmfordecidingheuristicvaluesforstatesbasedontheirlikelihoodofleadingtoadatarace.Heuristicvaluesbecomeavailableaccordingtotheheuristics(WF,WW,ARA,AF)presentedinSection 4.2.1 ............ 50 4-6JRFcomponentsandtheirJPFcounterparts ................... 52 4-7ThedefaultJRFcongurationinjrf.jpfle ................... 53 4-8Internalrepresentationofh ............................. 55 4-9OverallsystemhierarchyofaJRFextensiontoJPFandtheirdataposition .. 57 4-10Modieddentionofhn+1withuntrackedvariables.OmittedactionsarethesameasshowninFigure 4-1 ............................ 60 4-11Modiedalgorithmfornon-volatilereadandwritewithlazyrepresentationofarrayelementsanduntrackedvariables ...................... 65 5-1PartialoutputfromJRFformodiedSimpleclass.Eightsimilartraceshavebeenomitted. ..................................... 71 5-2JRFoutputwhichexplainsthesourceoftheraceandsuggestshowtoeliminateit ............................................ 72 5-3Suggestchangetovolatileoratomicarray. .................... 73 5-4Findthesetofhappens-beforeedgesthroughsynchronizationactionsonpathpathInstr. ....................................... 75 5-5Suggestmovinginstruction ............................. 75 5-6ViagoFlagThread1notiesThread2whenobjectpublishisreadytobeused. ......................................... 76 5-7Partofthestatespaceshowingadataracefreepathandapathwithadatarace. .......................................... 77 8

PAGE 9

5-8Suggestasynchronizedblock. ........................... 78 5-9Thread2needtosynchronizeonlocktoaccessdata .............. 79 5-10Partofthestatespacewiththeunlockinbetweenthesourcestatementandthemanifeststatement ................................ 79 5-11Suggestchangingadifferentmemorylocationstovolatile ............ 80 5-12Partofthestatespacethatchangingdonetovolatilecaneliminatearaceonx 81 5-13Suggestperforminganacquireoperationthatcanaddthedataracememorylocationtohofthemanifestingthread. ....................... 82 5-14AcquiringhistoryofThread2showsThread3cangetracefreeaccessonxbyreadingdone. ................................... 83 5-15Partofthestatespacewiththeacquiringhistorythatguideshowtoeliminatetherace ........................................ 83 5-16ViagoFlagThread1notiesThread2whenobjectpublishisreadytobeused.Thread3canalsonotifyThread2bycheckingaeldoftheobjectpointedtobypublish. .................................... 86 5-17InstantiationofConditionsinLemma 9 basedonsampleprograminFigure 5-16 .Happens-beforeedgesformedbysynchronizationactionsareshownbylinesconnectingthematchingreleaseandacquireinstructions. ....... 87 5-18TheJRF-Eexecutionmodel:JRFpluginorstandalone .............. 88 5-19UnboundedQueuelibrary .............................. 90 5-20DisBarrierlibrary ................................... 91 5-21FairMessageapplicationusesUnboundedQueueandDisBarrier ........ 92 5-22ThefourstepsinmodularextensionofJRF .................... 94 5-23UnboundedQueuelibrarywithpreconditionannotation .............. 96 5-24Generateuniversalenvironmentwithallpossiblecombinationsofmethods.(choose*generatedifferentstatesinmodelchecking) .............. 98 5-25GenerateduniversalenvironmentforUnboundedQueue ............. 99 5-26PreconditionviolationsinFairMessagedetectedbyJRFmodularextension .. 101 6-1Abstractclassusedinthreelock-basedclosed-addresshashsets ....... 108 6-2HashSetimplementationthatusesasinglelock ................. 109 6-3HashSetimplementationthatusesaxedsizearrayoflocks .......... 110 9

PAGE 10

6-4CLHlock ....................................... 113 6-5VariationofCLHlockthatallowswaitingnodestotimeout ............ 114 6-6DataRaceoncursorinLockFreeDeque.DeqIteratorsincetheimplementationisnotthread-safe ................................... 120 6-7DataRaceonnextNodeinLockFreeQueue.QueueItrsincetheimplementationisnotthread-safe ................................... 122 6-8SyncCounterThreadusingabarrierwithanon-volatilevalueeldhasanraceonvalueeldsincetheupdatebyathreadafterbarriercannotbeoverwrittenbyanotherthreadwithoutcausingaWWrace ................... 126 6-9SimpleBarrierusinganAtomicIntegerisnotworkingproperlysincethebarrierisbroken ....................................... 127 6-10Syncbenchmarkcodefragmentwitharaceonshared cont ........... 131 6-11MonteCarlobenchmarkcodefragmentwitharaceonstaticeldUNIVER-SAL DEBUG ..................................... 132 7-1ComparisonoftheJRFresultswith/withoutthreadlocaloptimizationwhenDFSstrategyisconguredtondallracesbyexploringthefullsearchspace 136 7-2Comparisonoftheheuristicwitharandomsearchstrategy ........... 137 7-3Comparisonofdifferentheuristiccongurations .................. 138 7-4ThecomparisonofJRFandJRFmodularextensionfordifferentcongurationofFairMessage .................................... 139 7-5ComparisonofJRFwith/withoutthreadlocaloptimizationwithoriginalJPFwhenDFSstrategyiscongureandforcedtostopatthestatewhereJRFndtherstrace ................................... 141 10

PAGE 11

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyJAVARACEFINDER:PRECISEDATARACEDETECTORINARELAXEDMEMORYMODELByKyungHeeKimDecember2010Chair:BeverlyA.SandersMajor:ComputerEngineeringWidespreaduseofmulticorecomputersmakesmultithreadedprogramsubiquitous,andconcurrentprogrammingasamainsoftwareparadigm.Asdifcultassequentialprogrammingis,concurrentprogrammingisevenharder,andthecorrectnessofitismoredifculttoprove.Mostapproachestoreasoningaboutmultithreadedprograms,includingmodelchecking,maketheimplicitassumptionthatthesystembeingconsideredissequentiallyconsistent.However,thisisnotavalidassumptionformostcurrentmultiprocessor/coresystemsandthisfactisexposedtotheprogrammerformanyconcurrentprogramminglanguagesintheformofarelaxedmemorymodel.Forexample,theJavaMemoryModelonlypromisessequentiallyconsistentbehaviorsforprogramsthatarefreefromdataraces,makingtheabilitytodetectandeliminatedataracesessentialforsoundreasoningaboutJavaprograms.Towardthisend,weintroduceanewsummaryfunctionthatcapturestheinformationnecessaryforprecisedataracedetectionalongwithanefcientrepresentationofthefunctionthatallowsdataracedetectionbymodelchecking.Wealsointroducenovelsearchheuristicsspecializedfordataracedetectionthatleadtoshortercounterexamplepathsthanstandardsearchstrategies.TheideashavebeenimplementedinJavaRaceFinder(JRF),anextensiontothemodelcheckerJavaPathFinder(JPF).Incontrasttomanydataracedetectiontoolsthatcanonlydealwitharestrictedsetofconcurrentprogrammingidioms,suchaslock-basedsynchronization,JRFcorrectly 11

PAGE 12

handlesprogramsthatcontainmemorymodel-relevantfeatures,includingvolatileelds,nalelds,compareAndSwap,andstaticinitializationinadditiontobothintrinsicandextrinsiclocks.Asaresult,JRFispowerfulenoughtobeeffectivelyusedwithwait-freeandlock-freedatastructures.Inadditiontoprecisedataracedetection,thetoolprovidesspecicadviceforeliminatingdataracesfromaprogrambyanalyzingthecounterexampletraceprovidedbythemodelcheckerandtheacquiringhistoryrecordedbyourtool.Casestudiesofwidelyusedmultithreadedprogramsandconcurrentlibrarieshaveprovedtheusefulnessofourtoolandtheeffectivenessofouradvicetocorrectfoundraces.Finally,weprovidedanextendedframeworktoguaranteetherace-freeuseofalibraryusingalibraryprogrammer'spreconditionannotationsandthemodularvericationoftheircorrectnessusinganassume-guaranteereasoning.Thishasbeenappliedtovariousconcurrentlibrariesandhasveriedthecorrectpreconditionssuccessfully.Onceaconcurrentprogramisproventobefreefromadatarace,standardmodelcheckingtechniquescanbesoundlyusedtocheckotherpropertiesofinterest.Ourapproachsuccessfullyaddressestherelaxedmemorymodelissueinmodelcheckingthroughprecisedataracedetection. 12

PAGE 13

CHAPTER1MOTIVATIONThecurrenttrendinmulticorecomputersputsmoreemphasisonconcurrentprogramming.Inmostprogrammingenvironments,however,thecorrectnessofaconcurrentprogram'ssafetydependsontheprogrammer'sabilitytocorrectlyordereveryaccesstoshareddata.Unfortunately,thisisdifcult,andnopreviousresearchhascorrectlyimplementedvericationofthisconcept.Virtuallyallapproachestoreasoningaboutthebehaviorofconcurrentprograms,boththeinformalreasoningpracticedbyprogrammerswritingaconcurrentprogramandtheformalmethodsandtools,suchasmodelcheckers,startwiththeassumptionthatprogramexecutionsaresequentiallyconsistent(SC)[ 5 ].InaSCexecution,aconcurrentprogrambehavesasifallofitsatomicactionsoccurinsomeglobalorderthatisconsistentwiththeprogramorderoneachthread.Inparticular,allthreadsseevalueswrittentomainmemoryinaconsistentorder.Incontrasttothiswidelyacceptedassumptioninthevericationphase,modernprogrammingenvironmentsdonotguaranteesequentialconsistency.Commonoptimizationsbythecompilerandthehardwarethatsignicantlyspeedsupprogramswithoutaffectingtheirsequentialsemanticsarenotnecessarilybenigninaconcurrentenvironment.Asanexample,considerthefollowingprogramfragmentwhereresultanddonearevisibletomultiplethreads: result=computation();done=true;Thevariabledoneisinitiallyfalseandnotaccessedbycomputation(),whichupdatesresultandpossiblyhasothersideeffects.Sincethetwostatementsareindependent,theordercouldbereversedwithoutchangingthesequentialsemantics.However,ifthisfragmentoccursinaconcurrentprogramanddoneisintendedtobeaagtootherthreadsthatcomputation()isnished,thenreversingtheordercouldresultinanother 13

PAGE 14

threadndingdone==true,andseeingastatereectinganincompleteexecutionofcomputation().ThisscenarioisnotSC.Architecturesprovidelowlevelinstructionsthatcanbeusedtopreventreorderingbythehardwareandaretypicallyinsertedintheobjectcodeasaresultofsynchronizationinstructionsintheprogramsource.Compilerscanrefrainfromcertainoptimizationsthatmaycausesequentiallyinconsistentbehavior.Althoughitwouldmakeconcurrentprogrammingmucheasiertousecompileranalysistodeterminewhatneedstobedone,thiscurrentlyisnotpracticalandtheprogrammerisexpectedtoinsertsufcientsynchronizationtoensuresequentialconsistency.Exactlyhowthreadsinteractwithmemoryandhowtheprogrammercancontrolthisisdenedbyamemorymodel.Traditionally,memorymodelshavebeendenedforarchitectures,butmorerecentlymemorymodelshavebecomepartofaprogramminglanguage'ssemanticsandmemorymodelshavebeendenedforlanguagesincludingJava[ 6 ,Chapter17],.netbasedlanguagessuchasC#[ 7 ,PartitionI,section12.6],C++[ 8 ],andOpenMP[ 9 ].InJava,C#,andC++,dataracesarethesituationsthatcanleadtonon-SCexecutions.ThetermdataracehasoftenbeenusedwherethedenitionandconsequencesaresubtlydifferentfromthoseofdataracesinthecontextoftheJMM.Twomemoryaccessesbydifferentthreadsonthesamelocationaresaidtoconictwhenatleastoneisawrite,andadataracehasbeendenedtobeasituationwhereconictingoperationsarenotorderedbysynchronization.Inasequentiallyconsistentsystem,dataracesmayindicatesomesortofconcurrencyrelatednon-determinismthatmayormaynotaffecttheoverallcorrectnessoftheprogram.Forexample,inanSCsystemintheexamplegivenearlier,theaccessestodonewouldbeconsideredabenigndatarace, 1 whileitisabugwithpotentiallyseriousconsequencesinaprogramexecuting 1 SCrulesoutreorderingandtheaccessitselfisatomic. 14

PAGE 15

undertheJMM.AccordingtothememorymodelsofJavaandC#(butnotC++),theracecouldbeeliminatedbymarkingdoneandresultwiththevolatilekeyword.TheJMMconstrainsthebehaviorofJavaprogramswithdataraces,soonecould,inprinciple,verifythatadataraceisbenign.Thisisquitedifcultandbestlefttoexperts.IntheC++memorymodel,thebehaviorofprogramswithdataracesisundened,thusnodataracescanbeviewedasbenigninthatlanguage.Inthecontextofarelaxedmemorymodel,dataracesarealmostalwaysseriousbugs 2 .Similarly,aprogrammaybefreeofdataracesinthesenseoftheJMM,andthusguaranteesequentialconsistency,whilestillcontainingconcurrencyrelatederrors.Forexample,supposeaclassrepresentingabankaccountcontainsaeldforthebalanceandoffersadepositmethodthatexecutesbalance=balance+amount.IntheJMM,ifbalanceismarkedvolatile(orifeachoftheaccessestothiseldoccursinsideitsowncriticalsectionimplementedusingacommonlock)thentheprogramwillnothaveadatarace,butitwillstillbeincorrectduetothefactthattheentiredepositmethodisnotatomic.Someauthorswouldcallthiserrorarace,addingtotheconfusionaroundtheterm.Inthisdissertation,wewillusethetermdataraceasdenedintheJMManddiscussedinmoredetailinSection 3.1 withthegoalofdetectingsituationsthatleadtonon-SCbehavior.TheJMMsatisesanimportantfundamentalpropertyformemorymodels[ 11 ]: ProgramswhoseSCexecutionshavenoracesmusthaveonlySCexecutions.Asaresult,wecanassumeSC,andthususemodelcheckingtodemonstratedataracefreedom.Then,forprogramswithoutdataraces,standardmodelcheckingtechniquescansoundlybeusedtondothertypesofconcurrencyrelatederrors.Neverthelessvarioustoolstodetectdataraceshavebeendeveloped,mostofthemdonotpreciselyndraceconditionsasdenedbyawell-denedmemorymodel.Rather 2 Lazyinitialization[ 10 ],isthesinglewell-known,reasonablypracticalprogrammingidiomexhibitingbenigndataracesinJava. 15

PAGE 16

theyidentifyeasy-to-detectsituationsthatmayindicateadatarace.Forexample,manyapproaches(includingoneofthedataracedetectionextensionsinthecurrentJPFdistribution)attempttoensurethatforallsharedvariables,allaccessestoaparticularvariableareprotectedbyacommonlock.Whilethisconditionissufcientfordataracefreedom,itisnotnecessary.Incontrasttoourapproach,toolsthatchecklockusagecannoteffectivelyanalyzeprogramsusingimportantidiomsthatarenotbasedonlockingincludingwait-freeandlock-freealgorithms.Providedthatthesealgorithmsbecomesmoreimportantaccordingtothewide-spreaduseofmulticores,theabilitytohandlethemcorrectlyisasignicantadvantageinaracedetector.WedescribeatoolthatusesmodelcheckingtopreciselydetectdataracesinJavaprogramsinordertohelpprogrammersensuresequentialconsistency.Modelcheckingwasoriginallydesignedtobeusedatthehardwarelevel,butthesedays,softwaremodelcheckingbecomespopulartocheckpropertiesinconcurrentprograms.JPFiswidelyusedmodelcheckerforJavabytecode,andhasbeenappliedtovariousapplicationsincludingseveralmissioncriticalapplications.However,asdiscussedabove,oneoftheproblemofusingJPFtocheckthecorrectnessofaconcurrentprogramisthatJPFtakesintoaccountSCexecutionsonly.JPFisunabletondpropertyviolationsinanon-SCexecution.Forexample,considerthefollowingsimpletwo-threadedjavaprogramusingtheabovecodefragment. publicclassSimple{staticintx;staticbooleandone=false;publicstaticvoidmain(String[]args){(newOtherThread()).start();x=1;done=true;} 16

PAGE 17

staticclassOtherThreadextendsThread{publicvoidrun(){while(!done){/*spin*/}assert(x==1);}}}TheexpectedmodelcheckingresultwouldbetondanassertionviolationatOtherThreadwhentheproperty"done=)(x==1)"isviolatedinanypossibleexecutionpath.However,JPFendswithnoerrorforSimple,eventhoughthereisoneaccordingtoJMM.TheresultforthedifferentversionofSimplewithdoneasavolatile,inwhichthepropertyalwaysholds,isthesameincurrentJPF.ThisshowsthatJPFcannotbeusedtosoundlyproveconcurrentprogramsafety.Weaimedtondaformalmethodtodistinguishthetwoprogramsandidentifyaprogramwithasoundresult.Inaddition,themostcommonprobleminmodelcheckingisastate-spaceexplosionthatobstructsitfrombeingapracticalcorrectnessproofmethod.Ourtoolincludesthefeaturestoleveragethisproblemduringmodelcheckingtosearchforarace,althoughitisimpossibletothoroughlysolveit.Thepurposeistominimizethechanceofstate-spaceexplosionbeforeadataracedetectionandweappliedheuristicsindeterminingsearchorderstodetectaraceearlierifoneexists.Wealsousedseveraloptimizationmethodsincludinglazyrepresentationofarrayelementsandthreadlocalexclusiontosavetemporalandspatialresources.Furthermore,wealsosuggestcodechangestocorrectaracetoaddresstheproblemofanalyzingfoundraces.Modelcheckinggeneratesacounterexamplealongwithapropertyviolation,andthecounterexamplepathanalysisidentiesthesourceofarace.Accordingtoourexperienceinseveralcasestudies,moreexplanationofadataraceotherthantheracelocationisinevitable. 17

PAGE 18

Modularizationisanotherwidelyusedapproachtoleveragethestate-spaceexplosionproblem.Theideaistodecomposethesystemundertestintosmallermodulesandapplythevericationalgorithmintothem.Theindividualvericationresultsarecombinedattheendtoconcludetheoverallresultgiventhatcertainsafetyconditionsaresatised.Weextendedourframeworktoincludeamodularracecheckingcapability.Inadditiontothedecompositionoftheracedetectionoverhead,weaddressedtheproblemofaforeigncodeexecutionintherace-freeguarantee.Javalibrariesaredistributedintheformofabytecodeingeneralandthedataracefreedomofthoseforeigncodesaredeterminedsolelybytheappropriateuseinanapplicationthatmakesuseofthem.Eitherbecauseofalackofavailableinformationabouttheinternalimplementationorbecauseofalackofattentioninapplyingthem,theoveralldataracefreedomoftheforeignlibraryisvulnerable.Toachievethetwogoalsofdecomposingtheracedetectionintomodulesandspecifyingthedesigndecisionsofaforeigncoderegardingitsdataracefreedom,wesuggestedtheuseofrace-freepreconditions.Thepreconditionsattachedtothebytecodeandannotatedbythelibrarydeveloperareveriedautomatically.Ourframeworkensuresthatpreconditionsarecorrectandsatised.Thelackofpreconditionviolationguaranteestheracefreedomofaforeigncodeanditsusage.Theremainderofthedissertationisorganizedasfollows:InChapter 2 ,webeginwiththerelatedworkondataraceandmodelchecking.InChapter 3 ,weintroducetheJMMandgiveabriefoverviewofitsformaldenition,includinganewresultshowingthataweakerconditionthancompletedataracefreedomissufcienttoensuresequentialconsistency.WealsointroducemodelcheckingandJPFinthischapter.Chapter 4 describesourapproachusedinJRF.Weintroduceasummaryfunctionthatcanbeusedtocapturethenecessaryinformationtopreciselyrecognizedataracesduringmodelcheckingandnewsearchheuristicsbasedonacarefulanalysisofthepropertiesofdataraces.Aforementionedideasareimplementedina 18

PAGE 19

modelcheckerbyextendingJPF.Weintroduceaspaceefcientrepresentationofthesummaryfunctionandseveraloptimizationtechniques,includinghowtorepresentanarraydatastructureandhowtohandlethreadlocalmemories.TheextensionstoJRFtomakeitmorepowerfularedescribedinChapter 5 .Weusedcounterexamplepathandacquiringhistorytoanalyzearaceandsuggestappropriatecodechangestoeliminateit.Weintroduceonemoreextension,modularrace-freepreconditionverication,inthischapter.CasestudiesfollowinChapter 6 ,andChapter 7 discussestheoverallperformanceofJRF.WepresentoutconclusionsinChapter 8 19

PAGE 20

CHAPTER2RELATEDWORK 2.1DataRaceDetectionVarioustoolsandapproachestoavoidanddetectraceconditionsanddataraceshavebeendescribedintheliterature,butthoseexistingtoolstodetectorpreventraceconditionsarelimitedinfundamentalways.Manytoolsfordataracedetectionoravoidancedonotstartfromthememorymodelanditsdenitionofadatarace,butinsteadlookforconditionsthatimplydataracefreedomandareeasiertocheck.Forexample,manyapproachesattempttoensurethatallaccessestoaparticularsharedvariableareprotectedbyacommonlock. 1 WhilethesemanticsofthelockprimitiveinJavaimplythatthisconditionissufcientfordata-racefreedom,itisnotnecessaryandrequiringitrulesoutincreasinglyimportantprogrammingidiomsincludingwait-freeandlock-freealgorithms.Previousapproachesusuallyfellintothreecategories:usingatypesystemwithadditionalannotations,dynamicracedetectionbasedonthelocksetalgorithmorvectorclockalgorithmtocomputethehappens-beforerelation,andhybridmethodswithcombinedstaticanddynamicinformation.Typebasedapproachesextendthetypesystemofaconcurrentprogramminglanguagetomarksharedvariablesasguardedbyaparticularlock,andtoensurethatthelockisheldwhenthevariableisaccessed[ 12 13 ].Adisadvantageofthetypebasedapproachistheburdenontheprogrammertoprovidethenecessarytypeannotations.Whetherornotthisisasignicantdisadvantagefornewprogramswheretheannotationscanbewrittenwhilethecodeisbeingdevelopedisarguable,butthisclearlylimitstheusefulnessoftheapproachforexistingsystems.Someworkattemptstoimprovethesituationbyinferringmanyofthenecessaryannotations[ 14 ].Most 1 ThisiswhatisdonebytheDataRaceCheckerthatiscurrentlypartoftheJPFdistribution. 20

PAGE 21

approachesusingadditionaltypeannotationsareconservativeandproducelargenumbersoffalsepositives.Thus,thetoolsimplementingthisapproachpaymuchattentiontoeffectivelyeliminatingthem.[ 13 15 ]areotherdialectsofjavaforpreventingadataracethroughatypesystem.Insteadofdetectingpossibleraces,thosetypesystemsforceallshareddataaccessestobeguardedbyalock;synchronizationisnotaccomplishedbyprogrammer'scode,butbythetypesystemitself.Theseapproachesguaranteetheracefreedomforprogramssatisfyingsuggestedtypingrules,butcannotbeusedforlegacycodes.Racedetectiontoolsbasedonstaticanalysistechniquestypicallysacricecompleteness,inthesensethattheycanonlydealwithaparticularsetofprogrammingidioms,andthusdisallowlegaldata-racefreeprograms.Sometoolsdeliberatelysacricesoundnessaswell,failingtoidentifycertaindataraces.Forexample,Chord[ 16 ],whichcanhandlelexically-scopedlock-basedsynchronization,fork/joinsynchronization,andwait/notify,startsbyconstructingasupersetofpossibleconictingoperations,thenltersthissetusingasequenceofanalyses,andreportsapossibledataraceforallremainingpairs.AlthoughChordisbothunsoundandincomplete,itappearstobeextremelywell-engineeredandinpracticehasshownsignicantutilitybyidentifying387concurrencyrelatederrorsinasetofwidelyusedopensourceprograms.Anotherexampleistherccchecker[ 12 ]asrecentlyresurrectedandextendedfortheMobiusproject[ 17 ].Thistoolusesatypetheorybaseapproach(whichrequiresannotationsbytheusers)toensurethatlockingisdonecorrectly.Initsmostrecentincarnation,italsorecognizesthatvolatilevariablesdonotneedtobeprotectedbylockstoavoiddataraces.However,inwhateverform,thetoolcannotdealwithhappens-beforeedgesobtainedviatransitivityandgeneratesfalsepositivesasaresult.Toolsthatperformdynamicracedetectionlookforracesinparticularexecutionsoftheprogram.Thedisadvantageisthatdynamictoolsonlydetectproblemsinthetestcasesthatareactuallyexamined.Thesearetypicallybasedonmaintainingvector 21

PAGE 22

clocksorthelock-setalgorithmwithcheckstoseeifeverysharedvariableaccessisconsistentlylocked.Eraser[ 18 ]isaninuentialexampleofalock-setbaseddetector.Thissystempredatesprogramminglanguageswithwell-denedmemorymodelsandwasbasedontheideathataccessestosharedvariablesshouldconsistentlybelocked.Inpracticethatrequirementistoostrict,andEraserandsubsequentlock-basedtoolstypicallyincorporatespecialcases,suchasthesituationwhereasinglethreadaccessesandmodiesavariableorobjectforsomeperiodoftimeafterwhichitisnolongermodied,butcanbereadbymultiplethreads.Sincethesetoolsdonotcheckforsafepublication,theyareunsoundintheJMM.VeryrecentworkextendingtheEraseralgorithmincludes[ 19 ],whichusesaspect-orientedprogrammingtechniquestoinstrumenttheprogramatthesourcecodelevelwithnewpoint-cutslock,unlock,andmaybeshared.ThedynamictoolmostcloselyrelatedtooursisGoldilocks[ 20 ].Theso-calledlocksetalgorithmdescribedthereusesarelationthatisverysimilartotheinverseofsummaryfunctionhexplainedinSection 4.1 .Inotherwords,thegoldilocksalgorithmmaintainsafunctionforeachvariablethatindicateswhichthreadscanaccessthevariable.Aswithalltoolsperformingdynamicanalysis,therequiredinstrumentationoftheprogrammaychangeitsbehaviorandthetoolislimitedtoonlyanalyzingpathsthathappentobetested.ThestandardJPFdistributionincludestworacedetectingtools,RaceDetectorandPreciseRaceDetector.Theformerimplementsthelocksetalgorithmbasedontheassumptionthateverysharedeldisprotectedbyacommonlock.Thislackstheabilitytocheckhappens-beforeorderingotherthanbylocking.Thelatterchecksforadataracebasedonagenericdenitionofarace,ratherthanstartingwiththeJMM.Ateachchoicegenerator,itchecksiftherearemorethantwothreadchoicestryingtoaccessthesamememorylocationandifatleastoneofthemisanupdateaccess.Readandwriteaccessestoavolatileeldarealsodetectedasarace.Asimplemodicationto 22

PAGE 23

recognizethatvolatilevariablesarenotinvolvedindataraceswouldeliminatesome,butnotall,falsealarms.Anysituationwheretransitivityofhappens-beforeedgesmakesanaccesssafewouldbereportedasafalsealarm. 2.2ModelCheckingRecentworkhasincorporatedmemory-modelawarenessintoprogramanalysistools.TheCheckFence[ 21 ]systemveriesthataprogramexecutedonasystemwithahardware-levelrelaxedmemorymodelisobservationallyequivalenttoasequentialexecutionbytranslatingCimplementationcodeandthetestprogramintoaSATformulathatisthengiventoaSATsolver.Otherconstraint-basedapproachesaredescribedin[ 22 23 ].Severalstudies[ 24 26 ]haveincorporatedmemorymodelawarenesstomodelchecking.Theapproachespresentedin[ 24 26 ]canverifysequentialconsistency.[ 24 ]considersahardware-levelmemorymodelandusesboundedmodelcheckingandCHESS,astatelessmodelchecker.[ 27 ]Therefore,theyusevector-clockstocapturethehappen-beforerelation.[ 26 ]considersC#'smemorymodelandabytecode-levelstate-basedmodelcheckertailoredforC#.Thetechniquepresentedin[ 25 ]guidesthemodelcheckeringeneratingasubset(i.e.,underapproximatestheJMM)ofprogramexecutionsvaryingduetoinstructionreorderingallowedintheJMM.However,thistooldoesnotdetectdataraces.Debuggingistediouswork,soprovidingautomateddebuggingsupportforprogrammerscansavemuchtimeandeffort.Oneintuitiveideaistocomparefailingexecutionswithsuccessfulonesandusethedifferencesbetweenthetwotoexplainandlocalizethefault.Thisconcepthasbeenexploredinvariousstudies;some[ 28 32 ]usemodelcheckingtogeneratetheexecutions,whereasothers[ 33 36 ]usetesting.Theapproachin[ 29 31 34 35 37 ]istocompareasetofsuccessfultraceswithasetoferroneousonestolocalizetheerrorsortofocusthedebuggingprocessonarelativelysmallpartoftheprogram.[ 29 ]considersbothtransitionandinvariantdifferencesonsuccessfultracesaswellasthecounterexamplepaths.Itprovidesfeedbackonhow 23

PAGE 24

successfultracescanbetransformedintocounterexamplepaths.Incontrast,ouranalysisisaimedatprovidingfeedbackonhowtotransformacounterexamplepathintoapossiblysuccessfultrace.[ 30 ]generatesmultipleerrortraceshavingindependentcausesandforeacherrorcausereportsasingleerrortrace.[ 37 ]usesdynamicanalysisandmachinelearningtoclassifyprogrampropertiesasfault-revealingandnon-fault-revealingandreportsprograminvariantsthatareinthefault-revealingset.[ 28 38 ]focusonerrortracesonly.[ 28 ]slicesacounterexamplepathtondthestatementsthatdirectlyorindirectlyaffectthefailure.[ 38 ]computesthetransactionalhappens-beforeedgesonthedynamicallygeneratedexecutiontracestodetectblocksthatcannotpreservetheiratomicityandhencecannotbeserialized.Modularmodelcheckingisapowerfulreductiontechniquebasedonassume-guaranteereasoningusingdivideandconquertoachievemodelcheckingscalability.TheCalvinchecker[ 39 40 ]isamodularapproachusingassume-guaranteemodelchecking.Itusesuserspecicationsaboutenvironmentassumptionstoconstrainthreadinteractionsbasedonlocking.In[ 40 ],theyinferredenvironmentassumptionsautomaticallytorelievetheburdenofprovidingtheminaloosely-coupledmultithreadsystem.[ 41 42 ]tookprogramspecicationsinasetofstatetransitiondiagrams(STDs)asinputandanalyzedeachcomponentseparately,thenmodeledthemassynchronousreactivesystems(SRS)synchronizedbyevents.Theresultfortheentiresystemisdeducedfromtheindividualanalysisresult.Modelcheckingisolatedcomponentsneedsamodelofanenvironmentinteractingwiththemand[ 41 ]usedthemostgeneralenvironment,calleduniversalenvironment,andprovidedautomaticassumptiongenerationtotakeintoaccountparallelcomponents.[ 42 ]UnliketheCongurationinthisapproach,summaryfunctionhintroducedinSection 4.1 dependsontheinterleavingofeachmodule'sinternalmicrostep,soglobalstatecannotbemodeleduniquely.InourmodularextensionexplainedatSection 5.2 ,werestrictedtheglobalstateandassumptionsto 24

PAGE 25

onlyincludehformemoriesdenedintheuniversalenvironmentandnon-internaleldsofeachmodule.Moreworkrelatedtoourmodularextensionapproachisdescribedin[ 43 44 ].[ 43 ]implementedautomaticassume-guaranteemodelcheckingusingalearningalgorithmtosynthesizeassumptions.[ 44 ]focusedongeneratingprecisecomponentinterfaceswithoutadditionalenvironmentinformationbasedonlearningduringstatespacetraversal.[ 45 ]generatesenvironmentsforcomponentsautomaticallyusingside-effectsandpoints-toanalysesinmodularmodelchecking.[ 46 ]usedaninterfacegrammartogeneratecomponentstubstouseincompositionalmodelchecking.TheBandera[ 47 ]implementedautomaticenvironmentgenerationfromtheenvironmentassumptions.EnvironmentassumptionsareprovidedasanLTLformulaoraregularexpressiontoordertheactionsinaprogram.Thosegeneratedenvironmentsaredifferentfromouruniversalenvironmentsinthattheirenvironmentsareanabstractionoftheentiresystemtoapproximatethebehavioroftherestoftheapplicationratherthanmostgeneralexecutionenvironmentsthatcancoverallpossibleconcreteusagescenarios.Thetargetofourenvironmentisdifferentinthatthemostimportantrequirementistoaddnoadditionalhappens-beforeorderratherthantoprovidethespecicexecutioncontext.Wedonotrestricttheenvironmentofthemoduleaccordingtothespecicationinthissense.[ 48 ]suggestedusingstaticdataowanalysistogeneratethemostgeneralenvironmentofanopenreactivesystem.[ 49 ]automatedthegenerationofareasonablebehaviormodelofthearticialenvironmentusingstaticanalysisfocusingonthelevelofparallelism.Noneoftheabovementionedapproachesdealswithdataracedetection.Therace-freepreconditionssuggestedinSection 5.2 resembletheannotationsintypebasedracedetectionapproaches.TheRaceFreeJavatypesystempresentedin[ 14 ]hadamethodpreconditionannotationrequirestospecifynecessarylocksetinformationatthemethodentry.Thislocksetisusedtostaticallychecktheguarded byclausesinthemethodbody.TheParameterizedRaceFreeJavain[ 50 51 ]extends 25

PAGE 26

RaceFreeJavawithownershipparameterandisbasedontheconceptofobjectownership.Everymethodisrelatedwithrequiresclauseandlocaltypeinferencereducestheburdenofannotations.ExtendedParameterizedRace-FreeJavain[ 52 ]allowsamoreaccurateanalysisofraceconditionsandcombinesitwithanatomicitycheck.JAC[ 15 ]distinguishesapreconditionannotationifandaguardannotationwhen.Aguardannotationdelaystheexecutionoftheinvokedmethoduntiltheconditionissatised.ThesynchronizationinJACisdeterminedbythetypesystemandifpreconditionisforspecifyingthelegalenvironmentsotherthanlocksettoexecutethemethod.Ourrace-freepreconditionsinSection 5.2 includeasummaryfunctionhaswellasanexplicitorimplicitlocksetandareveriedduringmodelcheckingratherthanstaticallycheckedbyatypesystem. 26

PAGE 27

CHAPTER3FOUNDATION 3.1TheJavaMemoryModelThenewJavaMemoryModel(JMM)revisedaspartofJava5.0in2005[ 6 53 ],denesthelegalbehaviorforamultithreadedJavaprogramanditguaranteessequentialconsistencywhenaprogramisfreefromadatarace.Morespecically,JMMdenesthelegalbehaviorofcorrectlysynchronizedprogramsandout-of-thin-airguaranteesforincorrectprograms.Thesemanticsofamemorybarrierusingavolatileeld,immutableobjectusingnaldeclaration,andcausalityanddependenciesareexplainedbothformallyandinformallytodeterminelegalexecutionsusingcommititerations.Ateachiteration,JMMcommitsasetofmemoryactionsthatareconsistentwithalreadycommittedactionsandformsasetoflegalexecutions.TherstrequirementofJMMistoguaranteesequentialconsistencyforcorrectlysynchronizedprograms.Correctlysynchronizedprogramsareprogramswithoutanydataraceinallsequentiallyconsistentexecutionsofthem.Forexample,thefollowingcodefragmentcontainsadataraceonxandyinanysequentiallyconsistentexecution,thustheresultr2==2,r1==1,whichisnotallowedundersequentialconsistency,ispossible[ 53 ]. //initially,x=y=0;//Thread1//Thread2r2=x;r1=y;y=1;x=2;Conversely,whenbothxandyaredeclaredasvolatilevariables,sinceallmemoryactionsonvolatilesaresynchronizationactions,theprogramiscorrectlysynchronizedandtheaboveresultisprohibited.Inaddition,JMMalsoguaranteesnoout-of-thin-airforincorrectprogramswithadatarace.ThisensuresJMMtobesecureandsafeevenforincorrectprograms,which 27

PAGE 28

isnotthecaseinothermemorymodels.Anexampleofout-of-thin-airresultisasfollows[ 53 ]: //initially,x=y=0;//Thread1//Thread2r1=x;r2=y;y=r1;x=r2;//r1==r2==42InJMM,eventhoughtheaboveprogramisnotcorrectlysynchronized,itisguaranteedthatavaluesuchas42willnotappearinr1orr2.Thereadandwriteofavolatilevariablearesynchronizationactionsandfunctionasamemorybarriertoallowallsubsequentmemoryactionstoobservemostup-to-datevalues.AnexampleistheclassSimpleinChapter 1 withdonedeclaredasvolatile.ThereadofdoneinOtherThreadisamemorybarrierandthesubsequentreadofxisguaranteedtoseemostrecentwriteofvalue1.Finaleldscanbeusedtocreatethread-safeimmutableobjectsandguaranteetoseethecorrectlyinitializedvaluefornaleldsthemselvesandallreachableobjectsusingthem.ThecausalityinJMMrestrictscausalcycles,suchasaboveout-of-thin-airresultcode,butallowsthatresultedfromcompileroptimizationsusingglobalanalysisanddependencebreakingasfollows[ 53 ], //initially,x=y=0;//Thread1//Thread2r1=x;r3=y;r2=r1|1;x=r3;y=r2;//globalanalysisallowsr1==r2==r3==1 28

PAGE 29

//initially,x=y=0;//Thread1//Thread2r1=x;r3=y;r2=x;x=r3;if(r1==r2)y=2;//redundantreadeliminationallowsr1==r2==r3==2Tosatisfythisrequirement,JMMallowsthecommitmentofanyoftheuncommittedwritesthathasthesameaddressandvaluewithalreadycommittedwritesandanyoftheuncommittedreadsthatreadapreviouslycommittedwriteinboththejustifyingexecutionandtheexecutionbeingjustied.Thepurposeofthisrequirementistopreventacommittedactionincludingonethatdependsonanuncommitteddatarace.Intherestofthissection,wegiveanoverviewoftheformaldenitionoftheJMM.Forthemostpart,ourtreatmentfollowsthatof[ 54 ], 1 which,inturn,isbasedontheoriginalspecicationoftheJMMgivenin[ 6 53 ]. 2 Anactionisamemory-relatedoperationathatbelongstoasinglethreadThread(a).Anactionaffectsvariablevormonitor(lock)mandhasakind,whichisoneofthefollowing:volatilereadfromv,volatilewritetov,non-volatilereadfromv,non-volatilewritetov,lockingoflockm,unlockingoflockm,startingathreadt,detectingterminationofathreadt,andinstantiatinganobjectwithasetofvolatileelds,volatiles,andasetofnon-volatileelds,elds.Alloftheseactiontypes,withtheexceptionofnon-volatileread,non-volatilewrite,andobjectinstantiation,aresynchronizationactions. 1 Ourtreatmentexplicitlyhandlesobjectinstantiation.2 Themostimportantdifferencesbetween[ 53 ]and[ 54 ]arethatthelatterrequiresthatthetotalorderforSCexecutionsbeconsistentwithboththesynchronizationorderandprogramorder(asopposedtojusttheprogramorder),formulatesthesemanticsintermsofniteexecutions,andignoresexternalactions. 29

PAGE 30

AnexecutionEisgivenbyatuplehA,P,po!,so!,W,Viwhere Aisanitesetofactions, Pisaprogram, po!,theprogramorder,isapartialorderonAobtainedbytakingtheunionoftotalordersrepresentingeachthread'ssequentialsemantics, so!,thesynchronizationorder,isatotalorderoverallofthesynchronizationactionsinA, V,thevaluewrittenfunction,assignsavaluetoeachwrite,and W,thewrite-seenfunction,assignsawriteactiontoeachreadactionsothatthevalueobtainedbyareadactionrisV(W(r)).Executionsaresubjecttocertainwell-formednessconstraints,butitisnotrequiredthatthewrite-seenfunctionreturnsthemostrecentwritetothevariableinquestionorthatthewrite-seenfunctionsforactionsondifferentthreadsareconsistent,thusallowingvarioussortsofsequentiallyinconsistentbehavior.Thesynchronizes-withrelationonactions,denotedsw!,isgivenbelow. Anunlockactiononamonitorlockmsynchronizes-withallsubsequentlockactionsonmbyanythread. Awritetoavolatilevariablevsynchronizes-withallsubsequentreadsofv. Theactionofstartingathreadsynchronizes-withtherstactionofthenewlystartedthread. Thenalactioninathreadsynchronizes-withanactioninanyotherthread(e.g.,join,orinvokingtheisAlive()method)thatdetectsthethread'stermination. Thewritingofdefaultvaluesofeveryobjecteldsynchronizes-withtherstaccessoftheeld.Inthedescriptionsabove,subsequentisdeterminedbythesynchronizationorder. 30

PAGE 31

Well-formednessconstraints 3 onexecutionsincludesuchunsurprisingrequirementsastypecorrectness,correctbehavioroflocks,consistencywiththesequentialsemanticsoftheprogram,andconsistencyofvolatilereadsandwriteswiththesynchronizationorder.Inaddition,awell-formedexecutionsatiseshappens-beforeconsistencywherehappens-beforeorderisatransitive,irreexivepartialorderontheactionsinanexecutionobtainedbytakingthetransitiveclosureoftheunionofsw!andpo!.Happens-beforeconsistencymeansthatareadrofvariablevisallowedtoseetheresultsofawritew=W(r)providedthat risnotorderedbeforew,i.e.:(rhb!w). Thereisnointerveningwritew0tov,i.e.,:9w0:whb!w0hb!r.Twooperationsconictifneitherisasynchronizationaction,theyaccessthesamememorylocation,andatleastoneisawrite.Adataraceisdenedtobeapairofconictingoperationsnotorderedbyhb!.Asequentiallyconsistent(SC)executionisonewherethereisatotalorder,sc!,ontheactionsconsistentwithpo!andso!andwhereareadrofvariablevseestheresultsofthemostrecentprecedingwritew,i.e., wsc!r, Thereisnointerveningwritew0tov,i.e.,:9w0:wsc!w0sc!r.AJavaprogramiscorrectlysynchronizedifallsequentiallyconsistentexecutionsaredataracefree. 3 Inadditiontothewell-formednessconditions,legalexecutionsaccordingtotheJMMarealsorequiredtosatisfyadditionalcausalityconditionsthatconstrainthebehaviorofprogramswithdataraces,thusprovidingcertainsafetyguaranteesforprogramwithraces.Ourgoalistodetectdataracessothattheycanbeeliminatedratherthanreasonaboutthepropertiesofprogramswithdataraces,consequently,theseconditionsarenotrelevantfortheworkdescribedinthispaper. 31

PAGE 32

Theorem1. Anylegalexecutionofawell-formedcorrectlysynchronizedprogramissequentiallyconsistent.ThisisapropertyoftheJMM[ 6 53 ]andisequivalenttoTheorem1in[ 54 ],whereaproofcanbefound.Theorem 1 iscrucialforjustifyingourapproach.Itmeansthatwecanuseamodelchecker,whichassumesSC,tocheckwhethertheprogramiscorrectlysynchronized,andifso,soundlyusethemodelcheckertocheckotherpropertiesasdesired. 3.1.1ModelCheckerPropertiesSincewewanttouseamodelcheckertocheckpropertiesrelatedtotheJMM,weneedtorelatethepathsgeneratedbyamodelcheckertotheexecutionsdenedbytheJMM.Pathsgeneratedbyamodelcheckeraretotallyorderedandsequentiallyconsistentsothatthevaluereadbyeachreadisthevaluewrittenbythemostrecentwritetothatvariable.AmodelcheckerforJavawillusuallyincludebyte-codelevelinstructions,someofwhicharenotvisibletootherthreads,andthusdonotcorrespondtoactionsintheJMM.Apathcorrespondstoanexecutionifitcontainsalloftheactionsintheexecution,thetotalorderisconsistentwithbothpo!andso!,andthesamevaluesarereadandwrittenforeachaction.Moreformally,apathPath=hApath,path,Vpath,WpathiwhereApathisasetofactions(whichmayincludelocalactionsnotvisibletootherthreads),pathisatotalorderonApathcorrespondstoexecutionE=hA,P,po!,so!,W,ViifApathjA=A,pathjpo!=po!,pathjso!=so!,Vpath=V,andWpath=W.WesaythatamodelcheckerissoundwithrespecttotheJMMifeverypathgeneratedbyamodelcheckerforprogramPcorrespondstoawell-formed,sequentiallyconsistentexecutionofP. Lemma1. Ifapathgeneratedbyasoundmodelcheckercontainsadatarace,thenthereisadataraceinthecorrespondingexecution.Twopathsareequivalentw.r.t.anexecutionEwithactionsAiftheybothcorrespondtoEandtheeventsinAappearinthesameorderinbothpaths. 32

PAGE 33

Lemma2. Twoexecutionequivalentpathswillexhibitthesamesetofdataraces.AmodelcheckeriscompletewithrespecttotheJMMifforeverysequentiallyconsistentexecutionE,atleastonecorrespondingpathisgenerated.FromTheorem 1 andtheabovedenitionsandlemmas,ifallpathsgeneratedbyamodelcheckerthatissoundandcompletewithrespecttotheJMMforaprogramParedataracefree,soarethesequentiallyconsistentexecutionsofP.Ifamodelcheckerissoundbutnotcomplete,itmaystillbehelpfultondbugs(dataraces)thatmayleadtosequentiallyinconsistentbehavior,butnottoverifythataprogramwillbesequentiallyconsistent.Boundingand(potentially)partial-orderreductionmayaffectthecompletenessofamodelchecker. 3.1.2AWeakenedConditionforSequentialConsistencyDataracesinpathscanbeclassiedintothreekinds:WR,WW,andRW,meaningthattheconictingoperationsareawritefollowedbyaread,awritefollowedbyawrite,andareadfollowedbyawrite,respectively.Wesaythatprogramisweaklycorrectlysynchronizedifallsequentiallyconsistentexecutionsaredatarace-freeorwhosecorrespondingpathscontainonlyRWdataraces. Theorem2. AnylegalexecutionEofawell-formedweaklycorrectlysynchronizedprogramissequentiallyconsistent.TheproofrequiresonlyminormodicationstotheproofofTheorem 1 in[ 54 ].ThemostimportantpartofourproofisaweakerversionofLemma 4 .Lemmas 3 and 5 donotrelyondataracefreedomandtheproofsgivenin[ 54 ]requirenomodication. Lemma3. IfpisapartialorderonAandtisatotalorderonSA,thenq=(p[t)+isapartialorderonA. Lemma4. Foranywell-formedexecutionofaweaklysynchronizedwell-formedprogramP,ifeachreadseesawritethathappens-beforeit,theexecutionissequentiallyconsistent. Proof. LetE=hA,P,po!,so!,W,Vibeawell-formedexecutionofP. 33

PAGE 34

FromLemma 3 ,(po![so!)+isapartialorder.Lettbeatopologicalsortof(po![so!)+.Sincetisatotalorderonawell-foundedset,itiswell-founded.Wewillprovesequentialconsistencyusingwell-foundedinductionont.Now,supposewehaveareadrinEandallreadsxtrexceptrseethemostrecentwrite.LetwbethemostrecentwritetothevariablereadbyrpriortorsothatW(r)twtr,andfromthehypothesis,W(r)hb!r.Inawell-formedexecution[ 6 ,Section17.4.7],[ 54 ,rule9indenition6],Wmustbeconsistentwithhb!,thusr6hb!W(r)andifW(r)hb!whb!rthenW(r)=w.Therearethreepossibilities: 1. rseesthemostrecentwriteinEw.r.t.t(i.e.W(r)=w) 2. thereisaWWracebetweenW(r)andwinEw.r.t.t 3. thereisaWRracebetweenwandrinEw.r.t.tNow,consideranexecutionE0thatcontainsalltheactionsofEpriortorw.r.t.t.LetA0=fxjxtrgandE0=hA0,P,po!jA0A0,so!jA0A0,W[r!w],Vi.FromLemma 5 ,E0isawell-formedexecution.Fromtheinductionhypothesisandchoiceofwtobethemostrecentwritetothevariablereadbyr,E0issequentiallyconsistent.BecauseofthewaythatE0wasconstructed,ifthereisaWWracebetweenW(r)andwinEw.r.t.t,thenthereisaWWracebetweenW(r)andwonasequentiallyconsistentpathinE0.Similarly,ifthereisaWRracebetweenwandrinE,thenthereisaWRbetweenwandronasequentiallyconsistentpathofE0.Ineithercase,wehaveasequentiallyconsistentpathwitharace,whichcontradictstheassumptionthatPisweaklycorrectlysynchronized.ThusweconcludethatrseesthemostrecentwriteinE. Lemma5. LetPbeawell-formedprogram,hA,P,po!,so!,W,Viawell-formedexecution,hb!itshappens-beforeorder,tatotalorderonA,r2Aareadactionofvariablev,andw2Aawriteactiontovsuchthat: tisconsistentwithso!andhb!, 34

PAGE 35

foreveryreadr2AwehaveW(r)hb!r, wisthemostrecentwritetorint,i.e.wtrandforallwritesw0tovariableveitherw0tworrtw0.LetA0befxjxtrg.ThentheexecutionhA0,P,po!jA0A0,so!jA0A0,W[r!w],Viisawell-formedexecution. Proof. Givenin[ 54 ]. TherestoftheproofreliesonthedenitionofalegalexecutionintheJMM.Inordertomakethepresentationself-contained,wewillrepeatthedenitionofalegalexecutionfrom[ 54 ,Denition7].Denition:LegalExecutionAwell-formedexecutionE=hA,P,po!,so!,W,Viwithhappens-beforeorderhb!islegalifthereisanitesequenceofsetsofactionsCiandwell-formedexecutionsEi=hAi,P,poi,soi,Wi,Viiwithhappens-beforeorderhbisuchthatC0=,Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1Ciforalli>0,SCi=A,andforeachi>0,thefollowingaresatised: 1. CiAi 2. hbijCi=hb!jCi 3. soijCi=so!jCi 4. VijCi=VjCi 5. WijCi)]TJ /F16 5.978 Tf 5.76 0 Td[(1=WjCi)]TJ /F16 5.978 Tf 5.75 0 Td[(1 6. Forallreadsr2Ai)]TJ /F3 11.955 Tf 11.95 0 Td[(Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1,Wi(r)hbir 7. Forallreadsr2Ci)]TJ /F3 11.955 Tf 11.95 0 Td[(Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1,Wi(r)2Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1andW(r)2Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1Theseconditionsareintendedtoruleoutcausalloopsthatcouldleadtosituationswheredataracescausevaluesreadtoappearoutofthinair.Now,wereturntotheproofoftheTheorem 2 .FromLemma 4 ,itissufcienttoshowthateveryreadinalegalexecutionEofaweaklycorrectlysynchronizedprogramseesawritethathappens-beforeit. 35

PAGE 36

SinceEislegal,thereisacommittingsequencefCi,EigjustifyingE.WeshowbyinductiononithatforallreadsinCi,W(r)hb!r.Thebasecase,C0=holdstrivially.Now,weassumeforallreadsinr2Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1,wehaveW(r)hb!r,andshowthatforanyreadr2Ci,wehaveW(r)hb!r.Fromtheinductionhypothesisandlegalityrules2and5,wegetWi(r)hbirforallreadsr2Ci)]TJ /F5 7.97 Tf 6.58 0 Td[(1.Fromlegalityrule6,wegetforallreadsr2Ai)]TJ /F3 11.955 Tf 12.14 0 Td[(Ci)]TJ /F5 7.97 Tf 6.58 0 Td[(1,Wi(r)hbir,andsinceCiAiandCi)]TJ /F5 7.97 Tf 6.58 0 Td[(1Ci,wehaveforallreadsr2Ci,Wi(r)hbir.Fromlegalityrule7,forallreadsr2Ci=Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1,wehaveWi(r)2Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1andW(r)2Ci)]TJ /F5 7.97 Tf 6.58 0 Td[(1,andthusfromrule5,Wi(r)=W(r).Fromrule2,wecanconclude,forallreadsr2Ci=Ci)]TJ /F5 7.97 Tf 6.59 0 Td[(1,W(r)hb!randthuswiththeinductionhypothesisthedesiredresultforallreadsr2Ci,W(r)hb!r.SinceallreadsinAbelongtosomeCi,thisimpliesthateveryreadinEseesawritethathappensbeforeit.Sincecorrectsynchronizationimplyingsequentialconsistencyisconsideredafundamentalpropertyofmemorymodels[ 11 ],thisresultisofintrinsicinterest.AswillbediscussedinChapter 4 ,ignoringRWdataraceswillallowustogiveamoreefcientimplementationofJRFwithoutsacricingtheabilitytoverifysequentialconsistency. 3.2ModelCheckingandJavaPathFinderModelcheckingisaformalvericationmethodforaconcurrentsystem.Thesystemundertestwillbeveriedusingthecorrectnessspecicationgivenintemporallogicbyexhaustivelysearchingallpossibleinterleavingofthreads.Traditionally,modelcheckingiswidelyusedinhardwarecircuitdesigntoverifythebehaviorisconsistentwiththedesignspecicationbutrecently,softwaremodelcheckersareacceptedaspowerfulandusefulautomatedtoolsthankstotheiradvantages[ 55 ].Unlikelogicalinferenceusingatheoremprover,modelcheckingdoesnotrequireaproofandiseasytoexpressconcurrencypropertiesinlogics.Inaddition,modelcheckersprovidecounterexampleswhenthepropertyisviolated.However,becauseitisanexhaustiveapproach,they 36

PAGE 37

inevitablysufferstatespaceexplosion.Inamodernpracticalsystem,therearetoomanythreadsandpossibledatachoices.Techniquestoaddresstheproblemhaveintroducedsuchasapartialorderreduction,symbolicrepresentationofstatesandtransitions,andcompositionalmodelchecking[ 56 ].Partialorderreductionisatechniqueusingabstractionofstatestoreducetotalnumberofstates.Givenaproperty,partialordermethodsexploreonlyareducedpartoftheglobalstatespacethatissufcientforcheckingthegivenproperty[ 57 ].Thesimplestpartialorderreductionistoreducetheindependenttransitions.Symbolicmodelcheckingavoidsexplicitenumerationofstatesusingabinarydecisiondiagram(BDD),acanonicalformofrepresentingbooleanformula[ 58 ].Thecompositionalapproachisaneffectivetechniquetoachievescalabilityincomplexsystemverication.Insteadofexploringtheentirestatespaceofthesystemundertest,modularmodelcheckingdecomposesthesystemintosmallerindependentmodulesandappliesavericationalgorithmtoeachindividualmodulewithanenvironmentmodel.Onceeverymoduleischeckedwithasetofassumptionsoftheenvironmentmodel,theseassumptionsareveriedontheenvironmenttodischargethem.Thisassume-guaranteereasoningiswidelyusedwithenvironmentgenerationinautomaticvericationapproaches.Modularmodelcheckingbasedonthistechniquegenerallyrequiresadditionalinformationprovidedmanuallyaboutthemodule'senvironment.Thismanualannotationisanerror-proneprocessandtherewereattemptstoautomatetheassumptiongeneration[ 41 42 ].Onemorerequirementusingthisparadigmistorestrictthegeneratedenvironmentsufcientlytoavoidfalsepositives.[ 43 44 ]trytomakethoseinterfacesprecisethroughlearningsearchspace.Boundedmodelchecking[ 59 ]considersonlyaniteprexofapathwithlengthkbyunrollingthenitestatemachineforksteps.Theclosedsysteminmodelcheckingnaturallyhasaboundinthenumberofthreadsandthenumberofrepeatediterations.Thisisanotherlimitationusingmodelcheckingtocheck 37

PAGE 38

propertiesinopensystemssuchaslibraries.TheboundconstraintinJRFmodularextensionisexplainedinSection 5.2 .AmongdifferentapproachestodetectdataracesdiscussedinSection 2.1 ,wechoosemodelcheckingwithindirectcodeinstrumentationtoimplementsummaryfunctionhdescribedinSection 4.1 .Firstofall,weruledoutadynamicapproachduetoitsweakness:itonlydetectsracesinacurrentexecution.WealsoruledoutdirectcodemodicationthroughannotatingatargetprogramwithcodesimplementingouralgorithminSection 4.1 .Itisundesirablethattheprogramsemanticswouldbechangedby ADirecthannotationthroughanextraannotationtool BIndirecthannotationthroughamodelcheckerinternalfunctionFigure3-1. Operationalmodelofdirectvsindirectannotationinmodelchecking 38

PAGE 39

additionalcode.Figure 3-1 showstwooperationalmodels, 3-1A oneusingdirectcodeannotationtotargetapplicationandtheother 3-1B indirecthannotationaccomplishedbymodelcheckerinternalfunction.Thelatterdoesnotchangethenumberofstatestosearchandwillnishinonepasswithoutpreprocessingincodemanipulationtool.TheJPF[ 60 ]modelchecksJavabytecodebyreadingJavaclasslesandsimulatingtheirexecutionusingitsownvirtualmachinewithon-the-yvericationofspeciedproperties.ApropertyviolationisreportedbyJPFalongwithacounterexample,theexecutionpaththatledtotheviolation.JPFisanexplicitstatesoftwaremodelchecker,andprovidesalistenerinterfacethatweusedtoextenditsfunctionalityforJRFtoavoiddirectcodeinstrumentation.Theinterfaceprovidesasetofcallbackfunctionsallowinglowleveloperations,suchasobjectcreation,objectlockingandunlocking,thestartofanewthread,andeachexecutionofaninstructiontobeinterceptedandaugmentedwithuser-suppliedcode.ThedefectsJPFiscapableofhandlingareuserspecicpropertiesprovidedbyprogrammers,aswellasdeadlocksandunhandledexceptions.ArecentreleaseofJPFincludesseveralextensionsinseparateruntimemodulesthataremaintainedastheirownprojects.Examplesofinterestingextensionsarejpf-concurrentandjpf-symbcprojects.Thejpf-coreproject,whichimplementsvirtualmachineandcoremechanisms,doesnotsupportthefullfunctionalitiesofthestandardJavalibrary.Mostofall,thejava.util.concurrentpackageisanimportantcoremoduleinconcurrentdatastructures,butonlyispartiallyimplementedinjpf-core.Thejpf-concurrentprojectisanoptimizedextensionofjava.util.concurrentandcurrentlyimplementshalfoftheoriginalconstructs.Thejpf-symbcisthesymbolicexecutionframeworkofJPF.Itusesacongurableinstructionset,calledfactory,tochangetheinstructionsemanticsfromimmediatevaluetosymbolicexecution. 39

PAGE 40

CHAPTER4JAVARACEFINDER 4.1TheSummaryFunctionhInthissection,weintroduceafunctionhthatsummarizeshb!ateachpointinanSCexecution,allowingdataracestobedetectedastheyoccurduringmodelchecking.LetAddrbethesetof(abstract)memorylocationsrepresentingnon-volatilevariablesintheprogram,SynchAddrbethesetof(abstract)memorylocationsrepresentingvariableswithvolatilesemanticsandlocks,andThreadsbethesetofthreads.Wesummarizethehappens-beforerelationasafunctionh:SynchAddr[Threads!2Addrthatmapsthreadsandsynchronizationvariablestosetsofnon-volatilevariablessothatx2h(t)meansthatthreadtcanreadorwritevariablexwithoutcausingaWWorWRdatarace.ForanitesequentiallyconsistentexecutionEofprogramPthathasasetofstaticnon-volatilevariablesstatic(P),letEnbetheprexofEoflengthn,i.e.,thesequenceofactionsa0,a1,...,an)]TJ /F5 7.97 Tf 6.59 0 Td[(1,andhnbethevalueofhafterperformingalltheactionsinEn.Weassumethatthreadmainisthesinglethreadthatinitiatestheprogram.Initially, h0=z.ifz=mainthenstatic(P)else(4)Thewaythathn+1isobtainedfromhndependsontheactionan.First,wedenefourauxiliaryfunctions;release,acquire,invalidate,andnew.Thefunctionrelease(t,x)takesasummaryfunctionhandyieldsanewsummaryfunctionbyupdatingh(x)toincludethevalueofh(t).Itisusedwithactionsbythreadtthatcorrespondtothesourceofasw!edge,forexample,writingavolatilevariablex,releasinglockx,startingthreadx,etc. release(t,x)hb=h[x7!h(t)[h(x)](4) 40

PAGE 41

Thefunctionacquire(t,x)takesasummaryfunctionhandyieldsanewsummaryfunctionobtainedfromhbyupdatingh(t)toincludethevalueofh(x).Itisusedinactionsthatformthedestinationofasw!edge,forexample,readingavolatilex,lockinglockx,andjoiningordetectingterminationofthreadx. acquire(t,x)hb=h[t7!h(t)[h(x)](4)Thefunctioninvalidateyieldsanewsummaryfunctionbyremovingxfromh(z)forallz6=t.Itisusedinactionswherethreadtwritesnon-volatilex.invalidate(t,x)hb=z.if(t=z)thenh(z)elseh(z)nfxgThefunctionnewisusedtoincorporateanewlyinstantiatedobjectintothesummaryfunction.Ityieldsanewsummarybyaddingtheseteldstothevalueofh(t)andinitializingthepreviouslyundenedvaluesofhforthenewvolatilevariables. new(t,elds,volatiles)hb= (4) z.if(t=z)thenh(t)[eldselseif(z2volatiles)thenfgelseh(z)Wedene norace(x,t)=x2h(t)(4)Thedenitionofhn+1,whichdependsonhnandactionan,isgiveninFigure 4-1 .Todetectdataracesduringmodelchecking,wemaintainhandchecknorace(x,t)beforethereadingorwritingofnon-volatilexbythreadt.Whenthisconditionholdsforallnon-volatilereadsandwritesinanexecution,wesaytheexecutionish-legal. 41

PAGE 42

actionanbythreadthn+1 writeavolatileeldvrelease(t,v)hn readavolatileeldvacquire(t,v)hn lockthelockvariablelckacquire(t,lck)hn unlockthelockvariablelckrelease(t,lck)hn startthreadt0release(t,t0)hn jointhreadt0acquire(t,t0)hn t'.isAlive()if(t0.isAlive())hnelse(acquire(t,t0)hn) writeanon-volatileeldxinvalidate(t,x)hn readanon-volatileeldxhn instantiateanobjectcontainingnon-volatileeldseldsandvolatileeldsvolatilesnew(t,elds,volatiles)hn Figure4-1. Denitionofhn+1 Wecanproveseveralfactsabouth-legalexecutions.Inthefollowing,weuselast(E)todenotethelastelementofnitesequenceEandEnjw(x)[new(x)todenotethesubsequenceofactionsinEnthatwritetoxplustheinstantiationaction.Therst,ratherobviouslemmaconrmsourintuitionthatnon-staticvariablesbelongingtoobjectsthathavenotbeeninstantiatedyethavenotbeenwritteninanyh-legalexecutionandwillserveasthebasecaseforinductiveproofsofotherresults. Lemma6. ForanSCexecutionE,andnon-staticx,ifxhasnotbeeninstantiated,Enjw(x)[new(x)isempty.Thefollowingtwolemmas,fornon-staticandstaticvariables,respectively,saythatatanypointinanh-legal,SCexecution,themostrecentthreadtowriteanon-volatilevariablecanaccessitwithoutcausingaWRorWWdatarace. Lemma7. Foranh-legalSCexecutionEnandnon-volatile,non-staticvariablex,ifxhasbeeninstantiatedinEnandt=thread(last(Enjw(x)[new(x))),thenx2hn(t). Proof. Theproofisbyinduction.SincexcannothavebeeninstantiatedinE0,thebasecaseholdstrivially.Now,assumethelemmaholdsfornandshowthatitholdsforn+1.Therearefourcases: 42

PAGE 43

xhasnotbeeninstantiatedinEnandactionandoesnotinstantiateit.Thenthelemmacontinuestoholdtrivially. xhasnotbeeninstantiatedinEnandactionanisanactionofthreadtthatinstantiatesx.Thenlast(En+1jw(x)[new(x))=anandfromtheruleforobjectinstantiationinFigure 4-1 ,x2hn+1(t). xhasbeeninstantiatedinEn,x2hn(t)wheret=thread(last(Enjw(x)[new(x)))andthread(an)=t.FromFigure 4-1 ,therearenoactionsperformedbythreadtthathavetheeffectofremovingitemsfromhn(t)toobtainhn+1(t). xhasbeeninstantiatedinEn,x2hn(t)wheret=thread(last(Enjw(x)[new(x)))andthread(an)6=t.FromFigure 4-1 ,eitherx2hn+1(t)oranwritestoxwithx2hn(thread(an)).Thisfalsiest=thread(last(Enjw(x))whileestablishingthread(an)=thread(last(En+1jw(x)).Sincetheexecutionish-legal,x2hn+1(thread(an). Lemma8. Foranh-legal,SCexecutionEnandnon-volatile,staticvariablex,ift=(ifEnjw(x)6=thenthread(last(Enjw(x))elsemain)thenx2hn(t). Proof. Againtheproofisbyinduction.Sincexisstatic,x2h0(main)andthebasecaseholds.Now,assumethelemmaholdsforEnandshowthatitholdsforEn+1.Therearetwocasesthataresimilartothelasttwocasesinlemma 7 x2hn(t)wheret=(ifEnjw(x)6=thenthread(last(Enjw(x))elsemain)andthread(an)=t.FromFigure 4-1 ,therearenoactionsperformedbythreadtthathavetheeffectofremovingitemsfromhn(t)toobtainhn+1(t). t=(ifEnjw(x)6=thenthread(last(Enjw(x))elsemain)andthread(an)6=t.FromFigure 4-1 ,eitherx2hn+1(t)oranwritestoxwithx2hn(thread(an)).Thisfalsiest=(ifEnjw(x)6=thenthread(last(Enjw(x))elsemain)whileestablishingthread(an)=thread(last(En+1jw(x)).Sincetheexecutionish-legal,x2hn+1(thread(an). Thenextlemmaformsthebasisofoursoundnessproof.Inthislemma,ahb=)166(!bindicateseitherahb!bora=b. 43

PAGE 44

Lemma9. LetEbeawell-formed,h-legalSCexecution.Thenforallnon-volatilesx,allthreadst,allvolatilesv,andalln,x2hn(t))Enjw(x)[new(x)hb=)166(!last(Enjt)^x2hn(v))Enjw(x)[new(x)hb=)166(!last(Enjw(v))whereforsetS,Shb=)166(!smeans(8s0:s02S:s0hb=)166(!s). Proof. Theproofisbyinduction.Forthebasecase,wehaveE0,whichdoesnotcontainanyactionsandwhereh(main)=static(P)andhisundenedforallotherarguments.SinceE0isempty,therightsidesofbothimplicationsaretriviallytrue.Now,weassumethepropertyforEnandshowitholdsforEn+1.Wemustconsidereachkindofaction. Writeavolatileeldvbythreadt.hn+1(t)=hn(t),andduetoprogramorderandthehypothesis,Enjw(x)[new(x)hb!last(En;write(v,t)jt).Thesecondrequirementissimilartox2hn(v).Forthenewadditions,wealreadyhavex2hn(t)!Enjw(x)[new(x)hb!last(En;write(v,t)jt).Sincethiswritewillbecomelast(Enjw(v)),theconditionisreestablished. Readavolatileeldvbythreadt.Thishastheeffectofaddingtheelementssatisfyingx2hn(v)tohn+1(t).Sinceforanysuchx,allwritesofxhb!thelatestwriteofv,whichhb!thereadwhichisthecurrentactiononthreadt,andthusbecomeslast(En+1jt),thusreestablishingthecondition. Lockthelocklck.Thesituationisanalogoustoreadingavolatile Unlockthelocklck.Thesituationisanalogoustowritingavolatile Startthreadt0.Thiscaseissomewhatpeculiarsinceitinvolvestwothreads,andoneofthemdoesnothaveanyactions.Todealwiththis,weconsiderthattherstactionofathreadisasyntheticactionwherethestartactionbythestartingthreadhappens-beforethisaction.Then,bytransitivityofhb!,therstconditionisreestablished. Jointhreadt0.Thiscasefollowseasilyfromthetransitivityofhb!. Detectingterminationt'witht'.isAlive().Similartojoin. 44

PAGE 45

Writeanon-volatileeldxbythreadt.Inthiscase,bothconditionsareviolateditneednolongerbethecase,forexample,thatthatallwritesofxhappenbeforethelaststatementofthreadt'.Thesolutionistoremovexfromhn+1(t0),thusreestablishingthecondition.Sincewerequirex2h(t)beforeperformingthewrite,therequiredconditionforthreadtwillbemaintained. Readanon-volatileeldx.Thisdoesnotaffectthesecondcondition,andextendingEnwithareadoperationpreservestherst. Instantiateanobjectcontainingnon-volatileeldsbythreadtnon-volatileeldseldsandvolatileeldsvolatilesTheinstantiateaddsthenewnon-volatilestohn+1(t)whileextendingEnwiththeinstantiateaction,thuspreservingtherstcondition.Thesecondconditionisnotaffectedbymerelydeninganewh(v)tobeempty. Thelemmatellsusthatifx2hn(t),thenalltheprecedingwritestoxhappen-beforethelatestactiononthreadt.Sincetherearenowritestoxbetweenlast(Enjt)andan,byprogramorder,last(Enjt)hb!an,andbytransitivityallwritestoxhb!an.Asaresult,ifanisawritebythreadttox,theactioncanbeperformedwithoutcausingaWWdatarace.Thenexttheorem,whichfollowseasilyfromLemma 9 justiesourapproach. Theorem3. Ifallsequentiallyconsistentexecutionsofawell-formedprogramareh-legal,thentheprogramisweaklycorrectlysynchronizedandthusallitslegalexecutionsaresequentiallyconsistent. 4.2DataRaceSpecicSearchHeuristicModelcheckingisawaytoverifyprogramcorrectnessbyexhaustivelyexploringallpossiblestatesofamultithreadedprogram.Themostwell-knownproblemofmodelcheckingisastate-spaceexplosionandthislimitationmakesJPFnotabletobescalable.Ourrstapproachtothisproblemwasprogramslicing.Theprimaryideawastodividethewholeprogramintosmallsliceswithminimumcodesinthemandwerecapableofconcludingifaspecicdatahasaraceornot.WehadusedIndusJavaslicer[ 61 ]inourpreliminaryexperiment,buttheresultwasnotasusefulaswehad 45

PAGE 46

expected.Theconcurrentprogramsarecomplicatedandhardtosliceintosmallpieces.Althoughanon-volatiledataaccessisfreefromothernon-volatiledataraceanalysisinthatthehofanon-volatileisindependentofthatofothernon-volatiles,wecannotexcludethoseaccessesiftheychangethecontrolow.Anotherhinderancewasthedifcultiesofincorporatingseveraltools.JPFandIndushavetheirownrequirementsaboutthespecicversionofJavaplatformandotherthird-partytools.Thenextapproachwastouseamethodsummarytosummarizetheh-relatedbehaviorofmodules.Insteadofexploringallpossiblestates,weintendedtosearchthestatesinamethodsummary.However,theformalizationofamethodsummarywasnotsuccessful,andtheworkloadtoimplementthismechanismintoJPFwasalsoproblematic.Inthissection,weexplaintheheuristicsearchalgorithmsthatincorporatetherationalesabouthowtodetectaraceearlierbychoosinganexecutionpathwithhigherprobabilityofarace.Section 4.2.1 includesamotivatingsmallexample,andChapter 6 hasmoresupportingexperimentalresults. 4.2.1HeuristicsContemporarymodelcheckingresearchsuggestabetterwaytosolvethescalabilityproblem.Insteadofreducingthenumberofstatestovisit,thefocusistonderrorearlierifoneexists.Itisstillnotguaranteedtoterminate,butthesuccessfuldetectionoferrorcanbeincreasedbychangingtheorderofstatesearchingaccordingtoheuristics.Ateachstate,amodelcheckerchoosesthenextstatefromallpossiblecandidatesandchecksthepropertiesthatshouldbesatisedatthatpointintheprogram.Whentherearenocandidatesforanextstateleft,itbacktrackstocheckanyremainingschedulingsequences.Theorderthatstatesaretraversedduringmodelcheckinginuenceshowmanystatesmustbevisitedbeforeerrorsarefound.Sinceadataracerequirestheinteractionoftwothreads,asearchstrategywithmorethreadsinterleavingislikelyto 46

PAGE 47

ndadataraceearlierthanadepth-rstsearch(DFS)strategy,whichhasminimuminterleavingthroughsequentialschedulingofthreadsatthebeginning.Ratherthansimplyincreasingthreadinterleaving,wealsoconsiderthenatureofdataracesandproposethefollowingheuristics,whichdependonthecurrentvalueofh,tochoosethenextstate.Inthedescriptions,currreferstothecurrentthread. Writes-rst(WF):Althoughadataracemayoccurateitherareadorawrite,thesourceofadataraceinvolvingmemorylocationmisawriteofmthatcausesaninvalidateoperationstoremovemfromh(t)forallt6=curr.Anyfuturereadorwritetombysomeotherthreadresultsinadataraceunlessanappropriatesynchronizationactionhasoccurred.Thusthisheuristicprioritizeswriteoperations. Watch-written(WW):Iftherehasbeenawriteonamemorylocation,m,itispossiblethatafuturereadorwriteonmbyanotherthreadwillresultinadatarace.Thus,thisheuristicprioritizesoperationsonamemorylocationthathasrecentlybeenwrittenbyadifferentthread. Avoidrelease/acquire(ARA):Adataracefreeprograminvolvesappropriatelylocatedmatchingreleaseandacquireoperations.Althoughprogramswithdataracesmayalsohaveacquireandreleaseoperations,theexistenceoftheseonapathmayindicatealowerprobabilityoftheexistenceofadatarace.Thisheuristicprioritizesoperationsonthreadsthatdonothavearecentacquireoperationprecededbyamatchingreleaseontheexecutionpath. Acquire-rst(AF):Whenanacquireoperationisexecutedafteramatchingrelease,ahappens-beforeedgeiscreatedonthecurrentpath.However,iftheacquireisexecutedbeforethematchingreleasestatementthenthisdoesnotresultinanhappensbeforeedgeandcannotpreventadatarace.Thisheuristicprioritizesacquireoperationsthatdonothaveamatchingreleasealongtheexecutionpath.Thissituationoftencorrespondstosituationsofunsafepublicationofanotherwisecorrectlysynchronizedobject. 1 Themainpurposeoftheseheuristicsistominimizethehappens-beforeorderingsthatpreventsadataracetooccur.Noheuristicisdisjointandcouldbecombinedto 1 Publicationofanobjectistheactofmakingitsreferencevisibletootherthreads.Unsafepublication(section3.5of[ 62 ])canleadtoapartiallyconstructedobjectbecomingvisibletootherthreads. 47

PAGE 48

Initially,flag0=flag1=turn=shared=0;/*allfieldsarenon-volatile*/Thread1Thread2=================================================================================s1:flag0=1;s6:flag1=1;s2:turn=1;s7:turn=0;s3:while(flag1==1\&turn==1){/*spin*/}s8:while(flag0==1\&turn==0){/*spin*/}s4:shared++;s9:shared++;/*criticalsection*//*criticalsection*/s5:flag0=0;s10:flag1=0;Figure4-2. OneiterationofPeterson'sAlgorithm maketheheuristicmorepowerful.TheexperimentalresultfordifferentcongurationofthemisdiscussedinChapter 6 .AfragmentofthewellknownPeterson'salgorithminFigure 4-2 showstheadvantageoftheWFheuristicoveradepth-rstsearch.ThesearchspaceusingDFSforthisexampleisshowninFigure 4-3A ,andtheWFheuristicsearchinFigure 4-3B .DFS-basedmodelcheckingtakessevenstatestondthedataraceonturn,whilethe ADFSalgorithm BHEURISTICalgorithmFigure4-3. ModelcheckingofPeterson'salgorithmusingdifferentsearchstrategies 48

PAGE 49

heuristicsearchndsthesameraceaftervisitingonlyfourstates.Thecounterexamplepathisshorterandeasiertoanalyze. 4.2.2AlgorithmFigure 4-4 showstheheuristicsearchalgorithmusedinJRF.Itstoresthestatesinamaxpriorityqueue,whereapriorityisthesumoftheheuristicvalueandthedepthofagivenstatetimesthemaxheuristicvalue(MAX=WRITEwritten by other).Theheuristicvaluesonlyaffectthechoiceofanextstateamongchildrenofthecurrentstate.Oncethenextstateischosenandadvanced,itsnewlygeneratedchildrenalwayshavehigherprioritythanthestatesremaininginthequeue.Thisguaranteesthatthehighestprioritychildrenandtheirdescendantswillbeexploredrst.Thesearchalgorithmvisitsthestatesindescendingpriorityorderuntilitreachesanerrorstateornomorestatesareleft.ThealgorithmforcomputingtheheuristicvalueforastateisgiveninFigure 4-5 .Theheuristicvaluesaredeterminedbyconsideringthedataracerelatedheuristics. AlgAlgorithmHeuristicSearchQ:maxpriorityqueues,s0:statevalue:integerQ emptyput(initialstate,maxinteger)intoQwhileQnotemptydo(s,value) removefromQifsisanerrorstatethenprint(error)breakforeachsuccessors0ofsdoifs0isnotmarkedthenmarks0put(s0,HeuristicValue(s')+depthMAX)intoQFigure4-4. Heuristicsearchalgorithm.StatesareprioritizesbasedontheirheuristicvalueascomputedinFigure 4-5 andthesearchdepth. 49

PAGE 50

AlgorithmHeuristicValue(s:state)v:variableifsreachedviawritetonon-volatilevthenifvmostrecentlywrittenbyanotherthreadthenreturn8/*WRITEwritten by other(WF_WW)*/elsereturn7/*WRITEwritten by self(WW)*/ifsreachedviareadfromnon-volatilevthenifvmostrecentlywrittenbyanotherthreadthenreturn6/*READwritten by other(WF)*/elsereturn5/*READwritten by self(WF^WW)*/ifsreachedviareadfromavolatilevorlockinganobjectnotreleasedbeforethenreturn4/*ACQUIREwithout prior release(ARA)*/ifsreachedviareadfromavolatilevorlockinganobjectreleasedbeforethenreturn2/*ACQUIREwith prior release(ARA_AF)*/ifsreachedviavolatilewriteorunlockinganobjectthenreturn1/*RELEASE(AF)*/elsereturn3/*OTHER(ALL)*/Figure4-5. Algorithmfordecidingheuristicvaluesforstatesbasedontheirlikelihoodofleadingtoadatarace.Heuristicvaluesbecomeavailableaccordingtotheheuristics(WF,WW,ARA,AF)presentedinSection 4.2.1 Non-volatileWRITEhashigherheuristicvalue(8and7)thannon-volatileREAD(6and5,respectively)wheneverwrites-rstiscongured.Non-volatilewritten by other(8and6)isgreaterthannon-volatilewritten by self(7and5,respectively)withwatch-written.acquire-rstandavoidrelease-acquirealsodeterminethe1,2,and4hierarchy.Non-volatileaccesses(5through8)arealwaysthebestcandidatesforadatarace,andacquire/releaseaccessesthatgeneratehappens-beforeedgesshouldbeavoidedforaslongaspossible.ACQUIREwithout prior releaseischosenearlierthanregularoperationstoavoidafuturerelease-acquirepossibility.Themainpurposeofthoseheuristicsistofollowpathsthatminimizethehappens-beforeorderingssinceitisthelackofhappens-beforeorderingsthatcausesdataraces.TheheuristicscanbeusedtogetherandtheheuristicsearchalgorithmgiveninFigure 50

PAGE 51

4-5 canbeconguredinvariouswaysbyturningonandoffeachofthe4heuristics:WF,WW,AF,ARA.TheexampleseenearlierinFigure 4-3B showsthesearchpathwithonlywrite-rstcongured.Forexample,ifthechoiceofheuristiciswrites-rstandwatch-written,theonlyavailableheuristicvaluesareWRITEwritten by other,WRITEwritten by self,READwritten by other,READwritten by self,andOTHER.IncomparisonwithDFS,theheuristicsearchalgorithmrequiresmorememory.AmodelcheckerusingDFSonlyadvancesandbacktracks,whileaheuristicsearchgeneratesandstoresallpossiblechildstatesbeforeadvancing,andthenlaterrestoresthem.Althoughonlysofhrelateddataarestoredduringstateadvanceandbacktrack,acompletecopyofthehissavedwhenthechildstatesaregenerated.Heuristicsearchalsotendstotakemoretimebecauseitvisitsmorestatesingeneral.AswecanseefromFigure 4-3 ,DFSonlygenerates7statestondtherace,buttheWFheuristicsearchgenerates8statessinceittakesintoconsiderationallchildrenofthecurrentstatetodecidenextone.Theadvantageofheuristicsearchisthat,inmostcases,ittendstonddataraceswithashortercounterexamplepaththanDFS.Thisisimportantbecause,inourexperience,reasoningaboutthecauseofadataraceusingthecounterexamplepathisnotastraight-forwardtask,especiallywhenthelengthofthepathisfairlylong.ExperimentalresultsandfurtherdiscussionaboutunderstandingthecounterexamplepatharegiveninChapter 6 4.3ImplementationThecoreofJRFistomaintainarepresentationofthesummaryfunctionhdescribedinSection 4.1 ;thelistenercodeinterceptsrelevantinstructionsandupdatestherepresentation,asdescribedinFigure 4-1 .Inaddition,thenoracepropertyischeckedpriortoallnon-volatilereadsandwrites.Inadditiontotheindirectcodeinstrumentationtomaintainsummaryfunctionh,JRFimplementsefcientsearchheuristicsinordertodecreasethenumberofvisitedstatesbeforetheracedetection.JRFalsousestechniquestoleveragetheoverheadof 51

PAGE 52

bookkeepingadditionalinformationsuchashandacquiringhistory;lazyrepresentationofarrayelementsandhbitmapentries,storingofhintoastack,andexcludinganuntrackedvariableandthreadlocalmemoriesfromthehentry. Figure 4-6 showsthecomponentsofJRF.AllcomponentsareimplementedasacongurablemoduletoallowthemostexibilityinusingJRF.InFigure 4-7 ,thedefaultcongurationofJRFfromajrf.jpflecanbefound.ThedirectorystructureofaJRFdistributionfollowstheguidelinesofJPFruntimemodules.ThefollowingsectionsdescribetheimplementationdetailsofeachJRFmodule. Figure4-6. JRFcomponentsandtheirJPFcounterparts 52

PAGE 53

Figure4-7. ThedefaultJRFcongurationinjrf.jpfle 53

PAGE 54

4.3.1RepresentationofhInthissection,wedescribearepresentationofhthatissuitableforimplementationinamodelcheckingtool.Thiscontextrequiresspaceefciency,efcientupdating,and,sincemodelcheckinginvolvesbacktracking,awaytoefcientlysaveandrestorepreviousincarnations.Wetakeadvantageofthefactthat,inJava,threadsandlocksarealsoobjectsandhandleelementsofSynchAddr,Thread,andAddruniformlyasmemorylocations.RecallthathmapsSynchAddr[Threadto2Addr.Thismappingisimplementedasanarrayofbitvectors.EachSynchAddr[Threadsisgivenauniqueindex, 2 conceptuallycorrespondingtoarow,andAddranindexconceptuallycorrespondingtoacolumn.Thennorace(t,x)holdswhenh[row(t),column(x)]==1. WhenanelementofSynchAddr[Threadsiscreated,anelementofthearrayofbitvectorsisreserved.WhenanelementofAddriscreated,anindexisassigned,butspaceinthecorrespondingbitvectorisnotallocateduntilitisactuallyused.Thisimpliesthatthebitvectorsaredynamicallyresized.Theacquireandreleaseoperationscanbeimplementedasasinglesetunionstep,andthenoracecheck,whichisusedmostoften,canbedonewithbitmasking.Theinvalidateoperation,however,involvesanumberofrowsofsetoperations.Fortunately,jSynchAddr[ThreadsjisfarlessthanjAddrj.Inourexperiments,theformerisontheorderoftens,whilethelattercaneasilyexceedthousands. 2 Eachmemorylocationisgivenauniquekeyconstructedfromthenameoftheclass,theinstancenumber,andtheeldname(orarrayindexifthelocationinquestionisanarrayelement).ThekeysareinturnmappedtothecorrespondingindexinthebitvectorforelementsofAddr,andthearrayholdingthebitvectorsforelementsofSynchAddr[Thread. 54

PAGE 55

Figure4-8. Internalrepresentationofh Thechangesinharestoredinaseparatestackinordertorestorethevaluetoanearlierstatewhenthemodelcheckerbacktracks.Figure 4-8 illustratesthebitmapforh,andahistorystackusedtostores.Thesofhincludethechangesfollowingacquire,release,invalidate,andallocation,includingreuseofgarbagecollectedmemoryandinstantiationinastaticinitializer.Inaddition,thechangesinprior releaseandwritten byinformation,explainedinSection 4.3.2 ,arealsostoredinthehhistorystack.Duringstatebacktracking,theseareundonetorecoverhtopreviousvalues.Therstrepresentationofhhadusedjava.util.HashMapinsteadofabit-vector,andthechangeshadbeenmanagedbyamodelcheckerratherthanstoringinaseparatestack.Fromthefactthatjava.util.HashMapstoresthejava.util.Entrysinanarrayofalinkedlist,andeachentrystoresamappingfromakey,SynchAddr[Thread,tovalue,Addr,thebit-vectorrepresentationwasexpectedtoutilizethespaceefcientlyduetotheextraspacetomaintainheavyweightJavaclassstructures.Experimentalresults 55

PAGE 56

conrmourexpectationandthebit-vectorwithstackrepresentationoutperformsthejava.util.HashMaprepresentationcombinedwithmodelcheckermanagement. 4.3.2TheListenerImplementationJPFsupportsaListenerinterfacethatcanbeusedtoextenditsfunctionality.TheinterfacenotieslowleveleventsattheJPFjavavirtualmachinelevelthroughpreregisteredcallbackfunctions.TypesoftheseeventsareVMrelatedevents,suchasinstructionExecuted,threadStarted,andobjectLocked,searchrelatedevents,suchassearchAdvanced,searchBacktrackedandpropertyViolated;thoseeventsaredenedinVMListenerandSearchListenerrespectively.JRFlistenerinheritsPropertyListenerAdapter,whichimplementsbothVMListenerandSearchListenerinterfaces.CallbackfunctionsinheritedfromSearchListenermanagethestackstructuretostoreofh,andcallbacksfromVMListenermanageh,asdescribedinSection 4.3.1 .Theoperationsacquire,release,invalidateandassertingnoraceareperformedasappropriatewhenexecutionofmemorymodelrelatedinstructionsoccur.ThehierarchicalstructureofthesystemisshowninFigure 4-9 .ThelistenerliesatthesamelevelasotherJPFcode,outsidethetargetmodelclasses.ToimplementtheheuristicsearchalgorithmdescribedinSection 4.2.2 ,weusethefollowingtwoauxiliaryfunctions: written by:AddrThread!Boolean prior release:SynchAddr!Booleanwritten by(m,t)returnstruewhenthemostrecentwritetothegivenmemorylocationmwasdonebythreadt,andprior release(m)returnstruewhenthegivenmemorylocationmwasreleasedatleastoncepreviously.Inanearlierincarnationofourtool,weannotatedthebytecodewiththeseassertionsandthencheckedforassertionviolationsusingstandardJPF.Thelistener-basedapproachtoextendingJPFdescribedaboveprovedtobebothmoreefcientandmoreexible. 56

PAGE 57

Figure4-9. OverallsystemhierarchyofaJRFextensiontoJPFandtheirdataposition 4.3.3PruningtheSearchSpaceJPF,likemostmodelcheckers,usestechniquestoprunethesearchspace.JPFgiveseachstateastatenumberdeterminedbythevaluesofallvariablesinthestaticanddynamicareaalongwiththreadstatessuchaslockandprogramcounterinformation.Ifastateisencounteredwhosestatenumberhasalreadybeenseen,JPFdoesnotexplorethatstate'ssubtreeagain.However,stateswiththesamestatenumbermayhavedifferenthistoriesthatresultindifferentvaluesofh,makingthisparticularoptimizationunsoundfordataracedetection.Intheremainderofthissection,wedevelopacondition,thatwhensatised,doesallowthesearchspacetobesafelyprunedwhenstateshavethesameJPFstatenumberbutdifferenthvalues.IftwostateshavethesameJPFstatenumber,theywillhaveidenticalsubtreesinthesearchspace,exceptpossiblyforthevaluesofh.Letfuture:State!2SynchAddr[Threads[Addrbeafunctionmappingstatestothesetofmemorylocationswhosehvaluesarereadorwrittenwhenprocessingthesubtree.If,for 57

PAGE 58

example,atransitionbetweens0toachildstates00inthesubtreeofsisareadofxbythreadt,thenbothxandtareinfuture(s).Iftwostatess1ands2havethesamestatenumber,thenfuture(s1)=future(s2).Lemma 10 saysthatiftwostateshavethesameJPFstatenumber,andoneofthemhasanhfunctionvaluethatisasubsetoftheotherforallofthememorylocationsandthreadsthatwillbeaccessedintheir(common)subtree,thenthatsubsetpropertyholdsintheentiresubtree. Lemma10. Supposes1ands2havethesamestatenumberandthevalueofhatthesestates,hs1andhs2,respectivelysatisfyhs1jfuture(s1)hs2jfuture(s1).Thenforallcorrespondingstatess10ands20inthesubtreesofs1ands2,hs10jfuture(s1)hs20jfuture(s1). Proof. TheproofisbyinductiononthelengthofthepathsincealloftheactionsdenedinFigure 4-1 preserve. Theorem 4 justiespruningofthesearchspace. Theorem4. Supposethats1isastateinthesearchtreewhosesubtreehasbeenfoundtobefreefromdataraces.Ifs2hasthesamestatenumberass1andhs1jfuture(s1)hs2jfuture(s1),thentherearenodataracesinthesubtreeofs2. Proof. followsimmediatelyfromlemma 10 andthedenitionofnorace. 4.3.4UsingtheModelJavaInterfaceJavaprogramsrelyonanumberofplatformdependentfunctions,includingthreadimplementationandlowlevelsynchronizationprimitivesthatareimplementedinnativecode.TheModelJavaInterface(MJI)isprovidedtoallowJPFtohandlethesesituations:nativecodeisexecutedbythehostJVMandnotmodelcheckedbyJPF.Unfortunately,thismeansthattheh-relateddatastructuresarenolongercorrectafterexecutingnativecode.WeextendedDefaultFieldFactoryclassofJPFtoimplement 58

PAGE 59

newJRFFieldFactoryclasstoincludethenecessarycallstohmanipulationcode. 3 ToenablecompletecoverageofJavafeaturesbyJRF,itwasalsonecessarytoextendMJItoincludesomeclassesmissingfromthejava.util.concurrent.atomicpackageintheJPFdistribution,includingallclassessupportingatomicarrays.Manylock-freedatastructuresusetheseclasses[ 2 3 63 64 ].Exceptfornalizers,thecurrentimplementationofJRFcorrectlyhandlesallJavalanguagefeaturesrelatedtotheJMM. 4.3.5ProblemswithStateBacktrackingTherewereseveralissueswithstatebacktrackingthatmadethetaskofextendingJPFlessstraightforwardthanonemightexpect.StaticinitializerscomplicatethemanagementofhsincethestatebacktrackingschemebuiltintoJPFdoesnotreloadclassesorre-initializetheirstaticelds,evenwhenthestateisbacktrackedtoapointbeforetheclasswasloaded.Thus,itisnecessarytoidentifywhichmemorylocationsareallocatedinastaticinitializer.Theselocations,iftheyhavenotbeenupdated,shouldbeaccessiblebyallthreads,includingthosecreatedlaterwithoutcausingadatarace.WemaintainedanothersetofAddr,h(static initializer),wherestatic initializerisasyntheticthreadrepresentingtheclassloaderthread.Thisrepresentsthelocationscorrespondingtostaticvariablesthathavenotyetbeenupdatedoutsidethestaticinitializerfortheclass.Inadditiontotherelease(parent thread,child thread),thethreadstartoperationshouldalsoperformrelease(static initializer,child thread).AnothercomplicationinthelistenerimplementationistheunpredictablegarbagecollectionbyJPF.Whenanobjectisgarbagecollectedduringstatesearch,itisnolongerinusealongthatpathandmightbereusedforanewobject.Thisproblematicsituationoccurswhentheuniquekeyfortheobjectisnolongerunique.Theoriginal 3 BeforeJPFversion4,ithadbeennecessarytomanuallymodifytheirModelJavaInterfacestoreectthewaythetargetfunctionsupdateh.Thishadbeenapainstakingtaskandhardtomanipulate.ThenewFactorystructureadoptedafterJPFversion5allowsmaximumcongurabilityincludingextendablebytecodefactoryandeldfactory. 59

PAGE 60

object,whichisstillusedinotherstoredpaths,sharesitskeywithanewobject,resultinginincorrectsharingofh.Statebacktrackingwillusethehofthenewobjectunlessitisproperlyrestoredtothevaluebeforegarbagecollection.Thehhistorystackstoresthenecessarygarbagecollectionandreallocationinformation. 4.3.6UntrackedVariablesJRFofferstheoptiontomarkindividualnon-volatilelocationsasuntracked,sothatreadsandwritesofthesevariablesdonothaveanyeffectonhandnorace(x,t)isnotchecked.Inotherwords,weonlyrequirenorace(x,t)forx2Addrnuntrwhereuntristhesetofvariablesmarkedasuntracked.Thedenitionofh0ischangedto h0=z.ifz=mainthenstatic(P)nuntrelse(4)andwemodifyhn+1inFigure 4-1 asshowninFigure 4-10 .Thefollowinglemmajustiestheuseofuntrackedvariables. Lemma11. ConsideranexecutionEnandlethnbethesummaryfunctionwithallx2Addrtracked,andhuntrnbethesummaryfunctionforthesameexecutionwherethethevariablesinuntrAddrareuntracked.Then 8u2(SynchAddr[Threads):hn(u)nuntr=huntrn(u).(4) actionanbythreadthn+1 writeanon-volatileeldx=2untrinvalidate(t,x)hn writeanon-volatileeldx2untrhn instantiateanobjectcontainingnon-volatileeldseldsandvolatileeldsvolatilesnew(t,eldsnuntr,volatiles)hn Figure4-10. Modieddentionofhn+1withuntrackedvariables.OmittedactionsarethesameasshowninFigure 4-1 60

PAGE 61

Proof. Theproofisbyinductiononn.Thebasecasefollowsimmediatelyfromthedentionsofh0giveninEquations 4 and 4 .Thecaseforreadnon-volatilefollowsimmediatelyfromtheinductionhypothesis.Therulesforwritevolatile,readvolatile,lock,unlock,start,join,andisAliveallhavetheformhn+1=hn[u7!hn(u0)[hn(u00)]forsomeu,u0,andu00,sowecangiveasingleproofforallofthem.hn+1(u)nuntr=(hn(u0)[hn(u00))nuntr=hn(u0)nuntr[hn(u00)nuntr=bytheinductionhypothesishuntrn(u0)[huntrn(u00)=huntrn+1(u)Forinvalidate(t,x)wherex2untr,hn+1(u)nuntr=(if(t=u)thenhn(u)elsehn(u)nfxg)nuntr=if(t=u)thenhn(u)nuntrelsehn(u)nfxgnuntr=if(t=u)thenhuntrn(u)elsehn(u)nfxgnuntr=if(t=u)thenhuntrn(u)elsehn(u)nuntr=if(t=u)thenhuntrn(u)elsehuntrn(u)=huntrn(u)=huntrn+1(u) 61

PAGE 62

Forinvalidate(t,x)wherex=2untr,hn+1(u)nuntr=(if(t=u)thenhn(u)elsehn(u)nfxg)nuntr=if(t=u)thenhn(u)nuntrelsehn(u)nfxgnuntr=if(t=u)thenhuntrn(u)elsehn(u)nfxgnuntr=if(t=u)thenhuntrn(u)elsehuntrn(u)nfxg=huntrn+1(u)Thenalcaseisnew.hn+1(u)nuntr=(if(t=u)thenhn(t)[eldselseif(u2volatiles)thenfgelsehn(u))nuntr=(if(t=u)then(hn(t)nuntr)[(eldsnuntr)elseif(u2volatiles)thenfgelsehn(u)nuntr)=(if(t=u)thenhuntrn(t)[(eldsnuntr)elseif(u2volatiles)thenfgelsehuntrn(u))=new(t,eldsnuntr,volatiles)huntrn=huntrn+1 62

PAGE 63

Aclasscanbemarkedastrusted,whichwillresultinallofitsprivatenon-volatilesbecominguntracked.Apackagecanbemarkedtrusted,whichwillresultinallpackage-privatenon-volatilevariablestobecomeuntracked.Therearetwodifferentmotivationsfornottrackingvariables.TherstallowsJRFtobeusedwithprogramsthatcontainso-calledbenignraces.TheJMMprovidesastrongguarantee,namelysequentialconsistencyforproperlysynchronizedprograms.Italsoconstrainsprogramswithdataracesinordertoprovidesomeminimalsecurityguarantees.Theseguaranteesincludetypesafetyandtheguaranteethattherearenoout-of-thin-airvalues.Asaresult,itispossible,inprinciple,forprogrammerstowriteandreasonaboutthecorrectnessofprogramsthatcontaindataraces.Ifthepresenceofaracecannotcauseaprogramtoviolateitsspecication,evenwhenthenon-SCbehaviorallowedbytheJMMisexhibited,itisconsideredtobebenign.Occasionallybenignracesareallowedinprogramsforperformancereasons.Reasoningaboutprogramswithracesisquitedifcultandshouldbeconsideredtobeajobforexpertsonly.Mostprogrammersshouldwriterace-freeprograms.JRFdoesnotsupportreasoningaboutprogramswithracesinthesensethatonecannotconstructaprogramwithdataraces,submitittoJRF,andexpectthemodelcheckertohaveexploredthestatesthatonlyoccurinsequentiallyinconsistentexecutionsallowedbytheJMM.Markingthelocationinvolvedinabenignraceasuntrackedallowstherestoftheprogramtobeanalyzedforraces.Itistheprogrammer/JRFuser'sresponsibilitytoensurethatwhenaraceisignoredbymarkingaelduntracked,itisindeedbenignandthatitseffectsaresufcientlyencapsulatedthattherestofthemodelcheckingwillbesound.ThesecondmotivationfornottrackingcertainvariablesistoimprovethescalabilityofJRF.Markingclassesorpackagesastrustedisaconvenientwaytoreducethetimeandmemoryrequirementsbynotcheckingclassesthatwearewillingtoassumedonothaveraces,oronlybenignraces.JRFdoes,however,continuetotrackhappens-before 63

PAGE 64

edgesinvolvingvolatileeldsandlocksdenedintrustedclassesandthenon-volatileeldsaccessibleoutsideoftrustedclasses(orpackages)inordertocontinuetopreciselydetectdataracesinorcausedbyotherclasses.Forexample,acommonwaytosafelypublishobjectsistomakethemavailabletootherthreadsbypassingthemthroughadatastructure,suchasaqueue,whoseimplementationisprovidedinthejava.util.concurrentpackagewheretheinsertionofanobjecthappens-beforeremovaloftheobject.Ifthepackageismarkedastrusted,whilethehappens-beforeedgesrelatedtoinsertingandremovingobjectswillbepreserved,nocheckingwillbedoneontheinternalnon-volatilevariablesintheclassunlesstheyarevisibleoutsidethetrustedcode.BymarkingtheclassesinthestandardJavareleaseastrusted,asignicantreductioninthetimeandspacerequirementstomodelcheckanapplicationclasscanbeachieved.JRFsetsjava.lang,java.util,java.io,JPF INITIALIZER,sun.misc,andgov.nasa.jpfasdefaultuntrackedpackages. 4.3.7LazyRepresentationofArrayElementsIntherepresentationofhdescribedinSection 4.3.1 ,everymemorylocationrequiresanentryinthehtable.Thisalsoappliestoarrays,whichrequireanentryforeacharrayelement. 4 JRF,however,usesasinglehentrytoabstractlyrepresenttheentirearrayuntilsuchtimethatanindividualelementisupdatedandobtainsadifferentvalueforh.Thiswouldoccurwhentheabstractarraylocationisnotinthehsetofthecurrentthreadorisincludedinsomeotherhsetwhenupdated.Atthatpoint,anadditionalentryinthehtableforthatelementisallocated.Theabstractlocationisstillusedfortheremainingelements.Thelazyrepresentationofarrayelementssavesacquireandreleasetimeaswellasspace.Forinstance,anintegerarrayof 4 ThereisnowayinJavatomaketheindividualelementsofanormalarrayhavevolatilesemantics.Markingthearraydeclarationwiththevolatilekeywordgivesvolatilesemanticstothereferencetothearray,butnottheindividualelements.ThisisacommonmisunderstandingoftheJMMandafrequentsourceoferrors. 64

PAGE 65

Algorithmnon-volatileread(x:memorylocation)t:currentthreadifx2untrthenreturn;elseifxisithelementofanarrayathenifa[i]hashrepresentationthennorace(a[i],t);else/*useabstraction*/norace(a,t);else/*notanarray*/norace(x,t);Algorithmnon-volatilewrite(x:memorylocation)t:currentthreadifx2untrthenreturn;elseifxisithelementofanarrayathennorace(a,t);/*checkaccessibilityrst*/ifa[i]hashrepresentationtheninvalidate(t,a[i]);else/*useabstraction*/if8v2SynchAddr[Threadswithv6=t,a=2h(t)thenreturn;/*invalidate(t,a)isredundant*/elsenew(t,a[i],;);/*allocatehfora[i]*/else/*notanarray*/invalidate(t,x)Figure4-11. Modiedalgorithmfornon-volatilereadandwritewithlazyrepresentationofarrayelementsanduntrackedvariables sizenrequiresn+1differenthentrieswithoutlazyrepresentation,butJRFinitiallyallocatesonlyonehentry.Allhaccessesforarrayelements,suchasnoraceandinvalidate,usethisvalueuntiladditionalentriesarecreated.Figure 4-11 summarizesthehmanipulationfornon-volatilereadandwriteoperations.TherepresentationofstringsinJavaprovidesstrongmotivationforlazyrepresentationofarrayelements.AstringisrepresentedintheStringclassusingachararray. 65

PAGE 66

Althoughthejava.lang.Stringclassisimmutable 5 ,stringconcatenationwiththe+operator,utilizesthejava.lang.StringBuilderclass,whichisnotimmutable.Thechararrayusedtorepresentthecontentscouldbechangedbycallingtheirpublicmethods,suchasappendandreplace.Ifwehadaprintmessageoflength100concatenatedusing+,JRFwithoutlazyarrayrepresentation,100entrieswouldbeneededinthehtable.Awriteofavolatilevariablevbythecurrentthreadwouldaddthose100memorylocationstothehsetofv,andlaterthosewouldbepropagatedtootherthreadsthatreadveventhoughtheyneitherreadnorwritethemessage.Inpractice,withoutlazyarrayrepresentation,stringsbecomeasignicantconsumeroftimeandspaceinJRF. 4.3.8ThreadlocalOptimizationIfwehaveknowledgeofwhichmemorylocationsarenotshared,thiscanbeusedtoreducetheoverhead.Inparticular,sinceadataraceonanon-volatilevariablebydenitioninvolvestwothreads,anon-staticvariablethatisonlyaccessedbythethreadthatinstantiateditcanneverbeinvolvedinadatarace.Staticvariablesareaccessedbyaclassloaderthreadduringstaticinitialization.Theyarethenaccessedbyatmostoneapplicationthread,theycanalsoneverbeinvolvedinadatarace.ThisisbecausetheJMMguaranteesthatstaticinitializationisguaranteedtohappen-beforetherstaccessofavariable,sayx,byanyapplicationthread,sayt,thusestablishingnorace(t,x)forthataccess.Then,norace(t,x)willonlybefalsiedbyaninvalidateoperationcausedbyanotherthreadwritingx,whichdoesnotoccursincetistheonlyapplicationthreadtoaccessx.Asaresult,wedonotneedtomaintainhdataorchecknoracefornon-volatilesthatareonlyaccessedbyasinglethread.Ifourknowledgeofsharingisimprecise,itmayleadtounnecessaryoverhead(ifwedonotrecognizeathreadlocal 5 TheStringclassisusuallyconsideredtobeimmutable,althoughtechnicallyitisnot.ThehashcodeinitializationisperformedlazilywithabenigndataracethatreliesontheJMMnaleldsemanticsforcorrectness. 66

PAGE 67

variableandtreatitasshared)oramisseddatarace(ifwetreatasharedvariableasthreadlocal.)Thesituationwithvolatilesandlocksislessobvious,buttheoutcomeissimilar.Ifweknowthataparticularvariableorlockxisonlyaccessedbyasingleapplicationthread,thennorace(x0,t0)doesnotdependonh(x). Lemma12. Supposethatvariableorlockxisaccessedbyonlyoneapplicationthreadt.Thenh(x)h(t). Proof. Thepropertyistrueinitiallysinceh(x)=.ItiseasytoseethatitismaintainedbyallactionsinFigure 4-1 Theorem5. Ifavolatilevariableorlockxisonlyaccessedduringanexecutionbythreadt,thenforanythreadt0,h(t0)doesnotdependonh(x). Proof. Theresultistrueinitially.Bylemma 12 ,itismaintainedwhent0=t.Fort06=t,wenotethattheonlysuchdependencywouldhavetobeintroducedbyacquire(t0,x).However,thisisaresultoft0readingx,whichdoesnothappen. Ifknowledgeofsharingoflocksandvolatilesisimprecise,itmayleadtounnecessaryoverhead(ifalocalvariableisconsideredshared)orapossiblefalsealarm(ifasharedvariableisconsideredtobelocal)butwillnotresultinadataracebeingmissed.Applyingthethreadlocaloptimizationrequiressomehowdeterminingwhichvariablesareshared.PossibilitiesincluderunningJPF(aslightlymodiedversionthatmaintainssharinginformation),performingsomesortofstaticanalysis,orallowingtheusertospecifythreadlocalvariablesandverifyingthechoices. 4.3.9BenignRaceInmostcases,programmersshouldwriteprogramswithoutdataraces.However,sincetheJMMconstrainsprogramswithraces,itis,inprinciple,possibletoprovethat 67

PAGE 68

adataraceisbenignandtoallowdataracesissometimesdesirableforperformancereasons. 6 JRFprovidesthemechanismtoignorecertaindataracesbyidentifyingthemasbenign.Thenotionofabenigndataraceisdenedbythecircumstances.Insomeapproachesadataraceisconsideredbenigniftheexecutionsarestillsequentiallyconsistent,or,insomeappropriatesense,equivalenttosequentiallyconsistentexecutions.Thishastheadvantagethatthespecicationisimplicit,butmayover-constraintheprogram.Inamodelchecker,itisnaturaltoconsiderracestobebenignwithrespecttoaspecicationprovidedintheformofacheckableproperty.Thus,abenignracemayallowexecutionsthatarenotequivalenttoasequentiallyconsistentexecution,providedtheystillsatisfythegivenproperty.Wedonotlimitthesemanticofabenignraceinourtool,instead,weprovidethewaytomarkspecicprogramcodeasabenignrace.Conceptually,abenignraceandanuntrackedvariablehavesameeffectonthereportedracekind;toignoretherelevantrace.Thebenignraceisrelatedtoaspecicprogrampoint,whereasanuntrackedvariableisaboutamemorylocation.Anotherdifferenceistheresourcesavings.Anuntrackedvariableconsumesneitherthespaceforhentrynorthetimetoexecutenoraceandinvalidate.Ontheotherhand,abenignracerequiresboth,andhasnoresourcesaving.UsersofJRFwouldchooseabenignraceforanintentionalraceonvariablesthatisalsoaccessiblefromotherclassesorpackages.Whenavariableisdeclaredclassprivateorimmutable,ashashcodeinStringclass,useofanuntrackedvariablebetterutilizestheresourcesandthishasnoriskofbeing 6 PerhapsthemostcommonexampleofabenignracepatterninJavaprogramsislazyinitializationofeldsaccessedwithoutsynchronization,asinTgetX()fif(x==null)freturnf(....);gelsereturnx;gwherexisnotvolatileandfonlydependsonnaleldsandconstants.Thispatternisfoundintheinitializationofthehashcodevalueinthejava.lang.Stringclass. 68

PAGE 69

involvedinaracethroughthird-partycodes.ThedefaultJRFcongurationincludestwobenignraces:java/util/concurrent/locks/AbstractOwnableSynchronizer.java:56andjava/lang/String.java:652.OurexperimentinSection 6.5 foundonebenignracewhichisaredundantupdateonsharedeldUNIVERSAL DEBUGinmontecarlobenchmark. 69

PAGE 70

CHAPTER5EXTENSION 5.1EliminatingDataRaceUsingCounterexampleAlthoughtheoriginalJPFprovidesthesequenceofstatements(thecounterexamplepath)thatleadstoapropertyviolation,dataraceinourtool,itrequiresatediousmanualefforttoparsetheinformationhiddeninthecounterexampletondtheinterleavingsequenceofthethreadsandthereasonwhythedataraceoccurred.Forinstance,thecounterexamplepathforaslightlymodiedversionofSimpleclassinChapter 1 isshowninFigure 5-1 .Bothxanddoneareinvolvedinraces.Theomittedoutputgivessimilarresultsforadditional(8more)detectedraces.Tounderstandwhatcausedthedetectedrace,weneedtodecodethecounterexamplepathgivenas"trace#1".Clearly,thisisatediousexercise,evenforthissimpleprogramwherethelengthofthecounterexamplepathisonlysix.Thepathlengthmaybeseveralhundredinrealisticexamples.Incontrast,Figure 5-2 showstheoutputoftheanalysisproducedbyJRF.Foreachuniqueracefound,theracesourcestatement,theracemanifeststatement,andsuggestionsforcodemodicationsthatwilleliminatethatracearegiven.Notethatthetoolrecognizedthatmarkingdoneasvolatileissufcienttoeliminatetheraceonxalso.Simpleclasshasmaxdepth6,butourexperimentalresults(Chapter 6 )easilygrewtohundredsandeventhousandsinmaxdepth.Decodingalltransitionstotracehappens-beforerelationsinthiscounterexamplepathisatediousanderror-proneprocedure.Furthermore,thetracedoesnotincludehinformationfromexecutionpathsotherthanthecounterexample.Inmostcases,thoseexecutionpathswithouterrorssetagoodexampleofaracefreepattern.Weprovideacounterexampleanalysisandanexplanationofhowtoxtheracebasedonthehinformationfrombothacounterexamplepathandotherracefreeexecutionpaths. 70

PAGE 71

Figure5-1. PartialoutputfromJRFformodiedSimpleclass.Eightsimilartraceshavebeenomitted. 5.1.1AnalysisSinceadataraceisdenedtobethelackofahappens-beforeedge,wecanleveragetheinformationinhtoexplainwhythereisnohappens-beforeedgebetweenthestatementthatcausedthedatarace,whichwecallthesourcestatement,andthe 71

PAGE 72

Figure5-2. JRFoutputwhichexplainsthesourceoftheraceandsuggestshowtoeliminateit manifeststatementwherethedataracehasoccurred,i.e.,whereassertnoracehasfailed.Thisinformationcanbeusedtoprovidesuggestionsforwaystoeliminatethedataracebycreatingahappens-beforerelationshipbetweenthosestatements.Toimprovethequalityofthesuggestions,wealsomaintaintheacquiringhistory,AcquireHis:Addr)2(SynchAddr[Threads)Threads.Foramemorylocationm,(v,t)2AcquireHis(m)meansthat,thusfaratsomepointinthecomputation,threadtperformedanoperationonvthatresultedinmbeingaddedtoh(t).Theactionsbythreadtthatwouldresultin(v,t)beingaddedtoAcquireHis(m)forsomemcouldbereadingv,lockingv,orjoiningv,wherevisavolatileeld,lock,orthread,respectively.Incontrasttothesummaryfunctionh,whichonlyappliestoaparticularpath,theAcquireHisiscumulativeandcontainsinformationfromallexploredpaths.JRFcurrentlyprovidesfourtypesofsuggestions: Changethevariablethatparticipatedinthedataracetovolatileortouseajava.util.concurrent.atomicarrayifthevariableisanarrayelement. Movethestatementthatcausedthedataracebeforethestatementthatisthesourceofanexistinghappens-beforeedge. 72

PAGE 73

Acquireaparticularlock(eitherintrinsicorextrinsic)beforethemanifestingstatement. Performthesametypeofacquireoperationonanagentmemorylocationashasbeenaccomplishedearlierbysomethread. 5.1.2AlgorithmsThissectiondescribesthealgorithmsusedforgeneratingsuggestions.WhenJRFdetectsadatarace,theraceeliminatorusesthecounterexamplepath(pathInstr),thehinformationforthecounterexamplepath(pathHB),theacquirehistory(AcquireHis),thepositionofthemanifeststatement(raceManifestIndex),andthepositionofthesourcestatement(raceSourceIndex)onthecounterexamplepath.Theprovidedsuggestionsonlyeliminatethedataraceonthecurrentexecutionpathanditispossiblethataftertheuserfollowsthesuggestionandmakesthesuggestedchanges,JRFwillndadifferentdatarace.Thusthetoolwillincrementallyguidetheuseruntilnomoredataracescanbefound. Changea(non-arrayelement)variabletovolatileorimplementwithanatomicclass.DuetothesemanticsofvolatilevariablesinJava,changingavariableinvolvedinaracetovolatileisalwayssufcienttoeliminateadataraceinvolvingthatvariable.Sincevolatilevariablesinhibitcompileroptimizationsandaccessestovolatilesincur AlgorithmmakeChangeToVolatileSuggestions(raceManifestIndex,raceSourceIndex)integerraceManifestIndex,raceSourceIndexletvdenotethevariableaccessedbyinstructionsatraceManifestIndexandraceSourceIndexifvisanarrayelementthenprint"Useatomicarray..."elseprint"Makevvolatile"Figure5-3. Suggestchangetovolatileoratomicarray. 73

PAGE 74

runtimeoverhead,thetrivialwayofeliminatingracesbymakingeverythingvolatileisundesirable.Changingavariabletovolatilelikelytobethemostappropriateinsituationswherethisvariableisbeingusedforpublication(i.e.makingthereferencetoanewobjectinstancevisibletootherthreads).Unsafepublication[ 62 ]isacommonerrorinconcurrentJavaprogramswrittenbyprogrammerswithoutagoodunderstandingoftheJMMandcanleadtoasituationwhereanotherthreadseesapartiallyinitializedobject.Anotherguaranteedsolutionistoreplacethevariablewithanal 1 referencetoaninstanceoftheatomicclasscorrespondingtothevariable'stypeinthejava.util.concurrent.atomicpackage.Forexample,replaceanintvariablewithaninstanceofthejava.util.concurrent.atomic.AtomicIntegerclass.Theseclassesarelessconvenientthanvolatilesbecausetheymustbeaccessedwithgetandsetmethodsandarethebetterchoiceonlyifthelock-freeatomicupdatemethodstheyprovideareneeded.TheseatomicupdatemethodsincludecompareAndSet,whichisfrequentlyusedinlock-freealgorithms,andwhereappropriateforthetype,methodssuchasgetAndAdd,addAndGet,getAndIncrement,etc. Changeanarraytoanatomicarray.Atomicarraysforvariouselementtypesareprovidedintthejava.util.concurrent.atomicpackage.Theyprovidevolatilesemanticsforarrayelementsandthusthischangeisalwayssufcienttoeliminatedataracesinvolvingarrayelements.ArraysareobjectsinJavaandafrequenterroristomarkanarrayreferencevolatilewithoutrealizingthatthisdoesnotprovidevolatilesemanticsfortheaccessestotheelements. Movesourcestatement.Dataracescansometimesbeavoidedbyplacingasourcestatementbeforeastatementthatisthesourceofahappens-beforeedge, 1 Finaleldsmustbesetintheconstructor,cannotbemodied,andhavespecialsemanticsintheJMM.Notethatthevalueencapsulatedintheatomicobjectcanchange,justnottheobjectitself. 74

PAGE 75

asshowninthesecondpathofthestatespaceinFigure 5-7 .ThealgorithminFigure 5-5 rstcallsthendHBEdgesalgorithminFigure 5-4 tocomputeallthehappens-beforeedgesthatresultfromsynchronizationactionsonthecounterexamplepath.Ahappens-beforeedgeisapairofinstructionswherethereleaseinstructionisthesourcevertexandthematchingacquireinstructionisthedestinationvertex.Instructionsareidentiedbytheirpositionsonthecounterexamplepath.Afterhaving AlgorithmndHBEdges(pathInstr):SetofintegerpairsStackpathInstrSetofintegerpairsHBEdges ;forindexDestfromsize(pathInstr)to1doifpathInstr(indexDest)isanacquirethenforindexSourcefromindexDest-1to1doifpathInstr(indexSource)isareleasematchingpathInstr(indexDest)thenHBEdges HBEdges[(indexSource,indexDest)breakreturnHBEdgesFigure5-4. Findthesetofhappens-beforeedgesthroughsynchronizationactionsonpathpathInstr. AlgorithmmakeMoveSourceInstructionSuggestions(pathInstr,raceManifestIndex,raceSourceIndex)StackpathInstrintegerraceManifestIndex,raceSourceIndexSetofintegerpairsHBEdges ndHBEdges(pathInstr)foreachpairp=(index1,index2)2HBEdgess.t.(raceSourceIndex,raceManifestIndex)intersectspANDindex1
PAGE 76

Figure5-6. ViagoFlagThread1notiesThread2whenobjectpublishisreadytobeused. allpairsrepresentingthehappens-beforeedgesonthecounterexamplepath,theFigure 5-5 algorithmcomparesthe(sourcestatement,manifeststatement)pair(raceSourceIndex,raceManifestIndex)withallotherpairsfromthesetofhappens-beforeedges.Ifmovingthesourcestatementbeforeastatementthatisthesourceofahappens-beforeedgeandhasbeenexecutedbeforethesourcestatementcreatesahappens-beforeedgebetweenthesourcestatementandthemanifeststatement,themoveissuggested.Asanexample,considertheprograminFigure 5-6 wheregoFlagandpublisharesharedvariables.Sincepublishisareference,theobjecttowhichitreferscanalsobeaccessedbyboththreads. 2 Thread1createsanobjectatlines1thatiscurrentlyaccessibleonlytoitself.Thenitpublishestheobjectbystoringthereferenceinasharedvariablepublishatlines2.Thestateoftheobjectisupdatedatlines3andthesharedvariablegoFlagissettotrueatlines4declaringthattheobjectdescriptorhasbeensetandcansafelybereadbyotherthreads.Thread2checkswhetherpublishisnotnullat 2 SinceJPFisworkingatthebytecodelevel,accessingaeldpotentiallyinvolvestwobytecodeinstructionsonetogetareferencetotheobjectandonetoaccesstheeld. 76

PAGE 77

Figure5-7. Partofthestatespaceshowingadataracefreepathandapathwithadatarace. linet1and,ifso,spinsuntiltheglobalagbecomestrueatlinet2andreadstheobjectdescriptorinlinet3.WhenthecodeisanalyzedusingstandardJPF,noerrorsarereported.However,thisresultisunsoundsinceanassertionfailureatlinet4islegalaccordingtoJavasemantics.Theprogramcontainsadataracebetweenwritingtheobject'sdescriptorbyThread1andreadingitbyThread2,thusSCsemanticsisnotensuredanditwouldbelegalforliness3ands4tobereordered.JRFcorrectlyreportsadataraceforthisprogram.Figure 5-7 showspartofthestatespaceoftheexample.AslongasJRFdoesnotrunoutofmemoryortheuseroutofpatience,itcanexploreallpossiblepathsinthestatespace.Therstpathdoesnotexhibitadatarace.WhenJRFexecutest2onthe2ndpaththatisexplored,adataraceismanifested(e.g.,assertnoracefails), 77

PAGE 78

thecurrentlyexploredpathisreportedtotheuserasacounterexample,andJRFterminates.Onewayofeliminatingthisparticulardataraceismakingtheglobalag,goFlag,volatile,thuscreatingahappens-beforeedgebetweens4andt2.Anotherwayistomoves4befores2andcreateahappens-beforeedgebetweens4andt2,whichfollowsfromthetransitivepropertyofthehappens-beforerelationship:s4hb!s2,s2hb!t1,ands2hb!t1impliess4hb!t2.Oncethischangeismade,anewdataracebetweenthewriteats3andthereadatt3isexhibited.Thiscanbeeliminatedbymovings3befores2.Atthispointtheexampleisbothcorrect(theassertionwillneverfail)andcontainsnodataraces. Useasynchronizedblock.Usingconsistentlockingisonewayofcreatinghappens-beforeedgesbetweenaccessestoshareddata.UsingsynchronizedblocksisonewayofimplementinglockinginJava.Figure 5-8 ndsallthelocksthatarereleasedafterthesourcestatementandbeforethemanifeststatementandsuggestsprotectingthemanifeststatementwiththeselocksbyreferringtothespecicsourcelinesthatperformthelocking. AlgorithmmakePutInSynchronizedBlockSuggestions(pathInstr,raceSourceIndex,raceManifestIndex)StackpathInstrintegerraceSourceIndex,raceManifestIndexSetofInstructionLocationssyncLoc ;forindexfromsize(pathInstr)toraceSourceIndex+1doifpathInstr(index)isaMONITOREXITinstructionORRETURNinstructionofasynchronizedmethodthenletlocdenotethesourcelineforpathInstr(index)syncLoc syncLoc[flocgforeachsourcelineloc2syncLocdoprintPutinstructionpathInstr(raceManifestIndex)insynchronizedblockasinlinelocFigure5-8. Suggestasynchronizedblock. 78

PAGE 79

Figure5-9. Thread2needtosynchronizeonlocktoaccessdata Figure 5-9 showsanexampleinwhichThread1acquiresalockbeforeaccessingtheshareddatadatawhereasThread2doesnotacquireanylockbeforeaccessingdata.Figure 5-10 showsthecounterexamplepaththatmanifeststhedataraceondata.Ats3Thread1unlockslockbeforethemanifeststatementt1.ThedataracecanbeeliminatedbymakingThread2acquirelockbeforet1. Changeothermemorylocationstovolatileoruseatomicarrays. Figure5-10. Partofthestatespacewiththeunlockinbetweenthesourcestatementandthemanifeststatement 79

PAGE 80

Onewayofcreatingahappens-beforeedgebetweenthesourceandthemanifeststatementistocreateahappens-beforeedgebetweenapairofstatements(s1,s2)thatcomebetweenthesourceandthemanifeststatementintheexecutionsequence,i.e.,(source,...,s1,s2,...,manifest).Forthistoworkweneedsourcehb!s1ands2hb!manifest.Ifourprogrammodicationsestablishs1hb!s2,thenbythetransitivityofthehappens-beforerelation,wewillhavesourcehb!manifest.Ifs1ands2arethewriteandreadofavariablev,respectively,thenchangingvtovolatilecreatesahappenbeforeedgebetweens1ands2.Figure 5-11 showsthealgorithmforcheckingthehappens-beforerelationandthealgorithmforthistypeofsuggestionusingit. AlgorithmisHappensBeforeOrdered(sourceIndex,destIndex,hbEdges):booleanintegersourceIndex,destIndexSetofintegerpairshbEdgesifsourceIndexanddestIndexhavesameexecutingthreadthenreturntrue;if(sourceIndex,destIndex)2hbEdgesthenreturntrue;foreach(s1,s2)2hbEdgesifisHappensBeforeOrdered(sourceIndex,s1,hbEdges)andisHappensBeforeOrdered(s2,destIndex,,hbEdges)thenreturntrue;returnfalse;AlgorithmmakeChangeOtherToVolatileSuggestions(manifestIndex,sourceIndex,hbEdges)integermanifestIndex,sourceIndexSetofintegerpairshbEdgesforeachwriteofvats1betweensourceIndexandmanifestIndexdoifthereexistsareadofvats2betweens1andmanifestIndexandisHappensBeforeOrdered(sourceIndex,manifestIndex,hbEdges[(s1,s2))thenifvisanarrayelementthenprint"Useatomicarray..."elseprint"Makevvolatile"Figure5-11. Suggestchangingadifferentmemorylocationstovolatile 80

PAGE 81

Figure5-12. Partofthestatespacethatchangingdonetovolatilecaneliminatearaceonx ConsiderthemodiedSimpleclasswithtwothreadssharingtwovariables:doneandx.IfJRFisconguredwiththreshold>1,itispossibletondacounterexamplethatshowsadataracemanifestedinstatementt2asshowninFigure 5-12 .Itturnsoutthatbetweenthesourcestatement(s1)andthemanifeststatement(t2),thereisawriteofdonefollowedbyreadofdone.Sinces1ands2areexecutedbyThread1andt1andt2areexecutedbyThread2,changingdonetovolatilecreatesahappens-beforeedgebetweens1andt2andeliminatesthedatarace.Inourexperience,suggestionsfromthisclassareoftenthemostappropriatesolutioninlock-freealgorithmsthatexhibitdataracesonmultiplevariables. Performthesamesynchronizationoperation.JRFkeepstrackoftheacquiringhistorytoallowdeterminationofhowhappens-beforeedgeswerecreatedfornon-racyaccessestoamemorylocation.Formally,wedenetheacquiringhistoryasafunction:AcquireHis:Addr)2(SynchAddr[Threads)Threads.Foramemorylocationm,(v,t)2AcquireHis(m)meansthatatsomepointinthecomputationsofar,threadtperformedanoperationonvthatresultedinmbeingaddedtoh(t).Theactionsbythreadtthatwouldresultin(v,t)beingaddedtoAcquireHis(m),forsomemcouldbe 81

PAGE 82

AlgorithmmakePerformSameAcquireSuggestions(AcquireHis,m)MappingofmemoryLocationtoSetof(ThreadId,agentLoc)AcquireHisMemoryLocationm,locforeach(t,loc)2acquireHistory[m]doiflocisreferencetothreadandm2h(loc)thenprintjointhreadlocbeforemanifestinstruction"elseiflocisafieldandm2h(loc)thenprintreadfieldlocbeforemanifestinstruction"elseifm2h(loc)thenprintlocktheobjectlocbeforemanifestinstruction"Figure5-13. Suggestperforminganacquireoperationthatcanaddthedataracememorylocationtohofthemanifestingthread. readingv,lockingv,orjoiningv,wherevisavolatileeld,lock,orthread,respectively.Incontrasttothesummaryfunctionh,whichonlyappliestoaparticularpath,theAcquireHisiscumulativeandcontainsinformationfromalltheexploredpaths.Figure 5-13 ndsouthowpreviousaccessestothedataracememorylocation(m)havebeenorderedbythehappens-beforerelationandsuggestsperformingthesameacquireoperation.Threepossibleacquireoperationchoicesareread,lock,andjoinaccordingtothetypeofmemorylocation.Ifthememorylocationisaeld,thenitmustbevolatileandthecorrespondingacquiringoperationistoreadit.Ifthememorylocationisalock,theacquireoperationistolockit.Whenthememorylocationisareferencetothread,joiningitservesasanacquire.TheexampleinFigure 5-14 motivatestheuseoftheacquiringhistory.Inexecutionsequence(r1,r2,s1,s2,t1)asshowninFigure 5-15 ,thereisadataracebetweenr1,awriteofxbyThread1,andt1,thereadofxbyThread3.ItshouldbenotedthatThread2alsoperformsareadofxbutitdoesnotresultinadatarace.ThereasonisThread2readsvolatiledonebeforereadingxandthisgeneratesahappens-beforeedgebetweenthewriteofxbyThread1andthereadofxbyThread2.Theacquiringhistory 82

PAGE 83

Figure5-14. AcquiringhistoryofThread2showsThread3cangetracefreeaccessonxbyreadingdone. storesthisinformationandJRFusesittosuggestthatThread3readsvolatiledonebeforereadingxtoeliminatethedatarace.Thesuggestionsgeneratedbytheabovealgorithmsareguaranteedtoeliminatethedataraceonthepathwheretheracewasdiscovered.Theprecisionofthesuggestionsareimprovedbylteringthesetofsuggestionstoonlyincludethosethatappearonallofthepaths.Giventhesetofsuggestions,theprogrammerdeterminesthebestsolutionandimplementsit.JRFshouldthenbererun. Figure5-15. Partofthestatespacewiththeacquiringhistorythatguideshowtoeliminatetherace 83

PAGE 84

5.1.3TheoreticalResultsInthissection,weprovethatmodifyingtheprogramaccordingtothesuggestionsgeneratedbyourtooldoesnotremoveanyhappens-beforeedgesthatexistedbeforethemodication.Specically,Theorem 6 andTheorem 7 showthatthisisthecaseforallexecutionpathsconsideringchanginganon-volatiletovolatileandthataddinganewsynchronizationactionviaputtinginasynchronizedblockorfollowinganacquiringhistory-basedsuggestion,respectively.Theorem 10 showsasimilarresultforthemovesuggestiononlyonthecounter-examplepathundercertainconditions. Theorem6. Changinganon-volatilevariabletoavolatilevariabledoesnotremoveanyoftheexistinghappens-beforeedgesthatresultfromsynchronizationactionsonanyoftheexecutionpaths,butitmayintroduceadditionalhappens-beforeedges. Proof. Accessingnon-volatilevariablesarenotsynchronizationactions,sotheycannotinvolveinthecreationofhappens-beforeedgesresultingfromsynchronizationactions.Onceanon-volatilevariableischangedtoavolatilevariable,thewriteaccesseswillbecomereleasestatements,sothereadaccesseswillbecomeacquirestatementsandmatchingreleaseandacquirepairs,ifany,willcreatehappens-beforeedges. Theorem7. Changingaprogrambyaddingasynchronizationaction(joiningathread,acquiringalock,orreadingavolatilevariable)thatinvolvesanexistingmemorylocationdoesnotremoveanyoftheexistinghappens-beforeedgesthatresultfromsynchroniza-tionactionsortheprogramorderonanyoftheexecutionpaths. Proof. Anexistinghappens-beforeedgethatresultsfromsynchronizationactionscanberemovedonlybychangingthesourceorthedestinationstatementofthehappens-beforeedge.Addingasynchronizationactiondoesnotchangesuchastatement.Also,anexistinghappens-beforeedgethatresultfromprogramorderdoesnotchangeasaresultofaddingasynchronizationactionbecausehappens-before 84

PAGE 85

isatransitiverelationandallsuchexistinghappens-beforeedgeswouldbepreservedduetotransitivity. Theorem8. Movinganinstructionthataccessesanon-volatilevariabledoesnotremoveanyoftheexistinghappens-beforeedgesthatresultfromsynchronizationactionsonanyoftheexecutionpaths. Proof. SamereasoningasinproofofTheorem 6 Lettid(i)denotetheidofthethreadthatexecutedinstructionionagivenpath. Theorem9. Movingadataracesourceinstruction,writexatsteph,beforeaninstructionthatisthesourceofahappens-beforeedge,representedby[d,i]whered
PAGE 86

Figure5-16. ViagoFlagThread1notiesThread2whenobjectpublishisreadytobeused.Thread3canalsonotifyThread2bycheckingaeldoftheobjectpointedtobypublish. d
PAGE 87

AInstantiationofCondition1 BinstantiationofCondition2Figure5-17. InstantiationofConditionsinLemma 9 basedonsampleprograminFigure 5-16 .Happens-beforeedgesformedbysynchronizationactionsareshownbylinesconnectingthematchingreleaseandacquireinstructions. pathinJRFandanalyzeitlaterusingthisstoredinformation.ThisapproachisclosetotheJPFextensionjpf-trace-serverthatprovidesstoring,querying,andanalysisofexecutiontrace.Unfortunately,thisisanon-goingprojectandnotavailabletoJRFat 87

PAGE 88

Figure5-18. TheJRF-Eexecutionmodel:JRFpluginorstandalone thistime.Moreover,JRFneedsmoreinformationinadditiontothecounterexampletracetoanalyzearace,suchasmemorypathelements,acquiringhistory,andh.WestoreallandonlynecessaryinformationataracedetectiontoaleandlaterstandaloneJRF-Eloadsandanalyzesittoproducesuggestions.Inaddition,whenweonlylookforonerace,itiseasiertousetodetectandanalyzethatatonepass.Thisiscongurableinjrf.jpfusingjrf.reporter.standalone. 4 Figure 5-18 showsthestructureofjrf.reporterandJRF-Eextension.EachcounterexampletraceisdirectedtolewithoutanyanalysiswhenJRFdetecteditinstandalonemode.JRF-Ewillanalyzetracesandcategorizethemintodifferentraces,bookkeepingthesuggestionsaccordingly.Additionalinformation,suchashappens-beforeedgesinthecounterexamplepathandacquiringhistory,areavailabletoexplaintheracebetter. 4 TheGUIversionofjrf.reporterwillprovideauser-friendlyenvironmenttoJRF-E. 88

PAGE 89

5.2ModularVericationoftheCorrectnessofRace-FreePreconditionsTheforeigncodeinaconcurrentprogramcomplicatesthevericationoftheracefreedom.Thelibraryprogrammer'sspecicdesigndecisiontoavoidadataraceisdeliveredasaninlinecommentoraseparatedesigndocumenttotheapplicationprogrammerthatmakesuseofit.Naturally,thisinformationisnotalwaysavailabletotheuserduetothefactthatmostforeigncodesaredistributedinanintermediateformatsuchasJavabytecode,ratherthananoriginalsourceformatwithcomments.Underthesecircumstances,itisdangeroustorelysolelyontheapplicationprogrammer'sabilitytouselibrariesconsistentlyforadatarace-freeguarantee.Fromlibraryprogrammers'perspective,theyarealsoinneedofamechanismtospecifydesigndecisionsthatissafelydeliverabletousersandautomaticallycheckableinaconcreteapplicationenvironment.Theannotatedpreconditionsforthelibraryroutineswouldplaythisrole.Thelibraryprogrammerannotatestheconstraintsofeachlibraryroutineasrace-freepreconditionsandtheapplicationprogramanalysiswillautomaticallycheckifthesepreconditionsaresatisedatallinvocationsoftheroutine.Whenallpreconditionsaresatisedatallinvocationsofalibraryroutine,thereisnoneedtoworryaboutadataraceinsidethelibrarycodeusedinanapplication.Inthisway,wecanverifytheracefreedomoftheentiresystemusingmodularracecheckingofindividuallibrariesbyassume-guaranteereasoning.Thepreconditionscanbeeitherapriorlockingrequirementoracurrentthread'shinclusionrequirement.Forexample,assumeUnboundedQueue[ 1 ]andDisBarrier[ 1 ]inFigures 5-19 and 5-20 areforeignlibraries,andanapplicationFairMessage 5 inFigure 5-21 usesthese. 5 ThiscodeisslightlymodiedfromthejunittestdriverforUnboundedQueuein[ 1 ]toincludeacalltoDisBarrier 89

PAGE 90

publicclassUnboundedQueue{privatestaticfinalintEMPTY=java.lang.Integer.MIN_VALUE;publicfinalReentrantLockenqLock=newReentrantLock(),deqLock=newReentrantLock();Nodehead,tail;publicUnboundedQueue(){head=newNode(EMPTY);tail=head;}publicintdeq()throwsEmptyException{intresult;deqLock.lock();try{if(head.next==null)thrownewEmptyException();result=head.next.value;head=head.next;}finally{deqLock.unlock();}returnresult;}publicvoidenq(intx){if(x==EMPTY)thrownewNullPointerException();enqLock.lock();try{Nodee=newNode(x);tail.next=e;tail=e;}finally{enqLock.unlock();}}publicintsize(){//requires:lockenqLockanddeqLockbeforecallinginti=(head==tail?0:1);for(Nodetmp=head.next;tmp!=null&&tmp!=tail;tmp=tmp.next,++i);returni;}protectedclassNode{finalintvalue;volatileNodenext;Node(intx){value=x;next=null;}}}Figure5-19. UnboundedQueuelibrary InUnboundedQueue,thenon-volatilesharedeldsheadandtailareprotectedbyexplicitlockdeqLockandenqLock,respectively.However,thesize()methodbreaksthis 90

PAGE 91

publicclassDisBarrier{finalintsize;intlogSize;finalNode[][]node;finalThreadLocalmySense;finalThreadLocalmyParity;publicDisBarrier(intcapacity){...}publicvoidawait(){...}privateclassNode{volatilejava.util.concurrent.atomic.AtomicBoolean[]flag;Nodepartner;Node(){AtomicBoolean[]flag=newAtomicBoolean[2];flag[0]=newAtomicBoolean(false);flag[1]=newAtomicBoolean(false);this.flag=flag;//addedtoensuresafepublicationofflag[]}}}Figure5-20. DisBarrierlibrary constraintandrequirespriorlockingtobothlocksbeforeitsinvocation.Thisconstraintiscommented,asshowninFigure 5-19 ,buttheapplicationprogrammerwasnotawareofit.TheFairMessageprogramhasadataraceonheadandtail.AlthoughitispossibletouseJRFtodetecttheseraces,itishardtocorrectthisunlessgivenmoreinformation,assuggestedinSection 5.1 .LetusassumethatthelibraryUnboundedQueuehasannotatedpreconditionenqLock2lockset(current thread)^deqLock2lockset(current thread)atsize().OnemoreassumptionisthatthelibrarydeveloperhasalreadyveriedthatthispreconditionwouldruleoutanyraceinUnboundedQueueinanyapplicationenvironment.Giventhesetwoassumptions,itismorestraightforwardtodetectapreconditionviolationinsize()thandataracesonheadandtail.Itisalsopossibletoexcludehead,tail,and 91

PAGE 92

publicclassFairMessage{publicstaticvoidmain(String[]args){(newFairMessage()).run();}UnboundedQueuequeue=newUnboundedQueue();DisBarrierbar=newDisBarrier(NUM_THREAD);staticfinalintNUM_THREAD=2;staticfinalintPER_THREAD=2;privatevoidrun(){assert(queue.size()==0);for(inti=0;i
PAGE 93

5.2.1AnalysisThemodularextensionofJRFrequiresfourstepstoaccomplishthecompositionalverication.Figure 5-22 outlinesthefourstepsofourmodularracedetection.First,thelibrarydevelopershouldprovideappropriatepreconditionstoconveyhis/herdesigndecisionsregardingthedatarace-freeguarantee.Thisincludestheguardingexplicitandimplicitlockandhrequirementattheentrytoeachmethod.Thisannotationprocessisfollowedbyuniversalenvironmentgeneration.Second,theJRFmodularextensiongeneratesauniversalenvironmentofthetargetlibrarywithminimumhappens-beforeorderingsthatsatisfygivenpreconditionsandthemodelcheckingconstraints.ThethirdstepistomodelcheckthelibrarywiththegeneratedcontextusingthehabstractionofJRF.Wealsocheckthepreconditionsateachinvocationofthelibrarymodules.Whenahpreconditionisviolated,weenforceitbyexpandingh.Whenalockorsynchpreconditionisviolated,wesimplyignorethepath.Weuseboundedmodelcheckingwithkasthemultipleofthenumberofconcurrentthreadsandthedepthofeachmethodtoconstrainthesearchspace.Thisstepmayormaynotdiscoverarace.Adjustthepreconditionwhenitcaneliminatethefoundraceormodifythelibrarycodeitselfwhenpreconditionchangecannoteliminatetherace.Inthatcase,itisabuginalibraryratherthanaproblemofconstrainingtheusage.Repeatthethirdstepasmanytimesasneededuntilnoraceisleft.Thesethreestepsareperformedbytheoriginallibrarydeveloperbeforereleasingthelibrarytoothers.Thelaststephappensintheapplicationcontext.Theapplicationprogrammerchecksthepreconditionsateverylibrarymethodinvocationasinthepreviousstep.However,JRFdoesnotmanagehforthelibraryinternalelds.Ifanypreconditionviolationisreported,thisisaninconsistentuseofalibrarymethodanditisuptotheapplicationprogrammertodecidehowtohandleit.Sinceapreconditionviolationdoesnotimplythepresenceofarace,theapplicationprogrammermighttrytouseJRFwithoutamodularextension.Ifthereisnopreconditionviolation,wecanconcludetheapplication 93

PAGE 94

APreconditionannotation BUniversalenvironmentgeneration CVerifypreconditioncorrectness DCheckpreconditionincompositionFigure5-22. ThefourstepsinmodularextensionofJRF 94

PAGE 95

isfreefromadataraceonlibrarymoduleinternalelds.WhenJRFreportsnoraceonnon-volatileeldsdenedoutsidethelibrarymodule,theentiresystemisguaranteedtobesequentiallyconsistentwithoutcheckinglibraryinternalnon-volatiles.WecanexpectbetterscalabilityusingJRFwhenthememoryandtimeoverheadrequiredtomanagelibraryinternalnon-volatilesoutweighthosetocheckmethodpreconditions. 5.2.2AlgorithmThissectiondescribesthealgorithmsusedineachstepinFigure 5-22 Step1.Annotateenvironmentconstraintsandmethodpreconditionstoalibraryprogram.Whendevelopersimplementealibrary,theymadedesigndecisionsonhowtoorderconictingaccesses.Preconditionssavethisinformationandconstraintheillegaluseofthelibrary.Weneedtoboundthenumberofconcurrentthreadsinvokingthemethodsofanobjecttoconstrainthenumberofthreadsinmodelchecking.Thisboundisinevitableinmodelcheckingtolimitthesearchspace.Inaddition,thedepthconstraintisusedtoboundtherecursivecalltoamethod.Thepreconditioncanbethehrequirement,explicitlock,orsynchronizedblock.Methodscanhaveanarbitrarynumberofpreconditions.Thederivingrulesareasfollows: :=:=threads bound:=threads bound:=:=depth bound:=j_:=h(field)jlock(field)jsynch(field)threads boundisdenedforeachobjectincludingstaticobjectrepresentedbytheclassitselfandboundsthenumberofthreadsconcurrentlyrunningmethodsofthis 95

PAGE 96

instance.Inadditiontothisbound,individualmethodhasadepth boundasallowednumberofrecursivecalls.Whenathreadtriestoinvokethemethodandtherearealreadydepth boundstackframesinworkingthreadsstackarea,thiscallviolatesthegivenboundandisrevoked.Inmostcases,theminimumpreconditionrequirementtoinvokealibrarymethodiswhethertheobjectisaccessibleinthecurrentthread;tohavetheobjectitselfinhofthecallingthread.Thisissafepublicationpropertyandcanbespeciedbyh(this)precondition.Apriorlockingrequirementcanbespeciedusingasimilarnotation. @ObjectConstraint(threads_bound=3)publicclassUnboundedQueue{...@MethodConstraint(depth_bound=1)publicUnboundedQueue(){...}@MethodConstraint(depth_bound=2)@Precondition(h={"CURRENT_THREADWITHTHIS"})publicintdeq()throwsEmptyException{...}@MethodConstraint(depth_bound=2)@Precondition(h={"CURRENT_THREADWITHTHIS"})publicvoidenq(intx){...}@MethodConstraint(depth_bound=2)@Precondition(h={"CURRENT_THREADWITHTHIS"},lock={"enqLock","deqLock"})publicintsize(){...}...}@ObjectConstraint(threads_bound=3)publicclassDisBarrier{...@MethodConstraint(depth_bound=1)publicDisBarrier(intcapacity){...}@MethodConstraint(depth_bound=2)@Precondition(h={"CURRENT_THREADWITHTHIS"})publicvoidawait(){...}...}Figure5-23. UnboundedQueuelibrarywithpreconditionannotation 96

PAGE 97

Forexample,whenamethodneedstolockpriorLockeld,whichisaninstanceofjava.util.concurrent.locks.ReentrantLock,beforeinvokingit,ithaslock(priorLock)asaprecondition.Whentherequirementistobeinvokedinsideasynchronizedblockontheobjectitself,thepreconditionwouldbesynch(this).Preconditionsarecompositionalasgiveninthederivingruleas.Forinstance,thepreviouspriorlockexamplewouldhaveapreconditionsh(this)andlock(priorLock)sincesafepublicationisthenecessarypreconditionatalltimes.Itisalsopossibletohavechoicesinaprecondition.WhenamethodeitherrequirestolockpriorLockbeforecallingitortosynchronizeonthesameeldpriorLock,thepreconditioncanbespeciedaslock(priorLock)_synch(priorLock).Thepreconditionisregardedassatisedwheneitheroneissatised.Figure 5-23 showstheconstraintsandpreconditionsannotatedtothelibraries.BothUnboundedQueueandDisBarrierhaveannotatedwithaconstraintof3concurrentthreadsand2callstoeachmethods.Notethataconstructorhaveadepth boundonesinceitiscalledonlyoncepereachinstance. Step2.generateuniversalenvironmentwithminimumhappens-beforeorder.Thoughthepreconditionsareannotatedbytheoriginalprogrammerwhoknowsbetterthananyoneelseaboutthelibrary,itisverydifculttobesurethattheconditionsarestrongenoughtoguaranteeracefreedom.Toprovethatthosepreconditionsarecorrectchoicesusingmodelcheckingwhereweneedaclosedsystem,wegeneratethemostgeneralexecutionenvironmentmeetingtheenvironmentconstraints.ThealgorithmtogeneratethisuniversalenvironmentisinFigure 5-24 .Wespawnonethreadforachoiceofconstructorstoinstantiateanobjectandassignittoanon-volatilereference.Thenon-volatilereferenceisusedtoaddnoadditionalhappens-beforeorderinauniversalenvironment.Notethatthesafepublicationrequirementwillbefullledatthevericationstep.Wealsospawnthreads boundthreadsandfordepth boundtimes,maketheminvokeoneofthemethods.All 97

PAGE 98

AlgorithmgenerateUniversalEnvironment(object annotation,method annotations)declareanon-volatilereferencetoanobject;choose*aconstructordefinedinmethod annotationsspawnathreadwhichinvokestheconstructorwithallparametersassymbolicandassignnewinstancetothenon-volatilereference;forifrom1toobject annotation.threads boundspawnthreadwhichforjfrom1tomax(method annotation.depth bound)choose*non-internalmethodmdefinedinmethod annotationsforeachlockofmlockonthefieldforeachsynchofmsynchronizeonthefieldforeachlockin_ofmchoose*lockonthefieldorskipforeachsynchin_ofmchoose*synchronizeonthefieldorskipinvokesmwithallparametersassymbolicforeachlockofmunlockthefieldforeachlockin_ofmunlockonthefieldorskipaccordinglyFigure5-24. Generateuniversalenvironmentwithallpossiblecombinationsofmethods.(choose*generatedifferentstatesinmodelchecking) parametersaresymbolicvariablessoastopreventspecifyinganinputdirectly.Inthismanner,therewillbe(1+threads bound)spawnedthreadsinadditiontothemainintheuniversalenvironment.ThegenerateduniversalenvironmentforUnboundedQueueisshowninFigure 5-25 .Whenmorethanonelocksarespeciedaspreconditions,thereisachanceofadeadlockinaconcreteenvironmentbutthegenerateduniversalenvironmentisfreefromthisproblemsincethealgorithmordersthelocksaccordingtothespeciedorder.Whenaconcreteenvironmenthappentohaveadeadlock,itwillbedetectedbyJPFduringStep3. 98

PAGE 99

publicclassUnboundedQueueVerify{UnboundedQueueobj;@Symbolic("true")intsym0;@Symbolic("true")intsym1;publicstaticvoidmain(String[]args){newUnboundedQueueVerify().doTest();}voiddoTest(){for(inti=0;i<1;++i)newGroup1Thread().start();for(inti=0;i<3;++i)newGroup2Thread().start();}classGroup1ThreadextendsThread{publicvoidrun(){for(inti=0;i<1;++i){intoption=gov.nasa.jpf.jvm.Verify.getInt(1,1);if(option==1){obj=newUnboundedQueue();}}}}classGroup2ThreadextendsThread{publicvoidrun(){for(inti=0;i<2;++i){while(obj==null);intoption=gov.nasa.jpf.jvm.Verify.getInt(1,3);if(option==1){obj.deqLock.lock();obj.enqLock.lock();try{obj.size();}finally{obj.enqLock.unlock();obj.deqLock.unlock();}}elseif(option==2){if(i==0)obj.enq(sym0);elseif(i==1)obj.enq(sym1);}elseif(option==3){try{obj.deq();}catch(jrfm.UnboundedQueue.EmptyExceptione){}}}}}}Figure5-25. GenerateduniversalenvironmentforUnboundedQueue 99

PAGE 100

Step3.Verifytheracefreedomforexecutionswithallpreconditionssatised.TheannotatedlibraryfromStep1andtheuniversalenvironmentfromStep2willbefedtoJRFmodularextensiontocheckbothpreconditionsandraces.Whenapreconditionisviolatedattheentryofanymethodduringmodelchecking,wewillignorethepathunlessthepreconditionishtype.WehandlethehpreconditionviolationdifferentlyfromotherssincetheuniversalenvironmentdidnottakeitintoaccountasshowninFigure 5-24 .Weupdatethehofthecurrentthreadathtypepreconditionviolationbyaddingthefieldandallmemorieslastupdatedbythesamethreadasfieldwhenithadhappenedbeforethelastupdateoffielditself.JRFwillcheckaraceasbefore.Whennoraceisfoundattheend,wecanconcludethatthepreconditionsarecorrectandthelibraryisveriedtoguaranteethattheinternaleldswillbefreefromadataraceforanyconcreteenvironmentwithallpreconditionssatised.Ifaraceisfoundatthisstep,thepreconditionsarenotstrongenoughforarace-freeguarantee.ThisstepshouldberepeatedeitherbyaddingmorepreconditionsormodifyingthelibrarycodeusingtheadvicesuggestedinSection 5.1 untilnoraceisleftasfollows: ======================================================JRFM-VerifyModuleresults======================================================noracefoundandpreconditionsin"UnboundedQueue"areverified... Step4.Checkpreconditionsoflibrarymethodsinanapplicationprogram.Thelibraryisreadyatthispointandanapplicationprogrammerwilluseitinaconcreteenvironment.JRFcanbeusedtopreciselydetectaraceatthisstep,but,asdescribedearlier,theannotatedpreconditionwouldbetterguidetheapplicationprogrammertoaconsistentuseofalibrary.Themodularextensionwillruleouttheinternaleldsfromh.Instead,atallmethodinvocationsofthelibrary,thepreconditionsarechecked.Whenallpreconditionsaresatisedatallinvocationsoflibrarymethods,thelibrarymodulesareconsistentlyusedasrecommendedandguaranteedtobefreefromarace 100

PAGE 101

======================================================JRFM-ComposeModuleresults======================================================preconditionviolation#0in"jrfm.UnboundedQueue.size()"thelockpreconditionofmethod(size)"enqLock,deqLock"isviolated.at"System.out.println("queuesize="+queue.size());"in"jrfm.FairMessage.run(FairMessage.java:23)"======================================================preconditionviolation#1in"jrfm.UnboundedQueue.size()"thelockpreconditionofmethod(size)"enqLock,deqLock"isviolated.at"assert(queue.size()==0);"in"jrfm.FairMessage.run(FairMessage.java:17)"...Figure5-26. PreconditionviolationsinFairMessagedetectedbyJRFmodularextension insidethelibrarymodule.Whenapreconditionisviolatedandmorethanonethreadhasexecutedthemethodsofanobject, 6 itmeansthereisapossibilityofarace.Thepresenceofapreconditionviolationdoesnotautomaticallymeanthepresenceofarace,though.Ratheritmeansthelibraryisusedinaninconsistentwaythatviolatesthedeveloper'sdesign.Itistheapplicationprogrammer'schoiceabouthowtohandleapreconditionviolation.HemaychangethecodetosatisfythepreconditionoruseJRFwithoutmodularextensiontopreciselydetectadatarace.WhenJRFitselfdetectsnorace,theapplicationisfreefromaraceinspiteofthepreconditionviolation.Figure 5-26 showsthepreconditionviolationinFairMessagedetectedbyJRFmodularextension. 5.2.3TheoreticalResultsInthissection,wejustifythesoundnessofmodularracecheckingwithrespecttotherace-freeguarantee.TheexperimentalresultisinSection 7.1 .Denition:EquivalentExecutionsw.r.t.asetofactionsTwowell-formedexecutions,E1=hA1,P1,po!1,so!1,W1,V1iandE2=hA2,P2,po!2,so!2,W2,V2i,areequivalentw.r.t.asetofactionsAwhichisasubsetofA1andA2ifA1jA=A2jA,po!1jA=po!2jA,andso!1jA=so!2jA. 6 Whenonlyonethreadisaccessingthemethodsofanobjectincludingtheconstructors,theobjectisthreadlocalandfree-fromaraceregardlessofthepreconditionviolation. 101

PAGE 102

SupposeLisalibrarywithinternalnon-volatileseldsLthatisonlyaccessibleinthemethodsofL.LetusassumeauniversalenvironmentUwithboundingconstraintBandanyarbitraryconcreteexecutioncontextCofLsatisfyingtheboundingconstraintBwithEU=hAU,PU,po!U,so!U,WU,VUiandEC=hAC,PC,po!C,so!C,WC,VCidenotearbitrarysequentiallyconsistentexecutionsofUandC,respectively.WhenallsuchECsatisfythepreconditionsPofLinboundBandallEUhavenoraceoneldsL,itisguaranteedthatallEChasnoraceoneldsLingivenboundB.Thefollowingtwolemmaswilljustifythis. Lemma13. ForanarbitraryEC,thereisalwaysanequivalentexecutionEU,whichisnotignoredatstep3,w.r.t.AL,whereALiseitherthesetofactionsinLortheactionssatisfyingthelockandsynchpreconditionsofL. Proof. Theproofisbycontradiction.LetusassumethatanexecutionEC1hasnoequivalentexecutioninthesetofEUw.r.t.AL,whichisnotignored.GiventhatEC1satisesallpreconditionsandconstraintsofL,wecanconstructanexecutionEU1correspondingtoapathoftheenvironmentUbychoosingthesameactionsasEC1ateachtransition.Moreover,giventhattheparametersarerepresentedsymbolically,wecanchoosethesameWandVforEU1asEC1,i.e.,EU1=hAC1,PU,po!C1,so!C1,WC1,VC1iSuchatransitionchoiceisalwaysavailableinauniversalenvironmentbecauseECsatisestheboundingconstraintBandthereareenoughtransitionchoicesinUtocoveralldifferentinterleavingswithinB.Thevericationruleatstep3guaranteesthatthispathisnotignoredsincethispathsatisesalllockandsynchpreconditionsandsanddepth boundconstraints.ThiscontradictstheassumptionthatthereisnosuchEU1inU. Lemma 14 showsthatthehthatincludesthenon-volatileeldsinalibraryisminimalinuniversalenvironment.Inotherwords,whenanon-volatileeldisinthehofacurrentthreadatsomeexecutionstepintheuniversalenvironment,itisguaranteed 102

PAGE 103

tobeinthehofacurrentthreadattheequivalentexecutionstepinanyequivalentconcreteenvironment. Lemma14. Supposeh)]TJ /F5 7.97 Tf 6.58 0 Td[(1istheinverseofhwhereh)]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)isthesetofmemorylocationsv2(SynchAddr[Threads)suchthatx2h(v).WhenhUandhCdenotehfortwoequivalentexecutions,EUandECw.r.t.AL,respectively,8x2eldsL,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)holdsforallprexesofEUjAL. Proof. TheproofisbyinductiononthelengthoftheprexofEUjAL.Basis.Wehavelength0prexofEUjAL.SincenoactioninALhappens,8x2eldsL,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)=(hC))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)=.InductiveStep.Assume8x2eldsL,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)holdsfor(EUjAL)n.Whenthe(n+1)thactionisanactionsatisfyingthelockandsynchpreconditionsofL,the(n+1)thactioniseitherreleaseoracquireand(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)ispreservedbythehupdateruleinFigure 4-1 .Whenthe(n+1)thactioniseitherrelease,acquire,invalidate,orhirrelevantactions,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)ispreservedbythehupdateruleinFigure 4-1 .Otherwise,the(n+1)thactioniseitheraninstantiationorapublicationoftheLobjectoraninvocationofamethodinL. WhentheactionisaninstantiationoftheLobject,itwilladdtheinstantiatingthreadtoboth(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)and(hC))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)forallxineldsL.Thispreserves(hU))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x). Whentheactionisapublication,giventhatthereferenceinUisdenedasnon-volatile,thispublicationwillnotchange(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x).IfthereferenceinCisavolatile,thiswilladdthecurrentthreadinto(hC))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x).Otherwise,(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)remainsthesame.Thispreserves(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x). Atamethodinvocation,whenhpreconditionisviolatedinU,thiswilladdtheviolatedeld,supposetheeldisf,andallothermemorylocationsYwrittenpriortothatbythesamethreadttothehofcurrentthreadcurrentT.SincethememoriesinYwerelastupdatedbyt,Yh(t).(hU))]TJ /F5 7.97 Tf 6.58 0 Td[(1(f)jn+1 (hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(f)jn[fcurrentTgand(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)jn+1 (hU))]TJ /F5 7.97 Tf 6.58 0 Td[(1(y)jn[fcurrentTgforally2Y.WhenhpreconditionisnotviolatedinU,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)jn+1=(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)jn. 103

PAGE 104

TheassumptionguaranteesthatthehpreconditionissatisedinCandfcurrentTg(hU))]TJ /F5 7.97 Tf 6.58 0 Td[(1(f)jn+1.IftiscurrentT,fcurrentTg(hU))]TJ /F5 7.97 Tf 6.58 0 Td[(1(y)jn+1forally2Y.Whent6=currentT,fhasbeenaddedtocurrentTafterthelastwriteoffthroughacquireofvbycurrentTprecededbyreleaseofvbyt.Atthetimeofreleaseofvbyt,ffg[Yhadbeenaddedtoh(v)sinceffg[Yhadbeeninh(t).FollowingacquireofvbycurrentTwouldhaveaddedffg[Ytoh(currentT).ThisconcludesthatfcurrentTg(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(f)jn+1andfcurrentTg(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(y)jn+1forallyinY.Inbothcases,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)ispreserved. Theorem 11 justiesthepreconditionsthatwereveriedinUcanguaranteethedataracefreedomoninternaleldsofLinC. Theorem11. Whenasetofpreconditionsareveriedtobecorrectinuniversalenviron-mentsUwithboundingconstraintB,anyarbitraryconcreteenvironmentCwithinBisguaranteedtobefreefromadataraceonanyinternaleldsofLifallpreconditionsaresatisedinallsequentiallyconsistentexecutionsECofC. Proof. Theprooffollowsimmediatelyfromlemma 14 .Since8x2eldsL,(hU))]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)(hC))]TJ /F5 7.97 Tf 6.58 0 Td[(1(x)holdsforallprexesofEUjALandx2hU(t)at(EUjAL)nguaranteesx2hC(t)at(ECjAL)n. SinceCisprovedtohavenoraceoneldsL,Lcanbetrusted,asdiscussedinSection 4.3.6 ,andsafelyexcludedfromhwithouthurtingthesoundnessofJRFinC. 104

PAGE 105

CHAPTER6CASESTUDIESOurextensiontoJPF 1 makeitpossibletosoundlyanalyzecomplex,highlyconcurrentdatastructuresthatdonotnecessarilyuselocking,oruseamixtureoflockingandothersynchronizationidioms.InorderforJPFtobesoundwhenappliedtotheseprograms,theprogrammustbefreeofdataraces.Lock-freeprogramstypicallyusevolatilevariablesalongwithinstancesofclassesinthejava.util.concurrent.atomicpackagetocreatethenecessaryhappens-beforeedgestopreventdataraces.ThischapterwillcoverourexperienceofapplyingJRFinvariousprogramsfromconcurrentdatastructurestoalargeapplicationframework.Testingwasperformedonai386/8processorwith32GBramusingLinux/2.6.32-24-genericOS,JPFversion5,andSunMicrosystemsInc./1.6.0 16Javawith2GBJVMheapmemory. 6.1SimpleExamplesWestartedourexperimentwithsimpleexamplesthatareproventoberacefree.AftertherstsuccessfulvericationofaracefreedomusingJRF,weintentionallyseededracestothemandcheckedifJRFcoulddetectthem.JRFwastestedforvarioustypesofraces;movedashareddataaccessoutsideasynchronizedblock,omittedlocking,inconsistentuseofalock,andchangedavolatilevariabletoanon-volatile.Theprogramsusedinthisphasewasfrom[ 65 ],andoriginallyimplementedinZing[ 66 ].ThetranslationfromZingtoJavawasdonemanuallyfortotalsevenprograms,andtheyhadcodesizesfrom43linesto514lines.Briefexplanationabouteachapplicationisasfollows, 1 ThestandardJPFdistributionincludestworacedetectingtools:RaceDetectorandPreciseRaceDetector.Neithercorrectlydealswithprogramswheredataracesareavoidedbyusingvolatilevariableorclassesfromthejava.util.concurrent.atomicpackage. 105

PAGE 106

IndependentWorker1:themainthreadcreatestwothreadseachofwhichallocatesitsownlistandtraversesitwithoutanysharingofadata. IndependentWorker2:themainthreadcreatestwothreadsandtwolistsandletthemtraversesoneofthelists. FileSystem:astaticarrayeldissharedamongthreads,buteachthreadaccessesdisjointelementsofthearray.Eventhoughthecontainerarrayissharedamongdifferentthreads,individualdataelementofitisthread-local. Indexer:multithreadsindexeachelementoftheglobalsharedarraytable[]usingahashvalue.thesameastheFileSystem,individualdataelementoftheglobalstaticarrayisthread-local. HaltException:producerandconsumerthreadssharealock-protectedbuffer. IOManager:themainthreadcreatethreemorethreadstomodelanI/ORequestPacket(IRP);NtReadFiletoexecuteIRP,IopCloseFiletocancelit,andModelProcessCompletionPortEntriestopost-processeithersuccessfulorcanceledIRP.Theysynchronizewitheachotherusingbothlock-protectedqueuesandvolatilesharedvariables. BlinkTree:thisisaB-linktreeimplementationwithcongurablenumberoflisttesterthreads.Eachnodeisprotectedbyaspeciclock,andthetreebalancingmakesthisprotectinglocktodynamicallychange. 6.2Herily-ShavitExamplesToevaluatetheusefulnessofJRFanditscounterexampleanalysisandtoempiricallyexplorethebehaviorofthesearchheuristics,weusedJRFtoanalyzeanextensivesetofconcurrentobjectsdescribedinthearecentlypublishedandhighlyregardedtextbookbyHerlihyandShavit[ 1 ].Javaimplementationswereobtainedfromthebook'swebsiteandallthecodeinthissectionisderivedfromtheseprograms.OurtestsuiteincludessixbarrierimplementationsfromChapter17,eighthashsetsfromChapter13,velistimplementationsfromChapter9,fourmutexcodesfromChapter2,sixlock-freequeue-basedspinlockimplementationsfromChapter7,fourmonitorandblockingsynchronizationtestsfromChapter8,ninedifferentqueueimplementationsfromChapter10,nineschedulingandworkdistributionalgorithmsfromChapter16,andfourpriorityqueuesfromChapter 106

PAGE 107

15.[ 1 ]Inaddition,onceJRFfoundarace,wealsocorrecteditandreranJRFtondanyremainingraces.Wefound19racesfromatotal68testprogramsand14otherapplicationassertionviolations,suchasadeadlock.ThepurposeofpublishingthisworkisinnowaymeanttodisparagetheveryimpressiveandvaluableworkofHerlihyandShavit.Rather,analyzingtheirexamplesallowsustodemonstratethatJRFisapracticalandusefultoolforndingdataraces,andthattaskisworthsupportingwithatoolbecauseitisdifcult.TheJavaimplementationswereprovidedintheformofaclassimplementingthealgorithmwithatestclassimplementingJUnittestcases.ToavoidhavingtheJPFmodelchecktheJUnitframework,wemodiedthetestclassestoinvokethetestmethodsdirectlyandsignicantlyreducedthenumberofthreadsandnumberofiterationsforthetests.Inafewcases,suchastheTOLockclassgiveninFigure 6-5 ,wefounditdesirabletomodifythesourcecodeoftheclassbeingtestedtoeliminatecallstoSystem.nanoTime.Inalmostallcases,dataraceswerefoundveryquickly,withinafewseconds.Oncetherewerenomoredataraces,runninganexampletocompletioncouldtakeaprohibitiveamountoftime,withnoindicationofhowmuchlongeritwouldbeanother30seconds,severalhours,orseveraldays.Inmostcases,theexecutionterminated,orwascanceledbeforeitranoutofmemory.StandardJPFwithoutourextensions(usingthedefaultproperties)showedthesamelackofscalability.SomeexamplesusingstandardJPFranforseveraldayswithoutterminating.Asaresultitwasdifculttoobtainmeaningfuldataforvalidcomparisons,butthedatawedohaveindicatethatourextensionscurrentlyincreasetheexecutiontimebyafactoroftwotothree.Generally,dataracesarefoundquicklybecauseinmostprogramstheytendtooccuronmultiplepaths.Toillustratehowtheprocessofusingourtoolcanwork,wedescribeaselectedsetofcasestudiesindetailincludingseveralhashsetimplementationsfromChapter13andtwolock-freequeue-basedspinlockimplementationsfromChapter7. 107

PAGE 108

6.2.1Examples 6.2.1.1ConcurrentHashSetsWewilllookatthreedifferentclosed-addresshashsetimplementationsthatuselockinginthreedifferentways.CoarseHashSetusesasinglelock,StripedHashSetusesaxed-sizearrayoflocks,andRefinableHashSetusesaresizablearrayoflocks.AllthreearebasedontheabstractBaseHashSetclass.AnextractofthisclassisgiveninFigure 6-1 .Theimplementationsofthecontainsandremovemethodsare publicabstractclassBaseHashSet{protectedList[]table;protectedintsize;publicBaseHashSet(intcapacity){size=0;table=(List[])newList[capacity];for(inti=0;i();}}publicbooleancontains(Tx){...}publicbooleanadd(Tx){booleanresult=false;acquire(x);try{intmyBucket=Math.abs(x.hashCode()%table.length);result=table[myBucket].add(x);size=result?size+1:size;}finally{release(x);}if(policy())resize();returnresult;}publicbooleanremove(Tx){...}publicabstractvoidacquire(Tx);publicabstractvoidrelease(Tx);publicabstractvoidresize();publicabstractbooleanpolicy();}Figure6-1. Abstractclassusedinthreelock-basedclosed-addresshashsets 108

PAGE 109

publicclassCoarseHashSetextendsBaseHashSet{finalLocklock;CoarseHashSet(intcapacity){super(capacity);lock=newReentrantLock();}/*doublethesetsize*/publicvoidresize(){intoldCapacity=table.length;lock.lock();..resizetableifoldCapacity==table.length...}finally{lock.unlock();}}publicfinalvoidacquire(Tx){lock.lock();}publicvoidrelease(Tx){lock.unlock();}publicbooleanpolicy(){returnsize/table.length>4;}}Figure6-2. HashSetimplementationthatusesasinglelock omittedforbrevity,buttheyusealockingschemeandhashcalculationsimilartotheaddmethod.Theacquire,release,resize,andpolicymethodsareabstractandmustbeimplementedinasubclass.Theacquireandreleasemethodsimplementthelockingmechanism,resizeresizestheHashSet,andpolicydeterminesifaresizeshouldbedone.Thesimplestsubclassusesasinglelock(Figure 6-2 ).Theacquireandreleasemethodssimplydelegatetothelockandunlockmethodsonasinglejava.util.concurrent.ReentrantLockinstance.AnalyzingthisprogramwithJRFrevealeddataracesonsizeandtable.lengthduetotheunguardedaccessestothesevariablesinthepolicymethodandtherstlineoftheresizemethodthatconictandracewithaccessesintheaddandremovemethods.Whiletherewereseveraloptionsforcorrectingtheproblem,wediditbymodifyingtheresizemethodsothatitrecheckspolicyinsteadofoldCapacity==table.length,eliminatingtheneedfortheunguardedassignmenttooldCapacity.Inaddition,thebodyofpolicywaslocked. 109

PAGE 110

publicclassStripedHashSetextendsBaseHashSet{finalLock[]locks;publicStripedHashSet(intcapacity){super(capacity);locks=newLock[capacity];for(intj=0;j
PAGE 111

presenthere.TheywereeliminatedsimilarlytowhatwasdonewiththeCoarseHashSetexample.RerunningJPFrevealedanewproblem.TheincrementofsizeintheaddmethoddenedinBaseHashSetisnolongeratomic,sincethreadsholdingdifferentlocksfordifferentbucketswillnotexcludeeachother.AgoodsolutiontothatproblemandonethatdoesnotrequireintroducingadditionallockingistochangethetypeofthesizevariablefrominttoAtomicInteger,sincetheAtomicIntegerclassprovidesanatomicgetAndIncrementmethodthathasthesamememorysemanticsasreadingandwritingavolatilevariable.Thisyieldsaracefreeprogram.Wealsonoticed,andJRFallowedustoverify,thatwhensizeisanAtomicInteger,lockingthebodyofpolicyisnolongerrequired.Theprogramnowusesamixtureofsynchronizationidioms.Thethirdlockbasedhashset,calledRefinableHashSet,supportsaresizablearrayoflocks.TheoriginalversionsufferedfromthesameproblemwiththesizevariableasStripedHashSet.OnceweusedtherevisedversionofBaseHashSet,wheresizeisanAtomicInteger,theprogramwasfreefromdataraces.Sincetheseclassesuselocksforsynchronization,someofthedescribedanalysiscouldhavebeendoneusingalock-setbasedracedetectiontool.However,thesewouldnothavebeenabletodealwiththeAtomicIntegertypeorallowustocondentlyremovethelockingfromthepolicymethod.Severalotherhashsetclasses,includingtheconcurrentopen-addresscuckoohashsetimplementations[ 67 ]CuckooHashSet,StripedCuckooHashSet,andRefinableCockooHashSet,weresuccessfullyanalyzedwithoutrevealinganydataracesorothererrors. 2 2 LockFreeHashSet,alock-freeclosed-addresshashmapimplementationbasedonrecursivesplit-orderedlist[ 63 ]astheunderlyingdatastructure,displayedanassertionfailurethatturnedouttobeanerrorinthetestdriver. 111

PAGE 112

6.2.1.2Queue-basedSpinLocksIntruemultiprocessors,itoftenmakessensetoimplementlocksbyhavingthreadsspin,i.e.,repeatedlycheckacondition,ratherthanblockandcauseacontextswitch.However,simplyhavingallwaitingthreadsreadasinglememorylocationisnotscalableinthemostcommonarchitecturesduetothedemandsitplacesonthesharedsystembus.Queue-basedlocksaredesignedsothatonlyonethreadspinsonanymemorylocation,andtheselocationsarenotlikelytobeinthesamecacheline.Inaddition,theselocksprovidefairnessandavoidthecriticalsectionunderutilizationinherenttoback-offschemes.AdetaileddiscussionoftheseissuescanbefoundinChapter7ofHerlihyandShavit'sbook[ 1 ].Fromthepointofviewofthememorymodel,lockalgorithmsmustsatisfytworequirements.Thealgorithmsthemselvesneedtobedata-racefree.Inaddition,theyneedtoprovideahappen-beforeedgebetweenareleaseofthelockandasubsequentacquiresothattheselockswillprovidethesamememorymodelrelatedsemanticsasJava'sintrinsiclocks.Thissectionwilllookattwoqueue-basedlockingalgorithms.TherstalgorithmiscalledaCLHlockafteritsoriginators,Craig,Hagersten,andLandin[ 68 69 ]andisshowninFigure 6-4 .Variabletailalwaysreferstothelastnodeinthequeue,oradummynodeifthequeueisempty.Eachnodecontainsaeldlocked.Waitingnodesformavirtuallinkedlist,andspinonthelockedeldoftheirpredecessor.Threadlocalvariablescannotbeinvolvedinadatarace(althoughasharedobjectreferredtobyaThreadlocalcouldbe),andtailisimplementedusinganAtomicReference,soitwillnotbeinvolvedinadatarace,andeveryaccess,suchasthegetAndSetcallinthelockmethodwillcreatehappen-beforeedges.AnalyzingthisclasswithJRF,however,revealsthatinsomeexecutions,thereisadataraceonthelockedeld.Thisracecanbeeliminatedbymakinglockedvolatile. 112

PAGE 113

publicclassCLHLockimplementsLock{AtomicReferencetail;//mostrecentlockholderThreadLocalmyNode,myPred;//thread-localvariablespublicCLHLock(){tail=newAtomicReference(newQNode());//initializethread-localvariablesmyNode=newThreadLocal(){protectedQNodeinitialValue(){returnnewQNode();}};myPred=newThreadLocal(){protectedQNodeinitialValue(){returnnull;}};}publicvoidlock(){QNodeqnode=myNode.get();//usemynodeqnode.locked=true;//announcestart//Makemethenewtail,andfindmypredecessorQNodepred=tail.getAndSet(qnode);myPred.set(pred);//rememberpredecessorwhile(pred.locked){}//spin}publicvoidunlock(){QNodeqnode=myNode.get();//usemynodeqnode.locked=false;//announcefinishmyNode.set(myPred.get());//reusepredecessor}...staticclassQNode{//Queuenodeinnerclasspublicbooleanlocked=false;}}Figure6-4. CLHlock ThenextalgorithmisavariationoftheCLHlockthatallowswaitingnodestotimeoutandabort.ThisversionalsoneedsitsQNodelockedeldtobevolatile.Sinceawaitingthreadispartofavirtuallistofnodes,itcannotsimplyabort.Instead,when 113

PAGE 114

publicclassTOLockimplementsLock{staticQNodeAVAILABLE=newQNode();AtomicReferencetail;ThreadLocalmyNode;publicTOLock(){tail=newAtomicReference(null);myNode=newThreadLocal(){..sameinitializationasCLHLock..}publicbooleantryLock(longtime,TimeUnitunit)throwsInterruptedException{longstartTime=System.nanoTime();longpatience=TimeUnit.NANOSECONDS.convert(time,unit);QNodeqnode=newQNode();myNode.set(qnode);//rememberforunlockqnode.pred=null;QNodepred=tail.getAndSet(qnode);if(pred==null||pred.pred==AVAILABLE){returntrue;//lockwasfree;justreturn}while(System.nanoTime()-startTime
PAGE 115

predecessor.TherelevantpartoftheoriginalcodeforthisclassisshowninFigure 6-5 .ThisclassisoneofthefewwherewechangedthecodeoftheclassitselftofacilitatetheanalysiswithJPF.InordertoavoidcalculationsinvolvingSystem.nanoTime(),theguardoftheloopwhile(System.nanoTime()-startTime
PAGE 116

Table6-1. Experimentalresultsfor[ 1 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. examplesearchjpf-h-stateslengthtimememvolatiles,non-statespruned(sec)(MB)locks,threadsvolatiles DisBarrierdfs1080/2810888136300heuristic740/03845436300bfs12970/1273611734736306 StaticTreeBarrierdfs580/865895839322heuristic880/04676539324bfs104828/2282>27*12457198839344 CoarseHashSetdfs582/463486542341heuristic1390/2341810660351bfs4950/50>9**10730058361 LockFreeHashSetdfs550/05567044380heuristic1810/046129943364bfs185935/181>12**30960541346 RenableHashSetdfs992/76561810049376heuristic1980/0562012949378bfs5619142/1516>15*10036199981382 StripedHashSetdfs582/463496346351heuristic840/02287560362bfs4810/52>9**13331854371 TCuckooHashSetdfs427724/2372>240*1230549155677heuristic10930/38>234*697938146676bfs33894/203>10*98361999120525 LazyList***dfs640/06497554353heuristic62260/1158822660101672379 OptimisticListdfs550/05587854349heuristic27200/2855589848762365bfs7155417/1340>16*17221199246325 Bakerydfs330/03343938423heuristic690/03455638410bfs6900/17229428638378 Filterdfs540/205554538300heuristic470/02533538310bfs3770/18194217838312 Petersondfs240/02423335307heuristic480/02323635300bfs3220/18172313235302 LockFreeQueuedfs250/02523229236heuristic320/01823429233bfs530/01533929233 ALockdfs220/02223335245heuristic480/02523535245bfs49722/28203819135245 CLHLock***dfs190/01913433250heuristic400/02123533250 MCSLockdfs170/01712933253heuristic430/02023233253bfs2040/2615149033253 CorrectedMCSLock***dfs170/01713135251heuristic210/01112734245 DEQueuedfs260/02623329245heuristic280/01823329241bfs340/01033329241 UnboundedQueuedfs350/03554546333heuristic1390/1291410846347bfs107811/3621233354862336 *JPFoutofmemoryandJRFendedgracefullywithresultreport**algorithmfoundapplicationassertionviolationbeforearaceisfound***bfsalgorithmranoutofJavaheapspaceandJPFfailedtoreporttheresult 116

PAGE 117

Table6-2. ExecutionresultofPreciseRaceDetectorforHerlihy-Shavittestsfrom 6-1 examplestatelengthtimememresult DisBarrierTest9999122sameraceStaticTreeBarrierTest4545122sameraceCoarseHashSetTest5833122sameraceLockFreeHashSetTest11263122applicationassertionerrorRenableHashSetTest9855123sameraceStripedHashSetTest5833122sameraceTCuckooHashSetTest505318228applicationassertionerrorLazyListTest22465127sameraceOptimisticListTest24155127sameraceBakeryTest28051123differentracefound*FilterTest9280330noraceLockFreeQueueTest7025122sameracePetersonTest12922122sameraceALockTest7720231noraceCLHLockTest9620122sameraceMCSLockTest5319122sameraceCorrectedMCSLockTest5319116falseraceonvolatileeldDEQueueTest16020122sameraceUnboundedQueueTest9635117samerace *Theresultsinboldfaceisfailuretodetectracesonvolatilearraywithnon-volatileelementaccesses. Table 6-4 summarizessuggestionsbyJRFforeachracefound.Mostraceswerecausedbynotusingvolatileoratomicarray.Thisisbecausethetargetcodeswereselectedfromconcurrentdatastructuresthatimplementlock-freealgorithms.Theprogrammerswhoimplementtheconcurrentlibrariesaretheexpertswhounderstandconcurrentprogrammingrulesandthreadinterleaving,andtendtomakefewconcurrencymistakescausedbymissingsynchronizationorinconsistentlocking. 6.3AminoConcurrentBasicBlocksTheAminoopensourcesoftwareproject[ 2 ]implementsconcurrentbuildingblocksinhighlyefcientandscalablecodes,andaimestosupportasetoflockfreecollectionclasses,parallelpatterns,andschedulingalgorithms.Aminoconcurrentbuildingblocksareimplementedintwolanguages,JavaandC/C++.TheJavaversionisimplementedasorg.aminopackagewith6subpackages,includingutility,algforparallelgraphalgorithm,searchingandsortingalgorithms,dsfordatastructures,suchasparalleltree,parallelgraph,andlockfreecollections,mcasformulti-CAS,patternformaster-worker 117

PAGE 118

Table6-3. Experimentalresultsfor[ 1 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. examplethresholdjpf-h-statesmaxJRFtimeJRF-Etimememvolatiles,non-racesstatesprunedlength(sec)(sec)(MB)locks,threadsvolatiles DisBarrier11080/28108807036300103310/184265275157363041001015641/36752681505726336304 StaticTreeBarrier*1580/86581005739322 CoarseHashSet**1582/4634807142341101698/2143837215769374 LockFreeHashSet1550/05570684438010792/2857107774438610011226/14661182610744406 RenableHashSet1992/7656192105493761023615/2956166232236338310031941/52962953624276409 StripedHashSet**1582/46349063463511016915/2353841515973384 LazyList1640/0641008654353101162/566620811954355100356133/793771088124168362 OptimisticList1550/055807454349101108/925922812354351100734911/26935959724412102407 Bakery1330/03331403842310360/2344163938430100710/365191817238492 Filter1540/205552473830010720/227063255383041001970/254811830410238320 LockFreeQueue1250/02510292923610340/8292033292381002540/454381157629238 Peterson1240/02420343530710290/4262233353191001410/210399207335337 ALock1220/02220313524510250/2232134352451001610/22636997235245 CLHLock1190/01910323325010240/42120333325510018382/528311838033255 MCSLock1170/017103233253105912/8226404933253100151161/671301839433279 CorrectedMCSLock*1170/017102935251 DEQueue1260/02620322924510500/26353138292451002740/4643713118029246 UnboundedQueue1350/03550554633310400/43765524633510017521/27545315213046348 *JRF-Eranoutofmemorybeforedetectingthresholdraces**JRF-Eendedbyapplicationassertionerrorbeforedetectingthresholdraces pattern,andschedulerforschedulingthreads.Standarddistributiondownloadedfrom[ 2 ]includes20junittestsand10examplesusingtheconcurrentbuildingblocks. 118

PAGE 119

Table6-4. JRF-EsuggestionsfromcounterexampleandacquiringhistoryanalysisforHerlihy-Shavitexampleswitharace testsuiteraceeldanalysisofclass DisBarrierag[]ofNodeuseatomicarrayforag[]log[]ofDisBarrieruseatomicarrayforag[]orlog[] StaticTreeBarriersenseofStaticTreeBarriemakesensevolatilelog[]ofStaticTreeBarrieuseatomicarrayforlog[],makesensevolatile CoarseHashSetsizeofCoarseHashSetmakesizevolatile,lockthelock LockFreeHashSetbucket[]ofLockFreeHashSetuseatomicarrayforbucket[]headofBucketListmakenextvolatile,useatomicarrayforbucket[]nextofNodemakenextorheadofBucketListvolatile,useatomicarrayforbucket[] RenableHashSetsizeofRenableHashSetmakesizevolatile,lockthelocks[] StripedHashSetsizeofStripedHashSetmakesizevolatile,lockthelocks[] TCuckooHashSettable[]ofTCuckooHashSetuseatomicpackagefortable[],lockthelocks[][] LazyListTestnextofNodemakenextvolatilemarkedofNodemakemarkedornextvolatilekeyofNodemakekey,marked,ornextvolatile,lockofNodemakelockornextvolatile OptimisticListnextofEntrymakenextvolatile,lockthelockkeyofEntrymakekeyornextvolatile,lockthelock Bakerylabel[]ofBakeryuseatomicarrayforlabel[]andag[],makecounterofLabelvolatilecounterofLabelmakecounteroridofLabelvolatile,useatomicarrayforlabel[]ag[]ofBakeryuseatomicarrayforag[] Filterlevel[]ofFilteruseatomicarrayforvictim[]orlevel[]victim[]ofFilteruseatomicarrayforvictim[]counterofFiltermakecountervolatile,useatomicarrayforvictim[]orlevel[] PetersonvictimofPetersonmakevictimorcountervolatile,useatomicarrayforag[]ag[]ofPetersonuseatomicarrayforag[]counterofPetersonmakecountervolatile,useatomicarrayforag[] LockFreeQueuetailofLockFreeQueuemaketailvolatileheadofLockFreeQueuemaketailvolatileitems[]ofLockFreeQueueuseatomicarrayforitems[],maketailvolatile ALockag[]ofALockuseatomicarrayforag[],makevalueofEntryvolatilecounterofALockmakecounterorvalueofEntryvolatile,useatomicarrayforag[] CLHLocklockedofQNodemakelockedvolatilecounterofCLHLockmakelockedofQNodeorcountervolatile MCSLocknextofQNodemakelocked,next,orcounterofQNodevolatilecounterofMCSLockmakelockedofQNodeorcountervolatilelockedofQNodemakelockedofQNodeornextofQNodevolatile CorrectedMCSLockcounterofMCSLockmakelockedofQNodeorcountervolatile DEQueuebottomofDEQueuemakebottomvolatilemap[]ofDeQueueuseatomicarrayformap[] UnboundedQueuenextofNodemakenextvolatile,locktheenqLockvalueofNodemakevalueornextvolatile,locktheenqLock *boldentryindicatesthemostappropriatesolution. 6.3.1ExamplesTheorg.amino.ds.lockfree.LockFreeDequeimplementsadoublyendedqueueusingaCompare-And-Setbasedlockfreealgorithm.Thelockfreealgorithmuses 119

PAGE 120

publicclassLockFreeDequeextendsAbstractQueueimplementsDeque{.../***Iteratordefinitionofdeque.ThisiteratorisNOTthread-safe**/privateclassDeqIteratorimplementsIterator{privateDequeNodecursor=anchor.get().left;publicbooleanhasNext(){returncursor!=null;}publicEnext(){if(cursor==null)thrownewNoSuchElementException();Eresult=cursor.data;cursor=cursor.right.get();returnresult;}publicvoidremove(){thrownewUnsupportedOperationException();}}...}Figure6-6. DataRaceoncursorinLockFreeDeque.DeqIteratorsincetheimplementationisnotthread-safe anAnchorTypeobjectwithleftandrightpointers,astatuseld,andanumberofelementsindeque.Oneanchorisdenedforeachdeque,andisimmutable.Itusesjava.util.atimic.AtomicIntegerFieldUpdatertochangethestatuseld.Inaddition,ananchorforadequeisupdatedusingjava.util.atomic.AtomicReferencewrapping.ThereisaraceinthisclassimplementingiteratorwhentestedforIteratorTest.testSameIteraterUsedByMT,whichisatestforaniteratorsharedbymultithreads.ThecodeforiteratorisgiveninFigure 6-6 .Aminolockfreecomponentsdonotsupportiterator.remove()toprovidebetterperformanceforfrequentlyusedoperationssuchasaddFirst,addLast,pollFirst,andpollLast. 120

PAGE 121

Theorg.amino.ds.lockfree.LockFreeQueueisalockfreeFIFOqueue.Italsousestwopointers,prevandnext,insteadofastandardsinglylinkedlist,storesheadandtailofthequeueinavolatileeld,andisupdatedusingjava.util.atomic.AtomicReferenceFieldUpdater.QueueItrforthisclassisalsonotthreadsafeandhasaraceonitsnextNodeeldwhenthesameiteratorisusedbymultithreads.ThecodeisgiveninFigure 6-7 .org.amino.ds.lockfree.LockFreeBlockQueueimplementstheblockingversionofthelockfreeFIFOqueue.ThisusesthesamedatastructuretoimplementtheFIFOqueue,andblockswhenanimmediatepollorputisunavailable.LockFreeBlockQueuedoesnotsupportaniterator.org.amino.ds.lockfree.LockFreePriorityQueueisanotherimplementationofconcurrentqueueimplementation.Itusesanarraylistofjava.util.atomic.AtomicMarkableReferencetorepresentnextlevels.OneiteratorcannotbesharedamongmultiplethreadssincePQueueIterator.cursorisnotsharedproperly,asinFigure 6-6 org.amino.ds.lockfree.EBDeque,aneliminationbackoffdeque,implementsthesamealgorithmasorg.amino.ds.lockfree.LockFreeDequetogetherwithconsideringeliminationbackoff.Thisisforhighcontentioncases,andusesorg.amino.utility.EliminationArrayinitsimplementation.Itsmainideaistoreducethenumberofmodicationtothemaindatastructurethroughmaintainingtwoarraystostoretwotypesofoperations:addandremove.Inadditiontolockfreequeue,lockfreelist,lockfreeorderedlist,lockfreeset,lockfreevector,andlockfreedictionaryareprovidedintheorg.amino.da.lockfreepackage.Whentestedwithjpf-corewithoutjpf-concurrent,bothdirectedandundirectedGraphTestndaraceonjava.util.concurrent.ConcurrentHashMap$KeySetfromthecodejava.util.concurrent.ConcurrentHashMap.Thereasonisjpf-coredoesnot 121

PAGE 122

publicclassLockFreeQueueextendsAbstractQueueimplementsQueue,Serializable{...privateclassQueueItrimplementsIterator{privateNodenextNode;privateNodelastRet;privateEnextItem;QueueItr(){lastRet=null;advance();}privateEadvance(){lastRet=nextNode;Ex=nextItem;/*valueofnextnode*///ppointtonextvalidnodeNodep=(nextNode==null)?first():nextNode.getNext();while(true){//reachtheendif(p==null){nextNode=nextItem=null;returnx;}Eitem=p.value;if(item!=null){/*pisavalidnode*/nextNode=p;nextItem=item;returnx;}else/*skipovernulls*/p=p.getNext();}}publicbooleanhasNext(){returnnextNode!=null;}publicEnext(){if(nextNode==null)thrownewNoSuchElementException();returnadvance();}publicvoidremove(){thrownewUnsupportedOperationException();}}...}Figure6-7. DataRaceonnextNodeinLockFreeQueue.QueueItrsincetheimplementationisnotthread-safe 122

PAGE 123

implementjava.util.concurrent.ConcurrentHashMapproperly.OrderedListExamplefromtheexamplespackagealsondsabenignraceinjava.lang.Stringclass. 6.3.2ExperimentalResults 6.3.2.1RaceDetectioninJRFTable 6-5 comparesJRFresultswithdifferentsearchstrategiesfortheexamplesfrom[ 2 ]withraces.TheresultshowsthatsuggestedheuristicsperformsbetterthanDFSexcludingQueueTest.BFSreturnstheoptimalcounterexamplepathwhenthereisaraceandJRFheuristicsearchperformsclosetothisoptimalcaseinTable 6-5 .ItisalsonoticeablethatheuristicsearchuniquelyndsaraceinLockFreeSetwhileDFSandBFSranoutofmemoryafterverylongcomputation.Table 6-6 summarizestheresultsofJPFPreciseRaceDetectorfortheexamplesinTable 6-5 .JRFheuristicsearchgivesshortercounterexamplepathsalleightcases. Table6-5. Experimentalresultsfor[ 2 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. examplesearchjpf-h-stateslengthtimememvolatiles,non-statespruned(sec)(MB)locks,threadsvolatiles IteratorTestdfs260/0263213891564(EBDeque)heuristic250/01354591562bfs1210/0115215990558 IteratorTestdfs260/02686558458(LockFreeDeque)heuristic250/01344758456bfs1210/0113412657452 IteratorTestdfs380/038107152446(LockFreeList)heuristic500/02155551440bfs4360/01911928151440 IteratorTestdfs380/038106552458(LockFreeOrderedList)heuristic500/02165151452bfs4360/01912327951452 IteratorTestdfs330/0337020692960(LockFreePriorityQueue)heuristic310/016188792958bfs2140/01438438391954 IteratorTestdfs610/061209957518(LockFreeQueue)heuristic330/01745155512bfs8300/01336843254508 IteratorTestdfs530/053*36685204311382297(LockFreeSet)heuristic490/02135542211412301bfs3470/0>17*19895199911382297 QueueTestdfs140/01433544314heuristic280/01534144314bfs550/31064843320 *JPFoutofmemoryandJRFendedgracefullywithresultreport 123

PAGE 124

Table6-6. ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-5 examplestatelengthtimememresult IteratorTest(EBDeque)4021117sameraceIteratorTest(LockFreeDeque)4021123sameraceIteratorTest(LockFreeList)5233222sameraceIteratorTest(LockFreeOrderedList)5233222sameraceIteratorTest(LockFreePriorityQueue)5627128sameraceIteratorTest(LockFreeQueue)28054328sameraceIteratorTest(LockFreeSet)818793347differentraceQueueTest7720116samerace Specically,JRFsignicantlyoutperformsPreciseRaceDetectorinexampleLockFreeSetwithcounterexamplelength21comparingto793. 6.3.2.2RaceAnalysisinJRF-EInthissection,theexperimentperformedwithcounterexampleanalysisturnedon.Table 6-7 aretheexperimentalresultswiththresholdracesas1,10,and100excludingLockFreeSetsinceitranoutofmemorywithDFSstrategy.Table 6-8 summarizessuggestionsbyJRFforeachracefound. Table6-7. Experimentalresultsfor[ 2 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. examplethresholdjpf-h-statesmaxJRFtimeJRF-Etimememvolatiles,non-racesstatesprunedlength(sec)(sec)(MB)locks,threadsvolatiles IteratorTest1260/02633013991564(EBDeque)10360/142758917291571100470/2228811422891582 IteratorTest1260/026806558458(LockFreeDeque)10360/14271478058465100470/2228191210558476 IteratorTest1380/0381006652446(LockFreeList)10480/14391459252453100590/22401889552464 IteratorTest1380/0381006552458(LockFreeOrderedList)10480/14391559952465100590/22401999852476 IteratorTest1330/033892222796995(LockFreePriorityQueue)10470/203413715924893978100640/383518725226092978 IteratorTest1610/06120010257518(LockFreeQueue)10760/671275121635331001260/14671765823863533 QueueTest1140/01420394431410310/162270564533510015054/3024743613047355 124

PAGE 125

Table6-8. JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 2 ]exampleswitharace testsuiteraceeldanalysisofclass Iterator(EBDeque)cursorofDeqIteratormakecursorvolatile Iterator(LockFreeDeque)cursorofDeqIteratormakecursorvolatile Iterator(LockFreeList)nextofListItrmakenextvolatilecurofListItrmakecur,nextvolatileprevofListItrmakecur,prevvolatile Iterator(LockFreeOrderedList)nextofListItrmakenextvolatilecurofListItrmakecur,nextvolatileprevofListItrmakecur,prevvolatile Iterator(LockFreePriorityQueue)cursorofPQueueIteratormakecursorvolatile Iterator(LockFreeQueue)nextNodeofQueueItrmakenextNode,nextItemvolatilenextItemofQueueItrmakenextItemvolatilelastRefofQueueItrmakelastRef,nextNodevolatile Iterator(LockFreeSet)nextofCompositeStateHoldmakenextvolatilecurofCompositeStateHoldmakenext,curvolatileprevofCompositeStateHoldmakeprevvolatile QueueprevofNodemakeprevvolatile *boldentryindicatesthemostappropriatesolution. 6.4GoogleConcurrentPackageThenexttargetofourexperimentwastwoconcurrentdatastructurepackagesfrom[ 3 64 ]. 6.4.1ConcurrentAdtExperimentFramework[ 64 ]isaframeworkforanexperimentoflock-freeandwait-freeconcurrentdatastructures.Themaingoalofthisframeworkisaperformancecomparisonofeachconcurrentabstractdatatype;ConcurrentStack,IlConcurrentStack,IlConcurrentStack v2,andConcurrentLinkedQueue.ThetestframeworkiscomposedofmultipleProducersandaConsumer.ConcurrentStackisasimplesinglylinkedlistwithstacktopsavedusinganjava.util.concurrent.atomic.AtomicReferencewrapper.ConcurrnetLinkedQueueisansinglylinkedlistwithtwojava.util.concurrent.atomic.AtomicReferencewrappedpointers,headandtail.OneimportantdifferencebetweenitandConcurrentStackistheuseofjava.util.concurrent.atomic.AtomicReferencewrappedpointerforthelinkedlist.Unlikeastackwithonlyonepointer,concurrentaccessestoaqueueupdatestwopointersatthesametime,andthisrequiresatomicupdateofbothpointersandtheirinternallinks.IlConcurrentStackandIlConcurrentStack v2 125

PAGE 126

usesthestatuseld.Thestatusofanodeiseitherstableorunstable,andanunstablenodeneedshelpfromthefollowingstackoperations.Thestatuseldhasajava.util.concurrent.atomic.AtomicReferencewrapperandformsahappens-beforerelationamongaccesses.Noneofthefourtestsfoundanyrace,butdidndoneNullPointerExceptioninConcurrnetLinkedQueue. 6.4.2GoogleConcurrentDataStructuresWorkshopBarrierThesecondstructureisfrom[ 3 ],abarrierimplementationfromaconcurrentprogrammingworkshopbyRoeiRavivandJonathanSeroussi.Ithasimplementationsof12barriers:simpleBarrier,senseBarrier,treeBarrier,staticTreeBarrier,lockBarrier,cyclicBarrier,senseBarrierWithWait,linearSenseBarrier,linearSenseBarrierVolatile,linearSenseBarrierVolatileWithBackoff,binaryStaticTreeBarrier,andsplittedSenseBarrier. publicclassSyncCounterThreadextendsThread{privateCounterWithBarrier_syncedCounter;publicSyncCounterThread(CounterWithBarriersyncedCounter){_syncedCounter=syncedCounter;}@Overridepublicvoidrun(){intcachedVal;intcountTo=Main.getCountTo();for(intiterNum=0;iterNum
PAGE 127

Unfortunately,8of12barrierimplementationshadaraceonaccessingsharedvalueofcounter.AsshowninFigure 6-8 ,abarriercannotbeusedtoorderthistypeofRWsequencessinceaWWraceexists.ThesimplestbarrierimplementationinFigure 6-9 alsohasaproblem.Sincethesizeins3isresetfornextuserightafterthebarrierconditionsatised(s2),anotherthreadthatisspinats5maymissit.Asanexample,twothreadsT1andT2callawait()toreachabarrier,andanexecutionsequence(T1:s1)(T2:s1,s2,s3)(T1:s4,s5)willmakeT2spinats5forevereventhoughthebarrierisalreadyreached.AnotherproblemJRFfoundistheuseofvolatilearray.Anarraydeclaredwithavolatilekeyworddoesnotguaranteevolatilesemanticsinaccessingitselements.Instead,java.util.concurrent.atomicpackagereplacesvolatilearrayelementsemantics.LinearSenseBarrierVolatileextendBaseLinearSenseBarrierVolatile,whichhasaeld threadDoneArraydeclaredtobeprivatevolatileboolean threadDoneArray[].InLinearSenseBarrierthesameeldisdeclaredevenwithoutvolatile,andalsohasarace. publicclassSimpleBarrierimplementsBarrier{AtomicIntegercount;intsize;publicSimpleBarrier(intn){this.count=newAtomicInteger(n);this.size=n;}@Overridepublicvoidawait(){intposition=count.getAndDecrement();/*s1*/if(position==1){//IfI'mlast.../*s2*/count.set(size);//resetfornextuse/*s3*/}else{//otherwisespin/*s4*/while(count.get()!=0){}/*s5*/}}}Figure6-9. SimpleBarrierusinganAtomicIntegerisnotworkingproperlysincethebarrierisbroken 127

PAGE 128

6.4.3ExperimentalResults 6.4.3.1RaceDetectioninJRFTheexperimentalresultsof[ 3 ]tenbarriertestsaregiveninTable 6-9 .Inthiscase,heuristicsearchndssevenshortercounterexamplesoutoftenexamplesandallBFSsearchesarefailedtondarace.Table 6-10 summarizestheresultsofJPFPreciseRaceDetectorfortheexamplesinTable 6-9 .Insevenoutoftenexamples,PreciseRaceDetectorndsfalseracesonvolatileeldsandmissestworacesonvolatilearrayelementaccesses. Table6-9. Experimentalresultsfor[ 3 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. examplesearchjpf-h-stateslengthtimememvolatiles,non-statespruned(sec)(MB)locks,threadsvolatiles LinearSenseBarrierVolatiledfs390/283964543372heuristic1010/03966143368bfs9890192/461>17*2195198539356 LinearSenseBarrierdfs400/284064944400heuristic1040/04065844412bfs9682260/758>15*6360198340376 SimpleBarrierdfs361/332954538335heuristic530/02133634337bfs11939252/655>16*6502198636341 SenseBarrierdfs480/384876541362heuristic890/03755541376bfs10606222/451>15*2073199037346 SenseBarrierWithWait**dfs680/386885241360heuristic950/03955741376 TreeBarrierdfs710/6271128144386heuristic1280/04986444396bfs9896252/615>15*4932197639371 LockBarrierdfs340/03475258406heuristic1010/035107754407bfs9374219/447>15*7342199349400 CyclicBarrierdfs250/02523337334heuristic680/02434433336bfs13250253/610>16*7285197834344 BinaryStaticTreeBarrierdfs600/3860107847391heuristic1350/052107047403bfs9328213/551>15*6341198142378 SplittedSenseBarrierdfs710/6071149149383heuristic1210/050109349396bfs8450205/514>14*6260198443371 *JPFoutofmemoryandJRFendedgracefullywithresultreport**bfsalgorithmranoutofJavaheapspaceandJPFfailedtoreporttheresult 128

PAGE 129

Table6-10. ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-9 examplestatelengthtimememresult LinearSenseBarrierVolatile4242122falseraceonvolatileeld*LinearSenseBarrier4444122falseraceonvolatileeld*SimpleBarrier4636122sameraceSenseBarrier2828123falseraceonvolatileeldSenseBarrierWithWait3838122falseraceonvolatileeldTreeBarrier4444123falseraceonvolatileeldLockBarrier2626122sameraceCyclicBarrier2020122sameraceBinaryStaticTreeBarrier4343122falseraceonvolatileeldSplittedSenseBarrier4545123falseraceonvolatileeld *Theresultsinboldfacearefailuretodetectaraceonvolatilearraywithnon-volatileelementaccesses 6.4.3.2RaceAnalysisinJRF-EInthissection,theexperimentperformedwithcounterexampleanalysisturnedon.Table 6-11 aretheexperimentalresultswiththresholdracesas1,10,and100excluding Table6-11. Experimentalresultsfor[ 3 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. examplethresholdjpf-h-statesmaxJRFtimeJRF-Etimememvolatiles,non-racesstatesprunedlength(sec)(sec)(MB)locks,threadsvolatiles LinearSenseBarrierVolatileTest1390/28396614337210530/385485664337210027761/66167485815143394 LinearSenseBarrier1400/28406484440010560/385793674442410030363/71172603717644514 SimpleBarrier*1361/332954438335 SenseBarrier1480/38487574136210575/694993584136210040989/100950713220641380 SenseBarrierWithWaitTest1680/386885441360107911/107691066941360100522195/151970836318041378 TreeBarrier1710/627112804438610808/102721427444386100181250/103272582421544386 LockBarrier1340/0346655840610400/63776775840610014077/36543606717662407 CyclicBarrier1250/0252253733410410/143131583733410043051/84536362016238346 BinaryStaticTreeBarrier1600/386010664739110870/82781658847391100929283/2601782134826352483 SplittedSenseBarrier1710/6071147249383108014/118721888249383100175258/104672737317149383 *JRF-EranoutofJavaheapspacebeforethresholdracesandJPFfailedtoreporttheresult 129

PAGE 130

Table6-12. JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 3 ]exampleswitharace testsuiteraceeldanalysisofclass LinearSenseBarrierVolatile valueofCounterWithBarriermake valuevolatile,useatomicarrayfor threadDoneArray[] threadDoneArray[]ofuseatomicarrayfor threadDoneArray[],BaseLinearSenseBarrierVolatilemake valueofCounterWithBarrier,valueofEntryvolatile LinearSenseBarrier valueofCounterWithBarriermake valuevolatile,useatomicarrayfor threadDoneArray[] threadDoneArray[]ofuseatomicarrayfor threadDoneArray[],BaseLinearSenseBarriermake valueofCounterWithBarrier,valueofEntryvolatile SimpleBarrier valueofCountermake valuevolatile SenseBarrier valueofCountermake valuevolatile SenseBarrierWithWait valueofCountermake valuevolatile TreeBarrier valueofCountermake valuevolatile LockBarrier valueofCountermake valuevolatile CyclicBarrier valueofCountermake valuevolatile BinaryStaticTreeBarrier valueofCountermake valuevolatile SplittedSenseBarrier valueofCountermake valuevolatile *boldentryindicatesthemostappropriatesolution. LockFreeSetsinceitranoutofmemorywithDFSstrategy.Table 6-12 summarizessuggestionsbyJRFforeachracefound. 6.5JavaGrandeForumTestSuiteJavaGrandeForumbenchmarksuitefrom[ 4 ]iswidelyusedintheexperimentofseveralliteratureresearchprojects.Sincethebenchmarkoriginallyaimstomeasuretheperformanceofgrandeapplicationswhichrequireintensivememoryorcomputingpower,itcannotbeusedinamodelcheckingtechniqueasitis.TomodelcheckthoseprogramsincludedinJavaGrandeForumbenchmarkmultithreadedbenchmarkversion1.0[ 4 ],wedramaticallyreducedthenumberofthreads,numberofiterations,andsizeofdatainourexperiment.Firstsectionof[ 4 ]testsuitesarefortestinglowleveloperationsuchasfork-joinofthreadsandsynchronizationusingabarrierorasynchronizedblock.JGFForkJoinBenchteststheperformanceofthreadforkingandjoining,andeachthreadhasnointeractions,sonoracefound.JGFBarrierBenchaimstomeasuretheperformanceofbarriersynchronization.ItusedalockfreeTournamentBarriertosynchronizethepartialcomputationthreads.Volatilearray,causesaraceonitselementaccess,isusedasIsDone[]ofwhicheachelementmarksthearrivalofthecorrespondingthread. 130

PAGE 131

classCounterClass{intshared_cont;}classSyncObjectRunnerimplementsRunnable{intid,size;CounterClasscont;publicSyncObjectRunner(intid,CounterClasscont,intsize){this.id=id;this.cont=cont;this.size=size;}publicvoidrun(){for(inti=0;i
PAGE 132

publicclassUniversal{...privatestaticbooleanUNIVERSAL_DEBUG;...publicUniversal(){super();this.DEBUG=true;this.UNIVERSAL_DEBUG=true;this.prompt="Universal>";}...}Figure6-11. MonteCarlobenchmarkcodefragmentwitharaceonstaticeldUNIVERSAL DEBUG distributedintothreadsandnosharingofdatainvolved.ThelastbenchmarkisJGF-SparseMatmult,computesamatrixmultiplicationstoredinasparsematrixstoredincompressedrowformat.Thecomputationisdistributeduniformlyintomultithreads,andresultvectorsharingisavoidedusingsortedarray.Finalsectionhaslargescaleapplications.JGFMolDynBenchisadynamicsimulationofmolecularparticles.Eachthreadinthisbenchmarkgenerateparticles,calculatevelocitiesofmoves,moveparticlesandupdateforces,andthencomputefullpotentialenergyafterbarriertosynchronizeallcomputations.SincetheoriginalprogramwaswritteninFortran,thebenchmarkisnotprogrammedwithanobjectorientedconcept.AsJGFLUFactBench,volatilearrayIsDone[]inTournamentBarrierisinvolvedinarace.JGFMonteCarloBenchusesaMonteCarloalgorithminanancialsimulation.IthasabaseclasscalledUniversalusedasacentralizedrepositoryforallthefunctionalitiesofmontecarloalgorithm,butunfortunately,sharedastaticeldUNIVERSAL DEBUGamongthreadswithoutanyprotectionasgiveninFigure 6-11 3 3 Thisisanexampleofbenignracesincethesharingoftheeldisredundantupdatethatdoesnotaffectthesemanticsoftheprogram. 132

PAGE 133

6.5.1ExperimentalResults 6.5.1.1RaceDetectioninJRFTheexperimentalresultsof[ 4 ]tenbarriertestsaregiveninTable 6-13 .Inthiscase,heuristicsearchndsfourshortercounterexamplesoutofsixexamples.AswecandeducefromtheTable 6-13 ,themorestatesarevisitedbeforearacedetection,theshortercounterexamplesarefoundbyheuristicsearchcomparingtoDFS.Table 6-14 summarizestheresultsofJPFPreciseRaceDetectorfortheexamplesinTable 6-13 .Infouroutofsixexamples,PreciseRaceDetectormissesaraceonavolatilearrayelement. 6.5.1.2RaceAnalysisinJRF-EInthissection,theexperimentperformedwithcounterexampleanalysisturnedon.Table 6-15 aretheexperimentalresultswiththresholdracesas1,10,and100excludingLockFreeSetsinceitranoutofmemorywithDFSstrategy.Table 6-16 summarizessuggestionsbyJRFforeachracefound. Table6-13. Experimentalresultsfor[ 4 ]examplescontainingaracefoundbyJRF.ResultswithoutthreadlocaloptimizationforDFS,heuristicsearch,andBFSaregiven. examplesearchjpf-h-stateslengthtimememvolatiles,non-statespruned(sec)(MB)locks,threadsvolatiles BarrierBenchdfs870/1887137945474heuristic1390/072118745474bfs15610/03232842843446 SyncBenchdfs1110/101031710548507heuristic400/02222737251bfs3370/1131712637252 lufactdfs340/183443935274heuristic340/01923435274bfs920/01665635274 sordfs150/61585242690heuristic680/03587442689bfs250/0764242689 moldyn*dfs28210/1874282159871937521heuristic18960/095027961737575 montecarlodfs860/0863617464919heuristic1780/0902314464998bfs1510/20173011952609 *bfsalgorithmranoutofJavaheapspaceandJPFfailedtoreporttheresult 133

PAGE 134

Table6-14. ExecutionresultofPreciseRaceDetectorforexamplesfromTable 6-13 examplestatelengthtimememresult BarrierBench4885302851none*SyncBench109102122sameracelufact35830334none*sor22750334none*moldyn14989480073221139none*montecarlo88687335samerace *Theresultsinboldfacearefailuretodetectaraceonvolatilearraywithnon-volatileelementaccesses Table6-15. Experimentalresultsfor[ 4 ]examplescontainingaracefoundbyJRF-E.ResultswithoutthreadlocaloptimizationusingDFSandthresholdas1,10,100aregiven. examplethresholdjpf-h-statesmaxJRFtimeJRF-Etimememvolatiles,non-racesstatesprunedlength(sec)(sec)(MB)locks,threadsvolatiles BarrierBench1870/188713010345474101280/741022721374553510016670/46501026372228445556 SyncBench11110/1010318011548507101310/261132501284850710057294/1500113229527348507 lufact1340/183440393527410720/30727149352761003310/152331296415635300 sor1150/61580574269010770/66774313130426911001590/1421597828822242691 moldyn128210/1874282163832723375211028610/188828616632957253752110031360/20643136697362175437521 montecarlo1860/0863821816491910910/4885019190649841001661/77125239202297661103 Table6-16. JRF-Esuggestionsfromcounterexampleandacquiringhistoryanalysisfor[ 4 ]exampleswitharace testsuiteraceeldanalysisofclass BarrierBenchIsDone[]ofTournamentBarrieruseatomicarrayforIsDone[] SynchBenchshared countofCounterClassmakeshared countvolatile lufactIsDone[]ofTournamentBarrieruseatomicarrayforIsDone[] sorsync[]ofSORuseatomicarrayforsync[]A[][]ofRandomMatrixuseatomicarrayforA[][],sync[] moldynIsDone[]ofTournamentBarrieruseatomicarrayforIsDone[] montecarloUNIVERSAL DEBUGofUniversalmakeUNIVERSAL DEBUGvolatile* boldentryindicatesthemostappropriatesolution.*Thisisabenignraceinredundantwrites. 134

PAGE 135

CHAPTER7DISCUSSION 7.1PerformanceTocomparetheperformanceofvariouscongurationofJRF,weusedselectedsetofexamplesdescribedinChapter 6 .TheyareDisBarrier,Filter,Peterson,andDEQueuefromSection 6.2 ,LockFreeList,LockFreePriorityQueue,andLockFreeSetfromSection 6.3 ,LinearSenseVolatileBarrierandBinaryStaticTreeBarrierfromSection 6.4 ,andsynch,sor,andmoldynfromSection 6.5 7.1.1ThreadlocalOptimizationFigures 7-1 comparestheperformancewithandwithouttheoptimizationofthreadlocaloptimizationdescribedinSection 4.3.8 .WetestedfortheexampleswithracesusingDFSstrategyandconguredJRFtoexplorethefullsearchspacetondallpresentingracesratherthantostopattherstracedetection.Theresultsshowthatsignicantimprovementsinbothexecutiontimeandmemoryrequirementscanbeobtainedwheninformationaboutwhichvariablesarethreadlocalisavailable.Fortheoptimizedversion,informationaboutthreadlocalitywasobtainedbyaprerunofJPF,withtheonlymodicationtothestandarddistributionbeingtosavethesharinginformation.Inseveralexamples,theprerunranoutofmemorybeforeterminating.SinceprogrammerswilltypicallywanttouseJPFtoeliminatenormalprogrammingerrorsbeforeusingJRFtodetectdataraces,thisisareasonableapproach.Otherpossibilitieswillbeexploredinfuturework.Withthreadlocaloptimization,JRFsuccessfullyfoundallracesinsomeofexamplespreviouslyfailedtogetresultduetotheoutofmemory:Iterator LockFreeList,Iterator LockFreePriorityQueue,LinearSenseVolatileBarrier,synch,andsor. 7.1.2HeuristicSearchInordertoevaluatetheeffectivenessoftheknowledgeofdataracesbuiltintotheheuristics,wealsocomparedtheheuristicstrategywitharandomchoicestrategy.The 135

PAGE 136

Aexecutiontime BmemoryconsumptionFigure7-1. ComparisonoftheJRFresultswith/withoutthreadlocaloptimizationwhenDFSstrategyisconguredtondallracesbyexploringthefullsearchspace randomchoicestrategywillalsofallbetweenDFSandBFS.Figure 7-2 comparesthememory,time,andpathlengthofheuristicwitharandomchoice.Inthecaseofrandom,thelineshowstheaveragevalueobtainedin100trialswiththestandarddeviation 136

PAGE 137

Figure7-2. Comparisonoftheheuristicwitharandomsearchstrategy 137

PAGE 138

indicated.Theseresultsdonotshowasignicantbenetfortheheuristic,butmostofthetimetheheuristicissomewhatbetterthanrandominthequalityoftheresult(shortercounterexamplepathlength)withnoticeablesavinginmemoryandtimesused. Athenumberofbestsolutions(solutionswiththeshortestcounterexampleforthetestcase)pereachheuristiccongifuration BthesumofallcounterexamplelengthspereachheuristiccongurationFigure7-3. Comparisonofdifferentheuristiccongurations 138

PAGE 139

RecallthatseveralheuristicswerepresentedinSection 4.2.1 .Wetestedvariouscongurationsoftheheuristicchoicesandtheresultsshowthatusingallfourheuristicsor(WW,WF)performbestformostcasesintermsofthecounterexamplelength:(WW,WF)congurationgives24bestsolutionsforour41testcases,whichisonemorethan(ARA,AF,WF,WW),butgiveslongestpathforoncecase,moldyn.TheexperimentalresultscomparingdifferentheuristiccongurationsaregiveninFigure 7-3 .Fig 7-3A showswhichheuristiccongurationfoundthemostbestsolutionswhentheshortestcounterexamplewaschosenasthebestsolution.Fig 7-3B showsthesumofallcounterexamplelengthsforallraceexampleswehadtested.The(ARA,AF,WF,WW)congurationwasusedfortheheuristicresultsinFigure 7-2 andTable 6-1 6-5 6-9 ,and 6-13 7.1.3ModularExtension Figure7-4. ThecomparisonofJRFandJRFmodularextensionfordifferentcongurationofFairMessage 139

PAGE 140

Inthissection,webrieyshowtheexperimentalresultofJRFmodularextensionusingFairMessageinSection 5.2 .Duringthemodularextensionvericationstep(step3)withDisBarrier,weidentiedtheunsafepublicationofflaginNodeconstructorandcorrectitusingthis.flag=flag;.ThishadbeenunidentiedusingJRFsincetheconcreteenvironmentwehadusedhappentohaveahappens-beforeorderthere.InFigure 7-4 ,JRFwithmodularextensionoutperformsJRF.UnboundedQueueApplyandDisBarrierApplyarethejunitdriverwhichweusedinSection 6.2 .ThelibrariesUnboundedQueueandDisBarrierinSection 5.2 arecorrectedaccordingtothesuggestioninTable 6-4 .Theresultshowsthatforallthreecases,JRFmodularextensionoutperformsJRF.Inaddition,FairMessagesavesmoretimeandmemorythanUnboundedQueueApplyandDisBarrierApplysinceitusesbothlibrariesandthesavingsarecompositional.ThenumberofstatesgeneratedbyJPFremainsthesameasweexpectedbuttotalnumberofstatesvisitedinJRFmodularextensionislessthanJRFbecauseofthesmallerhinallcases.ItisclearthatthemessagefromJRFmodularextensionforFairMessageinFigure 5-26 inSection 5.2 iseasiertounderstandthanthemessagefromJRForJRF-E. ======================================================JRFresults======================================================datarace#1edu.ufl.cise.jrf.util.HBDataRaceExceptionatTHREAD(java.lang.Thread@fromnull)toMEMORY(jrfm.UnboundedQueue@.tailfrom"volatileUnboundedQueuequeue=newUnboundedQueue();"atjrfm/FairMessage.java:10in())inINSTRUCTION(getfield)ofSOURCE("for(Nodetmp=head.next;tmp!=null&&tmp!=tail;tmp=tmp.next,++i);"atjrfm/UnboundedQueue.java:72)...======================================================JRF-Eresults______________________________________________________analyzecounterexampledataracesourcestatement:"putfield"atjrfm/UnboundedQueue.java:58:"tail=e;"bythread1dataracemanifeststatement:"getfield"atjrfm/UnboundedQueue.java:72:"for(Nodetmp=head.next;tmp!=null&&tmp!=tail;tmp=tmp.next,++i);"bythread0Changethefield"jrfm.UnboundedQueue@.tailfrom"volatileUnboundedQueuequeue=newUnboundedQueue();"atjrfm/FairMessage.java:10in()"tovolatile.Lock"java.util.concurrent.locks.ReentrantLock@from"enqLock=newReentrantLock();"atjrfm/UnboundedQueue.java:25in()"beforeaccessing(jrfm.UnboundedQueue@.tail)... 140

PAGE 141

7.2OverheadInordertoshowtheoverheadofJRFcomparedwithstandardJPF,weusedthesameexamplesexceptIterator LockFreeSetwhichrunsoutofmemoryandranoriginalJPFwithaslightchangetomakeitstopatthesamestateasJRF.Figure 7-5 representsthetimeandmemoryconsumedbyoriginalJPF,JRFwithoutthreadlocaloptimization,andJRFwiththreadlocaloptimizationrespectively.ThedefaultofDFSwasusedforallcases. Figure7-5. ComparisonofJRFwith/withoutthreadlocaloptimizationwithoriginalJPFwhenDFSstrategyiscongureandforcedtostopatthestatewhereJRFndtherstrace 141

PAGE 142

CHAPTER8CONCLUSIONAmultithreadedconcurrentsystemisthemainprogrammingenvironmentinmoderncomputerarchitectureandGUI-basedapplicationsmakeitinevitablethatwewillusemultiplethreadsinteractingwitheachotherthroughsharedmemories.Thisputsemphasisonthevericationofaconcurrentprogram,butunfortunatelythereisnoexistingapproachtopreciselyaddressthisproblem.Eventheexistinganalysistoolsusedtoreasonaboutconcurrentprogrambehaviorarebasedonafalseassumptionaboutsequentialconsistency,whichisidealbutnotrealistic.Itisverydifculttounderstandamemorymodelprecisely,buttheconsequenceofmisunderstandingcanbedevastatingtoprogramcorrectness.Inthisdissertation,wehaveaddressedthisproblemandprovidedamethodtohelpprogrammersndconcurrencybugsrelatedtoarelaxedmemorymodel.First,weprovedthataprogramwithoutadataracecoulduseexistingmodelcheckerstoverifyitsproperties.Uponthisresult,wereducedtheproblemsofarelaxedmemorymodelandasequentialconsistencyissueintoadataracedetectionproblem.Wehavedescribedanapproachbasedonmaintainingafunctionsummarizingthehappens-beforerelationthatcanbeusedinamodelcheckertopreciselydetectdataraces.WeimplementedthisintoanexistingJPFmodelcheckerandsuggestedseveraltechniquestomakeitmoreefcient.Inaddition,weintroducednewsearchheuristicsbasedonacarefulanalysisofdataracesthatleadstoshorterandmoreeasilyunderstoodcounterexamplepaths.Second,wedevelopedtechniquestoanalyzethepaththatndsaracetomakeiteasiertoidentifywhattheproblemis,andalsotosuggestmodicationstothecodetoeliminateraces.Withoutathoroughunderstandingofthememorymodel,itisevenhardertocorrecttheprogramwithadataracethantondone.Oursuggestionsarepreciseandconsistentsincetheybasedontheanalysisofthehappens-beforerelationsandinformationgatheredfrompathswithoutarace.Wehaveidentiedimportant 142

PAGE 143

concurrencybugpatternsduringthecasestudiesandthisimprovedtheanalysisofracestoprovidebettersuggestions.Inaddition,weaddressedtheproblemofforeigncodeinarelaxedmemorymodel.Withrestrictedinformationabouttheinternalimplementationdetails,theprogrammereasilyviolatestheconsistentusagepatterns.Weprovidedamethodtoautomaticallychecktheconsistencyofforeignlibraryinvocationsusingpreconditions.Thisapproachwasimplementedusinganassume-guaranteetechniqueanddecomposedtheburdenofracedetectionintosmallermodules.Theprogrammerofaforeignlibrarymodulehadamechanismtochecktheirpreconditions'correctnessandtheuserofthisforeigncodehadamechanismtochecktheirconsistencytowardthem.TheideashavebeenimplementedinJRF,anextensionofJPFthatdetectsdataraces.Thisisimportant,becausestandardJPFisunsoundforprogramsthatcontainraces.JRFhasbeenshowntobeusefulinawiderangeofimportantconcurrentdatastructures.Incontrasttomostotherapproaches,JRFispreciseandcandealwiththewealthofsynchronizationactionsintheJavaprogramminglanguage.OurideasareimplementedinaJavarelaxedmemorymodelanditsmodelcheckerJPF,butwebelievethatourapproachcanbeappliedtoothermemorymodels,suchasC#andC++,withoutdifcultyandleftasafuturework.Themaincontributionsofthisworkcanbesummarizedas: AweakerrequirementforaJavaprogramtobeSC. Asummaryfunctionthatcapturesthenecessaryhappens-beforerelationalongwithasoundnessproof. Anefcientrepresentationofthesummaryfunction. Datarace-specicsearchheuristics. Anexplanationofadataracecounterexamplebyidentifyingmissinghappens-beforeedges. Suggestionsforcodemodicationstoeliminatethedataracebyanalyzingtheprioracquiringhistory. 143

PAGE 144

Preconditionsofalibrarymoduletoensuredataracefreedomofinternaleldsandanautomatictooltoverifytheircorrectnessinagenerateduniversalenvironment. Atooltocheckthepreconditionsatalibrarymoduleinvocation. Amemorymodel-awareextensiontoJPF,thusmakingJPFsoundforJMMandshowingfeasibilityofextendingitfornewmemorymodels. Soundanalysisoflock-freeandwait-freeprotocols,suchasdataraceavoidanceusinganycombinationofintrinsicandextrinsiclocks,volatilevariables,join,barriers,compareAndSetoperations,andtransmittingvaluesthroughconcurrentdatastructures. CasestudiesofapplyingJRFtovariousconcurrentprogramsfromconcurrentbuildingblockstothewidelyusedJavadevelopmentframework. 144

PAGE 145

REFERENCES [1] M.HerlihyandN.Shavit,TheArtofMultiprocessorProgramming,MorganKaufmann,2008. [2] Aminoconcurrentbuildingblocks,http://amino-cbbs.sourceforge.net/. [3] Googleconcurrentdatastructuresworkshopbarriers,http://code.google.com/p/concurrent-data-structures-workshop-barriers/. [4] TheJavaGrandeForumbenchmarksuite,http://www2.epcc.ed.ac.uk/computing/research activities/java grande/index 1.html. [5] L.Lamport,Howtomakeamultiprocessorcomputerthatcorrectlyexecutesmultiprocessprograms,IEEETransactionsonComputers,vol.C-28,no.9,pp.690,September1979. [6] J.Gosling,B.Joy,G.Steele,andG.Bracha,JavaLanguageSpecication,AddisonWesley,3rdedition,2005. [7] ECMAInternational,StandardECMA-335-CommonLanguageInfrastructure(CLI),4edition,June2006. [8] H.-J.BoehmandS.V.Adve,Foundationsofthec++concurrencymemorymodel,SIGPLANNot.,vol.43,no.6,pp.68,2008. [9] G.BronevetskyandB.R.deSupinski,Completeformalspecicationoftheopenmpmemorymodel,Int.J.ParallelProgram.,vol.35,no.4,pp.335,2007. [10] J.Manson,Data-race-fullazyinitializationforperformance,December2008. [11] V.A.Saraswat,R.Jagadeesan,M.Michael,andC.vonPraun,Atheoryofmemorymodels,inPPoPP'07:Proceedingsofthe12thACMSIGPLANSym-posiumonPrinciplesandPracticeofParallelProgramming,NewYork,NY,USA,2007,pp.161,ACMPress. [12] C.FlanaganandS.N.Freund,Type-basedracedetectionforJava,inPLDI'00:ProceedingsoftheACMSIGPLAN2000ConferenceonProgrammingLanguageDesignandImplementation,NewYork,NY,USA,2000,pp.219,ACMPress. [13] D.F.Bacon,R.E.Strom,andA.Tarafdar,Guava:Adialectofjavawithoutdataraces,inObject-OrientedProgramming,Systems,Languages,andApplications(OOPSLA,2000,pp.382. [14] M.Abadi,C.Flanagan,andS.N.Freund,Typesforsafelocking:Staticracedetectionforjava,ACMTrans.Program.Lang.Syst.,vol.28,no.2,pp.207,2006. 145

PAGE 146

[15] K.-P.LohrandM.Haustein,Jac-minimizingthedifferencesbetweenconcurrentandsequentialjavacode,Tech.Rep.TRB-05-xx,FBMathematikundInformatik,FUBerlin,2005. [16] M.Naik,A.Aiken,andJ.Whaley,EffectivestaticracedetectionforJava,inPLDI'06:Proceedingsofthe2006ACMSIGPLANConferenceonProgrammingLanguageDesignandImplementation,NewYork,NY,USA,2006,pp.308,ACMPress. [17] Mobiusconsortium.deliverabled3.3:Preliminaryreportonthread-modularverication,March2007,http://mobius.inria.fr. [18] S.Savage,M.Burrows,G.Nelson,P.Sobalvarro,andT.Anderson,Eraser:adynamicdataracedetectorformultithreadedprograms,ACMTrans.Comput.Syst.,vol.15,no.4,pp.391,1997. [19] E.BoddenandK.Havelund,Racer:effectiveracedetectionusingaspectj,inISSTA'08:Proceedingsofthe2008internationalsymposiumonSoftwaretestingandanalysis,NewYork,NY,USA,2008,pp.155,ACM. [20] T.Elmas,S.Qadeer,andS.Tasiran,Goldilocks:araceandtransaction-awareJavaruntime,inPLDI'07:Proceedingsofthe2007ACMSIGPLANconferenceonProgrammingLanguageDesignandImplementation,NewYork,NY,USA,2007,pp.245,ACM. [21] S.Burckhardt,R.Alur,andM.M.K.Martin,Checkfence:Checkingconsistencyofconcurrentdatatypesonrelaxedmemorymodels,inPLDI,2007. [22] G.Gopalakrishnan,Y.Yang,andH.Sivaraj,QBornotQB:Anefcientexecutionvericationtoolformemoryorderings,inCAVLNCS3114,2004. [23] Y.Yang,G.Gopalakrishnan,G.Lindstrom,andK.Slind,Nemos:Aframeworkforaxiomaticandexecutablespecicationsofmemoryconsistencymodels,inIPDPS,2004. [24] S.BurckhardtandM.Musuvathi,Effectiveprogramvericationforrelaxedmemorymodels,inCAV'08:Proceedingsofthe20thinternationalconferenceonComputerAidedVerication,Berlin,Heidelberg,2008,pp.107,Springer-Verlag. [25] A.De,A.Roychoudhury,andD.D'Souza,Javamemorymodelawaresoftwarevalidation,inPASTE'08:Proceedingsofthe8thACMSIGPLAN-SIGSOFTworkshoponProgramanalysisforsoftwaretoolsandengineering,NewYork,NY,USA,2008,pp.8,ACM. [26] T.Q.HuynhandA.Roychoudhury,Memorymodelsensitivebytecodeverication,Form.MethodsSyst.Des.,vol.31,no.3,pp.281,2007. [27] M.Musuvathi,S.Qadeer,andT.Ball,Chess:Asystematictestingtoolforconcurrentsoftware,Tech.Rep.MSR-TR-2007-149,MicrosoftResearch,2007. 146

PAGE 147

[28] S.Basu,D.Saha,andS.A.Smolka,Localizingprogramerrorsforcimpledebugging,inFORTE,2004,pp.79. [29] A.GroceandW.Visser,Whatwentwrong:Explainingcounterexamples,inSPINWorkshoponModelCheckingofSoftware.2003,pp.121,Springer. [30] T.Ball,M.Naik,andS.K.Rajamani,Fromsymptomtocause:localizingerrorsincounterexampletraces,inPrinciplesofProgrammingLanguages,2003,pp.97. [31] A.Groce,Errorexplanationwithdistancemetrics,inToolsandAlgorithmsfortheConstructionandAnalysisofSystems,2004,pp.108. [32] A.Groce,D.Kroening,andF.Lerda,Understandingcounterexampleswithexplain,inComputer-AidedVerication.2004,pp.453,Springer.LNCS. [33] M.RenierisandS.P.Reiss,Faultlocalizationwithnearestneighborqueries,inASE,2003,pp.30. [34] H.CleveandA.Zeller,Locatingcausesofprogramfailures,inICSE,2005,pp.342. [35] T.WangandA.Roychoudhury,Automatedpathgenerationforsoftwarefaultlocalization,inASE,2005,pp.347. [36] N.Gupta,H.He,X.Zhang,andR.Gupta,Locatingfaultycodeusingfailure-inducingchops,inASE,2005,pp.263. [37] Y.BrunandM.D.Ernst,Findinglatentcodeerrorsviamachinelearningoverprogramexecutions,inICSE,2004,pp.480. [38] C.Flanagan,S.N.Freund,andJ.Yi,Velodrome:asoundandcompletedynamicatomicitycheckerformultithreadedprograms,inPLDI,2008,pp.293. [39] C.Flanagan,S.Qadeer,andS.A.Seshia,Amodularcheckerformultithreadedprograms,inInCAV02:ComputerAidedVerication.2002,pp.180,Springer. [40] C.FlanaganandS.Qadeer,Assume-guaranteemodelchecking,Tech.Rep.,2003. [41] C.d.l.Riva,J.Tuya,andJ.R.d.Diego,Translatingsa/rtmodelstosynchronousreactivesystems:Anapproximationtomodularvericationusingthesmvmodelchecker,inPSI'99:ProceedingsoftheThirdInternationalAndreiErshovMemo-rialConferenceonPerspectivesofSystemInformatics,London,UK,2000,pp.493,Springer-Verlag. [42] C.delaRivaandJ.Tuya,Modularmodelcheckingofsoftwarespecicationswithsimultaneousenvironmentgeneration,inAutomatedTechnologyforVerication 147

PAGE 148

andAnalysis,F.Wang,Ed.,vol.3299ofLectureNotesinComputerScience,pp.369.SpringerBerlin/Heidelberg,2004. [43] C.Pasareanu,D.Giannakopoulou,M.Bobaru,J.Cobleigh,andH.Barringer,Learningtodivideandconquer:applyingthel*algorithmtoautomateassume-guaranteereasoning,FormalMethodsinSystemDesign,vol.32,pp.175,2008,10.1007/s10703-008-0049-6. [44] D.GiannakopoulouandC.S.Pasareanu,Interfacegenerationandcompositionalvericationinjavapathnder,inFASE'09:Proceedingsofthe12thInterna-tionalConferenceonFundamentalApproachestoSoftwareEngineering,Berlin,Heidelberg,2009,pp.94,Springer-Verlag. [45] O.TkachukandM.B.Dwyer,Adaptingsideeffectsanalysisformodularprogrammodelchecking,SIGSOFTSoftw.Eng.Notes,vol.28,no.5,pp.188,2003. [46] G.HughesandT.Bultan,Interfacegrammarsformodularsoftwaremodelchecking,inISSTA'07:Proceedingsofthe2007internationalsymposiumonSoftwaretestingandanalysis,NewYork,NY,USA,2007,pp.39,ACM. [47] O.TkachukandM.B.D.O.Cis,Automatedenvironmentgenerationforsoftwaremodelchecking,inInProceedingsofthe18thInternationalConferenceonAutomatedSoftwareEngineering,2003,pp.116. [48] C.C.Loyola,C.Colby,P.Godefroid,andL.J.Jagadeesan,Automaticallyclosingopenreactiveprograms,inInProceedingsof1998ACMSIGPLANConferenceonProgrammingLanguageDesignandImplementation.1998,pp.345,ACMPress. [49] P.Parizek,J.Adamek,andT.Kalibera,Automatedconstructionofreasonableenvironmentforjavacomponents,Electron.NotesTheor.Comput.Sci.,vol.253,no.1,pp.145,2009. [50] C.Boyapati,R.Lee,andM.Rinard,Ownershiptypesforsafeprogramming:Preventingdataracesanddeadlocks,2002. [51] R.AgarwalandS.D.Stoller,Typeinferenceforparameterizedrace-freejava,inInProceedingsoftheFifthInternationalConferenceonVerication,ModelCheckingandAbstractInterpretation.2004,pp.149,Springer-Verlag. [52] A.Sasturkar,R.Agarwal,L.Wang,andS.D.Stoller,Automatedtype-basedanalysisofdataracesandatomicity,inProc.ACMSIGPLAN2005SymposiumonPrinciplesandPracticeofParallelProgramming(PPoPP).June2005,ACMPress. [53] J.Manson,W.Pugh,andS.V.Adve,TheJavamemorymodel,inPOPL'05:Proceedingsofthe32ndACMSIGPLAN-SIGACTSymposiumonPrinciplesofProgrammingLanguages,NewYork,NY,USA,2005,pp.378,ACMPress. 148

PAGE 149

[54] D.AspinallandJ.Sevcik,FormalisingJava'sdata-race-freeguarantee,inTPHOLs2007(LNCS).2007,vol.4732,pp.22,Springer. [55] E.M.C.Jr.,O.Grumberg,andD.A.Peled,ModelChecking,TheMITPress,1999. [56] E.A.Emerson,Thebeginningofmodelchecking:Apersonalperspective,pp.27,2008. [57] P.Godefroid,Partial-ordermethodsforthevericationofconcurrentsystems-anapproachtothestate-explosionproblem,1995. [58] K.L.McMillan,SymbolicModelChecking,KluwerAcademicPublishers,Norwell,MA,USA,1993. [59] A.Biere,A.Cimatti,E.M.Clarke,O.Strichman,andY.Zhu,Boundedmodelchecking,2003. [60] JavaPathnder,http://javapathnder.sourceforge.net/. [61] Indus,http://indus.projects.cis.ksu.edu/. [62] B.Goetz,T.Peierls,J.Bloch,J.Bowbeer,D.Holmes,andD.Lea,JavaConcur-rencyinPractice,AddisonWesleyProfessional,2006. [63] O.ShalevandN.Shavit,Split-orderedlists:lock-freeextensiblehashtables,JournaloftheACM,vol.53,no.3,pp.379,2006. [64] Concurrentadtexperimentframework,http://code.google.com/p/concurrentadt/. [65] T.Elmas,S.Qadeer,andS.Tasiran,Preciseracedetectionandefcientmodelcheckingusinglocksets,Tech.Rep.TechnicalreportMSR-TR-2005-118,MicrosoftResearch,2006. [66] Zing,http://research.microsoft.com/en-us/projects/Zing/. [67] M.Herlihy,N.Shavit,andM.Tzafrir,Concurrentcuckoohashing,Tech.Rep.,BrownUniversity,2007. [68] T.Craig,BuildingFIFOandpriority-queueingspinlocksfromatomicswap,Tech.Rep.TR93-02-02,DepartmentofComputerScience,UniversityofWashington,1993. [69] P.Magnussen,A.Landin,andE.Hagersten,Queuelocksoncachecoherentmultiprocessors,inProceedingsoftheEighthInternationalSymposiumonParallelProcessing(IPPS),1994,pp.165. 149

PAGE 150

BIOGRAPHICALSKETCH KyungHeeKimreceivedherbachelor'sdegreeincomputerscienceandengineeringatPohangUniversityofScienceandTechnologyinKoreain1995.SheworkedfortheSamsungElectronics,Inc.,Koreafrom1995to1999.ShewasinvolvedinthedevelopmentofanextgenerationRealtimeoperatingsystem,transportingpJavaontopofpSOSrealtimeoperatingsystem,andthedesignofanexecutionunitfordigitaltelevision.ShealsoworkedforAlticastInc.inKoreainimplementingtheindependentdataserverforthedigitalbroadcastingsystemofKoreanBroadcastingSystem(KBS).Since2005,shehasbeenconductingresearchwithDr.BeverlyA.SandersinDepartmentofComputerandInformationScienceandEngineeringattheUniversityofFlorida.Herresearchinterestsareconcurrentprogramming,formalverication,staticanalysis,andsoftwaremodelchecking. 150