Citation
Exploring Cost-Effective Resource Management Strategies In The Age Of Utility Computing

Material Information

Title:
Exploring Cost-Effective Resource Management Strategies In The Age Of Utility Computing
Creator:
Zhao, Han
Publisher:
University of Florida
Publication Date:
Language:
English

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Computer Engineering
Computer and Information Science and Engineering
Committee Chair:
Li, Xiaolin
Committee Members:
Chen, Shigang
Figueiredo, Renato Jansen
Fortes, Jose A
Fang, Yuguang
Graduation Date:
5/4/2013

Subjects

Subjects / Keywords:
Auctions ( jstor )
Budget allocation ( jstor )
Cloud computing ( jstor )
Envy ( jstor )
Hypergraphs ( jstor )
Prices ( jstor )
Pricing ( jstor )
Scheduling ( jstor )
Simulations ( jstor )
Tenants ( jstor )
cloud
computing
distributed
management
middleware
optimization
resource
scheduling
utility

Notes

General Note:
With the rapid progress of computing, storage and networking technologies, distributed computing paradigms have undergone profound changes in the past decade. We are entering an era of "Everything-as-a-Service" where resources are shared at an unprecedented scale and deliver agile, metered computing services to both business and scientific communities. The so called utility computing model, built upon cloud computing infrastructures, becomes ubiquitous in the enterprise IT landscape. It is therefore of paramount importance to devise efficient resource management strategies to scale with the growth of the system. However, the problem of managing resource allocations in a utility computing environment is challenging because both resources and administrative parties who operate these resources feature diverse heterogeneity. As utility computing proliferates, scalable resource sharing platform instantiated on multiple resource providers become cheaper and more accessible. As a result, strategy design for resource management in a utility computing model should equally address the heterogeneous interests of various involved parties who pursue maximum economic benefits. As a result, an inter-disciplinary research approach that combines economic models in social computing scenarios with algorithmic design in computer science becomes a viable option for researchers to build cost-effective resource scheduling strategies in utility computing. We recognized three fundamental issues that govern the exploration in cost-effective resource management strategies in this dissertation. (1) The flourish of virtualization technology enables more flexible resource aggregation and presents an exponential search space for optimization. (2) The heterogeneous nature of user interests has direct impact on resource management decisions. (3) Financial costs play an important role in determining the achievable application performance. To address these issues, we develop several resource management strategies that achieve cost-effectiveness and flexibility with regard to various scheduling contexts in utility computing. Our study seeks to investigate economic models and their implication to the utility-oriented scheduling problems. The proposed research highlights the heterogeneity challenge presented in utility and cloud computing. Concentrating on the strategy design space of resource customers, our study for cost-effective resource management strategy progressively evolve towards better efficiency and flexibility. Specifically, this dissertation include the following main scientific contributions: (1) development of optimal resource rental planning models in a utility computing environment, based on linear integer programming and stochastic optimization techniques; (2) design of a suite of efficient and fair resource trading protocols, allowing the distributed system to benefit from utility-driven resource trading activities; and (3) implementation of an experimental market-oriented resource sharing platform integrating cloud resource management with eBay's transaction model. The study presented in this dissertation improves upon existing research as it targets at cost-effective design and accommodates flexibility in service provisioning and acquisition in utility computing.

Record Information

Source Institution:
UFRGP
Rights Management:
Copyright Zhao, Han. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Embargo Date:
5/31/2015

Downloads

This item has the following downloads:


Full Text

PAGE 1

EXPLORINGCOST-EFFECTIVERESOURCEMANAGEMENTSTRATEGIESINTHEAGEOFUTILITYCOMPUTINGByHANZHAOADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2013

PAGE 2

c2013HanZhao 2

PAGE 3

TomywifeXinxinLiuforherconstantsupportandcaringthroughoutmygraduatestudy 3

PAGE 4

ACKNOWLEDGMENTS Iwouldliketoexpressmysincereappreciationtothemanywhohavehelpedmethroughthegraduateschool.Firstandforemost,Ithankmyadvisor,Dr.Xiaolin(Andy)Li.Itwouldbeimpossibleformetoachievemygoalwithouthissoundadvice,patience,generosityandencouragementthroughoutmystudyatbothUniversityofFloridaandOklahomaStateUniversity.IalsocannotimaginehavingspentthelastfewyearsatUniversityofFloridawithoutlearningfromandworkingwithDr.Yuguang(Michael)FangandDr.RenatoFigueiredo,twoofthebestmentorsIevermet.Ihavefoundthemtobeoutstandingteachingmentors,researchcollaborators,andsourcesofpersonaladvice.IwouldalsoliketoexpressmyheartfeltthankstoDr.JoseFortesandDr.ShigangChenforservingonmydissertationcommittee.Iamveryfortunatetohearfromthesetwoscholarsandlearnedhowtoconductindependentresearchand,asimportantly,howtoenjoyit.BeforetransferringtoUniversityofFlorida,Ihadthegoodfortuneofworkingwithmanyexcellentmentorswhotaughtmethevalueofresearch.IhaveDr.JohnChandler,Dr.K.M.George,Dr.DouglasR.Heisterkamp,Dr.SubhashKak,Dr.NohpillPark,andDr.JohnsonThomastothankforfouryearsofacademicandpersonalgrowthwhiledoingresearchatOklahomaStateUniversity.Duringsummer2010,IhavespentthreemonthsworkingasaresearchinternatIBMThomasJ.WatsonResearchCenter,Hawthorne,NY.IamveryfortunatetohaveDr.ChaiWahWuandDr.PeterWesterinkasmyresearchmentors.AlsoIwouldliketothankDr.PaoloDettori,Dr.JulioNogima,andDr.FrankSchaffafortheirsupporttomyinternshipwork.IalsogreatlyappreciatemyenjoyablecollaborationwithDanielSmilkovduringmyinternshipperiodatIBMresearch.Myyearsatgraduateschoolwouldhavebeenmuchlessenjoyablewithoutthefriendshipandgoodwitofmylabmates,HuanyuZhao,XinYang,ZeYu,MinLi,KaikaiLiu,DiWang,LiYu,RuiYang,XingMao,ShivamTivari,andthelistgoesonandon. 4

PAGE 5

Laughterandmanygreatpartiestogetherhavewitnessedourfriendship.ThankstoMiaoPan,HaoYue,andLinkeGuo,etc.atWirelessNetworksLabformanyvaluablediscussionsonresearchandpersonallife.Mostofall,IwouldliketodevotemyspecialthankstomywifeXinxinwhoisnowaPh.D.inthedepartmentofCISEatUniversityofFlorida.Sheismymostintimatesoulmateandalwaysgivesmeunconditionalsupportthroughoutmyyearsatgraduateschool.Mylifewouldmessupwithoutherlendingahelpinghandforhousework.Besides,shesharedthejoysandtearswithmeaswehavecollaboratedinmanyresearchprojectsandhavespentnumeroussleeplessnightstogethercatchingprojectandpaperdeadlines.Thosearesurelythemostmemorablemomentsinmywholelife.Iwouldalsolovetosaythankyoutomylovingparents,fortheirconstantsupportandformakingmewhoIam.IthanktheUniversityofFloridaandOklahomaStateUniversityfortheirgeneroussupportofmygraduatestudies.MyworkhasalsobeensupportedbyNationalScienceFoundationgrantCCF-0953371,CCF-1128805,OCI-0904938,CNS-0709329,andCNS-0916391. 5

PAGE 6

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 9 LISTOFFIGURES ..................................... 10 ABSTRACT ......................................... 12 CHAPTER 1INTRODUCTION ................................... 14 1.1Motivation .................................... 14 1.2SummaryofRelatedWork .......................... 15 1.2.1Socioeconomicapproachforresourcemanagement ........ 16 1.2.2Managingresourceincloudandutilitycomputing .......... 17 1.3Contribution ................................... 18 1.4Organization .................................. 22 2FINE-GRAINEDRESOURCERENTALPLANNING ............... 23 2.1Background ................................... 23 2.2PriorWork .................................... 26 2.3Fine-GrainedResourceRentalPlanning ................... 27 2.3.1SystemModel .............................. 28 2.3.2Motivationforne-grainedresourcerentalplanning ......... 29 2.3.3Optimizingplanningfordeterministicinstancepricing ........ 30 2.3.4Solutiontodeterministicpricingresourcerentalplanning ...... 34 2.3.5Evaluationofdeterministicpricingresourcerentalplanning .... 35 2.4DealingwithSpotPricingUncertaintyinCloud ............... 38 2.4.1PredictiveplanninginAmazonspotmarket ............. 39 2.4.1.1Introduction .......................... 39 2.4.1.2Methodology ......................... 40 2.4.2Stochasticplanningforspotpricingmarket .............. 45 2.4.2.1Solutionoverview ...................... 45 2.4.2.2Bid-dependentdynamicsampling ............. 46 2.4.2.3Transformingusingmultistagerecourse .......... 47 2.4.2.4DeterministicreformulationofSRRP ............ 48 2.4.2.5Polynomial-timesolutions .................. 49 2.4.2.6Evaluationofstochasticrentalplanningmodel ...... 51 3TOWARDSEFFICIENTANDFAIRRESOURCETRADING ........... 54 3.1Background ................................... 54 3.2Priorwork .................................... 56 6

PAGE 7

3.3ADistributedResourceTradingFrameworkforCommunityCloud ..... 58 3.3.1Systemmodel .............................. 58 3.3.2Problemstatement ........................... 60 3.4Budget-unawareResourceTradingProtocol ................. 62 3.4.1Designpreliminaries .......................... 62 3.4.2Amultiagent-basedoptimizationapproachforresourcetrading .. 64 3.4.3Protocolimplementation ........................ 66 3.4.3.1Localviewestablishment .................. 67 3.4.3.2Dealnegotiation ....................... 67 3.4.3.3Messagebroadcasting ................... 68 3.4.3.4Convergencedetection ................... 68 3.4.3.5Furtherdiscussion ...................... 69 3.5Budget-awareResourceTradingProtocol .................. 69 3.5.1Modelingresourcetradingusingadirectedhypergraph ....... 69 3.5.2Optimalstructuresindirectedhypergraph .............. 71 3.5.3Protocoldesign ............................. 73 3.6PerformanceEvaluation ............................ 75 3.6.1Simulationsettings ........................... 75 3.6.2EvaluationofBuMRT .......................... 76 3.6.3Convergenceanalysis ......................... 77 3.6.4EvaluationofBaMRT .......................... 78 3.6.5Sensitivityanalysis ........................... 79 4CloudBay:CLOUDMIDDLEWAREFORSCALABLERESOURCESHARING 82 4.1Summary .................................... 82 4.2Background ................................... 82 4.3PriorWork .................................... 85 4.4DesignOverview ................................ 86 4.4.1Architecture ............................... 86 4.4.2Usecaseillustration .......................... 87 4.4.3Resourcevirtualizationtools ...................... 88 4.4.4Autonomicresourcepooling ...................... 89 4.5Market-drivenServiceScheduling ...................... 91 4.5.1Resourceandservicerequestmodels ................ 91 4.5.2Jobsubmission ............................. 93 4.5.3Economybootstrapping ........................ 94 4.5.4Interfaceforvaluationexpression ................... 94 4.5.5ServiceschedulinginCloudBay .................... 95 4.5.6Paymentaccounting .......................... 97 4.5.7Discussion ................................ 98 4.6Evaluation .................................... 99 4.6.1Evaluationforresourcepooling .................... 99 4.6.2Evaluationforservicescheduling ................... 101 7

PAGE 8

5CONCLUSIONANDFUTUREWORK ....................... 105 5.1Conclusion ................................... 105 5.2FutureWork ................................... 107 REFERENCES ....................................... 108 BIOGRAPHICALSKETCH ................................ 119 8

PAGE 9

LISTOFTABLES Table page 2-1Summaryofnotations ................................ 32 9

PAGE 10

LISTOFFIGURES Figure page 1-1Utility-driveresourcecustomerfacingresourcemanagementchallenges .... 18 1-2Designrevolution1:smartresourcerentalplanning ............... 19 1-3Designrevolution2:efcientandfairresourcetrading .............. 20 1-4Designrevolution3:exibleandopenresourcetradingmarket ......... 22 2-1Systemmodelforthecloud-basedresourcerentalplanningproblem ...... 28 2-2ExampleforecastdemandscheduleforaVS. ................... 30 2-3Flownetworkofthedeterministicpricingresourcerentalplanningproblem ... 34 2-4CostcomparisonforDRRPandresourcerentalwithoutplanning ........ 37 2-5SensitivityanalysisforDRRP ............................ 38 2-6Box-and-Whiskerdiagramforspotpricehistory .................. 41 2-7VariationofdailyspotpriceupdatefrequencyforVMclasslinux-c1-medium .. 41 2-8Histogramplotfortheselectedspotpricehistory(linux-c1-medium) ...... 42 2-9Datadecompositionfortheselecteddataseries ................. 43 2-10ACFandPACFfortheselectedseries ....................... 44 2-11Day-aheadpredictionfortheselectedseries ................... 45 2-12Anexampleofmultistagescenariotree ...................... 48 2-13Costcomparisonofpredictiveandstochasticplanning .............. 52 2-14ImpactofapproximationprecisionforSRRPsolution ............... 53 3-1Multinenancyresourcetrading:systemmodel ................... 59 3-2Acounterexampleofgloballyefcientallocation ................. 63 3-3Dealnegotiation ................................... 68 3-4Paymentmessagebroadcastalongabroadcasttree ............... 68 3-5Adirectedhypergraphmodelderivedfromanmnnhyperspace ...... 70 3-6Directedhypergraphandspanninghypertree ................... 73 3-7Performanceevaluationforbudget-unawarecase ................. 78 10

PAGE 11

3-8Convergenceanalysisforbudget-unawarecase ................. 78 3-9Performanceevaluationforbudget-awarecase .................. 80 3-10Impactofdifferentpaymentselectionstrategiesforbudget-awarecase. .... 81 3-11Impactofinitialbudgetassignmentforbudget-awarecase. ........... 81 4-1CloudBayarchitecture ................................ 86 4-2AsimpleusecaseforCloudBay .......................... 88 4-3Userinterfaceforviewingresourcesintheglobalpool .............. 90 4-4JobsubmissioninCloudBay ............................ 94 4-5Experimentforautonomicresourcepooling .................... 99 4-6Performanceevaluationforvirtualnetoworking .................. 100 4-7Evaluationoftheoverheadduetopreemption ................... 103 4-8ServicedelayfactorVS.Offeredprice ....................... 103 11

PAGE 12

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyEXPLORINGCOST-EFFECTIVERESOURCEMANAGEMENTSTRATEGIESINTHEAGEOFUTILITYCOMPUTINGByHanZhaoMay2013Chair:Xiaolin(Andy)LiMajor:ComputerEngineering Withtherapidprogressofcomputing,storageandnetworkingtechnologies,distributedcomputingparadigmshaveundergoneprofoundchangesinthepastdecade.WeareenteringaneraofEverything-as-a-Servicewhereresourcesaresharedatanunprecedentedscaleanddeliveragile,meteredcomputingservicestobothbusinessandscienticcommunities.Thesocalledutilitycomputingmodel,builtuponcloudcomputinginfrastructures,becomesubiquitousintheenterpriseITlandscape.Itisthereforeofparamountimportancetodeviseefcientresourcemanagementstrategiestoscalewiththegrowthofthesystem.However,theproblemofmanagingresourceallocationsinautilitycomputingenvironmentischallengingbecausebothresourcesandadministrativepartieswhooperatetheseresourcesfeaturediverseheterogeneity.Asutilitycomputingproliferates,scalableresourcesharingplatforminstantiatedonmultipleresourceprovidersbecomecheaperandmoreaccessible.Asaresult,strategydesignforresourcemanagementinautilitycomputingmodelshouldequallyaddresstheheterogeneousinterestsofvariousinvolvedpartieswhopursuemaximumeconomicbenets.Asaresult,aninter-disciplinaryresearchapproachthatcombineseconomicmodelsinsocialcomputingscenarioswithalgorithmicdesignincomputersciencebecomesaviableoptionforresearcherstobuildcost-effectiveresourceschedulingstrategiesinutilitycomputing. 12

PAGE 13

Werecognizedthreefundamentalissuesthatgoverntheexplorationincost-effectiveresourcemanagementstrategiesinthisdissertation. 1. Theourishofvirtualizationtechnologyenablesmoreexibleresourceaggrega-tionandpresentsanexponentialsearchspaceforoptimization. 2. Theheterogeneousnatureofuserinterestshasdirectimpactonresourceman-agementdecisions. 3. Financialcostsplayanimportantroleindeterminingtheachievableapplicationperformance. Toaddresstheseissues,wedevelopseveralresourcemanagementstrategiesthatachievecost-effectivenessandexibilitywithregardtovariousschedulingcontextsinutilitycomputing.Ourstudyseekstoinvestigateeconomicmodelsandtheirimplicationtotheutility-orientedschedulingproblems.Theproposedresearchhighlightstheheterogeneitychallengepresentedinutilityandcloudcomputing.Concentratingonthestrategydesignspaceofresourcecustomers,ourstudyforcost-effectiveresourcemanagementstrategyprogressivelyevolvetowardsbetterefciencyandexibility.Specically,thisdissertationincludethefollowingmainscienticcontributions:(1)developmentofoptimalresourcerentalplanningmodelsinautilitycomputingenvironment,basedonlinearintegerprogrammingandstochasticoptimizationtechniques;(2)designofasuiteofefcientandfairresourcetradingprotocols,allowingthedistributedsystemtobenetfromutility-drivenresourcetradingactivities;and(3)implementationofanexperimentalmarket-orientedresourcesharingplatformintegratingcloudresourcemanagementwitheBay'stransactionmodel.Thestudypresentedinthisdissertationimprovesuponexistingresearchasittargetsatcost-effectivedesignandaccommodatesexibilityinserviceprovisioningandacquisitioninutilitycomputing. 13

PAGE 14

CHAPTER1INTRODUCTION 1.1Motivation Thefastdevelopmentofparallelanddistributedcomputingparadigms,drivenbyincreasingdemandforcomputingpowerandnetworkbandwidth,spursavarietyofmassivelydistributedcomputingplatforms,suchasP2P,cluster,andgridcomputingemerginginacademicandindustrialcommunities.Ofalltheresearchissuesintheeldofdistributedcomputing,resourcemanagement,whichconcernstheefcientandeffectiveacquisitionanddeploymentofcomputational,storageandnetworkingresources,isamongthemostimportantresearchtopics.Conventionalmethodsforresourcemanagement,mostlycentralized,aredifculttoadapttothegrowingcomplexityofmodernheterogeneousdistributedenvironments.Thisgrowingcomplexityismainlycausedbytwophenomena.First,thedistributedsystembecomesmorelooselycoupled,asitsscaleincreasesdramaticallyinthepastfewyears.Today'sdistributedcomputingplatformgrowsfromclustersinasinglelaboratorytomultiplegeographicallydistributedcomputationalsites,eachofthemcomposedofhundredsofcomputationalnodesandfeaturinghighautonomy.Second,computationaldevicesbecomemoreheterogeneousthaneverbefore.Manytypesofdevicesnowadaysarecapableofofferingcomputingpowerthatisonlyavailableonsupercomputersdecadeago,includingtabletcomputers,smartphones,gamingconsoles,etc.Asaresult,theincreasingscalabilityandheterogeneitybringmanychallengestoeffectiveresourcemanagementstrategydesign. Toaddressthesechallenges,researchershaveproposedmanypotentialresourcemanagementsolutionstoaccommodatethegrowingcomplexity,boththeoreticallyandpractically.Amongthem,oneapproachistousemarketorientedmechanismstoregulatetheschedulingdecisionmakingprocess.Themarketorientedmechanismsregarddistributedcomputingunitsaseconomicallyrationalindividualsinthehuman 14

PAGE 15

society,andtrytocharacterizethecooperationandcompetitionprocessusingeconomictheory.Hence,suchasocioeconomicapproachmostnaturallymodelsthesystemsinceitpreciselycapturetheessentialfeaturesofthemodernheterogeneousdistributedenvironments,andscaleswelltolargerandmorecomplexlooselycoupleddistributedplatforms. Recently,anewdistributedcomputingmodelcalledutilitycomputingbecomesubiquitousinenterpriseandcommerce,andhasdrawnsignicantattentionsfrommanyresearchers.Itoffersrental-basedaccesstoamassivepoolofcomputationalpowerandprovidesmeteringandaccountingfunctionsforresourceusage.Thisserviceprovisioningmodelhasquitealonghistory[ 1 ],butonlybecomepopularascloudcomputingprevails.Cloudcomputingeliminatestheheavyeconomicburdenofresourcesetupandoperationalcostforindustries,andliberatesthesoftwaredevelopmentprocessbyofferingon-demandserviceprovisioning.Theutilitycomputingmodelismostlybuiltonacloud-basedinfrastructure,anddenesaccountingpoliciesforresourceacquisition.Ithelpsusersgainaccesstocomputationalresourcesatatremendousscale.Whenwestepintotheageofutilitycomputing,weexpecttoseenewcomputingparadigmsimplementingEverything-as-a-Serviceandtobechargedbyreasonablerate.Mostimportantly,cost-effectiveresourcemanagementstrategiesarehighlydesirablesothatresourcesareareutilizedatlowcost,andasefcientlyaspossible.Again,theincreasingscalabilityandheterogeneitypresenttremendouschallenges,asoneneedstocopewithvaryingprovider-sideresourceavailabilityandpricing,aswellasuctuatinguser-sideapplicationcongurationanddemand.Theresearchpresentedinthisdissertationhighlightsthesechallengesandprovideasetofpossiblesolutionsforcost-effectiveresourcemanagement. 1.2SummaryofRelatedWork Inthissection,webrieyreviewrelatedresearchtoutility-basedresourcemanagementresourcestrategydesignintheliterature. 15

PAGE 16

1.2.1Socioeconomicapproachforresourcemanagement Inrecentyears,wehavewitnessedaburstofresearcheffortsthatstudytheapplicationofeconomicandgametheoryforresourcemanagementinheterogeneousdistributedsystems[ 2 4 ].Anearlyworkproposingcomputationaleconomyforresourceschedulingwasproposedin[ 5 ].Theauthorspresentedtwodifferentmarketstrategiesforagridcomputingenvironment,namelycommoditiesmarketandauction.Theadvantageofusingauctionsinacomputationaleconomyisthatit'sbenecialtodiscovercommoditypricesthroughstrategicmechanismdesign.Thistrendislargelycontributedtothefollowingobservations:designsimilarityofeconomicmarketmechanismsanddistributedsystemschedulingprinciples;androlesimilarityofrealisticrationalindividualsandegocentricheterogeneouscomputers.Therefore,market-orientedmethodsderivedfromgametheoryisextremelyhelpfulinmodelingbehaviorsofbenet-drivenagents.Methodsusinggametheoryconvergestosystemequilibriumstateonthebasisofrevenuemaximization.Thekeychallengeforresearchersistoidentifyasuitableobjectivefunctionthatdenestargetperformanceoptimizationintermofutility.Exampleapplicationsofeconomicmethodshavebeenproposedforvariousschedulingtopicsincludingbutnotlimitedtodynamicresourcesharing[ 6 ],workloadbalancing[ 7 8 ]andpromotingincentivesingridandP2Psystems[ 9 ].Therecentdevelopmentofcloudcomputingtechnologiesurgeresearcherstoinvestigatetheapplicationofauctionstomanageandschedulecloudresources[ 10 12 ]. Dependingonassumptionsforindividualcomputingresourcecontributor,themarketorientedmethodscanbecategorizedascooperativeornoncooperative.Cooperativemethods[ 13 ]takesadvantageofcooperativebehaviorsofindividualsitesforoptimalperformancescheduling.Ontheotherhand,noncooperativemethods[ 14 15 ]exploreinherentselshnessofcomputerpeersanddesignnegotiationstrategiesforutilitymaximization.Asthedistributedcomputingplatformsbecomemorelooselycoupled,weenvisionthatahybridarchitectureismoresuitabletomodelmodern 16

PAGE 17

distributedsystems,wherecooperationandnon-cooperationcoexistatvariousschedulingandmanagementlevels.Thisiswellconformedtodistributedcomputingplatformsinglobalscale,e.g.,P2PdesktopgridsystemssuchasCohesion[ 16 ]andOurGrid[ 17 ].Further,wearguethatcurrentliteratureofmarketorientedapproachessuffersfromthefollowingproblems:1)inaccuratemodelingofegocentricagentbehaviors;2)lackofefcientresourcemanagementmechanismdesignformulti-criteriaoptimization;and3)insufcientresearchinvestigationonideasfromotherdisciplines,mainlyfromeconomicandnancialelds.Alltheseissuesareaddressedinthisdissertation,andguideourresearchinexploringcost-effectiveresourcemanagementmechanisms. 1.2.2Managingresourceincloudandutilitycomputing Nowadays,cloudcomputingbecomesmoreprevalentanddrawssignicantattentionsfromtheHPCcommunitytosolveadvancedcomputationalproblems.Anumberofresearch[ 18 20 ]haveattemptedtostudythecost-benetofrunningcomputationalanddataintensiveapplicationsincloud.Itisevidentthatmovingandstoringlargedatasetincloudincurhugecostcomparabletothecomputingcost[ 18 ].Withregardtoresourceplanningincloud,variousoptimizationmodelsbasedon(non-)linearprogrammingwereproposed.Forexample,GoudarziandPedram[ 21 ]formulatedthemulti-dimensionalSLA-basedresourceallocationproblemasamixedintegernon-linearprogrammingproblem,andprovidedaheuristicsolutionbasedonforce-directedscheduling.QianandMedhi[ 22 ]presentedanoptimizationmodelforminimizingserveroperationalcostindatacenters.Inthisdissertation,weconsideramorecomprehensiveresourcerentalplanningmodelthattsinAmazon'sEC2pricingmodel. Inutilitycomputing,duetotheservice-orientedparadigmshift,resourcesbecomecommercializedindatacenters.Signicanteffortsweremadetoimplementeconomicsdrivenresourcemanagementinacloudenvironment,e.g.,usingauctions[ 23 ].Similar 17

PAGE 18

projectsaimingtorealizeavirtualcomputationalcloudacrossmultipledomainsincludeCometCloud[ 24 ]and4CaaSt[ 25 ].ThethirdpartofthisdissertationpresentsanovelsystemprototypedesigncalledCloudBay.TheroleofCloudBayisuniqueasitisnotdesignedtoserveasyetanotherIaaS,PaaS,orSaaSprovider,butrathertobridgethescatteredscienticcommunityinsupportofHPCapplications. 1.3Contribution Consideraresourcecustomerwishingtoobtainasetofcomputationalresourcesfromapubliclyaccessibleresourcepool,asillustratedinFigure 1-1 .Themajorchallengesincludehowtomeetthecomputationalservicedemandwhileatthesametime,reduceresourceacquisitioncost.Wemodeltheresourceusertobeutility-driventhatisinterestedinprotabletradingactivitiesandindependentlymakesmanagementdecisions.Thissectionsummarizesourdesignevolutionofcost-effectiveresourcemanagementstrategiesfortheutility-drivenresourcecustomer. Figure1-1. Utility-driveresourcecustomerfacingresourcemanagementchallenges Anoptimizationstrategydesignforcloudresourcerentalplanning Therstdesignfocusesoninteractionsbetweentheresourcecustomerandtheresourceprovider,asillustratedinFigure 1-2 .Inparticular,wefocusonoptimalstrategydesignforcloudresourcerentalplanning.CloudcomputingrevolutionizestheuseanddeploymentofITservicesandspurstheemergenceofApplicationServiceProviders(ASPs)whoprovidemanagedapplicationhostingservicesusingcloudresources. 18

PAGE 19

Withtheknowledgeofresourcepricingoptions,amajorissuefacedbyASPsishowtointelligentlyplanresourceusageforacertaintimehorizoninordertominimizerentalcostwhilemeetingtheprojecteddemandschedule.Wefoundthatlittleworkhasbeendevotedtoleverageapplicationelasticity(throughjobspawningandmigration)tolowerresourcerentalcost.What'smoreinterestingisthatresourcepricingcanbedynamic(e.g.,asweseeinAmazon'sspotinstancemarket),makingitmoredifculttochoosethebestresourcerentaloptionsunderuncertaintiesofresourceavailability. Figure1-2. Designrevolution1:smartresourcerentalplanning Therefore,weconductathoroughinvestigationofthisproblem[ 26 ].Usingamixedintegerlinearprogram,wesolvetheoptimalresourcerentalproblemunderdeterministicpricingconstraints.Further,weruntimeseriesanalysisonthecompletehistoryofAmazon'sspotpricevariations,andproposeanalternativesolutionthatappliesastochasticoptimizationapproach(multistagerecourse)totheresourcerentalplanningprobleminordertodealwiththestochasticpricingchallenge.ThroughthoroughinvestigationthatusesAmazon'sEC2marketasacaseofstudy,wedrawtheconclusionthatwecannotcountonpredictionsincethedatacorrelationisweak,andthestochasticoptimizationapproachisquiteeffectiveinhedgingagainstpricinguncertainties.Combiningthedeterministicandstochasticoptimizationapproaches,thisworkpresentsempiricalvaluesforASPstodeploycost-effectiveapplicationservicesincloud. 19

PAGE 20

Anefcientandfairresourcetradingframeworkforcommunitycloud Theseconddesigntakesonestepfurther,presentinganefcientandfairresourcetradingframeworkforacommunity-basedcloudcomputingenvironment,asdepictedinFigure 1-3 .Weproposeasetofmultitenancynegotiationprotocolstofacilitateresourcetradingactivities.Inamulti-tenantenvironment,itiscriticaltosatisfydifferentuserinterestsinafair,manageable,andproductiveway.Myresearchonthistopic[ 27 ]adoptedamulti-agentapproachthat:(1)modelstenantsasutility-driven,intellectualindividuals;(2)quantiesallocationbenetandlossusingwell-designedvaluationfunctions;and(3)layaresourcetradingframeworkthatevolvestowardsbetterallocationstatewhenagentsonlymakeself-benettradedecisions. Figure1-3. Designrevolution2:efcientandfairresourcetrading Inspiredbypreviousstudiesinarticialintelligence,wepresentauniedmodelthatusesadirectedhypergraphtosimultaneouslycaptureresourceallocationefciencyandunbalanceamongstagents.Whenbudgetconstraintpresents,weproposeasetofheuristicalgorithmsthatworkondistributedenvironmentandguideagentstospontaneouslytraderesources.Throughtheoreticalanalysis,weshowthatthesetradingactivitieswillimprovetheresourceallocationwithinthecommunitycloud.Thisresourcetradingframeworkisdesignedtoenhanceresourcemanagementinahighly 20

PAGE 21

collaborativeresourcesharingenvironment.Forexample,enablingresourcetradinginpopularcloudresourcemarketsuchasAmazonEC2willgrantcustomerswithsubleasingrights,makingitmoreattractivetocloudresourcecustomerswishingamorecost-effectiveresourceprovisioningsolution.Acloudmiddlewareforscalableresourcesharing Finally,wedesignCloudBay[ 28 ],acloud-basedmiddlewareplatformforcross-domainresourcesharing.ThisdesignaimsatprovidingaresourcetradingplatformthatletpeoplerentandleasecomputationalresourcesjustliketheybuyandselltheircommoncommoditiesoneBay.Itfeaturesaexiblearchitectureinwhichprivatelyownedresourcesformthenetworkedcomputingutilities,asillustratedinFigure 1-4 .Therefore,itismostchallengingtoourresearchoncost-effectiveresourcemanagementstrategydesign.WepresenttheprototypedesignofCloudBayasourinitialeffortsofimplementingHighPerformanceComputing(HPC)-as-a-Service.CloudBaydeploysHPCandcloudservices(e.g.,MPI,Hadoop)ondedicatedhostscontributedbyscienticcommunities.SimilartothePlanetLabproject,itprovidespurpose-builtsoftwarefromgroundup,includinganoperatingsystem.PrepackagedsoftwareservicesareencapsulatedinvirtualcontainerscalledCloudAppliances.TwoimportantservicesareconsideredasessentialbuildingblockstoenableCloudBay'sfunctionalities:(1)autonomicnetworkingservice,whichofferslabor-freeresourcebundlingbasedonavirtualizedP2Pnetworkinglibrary;and(2)preemptivejobschedulingservice,whichusesCondortomanagejobsubmission,checkpoitingandpreemption. WhatdistinguishesCloudBayfromotherresourcetradingplatformsliesinitsabilitytoaccommodatebothquality-sensitiveandcost-sensitiveservicerequests.CloudBay'sserviceschedulingschemeisinspiredbyeBay'stransactionmodel,wherecustomerscanchoosetobuy-it-noworbidforanitem.Forthisreason,CloudBayisviewedasaneBayofscienticcomputingresourcesthatallowsresearcherstorentorreleaseresourcesinaglobalresearchcloudjustastheybuyandselldailycommoditieson 21

PAGE 22

Figure1-4. Designrevolution3:exibleandopenresourcetradingmarket eBay.Dependingonuser'swillingnesstopay(WTP),CloudBayprioritizesuserservicerequestsandschedulethemaccordingly.Weproposeanovelauctionform[ 29 ]thatusesproxyiterativebiddingtoachieveefcientauctionoutcome.TheproposedutilitymodelmaintainsincentivecompatibilityasinVCGauction,andiscomputationallytractableinwinnerdetermination.WevalidatetheprototypeofCloudBayacrossavarietyofopenandprivatecloudplatforms,includinguniversityclusters,FutureGrid,andAmazonEC2.ResultsshowthatCloudBaymakesgooduseofidleresourcesandprovideseasy-to-accessresourcestoresearchersinafairmanner. 1.4Organization Thisdissertationorganizesasfollows.InChapter 2 ,weproposeanne-grainedresourcerentalplanningdesignforbothat-rateandspotmarkets.InChapter 3 ,weformulatearesourcetradingframeworkforacommunity-basedcloudcomputingenvironment.Chapter 4 describesourproof-of-conceptdesignandimplementationofaresourcesharingmarketplace.Finally,inChapter 5 ,wesummarizeourresearchndingsanddiscussfutureresearchdirections. 22

PAGE 23

CHAPTER2FINE-GRAINEDRESOURCERENTALPLANNING 2.1Background Theemergingcloudcomputingmodel,withitsvirtuallyinniteresourcesandelasticity,liberatesorganizationsfromtheexpensiveinfrastructureinvestment.Asaresult,moreandmoreApplicationServiceProviders(ASPs)recognizetheseparationbetweentheactualapplicationandtheinfrastructurenecessarytorunit,andbegintodeployapplicationsonresourcesrentedfrominfrastructureproviders.Forexample,foursquareusesAmazonEC2toperformanalyticsacrossmorethan5milliondailycheckins,andsaves53%incostswhilemaintainingscalabilityneeds[ 30 ].AccordingtoarecentforecastbyGartner[ 31 ],Software-as-a-ServiceandCloud-basedbusinessapplicationserviceswillgrowfrom$13.4billionin2011to$32.2billionin2016. Byadoptingcloudcomputing,amajorissuefacedbytheASPsishowtominimizetheresourcerentalcostwhilemeetingtheirapplicationservicedemand.Signicantresearcheffortshavebeendirectedtowarddevelopingoptimalresourceprovisioningschemestomeetservicerequirements(avoidthecostduetoover-provisioningandthepenaltyduetounder-provisioning)[ 32 36 ].Theseworks,althoughoffereffectiveresourceprovisioningcontrolsinresponsewithvaryingworkload,arestillcoarse-grainedintermsofexploringapplicationelasticitywithregardtodifferentresourcepricingoptions.Webelievethatane-grainedresourcerentalplanningschemeisneededtofurtherreduceASPs'operationalcost.Specically,weproposesane-grainedcontrolschemetoregulatetherentalactivitiesonatime-slottedbasis,exploringhourlychargingrateofvarioustypesofresources,inordertomeettheprojectedservicedemandandminimizeresourcerentalcostatthesametime.Complementarytopriorresourcescalingsolutions,thisworkopensuptremendousnewresearchopportunities,coinedasapplicationscaling,aimedtodevelopthemosteconomicresourcerentalplanwithoutcompromisingtheservice-levelagreement. 23

PAGE 24

Yetanotherobstacleliesintheuncertaintyofcomputationalresourcepricing.Thischallengeisencounteredinthespotresourcemarketemergedinrecentyears.Onaspotresourcemarket,dependingontheresourcesupplyanddemandlevel,theunitpriceofacomputationalinstanceisuctuatingallthetime.Forexample,inDecember2009,Amazonlauncheditsspotinstanceserviceandimplementedanauctionmechanismtodetermineinstancepricing.Sincespotinstancesleverageidlecyclesfromtheregularon-demandserverpool,theyareauctionedoffatapricemuchlowerthanthatoftheregularon-demandinstancemostofthetime.Asaresult,thisreal-timebiddingmarkethasattractedmanyASPswhowishtoincreaseservercapacityatlowcost,andthereisagrowingresearchinterestinutilizingspotinstanceservice.However,modelingandanalyzingspotinstancepricingislargelyneglectedduetothelackofinformation.Webelievethatourstudyishelpfultounderstandspotpricing,andmoreimportantly,howtoimproveresourceutilizationunderspotpricing. Inthischapter,werstdevelopane-grainedoptimalresourcerentalplanningmodelforelasticapplicationscalinginacloudresourcemarket.Inparticular,givenaforecastdemandschedule,theASPneedstoperiodicallyreviewtherunningprogressofthedeployedserviceandmakeoptimaljoballocationaswellasresourcerentaldecisionssoasnottowastemoneyonexcessivecomputation,storageordatatransfer.Ourrstcontributionistheformulationofadeterministicplanningmodelthatapproximatesresourcerentaldecisionmakingprocessovercertainplanninghorizon.Thesolutiontothismodelservesasaguidetomakecost-effectiveresourcerentaldecisionsinrealtime.Weshowthatourplanningmodelisespeciallyusefulforhigh-costVirtualMachine(VM)classes.ThisisbecausecostsavingfromourmodelprimarilycomesfromeliminatingunnecessaryjobrunningbydecreasingVMrentalfrequency.Fromthisperspective,ourmodelformulationisalignedwiththedynamiclot-sizingmodelcommonlyencounteredintheeldofproductionplanning. 24

PAGE 25

Thesecondcontributionoftheworkpresentedhereisthatweanalyzeandsolvethene-grainedcloudresourcerentalplanningproblemunderthepricinguncertaintychallenge.Inparticular,twopossiblesolutionsarejointlyexploredinthischapter.WesystematicallyanalyzethepredictabilityofAmazonEC2spotpricingandshowthatpredictioncannotprovideadequateapproximationtobeusedindeterministicplanningmodel.Forthepurposeofcomparison,weproposeamultistageresourcemodelforstochasticresourcerentalplanning.Thismodeldecomposesthestochasticprocessofdecisionmakingundervaryingpriceintosequentialdecisionmakingprocesseswiththeaidofpricedistributionatvariousstages.Assuch,thestochasticoptimizationproblemistransformedintoalarge-scaledeterministicoptimizationproblem.Wefurtherpresentapolynomial-timesolutiontotheproblembasedon[ 37 ].Throughsimulations,wehaveshownthatthestochasticplanningapproachismorecost-effectivethanpredictiveplanning. Insummary,thekeycontributionsofthischapterarelistedasfollows: Ane-grainedplanningapproachforcost-effectiveresourcerentalincloudresourcemarket. AMILPformulationfordeterminingtheoptimalresourcerentaloveraxedplanninghorizon. Asystematictime-seriesanalysisofthepredictabilityofspotpricingusingrealpricetracesofAmazon'sspotinstanceservice. Astochasticoptimizationsolutiontodealwithspotpriceuncertaintyinresourcerentalplanning. Therestofthechapterisorganizedasfollows.Section 2.2 surveystherelatedwork.InSection 2.3 ,wepresentthesystemmodel,provideadeterministicplanningmodelfortheproblem,giveoutanoptimalsolution,andevaluatetheperformanceofthedeterministicpricingresourceplanningapproach.Finally,inSection 2.4 ,weanalyzethepredictabilityofAmazonEC2spotpricingusingtime-seriesanalysistechniques, 25

PAGE 26

proposeastochasticoptimizationmodeltosolvetherentalplanningproblem,andperformsimulationstoevaluatetheperformanceofoursolution. 2.2PriorWork Nowadays,awidevarietyofcomputationalanddataintensiveapplicationsutilizecloudtotheirbenet.Therefore,itbecomesimperativetounderstandthecost-benetofrunningresource-demandingapplicationsincloudinordertomakecost-effectiveresourcerentaldecisions.Comparedwithrunningapplicationsonconventionalplatformssuchasgrid,cloudeliminatesup-frontsetupandoperationalcostfordistributedresources.However,movingandstoringlargedatasetincloudincursignicantcostcomparabletothecomputingcost[ 18 ],andeffortshavebeenmadetomitigatesuchcostincloud[ 38 39 ].Inthischapter,wepresentaplanningmodelthatoptimizesresourceusageforelasticapplicationswithcomprehensivecostconsiderations. Findinganoptimalresourceutilizationstrategyischallengingforbothcloudinfrastructureprovidersandapplicationserviceproviderswhorelyonrentedinfrastructure.Fromtheperspectiveofthecloudinfrastructureprovider,thechallengeishowtoreducetheoperationalcostandmaximizeleasingrevenue.Manyexistingresearchhasfocusedonthisaspect.ThegeneralproblemofminimizingresourceallocationcostwhilemeetingjobdemandisNP-hard[ 40 ].Resourceschedulingfortheemergingspotmarketwasproposedin[ 41 ].Theproposedframeworkincludes:(1)amarketanalyzerperiodicallyforecastingsupplyanddemand,(2)acapacityplannerdeterminingthespotpricebasedontheforecastresults,and(3)aVMschedulermaximizingtherevenuebysolvingaNLIPmodelfortheschedulingproblem.Fromtheperspectiveoftheapplicationserviceproviders,thechallengebecomeshowtominimizeresourcerentalcostwhilemeetingservicedemandfromcustomers.Manyresourceplanningschemesrelyonpredictiveworkloadassessment[ 35 42 ].Ourworktakesonestepfurtherthatpresentsane-grainedplanningcontrolmodelbasedontheforecastdemand.The 26

PAGE 27

proposedmodeltakesfullconsiderationofvariousresourcetypesandtheirassociatedcostswithinacloudresourcemarket,andstrivestondtheoptimaltradeoffpointinresourcerentalallocation. Thestochasticplanningmodelproposedinthischapterdealswiththepriceuncertaintyinthespotresourcemarket.Suchaspotmarketiseitherformedbymultipleresourceproviders[ 43 ]orbyasingleresourceprovider.AmazonEC2spotmarketisthemostrepresentativeexamplethatattractssignicantresearchattentions.Researchersareinterestedinutilizingspotinstancestotemporarilyaddcapacitytodedicatedclustersduringpeakperiods[ 44 ].Thebiggestconcernforutilizingspotinstancesisthatitishardtoguaranteeresourceavailability.Recentworks[ 45 46 ]addressedthisproblemusingstatisticalanalysis.Notably,Ben-Yehudaetal.[ 47 ]reverselyengineeredspotpricesbyconstructingasparecapacitypricingmodelbasedonexistingpricetraces.However,theeffectivenessoftheseapproachesisstillunclearduetounsubstantiatedassumptionsonAmazonEC2spotservice.Inthischapter,wetakeEC2asacasestudyandtargetsatageneralspotresourcemarketwherepricesaremarket-drivenandusersbidaccordingtotheirtruevaluations(simple-mindedassumption).Themostrelevantworkstothisstudyarepresentedin[ 48 ]and[ 49 ].In[ 48 ],theauthorspresentedanoptimalVMplacementalgorithmthatminimizesthecostofresourceprovisioninginamultiplecloudprovidersenvironment,andin[ 49 ],theauthorsproposedaprot-awaredynamicbiddingalgorithmtooptimizeASP'sprotsinEC2spotmarket.Ourwork'sapplicationscenarioisdifferentfrom[ 48 ],andwedevelopourmodelbasedonrealisticapplicationandpricetraces.Comparingwith[ 49 ],ourapproachproposesadifferentmodelthattakesstorageandnetworktransfercostintoaccountinadditiontocomputationalinstancebidding. 2.3Fine-GrainedResourceRentalPlanning Resourcerentalplanningentailstheacquisitionandallocationofcomputationalandstorageresourcestoapplicationssoastosatisfydemandoveraspeciedtime 27

PAGE 28

horizon.Ane-grainedcontrolschemeisproposedtooptimizerentaldecisiononatimeslottedbasis.Inthissection,wepresentthesystemmodel,formulatethene-grainedrentalplanningprobleminthecontextofthismodel,andexaminessolutionstosolvetheproblem. 2.3.1SystemModel WepresentascenariowhereanASPofferssomecomputationalanddataintensiveapplicationservices(exampleservicesaredatavisualization,dataanalytics,dataindexing,etc...)tocustomersoveranetwork.Insteadofusinglocalresources,thetasksofcomputationanddatastoragearecompletelyoutsourcedtoasharedresourcepooloperatedbysomeInfrastructure-as-a-Service(IaaS)provider(s),asshowninFigure 2-1 .Thedepictedsystemmodelresemblesabroadrangeofpracticalexamplesintoday'scloud-basedservicemarket.Forinstance,theASPcouldbemappedtosomeSoftware-as-a-Serviceproviderwhooffersroutinedataanalyticstoitscustomerrms,orsomeacademicinstitutionwhoprovidesscienticdatavisualizationservicestothegeneralpublic. Figure2-1. Systemmodelforthecloud-basedresourcerentalplanningproblem AsillustratedinFigure 2-1 ,resourceusageincursmonetarycosttotheASPinvariousforms.Rentalactivitiesarechargedthroughoutthelifecycleofthedeployedserviceasfollows.First,inputdatanecessarytoruntheprogramisimportedintothecloudfromthelocalstoragemedia,introducingnetworktransfer-incost.Next,anumber 28

PAGE 29

ofVMinstances(herebyreferredasVirtualServers,orVSforshort)arelaunchedtoperformdataprocessingtasks.EachofthemcostscertainmoneydependingonbothVSunitpriceandrentalduration.Afterthecomputationaljobsarecompleted,resultsandlogsaresavedtocloudstorage,andmaylaterbedumpedintolocalpersistentstorage.Manyoftenthedatasizeislarge(e.g.,imagesorvideos)andincurssignicantstorageandnetworktransfer-outcostfortheASP.Thestoragecostmayalsoapplytoinputdataalreadyfetchedintothecloudbutnotprocessedyet.Finally,highperformanceapplicationsoftenfeaturetremendousI/OrequirementsandsomeresourceproviderwillchargeforI/Oactivities.Whenperformingresourcerentalplanning,AnASPneedstoconsiderallcostsdescribedaboveinordertounderstandthecost-benetratioofpossiblechoices. 2.3.2Motivationforne-grainedresourcerentalplanning ConsideringanASPrentsanumberofvirtualserversfromthecloudresourcemarketforthepurposeofdataprocessingandpresentation.Inordertoachieveresourceauto-scalingforefcientresourceutilization,therststepistoidentifytheclientworkloadpatternandbuildaforecastdemandscheduleforeachVS.Whilethereareanumberofpublicationsaddressingthisproblem,e.g.,[ 50 52 ],theproblemofworkloadforecastingisoutofthescope.OncetheforecastdemandpatternisbuiltupforeachVS,theASPisabletoscheduleresourcerentalthroughjobaddition,replication,migrationandremoval.Itisbettertoperformresourcerentalplanningonaper-VSbasisratherthanschedulingforthewholeVScluster.Thisisbecauseane-grainedrentalcontrolallowsforbetterexibilityinallocatingresourcesmeetingservicedemand.AnexampledemandforecastofasingleVSisdepictedinFigure 2-2 .ThetopdashlinerepresentstheVS'smaximumloadhandlingcapacity.IftheVSstrictlyfollowsajobprocessingscheduleinaccordancewiththeforecastworkloadpattern(redsteps),computingcyclesarewasted.Forexample,theASPpaysfullrentfortheVSbetween2pmand3pm,butdoesnotfullyutilizetheserver'scomputingcapability 29

PAGE 30

shownintheshadedarea.Inthischapter,weproposetolettheASPconsolidatefuturecomputingjobsincurrentrentalslot,giventhattransferandstoragecostless.Ourapproachintroducesoperationalandmanagementsciencetechniquesintocloudcomputingandachievesbettercost-effectivenesscomparedtoon-demandresourceprovisioning. Figure2-2. ExampleforecastdemandscheduleforaVS. 2.3.3Optimizingplanningfordeterministicinstancepricing Therstresourcerentalplanningmodeltargetsatanon-demandresourcemarketwhereeachVScostsaxedamountofmoney.EachVSbelongstoaspecicVStypespecifyingthehardwareconguration.Weassumetheapplicationstobeelasticandcomposedofjobseasytoscalegracefullyandautomatically.Forexample,applicationsprocessingBags-of-Tasks(nojobdependencies).Similarto[ 53 ],weareinterestedinself-awaresolutionsthatcanplanresourceusageofcloudapplicationsundervariouspricing.TheplanninghorizonTisdividedintoxedtimeslotst=1,...,T.Wereferthestartofeachtimeslotasadecisionpoint.Ateachdecisionpoint,arentaloperationisperformedtoaccessthemostcost-effectiveresourceavailablefortheapplication. LetTbethesetofdecisionpoints.Thegoalofresourcerentalplanningistominimizethetotalrentalcostassociatedwithprocessingtheforecastworkloadover 30

PAGE 31

theplanninghorizonT.Inordertoaccomplishthisgoal,threesetsofvariablesareintroducedtoidentifytherentaldecisionstobemadeateachdecisionpoint.Therstsetofvariables,i,t,denotestheamountofdatatobeprocessedbytheapplicationduringtimeslottonatype-iVS.Next,attheendofslott,weusei,ttorepresentthedesiredstoragespaceforholdingthedata.Finally,letbinarydecisionvariablestdenoteifpoweringonatype-iVSisneededattimeslott.i,tandi,tspecifyhowtomakeuseofthecomputationalresourcestocontroltheapplicationprogress,whilei,tdeterminestheamountofstorageresourcestoreserveinacloudmarket.Ifallthesevariablesaredetermined,acontrolpolicyisformedtoguidetherentalactivitiesinthecloudmarketforoptimalresourceutilization. Anumberofcostparametersareassociatedwithourresourcerentaloptimizationproblem.Specically,therentalcost(processingcost)fortype-iVSintimeslottisCp(i,t),andthestoragerentalcostperdataunitforslottisCs(t).AspresentedearlierinSection 2.3.1 ,manyIaaSproviderschargenontrivialcostfordatatransferacrossthecloudboundary.Foreachtimeslott,letCio(t)betheI/Ocostfordatatransferfromandtothecloudstorage,andletC+f(t)andC)]TJ /F8 7.97 Tf -1.41 -8.28 Td[(fbethecostfortransferringintoandoutofthecloud,respectively.Inadditiontothecostparameters,weassumethecustomer'sdemandfunctionisD(),whereD(i,t)denotestheforecastworkloaddemandproleforatype-iVSinslott.Forreaders'reference,wesummarizethenotationusedthroughoutthechapterinTable 2-1 Withalltheprerequisites,weformulatetherentalpaymentfunctionfollowingalinearcostmodel.Morespecically,therentalcostislinearlyproportionaltotheconsumedresourceamountaswellastothedurationoftherentalperiod.Naturally,ourobjectivefunctionaimsatminimizingtherentalcostforeachtype-iVSovertheentireplanninghorizonT.Ateachdecisionpoint,axedrentalcostCp(i,t)ischargediftheASPdecidestorentonetype-iVS(i,t=1).Now,giventhepresenceofthiscomputationalresourcecost,theASPmaychoosetomakefulluseoftheVScapacitysoastomeet 31

PAGE 32

Table2-1. Summaryofnotations Variables i,toutputdatasizegeneratedbyonetype-iVSintimeslott i,tstoragespacefordataproducedbyonetype-iVSattheendofslott i,tbinarydecisionvariablerepresentingrentaldecisionofonetype-iVSintimeslott Parameters TSetoftimeslotsISetofVStypes Cp(i,t)VSrentalcost(pertype-iVSslotduration)Cs(t)Storagecost(perdataunitslotduration)Cio(t)I/Ooperationcost(perdataunitslotduration)C+f(t)Networktransfer-incost(perdataunitslotduration)C)]TJ /F8 7.97 Tf -1.41 -8.28 Td[(f(t)Networktransfer-outcost(perdataunitslotduration) D(i,t)Demandtobesatisedforonetype-iVSattheendofslott P(i)Averagebottleneckresourceconsumptionrate(perdataunitgenerated)foronetype-iVS Q(i,t)Bottleneckresourceavailableforonetype-iVSintimeslott iAverageoutput-to-inputratioforonetype-iVS(applicationspecic) theforecastworkloaddemandoveranumberoffuturetimeslots.However,doingsowillincreasethestorageandI/Ocostasmoreworkloadisprocessedearlierintime.Assuch,theplanningproblemarisesastheASPneedstocarefullytradeoffthecomputationalrentalcostversusstorageanddatamigrationcosts.Inproductionplanning,similarproblemsarerecognizedasthedynamiclot-sizingproblem.Thesolutiontothedynamiclot-sizingproblemdeterminestheoptimalfrequencyofsetupssoastominimizethetotalcostwithintheresourceanddemandconstraints.Inthecontextofcloudcomputing,weformulatetheplanningproblemunderxedresourcepricingastheDeterministicResourceRentalPlanning(DRRP)problem.DRRPmodelscloudresourcerentalonaper-VSbasis,formingane-grainedcontrolpolicyforrentalplanning.Thecompletemodelformulationisgivenasfollows. 32

PAGE 33

minXt2T(C+f(t)ii,t+(Cs(t)+Cio(t))i,t+C)]TJ /F8 7.97 Tf -1.42 -8.28 Td[(f(t)D(i,t)+Cp(i,t)i,t) (2)s.t.i,t)]TJ /F9 7.97 Tf 6.59 0 Td[(1+i,t)]TJ /F7 11.955 Tf 11.95 0 Td[(i,t=D(i,t),i2I,t2T (2)P(i)i,tQ(i,t),i2I,t2T (2)i,tBi,t,i2I,t2T (2)i,0=",i2I (2)i,t,i,t2R+,i2I,t2T (2)i,t2f0,1g,i2I,t2T (2) NotethattheobjectivefunctiondoesnottakeI/Oandstoragecostforinputdataintoaccount.Thisisbecauseweassumethatinputdataisbroughtintocloudontheytocompletethecomputationaljobs.Anotheroptionistocopyallinputdataonceandstorethemincloudthroughouttheentireplanninghorizon.Thedecisiononwhichoptionisbetterdependsonthedataaccesspatternandthedurationofplanninghorizon.Here,wesimplyassumethatinputdataistransfer-on-demandtosimplifythepresentation. Constraint( 2 )isanalogoustotheinventorybalanceconstraintinthedynamiclot-sizingproblem.Itsimplyspeciesthatworkloaddemandshouldbemetatanytimeslot.Atslott,thedatastoredattheprevioustimesloti,t)]TJ /F9 7.97 Tf 6.59 0 Td[(1,andthedatageneratedinthecurrentsloti,t,arecombinedtogethertoservetheforecastdemandproleemergedinthecurrenttimeslot,i.e.,i,t)]TJ /F9 7.97 Tf 6.58 0 Td[(1+i,tD(i,t).Theoverprovisioningamountbecomesthestorageamounti,tattheendoft.Theinitialstoragespaceissettobesomeconstant"inconstraint( 2 ),dependingonthespecicplanningscenario.Next,letP(i)betheaveragebottleneckresourceconsumptionrateforonetype-iVS,andlet 33

PAGE 34

Q(i,t)denotethebottleneckresourceavailableforonetype-iVSint,constraint( 2 )ensuresthattheworkloadprocessingratedoesnotsaturatetheavailablebottleneckresource. Constraint( 2 )isoftenreferredtoastheforcingconstraint.Itstatesthattherewillbenodatageneratedintifnorentaldecisionismade(i,t=0).Bissettobeaverylargeconstantthatexceedsthemaximumpossiblevalueofi,t.Finally,constraints( 2 )and( 2 )specifydomainsofthevariables. 2.3.4Solutiontodeterministicpricingresourcerentalplanning Inthissection,webrieydiscussaboutalgorithmsthatndanoptimalplanforobjectivefunction( 2 )subjecttoconstraints( 2 )to( 2 ).TheformulationisaMixedIntegerLinearProgram(MILP)thatisNP-completeingeneral.Whentheintegerconstraintisnotconsidered,theproblemcanbetransformedtotheminimumcostownetworkproblem[ 54 ].WeshowthetransformationtoownetworkinFigure 2-3 Figure2-3. Flownetworkofthedeterministicpricingresourcerentalplanningproblem AsshowninFigure 2-3 ,thenetworkhasT+1vertexes.Vertex0isthesourceofalldatageneration,andishereafterreferredtoasthesourcenode.Thet-thvertex(t=1,...,T)representsthet-thplanningtimeslot.Thesevertexesarereferredtoassinknodes.Directededgesarecategorizedintotwotypes.Thoseconnectingthesourcenodewithsinknodesarelabeledastheproductionarcs,andthoseconnectingtwosuccessivesinknodesarelabeledastheinventorycarryarcs.WealsoassignthesupplyvalueofPTt=1D(i,t)tothesourcenode,andthedemandvalueof)]TJ /F6 11.955 Tf 9.3 0 Td[(D(i,t)toeachsinknodet=1,...,T.Byestablishingthisownetwork,theinventorybalance 34

PAGE 35

constraint( 2 )istranslatedtotheowbalanceconstraintintheminimumcostowproblem.Theresourceconstraint( 2 )isenforcedbyimposingcapacitiesontheproductionandinventorycarryarcsineachperiod.Thecostsassociatedwitheacharcarenon-negative.Withlinearobjectivefunction,theresultingminimumcostowproblemscanbesolvedwiththeaidofresidualnetworks.Effectivealgorithmsexistforsolvingthetransformedownetwork,e.g.,cycle-cancelingalgorithm,successiveshortestpathalgorithm,andprimal-dualalgorithm.Wereferreadersto[ 54 ]fordetails. Whenintroducingthebinaryintegervariableandapplyingtheforcingconstraint 2 ,wecansolvetheproposedMILPwithstandardtechniques,e.g.,usingBranch-and-Bound(B&B)whichusessearchtreeforndingtheoptimalschedule.Inourperformanceevaluation,weusethestandardsolverofferedbyCPLEXTM[ 55 ]tosolveDRRP. 2.3.5Evaluationofdeterministicpricingresourcerentalplanning WeconsiderthreeVSclassesI=fc1.medium,m1.large,m1.xlargeg,andperformsimulationstoevaluatethesolutiontoDRRPbasedonrealisticpricingandapplication-usagescenarios.Therentalplanningdecisionsaremadeinanhourlybasis,spanningoverdailyplanninghorizon(24hours).TheMILPformulationissolvedusingtheownetworkbasedapproachpresentedinSection 2.3.4 ,andveriedbytheCPLEXTM[ 55 ]solverintegratedinAIMMS3.11[ 56 ].WesamplethehourlydataprocessingdemandfromanormaldistributionN(0.4,0.2)(expressedintheunitofGigaByte).Itisassumedthatthesoftwarerequiredbytheapplicationserviceshasbeenconguredonvirtualserversrentedfromthecloudmarket.Therefore,wedonottaketheinitialenvironmentpreparationintoaccount. 35

PAGE 36

ThecostparametersusedinmodelformulationsaresetaccordingtoAmazon'sEC2on-demandpricingpolicy1.Specically,thehourlyon-demandVSrentalcostsaref$0.2,$0.4,$0.8gforthethreeVSclasses.UsingElasticBlockStore(EBS),thestoragecostis$0.1perGB/month,and0.1permillionI/Ooperations.Theinboundandoutboundtransfercostis$0.1and$0.17perGB.Inordertoproviderealisticparameterestimatesinourproposedmodels,werefertoarecentpaper[ 20 ]studyingthecostandperformanceofrunningscienticworkowapplicationsonAmazonEC2.Basedonthe3-yearcostofamosaicservice(generatedbyanastronomicalapplicationMontage,see[ 57 ]fordetails)hostedonEC2,wenormalizetheI/Ocostto$0.2perGB,andsetito0.5foralli2I.Accordingtothedataprovidedin[ 20 ](runtime,inputandoutputvolume,etc.),thevirtualserversareabletooffersufcientresourcesforservingtherandomlygenerateddemand.Therefore,constraint( 2 )inDRRPisomitted. Werstshowthecost-savingadvantageofourproposedsolutionoverresourcerentalwithoutplanning.TheresultsareshownattheuppersideofFigure 2-4 .Inoursimulation,per-VScostsoverdailyplanninghorizonforbothschemesarecompared.Fromtheresults,wecanobservethatcostderivedfromsolvingDRRPissignicantlylowerthanthatoftheno-planningsolution.AsVSbecomesmorepowerful,thecostreductionbecomesmoresignicant.Especially,thecostreductionforVSofclassm1.xlargeachievesnearly50%dropoff.Thisisbecausecomparedtotheno-planningsolution,thecostreductionprimarilycomesfromthesavingofcomputationalcost(virtualserversareturnedoffincloudwhendemandissatisedbycacheddataincloudstorage).Therefore,moresavingisexpectedforhigh-costVSclasses.ThecoststructureforeachVSclassispresentedinthelowersideofFigure 2-4 .Theproportion 1AmazonhasdeclaredlowerpricingforEC2whenwepreparedthismanuscript.Sinceoursimulationisbasedon[ 20 ],thestudypresentedhereisbynomeansup-to-date,butservesasarepresentativecaseofstudy. 36

PAGE 37

ofcomputationalcostisrelativelystableinallthreeclasses.However,weobservethatmoremoneyisspentonI/OandstorageasVSbecomesmorepowerful.ThisisbecausemorepowerfulVSincurshigherVSrentalcosteachtimetherentaldecisionismade.Asaresult,anASPtendstoutilizecachingmoreoftentoservethecustomerdemandandrentsVSlessfrequently. Figure2-4. CostcomparisonforDRRPandresourcerentalwithoutplanning Next,weconductasensitivityanalysistothesolutionforDRRPandplottheresultsinFigure 2-5 .WedenecostratioasthecostofrentalplanningbasedonDRRPtothecostofresourcerentalwithoutplanning.Thebaseratio(67%)issettothecostratioofVSclassm1.largecalculatedinthelastsimulation.Fromthisbaseratio,werstvarytheweightsofI/Oandcomputationalcostgradually.Inonedirection,wekeeptheI/Ocostxedandincreasethecomputationalcostwithaxedstepof0.1,andthenweincreasetheI/Ocostintheotherdirectionsimilarly.TheresultshowedintheleftpartofFigure 2-5 clearlydemonstratethatthecostreductionachievedbysolvingDRRPbecomesmoresalientforexpensivecomputationalresources.Thisconclusionconrms 37

PAGE 38

theanalysiswepreviouslyprovided.TheimpactofdemandisinvestigatedintherightpartofFigure 2-5 .Inparticular,wealterthemeanofthedemanddistributionfrom0.2to1.6GB/hour.Asmoredemandisgeneratedforservices,thecomputationalresourcestendtobekeptbusyallthetimebecausethecurrentstoragecannotmeetthedemand.Asaresult,costreductionisnotnoticeableforheavyservicedemand. Figure2-5. SensitivityanalysisforDRRP 2.4DealingwithSpotPricingUncertaintyinCloud Inthissection,weextendtheresourcerentalplanningmodelbyincludingcostuncertainty.SuchuncertaintyisintroducedbymanyIaaSproviderswhoofferspotpricingoptionforidlecomputationalresources.Examplemarketscanbefoundin[ 58 ]and[ 59 ].Thepriceuctuationofspotresourcesovertimecreatestimeseriesdataforanalysis.UsingAmazon'sspotmarketasacaseofstudy,wetaketworoutestoattacktheresourcerentalplanningproblemwithspotpricinguncertainty.First,weapplytimeseriesforecastingtospotpricehistorycrawledfrom[ 60 ].Thepredictionresultsarefedintoourdeterministicplanningmodel(hereafterlabeledaspredictiveplaning).Next,weproposeanalternativeapproachthatleveragesthepricedistributioninformation(hereafterlabeledasstochasticplanning).Adynamicprogrammingalgorithmisalsopresentedtosolvethestochasticoptimizationproblem.Wecomparethetwoapproachesintheendofthissection,andconcludethatthestochasticplanningapproachsavesmorecostthanthepredictiveplanningapproach. 38

PAGE 39

Beforeweproceed,afewassumptionsneedtobeclaried.First,weassumethatASPswillbidtruthfullyinthespotresourceacquisitionprocess.Thisassumptionisinlinewiththeassumptionmadein[ 46 ].Withthisassumption,anASPwillnotbidstrategically.Infact,whetherstrategicbiddingishelpfultoachievesomedesiredlevelofresourceavailabilityiscontroversial.Ontheonehand,byexploitingpriorpricehistory,itisviabletooptimizebiddingusingprobabilisticmodelsforasinglebidder[ 45 ].Ontheotherhand,oneshouldalsoconsiderbiddingstrategiesofotherbiddersbeforemakingdecisions.Fromagametheoreticperspective,intentionallyoverbiddingorunderbiddingisnotadominantstrategy(e.g.,ifeverybidderoverbids,thespotpriceincreases,onlybenetingtheIaaSprovider).Second,anout-of-bideventoccurswhenanASP'sbidpriceislowerthanthespotprice.Ifsuchaneventhappens,theASPneedstorentthedesirednumberofvirtualserversfromtheregularon-demandresourcemarketinordertomeetthedemandrequirement. 2.4.1PredictiveplanninginAmazonspotmarket 2.4.1.1Introduction Inthischapter,weuseAmazon'sspotinstancemarketasacaseofstudyforpricepredictionandcostoptimization.LaunchedonDecember2009,Amazon'sspotinstancemarketoffersanewwaytopurchaseEC2instancesinadiscountrate.Itallowscloudcustomerstobidonunusedservercapacityandusethemaslongasthebidexceedsthecurrentspotprice,whichisupdatedperiodicallybasedonsupplyanddemand.Paymentinspotinstanceauctionisuniform,i.e.,allwinnersintheauctionwillpayaper-unitpriceequaltothelowestwinningbid(a.k.athespotprice).Whilerunningspotinstancessaveshugecost(typicallyover60%accordingto[ 61 ]),italsointroducessignicantuncertaintyforresourceavailability.Asaresult,previousresourcerentalplanningmodelbasedondeterministicresourcepricingdoesnotapply. Ifoneisabletoforecastspotpriceswithrelativelyhighaccuracy,thenthesepredictionscanbeusedtoinstantiatetheDRRPmodelpresentedinSection 2.3.3 39

PAGE 40

toobtainanear-optimalsolution.However,performingforecastingischallengingforcustomersbecausetheydonotpossesstheglobalinformationofsupplyanddemandasAmazondoes.In[ 41 ],theauthorsattemptedtopredictcustomerdemandfromtheviewofanIaaSprovider.Theyproposedasimpleauto-regressionmodelforpredictionbutnopredictionresultswerereportedduetothelackofrealisticdemandinformation.AnotherstudyonthepredictabilityofAmazon'sspotinstancepricewaspresentedin[ 46 ].Theirworkfocusedonachievingavailabilityguaranteewithspotinstances,andusedaquantilefunctionoftheapproximatenormaldistributiontopredictwhentheautocorrelationofcurrentandpastpriceisweak.Whentheautocorrelationisstrong,asimplelinearregressionpredictionmodelwasadopted.However,wefoundthatsuchanapproximationisinaccurateinsometestcasesthatcannotbetakenasagenericapproach.Inthissection,wewillassessthepredictabilityofspotinstancepricebasedonastatisticalapproach(ARIMA),andestimatethepredictionerrorsusingempiricaldataset. 2.4.1.2Methodology Wehavecollectedthehistoricaldata(publishedon[ 60 ])forspotpricevariationfromFebruary1,2010toJune22,2011.Thedatasourcerepresentsspotpricevariationsforlinuxinstancesinus-east-1region.Thedatasizeisapproximately100Krecords.Therststepinourinvestigationistoidentifytheoutliersintheoriginaldataset.Figure 2-6 plotsthebox-and-whiskerdiagramforthespotpricedatasetcorrespondingto4differentlinuxVMclasses.Theoutliersareidentiedasthosepointsbeyondthewhiskers(1.5IQR(interquartilerange)oftheupperquartile).WecanseethatmoreoutlierspresentinmorepowerfulVMclass,indicatingincreasingpricedynamicsinmorepowerfultypes.However,evenforthemostpowerfulinstance(c1.xlarge),thenumberofoutliersstillcontributesatrivialamounttotheoveralldataset(<3%). Havingtrimmedouttheoutliers,westillcannotapplystandardtimeseriesanalysisbecausethederiveddatasetisunequallyspacedwithinconsistentsamplingintervals, 40

PAGE 41

Figure2-6. Box-and-Whiskerdiagramforspotpricehistory asshowninFigure 2-7 .ItplotsthedailypriceupdatefrequencyforVSofclasslinux-c1-medium.Forthatreason,wefurtherconvertthedataintoequallyspacedtimeseriesdatawitharegularupdatefrequencyof24timesperday.Atthestartofeachhour,thespotpriceissettobethemostrecentupdatedpriceinthelasthour.Ifnoupdateappearsinthelasthour,thespotpriceisunchanged. Figure2-7. VariationofdailyspotpriceupdatefrequencyforVMclasslinux-c1-medium Wehaveperformedvariousexperimentsonthisconverteddataset,eachwithdifferenttimescaleofprediction(bothshort-termandlong-term).Duetothespacelimit,weonlyshowarepresentativepredictionresultforinstanceofclasslinux-c1-mediumoveraperiodoftwomonths.Specically,weusethedatarangingin[12/1/2010,1/31/2011]astheestimationdataset,anddatain2/1/2011asthevalidationdataset.In 41

PAGE 42

otherwords,thedatacollectedfromthetwo-monthhistoricalrecordsisusedtoprovidethenext-daypriceforecasting.InFigure 2-8 ,weplotthehistogramanddensityoftheselecteddata.Wealsorandomlygeneratethesamenumberofpointsfromanormaldistributioncharacterizedbythethreemainmeasuresinquantitativestatistics(mean,varianceandstandarddeviation),andplotthecurveinFigure 2-8 forcomparison.ExaminationoftheShapiro-Wilktestresult(omittedhere)veriesthatthepricingdatadoesnottthenormaldistribution. Figure2-8. Histogramplotfortheselectedspotpricehistory(linux-c1-medium) Inordertoidentifypatternsintheselectedseriesandperformprediction,weusetheARIMAapproachdevelopedbyBoxandJenkins[ 62 ],whichretainsgreatexibilityinrecognizingdatapatternsandisrelativelylightweightcomparedtomachinelearningtechniquessuchasarticialneuralnetworksorsupportvectormachines.TwocommonprocessesareusedinARIMAtoidentifythecorrecttimeseriespattern.TherstprocessistheAuto-Regressive(AR)processthatdecomposesobservationsintoarandomerrorcomponentandalinearcombinationofpriorobservations.ThesecondprocessiscalledtheMovingAverage(MA)process.InMA,eachobservationismadeupofarandomerrorcomponent,andalinearcombinationofpriorrandomerrors.Given 42

PAGE 43

atimeseriesofdataXt,thegeneralformofanARIMAprocessisgivenasfollows: (1)]TJ /F8 7.97 Tf 18.26 15.21 Td[(pXi=1iLi)(1)]TJ /F6 11.955 Tf 11.96 0 Td[(L)dXt=(1+qXi=1iLi)"t,(2) whereListhelagoperator,iandiaretheparametersforARandMAprocess,respectively,and"tareerrorterms.ThekeytotheARIMAmodelistoidentifyparametersp(ARparameter),d(differencingpass),andq(MAparameter)correctly.Thisisachievedthroughaseriesofsteps.First,weverifythatourtestdataseriesisstatisticallystationary(statisticalpropertiessuchasmeanandvarianceareconstantovertime),anddoesnotrequirefurtherdifferencing.ThedecompositionoftheselectedseriesispresentedinFigure 2-9 ,wheretheoriginaltimeseriesisdecomposedintothreeparts:trend,seasonal,andrandomnoise.Wecanseethatthetargetseriesdoesnotexhibitcleartrend,butadvertisescertaincyclicpatternasshownintheseasonaldecomposition.Forthatreason,wereviseourpredictionapproachbyemployingaSeasonalARIMA(SARIMA)modelwhichtakestheseasonalcomponentintoaccount.ItcanbeexpressedasSARIMA,(p,d,q)(P,D,Q)24,whichincludestheseasonalparametersforpriceprediction. Figure2-9. Datadecompositionfortheselecteddataseries ThenextstepforidentifyingtheSARIMAmodelparametersistoplotthecorrelogramsforautocorrelationfunction(ACF)andpartialautocorrelationfunction(PACF),asdisplayedinFigure 2-10 .Thesetwofunctionshelptodetecttrendandseasonalityofthe 43

PAGE 44

selectedseries.Notethatthex-axisisnormalizedbyfrequencysothat1.0correspondstolag=24.Fromthegraphswecanobservethat,theselectedserieshascertaindegreeofcorrelationwithitspastatcertainlagvalue,e.g.,lag=3,becausethesevaluesexceedthe95%condencelimit.However,suchacorrelationisnotstrongenoughsinceitsvalueisstillfarfromfrom1.0(whichindicatesperfectcorrelation). Figure2-10. ACFandPACFfortheselectedseries Finally,theidenticationofthemostappropriatemodelparametersisachievedbytheforecastpackagedevelopedinR[ 63 ].Intheforecastpackage,thecallingofauto.arimafunctionwillreturnthebestmodelaccordingtoAkaikeinformationcriterion(AIC)orBayesianinformationcriterion(BIC)values.Thefunctionperformsasearchoverpossiblemodelswithintheorderconstraintsprovided.Throughextensivetrials,wefoundthatmosttestseriestSARIMA(2,0,1or2)(2,0,0)24best.ThepredictionresultfortheselectedseriesisshowninFigure 2-11 .ThebluesolidpointsandtheredhollowpointsrepresentthepredictedandtheactualpricesonFebruary1st,2011.Theblacklinesrepresentspotpricevariationinthepast48hours.Weobservethatthepredictedpricesaremostlyhangingovertheaveragepriceline.Whilethismodelreturnstheleastpredictionerrorcomparedtoothermodels,itsmeansquaredpredictionerror(MSPE)isonlyslightlybetterthanthesimplepredictionusingtheexpectedmeanvalue. 44

PAGE 45

Figure2-11. Day-aheadpredictionfortheselectedseries 2.4.2Stochasticplanningforspotpricingmarket 2.4.2.1Solutionoverview Inadditiontothepredictiveplanningapproach,weproposeanalternativeapproachthattakesthestochasticnatureofthespotpricingintoaccount.WemodeltheuctuationofthespotinstancerentalcostCp(i,t)asastochasticprocessCpwithstatespaceS.CpisacollectionofS-valuedrandomvariablesonaprobabilityspaceindexedbythetimeslotsetT,i.e.,Cpforclass-iinstanceisacollection:fCp(i,t):t2Tg.Thetruevaluationsofthespotpricesovertheplanninghorizonarerepresentedbyset:fcCp(i,t):t2Tg.Thegoalofthestochasticresourcerentalplanningistooptimizetheexpectedoverallcostoverthecompletestateandprobabilityspace.Inparticular,theobjectivefunction( 2 )inDRRPcanbereformulatedasfollows: exp=ECpfXt2T(C+f(t)ii,t+(Cs(t)+Cio(t))i,t+C)]TJ /F8 7.97 Tf -1.42 -8.28 Td[(f(t)D(i,t)+Cp(i,t)i,tg,(2) whereexpistheexpectedtotalcost.Theoptimizationmodelnowbecomestominimize( 2 ),subjecttoconstraints( 2 ),( 2 ),( 2 ),( 2 ),( 2 ),and( 2 ).Wesummarizeoursolutiontostochasticresourceplanningasfollows. 1. GeneratebidpricescCp(i,t)fortheclass-iVSateveryt2T,basedonthetruevaluations. 2. Calculatethebaseprobabilitydistributionaccordingtothepricinghistory. 45

PAGE 46

3. Derivenewprobabilitydistributionsatallt2Taccordingtothebasedistributionandthebidprice. 4. Reformulateusingamultistagerecourseapproach,basedonthenewlygenerateddistributions. 5. Solvethedeterministicequivalentreformulation. Duetothepossibilityoflosingtheauction,theactualrealizationsofspotpricesarepossiblydifferentatmultipledecisionpoints.Step1),2)and3)summarizeoursolutiontothischallenge.Wecallourproposedapproachbid-dependentdynamicsampling.Aftercalculatingthedistributions,amultistageresourcemodelisusedtooptimizetheexpectedtotalcost. 2.4.2.2Bid-dependentdynamicsampling LetSibethenitestatespaceforthespotpriceofaclass-iVS.Abaseprobabilitydistributionisthesummarizeddiscreteprobabilitydistributionoveraselectedhistoricalpriceseries:Pr(Cp(i,t)=si),si2Si.Thisdistributioncannotbeusedinourstochasticoptimizationmodelbecauseitdoesnotincludetheriskofout-of-bid.Therefore,weproposetousethefollowingapproachtodynamicallygeneratetheprobabilitydistributionateverydecisionpointt.ThevaluesinthenitestatespaceSiissortedintheascendingorder(noequivalentvaluesarepresentinSi).Supposethexedon-demandcostisi.Ateachdecisionpoint,wekeepalltheprobabilitydistributionsforthosepricesinthebasedistributionwhosevaluesarelessthanthebidprices,i.e.,sicCp(i,t).Therestofthedistributionsissubstitutedbythefollowingprobabilityrepresentingthelikelihoodoftheout-of-bidevent. Pr(Cp(i,t)=i)=1)]TJ /F11 11.955 Tf 22.31 11.36 Td[(XsicCp(i,t)Pr(Cp(i,t)=si)(2) Notethatitisimpossibletogeneratetheprecisedistributionateachdecisionpointbecausewedonotknowtheactualrealizationofthespotpriceinadvance.Therefore,thedynamicallygenerateddistributionbasedontheASP'sbidpriceisan 46

PAGE 47

approximationtotheactualspotpricedistribution.However,stochasticplanningusingthisapproximateddistributionoutperformsdeterministicplanningusingxedcostparameters.Wewillillustratethispointaswellastheimpactofapproximationprecisiontostochasticplanninginthelaterpartofthissection. 2.4.2.3Transformingusingmultistagerecourse WeformulatetheproblemofStochasticResourceRentalPlanning(SRRP)asastochasticoptimizationproblem,andbuildamultistagerecoursemodeltosolvethisproblem.Themultistagerecoursemodelallowstheapplicationplannertoadoptadecisionpolicythatcanrespondtorandomeventsastheyunfold.Initially,decisionsaremadegivenpresentresources.Astimeevolves,possibleadjustments(recourseactions)becomeavailabletotheapplicationplanner.AstoSRRP,rentalplanningdecisionsatvariousdecisionpointsarerecoursevariables. Thedynamicstochasticspotpricesarerepresentedinamultistagescenariotree,G=(V,E),presentedinFigure 2-12 .AscenariotreehasT+1stages.Therststagerepresentsthecurrentstateoftheworld,andallsubsequentstagescorrespondtothefuturetimeslotswhennewinformationisavailabletotheapplicationplanner.Avertexvinstaget2Tstandsforthestateofthesystemthatcanbedistinguishedbyinformationavailableuptostaget.Eachvertexv2V,excepttherootvertex(indexedasv=0),hasauniqueparentvertex(v).Theprobabilityassociatedwiththestaterepresentedbyvertexvispv.Let(v)denotethetimestageofvertexvinthetree,wehave:P(v)=tpv=1.Eachnon-leafvertexvistherootofthesubtree:G(v)=(V0V,E0E)containingalldescendantsofvertexv.ThecompletetreeisrepresentedbyG=G(0). LetthesetofleafverticesofG(0)beL,andletthesetofverticesonthepathfromtheroottovertexvbeP(v).Ifv2L,thenP(v)representsascenariooftheproblemdescribingajointrealizationofthestochasticparametersoverallstages.Otherwise,P(v)denotesapartialrealizationoftheproblemuptothestage(v).Withthenotations 47

PAGE 48

Figure2-12. Anexampleofmultistagescenariotree denedabove,adecisionvariableXi,tdenedinthedeterministicproblemisreplacedbyasetofscenario-dependentdecisionvariables(recoursevariables)presentedbelow. Xi,t)fXi,vj(v)=tg,t2T(2) ThemultistagescenariotreeisperfectlybalancedbecauseeachpathfromroottoleafvertexhasthesamelengthT.However,thenumbersofpossiblestatesappearedineachstagearenotnecessarilyequalbecauseofthebid-baseddynamicsamplingprocesspresentedinSection 2.4.2.2 .GivenascenariotreewithascenariosetS,theASPwishestosetapolicythatmakesdifferentresourcerentaldecisionsunderdifferentscenarios.ForascenarioSj2S,decisionsmadeatstagetifencounteredbyscenarioSjisavector: fi,v,i,v,i,vg,v2Sj(2) Thesolutionmustconformtotheowofavailableinformation(non-anticipativity).Itguarantiesthatdecisionsdonotrelyoninformationthatisnotyetavailable. 2.4.2.4DeterministicreformulationofSRRP Havingbuiltthemultistagerecoursemodel,wederiveadeterministicequivalentformulationofSRRP.Inthereformulation,thetime-dependentdecisionvariablesareeliminated.Thenewformulationintroducesasetofnewvariablesthatareindexedby 48

PAGE 49

theverticespresentedinG(0).Eachvariableindexedbyvertexvisassociatedwithaprobabilitypv.Assuch,thegoalofresourcerentalplanningistosolveMILPwithregardtothescenariotree.ThecompletedeterministicequivalentformulationofSRRPisgivenbelow: minXv2Vpv(C+f((v))ii,v+(Cs((v))+Cio((v)))i,v+C)]TJ /F8 7.97 Tf -1.41 -8.28 Td[(f((v))D(i,(v))+Cp(i,(v))i,v) (2)s.t.i,(v)+i,v)]TJ /F7 11.955 Tf 11.95 0 Td[(i,v=D(i,(v)),i2I,v2V (2)P(i)i,vQ(i,v),i2I,v2V (2)i,vBi,v,i2I,v2V (2)i,0=",i2I (2)i,v,i,v2R+,i2I,v2V (2)i,v2f0,1g,i2I,v2V (2) 2.4.2.5Polynomial-timesolutions Sincevariablesateacht2Tareassociatedwithanumberofpossiblerealizations,solvingSRRPisequivalenttosolvingalarge-scaleMILP.Thereexistsanumberofstandardtechniquestosolvethisproblem,forexample,usingBendersdecomposition[ 64 ].However,duetothehugesearchspaceforoptimization,theyareonlysuitableforperformingshort-termresourcerentaldecisions.Forexample,weusedCPLEXtosolveSRRPfora6-hourplanninghorizononamachinewithIntelCorei5-650CPU(4MCache,3.20GHz)and3.2GBmemoryona32-bitOS.Ittookafewminutestogenerate 49

PAGE 50

thesolution.Whenmorescenarioswereincorporated,thecomputationaltimebecameunbearableforpracticaluse. Fortunately,anumberofpolynomialalgorithmswereproposedtosolvethestochasticlot-sizingproblem(SLSP)inmanufacturingandoperationsresearch.Amongthem,themostnotablesolutionsareabranch-and-cutalgorithmdevelopedbyGuanetal.[ 65 ],acubic-timedynamicprogrammingalgorithmdevelopedbyHuangandKucukyavuz[ 66 ],andlaterimprovedtoquadratictimebyJiangandGuan[ 37 ].Allofthemarepolynomial-timealgorithmswithrespecttothenumberofnodesonthescenariotree.Inordertodevelopapolynomial-timealgorithmforSRRP,weneedtotakecareofafewthings.First,thegeneralformulationofSLSPincludesmultiplestochasticvariables,e.g.,demand,productionandinventorycost,andorderleadtime.Therefore,SRRPcanbetreatedasaspecialinstanceofSLSPwithzeroorderleadtime.Second,givenapositiveinitialstorage(constraint 2 ),wecanconstructanequivalentproblemwithoutinitialinventorybyusing"tosatisfythedemandsemergedinrst,second,...stagesuntilitisdepleted.Finally,notethatDRRPisaspecialcaseofSRPPwithleafnodenumberequaltoone.ArelaxationoftheWagner-Withinproperty[ 67 ],calledthesemi-Wagner-Withinproperty,becomesthekeytodeveloppolynomial-timealgorithmforSRRP.Thesemi-Wagner-Withinpropertyisstatedasfollows. Proposition1(Semi-Wagner-WithinProperty[ 66 ]). Thereexistsanoptimalsolution(0,0,0)ofSRRPsuchthatif0i,v>0forv2V,thenwehave:0i,(v)+0i,v=D(i,(m)))]TJ /F6 11.955 Tf 11.96 0 Td[(D(i,((v))) forsomem2G(v). Basedonthisproposition,wecanuseavaluationfunctiontorepresentthetotalamountoftheproductionplacedinthenodesalongP((v))satisfyingthecumulativedemandsfromtheroottov,foreachv2V.Thevaluationfunctionscanbecalculated 50

PAGE 51

usingbackwardrecursion,andtheoptimalsolutionisgivenbytheresultofthevaluationfunctionoftherootnode.Duetothespacelimitation,wedonotgiveoutthealgorithmindetail.Forthecomputationaleffectiveness,readerscanreferto[ 65 ]foradetailedcomputationalexperimentsthatcomparingthebranch-and-cutalgorithmwiththedefaultCPLEXMILPsolver. 2.4.2.6Evaluationofstochasticrentalplanningmodel Inthissection,weperformsimulationstoevaluatethesolutiontoSRRPmodel.Thesimulationsettingisbasedonrealisticspotpricinghistoryandapplication-usagescenariopresentedinSection 2.3.5 .First,imagineanoraclewhoknowsallthefuturerealizationsofspotpricesinadvance,andtakesthemasinputstotheDRRPmodel.Wedenotethecostgeneratedbythismethodastheidealcasecostforne-grainedresourcerentalplanning.Wethencomputetheoverpaypercentagesagainsttheidealcasecostforallotherapproaches.ThepricedistributionisdrawnfromthesamerepresentativedatasetdescribedinSection 2.4.1.2 ,paragraph3.TheresultsareplottedinFigure 2-13 .Here,weusethepredictionvaluesobtainedfromtheapproachdescribedinSection 2.4.2 asthebidprices,becausetheyarethebestapproximationvalueswecanobtainusingstatisticalanalysisofpastpricehistory.ThecostderivedbysolvingSRRPusingforecastpricesislabeledasstochasticplanning,andthecostofsolvingitsDRRPcounterpartandthecostofusingon-demandvirtualserversarelabeledaspredictiveplanningandon-demand-deterministic,respectively.Itisnotsurprisingtoseethatthedeterministicplanningschemeusingon-demandvirtualinstancesyieldsthemostoverpay.Inaddition,stochasticplanningismorecostefcientthanpredictiveplanningforallthreeVStypes.Thisisbecauseplanningusingpricedistributionsismoreadaptivetotheuncertainavailabilityofspotresourcesthandeterministicplanning,andtheapproximationerrorsintroducedbybiddingaredilutedbyne-grainedscenariodivisionateachdecisionpoint.Whenconsideringthepricedistributionateverydecisionpoint,stochasticplanningbetterhedgesagainsttherisk 51

PAGE 52

oftheunexpectedout-of-bideventcomparedtorentalplanningbasedonforecastingvaluesinpredictiveplanning.WealsomimicacommonbidstrategythatASPsbidaxedpriceequaltotheexpectedmeanpriceofthehistoricaldata,andcompareitscostderivedbystochasticandpredictplanning.TheresultsshownonFigure 2-13 drawthesameconclusionthatstochasticplanninghasbettercostadvantage. Figure2-13. Costcomparisonofpredictiveandstochasticplanning Next,weinvestigatetheimpactofbidpriceapproximationprecisiontothestochasticplanningapproachwithregardtocostreductionforVStypec1.medium.ThisevaluationisnecessarybecauseaccordingtoSection 2.4.2.1 ,thesolutionqualityofstochasticplanningiscloselyrelatedtothetruevaluationcCp(i,t),whichisinaccurateinnaturewithrespecttotheactualspotprice.Takingthecostderivedbyactualrealizationofspotpriceasthebaselinecost,wecreatearticialbidpricesthatare+=)]TJ /F6 11.955 Tf 12.75 0 Td[(2%to10%1deviatedfromtheactualpricerealizations,andmeasurethecostdeviationfromthebaselinecostintroducedbytheapproximationerrors.TheresultsconvertedtopercenterrorsagainstthebaselinecostareplottedinFigure 2-14 1pricesthataremorethan+=)]TJ /F6 11.955 Tf 12.51 0 Td[(10%fromtheactualpricesareoutoftheactualpricerange 52

PAGE 53

Clearly,theerrorsincreaseasapproximationbecomeslessaccurate.Weusethemeansquaredpredictionerror(MSPE)tomeasuretheapproximationerrors.TheMSPEofourbestapproximationachievedbasedonthemethodpresentedinSection 2.4.2 fallsbetweenthatof2%and4%deviationofthemodel.However,theactualpercenterrorusingourapproximationis)]TJ /F6 11.955 Tf 9.3 0 Td[(12%fromthebaselinecost.Apossibleexplanationisthatourapproximationspresentamixtureofover-andunder-estimationsoftheactualpricerealizations,thusaredifferentfromthepatternofthearticialapproximatedbidpriceswecreatedinthesimulation.Inconclusion,ifonebidsaccordingtothebestapproximationresultinpractice,thepercentageerrorintroducedbyapproximationisgenerallyacceptable. Figure2-14. ImpactofapproximationprecisionforSRRPsolution 53

PAGE 54

CHAPTER3TOWARDSEFFICIENTANDFAIRRESOURCETRADING 3.1Background Theever-increasingdemandforcomputingpowermotivatesrecentadvancesinmulti-coreprocessingandhigh-speednetworking,anddrivestheemergenceofdistributedcomputingplatformsthatspanavarietyofheterogeneousdevicesacrosstheInternet.Withtheaidofhardwarevirtualization,theseplatformsarecapableofofferingrent-on-demandresourceleasingservicesthatletendusersinstantlyaccessavastpoolofresources,knownasthecloudcomputingmodel.Currentcloudcomputingmodelismostlyvendordriven,withusershavingnocontroloverthedataorthetechnologysupportedbythecloud.Suchavendor-drivenmodel,althoughconvenienttouse,alsobringsmanyissuestolight,e.g.,failureofmonocultures,tradeoffbetweenconvenienceandcontrol,andconcernsaboutenvironmentalimpact[ 68 ].Toaddresstheseissues,researchershaveproposedanalternativemodelthatprovidesacollaborativeresourcesharingplatformcalledcommunitycloud[ 69 71 ].Differentfromthecentralizedvendormodel,community-basedcloudcomputingleveragesunder-utilizednetworkedprivateresourcesforinfrastructuresupport.Tenantswithinthesamecommunitycloudtypicallysharecommonsecurityandcomplianceconcerns,andmaydelegatemanagementtosometrustedthird-partyorganization. Similartothecentralizedvendor-drivenmodel,thecommunity-basedmodelprovisionscomputationandstorageresourcesasmeteredservices.Therefore,thedesigngoalofthecommunitycloudshouldnotonlyfocusonthequalityofcomputingservice,butshouldequallyaddresstheeconomicaspectsuchthattenantsreceivecost-effectivecloudserviceprovisioning.Whilemanagingresourceallocationisrelativelystraightforwardinthecentralizedvendor-drivenmodel(e.g.,Amazon'son-demandandspotinstancepricing),itisparticularlychallengingduetotheheterogeneityinthemultitenancyenvironment.Inacommunitycloud,wearefacinga 54

PAGE 55

freemarketwheretenantsareonlyincentivizedtoacceptprotableresourceexchange.Asaresult,awelldesignedmultitenancyresourcetradingprotocolishighlydesirabletoeffectivelyregulatethemanagementofresourceallocation. Inthischapter,westudythedistributedresourcetradingprobleminacommunity-basedcloudcomputingenvironment,andproposeasetofmultitenancyresourcetradingprotocolstojointlyoptimizeresourceallocationefciencyandfairness.Specically,betterefciencyreferstotheincreasedaggregatevaluationsofallthetenants,andbetterfairnessisinterpretedasreducedenvybetweeneverypairwisecombinationoftenants.Oursolutionfollowsamarket-orienteddesignprinciple,anddevelopsadirectedhypergraphmodeltointegratethesetwoseeminglyconictingdesignobjectivesintooneuniedresourcetradingframework.OursolutiondirectlyextendstheworkofChevaleyre[ 72 ],andfurtheraddressesthechallengeofbudgetlimitedresourcetrading.Withsystematicanalysisoftheresourcetradingmarket,asetofheuristic-baseddistributedresourcetradingprotocolsaredevelopedandevaluated. Thecomprehensivestudypresentedinthischapterhasbroadutilityinthegrowingworldofeverything-as-a-service,becauseitcharacterizestheextenttowhichindependentandself-interestedtenantsinteractwitheachother.Ouranalysisshowsthatincentivepreservingtaskexchangestendtobenetthesystem,bothfromaglobalviewoftheoverallserviceefciencyandfromalocalviewoftheimprovedservicequalityvaluation.Moreover,theproposedresourcetradingapproachesarecomplementarytothevendor-drivencloudcomputing.Forexample,consideruserAlicerentsavirtualmachinefromAmazonwithreservedinstancepricing.AfterAlicenishesherjobandbeforetheleaseexpires,AlicemightsubleasethisvirtualmachinetoBobinordertopartiallycompensateforherresourcerentalcost. Thecontributionofthisstudyisprimarilyfour-folded,andissummarizedasfollows. Createadistributedmarket-orientedframeworkformodelingthemultitenancyresourcetradingproblemincommunitycloud. 55

PAGE 56

Leverageamultiagent-basedtechniquetosolvetheoptimalresourceallocationfordistributedbudget-freeresourcetrading. Proposeanoveldirectedhypergraphmodeltofacilitatetheanalysisofbudgetconstrainedresourcetrading. Developeffectiveheuristic-basedprotocolsforresourcetradinggiventhepresenceofbudgetlimitation. Therestofthechapterisorganizedasfollows.Section 3.2 presentsanoverviewoftherelatedwork.InSection 3.3 ,wedescribetheproblemsettingandformulatetheresourceallocationobjectives.InSection 3.4 ,weintroduceamultiagent-basedtechniquetoachieveoptimalresourcetradingefciencyandfairness.Section 3.5 furtherinvestigatesallocationstrategieswithlimitedbudget.Weproposeanoveldirectedhypergraphmodelanddevelopaseriesofdistributedresourcetradingprotocolsbasedonheuristicapproaches.Finally,Section 3.6 showssimulationresultsandanalyzestheirimplications. 3.2Priorwork Thestudydescribedinthischapterpresentsdistributedprotocoldesigntojointlyoptimizeresourcetradingefciencyandfairness.Astheorganizationofdistributedresourceevolvestowardsamorehierarchicalarchitecture[ 73 ],distributedalgorithmsdesignedforsolvingcombinatorialmulti-criteriaoptimizationproblemsbecomemoreattractive.Commonoptimizationtechniquesincludemachinelearning[ 74 ],evolutionaryalgorithms[ 75 ],swarmintelligence[ 76 ],andsocialeconomyapproaches[ 77 78 ].Alltheseapproachesshareacommonavorthatinvolvesinteractingentitiesevolvingtowardstheoptimalsolution(byfollowingcertainlearningornegotiationrules).Ourproposedapproachfallsintothecategoryofsocialeconomyapproaches.Theyarebuiltbasedontheobservationthatresourcemanagementindistributedsystemssharescommonfeatureswithcommodityallocationsdrivenbymarketpowerintheeconomicstudy.Itiswidelyadoptedtocreateacomputationaleconomyforgridcomputing[ 79 80 ]andtheemergingcloudcomputing[ 10 81 ].Inanearlystudy,Wolskietal.[ 5 ] 56

PAGE 57

presentedtwodifferentmarketstrategiesforcontrollingresourceallocation,namelycommoditiesmarketsandauction.Thecommoditiesmarketsstrategytreatsdisparateresourcesasinterchangeablecommodities,whileauctionrequiresorchestrationfromacentralizedauctioneerforcollectingbidsanddeterminingwinners.Ourproposedresourcetradingframeworkisdesignedforacommunitycloudenvironment,andbelongstothecommoditiesmarketcategory.Inparticular,weproposeaP2Presourcetradingmarketformanagingcloudresourceallocation.Exampleresearchrelatedtothisnotionincludes[ 82 ]and[ 83 ].In[ 82 ],aP2Pdatareplicationsystemwasproposedtoimprovefault-toleranceofdigitalcollectionsinlibrary.In[ 83 ],theauthorsproposedamultiplecurrencyeconomythatanypeercanissueitsowncurrency.Differentfromtheirdesign,peersdirectlyexchangeresourcesinourdistributedresourcetradingdesign. Inthischapter,twoeconomicmetricsareusedtoquantifythequalityofanallocation:efciencyintermsofoverallsocialwelfare,andfairnessintermsofenvy-freeness.Themetricofefciencyisimportanttocharacterizetheachievablesystemperformance,andwasstudiedinanumberofpublications[ 29 84 85 ].Ontheotherhand,themetricoffairnesshighlightsindividual'sutilitysuchthateachindividualachievesthemaximumcontentmentofitsallocatedshare[ 86 ].Comparedtoefciency,theenvy-freefairnesshasgenerallyreceivedfarlessattentions.Arelatedworktargetinggridcomputingwasfoundin[ 87 ].Usinggametheory,theauthorstackledwithamulticriteriaoptimizationproblemwiththeaidofaxiomatictheoryofequity.Theauthorsconcludedthatforfairandfeasibleschedulingonglobalscalecomputationalgrid,astrongcommunitycontrolisrequired.Theresearchconductedinthischapterapproachesthemulticriteriaoptimizationproblemfromadifferentangle,andfurtherinvestigateshowtobalancethetwometricsamongstbudget-awaredistributedtenants. Ourproposedprotocolsbuildonadirectedhypergraphmodel.Ahypergraphisanextensionofthegraphconceptthatoneedge(calledahyperedge)canconnectanarbitrarysetofverticesratherthantwo.Ahypergraphmodelisexibleandinformative 57

PAGE 58

touseinalgorithmdesignasitgeneralizesthegraph.Forthatreason,itbecomesattractivetoimprovealgorithmperformanceinvariousresearchdomains,e.g.,pagereputationcomputationforsearchengines[ 88 ],cellularmobilecommunication[ 89 ]andmemorymanagement[ 90 ].Forlarge-scalescienticcomputing,CatalyurekandAykanat[ 91 ]proposedamultilevelpartitioningapproachformappingrepeatedsparsematrix-vectorcomputationstomulticomputersusinghypergraph.Theirapproachsignicantlyreducescommunicationoverheadswhileachievingdrasticallyimprovedmappingresults.Intheirhypergraphmodel,hyperedgesrepresentafnityamongsubsetsofthedata,andtheweightsreectthestrengthofthisafnity.Wemodeltheresourcetradingprobleminasimilarmannerthataimstooptimizetheaggregateweightsofthedirectedhypergraphmodel. 3.3ADistributedResourceTradingFrameworkforCommunityCloud Thissectionpresentsthedesignoverviewofadistributedresourcetradingframeworkforthecommunitycloud.InSection 3.3.1 ,wedepicttheresourcetradingsystemmodel.InSection 3.3.2 ,weclarifytheproblemassumptions,denethegoalsforresourcetrading,andformulatetheproblem. 3.3.1Systemmodel ConsiderascenariowhereanumberofhighlyautonomoustenantsconnectedinaP2Pmanner,eachholdingasetofindivisibleresources.Aresourceisanabstractionofhardwarebundleorsoftwareservice,e.g.,VirtualMachine(VM),computationaltime,etc.Theseresourcesformapubliclyaccessibleresourcepool,andtheyarecompletelyallocatedtoallthetenantsinitially,asdescribedinFigure 3-1 .Alltenantsformacollaborativecommunitywithcommonpurposesandconcerns.TheunderlyingP2Pcommunicationinfrastructureensuresthateverytenantisabletotalktoeveryothertenantwithinthesamecommunity(theymaynotcommunicatedirectly,butthereisatleastonecommunicationpathbetweeneverypairwisetenantsonthetopology).Forthisstudy,wedonotconsiderdynamictenantsjoinandleave.Wealso 58

PAGE 59

assumethatthedistributedsystemisreliable.Anyresourcecanbeassignedtoanytenant,incurringcertainbenetandcostthatmayvarydependingonthespecicresource-tenantassignment.Eachtenantcanbeinvolvedinanynumberofresourcetradingactivities,followingthespecictenantnegotiationprotocol.Thedistributedresourcetradingresultsinaremappingofresourcestotenants.Wecalleachinstanceofsucharesourceremappingamatchmaking.Tenantsareincentivizedtopurchaseunder-utilizedresourcesfromthetenantswhocurrentlyholdthem.Asaresult,thesystemevolvestowardsbetterresourceutilizationinthelongrun. Figure3-1. Multinenancyresourcetrading:systemmodel Formally,letP=fp1,...,pngbethenitesetoftenants,andletR=fr1,...,rmgbethenitesetofindivisibleresources.TypicallywehavejRj>jPj.This,however,isnotnecessarilyalwaysthecase,i.e.,sometenantsmayobtainemptyallocation.AmatchmakingisdenedasamappingA:P!2R.Morespecically,wehavethefollowingdenition: Denition1. Matchmaking:AmatchmakingA=fA1,A2,...,AngisamappingA:P!2Rsatisfying:AiTAj=;,andSAi=A. 59

PAGE 60

TheconditionofSAi=Aensuresthatthenalmatchmakingresultisacompleteallocation. 3.3.2Problemstatement Foreachtenant,weassumeaprivatevaluationmodel,indicatingthattenantsaremutuallyblindtoeachotherandevaluateindividualallocationindependently.ThevaluationofpiisdenedbythevaluationfunctionVi(),Vi(;)=0andVi(Ai)Vi(Ai)forallAiAi.Moreover,weassumethevaluationfunctionismodular,i.e.,Vi(Ai[Aj)=Vi(Ai)+Vi(Aj))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai\Aj)forallAi,AjA. Ourrstgoalfordistributedresourcetradingprotocoldesignistoachieveoptimalmatchmakingefciencysuchthatthesocialwelfare,i.e.,!=Pni=1Vi(Ai),ismaximized. Denition2. Efciency:Let)]TJ /F1 11.955 Tf 10.1 0 Td[(bethesetofallpossiblematchmakingresults,anefcientmatchmakingisanallocationA=fA1,A2,...,Angthatmaximizesthesocialwelfare:!max=maxA2)]TJ /F11 11.955 Tf 7.31 10.76 Td[(Ppi2PVi(Ai). Theefciencycriterionreectstheoverallsystemperformance.Forexample,supposetherearetworesources,onewith2cores+1Gmemoryandtheotheronewith1core+2Gmemory,alsoassuminguserAlicehasaCPU-boundjobanduserBobhasamemory-boundjob.Therefore,AlicehashighervaluationfortherstresourcewhileBobprefersthesecondresource.ByassigningtherstresourcetoAliceandthesecondtoBob,theaggregatevaluationismaximized,andthesystemfeaturesbestjobturnaroundtime. Wedenearesource-bundleasacollectionofoneormoreresourcesheldbyanytenantpi,i.e.,aresource-bundleisannon-emptysubsetofAi.WedeneaDealasthebasiceventinthemultinenancyresourcetradingframework.Adealrepresentstheprocessofresource-bundletransferfromonetenanttoanother.Inordertoacquireresourcesfromanothertenant,certainamountofcompensationisnecessarytocompletethedeal.APaymentFunction'i,jdenesthiscompensationamountpipays 60

PAGE 61

topj.If'i,jisnegative,thenpireceivesmoneyfrompj.Eachtenantkeepsarecordofitspaymenthistory.Formally,wedenepi'sBalanceasthesummationofitswithdrawalsanddepositsinalldealspiisinvolvedin:i=P'i.Alltenantsareutility-driventhatseektomakeprotateachdeal.Formally,supposeafteradeal,theallocationofpibecomes~Ai,adealmustbeaRationalDeal(RD)ifandonlyifVi(~Ai))]TJ /F3 11.955 Tf 12.04 0 Td[(Vi(Ai)'i,jforallpi2P.Notethattherequirementofrationaldealappliestobothtenantsinvolvedinthedeal,thusisabilateralconstraint.TheUtilityofpiisgivenasUi(Ai)=Vi(Ai))]TJ /F7 11.955 Tf 11.95 0 Td[(i. Thesecondgoalofourprotocoldesignistopromotefairnesswithinthesystem.Byassociatingthevaluationandpaymentfunction,fairnessdenotestoenvy-free[ 92 ]amongstalltenants,indicatingthatnotenantwouldgetbetteroffbyswappingitsallocationwithanotherpeerthougharationaldeal.Specically,thedenitionofafairallocationisgivenasfollows. Denition3. Fairness:Let)]TJ /F1 11.955 Tf 10.1 0 Td[(bethesetofallpossiblematchmakings,amatchmakingresultischaracterizedasfairiffthereexistsA=fA1,A2,...,Ang2)]TJ /F1 11.955 Tf 10.1 0 Td[(suchthat:a)8pi,pj2P,piandpjhasdirectconnection;andb)Vi(Ai))]TJ /F7 11.955 Tf 11.95 0 Td[(iVi(Aj))]TJ /F7 11.955 Tf 11.96 0 Td[(j. Thefairnesscriterionisinlinewiththeenvy-freedenitiongivenoutin[ 72 ]thattakestransferableutilityintoaccount.TheauthorsprovedthataEfcientandEnvy-Free(EEF)statealwaysexists.Here,wefurtherextendtheirresultbyaddingtopologyconstrainttothefairnessdenition.Ourdenitionlimitsenvy-freestatestoneighboringtenants.Thisisjustiableastheunderlyingcommunicationtopologymightnotbeafullyconnectednetwork.Inaddition,acommonpracticeindistributedsystemsistoemployabudgettransfermechanismtoenforceincentivesforcommunitycontrol[ 93 ].Forexample,inP2Pandsocialnetworks,someformofdigitalcash,ornumericalreputationsrepresentingtrustrelationshipsmaybeusedforrewardingandpunishingcertainactions.Weformallydenebudgetconstraintasfollows. Denition4. Budget:Budgetbtiexpressesmaximumamountpiisabletoofferaftertdeals.Letb0ibetheinitialbudgetinitially,wehave: 61

PAGE 62

bti=b0i)]TJ /F7 11.955 Tf 11.95 0 Td[(ti Givenanyinitialallocation,thegoalofthisstudyistoinvestigatetowhatextentefciencyandfairnesscanbeachievedinthemultitenancyresourcetradingframeworkdescribedabove,andtodesignresourcetradingprotocolstoguidetenantinteractionsevolvingtowardssystem-wideefciencyandfairness.Weanalyzesituationswithandwithoutthebudgetlimitation.Fromnowon,welabelthescenariowithbudgetconstraintasbudget-aware,andrefertothelaterscenarioasbudget-unaware. 3.4Budget-unawareResourceTradingProtocol Inthissection,wedeveloparesourcetradingprotocolwithoutthepresenceofbudgetconstraint.Ourprotocoldesignisbasedonthemultiagent-basedresourceallocationoptimizationframeworkpresentedin[ 72 ]. 3.4.1Designpreliminaries Byfollowingcertainpaymentrules,wewillshowthattheresourcetradingprotocoliscapableofreachingtopology-wideefciencyaswellasenvy-freefairnessuponconvergence.Atopology-wideefcientallocationisanallocationsuchthatforeverytenant,theallocationforthesub-topologyconsistingofthattenantanditsdirectneighborsisefcient,i.e.,thematchmakingachievesmaximumsocialwelfareonthesub-topology.Weintroducetopology-wideefciencybecauseforapartiallyconnectedcommunicationtopology,agloballyefcientmatchmakingisnotguaranteedunlesstheorderofresourcetradingiscarefullyplanned.ThisisillustratedbytheexampledescribedinFigure 3-2 .Considerasystemwiththreetenantsandoneresourceinitiallyassignedtop1.Tenantshavedifferentvaluationsforthisparticularresource.Communicationlinksexistbetweenfp1,p2gandfp1,p3g.Bothp2andp3envyp1astheyhavehighervaluationfortheresource.Supposetheybothproposefortrade,andp1acceptsp3'sproposalrst.Thenalallocationisnotgloballyefcientwheretheresourceshouldbeassignedtop2. 62

PAGE 63

Figure3-2. Acounterexampleofgloballyefcientallocation Intheresourcetradingframework,eachtenantcompletestransactionswithneighborsusingonlyrationaldeals(RD),andobtainsorlosesresourcebundleaccordingly.AnRDindicatesthatthetransactionisbenecialforpushingresourcestotenantswhovaluethemmore.Infact,ANYsequenceofRDexecutionswillachieveefciencywithregardtotheunderlyingcommunicationtopology.Thisisduetothefollowingobservations:1)RDincreasessocialwelfareaccordingtoitsdenition;and2)ifnomoreRDispossible,thenthematchmakingmustreachthemaximumpossiblesocialwelfare.Givenmodularvaluationfunction,wehavethefollowingproposition. Proposition2(ConvergencetoEfciency[ 94 ]). AnysequenceofRDinvolvinganynumberofresourceexchangewilleventuallyyieldtotopology-wideefciency. ThereasoningbehindProposition 2 isfairlysimple.EachRDresultsinremappingofresourcestotenantswithhigherinterests.WhennoRDispossiblewithrespecttothecommunicationtopology,thesystemconvergestoatopology-wideefcientstate.AnotherimplicationisthatthenalstateisindependentoftheexecutionorderofRDs.NowsupposeafteranexecutionofanRD,thecurrentallocationbecomes~A.Sincethedealisbilaterallybenecialtobothtenantsinvolvedinthedeal,wecalculatethepaymentrangewiththefollowingequations. Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)'i,jVj(~Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(Aj))]TJ /F7 11.955 Tf 21.92 0 Td[('i,j(3) BysolvingEquation 3 ,theresultofthepaymentfunction'i,jfallsintotherangeof[Vj(Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(~Aj),Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)],whichdenestherationalpaymentrange. 63

PAGE 64

3.4.2Amultiagent-basedoptimizationapproachforresourcetrading Thissectionintroducesthetheoreticalfoundationofourmultitenancyresourcetradingprotocoldesign.ItismainlybasedonthetheoreticalframeworkdevelopedbyChevaleyreetal.[ 72 95 ]formultiagentsystems.Onecentralconclusionisthatresourceallocationefciencyandfairnesscanbesimultaneouslyachievedinamultiagentnegotiationframework.Tosupportthisconclusion,aproperpaymentfunctionwasselectedtodealwiththeincreasedsocialsurplus!(~A))]TJ /F7 11.955 Tf 12.36 0 Td[(!(A)aftereachdeal.Inparticular,apaymentfunctioncalledGloballyUniformPaymentFunction(GUPF)wasproposed.SupposeAand~AareallocationsbeforeandafteranRDexecution,respectively,theGUPFisdenedasfollows. GUPF:'i=[Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)])]TJ /F6 11.955 Tf 13.15 8.09 Td[([!(~A))]TJ /F7 11.955 Tf 11.95 0 Td[(!(A)] n(3) Equation 3 islabeledasgloballyuniformbecausethispaymentisimposedonalltenants.Fortenantswhodonotinvolvedinthedeal,Vi(~Ai))]TJ /F3 11.955 Tf 12.3 0 Td[(Vi(Ai)equalstozero,soeachofthemreceivesanequalshareofthesocialsurpluscreatedbythetradingactivity.NotethatGUPFiswithintheboundofrationalpayment(Equation 3 ),asillustratedbythefollowinganalysis.First,itisobviousthatsocialsurplusatleastdoesnotdecreaseafteranRDoccurs.Therefore!(~A))]TJ /F7 11.955 Tf 11.99 0 Td[(!(A)0.Next,weshowthatGUPFVj(Aj))]TJ /F3 11.955 Tf 11.96 0 Td[(Vj(~Aj).Accordingtothedenitionofsocialwelfarewehave: !(~A))]TJ /F7 11.955 Tf 11.96 0 Td[(!(A)=[Xk6=i,jVk+Vi(~Ai)+Vj(~Aj)])]TJ /F6 11.955 Tf 11.95 0 Td[([Xk6=i,jVk+Vi(Ai)+Vj(Aj)]=[Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)])]TJ /F6 11.955 Tf 11.95 0 Td[([Vj(Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(~Aj)]. Asaresult,thefollowinginequalitystands. [Vi(~Ai))]TJ /F3 11.955 Tf 11.95 0 Td[(Vi(Ai)])]TJ /F6 11.955 Tf 11.96 0 Td[([Vj(Aj))]TJ /F3 11.955 Tf 11.96 0 Td[(Vj(~Aj)]!(~A))]TJ /F7 11.955 Tf 11.96 0 Td[(!(A) n 64

PAGE 65

RearrangingthisinequalitygivesGUPFVj(Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(~Aj).Finallywehave: GUPF2[Vj(Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(~Aj),Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)]. InadditiontoGUPF,anone-offpaymentamountatinitialisintroduced.Theinitialpaymentamount,calledinitialequitabilitypayment,isdenedas:'0=Vi(A0i))]TJ /F18 7.97 Tf 13.2 5.48 Td[(!(A0) n.Themainpurposeforthispaymentfunctionistoleveltheplayingeld.ThenexttwotheoremsshowthatimposinginitialequitabilitypaymentandGUPFforresourcetradingleadstoefcientandfairmatchmaking.WerstshowthefollowinginvariantforindividualutilityholdsaftereveryRD. Theorem1. IfeachtenantpaysinitialequitabilitypaymentsatstartandpaysGUPFaftereachRDexecutes,thenalltenantssharethesameutility:Ui(Ai)=!(A) naftereachRD. ProofSketch. Thebalanceiofpiremainsinvariantaftereachdeal[ 72 ].Thisistrueforthebasecase,whereequitabilitypaymentisused.NextsupposeatallocationA,pi'sbalanceiequalstoVi(Ai))]TJ /F18 7.97 Tf 13.82 5.48 Td[(!(A) n,thenafterallocationchangesto~Ai,addingGUPF[(Vi(~Ai))]TJ /F3 11.955 Tf 11.96 0 Td[(Vi(Ai)))]TJ /F6 11.955 Tf 11.95 0 Td[((!(~A) n)]TJ /F18 7.97 Tf 13.15 5.48 Td[(!(A) n)]topi'sbalanceatAleadsto~i=Vi(~A))]TJ /F18 7.97 Tf 13.15 5.48 Td[(!(~A) n. Nowwiththisbalanceinvariant,theutilityofpiatallocationAiscomputedas: Ui(Ai)=Vi(Ai))]TJ /F7 11.955 Tf 11.95 0 Td[(i=Vi(Ai))]TJ /F6 11.955 Tf 11.95 0 Td[((Vi(Ai))]TJ /F7 11.955 Tf 13.15 8.08 Td[(!(A) n)=!(A) n Therefore,aftereachRD,theutilityofeachtenantpresentsanequalshareoftheoverallsocialwelfare. Withthisinvariant,weprovethefollowingtheorem.Notethatourversionisslightlydifferentfromthatpresentedin[ 72 ],aswetargetattopology-wideefciencyanduseamorestrictassumptionofmodulardomain. 65

PAGE 66

Theorem2(ConvergencetoEfciencyandFairness[ 72 ]). Whenallvaluationsaremodularandbudgetlimitationisnotaconcern,payinginitialequitabilitypaymentatstartandGUPFaftereachRDforeverypi2Pwillconvergetoamatchmakingstatethatachievesbothtopology-wideefciencyandenvy-freefairness. Proof. First,accordingtoProposition 2 ,whennoRDispossible,thenalmatchmaking_Aisinthetopology-wideefcientstate.Next,weshowthatthistopology-wideefcientallocationisalsointhestateofenvy-freefairness.Recallthattwotenantshavingenvyrelationshiponlyifthereexistsacommunicationlinkbetweenthem.Nowsupposepiandpjbeanytwotenantsdirectlyconnected.Accordingtothedenitionoftopology-wideefciency,transferringpj'sallocationtopiwillnotincrease!(_A).Therefore, Vi(_Ai)+Vj(_Aj)Vi(_Ai[_Aj). Asvaluationsaremodular,Vi(_Ai[_Aj)=Vi(_Ai)+Vi(_Aj))]TJ /F3 11.955 Tf 12.77 0 Td[(Vi(_Ai\_Aj).Since_Ai\_Aj=;,wehave: Vj(_Aj)Vi(_Aj),Vi(_Ai))]TJ /F6 11.955 Tf 11.96 0 Td[((Vi(_Ai))]TJ /F7 11.955 Tf 13.15 8.09 Td[(!(_A) n)Vi(_Aj))]TJ /F6 11.955 Tf 11.95 0 Td[((Vj(_Aj))]TJ /F7 11.955 Tf 13.15 8.09 Td[(!(_A) n). FromTheorem 1 ,Vi(_Ai))]TJ /F18 7.97 Tf 13.69 5.48 Td[(!(_A) nequalstoiandVj(_Aj))]TJ /F18 7.97 Tf 13.69 5.48 Td[(!(_A) nequalstojatnalallocation_A.Therefore,wehaveVi(_Ai))]TJ /F7 11.955 Tf 12.77 0 Td[(iVi(_Aj))]TJ /F7 11.955 Tf 12.77 0 Td[(jforeverycombinationofpairwisetenants. 3.4.3Protocolimplementation Wedevelopahybridcentralized/distributedresourcetradingprotocolbasedonthetheoreticallyoptimalnegotiationframework.Fromthispoint,wenamethisprotocolBudget-unawareMultitenancyResourceTrading(BuMRT).InBuMRT,eachtenanthas 66

PAGE 67

itsownexecutioncycle.SinceaftereachexecutionofanRD,alltenantsareimposedapaymentamountdenedbyGUPF,wealsoneedabanktoredistributethewealthacrossthesystem.Thisrolecanbeassignedtosometenant,referredtoasthebanknode.Thebanknodeisalsoinchargeofothercoordinatingactivitiessuchascharginginitialequitabilitypaymentanddetectingconvergence. 3.4.3.1Localviewestablishment Foreachtenant,therststepintheexecutioncycleistoestablishthelocalviewofenvyrelationshipwithothertenants.Specically,eachtenantsetsupanagentprocessanddelegatesresourcetradingtoit.Theagentprocesswillperiodicallyprobeneighboringagentswhorespondwithmessagesencapsulatingthetruevaluationoftheholdingresources.Whentheagentnoticessomeresourceintriguesmoreinterestthanitsneighbor,thisinformationwillberecordedbyaddingtheneighbortoatradingpartnerlist.Notethatourcurrentdesigndoesnotallowmaliciousagentprocesstodeliberatelyreportundervaluedresourcebundlemessages.Howtopreventsuchbehaviorsisstillanopenquestiontobeexploredinthefuture. 3.4.3.2Dealnegotiation Eachtenantrunstwoseparateprocesses,eachwithdifferentpurposes.Afterthelocalviewisinstalled,oneprocessisinchargeofinitiatingresourcetradingrequeststothosetenantsinthetradingpartnerlist,andtheotherprocessusesaproposalqueuetoactivelyhandleincomingresourcetradingrequests,asshowninFigure 3-3 .Whenanagreementisestablished,thetwotenantsstarttotransferresourcesandassociatedpayments.ThepaymentamountdeterminedbyGUPFisalsobroadcastedtoothers.Weshouldensurethatthisprocessisatomicsuchthatintheeventoffailure,thetradecanberolledbackasifitneverhappens.Alsonoticethattheresourcetransferoperationshouldbeeasytoaccomplish,e.g.,simplytransferringVMauthenticationratherthanmovingitacrossphysicalresources. 67

PAGE 68

Figure3-3. Dealnegotiation 3.4.3.3Messagebroadcasting BasedonanalysispresentedinSection 3.4.2 ,eachRDgeneratesapaymentvaluecalculatedbyGUPF,andthisvalueneedstobedisseminatedtoallthenon-participatingtenantsinthenetwork.Therefore,afteranRDisnished,wedesignatethetradeproposertobroadcastthecorrespondingGUPFvaluetoothertenantsexceptforthetwotenantsinvolvedinthetransaction.Inordertonotoodthenetwork,weadoptaspanning-treeprotocol[ 96 ]forbroadcasting,asexempliedbyFigure 3-4 .Whenthisprocessisdone,alltenantscommunicatewiththebanknodeforredistributingofthetotalwealthwithinthesystem.NotethattheexecutionorderofRDsdoesnotaffecttheconvergence.Therefore,anRDcanstartbetweenpairwisetenantswithoutwaitingforotherRDsnish.AnRDisclosedaslongastheassociatedpaymentaccountingandbookkeepingisdoneforalltenants. Figure3-4. Paymentmessagebroadcastalongabroadcasttree 3.4.3.4Convergencedetection Atradingagentseekingforpotentialtransactionsismarkedasactive.WhennoRDispossible,theagentturnstoadormantstate.Theeventofreceivingatradeproposalwilltriggertheagenttobecomeactiveagain.Whenallagentsbecomedormant, 68

PAGE 69

thematchmakingresultachievestopology-wideefciencyandenvy-freefairnesssimultaneously.Detectingtheconvergencetothisoptimalstateposesaclassicalproblemofterminationdetectionthathasarichliterature[ 97 ].Weadoptasimpleversionofthedistributedterminationdetectionalgorithmbasedonthatdescribedin[ 98 ]. 3.4.3.5Furtherdiscussion ThehybridresourcetradingprotocolissuitabletobeimplementedusingtheZookeeperservices[ 99 ].ZookeeperimplementsaPaxosstatemachine,andadeploymentoftheZookeeperprovidesservicesforhighlyreliabledistributedcoordination.AlthoughBuMRTdoesnotrequireRDserialization,itisstillbenetfrommanyusefulfunctions,e.g.,atomicbroadcastandleaderelection.ImplementingBuMRTanddeploytheprotocoltorealisticP2Penvironmentarepartofourfutureresearchplan. 3.5Budget-awareResourceTradingProtocol 3.5.1Modelingresourcetradingusingadirectedhypergraph Whenbudgetconstraintisimposed,theconvergencetotheoptimalmatchmakingstatemightnotexist.Inthissection,wedevelopadirectedhypergraphmodelforcommunity-basedcloudresourcetrading.Ahypergraphisageneralizationofthe2Dgraphthatanedgecanconnectasetofvertices.Ifthehypergraphisdirectional,anedge(a.k.a.ahyperarc)connectsahypernode(head)withasetofhypernodes(tailset).Themotivationbehindthedirectedhypergraphmodelliesinitsimplicationforone-to-manyrelationship.A2Dgraphmerelymodelsconnectivityamongtenants,butcannotrepresenttaskallocationandenvyrelationshipamongthem.Adirectedhypergraphismoreinformative,succinctlycapturingthescenariothataresourceisheldbysometenant,butinspiresmoreinterestfromsomeothertenantseachholdingasetofresources. Weproposetwomatricestobuildupahyperspace.TherstmatrixisanAllocationMatrix(AM).Itisanmnmatrixthattakesbinaryvalues,representingcurrentresourcematchmakingstateforalltenants.Eachentryi,jinAMisdenedasfollows. 69

PAGE 70

i,j=8><>:1tenantjholdsresourcei0otherwise(3) ThesecondmatrixisanEnvyMatrix(EM)representingcurrentmatchmakingunfairness(orenvyrelationship).Supposewehavetwotenants,AliceandBob.BobissaidtoenvyAlicewhenBobhashighervaluationforsomeresourcecurrentlyallocatedtoAlice.Again,weusebinaryvaluestorepresenttheenvyrelationships.Formally,AnEnvyMatrixisannmatrixdenedasfollows. "i,j=8><>:1piisenviespj0otherwise(3) Figure3-5. Adirectedhypergraphmodelderivedfromanmnnhyperspace Combiningtheallocationmatrixandtheenvymatrix,wearereadytounifyingallocationandenvyrelationshipsintoonedirectedhypergraphmodel.Werstcreateathree-dimensionalspace,mnn,asshownintheleftsideofFigure 3-5 .AdirectedhypergraphH=(V,E)iscomposedofanitenon-emptysetVofhypernodesandanitenon-emptysetEofhyperarcs.Usingthecoordinatesofthehyperspace,we 70

PAGE 71

denethehypernodeasfollows.Ahypernodev(Equation 3 )isathree-tuple(x,y,z),wherex2Rrepresentstheresource,y2Rrepresentsthetenantcurrentlyholdingx,andzissometenanthasenvyrelationshipwithy,i.e.,z2P,"y,z=1.Ahyperarce(Equation 3 )isapair,whereTVisthetailofeandh2VnTisitshead.ThetailsetTincludesthosehypernodeswhosehosttenantsinvolvedinenvyrelationshipswiththehostofthehead. Hypernode:Ahypernodeisathree-tuple: v=(x,y,z)2Vs.t.x2Ry2P,andx,y=1z2P,and"y,z=1(3) Hyperarc:Ahyperarce2Eisanorderedpairiff: e=2Es.t.h=(x1,y1,z1)2Vv=(x2,y2,z2)2TVy2=z1(3) EachtenantcanestablishalocalviewofthedirectedhypergraphbyfollowingtheprocedurepresentedinSection 3.4.3.1 .Thehyperarcsimplypotentialtransactionstobenegotiated.Inadistributedenvironment,whenonetransactionisaccomplishedusinganRD,resourceallocationchangeswhichmightaffectotherresourcetradingactivities.Buildingadirectedhypergraphisthushelpfultoevaluatethequalityoftradingselections. 3.5.2Optimalstructuresindirectedhypergraph Whenthelocalviewofthedirectedhypergraphisestablished,eachtenanthastwoactionstointeractwithothers:proposedealandacknowledgedeal.Whenmultiple 71

PAGE 72

proposalsarrive,theresourcecanonlybetradedwithoneproposer.Fromaglobalpointofview,weareinterestedinndinganoptimalstructureinthedirectedhypergraphmodelthatrepresentsthepotentialimpactofatenant'sdealselectionstrategytosuccessivetransactions.Ifweassociateeachhyperarcwithcertainweight(e.g.,valuationgainorenvydegreedrop,aswillbeintroducedlater),wewouldliketondoutsomeoptimalstructurethatminimizes/maximizestheaggregateweightswithregardtothespecicstructure.Forexample,examiningtheimpactofanofferselectiondecisionwithregardtotheresourceallocationcanbeviewedasndingtheoptimalspanninghypertreewithinthedirectedhypergraph.Specically,letRbetherootsetofahypertree,wedeneahypertreeTR=(R[N,E)rootedatRfollowingthreeconditions:1)cycle-free;2)R\N=;;and3)eachv2NhasexactlyoneenteringhyperarcandnohyperarchasanodeofRasitshead.Withthisdescriptionofahypertree,wedeneaspanninghypertreeasfollows. SpanningHypertree:AspanninghypertreeofH=(V,E)isdenedas: TR=(V,ET)s.t.ETE(Te[he)6R,8e2EnET(3) AnexampleofaspanninghypertreeisdepictedinFigure 3-6 .Ifahyperarceexistssuchthat(Te[he)R,wecanuseittospantherootsetR,anddecreasethecardinalityofRbyone.Therefore,itguaranteestheminimalityoftherootsetR.However,ndinganoptimalspanninghypertreeisNP-completeingeneral[ 100 ].Thereareotherpossibleapplicationforthedirectedhypergraphmodel,e.g.,ndinganoptimalsetofnon-conictingofferselectionscanbetransformedtondinganoptimaledgecoverinthedirectedhypergraph.Wewillleavethemtofutureexploration.Inthenextsection,wewilldevelopasetofheuristic-basedprotocolsforbudget-awareresourcetradinginacommunitycloudenvironment. 72

PAGE 73

Figure3-6. Directedhypergraphandspanninghypertree 3.5.3Protocoldesign Whenproposingforresourcetrade,atenantrationallycalculatesitspaymentamount.Whenthebudgetlimitationbiisimposedonpi2P,therationalpaymentamount'i,jfortradeproposalisintherangeof: 'i,j2[Vj(Aj))]TJ /F3 11.955 Tf 11.95 0 Td[(Vj(~Aj),minfVi(~Ai))]TJ /F3 11.955 Tf 11.95 0 Td[(Vi(Ai),big].(3) AccordingtoanalysisinSection 3.4.2 ,resourceallocationinthecommunitycloudevolvestowardsefcientandfairstatewhentenantspayinitialequitability'0andGUPFinBuMRT.However,whenbudgetlimitationpresents,tenantsdonotalwaysabidebytheseroutinepayments.Therefore,weareinterestedininvestigatingthetransitionofresourceallocationstates,whentenantspaydifferentamountsaslongastheamountsfallintherangeofEquation 3 .Inthissection,weproposeaseriesofheuristic-basedBaMRTs.Theseprotocolsconnethetradingactivitiesofeachtenanttoneighboringpeers,allowingthemtoconductlocalnegotiations.However,Theyaredifferentwitheachotherintermsoftradingselectioncriterion.ThecompletedescriptionoftheproposedBaMRTprotocolsareillustratedinProtocol 1 Tenantsdelegatetradingcontrolstotradingagentswhoperformtwobasicoperationsperiodically:proposingtradeandselectingoffer.Whenproposingatrade,theagentsimplyselectstheneighboringpeerwhoheenviesmostasthetradingpartner.In 73

PAGE 74

Protocol1:(a)V-BaMRT(b)E-BaMRT(c)P-BaMRT begin forpi2Pdo Establisheslocalviewwithneighboringpeers; // --TradeProposal--whilepihasatleastoneenviousneighbordo a)Sortspotentialtransactionsbasedonenvydegree;b)Selectspjwiththehighestenvydegreedrop;c)SelectspaymentwithintherangedenedbyEquation 3 ;ifpjacceptsofferthen Makepayments;Removespjfromitsenvylist // --OfferSelection--whileconictingoffersarrivaldo Selectsofferwith;8><>:(a)highestsocialwelfaregain,or(b)largestenvydegreedecrease,or(c)highesttransactionprotsAcceptsoffer;Receivespaymentsandupdateslocalview; ordertoquantifythematchmakingunfairnessbetweenpairwisetradingpartners,weusethefollowingequationtodenetheenvydegreeonaparticularhyperarc. i,j=maxfUi(~Ai,~i))]TJ /F3 11.955 Tf 11.95 0 Td[(Ui(Ai,i),0g(3) Thetradingagentmayselectanypaymentamountwithintherationalrange.Atenantcansetupapredenedpaymentpolicyforthetradingagent.Forexample,aconservativepolicyresultsinresourceacquisitionwithlowcost,whileanaggressivepolicyhelpsfundingpeertenantstoconductfurthertrades,andmightbenetmoreinreturn.Wewillevaluatedifferentpaymentpoliciesinperformanceevaluation.Whenmultipleoffersarrive,eachtradingagentneedstocarefullyevaluatetradingdecisionswithalocalviewofthedirectedhypergraphmodel.Thisisespeciallyimportantwhen 74

PAGE 75

offersconictwitheachothersincetheresourcecanonlybegrantedtooneneighbor.Inourdesign,eachtradingagentemploysahillclimbingtechniquetonegotiateresourcetradingwithneighboringpeers.Thehillclimbingalgorithmisfastandeffectiveinndingalocaloptimalmatchmaking.Thelocaloptimalofferselectiondecisionmustberational,asthepaymentamountsconformingtoRDincreasetheoverallsocialwelfare(Proposition 2 ).Inotherwords,ifatradeoccurs,theallocationefciencyisreinforced,andthecorrespondingenvyrelationshipbetweenthetradingpartiesiseliminated. WeproposethreeversionsofBaMRTinfavorofdifferenttradingselectioncriterion.Eachofthemfollowsdifferentpathstoreachthelocalminimum.TherstversionlabeledasValuationorientedBaMRT(V-BaMRT),lettradingagentsselecttradeswiththehighestsocialwelfaregain.Inthesecondversion,eachagentselectstheneighboringpeerwhoheenviesmostasthetradingpartner.WelabelthisversionofBaMRTasEnvyorientedBaMRT(E-BaMRT).Finally,weproposeProtori-entedBaMRT(P-BaMRT),inwhichagentsselectoffersthatwillbringinthehighesttransactionprots(denedasthedifferenceofpaymentandgainedvaluation).TheseprotocolsworksimilarlytoBuMRTexceptthatitdoesnotrequiremessagebroadcastingtoredistributesocialwealthwithinthecommunity. 3.6PerformanceEvaluation Inthissectionweinvestigatetheperformanceoftheproposedprotocolsthroughthreedifferentsetsofsimulations.First,weimplementBuMRTandvalidateitsachievableefciencyandfairness.Inthesecondsetofsimulations,threeversionsofBaMRTpresentedinProtocol 1 arecomparedinvariousnorms.Finally,weevaluatetheperformanceimpactofdifferentpaymentselectionpolicyandinitialbudgetsettingsforBaMRT. 3.6.1Simulationsettings Weinstantiatethematchmakingframeworktoageneralizeddistributedcomputingenvironment,andimplementtheresourcetradingprotocolsusingSimGrid[ 101 ]. 75

PAGE 76

Thecoreschedulingandcommunicationfunctionsareimplementedusingtheapplication-levelsimulationinterfacesprovidedbytheMSGmoduleofSimGrid.Acommunitycloudplatformwith20computationalnodes(tenants)issimulated.Wealsocreates800synthetictaskunits(resources).Tocreateaheterogeneousplatform,weassigndifferentcomputationalandnetworkingsettingstothecomputationalnodes.Assuchthesametaskunitpresentsdifferentvaluestodifferentnodes.InSimGrid,thisinformationisencapsulatedinseparateXMLles.Nodei'ssatisfactionofitscurrentallocationisquantiedbyaconcavevaluationfunctionVi(),whereVi(x)denestheutilityofnodeiobtainingxtasks[ 102 ].Theconcavityassumptionindicatesthatthemarginalvaluationdiminisheswhentheallocationincreases.Specically,weusethefollowingconcavefunctiontorepresentvaluation, Vi(x)=cxr,(3) wheretheconstantcoefcientcissetto10.0,andrisrandomlygeneratedintherangeof(0.2,0.6). Weprimarilyusefourmetricstoevaluatetheperformanceoftheproposedprotocols.First,weusesocialwelfaretoquantifytheallocationefciency.Next,inordertovalidatefairness,thetotalenvydegreeamongstallnodesisrecordedaftereachtransaction.Inaddition,twonodesthatenvyeachotherformanenviouspair.Thetotalnumberofenviouspairsisalsocountedthroughoutthenegotiationprocess.Finally,wemeasuresystemprotasanindicationofsystem'ssideutility.Foreachtransaction,theprotearnedisthedifferenceofbuyer'svaluationandtheassociatedpaymentamount.Thesystemprotisthusdenedasthecumulativeprotearnedinalltransactions. 3.6.2EvaluationofBuMRT Intherstsetofsimulations,nodesnegotiatewitheachotherusingBuMRTuntilconvergenceisreached.TheresultsareplottedinFigure 3-7 .Atthestartofeachsimulation,800taskunitsarerandomlymappedto20nodes.Wegeneratethree 76

PAGE 77

topologyprolesrepresentingdifferentnetworkcongurations.Thersttopologyprole(labeledasfullyconnected)describesafullyconnectedmeshnetwork,andtherestprolesdescribetworelativelysparsenetworktopologies.Thefullyconnectedtopologyhasatotalnodedegreeof2019=380.Thenodedegreesoftheothertwoprolesarenormalizedrelativetothefullyconnectedprole.Weusethesenormalizedvalues,0.45and0.72,torepresenttheconnectivityofbothproles.Inordertovalidateefciency,wealsoimplementaself-adaptiveauctionalgorithm[ 29 ]thatachievesmaximumsocialwelfarewhentasksareallocated.Thisresult,labeledasoptimalinFigure 3-7A ,denestheglobaloptimalsocialwelfare.FromFigure 3-7A ,weobservethatinalltopologyproles,theoverallsocialwelfareincreasesallthetimeandconvergesafteraround24transactions.Inaddition,forthefullyconnectednetwork,thenalallocationachievesthemaximumsocialwelfarewhenconverges.Figure 3-7B andFigure 3-7C showthatallsimulationsconvergetofairstatewhereallenvyrelationsareeliminated.Notethataftereachtransaction,bothenvydegreeandenviouspairnumberdonotnecessarilydecrease.Thiscanbeexplainedasfollows:althoughtheoverallunfairnesswillbeeliminatedeventually,eachsingletransactiononlyeliminatesenvybetweenthetwotradingpartners,butmaycreateenvyrelationshipbetweenotherpairs.AnotherinterestingobservationforFigure 3-7 isthattheinitialmatchmakingunfairnessiscloselyrelatedtothenetworkconnectiondegree.Thisisbecauseenvyrelationismorelikelytopresentifmorenodesareconnected.Moreover,moreconnectednetworkalsomeansmoreopportunitiesfortaskstobeassignedtonodeswhovaluethemmore.Therefore,theachievablelocalefciencyismorelikelytoincreaseasthenetworkbecomesmoreconnected. 3.6.3Convergenceanalysis Toanalyzetheconvergencerateofthebudget-unawareresourcetradingprocess,weconductmultiplesimulationsforvaryingproblemscales.Thetopologyisxedtoafullyconnectednetwork,andweusethesameinitialallocationforfaircomparison. 77

PAGE 78

AMeasurementofefciency BMeasurementoffairness:envydegree CMeasurementoffairness:envi-ouspair Figure3-7. Performanceevaluationforbudget-unawarecase BecauseBuMRThasnoxedorderofRDexecution,weaveragetheresultsof20simulationrunsforeachdatapoint.Initially,thenumberofnodesiskeptxedto20,andthesynthetictaskunitnumberisincreasedfrom50to80withstepsizeof5.Next,wemaintainthetaskunitnumbertobe80andincreasenodenumberfrom10to70.TheresultisdisplayedinFigure 3-8 .Weobservethatwhiletheconvergencerateishardlyaffectedbythetaskunitnumber(leftofFigure 3-8 ),itsteadilyincreasesassystemscalesup(rightofFigure 3-8 ).Hence,wereachtheconclusionthattheconvergencerateisnotdependentontheresourcestobeallocated,butratheraffectedbyhowmanytenantsparticipatingintheresourcetradingprocess. Figure3-8. Convergenceanalysisforbudget-unawarecase 3.6.4EvaluationofBaMRT Next,weaddbudgetlimitationtoeachnodeandcomparetheperformanceofdifferentversionsofBaMRTpresentedinSection 3.5 .Thenodeandthetaskunit 78

PAGE 79

numberaresettobe20and800respectively.Basedontheanalysisoftheaveragetransactionpaymentrange,weassigneachnodeaninitialbudgetof100.Whenatransactioniscompleted,thenodewhomakespaymentwilldeductthecorrespondingamountfromitsbalance.Conversely,itstradepartnerwilladdthesameamounttoitsbalance.Forthepurposeoffaircomparison,allsimulationsareconductedusingthesamesetting(valuationfunctionsandinitialallocation).Allsimulationsuseasamefully-connectednetwork.ThecomparisonresultsareexhibitedinFigure 3-9 .Fromtheseresults,wedrawtheconclusionthattheperformanceofeachprotocolisprimarilyinuencedbytheofferselectionstrategy.InV-BaMRT,theofferbringsthemostsocialwelfaregrowthisselected.ThereforeinFigure 3-9A weobservethatV-BaMRTleadstothehighestlocalefciencywhenconverged.Similarly,Figure 3-9C and 3-9D showthatE-BuMRTperformsbetterinpromotingfairness.Andnotsurprisingly,theoverallprotsgainisinfavorofU-BuMRT,asshowninFigure 3-9B 3.6.5Sensitivityanalysis Inthissection,weinvestigatetheimpactofdifferentpaymentselectionstrategiesandinitialbudgetsettings.ThecongurationparametersarekeptthesameasinSection 3.6.4 AsanalyzedinSection 3.5.3 ,eachtenantcansetuparbitrarypaymentpolicyforthetradingagent.Aconservativepolicyresultsinresourceacquisitionwithlowcost,whileanaggressivepolicyhelpsfundingothertenantstoconductmoretradingactivities.Whichpolicygivesbetterresultdependsontheofferselectionstrategiesandinitialbudgetdistribution.Wemodifythesimulationcodetoleteachnodeselectpaymentamountwithintheallowedrangedeterministically.Specically,letthepaymentselectionrangebe(low,high),wedevisethreedeterministicpaymentselectionstrategiesforevaluation: Aggressive:payment=low+0.75(high)]TJ /F3 11.955 Tf 11.96 0 Td[(low) Modest:payment=low+0.5(high)]TJ /F3 11.955 Tf 11.95 0 Td[(low) 79

PAGE 80

AEfciencyimprovement BProtgain CFairnessimprovement:envydegree DFairnessimprovement:enviouspair Figure3-9. Performanceevaluationforbudget-awarecase Conservative:payment=low+0.25(high)]TJ /F3 11.955 Tf 11.96 0 Td[(low) WecomparetheaggregateprotsofthesysteminFigure 3-10 .Eachvalueistheaverageresultof20simulationruns.Theresultsuggeststhatmoreaggressivebiddingbehaviorwillresultinhighersystemprotsatconvergence.Thiscanbeexplainedthatifallnodesofferhigherateachdeal,morenodeswillgetfundedthatleadtomoretransactions.Asaresult,themicro-economyofthesmallcomputingcommunityisboosted. Finally,wealtertheinitialbudgetassignmentandmeasureitsimpacttothesystemenvydegree.Takinginitialbudgetof100tobethebasecase(markedasX),thestartupfundforeachnodeisalteredfrom0.5to2timesof100.Againweaveragetheresultof20simulationruns.ThecomparisonisvisualizedinFigure 3-11 .Weobservethatforthecaseofabundantinitialfundassignment,theconvergencevalueiscloseto 80

PAGE 81

Figure3-10. Impactofdifferentpaymentselectionstrategiesforbudget-awarecase. thatachievedbyBuMRT.Whentheinitialbudgetreaches200,allprotocolsconvergetoanenvydegreeof0asiftherearenobudgetconstraint.Onthecontrary,forapoorlyfundedcomputingcommunity,thetradingactivitiesaremorelikelytofreezeduetolackofbudget,resultinginpotentiallongerconvergencetimeandhigherdegreeofunfairness. Figure3-11. Impactofinitialbudgetassignmentforbudget-awarecase. 81

PAGE 82

CHAPTER4CLOUDBAY:CLOUDMIDDLEWAREFORSCALABLERESOURCESHARING 4.1Summary ThischapterpresentsCloudBay,anonlineresourcetradingandleasingplatformformulti-partyresourcesharing.Itisaproof-of-conceptdesignbridgingthegapbetweenresourceprovidersandresourcecustomers.WiththehelpofCloudBay,theuntappedcomputingpowerprivatelyownedbymultipleorganizationsisunleashed,formingacommunitycloudresourcepool.Itpresentsthemostchallengetoourexplorationofcost-effectiveresourcemanagementstrategydesign.Followingamarket-orienteddesignprinciple,CloudBayprovidesanabstractionofasharedvirtualresourcespaceacrossmultipleadministrationdomains,andfeaturesenhancedfunctionalitiesforscalableandautomaticresourcemanagementandefcientserviceprovisioning.CloudBaydistinguishesitselffromexistingresearchandcontributesinanumberofaspects.First,itleveragesscalablenetworkvirtualizationandself-congurablevirtualappliancestofacilitateresourcefederationandparallelapplicationdeployment.Second,CloudBayadoptsaneBay-styletransactionmodelthatsupportsdifferentiatedserviceswithdifferentlevelsofjobpriorities.Forcost-sensitiveusers,CloudBayimplementsanefcientmatchmakingalgorithmbasedonauctiontheoryandenablesopportunisticresourceaccessthroughpreemptiveservicescheduling.TheproposedCloudBayplatformstandsbetweenHPCservicesellersandbuyers,andoffersacomprehensivesolutionforresourceadvertisingandstitching,transactionmanagement,andapplication-to-infrastructuremapping.Inthischapter,wepresentthedesigndetailsofCloudBay,anddiscusslessonsandchallengesencounteredintheimplementationprocess. 4.2Background TheemergingcloudcomputingparadigmreshapesthewayITservicesaredeliveredwithitsabilitytoelasticallygrowandshrinktheresourceprovisioningcapacity 82

PAGE 83

ondemand.Itlaunchesanewchapterfore-scienceande-engineeringapplicationsthatoffersHighPerformanceComputing(HPC)atscale.Forexample,therecentpublishedtop500listincludesAmazon'sEC2virtualclustercomposedofoveronethousandcc2.8xlargeinstances[ 103 ].InordertorealizeHPC-as-a-servicewiththefullpotentialofcloudcomputing,itisbesttotakeadvantageofresourcesinanopenmarketplaceacrossmultipleclouds[ 104 ].However,twomajorchallengesstillremaintobeaddressed.First,althoughendusersareliberatedfromthearduoustaskofresourceconguration,thisburdenistransferredtocomputationalresourceproviders.Existingworkeitherlimitsservicetolocalareaconnectivity[ 105 ],orrequiresnontrivialresourceandnetworkingsetupamongallresourcecontributors[ 17 ].Second,therelacksaexibleapplication-to-infrastructuremappingmechanismthataccommodatesdifferentiatedservicerequirementsandatthesametime,maintainshighefciencyforresourceallocationacrossmultipleclouds.Finally,itiscriticaltoimplementafairpricingschemeinamulti-partycloudenvironmentforbothresourcesellersandcustomers. Toovercomethesehurdles,weproposeCloudBayasafull-edgedsolutionforcomputationalresourcesharingandtradinginancommunity-basedcloudenvironment.CloudBayaddressestherstchallengebyincorporatingdecentralizedself-congurablenetworkingandself-packagingcloudtoolsets.Thisdesignbreaksthebarrierofproprietarycloudsandreduceseffortsforresourcejoining,maintenanceandquery.Italsohelpscloudresourcecustomerstodeployandmaintaintheirapplicationsusingthesharedcloudinfrastructure.Toaddressthesecondchallenge,CloudBayimplementsaneBay-styletradingmechanism.Specically,userrequestsareclassiedasquality-sensitiveandcost-sensitivedependingontheofferstheusersarewillingtomake.TheserviceschedulerinCloudBayassignshigherprioritiestoquality-sensitiveservicerequests,andallowsopportunisticprovisioningofunder-utilizedresourcesthroughpreemptiveapplicationexecution.Thecompetitionamongcost-sensitiveservicerequestsareresolvedbyanefcientauctionmechanismthatguaranteesresource 83

PAGE 84

accessforthoseuserswhovaluethemthemost.ServiceschedulinginCloudBayalsosupportsdistributedsitesbidforjobsforoptimalsystem-wideperformance.Ourproposedmarket-drivensolutiondiffersfromAmazon'son-demandandspotIaaSinthefollowingtwoaspects:(1)CloudBayismoreexiblethanAmazon'sspotmarketbecauseitallowsforpartiallyfulllinguserrequests,whereasEC2spotmarketonlysupportsall-or-noneresourceacquisition.ThisfeatureisusefulasHPCusersoftenhavefuzzyresourcedemand[ 106 ].(2)ResourceauctioninCloudBayisbasedonanovelAusubelauctionmodelthatencouragestruthfulbidding(i.e.,biddersbidbasedontheirtruevaluation),andachievesVickreyefciencycomparedwithAmazon'sspotmarketauction.ThedevelopmentofCloudBayisstillinprogressandmorefeatureswillbeaddedinfuture.WebelievethattheexploratoryinvestigationpresentedinthisstudycanopenupsignicantperspectivesofmergingHPCandcloudcomputinginthelongrun. Inthisstudy,wedemonstratethatthefollowingfeaturesrenderCloudBayafavorableimplementationforHPC-as-a-serviceinanopencloudenvironment: Scalableresourcefederation:LeveragingP2P-basedvirtualnetworking,CloudBayachievesscalableresourcebridgingbydisseminatingroutinginformationinadecentralizedfashion. Self-congurableresourceprovisioning:WedevelopanumberofprogramstoautomatenetworkcongurationandapplicationdeploymentinCloudBay.Ourworkgreatlysimpliesthetaskofresourceprovidersandprovidestimelyservicestoendusers. Fairresourceallocation:Afairallocationofresourcesallowstheservicequalitiesreceivedbyenduserstobeproportionaltothevaluestheypay.InCloudBay,weimplementanefcientmarket-drivenmatchmakingmechanismtoachievethisgoal. Flexibleresourceusage:CloudBayaccommodatesavarietyofresourceusagemodelsandoffersdifferentiatedlevelsofservicestoendusers.Forexample,itcansupportbothrigidandexibleparallelapplicationexecution. Therestofthechapterisorganizedasfollows.WesurveytherelatedsystemdesignandimplementationinSection 4.3 .InSection 4.4 ,weprovideanoverviewof 84

PAGE 85

CloudBayandintroducethedesignandimplementationofvirtualizationtoolsfacilitatingresourcesharing.InSection 4.5 ,weexplainthedetailsofthejobschedulingalgorithmsinCloudBay.TheevaluationresultsofourprototypeCloudBayimplementationarepresentedinSection 4.6 4.3PriorWork Therehaslongbeensignicantinterestininvestigatingtheapplicationofeconomicapproachesforresourcemanagementindistributedsystems.AccordingtoWolskietal.[ 5 ],twotypesofmarketstrategyarecommonlyusedinacomputationaleconomy,namelycommoditiesmarketsandauctions.Auctionsaresimpletoimplementandareefcienttoselloffcomputingcyclestocontendingusers.Therefore,auctionsachievedwideapplicationsinearlycomputationalecosystemssuchasSpawn[ 107 ],Popcorn[ 108 ],andTycoon[ 78 ].InFaucets[ 109 ],auctionisconductedtodeterminetheoptimalplacementofjobsoncomputeservers.AnotherearlyworkwasNimrod/G[ 3 ],wheregridresourceswereallocatedbasedonuser-negotiatedcontractswiththeresourcesellers.Mostsystemsweredesignedforearlydistributedcomputinginfrastructuresuchasdedicatedclustersandcomputationalgrids,anddidnotaccountforthelatesttechnologyadvanceinnetworkingandhardwarevirtualization. Intheeraofcloudcomputing,duetotheservice-orientedparadigmshift,market-drivendistributedsystemsbecomecommercializedinthenext-generationdatacenters.However,theroleofCloudBayisnottoserveasyetanotherIaaS,PaaS,orSaaSprovider,butrathertobridgethescatteredHPCresourcesandthescienticcommunityinsupportofHPCapplicationdevelopmentanddelivery.ThemostrelatedworktoCloudBaywasproposedin[ 23 ],wheretheauthorsbuiltanexperimentalresourcemarketinsideGoogleInc.Themajordifferencesbetweenthetwoworksarepresentedasfollows: Deploymentscope:TheresourcemarketcreatedbyCloudBaycanspanmultiplenetworking-layerdomains,whereasin[ 23 ],theresourcemarketwasbuiltuponresourcesconnectedbyaintra-companynetwork.Thecapabilityoftraversing 85

PAGE 86

NAT/rewallsusingP2PvirtualnetworkingearnsCloudBayawiderdeploymentscope. Schedulingmodel:CloudBaysupportsimmediateserviceschedulingwithtransparentjobpreemption,whereasin[ 23 ],auctionswereconductedinaperiodicmanner,andcannotbetriggerediftheauctioneerdidnotcollectenoughbids.Asaresult,thepossiblechanceofresourceutilizationduringthetimeofbidwindowislost. Pricingalgorithm:Inadditiontotheauctionprocedure,CloudBayadoptsanincentive-compatiblepaymentschemetoregulatebidderbehaviors. Besidestheauctionapproach,manyresearchersattemptedtodesignincentive-compatibleresourceallocationmechanismsforindividuallyrationalmarketparticipants.Forexample,TeoandMihailescu[ 110 ]developedastrategy-proofpricingschemeformultipleresourcetypeallocations.In[ 111 ],CarrollandGrosudesignedanonlineschedulingalgorithmMPJSformalleableparalleljobswithindividualdeadlines.Thesemethodsareeffectivefordistributedsettingswhereagentsareindividualrationalandarenon-cooperativeonthedynamicmarket. 4.4DesignOverview 4.4.1Architecture Figure4-1. CloudBayarchitecture Figure 4-1 depictsthearchitectureofCloudBay.ThedesigngoalofCloudBayistoprovideasuiteoftoolsthatfacilitatecomputationalresourcesharingandenhanceapplication-to-infrastructuremapping.Tofulllthisgoal,CloudBayisdesignedasaservice-orientedarchitecturethatseamlesslybridgesthegapbetweenapplications 86

PAGE 87

andresources.First,CloudBayprovidesresourcevirtualizationservicesontopofthebarehardware,including:(1)aP2Pvirtualnetworkingtoolthatsupportsscalableandcross-domainresourcestitching;and(2)anapplication-awareVMimagecalledCloudAppliancethatpackagesgrid/cloudcomputingtoolsetsandself-congurablenetworkingfacilities.Withthesupportofthevirtualizationservice,computationalresourcesresidingondifferentdomainscanbeeasilyconnectedtogethertoformanad-hocclusteroverwide-areanetworks. Next,accompaniedbythevirtualizationservices,CloudBayoffersmarket-orientedresource-requestmatchmakingservicesforbothquality-sensitiveandbudget-sensitiveusers.Thecorefunctionalitiesinclude:(1)anaccountmanagermanagingresourcesellerandbuyeraccounts;(2)atransactionnegotiatorthathelpstoarrangeuserrequestsbasedonthesupplyanddemandlevelofthecurrentresourcemarket;(3)anauctionenginethatresolvesresourcecompetitionwhennecessary;and(4)apaymentcollectorthatcollectsfeesforresourcerental.Wewillcoverthedetailsofthemarket-drivenserviceschedulingschemeinSection 4.5 Finally,CloudBayoffersavarietyofpopularprogrammingmodelsfordeployingandrunningdistributedapplications.Thisisachievedbyinterfacingwithpre-packagedsoftwaresupportingapplicationcompilation,run-timecongurationandjobmanagement.Forexample,thecurrentimplementationofCloudApplianceimagepackagesMPIlibraryandMyHadoop[ 112 ]forHPCapplicationtuningandrunning.Additionalfunctionalitiessuchasinterfacingwithusers,monitoringandprolingaretraversaltotheentireCloudBayservicestack. 4.4.2Usecaseillustration Figure 4-2 illustratesasimpleworkingscenarioinCloudBay.Ausersubmitsabidrequest(detailedinSection 4.5.1 )totheCloudBayserverseekingtoaccessresourceswithinhisbudgetconstraint(step1).TheCloudBayserveracceptsthebidrequestandplacesittogetherwithotherincompletebidrequestsinthesystem.Iftherequestcannot 87

PAGE 88

Figure4-2. AsimpleusecaseforCloudBay besatisedbythecurrentresourcesupply,adynamicascendingauctionislaunchedbythecentralizedauctionengine(step2).Supposethisuserwins6VMinstancesasaresultoftheauction,theCloudBayserverwillautomaticallyprovideconnectivitythatbundlestheallocatedinstancesintoacluster(step3).Thebundlenowbecomesinvisibletootherusersandisisolatedfromotherresourcesinthesystem.CloudBayemploysCondor[ 113 ]tomanageusersubmittedjobs,andrandomlydesignatesanodewithinthewinningbundleastheheadnode.Thejobsubmittedbytheuserwillbeforwardedtoalocalclientnoderunningthecondor schedddaemon,andCloudBaywillletCondortakeovertherestofthework(step4). 4.4.3Resourcevirtualizationtools Thissectionintroducesourpreviousworkonplatform,resourceandnetworkvirtualization.Thesetechniquesformthebasisofscalableandself-congurableresourcesharinginCloudBay.Sincethisstudymainlyfocusesonresourcemanagementandserviceschedulingissues,weonlypresentanoverviewofthesevirtualizationtools,andreferthereadersto[ 114 116 ]forimplementationdetails. IP-over-P2P(IPOP)[ 114 115 ] .CloudBayisdesignedtoprovideinfrastructuresupporttoscaleuptolargenumbersofgeographicallydistributedresourcesoverwide-areanetworks.Toaddressthisrequirement,wedevelopedIPOP,aP2P-based 88

PAGE 89

self-congurabletoolfornetworkvirtualization.CloudBayusesIPOPtoenableelasticresourceprovisionandrelinquish,andattainsthefollowingbenets:(1)scalablenetworkmanagement,becauseroutinginformationisself-conguredanddisseminatedinadecentralizedmanner.(2)resilienttofailure,becauseP2Pnetworksoffermorerobustnessagainstfailurethanacentralizedapproach.(3)easyaccessibility,duetoIPOP'sabilitytotraverseNAT/rewalls. Cloudappliance .CloudAppliancedirectlyextendsourpreviousworkofGridAppliance[ 116 ].Itpackagescloudcomputingtoolsetsintoanapplication-awarevirtualmachineimage(availableinVMware,VirtualBoxandKVM),andsupportson-demandresourceclustering.AresourceprovidermaychoosetolaunchaCloudApplianceonthephysicalhostmachine,whichwillautomaticallyplacethecontributedresourcesliceintotheglobalCloudBayresourcepool.Alternatively,aresourceprovidermayalsochoosetoinstallseparateCloudBaypackageonthey(e.g.,thepackagegrid-appliance-baseoffersvirtualnetworkingfunctionalityandcanbeinstalledfromubuntu).Inessence,aCloudApplianceisanintegratedmiddlewarethatencapsulatesafulljobschedulingsoftwarestack.Ithidestheheterogeneityofvariouscloudplatformsandprovidesauniforminterfacetodifferentcloudresourceproviders.CloudAppliancealsoallowsresourcecustomerstorununmodied,binarysoftwareexecutableswithoutimposingplatform-specicAPIsthatapplicationsmustbeboundto.SchedulingserviceinCloudAppliancedirectlyinterfaceswiththeCondorschedulerforjobmanagement.Finally,CloudApplianceofferssandboxingsecuritysuchthatundesirablebehaviorsareconnedtoanisolatedVMinstance. 4.4.4Autonomicresourcepooling ThissectionpresentstheimplementationdetailsofresourcepoolinginCloudBay.Resourcepoolinginvolvesdevelopmentof:(1)acentralizedresourcepoolaccessibletoallusers;and(2)anisolatedresourcepoolallocatedtoaparticularuser.Ourimplementationusesacentralizedapproachtoprovideautonomicservicesforresource 89

PAGE 90

congurationandmanagement.Specically,aresourcemanagerprocess,runningside-by-sidewiththeCondorcentralmanager,isimplementedontheCloudBayserverthathelpstomonitorandmanageactiveresourcesinthesystem.WeautomatetheresourcejoiningprocessbypackingabootingscriptwritteninPythonintotheCloudApplianceVMimage.TocontributeaVMinstantiatedbytheCloudAppliance,aresourceproviderrstsubmitsaresourcejoinrequestfromawebinterface,andthendownloadsacertiedcongurationleboundtotheVM.ThisprocessistermedasoppyinsertioninCloudBay. Figure4-3. Userinterfaceforviewingresourcesintheglobalpool ThefrontendofCloudBayisimplementedusingDjango[ 117 ],allowinguserstoeasilyinteractwiththeserver.Figure 4-3 showsaresourcesummarypagethatreturnsthecondor statusresultusingCondorSOAPAPI.Whenaresourcebundleisallocatedtosomerequest,theresourcemanagerprocesswillcreateanewcongurationle(oppy)foreachVMwithinthebundle.Inourpreviousimplementation[ 118 ],usershavetomanuallyconguretheallocatedresourcebundlethroughthewebinterfaces.WhereasinCloudBay,theresourcemanagerautomaticallylocatestheVMsbasedontheiraddressesontheIPOPvirtualnetworkandtransferstheoppiestothemviascp.ThisautonomicoppyinsertionprocessenablesCloudBaytoformanisolatedresourcepooluponrequestandgreatlysimpliesresourceallocation. 90

PAGE 91

4.5Market-drivenServiceScheduling ThissectionpresentsthedesigndetailsforuserserviceschedulinginCloudBay.Duetospacelimitation,wefocusontheeBay-styledifferentiatedserviceprovisioning,HPCjobsubmission,andauctionmechanismdesign.ThedetailsaboutCloudBayeconomybootstrappingandinterfacesforvaluationexpressionareomitted.Thesedetailswillbeaddressedinfutureresearch. 4.5.1Resourceandservicerequestmodels WeconsidertheresourcepoolofCloudBayconsistingofdedicatedandhigh-performancecomputingandstoragefacilities(e.g.,clustersandnetworksharedlesystems)thatspanacrossorganizationalandnationalboundaries.LeveragingtechniquespresentedinSection 4.4 ,thesefacilitiesareeasytoconfederatewithinacommonresourcenamespace,formingwhatisreferredtoasasciencecloud.NotethatCloudBaydoesnottargetatnon-dedicatedandcheapresourceinthevolunteercomputingmodelbecauseitishardtoguaranteequalityofserviceforquality-sensitiveHPCusersinahighlydynamicenvironment.OntheresourcemarketformedbyCloudBay,resourceproviderspartitiontheirresourcesintostandardsizedresourceslicesthatareinstantiatedusingCloudAppliances,anddelegatethetaskofnegotiationandsellingtoCloudBay.TheresourcerentalmodelinCloudBayissimilartothatusedinAmazonEC2,whereuserspurchasecomputingservicesintheunitofinstancehours.However,ratherthanprovidingIaaSwhereusershavecompletecontrolovertheallocatedVMsandbuildtheirownsoftwarestacks,CloudBayismorePaaS-orientedthatpacksacomputingplatformandjobmanagementfunctionalitiesasaservice. LetRbethesetofVMinstanceswithintheglobalresourcepool.CloudBayallowsforVclassesofVMinstancestobecreatedbyresourceproviders(e.g.,small,mediumandlargeVMinstances).Allinstanceswithinthesameclass,i.e.,Rv2R,v2V,havehomogeneouscongurations.WedenotethesetofuserrequestsbyU,eachrequestU2UislimitedtoasetofVMinstanceswithinthesameclass.Ifauserwishesto 91

PAGE 92

runajobonasetofheterogeneousresources,hecansimplycreatearequestgroupinCloudBaythatbundlestheVMsgrantedbyalltherequestssharingthesamejobconguration. Theneedfordifferentiatedserviceprovisioningisimminentbecauseitimprovesutilizationoftheinfrastructure.TraditionalHPCcentersallowdifferentjobpriorityclassesandusebackllingscheduling[ 119 ]toreducefragmentationofsystemresources,whilemodernIaaSprovidersincloudcomputingtendtojointlyscheduleon-demandandopportunisticresourcerequests,asisthecaseofAmazon'slaunchofspotmarketinadditiontotheon-demandservice.AsHPCmergeswithcloudcomputing,thequestionbecomes,howtoimplementthedifferentiatedrequestmodelinmodernHPCcentersequippedwithcloudinfrastructure?InCloudBay,wedevelopaserviceschedulingapproachinspiredbythetransactionmodelusedineBay.Beforeweproceedtodescribeourapproach,weclarifytheassumptionsandspecicationsoftheuserrequestmodelinthenextfewparagraphs. CloudBayadoptsamarket-orientedapproachforresourcemanagement.Inparticular,resourcepricinginCloudBayisdrivenbysupplyanddemandonthemarket.Whenuserdemandisgreaterthanresourcesupply,resourcepricesincreasethatonlythoseresourceaccessrequestswithsufcientrentalpricesaresatised.Ontheotherhand,whenuserdemandfallsbelowthesupplylevel,resourcepricesdecreasethatonlythoseresourceswithsufcientlylowleasingpricesareallocated.Thesetwocasesaresymmetricthatwecansimilarlyusesell-it-nowandbidoptionstodifferentiatedifferenttypesofresourcesellers.Inthischapter,wewillfocusonthecasewhendemandisgreaterthanresourcesupply. WedenetwotypesofuserrequestsinCloudBay. buy-requestsubmittedbyquality-sensitiveusersandisanalogoustotheoptionofbuy-it-nowoneBay.Thesubmittedjobislikelytobeassociatedwithadeadline,andtheinterruptioninserviceisgenerallyundesirable(non-preemptive). 92

PAGE 93

Notethatwecannotpromiseimmediateaccesstotheresourcebecausethesystemmightbecomesocongestedllingwithnon-preemptivejobs. bid-requestsubmittedbybudget-sensitiveusersandisanalogoustothosewhobidongoodsoneBaywishingtondadeal.Thereisnodeadlineassociatedwithbid-requests.Thebiddermayspecifyaexpecteddurationofjobexecution,orsimplyletitruntocompletion.Thejobsarecharacterizedasfailureresilientthatinterruptioninservicedoesnotcompromisethecomputationintegrity. LetUbstandforabidrequest,andletUqstandforabuy-request,Ub,Uq2U.Abuy-requestUqconsistsofthenumberofVMinstancestoboot,andtheexpectedrentingduration.Theexpressionofabid-requestisslightlymorecomplexbecauseitdenesexiblecongurationparameters.Abid-requestisatuplecomposedoffourelements:Ubi=fv,pi,ni,ig,wherev2VistherequestedVMclass,piisthebidpriceforaunitVMinstanceinunittime,niistherequestedVMnumber,andiisthedesiredrentingduration. Givenamixtureofthetwotypesofuserrequests,thegoalofserviceschedulingistoachievefairpricingwhilemaintaininghighutilizationoftheinfrastructure.Withdifferentcontext,marketfairnesscouldhavedifferentmeanings.Herebyfairpricingwemean:(1)jobsassociatedwithhighbidsshouldtakeprecedenceoverlow-bidjobs;and(2)marketpriceofresourcesisnotover-orunder-valuated.Byhighinfrastructureutilizationwemeanthatthematchmakingserviceshouldmakeresourceallocationdecisionsinatimelymanner,andgrantsresourceaccessrightstoenduserswheneverthereisachance.WewillillustratethedesigndetailsofserviceschedulinginCloudBayinthelatersections. 4.5.2Jobsubmission CloudBaydirectlyinterfaceswithCondorforjobmanagementbecauseofCondor'sabilitytosupportbothdedicatedandopportunisticjobexecution.WecreateauniformwebinterfacethatallowsuserstouploadexecutablesandjobcongurationlestotheCloudBayserver.ThejobsubmissionprocessinCloudBayisillustratedinFigure 4-4 .First,whenaresourcebundleisallocatedtoservearequest,theCloudBayserverwill 93

PAGE 94

sendthejobtoagatewaynodewithinthewinningbundlerunningthecondor schedddaemon.Afterthat,thelocalCondorserverwillfetchthejobinformationandschedulethejobinthelocalpool(seetheleftofFigure 4-4 ).Ifthisjobgetspreemptedsometimelater,theCloudBayserverwillstorethecomputingstate(throughcheckpointing)aswellastheoriginaljobconguration.Supposeafterawhile,anewpoolofresourcesbecomeavailableagain,theCloudBayserverwillredirectthejobinformationtoagatewaynodeinthenewpooltoresumethejobexecution(seetherightpartofFigure 4-4 ). Figure4-4. JobsubmissioninCloudBay 4.5.3Economybootstrapping CloudBaycustomersbuycomputingservicesusingvirtualcurrenciescirculatedinthesystem.TherecentemergenceoftheBitcoin[ 120 ]systemseemstoprovideaplausiblesolutiontotheimplementationofthevirtualcurrenciesusedinCloudBay.Thisisbecauseduetotheunderlyingcommunicationinfrastructure,thetransactionmodelusedinCloudBayisP2Pinnature,whichmatcheswellwithBitcoin'sdesignprinciple.Forresourceproviders,CloudBayadoptsaclosed-loopfundingpolicy[ 78 ]toencouragecontribution,i.e.,eachproviderisassignedaninitialallotmentoffundsatjointime,andearnsfundsbyprovidingHPCservicestoresourcecustomers. 4.5.4Interfaceforvaluationexpression CloudBayplanstopresentaexibleinterfaceforbidders.SinceeachbidrequestsasetofhomogeneousVMinstanceswithinthesameVMclass,thereisnoneedtoemployacomplexbiddinglanguage(e.g.,ORorXOR)designedforthegeneralformofcombinatorialauctions.Formally,letUbstandforabidrequest,andletUqstandfora 94

PAGE 95

buy-request,Ub,Uq2U.Abuy-requestUqconsistsofthenumberofVMinstancestoboot,andtheexpectedrentingduration.Theexpressionofabid-requestisslightlymorecomplexbecauseitdenesexiblecongurationparameters.Abid-requestisatuplecomposedoffourelements:Ubi=fv,pi,ni,ig,wherev2VistherequestedVMclass,piisthebidpriceforaunitVMinstanceinunittime,niistherequestedVMnumber,andiisthedesiredrentingduration.Inaddition,CloudBayallowsuserstoexpresspartialinterestsforthedesiredresourcebundle.Thisisusefulwhenusersexpectdiminishingreturnsforallocatedresourcebundle.Inthecontextofparallelcomputing,asscaleincreases,performancedegradationduetocoordinationoverheadiscommonlyencountered.However,existingsystemsimplementingcomputingeconomydonotallowuserstoexpressdiminishingvaluationformarginalresource.CloudBaydesignsaninterfacesimilartoeMediator[ 121 ]thatsupportsgraphicalprice-quantityexpressionuponbidders'request. 4.5.5ServiceschedulinginCloudBay TheprocedureforrequestschedulinginCloudBayissummarizedinAlgorithm 2 .Inordertoeliminaterequestqueueing,thetransactionnegotiatortriestomakeanallocationdecisionwheneveraservicerequestarrives.Anincomingrequestissuedbysomequality-sensitiveusertakesprecedenceoverallbidrequestsandgainaccesstothedesiredresourcebundlewheneverpossible(line3to12).Ontheotherhand,ifanincomingrequestisofbidtype,itisscheduledtocompeteforresourceswithotherbidrequestswhencurrentresourcesupplycannotmeetitsdemand.Theauctionenginewilltriggeratwo-stageAusubelauction(line17)toresolvethecompetition. TheoriginalAusubelauction(alsoknownastheefcientascendingauction)wasproposedin[ 122 ],andpossessestwoappealingpropertiesthatmakeitagoodmatchforourdesigngoal.First,itiscomputationallytractable.Second,itemploysanon-linearpaymentmethodtoeliminatetheincentivesofstrategicbidbehaviors.However,wecannotdirectlyapplytheoriginalAusubelauctiontoourschedulingcontextbecauseof 95

PAGE 96

Algorithm2:RequestschedulinginCloudBay begin examineincomingrequesttypecasebuy-request ifsupplydemandthen allocateVMsasrequested elseif9unnishedbidjobsANDtheiraggregateresourceoccupationdemandthen preemptjobsfromlow-bidtohighuntildemandissatisedelse negotiatewiththeuserwithtwooptionstryatalatertimepaylargeneforimmediateresourceaccess casebid-request ifsupplydemandthen allocateVMsasrequestedandcollectpaymentsaccordinglyelse startatwo-stageAusubelauction,reconsiderbid-requestsforallincompletejobsallocateaccordingtotheauctionresult thefollowingdifculties:(1)Ausubelauctionusesiterativepriceadjustmenttobalancemarketdemandandsupply.Inpracticalalgorithmicdesign,theconvergencetomarketequilibriumstatemighttakelongtimeduetopriceoscillatingaroundthemarketclearingprice.Thereasonbehindthisisthatit'simpossibletodeterminethesteplengthforpriceadjustmentunlessweknowthesearchstoppoint(themarketclearingprice)inadvance.(2)Somebiddershaveall-or-noneresourceacquisitionpreference.Theymaysuddenlydropoutoftheauctionwhenpriceisadjusted.Ifthatisthecase,themarketequilibriumstatemaynotexistatall.Inordertodetermineresourceallocation,wehavetoextendthefeasibleregionforthesolution.Specically,supposenbiddersbidformVMinstancesofcertainclassv.Leteachbidder'sdemandbedtiatauctionroundt(theauctionisiterative).WerelaxtheconvergenceconditionofPni=1dti=mtoPni=1dtim. 96

PAGE 97

Notethatsuchrelaxationwillresultinefciencyloss.However,asweusebacktrackingtondtheclosestpointtoequilibriumstate,suchlossisrelativelysmall. IntheoriginalAusubelauction,thepaymentcalculationiscarriedoutalongwiththeproceduretosearchforthemarketequilibriumprice.Weproposeatwo-stageAusubelauctiontoovercometherstdifculty.Intherststage,thealgorithmquicklylocatesanalmarketprice.Withthisinformation,wecandecidethepriceadjustmentstepandsimulatetheoriginalAusubelpaymentcalculationprocedureinthesecondstage.Weassumeuser'svaluationtoresourcebundleismonotonicandstrictlyconcave,i.e.,allocatedresourcesexhibitdiminishingrewardstousers.ForagivenVMclassv,letpki(weomitvforbrevityofnotations)beuseri'sbidpriceforthekthallocatedinstanceunittime.Toobtainthemarketclearingprice,weperformabinarysearchonasortedlistofsuchbidprices.Whentwobidssubmittedbytwodifferentuserstiewitheachother,thealgorithmassignshigherprioritytothebidsubmittedatanearlierwallclocktime.Ifthealgorithmfailstoconvergetoamarketclearingprice,itwillbacktracktondthebestfeasibleallocationyieldingPni=1dtim.Thenalallocationforeachuserisdeterminedbyevaluatingthemarginalbidvectorusingthereturnednalmarketprice. 4.5.6Paymentaccounting Inthesecondstage,theauctionenginesimulatestheauctioneer-biddercommunicationsasproposedintheoriginalAusubelauction[ 122 ]inaniterativemanner.Thepaymentcollectorinteractswiththeauctionengineinordertocalculatepaymentamountsforallbidders.Webrieysummarizethepaymentaccountingmethodasfollows.First,ateachroundt,theauctioneercalculatestheaggregatereservedbundletiforbidderibycomparingthemarketsupplyagainsttheaggregatedemandfromi'sopponents: ti=maxf0,m)]TJ /F11 11.955 Tf 11.95 11.36 Td[(Xj6=idtjg(4) 97

PAGE 98

Accordingly,theroundreservedbundleisdenedasthedifferenceoftheaggregatereservedbundleatadjacentrounds: 1i=1iti=ti)]TJ /F7 11.955 Tf 11.96 0 Td[(t)]TJ /F9 7.97 Tf 6.59 0 Td[(1i(t>1)(4) Notethatti0becausetheaggregatedemandfromi'sopponentsisweaklydiminishing.Ifti>0,thenthisamountofallocationisreferredtoasclinchedbybidderiatcurrentroundpricept.SupposeiwinsAiatthenalroundT,thetotalpaymentofiiscalculatedas: Pi(Ai)=TXt=1ptti(4) Accordingly,theauctionrevenueQforthenalallocationAisgivenby: Q(A)=nXi=1Pi(Ai)(4) OnevirtueoftheAusubelauctionisthatitreplicatestheoutcomeofthestaticVickreyauction.Thispropertyisdesirablebecauseuntruthfulusersexperiencedegradedperformanceincomputingmarkets[ 123 ].Theproposedauctionisincentivecompatible(proofdetailedin[ 122 ]),andresultsinfairmarketpricinguponconvergence. 4.5.7Discussion Ourschedulingdecisionismadeonrequest.Thismightcauseconstantthrashingofthelow-bidrequests.Infact,suchaneffectisthetradeofftoreducedresourceutilizationinperiodicscheduling.Toalleviatethisproblem,wecancompensatethepreemptedlow-bidjobsforasmallamount.Asthecompensationaccumulates,thejobbecomesmoreresilienttopreemption.Thisisaninterestingtopicbecausedoingsoseemstoviolateourdesigngoaloffairness.Duetothespacelimitationwewillnotdiscussthesolutionfurther. 98

PAGE 99

4.6Evaluation 4.6.1Evaluationforresourcepooling WedevelopanddeployanexperimentalCloudBayplatformcomposedof32VMinstances,with20ofthemsetuponFutureGrid[ 124 ],8onAmazonEC2,and4onlocallabmachinesattheUniversityofFlorida.Eachinstanceisequippedwith1.5Gmemoryand1virtualCPUcorerunningat2.66GHz,andispre-conguredwithCondorsupportingbothdedicatedandopportunisticscheduling.TheCloudBayserverprocessisimplementedandrunonaseparatemachinethatalsoworksastheheadnodefortheglobalCondorresourcepool. Figure4-5. Experimentforautonomicresourcepooling First,weexaminethesetuptimeforcreatinganisolatedbundleofVMinstances.Inparticular,thesetuptimeisthetimeelapsedfromthemomentanallocationdecisionismadeuntilallresourcesinthebundleareshownusingthecondor statuscommand.AccordingtoSection 4.4.4 ,thesetuptimecomprises:(1)generatingoppylefornetworkconguration;and(2)notifyingtheVMinstancewithinthewinningbundleabouttheinformationofthenewCondorheadnodebytransferringtheoppyleandmodifythelocalCondorconguration.Figure 4-5 showsthemeasurementofbundlesetup 99

PAGE 100

AWordCount BTerasort Figure4-6. Performanceevaluationforvirtualnetoworking timeformultipleVMinstancescrossthreedifferentsitesandonFutureGridonly.WeobservethatthesetuptimedisplaysanincreasingtrendasthenumberofVMinstancesincreasesforbothexperiments.Sinceclouduserstypicallyrequestresourcesoverhours,theexperimentresultsindicatethatautomaticresourcepoolinginCloudBayimposesatrivialoverheadtothetotalresourcerentalperiod. Next,weinvestigatetheperformanceofCloudBayvirtualnetworkingbystresstestingtheHadoopclusterwithandwithoutIPOPvirtualnetwork,respectively.Specically,wedeployaCloudBayHadoopcluster,withtwoVMinstanceshostedontheUFcampusnetwork,andtheothertwoVMinstanceshostedonAmazon'sEC2platform.AllinstanceshavethesameresourcecongurationwithEC2'sm1.largeinstancetype.Forthepurposeofperformancecomparison,wealsosetupaEC2homogeneousHadoopclusterconnectedbyEC2'sinternalnetwork.TwoMapReduceprograms,wordcountandterasort,areselectedasbenchmarkprograms.Foreachprogram,wevarytheinputlesizefrom0.5Gto2.5G,andmeasurethecompletiontimeofallthemapandreducetasks.TheresultsareshowninFigure 4-6A andFigure 4-6B .Fromthegure,weobservethattheheterogeneousnetworkingenvironmentinCloudBayvirtualclusterachievesbroaderdeploymentscopeatthecostofdegradedexecutiontime.UsingtheHadoopmonitoringtool,weobservethatthetwolocalnodesgreatly 100

PAGE 101

straggletheprogramprogressduetotheintermediatedatatransferfromtheEC2site(themasternodeislocatedatEC2side).Inaddition,theperformancegapintermsofcompletiontimedifferenceisrelativelyconsistentinthewordcountprogram,butincreasessignicantlyastheinputdatasizegrows.Thisphenomenonisprimarilycontributedtothedifferenceofintermediatedatatransferbetweenthemapandthereducephase.Forwordcount,thesizeofthewordlistgeneratedfromthemaptasksisalmostthesameforallinput1,whileforterasort,thesizeoftheintermediatedataforthereducetasksisincreasingallthetime.Asthedatasharingproblembecomesmoreseriousinavirtualcloudenvironment[ 125 ],theresearchforlocation-awareschedulingmechanismfordata-intensiveapplicationsisthereforeimperativeinthefuturedevelopmentofCloudBay. 4.6.2Evaluationforservicescheduling ThissectionstudiesCloudBay'sabilitytoschedulemixed-typeservicerequests.Ourinvestigationanswerstwoquestionsfromdifferentperspectives.First,fromtheperspectiveoftheresourceproviders,weareinterestedinunderstandinghowmuchresourcetimeisconsumedbyfrequentpreemptionsoflow-bidservicerequests(pre-emptionoverhead).Next,fromtheperspectiveoftheendusers,weareconcernedabouttheperceivedlagofservicecompletion(servicedelay)againstthewillingnesstopayfortheservice(offeredprice). TheCloudBayplatformisinprototypetestingstageanddoesnotaccumulateenoughuserbase.Therefore,ourevaluationissimulation-based.Weimplementadiscrete-eventsimulatorusingtheSimpy[ 126 ]simulationpackagebasedonPython.Inthesimulation,wecreate512single-coreVMinstancestoserveincominguserrequests.Eachrequestcanaskforupto32instancesforrunningapplications.TherequestedVMnumberperrequestisuniformlydistributedintherangeof(0,32].Forbuy-requests,the 1Wesimplyappendthesametexttogeneratelargersizeofinput 101

PAGE 102

resourcereservationpriceinaunitoftimeissetto20.AccordingtoAmazon'sspotpricehistory[ 60 ],wesettheofferedpricesforbid-requeststofallintherangeof(0,20),andfollowsanormaldistributionwith=8and=4.ThejobarrivalprocessisassumedtofollowaPoissondistribution.Byvaryingtherateparameter,wecansimulatesystembehaviorsunderdifferentworkloads.Wegeneratesyntheticuser-requestedresourceusagetimesbasedonarealisticworkloadscenariodescribedin[ 127 ].TheworkloadtracesincludeaCondorworkloadfromtheUniversityofNotreDame,andanon-demandIaaScloudworkloadfromtheUniversityofChicagoNimbussciencecloud.Basedonthesetraces,therequestedtimesaresetspanningarelativelylongperiodoftime(e.g.,atypicalrequestwillaskforresourcerentaloverseveralhours). Intherstsetofsimulations,weassumethepreemptiontimeislinearlyproportionaltothenumberofVMinstancestorelinquishandreset.Thepreemptionprocessincludesthetimetosaveprogramstate(checkpointing),restartthenetworkingcongurationprocessandrecongurelocalCondorservice.ThisprocesscantakeseveralminutesforrepoolingalargenumberofVMinstances.Figure 4-7 showstheresultscalculatedoveraone-monthperiodsimulationrun.Wevarythepercentageofbidrequeststogeneratedifferentowsofincomingrequests.Thelabelsofhigh,medium,andlowworkloadcorrespondtotheaveragesystemutilizationof83.3%,66.5%,and53.4%,respectively.Notethatthepresentedresultsarerelativemeasurements,e.g.,thebidoverheadismeasuredasthepreemptionlosswithregardtothetotalresourcetimeoccupiedbybid-requests,nottoresourcetimeoccupiedbyallrequests.Therefore,theoveralloverheadisapproximatelytheweightedsumofbid-requestandbuy-requestoverhead.Whenlessbidrequestsarepresent,theyaresubjecttofrequentpreemptionbythedominantbuyrequests.Asaresult,weobservespikesattheinitialphaseforbidrequests.However,theoveralloverheadisrelativelystableinallthetestedscenarios,contributingaround1.8%tothetotalbusyresourcecycles,indicatingthatCloudBayissuitableforprocessinghighthroughputservicerequestsinanopencloudenvironment. 102

PAGE 103

Figure4-7. Evaluationoftheoverheadduetopreemption Figure4-8. ServicedelayfactorVS.Offeredprice Inthesecondsetofsimulations,wecreate2,000syntheticrequestsandinvestigatetheaverageservicedelaywithregardtodifferentuserbidprices.Theservicedelayfactorisdenedastheratiooftheactualservicecompletiontimetotheuserrequestedtime.Afactorof1.0meansthereisnoservicedelay.Weconductvesimulationrunswithvaryingpercentagesofbidrequestfrom30%to70%.Foreachrun,thesystemutilizationaveragesataround83%,andthetotalsimulatedtimeisabout50days.The 103

PAGE 104

resultsareshowninFigure 4-8 .Asweexpected,higherbidpriceleadstolessservicedelayingeneral.However,wealsoobserveafewirregularpointsonthegure,andthelessbidrequestsacurvegets,themorewrinkledacurveexhibits.Thiscanbeexplainedasfollows:(1)alow-bidrequestmightgetscheduledwithoutblockingsimplybecausethereareavailableslotsinthesystem;(2)thebidpriceisrandomlygeneratedforeachrequestsuchthatthenumberofbidrequestsforaparticularpriceisinsufcient.Ingeneral,wecanconcludethatCloudBayachievesfairresourceallocationforservingdifferentiateduserrequests. 104

PAGE 105

CHAPTER5CONCLUSIONANDFUTUREWORK 5.1Conclusion Inthisdissertation,wehavethoroughlyexploredthedesignspaceforcost-effectiveandexibleresourcemanagementstrategiesinutilitycomputing,andhaveproposedafewnovelsolutionstoaddressthechallengesofscalabilityandheterogeneity.Inthesecondchapter,weinvestigatedtheproblemofne-grainedresourcerentalplanninginacloudenvironment,anddevelopedsolutionsforbothdeterministicandstochasticresourcepricingsettings.Ouroptimizationmodelswerebasedonathoroughrentalcostanalysisofelasticapplicationdeploymentincloud.Whenresourcepricingisxed,weobservedthecosttradeoffbetweencomputingandstorageemergesintime-slottedresourceprovisionscheduling.Basedonthisobservation,weformulatedadeterministicoptimizationmodelthateffectivelyminimizesrentalcostofvirtualserverswhilecoveringcustomerdemandovercertainplanninghorizon.Inaddition,wetookonestepfurthertoanalyzethepredictabilityofspotresourcepricesusingAmazon'sspotinstancepricetrace,andproposedanalternativestochasticoptimizationmodelthatseekstominimizetheexpectedresourcerentalcostgiventhepresenceofspotpriceuncertainty.Simulationsbasedonrealisticsettingsclearlydemonstratedtheadvantageofthestochasticoptimizationapproachoverthepredictiveapproachinrentalcostreduction.Wealsostudiedtheimpactofvariousparametersettingsontheperformanceofbothmodels.Webelievetheproposedned-grainedapproachesoffereffectivemeansforresourcerentalplanninginpractice. Inthethirdchapter,wehavestudiedtheresourcetradingprobleminacommunity-basedcloudcomputingenvironment.Thegoalofthestudypresentedinthischapteristoinvestigatetheinteractionsamongindependentandrationaltenants,andtoestablisheffectiveandeasy-to-implementnegotiationprotocolsforsystem-wideallocationefciencyandfairness.Towardsthisgoal,werstadoptedamultiagent-based 105

PAGE 106

optimizationframeworkandanalyzedtheoptimalresultswithoutconcerningaboutbudgetlimitation.Next,weproposedanoveldirectedhypergraphmodelthatcombinesallocationandenvyrelationshipinathree-dimensionalhyperspace.Thismodeleffectivelycapturedtheimpactoftradingselectiondecisionsfromaglobalpointofview.Whenbudgetlimitationisimposedoneachtenant,wedevelopedasetofdistributedresourcetradingprotocolsbasedonheuristicapproaches.Simulationresultsshowthattheproposedprotocolsperformwellinawiderangeofsettings.Weexpectthatthecomprehensivestudypresentedinthischapterwouldopennewvistasfordesigningeffectiveresourcemanagementstrategiesinutilitycomputingenvironments. Finally,wepresentedCloudBay,anovelresourcesharingmiddlewarestackcomposedofresourcemanagementsoftwarestackfromgroundup.Equippedwithvirtualnetworkingandapplication-awarevirtualappliances,CloudBayachievesad-hocself-organization,discoveryandgroupingofdistributedresourceswithoutincurringextradeploymentandmanagementeffortsfrombothresourceprovidersandendusers.Moreover,CloudBayimplementsamarket-drivenserviceschedulingpolicythataccommodatesamixtureofuserrequestmodels,andefcientlydistributesidleresourcestousersinacost-effectivemanner.Thepricingandpaymentaccountingpoliciesboostsutilitiesformultipletradingparties,andguaranteesincentivecompatibilityforbidders.UtilizingservicesprovidedbyCloudBay,researcherswithdomainknowledgecancomfortablydeploytheirparallelapplicationsusingpopularparallelprogrammingmodelsonaresourcebundleassembledfrommultipleorganizations.Wehavealreadydeployedvirtualappliancesacrossavarietyofopenandprivatecloudplatforms,includinguniversityclusters,FutureGrid,andAmazonEC2.WeexpectthatourexperiencesgainedfromthedesignandimplementationofCloudBaywouldopenanewresearchavenueforrealizingHPC-as-a-service,andpushtheboundaryfornewcloudcomputingusagemodels. 106

PAGE 107

5.2FutureWork Theresearchofdistributedsystemsencompassesmanyareasofcomputerscienceandisamongthefastestdevelopingeldsinthepastdecade.Asresourcemanagementneedstocopewiththegrowingcomplexityofthedistributedsystems,theexplorationpresentedinthisdissertationisjustastartingpoint.Weexpectthedesignspacetobegrowingtremendouslyasdistributedsystemsscale.Inthefuture,weplantocontinuouslyexploreandimproveresourcemanagementindistributedsystemsinvolvingmutuallydistrustfulcomponents.Anotherpotentialresearchdirectionistoinvestigatetheemergingresearchproblemsintheintersectionofmobilecomputingandcloudcomputing.Adistributedcomputingsystemcomposedoftraditionalcomputingdevicesandmobilecomputingdevicesposesmorechallengestoresourcemanagement.Asmobiledevicesarearchitecturallyheterogeneousandresourceconstrained,theybecomemoreandmorerelieduponthecloudcomputinginfrastructure.Inthelongrun,weexpectourresearchtoextendtowardsscopesbeyonddesktopsanddatacenterclusters,andtacklechallengesemergedinvariousformsofdistributedsystems. 107

PAGE 108

REFERENCES [1] S.GarnkelandH.Abelson,ArchitectsoftheInformationSociety:35YearsoftheLaboratoryforComputerScienceatMIT,ser.ArchitectsoftheInformationSociety:35YearsoftheLaboratoryforComputerScienceatMIT.MitPress,1999. [2] E.Anshelevich,A.Dasgupta,J.Kleinberg,E.Tardos,T.Wexler,andT.Roughgarden,Thepriceofstabilityfornetworkdesignwithfaircostallocation,inProceedingsofthe45thAnnualIEEESymposiumonFoundationsofComputerScience(FOCS04),2004,pp.295. [3] R.Buyya,D.Abramson,andJ.Giddy,Nimrod/g:anarchitectureforaresourcemanagementandschedulingsysteminaglobalcomputationalgrid,inPro-ceedingsofthefourthInternationalConference/ExhibitiononHighPerformanceComputingintheAsia-PacicRegion,2000,pp.283. [4] L.Amar,A.Mu'alem,andJ.Stoer,Ontheimportanceofmigrationforfairnessinonlinegridmarkets,inProceedingsofthe7thinternationaljointconferenceonAutonomousagentsandmultiagentsystems(AAMAS08),2008,pp.1299. [5] R.Wolski,J.S.Plank,J.Brevik,andT.Bryan,G-commerce:Marketformulationscontrollingresourceallocationonthecomputationalgrid,inProceedingsofthe15thInternationalParallel&DistributedProcessingSymposium(IPDPS2001),2001,pp.46. [6] K.Rzadca,D.Trystram,andA.Wierzbicki,Fairgame-theoreticresourcemanagementindedicatedgrids,inProceedingsoftheSeventhIEEEInterna-tionalSymposiumonClusterComputingandtheGrid(CCGRID07),2007,pp.343. [7] D.Ferguson,Y.Yemini,andC.Nikolaou,Microeconomicalgorithmsforloadbalancingindistributedcomputersystems,in8thInternationalConferenceonDistributedComputingSystems,1988,pp.491. [8] D.GrosuandA.T.Chronopoulos,Noncooperativeloadbalancingindistributedsystems,J.ParallelDistrib.Comput.,vol.65,no.9,pp.1022,2005. [9] K.Ranganathan,M.Ripeanu,A.Sarin,andI.T.Foster,Incentivemechanismsforlargecollaborativeresourcesharing,inCCGRID,2004,pp.1. [10] R.Buyya,S.Pandey,andC.Vecchiola,Cloudbustoolkitformarket-orientedcloudcomputing,inProceedingsofthe1stInternationalConferenceonCloudComputing(CloudCom'09),2009,pp.24. [11] I.Fujiwara,K.Aida,andI.Ono,Applyingdouble-sidedcombinationalauctionstoresourceallocationincloudcomputing,in10thIEEE/IPSJInternationalSymposiumonApplicationsandtheInternet(SAINT'10),2010,pp.7. 108

PAGE 109

[12] S.ZamanandD.Grosu,Combinatorialauction-basedallocationofvirtualmachineinstancesinclouds,inIEEECloudCom,2010,pp.127. [13] R.Ranjan,M.Rahman,andR.Buyya,Adecentralizedandcooperativeworkowschedulingalgorithm,ClusterComputingandtheGrid,IEEEInternationalSymposiumon,vol.0,pp.1,2008. [14] Z.HanandK.J.R.Liu,Noncooperativepower-controlgameandthroughputgameoverwirelessnetworks,IEEETransactionsonCommunications,vol.53,no.10,pp.1625,2005. [15] Y.-K.Kwok,S.Song,andK.Hwang,Selshgridcomputing:game-theoreticmodelingandNASperformanceresults,inCCGRID,2005,pp.1143. [16] S.Schulz,W.Blochinger,M.Held,andC.Dangelmayr,Cohesion-amicrokernelbaseddesktopgridplatformforirregulartask-parallelapplications,FutureGenerationComputerSystems,vol.24,no.5,pp.354,2008. [17] W.Cirne,F.Brasileiro,N.Andrade,L.Costa,A.Andrade,R.Novaes,andM.Mowbray,Labsoftheworld,unite!!!JournalofGridComputing,vol.4,no.3,pp.225,2006. [18] E.Deelman,G.Singh,M.Livny,B.Berriman,andJ.Good,Thecostofdoingscienceonthecloud:themontageexample,inProceedingsofthe2008ACM/IEEEconferenceonSupercomputing(SC'08),2008. [19] C.Vecchiola,S.Pandey,andR.Buyya,High-performancecloudcomputing:Aviewofscienticapplications,inProceedingsofthe200910thInternationalSymposiumonPervasiveSystems,Algorithms,andNetworks(ISPAN'09),2009,pp.4. [20] G.B.Berriman,E.Deelman,G.Juve,M.Regelson,andP.Plavchan,Theapplicationofcloudcomputingtoastronomy:Astudyofcostandperformance,CoRR,2010. [21] H.GoudarziandM.Pedram,Multi-dimensionalsla-basedresourceallocationformulti-tiercloudcomputingsystems,inIEEECloud2011,2011. [22] H.QianandD.Medhi,Serveroperationalcostoptimizationforcloudcomputingserviceprovidersoveratimehorizon,inProceedingsofthe11thUSENIXconferenceonHottopicsinmanagementofinternet,cloud,andenterprisenetworksandservices(Hot-ICE'11),2011. [23] M.Stokely,J.Winget,E.Keyes,C.Grimes,andB.Yolken,Usingamarketeconomytoprovisioncomputeresourcesacrossplanet-wideclusters,inPro-ceedingsofthe2009IEEEInternationalSymposiumonParallel&DistributedProcessing(IPDPS2009),2009,pp.1. [24] CometCloud,Available:http://nsfcac.rutgers.edu/CometCloud/,2010. 109

PAGE 110

[25] CaaSt,Available:http://4caast.morfeo-project.org/,2012. [26] H.Zhao,M.Pan,X.Liu,X.Li,andY.Fang,Optimalresourcerentalplanningforelasticapplicationsincloudmarket,inProceedingsofthe26thIEEEInternationalParallelandDistributedProcessingSymposium(IPDPS'12),2012. [27] H.Zhao,X.Liu,andX.Li,Hypergraph-basedtask-bundleschedulingtowardsefciencyandfairnessinheterogeneousdistributedsystems,inProceedingsofthe24thIEEEInternationalParallelandDistributedProcessingSymposium(IPDPS'12),2010,pp.1. [28] H.Zhao,Z.Yu,S.Tiwari,X.Mao,K.Lee,D.Wolinsky,X.Li,andR.Figueiredo,Cloudbay:Enablinganonlineresourcemarketplaceforopenclouds,inUtilityandCloudComputing(UCC),2012IEEEFifthInternationalConferenceon,2012,pp.135. [29] H.ZhaoandX.Li,Efcientgridtask-bundleallocationusingbargainingbasedself-adaptiveauction,inProceedingsofthe9thIEEE/ACMInternationalSym-posiumonClusterComputingandtheGrid(CCGrid2009),vol.0,2009,pp.4. [30] AWSCaseStudies,Available:http://aws.amazon.com/solutions/case-studies/. [31] MarketTrends:PlatformasaService,Worldwide,2012-2016,2H12Update,ID:G00239236,5October2012. [32] B.UrgaonkarandA.Chandra,Dynamicprovisioningofmulti-tierinternetapplications,inProceedingsoftheSecondInternationalConferenceonAuto-maticComputing(ICAC'05),2005,pp.217. [33] J.Zhang,J.Kim,M.Yousif,R.Carpenter,andR.J.Figueiredo,System-levelperformancephasecharacterizationforon-demandresourceprovisioning,inProceedingsofthe2007IEEEInternationalConferenceonClusterComputing(CLUSTER'07),2007,pp.434. [34] P.Padala,K.G.Shin,X.Zhu,M.Uysal,Z.Wang,S.Singhal,A.Merchant,andK.Salem,Adaptivecontrolofvirtualizedresourcesinutilitycomputingenvironments,SIGOPSOper.Syst.Rev.,vol.41,pp.289,March2007. [35] Z.Gong,X.Gu,andJ.Wilkes,Press:Predictiveelasticresourcescalingforcloudsystems,in2010InternationalConferenceonNetworkandServiceManagement(CNSM'10),2010,pp.9. [36] R.Buyya,R.Ranjan,andR.Calheiros,Intercloud:Utility-orientedfederationofcloudcomputingenvironmentsforscalingofapplicationservices,inAlgo-rithmsandArchitecturesforParallelProcessing,ser.LectureNotesinComputerScience,2010,vol.6081,pp.13. 110

PAGE 111

[37] R.JiangandY.Guan,Ano(n2)-timealgorithmforthestochasticuncapacitatedlot-sizingproblemwithrandomleadtimes,Oper.Res.Lett.,vol.39,no.1,pp.74,2011. [38] D.Yuan,Y.Yang,X.Liu,andJ.Chen,Acost-effectivestrategyforintermediatedatastorageinscienticcloudworkowsystems,in2010IEEEInternationalSymposiumonParallelDistributedProcessing(IPDPS'10),2010,pp.1. [39] H.M.Monti,A.R.Butt,andS.S.Vazhkudai,Catch:Acloud-basedadaptivedatatransferserviceforhpc,in2011IEEEInternationalParallelDistributedProcessingSymposium(IPDPS'11),2011,pp.1242. [40] V.T.Chakaravarthy,G.R.Parija,S.Roy,Y.Sabharwal,andA.Kumar,Minimumcostresourceallocationformeetingjobrequirements,in2011IEEEInternationalParallelDistributedProcessingSymposium(IPDPS'11),2011,pp.14. [41] Q.Zhang,E.Gurses,R.Boutaba,andJ.Xiao,Dynamicresourceallocationforspotmarketsinclouds,inProceedingsofthe11thUSENIXconferenceonHottopicsinmanagementofinternet,cloud,andenterprisenetworksandservices(Hot-ICE'11),2011. [42] A.K.Mishra,J.L.Hellerstein,W.Cirne,andC.R.Das,Towardscharacterizingcloudbackendworkloads:insightsfromgooglecomputeclusters,SIGMETRICSPerform.Eval.Rev.,vol.37,no.4,pp.34,2010. [43] N.Chohan,C.Castillo,M.Spreitzer,M.Steinder,A.Tantawi,andC.Krintz,Seespotrun:usingspotinstancesformapreduceworkows,inProceedingsofthe2ndUSENIXconferenceonHottopicsincloudcomputing(HotCloud'10),2010,pp.7. [44] M.Mattess,C.Vecchiola,andR.Buyya,Managingpeakloadsbyleasingcloudinfrastructureservicesfromaspotmarket,inProceedingsofthe2010IEEE12thInternationalConferenceonHighPerformanceComputingandCommunications(HPCC'10),2010,pp.180. [45] A.Andrzejak,D.Kondo,andS.Yi,Decisionmodelforcloudcomputingunderslaconstraints,inProceedingsofthe2010IEEEInternationalSymposiumonModeling,AnalysisandSimulationofComputerandTelecommunicationSystems(MASCOTS'10),2010,pp.257. [46] M.MazzuccoandM.Dumas,Achievingperformanceandavailabilityguaranteeswithspotinstances,inProceedingsofthe13thInternationalConferenceonHighPerformanceComputingandCommunications(HPCC'11),2011. [47] O.AgmonBen-Yehuda,M.Ben-Yehuda,A.Schuster,andD.Tsafrir,Deconstructingamazonec2spotinstancepricing,inIEEEThirdInternationalConferenceonCloudComputingTechnologyandScience(CloudCom2011),2011,pp.304. 111

PAGE 112

[48] S.Chaisiri,B.-S.Lee,andD.Niyato,Optimalvirtualmachineplacementacrossmultiplecloudproviders,inIEEEAsia-PacicServicesComputingConference(APSCC'09),2009,pp.103. [49] Y.Song,M.Zafer,andK.-W.Lee,Optimalbiddinginspotinstancemarket,inIEEEINFOCOM2012,2012,pp.190. [50] E.Caron,F.Desprez,andA.Muresan,Forecastingforgridandcloudcomputingon-demandresourcesbasedonpatternmatching,in2010IEEESecondInter-nationalConferenceonCloudComputingTechnologyandScience(CloudCom),2010,pp.456. [51] J.Tirado,D.Higuero,F.Isaila,andJ.Carretero,Predictivedatagroupingandplacementforcloud-basedelasticserverinfrastructures,in201111thIEEE/ACMInternationalSymposiumonCluster,CloudandGridComputing(CCGrid),2011,pp.285. [52] N.Roy,A.Dubey,andA.Gokhale,Efcientautoscalinginthecloudusingpredictivemodelsforworkloadforecasting,in2011IEEEInternationalConfer-enceonCloudComputing(CLOUD),2011,pp.500. [53] A.Demberel,J.Chase,andS.Babu,Reectivecontrolforanelasticcloudapplication:anautomatedexperimentworkbench,inProceedingsofthe2009conferenceonHottopicsincloudcomputing(HotCloud'09),2009. [54] R.K.Ahuja,T.L.Magnanti,andJ.B.Orlin,Networkows:theory,algorithms,andapplications.Prentice-Hall,Inc.,1993. [55] IBMILOGCPLEXoptimizer[online],Available:http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/. [56] AIMMSOptimizationSoftware,Available:http://www.aimms.com/. [57] J.C.Jacob,D.S.Katz,G.B.Berriman,J.C.Good,A.C.Laity,E.Deelman,C.Kesselman,G.Singh,M.Su,T.A.Prince,andR.Williams,Montage:agridportalandsoftwaretoolkitforscience-gradeastronomicalimagemosaicking,Int.J.Comput.Sci.Eng.,vol.4,pp.73,July2009. [58] EC2SpotInstance,http://aws.amazon.com/ec2/spot-instances/. [59] SpotCloud,http://www.spotcloud.com/. [60] CloudExchange,http://www.cloudexchange.org/. [61] HowtorunMapReduceinAmazonEC2spotmarket,Available:http://huanliu.wordpress.com/2011/06/22/how-to-run-mapreduce-in-amazon-ec2-spot-market/. [62] G.E.P.BoxandG.Jenkins,TimeSeriesAnalysis,ForecastingandControl.Holden-Day,Incorporated,1990. 112

PAGE 113

[63] forecastpackageforR[online],Available:http://robjhyndman.com/software/forecast/. [64] J.R.Birge,Decompositionandpartitioningmethodsformultistagestochasticlinearprograms,OperationsResearch,vol.33,no.5,pp.989,1985. [65] Y.Guan,S.Ahmed,G.L.Nemhauser,andA.J.Miller,Abranch-and-cutalgorithmforthestochasticuncapacitatedlot-sizingproblem,MathematicalProgramming,vol.105,pp.55,2006. [66] K.HuangandS.Kucukyavuz,Onstochasticlot-sizingproblemswithrandomleadtimes,OperationsResearchLetters,vol.36,no.3,pp.303308,2008. [67] H.M.WagnerandT.M.Whitin,Dynamicversionoftheeconomiclotsizemodel,Manage.Sci.,vol.50,pp.1770,dec2004. [68] G.BriscoeandA.Marinos,Digitalecosystemsintheclouds:Towardscommunitycloudcomputing,in3rdIEEEInternationalConferenceonDigitalEcosystemsandTechnologies(DEST2009),2009,pp.103. [69] EucalyptusCommunityCloud,URL: http://www.eucalyptus.com/eucalyptus-cloud/community-cloud [70] FederalCommunityCloud,URL: http://www-304.ibm.com/industries/publicsector/us/en/contentemplate1/!!/xmlid=207581 [71] A.MarinosandG.Briscoe,Communitycloudcomputing,CoRR,vol.abs/0907.2485,2009. [72] Y.Chevaleyre,U.Endriss,S.Estivie,andN.Maudet,Reachingenvy-freestatesindistributednegotiationsettings,inProceedingsoftheTwentiethInterna-tionalJointConferenceonArticialIntelligence(IJCAI2007),January2007,pp.1239. [73] Y.-K.Kwok,K.Hwang,andS.Song,Selshgrids:Game-theoreticmodelingandnas/psabenchmarkevaluation,IEEETrans.ParallelDistrib.Syst.,vol.18,pp.621,2007. [74] J.Perez,C.Germain-Renaud,B.Kegl,andC.Loomis,Multi-objectivereinforcementlearningforresponsivegrids,JournalofGridComputing,vol.8,pp.473,2010. [75] A.diCostanzo,M.D.deAssuncao,andR.Buyya,Harnessingcloudtechnologiesforavirtualizeddistributedcomputinginfrastructure,IEEEInternetComputing,vol.13,pp.24,2009. [76] S.Pandey,L.Wu,S.M.Guru,andR.Buyya,Aparticleswarmoptimization-basedheuristicforschedulingworkowapplicationsincloud 113

PAGE 114

computingenvironments,inProceedingsofthe24thIEEEInternationalCon-ferenceonAdvancedInformationNetworkingandApplications(AINA2010),2010,pp.400. [77] O.RegevandN.Nisan,ThePOPCORNmarket.onlinemarketsforcomputationalresources,DecisionSupportSystems,vol.28,no.1-2,pp.177,2000. [78] K.Lai,L.Rasmusson,E.Adar,L.Zhang,andB.A.Huberman,Tycoon:Animplementationofadistributed,market-basedresourceallocationsystem,MultiagentGridSyst.,vol.1,pp.169,2005. [79] D.Abramson,R.Buyya,andJ.Giddy,Acomputationaleconomyforgridcomputinganditsimplementationinthenimrod-gresourcebroker,FutureGener.Comput.Syst.,vol.18,pp.1061,2002. [80] R.Buyya,D.Abramson,andS.Venugopal,Thegrideconomy,ProceedingsoftheIEEE,vol.93,no.3,pp.698,2005. [81] B.Song,M.Hassan,andE.-N.Huh,Anovelcloudmarketinfrastructurefortradingservice,inProceedingsofthe2009InternationalConferenceonCompu-tationalScienceandItsApplications(ICCSA2009),2009,pp.44. [82] B.CooperandH.Garcia-Molina,Peer-to-peerresourcetradinginareliabledistributedsystem,intheFirstInternationalWorkshoponPeer-to-PeerSystems(IPTPS2001),2002,pp.319. [83] D.A.TurnerandK.W.Ross,ALightweightCurrencyParadigmfortheP2PResourceMarket,inSeventhInternationalConferenceonElectronicCommerceResearch,2004. [84] P.Gradwell,Distributedcombinatorialresourcescheduling,in1stInternationalWorkshoponSmartGridTechnologies,2005. [85] H.Boche,S.Naik,andT.Alpcan,Characterizationofnon-manipulableandparetooptimalresourceallocationstrategiesforinterferencecoupledwirelesssystems,inProceedingsofthe29thconferenceonInformationcommunications(INFOCOM2010),2010,pp.2417. [86] J.EdmondsandK.Pruhs,Balancedallocationsofcake,inProceedingsofthe47thAnnualIEEESymposiumonFoundationsofComputerScience(FOCS2006),2006,pp.623. [87] K.Rzadca,D.Trystram,andA.Wierzbicki,Fairgame-theoreticresourcemanagementindedicatedgrids,inProceedingsoftheSeventhIEEEInterna-tionalSymposiumonClusterComputingandtheGrid(CCGrid2007),2007,pp.343. 114

PAGE 115

[88] K.Berlt,E.S.D.Moura,A.L.D.C.Carvalho,M.Cristo,N.Ziviani,andT.Couto,Ahypergraphmodelforcomputingpagereputationonwebcollections,inBrazilianSymposiumonDatabases,2007,pp.35. [89] S.SarkarandK.Sivarajan,Hypergraphmodelsforcellularmobilecommunicationsystems,IEEETransactionsonVehicularTechnology,vol.47,no.2,pp.460,1998. [90] S.Krishnamoorthy,U.Catalyurek,J.Nieplocha,A.Rountev,andP.Sadayappan,Hypergraphpartitioningforautomaticmemoryhierarchymanagement,inProceedingsofthe2006ACM/IEEEconferenceonSupercomputing(SC2006),2006. [91] U.CatalyurekandC.Aykanat,Hypergraph-partitioning-baseddecompositionforparallelsparse-matrixvectormultiplication,IEEETrans.ParallelDistrib.Syst.,vol.10,no.7,pp.673,1999. [92] S.J.BramsandA.D.Taylor,Fairdivision:fromcake-cuttingtodisputeresolution.CambridgeUniversityPress,1996. [93] N.Laoutaris,L.J.Poplawski,R.Rajaraman,R.Sundaram,andS.-H.Teng,Boundedbudgetconnection(bbc)gamesorhowtomakefriendsandinuencepeople,onabudget,inProceedingsofthetwenty-seventhACMsymposiumonPrinciplesofdistributedcomputing(PODC2008),2008,pp.165. [94] S.Estivie,Y.Chevaleyre,U.Endriss,andN.Maudet,Howequitableisrationalnegotiation?inProceedingsofthefthinternationaljointconferenceonAu-tonomousagentsandmultiagentsystems(AAMAS2006),2006,pp.866. [95] Y.Chevaleyre,U.Endriss,andN.Maudet,Allocatinggoodsonagraphtoeliminateenvy,inProceedingsofthe22ndAAAIConferenceonArticialIntelli-gence(AAAI2007),July2007,pp.700. [96] B.Awerbuch,Optimaldistributedalgorithmsforminimumweightspanningtree,counting,leaderelection,andrelatedproblems,inProceedingsofthenineteenthannualACMsymposiumonTheoryofcomputing(STOC1987),1987,pp.230. [97] J.MatochaandT.Camp,Ataxonomyofdistributedterminationdetectionalgorithms,J.Syst.Softw.,vol.43,no.3,pp.207,1998. [98] DistributedTerminationDetection,URL: http://www.risc.jku.at/software/daj/TermDetect/ [99] P.Hunt,M.Konar,F.P.Junqueira,andB.Reed,Zookeeper:wait-freecoordinationforinternet-scalesystems,inProceedingsofthe2010USENIXconferenceonUSENIXannualtechnicalconference,2010,pp.11. 115

PAGE 116

[100] I.TomescuandM.Zimand,Minimumspanninghypertrees,DiscreteAppliedMathematics,vol.54,no.1,pp.67,1994. [101] H.Casanova,A.Legrand,andM.Quinson,SimGrid:aGenericFrameworkforLarge-ScaleDistributedExperiments,in10thIEEEInternationalConferenceonComputerModelingandSimulation(UKSIM2008),2008. [102] R.T.Ma,D.M.Chiu,J.C.Lui,V.Misra,andD.Rubenstein,Onresourcemanagementforcloudusers:Ageneralizedkellymechanismapproach,ElectricalEngineering,Tech.Rep.,May2010. [103] HPConAWS,http://aws.amazon.com/hpc-applications/,2012. [104] O.Krieger,P.McGachey,andA.Kanevsky,Enablingamarketplaceofclouds:VMware'svClouddirector,SIGOPSOper.Syst.Rev.,vol.44,pp.103,2010. [105] H.Abbes,C.Cerin,andM.Jemni,Bonjourgrid:Orchestrationofmulti-instancesofgridmiddlewaresoninstitutionaldesktopgrids,inProceedingsofthe2009IEEEInternationalSymposiumonParallel&DistributedProcessing(IPDPS'09),2009,pp.1. [106] C.BaileyLee,Y.Schwartzman,J.Hardy,andA.Snavely,Areuserruntimeestimatesinherentlyinaccurate?inProceedingsofthe10thinternationalconfer-enceonJobSchedulingStrategiesforParallelProcessing(JSSPP'04),2005,pp.253. [107] C.Waldspurger,T.Hogg,B.Huberman,J.Kephart,andW.Stornetta,Spawn:adistributedcomputationaleconomy,IEEETransactionsonSoftwareEngineering,vol.18,no.2,pp.103,1992. [108] O.RegevandN.Nisan,ThePOPCORNmarketanonlinemarketforcomputationalresources,inProceedingsoftherstinternationalconferenceonInformationandcomputationeconomies(ICE'98),1998,pp.148. [109] L.Kale,S.Kumar,M.Potnuru,J.DeSouza,andS.Bandhakavi,Faucets:efcientresourceallocationonthecomputationalgrid,inProceedingsofthe2004InternationalConferenceonParallelProcessing(ICPP'04),2004,pp.396. [110] Y.M.TeoandM.Mihailescu,Astrategy-proofpricingschemeformultipleresourcetypeallocations,inProceedingsofthe2009InternationalConferenceonParallelProcessing(ICPP'09),2009,pp.172. [111] T.E.CarrollandD.Grosu,Incentivecompatibleonlineschedulingofmalleableparalleljobswithindividualdeadlines,inProceedingsofthe201039thInterna-tionalConferenceonParallelProcessing(ICPP'10),2010,pp.516. [112] S.Krishnan,M.Tatineni,andC.Baru,myhadoop-hadoop-on-demandontraditionalhpcresources,SanDiegoSupercomputerCenter,UniversityofCaliforniaSanDiego,Tech.Rep.SDSCTR-2011-2,2011. 116

PAGE 117

[113] CondorProjectHomePage,http://research.cs.wisc.edu/condor/,2009. [114] A.Ganguly,A.Agrawal,O.P.Boykin,andR.Figueiredo,IPoverP2P:Enablingself-conguringvirtualIPnetworksforgridcomputing,inProceedingsofthe20thinternationalconferenceonparallel&distributedprocessing(IPDPS'06),2006. [115] D.I.Wolinsky,Y.Liu,P.S.Juste,G.Venkatasubramanian,andR.Figueiredo,Onthedesignofscalable,self-conguringvirtualnetworks,inProceedingsoftheConferenceonHighPerformanceComputingNetworking,StorageandAnalysis(SC'09),2009,pp.13:1:12. [116] D.WolinskyandR.Figueiredo,Experienceswithself-organizaing,decentralizedgridsusingthegridappliance,inProceedingsofthe20thinternationalsympo-siumonHighperformancedistributedcomputing(HPDC'11),2011. [117] Django,https://www.djangoproject.com/,2012. [118] D.I.WolinskyandR.Figueiredo,Gridapplianceuserinterface, http://www.grid-appliance.org ,September2009. [119] B.G.LawsonandE.Smirni,Multiple-queuebackllingschedulingwithprioritiesandreservationsforparallelsystems,SIGMETRICSPerform.Eval.Rev.,vol.29,pp.40,2002. [120] S.Nakamoto.(2010)Bitcoin:APeer-to-PeerElectronicCashSystem.[Online].Available: http://bitcoin.org/bitcoin.pdf [121] T.Sandholm,eMediator:anextgenerationelectroniccommerceserver,inProceedingsofthefourthinternationalconferenceonAutonomousagents(AGENTS'00),2000,pp.341. [122] L.M.Ausubel,Anefcientascending-bidauctionformultipleobjects,AmericanEconomicReview,vol.94,no.5,pp.1452,2004. [123] S.Shudler,L.Amar,A.Barak,andA.Mu'alem,Theeffectsofuntruthfulbidsonuserutilitiesandstabilityincomputingmarkets,inproceedingsofthe10thIEEE/ACMInternationalConferenceonCluster,CloudandGridComputing(CCGrid'10),2010,pp.205. [124] FutureGrid,Available:http://futuregrid.org/,2012. [125] M.Zaharia,A.Konwinski,A.D.Joseph,R.Katz,andI.Stoica,Improvingmapreduceperformanceinheterogeneousenvironments,inOSDI'08,2008,pp.29. [126] Simpy,Available:http://simpy.sourceforge.net/,2012. 117

PAGE 118

[127] P.Marshall,K.Keahey,andT.Freeman,Improvingutilizationofinfrastructureclouds,inProceedingsofthe11thIEEE/ACMInternationalSymposiumonCluster,CloudandGridComputing(CCGrid'11),2011,pp.205. 118

PAGE 119

BIOGRAPHICALSKETCH HanZhaogrewupinthecityofDalian,China,wherehegraduatedfromDalianYumingSeniorHighSchoolinyear2002.HeearnedhisB.E.inSoftwareEngineeringfromJilinUniversity(China)insummer2006,hisM.S.inComputerSciencefromOklahomaStateUniversityinfall2010,andexpectstoreceivehisPh.D.inComputerEngineeringfromUniversityofFloridainMay2013.Duringsummer2010,heworkedasaresearchinterninIBMT.J.WatsonResearchCenter,NY.Hisresearchinterestsincludeparallelanddistributedcomputing,cyber-physicalsystems,andcomputernetworking.HeisastudentmemberofACMandIEEE. 119