<%BANNER%>

Data Mining Methods with Applications in Biomedicine

MISSING IMAGE

Material Information

Title:
Data Mining Methods with Applications in Biomedicine
Physical Description:
1 online resource (103 p.)
Language:
english
Creator:
Korenkevych, Dmytro
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Industrial and Systems Engineering
Committee Chair:
Pardalos, Panagote M
Committee Co-Chair:
Momcilovic, Petar
Committee Members:
Boginskiy, Vladimir L
Bliznyuk, Nikolay A

Subjects

Subjects / Keywords:
brain -- classification -- data -- networks -- prediction
Industrial and Systems Engineering -- Dissertations, Academic -- UF
Genre:
Industrial and Systems Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Biomedical applications present rich opportunities and a high demand for data mining techniques. Biological processes and functions of a human body are very complex and many of them are not well understood. Therefore there are no suitable analytical or modeling tools that could be used for precise description of these processes. In contrast, data mining techniques can provide insights and discover empirical patterns hidden in the data that can shed light onto physiological nature of underlying processes. With the development of modern information systems and medical data acquisition techniques the amount of available data has exploded exponentially, creating the need for specialized methods and algorithms. The objective of this work is to explore data mining methods and algorithms that are well suited for biomedical data analysis, and to adapt these method to solve some of the practical problems arising in biomedical domain.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Dmytro Korenkevych.
Thesis:
Thesis (Ph.D.)--University of Florida, 2013.
Local:
Adviser: Pardalos, Panagote M.
Local:
Co-adviser: Momcilovic, Petar.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2015-08-31

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2013
System ID:
UFE0045825:00001

MISSING IMAGE

Material Information

Title:
Data Mining Methods with Applications in Biomedicine
Physical Description:
1 online resource (103 p.)
Language:
english
Creator:
Korenkevych, Dmytro
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Industrial and Systems Engineering
Committee Chair:
Pardalos, Panagote M
Committee Co-Chair:
Momcilovic, Petar
Committee Members:
Boginskiy, Vladimir L
Bliznyuk, Nikolay A

Subjects

Subjects / Keywords:
brain -- classification -- data -- networks -- prediction
Industrial and Systems Engineering -- Dissertations, Academic -- UF
Genre:
Industrial and Systems Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Biomedical applications present rich opportunities and a high demand for data mining techniques. Biological processes and functions of a human body are very complex and many of them are not well understood. Therefore there are no suitable analytical or modeling tools that could be used for precise description of these processes. In contrast, data mining techniques can provide insights and discover empirical patterns hidden in the data that can shed light onto physiological nature of underlying processes. With the development of modern information systems and medical data acquisition techniques the amount of available data has exploded exponentially, creating the need for specialized methods and algorithms. The objective of this work is to explore data mining methods and algorithms that are well suited for biomedical data analysis, and to adapt these method to solve some of the practical problems arising in biomedical domain.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Dmytro Korenkevych.
Thesis:
Thesis (Ph.D.)--University of Florida, 2013.
Local:
Adviser: Pardalos, Panagote M.
Local:
Co-adviser: Momcilovic, Petar.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2015-08-31

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2013
System ID:
UFE0045825:00001


This item has the following downloads:


Full Text

PAGE 1

DATAMININGMETHODSWITHAPPLICATIONSINBIOMEDICINEByDMYTROKORENKEVYCHADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2013

PAGE 2

c2013DmytroKorenkevych 2

PAGE 3

Tomyparents 3

PAGE 4

ACKNOWLEDGMENTS IwouldliketothankallthepeoplewithwhomIworkedandstudiedtogetherduringmyPhDprogram.Ithasbeenachallengingandanexcitingtimeandyoumadeitatrulyspecialexperience.IwouldliketoacknowledgemyscienticadvisorDr.PanosM.Pardalosforallhissupportandguidanceandforconstantencouragementtodoexcellentwork.Also,Iwouldliketoacknowledgemymasterthesisadvisor,professorVladimirShylofromtheInstituteofCyberneticsinKiev,Ukraine,whointroducedmeintoworld-classscienticresearchandopenedformeawidehorizonofscienticcareeropportunities.IwouldliketothankmyresearchcollaboratorsPetarMomcilovic,FrankSkidmore,SyedMujahid,AzraBihorac,TezcanOzrazgatBaslantiandAlexSavenkov,withwhomithasbeenapleasuretowork. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 10 CHAPTER 1INTRODUCTION ................................... 11 1.1TheRoleofDatainModernScience ..................... 11 1.2DataMininginBiomedicalDomain ...................... 12 1.3ThesisStructure ................................ 13 2NETWORKMODELSINCOMPUTATIONALNEUROSCIENCE ......... 14 2.1ReviewonHumanBrainNetworksModeling ................. 14 2.1.1Background ............................... 14 2.1.2SmallWorldNetworks.ModelsandProperties. ........... 15 2.1.2.1Characteristicpathlengthandclusteringcoefcient. ... 17 2.1.2.2Networkefciencyandcost. ................ 19 2.1.3BrainNetworks ............................. 28 2.2ConnectivityBrainNetworksBasedonWaveletAnalysis .......... 30 2.2.1Background ............................... 30 2.2.2Methods ................................. 32 2.2.2.1Functionalimagingparameters ............... 34 2.2.2.2Waveletcorrelationanalysisandgraphconstruction ... 35 2.2.3Results ................................. 39 2.2.3.1Globalmeanefciency ................... 39 2.2.3.2Meannodalefciency,top30nodes ............ 40 2.2.3.3Networkefciencydecrement ............... 41 2.2.3.4Prediagnosticsubject .................... 42 2.3OptimalPropertiesofHumanBrainNetworks ................ 42 2.3.1GraphAnalysis ............................. 43 2.3.2Results ................................. 45 2.3.2.1Clusteringcoefcient .................... 45 2.3.2.2Networkefciency ...................... 46 2.3.2.3Characteristicpathlength .................. 47 3PROBABILISTICCLASSIFIERSINBIOMEDICALAPPLICATIONS ....... 49 3.1Background ................................... 49 3.2TheProblemofClassication ......................... 50 5

PAGE 6

3.3ClassicationAlgorithmsWithProbabilisticOutputs ............ 52 3.3.1LogisticRegression ........................... 52 3.3.2BayesianLearning ........................... 53 3.3.3GeneralizedAdditiveModels ..................... 56 3.4ObtainingPosteriorProbabilitiesForNon-ProbabilisticClassiers ..... 58 3.5EvaluatingthePerformanceofaClassier .................. 62 3.6AssessingMortalityRiskinPost-OperativePatients ............ 69 3.6.1Background ............................... 69 3.6.2ProbabilisticModelBasedonLogisticFunction ........... 71 3.6.2.1Probabilisticmodel ...................... 71 3.6.2.2Riskfactorsanalysis ..................... 73 3.6.2.3Assessingthetofthemodel ................ 81 3.6.2.4Results ............................ 82 3.6.3GeneralizedAdditiveModels ..................... 86 3.6.3.1Probabilisticmodel ...................... 86 3.6.3.2Riskfactorsanalysis ..................... 87 3.6.3.3Results ............................ 88 4CONCLUSIONS ................................... 95 REFERENCES ....................................... 97 BIOGRAPHICALSKETCH ................................ 103 6

PAGE 7

LISTOFTABLES Table page 2-1Demographicsofthedataset. ........................... 33 3-1Weightsfwigassociatedwithriskfactors. ..................... 72 3-2CharacteristicsofthepatientsinthedatasetwithRRTpatientsexcluded. ... 84 3-3Modeldiscriminativeability. ............................. 85 3-4Featureselectionanddegreesoffreedom. .................... 89 3-5Vuong'stestformodelcomparison. ......................... 89 3-6Modeldiscriminativeability(GAMs). ........................ 90 7

PAGE 8

LISTOFFIGURES Figure page 2-1Smallworldversusregular/randomnetworks[ 81 ]. ................ 18 2-2Clusteringcoefcientversusmeanshortestpathlength[ 81 ]. .......... 19 2-3GlobalandlocalefcienciesforWattsandStrogatzmodel[ 48 ]. ......... 24 2-4FunctionsElocal(p)=Elocal(0)andEglobal(0)=Eglobal(0)exhibitbehavior,similartoL(p)andC(p)respectively[ 48 ]. .......................... 25 2-5PlotsofEglobalandElocalasfunctionsofCostforsecondmodel[ 48 ]. ...... 26 2-6Eglobal,ElocalandCostasfunctionsofpforthethirdmodel. ............ 27 2-7Eglobal,ElocalandCostasfunctionsofpforthefourthmodel. ........... 28 2-8Geometricalrepresentationoftimeseries. ..................... 37 2-9Connectivitynetworks ................................ 38 2-10GlobalefciencyvaluesforControlandParkinsonsets. ............. 39 2-11Top30nodalefciencies .............................. 40 2-12MeanDecrementinEfciencyAcrossNodes. ................... 41 2-13Networkwithhighestclusteringcoefcientandlowestcharacteristicpathlengthoforder10andsize24. ............................... 45 2-14Clusteringcoefcientversusnetworkdensity. ................... 46 2-15Networkefciencyversusnetworkdensity. .................... 47 2-16Characteristicpathlengthversusnetworkdensity. ................ 48 3-1SupportVectorMachinemethod. .......................... 58 3-2AnexampleofaROCcurve. ............................ 64 3-3Atoyexampledecisiontree[ 77 ]. .......................... 67 3-4AdecisioncurveforSVIdata. ............................ 68 3-5ROCcurvesdemonstratingmodelpredictingperformanceondifferentsetsoffeatures. ....................................... 72 3-6Truepositiveandtruenegativeratesofthemodeldependingonthethreshold. 73 3-7Learnedriskfactorsweightsfwig. ......................... 74 8

PAGE 9

3-8Oddsratioscomputedwithrespecttoanapriorioddsinthedataset(sashedlineonthegures). .................................. 75 3-9ProcedurestreebasedonICD9codes. ...................... 78 3-10Agevaluesdistributions ............................... 78 3-11LearnedmixtureoftwoconditionaldistributionsformaxCr/minCrvalues. .... 80 3-12LearnedmixtureoftwoconditionaldistributionsforrecentCr/minCrvalues. .. 81 3-13Comparisonbetweenpredictedscoreandestimatedrisk ............ 82 3-14Thelogoddsratiosforin-hospitalmortalitybasedonsCrfactors. ........ 86 3-15OddsratiosforparticularsCrchangepatternscomputedwithrespecttoanapriorioddsinthedataset(0.02). .......................... 86 3-16NonlinearfunctionsforsCrfactorspredictingin-hospitalmortality. ........ 91 3-17NonlinearfunctionsforsCrfactorspredicting90-daysmortality. ......... 92 3-18Nonlinearfunctionsforpredictingin-hosp.mortality. ............... 93 3-19Nonlinearfunctionsforpredicting90-daysmortality. ............... 93 3-20Oddsratioswith95%predictiveintervalsforsCrchangepatterns(in-hosp.mortality). ....................................... 94 3-21Oddsratioswith95%predictiveintervalsforsCrchangepatterns(90-daysmortality). ....................................... 94 9

PAGE 10

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyDATAMININGMETHODSWITHAPPLICATIONSINBIOMEDICINEByDmytroKorenkevychAugust2013Chair:PanosM.PardalosMajor:IndustrialandSystemsEngineering Biomedicalapplicationspresentrichopportunitiesandahighdemandfordataminingtechniques.Biologicalprocessesandfunctionsofahumanbodyareverycomplexandmanyofthemarenotwellunderstood.Thereforetherearenosuitableanalyticalormodelingtoolsthatcouldbeusedforprecisedescriptionoftheseprocesses.Incontrast,dataminingtechniquescanprovideinsightsanddiscoverempiricalpatternshiddeninthedatathatcanshedlightontophysiologicalnatureofunderlyingprocesses.Withthedevelopmentofmoderninformationsystemsandmedicaldataacquisitiontechniquestheamountofavailabledatahasexplodedexponentially,creatingtheneedforspecializedmethodsandalgorithms.Theobjectiveofthisworkistoexploredataminingmethodsandalgorithmsthatarewellsuitedforbiomedicaldataanalysis,andtoadaptthesemethodtosolvesomeofthepracticalproblemsarisinginbiomedicaldomain. 10

PAGE 11

CHAPTER1INTRODUCTION 1.1TheRoleofDatainModernScience Intheageofmoderncomputingfacilitiesanddataacquisitiontechniquesthewayscienticndingsandinferencearemadeischanging.Thetraditionalsciencewasbuiltaroundtheprinciple:hypothesis!model!validationthroughtheexperiments.Oncethemodelwasconrmed,theconclusionsandtheknowledgeaboutthesubjectwereobtainedbasedontheproposedmodel.Dataservedprimarilyasamodelvalidation/rejectiontool.Ithasalwaysbeenrecognizedthatthemodelsareimpreciseandatbestcanbeconsideredasapproximationstotherealworld.AsstatisticianGeorgeBoxproclaimed,Allmodelsarewrong,butsomeareuseful.Withmodernworld'svastamountsofdatathetraditionalapproachbecomesobsoleteincertainareasofscience.Thereisnoneedtodevelopanddrawknowledgefromapproximatemodelsanymore.Theknowledgecanbeobtaineddirectlyfromthedatawithoutintermediariesbyusingspecializedcomputationalmethods.Onecananalyzethedatawithouthypothesesaboutwhatitmightrepresentbylettingdataminingalgorithmsndpatternswheresciencecannot.Theamountsofdatacollectedmakesthebestvalidationforobtainedempiricalknowledge.IfthedatashowsthereisarelationshipbetweenXandY,thereisnoneedtoprovideatheoreticalproof:wecanbecondentthatthedatatellthetruth. Theproblemofextractingknowledgefromthedatahasbeenstudiedformanydecades.Researchesfromdifferentdisciplineshavecontributedtotheeldofdataanalysisandinformationextraction.InstatisticaldisciplinesthiseldisoftenreferredasDataMining,whileincomputersciencedisciplinesMachineLearningisamorecommonterm.Theproblemsandtheobjectivesinthisdomaindifferfromapplicationtoapplicationbutthecommonpropertyisalargeamountofdatawithnoclearoreasilyobtainablewaytolearntheinformationofinterest. 11

PAGE 12

1.2DataMininginBiomedicalDomain Biomedicalapplicationspresentrichopportunitiesandahighdemandfordataminingtechniquesforseveralreasons.First,thebiologicalprocessesandfunctionsofahumanbodyareverycomplexandmanyofthemarenotwellunderstood.Thereforetherearenosuitableanalyticalormodelingtoolsthatcouldbeusedforprecisedescriptionoftheseprocesses.Incontrast,dataminingtechniquescanprovideinsightsanddiscoverempiricalpatternshiddeninthedatathatcanshedlightontobiologicalnatureofunderlyingprocesses.Second,withthedevelopmentofmoderninformationsystemsandmedicaldataacquisitiontechniquestheamountofavailabledatahasexplodedexponentially,creatingtheneedforspecializedmethodsandalgorithmsinordertoprocessitefciently. Dataminingmethodsandapproachescanbeclassiedintoseveralcategories,includingdatapreprocessing,unsupervisedlearning,supervisedlearningandfeatureextraction.Althoughinbiomedicalapplicationssupervisedlearningmethodsareusuallyofultimateinterest,theirsuccessfulapplicationisimpossiblewithoutpriordatapreprocessing,dimensionalityreductionanduseofunsupervisedlearningmethods.Generallybiomedicaldataarehighlycontaminatedwithnoise,missingvalues,errorsandartifactsduetothewaydataacquisitionprocessiscurrentlysetinclinics.Highdimensionalityandredundancyofavailabledatamakesitimpossibletodirectlyapplysupervisedlearningalgorithmsinordertostudytheeffectofinterest.Dimensionalityreductionandfeatureselectionmethodsmustbecarefullyappliedinordertotransformthedataintosuitableforanalysisform.Thecomplexityofunderlyingprocessesrepresentedinbiomedicaldatamakessimpleapproachesandstraightforwardmethodstoperformpoorlyinbiomedicalapplications.Sophisticatedalgorithmsbasedoncomplexmathematicalconceptsareusuallyneededinordertoachievedesirableresults.Therefore,biomedicaldomaindrivesthedevelopmentofspecializeddataminingmethodsandapproachesthatwouldbeabletofullltheneedsofmodernmedicalcare. 12

PAGE 13

1.3ThesisStructure Theobjectiveofthisworkistoexploredataminingmethodsandtheirapplicationtosolvepracticalproblemsfrombiomedicaldomain.Thethesisisorganizedasfollows.InChapter 2 wewillexploretheconceptofnetworkmodelsofahumanbrainandwilldiscusstheapproachestoconstructsuchmodelsfromtherealdata.InSection 2.2 wewillapplythisconcepttoaproblemofParkinson'sdiseasediagnostics.InChapter 2.3 wewillexploreseveralcriticalpropertiesofhumanbrainnetworksandwillshowthathumanbrainexhibitsnearlyoptimalbehaviorinthecontextofinformationtransferandresilience.InChapter 3 wewillexploreprobabilisticclassicationalgorithmsusedinbiomedicaldomainandmethodstoassesstheperformanceofsuchalgorithms.InSection 3.6 wewillapplyprobabilisticclassicationalgorithmsinordertosolveaproblemofpredictingin-hospitalmortalityinpost-operativepatients. 13

PAGE 14

CHAPTER2NETWORKMODELSINCOMPUTATIONALNEUROSCIENCE 2.1ReviewonHumanBrainNetworksModeling 2.1.1Background Humanbrainisoneofthemostcomplexobjectsinnature.Despitedecadesofstudying,understandingofitsdynamicsandfunctionprinciplesisstillamongthelargestchallengesinmodernscience.Modernmedicalexaminationdevicesprovidevastamountofinformationaboutbrainstructureandfunction.Duetothecomplexityofunderlyingprocessesthesedatapresentaninterestingandcomplicatedproblemtothedataminingeld.Anotherspecicofbrainactivitydataisalargeamountofnoiseandartifactsduetothedifcultytoaccessbraindirectlywithrecordingdevices.Thechallengealsopartiallyliesinthefactthatitisdifculttocollectsufcientlylargenumberofhumanbraindatarecordings.Thetypicaldatasetscontainarelativelysmallnumberofsubjects(e.g.patients),whereeachsubjectisdescribedbyalargehigh-dimensionaldataset.ProblemsofthissortareespeciallydifculttosolveduetothesocalledCurseofDimensionalityandovertting. Amongthenovelapproachestoanalyzehumanbrainactivitydataisnetworkmodeling.Overthelastdecadesnetworkmodelshavebeenwidelyappreciatedasanessentialpartinstudyingcomplexsystems[ 71 ],[ 4 ],[ 16 ].Networksprovideinformationonwhichsystemcomponentsinteractwitheachotherputtingasidethemechanismsoftheseinteractions.Thedynamicsofthesystemistightlyconnectedtothestructureofunderlyingnetwork.Inmanycasesitiseasytodescribethebehavioroftinysystemcomponentsonmicroscopiclevel,butmuchmorechallengingtounderstandcomplexmacroscopicsystemprocesses.Studyingthetopologicalpropertiesofsystemnetworkscanprovideaninsightaboutthesystem'shighlevelstructureandfunction. Thehumanbrainisatypicalexampleofacomplexsystem.Althoughthebasicmechanismsandfunctionprinciplesonsingleneuronslevelareunderstood,notmuch 14

PAGE 15

progresshasbeenachievedinunderstandingglobalbrainprocessesandfunction.Itsabilitytogatherandintegrateinformationfromnumeroussourcesandgenerateresponsesistightlyconnectedtothecomplicatedstructureofneuralconnectionsnetwork[ 20 ],[ 72 ].Studyingneuralactivityonthehighlevelofinteractionpatternsbetweenlargeneuronpopulationsandsegregatedanatomicalregionsmayhelptounderstandfundamentallawsofbrainfunction. Ithasbeenfound,thatmanycomplexnetworksinphysicsandbiologyfollowcommonorganizationalprinciples[ 6 ],[ 5 ],[ 66 ].Thesenetworksaredenselyconnectedonthelocal(neighborhood)level,butatthesametimehavecertainnumberoflongrangeconnectionslinkingnodes,whichotherwisewouldbefarapart.WattsandStrogatzintheirseminalpaper[ 81 ]calledsuchnetworkssmall-worlds,byanalogywithsmall-worldphenomenondescribedbyMilgramin1960s.Afterthisrstpapernumerousstudieshavereportedsmall-worldpropertiesindifferentkindsofnetworks,inparticular,brainnetworks. Small-worldnetworksareattractivemodelsofthebrain,sincetheytendtooptimizetheefciencyofparallelinformationtransferthroughthenetwork,andatthesametimeremainfaulttolerant,i.e.resilienttodamageofsinglenodesorconnections[ 48 ],[ 7 ],[ 41 ],[ 46 ],[ 50 ].Thelocaldensityofbrainconnectionsisratherintuitivelyexpected,sincelongwiringconnectionsareexpensivefrommaterialandenergypointsofview[ 23 ],[ 24 ],[ 45 ],[ 22 ].However,computationalstudiesshowed,thatbraindoesnotexhibitpurelylocalconnectivitystructure,havingcertainnumberofconsistentlongrangeconnectionsbetweendistantareas.Thissuggests,thatbrainnetworkspossessneitherrandomnorcompletelyregularstructure.Multiplestudieshaveshown,thattopologyofbrainnetworksatallscaleshassmallworldattributes. 2.1.2SmallWorldNetworks.ModelsandProperties. Thesmall-worldphenomenonrstwasdenedinsocialsciences,anareawheresuchnetworksnaturallyarise.In1960psychologistfromYaleUniversityStanleyMilgram 15

PAGE 16

hasconductedseriesofexperiments,wherehetriedtoestimateaveragedistancethroughtheacquaintancesbetweenpairsofpeopleintheworld[ 55 ].Resultsofhisexperimentswhereratherstriking.Hehasfound,thataveragedistancethroughtheacquaintancesbetweenanytwopeopleiscloseto6.Keepinginmindthetotalsizeofworld'spopulation,thisincrediblylownumberindicates,thatthesocialnetworkofacquaintanceshassomeveryspecialstructure.Milgramhascalledthisndingasmall-worldphenomenon.Suchshortsequencesofacquaintancesbetweenpairsofpeopleareespeciallysurprising,becausesocialnetworksareknowntohavelocallyhighdensehighclusteredstructure,andmostofthepairwisedistancesbetweenpeoplefromdifferentclustersareexpectedtobelong. MathematicallyMilgram'sndingscanbeformulatedaslowvalueoftheaverageshortestpathlengthbetweennodesinthedense,highlyclusterednetwork.Sinceinconsideredsocialnetworktheconnectionsaresimplybinary:twonodesareeitherconnectedornot,itisthetopologyofthenetwork,thatmakesitsmall-world.Similarpropertieslaterwerefoundinnumeroussocial,economicandbiologicalnetworks.Thisgaverisetoaninterestinstudyingthepropertiesandtopologyofsmallworldnetworks. Therearetwomainapproachesinquantifyingsmallworldnetworkproperties.Onecomprisesevaluationofnetworkcharacteristicpathlengthandclusteringcoefcient,anotherfocusesonquantifyingnetworkefciencyproperties.Smallworldnetworkshavebeenshowntohavelowvaluesofcharacteristicpathlengthwhilemaintaininghighclusteringcoefcient.Thesepropertiescanbereformulatedintermsofoneintegratedmeasureofnetworksefciency:smallworldnetworkshavehighvaluesofnetworksefciencyonbothglobalandlocallevels.Belowwegiveformaldenitionstothesemeasuresanddiscusstherelationshipsbetweenthem.Also,wegiveareviewofmodelsofnetworktopologywhichgiverisetosmallworldattributes. 16

PAGE 17

2.1.2.1Characteristicpathlengthandclusteringcoefcient. ConsideranetworkG=fN,Eg,whereNissetofnodes,andEisasetofedges.Letdijbethelengthoftheshortestpathbetweennodesiandjthroughtheconnectionsinthenetwork.Thislengthcomputationallycanbedeterminedbyvarietyofsimplealgorithms,likebreadth-rstsearch,orDijkstra'salgorithm.ThecharacteristicpathlengthLofthenetworkGisdened,asL=1 n(n)]TJ /F5 11.955 Tf 11.95 0 Td[(1)X(i,j)2Adij, wherenisasizeofsetN(numberofnodesinthenetwork). ConsideranarbitrarynodeiofthenetworkG.LetGibeasubgraphofG,containingnodes,adjacenttoi.ThelocalclusteringcoefcientCiisdened,asCi=#ofarcsinGi ki(ki)]TJ /F5 11.955 Tf 11.96 0 Td[(1), wherekiisanumberofnodesinGi.Therefore,CiisafractionofarcsinsubgraphGidividedbymaximumpossiblenumberofarcs,whenGiisacompletegraph.TheglobalclusteringcoefcientCisdenedasameanoflocalclusteringcoefcients:C=1 nXi2NCi. Thecharacteristicpathlengthmeasuresthetypicaldistancebetweenpairsofnodes,whichistheglobalpropertyofthenetwork,whileclusteringcoefcientmeasurestheaveragecliquishness,orlocalconnectednessofneighborhoodsinthenetwork,whichisalocalproperty.Characteristicpathlengthandclusteringcoefcientareapplicabletotheconnectednetworkswithbinaryweightsofedges.Twotypicalfamiliesofbinarynetworkswithsimplyexpressedtopologyareregularnetworksorlatticesandrandomnetworks.Regularnetworkscanbethoughtofasnetworks,wherenodesareplacedonagridandtheneighboringnodesareconnectedbyanedge.Randomnetworks,ontheotherhand,arethenetworks,whereconnectionsarethrown 17

PAGE 18

betweennodesrandomlyaccordingtouniformdistribution.Thetopologyofsmall-worldnetworksliessomewherebetweenthesetwoextremecases.AsimplemodelofbinarynetworksexhibitingsmallworldtopologywasproposedbyWattsandStrogatz[ 81 ].Thenetworksweregeneratedfromregularnetworksbyrandomrewiringsomeoftheedges,introducingdegreeofdisorderintonetworktopology.Bychangingthenumberofrewirededgesthenetworkscanbetunedbetweenregularandrandomstates,passingthroughtherangeofintermediatetopologiesinbetween(Figure 2-1 ).Forcertain Figure2-1. Smallworldversusregular/randomnetworks[ 81 ]. rangeofrewiringprobabilityptheresultingnetworkswerehighlyclustered,likeregularlattices,yethadsmallcharacteristicpathlengths,likerandomgraphs(Figure 2-2 ).WattsandStrogatzcalledthesenetworkssmall-worldsbyanalogywiththesmall-worldphenomenon[ 81 ].Itturnsoutthatthesefeaturesarepresentinmanyrealworldnetworks.FirstsuchresultswereobtainedbyWattsandStrogatzinthesamework[ 81 ],weretheyconsideredcollaborationnetworkofactorsinlms,theelectricalpowergridofthewesternUnitedStatesandtheneuralnetworkofthenematodewormC.elegans.Allthreerealnetworksofcompletelydifferentnatureexhibitedsmall-worldproperties,havinglowcharacteristicpathlengthandhighclusteringcoefcients.Thisindicates,thatthesmall-worldphenomenonisnotmerelyacuriosityofsocialnetworksnoranartefactofanidealizedmodel,butiscommoninnetworksfoundinnature. 18

PAGE 19

Figure2-2. Clusteringcoefcientversusmeanshortestpathlength[ 81 ]. 2.1.2.2Networkefciencyandcost. Characteristicpathlengthandclusteringcoefcientareusefulonlywhenstudyingunweighted(binary)connectednetworkswithrelativelysmallnumberofedges.Clearly,itdoesn'tcoverwholevarietyofrealnetworks.Inmostcasestheinformationwhetherthereisorthereisnoaconnectionbetweentwonodesispoorandnotsufcientinordertodescribethenetworkwithsuitableprecision.Backtothenetworkexamples,consideredabove,inthecaseoflmsactorsthebinaryapproximationonlyprovidesknowledgewhetheractorsparticipatedinsomemovietogether,ornot,anddoesn'ttakeintoaccountthefact,thattwoactorsthathavedonetenmoviestogetherareinamuchstrongerrelationthantwoactorsthathaveactedtogetheronlyonce.Inthecaseoftheneuralnetworkthenumberofjunctionsconnectingacoupleofneuronscanvaryalot.Aweightednetworkismoresuitedtodescribesuchsystemandcanbebuiltbysettingthelengthoftheconnectionproportionaltotheinversenumberofjunctionsbetweennodes.Theelectricalpowergridnetworkisclearlyanetworkwherethegeographicaldistancesplayafundamentalrole. 19

PAGE 20

Thegeneralizationonarbitrarynetworkscanbemadebyusingmeasureofsocallednetworkefciency.NetworkefciencywasintroducedbyLatoraandMarchioriin[ 47 ],[ 48 ],wheretheyproposedtoexpresssmall-worldpropertiesofnetworksthroughitsinformationtransfercapabilities.Theefciencyoftransmittinginformationbetweentwonodesinthenetworkisinverselyproportionaltothedistancebetweenthem(lengthofshortestpathbetweengivennodes).Itturnsout,thatsmall-worldpropertiesofthenetworkcanbeformulatedintermsoftheefciency,whichplaystheroleofcharacteristicpathlengthLandclusteringcoefcientC.Theformaldenitionisfollowing ConsideranetworkGasaweightedandpossiblynon-connectedandnon-sparsenetwork.Let(a)ijbetheadjacencymatrix,containingtheinformationabouttheexistenceofedgesbetweennodes,anddenedasforthetopologicalnetworkasasetofnumbersaij=1forpairsofnodes(i,j),connectedbyanedge,andaij=0forallotherpairs.Let(l)ijbethematrixofweightsassociatedwitheachconnection,whereelementlijcontainsinformationabouttheweightofedgebetweennodesiandj.Fortheweightednetworktheshortestpathlengthdijbetweennodesiandjisdenedasaminimumsumofweightsofedgesoverallthepossiblepathesinthegraphfromitoj.Suppose,thateverynodeinthegraphsendsinformationthroughtheedges.Onecanassume,thattheefciencyeijofinformationtransferbetweennodesiandjisproportionaltotheinverseoftheshortestpathlengthfromitoj,i.e.eij1 dij8i6=j.Ingeneralthisisareasonableapproximation,althoughinparticularcasesotherrelationscanbeused[ 48 ].TheaverageefciencyofthegraphisdenedasE(G)=1 N(N)]TJ /F5 11.955 Tf 11.95 0 Td[(1)Xi6=j2G1 dij. ThevalueofEgivenbythisformulamayvaryintherange[0;1).ItispossibletonormalizeitbyconsideringtheextremecaseofcompletegraphwhichhasallN(N)]TJ /F9 7.97 Tf 6.59 0 Td[(1) 2edges.Inthiscasetheinformationisbeingtransferredinthemostefcientway,anddij=lij8i6=j.IfwewilldivideactualEbyEcompleteofthecompletegraph,wewillobtain 20

PAGE 21

avaluefrom[0;1]range.Althoughthevalue1canbereachedonlywheneverypairofnodesinthenetworkareconnectedwithanedge,realnetworksoftenpossesshighvaluesofnormalizedefciencyE. Thenetworkefciencycanbemeasuredonboth,localandgloballevels.Theglobalmeaningisdened,asEglobal=E(G) E(Gcomplete), wherethenormalizationtermE(Gcomplete)isanefciencyoftheextremecaseofcompletegraph.Arguably,EglobalplaysrolesimilartocharacteristicpathlengthL,andittellsus,howefcientisnetworkinparallelinformationtransferring.InthecaseofdisconnectedgraphL=1,whileEglobalalwaysremainsnitevalue. LetGibeasubgraphofnodes,adjacenttonodei,notincludingiitself.Thelocalnetworkefciencyisdened,asmeanofglobalefcienciesforsubgraphsGioveralli2V:Eloc=1 NXi2GE(Gi) E(Gcompletei), whereGcompleteiisacompletegraphcontainingsameverticesasGi.Elocisameanoflocalefcienciesinthegraph,andthereforeitplaysarole,similartotheclusteringcoefcient.ThelocalefciencyElocaltellsus,howfaulttolerantoursystemis,i.e.howefcientisthecommunicationbetweenneighborsofi,wheniisremoved.HighvaluesofElocalmean,thatifwewillrandomlydeletenodesfromthenetwork,theconnectivityofthenetworkwillnotbeaffectedstrongly. Thegeneralizeddenitionofsmall-worldbuiltintermsofthecharacteristicsofinformationowatglobalandlocallevelisfollowing:asmall-worldnetworkisanetworkwithhighEglobalandEloc,i.e.veryefcientbothinglobalandlocalcommunication.Thisdenitioncanbeappliedbothforunweightedandforweightednetworks,aswellastodisconnectedand/ornonsparsenetwroks. 21

PAGE 22

Inordertounderstandtherelationoftwoapproachestodescribesmallworldnetworks,itisnecessarytostudythecorrespondencebetweenEandL,C.AlthoughbothofmeasuresLandEarerelatedtotheefciencyofowtransferringthroughthenetwork,oneshouldnote,that1=Lmeasurestheefciencyofsequentialsystem,likeasystemwherethereisonlyonepathofinformationgoingalongthenetwork,whereasEisameasureofparallelsystems,whereinformationisbeingtransferredthroughalltheedgesinthenetwork. IncertaincasesmeasureLworksreasonably:itcanbeshown,that1=LisareasonableapproximationofEglobwhentherearenotsignicantdifferencesamongthedistancesinthegraph[ 48 ].Howeveritfails,likeeveryapproximation,toproperlydealwithallcases.Consideredthelimitcase,whereanodeisisolatedfromthesystem.Forneuralnetwork,thiscaninterpretedforexampleasthedeathofaneuron.Inthiscase,1=Ldropstozero,since(L=+1),buttheoverallefciencyofthesystemisnotaffectedthatstrongly:infact,thebraincontinuestowork,asalltheotherneuronscontinuetoexchangeinformation;only,theefciencyisjustslightlydecreased,sincethereisoneneuronless.ThisslightchangeisproperlytakenintoaccountusingEglob. SimilarrelationshipexistsbetweenclusteringcoefcientCandlocalefciencyEloc.ItcanbeshownthatclusteringcoefcientC,inthecaseofundirectedbinarygraphs,isalwaysareasonableapproximationofEloc.Itcanbeconcludedthereforethattherearenotwodifferentkindsofcharacteristicstoconsiderwhendescribinganetworkonthelocalandgloballevels.Existsjustonegeneralconcept:theefciencyofinformationtransfer. Anotherimportantpropertyofanetworkisitscost[ 48 ].Thismeasurecomesintoplayparticularlywhendealingwithrealsystems,modeledbyweightednetworks.Intuitivelyclear,thatthemoreedgesthereareinthenetwork,thehigheristhenetwork'sefciency.Ontheotherhand,inrealsituationsthereisapricetopayfornumberandlength(weight)ofedges.Inparticulartherewirededgesthatcausethefastdropof 22

PAGE 23

characteristicpathlengthvalueLintheWattsandStrogatzmodelconnectatnocostnodesthatwouldotherwisebefarapart.Inrealisticmodel,suchconnectionswouldbeexpensivetocreateandincertaincasesthiscostcouldovercomethegaininnetworkefciency,providedbyaddingtheseconnections.Thereforeitisnecessarytoconsiderweightednetworks,whereweightswouldallowustodifferentiatebetweendifferentedgesinthenetworkandtointroduceameasuretoquantifythecostofanetwork. Formally,thecostofanetworkGisdenedas:Cost(G)=Pi6=j2Gaij(lij) Pi6=j2G(lij) Thefunctionhereidentiesthecostneededtocreateaconnectionofagivenlengthlij.Note,thatthevalueoftotalcostisnormalizedbydividingitbythecostofthecompletegraph,whichcontainsallthepossibleedges.Becauseofsuchanormalization,thefunctionneedsonlytobedeneduptoamultiplicativeconstant,andthemeasureCost(G)takesvaluesfromtheinterval[0,1],themaximumvalue1isreachedforGcomplete,i.e.whenalltheedgesarepresentinthegraph.Cost(G)isnothing,butthenormalizednumberofedges2K=N(N)]TJ /F5 11.955 Tf 13.39 0 Td[(1)inthecaseofanunweightedgraph.Inmanycasesitisreasonabletotakefunctionastheidentityfunction:(x)=x.Infactsuchfunctionisappliedincaseofunweightednetworksandinmanycasesoftherealnetworks,wherethecostofaconnectionisproportionaltoitslength(forexampleEuclidiandistance).InsuchcasesthedenitionofthecostreducestoCost(G)=Pi6=j2Gaijlij Pi6=j2Glij. Nowwecandeneaneconomicsmall-worldnetworkasanetworkwithhighlocalandglobalefcienciesandlowcost.Inordertoillustratetheuseofnewquantitiesinsomeabstractandpracticalexamples,LatoraandMarchiorihavestudiedseveralmodelsofrealandrandomlygeneratednetworks,startingfromtheoriginalWattsand 23

PAGE 24

Strogatzmodel,andproceedingtothemodelsofweightednetworks.Wewillbrieyreviewthesemodelshere,sincetheyaregoodillustrationstothediscussedmeasuresofsmallworlds.TheWattsandStrogatz[ 81 ]networkmodelisbinary,i.e.weightof Figure2-3. GlobalandlocalefcienciesforWattsandStrogatzmodel[ 48 ]. everyedgeisequalto1.Aninitialregularnetworkwasrewiredwithcertainprobabilityp,givingrisetoarangeofnetworkswithacertaindegreeofrandomness,dependingonvalueofp,inparticular,small-worldnetworks.Sincethetotalcostofanetworkisproportionaltothenumberofconnections,thecostofrewirednetworksdoesn'tchange.LatoraandMarchiorihavefound,thatforp=0(regularnetwork),theglobalefciencyEglobalwaslow,whilethelocalefciencyElocalpossessedhighvalues.Thismeans,thatregularnetworksareinefcientinglobalinformationtransferring,butefcientonalocallevel.Thesituationwasrevertedforp=1,whichcorrespondstoacompletelyrandomnetworks.ThelocalefciencyElocalwascloseto0,andtheglobalefciencyEglobalassumedamaximumvalue.Theeconomicsmall-worldbehaviorappearedfortheintermediatevaluesofp(Figure 2-3 ).AsincasewithcharacteristicpathlengthLandclusteringcoefcientC,introductionofseverallong-rangeconnectionssignicantlyincreasedglobalefciencyEglobal,whilethelocalefciencyElocalremainedalmost 24

PAGE 25

unaffected.Moreover,thebehavioroffunctionsElocal(p)=Elocal(0)andEglobal(0)=Eglobal(0)wasverysimilartothebehaviorofL(p)andC(p)(Figure 2-4 ). Figure2-4. FunctionsElocal(p)=Elocal(0)andEglobal(0)=Eglobal(0)exhibitbehavior,similartoL(p)andC(p)respectively[ 48 ]. InthesecondmodelLatoraandMarchiorihaveproposeddifferentfromreportedin[ 81 ]approachtogeneratearticialsmall-worldnetworks.StartingfromasetofnodesNandemptysetofedgestheedgeswererandomlyaddedtothenetwork,untilthenetworkbecameconnected.Thesenetworkswerebinary,likenetworksintherstmodel,butunlikethosethecostofgivennetworkswasnotconstant,andwasgrowingmonotonicallywithincreasingofnumberofaddedconnections.Givenprocedureweregivingrisetosmall-worldnetworksforvaluesofcost0.5)]TJ /F5 11.955 Tf 12.02 0 Td[(0.6,butitfailedtogenerateeconomicsmall-worldnetworks,whicharenetworkswithsmall-worldpropertiesandsmallnumberofconnections(Figure 2-5 ). WhileinModel1andModel2thelongrangeconnections(orshortcuts)havebeingrewiredatnochangeincost,whichisunrealisticapproachinrelationtorealworldnetworks,inthisthirdmodelLatoraandMarchiorihaveintroducedlengthesofconnectionsinordertoaddressthislimitation.Theyhaveusedthesamerandomrewiringprocedure,butaftereachrewiringthelengthofarewirededgehasbeing 25

PAGE 26

Figure2-5. PlotsofEglobalandElocalasfunctionsofCostforsecondmodel[ 48 ]. changedfrom1to3,andinthiswaythetotalcostofthenetworkwasincreasing.Thenetworkinthismodelthereforewasweighted,withweightsofconnectionsrepresentinglengthsofedgesbetweennodes.Thesmallworldbehaviorinthesenetworkswasstillpresent.ForsmallvaluesofptheglobalefciencyEglobalgrewrapidly,whilelocalefciencyElocaldidn'tchangemuch.However,unliketheprevioustwomodels,thefunctionEgloabl(p)wasnotsimplymonotonicallyincreasing.Contrarilytoit,thecostofthenetworkwasmonotonicallyincreasingwithrewiringprobabilityp.Attherangeofp,wheregeneratednetworkspossessedsmall-worldproperties,thecostofthesenetworkswaslow,whichmeansthenetworkswerenotonlysmall-worldsbutalsoefcient.TherelativebehaviorofElocal(p),Eglobal(p)andCost(p)howeverisrathercomplex,andcan'tbeexpressedinsimpleterms.Incontrasttothesecondmodel,thisgeneralizedrewiringapproachwasabletocreateeconomicalsmall-worldnetworks,byonlyrewiringoftheedges.Therefore,thestructure(topology)ofthenetworkplaysarelevantrolenotonlyinitsefciency,butalsoineconomicalproperties. Inordertobuildamorerealisticmodel,LatoraandMarchioritookinconsiderationgeometricaldistancesbetweennodes.Inthisfourthmodeltheyhaveplacedthenodesofthenetworkaroundthecircleontheplane.Thelengthoftheconnectiontheyhave 26

PAGE 27

Figure2-6. ThethreequantitiesEglobal,ElocalandCostasfunctionsofpforthethirdmodel.Theeconomicsmall-worldbehaviorappearsforp0.1[ 48 ]. dened,asaEuclidiandistancebetweencorrespondingnodes.Forthepointsonthecirclethisdistanceisequalto: lij=2sin(ji)]TJ /F3 11.955 Tf 11.96 0 Td[(jj=n) 2sin(=n), ifwewillassume,thatthenodesareplacedaroundthecircleaccordingtotheirindicesi,andthatthelengthofthearcbetweenneighborpointsisequalto1.Theresultsofanalyzesofnetworksgeneratedbythisprocedurewereconsistentwiththeresultsfromthepreviousmodel.Therewasarangeofvaluesofp,whereglobalefciencyEglobalandlocalefciencyElocalattainedhighvalues,whilethecostwasrelativelylow,indicatingeconomicsmall-worldpropertiesofthenetwork.Also,thebehavioroffunctionEglobal(p)wasnotsimplymonotonicallydecreasing,butmorecomplex.Asaconclusion,thismodelandModel3indicate,thattheeconomicsmall-worldpropertiesarenotonlyan 27

PAGE 28

attributeofthetopologicalabstractions,butarepresentalsointheweightednetworks,wherethephysicaldistanceplaysroleandtherewiringofconnectionshasacost. Figure2-7. ThethreequantitiesEglobal,ElocalandCostasfunctionsofpforthefourthmodel.Theeconomicsmall-worldbehaviorappearsforp0.02)]TJ /F5 11.955 Tf 11.95 0 Td[(0.04[ 48 ]. 2.1.3BrainNetworks Earlierstudiesofthebrainfunctionhavefocusedmainlyeitheronlocalordistributedproperties.Modernworksshiftedtheattentiontothestructureanddynamicsoflargescaleneuronalnetworks[ 30 ],[ 65 ].Inthissectionwewilldescribeseveralcharacteristicsofbrainnetworks,inparticular,theirsmallworldproperties. Withafewexceptionsmostpartofcommunicationbetweennervecellsisperformedthroughthephysicalconnections.Sometimesthisconnectionsmaylinkcellsseparatedbyalongdistances.Signalsgeneratedbyneuronsandtransmittedthroughtheseconnectionscompriseaseriesofactionpotentialsorspikes.Thereceptionofanactionpotentialatasynapticjunctionlaunchesnumerousbiochemicalandbiophysicalprocesses,resultingingeneratinganoutputspike,transmittedalongtheneuron's 28

PAGE 29

axon.Neuronsinthecerebralcortexformadensenetworkwiththousandsofinputandoutputconnections[ 65 ].Byquantitativeestimations,thehumancerebralcortexcontainsroughly8109neuronsand71013connections[ 56 ].Thetotallengthofallconnectionsinthehumanbrainisestimatedbetween105and107km.However,havingthistremendousnumberofconnections,brainnetworksremainverysparsewithafractionofpresentconnectionsofaround10)]TJ /F9 7.97 Tf 6.59 0 Td[(6.Itturnsout,thatthesesparsebrainnetworksarenotrandom,buthavespecialstructurepatterns.Neuronstendtoconnectmainlytotheneighborneuronsforminglocalgroups,andthuslocalconnectivityinneuralnetworkismuchhigher,thanitwouldbeexpectedinrandomnetwork[ 10 65 69 ]. Neuralbrainnetworkscanbeanalyzedatdifferentlevelofscale[ 67 ].Atmicroscopicscale,neuronsformspecialsetsofconnections,calledlocalcircuits.Atahigherlevelofscale,populationsofneuronscommunicatethroughthebundlesofconnections,formingnetworkswithinsinglecorticalareas.Connectionpatternsformedbytheselocalnetworksaredenedbyspecictaskeachareaisresponsiblefor.Finally,atanentirebrainlevel,thelargescalenetworksareformedbyinterconnectionslinkingdifferentbrainareas[ 65 ]. Sofarwewerediscussingneuralconnectivity,i.e.structural(anatomical)brainnetworksformedbyphysicalconnectionsbetweenneurons.Anotherimportantaspectofbrainstudyisneuralactivityanddynamicsofbrainprocesses.Neuralactivityalsocanbemodeledbyanetwork,socalledfunctionalconnectivitybrainnetwork[ 64 ],[ 32 ].Theconnectionsinfunctionalnetworkaredenednotbyphysicalpathesinthebrain,butbydependenciesbetweenneuralsignalsindifferentpartsofthebrain.Iftwopopulationsofneuronsexchangeinformation,thesignalpatternsproducedbythesepopulationsarelikelytobestatisticallydependent,forexamplecorrelatedintime.Itiscommonlyrecognizedthatinrestinghumanbrainexistsconsistentfunctionalnetworkbetweencorticalbrainregions(socalleddefaultmodenetwork)[ 15 26 31 59 60 76 ]. 29

PAGE 30

Ithasbeenproposed,thatfunctionalnetworksfromaphysiologicalbasisforinformationprocessingandmentalcognition[ 17 20 21 54 ].Oneofthequestionsarisinghere,ishowdostructuralandfunctionalnetworksrelatetoeachother?Nodoubts,thatpatternsofneuralactivityaredenedandshapedbyneuralconnectivity,andfunctionalnetworksinlargepartresemblestructuralbrainnetworks[ 63 ],[ 40 ].Butsometimesfunctionalnetworkscantellusmoreaboutprocessesinthebrainthenjustmapofphysicalneuralconnections.Certainbrainareas,whicharenotconnecteddirectly,communicatewitheachotherthroughotherareas,andthesecommunicationsarereectedinthefunctionalnetwork.Therefore,functionalnetworkcomprisesstructuralnetwork,butalsocontainsconnectionsbetweenthoseareaswhicharenotconnectedinstructuralnetwork,butlinktoeachotherdynamically,exhibitingrelatedneuralactivity. Itisnaturallytoassume,thatifstructuralnetworksinlargepartdenesfunctionalnetwork,theyshouldhavesimilartopologicalproperties.Andinfactthiswasconrmedbymanystudies,whichreportedsmall-worldpropertiesinbothgroupsofnetworks.Historically,rststudiesonbrainnetworkpropertieswereperformedonstructuralnetworks,inparticularthoseofmammalianbrain.Furtherinthissectionwewillrstreviewresultsonstructuralbrainnetworks,andthenwilldiscussfunctionalnetworksproperties. 2.2ConnectivityBrainNetworksBasedonWaveletAnalysis 2.2.1Background Magneticresonanceimaging(MRI)isatechniqueforlookingatmagneticdipoleinteractionsthatcanbeadjustedtoprovidevastamountsofinformationabouttheinternalstructureandfunctionofthebrain.Variousimagingtechniquesusingdenedpulsesequencescandelineatebertractsbetweendifferentregions(forexamplewithdiffusion-weightedMRI),whileothertechniquescandenevariousstructuralelements.FunctionalMRI(fMRI)generatestemporalinformationaboutfunctionalbrainnetworks. 30

PAGE 31

Alterationinpulsesequencecanthereforeprovidedifferentandoverlappingdetailaboutbrainstructureandfunction. InordertobetteranalyzethevastamountofinformationthatcanbegeneratedbyMRI,multiplemethodsofevaluatingthedatawillproveuseful.Inthispaper,wehavefocusedonanalyzingtemporalinformationgeneratedbyrestingfMRI.Multiplepriorstudiesindicatethatcertainbrainregionsposseshighlycorrelatedneuralactivityintimeandfrequencydomains[ 1 15 53 ].Recently,EdBullmoreet.al.[ 1 ]describedusingnetworkmodelingtechniquestomodelbrainactivityinthetemporaldomain.Justasanatomicconnectivitycanbedescribedandmappedusingtractographicmethods[ 84 ],functionalconnectivitybetweenregionscanbeassessedovertimetocreatearegionbyregiontemporalconnectivitymap.Unlikemoretypicaleventrelatedfunctionalimaginganalytictechniques,networkconnectivityanalysiscanbereadilyappliedtobrainrestingstates.Thisfeaturemayprovideclinicalrelevancybyallowinganalysisoftemporalbraininteractionsinthetypesofconditionsthatmightbereadilyavailableinthevastnumberofclinicalscannersincommonuse. Wewishedtoevaluateifusingfunctionalconnectivitytoderivebrainnetworkscouldprovideusefulinformationinadiseasestate.WechosetoanalyzethisstateinParkinson'sdisease,adiseasethatisnotableforhavinganumberofrestingstateanomaliesincludingtremor,rigidity,akinesiaandbradykinesia,andmotorfreezing.AlthoughanumberofauthorshavedenedsomechangesinanatomicconnectivityandstructurethatmayoccurinParkinson'sdisease,nobiomarkerhasyetbeendened.Bullmoreetal.[ 1 ]showedthatdopamineblockadeinhealthyadultsresultedinanalterationinnetworkcostandefciency.WereasonedthereforethatParkinson'sdisease,whichischaracterizedinpartbydecreaseddopaminergicactivity,mightshowalterednetworkefciencyandcost. Wewishedtousetemporalconnectivityasamethodofdeningtherelationshipbetweenvariousbrainregions.Inthisconception,tworegionsofthebrainare 31

PAGE 32

consideredtobeconnectediftheyshowatemporalinteraction.Nodesinourcaseweredeneda-prioriusingapre-denedatlas.Eachbrainfunctionaldivisionwasconsideredanode,withthecorrespondingfunctionalconnectionsbetweennodesasservingasedgesinthenetwork.Avarietyofmeasuresdeningvariousaspectsofconnectivitycanbeusedtomaptheinteractionbetweenthenodes.Inourcase,wedecidedtomeasurenetworkefciencyandeconomicalproperties.WepredictedthatindividualswithParkinson'sdiseasewouldshowaglobaldecreaseinnetworkefciencyandincreasednetworkcost. 2.2.2Methods Sample:19healthycontrolsubjectswithoutParkinson'sdiseaseand21subjectswithParkinson'sdiseasewereinitiallyrecruitedtoparticipateinthisstudy.Healthycontrolsubjectswererecruitedfromthecommunityorthroughadatabaseofolderhealthyadultswhoexpressedaninterestinparticipatinginresearch.SpousesofindividualsofParkinson'sdisease(4subjects)whodidnotdisplayevidenceofanyneurologicdiseasewerealsoincludedinourhealthycontrolpool.IndividualswithParkinson'sdiseasewererecruitedfromamovementdisordersclinicattheUniversityofFloridaandfromtheNorthFloridaSouthGeorgiaVAmedicalcenter.Healthycontrolswereindividualswithoutevidenceofmovementdisorder.IndividualswithParkinson'sdiseasewerediagnosedwithParkinson'sdiseasebyafellowshiptrainedmovementdisordersspecialist.Subjectswerematchedonthebasisofage.AllsubjectswithParkinson'sdiseasewereoffmedicationfor12-18hourspriortothefunctionalimaging.Allsubjectsunderwentaneurologicexaminationandcognitivetestingpriortoimaging. Aninitialdataqualitystepwasperformedthatincludedevaluatingsubjectdataforexcessivemotion.Aftermaskingtocreateabrain-onlymask,overallimagevariabilitywasanalyzedandstandarddeviationofvariabilityovermeansignalwascalculatedacrossthesample.Functionalimageswithpeak(brainonly)variabilityof>1.5%wereexcludedfromoursample.Visualinspectionshowedthatsubjectswithelevated 32

PAGE 33

Table2-1. Demographicsofthedataset. healthycontrolsPDpatients Number1514Age6513629Gender6female9male2female12maleCognitiveProleMMSE293273MOCA273253TrailMakingTestA2785433TrailMakingTestB893014797DiseaseProle(Parkinson'sDisease)UPDRSTotal5920PartA(Mood/Cognition)42.7PartB(ActivitiesofDailyLiving)137.7PartC(MotorExaminationScale)3911PartD(MedicationComplications)3.32.1 variabilitywerealsosubjectsthathadvisibleheadmotionduringthefunctionalrun.Inaddition,individualswithParkinson'sdiseasewhowerenotedtohavesignicantfacialtremoronclinicalexaminationwerealsonotedtobeamongthecategoryofindividualswithincreasedvariability.Thisstepresultedinexclusionof3healthycontrolsfromthesampleand7individualswithParkinson'sdisease.Subsequenttoourinitialevaluation,oneindividualwhopresentedtousearlyintherecruitmentasahealthycontrollaterreturned8monthslaterwithanewonsetofresttremorandmildbradykinesia,andwasdiagnosedwithearly(levodoparesponsive)Parkinson'sdisease.Thissubjectwasalsoexcludedfromourhealthycontrolpopulation. Analysisofthedatacommencedafterinitialexclusionofsubjectswithvisibleorpresumedexcessivemotionasidentiedabove.Wesubsequentlyfurtheranalyzed15healthycontrolsand14individualswithParkinson'sdiseaseforthisstudy.Ofnote,giventheabove,oursamplewasrelativelyenrichedinindividualswithakinetic-rigidvariantofParkinson'sdisease.DemographicsofoursampleisshowninTable 2-1 33

PAGE 34

2.2.2.1Functionalimagingparameters FunctionalimagingwasperformedonaPhilipsAchieva3.0Teslascanner(PhilipsMedicalSystems,Best,TheNetherlands)withtheparameters:TR=2000ms,TE=30ms,FOV=240mm,slicethickness=3.8mm,Gap=0,FlipAngle=80degree,totally36slices,paralleltoAC-PCline,acquisitionmatrix=64x64.ThreedimensionalstructuralimageswereacquiredwithTR=8.05ms,TE=3.68ms,FOV=240mm,FlipAngle=8degree,slicethickness=1.0mm,voxelsize=1.0mmx1.0mmx1.0mm,acquisitionmatrix=256x256,totally144160slicestocoverthewholebrain.Therst5volumesofthefunctionalimagewerediscardedtoallowforT1saturationeffects,leaving175volumesforeachsubjectavailableforanalysisoftherestingstateconnectivity. Eachdatasetwascorrectedforgeometricdisplacementduetoheadmotionandco-registeredwiththeMontrealNeurologicInstitutegradient-echoechoplanarimaging(EPI)template.First,volumeregistrationandmotioncorrectionwasperformed.A3-dimensionalvolumeregistrationwasperformedthat,usingaleastsquaresmethod,co-registeredeachvolumetothebaselinevolume.Resultantestimatesofheadmotionwereobtained.Theseestimatesoftranslationsandrotations(x,y,z,roll,pitch,andyaw)wereusedtofurthercorrectthetimeseriesonavoxelbyvoxelbasisusingregression.Asnotedabove,aninitialdataqualitystepremovedindividualswithexcessiveheadmotionandinbothourPDandcontrolsubjectsheadmotionwaslessthan5mm;sampleswereadditionallymatchedintermsofoverallmeanheadmotioninthesample.Theresidualsoftimeseriesaftermotioncorrectionconstitutedthesetofvoxelbyvoxeltimeseriesthatwerefurtheranalyzed. SimilartothemethodsreportedbyBullmoreet.al.[ 1 ],theanatomicallylabeledtemplateimagevalidatedbyTzourioMazoyeret.al.[ 73 ]wasdownloadedandtransformedintoanAFNIBRIKformat.Datawasnotspatiallysmoothed,andregionalparcellationwasperformed.Thisparcellationdividedeachhemisphere,includingthecerebralhemispheresandthecerebellum,into116differentanatomicalregionsof 34

PAGE 35

interest.Regionalmeantimeseriesoverallvoxelsineachoftheregionswascomputedandconstitutedthesetofregionalmeantimeseriesusedforwaveletcorrelationanalysis. 2.2.2.2Waveletcorrelationanalysisandgraphconstruction AlthoughMRIsignalcanbeconsideredasanindicatorofneuralactivity,itcannotbeuseddirectlywithoutinitialpreprocessing.Therearecertainirrelevantprocesses,whichcanmakeanimpactonMRImeasurements.AlongwithactualneuralactivityMRIsignaluctuationsarecausedbypatientheadmovementsduringtheMRIsession,cardioandrespiratoryrhythms,noiseetc.Inordertoremovetheseeffects,weusemaximaloverlapdiscretewavelettransform(MODWT)[ 57 ].WeremovemotioncomponentbystandardAFNIprocedure[ 80 ],basedonregressionmodel,anddecomposeresultingsignalintowaveletcoefcientsat3scales.TherearemanyreasonsinsupportofusingwaveletsinfMRIsignalprocessing.Itwasshown,thatcorticalfMRItimeseriesposseslongmemoryintime,ora1=fpowerspectruminthefrequencydomain[ 53 ],andthisarguesagainstmeasuringfunctionalconnectivitybetweenapairoffMRItimeseriesbyestimationoftheircorrelationinthetimedomain,ortheircoherenceinthefrequencydomain,becausebothtimeandfrequencydomainestimatorsofassociationarenotproperlyestimableforlongmemoryprocesses[ 11 ].Ontheotherhand,thewaveletcoefcientsofalongmemoryprocessarestationary,andwavelet-basedestimatorsofthecorrelationbetweentwolongmemoryprocesseshavebeenshowntohavedesirablestatisticalproperties[ 19 ].Anotherreasoningofwavelet-basedcorrelationanalysis,sharedincommonwithapproachesinthefrequencydomain,isthatitallowsustolookatthefunctionalsimilaritybetweenbrainregionsbasedonactivityinadenedfrequencyintervalorwaveletscale.Thisisattractiveforanapplicationtoresting-statefMRIdatabecauseitwasshown,thatfunctionalconnectivitybetweenregionsistypicallygreatestatfrequencieslessthan 35

PAGE 36

0.1Hz,whereasnon-neuralsourcesofrestingstatecorrelations,suchasaliasedcardiorespiratoryeffects,aresubtendedbyhigherfrequencies[ 25 ]. Ourdatasetconsistsof29subjects,15healthycontrolsand14parkinsonpatients.Eachpatientwasimagedeach2secondsoverthe6minutesinterval.First6timepointswerediscardedinorderforsubjectstoadjusttocircumstances.Eachsignalvectorthereforecontains174components.Foreachpatientwebuildnetworkof116nodes,inaccordancetoMNIbraintemplate. Wedenestrengthoftheconnectionbetweenregionsascorrelationbetweensecondlevelwaveletcoefcientsofaveragedsignals,whichcorrespondsto0.06-0.12Hzfrequencyinterval.ConsidertworegionsA,BandcorrespondingnodesinnetworkVAandVB.Thenthestrengthoftheconnectionis F(A,B)=PiW2,i(fA)W2,i(fB) r Pi(W2,i(fA))2Pi(W2,i(fB))2, wherefA=1 jAjXx2Af(x),fB=1 jBjXx2Bf(x), andW2(f)-leveltwowaveletcoefcientsofsignalf. Wewouldliketointroduceweightsofedgesinthenetworkinsuchway,thattheywouldrepresentdistancebetweennodes.Thengraphtheoryconcepts,suchasshortestpathlength,diameterofthegraph,costefciencyandsoon,wouldbemeaningful.Intuitively,thestrongertheconnectionbetweentworegions,thesmallerisdistancebetweencorrespondingnodes,andviseversa.Considerfollowinggeometricalrepresentation.Eachwaveletcoefcientsvector)648(!W=(w1,w2,...,wn)canberepresentedbyapointinn-dimensionalspace(Figure 2-8 ). Note,thatwaveletcoefcientvectorspossesfollowingproperty:meanofitscomponentsiszero[ 57 ].Considertwovectors)777(!x=(x1,x2,...,xn),)777(!y=(y1,y2,...,yn)withzeromeancomponents,andcorrespondingpointsX,Yinn-dimensionalspace. 36

PAGE 37

Figure2-8. Geometricalrepresentationoftimeseries. Thecorrelationcorr(x,y)isnothingbutcosineofanglebetweenvectors)496()497(!OXand)476()476(!OY.Thecorrelationwillnotchange,ifwewillnormalizegivenvectors.FornormalizedvectorsxandyEuclidiandistancebetweentheirendpointsisgivenbyjjx)]TJ /F3 11.955 Tf 11.95 0 Td[(yjj2=jjxjj2)]TJ /F5 11.955 Tf 11.95 0 Td[(2(x,y)+jjyjj2=2(1)]TJ /F5 11.955 Tf 11.96 0 Td[((x,y)), wherescalarproduct(x,y)isequaltocorrelationbetweenxandy.Therefore1)]TJ /F3 11.955 Tf -423.91 -23.91 Td[(corr(a,b)mayserveasagoodmeasureofdistancebetweentimeseriesaandb. Accordingtotheseconsiderations,wedeneweightofedgebetweennodesVAandVBasw(A,B)=1)]TJ /F3 11.955 Tf 11.96 0 Td[(F(A,B). TheresultingnetworksforControlandParkinsonsetsarepresentedinFigure 2-9 .Thethesizeandcolorofthenodescorrespondstoitsdegreeinthenetwork.Asonecansee,therearesignicantdifferencesbetweennetworks,whichwillbediscussedindetailsbelow. 37

PAGE 38

Figure2-9. Connectivitynetworks.A)Controlgroup.B)Parkinson'sgroup. Itwasshown,thatbrainconnectivitynetworksgenerallydemonstrateeconomicalsmall-worldpropertiesofhighglobalandlocalefciencyforlowcost[ 1 ].Wepredicted,thatParkinsonbrainnetworksposseslowerefciencyatglobalandlocallevels.Wemeasuredglobalandlocalefciencyofparallelinformationprocessing,asafunctionofcost(distance).Theinverseoftheharmonicmeanoftheminimumpathlengthbetweeneachpairofnodesisameasureoftheglobalefciencyofparallelinformationtransferinthenetwork[ 48 ]: Eglobal=1 N(N)]TJ /F5 11.955 Tf 11.96 0 Td[(1)Xi6=j2G1 Lij. 38

PAGE 39

Analogously,onecandenethenodalefciencyastheinverseoftheharmonicmeanoftheminimumpathlengthbetweenagivennodei,andallothernodesinthenetwork:Enodal(i)=1 N)]TJ /F5 11.955 Tf 11.96 0 Td[(1Xj2G1 Lij. 2.2.3Results 2.2.3.1Globalmeanefciency Meanefciencyacrossallnodeswasdecreasedincontrolswas(1.85+/-0.57),comparedto(1.12+/-0.55)inindividualswithPD(p=0.0017,Figure 2-10 ). Figure2-10. GlobalefciencyvaluesforControlandParkinsonsets.ThemeanvaluesC=1.85,P=1.12andthep-valueoftwo-tailedt-testis0.0017. ThedecreaseinefciencywasFigure 2-10 ,GlobalEfciencynotedtobeapropertyoverallofthenetworksoftheindividualsdiagnosedwithParkinson'sdisease,andamongthePDsamplediseaseseverityasmeasuredbytheUniedParkinson'sDiseaseRatingScale(UPDRS)wasnotnotedtobesignicantlyassociatedwithglobalefciencyscores.AgeandCognitionweresimilarlynotspecicallyassociatedwithglobalefciency. 39

PAGE 40

2.2.3.2Meannodalefciency,top30nodes Weadditionallyevaluatedthemeanefcienciesofthetop30nodesinthecontrolsample,andcomparedtheseefcienciestoefcienciesintheParkinson'sdiseasesample(seegure 2-11 ). Figure2-11. Top30nodalefciencies.A)Controlgroup.B)Parkinson'sgroup. Asexpected,thePrecuneus,Cuneus,SuperiorParietal,SuperiorOccipital,andMiddleFrontalregionswereamongthetop30nodesinhealthycontrols;theseregionsallcontainaspectsoftherestingnetworkthathasbeendescribedastheDefaultModeNetwork.13TheleftSupplementaryMotorCortex,contiguousPrecentralRegions,theCalcarinecortices,secondaryvisualregions,andthecertainregionswithinthecerebellumwerealsonotedtohaveincreasedefciency.IndividualswithParkinson'sdiseasewerenotedtohavesubstantialdecreasesinefciencyacrossallofthesenodes.Meanefciencyamongthetop30nodeswasalsosubstantivelydifferentinthePDsubjects(2.0+/-0.57)comparedtothehealthycontrols(2.74+/-0.78),p=0.0056(Figure 2-11 ). 40

PAGE 41

2.2.3.3Networkefciencydecrement Wewerecuriousastotheaveragedecrementacrossallnodesinoursample,andsowearrayednodesfrommosttoleastefcient.Figure 2-12 displaysdecrementinefciencyfrommosttoleastefcientnodeacrossthesample.DecreasedefciencyinthePDsamplewasnotedtobeparticularlyacuteinthemostefcientnodesinthesample,whilelessefcientnodeswerenotedtobemoresimilarinthesample. Figure2-12. MeanDecrementinEfciencyAcrossNodesforControl(redline)andParkinson(blueline)subjects. Ascanbenotedinthegure,meanefciencyinthePDsubjectsrangedbetweenapproximately1.2and1.8,inthehealthycontrolsthismeanrangedbetweenapproximately1.2and3.Whentheslopeofefciencywascalculatedacrossnodes,itwasnotedthatmeandecrementofefciencyacrossnodeswassignicantlydifferentwhencomparingPDsubjects(Slope=0.76+/-0.61)andcontrolsubjects(Slope=1.30+/-0.64;p=0.03). 41

PAGE 42

2.2.3.4Prediagnosticsubject Asnotedinthemethodssection,oneindividualearlyinourenrollmentobtainedarestingscanasahealthycontrol.Thesubjecthadnoalterationinactivitiesofdailyliving,noREMbehaviorsorrestlesslegs,hadnormaltone,nobradykinesia,andnonotedrestingtremors.UPDRSmotorscore(andtotalscore)forthissubjectwasdocumentedas0(noclinicallyappreciabledisease).Thesubjectwasfollowedandhadnofurtherappreciablesymptomsotherthan4moreincidencesduringwhichrestingtremorwasnotedbyaspouse,untilabout6monthsafterenrollmentinthestudywhenhebegannotingsomebradykinesiaandamildalterationingait.Approximately8monthsafterourrestingscanthesubjectwasdiagnosedwithearly,levodopa-responsiveidiopathicParkinson'sdisease.Inthissubject,meannodalefciencywasnotedtobesubstantiallybelowthatofothercontrolsat0.42.Meandecrementinefciency(slope)forthesubjectwas(0.20),alsowellbelowmeancontrolslope. 2.3OptimalPropertiesofHumanBrainNetworks Anumberofstudieshavereportedsmallworldpropertiesinbrainnetworksmeaningthatthesenetworkshavehighclusteringcoefcientpairedwithlowmeanshortestpathlength.Thesepropertiesarenotonlyafunctionofthehumanbrain,buthavebeenshowntooccurinmultipledifferenttypesofbrainnetworksoperatingatdifferentscales.ForexampletheneuronalnetworkofCaenorhabditiseleganshassmall-worldtopologyatamicroscopicsingleneuronscale[ 81 ].Anatomicalconnectivitynetworksofcatandmonkeycorticesalsoexhibitsmall-worldpropertiesatamacroscopicscale[ 38 ].Small-worldpropertieshavealsobeenreportedingraphsobtainedfromfunctionalconnectivitymatricesmeasuredatamacroscopic(regional)scaleinmonkeyandhumanneurophysiologicaldata[ 70 ],[ 68 ],[ 61 ]andatavoxelscaleinhumanfunctionalmagneticresonanceimaging(fMRI)data[ 29 ].SmallworldpropertieswerealsoreportedinanatomicalregionsscalefunctionalhumanbrainnetworksderivedfromfMRIdata[ 2 ]. 42

PAGE 43

Inthemajorityofstudiesresearchersevaluatedsmallworldcharacteristicsofhumanbrainnetworksandcomparedthemtothoseineitherrandomorregularnetworksofrespectiveorderandsize.Ithasbeenshownthatgenerallysmallworldnetworksexhibitcharacteristicpathlengthsimilartorandomnetworks,andclusteringcoefcientsimilartoregularnetworks[ 81 ],[ 48 ].Basedonthecomparisonwiththesetwoextremes(randomandregular)brainnetworkswerereportedtohavesmallworldtopology.Wedecidedtocomparealsothepropertiesofhumanbrainnetworkstothoseintheoptimalsmallworldnetwork,whichingeneralisneitherrandomnorlattice.Byoptimalwemeanherethenetworkthatmaximizesorminimizesparticularnetworkpropertyoverallnetworksofgivensizeandorder.Toourknowledgethisquestionhasnotbeenaddressedintheliterature. Barmpoutisetal.[ 8 ],[ 9 ]hasshownthatnetworkswithmaximalclusteringcoefcienthavealsominimalshortestpathlengthandmaximalnetworkefciencyoverallnetworksofgivensizeandorder.Thereforethesenetworksoptimizeallsmallworldattributesatthesametimeandinasenseareoptimalsmallworlds.Barmpoutisetal.havealsodescribedanalgorithmforgeneratingsuchnetworksandaprocedureforfastgeneratingapproximateoptimalnetworksoflargesize. InthisstudywecomparefunctionalhumanbrainnetworksderivedfromfMRIdatatotheoptimalsmallworldnetworksofrespectivesize.Theproceduresfornetworkderivationsaredescribedabove. 2.3.1GraphAnalysis ConsideranundirectedgraphG=fV,Eg,whereVisasetofverticesandEisasetedges.Theorderofagraphisanumberofitsverticesn=jVj.Thesizeofagraphisanumberofitsconnectionsm=jEj.AcharacteristicpathlengthLisdenedasameanshortestpathlengthbetweenallpairsofverticesinthegraph: L=1 )]TJ /F4 7.97 Tf 5.48 -4.38 Td[(N2Xi,j2Vdij, 43

PAGE 44

whereNisanumberofverticesinthegraphanddijisalengthofashortestpathbetweenverticesiandj. AlocalclusteringcoefcientCiofavertexiisdenedasanumberofconnectionsbetweentheneighborsofidividedbythetotalpossiblenumberofconnections:Ci=mi )]TJ /F4 7.97 Tf 5.48 -4.38 Td[(ki2, wherekiisanumberofverticesadjacenttovertexi,miisanumberofconnectionsbetweenthesevertices.AclusteringcoefcientofthegraphGisdenedasameanlocalclusteringcoefcientacrossthevertices: C=1 NXi2VCi. Theefciencyofthenetworkisdenedasameaninverseoftheshortestpathlengthbetweenallpairsofvertices: E=1 )]TJ /F4 7.97 Tf 5.48 -4.38 Td[(N2Xi,j2V1 dij. Thismeasurecharacterizestheefciencyofinformationtransferthroughthenetwork. Thesmallworldnetworksaredenedasnetworksthathavelowcharacteristicpathlengthandhighclusteringcoefcient[ 81 ].Thisdenitionisnotveryprecise,sincelowandhighcanbetreateddifferentlyindifferentnetworks.Conventionally,ifthecharacteristicpathlengthinthenetworkiscomparabletothatinrandomnetworksofthesameorderandsize,andclusteringcoefcientishigherthantheoneinrandomnetworks,thenetworkisconsideredtobesmallworld. Inthisstudyhoweverwewishedtocompareournetworkstooptimalsmallworldnetworks.Barmpoutisetal.[ 8 ]havedescribedthestructureofsuchnetworksandprovidedanalgorithmtogeneratesuchnetworksforarbitraryordernandsizem.An 44

PAGE 45

exampleofoptimalsmallworldnetworkoforder10andsize24ispresentedonFigure 2-13 Figure2-13. Networkwithhighestclusteringcoefcientandlowestcharacteristicpathlengthoforder10andsize24. Theoptimalnetworkshowevercannotbegeneratedefcientlyforlargesizes.Wehaveusedapproximateproceduredescribedin[ 8 ]togeneratenetworksclosetooptimal.Wecompareddescribedabovesmallworldpropertiesofbrainnetworkstotheonesinapproximateoptimalnetworksofthesameorderandsizeandtotheonesinrandomnetworksgeneratedbyrandomlyuniformlydistributingconnectionsacrossthenetwork. WehavegeneratedapproximateoptimalnetworksusingthePythoncodekindlyprovidedusbyDr.Barmpoutis. 2.3.2Results 2.3.2.1Clusteringcoefcient Theclusteringcoefcientisameasurethatshowshowwellanetworkisconnectedonthelocallevel.Italsoindicatesthenetworksrobustnesstorandomandtargetedattacks.Networkswithhighclusteringcoefcientgenerallyaremoreresilientandrequiremoreelementstobedeletedinordertomakethenetworkdisconnected.Wehavecomputedaveragedover16patientsclusteringcoefcientinbrainnetworkswith 45

PAGE 46

densityvariedfrom0.1to0.5.Foreachnetworksizewehavegeneratedanapproximateoptimalnetworkand50randomnetworks(foreachrandomnetworkweevaluatedclusteringcoefcientandreporttheaveragedover50generatednetworksvalue).TheresultingplotsofclusteringcoefcientsarepresentedonFigure( 2-14 ). Figure2-14. Clusteringcoefcientversusnetworkdensity.Bluelinerepresentsaveragedclusteringcoefcientover16brainnetworks.Redlinerepresentsclusteringcoefcientinapproximatedoptimalnetwork.Greenlinerepresentsaveragedclusteringcoefcientinrandomnetworks. Asonecansee,brainnetworksdemonstrateveryhighclusteringcoefcientatallvaluesofdensity.Forsomevaluesofdensitybrainnetworksclusteringcoefcientisevenhigherthantheoneinapproximatedoptimalnetworks,whichmeansthatit'sveryclosetothetrueoptimal.Therandomnetworkshavesignicantlylowerclusteringcoefcientcomparedtobrainandapproximateoptimalnetworksatallvaluesofdensity. 2.3.2.2Networkefciency Thenetworkefciencymeasuresthecapabilityforparallelinformationtransferandprocessingthroughthenetworkandistightlyrelatedtothecharacteristicpathlength[ 48 ].Asinthecasewithclusteringcoefcientwehavecomputednetworkefciencyinbrainnetworksoverthesetofnetworkdensityvaluesvariedfrom0.1to0.5.For 46

PAGE 47

eachnetworksizewehavegeneratedanapproximateoptimalnetworkand50randomnetworks.Wethencomputedaveragedefciencyfortheapproximateoptimalnetwork,aswellasmeanefciencyofthe50randomnetworks.Wecomparedthesetothemeancomputedclusteringcoefcientofourbrainnetwork.TheresultsarepresentedonFigure( 2-15 ). Figure2-15. Networkefciencyversusnetworkdensity.Bluelinerepresentsaveragednetworkefciencyover16brainnetworks.Redlinerepresentsnetworkefciencyinapproximatedoptimalnetwork.Greenlinerepresentsaveragednetworkefciencyinrandomnetworks. Theresultsshowthatbrainnetworkshavelowerefciencythantherandomandapproximateoptimalnetworks,especiallyinsparsenetworks(networkswithlownetworkdensity). 2.3.2.3Characteristicpathlength Thecharacteristicpathlengthrepresentsthetypicaldistancebetweennodesinthenetworkanditisrelatedtotheefciencyofthenetworkintransferringinformation.Unlikeclusteringcoefcientandnetworkefciency,characteristicpathlengthisdenedonlyforconnectednetworks(itisinnitefordisconnectednetworks).Ingeneratedrandomandbrainnetworkssomeisolatedverticesordisconnectedcomponents 47

PAGE 48

happen,whichpreventedusfromevaluatingthecharacteristicpathlengthdirectly.Thisisadrawbackofacharacteristicpathlengthmeasure,sinceaddingoneisolatednodetoanetworkofarbitrarysizebringsthismeasuretoinnity,althoughonenodeshouldnotcausesignicantimpactonglobalnetworkproperties(thisphenomenonhasbeendiscussedindetailbyLatoraandMarchiori[ 48 ]).Inordertodealwiththisissuefordisconnectednetworkswehaveevaluatedthecharacteristicpathlengthofthelargestconnectedcomponent,whichinmostcaseswasclosebysizetothesizeofthewholenetwork.TheresultingplotsofclusteringcoefcientsarepresentedonFigure( 2-16 ). Figure2-16. Characteristicpathlengthversusnetworkdensity.Bluelinerepresentsaveragedcharacteristicpathlengthover16brainnetworks.Redlinerepresentscharacteristicpathlengthinapproximatedoptimalnetwork.Greenlinerepresentsaveragedcharacteristicpathlengthinrandomnetwork. Asonecansee,brainnetworksexhibithighercharacteristicpathlengththanthatinrandomandapproximateoptimalnetworks. 48

PAGE 49

CHAPTER3PROBABILISTICCLASSIFIERSINBIOMEDICALAPPLICATIONS 3.1Background Theproblemofextractingknowledgefromthedatahasbeenstudiedformanydecadesnow.Researchesfromdifferentdisciplineshavecontributedtotheeldofdataanalysisandinformationextraction.InstatisticaldisciplinesthiseldisoftenreferredasDataMining,whileincomputersciencedisciplinesMachineLearningisamorecommonterm.Theproblemsandtheobjectivesinthisdomaindifferfromapplicationtoapplicationbutthecommonpropertyisalargeamountofdatawithnoclearoreasilyobtainablewaytolearntheinformationofinterest[ 14 ],[ 36 ],[ 74 ]. Biomedicalapplicationspresentrichopportunitiesandahighdemandforthedataminingtechniquesforseveralreasons.First,thebiologicalprocessesandfunctionsofahumanbodyareverycomplexandmanyofthemarenotwellunderstood.Thereforetherearenosuitableanalyticalormodelingtoolsthatcouldbeusedforprecisedescriptionoftheseprocesses.Incontrast,dataminingtechniquescanprovideinsightsanddiscoverempiricalpatternshiddeninthedatathatcanshedlightontobiologicalnatureofunderlyingprocesses.Second,withthedevelopmentofmoderninformationsystemsandmedicaldataacquisitiontechniquestheamountofavailabledatahasexplodedexponentially,creatingtheneedforspecializedmethodsandalgorithmsinordertoprocessitefciently. Oneoftheimportantproblemsindataminingisaproblemofclassication[ 14 ].Givenasetofobjectsandasetofclasslabelsknownforalimitedsubsetofobjects,thetaskistoreconstructtheclasslabelsforalltheobjectsintheset.Inthecontextofbiomedicalapplicationstheobjectsareusuallypatientsfromthecohortbeingstudiedandtheclassesmayrepresentmedicalconditions,complicationsafterthesurgery,futureoutcomesofthetreatmentetc.Inpracticalapplicationsavailabledataisalmostneversufcienttoprovidefulldescriptionoftheobject,andthisisparticularlythe 49

PAGE 50

caseinbiomedicaleld.Onehastooperatewithlimitedknowledgeabouttheobjectsthatdescribesomeparticularpropertiesrelevanttotheproblem.Thereforeitisrarelypossibletosolveclassicationproblemexactly.Theobjectivethenbecomestodeveloptechniquesforpredictingclasslabelsapproximately,providingminimalnumberofmisclassicationerrors.DuetoinherentuncertaintyinpredictedclasslabelscausedbyalimitedavailableinformationtheposteriorprobabilitiesP(classjobject)areoftenofinterest.Apartfromestimatingthecondencesoverclassication,posteriorprobabilitiesareusefulwhendealingwithunbalancedlossesorincaseswhereclassierismakingasmallpartoftheoveralldecisionontheobject[ 58 ],[ 33 ],[ 51 ].Inthisworkwewillrefertothealgorithmsthatestimatetheseprobabilitiesfromtheavailabledataastoprobabilisticclassicationalgorithmsorprobabilisticclassiers. 3.2TheProblemofClassication Theproblemofclassicationisaparticularcaseofawiderproblemofmachinelearning.ConsiderasetofobjectsX,asetoflabelsYandassumethatexistsanobjectivefunctiony:X!Y,valuesofwhichyi=y(xi)areknownonlyforthelimitedsubsetofobjectsfx1,...,xlgX.AsetofpairsXl=(xi,yi)li=1iscalledatrainingset(trainingsample).Theproblemofmachinelearningliesinreconstructingthefunctionalrelationbetweenobjectsandlabels.Inotherwords,oneshouldconstructafunctiona:X!Ysatisfyingfollowingconditions: Functionashouldallowefcientcomputationalimplementation.Inthiscasewewillcallitanalgorithm. Algorithma(x)hastoreconstructgivenlabelsfortheobjectsoftrainingsample:a(xi)=yi,i=1,...,l.Heretheequalitycanbetreatedasanapproximatedependingonparticularproblem. Therecanbedifferentaprioriconstrainsimposedonanalgorithma(x)suchasa(x)hastobesmooth,continuous,monotonicetc.Incertaincasesanalgorithmmodelcanbeimposed,sothatfunctiona(x)isdenedaprioriuptothevaluesoftheparameters. 50

PAGE 51

AlgorithmashouldexhibitageneralizationabilitymeaningitshouldbeabletoreconstructobjectivefunctionuptoacertainprecisionnotonlyonthetrainingsetbutonentiresetX. DependingonthenatureofsetYmachinelearningproblemscanbedividedintofollowinggroups.IfY=Rthantheproblemiscalledaregressionproblem.IncaseswhenXdescribespastbehavioroftheobjectandYdescribesitsfuturebehavior,themachinelearningproblemiscalledaforecastingproblem.Finally,ifY=f1,2,...,MgwhereMispositiveinteger,thentheproblemiscalledaclassicationproblemwithMclasses. Theproblemofclassicationhasbeenstudiedfordecadesnowandanumberofalgorithmshavebeendeveloped.Intheoriginalformulationofaclassicationproblemanalgorithmaisexpectedtopredicttheclasslabelsthemselves.However,aswasnotedbeforeduetotheuncertaintyofthesepredictedlabelsitisoftenmoreusefultoobtainposteriorprobabilitiesP(yjx)wherex2Xandy2Y.Posteriorprobabilitiesarerequiredwhenaclassiercontributestoapartoftheoveralldecisionasinensembleclassicationmodels.Incertainsituationsthecostofamisclassicationerrorisvastlydifferentfordifferentclasses.Hereagainposteriorprobabilitiesaremoresuitableastheyallowexperttoimposeathresholdbasedonwhichclassicationwillbeperformedwithabiastowardsaparticularclass. Arangeofclassicationalgorithmsattempttoestimateposteriorprobabilitiesnaturallyinthetrainingprocess,suchaslogisticregressionoraBayesianlearningmethod[ 36 ].Othertechniques,particularlythoseexploitinggeometricalpropertiesofthedata(e.g.SupportVectorMachines),usuallyprovidetheclasslabelsthemselves.Theareseveralapproachesproposedtoobtainposteriorprobabilitiesfromtheoutcomesobtainedbysuchalgorithms.InSection 3.4 wewilldiscussseveralapproachestoobtainprobabilisticoutputsforSupportVectorMachine(SVM),oneofthemostwidelyusedclassicationalgorithms[ 75 ]. 51

PAGE 52

3.3ClassicationAlgorithmsWithProbabilisticOutputs 3.3.1LogisticRegression Logisticregressionisprobablythemostwidelyusedmethodinbiomedicalapplicationsforpredictingbinaryoutcomes(classicationproblemswith2classes).Logisticregressionhasbecomeapopulartoolalsoinsocialsciences,marketingandbusinessapplications.Itscommonrecognitionisduetorelativesimplicityandeasyinterpretabilityofmodelparameters[ 3 ]. LetfY:yl2f0,1g8lgbeabinarysetofclassesandXbeasetofobjects,whereeachXl2Xisdescribedbymfeaturesfxl1,...,xlmg.Thelogisticregressionmodelisgivenby: P(Y=1jX=Xl)=exp(0+Pmi=1ixli) 1+exp(0+Pmi=1ixli).(3) Equivalently,thelogodds,calledthelogitfunction,hasthelinearrelationshiplogit(P(Y=1jX=Xl))=0+mXi=1ixli.Themagnitudeofthecoefcientihasthefollowinginterpretation:eiisanincreaseinoddsforevery1-unitincreaseinvalueofxli-thei-thfeature.Therefore,thecoefcientiindicatetheimpactofafeaturexliontheoverallpredictionontheobjectXl. Logisticregressionmodelsareusuallytbymaximumlikelihoodprinciple,usingtheconditionallikelihoodofYgivenX.Thelog-likelihoodfunctioncanbewrittenas: l()=XlyllogP(Y=1jXl,)+(1)]TJ /F3 11.955 Tf 11.96 0 Td[(yl)logP(Y=0jXl,),(3) Maximizationoflwithrespecttoprovidesthevaluesfortheparametersforthettedmodel.Therearetwowaystosolve( 3 ).Oneistodirectlyapplyiterativeoptimizationmethodforsolvingapproximatelysmoothnonlinearoptimizationproblems,suchasinterior-pointmethod.Theotheroneistoderiveoptimalityconditionsfor( 3 )and 52

PAGE 53

thentoattemptsolvingobtainedequations.Taking( 3 )intoaccountandassumingxl0=18l,( 3 )canberewrittenas: l()=mXi=0(Xlylxli)i)]TJ /F8 11.955 Tf 11.96 11.36 Td[(Xllog(1+exp(mXi=0ixli)).(3) Thelikelihoodequationsresultfromsetting@l()=@=0.Since @l() @i=Xlylxli)]TJ /F8 11.955 Tf 11.95 11.36 Td[(Xlxliexp(Pjjxlj) 1+exp(Pjjxlj), thelikelihoodequationsare Xlylxli)]TJ /F8 11.955 Tf 11.96 11.36 Td[(Xl^lxli=0,i=0,...,m,(3) where^l=exp(Pj^jxlj)=(1+exp(Pj^jxlj))isthemaximumlikelihoodestimateofP(Y=1jX=Xl).Theequationsarenonlinearandrequireiterativesolution.Thecommonapproachforsolvingequations( 3 )isaNewton-Raphsonmethod[ 3 ]. Severalresultsestablishexistenceofestimatesforlogisticregressionmodel.Thelog-likelihoodfunctionforlogisticregressionmodelsisstrictlyconcave[ 3 ].Maximumlikelihoodestimatesexistandareuniqueexceptforboundarycases[ 82 ].Estimatesdonotexistormaybeinniteincaseswhentheclassesarelinearlyseparable(e.g.existsahyperplanethatperfectlyseparatestwoclasses). 3.3.2BayesianLearning Bayesianapproachtoclassicationisbasedonthetheoremstatingthatifprobabilitydensityfunctionsofeachclassareknown,theclassicationalgorithmcanbeformulatedexplicitlyinananalyticalform.Moreover,thisalgorithmisoptimal,e.g.providesminimalprobabilityofaclassicationmistake.Inpracticeclassprobabilitydensityfunctionsusuallyarenotknownandshouldbeestimatedfromthetrainingsample.Asaresult,theBayesianalgorithmisnotoptimalinrealsituationssincethedensityfunctionsestimatesareapproximate. 53

PAGE 54

Consideraprobabilisticformulationofaclassicationproblem.GivenasetofobjectsXandanitesetofclassesY.Assume,thatthesetXYisaprobabilisticspacewithknownprobabilitydensityp(x,y)=P(y)p(xjy).ProbabilitiesofobservingobjectsofeachclassPy=P(y)areknownandarecalleda-prioriprobabilitiesoftheclasses.Probabilitydensityfunctionsofeachclasspy(x)=p(xjy)arealsoknownandarecalledclasslikelihoodfunctions.Theobjectiveistodesignanalgorithma(x)minimizingprobabilityofaclassicationmistake. Consideranarbitraryalgorithma:X!Y.ItsplitsthesetXintonon-overlappingsubsets:Ay=fx2Xja(x)=yg,y2Y.Probabilityofobservinganobjectofclassy,thatwouldbeclassiedbyanalgorithmaasclasss,isPyP(Asjy).Ify=s,thisgivesprobabilityofacorrectclassication.Ify6=s,thisgivesprobabilityofaclassicationmistake.Dependingontheproblem,thelossesassociatedwithdifferentclassicationmistakesmaybedifferent.Foreachpair(y,s)2YYdenoteysthelossassociatedwithassigningobjectofclassytoclasss.Itisnaturaltoassumeyy=0andys>0ify6=s. Thefunctionalofanaverageriskforanalgorithmaisanexpectedvalueofthelossduetomisclassicationerrors: R(a)=Xy2YXs2YysPyP(Asjy). Incasewhenthelossisthesameforallclasses(ys=1),theaverageriskfunctionalbecomessimplyaprobabilityofamisclassicationerror. ThefollowingresultestablishesBayesoptimaldecisionrule(seeforexample[ 14 ]). Theorem3.1. Ifa-prioriprobabilitiesPyandlikelihoodfunctionspy(x)areknown,thantheexpectedriskR(a)isminimizedbythealgorithm: 54

PAGE 55

a(x)=argmins2YXyinYysPypy(x). Ifthelossesduetomisclassicationdependsonlyonthetrueobjectclass,meaningyy=0andys=yforally,s2Y,theBayesianoptimaldecisionrulebecomessimpler: a(x)=argmaxy2YyPypy(x). TheBayesiandecisionrulecanalsobeformulatedintermsofposteriorprobabilitiesP(yjx).AccordingtotheBayesianrule: P(yjx)=p(x,y) p(x)=py(x)Py Ps2Yps(x)Ps. Theexpectedlossassociatedwiththeobjectxintermsofposteriorprobabilitieshastheform: R(x)=XyyP(yjx). Theoptimaldecisionrulethenbecomes: a(x)=argmaxy2YyP(yjx). Ifthelossesforallclassesareequal,thisrulesstatesthatanobjectxshouldbeclassiedtoaclasswithmaximalposteriorprobabilityP(yjx). Inrealapplicationstheproblembecomestoestimatea-prioriclassprobabilitiesPyandclassprobabilitydensityfunctionsp(xjy).WhileforlargesamplesprobabilitiesPycanbeestimatedrelativelycloselyby ^Py=ly l,ly=jXlyj,y2Y, 55

PAGE 56

theproblemofestimatingp(xjy)ismuchmoredifcult.Forthisreasoninpracticetheassumptionoffeaturesindependenceisoftenmade,resultinginso-calledNaiveBayesianclassier.Underthisassumptiontheprobabilitydensitypy(x)becomes:py(x)=p1y(x1)p2y(x2)...pmy(xm). Thissimpliestheproblemsignicantly,sinceitismucheasiertoestimatemone-dimensionaldensityfunctionsratherthananm-dimensionalone. 3.3.3GeneralizedAdditiveModels Generalizedadditivemodels(GAMs)extendlinearmodelsandinparticular,logisticregression,byreplacingthelinearform+Pixiiwiththeadditiveform+Pifi(xi),wherefiarenonlinearsmoothfunctions[ 37 ].Asimpleadditivemodelwithalinearlinkfunctionhastheform Y=+mXi=1fi(xi)+", where"isanerrortermwithzeromean. Thefunctionsfiandparameterarelearnedbyminimizingpenalizedsumofsquaresoverthetrainingdataset: minf,nXl=1(yl)]TJ /F6 11.955 Tf 11.96 0 Td[()]TJ /F4 7.97 Tf 17.07 14.94 Td[(mXi=1fi(xli))2+mXi=1iZfi(ti)2dti,(3) wherethei0aretuningparameters.Itcanbeshownthatthesolutionto( 3 )isanadditivecubicsplinemodel,whereeachfunctionfiisacubicsplineoverthefeatureXiwithknotsateachuniquevaluexli,l=1,...,n.Estimationofandfi,i=1,...,mcanbedonewiththebackttingalgorithm[ 37 ]. Inthecontextofpredictingbinaryclassesthegeneralizedadditivemodeltakestheform:logit(P(Y=1jX=Xl))=+Xifi(xli). 56

PAGE 57

Estimationofandfi,i=1,...,misaccomplishedbyalocal-scoringalgorithmforttingaweightedadditivemodel[ 37 ]. GAMsarepowerfultoolsinsolvingclassicationproblems.AlthoughtheymightresemblediscussedbeforelogisticregressionandthenaiveBayesianclassicationalgorithms,theiradvantageisthattheyareabletocapturethedependenciesbetweenthevariablesinthemodel.Duetothewaythefunctionsfiarelearnedinthelocalscoringalgorithm,theimpactofonevariableontotheoutcomeaffectsimpactsofothervariables,whichisreectedintheresultingfunctionsfi.However,aswitheveryclassicationalgorithm,ifappliedwithoutcautionGAMsmayleadtoapoorclassicationperformanceonthetestingset.Becauseoftheirgreatexibility,itiseasytoovertthedatawhentrainingGAMsandhencetoobtainpoorgeneralizingabilityoftheresultingalgorithm.Thiscanbecontrolledbyadjustingthedegreesoffreedomofcorrespondingcubicsplinesthatconstitutefunctionsfiinthemodel.Degreeoffreedomofasplineindicatesit'scurvature.Thelowerthedegreeoffreedom,thesmootheristhespline.Degreeoffreedomequalto1correspondstoalinearfunction.Highdegreesoffreedomcorrespondtoahighlynonlinearsplines,whichinextremecasecanpassexactlythrougheverypointinthedataset.HastieandTibshiraniproposedseveralpossiblemethodsforautomaticselectionofdegreesoffreedomofthesplinesinthemodel(see[ 37 ],sec.6.9).Onecannotsimplyminimizetheerrororthedeviance(likelihood-ratiostatistics)D(y,)ofthemodel,asthiswillleadtoahighlyovertsolution.Instead,oneoftheapproachesistousecross-validation,leavingoneobjectoutofsample,trainingmodelontheremainingdataandtestingtheresultontheleftoutobject.Let^ibethepredictedposteriorprobabilityati-thobjectobtainedbyleavingthei)]TJ /F1 11.955 Tf 9.3 0 Td[(thobjectoutofthesample.Thenthecrossvalidateddevianceis: CV=1 nnXi=1D(yi;^i).(3) Thesplineparametersareselectedbyminimizing( 3 ). 57

PAGE 58

AnotherapproachistominimizeAkaikeInformationCriterion(AIC),whichisdenedasAIC=D(y;^)=n+2df=n,wheredfaretotaldegreesoffreedomofthemodel(see[ 37 ],sec.6.8fordetails). 3.4ObtainingPosteriorProbabilitiesForNon-ProbabilisticClassiers Differentclassicationalgorithmsexploittheproblempropertiesofdifferentnatureandrelyondifferentaspectsofthedatasetwhenmakingclassicationdecision.Currentlytherearenouniformlybestalgorithmthatwouldbesuperioronallpracticalproblems.Differentalgorithmsturnouttobemostsuitableindifferentapplications.Notalloftheclassicationalgorithmshoweverproduceposteriorprobabilities.Thequestionarisestherefore,isitpossibletoderiveoratleasttoestimateposteriorprobabilitiesfromoutputsofnon-probabilisticclassicationalgorithms?WewilldiscussthisproblemforthecaseofSupportVectorMachine(SVM)methodwhichisoneofthemostpopularandwidelyusedclassicationalgorithms[ 75 ].Manyoftheapproachespresentedherecanbeeasilyextendedtootherclassicationalgorithms. Initscoreconceptthesupportvectormachine(SVM)algorithmndsahyperplanethatseparatesobjectsoftwoclasseswithamaximummargin(Figure 3-1 ).Inthekernel Figure3-1. SupportVectorMachinemethod.Thedistancebetweendashedhyperplanesisproportionaltothemarginbetweenclasses. versionofSVMthehyperplaneliesinthehigh-dimensionalkernelspace.TrainingofanSVMminimizesapproximationtothetrainingmisclassicaitonrateandmaximizesthemarginbetweenclasses(distancebetweensupporthyperplanes): 58

PAGE 59

minCXl(1)]TJ /F3 11.955 Tf 11.95 0 Td[(ylfl)+khkF, wherefl=h(xl)+b,h(x)=Plyllk(xl,x)isanoutputofanSVM(distancefromobjectxltotheseparatinghyperplane)andkhkFisanormofhinthekernelspace. In[ 74 ]sec.11.11,Vapnikproposedtoestimatedconditionalprobabilitydensityinthefeaturespacealongthelinex=x0+e(t)]TJ /F3 11.955 Tf 11.96 0 Td[(x0)passingthroughapointofinterestx0.ForthispurposeeachobjectXlfromthedatasetisdecomposedintotwocomponents(tl,ul),wheretl=XleisaprojectionofthevectorXlontodirectioneandulisanorthogonalcomplimentofthevectoretltothevectorXl.LetX0=(t0,u0).VapnikproposestosolveequationZt0p(y=1jt,u0)dF(tju0)=F(y=1,tju0)byintroducingapproximationsFl(tju0)=lXi=1i(u0)(t)]TJ /F3 11.955 Tf 11.96 0 Td[(ti),Fl(y=1,tju0)=lXi=1i(u0)(t)]TJ /F3 11.955 Tf 11.96 0 Td[(ti)(yi),i(u0)=g(jjui)]TJ /F3 11.955 Tf 11.95 0 Td[(u0jj) Pli=1g(jjui)]TJ /F3 11.955 Tf 11.96 0 Td[(u0jj),whereg(u)isaParzenkerneland(yi)=[y1=1].Themethodforsolvingtheseequationscanbefoundin[ 74 ],sec.7.3.ItrequiressolvingalinearsystemateachSVMevaluation.Inaddition,theresultingposteriorprobabilitiesarenotmonotonicwithf[ 58 ]. Letf(x)beanunthresholdedoutputofSVM.IncaseoflinearSVMf(x)isadistancefromthepointxtotheseparatinghyperplane.OneapproachproposedbyWahbaetal[ 79 ]consistsindeningposteriorprobabilitiesas: 59

PAGE 60

P(y=1jx)=1 1+exp()]TJ /F3 11.955 Tf 9.29 0 Td[(f(x)). TheSVMalgorithmisthatlearnedviamaximizationofalikelihoodfunctionmaxnXi=1yilogpi+(1)]TJ /F3 11.955 Tf 11.95 0 Td[(yi)log(1)]TJ /F3 11.955 Tf 11.96 0 Td[(pi)), wherepi=p(xi).ThedrawbackofthisapproachisthatitdoesnotproducesparseSVMs,meaningthatalargenumberoffeatures(kernelsincaseofkernelSVM)willbeusedinanalmodel. AnextensiontothisapproachhasbeenproposedbyPlattetal.AccordingtoBayes'rule,theposteriorprobabilitycanbecomputedas: P(y=1jf)=p(fjy=1)P(y=1) Pi=0,1p(fjy=i)P(y=i). whereP(Y=i),i=0,1arepriorprobabilitiesthatcanbecomputedfromthedataset.Theposteriorprobabilitythereforeisafunctionoffandhasaform: P(Yjf)=1 1+exp(af2+bf+c).(3) OneissuewiththisapproachisthattheassumptionofGaussianclass-conditionalisoftenviolated(seePlatt,Fig.1).TodealwiththisissuePlattproposedinsteadofestimatingclass-conditionaldensitiesP(fjY)totaparametricmodeldirectlytoposteriorprobabilitiesP(Yjf).Basedontheempiricaldataobservationsauthorproposedtouseamodelinaformofasigmoid:P(Y=1jf)=1 1+exp(Af+B). Thismodelisdifferentfromdescribedabove,sincetwoparametersaretrainedseparatelyhere.Authorsproposetotthemodelviamaximumlikelihoodfromatraining 60

PAGE 61

dataset(yl,fl).TheparametersAandBarederivedbysolvingmaximizationproblem:maxnXl=1yllogpl+(1)]TJ /F3 11.955 Tf 11.95 0 Td[(yl)log(1)]TJ /F3 11.955 Tf 11.95 0 Td[(pl)), wherepl=1=(1+exp(Afl+B)). Grandvaletetal.proposedtoconsiderposteriorprobabilitiesP(YjX)asasemiparametricmodelp(yjx,)=g(x,)+"(x),whereg(x,)isaparametricpartofthemodeland"(x)isanonparametricnuisancefunction[ 33 ].Theparametersareestimatedbysolvingthefollowingmaximumlikelihoodproblem: min,")]TJ /F8 11.955 Tf 11.29 11.35 Td[(Xltilog(p(1jxl,))+(1)]TJ /F3 11.955 Tf 11.96 0 Td[(ti)log(1)]TJ /F3 11.955 Tf 11.95 0 Td[(p(1jxl,))s.t.p(1jx,)=g(x,)+"(x)0p(1jx,)1")]TJ /F5 11.955 Tf 7.09 -4.34 Td[((x)"(x)"+(x) where")]TJ /F5 11.955 Tf 7.08 -4.34 Td[((x)and"+(x)areuserdenedfunctionsthatplaceconstraintsonnon-parametricterm"(x). InthecontextofSVMtheaboveproblemreducestosolving g(x)+"+(x)=exp)]TJ /F5 11.955 Tf 9.3 0 Td[((c0+c1[1)]TJ /F5 11.955 Tf 11.96 0 Td[((f(x))]+)1)]TJ /F3 11.955 Tf 11.96 0 Td[(g(x))]TJ /F6 11.955 Tf 11.96 0 Td[(")]TJ /F5 11.955 Tf 7.08 -4.94 Td[((x)=exp)]TJ /F5 11.955 Tf 9.3 0 Td[((c0+c1[1+(f(x))]+)s.t.0g(x)+")]TJ /F5 11.955 Tf 7.09 -4.34 Td[((x)10g(x)+"+(x)1")]TJ /F5 11.955 Tf 7.08 -4.34 Td[((x)"+(x) 61

PAGE 62

withrespecttog,")]TJ /F1 11.955 Tf 10.41 -4.34 Td[(and"+.Heref(x)denotesSVMoutputs.Givenproblemhasasolutionfordifferentg,")]TJ /F1 11.955 Tf 10.41 -4.34 Td[(and"+,whichgivesexibilityinchoosingthemostappropriatefunctionsforaparticularapplication.Grandvaletetal.discusstheselectionofg,")]TJ /F1 11.955 Tf 10.41 -4.34 Td[(and"+forthecaseofunbalancedclassication. 3.5EvaluatingthePerformanceofaClassier Evaluatingtheperformanceofaclassicationalgorithmisanimportantstepinrealdataanalysis.Thereareanumberofapproachestotacklethisproblem,andwewilldiscusssomeofthemhere.Generallyacceptedframeworkistosplittheavailabledatasetintotwoparts-trainingandtesting.Thealgorithmisthenbuiltuponthetrainingpartandallthederivationsregardingitsperformancearemadebasedonthetestingpart. Thesimplestapproachistoestimatetheaccuracyoftheclassier,whichisdenedasaratioofthetotalnumberofcorrectlyclassiedobjectstothetotalnumberofobjectsinthetestingdataset: A=PlinX[yl=a(xl)] jXj. Incaseofaprobabilisticclassieroneshouldrstimposeathresholdontheposteriorprobabilityvalues,basedonwhichobjectswouldbeclassiedtooneoranotherclass.Althoughtheaccuracymeasurecanserveasagoodindicatorofanalgorithmperformanceincasesofbalancedclasses(wherethenumbersofobjectsindifferentclassesarecomparable),itprovideserroneousinsightsinproblemswithunbalancedclasses.Forexample,inthecasestudydiscussedattheendofthischapter,thetwoclassesconstitute98%and2%ofthetotalnumberoftheobjectsinthedatasetrespectively.Thereforeaclassierthatblindlyassignseveryobjecttoaclass1wouldinaveragedemonstrateaccuracyof98%. Moreinsightfulmeasuresofaclassierperformancearesensitivity(ortruepositiverate)andspecicity(truenegativerate),whicharedenedasclassicationaccuraciesineachparticularclass: 62

PAGE 63

sensitivity=Pl[yl=1ja(xl)=1] Pl[yl=1]; specicity=Pl[yl=0ja(xl)=0] Pl[yl=0]. Again,athresholdonposteriorprobabilityvaluesshouldbepredenedinordertoassessthesemeasures.Sensitivityandspecicitycanprovideinsightfulconclusionsregradingthealgorithmperformanceincasesofbalancedandunbalancedclasses.Forinstanceintheexamplediscussedabovethealgorithmthatblindlyclassiesalltheobjectstoclass1wouldobtainsensitivityof100%butspecicityof0%,whichindicatesthatthealgorithmisnotsuitablefortheproblem. Onelimitationofdiscussedsofarperformancemeasuresliesinthefactthatanarbitrarydiscriminatingthresholdmustbepreselectedinordertoassesstheclassicationaccuracies.Infact,inrealapplicationsthereisalwaysatradeoff:thehigherthesensitivity(meaningtheloweristhethresholdtoclassifyobjectstoclass1),theloweristhespecicityandviceversa.Itisnotalwaysclearwhatthresholdvalueisthemostappropriateinagivenproblem(anditoftendependsonthecostsofmisclassicationerrors,whichmaybenoteasytoevaluate).OnewaytoeliminatetheneedofthearbitrarilythresholdistoconstructaReceiverOperatingCharacteristic(ROC)curve.AnROCcurveisagraphicalplotofsensitivityversus(1-specicity)computedforeachpossiblethresholdvaluefrom[0,1].Sinceinrealproblemsthedatasetsarenite,onlyanitenumberofdifferentthresholdvaluesneedtobeassessed,namelytheuniquevaluesofpredictedposteriorprobabilitiesforobjectsinthedataset.Eachparticularthresholdvaluecorrespondstooneparticularvalueofsensitivityandtooneparticularvalueofspecicity,andthereforegeneratesonepointontheROCplot.Movingthroughthethresholdvaluesfrom0to1wegenerateasequenceofpointsthatformanROCcurve(Figure 3-5 ). 63

PAGE 64

Figure3-2. AnexampleofaROCcurve.Theidealclassiercorrespondstoapoint(1,0)atthecorneroftheplot.Thecompletelyrandomclassiercorrespondstoalinefromorigintothepoint(1,1).Curvesthatliesintheupperpartoftheplotcorrespondtoclassiersthatareperformingbetterthanrandom. Anidealclassierthatwouldyield100%sensitivityand100%specicitywouldproduceapointintheupperleftcornerwithcoordinates(1,0)attheplot.Acompletelyrandomguesswouldproduceasequenceofpointsarounddiagonal(0,0)-(1,1)line.Curvesthatlieintheupperpartoftheplotcorrespondtoclassiersthatperformbetterthanrandom.Thehigheristhecurve,thebetterisoverallperformanceoftheclassier.TheareaundertheROCcurve(alsoknownasc-statistics)providesagoodmeasureoftheclassierperformance.It'svalueliesintherangeof[0.5,1]withvalueof1correspondingtoanidealclassierandvalueof0.5correspondingtoarandomguess.ItcanbeshownthattheareaundertheROCcurveiscloselyrelatedtoaprobabilitytorankarandomobjectofclass1higherthanarandomobjectofclass0[ 34 ].FurtherinsightsandadditionalanalysiswithROCcurvescanbefoundin[ 27 ],[ 35 ]. Theapproachesdiscussedsofarassessthediscriminativeabilityoftheclassier,meaninghowwellcantheclassierdiscriminateobjectsfromdifferentclasses.Theydon'ttellusmuchhoweverabouthowaccuratelyaretheposteriorprobabilitiesestimated.Aclassierthatassignsarandomvaluebetween0and0.5toalltheobjects 64

PAGE 65

fromclass0andarandomvaluebetween0.5and1toalltheobjectsfromclass1inthedatasetwouldyieldaperfectdiscriminationbetweentwoclasses,butobviouslywronginsightsabouttheposteriorprobabilities.OneapproachtoestimatetheaccuracyoftheclassierintermsofthepredictedposteriorprobabilitiesisaHosmer-Lemeshowgoodness-of-ttest[ 42 ].Theideaofthetestliesingroupingobjectswithsimilarpredictedposteriorprobabilityvaluesandthencomparingtherateoftheobjectsfromclass1ineachgrouptothepredictedaverageposteriorprobabilityinthatgroup.Letnbeanumberofobjectsandgbeanumberofgroups.Wewillsplitobjectsintogroupssuchthattherstgroupwouldcontainn1=n=gobjectswiththesmallestvalues,andthelastgroupwouldcontainng=n=gobjectswiththelargestvaluesofpredictedposteriorprobability.TheHosmer-Lemeshowstatistic^Cisdenedas ^C=gXk=1(ok)]TJ /F3 11.955 Tf 11.95 0 Td[(nkpk)2 nkpk(1)]TJ /F5 11.955 Tf 12.1 0 Td[(pk), wherenkisthenumberofobjectsinthek)]TJ /F1 11.955 Tf 9.29 0 Td[(thgroup,ok=nkXj=1yj isthenumberofobjectsofclass1ink)]TJ /F3 11.955 Tf 11.95 0 Td[(thgroup,andk=nkXj=1pj=nk istheaverageestimatedprobability. HosmerandLemeshowhaveshownthatthedistributionofthestatistic^Ciswellapproximatedbythechi-squaredistributionwithg)]TJ /F5 11.955 Tf 10.28 0 Td[(2degreesoffreedom,2(g)]TJ /F5 11.955 Tf 10.28 0 Td[(2).Theconclusionregardingthemodeltnowcanbedrawnfromthep-valueofthe2(g)]TJ /F5 11.955 Tf 12.33 0 Td[(2)distributionatpoint^C.SmallvaluesofC-statisticcorrespondtolargevaluesofp.Incontrast,smallvaluesofpindicatethatthemodeldoesnottthedataandthepredictedposteriorprobabilitiesarenotaccurate. 65

PAGE 66

Apartfrompurelymathematicalassessmentofaclassicationalgorithmperformance,practicalconsiderationsshouldalsobetakenintoaccount.Itisimportanttorealize,whatwouldbethebenetofimplementinggivenclassierinapracticalclinicalsetting[ 49 ].Theresultsofaclassicationalgorithmareusedinmakingclinicaldecisions.Dependingontheriskpredictedbyanalgorithmclinicianswoulddecidewhethertoproceedwithcertainactions.Therefore,onlyseveralclinicallyrelevantthresholdsofrisk,thattriggerparticularclinicaldecisions,areofpracticalimportance.Thevariationsandimprecisionsinvaluesofpredictedprobabilityarenotimportantaslongasthepredictiontriggersthesameclinicaldecisions.Theevaluationofconsequencesofapplyingapredictionalgorithmusuallyreducestoevaluationofexpectedbenetsandharmofmakingclinicaldecisionsaccordingtothealgorithmoutput. Thereareseveraldecision-analyticmethodsthatidentifypossibleconsequenceofaclinicaldecisionandsimulatetheexpectedoutcomesofalternativeclinicalcarestrategies.Suchanalysisrequiresexplicitevaluationofhealthoutcomes,suchasnumberofcomplicationsprevented,qualitylife-yearssaved[ 43 ],[ 77 ].Thealgorithmthatmaximizestheoutcomeofinterest(notnecessarilyaccuracy,ROCetc.)isconsideredtheoptimaloneinthissetting.Suchmethodshoweverrequireadditionaldata,suchasonproceduralcosts,whichareusuallynotfoundintheavailabledataset. Vickersetal.proposedamethodcalleddecisioncurvesanalysisforevaluatingclinicalrelevanceofaprobabilisticclassierthatcanbeapplieddirectlytothetestingdatasetanddoesnotrequireadditionalinformation[ 77 ].Tobemorespecic,inthefollowingdiscussionwewillassumethatwedealwithadiseasediagnosisproblem.Inordertointroducetheconceptofthemethod,authorsprovidedthefollowingsimpleexample.Considerapatientdecidingwhethertotakeatreatmentforaspecicdisease.Letpbeaprobabilityofadiseaseanda,b,canddbebenetsassociatedwitheachpossibleoutcome(Figure 3-9 ).Assumethatthereisaprobabilisticmodelavailable.Ifthepredictedriskiscloseto0,thepatientwilllikelydecidenottotakeatreatment.If 66

PAGE 67

thepredictedriskiscloseto1,thepatientwilldecidetobetreated.Howeveratsomeprobabilityptbetween0and1thepatientwillbeunsurewhethertobetreatedornot,sincetheexpectedbenetfromthetreatmentwouldbeequaltotheexpectedbenetofavoidingthetreatment. Figure3-3. Atoyexampledecisiontree[ 77 ]. Itiseasytoderive,that: pta+(1)]TJ /F3 11.955 Tf 11.96 0 Td[(pt)b=ptc+(1)]TJ /F3 11.955 Tf 11.95 0 Td[(pt)d))a)]TJ /F3 11.955 Tf 11.96 0 Td[(c d)]TJ /F3 11.955 Tf 11.96 0 Td[(b=1)]TJ /F3 11.955 Tf 11.96 0 Td[(pt pt. Hered)]TJ /F3 11.955 Tf 12.52 0 Td[(bistheconsequenceofbeingtreatedunnecessarily(theharmfromafalsepositiveresult),a)]TJ /F3 11.955 Tf 13.15 0 Td[(cistheconsequenceofavoidingtreatmentwhenitwouldbebenecial(theharmfromafalsenegativeresult). Withoutlossofgeneralityonecanassume,thatavalueofatrue-positiveresulta)]TJ /F3 11.955 Tf 10.63 0 Td[(cis1.Thevalueofafalsepositiveresultb)]TJ /F3 11.955 Tf 12.05 0 Td[(disthen)]TJ /F3 11.955 Tf 9.3 0 Td[(pt=(1)]TJ /F3 11.955 Tf 12.05 0 Td[(pt).Wecancalculatethebenetas: Netbenet=truepositivecount n)]TJ /F5 11.955 Tf 13.15 8.09 Td[(falsepositivecount npt 1)]TJ /F3 11.955 Tf 11.95 0 Td[(pt. Thedecisioncurveisthenbuiltbyfollowingthesteps: 1. Choosevalueforpt. 67

PAGE 68

2. Calculatenumberoftruepositiveandfalsenegativeresultsatadiscriminatingthresholdofpt. 3. Calculatethenetbenetofthepredictionalgorithm. 4. Varyptoverappropriaterangeandrepeatsteps2and3. 5. Plotnetbenetagainstpt. 6. Repeatsteps1through5foreachalgorithmunderconsideration. Vickersetal.demonstratedthemethodontheproblemofpredictingseminalvesicleinvasion(SVI)inprostatecancerpatients.TheresultingdecisioncurvehasaformshownonFigure 3-4 .Thesolidcurvecorrespondstoastrategyofmakingdecisionsbasedonaprobabilisticalgorithm.Thehorizontallinecorrespondstoastrategyoftreatingnoone,whilethestraightdashedlinecorrespondstoastrategyoftreatingeveryone.Thedecisioncurvedemonstratestherangeforptwheretheprobabilistic Figure3-4. AdecisioncurveforSVIdata.Thesolidcurvecorrespondstoastrategyofmakingdecisionsbasedonaprobabilisticmodel.Thehorizontallinecorrespondstoastrategyoftreatingnoone.Thestraightdashedlinecorrespondstoastrategyoftreatingeveryone[ 77 ]. algorithmisofpracticalvalue.FromFigure 3-4 itcanbeconcluded,thatinthecaseconsideredbyauthorstheprobabilisticmodelismostlybenecialintheptrangeof2%-20%,andisnearlyuselesswhenpt<2%orpt>40%.Inthetwolattercasestrivialstrategiesoftreatingeveryoneornoonerespectivelyprovidethesamebenetasthe 68

PAGE 69

strategybasedonprobabilisticmodel,althoughthemodelisfarsuperioraccordingtoanyofthemathematicalaccuracytestsdiscussedearlierinthissection. Ifseveralprobabilisticalgorithmsshouldbecomparedinordertopicktheoptimaloneforaparticularapplication,thedecisioncurvesshouldbecalculatedforeachofthealgorithmandplottedatthesameplot.Thealgorithmprovidinghighestbenetintherelevantrangeforptshouldbeselected. 3.6AssessingMortalityRiskinPost-OperativePatients 3.6.1Background Thepostoperativeacutekidneyinjury(AKI),characterizedbyevensmallincreaseinroutinelymeasuredserumcreatinine(sCr)level,hasbeendemonstratedtobenotonlyoneofthemostcommoncomplicationaffectingupto30%ofsurgicalpatients,butalsoassociatedwith2to10foldincreasein-hospitalmortalityanddecreasedsurvivalupto15yearsaftersurgery[ 12 ],[ 13 ],[ 39 ].Intraoperativeinterventionsthatcouldhelppreventpostoperativecomplicationsareroutinelyappliedwithoutconsiderationofapatient'sindividualriskproleorarenotappliedatallbecauseriskiscompletelymissed.Weaimedtodevelopacomprehensivemodelthatreectsapatient'sphysiologicalstatusandispredictiveofin-hospitalmortality.Inaddition,wewishedtostudytheassociationbetweenpatternsofsCrlevelchangeandin-hospitalmortalityrisk.Anelevatedserumcreatinine(sCr)levelinbloodiscommonlyrecognizedasanindicatorofAKI.However,thedegreetowhichpatternsofsCrchangeareassociatedwithin-hospitalmortalityisunknown.Studydesign WeperformedaretrospectivestudyinvolvingpatientsadmittedtoShandsHospital(Gainesville,FL)from2000through2010.Foreachpatientwhounderwentasurgerydetailedclinicalandoutcomedatawerecollected.Specicdataelementsincludetheadmissiontype,thesurgeryproceduretype,patientage,race,gender,morbidityscore,providerIDandclinicalandphysiologicalvariablesduringpatient'sstayinthe 69

PAGE 70

hospital.Weassociatedthein-hospitalmortalityriskwithsCrchangepatternbyusingamultivariateprobabilisticmodel[ 62 ].InadditionweperformedstatisticalanalysisusingGeneralizedAdditiveModels(GAMs)[ 37 ].Patientselection Patientswhowereolderthan17yearsofageandwhounderwentsurgerywereeligibleforinclusioninouranalysis.Variablesandmodels Theprimaryoutcomevariablewasin-hospitalmortality.Thesecondaryoutcomevariablewas90-daysmortality(countingfromtheadmissiondate).TheprimaryexposurevariablewasatimeseriesofsCrmeasurementstakenduringpatient'sstayinthehospital.Covariatesincludedvariablesspeciedaprioriaspotentialcontributorstomortalityrisksonthebasisofexpertjudgement.Patient-levelcovariatesincludedadmissiontype;thesurgeryproceduretype;patientsage,race,gender;ICUadmission(yes/no),morbidityscore(ameasureoftheseverityofillnessrangingfrom0to16,withhigherscoresindicatingmoresevereillnessandahigherriskofdeath);basesCrlevelmeasuredatpatient'sadmissiontothehospital;thelengthofthehospitalstaybeforethesurgery;providerID.FortheGAMapproachweconsideredthreemodelsbasedondifferentrepresentationsofsCrtimeseries.Firstmodel(referencedasModel1)includedincludedthemaximumsCrvalueandthelastavailablesCrvalue.Secondmodel(referencedasModel2)includeddifferencesbetweenmaximumsCrvalue,lastavailablesCrandbasesCrvalue.Thirdmodel(referencedasModel3)includedratiosofmaximumsCrvalueandlastavailablesCrvaluetobasevalue.AllthreemodelsalsoincludedbasesCrvalueasaseparatefeature.Inaddition,weconsideredamodelthatincludesonlypreoperativedata(nosCrfactors)referencedasModel0.Vericationanalysis Toverifytheresultsofouranalysis,weconducteda70/30crossvalidationprocedure.Thedatasetwasrandomlysplitinto70%oftrainingand30%oftesting 70

PAGE 71

data.Themodelparameterswerelearnedfromthetrainingpartandthereportedresultswereobtainedfromapplyingthemodeltothetestingpart.Theprocedurewasperformed100timesforeachanalysis.Statisticalanalysis MultivariatemodelingoftheassociationbetweenthesCrpatternandin-hospitalmortalitywasperformedusingGAMswithlogisticlinkfunction,withadjustmentforcovariatesspeciedaspotentialcontributorstotheoutcomeasdescribedabove(seeAppendixfordetails).WeusedRandMATLABsoftwaretoperformtheanalysis.Categoricalvariablesweremodeledwithconditionalprobabilitiesforapatienttohaveaparticularvariablevalueconditioningontheoutcome.ThesurgerytypewasmodeledbasedonICD9codeswithaforeststructure,whereeachnoderepresentsagroupofprocedures,withrootsrepresentingmostgeneralgroupsofproceduresandleafnodesrepresentingspecicprocedures.Thecontinuousfunctionsweremodeledwithcubicsplineswithdegreesoffreedomestimatedbymaximizingrestrictedlikelihoodfunction[ 83 ].ThevalidityandaccuracyofthemodelwereassessedviaROCcurveandHosmer-Lemeshowgoodness-of-ttest[ 42 ].DifferentmodelswerecomparedusingVuong'stestfornon-nestedmodels[ 78 ]. 3.6.2ProbabilisticModelBasedonLogisticFunction 3.6.2.1Probabilisticmodel Weestimatedprobabilityofin-hospitalmortality(inthehospital,C=1,otherwiseC=0)byusingthefollowingmodel[ 62 ]: P(C=1jX=x)= 1+exp w0+mXi=1wigi(xi)!!)]TJ /F9 7.97 Tf 6.58 0 Td[(1;(3) here,misthenumberofriskfactors,X=(X1,...,Xm)aretheriskfactors,x=(x1,...,xm)arethevaluesofthesefactors,giisanonlinearriskfunctionassociatedwiththeithriskfactors(detailedinNonlinearriskfunctionsbelow),w0=lnP(C=1)=P(C=0)istheapriorioddsratio.Theweightsfwigindicateimportanceof 71

PAGE 72

Figure3-5. ROCcurvesdemonstratingmodelpredictingperformanceondifferentsetsoffeatures.A)Allfeatures.B)Preoperativefeatures. Table3-1. Weightsfwigassociatedwithriskfactors. Riskfactorweight(95%CI) procedure0.6492(0.62960.6993)admissiontype0.4103(0.38030.4580)age0.9912(0.91051.0567)doctorID0.5400(0.51850.6039)morbidity0.2566(0.19340.3526)gender0.5669(0.31890.7597)maxCr/minCr0.2527(0.22340.3160)recentCr/minCr0.6683(0.62410.6945) correspondingriskfactorsandarelearnedfromthedata(seeNonlinearriskfunc-tions). Givenvaluesoftheriskfactors,( 3 )providesaprobabilisticscoreinthe[0,1]interval,withahighscoreindicatingahighriskofmortality(inthehospital).FordifferentsubsetsofvariablestheresultingROCcurvesarepresentedinFigure 3-5 Thedependencebetweentruepositive/truenegativeratesandthethresholdispresentedonFigure 3-6 Thedistributionofriskfactorsweightsoverthe100samplesof70/30crossvalidationisshowninTable 3-1 andinFigure 3-7 72

PAGE 73

Figure3-6. Truepositiveandtruenegativeratesofthemodeldependingonthethreshold. 3.6.2.2Riskfactorsanalysis Nonlinearriskfunctions.Undertheassumptionofstatisticalindependenceoftheriskfactors,functionsfgigandweightsfwigsatisfy: gi(xi)=lnP(Xi=xijC=0) P(Xi=xijC=1); (2)w0=lnP(C=1) P(C=0);wi=1,i=1,...,m. Indeed,assume,thatallweightswjareequalto1.Assumealso,thatthefeaturesxjareindependentrandomvariables.Then, 73

PAGE 74

Figure3-7. Learnedriskfactorsweightsfwig.Oneachbox,thecentralredmarkisthemedian,theedgesoftheboxarethe25thand75thpercentiles,thewhiskersextendtothemostextremedatapointsnotconsideredoutliers,andtheoutliersaremarkedwithredcrosses. P(C=1jx1,...,xm)=(1+exp(w0+mPi=1gi(xi))))]TJ /F9 7.97 Tf 6.58 0 Td[(1) (3) )exp(w0+mPi=1gi(xi))=1 P(C=1jx1,...,xm))]TJ /F5 11.955 Tf 11.96 0 Td[(1==P(C=0jx1,...,xm) P(C=1jx1,...,xm)=P(C=0,x1,...,xm) P(x1,...,xm)P(x1,...,xm) P(C=1,x1,...,xm)==P(x1,...,xmjC=0) P(x1,...,xmjC=1)P(C=1) P(C=0). Takinglogarithmofbothsidesgives w0+mXi=1loggi(xi)=logP(x1,...,xmjC=0) P(x1,...,xmjC=1)+logP(C=1) P(C=0).(3) Sincefeaturesxiareassumedtobeindependent,P(x1,...,xmjC)=mQi=1P(x1jC).Therefore, w0+mXi=1loggi(xi)=mXi=1logP(xijC=0) P(xijC=1)+logP(C=1) P(C=0).(3) 74

PAGE 75

Note,thatinthisfunctionalequationthevariablesxiareseparated,andweimmediatelygetequation( 2 ). Inordertoaccountforstatisticaldependenciesbetweenriskfactorstheweightsfwigareadjusted,whilekeepinggi-sasin( 2 ).Inparticular,weobtainthevaluesofweightswifromthedatabysolvingthefollowingoptimizationproblem: maxw1,...,wmXk2f0,1gXj:Cj=klnP(Cj=kjXi=xji,...,Xi=xjm);(3) hereprobabilityiscomputedbasedon( 3 ),indexjdenotespatientsinthedataset,Cj2f0,1gisanoutcomeforthejthpatientandxjiisavalueofithriskfactorforthejthpatient. Obtainedweightsandnonlinearriskfunctions,togetherwith( 3 )provideanassociationbetweenriskfactorsvaluesandestimatedmortalityrisk(Figure 3-14 ). Figure3-8. Oddsratioscomputedwithrespecttoanapriorioddsinthedataset(sashedlineonthegures). Categoricalriskfactors.Thefollowingarethecategoricalvariablesinourdataset:procedure(seedetailsinProcedureriskfactor),admissiontype,gender,race,day 75

PAGE 76

ofsurgery,morbidityscore,attendingdoctorID,ICUadmission.Wesubstitutedthevaluesxiofcategoricalvariableswiththeratioslog[P(Xi=xijC=1)=P(Xi=xijC=0)],whereP(Xi=xjC=c)=#fj:Cj=c,xji=xg=#fj:Cj=cg.Wethentreatedeachcategoricalvariableasanorderedvariable.Incaseofclassicationtreessuchsubstitutiongivestheoptimalsplits,intermsofcross-entropyorGiniindex([ 36 ],p.310).Basedonourexperiments,suchmodelingofcategoricalvariablesprovideslessoverttingthanwhenusingbinarydummyvariables. InordertoobtainareliableestimateofP(Xi=xjC=c),riskfactorcategorieswithfewerthan100recordsweregroupedtogetherandlabeledother.Thisothergroupwasfurthersplitintoseveralsubgroupswhereeachsubgroupcontainedcategorieswithsimilarproportionsofpatientsformdifferentclasses.Thiswasachievedbyperformingk-meansclustering([ 14 ],p.424)onthesetofcategoriesinothergroup.Wesetthenumberofclustersto5.ThecomputationswereimplementedinMATLABsoftwareusingbuilt-inkmeansfunction. Continuousriskfactors.Forcontinuousriskfactors,wecomputedfunctionsfgigas gi(x)=lnf0i(x) f1i(x),(3) wherefkiisanestimatedprobabilitydensityfunctionofvaluesofithfactorinpatientswiththeoutcomeC=k. Thefunctionsffkigwereobtainedaccordingtomaximumlikelihoodprinciple.Inparticularfortheithriskfactorwithvaluesintheinterval[l1i,l2i]wehave: fki(x)='(x)]TJ /F3 11.955 Tf 11.96 0 Td[(h,), 76

PAGE 77

wheref',h,g=argmax'2,,hLki(',,h),Lki(',,h)=Xj:Cj=kln'(xji)]TJ /F3 11.955 Tf 11.96 0 Td[(h,) F'(l2i,))]TJ /F3 11.955 Tf 11.95 0 Td[(F'(l1i,); here'2,whereisafollowingsetofcontinuousparametricprobabilitydensityfunctionsavailableinMATLAB:exponential,extremevalue,gamma,inverseGaussian,logistic,loglogistic,lognormal,Nakagami,normal,Rayleigh,Rician,Weibull;isavectorofparametersofaparticulardistribution;hisahorizontalshiftparameter;F'isacumulativedistributionfunctionthatcorrespondsto';l1iandl2iarethelowerandtheupperlimitsofvaluesoftheithriskfactor,respectively. Proceduresriskfactor.Thesurgerytypesinourdatasetweredescribedby4-digitICD9codes http://icd9cm.chrisendres.com/index.php?action=procslist .Theseprex-basedcodesprovidedetailedinformationabouttheproceduresfromoneside,andcreateaverylargenumberofdifferentproceduretypesfromanother.Someoftheseprocedureshadonlyafewpatientswhohadthattypeofsurgery.Estimationofprobabilitiesbycountingnumberofsuchpatientsineachclassisnotreliable.Thus,wecombinedprocedureswithsmallnumberofpatientsintogroupsofproceduresbasedontheirsimilarityaccordingtotheICD9classication.Inparticular,wecreatedatreewhereeachnodencorrespondstoacertaingroupoftheproceduresandisdescribedbyasequenceofdigitssn(lengthvariesfrom2to4);andeachsuccessorofagivennodehasacodegeneratedbyaddingtosnoneadditionaldigitfromtheright.Foreachleafnode,wehaveassignedanumberofpatientswhohadasurgerytypedescribedbythecodeofthisnode,andforeachnon-leafnode(suchnodesrepresentgeneralclassesofprocedures)wehaveassignedanumberofpatientswho'ssurgerytypebelongstothisclass.ProcedureswereaggregateduptothetopleveloftheICD9hierarchy(18basicproceduresclasses)suchthateachprocedure/groupofprocedurescontainedatleast 77

PAGE 78

Figure3-9. ProcedurestreebasedonICD9codes. 100patients.Weenumeratedobtainedsetofprocedures/groupsofproceduresandtheenumerationindexwastakenasadiscretefeatureinourmodel. Ageriskfactor.Thedistributionofpatientsinourdatasethasbimodalstructure Figure3-10. Agevaluesdistributionsfordifferentoutcomes.Thedistributionfunctionswerelearnedsequentially:rstwelearneddistributionfunction'1overthelargermode(dashedline).Wethenextrapolatedlearnedcurveoverthesmallermodeandlearneddistributionfunction'2overtheresidualdata.Theresultingdistributionfunctionwecomputedasaweightedmixture'1+(1)]TJ /F6 11.955 Tf 11.96 0 Td[()'2. 78

PAGE 79

(Figure 3-10 )withminimalvalueinthedatasetequalto17,likelyduetothepresenceofalargeuniversityinGainesville,FL.Hence,weestimatedtheprobabilitydensityfunctionsforageriskfactorintwosteps.First,welearnedprobabilitydensityfunctions'k1overthelargermode,containingallvalueshigherthan28(visualboundarybetweenthemodes)forpatientswithoutcomeC=k,k=f0,1g(dashedlinesonFigure 3-10 ).Second,weextrapolatedobtaineddensityfunctionsoverthesmallermodeandcomputedfollowingresidualhistograms:hk(x)=#fj:xjage=x,Cj=kg)]TJ -97.71 -30.68 Td[()]TJ /F6 11.955 Tf 11.96 0 Td[('k1(x)#fj:xjage>28,Cj=kg 1)]TJ /F3 11.955 Tf 11.95 0 Td[(Fk'1(28), wherex=17,...,28,k2f0,1gandFk'1isthecorrespondingcumulativedistributionfunction.Third,weestimatedtheprobabilitydensityfunctions'k2,k2f0,1gthatcorrespondtohk(x).Finally,overalldensityfunctionsfkiaregivenbyaweightedmixtureof'k1and'k2: fkage(x)=k'k1(x)+(1)]TJ /F6 11.955 Tf 11.95 0 Td[(k)'k2(x), wherek=k1 k1+k2,k1=#fj:xjage>28g 1)]TJ /F3 11.955 Tf 11.95 0 Td[(Fk'1(28),k2=P28x=17hk(x) Fk'2(28))]TJ /F3 11.955 Tf 11.96 0 Td[(Fk'2(17). sCrriskfactors.ThesCrmeasurementstimeseriesinourdatasetwassupplementedwithadateandtimewheneachmeasurementwastaken.IndifferentpatientssCrtimeserieshaddifferentlengthandsometimescontainedmissingvalues(timespanbetweentwoconsecutivemeasurementswasmorethanaday).Moreover,incertainpatientsmeasurementsweretakenseveraltimesaday,whileinsomepatientstheyweretaken 79

PAGE 80

onceinseveraldays.InordertoincorporatesCrdataintoourmodelwetransformedsCrtimeseriesintoasetofunivariateriskfactors.ForeachsCrtimeserieswedenedmaxCrasthemaximumsCrvalueinthetimeseries,minCrastheminimumsCrvalueinthetimeseriesandrecentCrasthemostrecentsCrvalueinthetimeseries. Weconsideredawiderangeofdifferentriskfactors,includingmaxCrandminCrvalues,linearcombinationsofsCrtimeseriesvalues,areaundersCrtimeseriescurve,etc.ThefactorsthatprovidedthehighestaccuracyintermsofAUCROCweretheratiosmaxCr/minCrandrecentCr/minCr.AddingothersCrriskfactorstothesetofgiventwofactorsdidnotimprovethemodelaccuracyintermsofAUCROC. Inaddition,weconsideredriskfactorsthatcharacterizethedurationofakidneyinjury.SimilartoRIFLEweselectedathresholdonsCrvaluesanddenedkidneyinjurywheneversCrvalueexceedsthisthreshold.Anumberofmeasurementsthatquantifythedurationofaninjurywereconsidered,buttheydidnotprovideanimprovementinmodelaccuracy,measuredbyAUCROC. Figure3-11. LearnedmixtureoftwoconditionaldistributionsformaxCr/minCrvalues. 80

PAGE 81

Figure3-12. LearnedmixtureoftwoconditionaldistributionsforrecentCr/minCrvalues. ThedistributionsofvaluesforthemaxCr/minCrandrecentCr/minCrfactorshadspecialnon-smoothform.ThereisasignicantnumberofpatientsinthedatasetwithastablelevelofsCrthroughouttheirstayinhospital,andthereforebothfactorsareequalto1forthesepatients.Asaresultsthedistributionsofvaluesofthefactorshavemodesat1. WecomputedfunctionsgformaxCr/minCrandrecentCr/minCrfactorsas: g(x)=8>><>>:ln#fj:xjsCr=1,Cj=1g #fj:Cj=1g#fj:Cj=0g #fj:xjsCr=1,Cj=0g,x=1lnf1sCr(x) f0sCr(x)#fj:xjsCr>1,Cj=1g #fj:Cj=1g#fj:Cj=0g #fj:xjsCr>1,Cj=0g,x>1 wherefksCr,k=f0,1gareparametriccontinuousprobabilitydensityfunctions(seeContinuousriskfactorsabove)learnedoverthesetofsCrvaluesthataregreaterthan1. 3.6.2.3Assessingthetofthemodel Accordingtoanumberofstudies,itisnotsufcienttocomputetheareaunderROCcurveformakingadecisionwhetherthemodeltsthedata(referenceshere).The 81

PAGE 82

areaunderROCcurveprovidesonlyinformationondiscriminativeabilityofthemodel,e.g.aprobabilitythatthemodelwillrankapatientfromtheclassC=1higher,thanapatientfromtheclassC=0,andtellsnothingaboutthemodelaccuracy(howaccuratelytherisksofmortalityareestimated).InordertoassesstheaccuracyofthemodelwehaveperformedaHosmer-Lemeshovtest.Toprovidefurtherinsightswecomparedpredictedrisksprovidedbyourmodelwiththerisksestimatedfromthedatainseparategroupsofpatients.Werankedallthepatientsaccordingtothemodelpredictedscore,andsplittheminto100percentiles,eachcontaining457patients.IneachpercentilewecomputedmeanpredictedscoreandanestimatedriskasaratioofnumberofpatientswithoutcomeC=1tothetotalnumberofpatientsinthepercentile.TheresultsforproposedmodelandforlogisticregressionarepresentedinFigure 3-13 Figure3-13. Comparisonbetweenpredictedscoreandriskestimatedfromeachpercentileofthedatasetinprobabilisticmodelandlogisticregression.Bluelinecorrespondstothescorepredictedbythemodel,redlinecorrespondstomortalityriskestimatedfromeachdatapercentile. 3.6.2.4Results CharacteristicsofPatients.Ourdatasetconsistsof60,074patientswhounderwentsurgeryinShandsHospitalfrom2000through2010.Eachpatientwascharacterizedby 82

PAGE 83

21riskfactorsapartfromsCrtimeseries.WedidnotconsiderpatientswitheGFRvaluebelow65,andpatientswhohadsurgerymorethan14daysaftertheadmissiondate.Inaddition,patientswithunlikelyextremechangesinsCrvalue,whichinalllikelihoodwerecausedbyamistakeduringthedatacollection,werelteredoutbasedontwocriteria:a)patientswiththreeconsequentsCrmeasurements,wherethethirdvaluedeviatefromtherstonenomorethanby20%andthesecondvalueiseithervetimesgreaterorsmallerthantherstone;b)patientswitheitherten-foldincreaseordecreaseintwoconsequentsCrmeasurements.Theresultingsetof45,722patientswasusedinouranalysis(Table 3-2 ).Wenote,thatinthedatasetsomepatientshavemultiplesCrmeasurementsperday,andsomepatientshavedayswithnosCrmeasurements. Fromthisinitialsetthefollowingfactorswereincludedintoourmodel:proceduretype,dayofsurgery,admissiontype,age,gender,race,attendingdoctorID,morbidityscore,sCrtimeseries. ModelFitandAccuracy.Weperformed100runsof70/30cross-validationanalysis.ThemodelaccuracywasassessedviaareaunderReceiverOperatorCharacteristic(ROC)curve.Meanvaluesforareaunderthecurve(AUC)togetherwith95%condenceintervalsarepresentedinTable 3-6 .Basedon100runs70/30crossvalidationobservations,sCrfactorsprovidedanimprovementtothemodelaccuracy(AUC0.918vs.AUC0.835;p<0.001).Inaddition,therecoverypatternrepresentedbyrecentCr/minCrfactorprovidedadditionalaccuracyimprovementwithrespecttothemaxCr/minCrfactorthatrepresentsseverityofAKI(AUC0.855vs.AUC0.834;p<0.001).Finally,inthemodeladjustedforpreoperativedata,addingrecentCr/minCrfactoralsoimprovedmodelaccuracywithrespecttomaxCr/minCrfactoronly(AUC0.918vs.AUC0.910;p<0.001). Importanceofriskfactors.Weperformedfeatureselectionprocedurewiththegreedybackwardeliminationmethod.Ateachstepweeliminatedafeatureresultinginthesmallestreductioninthemodelpredictiveperformance(measuredasanareaunder 83

PAGE 84

Table3-2. CharacteristicsofthepatientsinthedatasetwithRRTpatientsexcluded. In-hospitalmortality90-daysmortalityCharacteristicYes(N=726)No(N=35,961)Yes(N=1,350)No(N=35,337) Procedure:1454ICD9codesProceduredayMedian(25th,75th)1(0,3)0(0,1)1(0,3)0(0,1)LengthofStayMedian(25th,75th)12(6,22)6(4,10)10(6,19)6(4,10)Adm.type,n(%)routineelective159(22)20,857(58)418(31)20,848(59)emergency566(78)15,103(42)69931(69)14,488(41)AgeMedian(25th,75th)61(46,72)53(41,65)63(52,73)53(40,65)Gender(female),n(%)319(44)17,980(50)594(44)17,668(50)Race,n(%)White588(81)29,128(81)1,107(82)28,622(81)Black64(11)4,315(12)148(11)4,240(12)Hispanic17(3)1,078(3)27(2)1,060(3)Other12(2)720(2)28(2)706(2)Datamissing17(3)360(1)40(3)706(2)NumberofICUadmissionsMedian(25th,75th)1(1,1)0(0,1)1(0,1)0(0,1)NumberofavailableCrmeasurementsMedian(25th,75th)16(9,29)4(2,9)12(6,23)4(2,9)BasesCrvalueMedian(25th,75th)0.74(0.6,0.9)0.76(0.6,0.9)0.74(0.61,0.89)0.76(0.63,0.90)AKIRIFLE,n(%)stage0138(19)26,970(75)526(39)26,502(75)stage1210(29)6,113(17)364(27)6,007(17)stage2188(26)2,157(6)256(19)2,120(6)stage3188(26)719(2)202(15)706(2)eGFR epiMedian(25th,75th)93.4(83.1,108.3)98.0(86.4,112.0)93.5(83.3,107.0)98.6(86.4,112.0)Admittingdoctor:505doctorIDsAttendingdoctor:463doctorIDsCCI(morbidityscore)Median(25th,75th)2(1,3)1(0,2)2(1,5)1(0,2)RRT(dialysis),n(%)175(24)136(0.4)163(12)148(0.4)MultiplesCrvaluesperday,n(%)668(92)12960(36)1240(92)12388(35)Avrg.#ofsCrvaluesperdayMedian(25th,75th)1.4(1.1,1.9)0.8(0.5,1.1)1.2(0.9,1.5)0.8(0.5,1.1) theROCcurvewith70/30crossvalidation).Removingdayofsurgery,genderandracecausedlessthan0.05%reductioninmodelaccuracy(evaluatedwithAUCROC),whereasremovinganyotherfeatureresultedinmorethan0.1%decreaseofaccuracy.Therefore,weeliminateddayofsurgery,genderandracefromthenalsetofriskfactors.Thenalsetofriskfactorsconsistedofadmissiontype(emergencyorroutineelective),proceduretype,age,morbidityscore,operatingdoctorIDandsCrtimeseries. AssociationbetweensCrpatternsandmortality.WehaveestablishedaquantitativedependencebetweenthepatternsofsCrandin-hospitalmortality.InordertoincorporatesCrtimeseriesintoourmodel,weconsideredarangeofunivariateriskfactorsderivedfromthetimeseries(seeAppendixfordetails).ForeachsCrtimeserieswedened 84

PAGE 85

Table3-3. Discriminativeability(AUCROC)with95%predictiveintervalsandgoodnessoft(Hosmer-Lemeshowstatistic)ofthemodels.TheHosmer-Lemeshowtest[ 42 ]wasperformedwith100groups. RiskfactorsincludedProb.modelAUC(95%CI) Preop.0.835(0.817-0.854)Hosmer-Lemeshowstatisticss=95.06;p=0.57Preop.+maxCr/minCr0.900(0.888-0.911)Hosmer-Lemeshowstatisticss=84.33;p=0.84Preop.+recentCr/minCr0.910(0.897-0.923)Hosmer-Lemeshowstatisticss=72.19;p=0.98Preop.+bothsCrfactors0.917(0.906-0.929)Hosmer-Lemeshowstatisticss=68.94;p=0.99maxCr/minCronly0.834(0.816-0.852)Hosmer-Lemeshowstatisticss=1,818;p=0recentCr/minCronly0.791(0.764-0.818)Hosmer-Lemeshowstatisticss=112.88;p=0.14bothsCrfactors0.855(0.837-0.873)Hosmer-Lemeshowstatisticss=137.31;p=0.01 maxCrasthemaximumsCrvalueinthetimeseries,minCrastheminimumsCrvalueinthetimeseriesandrecentCrasthemostrecentsCrvalueinthetimeseries.TworiskfactorsprovidinghighestmodelaccuracywereselectedtorepresentthesCrtimeseriesinanalmodel.TheseweretheriskfactorsrepresentingseverityofAKI(computedasmaxCr/minCr)andsubsequentrecovery(computedasrecentCr/minCr).Inordertoillustratethedegreetowhicheachfactorisassociatedwithmortality,wecomputedoddsratiosasfunctionsofthevaluesoftheriskfactors.Theoddsratioof1correspondstoapriorioddsinthedataset(907(#deceased)/44815(#discharged)0.02).ForarangeofvaluesoftheselectedsCrriskfactorstheoddsratiosarepresentedinFigure 3-14 AsetofspecicsCrpatternsispresentedinFigure 3-20 .Numbersovereachpatternpresentoddsratioslearnedfromthecompletedatasetand95%condenceintervalsassessedwith100runsofa70/30crossvalidationprocedure. 85

PAGE 86

Figure3-14. Thelogoddsratiosforin-hospitalmortalitybasedonsCrfactors.A)Thelogoddsratiosforin-hospitalmortalitybasedonsCrfactors;theratiosarecomputedwithrespecttoana-priorioddsinthedataset(0.02).B)DistributionofpatientswithrespecttothevaluesofsCrfactors. Figure3-15. OddsratiosforparticularsCrchangepatternscomputedwithrespecttoanapriorioddsinthedataset(0.02). 3.6.3GeneralizedAdditiveModels 3.6.3.1Probabilisticmodel Weestimatedprobabilityofmortality(C=1,otherwiseC=0)byusingGeneralizedAdditiveModel[ 37 ]: 86

PAGE 87

logitP(C=1jX=x)=+mXi=1fi(xi),(3) wheremisthenumberofriskfactors,X=(X1,...,Xm)aretheriskfactors,x=(x1,...,xm)arethevaluesofthesefactors,fiisanonlinearriskfunctionassociatedwiththeithriskfactorandisafreeterm. Givenvaluesoftheriskfactors,( 3 )providesaprobabilisticscoreinthe[0,1]interval,withahighscoreindicatingahighriskofmortality.AllcomputationsweredoneinRsoftwareusingthemgcvpackage[ 83 ]. 3.6.3.2Riskfactorsanalysis NonlinearRiskFunctions.Nonlinearriskfunctionsfiwereestimatedwithcubicsplines([ 37 ],p.140).Thedegreesoffreedomforeachsplinewereestimatedbymaximizingrestrictedlikelihoodfunction[ 83 ].Degreesoffreedomcharacterizeacurvatureofaspline,withvalue1correspondingtoalinearfunction([ 36 ],p.153).Riskfactorswithestimateddegreesoffreedomcloseto1(seeTable 3-4 )werenotsmoothedinanalmodel,insteadtheoriginalvaluesofriskfactorsxiwereused.Therefore,thenalmodelhasthefollowingform: logitP(C=1jX=x)=+Xi2Iwixi+Xi62Ifi(xi),(3) whereIisasetofriskfactorswithestimateddegreesoffreedomcloseto1andwiisalinearweightoftheithriskfactor. Categoricalriskfactors.InordertoobtainareliableestimateofP(Xi=xjC=c),riskfactorcategorieswithfewerthan100recordsweregroupedtogetherandlabeledother.Thisothergroupwasfurthersplitintoseveralsubgroupswhereeachsubgroupcontainedcategorieswithsimilarproportionsofpatientsformdifferentclasses.Thiswasachievedbyperformingk-meansclustering([ 14 ],p.424)onthesetofcategoriesinothergroup.Wesetthenumberofclustersto5.ThecomputationswereimplementedinMATLABsoftwareusingbuilt-inkmeansfunction. 87

PAGE 88

Procedureriskfactor.Procedureriskfactorwasanalyzedasdescribedintheprevioussubsection. sCrriskfactors.ThesCrmeasurementstimeseriesinourdatasetwassupplementedwithadateandtimewheneachmeasurementwastaken.IndifferentpatientssCrtimeserieshaddifferentlengthandsometimescontainedmissingvalues(timespanbetweentwoconsecutivemeasurementswasmorethanaday).Moreover,incertainpatientsmeasurementsweretakenseveraltimesaday,whileinsomepatientstheyweretakenonceinseveraldays.TheprecisionofthesCrvaluesinourdatasetwas0.1.Thisarticialdiscretizationresultedinhighlynon-smoothvaluesofsCrfactors,especiallyfactorscomputedasratiosbetweensCrvalues.Inordertoreducethiseffectweperformedstatisticalsmoothingsimilartoadditivesmoothing[ 52 ]priortofurtheranalysis,replacingeachsCrvaluexbyauniformrandomvalueinarange[x-0.05,x+0.05]. 3.6.3.3Results ModelSelection.Thediscussedabovelistofriskfactorswasinitiallyaddedtoeachoftheconsideredfourmodels.WethenperformedfeatureselectionusingBackward-StepwiseSelectionmethodbasedonAICcriteria([ 36 ],p.231).Theresultingsetsoffactorsforeachmodeltogetherwithestimateddegreesoffreedom(seeNonlin-earRiskFunctionsfordetails)arereportedinTable 3-4 Tocomparedifferentmodels,weperformedVuoung'stest[ 78 ].ThecomputationswereperformedinRsoftwareusingthepsclpackage[ 44 ].TheresultsofthetestarepresentedinTable 3-5 Asexpected,allthreemodelswithsCrriskfactorsexhibitedsimilarperformance.ThisisduetothefactthatsCrvariablessetsusedinthemodelscontainthesameamountofinformation(fullydescribedbymaxCr,lastCrandbaseCr).Forpredicting90-daysmortality,thedifferencebetweenmodelsisnotstatisticallysignicant,howeverModel3doesnotincludebaseCrfactorinthiscase(Table 3-4 ),whichmakesitsimpler 88

PAGE 89

Table3-4. Featureselectionanddegreesoffreedom.Adegreeoffreedomequalto1indicatesthatalinearfunctionofthecorrespondingvariablewasusedinthemodel.Thesymbolindicatesthatthefactorwasdiscardedduringthefeatureselectionprocedure. Model0Model1Model2Model3FeaturesIn-hospital90-daysIn-hospital90-daysIn-hospital90-daysIn-hospital90-days CategoricalProcedure5.63.56.24.26.24.16.24.1Admissiontype11111111Gender1111111Race1111111Attendingdoctor5.44.45.13.75.03.75.03.8ICUadmission11111111Morbidity1.28.81.98.81.98.81.68.8DayofsurgeryRRTContinuousAge4.04.04.04.04.04.04.04.0baseCr2.712.02.01maxCr5.83.5lastCr5.65.7maxCr-baseCr3.93.2lastCr-baseCr6.56.4maxCr/baseCr4.03.5lastCr/baseCr6.16.6 Table3-5. Vuong'stestformodelcomparison.Thevalueintheintersectionofi-throwandj-thcolumncontainsresultsoftestonthe(modeli,modelj)pair.Negativevalueindicatesthatmodeliprovidesbettertthanmodelj.Thep-valueindicatesstatisticalsignicanceofthedifferencebetweenmodels,withsmallvaluescorrespondingtoasignicantdifference. In-hospital/90-daysmortality Model1Model2Model3Model0-10.69(p<0.001)/-9.11(p<0.001)-9.32(p<0.001)/-9.1(p<0.001)-10.27(p<0.001)/-9.13(p<0.001)Model10.75(p=0.22)/-1.34(p=0.09)2.01(p=0.02)/-0.15(p=0.44)Model22.11(p=0.02)/1.37(p=0.09) andhencemoreattractive.TotestthesignicanceofbaseCrfactorinModel1andModel2forpredicting90-daysmortality,wecomputedthedifferencebetween10thand90thpercentilesofbaseCrriskfunction.Thisistherangecontaining80%ofthepatientsexcludingextremecases.ForModel1thisdifferencewas0.22,whichcorrespondsto1.25changeintheoddsratio;forModel2thedifferencewas0.81,whichcorrespondsto2.23changeintheoddsratio. Inordertoevaluatethedifferencesbetweenmodels,wecompared95%predictiveintervalsforthescoresproducedbythreemodelsforeachindividualpatient.Wecountednumbersofpatientsforwhichtheseintervalsdonotoverlap.Outoftotal36,687patientstherewere11patientswithnooverlapwhencomparingModel1andModel2, 89

PAGE 90

Table3-6. Discriminativeability(AUCROC)with95%predictiveintervalsandgoodnessoft(Hosmer-Lemeshowstatistic)ofthemodels.TheHosmer-Lemeshowtest[ 42 ]wasperformedwith100groups. AUCROC(95%PI)Hosmer-Lem.testTrainingTestingStatisticp-value Model0In-h.mortality0.893(0.886,0.901)0.850(0.826,0.874)101.540.3890-daysmortality0.857(0.849,0.865)0.842(0.826,0.858)125.070.03Model1In-h.mortality0.945(0.941,0.949)0.925(0.911,0.938)76.960.9490-daysmortality0.872(0.864,0.880)0.858(0.843,0.874)118.580.08Model2In-h.mortality0.943(0.939,0.948)0.923(0.911,0.937)88.240.7590-daysmortality0.872(0.865,0.880)0.859(0.843,0.875)110.860.18Model3In-h.mortality0.944(0.940,0.948)0.924(0.911,0.937)63.080.9990-daysmortality0.872(0.864,0.880)0.859(0.843,0.875)109.950.19 159patientswithnooverlapwhencomparingModel1andModel3and14patientswithnooverlapwhencomparingModel2andModel3.Theseresultsagainsuggestthatallthreemodelsproducesimilarscores. ModelFitandAccuracy.Foreachofthemodelsaboveweperformed100iterationsof70/30crossvalidationprocedure.Ateachiterationthedatasetwasrandomlysplitinto70%trainingand30%testingpartsandthemodelswerelearnedusingtrainingpartonly.Separatemodelswerelearnedtopredictin-hospitalmortalityand90-daysmortality.ForeachofthemodelswecomputedtheareaunderROCcurveontrainingandtestingparts.Also,foreachmodelweperformedHosmer-Lemeshowgoodness-of-ttest[ 42 ].TheresultsarepresentedinTable 3-6 Basedon100runsof70/30crossvalidationobservations,sCrfactorsprovidedanimprovementtothemodelaccuracy(AUC0.922vs.AUC0.850;p<0.001).ThissupportstheresultsobtainedwithVuong'stest(Table 3-5 ). AssociationbetweensCrpatternsandmortality.TheriskfactorsdescribingsCrtimeseriesthatweconsideredrepresentthefollowingthreeproperties:severityofAKI(e.g.maxCr/baseCr,maxCr-baseCr),recovery(e.g.lastCr/baseCr,lastCr-baseCr)andbasesCrlevel(baseCr).Inordertoillustratethedegreetowhicheachfactoris 90

PAGE 91

associatedwithmortality,wecomputedoddsratiosasfunctionsofthevaluesoftheriskfactors.Theoddsratioof1correspondstoapatientwithnochangeinsCr.ForarangeofvaluesoftheselectedsCrriskfactorstheoddsratiosforin-hopditaland90-daysmortalityarepresentedonFigures 3-16 and 3-17 respectively.ThenonlinearfunctionsflearnedfromthedataareshownonFigures 3-18 and 3-19 Figure3-16. NonlinearfunctionsforsCrfactorspredictingin-hospitalmortality.A)ValuesofnonlinearfunctionsfiforsCrfactorsforModel3within-hospitalmortalitytakenastheoutcome.Highvaluescontributetoahighoverallriskofmortality.Theshadedareasrepresent95%predictiveintervalsforthefunctionvalues.B)Thelogoddsratiosforin-hospitalmortalitybasedonsCrfactorsforModel3;theratiosarecomputedwithrespecttoapatternwithnochangeinsCr.C)DistributionofpatientsbasedonthevaluesofsCrfactorsusedinModel3. AsetofspecicsCrpatternsispresentedinFigure 3-20 .Numbersovereachpatternpresentoddsratioslearnedfromthecompletedatasetand95%predictiveintervalsassessedwith100runsofa70/30crossvalidationprocedure. 91

PAGE 92

Figure3-17. NonlinearfunctionsforsCrfactorspredicting90-daysmortality.A)ValuesofnonlinearfunctionsfiforsCrfactorsforModel3with90-daysmortalitytakenastheoutcome.Highvaluescontributetoahighoverallriskofmortality.Theshadedareasrepresent95%predictiveintervalsforthefunctionvalues.B)Thelogoddsratiosforin-hospitalmortalitybasedonsCrfactorsforModel3;theratiosarecomputedwithrespecttoapatternwithnochangeinsCr.C)DistributionofpatientsbasedonthevaluesofsCrfactorsusedinModel3. 92

PAGE 93

Figure3-18. NonlinearfunctionsforModel3within-hospitalmortalitytakenasanoutcome.Theshadedarearepresents95%predictiveintervalforthefunctionsvalues. Figure3-19. NonlinearfunctionsforModel3with90-daysmortalitytakenasanoutcome.Theshadedarearepresents95%predictiveintervalforthefunctionsvalues. 93

PAGE 94

Figure3-20. Oddsratioswith95%predictiveintervalsforparticularsCrchangepatternsforModel3computedwithrespecttoapatternwithnochangeinsCr.In-hospitalmortalityistakenastheoutcome. Figure3-21. Oddsratioswith95%predictiveintervalsforparticularsCrchangepatternsforModel3computedwithrespecttoapatternwithnochangeinsCr.90-daysmortalityistakenastheoutcome. 94

PAGE 95

CHAPTER4CONCLUSIONS Inthiswork,weexploreddataminingalgorithmsandtheirusetosolvepracticalproblemsfrombiomedicaldomain.Weprovidedanoverviewtosomeoftheexistingtechniquesandemployedseveralalgorithmstosolveproblemsofhumanbrainrecordingsdataanalysisandpredictingclinicaloutcomes. Intherstpartofourworkweconstructedafunctionalnetworkmodelofahumanbraininordertoreducethedimensionalityofthedataandtoextractrelevantinformationofinterest.Graphtheoreticalanalysisprovidesamethodoflookingathumanbraintimeseriesdata,usingasystemsapproachtoextendandexpandmorereductionistapproachesthatfocusonregionsofinterest,speciccircuits,orbehaviorallyconstrainedrepetitiveresponsestostimuli[ 28 ].Graphmetricanalysisintherestingstateshowsrepeatabilityandreliabilityintest-retestanalysis.Graphanalysiscanuseanumberofmathematicaltoolstoconstructthewholebraingraph;oneapproachistousewaveletanalysistechniques[ 18 ].Inthiswork,weshowedthatgraphanalytictechniquesandwaveletanalysiscanbeusedtodetectgroupdifferencesinnetworkefciencybetweenindividualswithadiseasestateandindividualswithoutthediseasestate.Becauseourtechniquedevelopsmetrics(suchasglobalandlocalornodalefciencies)fromtheentirebrain,thetechniquemightbeamenablewithfurtherdevelopmentfordiagnosticpurposes.Inaddition,ourresultssuggestthathumanbrainisorganizedinamannerinwhichitgiveshigherprioritytotheresilienceofthenetworkwithrespecttotheefciencyofthenetwork.Ourndingsmayrelatetohowwemeasuredbrainfunction.Inoursubjects,wemeasureddynamicnetworkpropertiesintherestingstate.Subjectswereatrest,awake,witheyesclosed,andinstructedtolettheirmindswander.Itispossiblethatduringdirectedactivitythatnetworkefciencywouldimproverelatedtothenecessitytofocusonaparticulartask.Networkclustering,efciency,and 95

PAGE 96

pathlengthmaythereforeshiftdynamicallydependentontask,andfurtherresearchwouldbeneededtoexplorethishypothesis. Inthesecondpartofourworkweemployedtwoprobabilisticclassicationalgorithmsinordertosolveaproblemofpredictingmortalityinpost-operativepatients.Theresultsprovidedbybothalgorithmsshowthatserumcreatininelevelinthebloodprovidesrelevantcomplimentaryinformationonmortalityriskinpostoperativepatients.Inparticular,addingsCrriskfactorstothepreoperativeriskfactorssignicantlyincreasesmodelsaccuracyanddiscriminativeability.Meta-featuresderivedfromthesCrtimeseriessuggestthattheimpactonmortalityriskislargelydependentonseverityofpatient'skidneycondition(reectedbythemaximumsCrvalue),andconsequentrecovery(reectedbythemostrecentsCrvalue).Basedonthesemeta-features,aquantitativerelationshipbetweensCrtimeseriesandmortalityriskwasestablished.FromthecomputationalresultsonecanseethatGAMswasabletoobtainhigheraccuracyanddiscriminationabilitythanprobabilisticmodelbasedonlogisticfunction.Inparticular,GeneralizedAdditiveModelwasabletocapturethedependenciesbetweensCrmeta-featuresinthedata,asreectedonFigures 3-16 3-17 ,unlikethemodelbasedonlogisticfunction.ThisisduetothefactthatthenonlinearfunctionsinGAMswerelearnedsimultaneouslywithbackttingalgorithm,whileinthelogisticfunctionbasedmodelriskfunctionswerelearnedindependentlyfromeachother.ThemodelperformancewasassessedbyevaluatingmodeldiscriminativeabilitywithROCcurveandmodelaccuracywithHosmer-Lemeshowtest.Carefulpreprocessingandanalysisofthedataprovidedhighaccuracyofthemodel.Theeffectofover-ttingwasreducedbycontrollingtheparametersofthelearnedsplinesinGAMsandgroupingofcategoricalvariables.Thefeatureselectionprocedurewasperformedbasedonlikelihoodratiocriteria.Theprocedureallowedtoidentifyfeaturesthataremostrelevanttotheoutcomeofinterest. 96

PAGE 97

REFERENCES [1] Achard,S.,&Bullmore,E.(2007).Efciencyandcostofeconomicalbrainfunctionalnetworks.PLoScomputationalbiology,3(2). [2] Achard,S.,Salvador,R.,Whitcher,B.,Suckling,J.,&Bullmore,E.(2006).Aresilient,low-frequency,small-worldhumanbrainfunctionalnetworkwithhighlyconnectedassociationcorticalhubs.JournalofNeuroscience,26(1),63. [3] Agresti,A.(2002).Categoricaldataanalysis,vol.359.Wiley-interscience. [4] Albert,R.,&Barabasi,A.(2002).Statisticalmechanicsofcomplexnetworks.Reviewsofmodernphysics,74(1),47. [5] Amaral,L.,&Ottino,J.(2004).Complexnetworks:Augmentingtheframeworkforthestudyofcomplexsystems.TheEuropeanPhysicalJournalB-CondensedMatter,38(2),147. [6] Barabasi,A.,&Oltvai,Z.(2004).Networkbiology:understandingthecell'sfunctionalorganization.NatureReviewsGenetics,5(2),101. [7] Barahona,M.,&Pecora,L.(2002).Synchronizationinsmall-worldsystems.PhysicalReviewLetters,89(5),54101. [8] Barmpoutis,D.,&Murray,R.(2010).Networkswiththesmallestaveragedistanceandthelargestaverageclustering.ArxivpreprintarXiv:1007.4031. [9] Barmpoutis,D.,&Murray,R.(2011).Extremalpropertiesofcomplexnetworks.ArxivpreprintarXiv:1104.5532. [10] Bassett,D.,&Bullmore,E.(2006).Small-worldbrainnetworks.Theneuroscientist,12(6),512. [11] Beran,J.(1994).Statisticsforlong-memoryprocesses,vol.61.Chapman&Hall/CRC. [12] Bihorac,A.,Delano,M.,Schold,J.,Lopez,M.,Nathens,A.,Maier,R.,Layon,A.,Baker,H.,&Moldawer,L.(2010).Incidence,clinicalpredictors,genomics,andoutcomeofacutekidneyinjuryamongtraumapatients.Annalsofsurgery,252(1),158. [13] Bihorac,A.,Yavas,S.,Subbiah,S.,Hobson,C.,Schold,J.,Gabrielli,A.,Layon,A.,&Segal,M.(2009).Long-termriskofmortalityandacutekidneyinjuryduringhospitalizationaftermajorsurgery.Annalsofsurgery,249(5),851. [14] Bishop,C.,etal.(2006).Patternrecognitionandmachinelearning,vol.4.springerNewYork. 97

PAGE 98

[15] Biswal,B.,Yetkin,F.,Haughton,V.,&Hyde,J.(1995).Functionalconnectivityinthemotorcortexofrestinghumanbrainusingecho-planarMRI.MagneticResonanceinMedicine,34(4),537. [16] Borner,K.,Sanyal,S.,&Vespignani,A.(2007).Networkscience.Annualreviewofinformationscienceandtechnology,41(1),537. [17] Bressler,S.(1995).Large-scalecorticalnetworksandcognition.BrainResearchReviews,20(3),288. [18] Bullmore,E.,Fadili,J.,Breakspear,M.,Salvador,R.,Suckling,J.,&Brammer,M.(2003).Waveletsandstatisticalanalysisoffunctionalmagneticresonanceimagesofthehumanbrain.Statisticalmethodsinmedicalresearch,12(5),375. [19] Bullmore,E.,Fadili,J.,Maxim,V.,Sendur,L.,Whitcher,B.,Suckling,J.,Brammer,M.,&Breakspear,M.(2004).Waveletsandfunctionalmagneticresonanceimagingofthehumanbrain.NeuroImage,23,S234S249. [20] Bullmore,E.,&Sporns,O.(2009).Complexbrainnetworks:graphtheoreticalanalysisofstructuralandfunctionalsystems.NatureReviewsNeuroscience,10(3),186. [21] Buzsaki,G.(2006).RhythmsoftheBrain.OxfordUniversityPress,USA. [22] Buzsaki,G.,Geisler,C.,Henze,D.,&Wang,X.(2004).Interneurondiversityseries:circuitcomplexityandaxonwiringeconomyofcorticalinterneurons.TRENDSinNeurosciences,27(4),186. [23] Cherniak,C.(1994).Componentplacementoptimizationinthebrain.JournalofNeuroscience,14(4),2418. [24] Chklovskii,D.,Schikorski,T.,&Stevens,C.(2002).Wiringoptimizationincorticalcircuits.Neuron,34(3),341. [25] Cordes,D.,Haughton,V.,Arfanakis,K.,Wendt,G.,Turski,P.,Moritz,C.,Quigley,M.,&Meyerand,M.(2000).MappingfunctionallyrelatedregionsofbrainwithfunctionalconnectivityMRimaging.AmericanJournalofNeuroradiology,21(9),1636. [26] Deco,G.,Jirsa,V.,&McIntosh,A.(2011).Emergingconceptsforthedynamicalorganizationofresting-stateactivityinthebrain.NatureReviewsNeuroscince. [27] DeLong,E.R.,DeLong,D.M.,&Clarke-Pearson,D.L.(1988).Comparingtheareasundertwoormorecorrelatedreceiveroperatingcharacteristiccurves:anonparametricapproach.Biometrics,(pp.837). [28] Deuker,L.,Bullmore,E.T.,Smith,M.,Christensen,S.,Nathan,P.J.,Rockstroh,B.,Bassett,D.S.,etal.(2009).Reproducibilityofgraphmetricsofhumanbrainfunctionalnetworks.Neuroimage,47(4),1460. 98

PAGE 99

[29] Eguiluz,V.,Chialvo,D.,Cecchi,G.,Baliki,M.,&Apkarian,A.(2005).Scale-freebrainfunctionalnetworks.Physicalreviewletters,94(1),18102. [30] Finger,S.(2001).Originsofneuroscience:Ahistoryofexplorationsintobrainfunction.OxfordUniversityPress,USA. [31] Fox,M.,&Raichle,M.(2007).Spontaneousuctuationsinbrainactivityobservedwithfunctionalmagneticresonanceimaging.NatureReviewsNeuroscience,8(9),700. [32] Fries,P.(2005).Amechanismforcognitivedynamics:neuronalcommunicationthroughneuronalcoherence.TrendsinCognitiveSciences,9(10),474. [33] Grandvalet,Y.,Mariethoz,J.,&Bengio,S.(2005).Aprobabilisticinterpretationofsvmswithanapplicationtounbalancedclassication. [34] Hanley,J.,&McNeil,B.(1982).Themeaninganduseoftheareaunderareceiveroperatingcharacteristic(roc)curve.Radiology,143:1,29. [35] Hanley,J.A.,McNeil,B.J.,etal.(1983).Amethodofcomparingtheareasunderreceiveroperatingcharacteristiccurvesderivedfromthesamecases.Radiology,148(3),839. [36] Hastie,T.,Tibshirani,R.,Friedman,J.,&Franklin,J.(2005).Theelementsofstatisticallearning:datamining,inferenceandprediction.TheMathematicalIntelligencer,27(2),83. [37] Hastie,T.J.,&Tibshirani,R.J.(1990).Generalizedadditivemodels,vol.43.Chapman&Hall/CRC. [38] Hilgetag,C.,Burns,G.,O'Neill,M.,Scannell,J.,&Young,M.(2000).Anatomicalconnectivitydenestheorganizationofclustersofcorticalareasinthemacaquemonkeyandthecat.PhilosophicalTransactionsoftheRoyalSocietyB:BiologicalSciences,355(1393),91. [39] Hobson,C.,Yavas,S.,Segal,M.,Schold,J.,Tribble,C.,Layon,A.,&Bihorac,A.(2009).Acutekidneyinjuryisassociatedwithincreasedlong-termmortalityaftercardiothoracicsurgery.Circulation,119(18),2444. [40] Honey,C.,Kotter,R.,Breakspear,M.,&Sporns,O.(2007).Networkstructureofcerebralcortexshapesfunctionalconnectivityonmultipletimescales.ProceedingsoftheNationalAcademyofSciences,104(24),10240. [41] Hong,H.,Choi,M.,&Kim,B.(2002).Synchronizationonsmall-worldnetworks.PhysicalReviewE,65(2),26139. [42] Hosmer,D.,&Lemesbow,S.(1980).Goodnessofttestsforthemultiplelogisticregressionmodel.CommunicationsinStatistics-TheoryandMethods,9(10),1043. 99

PAGE 100

[43] Hunink,M.M.,Glasziou,P.P.,Siegel,J.E.,Weeks,J.C.,Pliskin,J.S.,Elstein,A.S.,&Weinstein,M.C.(2001).Decisionmakinginhealthandmedicine:integrat-ingevidenceandvalues.CambridgeUniversityPress. [44] Jackman,S.(2012).pscl:ClassesandMethodsforRDevelopedinthePoliticalScienceComputationalLaboratory,StanfordUniversity.DepartmentofPoliticalScience,StanfordUniversity,Stanford,California.Rpackageversion1.04.4. URL http://pscl.stanford.edu/ [45] Klyachko,V.,&Stevens,C.(2003).Connectivityoptimizationandthepositioningofcorticalareas.ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,100(13),7937. [46] Lago-Fernandez,L.,Huerta,R.,Corbacho,F.,&Siguenza,J.(2000).Fastresponseandtemporalcoherentoscillationsinsmall-worldnetworks.PhysicalReviewLetters,84(12),2758. [47] Latora,V.,&Marchiori,M.(2001).Efcientbehaviorofsmall-worldnetworks.PhysicalReviewLetters,87(19),198701. [48] Latora,V.,&Marchiori,M.(2003).Economicsmall-worldbehaviorinweightednetworks.TheEuropeanPhysicalJournalB,32(2),249. [49] Localio,A.,&Goodman,S.(2012).Beyondtheusualpredictionaccuracymetrics:reportingresultsforclinicaldecisionmaking.Annalsofinternalmedicine,157(4),294. [50] Lu,J.,Yu,X.,Chen,G.,&Cheng,D.(2004).Characterizingthesynchronizabilityofsmall-worlddynamicalnetworks.CircuitsandSystemsI:RegularPapers,IEEETransactionson,51(4),787. [51] Madevska-Bogdanova,A.,Nikolik,D.,&Curfs,L.(2004).Probabilisticsvmoutputsforpatternrecognitionusinganalyticalgeometry.Neurocomputing,62,293. [52] Manning,C.D.,Raghavan,P.,&Schutze,H.(2008).Introductiontoinformationretrieval,vol.1.CambridgeUniversityPressCambridge. [53] Maxim,V.,Sendur,L.,Fadili,J.,Suckling,J.,Gould,R.,Howard,R.,&Bullmore,E.(2005).Fractionalgaussiannoise,functionalmriandalzheimer'sdisease.Neuroimage,25(1),141. [54] McIntosh,A.(2000).Towardsanetworktheoryofcognition.NeuralNetworks,13(8-9),861. [55] Milgram,S.(1967).Thesmallworldproblem.Psychologytoday,2(1),60. [56] Murre,J.,&Sturdy,D.(1995).Theconnectivityofthebrain:Multi-levelquantitativeanalysis.BiologicalCybernetics,73(6),529. 100

PAGE 101

[57] Percival,D.,&Walden,A.(2006).Waveletmethodsfortimeseriesanalysis.CambridgeUnivPr. [58] Platt,J.,etal.(1999).Probabilisticoutputsforsupportvectormachinesandcomparisonstoregularizedlikelihoodmethods.Advancesinlargemarginclassi-ers,10(3),61. [59] Raichle,M.(2010).Thebrain'sdarkenergy.ScienticAmericanMagazine,302(3),44. [60] Raichle,M.,MacLeod,A.,Snyder,A.,Powers,W.,Gusnard,D.,&Shulman,G.(2001).Adefaultmodeofbrainfunction.ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica,98(2),676. [61] Salvador,R.,Suckling,J.,Coleman,M.,Pickard,J.,Menon,D.,&Bullmore,E.(2005).Neurophysiologicalarchitectureoffunctionalmagneticresonanceimagesofhumanbrain.CerebralCortex,15(9),1332. [62] Saria,S.,Rajani,A.,Gould,J.,Koller,D.,&Penn,A.(2010).Integrationofearlyphysiologicalresponsespredictslaterillnessseverityinpreterminfants.ScienceTranslationalMedicine,2(48),48ra65ra65. [63] Singer,W.(1995).Developmentandplasticityofcorticalprocessingarchitectures.Science,270(5237),758. [64] Singer,W.(1999).NeuronalSynchrony:AVersatileCodeReviewfortheDenitionofRelations?Neuron,24,49. [65] Sporns,O.(2002).Networkanalysis,complexity,andbrainfunction.Complexity,8(1),56. [66] Sporns,O.(2010).NetworksoftheBrain.TheMITPress. [67] Sporns,O.,Tononi,G.,&Edelman,G.(2000).Theoreticalneuroanatomy:relatinganatomicalandfunctionalconnectivityingraphsandcorticalconnectionmatrices.CerebralCortex,10(2),127. [68] Stam,C.(2004).Functionalconnectivitypatternsofhumanmagnetoencephalographicrecordings:a[]small-world'network?Neuroscienceletters,355(1-2),25. [69] Stam,C.,&Reijneveld,J.(2007).Graphtheoreticalanalysisofcomplexnetworksinthebrain.NonlinearBiomedicalPhysics,1(1),3. [70] Stephan,K.,Hilgetag,C.,Burns,G.,O'Neill,M.,Young,M.,&Kotter,R.(2000).Computationalanalysisoffunctionalconnectivitybetweenareasofprimatecerebralcortex.PhilosophicalTransactionsoftheRoyalSocietyofLondon.SeriesB:BiologicalSciences,355(1393),111. 101

PAGE 102

[71] Strogatz,S.(2001).Exploringcomplexnetworks.Nature,410(6825),268. [72] Swanson,L.(2003).Brainarchitecture:Understandingthebasicplan.OxfordUniversityPress,USA. [73] Tzourio-Mazoyer,N.,Landeau,B.,Papathanassiou,D.,Crivello,F.,Etard,O.,Delcroix,N.,Mazoyer,B.,&Joliot,M.(2002).AutomatedanatomicallabelingofactivationsinSPMusingamacroscopicanatomicalparcellationoftheMNIMRIsingle-subjectbrain.Neuroimage,15(1),273. [74] Vapnik,V.(1998).Statisticallearningtheory.Wiley. [75] Vapnik,V.(1999).Thenatureofstatisticallearningtheory.Springer. [76] Varela,F.,Lachaux,J.,Rodriguez,E.,&Martinerie,J.(2001).Thebrainweb:phasesynchronizationandlarge-scaleintegration.Naturereviewsneuroscience,2(4),229. [77] Vickers,A.J.,&Elkin,E.B.(2006).Decisioncurveanalysis:anovelmethodforevaluatingpredictionmodels.MedicalDecisionMaking,26(6),565. [78] Vuong,Q.(1989).Likelihoodratiotestsformodelselectionandnon-nestedhypotheses.Econometrica:JournaloftheEconometricSociety,(pp.307). [79] Wahba,G.,etal.(1999).Supportvectormachines,reproducingkernelhilbertspacesandtherandomizedgacv.AdvancesinKernelMethods-SupportVectorLearning,6,69. [80] Ward,B.D.(2002).Deconvolutionanalysisoffmritimeseriesdata.Milwaukee,WI:BiophysicsResearchInstitute,MedicalCollegeofWisconsin. [81] Watts,D.,&Strogatz,S.(1998).Collectivedynamicsofsmall-worldnetworks.Nature,393(6684),440. [82] Wedderburn,R.(1976).Ontheexistenceanduniquenessofthemaximumlikelihoodestimatesforcertaingeneralizedlinearmodels.Biometrika,63(1),27. [83] Wood,S.N.(2011).Faststablerestrictedmaximumlikelihoodandmarginallikelihoodestimationofsemiparametricgeneralizedlinearmodels.JournaloftheRoyalStatisticalSociety(B),73(1),3. [84] Zalesky,A.,Fornito,A.,Harding,I.H.,Cocchi,L.,Yucel,M.,Pantelis,C.,&Bullmore,E.T.(2010).Whole-brainanatomicalnetworks:doesthechoiceofnodesmatter?Neuroimage,50(3),970. 102

PAGE 103

BIOGRAPHICALSKETCH DmytroKorenkevychwasborninDneprodzerjinsk,Ukraine.Hereceivedhisbachelor'sandmaster'sdegreesinappliedmathematicsandphysicsfromtheMoscowInstituteofPhysicsandTechnology(MIPT)inMoscow,Russia,in2007and2009respectively.InAugust2009hejoinedthegraduateprogramattheDepartmentofIndustrialandSystemsEngineeringatUniversityofFlorida. 103