<%BANNER%>

Reproducing Kernel Hilbert Spaces for Point Processes, with Applications to Neural Activity Analysis

Permanent Link: http://ufdc.ufl.edu/UFE0022471/00001

Material Information

Title: Reproducing Kernel Hilbert Spaces for Point Processes, with Applications to Neural Activity Analysis
Physical Description: 1 online resource (194 p.)
Language: english
Creator: Paiva, Antonio
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Point processes are stochastic random processes, yet a realization consists of a set of randomly distributed event locations. Hence, the peculiar nature of point process has made the application of conventional signal processing methods to their realizations difficult and imprecise to apply from first principles. Statistical descriptors have been extensively studied in the point process literature, and thus provide accurate and well founded methodologies to point process analysis by estimating the distributions necessary to characterize the process. But such methodologies face serious shortcomings when the interactions among multiples point processes need to be considered simultaneously, since they are only practical using an assumption of independence. Nevertheless, processing of multiple point processes is very important for practical applications, such as neural activity analysis, with the widespread use of multielectrode array techniques. This dissertation presents a general framework based on reproducing kernel Hilbert spaces (RKHS) to mathematically describe and manipulate point processes. The main idea is the definition of inner products (or point process kernels) to allow signal processing with point process from basic principles while incorporating their statistical description. Moreover, because many inner products can be formulated, a particular definition can be crafted to best fit an application. These ideas are illustrated by the definition of a number of inner products for point processes. To further elicit the advantages of the RKHS framework, a family of these inner products, called the cross-intensity (CI) kernels, is further analyzed in detail. This particular inner product family encapsulates the statistical description from conditional intensity functions of spike trains, therefore bridging the gap between statistical methodologies and the need for operators for signal processing. It is shown that these inner products establish a solid foundation with the necessary mathematical structure for signal processing with point processes. The simplest point process kernel in this family provides an interesting perspective to other works presented in the literature, since the kernel is closely related to cross-correlation. These theoretical developments also have important practical implications, with several examples shown here. The RKHS framework is of high relevance to the practitioner since it allows the development of point process analysis tools, with the emphasis given here to spike train analysis. The relation between the simplest of the CI kernels and cross-correlation exposes the limitations of current methodologies, but also brings forth the possibility of using the more general CI kernels to cope with general point process models. From a signal processing perspective, since the RKHS is a vector space with an inner product, all the conventional signal processing algorithms that involve inner product computations can be immediately implemented in the RKHS. This is illustrated here for clustering and PCA, but many other applications are possible such as filtering.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Antonio Paiva.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Principe, Jose C.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2009-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022471:00001

Permanent Link: http://ufdc.ufl.edu/UFE0022471/00001

Material Information

Title: Reproducing Kernel Hilbert Spaces for Point Processes, with Applications to Neural Activity Analysis
Physical Description: 1 online resource (194 p.)
Language: english
Creator: Paiva, Antonio
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Point processes are stochastic random processes, yet a realization consists of a set of randomly distributed event locations. Hence, the peculiar nature of point process has made the application of conventional signal processing methods to their realizations difficult and imprecise to apply from first principles. Statistical descriptors have been extensively studied in the point process literature, and thus provide accurate and well founded methodologies to point process analysis by estimating the distributions necessary to characterize the process. But such methodologies face serious shortcomings when the interactions among multiples point processes need to be considered simultaneously, since they are only practical using an assumption of independence. Nevertheless, processing of multiple point processes is very important for practical applications, such as neural activity analysis, with the widespread use of multielectrode array techniques. This dissertation presents a general framework based on reproducing kernel Hilbert spaces (RKHS) to mathematically describe and manipulate point processes. The main idea is the definition of inner products (or point process kernels) to allow signal processing with point process from basic principles while incorporating their statistical description. Moreover, because many inner products can be formulated, a particular definition can be crafted to best fit an application. These ideas are illustrated by the definition of a number of inner products for point processes. To further elicit the advantages of the RKHS framework, a family of these inner products, called the cross-intensity (CI) kernels, is further analyzed in detail. This particular inner product family encapsulates the statistical description from conditional intensity functions of spike trains, therefore bridging the gap between statistical methodologies and the need for operators for signal processing. It is shown that these inner products establish a solid foundation with the necessary mathematical structure for signal processing with point processes. The simplest point process kernel in this family provides an interesting perspective to other works presented in the literature, since the kernel is closely related to cross-correlation. These theoretical developments also have important practical implications, with several examples shown here. The RKHS framework is of high relevance to the practitioner since it allows the development of point process analysis tools, with the emphasis given here to spike train analysis. The relation between the simplest of the CI kernels and cross-correlation exposes the limitations of current methodologies, but also brings forth the possibility of using the more general CI kernels to cope with general point process models. From a signal processing perspective, since the RKHS is a vector space with an inner product, all the conventional signal processing algorithms that involve inner product computations can be immediately implemented in the RKHS. This is illustrated here for clustering and PCA, but many other applications are possible such as filtering.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Antonio Paiva.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Principe, Jose C.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2009-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022471:00001


This item has the following downloads:


Full Text

PAGE 1

REPRODUCINGKERNELHILBERTSPACESFORPOINTPROCESSES,WITHAPPLICATIONSTONEURALACTIVITYANALYSISByANTONIOR.C.PAIVAADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2008 1

PAGE 2

c2008AntonioR.C.Paiva 2

PAGE 3

Tomyfamily,foralltheirloveandcaring 3

PAGE 4

ACKNOWLEDGMENTS Iwouldliketoexpressmygratitudetomysupervisor,Dr.JoseC.Principe,forhavingacceptedmeashisstudent,andforhisexperiencedguidanceandadvice.Hisincentivetocreativity,breathofknowledge,andcriticalreachingthinkingare,Ibelieve,someofthemostvaluablelessonsIwillretainfrommydoctoraleducation.Withouthim,thisdissertationwouldnothavebeenpossible.IalsothankDr.JohnG.Harris,forservingasmycommitteemember,hisinterestinmyresearch,andprovidinganessentialpracticalperspectivetomuchofmywork.IalsothankDr.JustinC.Sanchezforhisvaluabletimetoreadandcommentonmanyoftheresultsshownhere.Hisexpertiseonneuralactivityanalysisandoftencomplementaryperspectivecanbeencounteredthroughoutthisdissertation.IalsothankDr.JianboGaoforalltheadviceandinterestinservinginmycommittee.IamforeverindebtedtoDr.FranciscoVaz,forrstcreatingtheopportunityformetocometoCNELandforallthehelpinobtainingfundingfromFCT.IwillneverforgetthatwithoutDr.Vaz'sassistance,IwouldhavemissedthewonderfulopportunitytogetaPh.D.attheUniversityofFlorida.MyfriendsandcolleaguesatCNELdeservecreditformanyofthejoysandforsharingthistortuouspathtoobtainadoctoraldegree.Inparticular,IthankIlPark(a.k.a.,Memming)formanyofthecontributionstothisresearch,Dr.YiwenWang,AysegulGunduz,ShalomDarmanjian,WeifengLiu,andDr.HuiLiuforallthefunmomentsandmanydiscussionsaboutresearchandlife.Last,butnotleast,Ithankmyfamilyfortheirloveandcaring,andalwaysbeingbyside,supporting,andcheeringmeupwhenIfeltdown. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................. 4 LISTOFFIGURES .................................... 9 LISTOFTABLES ..................................... 12 ABSTRACT ........................................ 13 CHAPTER 1INTRODUCTION .................................. 15 1.1GeneralMotivation ............................... 15 1.2ProblemStatement ............................... 17 1.3MainContributions ............................... 19 1.4Outline ...................................... 21 2POINTPROCESSESANDSPIKETRAINMETHODS ............. 23 2.1HistoryofPointProcessTheory ........................ 23 2.1.1LifeTablesandSelf-RenewingAggregates ............... 24 2.1.2CountingProblems ........................... 26 2.1.3CommunicationsandReliabilityTheory ................ 27 2.1.4DensityGeneratingFunctionsandMomentDensities ........ 28 2.1.5OtherTheoreticalDevelopments .................... 28 2.2RepresentationsandDescriptorsofPointProcesses ............. 30 2.2.1EventDensitiesandDistributions ................... 30 2.2.2CountingProcesses ........................... 32 2.2.3RandomProbabilityMeasures ..................... 32 2.2.4GeneratingFunctionalsandMomentDensities ............ 33 2.3SpikeTrainsasRealizationsofPointProcesses ................ 34 2.4AnalysisandProcessingofSpikeTrains .................... 35 2.4.1IntensityEstimation .......................... 35 2.4.1.1Binning ............................ 35 2.4.1.2Kernelsmoothing ....................... 37 2.4.1.3Nonparametricregressionwithsplines ........... 38 2.4.1.4Trialaveraging ........................ 39 2.4.2MethodsforSpikeTrainAnalysis ................... 40 2.4.3ProcessingofSpikeTrains ....................... 42 2.4.3.1Linear/nonlinearmodels ................... 43 2.4.3.2Probabilisticmodels ..................... 44 2.4.3.3Statespacemodels ...................... 45 2.4.3.4VolterraandWienermodels ................. 46 5

PAGE 6

3INNERPRODUCTSFORPOINTPROCESSES,ANDINDUCEDREPRODUCINGKERNELHILBERTSPACES ............................ 49 3.1InnerProductforEventCoordinates ..................... 49 3.2InnerProductsforPointProcesses ...................... 52 3.2.1LinearCross-IntensityKernels ..................... 53 3.2.2NonlinearCross-IntensityKernels ................... 56 3.3PropertiesofCross-IntensityKernels ..................... 58 3.3.1PropertiesofLinearCross-IntensityKernels ............. 58 3.3.2PropertiesofNonlinearCross-IntensityKernels ............ 60 3.4EstimationofCross-IntensityKernels ..................... 62 3.4.1EstimationofGeneralCross-IntensityKernels ............ 62 3.4.2EstimationofthemCIKernel ..................... 63 3.4.3EstimationofNonlinear(Memoryless)Cross-IntensityKernels ... 65 3.4.3.1EstimationofI ....................... 66 3.4.3.2EstimationofthenCIkernel,Iy .............. 66 3.5RKHSInducedbytheMemorylessCross-IntensityKernelandCongruentSpaces ...................................... 69 3.5.1SpaceSpannedbyIntensityFunctions ................. 70 3.5.2InducedRKHS ............................. 71 3.5.3MemorylessCIKernelandtheRKHSInducedby ......... 72 3.5.4MemorylessCIKernelasaCovarianceKernel ............ 73 3.6PointProcessDistances ............................ 74 3.6.1NormDistance .............................. 75 3.6.2Cauchy-SchwarzDistance ........................ 75 3.6.3SpikeTrainMeasures .......................... 76 4ASTATISTICALPERSPECTIVEOFTHERKHSFRAMEWORK ...... 78 4.1GeneralizedCross-CorrelationandthemCIKernel ............. 78 4.2RelevanceforStatisticalAnalysisMethods .................. 81 4.2.1RelationtotheCrossIntensityFunction ............... 81 4.2.2RelationtotheSpikeTriggeredAverage ................ 83 4.2.3IllustrationExample .......................... 83 5APPLICATIONSINNEURALACTIVITYANALYSIS .............. 87 5.1GeneralizedCross-CorrelationasaNeuralEnsembleMeasure ........ 87 5.2EmpiricalAnalysisofGCCStatisticalProperties .............. 89 5.2.1RobustnesstoJitterintheSpikeTimings ............... 89 5.2.2SensitivitytoNumberofNeurons ................... 91 5.3InstantaneousCross-Correlation ........................ 94 5.3.1StochasticApproximationofGCC ................... 96 5.3.2ICCasaNeuralEnsembleMeasure .................. 97 5.3.3DataExamples ............................. 97 5.3.3.1ICCasasynchronizationmeasure .............. 98 6

PAGE 7

5.3.3.2Synchronizationofpulse-coupledoscillators ........ 100 5.3.3.3Analysisofneuralsynchronousactivityinmotorneurons 103 5.4Peri-EventCross-CorrelationOverTime ................... 109 5.4.1Method .................................. 109 5.4.2DataExamples ............................. 111 5.4.2.1Simulation ........................... 112 5.4.2.2Event-relatedmodulationofsynchronousactivity ..... 114 6CLUSTERINGOFSPIKETRAINS ........................ 118 6.1Algorithm .................................... 118 6.2ComparisontoFellous'ClusteringAlgorithm ................. 121 6.3Simulations ................................... 125 6.3.1ClustersCharacterizedbyFiringRateModulation .......... 125 6.3.2ClustersCharacterizedbySynchronousFirings ............ 127 6.3.3ClusteringofRenewalProcessesbymCIandnCIKernels ...... 131 6.4ApplicationforNeuralActivityAnalysis ................... 133 7PRINCIPALCOMPONENTANALYSIS ...................... 136 7.1OptimizationintheRKHS ........................... 137 7.2OptimizationintheSpaceSpannedbytheIntensityFunctions ....... 140 7.3Results ...................................... 142 7.3.1ComparisonwithBinnedCross-Correlation .............. 142 7.3.2PCAofRenewalProcesses ....................... 147 8CONCLUSIONANDTOPICSFORDEVELOPMENT .............. 152 8.1Conclusion .................................... 152 8.2TopicsforFutureDevelopments ........................ 155 APPENDIX ABRIEFINTRODUCTIONTORKHSTHEORY .................. 158 BACOMPARISONOFSPIKETRAINMEASURES ................ 160 B.1Introduction ................................... 160 B.2BinlessSpikeTrainDissimilarityMeasures .................. 162 B.2.1Victor-Purpura'sDistance ....................... 162 B.2.2vanRossum'sDistance ......................... 164 B.2.3Schreiberetal.InducedDivergence .................. 166 B.3ExtensionoftheMeasurestoMultipleKernels ................ 168 B.4Results ...................................... 172 B.4.1DiscriminationofDierenceinFiringRate .............. 174 B.4.2DiscriminationofPhaseinFiringRateModulation ......... 177 B.4.3DiscriminationofSynchronousFirings ................. 180 B.5FinalRemarks .................................. 182 7

PAGE 8

REFERENCES ....................................... 185 BIOGRAPHICALSKETCH ................................ 194 8

PAGE 9

LISTOFFIGURES Figure page 1-1Aninnerproductisanelementaryoperationinsignalprocessingandpatternrecognition. ...................................... 18 1-2Outlineofthedissertation. .............................. 21 2-1Arealizationofapointprocessandthecorrespondingcountingprocess. .... 31 2-2Anexampleofasingle-neuronextracellularvoltagerecordingshowingafewactionpotentials. ................................... 34 2-3Estimationoftheintensityfunctionbydierentprocedures. ........... 36 2-4DiagramoftheVolterra/Wienermodelforsystemidentication. ......... 46 3-1EstimationofdierenceofintensityfunctionsforevalutionofnonlinearkernelinEquation 3{13 ................................... 67 3-2RelationbetweentheoriginalspaceofpointprocessesP(T)andthevariousHilbertspaces. .................................... 69 4-1Datasetforestimationofcross-intensityinducedeects. ............. 84 4-2Spike-triggeredaverageoftheintensityfunctionofpjandlaggedmCIkernelestimatedwiththeLaplacianandGaussiankernels. ................ 86 5-1ChangeinGCCversusjitterstandarddeviationinsynchronousspiketimings. 90 5-2VarianceofGCCversusthenumberofspiketrainsusedforspatialaveraging. 92 5-3MeanandstandarddeviationofGCCversusthenumberofspiketrainsusedforspatialaveragingfordierentsynchronylevels. ................. 93 5-4DiagramoutliningtheideaandprocedureforthecomputationoftheICC. ... 95 5-5AnalysisofthebehaviorofICCasafunctionofsynchronyinsimulatedcoupledspiketrains. ...................................... 99 5-6Evolutionofsynchronyinaspikingneuralnetworkofpulse-coupledoscillators. 101 5-7Zero-lagcross-correlationcomputedovertimeusingaslidingwindow10binslong,andbinsize1ms(top)and1.1ms(bottom). ................. 102 5-8ICCandneuronringrasterplotonasinglerealization,showingthemodulationofsynchronyaroundtheleverpresses. ....................... 104 5-9Windowedcross-correlationofselected6pairsofneurons,forthesamesegmentsshowninFigure 5-8 ................................. 105 9

PAGE 10

5-10Spatiallyaveragedwindowedcross-correlation,forthesamesegmentsshowninFigure 5-8 ....................................... 106 5-11TrialaveragedICCandcross-correlationtimelockedtoleverrelease. ...... 108 5-12Modulationofintensitywiththeeventforeachneuron. .............. 111 5-13CenteredPECCOTforthethreeneuronpairsaroundthelever. ......... 112 5-14CenteredJPSTHforeachneuronpair. ....................... 113 5-15CenteredPECCOTaroundtheleverpressonsetofmotorneurons. ....... 115 5-16CenteredPECCOTaroundtheleverpressonsetofmotorneurons(colorcodedimage). ........................................ 116 6-1ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithmfortwoclusters. .................... 123 6-2ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithmforthreeclusters. ................... 123 6-3ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithmforveclusters. .................... 124 6-4Clusteringperformanceasafunctionofthephasedierenceinthe(sinusoidal)intensityfunction. .................................. 126 6-5Clusteringperformanceasafunctionofthesynchronylevelbetweenpointprocesswithinclusterinthejitter-freecase. ......................... 128 6-6Clusteringperformanceasafunctionofthejitterstandarddeviation. ...... 129 6-7ComparisonofclusteringperformanceusingmCIandnCIkernelsforathreeclusterproblem. .................................... 132 6-8Clusteringofneuralactivityfollowingaleverrelease. ............... 134 7-1SpiketrainsusedforevaluationofthePCAalgorithm. .............. 142 7-2EigendecompositionofthecenteredGrammatrix~I. ................ 143 7-3Firsttwoprincipalcomponentfunctionsinthespaceofintensityfunctions. ... 144 7-4ProjectionontothersttwoprincipalcomponentsusingmCIkernel. ...... 144 7-5Eigendecompositionofthecovariancematrix. ................... 146 7-6Projectionofspiketrainsontothersttwoprincipalcomponentsofthecovariancematrixofbinnedspiketrains. ............................ 146 7-7SpiketrainsfromrenewalpointprocessesforcomparisonofmCIwithnCIkernel. 148 10

PAGE 11

7-8EigendecompositionoftheGrammatrix,forthemCIandnCIkernels. ..... 149 7-9ProjectionofrenewalspiketrainsontothersttwoprincipalcomponentsusingmCIandnCIkernels. ................................. 150 B-1Typicalexperimentalsetupforclassicationusingspiketraindissimilarities. .. 161 B-2Spiketrainandcorrespondinglteredspiketrainutilizingacausalexponentialfunction(Equation B{4 ). ............................... 164 B-3KernelsutilizedinthisstudyandthecorrespondingKqfunctioninducedbyeachofthekernels. .................................. 170 B-4EstimatedpdfofthemeasuresforeachkernelconsideredandcorrespondingttedGaussianpdf. .................................. 172 B-5Valueofthedissimilaritymeasuresforeachkernelconsideredasafunctionofthemodulatingspiketrainringrate. ........................ 174 B-6Discriminantindexofthedissimilaritymeasuresforeachkernelasafunctionofthemodulatingspiketrainringrate. ........................ 175 B-7Valueofthedissimilaritymeasuresforeachkernelintermsofthephasedierenceoftheringratemodulation. ............................ 177 B-8Discriminantindexofthedissimilaritymeasuresforeachkernelintermsofthephaseoftheringratemodulation. ......................... 178 B-9Valueofthedissimilaritymeasureforeachkernelasafunctionofthesynchronyamongspiketrains. .................................. 180 B-10Discriminantindexofthedissimilaritymeasuresforeachkernelintermsofthesynchronybetweenspiketrains. ........................... 181 11

PAGE 12

LISTOFTABLES Table page 1-1Commonformsofneuralactivityrecordingsandtheirproperties. ........ 16 3-1OutlineofthealgorithmforestimationofthenCIkernel,Iy. ........... 68 6-1Step-by-stepdescriptionofthealgorithmforclusteringofspiketrains.Thesearebasicallythestepsofthespectralclusteringalgorithm. ............ 120 12

PAGE 13

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyREPRODUCINGKERNELHILBERTSPACESFORPOINTPROCESSES,WITHAPPLICATIONSTONEURALACTIVITYANALYSISByAntonioR.C.PaivaAugust2008Chair:JoseC.PrncipeMajor:ElectricalandComputerEngineeringPointprocessesarestochasticrandomprocesses,yetarealizationconsistsofasetofrandomlydistributedeventlocations.Hence,thepeculiarnatureofpointprocesshasmadetheapplicationofconventionalsignalprocessingmethodstotheirrealizationsdicultandimprecisetoapplyfromrstprinciples.Statisticaldescriptorshavebeenextensivelystudiedinthepointprocessliterature,andthusprovideaccurateandwellfoundedmethodologiestopointprocessanalysisbyestimatingthedistributionsnecessarytocharacterizetheprocess.Butsuchmethodologiesfaceseriousshortcomingswhentheinteractionsamongmultiplespointprocessesneedtobeconsideredsimultaneously,sincetheyareonlypracticalusinganassumptionofindependence.Nevertheless,processingofmultiplepointprocessesisveryimportantforpracticalapplications,suchasneuralactivityanalysis,withthewidespreaduseofmultielectrodearraytechniques.ThisdissertationpresentsageneralframeworkbasedonreproducingkernelHilbertspaces(RKHS)tomathematicallydescribeandmanipulatepointprocesses.Themainideaisthedenitionofinnerproducts(orpointprocesskernels)toallowsignalprocessingwithpointprocessfrombasicprincipleswhileincorporatingtheirstatisticaldescription.Moreover,becausemanyinnerproductscanbeformulated,aparticulardenitioncanbecraftedtobesttanapplication.Theseideasareillustratedbythedenitionofanumberofinnerproductsforpointprocesses.TofurtherelicittheadvantagesoftheRKHSframework,afamilyoftheseinnerproducts,calledthecross-intensity(CI)kernels,is 13

PAGE 14

furtheranalyzedindetail.Thisparticularinnerproductfamilyencapsulatesthestatisticaldescriptionfromconditionalintensityfunctionsofspiketrains,thereforebridgingthegapbetweenstatisticalmethodologiesandtheneedforoperatorsforsignalprocessing.Itisshownthattheseinnerproductsestablishasolidfoundationwiththenecessarymathematicalstructureforsignalprocessingwithpointprocesses.Thesimplestpointprocesskernelinthisfamilyprovidesaninterestingperspectivetootherworkspresentedintheliterature,sincethekerneliscloselyrelatedtocross-correlation.Thesetheoreticaldevelopmentsalsohaveimportantpracticalimplications,withseveralexamplesshownhere.TheRKHSframeworkisofhighrelevancetothepractitionersinceitallowsthedevelopmentofpointprocessanalysistools,withtheemphasisgivenheretospiketrainanalysis.TherelationbetweenthesimplestoftheCIkernelsandcross-correlationexposesthelimitationsofcurrentmethodologies,butalsobringsforththepossibilityofusingthemoregeneralCIkernelstocopewithgeneralpointprocessmodels.Fromasignalprocessingperspective,sincetheRKHSisavectorspacewithaninnerproduct,alltheconventionalsignalprocessingalgorithmsthatinvolveinnerproductcomputationscanbeimmediatelyimplementedintheRKHS.ThisisillustratedhereforclusteringandPCA,butmanyotherapplicationsarepossiblesuchasltering. 14

PAGE 15

CHAPTER1INTRODUCTION 1.1GeneralMotivationAprimalquestioninanyworkaspiringforrelevanceiswhysuchworkisworthyofattention.Inthissectionweanswerthisquestion.Moreover,answeringthisquestionalsopreparesthereadertounderstand,andbetterappreciate,howtheproblemshouldbesolved,whichisdoneinthenextsection.Inaverybroadsense,onemightsaythatthisdissertationwasmotivatedbyadesiretounderstandhowthebrainworks.Or,morespecically,bythedesiretounderstandthebasicprinciplesbywhichthebrainrepresentsandcomputeswithinformation.Nevertheless,thisistoobroadofaquestiontotackle.Morethansimplytryingtoposeaphilosophicalquestion,orforthesakeoftheinterestinfundamentalneurophysiologyandneuroscience,weweretryingtosolveanengineeringchallenge.Thegoalwasdoproposeaframeworkforsignalprocessingwithneuralactivitywhichonecouldapplytodesignbetter(moreaccurateandreliable)brain-machineinterfaces(BMIs).Naturally,useofthisframeworkforBMIworkcouldgreatlybenetfromknowledgeoftheprinciplesofinformationrepresentationinthebrain.Moreimportantly,aframeworkforsignalprocessingcanprovidethemeanstodesignthenecessarytoolstosearchforthisunderstanding.Indeed,thismixofinterestswillbenoticeablethroughout.Whatdowemeanbyneuralactivityinthiswork?Brainactivitycanbeanalyzedusingmanyformsofneuralrecordings,namely:single-unitactivity(SUA),localeldpotentials(LFP),electro-corticogram(ECoG),electro-encephalogram(EEG),magneto-encephalogram(MEG),justtomentionthemostcommonlyused.Eachofthesesignalshasspecicproperties,forexample,intermsofspatialresolutionandcoverage,andsignal-to-noiseratio.Thegeneralideaisthatbetterpropertiesoftherecordingsaretypicallyobtainedattheexpenseofgreaterinvasiveness,whichisofparamountimportanceinpracticaluse.Table 1-1 reviewssomeoftheproperties. 15

PAGE 16

Table1-1. Commonformsofneuralactivityrecordingsandtheirproperties. RecordingInvasiveLocalresolutionSpatialcoverageSpectralrangeSNR SUAyesveryhighlocalizedhigh-frequencieshigh LFPyeshighbroadbroadbandhigh ECoGyeshighbroadbroadbandhigh EEGnolowbroadlow-frequencieslow MEGnolowbroadlow-frequencieslow InthisworkonlySUAwillbeconsidered.Thisisthemostinvasivemethod(togetherwithLFP)withtheneedtointroduceelectrodesperforatingthecortex.Ontheotherhand,fromanengineeringperspective,SUAhasthebestproperties,especiallyintermsofresolutionandSNR,andthereforederivedBMIshavethepotentialtoachievethebestresolution.Furthermore,BMIsstudiesbasedinthisformofrecordingalsohavethepotentialtodeepenyourunderstandingofhowisinformationrepresentedinthebrainandshouldprovideanupperboundontheachievableperformance.Finally,thisunderstandingcansuggesthowtoimprovethedesignofBMIsusinglessinvasiveneuralactivityrecordings.SUA-basedBMIsareattheforefrontofbraindecodingforbrain-machineinteraction.Thisisunderstandablesince,asstated,thisformofrecordinghasthebestsignalcharacteristics.However,unliketheotherrecordings,workingwiththisformofrecordingpresentsachallengeofitsownsinceSUAisarecordingoftheactivityofoneneuron,andneuronsareknowntocommunicatethroughelectricalpulses,calledspikes.Thus,informationisrepresentednotinavoltagewaveformasusualbutinsequencesofspikes,orspiketrains.Thechallengeisthatspiketrainsmustbemodeledasrealizationsofpointprocesses,forwhichthebasicsignalprocessingoperatorsarenotstraightforwardlydened.Thisisthegoalofthisdissertation.NoticethatalthoughBMIswerethemotivationtostartthisworkandareprimalapplications,theyarenotthefocusofthisdissertation.Infact,thisworkhasasubstantiallybroaderimpact,andforwhichBMIsareonlyoneapplication.Forexample,thismaybeof 16

PAGE 17

greatimportanceinothercomputationparadigms,suchasinliquidstatemachines(LSM)studies[ Maassetal. 2002 ],orspikingneuralnetwork(SNN)modelswhichhaverecentlyemergedasanewarticialneuralnetworksparadigminawaythatmorecloselymimicsthebrain[ MaassandBishop 1998 ; GerstnerandKistler 2002 ].Moreover,theimpactofthisworkmayevengobeyondtheseapplications,towhateverproblemwhereprocessingoranalysisofpointprocessesisrequired. 1.2ProblemStatementBeforemovingon,itisbenecialtotrytounderstandthechallengetackledhereandwhyprocessingwithpointprocessesisnotasstraightforwardasforcontinuous-ordiscrete-valuedrandomprocesses.Pointprocessesarestochasticrandomprocesses,yetarealizationconsistsofasetofrandomlydistributedeventlocations.Putdierently,forapointprocesstherandomnessisnotcontainedintheamplitude 1 butwhen(orwhere)theeventoccurs.Consequently,uponobservationofarealizationofapointprocessoneisnotinterestedintheeventsthemselvesbutonthemechanism/informationunderlyingthegenerationoftheevents.Pointprocessesplayaveryimportantroleinstatisticalmodelingandinferenceinawidevarietyofelds,suchas:biology,engineering,geography,physics,astronomy[ Snyder 1975 ,Section1.1forapplicationexamples].Ingeneraltheeventspaceofapointprocessescanbeone-dimensionalormultidimensional.However,hereweshalldealexclusivelywithone-dimensionalpointprocesses.Often,theeventspaceofone-dimensionalpointprocessesistime(asisthecaseforspiketrains).Forthisreason,wewilluse\eventlocations"and\eventtimes"interchangeablytorefertothecoordinatesofevents. 1 Actually,therearepointprocessmodels,calledmarkedpointprocesses,forwhichtheremaybeoneormorerandomvariablesassociatedwiththeevents.Inthiscase,theamplitudeoftheeventisaresultoftherandomnessintheserandomvariables.Nevertheless,theseareaspecialclassofpointprocessesandarenotconsideredinthiswork. 17

PAGE 18

. Filtering PCA Clustering Classication Innerproduct Convolution Projection Similarity Discriminantfunction Figure1-1. Aninnerproductisanelementaryoperationinsignalprocessingandpatternrecognition. Unfortunately,thepeculiarformulationofpointprocessesdoesnotallowfortheapplicationoftheusualsignalprocessingoperationstolter,eigendecompose,classifyorclusterpointprocessesortheirrealizations,whichareessentialtomanipulatingthesesignalsandextracttheinformationtheyconvey.Fromastatisticalperspective,pointprocessescanbewellcharacterizedandmanyrepresentationshavebeendevelopedintheliterature[ Snyder 1975 ; DaleyandVere-Jones 1988 ].SomeoftheserepresentationsanddescriptorswillbereviewedinChapter 2 .Themainlimitationofcurrentstatisticalapproachesisthatpointprocessesareanalyzedindependently,andindependenceneedtobetypicallyassumedtoavoidhandlingthehighdimensionaljointdistributionwhenmultiplepointprocessesareconsidered.Beforeattendingthequestionofhowtodosignalprocessingwithpointprocesses,letusrstconsiderwhatisnecessaryforsignalprocessing.Forltering,theoutputistheconvolutionoftheinputwithanimpulseresponse;forprincipalcomponentanalysis(PCA),oneneedstobeabletoprojectthedata;forclustering,theconceptofsimilarity(ordissimilarity)betweenpointsisneeded;andforclassication,itisnecessarytodenediscriminantfunctionsthatseparatetheclasses.However,carefulobservationrevealsthatalloftheseneededconceptsareeitherimplementeddirectlybyaninnerproductorcanbeconstructedwithaninnerproduct.Convolutionimplementsaninnerproductateachtimeinstantbetweenashiftedversionoftheinputfunctionandthesystems'impulse 18

PAGE 19

response.Projectionisinherentlyaninnerproductbetweentwoobjects.Aninnerproductisalsoasimilaritymeasure,anddissimilaritymeasures(suchasdistances)canbedenedgivenaninnerproduct.Discriminantfunctionscannotbeobtaineddirectlywithaninnerproductbut,aneuralnetworkcanbeusedtoapproximateittothedesiredprecision,withthelinearprojectionsinthePEsimplementedbysomegiveninnerproduct.Insummary,toobtainageneralframeworkforsignalprocessingandpatternrecognitionwithpointprocessesallthatitisneededisaninnerproductdenitionoperatingwithspiketrains.Itmustberemarkedthatitispossibletoimplementatleastsomeoftheaforementionedconceptswithoutdeninganinnerproduct.Forexample,distancesbetweenpointprocesseshavebeendenedwithoutexplicitlydeninganinnerproduct(Section 3.6.3 ).However,suchapproacheshavelimitedscopeanddonotprovideaconsistentandsystematicmathematicalframeworktodosignalprocessing,andtendtoobscurethepointprocessmodelassociatedwiththeoperation. 1.3MainContributionsBasedonthepreviousconsiderations,wecanstatethatforsignalprocessingwithpointprocessesallthatisneededisanappropriateinnerproduct.However,asbefore,deninganinnerproductofpointprocessesisnotstraightforward,buttherequiredmathematicalstructurefollowsonceoneisdened.Forthisreason,oneofthemaincontributionsofthisdissertationittosuggesthowinnerproductsofpointprocessescanbedened,estimatedfromrealizations,anddiscusssomeoftheirimplicationsandapplications.MostoftheconsiderationspresentedhereregardingdenitionsofinnerproductsforpointprocessesaredoneundertheformalismofreproducingkernelHilbertspace(RKHS)theory.Duetotheirequivalencethismeansthatinnerproductswillbedened 19

PAGE 20

askernels. 2 TheuseofRKHStheoryisdonetoassurethatthenecessarymathematicalstructureiswelldenedeveninsituationswheretheinnerproductisnotexplicitlydened,thusensuringgeneralitywithoutsacriceinrigor.Furthermore,operatingwithpointprocessesinanRKHSismoreconvenientsinceseveraldevelopmentsinsignalprocessingandmachinelearningcanbeimmediatelyincorporated.Therefore,thisprovidestheframeworkforthedevelopmentofacomprehensivesetofalgorithmsforanalysisandprocessingofpointprocesses.Althoughfrequentlyoverlooked,RKHStheoryisapivotalconceptinstatisticalsignalanalysisandprocessing[ Parzen 1967 ],andmachinelearning[ Scholkopfetal. 1999 ].InRKHStheory,kerneloperatorsdenoteinnerproductoperationsinaHilbertspace,whicharefundamentalforsignalprocessingtechniques,thusprovidingastrongmotivationfortheuseofkernelfunctions.Forinstance,thecross-correlationfunctionusedthroughoutstatisticalanalysisandsignalprocessing,includingthecelebratedWienerlter[ Haykin 2002 ],isavalidkernelandinducesaRKHSspace[ Parzen 1959 ].Infact,most(ifnotall)ofourunderstandingandeaseofmathematicaltreatmentofsecond-ordermethodscanbeobtainedfromthestudyoftheRKHSinducedbythecross-correlation.Inthisdissertation,severalkernels(thatis,innerproducts)foroperatingwithpointprocessesshallbeproposed.NoticethattheircorrespondingRKHSsareautomaticallydened.Twomainapproacheswillbefollowed.Therstderivesfromideasinkernelmethods,whereastheseconddenestheinnerproductinthespaceofintensityfunctionsofthepointprocesses.Bothmayplayanimportantroleinmethoddevelopments.Weshallmainlyfocusonthesecondapproachsincetheuseoftheconditionalintensityfunctionspermitstheinnerproducttoencapsulateacompletestatisticalcharacterization 2 Throughoutthisdissertationwewilloftenreferto`kernels'and`innerproducts'interchangeably.Inourcontext,unlesscautionedotherwise,theyshouldalwaysbeunderstoodasthesameconcept,althoughtheformershallbeoftenpreferredsinceitmakesexplicittheconnectiontoRKHStheory. 20

PAGE 21

. Chapter 2 :Pointprocessesandspiketrains Chapter 3 :RKHSsforpointprocesses Chapter 4 :AstatisticalperspectiveoftheRKHSframework Chapter 5 :Toolsforneuralactivityanalysis Chapter 6 :Clusteringofspiketrains Chapter 7 :PCAofspiketrains Figure1-2. Outlineofthedissertation. ofthepointprocess,providesabetterinsightofthepropertiesandlimitationsoftheinnerproductasadescriptorofthepointprocesses,andbecause,aswillbeshown,isclosestrelatedtocurrentmethodologies.Therelevanceoftheseconceptsareexempliedinapplications,wheresomeoftheseinnerproductsareutilized.Animportantcomponentofthisdissertationisalsothediscussionofimplicationsofthisworkwhich,byitsgenerality,providesinsightfulperspectivesinseveralmethodologiesdescribedintheliterature.Considerationsforimmediateimplicationsinthestate-of-the-artmethodsforspiketrainanalysisarealsopresented. 1.4OutlineThisdissertationisorganizedinroughlyfourparts.TherstcomprisesofChapter 1 andChapter 2 andprovidesthemotivation,establishestheproblemfromanoverallperspective,introducespointprocessesandspiketrains,andreviewspreviousapproches 21

PAGE 22

fortheiranalysis.Thesecondpart,inChapter 3 ,containsthemaintheoreticalcontributionsandiswherethekernelsforpointprocessesandthecorrespondingRKHSaredenedandanalyzed.Thethirdpartexploresamoregeneraldenitionofcross-correlationinspiredbyanRKHSconstructioninChapter 4 ,anditsmultipleconsequencesintermsofnewtoolsfortheexperimenterinChapter 5 withseveralexamplesofapplicationofthesetoolsinbothsimulatedandrealdatasets.ThispartissomewhatindependentofthetheoryinChapter 3 ,butindoingsothereaderwillmisstheimportantconnectionstothegeneralRKHSframeworkbeingpresented.Finally,thefourthpartshowstwoapplicationexamplesoftheRKHSframeworkformachinelearningbyshowinghowclusteringalgorithmsforspiketrainsmaybeeasilyderivedinChapter 6 ,andbyderivingfromrstprinciplestheprincipalcomponentanalysisalgorithmforspiketrains.ConclusionsanddiscussionofthisworkaregiveninChapter 8 ,alongwithadescriptionofpossibleideasforfuturedevelopmentsonthiswork. 22

PAGE 23

CHAPTER2INTRODUCTIONTOPOINTPROCESSESANDSPIKETRAINMETHODSInthischapter,webrieyintroducewhatpointprocessesareandhowtheyariseinanumberofproblems.Thisshallbedonerstinasomewhatinformalway,broadlyintroducingthereadertohistoricalproblemsthatgaverisetothestudyofpointprocessesinareviewmanner.Afterwards,theproblemofhowpointprocessesariseinneurophysiologyisdiscussedtoaimonsomeoftheimportantgoalsforthiswork.Then,manyofthetechniquesspecicallydevelopedtoanalyzespiketrainsarepresented,andwediscussthekeystrategiesutilizedtohandletheparticularitiesofpointprocessesandsomeofthetheirlimitations.Thisdiscussionwill,hopefully,allowthereadertohaveamoregeneralperspectiveandfurtherappreciatesomeoftheaccomplishmentsofthiswork. 2.1HistoryofPointProcessTheoryHere,abriefreviewofsomeofthehistoricaldevelopmentsinthetheoryofpointprocessesispresented.Thisisdoneherefortworeasons:tointroducethereadertosomeoftheterminologyandbasicconceptsinaninformalway,andshowcasesomeoftheapproachesdevelopedearlierthatarestillutilizedinstatisticalanalysisofpointprocesses.Foramoredetailedreviewthereaderisreferred,forexample,to DaleyandVere-Jones [ 1988 ,Chapter1].Althoughpointprocessescanbefoundinarelativelylargenumberofproblems,theprimordialideasanddevelopmentswherebeenmainlyassociatedwithfourareasofapplication,bychronologicalorder: lifetablesandself-renewingaggregates; countingproblems; communicationstheory;and particlephysicsandpopulationprocesses.Thersttwoapplicationsreallymotivatedtheinitialdevelopmentsinpointprocessanalysisandwheredevelopedinparallelwiththefundamentalideasofprobability(17thcentury),whereastheremainingtwowhereraisedinthepreviouscentury.Despitethis 23

PAGE 24

separationand,asexpected,itshouldbenoticedfromourpresentationthatthelatertopicswherestronglyaectedbytheearlierconceptsandterminology. 2.1.1LifeTablesandSelf-RenewingAggregatesLifetablesarerecordsutilizedindemographicsstudiesofapopulation.Simplyput,alifetableliststhenumberofindividualsfromapopulation,ortheirratio,thatsurvivetoagivenage.TherstknownlifetableisduetoJohnGrauntwhoin1662publishedthe\ObservationontheLondonBillsofMortality"(availableat[ Graunt 1662 ]).ThistablewasanalyzedatalatertimebyHuyghens(1629{1695)whoproposedthenotionofexpectedlengthoflife.Asecondlifetablewasconstructedin1693byHalleyusingdatafromthesmallercityofBreslau.ComparedtoGraunt'slifetable,thistablewasbettersinceHalleydidnothaveproblemswithdisease,immigrationandincompletedatathatplaguedGraunt'saccount.Lifetablesoccupiedmuchoftheeldofstatisticsofthattime,andwasdevelopedparalleltoadvancesinprobabilitytheory.Therearethreebasicdescriptors(orsummarystatistics)ofalifetable:therelativefrequencyofindividualssurvivingtoagivenageorsurvivorfunction;therelativefrequencyofindividualsthatdeceasedbetweentwoages,calledlifetimedistributionfunction;andtherelativefrequencyofindividualsthatdieafteracertainage,theso-calledhazardfunction.Theseconceptscanbewritteninformallyintermsofprobabilitiesas: (i) Survivorfunction:S(x)=Prflifetime>xg; (ii) Lifetimedistributionfunction:f(x)=limdx!01 dxPrflifetimeterminatesbetweenxandx+dxg; (iii) Hazardfunction:q(x)=limdx!01 dxPrflifetimeterminatesbetweenxandx+dxjlifetimexg: 24

PAGE 25

ActuallythesesameconceptsservedastherootforthedevelopmentsinprobabilitytheorybydeMoivre,EulerandLaplace.However,itwasnotuntilLaplace'swork,\APhilosophicalEssayonProbabilities,"thatthepreviousconceptsgainedamoreformalperspectiveintermsofprobabilities.Indeed,althoughdeMoivrehadsuggestedthatthesurvivorfunctionwoulddecreasewithconstantstepforagesbetween22and86,onlyafterLaplaceformalintroductionofprobabilitiesthethreeconceptswhereconnectedandmoreaccuratelythroughdistributions.Thebasicdistributionfunctionforlifetimehasbeentheexponentialfunction,f(x)=ex,x>0,correspondingtoaconstanthazardfunction,q(x)=.Thatis,theprobabilityofoccurrenceofaneventisindependentofpreviousevents.Amoreaccuratetisusuallyfoundbythepower-lawhazardfunctionwithaconstantadded,q(x)=B+Aex(A>0;B>0;>0),knownasGompertz-Makehamlaw.Itisoneofthemostwidelyusedfunctionsforttingalifetable.Othercommonlyuseddistributionsforlifetimemodelingare: Gamma:f(x)= ()x1ex; Lognormal:f(x)=1 p 2xe1 22(logx)2:Closelyrelatedtothestudyoflifetableswereproblemsinthestudyofstatisticaldemography,growth,mortalitytablesandinsurance.Intheinsurancecontextinparticular,theimportanceofmaintainingastable\portfolio";thatis,aself-regeneratingpopulationofindividuals,propelledthedevelopmentofthetheoryofself-renewingaggregates.Simplyput,thisproblemconcernsthestudyofevolutionofthehumanpopulationandthebalanceintermsofnumberofbirthsanddeaths.Aparticularlyrelevantconceptwastheideaofrenewaldensitycharacterizingtheprobabilityfortheneedofareplacementintimeinterval[t;t+dt).Inessence,thesesameideasservedasfoundationsforrenewaltheory. 25

PAGE 26

2.1.2CountingProblemsAnalternativerepresentationtostatisticallydescribepointprocessrealizationsistocountthenumberofeventsinintervalsorregionsoftheeventspace.Unlikeotherapproaches,countingistheonlyapproachthatlendsitselftoextensionandsystematicuseinspaceswithmorethanonedimension.Thebasicideaistodescribethepointprocessintermsofthedistributionofthenumberofeventsinagivenregionoftheeventspace.Sincethecharacterizingelementifthe\numberofevents"inthespace,discretedistributionsplayamajorroleinthestatisticalanalysisofpointprocessunderthisperspective(eventhoughthespaceiscontinuous).Theearliestreferencesofapplicationofacountingapproachtopointprocessesseemtobedueto Seidel [ 1876 ]whilestudyingtheoccurrenceofthunderstorms,and Abbe [ 1879 ]whichstudiedthenumberofbloodcellsinhaemocytometersquares.Noticethatthesecasestudiesdealtwithpointprocessesintwo-andthree-dimensionalspaces,respectively,whichjustiedtheneedforthisapproach.ThePoissondistributionisoneofthebestknownexamplesofdiscretedistributions,andisparticularlyimportantincountingproblemsofpointprocesses.In1838,Poissonhadincludedinhismonograph,\Recherchessurlaprobabilitedesjugementsenmatierescriminellesetmatierecivile,"thederivationofthePoissondistributionasthelimitcaseofthebinomialdistributionastheintervallength(orregionvolume)approacheszero.Interestingly,theworksofSeidelandAbbeoccurredafter,andapparentlyinanindependentmanner,fromPoisson'swork.Infact,thisisunderstandablesincePoisson'sresultdidnotgetwideattentionatthetime.Also,thefactthatitwasnotderivedinacountingprocesscontextmayexplainwhyitwasunknownorneglectedbySeidelandAbbe.AttentionwasonlydrawntoPoisson'sdistributionwhenin1898,VonBortkiewiczusedthedistributiontotseveralphenomenainhismonograph\DasGesetzderkleinenZahlen." 26

PAGE 27

SeveraladvancessucceededPoisson'swork,namelyongeneralizationandalternativestothePoissondistribution.Onenotablegeneralizationisthenegativebinomialdistributionderivedby GreenwoodandYule [ 1920 ]totaccidentstatistics.However,thenegativebinomialdistributioncanbeobtainedfromamixedPoissondistribution,inwhichtherateparameterisarandomvariablewithagammadistribution. 2.1.3CommunicationsandReliabilityTheoryCommunicationsandreliabilitytheoryaretwoofthemostimportantapplicationareasofpointprocessesinthepastcentury.ReliabilitytheorydevelopedmainlyafterWorldWarIIandconcernedtheestimationofthelifetimeofconnectedelements.Naturally,itabsorbedmuchoftheterminologyandconceptsderivedearlierforthestudyoflifetables.ThisapplicationwaspropelledbytheadvancesandgrowingindustryinelectronicsinthepostWWIIperiod.Communicationstheoryand,inparticular,queueingtheory,wasthesecondfundamentalapplicationofpointprocessestoengineering,withtheadventoftelephonetrunklines.Thelandmarkpaperinthesubjectwaspublishedby Erlang [ 1909 ]onthestudyofthenumberofcallsinaxedtimeinterval,forwhichErlangderivedadistribution.But,atthattime,ErlangdidnotrealizehisndingcorrespondstothePoissondistribution,onlymakingthiscorrectionin Erlang [ 1917 ].Actually,thedistributionderivedbyErlangisacontinuousprobabilitydistributionwhereasthePoissondistributionisdiscrete.Nevertheless,theErlangdistributionisaspecialcaseofthegammadistributionwheretheshapeparameterisanaturalnumber,andthegammadistributionhadalreadybeenderivedsometimeearlier.AnotherfundamentalcontributiontotheeldofqueueingtheorywasPalm'sthesisworkin1943onthestudyofintensityvariationsincommunicationstrac[ Palm 1988 ].Inhiswork,Palmprovidedadetailedanalysisofparticulartelephonetrunkingsystems,butalsothefoundationsofageneraltheoryforpointprocesswithfarreachingimpact. 27

PAGE 28

2.1.4DensityGeneratingFunctionsandMomentDensitiesPointprocesstheoryalsoprovedusefulforthestudyofparticlescatterinphysics.Duetothehigh-dimensionalityoftheproblem(morethan2dimensions)thegeneralapproachtocountingproblemswasused.However,insteadofutilizingdiscretedistributionsdirectly,theconceptsofgeneratingfunctionalsandmomentdensitieswereemployedsincetheyprovideamoreconvenienttreatmentoftheseproblems.Theseconceptswererstdevelopedby Yvon [ 1935 ],aphysicistlookingtocharacterizetheevolutionofparticlescatterdistributionsinexperimentalandtheoreticalphysicsstudies.Theseideasarerelatedtotheprobabilitygeneratingfunctionaldenedby G[h]=E(Yih(xi))=EexpZlogh(x)dNx;(2{1)whereh(x)issometestfunction,xiaretheeventlocations,andNxisthecountingmeasure.Foranitenumberofeventstheprobabilitygeneratingfunctionalallowsanexpansionintermsofmomentdensityfunctions(orproductdensities,asareperhapsmorecommonlyknown),characterizingthedistributionofthenumberofeventsandtheeventlocations.Oneofthemostimportantresultsinthisregardwasobtainedby Ramakrishnan [ 1950 ],whorstderivedexpressionsforthemomentsofthenumberofeventsintermsofproductdensitiesandStirlingnumbers.TheseideaswherelaterconsiderablyextendedbyRamakrishnan,JanossyandSrinivasan,amongothers,andappliedtonumerousphysicalproblems,suchascosmicrayshowers,forexample.Areviewofthisapproachcanbefoundin SrinivasanandVijayakumar [ 2003 ]. 2.1.5OtherTheoreticalDevelopmentsTheworkofPalmin1943[ Palm 1988 ]isoneofthelandmarksinthetheoryofpointprocessesinthelastcentury.Eventhoughithadwelldenedpracticalcontext,itestablishedthefoundationsforageneraltheoryofpointprocesses,andmanyofthecurrentterminology.Therearethreemajorcontributionsinhiswork.First,theconcept 28

PAGE 29

ofregenerationpointafterwhichapointprocess(orsystemresponsibleforthepointprocess)revertstoagivenstateandevolvesindependentlyofthepastbeforethestatecorrespondingtotheregenerationpointwasachieved.Relatedtothisidea,istheideaofaprocessaftereects,which,simplyput,describesthememorypropertyofaprocess.Thus,Poissonprocessesareprocesseswithoutaftereects,andrenewalprocesseshavelimitedaftereects.Second,thattwodistributionsareimportantindescribingstationarypointprocesses:thedistributionofthetimetothenexteventfromanarbitraryorigin,andthedistributionofthetimetothenexteventfromanarbitraryevent.ThesedistributionsarerelatedbythePalm-Khinchinequations.Third,apartialproofthelimittheoremforpointprocesses,whichstatesthatsuperpositionoflargenumberofindependentpointprocesstendstoaPoissonprocess.Palm'sworkpavedthewayfordevelopmentsby Wold [ 1948 ]onprocesseswithMarkovdependentintervals,whichconstitutethenextalternativetorenewalpointprocesses,andby Khinchin [ 1960 ]whogreatlyextendedandrenedPalm'swork.Thealternativeapproachwasthestudyofpointprocessesintermofprobabilitymeasuresonabstractspaces.ThiswasmotivatedbytheuseofcharacteristicfunctionalsproposedearlierbyKolmogorovtostudyrandomelementsinlinearspaces.Thisworkallowedforstudiesontheconvergenceofmeasuresonmetricspaces(whichalsooccurinpointprocesses),andservedasthebasisforthedevelopmentsmentionedearlieringeneratingfunctionalsandmomentdensities.Worthyofremarkarealsotheworksonthesecondhalfofthelastcenturyby Cox [ 1955 ]( CoxandIsham [ 1980 ]forareview)and Bartlett [ 1963 ].Theseauthorswereresponsiblefordevelopmentsinmethodsforstatisticaltreatmentofdatageneratedbypointprocesses.Forexample,CoxintroducedtheimportantclassofdoublystochasticPoissonprocesses,importantinthestudyofinhomogenousPoissonprocesses,andBartlettillustratedtheoreticallyhowsomemethodsoftimeseriesanalysiscouldbeadaptedtothepointprocesscontext. 29

PAGE 30

Onemustmustalsonotethecontributionsof Moyal [ 1962 ]whoestablishedthetheorypointprocessesinageneralstatespace,providingtherelationsbetweenproductdensitiesandprobabilitygeneratingfunctionals,aswellpointingoutseveralapplicationsthistheory. 2.2RepresentationsandDescriptorsofPointProcessesAsinformallyreviewedintheprevioussection,thereareroughlyfourdierentapproachestorepresentand/ordescribepointprocesses: 1. Eventdensitiesanddistributions; 2. Countingprocesses; 3. Randommeasures;and 4. Generatingfunctionalsandmomentdensities;Needlesstosaythattheserepresentationsareallcloselyinter-relatedandadescriptioninarepresentationmaybeconvertedtoanother.Thesearebrieypresentednext. 2.2.1EventDensitiesandDistributionsPointprocessescanbecharacterizedintermsofthedistributionsneededtospeciedthestatisticsofitsevents.Thisisperhapsthemostdirectapproachandreliesonthesamestatisticalprinciplesutilizetoquantifylifetables(Section 2.1.1 ).Noticethatingeneralmultiplesdistributionsmaybeneededtofullydescribeapointprocess.Thetwomostoftenusedstatisticsarethedensityofevents,calledratefunctionorsimplyintensityfunction,andtheinter-eventintervaldistribution.Thesespecifytheexpectednumberofeventsperspaceunitandthedistributionofthedierencebetweentwoadjacentevents,respectively.ThePoissonprocessisthesimplestofthecases,forwhichtheintensityfunctionprovideacompletedescription,andtheinter-eventintervalisinherentlyspeciedastheexponentialdistribution,sincethisdistributionisresponsibleforthememorylesspropertyofthePoissonprocess.Anotherexamplearetherenewalprocesses,whichgeneralizethePoissonprocesstogeneralinter-eventintervaldistributions,andthereforeboththeintensityfunctionandtheinter-eventintervaldistributionareneededtocharacterizethepointprocess.Butthesetwostatisticsdonotsuceingeneral. 30

PAGE 31

Figure2-1. Arealizationofapointprocessandthecorrespondingcountingprocess. Acompactdescriptioncanbeattainedbyusingtheconditionalintensityfunction[ Snyder 1975 ,Pg.238],denoted(tjHt),wheret2T(Tisthespaceofevents)andHt=ft1
PAGE 32

2.2.2CountingProcessesThroughouttheliteratureonpointprocesses,theconceptofcountingprocessisprevalentlyused.ThisisunderstandableforthereasonspresentedinSection 2.1.2 .Theperspectiveprovidedbyacountingprocessiseasilyinterpretableandtractableusingonlythetheoryofdiscretedistributions,andisextendibletomultidimensionalpointprocessesinasystematicmanner.ThefunctionNt(!),t2T,isacountingprocessandisdenedasthenumberofeventsuptolocationtfortherealization(Figure 2-1 ).Foreach!2,Nt(!)isapiecewise-constantfunctionoftwithunitjumpsattheeventcoordinates.However,noticethat,likestochasticrandomprocesses,Nt()isafunctionalrepresentationwhichbecomesawelldenedfunctionoftonlyforaxed!;thatis,agivenrealization.Acountingprocessisanattractiveformulationalsobecausethederivativeofitsexpectationover!atagiventmaybeinterpretedasthedensityofevents.Therefore,itprovidesawaytomapthespaceofeventstoadensityofevents,providinganequivalentrepresentationofthestatisticaldistributionsmentionedintheprevioussectionwithoutrequiringtheirformexplicitly. 2.2.3RandomProbabilityMeasuresRandommeasuresareanotherwaytoexpresspointprocesses.Letthespaceofevents,T,bealocallycompactHausdorspacewithaBorel-algebra,andNthesetoflocallynitecountingmeasuresonTwith-algebraN.Then,apointprocessonTisameasurablemap:N,fromaprobabilityspace(;B;P)tothemeasurablespace(N;N).ThatmeansthatforanysetS2T,(S)isarandomvariablecorrespondingtothenumberofeventsinS.Typically,thepointprocessrandommeasureiswrittenas ()=NXn=1tn();(2{4) 32

PAGE 33

wheredenotestheDiracmeasure, x(A)=8><>:1;x2A0;x62A;(2{5)Nisaninteger-valuedrandomvariable,andtnaretheeventsinT.Inessence,randommeasuresformalizemathematicallytheconceptsutilizedtobuildthecountingprocesses.Forthisreason,countingprocessesaresometimescalledcountingmeasuresintheliterature. 2.2.4GeneratingFunctionalsandMomentDensitiesAsmentionedearlier,thefoundationsforhandlingstochasticpopulationsofparticleshadalreadybeensetbeforeby Yvon [ 1935 ],buttherewheregapsinthetheory.Theproblemwasthatofstatisticallydescribingapointprocesscharacterizedbyanitesetofpointsorevents,sayX=fx1;:::;xNg,inastatespace.Asimpleprobabilitydescriptioncanbeobtainedintermsoftheprobabilitymassfunction(pmf)forthetotalnumberofpointsintherealization,Pr=Pr[N=r].Thepmfcanthenbeutilizedtowritethejointdistributionoverastatespaceofrealizations,r,whichinturnisspeciedintermsofthedensitiesfr(x1;:::;xr),withproperties: Pr[N(dx)=1]=f1(x)dx+o(dx);Pr[N(dx)>1]=o(dx);Pr[N(dx)=0]=1f1(x)dx+o(d):(2{6)Itmustbenotedthatthedensityf1isnotaprobabilitydensityfunction(pdf).Rather,itiscalledaproductdensityfunction,todistinguishitfromapdf.Thesecanbeutilizedtowritetheprobabilitygeneratingfunctionalintheform, G[]=E(NYi=1(xi))=P0+1Xr=1PrZr1(x1):::r(xr)fr(x1;:::;xr)dx1:::dxr;(2{7) 33

PAGE 34

Figure2-2. Anexampleofasingle-neuronextracellularvoltagerecordingshowingafewactionpotentials. or,equivalently,as G[1+]=1+1Xk=11 k!Zk1(x1):::r(xk)mk(x1;:::;xk)dx1:::dxk;(2{8)wherethemk'saretheproductdensities,with mk(x1;:::;xk)dx1:::dxkEfN(dx1):::N(dxk)g:(2{9)Inthisregard, Ramakrishnan [ 1950 ]wasthersttoderiveexplicitexpressionforthefactorialmomentsintermsofproductdensitiesandStirlingnumbers.Forthemthmomentitwasobtained EfN(dx)mg=mXk=1CmkZkfk(x1;:::;xk)dx1:::dxk;(2{10)wheretheweightcoecientsCmsaretheSterlingnumbers. 2.3SpikeTrainsasRealizationsofPointProcessesInneurophysiologyitiswidelyacceptedthatthefundamentalprocessingunitsofthebrain|theneuroncell|communicatethroughadiscretepulse-likewaveofvoltage,calledanactionpotential[ DayanandAbbott 2001 ].Aneuronreceivestheactionpotentialsinits,typically,largenumberofinputsynapsesandproducesanoutputofthesameformintheaxon,eventhoughinternallytothecellmembranetheactionpotentialisconvertedintoananalogpotentialchange.Actionpotentials,beingelectricalpulses,canbecapturedinsingle-neuronvoltagerecordings(Figure 2-2 ),whichrecordthevoltagedierentialtoadistant\ground" 34

PAGE 35

point,typicallytheskull.Despitetheinevitablenoiseintheserecordings,itiseasilyobservedthatactionpotentialshaveaverystereotypicalshape,withacharacteristicxedamplitudeandwidthassociatedwithagivenneuron.Thatistosaythat,fromaneurophysiologicalperspective,theactualshapeandmagnitudeofanactionpotentialisredundantbecauseitcontainsnoinformation,onlythemomentitoccurs.Asexplainedearlierthiskindofphenomenaisbestdescribedbyapointprocessmodel.Underthisperspectiveactionpotentialsaresimplycalled\spikes,"andtheseeventsareresponsiblefortransmittingtheinformationinandoutoftheneurononlythroughtheiroccurrence.Correspondingly,asequenceofspikesorderedintimeistermedaspiketrain.Sincespiketrainsalwayscorrespondtoanobservationormeasurementassociatedwithsomeunderlyingpointprocesstheyareconsideredtoberealizationsofapointprocessfromwhichonlythenumberofspikesandthemomentstheyoccurarerelevant. 2.4AnalysisandProcessingofSpikeTrainsSinceanalysisandprocessingofspiketrainsistheprimalmotivationforthiswork,forcompleteness,thissectionbrieyreviewsmanyoftheestablishedapproachesandmethodsutilizedintheirstudy. 2.4.1IntensityEstimationIntensityfunctionestimationisoneofthemostfundamentalproblemsinspiketrainsanalysis,sinceanintensityfunctionisafundamentaldescriptoroftheunderlyingpointprocess.Therearebasicallythreeapproachesforintensityfunctionofaspiketrain: 1. Binning; 2. Kernelsmoothing;and 3. Nonparametricregressionwithsplines. 2.4.1.1BinningBinningisthepredominantapproachincurrentspiketrainanalysisandprocessingmethods[ DayanandAbbott 2001 ].Statistically,itismotivatedbythecountingprocessrepresentationofapointprocess.Basically,thebinnedspiketrainisobtained 35

PAGE 36

Figure2-3. Estimationoftheintensityfunctionbydierentprocedures.(A)Spiketraintoestimatetheringrate.(B)Normalizedbinnedspiketrain,withbinsizet=100ms.(C){(E)Estimatedringratebykernelsmoothing,fortheGaussianfunction,Laplacianfunctionandexponentialdecayfunction,respectively.Thekernelsizeparameterwas100msforallthreesmoothingfunctions. bydiscretizingtimeandassigningthenumberofspikesoccurringinthetimequantizationinterval(i.e.,thebin)tothetimeinstant.Ifthebinsizeislargecomparedtotheaverageinter-spikeintervalthetransformationprovidesacrudeyeteectiveestimateoftheinstantaneousringrate.Forconsistentintensityfunctionestimation,thebinneddataisnormalizedbythebinsize(i.e.,thesizeofthequantizationinterval),althoughthislaststepisoftenskipped.Fromasignalprocessingperspective,binningisatransformationwhichmapstherandomnessinthespiketraincontinuoustimestructuretorandomnessintheamplitude 36

PAGE 37

oftheintensityestimation.Theuseofbinninghasclearadvantagesintermsofintuitiveunderstandingandeaseofpracticaluse,andthetransformationofpointprocessesintodiscreterandomprocessesallowsforthewealthofconventionalstatisticaltimeseriesanalysisandprocessingmethodstobeused.Ontheotherhand,thediscretizationoftimeaectstheresolutionintimeofanyanalysissubsequentlyperformedtotheresultingsignal.Thismeansthatanytemporalinformationinthespikeswithinandbetweenbinsisdisregarded,limitingthetypeofanalysisthatcanbesubsequentlydone.Thisisespeciallyalarmingforneurophysiologyusewhenanumberofrecentstudiessuggestthatneuronsspiketimingprecisionisonthesub-millisecondrange[ Wagneretal. 2005 ; CarrandKonishi 1990 ],andtheactualspiketimesencodeadditionalinformation[ Hatsopoulosetal. 1998 ; Vaadiaetal. 1995 ; MainenandSejnowski 1995 ].Moreover,areminiscentproblemisatwhattime-scaletoanalyzethedata.Thatis,whatbinsizetochoose?Noticethatthehard-limitingnatureoftherectangularwindowusedmakethisanevenhardertask. 2.4.1.2KernelsmoothingKernelsmoothingisanotherapproachtointensityestimation,andseemsthemethodofchoiceinthepointprocessesliterature.Themainadvantagecomparedtobinningisthattheprecisionintheeventlocationispreservedandfullyincorporatedintheintensityestimation.Ofcourse,thereareotherwaystoestimatetheintensityfunctionofapointprocess,mainlythroughsmoothing.Thatis,byconvolvingsomesmoothfunctionwiththespiketrain(seenasasumoftime-shiftedimpulses).Figure 2-3 illustratessomeofthesemethods.MoreelaboratedmethodsincludedBayesianandsplinettingtonormalizedbinneddata[ Kassetal. 2003 ; Venturaetal. 2002 ].Inanycase,theimprovementsintermsofresolutionand/orintheestimationoftheintensityfunctionthesemethodsmightprovidearemadeattheexpenseofmuchhighercomputationcomplexity.Inaddition,likeforthebinnedspiketrains,anyfurthercomputationisdonewithoutaclearunderstanding 37

PAGE 38

ofthemathematicalandstatisticalpropertiesofthespace.Arealizationofapointprocesscanbeinterpretedasasignalinacontinuousparameterspacecomposedasasumofimpulsescenteredattheeventcoordinates.Thus,spiketrainscanbewrittenas, s(t)=NXm=1(ttm);(2{11)whereNisthenumberofspikesandtmthespiketimesintherecordinginterval,and()denotestheDiracdelta.Then,theestimatedintensityfunctionisobtainedbysimplyconvolvings(t)withthesmoothingkernelh,yielding ^(t)=NXm=1h(ttim):(2{12)Noticethatthesmoothingfunctionmustintegrateto1sothattheestimatedintensityfunctionisconsistent(integraloftheestimatedintensityfunctionmustequalthenumberofspikes).Itshouldberemarkedthatbinningcanbeposedintermsofkernelsmoothing.Specically,binningofspiketrainscanbeputasatwostepprocedure: 1. Quantizethespiketimestoaprecisionoft=2,wheretisthebinsize; 2. Convolvethesequenceoftime-shiftedimpulsescenteredatthequantizedspiketimeswitharectangularwindowofwidtht.Thisviewmakesitclearthetimediscretizationinbinning.ThetwoapproachesareillustratedinFigure 2-3 forseveralsmoothingfunctions. 2.4.1.3NonparametricregressionwithsplinesArecentlyproposedmethodforintensityestimationisnonparametricregressionwithsplines[ Venturaetal. 2002 ; Kassetal. 2003 ].Thebasicpremiseintheuseofsplinesisthesmoothnessoftheintensityfunctions,whichistranslatedintoconstraintswithregardstowhichtheoptimizationalgorithmndstheestimatedintensityfunctionasaweightedcombinationofsplines.Inparticular,in Kassetal. [ 2003 ]theBayesianadaptiveregressionsplines(BARS)methodwasutilizedsinceitautomaticallyndsthe\knots"wherethe 38

PAGE 39

splinesarejoinedtogether,thusrenderingthemethodsbasicallyparameter-free.Althoughthismethoddoesnotrequirethechoiceofbinorkernelsize,unliketheprevioustwoapproaches,itcanonlybeutilizedinoinestudiesandithasmuchhighercomputationalcomplexityduetotheMonteCarlooptimizationapproachutilizedbyBARS.Insteadofbeingappliedtothedatadirectlysplinesmoothingcan,alternatively,beappliedtothebinnedspiketraintosmooththeestimatedintensityfunction.Indeed,oneofthemostimportantconclusionsby Kassetal. [ 2003 ]isthemuchgreaterdataeciencyinintensityestimationbysmoothing. 2.4.1.4TrialaveragingAnapproachoftenemployedinconjunctionwitheitherofthepreviousmethodsistrialaveraging.Thismeansthatifmultiplerealizationsoftheexperimentaltrialareavailable,andstationaryisassumesbetweentrials,thenonecanaveragetheestimatedintensityfunctionacrosstrialsforimprovedstatisticalrobustness.Thewidelyusedperi-stimulustimehistogram(PSTH)isanexampleoftrialaveragedintensityestimationusingbinning.Toimplementtrialaveragingthespiketrainsarersttimealignedwithrespecttothetimeastimulusisapplied,andthenonecanrstestimatetheintensityfunctionforeachtrialandthenaverageovertrialsor,conversely,condensethespiketrainsofalltrialstogetherandestimatetheintensityfunctionattendingforthenormalizationbythenumberoftrials.Oneoftheadvantagesoftrialaveragingisthat,fromthelimittheoremforpointprocesses,thecombinedspiketrainsapproacharealizationofaPoissonprocess,evenifthetrueunderlyingpointprocesscontainshistory.Putdierently,theestimatedintensityfunctionismorelikelyreectthetrueinstantaneousringrate,asintuitivelyexpected.Ontheotherhand,themaindicultywithtrialaveragingistheassumptionofstationaritybetweentrials.Thatit,isassumesthattheprocessgivingrisetothespiketraindidnotchange.However,giventhemanyfactorstheinuencethebrainactivity 39

PAGE 40

(memory,learning,plasticity,attentionfocus,etc.)thisisoftenadicultassumptiontojustify,especiallywhenthenumberoftrialsislarge. 2.4.2MethodsforSpikeTrainAnalysisConsideringonlyoneneuroninthebrainatatime,theinformationitconveysisexpressedthroughchangesintheringpattern(rateortemporalprecision).Inthiscase,oneneedstomeasurethestatisticsoftheobservedspiketraintoinfertheneuronalstate.Theintensityfunctioncapturestheneuroninstantaneousringrateisthereforeofgreatimportance.Anyofthemethodsdescribedintheprevioussectioncanbeutilizedbut,asmentioned,binningisthemostcommonmethod.Theperi-stimulustimehistogram(PSTH)isoftenusedforspiketrainanalysis[ Perkeletal. 1967a ; GersteinandAertsen 1985 ]. 1 ThePSTHisparticularlyusefultostudyandverifythepresenceofmodulationsintheneuralactivity(forexample,intermsoftheringrate)withregardstoatime-lockingstimulus.Neuronsinthebraingreatlyinteractwithneighboringneuronsthroughtheremany(ontheorderofthousands)synapticconnections.Howeverthepreviousapproach,althoughwellsuitedstatistically,doesnotscaleproperlytothesimultaneousanalysisofmultiplespiketrains.Forthis,independenceishabituallyassumed.Consequently,howtondandmeasureassociationorcouplingsbetweenneuronsisanothermajorproblemforwhichseveralmethodshavebeenproposed.Thecross-correlation[ Perkeletal. 1967b ; DayanandAbbott 2001 ]isprobablythemostwidelyusedtechniquetomeasureinteractionsbetweentwospiketrains. 1 Sometimestheperi-eventtimehistogram(PETH)ismentionedintheliterature.Conceptually,thePSTHandPETHarethesamething,althoughthetimemarkforalignmentofthespiketrainsisgeneralinthelattercase. 40

PAGE 41

IfNAandNBdenotesthebinnedspiketrains,thecross-correlation(orcorrelogram)isdenedas, RA;B[l]=EfNA[n]NB[nl]g'1 MMXn=lNA[n]NB[nl]:(2{13)whereMisthetotalnumberofbinsandNA[n],NB[n]arethenumberofspikesinthenthbin,respectively.Cross-correlationasastatisticalmeasureofsimilaritybetweenspiketrainswas\imported"fromrandomprocessesandcurrentlycanonlybeappliedtothebinnedspiketrains.Moreover,theexpectationimpliesaveragingovertimewhichlimitsitsusefulnessasadescriptoroftheevolutionofcorrelationasafunctionoftime,andintrinsicallyrequiresstationarityandergodicityovertheaveragingtimeinterval.Toaddressnon-stationary,cross-correlationisaveragedoversmallwindowsoftimewhichfurtherreducethetimeresolutionatthesacriceofstatisticalreliability.Thelimitedtemporalresolutionofcross-correlationleadtotheuseofothermethods.TheJPSTH GersteinandPerkel [ 1969 ]; Aertsenetal. [ 1989 ]; GersteinandPerkel [ 1972 ]isanotherwidelyusedtooltocharacterizetheevolutionofsynchronyovertimebetweentwoneurons.Thefundamentalideaisasmoothedtwo-dimensionalscatterdiagramoftheneuronalringsfromoneneuronwithrespecttotheother,andtime-lockedtoastimulus.AlthoughtheaveragingovertimeintheJPSTHisremoved(apartfromsmoothing),andthusprovidesmoredetailedinformationabouttime-dependentcross-correlationwithrespecttothestimulus,thisapproachrequirestrialaveraging.Therefore,oneneedstoassumestationaritybetweentrialswhich,forthesamereasonsgivenpreviously(Section 2.4.1.4 ),isanunrealisticassumption.Furthermore,theapproachrapidlybecomesunmanageableformorethanjustafewneuronssincetheanalysisisdoesinpairs(e.g.,16neuronsrequires120JPSTHplots).Thejointintervalhistogram(JIH)[ Rodiecketal. 1962 ]isasimilartooltoidentifycorrelationsbetweeninter-spikeintervalsforwhichsimilarconsiderationsmaybemade. 41

PAGE 42

Otherspiketrainanalysismethodsinthetimedomainincludeunitaryevents[ Grunetal. 2002a b ]andthegravitytransform[ Gersteinetal. 1985 ; GersteinandAertsen 1985 ].Unitaryeventsisastatisticalmethodtodetectcoincidentspikeactivityabovechance.Itdoessobycomparingthenumberofcoincidentspikeswiththeexpectednumberbychancefortheestimated\local"ringrate.However,likeothermethods,itissensitivetobinningandemploysalargemovingwindowanalysisforstatisticalreliability.Thegravitytransformtacklessomeoftheseproblems.Mainlybecauseitdoesnotrequirebinningandprovidesawaytovisualizetheevolutionofsynchronyovertime.However,itlacksastatisticalbaselinewhichlimitstheknowledgethatcanbeinferredfromtheanalysis.Severalmethodsforanalysisinthefrequency-domainhavealsobeenproposed.Forinstance,thepartialdirectedcoherence(PDC)[ BaccalaandSameshima 1999 ; SameshimaandBaccala 1999 ],andthemethodby Hurtadoetal. [ 2004 ].PDCemploysmultivariatetimeseriesanalysistotogetherwiththeideasbehindtheGrangercausalityconcepttoinferinter-dependenciesbetweenneurons.But,duetothetransformationintothefrequencydomain,thesemethodsoperateoverwindowsofdata.Therefore,theyrequirestationarityfortheanalysistobevalid,andthetimeresolutionisreducedasaconsequence. 2.4.3ProcessingofSpikeTrainsProcessingofspiketrainsisofgreatinterestfromaneurophysiologicalperspectivebutevenmoreimportantfromanengineeringpointofview.Mainlybecauseofthetremendousimplicationsforneuralprostheses,andinparticularfortheapplicationsinbrain-machineinterfaces(BMI).Intherecentyearscomputationwitharticialneuralnetworksofspikingneuronshasalsoemergedasanengineeringapplicationwheretoolstodosignalprocessingwithspiketrainsarenaturallyofgreatimportance.Anexampleistheemergingconceptofliquidstatemachines(LSM)proposedby Maassetal. [ 2002 ].LSMsusetheprinciples 42

PAGE 43

ofhigh-dimensionaldynamicalsystemstoperformcomputationwithrecurrentneuralnetworksofspikingneurons.Anotherexamplearespikingneuralnetworks(SNN)currentlybeingstudiedandappliedtolargenumberofproblems.Usingprocessingelementsthatmorecloselyresembletheactualneurons,thesenetworksextendthecomputationparadigmofarticialneuralnetworksinawaythatmorecloselymimicsthebrain[ MaassandBishop 1998 ; GerstnerandKistler 2002 ].Therearefourmainapproachescurrentlyutilizedforprocessingofspiketrains: 1. Linear/nonlinearmodels; 2. Probabilisticmodels; 3. Statespacemodels;and 4. Volterra/Wienermodels.Theliteratureonthesemethodswillnowbequicklyreviewed.Fromthereviewitwillbeclearlyshownthat,assaidabove,processingofspiketrainshasbeenlargelymotivatedandappliedtoneuralprostheses,whichisalsothe(long-term)motivationforthiswork.Inspiteofthat,wehopethatthereadermayrealizethewideimplicationsofthisworkbeyondthisrealmofproblems. 2.4.3.1Linear/nonlinearmodelsThesemodelsarethemostdirectapproachtowardsprocessingofspiketrains,sinceitutilizescurrenttimeseriesprocessingtechniques.Sothatthesemodelscanbedirectlyapplicablethespiketrainmustbetransformedintoadiscrete-timesignal,andthestandardapproachistoutilizebinning,sinceasexplainedearlierbinningimplementsthismapping.Itmustberemarkedthatthesecasesarepredicatedontheideathattheinformationtobeextractedisencodedinmodulationsoftheringrates[ Nicolelis 2003 ].Indeed,mostresultsreportedintheliteratureutilizebinnedspiketrainswithbinsize100ms,correspondingtoringrateestimation.ThesemodelshaveprovenparticularimportantinBMIssincetheoutputisadiscrete-timesignalofthemovementvariables.Atthecurrentstageofresearch,inmostoftheBMIexperimentalparadigmsthedesiredresponse(intendedmovement)isavailable.Therefore,theseparadigmslend 43

PAGE 44

themselvestosupervisedlearning,wheretheproblemiswellformulatedasasystemmodelingtask.TheWienerlter[ Haykin 2002 ]isthelinearmodeltypicallyemployed,havingbeenextensivelyutilizedinmanystudiesintheliterature[ Chapinetal. 1999 ; Wessbergetal. 2000 ; Carmenaetal. 2003 ].Fornonlinearmodeling,neuralnetworkshavebeenused.See Kimetal. [ 2006 ]; Sanchezetal. [ 2003 ]; Kim [ 2005 ]; Sanchez [ 2004 ]foracomparisonofmethods.Itisimportanttoremarkthat,becauseforthescenariosenvisionedforBMIsthedesiredresponsewillnotbeavailable,someattemptshavealsobeenmadetomovetowardstheuseofunsupervisedlearningmodels[ Darmanjianetal. 2007 ]. 2.4.3.2ProbabilisticmodelsProbabilisticmodelsattempttointerpretspiketrainscontentfromsomeprobabilisticmodel,oftenspecictoagiventask.Theworkby Georgopoulosetal. [ 1982 1988 ]representsalandmarkinspiketraindecoding,whenGeorgopoulossuggestedtheconceptofpopulationcoding.Georgopoulosshowedinacenter-outtaskthatifeachneuronisaassigneda\tunningcurve,"basicallydenotingthedistributionofthemovementangleasafunctionoftheneuron'sringrate,andbyaveragingacrossapopulationofneuronsahighprecisionisattained.Perhapsthemostimportantcontributionofthisworkwastoprovideevidencefortheimportanceofagroupofneuronsinconveyinginformationinareliableandeectivemanner.AnotherprobabilisticworthyofremarkistheBayesianapproachbyShenoy'sgroup(see,forexample, Shenoyetal. [ 2003 ]).Basedonthespecicexperimentalparadigm,astatespacemodelwithtransitionsdecodedbymaximumaposteriorprobability,wasproposed.Thisimpliedtheestimationofthemarginaldistributionsforeachneuronfromdata.ThesedistributionswerethencombinedusingBayes'theoremunderanindependenceassumption.Ineitheroftheseapproachesindependenceamongneuronsneedstobeassumed.Aswehadremarkedearlier,thisisoneofthemajorlimitationsofstatisticalmethods. 44

PAGE 45

2.4.3.3StatespacemodelsAnalternativetotheabovemethodsistoutilizestatespacemodelstogetherwithsequentialestimation.TheseapproachesmakeuseofBayesiantrackingtoprobabilisticallyinfertheevolutionofastatesequenceovertime.Roughlyspeaking,thisissimilartotheideaofhiddenMarkovmodels(HMMs)butforacontinuousstatespace.ThesimplestexampleofthismethodologyistheuseofKalmanlteringappliedtothebinnedspiketrains[ Wuetal. 2004 ].Kalmanlteringappliedtospiketrainprocessinghasnumerouslimitations:boththemodeldescribingtheevolutionthroughthestatespaceandthereadoutarelinear,alldistributionsareassumedtobeGaussian,andisappliedtobinnedspiketrainsonly.Undersimilarassumptionsbutappliedtothespiketrainsdirectly 2 hasbeenalsoproposedbyseveralgroups Edenetal. [ 2004 ]; Brownetal. [ 2001b ].Noticethatinthelattertheforward(encoding)modelisneededandhasbeenassumedtobeGaussian.Toavoidtheseassumptions,inrecentstudiesparticleltershavebeenusedwhichallowforarbitraryforwardmodels,non-linearevolutionthroughthestatespaceandnon-Gaussiandistributions.ParticleltercreatesaprobabilisticstatespacemodelforthedecoderwhichisrecursivelyandcontinuouslyadaptedthroughaBayesianapproachbasedonthelatestobservation.However,updateoftheprobabilisticmodelusedMonteCarlosequentialestimationwhichisnotonlyextremelycomputationalintensiveduetotherandomsamplingofthespace,butalsorequiresaprioriknowledgeofpropertiesoftheneuronsbeingmeasured. 45

PAGE 46

. ForwardVolterrakernels + Threshold Feedbackmodel Systemnoisen(t) ... Figure2-4. DiagramoftheVolterra/Wienermodelforsystemidentication.AMISO(multiple-inputsingle-output)congurationisshown.ForMIMOcongurations,basically,theMISOstructureisrepeatedforeachoutput. 2.4.3.4VolterraandWienermodelsSometimesthespiketrainanalysisorsignalprocessingproblemathandscanformulatedasasystemidenticationtask.Sincethebrainisknowntobehighlynonlinearthenamodelwithnonlinearmodelingabilityneedstobeutilized,suchasaneuralnetwork.However,iftheoutputistobeaspiketrainthentheVolterra/Wienermodelshavebeenutilized[ Marmarelis 2004 1993 ; Songetal. 2007 ].Thisisparticularlyimportantforthestudyofspecicneuralsystemsbyestimatingtheinput-outputmodelfromrecordedspiketrains,orinneuralprosthesesaimingtoreplaceoraidthefunctioningofafailingneuralstructure.Figure 2-4 depictsthearchitectureoftheVolterra/Wienermodel. 2 Morecorrectlysaid,thesequentialmethodsworkwithabinaryrepresentationofthespiketrain,equivalenttobinningwithaverysmallbinsize(1ms),whichcorrespondstoaBernoullirandomprocess. 46

PAGE 47

Atime-invariantsystemcanbeexpressedintermsoftheVolterraseriesas y(t)=h0+ZRh1(1)x(t1)d1+ZR2h2(1;2)x(t1)x(t2)d1d2+ZR3h3(1;2;3)x(t1)x(t2)x(t3)d1d2d3+=1Xn=0ZRnhn(1;:::;n)x(t1)x(tn)d1d3=1Xn=0Hn[x(t)];(2{14)whereH0[x(t)]=h0and Hn[x(t)]=ZRnhn(1;:::;n)x(t1)x(tn)d1:::dn(2{15)isthenthorderVolterrafunctional.OnecanthinkoftheVolterraseriesasaTaylorserieswithmemory.ThefunctionshnarecalledtheVolterrakernelsofthesystemandarecausal;thatis,hn(1;:::;n)=0,ifanyi<0,i=1;:::;n.Ingeneral,thesekernelsarenotuniqueforagivenoutput.However,ifsymmetryisimposedwithrespecttopermutationsofthei,thatis,ifhn(:::;i;:::;j;:::)=hn(:::;j;:::;i;:::),foralli;j=1;:::;n,thenitcanbeshownthattheVolterraseriesexpansionisunique.IntheVolterraseriestheoutputoftwodistinctfunctionalsisnot,ingeneral,orthogonal(i.e.,uncorrelated).HoweverintheWienerseriesthefunctionalsformacompleteorthonormalbasis.IntermsoftheWienerseriesthesystemcanbeexpressedas y(t)=1Xn=0Gn[x(t)];(2{16)whereGn[x(t)]aretheWienerfunctionals.ThecharacterizingfeatureoftheWienerfunctionalsistheirorthonormalityforzero-meanwhiteGaussiandistributedinput. 47

PAGE 48

TheWienerandVolterraseriesaretwoequivalentapproachestocharacterizeasystem.However,theorthogonalityofthebasisfunctionalscanbeusedto\isolate"eachofthetermintheseries,thisfacilitatingtheestimationofthecorrespondingkernel.Infact,iftheinputiszero-meanwhiteGaussiandistributed,theleadingWienerkernelscanbeobtaineddirectlybycross-correlationsbetweentheinput(atvariouslags)andoutput.Moreover,thereisamathematicalrelationbetweenthefunctionalsofthetworepresentations.Forneurophysiologicalstudies,however,theVolterraseriesasbeenmorewidelyusedsincetheestimatedkernelsprovideabetteranalyticaldescriptionoftheneurophysiologicalsystem[ Marmarelis 2004 ].Thisapproachhasverypowerfulsystemmodelingcapability.However,inpractice,italsopresentseveraldiculties:averylargenumberofcoecientsneedtobeestimated,especiallyforhigherorderkernels,thusneedinglargevolumesofdatafortheestimationofcross-correlations,anditestimationrequiresthattheinputiszero-meanwhiteGaussiandistributed.Moreover,thisapproachcanonlybeusedifthesystemisassumedstationaryandtime-invariant. 48

PAGE 49

CHAPTER3INNERPRODUCTSFORPOINTPROCESSES,ANDINDUCEDREPRODUCINGKERNELHILBERTSPACESAsmotivatedinChapter 1 ,thefundamentaloperatorforsignalprocessingisaninnerproductdenition.Inthischapterweintroduceseveralinnerproductsforpointprocess.Moreimportantthantheinnerproductdenitionsthemselveswewanttoillustratehowkernelsforpointprocessescanbedenedfollowingtodierenttwoapproaches.Afterwards,weproveseveralpropertiesofthekernelsdenedtodemonstratethatthekernelsarewell-posedandeachinducesacorrespondingreproducingkernelHilbertspace(RKHS)forcomputation.TherelationbetweentheRKHSinducedbyoneofthesekernelsandothersisanalyzedsinceitprovidesinsightonthefullpotentialofthesekernelsmaybeexplored.Theproblemofhowtoestimatethesekernelsfromrealizationsisalsoconsidered. 3.1InnerProductforEventCoordinatesDenotethemtheventcoordinateinarealizationofthepointprocessindexedbyi2Nastim2T,withm2f1;2;:::;NigandNithenumberofeventsintherealization.Tosimplifythenotation,however,theexplicitreferencetothepointprocessindexwillbeomittedifitisnotrelevantorobviousfromthecontext.Thesimplestinnerproductthatcanbedenedforpointprocessesoperateswithonlytwoeventcoordinatesatatime.Inthegeneralcase,suchaninnerproductcanbedenedintermsofakernelfunctiondenedonTTintothereals,withTtheeventspacewheretheeventsoccur.Letdenotesuchakernel.Conceptually,thiskerneloperatesinthesamewayasthekernelsoperatingondatasamplesinmachinelearning[ Scholkopfetal. 1999 ]andinformationtheoreticlearning[ Prncipeetal. 2000 ].Althoughitoperatesonlywithtwoevents,itwillplayamajorrolewheneverweoperatewithcompleterealizationsofpointprocesses.Indeed,theestimatorforoneofthepointprocesskernelsdenednextreliesonthissimplekernelasanelementaryoperationforcomputationorcompositeoperations. 49

PAGE 50

TotakeadvantageoftheframeworkforstatisticalsignalprocessingprovidedbyRKHStheory,isrequiredtobeasymmetricpositivedenitefunction.BytheMoore-Aronszajntheorem[ Aronszajn 1950 ],thisensuresthatanRKHSHmustexistforwhichisareproducingkernel.TheinnerproductinHisgivenas (tm;tn)=h(tm;);(tn;)iH=hm;niH:(3{1)wheremistheelementinHcorrespondingtotm(thatis,thetransformedeventcoordinate).Sincethekerneloperatesdirectlyoneventcoordinatesand,typically,itisundesirabletoemphasizeeventsinthisspace,thekernelisfurtherrequiredtobeshift-invariant.Thatis,forany2R, (tm;tn)=(tm+;tn+);8tm;tn2T:(3{2)Hence,thekernelisonlysensitivetothedierenceoftheargumentsand,consequently,wemaywrite(tm;tn)=(tmtn).Foranysymmetric,shift-invariant,andpositivedenitekernel,itisknownthat(0)j()j. 1 Thisisimportantinestablishingasasimilaritymeasurebetweeneventcoordinatessince,asusual,aninnerproductshouldintuitivelymeasuresomeformofinter-dependence.However,theconditionsposeddonotrestrictthisstudytoasinglekernel.Onthecontrary,anykernelsatisfyingtheaboverequirementsistheoreticallyvalidandunderstoodundertheframeworkproposedhere,althoughthepracticalresultsmayvary. 1 ThisisadirectconsequenceofthefactthatsymmetricpositivedenitekernelsdenoteinnerproductsthatobeytheCauchy-Schwarzinequality. 50

PAGE 51

Anexampleofafamilyofkernelsthatcanbeused(butnotlimitedto)aretheradialbasisfunctions[ Bergetal. 1984 ], (tm;tn)=exp(jtmtnjp);tm;tn2T;(3{3)forany0
PAGE 52

eventcoordinates.Thismeansthatsuchpointscannotbemappedbacktotheinputspacedirectly.Thisrestrictionhoweverisgenerallynotaproblemsincemostapplicationsdealexclusivelywiththeprojectionsofpointsinthespace,andifarepresentationintheinputspaceisdesireditmaybeobtainedfromtheprojectiontothemanifoldoftransformedinputpoints.Thekernelsdiscussedthisfaroperatewithonlytwoeventcoordinates.Asincommonlydoneinkernelmethods,kernelsoneventcoordinatescanbecombinedtodenekernelsthatoperatewithwholerealizationsofpointprocesses.Supposethatoneisinterestedindeningakernelonpointprocessrealizationstomeasuresimilarityintheeventpatterns[ ChiandMargoliash 2001 ; Chietal. 2007 ].Thiskernelcouldbedenedas V(pi;pj)=8>>>><>>>>:maxl=0;1;:::;(NiNj)NjXn=1(tin+ltjn);NiNjmaxl=0;1;:::;(NjNi)NiXn=1(tintjn+l);Ni
PAGE 53

Ratherthandoingthisdirectly,inthissection,generalinnerproductsforpointprocessesaredenedfromtheintensityfunctions,whicharefundamentalstatisticaldescriptorsofpointprocesses.Thisbottom-upconstructionofthekernelsforpointprocessesisunlikethepreviousapproachtakenintheprevioussectionandisrarelytakeninmachinelearning,butitprovidesdirectaccesstothepropertiesofthekernelsdenedandtheRKHStheyinduce.Thereisagreatconceptualdierencebetweenthetwoapproachestodesigninnerproductsforpointprocesses:fromkernelsoneventcoordinatesandfromconditionalintensityfunctions.Intherstcase,theinnerproductisdeneddirectlyforrealizationsofpointprocessesandthereforethefocusisplacedintheestimatorsfromdata,whereasinthesecondcasetheinnerproductisprimarilyastatisticaldescriptorforwhichtheproblemofestimationfromrealizationsneedstoaddressedlater.Althoughbothapproachesmayplayaveryimportantroleinspiketrainmethods,inthisdissertationwefocusofthesecondcase,presentedinthissection,sincewefeelitisamoreprincipledmethodology. 3.2.1LinearCross-IntensityKernelsIngeneral,tocompletelycharacterizeapointprocesstheconditionalintensityfunction(tjHt)isneeded,wheret2T=[0;T]denotesthetimecoordinateandHtisthehistoryoftheprocessuptotimet.Considertwopointprocesses,pi;pj2P(T),withi;j2N,anddenotethecorrespondingconditionalintensityfunctionsbypi(tjHit)andpj(tjHjt),respectively.AssumingthepointprocessesaredenedinaniteparameterspaceT,andtheboundednessoftheconditionalintensityfunctions,wehavethat ZT2(tjHt)dt<1:(3{7)Inwords,conditionalintensityfunctionsaresquareintegrablefunctionsonTand,asaconsequence,arevalidelementsofanL2(T)space.Obviously,thespacespannedbytheconditionalintensityfunctions,denotedL2(pi(tjHit);t2T),iscontainedinL2(T). 53

PAGE 54

Therefore,wecaneasilydeneaninnerproductoftheconditionalintensityfunctionsinL2(pi(tjHit);t2T)astheusualinnerproductinL2(T), I(pi;pj)=pi(tjHit);pj(tjHjt)L2(T)=ZTpi(tjHit)pj(tjHjt)dt:(3{8)Althoughwedenedtheinnerproductinthespaceofconditionalintensityfunctions,itisineectafunctionofthetwopointprocesses,andthusisakernelfunctioninthespaceofpointprocessesP(T).Theadvantageindeningtheinnerproductintermsoftheconditionalintensityfunctionsisthattheresultingkernelincorporatesthestatisticsofthepointprocessesdirectly.Moreover,thedenedkernelcanbeutilizedwithanypointprocessmodelsincetheconditionalintensityfunctionisacompletecharacterizationofthepointprocess[ CoxandIsham 1980 ].NoticehoweverthatEquation 3{8 denotesafunctionalinnerproductdenition,inthesensethattheconditionalintensityfunctionsareingeneralwelldenedfunctionsoftonlyforparticularrealizationsofthepointprocesses.Thedependenceoftheconditionalintensityfunctionsonthewholehistoryoftheprocessrendersestimationofthepreviouskernelintractablefromnitedata,asalmostalwaysoccursinapplications.Apossibilityistoconsidersimpliedpointprocessmodelswhichreducethenumbersofparametersneededtocharacterizetheconditionalintensityfunctions.Onecanconsider,forexample,that (tjHt)=(t;tt);(3{9)wheretisthespiketimeimmediatelyprecedingt.ThisrestrictedformgivesrisetoinhomogeneousMarkovinterval(IMI)processes[ KassandVentura 2001 ].Inthiswayitispossibletoestimatetheconditionalintensityfunctionsfromrealizationsofthepointprocesses,andthenutilizetheaboveinnerproductdenitiontooperatewiththem.Thispointprocessmodelisveryinterestingitissimpleyetgeneralenoughformodelingbeyond 54

PAGE 55

renewalprocesses,butsinceweaimtocomparethegeneralprinciplespresentedstartingfrommoretypicalapproachesitwillnotbepursuedinthispaper.Needlesstosay,thesameprinciplesdiscussedherecanbedirectlyutilizedinapplications,althoughattheexpenseofcomputationalcomplexityinthecomputationoftheinnerproductasdiscussedlater.AnotherwaytodealwiththememorydependenceistotaketheexpectationoverthehistoryoftheprocessHtwhichyieldssimplytheintensityfunctionsolelydependingontime.Thatis, pi(t)=EHitpi(tjHit):(3{10)Thisexpressionisadirectconsequenceofthegenerallimittheoremforpointprocesses[ Snyder 1975 ]which,asintroducedinChapter 2 ,statesthatifmultiplepointprocessesarecombinedtheyconvergetowardsaPoissonpointprocess.AnequivalentbutalternateperspectiveistomerelyassumedirectlyPoissonprocessestobeareasonablemodelfortheproblemathands.Thedierencebetweenthetwoperspectivesisthatinthesecondcasetheintensityfunctionscanbeestimatedfromsinglerealizationsinaplausibleandsimplemanner.Ineitherperspective,thekernelbecomessimply I(pi;pj)=ZTpi(t)pj(t)dt:(3{11)Startingfromthemostgeneraldenitionofinnerproductseveralkernelsfromconstrainedformsofconditionalintensityfunctionscanbeproposedforuseinapplications.OnecanthinkthatthedenitionofEquation 3{8 givesrisetoafamilyofcross-intensity(CI)kernelsdenedexplicitlyasaninnerproduct,asisimportantforsignalprocessing.SpecickernelsareobtainedfromEquation 3{8 byimposingsomeparticularformonhowtoaccounttothedependenceonthehistoryoftheprocessand/orallowingforanonlinearcouplingbetweenspiketrains.Twofundamentaladvantagesoftheconstructionmethodologyisthatitispossibletoobtainacontinuousfunctionalspacewherenobinning 55

PAGE 56

isnecessaryandthatthegeneralityoftheapproachallowsforinnerproductstobecraftedtotaparticularproblemthatoneistryingtosolve.Fromthesuggesteddenitions,thememorylesscross-intensity(mCI)kerneldenedinEquation 3{11 clearlyadoptsthesimplestformsincetheinuenceofthehistoryoftheprocessisneglectedbythekernel.Interestingly,thissimplekerneldenesanRKHSthatisequivalenttocross-correlationanalysissowidespread,forexample,inspiketrainanalysis[ Paivaetal. 2008 ],butthisderivationclearlyshowsthatitisthesimplestofthecases.Still,themCIkernelserveswellasanexampleoftheRKHSframeworksinceitprovidesabroadperspectivetoseveralotherworkspresentedintheliteratureandsuggestshowmethodscanbereformulatedtooperatedirectlywithpointprocesses.Inanycase,aswillbeshowninChapters 6 and 7 ,thederivedalgorithmsaretypicallyapplicableforanykernelonpointprocesses. 3.2.2NonlinearCross-IntensityKernelsThekernelsdenedintheprevioussectionarelinearoperatorsinthespacespannedbytheconditionalintensityfunctionsandaretheonesthatrelatethemostwiththepresentanalysismethods.However,pointprocesskernelscanbemadenonlinearbyintroducinganonlinearweightingbetweentheconditionalintensityfunctionsintheinnerproduct.Withthisapproachadditionalinformationcanbeextractedfromthedatasincethenonlinearityimplicitlyincorporatesinthemeasurementhigher-ordercouplingsbetweentheestimatedconditionalintensityfunctions.Thisisofespecialimportanceforthestudyofdoubly-stochasticpointprocesses,sincethenonlinearweightingkernelbasicallyaidsthepointprocesskerneltosensethehigher-odermomentsoftheintensityprocess.Intheexampleshownweshallconsider,foreaseofexposition,theintensityfunctionsdirectly(thatis,thememorylesscase).However,weremarkthatthemethodologyfollowedcanbeeasilyextendedtogeneralpointprocessmodelsasabove. 56

PAGE 57

ByanalogytohowtheGaussiankernelisobtainedfromtheEuclideannorm,wecandeneasimilarkernelforspiketrainsas I(pi;pj)=exp"pipj2 2#;(3{12)whereisthenonlinearweightingkernelsizeparameterandthenormnaturallyinducedbyalinearpointprocesskernel,pipj=q pipj;pipj,wasused.Thiskernelisclearlynonlinearonthespaceoftheintensityfunctions.Ontheotherhand,thenonlinearmappinginthiskerneldoesnotoperatedirectlyontheintensityfunctionsbutontheirnormandinnerproductandthushavereduceddescriptiveabilityonthecouplingoftheirtimestructure.AnalternatenonlinearCIkerneldenitionforpointprocessesis Iy(pi;pj)=ZTKpi(t);pj(t)dt;(3{13)whereKisasymmetricpositivedenitekernelwithkernelsizeparameter.Theadvantageofthisdenitionisthatthekernelmeasuresthepossiblynonlinearcouplingbetweenthepointprocesstimestructureexpressedintheintensityfunctions.ToverifythisconsiderasanexamplethattheGaussiankernel,K(x)=exp[x2=(22)],wasutilizedinthecomputationofthepointprocesskernel.TheGaussiankernelhasTaylorseriesexpansion K(x)=1Xn=0(1)n 2n2nn!x2=1x2 22+x4 84:::(3{14)Thus,thispointprocesskerneldependsonthenormofthespiketrains(denedthroughtthemCIkernel)butalsoonhigher-ordermomentsofthedierencebetweentheintensityfunctions.IntheremainderofthisdissertationweshallrefertothenonlinearCIkernelinEquation 3{13 asthenCIkernel. 57

PAGE 58

3.3PropertiesofCross-IntensityKernels 3.3.1PropertiesofLinearCross-IntensityKernelsInthissectionwepresentsomerelevantpropertiesofthelinearCIkernelsdenedinthegeneralforminEquation 3{8 .Inadditiontotheknowledgetheyprovide,theyarenecessaryforestablishingthattheCIkernelsarewelldened,induceanRKHSwiththenecessarymathematicalstructureforcomputation,andaidintheunderstandingofthefollowingsections.ThissectiondealsexclusivelywithCIkernelslinearinthespaceofconditionalintensityfunctions,unlessexplicitlystated,andthuslinearityshallbeimplicit.NonlinearCIkernelsarestudiedinthenextsection. Property3.1. ThelinearCIkernelsaresymmetric,non-negativeandlinearoperatorsinthespaceoftheintensityfunctions.BecausetheCIkernelsoperateonelementsofL2(T)andcorrespondtotheusualdotproductfromL2,thispropertyisadirectconsequenceofthepropertiesinherited.Morespecically,thispropertyguarantiestheCIkernelsarevalidinnerproducts. Property3.2. Foranysetofn1pointprocesses,theCIkernelmatrixI=266666664I(p1;p1)I(p1;p2):::I(p1;pn)I(p2;p1)I(p2;p2):::I(p2;pn)............I(pn;p1)I(pn;p2):::I(pn;pn)377777775;issymmetricandnon-negativedenite. Proof. ThesymmetryofthematrixresultsimmediatelyfromProperty 3.1 .Bydenition,amatrixisnon-negativedeniteifandonlyifaTIa0,foranyaT=[a1;:::;an]withai2R.So,wehavethat aTIa=nXi=1nXj=1aiajI(pi;pj);(3{15) 58

PAGE 59

which,makinguseofthegeneraldenitionforCIkernels(Equation 3{8 ),yields, aTIa=ZT0@nXi=1aipi(tjHit)1AnXj=1ajpj(tjHjt)!dt=*nXi=1aipi(jHit);nXj=1ajpj(jHjt)+L2(T)=nXi=1aipi(jHit)2L2(T)0:(3{16) Throughtheworkof Moore [ 1916 ]andduetotheMoore-Aronszajntheorem[ Aronszajn 1950 ],thefollowingtwopropertiesresultascorollariesofProperty 3.2 Property3.3. CIkernelsaresymmetricandpositivedenitekernels.Thus,bydenition,foranysetofn1pointprocessesandcorrespondingnscalarsa1;a2;:::;an2R, nXi=1nXj=1aiajI(pi;pj)0:(3{17) Property3.4. ThereexistsanHilbertspaceforwhichaCIkernelisareproducingkernel.Actually,Property 3.3 canbeobtainedexplicitlybyverifyingthattheinequalityofEquation 3{17 isimpliedbyEquation 3{15 andEquation 3{16 intheproofofProperty 3.2 .Property 3.2 ,Property 3.3 andProperty 3.4 areequivalentinthesensethatanyofthesepropertiesimpliestheothertwo.Inourcase,Property 3.2 isusedtoestablishtheothertwo.Themostimportantconsequenceoftheseproperties,explicitlystatedthroughProperty 3.4 ,isthataCIkernelinducesauniqueRKHS,denotedingeneralbyHI.IntheparticularcaseofthemCIkerneltheRKHSisdenotedHI. Property3.5. TheCIkernelsverifytheCauchy-Schwarzinequality, I2(pi;pj)I(pi;pi)I(pj;pj)8pi;pj2P(T):(3{18) 59

PAGE 60

Proof. Considerthe22CIkernelmatrix,I=264I(pi;pi)I(pi;pj)I(pj;pi)I(pj;pj)375:FromProperty 3.2 ,thismatrixissymmetricandnon-negativedenite.Hence,itsdeterminantisnon-negative[ Harville 1997 ,pg.245].Mathematically,det(I)=I(pi;pi)I(pj;pj)I2(pi;pj)0;whichprovestheresultofEquation 3{18 3.3.2PropertiesofNonlinearCross-IntensityKernelsWenowprovethatthenonlinearCIkernelsdenedinSection 3.2.2 arewelldened.Thatis,theydenoteinnerproductsinsomeRKHSofpointprocesses.Thetwofundamentalrequirementsarethatthepointprocesskernelsaresymmetricandpositivedeniteinthespaceofpointprocesses. Property3.6. ThepointprocesskernelI(denedinEquation 3{12 )isasymmetricpositivedenitekernelofpointprocesses. Proof. Thisfunctionisobviouslysymmetricasthesymmetryisinheriteddirectlyfromthepropertiesofthenorm.InlightofTheorem2.2inChapter3of Bergetal. [ 1984 ],toprovethefunctionispositivedeniteitsucestoprovethatthenormofthedierencebetweentwointensityfunctionsisnegativedenite.Bydenition,arealfunctionisnegativedeniteifandonlyifitissymmetricand nXi=1nXj=1cicj(xi;xj)0;(3{19)foralln2,fx1;:::;xngXandc1;:::;cn2KwithPni=1ci=0.Letn2.Foralln,considerthesetofpointprocessesfp1;:::;pngP(T),andc1;:::;cn2Rsuchthat 60

PAGE 61

Pni=1ci=0.Sincethefunctionissymmetricitremainstoprovethat nXi=1nXj=1cicjpipj20:(3{20)UsingthenorminducedbyoneofthelinearCIkernels,yields nXi=1nXj=1cicjpipj;pipj=nXi=1nXj=1cicjhpi;pii2pi;pj+pj;pj=nXi=1cihpi;pii!nXj=1cj!| {z }=02*nXi=1cipi;nXj=1cjpj++nXi=1ci!| {z }=0nXj=1cjpj;pj!=2nXi=1cipi0;(3{21)sincethenormis,bydenition,alwayspositive. Property3.7. ForanysymmetricpositivedenitekernelK,thenonlinearCIkernelIy(denedinEquation 3{13 )isasymmetricpositivedenitekernelofpointprocesses. Proof. ThesymmetryofIyisadirectconsequenceofthesymmetryofthekernelK.Denotebyfp1;:::;pngP(T)anysetofnpointprocesses,withn2,andconsidercoecientsa1;:::;an2R.ToprovethatIyispositivedeneoneneedstoshowthat nXi=1nXj=1aiajIy(pi;pj)0:(3{22)SubstitutingthedenitionofIyinthepreviousequationyields, nXi=1nXj=1aiajIy(pi;pj)=nXi=1nXj=1aiajZTK(pi(t);pj(t))dt=nXi=1nXj=1aiajZTDpi(t);pj(t)EHKdt;(3{23) 61

PAGE 62

wherethekernelKwassubstitutedbyitsinnerproductinthecorrespondingRKHSHK,andpi(t)denotesthetransformationoftheintensityfunctionvalueattimet(theargumentofK)intoHK.Utilizingthelinearityoftheintegralandtheinnerproductoperatorleadsto nXi=1nXj=1aiajIy(pi;pj)=ZT*nXi=1aipi(t);nXj=1ajpj(t)+HKdt=ZTnXi=1aipi(t)2HKdt0;(3{24)whichprovestheproperty. Thefollowingpropertyfollowsimmediatelyasacorollary: Property3.8. ThenonlinearCIkernels,IandIy,each (i) InduceanRKHS, (ii) Giverisetonon-negativedeniteGrammatrices,and (iii) VeriestheCauchy-Schwarzinequality.Thesearebecause,asstatedintheprevioussection,Property 3.8 (i),Property 3.8 (ii)andthepositivedenitenessofthekernelareequivalentfacts.Then,asshownintheproofofProperty 3.5 ,theCauchy-Schwarzfollows. 3.4EstimationofCross-IntensityKernelsTheproblemofestimatingthepreviouslydenedcross-intensitykernelsisnowconsidered.Recallthatthisproblemonlyposesitselfforkernelsdenedintermsofthestatisticaldescriptorsofpointprocesses,whereaspointprocesseskernelsbuiltfromkernelsoneventcoordinatesareineectestimators.Thiswillbeclearfromourpresentationand,infact,therelationshipbetweenperspectiveswillbeobservedinoneofthecases. 3.4.1EstimationofGeneralCross-IntensityKernelsFromthepointprocesskerneldenitions,isshouldbeclearthatforevaluationofCIkernelsoneneedstoestimatersttheconditionalintensityfunctionfromrealizationsofthepointprocesses.Apossibleapproachisthestatisticalestimationframeworkrecently 62

PAGE 63

proposedby Truccoloetal. [ 2005 ]forspiketrains.Briey,itrepresentsthepointprocessrealizationsasrealizationsofaBernoullirandomprocess,andthenutilizesageneralizedlinearmodel(GLM)totaconditionalintensityfunctiontothedata.Thisisdonebyassumingthatthelogarithmoftheconditionalintensityfunctionhastheform logpi(^tnjHin)=qXm=1mgm(m(^tn));(3{25)where^tnisthenthdiscrete-timeinstant,gm'saregeneraltransformationsofindependentfunctionsm(),m'saretheparameteroftheGLMandqisthenumberofparameters.Thus,GLMestimationcanbeusedunderaPoissondistributionwithaloglinkfunction.Thetermsgm(m(^tn))arecalledthepredictorvariablesintheGLMframeworkand,ifoneconsiderstheconditionalintensitytodependonlylinearlyonthehistoryoftheeventsthenthegm'scanbesimplydelays.Ingeneraltheintensitycandependnonlinearlyonthehistoryorexternalfactors.Basedontheestimatedconditionalintensityfunction,anyoftheinnerproductsintroducedinSection 3.2 canbeevaluatednumerically.Althoughquitegeneral,theapproachby Truccoloetal. [ 2005 ]hasamaindrawback:sinceqmustbelargerthattheaverageinter-spikeintervala(very)largenumberofparametersneedtobeestimatedthusrequiringlongspiketrains(>10seconds).Noticethatnon-parametricestimationoftheconditionalintensityfunctionwithoutgreatlysacricethetemporalprecisionrequiressmalltimeintervals,whichmeansthatqandthereforetherealizationsusedforestimationmusthavelongerduration.Inspiteofthesediculties,wemaintaintheimportanceoftheRKHSframeworkandthesepointprocesskerneldenitions.Formoreecientcomputationthesekernelsmaymakeuseofdevelopmentsinconditionalintensityfunctionestimationprocedureswillmayexpeditetheiruseinpracticalapplications. 3.4.2EstimationofthemCIKernelIntheparticularcaseofthemCIkernel,denedinEquation 3{11 ,amuchsimplerestimatorcanbederived.Wenowfocusonthiscase.Sinceweareinterestedinestimating 63

PAGE 64

themCIkernelfromsinglerealizationsofthepointprocesses,andforthereasonspresentedbefore,wewillassumethattherealizationsbelongtoPoissonprocesses.Then,usingkernelsmoothing[ Reiss 1993 ; DayanandAbbott 2001 ; Richmondetal. 1990 ]fortheestimationoftheintensityfunctionwecanderiveanestimatorforthepointprocesskernel.Again,theadvantageofthisrouteisthatastatisticalinterpretationisavailablewhilesimultaneousapproachingtheproblemfromapracticalpointofview.Moreover,inthisparticularcasetheconnectionbetweenthemCIkernelandwillnowbecomeobvious.Accordingtokernelsmoothingintensityestimation,givenarealizationofpointprocesspicomprisingofeventcoordinatesftim2T:m=1;:::;Nigtheestimatedintensityfunctionis ^si(t)=NiXm=1h(ttim);(3{26)wherehisthesmoothingfunction.Thisfunctionmustbenon-negativeandintegratetooneovertherealline(justlikeaprobabilitydensityfunction(pdf)).CommonlyusedsmoothingfunctionsaretheGaussian,Laplacianand-function,amongothers.Fromalteringperspective,Equation 3{26 canbeseenasalinearconvolutionbetweenthelterimpulseresponsegivenbyh(t)andtherealizationwrittenasasumofDiracfunctionalscenteredattheeventlocations.Inparticular,binningisnothingbutaspecialcaseofthisprocedureinwhichhisarectangularwindowandthespiketimesarerstquantizedaccordingtothewidthoftherectangularwindow(cf.Section 2.4.1.2 ).Moreover,itisinterestingtoobservethatintensityestimationasshownaboveisdirectlyrelatedtotheproblemofpdfestimationwithParzenwindows[ Parzen 1962 ]exceptforanormalizationterm,aconnectionmadeclearby DiggleandMarron [ 1988 ].Considerrealizationsofpointprocessespi;pj2P(T)withestimatedintensityfunctions^pi(t)and^pj(t)accordingtoEquation 3{26 .SubstitutingtheestimatedintensityfunctionsinthedenitionofthemCIkernel(Equation 3{11 )yieldsthe 64

PAGE 65

estimator, ^I(pi;pj)=NiXm=1NjXn=1(timtjn);(3{27)whereisthekernelobtainedbytheautocorrelationoftheintensityestimationfunctionhwithitself.AwellknownexampleforhistheGaussianfunctioninwhichcaseisalsotheGaussianfunction(withscaledbyp 2).Anotherexampleforhistheone-sidedexponentialfunctionwhichyieldsastheLaplaciankernel.Ingeneral,ifakernelisselectedrstandhisassumedtobesymmetric,thenequalstheautocorrelationofhandthushcanbefoundbyevaluatingtheinverseFouriertransformofthesquarerootoftheFouriertransformof.Theaccuracyofthispointprocesskernelestimatordependsonlyontheaccuracyoftheestimatedintensityfunctions.IfenoughdataisavailablesuchthattheestimationoftheintensityfunctionscanbemadeexactthenthemCIkernelestimationerroriszero.Despitethisdirectdependency,theestimatoreectivelybypassestheestimationoftheintensityfunctionsandoperatesdirectlyontheeventcoordinatesofthewholerealizationwithoutlossofresolutionandinacomputationallyecientmannersinceittakesadvantageofthetypicallysparseoccurrenceofevents.AsEquation 3{27 shows,ifischosensuchthatitsatisestherequirementsinSection 3.1 ,thenthemCIkernelultimatelycorrespondstoalinearcombinationofoperatingonallpairwisedierencesofeventcoordinates,onepairatatime.Inotherwords,themCIkernelistheexpectationofthelinearcombinationofpairwiseinnerproductsbetweeneventcoordinates.Putinthisway,wecannowclearlyseehowthemCIinnerproductestimatorbuildsupontheinnerproductforeventcoordinates,,presentedinSection 3.1 3.4.3EstimationofNonlinear(Memoryless)Cross-IntensityKernelsAsshownintheprevioussection,theestimatorofthemCIkernelresultsnaturallybysubstitutingthekernelintensityfunctionestimatorinthemCIkerneldenition.FortherelatednonlinearCIkernelspresentedinSection 3.2.2 thismatterisslightlymore 65

PAGE 66

complicatedduetothenonlinearityintroducedinthekerneldenition.InthissectionwebrieysuggesthowthesenonlinearCIkernelscanbeestimated. 3.4.3.1EstimationofIThenonlinearCIkerneldenedbyEquation 3{12 lendsitselftoaverysimpleestimator.Indeed,thiskerneldenitionreliesonthecomputationofthenaturalnorminducedbytheinnerproductassociatedwiththemCIkernel(butthenorminducedbyanyofthecross-intensitykernelscanbeconsideredingeneral).Theinducednormis pipj2HI=pipj;pipjHI=hpi;pii2pi;pj+pj;pj=kpik22pi;pj+pj2:(3{28)Therefore,thecomputationalbottleneckintheevaluationofthiskernelisthecomputationofthethreeinnerproductscorrespondingtothenorm.Fromthenon,oneonlyneedstocomputetheexponentialfunctionwiththisnorm(scaled)once.ForinhomogeneousPoissonprocesses,thiscanbeimmediatelydoneusingtheestimatorforthemCIkerneldescribedinSection 3.4.2 torstevaluatethenorm.Thus,thecomputationalcomplexityisofthesameorder. 3.4.3.2EstimationofthenCIkernel,IyEvaluationofthenonlinearCIkernelinEquation 3{13 ,however,doesnotbuildonourpreviousndings.ThereasonforthisisthatthekernelKnonlinearlyweighsthetemporalrelationshipbetweenthetwointensityfunctions,andthereforewecannotobtainananalyticalexpressingtotheintegralonthecombinationofsmoothingfunctions.Thuswewillproposeanestimatorwhichreliesonaparticularformoftheintensityfunctionestimator.Thekeyideaistosimplifytheproblembydividingtimeinintervalsduringwhichtheinteractionamongintensityfunctionsisconstant.Thisisachievedsimplybyutilizingarectangularpulseasthesmoothingfunction.Again,thefocushereintheinhomogeneousPoissoncase.Inthemoregeneralcaseofconditionalintensityfunction 66

PAGE 67

Figure3-1. EstimationofdierenceofintensityfunctionsforevalutionofnonlinearkernelinEquation 3{13 estimationusingtheGLMframework,thepointprocessismadediscrete-timewhichautomaticallyintroducesthissimplication.Nevertheless,intheestimatorpresentedtimeneedsnottobediscretized.Considerasymmetric,positivedeniteandshift-invariantkernelKandthatarectangularpulsesmoothingfunctionisusedforkernelsmoothingintensityestimation.IfKistakentobeshift-invariant(asoccursforthecommonlyusedkernels),thenitisonlysensitivetothedierenceofthearguments.Therefore,itiseasytoverifythatthereexistasmallnitenumberoftransitionsinthevalueofthedierencebetweenintensity 67

PAGE 68

Table3-1. OutlineofthealgorithmforestimationofthenCIkernel,Iy. Step1 Dene,Sy=[ti1;:::;tiNi;ti1+;:::;tiNi+;tj1;:::;tjNj;tj1+;:::;tjNj+];andthecorrespondingincrementalsequence,=[1;1;:::;1| {z }Nitimes;1;1;:::;1| {z }Nitimes;1;1;:::;1| {z }Njtimes;1;1;:::;1| {z }Njtimes]: Step2 SortSyinascendingorder,andapplythesamereorderingto. Step3 Setn=fNumberofnegativetimesinSyg,=Pn1i=1i,andIy=t0=0.Whilen<2(Ni+Nj)andSyn
PAGE 69

Figure3-2. RelationbetweentheoriginalspaceofpointprocessesP(T)andthevariousHilbertspaces.Thebi-directionaldouble-lineconnectionsdenotecongruencebetweenspaces. O(NiNj).Nevertheless,theproposedestimatorisquiteecientconsideringthattheintegralcannotbesimpliedanalytically. 3.5RKHSInducedbytheMemorylessCross-IntensityKernelandCongruentSpacesSomeconsiderationsabouttheRKHSspaceHIinducedbythemCIkernelandcongruentspacesaremadeinthissection.TherelationshipbetweenHIanditscongruentspacesprovidesalternativeperspectivesandabetterunderstandingofhowthemCIkernelcanbeutilizedforcomputationwithpointprocesses.Figure 3-2 providesadiagramoftherelationshipsamongthevariousspacesdiscussednext.SomeoftheserelationshipsextenddirectlytomoregeneralCIkernels.Therefore,althoughthissectionfocusonthespacesassociatedwiththemCIkernel,wewillmentionifsimilarconnectionsholdforotherpointprocesskernelswheneverapplicable. 69

PAGE 70

3.5.1SpaceSpannedbyIntensityFunctionsIntheintroductionofthemCIkerneltheusualdotproductinL2(T),thespaceofsquareintegrableintensityfunctionsdenedonT,wasutilized.Thedenitionoftheinnerproductinthisspaceprovidesanintuitiveunderstandingtothereasoninginvolved.L2(pi(t);t2T)L2(T)isclearlyanHilbertspacewithinnerproductdenedinEquation 3{11 ,andisobtainedfromthespanofallintensityfunctions.Noticethatthisspacealsocontainsfunctionsthatarenotvalidintensityfunctionsresultingfromthelinearspanofthespace(intensityfunctionsarealwaysnon-negative).However,sinceourinterestismainlyontheevaluationoftheinnerproductthisisofnoconsequence.ThemainlimitationisthatL2(pi(t);t2T)isnotanRKHS.ThisshouldbeclearbecauseelementsinthisspacearefunctionsdenedonT,whereaselementsintheRKHSHImustbefunctionsdenedonP(T).Despitethedierences,thespacesL2(pi(t);t2T)andHIarecloselyrelated.Infact,L2(pi(t);t2T)andHIarecongruent.Wecanverifythiscongruenceexplicitlysincethereisclearlyaone-to-onemapping,pi(t)2L2(pi(t);t2T)!pi(p)2HI;and,bydenitionofthemCIkernel, I(pi;pj)=pi;pjL2(T)=pi;pjHI:(3{29)Actually,thecongruencebetweenthetwospaceholdsforanylinearcross-intensitykernelsincetheinnerproductisthesameinbothspaces.ForthenonlinearCIkernels,forexample,thetwospacearestillcloselyrelatedbuttheinnerproductisnotdirectlyavailableinL2(pi(t);t2T)andthereforethetwospacesarenotcongruent.Adirectconsequenceofthebasiccongruencetheoremisthatthetwospaceshavethesamedimension[ Parzen 1959 ]. 70

PAGE 71

3.5.2InducedRKHSInSection 3.3.1 itwasshownthatthemCIkernelissymmetricandpositivedenite(Property 3.1 andProperty 3.3 ,respectively).Consequently,bytheMoore-Aronszajntheorem[ Aronszajn 1950 ],thereexistsanHilbertspaceHIforwhichthemCIkernelevaluatestheinnerproductandisareproducingkernel(Property 3.4 ).ThismeansthatI(pi;)2HIforanypi2P(T)and,forany2HI,thereproducingpropertyholds h;I(pi;)iHI=(pi):(3{30)Asaresultthekerneltrickfollows, I(pi;pj)=hI(pi;);I(pj;)iHI:(3{31)Writteninthisform,itiseasytoverifythatthepointinHIcorrespondingtoaspiketrainpi2P(T)isI(pi;).Inotherwords,givenanyspiketrainpi2P(T),thisspiketrainismappedtopi2HI,givenexplicitly(althoughunknowninclosedform)aspi=I(pi;).ThenEquation 3{31 canberestatedinthemoreusualformas I(pi;pj)=pi;pjHI:(3{32)ItmustberemarkedthatHIisinfactafunctionalspace.Morespecically,thatpointsinHIarefunctionsofpointprocesses;thatis,theyarefunctionsdenedonP(T).ThisisakeydierencebetweenthespaceofintensityfunctionsL2(T)explainedbeforeandtheRKHSHI,inthatthelatterallowsforstatisticsofthetransformedspiketrainstobeestimatedasfunctionsofspiketrains.InlightofProperty 3.4 andProperty 3.8 (i),similarconsiderationscanthedrawnforanyofthepointprocesskernelspresentedinthiswork.Naturally,thefunctionalspaceandcorrespondingfunctionalmappingwillbedierentfordierentkernels,butthesamemathematicalstructureexists.Sincethestructuretoperformcomputationisthesame,analgorithmderivedinthisspacecanbeutilizedusinganypointprocesskernel. 71

PAGE 72

3.5.3MemorylessCIKernelandtheRKHSInducedbyThemCIkernelestimatorinEquation 3{27 showstheevaluationwrittenintermsofelementarykerneloperationsoneventcoordinates.ThisfactaloneprovidesaninterestingperspectiveonhowthemCIkernelusestheeventstatistics.Toseethismoreclearly,considertobechosenaccordingtoSection 3.1 asasymmetricpositivedenitekernel,thenitcanbesubstitutedbyitsinnerproduct(Equation 3{1 )inthemCIkernelestimator,yielding ^I(pi;pj)=NiXm=1NjXn=1im;jnH=*NiXm=1im;NjXn=1jn+H:(3{33)Whenthenumberofsamplesapproachesinnity(sothattheintensityfunctionsand,consequentlythemCIkernel,canbeestimatedexactly)themeanofthetransformedeventcoordinatesapproachestheexpectation.Hence,Equation 3{33 resultsin I(pi;pj)= Ni NjEi;EjH;(3{34)whereEfig,Efigdenotestheexpectationofthetransformedeventcoordinatesand Ni; Njaretheexpectednumberofeventsinrealizationsfrompointprocessespiandpj,respectively.Equation 3{34 explicitlyshowsthatthemCIkernelcanbecomputedasa(scaled)innerproductoftheexpectationofthetransformedeventcoordinatesintheRKHSHinducedby.Inotherwords,thereisacongruenceGbetweenHandHIinthiscasegivenexplicitlyintermsoftheexpectationofthetransformedeventcoordinatesasG(pi)= NiEfig,suchthat pi;pjHI=G(pi);G(pj)H= Ni NjEi;EjH:(3{35) 72

PAGE 73

Recallthatthetransformedeventcoordinatesformamanifold(thesubsetofanhypersphere)and,sincethesepointshaveconstantnorm,thekernelinnerproductdependsonlyontheanglebetweenpoints.Thisistypicallynottruefortheaverageofthesepointshowever.Observethatthecircularvarianceofthetransformedeventcoordinatesforpointprocesspiis[ MardiaandJupp 2000 ] var(i)=Enim;imHoEi;EiH=(0)Ei2H:(3{36)So,thenormofthemeantransformedeventcoordinatesisinverselyproportionaltothevarianceoftheelementsinH.Thismeansthattheinnerproductbetweentwopointprocessesdependsalsoonthedispersionoftheseaveragepoints.Thisfactisimportantbecausedatareductiontechniques,forexample,heavilyrelyonoptimizationwiththedatavariance.Forinstance,kernelprincipalcomponentanalysis[ Scholkopfetal. 1998 ]directlymaximizesthevarianceexpressedbyEquation 3{36 [ Paivaetal. 2006 ]. 3.5.4MemorylessCIKernelasaCovarianceKernelInSection 3.3.1 itwasprovedthatthemCIkernelisindeedasymmetricpositivedenitekernel.AsreviewedinAppendix A Parzen [ 1959 ]showedthatanysymmetricandpositivedenitekernelisalsoacovariancefunctionofarandomprocessdenedintheoriginalspaceofthekernel(areviewoftheseideascanbefoundin Wahba [ 1990 ,Chapter1]).ThismeansthatforthemCI,andingeneralforanyofthepointprocesskernelsconsidered,thereexistsaspaceofrandomprocessesaredenedonP(T)forwhichthepointprocesskernelisacovarianceoperator.LetXdenotethisrandomprocess.Then,foranypi2P(T),X(pi)isarandomvariableonaprobabilityspace(;B;P)withmeasureP.AsprovedbyParzen,thisrandomprocessisGaussiandistributedwithzeromeanandcovariancefunction I(pi;pj)=E!fX(pi)X(pj)g:(3{37) 73

PAGE 74

Noticethattheexpectationisover!2sinceX(pi)isarandomvariabledenedon,asituationwhichcanbewrittenexplicitlyasX(pi;!),pi2P(T),!2.ThismeansthatXisactuallyadoublystochasticrandomprocess.Anintriguingperspectiveisthat,foranygiven!,X(pi;!)correspondstoanorderedandalmostsurelynon-uniformrandomsamplingofX(;!).ThespacespannedbytheserandomvariablesisL2(X(pi);pi2P(T))sinceXisobviouslysquareintegrable(thatis,Xhasnitecovariance).TheRKHSHIinducedbythemCIkernelandthespaceofrandomfunctionsL2(X(pi);pi2P(T))arecongruent.Thisfactisobvioussincethereisclearlyacongruencemappingbetweenthetwospaces.InlightofthistheorywecanhenceforwardreasonaboutthemCIkernelalsoasacovariancefunctionofrandomvariablesdirectlydependentonthespiketrainswithwelldenedstatisticalproperties.Alliedtoourfamiliarityandintuitiveknowledgeoftheuseofcovariance(whichisnothingbutcross-correlationbetweencenteredrandomvariables)thisconceptcanbeofgreatimportanceinthedesignofoptimallearningalgorithmsthatworkwithspiketrains.ThisisbecauselinearmethodsareknowntobeoptimalforGaussiandistributedrandomvariables.Asmentionedabove,similarconsiderationscanbemadeforanyofthepointprocesskernels,althoughtheGaussianrandomprocessesinthecovariancearedierentforeachsincetheycharacterizethestatisticsofthepointprocessmodelconsideredbythepointprocesskernel. 3.6PointProcessDistancesTheconceptofdistanceisveryusefulinclassicationandanalysisofdata,andpointprocessesarenoexception.Themainaimofthissectionistoshowthatinnerproductsforpointprocessescanbeutilizedtoeasilydenedistancesforpointprocessesinarigorousmanner,andindeednaturallyinduceatleasttwoformsofdistancesforpointprocesses.ThissectiondoesnotaimatproposinganyparticularmeasurebuttohighlightthisnaturalconnectionandconveythegeneralityofRKHSframeworkbysuggestinghow 74

PAGE 75

distancescanbeformulatedfrombasicprinciplesifneeded.Duetotheirrelevanceinneurophysiologicalstudies,theseideasarealsoparticularizedforthemCIkerneltoshowthatsomeofthemeasuresproposedinthiscontextaresimplyspecialcasesoftheRKHSframework. 3.6.1NormDistanceThefactthatHIisanHilbertspaceandthereforepossessesanormsuggestsanobviousdenitionforadistancebetweenpointprocesses.Infact,forthelinearcross-intensitykernels,sinceL2(T)isalsoanHilbertspacethisfactwouldhavesuced.Thedistancebetweentwopointprocessesor,ingeneral,anytwopointsinHI,isdenedas dND(pi;pj)=pipjHI=q pipj;pipjHI=q hpi;pii2pi;pj+pj;pj=q I(pi;pi)2I(pi;pj)+I(pj;pj):(3{38)wherepi;pi2HIdenotesthetransformedpointprocessesintheRKHS.FromthepropertiesofthenormandtheCauchy-Schwarzinequality(Property 3.5 )itimmediatelyfollowsthatdNDisavaliddistancesince,foranyspiketrainspi;pj;pk2P(T),itsatisesthethreedistanceaxioms: (i) Symmetry:dND(pi;pj)=dND(pj;pi); (ii) Positiveness:dND(pi;pj)0,withequalityholdingifandonlyifpi=pj; (iii) Triangleinequality:dND(pi;pj)dND(pi;pk)+dND(pk;pj):ThisdistanceisbasicallyageneralizationoftheideabehindtheEuclideandistanceinacontinuousspaceoffunctions. 3.6.2Cauchy-SchwarzDistanceThepreviousdistanceisthenaturaldenitionfordistancewheneveraninnerproductisavailable.However,asforotherL2spaces,alternativesmeasuresforpointprocessescanbedened.Inparticular,basedontheCauchy-Schwarzinequality(Property 3.5 75

PAGE 76

andProperty 3.8 )wecandenetheCauchy-Schwarz(CS)distancebetweentwopointprocessesas dCS(pi;pj)=arccosI(pi;pi)I(pj;pj) I2(pi;pj):(3{39)FromthesymmetryoftheinnerproductsandtheCauchy-SchwarzinequalityitfollowsthatdCSissymmetricandalwayspositive,andthusveriesthersttwoaxiomsofdistance.Moreover,sincedCSistheangulardistancebetweenpointsitalsoveriesthetriangleinequality.ThemajordierencebetweenthenormeddistancepresentedintheprevioussectionandtheCSdistanceisthatthelatterisnotanEuclideanmeasure.Indeed,becauseitmeasurestheangulardistancebetweenthespiketrainsitisaRiemannianmetric.ThisutilizesthesameideaexpressedinEquation 3{5 inpresentingthegeodesicdistanceassociatedwithanysymmetricpositivedenitekernel. 3.6.3SpikeTrainMeasuresSeveralspiketrainmeasureshavebeenproposedintheliterature[ VictorandPurpura 1997 ; vanRossum 2001 ; Schreiberetal. 2003 ]andtheyplayanimportantroleinneurophysiologicalstudies.Sincespiketrainsarerealizationsofpointprocessestheaboveideascanalsobeappliedtomeasuresimilarityordissimilaritybetweenspiketrains.Indeed,itisinsightfultoverifythattwowellestablishedspiketrainmeasurescanbeobtaineddirectlyasspecialcasesofthetwopointprocessdistancespresentedforthesimplestofthepointprocesskernelsconsidered,themCIkernel.SincetheinnerproductdenotesbythemCIkernelisdenedinL2(T),thenormdistancecouldobviouslyalsobeformulateddirectlyandwiththesameresultinL2(T).Then,ifoneconsidersthisperspectivewithacausaldecayingexponentialfunctionasthesmoothingkernelforintensityestimationthenweimmediatelyobservethatdNDcorresponds,inthisparticularcase,tothedistanceproposedby vanRossum [ 2001 ].Usinginsteadarectangularsmoothingfunctionthedistancethenresemblesthedistanceproposedby VictorandPurpura [ 1997 ],aspointedby SchrauwenandCampenhout [ 2007 ], 76

PAGE 77

althoughitsdenitionpreventsanexactformulationintermsofthemCIkernel.Finally,usingaGaussiankernelthesamedistanceusedby Maassetal. [ 2002 ]isobtained.Noticethatalthoughithadalreadybeennoticedthatothercost(i.e.kernel)functionsbetweenspiketimescouldbeusedinsteadoftheinitiallydescribed[ SchrauwenandCampenhout 2007 ],theframeworkgivenherefullycharacterizestheclassofvalidkernelsandexplainstheirroleinthetimedomain.Moreover,ultimatelythemCIkernelestimatorcanbeutilizedforecientcomputationusingtobetheLaplacian,triangular,orGaussiankernel,respectively,forthethreecasesjustdescribed.TheCauchy-Schwarzdistancecanalsobecomparedwiththe\correlationmeasure"betweenspiketrainsproposedby Schreiberetal. [ 2003 ].Infact,itcanbeobservedthatthelattercorrespondstotheargumentofthearccosineandthusdenotesthecosineofananglebetweenspiketrain,withnormandinnerproductcomputedwiththemCIkernelestimatorusingtheGaussiankernel.NoticethatSchreiber'setal.\correlationmeasure"isonlyapre-metricsinceitdoesnotverifythetriangleinequality.But,indCSthisisensuredbythearccosinefunction.Amoredetailedexpositionoftheseinter-relationshipscanbefoundinthecomparisonstudyinAppendix B 77

PAGE 78

CHAPTER4ASTATISTICALPERSPECTIVEOFTHERKHSFRAMEWORKThischapterprovidesanalternativeperspectivetotheRKHSframework,namely,byverifyingtheconstructionofanRKHSasobtainedfromconventionalstatisticaldescriptorsofinterdependence.Morespecically,itisshowntherelationbetweencross-correlationandRKHStheory,especiallynoticeablewhenitsgeneralizedformpresentedhereiscomparedtothemCIkernel.Thehopefullyinsightfulperspectiveprovidedinthischapterhasdirectconsequencesforstatisticalanalysismethods.Thisisexempliedinthesecondpartofthechapter,byshowingthatthemCIkernelcanbeutilizedtoformulateandestimatethecross-intensityfunction(CIF)[ Brillinger 1976 ]andspiketriggeredaverage(STA)[ DayanandAbbott 2001 ]ofonepointprocesswithregardstotheother.MorethatsimplyshowingtherelationshipforthemCIkernel,wetoaimtoexplicitlyshowthelimitationofcurrentapproachestothePoissonmodel,andincitefurtherdevelopmentsthroughtheperspectiveprovidedhere. 4.1GeneralizedCross-CorrelationandthemCIKernelBinnedpointprocessesarediscrete-timerandomprocesses.Therefore,asintroducedinSection 2.4.2 forspiketrains,thecross-correlationisdenedintheusualwayastheexpectationofthelaggedproductofthenumberofeventsperbin.Hence,assumingergodicity,thecross-correlationofbinnedpointprocessespiandpjishabituallyestimatedwith Cbinij[l]=1 MMXn=1Npi[n]Npj[n+l];(4{1)whereMisthenumberofbinsandNpi[n],Npj[n]arethenumberofeventsinthenthbinforpointprocessespiandpj,respectively.Equation 4{1 clearlyshowsthatCbinijisaninnerproductofthebinnedpointprocesses.InRKHStheorythemappingintotheRKHSisoftenunknown,butinthiscontextitisreadilynoticeablethatbinningimplementsthemapping.However,binningofpointprocessesdiscretizesthespaceofeventsandis 78

PAGE 79

thereforeundesirable.Thisraisesthequestionofwhatisbinningactuallydoing?And,correspondingly,canweutilizeabetterwaytodoit?Inessence,binningestimatesthedensityofeventsatagiventime,thatis,itattemptstoestimatetheinstantaneousringrate(apartfromanormalizationbythebinsize)[ DayanandAbbott 2001 ].Hence,initsgeneralform,cross-correlationcanbedeneddirectlyintermsoftheintensityfunctionsofthepointprocesses, Cij()=Epi(t)pj(t+)=limT!11 2TZTTpi(t)pj(t+)dt;(4{2)wherepi(t)andpj(t)denotestheintensityfunctionsofpointprocessespiandpj,respectively.Thisisafunctionalinnerproductinaninnitedimensionalspace.WemightthinkthatCbinijisnitedimensionalapproximationofthisfunctionalmeasure.Weshallrefertothisdenitionasthegeneralizedcross-correlation(GCC)[ Paivaetal. 2008 ],todistinguishfromthebinnedcounterpart.Inthestatisticalliteraturetheconventionalapproachforintensityfunctionestimationofpointprocessesiskernelsmoothing Reiss [ 1993 ],withclearadvantagesintheestimation Kassetal. [ 2003 ].SeeSection 2.4.1.2 forareviewonintensityestimationwithkernelsmoothing.So,iftheeventlocationsofapointprocesspiintheeventspaceP(T)=[0;T]aredenotedftim:m=1;:::;Nig,whereNiisthenumberofeventsofarealizationofpi,thekernelsmoothedestimatedintensityfunctionisgivenby ^pi(t)=NiXm=1h(ttim);(4{3)wherehisthesmoothingkernelfunctionwithsizeparameter.Substitutingtheseintensityestimationsinthedenitionofthegeneralizedcross-correlation(Equation 4{2 )andlimitingtheevaluationtothenitedomainoftheeventspace[0;T]yieldsthe 79

PAGE 80

estimator ^Cij()=1 TNiXm=1NjXn=1timtjn+; (4{4) whereisthekernelobtainedbytheconvolutionoftheintensityestimationkernelhwithitself,andisthekernelsize(orbandwidth)parameter.NoticethatCbinijisaspecialcaseofEquation 4{4 inwhichthespiketimesarerstquantizedandthentheGCCevaluatedwitharectangularkernel.Fromourpresentationitshouldbeclearthattheso-calledGCCequalsthemCIkernel,apartfromthenormalizationforthewidthoftheeventspace.ThisisclearlyobservablebycomparingtheGCCdenitioninEquation 4{2 withthemCIkerneldenitioninEquation 3{11 ,oralternativelyfromtheirestimatorsinEquation 3{27 andEquation 4{4 .Ofimmediateconsequencethisperspectivesuggestsadirectreplacementfor(binned)cross-correlationinpointprocessanalysis,withspiketrainanalysisinparticular.Asanexample,thisideahasbeenexploredtoconstructcontinuous-timecross-correlogramsforspiketrainanalysis[ Parketal. 2008 ]whichbenetofthedirectestimationonthespiketimes(i.e.,theeventcoordinates),thusprovidingmuchhigherprecision,andinafractionofthetimerequiredbyexplicitlysmoothing.Mostimportantly,theobservationofthisequivalenceoftheGCCtothemCIkernelrevealsthelimitationsofcurrentmethodologies.Thismeansthatallcurrentcross-correlationmethodshavedescriptivepoweratmostequivalenttoonlythesimplestofthecross-intensitykerneldenitionsgivenhere.ThemCIkernelcanaccuratelyquantifyatmostinteractionsintheratefunctions,equivalenttoainhomogeneousPoissonprocessmodel.Ontheotherhand,verifyingthiscloserelationshipbringsforththatcross-intensitykernelsareinfactcross-correlationoperatorsforgeneralizedpointprocessmodels.Therefore,webelieveCIkernelsrepresentthefutureofpointprocessanalysis. 80

PAGE 81

Forspiketrainanalysis,thekernelsizeinthemCIkernelhasaparticularusefulinterpretationinpractice.NoticethatEquation 4{3 canbeinterpretedastheconvolutionofthespiketrainwithawindowgivenbythesmoothingfunctionh,emulatingthesmoothingprocessintheneuroncellmembrane.Therefore,thesizeparameterdeterminesthesmoothingintroducedbyhandthekernel,andthusregulatesthescaleatwhichthemCIkernel(orGCC)estimatorinterpretstheneuronalcouplingexpressedintheintensityfunction,betweentheextremesofsynchronyinneuronrings(forsmallkernelsize)orringrate(largekernelsize). 4.2RelevanceforStatisticalAnalysisMethods 4.2.1RelationtotheCrossIntensityFunctionThecross-intensityfunction(CIF)isthesecond-orderassociationmomentbetweentwopointprocesses.Itwasoriginallyproposedby Brillinger [ 1976 ]andwasappliedtospiketrainanalysisbyBrillingerandcolleagues( Brillinger [ 1992 ]andreferencestherein)andothers[ Hahnloser 2007 ].Statistically,thecross-intensityfunction(CIF)istheconditionalprobabilityofaneventoccurringatagivenlocationintheeventspaceforapointprocesspjgiventheoccurrenceofaneventintheconditioningpointprocesspiatsomespeciclocation.Itisdenedas #pjjpi()=lim!0+1 Pr[NB(+tk;+tk+)=1jtk2pi];(4{5)whereNBisthecountingprocessassociatedwithpjandtk2piexpressesthattkisaneventofarealizationofpi.Naturally,theconditionalformulationofCIFmeansthat#pjjpi()isnotasymmetricfunction.Infact,notingthatthe(instantaneous)intensityfunctionofaninhomogeneousPoissonprocesspjcanbewrittenas pj(t)=lim!0+1 Pr[NB(t;t+)=1];(4{6) 81

PAGE 82

leadstotheobservationthatthedenitionofCIFdenesaconditionalintensityfunction, #pjjpi()=pj(+tkjtk2pi);(4{7)wheretkisthe\closest"eventofpito.FromEquation 4{7 resultsthatCIFcanbewrittenas #pjjpi()=pj(jpi)=EtAm2piB(+tAm)1 NANAXm=1B(+tAm):(4{8)Estimatingtheintensityfunctionofpjfromarealizationwithkernelsmoothing ^B(t)=NBXn=1h(ttBn);(4{9)andsubstitutingthisestimateinEquation 4{8 ,yieldsanestimatorforCIF ^#pjjpi()=^B(jpi)=1 NANAXm=1NBXn=1h(+tAmtBn):(4{10)Thisequationclearlyshowsthat#pjjpi()istheintensityfunctioninducedinpjthroughtheoccurrenceofaneventinpi.Notethattheerrorinthisestimatordependsonlyontheestimationoftheintensityfunctionofpjandtheexpectationovereventsofpi.Inotherwords,thisestimatorisunbiasedsincebothoperationscanbedoneexactlyforinnitedata.Conversely,similarargumentscanbeemployedtoderivethat ^#pijpj()=^A(jpj)=1 NBNBXm=1NAXn=1h(+tBmtAn):(4{11)ComparingEquation 4{10 andEquation 4{11 withthemCIkernelestimatorinEquation 3{27 itispossibletoverifythattheyarefundamentallythesameexpectforascaling(by1=NA),theintroductionofalagparameter,andtheuseofthesmoothingkerneldirectly(insteadofitsautocorrelation). 82

PAGE 83

4.2.2RelationtotheSpikeTriggeredAverageAnalternativeinterpretationthatstemsfromEquation 4{7 isthatCIFcanbethoughtofasaevent-triggeredexpectationoftheintensityfunctionofpjaroundeventsinpi.Thatis, #pjjpi()=pj(+tkjtk2pi)=E8tk2pipj(+tk)1 NANAXm=1pj(+tAm):(4{12)whereE8tk2pifgdenotestheexpectationoverallpossibleeventsofpi.ThisshowstheequivalencetotheCIFandthereforethatthemCIkernelcanalsobeutilizedforestimation.Inneurophysiologicalstudiesthiscorrespondstowhatiscalledthespike-triggeredaverage(STA)[ DayanandAbbott 2001 ].Simplyput,STAisaperi-eventdiagramofacontinuousquantityinwhichthesynchronizingeventsarethering(i.e.,spikes)fromaneuron.ItmusttheremarkedthatboththeCIFandSTAareconceptslimitedfordataanalysistothedescriptivepowerofintensityfunctions,andthustoPoissonmodels,ascanbeexpectedfromthecloserelationshiptothemCIkernel.However,asnotedearlierabouttherelationshipbetweenthemCIkernelandtheGCC,thisperspectivesuggeststhatthecross-intensitykernelscouldbeutilizedtogreatlyextendtheseconceptsbeyondPoissonpointprocesses. 4.2.3IllustrationExampleTherelationshipsjustdiscussedtheoreticallyarenowillustratedthroughasimplesimulation.ThesimulatedexamplewascraftedtoreplicatethedatasetofL3andL10neuronsfromexperimentswithAplysiautilizedby Brillinger [ 1992 ].Two10second-longspiketrainsweregeneratedasPoissonprocesses.piwasgeneratedasaninhomogeneousPoissonprocesswithrate20spk/s,andwasusedas 83

PAGE 84

(a) (b) (c)Figure4-1. (a) Modulationinringinducedinpjthroughthespikesinpi. (b) Spikesintimeofpjaroundtheoccurenceofspikesinpi(markedbytheverticaldottedline). (c) Firstsecondofreferencespiketrain,pi,intensityfunctionofpjwitheectsinducedbypi,andcorrespondingrealizationofpj. 84

PAGE 85

thereferencespiketrain(equivalencetoaL10neuron).ThegoaloftheCIFfunctionistostudycross-neuroninducedmodulationsintheintensityfunction.Morespecically,wheneveraspikeoccursinpiitintroducesamodulation(showninFigure 4-1(a) forthissimulation)intheintensityfunctionofpj(equivalenttoaL3neuron).Figure 4-1(c) depictsthismechanism,andcanbeperceivedintheresultingspiketrainsshowninFigure 4-1(b) .Asintroducedintheprevioussections,andshownbyEquation 4{7 ,theCIFcorrespondstoanaverageoftheintensityfunctionofpjwithrespecttothespikesinpi.ThisisshowninFigure 4-2(a) .Correspondingly,themCIkernelasafunctionofthelagevaluatedforthetwospiketrainsyieldsthesame(scaled)result.TheseareshowninFigure 4-2 (b) { (c) .NoticethatifthemCIkernelresultisdividedbytheaveragenumberofspikesofpiinaspiketrain(200spikes=(20spk/s)(10s),cf.Equation 4{10 )yieldsanaverageringrateof20spk/s,asexpected.Finally,ifoneestimatesthereversecondition,thatis,#pijpj(),Equation 4{11 provesthatfromanestimationstandpointthisismerelyamatterofmirroringthe,mCIkernelresult,iftheintensityestimationsmoothingfunctionissymmetric(asisimposedbytheelementarykernelusedtoestimatethemCIkernel).ThecausalrelationshipbetweentheneuronsisimmediatelyapparentfromthecausalityofthemCIkernelasafunctionofthelag. 85

PAGE 86

(a)Spike-triggeredaverageoftheestimatedintensityfunctionpj()withregardstoeventsofpi. (b)CIkernelestimatedwiththeLaplaciankernel (c)CIkernelestimatedwiththeGaussiankernelFigure4-2. (a) Averageintensityfunctionestimatedfromthe\trials"showninFigure 4-1(b) .Itcorrespondstoaspike-triggeredaverageoftheintensityfunctionofpjwithregardstothespikesinpi. (b) { (c) CIkernelasafunctionofthelagestimatedwithLaplacianandGaussiankernelsrespectively. 86

PAGE 87

CHAPTER5APPLICATIONSINNEURALACTIVITYANALYSISAsunequivocallystatedinthisdissertation'stitleanddetailedinchapter 1 ,thisworkcanbedirectlyappliedforneuralactivityanalysis,specicallyonspiketrainanalysis.FollowingtheconsiderationspresentedinChapter 4 ,wenowstudytheapplicationoftheseideasforspiketrainanalysis.Inessence,thischapterprovidessomeimmediatedevelopmentsforspiketrainanalysisforusebythepractitioner.Foreaseofdirectcomparisontocurrenttechniqueswewillbaseourpresentationinthegeneralizedcross-correlation(GCC),butrecallthatGCCandthemCIkernelarefundamentallythesameapartfromthenormalization.Therefore,theconsiderationsforfutureimprovementsareequallyapplicable,pendingonfuturedevelopmentsonconditionalintensityestimation. 5.1GeneralizedCross-CorrelationasaNeuralEnsembleMeasureByitsverynature,aspiketrainisrealizationofapointprocess(Section 2.3 ).Thereforeitshouldseemobviousthatallthetheorypresentedbeforecanbeapplied,asaspecicapplication,forspiketrainanalysis.Inthissection,someideasregardingtheuseofGCCforspiketraindataanalysisareputforward.Eveninthiscase,theperspectivepresentedinChapter 4 allowsfordevelopmentsobscuredbythecommonpresentationfoundintheliterature.Beforeproceeding,itmustberemarkedforthisapplicationthemeaningofthesizeparameterofthesmoothingfunction,orcorrespondinglyofthekernelutilizedintheGCCestimator.Inthiscasethesizeparameterhasawelldenedphysicalmeaning;itselectsthetimescaleatwhichtheanalysisistobeperformed.Inotherwords,thesizeparameteristobeselectedaccordingtotheringcharacteristicsknownapriorioftheneuronand/orthefeatureofinterestfortheanalysisathand.Animportantconsequenceoftheuseofkernelsmoothingforintensityestimationinthisframeworkisthatitseamlesslyintegratesthedierencesbetweenspikeratesandspiketimeswithout 87

PAGE 88

discretizationoftime.Putdierently,theuseofkernelsmoothingmakesiteasytozoomintothefeatureofinterestandputsthefocusonthetimestructureofthespiketrainsasthecentralparameterbeenquantiedasspiketrainsimilarity,Thischaracteristiccanbeveryusefulinspiketrainanalysis,forexample,tomeasuresynchronybetweenspiketrains.Incomputationalneuroscienceoneofthecommonlyuseddescriptorsoftherelationbetweentwospiketrainsissynchrony.Itisobviousthatsincetheinformationofspiketrainsiscontainedinthespiketimes,synchronyquantiesthisrelationshipsomewhateventhoughthereisnometricassigned.However,itisnottotallyfulllingasthedierentdenitionsintheliteraturedemonstrate:synchrony[ Freiwaldetal. 2001 ],synchronyatalag[ LindseyandGerstein 2006 ],polychronization[ Izhikevich 2006 ].Forthisreason,thedenitionofsynchronycanbesubstitutedbythegeneralconceptofsimilarityasmeasuredbyGCC(orthemCIkernel)aspropertime-scale.Moreover,theuseofpointprocessdistancesbetweentwospiketrainsasgiveninSection 3.6 allowsforafullfeaturedmetricspaceifnecessary.TomeasuresimilaritybetweenspiketrainstheGCCestimatorinEquation 4{4 isused.Likeanyestimator,theevaluatedvalueisarandomvariablewhichapproachestheexpectedvalueasmoredatabecomesavailable.Ontheotherhand,fromapracticalstandpointthelengthoftherecordingisoftenlimited.Anyway,itisdesirabletokeeptheintegrationintervaltoaminimumforimprovedresolution.Weproposetosolvethisproblemthroughensembleaveraging.IfMdenotesthenumberofensemblespiketrainsunderanalysis,theensembleaveragedGCCis, C()=2 M(M1)MXi=1MXj=i+1^Cij():(5{1)Inthisway,theintegrationintervalcanbereducedasthenumberofspiketrainsincreaseswithoutsacriceofthestatisticalaccuracy.Ingeneral,theaboveequationdependsonthelag(astheusualcross-correlation),butfortheanalysisdonenextthezerolagshallbeconsidered.Thiscorrespondstothesituationofsynchrony.Inpractice,one 88

PAGE 89

mightneedtotimealignthespiketrainsbyrstestimatingthelagusingthecontinuouscross-correlogram(CCC)[ Parketal. 2008 ],forexample.Ofcourse,animportantquestionthatmustbeconsiderediswhichspiketrainsshouldbeaveragedtogetherasconstituentsofthesameensemble.TheclusteringalgorithmpresentedinChapter 6 canbeofusetoanswerthisquestion. 5.2EmpiricalAnalysisofGCCStatisticalPropertiesThestatisticalpropertiesofGCCwithregardstojitterinthespiketimingsandthenumberofneuronsarenowanalyzed.ThebehaviorofGCCwithrespecttothesetwoparametersisveryimportantforspiketrainanalysis,especiallyinsynchronystudies.Inthefollowingexamplesthereistheneedtogeneratesimulatedspiketrainsunderdierentsynchrony(orcorrelation)conditions.Synchronousspiketrainsweregeneratedusingthemultipleinteractionprocess(MIP)proposedby Kuhnetal. [ 2003 2002 ].IntheMIPmodelaninitialspiketrainisgeneratedasarealizationofaPoissonprocess.Allspiketrainsarederivedfromthisonebycopyingspikeswithaprobability".Theoperationisperformedindependentlyforeachspikeandforeachspiketrain.TheresultingspiketrainsarealsoPoissonprocesses.Ifwastheringrateoftheinitialspiketrainthenthederivedspikestrainswillhaveringrate".Furthermore,itcanbeshownthat"isalsothecountcorrelationcoecient[ Kuhnetal. 2003 ].Adierentinterpretationfor"isthat,givenaspikeinaspiketrain,itquantiestheprobabilityofaspikeco-occurrenceinanotherspiketrain.Inthissense,weshallreferto"asthesynchronylevel.NotethatanalternativemannerofquantifyingsynchronycouldbethroughtheCSdistance,inwhichadistanceofzerocorrespondstoperfectsynchrony(i.e.,"=1). 5.2.1RobustnesstoJitterintheSpikeTimingsInaphysiologicalcontexttheideaofpreciselysynchronousspikesisunlikelytobefound.Thus,itisimportanttocharacterizethebehavioroftheGCCestimatorwhenjitterispresentinthespiketimings.ThiswasdonewithamodiedMIPmodelwherejitter,modeledaszero-meanindependentandidenticallydistributed(i.i.d.)Gaussian 89

PAGE 90

Figure5-1. ChangeinCIPversusjitterstandarddeviationinsynchronousspiketimings.Forthecasewithindependentspiketrains,theerrorbarsforonestandarddeviationarealsoshown.TheestimationkernelwastheLaplaciankernelwithsize2ms(top)and5ms(bottom). 90

PAGE 91

noise,wasaddedtotheindividualspiketimings.Theeectwasthenstudiedintermsofthesynchronylevelandkernelsize(of).Figure 5-1 showstheaverageGCCfor10MonteCarlorunsoftwospiketrains,10secondslong,andwithconstantringrateof20spikes/s.Inthesimulation,thesynchronylevelwasvariedbetween0(independent)to0:5(i.e.,50%ofspikesweresynchronous),andforakernelsizeof2msand5ms.Thejitterstandarddeviationvariedbetweentheidealcase(no-jitter)to15ms.ForasmallestimatorkernelsizetheGCCestimatormeasuresthecoincidenceofthespiketimings.Asaconsequence,thepresenceofjitterinthespiketimingsdecreasestheexpectedvalueofGCC.Nevertheless,theresultsinFigure 5-1 supportthestatementthatthemeasureisindeedrobusttolargelevelsofjitterwhencomparedtothekernelsize,andiscapableofdetectingtheexistenceofsynchronyamongneurons.Ofcourse,increasingthekernelsizedecreasesthesensitivityofthemeasureforthesameamountofjitter.Furthermore,itisalsoshownthatevensmalllevelsofsynchronycanbestatisticallydiscriminatedfromtheindependentcaseassuggestedbytheerrorbarsinthegure.(Thedierenceinscalebetweentheguresisaconsequenceofthenormalizationof,whichdependsonthekernelsize.) 5.2.2SensitivitytoNumberofNeuronsTheeectofthenumberofspiketrainsusedforensembleaveragingisnowanalyzed.Thiseectwasstudiedwithrespecttotwomainfactors:thesynchronylevelofthespiketrainsandthekernelsizeoftheGCCestimator.Intherstcase,thekernelsizewas2ms,whereasinthesecondcaseconsideredonlyindependentspiketrains.TheresultsareshowninFigure 5-2 fortheestimatedGCCaveragedoverallpaircombinationsofneurons.Thesimulationwasrepeatedfor1000MonteCarlorunsusing1secondlongspiketrainssimulatedashomogeneousPoissonprocesseswithringrate20spikes/s.Asillustratedinthegure,thevarianceintheGCCestimatordecreasesdramaticallywiththeincreaseinthenumberofspiketrainsemployedintheanalysis.Recallthatthenumberofpaircombinationsoverwhichtheaveragingisperformedincreaseswith 91

PAGE 92

Figure5-2. Variance(inlogscale)ofGCCversusthenumberofspiketrainsusedforspatialaveraging.TheestimationkernelwastheLaplaciankernel.(top)Theanalysiswasperformedfordierentlevelsofsynchronywithkernelsize2ms,and(bottom)fordierentvaluesofthekernelsizeforindependentspiketrains.InbothsituationsthetheoreticalvalueofGCCforindependentspiketrainsisshown(dashedline). 92

PAGE 93

Figure5-3. MeanandstandarddeviationofGCCversusthenumberofspiketrainsusedforspatialaveragingfordierentsynchronylevels,correspondingtotherstscenarioinFigure 5-2 M(M1),whereMisthenumberofspiketrains.Asexpected,thisimprovementismostpronouncedinthecaseofindependentspikestrains.Inthissituation,thevariancedecreasesproportionallytothenumberofaveragedpairsofspiketrains.ThisisshownbythedashedlineintheplotsofFigure 5-2 .Wheneverthespiketrainsarecorrelated,theimprovementonthevarianceoftheestimatorissmallerduetoanon-idealaveragingsituation,reachinganearlyextremesituationfor"=0:5whereensembleaveragingisalmostuseless.Inanycase,suchhighvaluesofsynchronyseemunlikelytobefoundinneurophysiologicalexperiments.TheseresultssupporttheroleandimportanceofensembleaveragingasaprincipledmethodtoreducethevarianceoftheGCCestimator.Finally,thesensitivityofGCCtothesynchronylevelshouldberemarked.InFigure 5-3 thestandarddeviationwassuperimposedtotheensembleaveragedGCC.Itisobservableacleardistinctionbetween,atleast,thefoursmallersynchronylevels,i.e.,"2[0;0:3].ThismeansthattheGCCestimatorhasahighdegreeofaccuracyinthis 93

PAGE 94

intervalwhenaveragedoveranumberofneuronsassmallas4,supportingourclaimthatGCCcanbeusedasasynchronyindex. 5.3InstantaneousCross-CorrelationTheGCCisamoregeneralformofcross-correlationthatdoesnotrequirebinningbutitstillneedsaniteintervalofdatatooperate.Itisthereforestilldependentonanergoricityassumption.Asafunctionoftime,theintegrandoftheGCC(Equation 4{2 ),whichweshallrefertoastheinstantaneouscross-correlation(ICC),providesamoreappropriaterepresentation.ICCisacontinuousfunctionofthespiketimingsanddescribestemporalstructureoftheinhomogeneousringsallowingforadirectassessmentofsimilarityintime.Onemightthinkofitasascalarinnerproductalongeachofthedimensionsindexedbytime.Therefore,theICCisdenedas ~cij(t;)=^pi(t)^pj(t+);(5{2)where^pi(t),^pj(t)aretheestimatedintensityfunctionsfromspiketrainscorrespondingtopointprocessespiandpj.Formethodologiesthatcanbeappliedonline,onlycausalintensityestimationsmoothingfunctionscanbeconsidered.Weproposetousetheexponentialfunction, h(t)=(1=)exp[t=]u(t);(5{3)whereu()isthestepfunction.Theexponentialfunctionprovidesbothgradedinteractionsandatimescalefortheintensityestimationbycontrollingthetimeconstant.Ofcoursetheideastobepresentedarenotlimitedtothedecayingexponentialsmoothingfunction,butitwaschosenforitsbiologicalplausibility,sinceitcanbeinterpretedasevokedpost-synapticpotentialsinaneuron,itswideusethroughoutneuroscience DayanandAbbott [ 2001 ],anditscomputationalsimplicity,sincecomputingthenextvaluedependsonlyonthepresentvalueandifaspikeoccursinthemeantime. 94

PAGE 95

Figure5-4. DiagramoutliningtheideaandprocedureforthecomputationoftheICC.OntopitisshowntwospiketrainsforwhichtheICCistobecomputed,followedbytheintensityestimationwiththedecayingexponentialfunction(representedbyH(s)).ThetwoestimatedintensityfunctionsarethenmultipliedtogethertoobtaintheICC.Thepositionofsynchronousspikesismarkedasredcirclesinthegure. Usingtheexponentialfunction,theintensityfunctionattimetestimatedfromaspiketrainis ^pi(t)=1 Xtimtexpttim u(ttim):(5{4)ThisisnothingbutthelteringofaspiketrainbyarstorderIIRlter.Then,theICCcanbecomputedbyinstantaneouslymultiplyingthetwoestimatedintensityfunctions.Noticethatthistwolayerevaluationprocesscanbecomputedveryeasily,andisespeciallysuitedforhardwareimplementation.ForsmallvaluesofthesizeparametertheICCquantiesstatisticallyourintuitionofsynchrony,gradedbythedecayingexponentialfunctionandfollowedbyacoincidencedetectionoperatorimplementedbytheproduct.Whentwoneuronsspikesynchronously 95

PAGE 96

theproductoftheestimatedintensitiesatthattimewillbehigh,withamaximumiftheyspikeexactlyatthesametime,butifthespikesareseparatedbymorethan5thentheICCisnearlyzero(Figure 5-4 ).Inthisrespect,theICCresemblesthe\gravityforce"inthegravitytransformframework Gersteinetal. [ 1985 ]; GersteinandAertsen [ 1985 ],butthepresentworkprovidesastatisticalinterpretationfortheestimatorandmuchbroaderperspectivenotavailablebefore. 5.3.1StochasticApproximationofGCCAstheformulationofICCsuggests,~cijisastochasticapproximationoftheGCCunderergodicity.ThisiseasilyveriedbytakingtheexpectationofEquation 5{2 overtime.Inparticular,theaverageICCoveratimeinterval[0;T]withaexponentialfunctionresultsis 1 TZ10~cij(t;)dt=1 T2NiXm=1NjXn=1Z1max(tim;tjn)expjtAmtBn+j ;(5{5)wheretheintegrationgoesuptoinnitytoaccountfortheinnitesupportoftheexponentialfunctionbutonlyspiketimesintheinterval[0;T]areincluded.Theevaluationoftheintegralinvolvesdeterminingwhichspikering,timortjn,occurslatertodeterminetheeectivelowerintegrationlimit.Solvingtheintegralforbothsituations(i.e.,timtjnortim>tjn),however,allowstoverifythatthedierencebetweenthetimeinstantsinbothsituationsispositive,whichcanbesummarizedintheformoftheLaplaciankernel.Thatis, 1 TZ10~cAB(t;)dt=1 TNAXm=1NBXn=11 2expjtAmtBn+j =1 TNAXm=1NBXn=1tAmtBn+=^CAB();(5{6)where,inthiscase,denotestheLaplaciankernel.NotethattheexponentialfunctiongivesrisetotheLaplaciankernelwhichveriesalltherequirementsfor^Cijtorepresentawelldenedinnerproduct.If,forexample,aGaussianfunctionofbandwidthhadbeen 96

PAGE 97

usedasthesmoothingfunctionforintensityestimationthentheresultingkernelwouldalsobeaGaussiankernelwithbandwidthp 2.However,withtheGaussianfunctionwewouldloosetheimportantadvantagesofeaseofcomputationandcausality. 5.3.2ICCasaNeuralEnsembleMeasureTheICCexploitsthetemporalnatureofthespiketrainsandenablesinstantaneousestimationofsynchronybecausenotemporalaveragingisdone.Thepricepaidisthat,forasinglepairsofneurons,variabilityinthespiketimesisdirectlytranslatedintotheICCandthusitsestimationisquite\noisy"duetoeventsoccurringbychance.InsteadofaveragingICCovertimewhichyieldstheGCCinatimeinterval,analternativewaytoreducethevarianceofthisestimatoristocomputetheexpectationovertheneuralensemble, c(t;)=Ef~cAB(t;)g;(5{7)whereEfgdenotestheexpectationoverallpairsofneurons.TheensembleaveragedICCisaspatio-temporalmeasureoftheensemblecooperationovertime.Inthisform,andduetotheexchangeoftimeforensembleaveraging,theICCiscapableofdetectingthepresenceofdynamiccellassembliesintheensemblewithhightemporalresolution.However,asinSection 5.1 ,itraisestheproblemofneuralselectiontoevaluatetheensembleaverage. 5.3.3DataExamplesThreeexamplesoftheapplicationofICCarenowpresented.ThersttwoareinsimulatedparadigmsandthethirdinarecordingofmotorneuronsfromtheM1cortexofratperformingabehavioraltask.IntheseexamplestheanalysisisfocusedonsynchronymainlybecauseitisanapplicationthatnaturallytakesadvantageofthehighresolutionintimeoftheICC,butweremarkthatICCcouldalsobeutilizedforstudiesofcorrelationsintheringratesinprinciple. 97

PAGE 98

5.3.3.1ICCasasynchronizationmeasureThemaingoalsofthisexampleare:rst,toshowthatthemeanvalueofICCissensitivetothesynchronylevelonthatdata,second,thatthismeasurementiseectiveforsingle-realizations,and,nally,toshowcasetheuseofGCCasasynchronyindex;inotherwords,adescriptorofthesynchronylevel.Forthisexample,wegenerated10homogeneousspiketrainsusingthemultipleinteractionprocess(MIP) Kuhnetal. [ 2003 ].TheMIPmodelallowsformultiplespiketrainstobegeneratedaccordingtoaselectedsynchronylevel,",whichisthecountcorrelationcoecientandquantiestheprobabilityofaspikeco-occurrenceinanotherspiketrain.Figure 5-5 showsonerealizationofthegeneratedspiketrainswithvaryinglevelsofsynchrony.Allsimulatedspiketrainshaveaverageringrate20spikes/s.ThegureshowstheICCaveragedforeachtimeinstantoverallpaircombinationsofspiketrains.Thetimeconstant,,oftheexponentialforintensityestimationwaschosentobe2ms.ToverifyEquation 5{6 ,thebottomplotshowstheaveragevalueofthemeanICC.Thiswascomputedwithacausal250mslongslidingwindowin25mssteps.Toestablisharelevanceofthevaluesshown,theexpectationandtheexpectationplustwostandarddeviationsarealsoshown,assumingindependencebetweenspiketrains.Themeanandstandarddeviation,assumingindependence,are1andq 1 2+121,respectively.TheexpectedvalueoftheICCforagivensynchronylevelis1+"=(2),withtheringrateofthetwospiketrains,andisalsoshownintheplotforreference.Finally,theensembleaveragedGCCcomputedforeachsecondofdataisalsoshown.ItisnoticeablefromthegurethattheICCestimatedsynchronyincreasesasmeasuredbyICC.Moreover,theaveragedICCisveryclosetothetheoreticalexpectedvalueandistypicallybelowthestatisticalupperboundunderanindependenceassumptionasgivenbythelineindicatingtheexpectationplustwostandarddeviations.ThedelayedincreaseintheaveragedICCisaconsequenceofthecausalaveragingofICC.Itisequally 98

PAGE 99

Figure5-5. AnalysisofthebehaviorofICCasafunctionofsynchronyinsimulatedcoupledspiketrains.(Top)Levelofsynchronyusedinthesimulationofspiketrains.(Uppermiddle)Rasterplotofrings.(Lowermiddle)EnsembleaveragedICC.(Bottom)TimeaverageofICCintheupperplotcomputedwithacausalrectangularwindow250mslonginstepsof25ms(darkgray).Forreference,itisalsodisplayedtheexpectedvalue(dashedline)andthisvalueplustwostandarddeviations(dottedline)forindependentneurons,togetherwiththeexpectedvalueduringmomentsofsynchronousactivity(thicklightgrayline),asobtainedanalyticallyfromthelevelofsynchronyusedinthegenerationofthedataset.Furthermore,themeanandstandarddeviationoftheensembleaveragedGCCscaledbyTmeasuredfromdatainonesecondintervalsisalsoshown(black). 99

PAGE 100

remarkabletoverifythatGCCmatchespreciselytheexpectedvaluesfromICCasgivenanalytically.ThisshowsasignicantadvantageoftheGCC/ICCasitcanbeusedforanalysisofdataprovidingnotonlydetectionabilitybutalsothepossibilitytoactuallymeasurethesynchronylevelwithahighdegreeofaccuracy. 5.3.3.2Synchronizationofpulse-coupledoscillatorsInthisexample,weshowthatICCcanquantifysynchronyinaspikingneuralnetworkofleaky-integrate-and-re(LIF)neuronsdesignedaccordingtoMirolloandStrogatz MirolloandStrogatz [ 1990 ] 1 andtheICCresultscomparefavorablywiththeextendedcross-correlationformultipleneurons.Thenetworkisinitializedinarandomconditionandisproventosynchronizeovertime(Fig. 5-6 ).Thesynchronizationisessentiallyduetoleakinessandtheweakcouplingamongtheoscillatoryneurons.TherasterplotofneuronringsisshowninFig. 5-6 .Therearetwomainobservations:theprogressivesynchronizationoftheringsassociatedwiththeglobaloscillatorybehaviorofthenetwork,andthelocalgroupingthattendstopreservelocalsynchronizationsthateitherentrainthefullnetworkorwashoutovertime,asexpectedfromtheoreticalstudiesofthenetworkbehavior MirolloandStrogatz [ 1990 ].TheICCdepictsthisbehaviorprecisely:thesynchronizationincreasesmonotonically,withaperiodoffastincreaseintherstsecondfollowedbyaplateauandslowerincreaseastimeadvances.Moreover,itispossibletoobserveintherst1.5stheformationofasecondgroupofsynchronizedneuronswhichslowlymergesintothemaingroup.Inaddition,theenvelopeofICCrevealsthecoherenceinthemembranepotentialsquantiedbytheinformationpotential(IP).TheIPisaninformationtheoreticquantityinverselyproportionalto 1 Theparametersforthesimulationare:100neurons,restingandresetmembranepotential-60mV,threshold-45mV,membranecapacitance300nF,membraneresistance1M,currentinjection50nA,synapticweight100nV,synaptictimeconstant0.1msandalltoallexcitatoryconnection. 100

PAGE 101

Figure5-6. Evolutionofsynchronyinaspikingneuralnetworkofpulse-coupledoscillators.(Top)Rasterplotoftheneuronrings.(Middle)ICCovertime.Theinsethighlightsthemergingoftwosynchronousgroups.(Bottom)Informationpotentialofthemembranepotentials.Thisisamacroscopicvariabledescribingthesynchronyintheneurons'internalstate. 101

PAGE 102

Figure5-7. Zero-lagcross-correlationcomputedovertimeusingaslidingwindow10binslong,andbinsize1ms(top)and1.1ms(bottom). entropy Prncipeetal. [ 2000 ].Itwascomputedwith IP=1 M2MXi=1MXj=1exp(d(i;j)=22)(5{8)with=75mV. 2 TheIPmeasuressynchronyoftheneuron'sinternalstate,whichisonlyavailableinsimulatednetworks.YettheresultsshowthatICCwasabletosuccessfullyandaccuratelyextractsuchinformationfromtheobservedspiketrains.InFig. 5-7 wealsopresentthezero-lagcross-correlationovertime,averagedthroughallpairwisecombinationsofneurons.Thecross-correlationwascomputedwithaslidingwindow10binslong,sliding1binatatime.Resultsareshownforbinsizesof1msand1.1ms.Itisnotablethatalthoughcross-correlationcapturesthegeneraltrendsofsynchrony,itmaskstheplateauandthenalsynchronyanditishighlysensitivetothebinsizeasshowninthegure,unlikeICC.Inotherwords,theresultsforthewindowedcross-correlationhighlighttheimportanceofworkingin\continuous"timeandwithouttimeaveragingforrobustspiketrainanalysis. 2 ThedistanceusedintheGaussiankernelwasd(i;j)=min(jijj;15mVjijj),whereiisthemembranepotentialoftheithneuron.Thiswrap-aroundeectexpressesthephaseproximityoftheneuronsbeforeandafterring. 102

PAGE 103

5.3.3.3AnalysisofneuralsynchronousactivityinmotorneuronsInthislastexample,theICCisutilizedtoanalyzethepresenceofsynchronousactivityinthemotorcortexofarat'sbrain.Throughouttheliterature,synchronousactivityhasbeenshowntoprovideadditionalinformationaboutmotormovementwhencomparedtoringratemodulationpatternanalysisalone,andincludingwhennoringratemodulationsarenoticeable[ Vaadiaetal. 1995 ; Hatsopoulosetal. 1998 ; Riehleetal. 1997 ].Indeed,synchronousneuralactivityseemstobeanwidespreadcharacteristicofthebrainandcanbefoundinanumberofcortices,suchastheauditory[ Wagneretal. 2005 ; CarrandKonishi 1990 ]andthevisualcortices[ Freiwaldetal. 2001 ],forexample.MultichannelneuronalringtimesfromamaleSprague-DawleyratweresimultaneouslyrecordedduringaconditionedbehavioraltaskattheUniversityofFloridaMcKnightBrainInstitute.Theratwaschronicallyimplantedwithtwo28arraysofmicro-electrodesplacedbilaterallyintheforelimbregionoftheprimarymotorcortex(1.0mmanterior,2.5mmlateralofbregma[ DonoghueandWise 1982 ]).NeuronalactivitywascollectedwithaTucker-Davisrecordingrigwithsamplingfrequencyof24414.1Hzanddigitizedto16bitsofresolution.Theringtimeswererecordedfromindividualneuronsspikesortedwithanonlinealgorithmemployingacombinationofthresholdingandtemplate-basedtechniques.Fromsorting,atotalof44singleneuronswererecorded,24neuronsfromthelefthemisphereand20neuronsfromtherighthemisphere.Simultaneously,theratperformedagono-goleverpressingtaskinanoperantconditioningcage(Med-Associates,St.Albans,VT,USA).Thetaskconsistedofchoosingandpressingoneoutoftwolevers(leftorright)dependingonaLEDvisualstimulustoobtainawaterreward.Thequeueandleverpresssignalswererecordedsimultaneouslywiththeneuralactivitywithsamplingfrequency381.5Hz.See Sanchezetal. [ 2005 ]foradditionaldetailsontheexperimentalconguration.ICCwasappliedtothisdatasettoinvestigateforthepresenceofsynchronousneuralactivityacrosstheensemble.Figure 5-8 showssometrialswiththeensembleICC.From 103

PAGE 104

Figure5-8. ICCandneuronringrasterplotonasinglerealization,showingthemodulationofsynchronyaroundtheleverpresses.TheICCwasaveragedthroughouttheneuronspairs,asgivenbyEquation 5{7 ,separatelyforeachhemisphere:left(blue)andright(green).Theleftplotsshowleftleverpressesandrightplotsshowrightleverpresses. 104

PAGE 105

Figure5-9. Windowedcross-correlationofselected6pairsofneurons,forthesamesegmentsshowninFigure 5-8 .Thecross-correlationwascomputedwitha200msslidingwindowover1msbins.Fouroftheneuronpairs,twofromeachhemisphere,areknowntosynchronizeandareshownindarkgrayandlightgraysolidlinefortheleftandrighthemispheres,respectively.Theremainingtwo,onefromeachhemisphere,donotsynchronizestronglyandareshownindottedline.Theleftplotsshowleftleverpressesandrightplotsshowrightleverpresses. 105

PAGE 106

Figure5-10. Spatiallyaveragedwindowedcross-correlation,forthesamesegmentsshowninFigure 5-8 .Thecross-correlationforeachneuronpairwascomputedwitha200msslidingwindowover1msbins.SimilartoICC,thespatialaveragewasdonethroughoutallneuronpaircombinations,separatelyforeachhemisphere:left(darkgray)andright(lightgray).Theleftplotsshowleftleverpressesandrightplotsshowrightleverpresses. 106

PAGE 107

theresultasystematicincreasewhentheleverisreleasedcanbeobserved.Moreover,whiletheleverwasbeingpressedtheensemblesynchronywasobservedtobesignicantlysmallerthansynchronybeforeandafter.Actually,inthisdataset,visualinspectionoftherasterplotswouldyieldsuchconclusions,butICCprovidesaquantitativemethodtotranslatethevisualevaluation.Furthermore,examiningtherasterplotswecanverifythepresenceofensemblesynchronizedactivityreoccurringinaperiodicmanneraftertheleverisreleased.NoticethattheensembleICCcapturesthepresenceofthisoscillatorysynchronizedactivity,seenintheenvelopeofICCinFigure 5-8 ,directlyfromasingletrialandwithhightemporalresolution.Forcomparison,weshowthecross-correlationcomputedatzero-lagwitha200msslidingwindowover1msbins[ Hatsopoulosetal. 1998 ];rst,foronlysomeselectedpairsofneurons(Figure 5-9 ),andthenspatiallyaveraged(Figure 5-10 )asproposedforICC.Althoughthepresenceofsynchronyisalsosuccessfullycapturedwithcross-correlation,thepresenceofanyperiodicmodulationinsynchronyisnotnoticeable.Thiscanbeexpectedsincethecross-correlationrequiresstationarityovertimeandusestimeaveragingtoreducetherandomnessoftheestimator.Thesetwofactorslteroutanyexistingperiodicitiesinthemodulation,whichmayrepresentagreatdealofinformation.Thisimposesupfrontalowerboundonthefrequenciesthatcananalyzed.ThiseectismostvisibleinFigure 5-10 wherespatialaveraginggreatlyimprovestheestimationasthevarianceoftheestimatorisreduced,butthetemporalaveragingpreventsthemodulationinsynchronytobeclearlynoticeable.ThesegureshighlighttheimportanceofthespatialaveragingproposedforICC,inoppositiontothetimeaveragingemployedincross-correlation.ThehightemporalresolutionoftheICCwillbewastedincaseswheretheexperimentalcharacteristicsdonotdisplayhightemporalsynchronyortheexperimentalconditionsdonotallowhighprecisionintemporalmeasurements.Onecaseistheaveragingacrosstrials.Manytimes,theresolutionofthetimemarkersisinsucientwithregardtothesampling 107

PAGE 108

Figure5-11. TrialaveragedICC(upperplot)andcross-correlation(lowerplot)timelockedtoleverrelease.ThetrialaveragedICCisshownforneuronsfromthelefthemisphere(lightgray)andrighthemisphere(darkgray).AlsoshownintheplotisthetrialaveragedICCsmoothedwitha200mslongrectangularwindowforneuronsfromthelefthemisphere(solidline)andrighthemisphere(dashedline).Thecross-correlationwascomputedwitha200msslidingwindowover1msbins.Thetriggeringeventismarkedintheguresbytimezero. 108

PAGE 109

rateoftheneuraldatacollection,ortheexperimentaleectsappearasynchronouswiththestimulus.However,eveninthiscasethesmoothingoftheICCwithalowpasslterwillprovideresultscomparabletothecross-correlationfunction.Toillustratethispoint,theICCanditslowpassversion(lteredwitharectangularwindow200mslong),andthe(spatiallyaveraged)cross-correlationwereaveragedthroughouttrialssynchronizedwithaleverpress.Theresultingperi-eventplotsareshowninFigure 5-11 .Fromthegures,onecanconcludethattheaveragedICCcontainsthesameinformationasthecross-correlationwherethemodulationofsynchronyattheleverpressisclearlyvisibleasmentionedearlier. 5.4Peri-EventCross-CorrelationOverTimeTheICCjustdescribedisasimpletooltodetectandcharacterizetheevolutionofcorrelationwithtime.DespitethesingletrialcapabilityofICC,itissometimesdesiredtocharacterizetheinteractionamongthetwoneuronsasafunctionoftheeventonset.Again,averagingovertimeisnotdesirable.Inthissectiontheperi-eventcross-correlationovertime(PECCOT)ispresented.ThePECCOTaimstobeatooltoanalyzeandvisualizetheevolutionofsynergisticinformationovertimeinaconvenientway. 5.4.1MethodThemaindicultyinestimatingcross-correlationisthatinpracticeonlystochasticestimatesoftheunderlyingintensityfunctionsareavailablefromspiketrains.Toobtainstatisticalreliability,thetraditionalapproachistoaveragetheinstantaneouscross-correlationintheargumentofexpectationoveratimeinterval.Theproblemwiththisapproachisthatittradestimeresolutionforstatisticalreliability.Amoreprincipledapproachistoaverageoverrealizations,asexpressedinthedenitionofcross-correlation.Therearefundamentallytwoprincipledapproachestoachievethis: (i) Averageovertheneuralensemble;or (ii) Averageovertrials. 109

PAGE 110

Eachoftheseapproachesimpliesaparticularassumptionandprovidesaspecictrade-o.Averagingovertheensemblerequiresthatmultiplespiketrainsareassumedpartofthesameensemble,whichmighthavetobefoundapriori,andonetrades\spatial"orensembleresolutionforstatisticalreliability.Conversely,averagingovertrialscanonlybeappliedtoparadigmswheretrialrepetitionisavailable,andalthoughitquantiesthecouplingforeachpairofneurons(high\spatial"resolution),itneedstoassumestationarityamongtrials(thatis,alltrialsarerealizationsofthesameunderlyingprocess).Inspiteofthat,inbothapproachesthetimeresolutionispreservedsincenointegration/averagingovertimeisinvolved.Theresultspresentedabovewherebasedonaveragingovertheensemble.Forexperimentalparadigmswithmultiplerealizations,thesecondapproachisnowconsidered.Insteadofaveragingovertime,thePECCOTaveragestheinstantaneouscross-correlation(ICC)overinstancesoftheevent.Asaconsequence,thePECCOTisabletocharacterizewithhightemporalresolutiontheinteractionsovertimeamongpairsofneurons.Thisisconceptuallysimilartohowtheperi-eventtimehistogram(PETH)isobtained,butherethequantityexpressesneuronalinteractions.ThereforethealgorithmforestimationofthePECCOTisasfollows: 1. Foreachrealizationoftheevent, (a) Estimatetheintensityfunctionofeachneuroninantimeintervalaroundtheeventonset,[T;T](zerocorrespondingtotheeventonset),accordingtoEquation 4{3 (b) Computetheinstantaneouscross-correlationforeachpairofneurons.Atthekthrealization,betweenneuronsiandj,theinstantaneouscross-correlationis,c(k)ij(t)=^(k)pi(t)^(k)pj(t);where^(k)pi(t);^(k)pi(t)aretheestimatedintensityfunctionsforthekthrealization. 2. Averagetheinstantaneouscross-correlationforeachpairofneuronsacrossrealizations. 110

PAGE 111

Figure5-12. Modulationofintensitywiththeeventforeachneuron. CarefulexaminingthealgorithmonemayrecognizethesameformthatleadstothemaindiagonaloftheJPSTH[ Aertsenetal. 1989 ]whichtypicallyexpressestheneuralinteractions.Thedierencehoweveristhatherethecomputationisdoneexplicitly,andthusmuchmoreeciently.Also,byfocusingonlyonthisfunction,analysisoftheoverallresultismuchsimplersincetheresultofallpairsofneuronsmaybesummarizedinasingleplot.Nevertheless,asfortheJPSTH,itisalsopossibletocomputeotherdiagonalsbyintroducingthedependencytoalagbetween^(k)pi(t)and^(k)pi(t).Moreover,thestatisticalprocedureproposedby Aertsenetal. [ 1989 ]fornormalizationoftheJPSTHcanbeappliedfornormalizationofthePECCOT,withtheintensityfunctionestimatedbykernelsmoothing. 5.4.2DataExamplesTwodataexamplesoftheanalysiswithPECCOTarenowshown.Firstasimulateddatasetisutilizedtoshowthemethoddoescapturethedesiredfeatureinthedata.InthesecondexamplethesamerecordingofmotorneuronsanalyzedinSection 5.3.3.3 wasanalyzedwiththePECCOT. 111

PAGE 112

Figure5-13. CenteredPECCOTforthethreeneuronpairsaroundthelever. 5.4.2.1SimulationToillustrateandvalidatethemethodjustproposedweconsiderasimplesimulatedexample.Threeneuronswithbaseringrate20spk/sweregenerated.Alloftheseneuronsmodulatedtheirringrateinthetimevicinityoftheevent,asshowninFigure 5-12 ,andheregeneratedwithaninhomogeneousPoissonmodel.Inaddition,neuronsAandBtendedtoresynchronouslyapproximately0.12sbeforetheevent.ThiscouplingwasintroducedinthegeneratedspiketrainsbyselectingthenearestspikeofAto0.12sbeforetheeventasareferenceandmovingtheclosestspikeinBtothesametime(witha1mszeromeanGaussianjitteradded),ifthetwospikesdierbylessthan50ms(baselineinter-spikeinterval).NeuronCspikedindependentlyofbothAandB.Atotalof100eventrealizations(trials)wheregenerated.TheconstructeddatasetwasanalyzedbyPECCOTwithaGaussiansmoothingfunctionofwidth=5ms.ThecomputedresultisshowninFigure 5-13 .Theresultwascenteredbyremovingtheexpectedcoincidencelevelsmerelyduetoratemodulations.The 112

PAGE 113

Figure5-14. CenteredJPSTHforeachneuronpair. 113

PAGE 114

PECCOTmarksthepresenceofsynchronousactivitybetweenneuronsAandBwithastrongpeakinthecross-correlationroughly0.12sbeforetheeventonset,asexpectedgiventheconstructionofthedataset.Moreover,theinstantaneouscross-correlationbetweenneuronCandothersdoesnotshowanysignicantpeak,onlytheeectofringratemodulations.Forcomparison,wealsocomputedtheJPSTHforthesameneuronpairs(showninFigure 5-14 )usingNeuroExplorer(Littleton,MA).Foreaseofcomparison,thebinsizewassetto5ms.Again,weobserveastrongpeakbetweenAandBapproximately0.12sbeforetheevent.Severalinteractionsarevisiblefortheothertwopairs.However,carefullyexaminingthescalesonenoticesthatthepeakisabouttwotimeshigherintherstcase.TheseresultshighlightthedicultyinanalyzingmultipleJPSTHplots,especiallywithanincreasingnumberofneuronpairs.Ontheotherhand,bydisplayingtheresultofallneuronpairsinasingleplotunderthesamescale,thePECCOTgreatlysimpliesthisanalysis. 5.4.2.2Event-relatedmodulationofsynchronousactivityThePECCOTisnowdemonstratedfortheanalysisofcouplingsintheneuronalringsofneuronsinforelimbregionofM1ofaratperformingabehavioraltask.ThesamedatasetasinSection 5.3.3.3 wasutilized.Specically,wewantedtoverifyiftheneurons'synchronousringpatternsmodulatedwithmovementonset.TotestthishypothesisthecenteredPECCOT 3 wascomputedinaneighborhoodoftwosecondsbeforeandaftertheleverpresses.ThesmoothingfunctionforintensityfunctionestimationwasaGaussianfunctionwithwidth=5ms.Forvisualizationpurposes,thecenteredPECCOTwasfurthersmoothedwithaGaussianwindowofwidth,=10ms.Toanalyzepossibledierencesinsynchronymodulationbetweenleftandright 3 Centeringwasutilizedtoremovetheeectofverydierentringratesandtheirmodulations. 114

PAGE 115

LefthemisphereRighthemisphere Figure5-15. CenteredPECCOTaroundtheleverpressonset.Thetwocolumnscorrespondtoneuronsfromtheleftandrighthemispheres,respectively,andtworowscorrespondtothesituationinwhicheithertheleftorrightleverwaspressed,respectively. leverpresses(sincethetwoleversareusuallypressedwithdierentpaws)andbetweenhemispheres,thesituationsareconsideredseparately.Atotalof93leftleverpressesand45rightleverpresseswereusedforaveraging.TheresultsareshowninFigure 5-15 andFigure 5-16 .IntherstgurePECCOTwasshownasinFigure 5-13 ,whileinthesecondweoptedtodisplaytheresultsintheformofacolorcodedgureduetothelargenumberofneuronpairs,makingiteasiertovisualizetheoverallmodulationandidentifythemostrelevantneuronpairs.Itcanbeobservedthatthesynchronyamongneuronsinthelefthemisphereisfarmorewidespreadthanintherighthemisphere,forbothleftorrightlevelpresses.Itcan 115

PAGE 116

LefthemisphereRighthemisphere Figure5-16. CenteredPECCOTaroundtheleverpressonset.LikeFigure 5-15 butinimageform.EachlinecorrespondstothePECCOTofapairsofneuronwithamplitudecolorcoded. beclearlyobservedthatinallsituationsthereisconsiderableinteractionamongneuronsbeforetheleverpressinstantandthattheseinteractionsarealmostentirelysuppressedimmediatelyafter.Approximatelyonesecondaftertheleverpressinstantthesynchronyincreasesagain.Interestingly,itshouldberemarkedthatthistimeintervalcorrespondsapproximatelytotheaveragedurationofaleverpress,afterwhichtheratreceivesawaterrewardifthecorrectleverwaspressed.Moreover,wenoticeleverpressspecicsynchronymodulationwithdepressionsaround1.4s,0.95s,0.8s,0.45sand0.3sbeforealeftleverpress,andamajordepressionaround1.25sbeforearightleverpress.Thesemodulationsarepresentatthesametimeinbothhemispheres.Also,intheimagesitisapparentthattheinteractionsbetweenneuronstendtobephaselockedandhaveaperiodiccomponent 116

PAGE 117

inthethetarange(3{8Hz).Althoughwehavenotinvestigatedthereasonforthisperiodicphaselockingofsynchrony,theseresultsmayprovidefurtherevidenceontheroleoflowfrequencyrhythmscommonlyfoundinmeso-andmacroscopicrecordingsas\clocksignals"forsynchronizationofmultiplebrainregions. 117

PAGE 118

CHAPTER6CLUSTERINGOFSPIKETRAINSHavinganRKHSframeworkforpointprocessesisimportantbecauseitfacilitatesthedevelopmentofnewmethodstooperatewithpointprocesses,andtheirrealizations.Moreover,allofthesemethodsaredevelopedunderthesameprinciplesprovidedbythisgeneraltheory.ToexemplifytheuseofpointprocesskernelsproposedundertheRKHSframework,inthefollowingweshowhowaclusteringalgorithmforspiketrainscanbeobtainednaturallyfromanyofthepointprocesskerneldenitionsherepresented.Comparingtheseideaswithpreviousclusteringalgorithmsforspiketrainswendthattheyresultinsimplermethods,derivedinanintegratedmanner,withaclearunderstandingofthefeaturesbeingaccountedfor,andgreatergenerality.Itmustberemarkedthatalthoughthischaptershallconsiderspiketrains,itisimmaterialtheexactnatureoftherealizationsofpointprocessestobeclustered.NotethattheprimaryemphasishereistoillustratetheeleganceandusefulnessoftheRKHSframeworkratherthanmerelyproposeanotheralgorithm.Inspiteofthat,itwillbeshownthroughmultiplesimulationsthatthespiketrainalgorithmpresentedhereperformsasgoodorbetterthanotheralgorithmsintheliteraturedespiteitssimplicity. 6.1AlgorithmIntheliteratureafewalgorithmshavebeenproposedforclusteringofspiketrains.Examplesarethemethodsproposedby Paivaetal. [ 2007 ]and Fellousetal. [ 2004 ].Bothofthesealgorithmsrelyonmeasuresbetweenspiketrains. Paivaetal. [ 2007 ]utilizedvanRossum'sdistance[ vanRossum 2001 ],butitispointedoutthatVictor-Purpura's(VP)distance[ VictorandPurpura 1996 1997 ]couldbeusedaswell.Inturn, Fellousetal. [ 2004 ]usedinsteadthe\correlation-basedmeasure"proposedby Schreiberetal. [ 2003 ].Nevertheless,asshowninSection 3.6.3 ,eitherofthemeasuresusedinthepreviousclusteringalgorithmscanbereformulatedintermsofthemCIkernel.Morethansimply 118

PAGE 119

areformulationofthedistances,thisraisesthequestion:\CantheRKHSframeworkbeutilizedtoderiveclusteringalgorithmsisanintegratedmanner?"Theanswerisyes.ForthepurposeofthisexamplewewillshowhowspiketrainkernelsdenedintheRKHSframeworkprovidethemeanstodoclusteringofspiketrains.Thealgorithmwillbebasedontheideasofspectralclustering,sincekernelsnaturallyquantifyanity.Spectralclusteringisadvantageousforthepurposeofthisexamplesincetheevaluationoftheanitybetweenspiketrainsbypointprocesskernelsandtheactualclusteringprocedureareconceptuallydistinct.Itispossibletoextendotherclusteringalgorithmsalthoughonemustintroducetheinnerproductdirectlyintothecomputationwhichslightlycomplicatesmatters.Spectralclusteringofspiketrainsoperatesintwomajorsteps.First,theanitymatrixofthespiketrainsiscomputed.Letfs1;s2;:::;sngdenotethesetofnspiketrainstobeclusteredintokclusters.Theanitymatrixisannnmatrixdescribingthesimilaritybetweenallpairsofspiketrains.Thesecondstepofthealgorithmistoapplyspectralclusteringtothisanitymatrixtondtheactualclusteringresults.Inparticular,thespectralclusteringalgorithmproposedby Ngetal. [ 2001 ]wasusedforitssimplicityandminimaluseofparameters.Theclusteringalgorithm,presentedstep-by-step,ispresentedinTable 6-1 .Thereaderisreferredto Ngetal. [ 2001 ]foradditionaldetailsonthespectralclusteringalgorithm.Clearly,thedeningstepfortheuseofthisalgorithmishowtoevaluateanitybetweenspiketrains.Sinceinnerproductsinherentlyquantifysimilarity,anyofthekernelsproposedcanbeused,andinparticularthemCIandnCIkernels,forwhichweprovideresults.Geometrically,thisroleofthekernelcanbeunderstoodsincetheinnerproductissensitivetothenormandangulardistanceofthetwospiketrainsintheRKHS.InthissituationtheanitymatrixissimplytheGrammatrixofthespiketrainscomputedwiththespiketrainkernel.Notethatthecross-correlation(CC)ofbinnedspiketrainsisinitselfaninnerproductofspiketrainsandthereforecouldbeusedaswell.Indeed, 119

PAGE 120

Table6-1. Step-by-stepdescriptionofthealgorithmforclusteringofspiketrains.Thesearebasicallythestepsofthespectralclusteringalgorithm. 1. ComputetheanitymatrixA2Rnnfromthenspiketrains.Theijthentryoftheanitymatrixisgivenby, aij=^I(si;sj);ifi6=j0;otherwise(6{1)where^I(pi;pj)denotestheestimatorofanypointprocesskernel,evaluatedforspiketrainssiandsj. 2. ConstructDasadiagonalmatrixwiththeithelementofthemaindiagonalequaltothesumofallelementsintheithrowofA(orcolumn,sinceAissymmetric).Thatis,di=nXj=1aij: 3. EvaluatethematrixL=(D1 2)A(D1 2): 4. Findx1;x2;:::;xk,thekeigenvectorsofLcorrespondingtothelargesteigenvalues,andformthematrixX=[x1;x2;:::;xk]2Rnk: 5. DeneY2RnkasthematrixobtainedfromXafternormalizingeachrowtounitnorm.Consequently,yij=xij q Pnj=1x2ij: 6. InterpretingYasasetofnpointsinRk,clusterthesepointsintokclusterswithk-meansorsimilaralgorithm. 7. Assigntotheithspiketrainthesamelabeloftheithpoint(row)ofY. 120

PAGE 121

Eggermont [ 2006 ]utilizedthisideainhisanalysis.However,binningquantizesthespiketimesandisthereforeintroducesboundaryartifactsintheanalysis,aswewillshowlater.Comparedtothemethodproposedby Paivaetal. [ 2007 ]thealgorithmshownhereissimplersincenotransformationtomapthedistanceevaluationtoasimilaritymeasurementandtheneedtoadjustthecorrespondingparameterisavoided.Sincedistancesarederivedconceptsand,usually,canbedenedintermsofinnerproducts,theapproachtakenismuchmorestraightforwardandprincipled.Moreover,thealgorithmcanbegeneralizedmerelybyusingadierentpointprocesskernel.EventhesimplemCIkernelexplicitlyunveilsabroaderpotentialofthealgorithm.Inparticular,unliketheformulationof Paivaetal. [ 2007 ]whichwasapparentlyrestrictedtoclusteringofspiketrainsbysynchrony,ourknowledgebasedonthemCIkernelrevealsthisisnottrue.Ratheritismerelyamatterofkernelsize.Furthermore,thereisacloseconnectionbetweenpointprocesskernelsandkernelsonspiketimes(i.e.,eventcoordinates),eitherbyconstructionorinestimation(asSection 3.4.2 elicits),andthussuggeststhatamultitudeofkernelsonspiketimescanbeusedinplaceoftheLaplaciankernelassociatedwithvanRossum'sdistance(cf.Section 3.6.1 ).Theseideasshallbeillustratednextwithsomesimulationexperiments. 6.2ComparisontoFellous'ClusteringAlgorithmTheclusteringalgorithmofspiketrainsby Fellousetal. [ 2004 ]isperhapsthemostwellestablishedmethodintheliterature.Therefore,thisalgorithmwillnowbecomparedwiththeclusteringalgorithmwejustdescribed.Thisallowstoassesswhichalgorithmisbetter,andifclusteringabilitymighthavelostinusingtheRKHSframework.Thealgorithmby Fellousetal. [ 2004 ]issomewhatsimilarinprincipletotheabovealgorithm,butwithimportantdierences.Forreference,theclusteringalgorithmisnowgiven.Thealgorithmoperatesinthreesteps: 121

PAGE 122

1. ComputethesimilaritymatrixusingSchreiber'setal.correlation-basedmeasure[ Schreiberetal. 2003 ]. 2. Reshapethesimilaritymatrixwithasigmoidfunctiontoincreasetheentropyofthehistogramofsimilarityvalues. 3. ApplyfuzzyC-means(FCM)tothesimilaritymatrixbytakingeachcolumn(orrow)asaninputpoint.Therststepcorrespondstothecomputationoftheanitymatrixthatwehaddescribedearlier.Thesecondstepwasmotivatedbytheworkof BellandSejnowski [ 1995 ]and,accordingtotheauthors,aimedtoimprovetheclusteringperformance.ThelaststepusesFCM(orfuzzyK-means;theyarethesame),toobtaintheactualclustering.Basically,thisusestheideathatneighboringspiketrainsarereciprocallycloseandthereforethesimilaritybetweentwospiketrainsissmallatthesamerow(orcolumn)ofthecolumnofthesimilaritymatrix.Forthecomparison,thesamesurrogatedatasetutilizedin Fellousetal. [ 2004 ]wasused.Thedatasetisavailableat http://www.cnl.salk.edu/~fellous/data/JN2004data/data.html .Thedatasetincludesthreescenarioswith2,3and5clusters.Thereareatotalof100situationsforeachscenariocorrespondingtomultiplelevelsofextraspikes(non-synchronousspikesaimedtoconfusetheclustering)andmultiplelevelsofjitterinthesynchronousspikes.Ineachsituation,thedatasetcomprises30MonteCarloruns,eachwith35spiketrainstobeclustered.BothclusteringalgorithmswereimplementedinMatlab.TheresultsforthealgorithmproposedwerecomputedwiththemCIkernelestimatorusingtheGaussiankernelwithwidth5ms,asindicatedin Fellousetal. [ 2004 ,pg.2992].Forreshapingofthesimilaritymatrixtheprocedurein Fellousetal. [ 2004 ,pg.2999]wasfollowed.FromFigure 6-1 ,Figure 6-2 ,andFigure 6-3 onecanclearlyverifythatthemethodproposedhereandusingthemCIkernelestimatorachievesmuchbetterperformance.Eventhoughtheresultsaresomewhatcomparableforthetwoclusterproblem,withadierencesmallerthan6%,forahighernumberofclustersthisimprovementisashigh 122

PAGE 123

Figure6-1. ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithmfortwoclusters.IntheleftcolumntheresultsfortheclusteringusingthemCIkernelwiththeGaussiankernelareshown.InthemiddlecolumntheresultsareforFellous'algorithm.Therightcolumnshowsthedierencebetweenthetwomethods(rstminussecond).Theupperrowshowstheresultsasafunctionofthejitterstandarddeviationformultiplenumberofextraspikes(legendontheright),andthebottomrowshowsthesameresultsfromthereciprocalperspective. Figure6-2. ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithm,likeFigure 6-1 ,butforthreeclusters. 123

PAGE 124

Figure6-3. ComparisonofclusteringperformancebetweentheclusteringalgorithmproposedhereandFellous'algorithm,likeFigure 6-1 ,butforveclusters. as25%forthreeclustersand50%forveclusters!Theprimaryreasonforthisshouldthedirectuseofthewholesimilarity/anitymatrixinthesecondmethod.Notethattheimplementationembedsthen-dimensionalsimilarityvectorsinann-dimensionalspace.Consequently,thisspaceisnecessarilysparse.Eventhoughitperformsacceptablyforasmallnumberofclusters,asthenumberofclustersisincreasedthesparsitywithinclusterforthedimensionallyofthespacegreatlyhinderstheclusteringperformance.OfcourseusingalargerkernelwouldmitigatetheproblemsomewhatbyintroducingcorrelationsamongdimensionsbutwouldlimittheanalysisfortheprobleminitiallyintendedbyFellousandcolleagues.Theroleorrelevanceofthesimilaritymatrixreshapingalwaysintriguedus.Thus,thiswasinvestigatedinoursimulation,althoughtheseresultsarenotshownforconciseness.Itwasfoundthatthistransformationhadaminimalimpact,andactuallytheresultstendedtobeslightlybetterwithoutit. 124

PAGE 125

6.3SimulationsThissectionaimsprimarilytocomparetheuseofmultiplepointprocesskernels.FirsttheclusteringusingthemCIkernel,thenonlinearkerneldenitioninEquation 3{12 ,andthe(binned)cross-correlation(CC).InSection 6.2 ,themCIandnCIkernelsarecomparedfortheclusteringofrenewalpointprocesses. 6.3.1ClustersCharacterizedbyFiringRateModulationInthissimulationexamplethedeningfeatureofeachclusterissimilarityintheintensityfunctionunderlyingeachspiketrain.Specically,thismeansthatforeachclusteranintensityfunctionwasgenerated,inthisparticularcasechosentobeasinusoidalwith1Hzfrequency.TheseintensityfunctionswerethenutilizedtogenerateonesecondlonginhomogeneousPoissonspiketrains.Sincethespiketrainsforeachclusterweregeneratedaccordingtothesameintensityfunction,ideally,theevaluationofthepointprocesskernelswouldyieldthemaximumvalueforspiketrainswithinclusterandadierentvaluefortheremainingspiketrains.However,sincethedataislimitedthereissomevarianceintheevaluationofthekernelwhichleadstoclusteringerrors.Ofcourse,ifthespiketrainsaremadelongerthisvariabilityisdecreasedandthereforetheclusteringperformanceisimproved.Theclusteringperformancealsodependsonhowdierentthetwoclustersare.Inourcasethedierentiatingcharacteristicbetweenclustersisthephasedierencebetweenthetwosinusoidalintensityfunctions.Inthesimulation,theclusteringperformancewasmeasuredasthevalueoftherelativephasewasvariedovertheinterval[0;180]instepsof20degrees.Foreachvalue,theclusteringperformanceresultswereaveragedover100MonteCarloruns,eachcomprising100spiketrainsrandomlydistributedoverthetwoclusters.Performanceresultsarealsogivenusingthreedierentkernelsizes25ms,50msand100ms,intheestimationofthepointprocesskernelsandvanRossum'sdistance.Thekernelsizeswerepurposelychosenlarge(thatis,ontheorderoftheaverageinter-spikeinterval)sincebytheproblemformulationitisknownthatthedistinguishingfeatureisasmoothintensity 125

PAGE 126

Figure6-4. Clusteringperformanceasafunctionofthephasedierenceintheintensityfunction.IntheleftcolumntheresultsfortheclusteringusingthemCIkernelwiththeLaplacianandrectangularkernelsareshown.Likewise,inthemiddlecolumnitisshowntheresultsforthealgorithmusingvanRossum'sdistance(=10)andthecross-correlation(topandbottomrows,respectively).TherightcolumnshowsthedierencebetweentheclusteringperformanceusingthemCIkernelandthecorrespondingmethodonthemiddlecolumn(ofthesamerow).Ineachplot,clusteringperformanceresultsareshownforthreekernelsizesspeciedinthelegend(forthecross-correlationinterpret\kernelsize"as\binsize"). 126

PAGE 127

function.Resultsforpointprocesskernelsusingarectangularkernel,withwidthgivenbythekernelsize,arealsoshownforcomparisonwithaCC-basedinnerproduct.Thegoalistoillustratethelimitationsincurredinadiscretetimerepresentationasimposedbybinning.Noticethattherectangular\kernel"isnotpositivedenite.Nevertheless,itcanbeutilizedinestimationjustlikethetanhfunctionisutilizedinkernelmethods[ Scholkopfetal. 1999 ].Figure 6-4 showstheclusteringperformanceresultsusingthemCIkernelevaluatedwithboththeLaplacianandrectangularkernels.Theseresultsarecontrastedwiththeapproachin Paivaetal. [ 2007 ],fortheoptimumsizeoftheGaussianfunction(=10),andutilizingCCastheinnerproduct.Notethatthesimilaritymeasureutilizedin Paivaetal. [ 2007 ],correspondsineecttothenonlinearpointprocesskerneldenitioninEquation 3{12 .TheLaplacianandrectangularkernelsusedtoevaluatethepointprocesskernelswereselectedtoapproximatethekernelfunctiononspiketimesimplicitinthemeasurewewerecomparingagainst.Asshowninthegure,theimplementationutilizingthemCIkernelnotonlyissimplerbutalsooutperformsthecompetingalgorithmsbyuptonearly10%.Thisimprovementismostnoticeableforsmallphasedierences;thatis,whendiscriminationamongclustersisthemostdicult.Mostimportantly,thegeneralityofpointprocesskernelsallowstoexperimentwithmanydierentkernelsonspiketimes.Inthisparadigm,betweentheLaplacianandrectangularkernels,thebestresultsareachievedwiththeLaplaciankernel.Anyway,itisshownthatevenutilizingtherectangularkerneltheperformancecanbeconsiderablyimprovedwithregardstotheuseoftheCC,onlybecausenobinningisutilized. 6.3.2ClustersCharacterizedbySynchronousFiringsIncontrasttothepreviousscenario,wenowconsiderthecasewhenclustersarecharacterizedthroughsynchronizedspikesamongtheirspiketrains.Inotherwords,adependencyisimposedintheunderlyingprocessgeneratingspiketrainswithinaclustersuchthataspikeisaddedsimultaneouslyintomorethanonespiketrainwithsome 127

PAGE 128

Figure6-5. Clusteringperformanceasafunctionofthesynchronylevelbetweenspiketrainswithinclusterinthejitter-freecase.TheresultsareshowninthesameformasforFigure 6-4 ,withresultsusingthemCIkernelintheleftcolumn,vanRossum'sdistanceandCCinmiddlecolumnanddierenceinperformanceintheright. probability.Sinceeachclusterisgeneratedindependentlysoaretheresultingspiketrainsbetweenclusters.Liketheprevioussimulation,theideahereistoparameterizethesynchronyofspiketrainswithinclustersandverifytheclusteringperformancebasedonthisparameter.Inourcase,thisisregulatedquitesimplybytheprobabilitythatthegeneratingprocessintroducesaspikeintomorethanonespiketrainatthesametime.Inthefollowingweshallrefertothisprobabilityasthe\synchronylevel,"denedasthe(expected)ratioofsynchronousspikeswithregardstotheoverallspikerate.Inourcasewemodeledthissituationthroughclusterwidesynchronousspikeswithanaverageoccurrence"spk/s, 128

PAGE 129

Figure6-6. Clusteringperformanceasafunctionofthejitterstandarddeviation.Again,theresultsareshowninthesamestructureasFigure 6-4 .However,foreachplotinthiscase,clusteringperformanceresultsareprovidedfortwovaluesofthesynchronyleveland,foreachsynchronylevel,forthreekernelsizesasindicatedinthelegend. where"andarethesynchronylevelandnalaveragespikerateforthespiketrains,respectively.By`clusterwidesynchronousspikes'wemeanthatsynchronousspikesareintroducedinallspiketrainswithinaclusteratthesametime.ThisprocessisthenaddedtoanindependentlygeneratedhomogeneousPoissonspiketrainwithaveragespikerate(1").NoticethattheresultingspiketrainsarestillPoissondistributedandwithaveragespikerate.However,thereadermightthinkthattheunderlyingintensityfunctionforeachclustershasmostofthetimeaconstantvalueof(1"),exceptatthetimesofthesynchronouswidespikeswherescaledimpulsesarepresentintegratingto".Itisworthpointingoutthatclusteringofspiketrainscorrespondingtothisparadigmisa 129

PAGE 130

commonproblem.Infact,thiswasthemainmotivationandapplicationfortheworkby Fellousetal. [ 2004 ].Twosituationswereconsideredforanalysis.Intherst,synchronousspikesmatchperfectlysothatthekernel(orbin)sizecanbemadeasclosetozeroasdesired.Indeedthebestperformanceisexpectedasthekernelsizeismadesmallersincethereisbetterdiscriminationoftruesynchronousspikesthanfromspikesthatoccurbychance.Conversely,asthekernelsizeisincreasedmorespikesoccurringbychanceareaccountedfor,thusincreasingthe\noise"andvariabilityofthemeasurement.Inthesecondcase,thesynchronousspikeswerejitteredindependentlywithzero-meanGaussiannoisebeforetheywereintroducedintoeachspiketrain.Unliketherstsituationwhichdepictsanimprovablescenario,thissituationaimsatunderstandinghowthealgorithmperformundersomevariabilityinthesynchronousspikesasisoftenencounteredinpractice.Forthesimulation,100spiketrainsweregeneratedatatimeaccordingtotheprocessdescribedbeforeanddistributedrandomlyovertwoclusters.Asaresultoftheprocessabove,spiketrainshadconstantaveragespikerateof20spk/sandwereonesecondlong.Resultswereobtainedbyaveragingover100MonteCarlorunsintherstsituation(nojitter)and500MonteCarlorunsinthesecondsituation(withjitter).Thisprocedurewasrepeatedforeachsynchronylevelandforthreedierentkernelsizes,2ms,5msand10ms.ComparedtotheexperimentinSection 6.3.1 ,inthiscasethekernelsizeswerechosensmallcomparedtotheaverageinter-spikeinterval(50ms)sinceintheparadigmformulationitwasstatedthatclusterswerecharacterizedbysynchrony.Alternatively,thiscanthethoughtofintermsoftheprobleminestimatingtheintensityfunctionwedepictedearlier,whichhappenstobeimplicitlytakenintoconsiderationbythemCIkernel.TheclusteringperformanceresultsaregiveninFigure 6-5 andFigure 6-6 ,forthejitter-freeandwithjittersituations,respectively.Inbothguresitcanbeobservedonceagainthattheclusteringresultsusingthepointprocesskernelsevaluatedwiththe 130

PAGE 131

Laplaciankernelarebetterthanwiththerectangularkernel.However,thealgorithmusingthemCIkernelperformssimilarlytothealgorithmbasedonvanRossum'sdistance,ineithersituation.InthecaseofthecomparisonwiththeCC-basedalgorithmthelatterperformsbetterinthenoise-freecase.However,Figure 6-6 showsthatthisisonlytrueforsmall(<2ms)standarddeviationsofthejitternoisesmaller.Asthejitterisincorporated,evensmallvariabilityinthesynchronyinthespikesleadstosignicantlossesintheperformanceusingCC.Thisshowstheparticularlysignicantnegativeimpactofusingbinnedspiketrainsforsynchrony-basedclusteringunderrealisticscenarios.Although,asremarkedabove,themethodby Paivaetal. [ 2007 ]utilizedoneofthenonlinearpointprocesskernelsproposed,theresultsarenotbetterthanwiththemCIkernel.ItmustbeemphasizedthatsuchbehaviorwasexpectedforthereasonspresentedinSection 3.2.2 .Basically,becausethenonlinearityplaysaminimalroleinextendingthecapabilitiesofthemCIkernelthisdenition,similartotheroleofsigmoidfunctionreshapinginFellous'algorithm.FortruenonlinearbehavioronthespaceofintensityfunctionsthenCIkernelsneedstobeused,withgreatmodelingadvantagesasshownnext. 6.3.3ClusteringofRenewalProcessesbymCIandnCIKernelsThegoalofthissimulationexampleistoshowtheimportanceofpointprocesskernelsthatgobeyondtherstcross-moment(i.e.,cross-correlation)betweenspiketrains.Forthisreason,weappliedthealgorithmproposedhereforclusteringofspiketrainsgeneratedashomogeneousrenewalpointprocesseswithagammainter-spikeinterval(ISI)distribution.ThismodelwaschosensincethePoissonprocessisaparticularcaseandthuscanbedirectlycompared.Athreeclusterproblemisconsidered,inwhicheachclusterisdenedbytheISIdistributionofitsspiketrains(Figure 6-7 (a)).Inotherwords,spiketrainswithintheclusterweregeneratedaccordingtothesamepointprocessmodel.Allspiketrainswere1slongandwithconstantringrate20spk/s.ForeachMonteCarlorun,atotalof100spiketrainsrandomlyassignedtooneoftheclustersweregenerated.Theresults 131

PAGE 132

(a)Inter-spikeinterval(ISI)distributionsden-ingeachcluster. (b)Examplespiketrainsfromeachcluster. (c)Clusteringresults.Figure6-7. ComparisonofclusteringperformanceusingmCIandnCIkernelsforathreeclusterproblem. 132

PAGE 133

statisticswereestimatedover500MonteCarloruns.ForboththemCIandnCIkernels,theGaussianfunctionwasusedassmoothingfunctionwithresultsforthreevaluesofthesmoothingwidth,2,10and100ms.Inaddition,theGaussiankernelwasutilizedforKinthecomputationofthenCIkernel,withresultsforkernelsizes=1and=10.TheresultsofthesimulationareshowninFigure 6-7 (c).Theclusterwithshapeparameter=1containedPoissonspiketrains,spiketrainswithshapeparameter=3weremoreregular,and=0:5gaverisetomoreirregular(i.e.\bursty")spiketrains.TheresultswiththemCIkernelareatmost1.4%better,onaverage,thanrandomselection.Thislowperformanceisnotentirelysurprisingsinceallspiketrainshavethesameconstantringrate.UsingthenCIkernelwiththelargersmoothingwidthyieldedanimprovementof14.7%for=10and18%for=1,onaverage.Smallervaluesofdidnotimprovetheclusteringperformance(=0:1resultedinthesameperformanceas=1),demonstratingthattheselectionofkernelsizeforthenCIkernelisnotveryproblematic.But,mostimportantly,theresultsshowthateventhoughtheformulationdependsonlyonthememorylessintensityfunctions,inpractice,thenonlinearkernelKallowsfordierentspiketrainmodelstobediscriminated.ThisimprovementisduetothefactthatKenhancestheslightdierencesintheestimatedintensityfunctionsduetothedierentpointprocessmodelexpressedinthespiketrains(Figure 6-7 (b)). 6.4ApplicationforNeuralActivityAnalysisToconcludethischapter,webrieypresentsomeresultsontheapplicationofthisalgorithmtotheneuralactivityanalyzedinChapter 5 .Asmentionedthen,clusteringbecoupledwiththeICCanalysistodeterminewhichneuronstoconsiderasanensemblesothataveragingovertheensemblecanbedone.AswasobservedinSection 5.3.3.3 ,thereisinterestingmodulationofsynchronyinthemotorneuronsabout0:250:4secondsaftertheleverisreleased.Therefore,clusteringwasappliedtothesetofspiketrains(oneforeachneuron)intheinterval[0:5;1:5](seconds)aftertheleverwasrelease,usingthemCIkernelwithaLaplacianestimation 133

PAGE 134

Figure6-8. Clusteringofneuralactivityfollowingaleverrelease,assuming4clusters.ThespiketrainscorrespondtothemomentsaftertheleverpressesinFigure 5-8 134

PAGE 135

kernelofwidth2ms.TheresultsareshowninFigure 6-8 forthesameleverpressesshowninFigure 5-8 ,considering4clusters.Oneofmajordicultieswhenapplyingclusteringtorealdatasetssuchasthisoneishowtochoosethenumberofclusters.Thisisgreatlycomplicatedbythefactthatallclusterssharesomesimilarity,andthusbecomesquitecomplicatedwheretoplaceaboundary.Inthiscase,thevaluewaschosenaftertryingvaluesfrom3to5.Forthisdataset,4clustersseemstoprovideagoodoveralldistinction(visuallyjudgedfromtherasterplot)betweenclusters.(Eectively,wetriedtopreventanytwoclustersfromlookingquitesimilar.)Theproblemwithestablishingaboundarymightsignifythatfuzzymethodsneedstobeutilized,maybesimplybyreplacingK-meansbyfuzzyK-meansintheeectiveclusteringstepofthespectralclusteringalgorithm.FromFigure 6-8 itcanbeveriedthattheclusteringalgorithmseparatesneuronsbasedonbothringrateandsynchrony,despitethesmallkernelsize.ThisisveryimportantbecauseifICCisappliedtoeachclusterthisresultsallowsfordierenttime-scalestobeutilizedforeachclusterandenhancesthevariousmomentswhensynchronyoccurswithinclusterandacrossclusters.Forexample,intherasterplotitcanobservedthemainsynchronyrhythmalsoshownintheICCplotsinFigure 5-8 (e.g.,redclusterintherstplot)but,inaddition,itrevealsotherhigher-frequencyrhythms(e.g.,yellowclusterintherstplot).TogetherwithICCanalysisforeachcluster,matchingtheICCwithLFPactivity,and/orsimplycorrelatingthesendingswiththespatialplacementofthemicro-electrodesmightrevealtherolethisneuronalcoupling. 135

PAGE 136

CHAPTER7PRINCIPALCOMPONENTANALYSISTofurtherillustratetheimportanceoftheRKHSframeworkshownhereforcomputationwithpointprocesses,inthefollowingwederivethealgorithmtoperformprincipalcomponentanalysis(PCA)ofrealizationsofpointprocesses,andofspiketrainsinparticular.AsinChapter 6 ,althoughweconsiderspiketrainsduetomainmotivationofthiswork,theideasareapplicabletoanyone-dimensionalpointprocess.ThePCAalgorithmwillbederivedfromtwodierentperspectives.First,PCAwillbederiveddirectlyintheRKHSinducedbyapointprocesskernel.ThisperspectiveshowstheusefulnessoftheRKHSframeworkforoptimization,andhighlightsthatoptimizationwithrealizationsofpointprocessesispossiblebythedenitionofaninnerproductforthepointprocessrealizations,andmorespecicallythroughthemathematicalstructureprovidedbytheRKHS.Thisisalsothetraditionalapproachinthefunctionalanalysisliterature RamsayandSilverman [ 1997 ]andhastheadvantageofbeingcompletelygeneral,regardlessoftheactualpointprocesskerneldenitionused.AwellknownexampleofdiscretePCAdoneinanRKHSiskernelPCA[ Scholkopfetal. 1998 ].InthesecondapproachwewillderivePCAinthespacespannedbytheintensityfunctionsutilizingtheinnerproductdenedinthisspace.Thus,thisperspectiveisapplicableonlyforlinearCIkernels.ThederivationshownhereconsidersthemCIkernelbutthesamecanbederivedintermsoftheconditionalintensityfunctionsforgenerallinearCIkernels.SinceforthesepointprocesskernelstheRKHSiscongruenttothisspacetheinnerproductsinthetwospacesareisometric,andthereforetheoutcomewillbefoundtobethesame.However,thisapproachhastheadvantagethatitexplicitlymakesavailabletheeigenfunctionsas(scaled)intensityfunctions.Thisisimportantinmanyneurophysiologicalstudiessincetheresearcherisofteninterestedinunderstandingtheundergoingprocessintheneuronalnetwork,asexpressedbytheintensityfunctions.Notethat,ingeneral,theeigenfunctionsarenotavailableintheRKHSbecausethe 136

PAGE 137

transformationtotheRKHSisunknown.However,thisapproachispossiblehereduetothelinearityofthespacespannedbytheintensityfunctionswiththeinnerproductwedened. 7.1OptimizationintheRKHSSupposewearegivenasetofspiketrains,fs1;s2;:::;sNg,forwhichwewishtodeterminetheprincipalcomponents.Computingtheprincipalcomponentsofthespiketrainsdirectlyisnotfeasiblebecausewewouldnotknowhowtodeneaprincipalcomponent(PC),however,thisisatrivialtaskinanRKHS.Letfsi2HI;i=1;:::;NgbethesetofelementsintheRKHSHIcorrespondingtothegivenspiketrains.Notethat,correctlyspeaking,denotesthetransformationforapointprocessintotheRKHS,andforwhichtheinnerproductisthepointprocesskernel.Inspiteofthat,thischapterdealsexclusivelywithpointprocessrealizationsandtherefore,withsomeabuseofnotation,sishallbeusedtodenotethe\transformedspiketrains."Then,theinnerproductofsi'sisineecttheestimatorofthepointprocesskernel.Denotethemeanofthetransformedspiketrainsas =1 NNXi=1si;(7{1)andthecenteredtransformedspiketrains(i.e.,withthemeanremoved)canbeobtainedas ~si=si:(7{2)PCAndsanorthonormaltransformationprovidingacompactdescriptionofthedata.DeterminingtheprincipalcomponentsofspiketrainsintheRKHScanbeformulatedastheproblemofndingthesetoforthonormalvectorsintheRKHSsuchthattheprojectionofthecenteredtransformedspiketrainsf~sighasthemaximumvariance.ThismeansthattheprincipalcomponentscanbefoundbysolvingthefollowingoptimizationproblemintheRKHS:afunction2HI(i.e.,:P(T)!R)isaprincipal 137

PAGE 138

componentifitmaximizesthecostfunction J()=NXi=1hProj(~si)i2kk21(7{3)whereProj(~si)denotestheprojectionoftheithcenteredtransformedspiketrainonto,andistheLagrangemultipliertotheconstraintkk21imposingthattheprincipalcomponentshaveunitnorm.Toevaluatethiscostfunctiononeneedstobeabletocomputetheprojectionandthenormoftheprincipalcomponents.However,inanRKHS,aninnerproductistheprojectionoperatorandthenormisnaturallydened.Thus,theabovecostfunctioncanbeexpressedas J()=NXi=1D~si;E2HIh;iHI1;(7{4)Becauseinpracticewealwayshaveanitenumberofspiketrains,isrestrictedtothesubspacespannedbythecenteredtransformedspiketrainsf~sig.Consequently,thereexistcoecientsb1;:::;bN2Rsuchthat =NXj=1bj~sj=bT~(7{5)wherebT=[b1;:::;bN]and~(t)=h~s1(t);:::;~sN(t)iT.SubstitutinginEquation 7{4 yields J()=NXi=1NXj=1bjD~si;~sjE!NXk=1bkD~si;~skE!+1NXj=1NXk=1bjbkD~si;~skE!=bT~I2b+1bT~Ib:(7{6) 138

PAGE 139

where~IistheGrammatrixofthecenteredspiketrains;thatis,theNNmatrixwithelements ~Iij=D~si;~sjE=si;sj=si;sj1 NNXl=1hsi;sli1 NNXl=1sl;sj+1 N2NXl=1NXn=1hsl;sni:(7{7)Inmatrixnotation, ~I=I1 N(1NI+I1N)+1 N21NI1N;(7{8)whereIistheGrammatrixoftheinnerproductofspiketrainsIij=si;sj,and1NistheNNmatrixwithallones.Thismeansthat~IcanbecomputeddirectlyintermsofIwithouttheneedtoexplicitlyremovethemeanofthetransformedspiketrains.FromEquation 7{6 ,ndingtheprincipalcomponentssimpliestotheproblemofestimatingthecoecientsfbigthatmaximizeJ().SinceJ()isaquadraticfunctionitsextremacanbefoundbyequatingthegradienttozero.Takingthederivativewithregardstob(whichcharacterizes)andsettingittozeroresultsin @J() @b=2~I2b2~Ib=0;(7{9)andthuscorrespondstotheeigendecompositionproblem 1 ~Ib=b:(7{10)ThismeansthatanyeigenvectorofthecenteredGrammatrixisasolutionofEquation 7{9 .Thus,theeigenvectorsdeterminethecoecientsofEquation 7{5 andcharacterizetheprincipalcomponents.Itiseasytoverifythat,asexpected,thevarianceoftheprojections 1 NotethatthesimplicationintheeigendecompositionproblemisvalidregardlessiftheGrammatrixisinvertibleornot,since~I2and~Ihavethesameeigenvectorsandtheeigenvaluesof~I2aretheeigenvaluesof~Isquared. 139

PAGE 140

ontoeachprincipalcomponentequalsthecorrespondingeigenvaluesquared.So,theorderingofspeciestherelevanceoftheprincipalcomponents.Tocomputetheprojectionofagiveninputspiketrainsontothekthprincipalcomponent(correspondingtotheeigenvectorwiththekthlargesteigenvalue)weneedonlytocomputeintheRKHStheinnerproductofswithk.Thatis, Projk(s)=hs;kiHI=NXi=1bkiDs;~siE=NXi=1bkiI(s;si)1 NNXj=1I(s;sj)!:(7{11)Weemphasizeoncemorethatnopropertyspecicofapointprocesskernelwasutilizedinthederivation.Indeed,itutilizesonlythelinearvectorspacestructureprovidedbytheRKHSforoptimizationandcomputation.Therefore,anyofthepointprocesskernelsproposedinthisdissertationcanbeutilized. 7.2OptimizationintheSpaceSpannedbytheIntensityFunctionsAsbefore,letfs1;s2;:::;sNgdenotethesetofspiketrainsforwhichwewishtodeterminetheprincipalcomponents,andfsi(t);t2T;i=1;:::;Ngthecorrespondingintensityfunctions.Themeanintensityfunctionis (t)=1 NNXi=1si(t);(7{12)andthereforethecenteredintensityfunctionsare ~si(t)=si(t)(t):(7{13)Again,theproblemofndingtheprincipalcomponentsofasetofdatacanbestatedastheproblemofndingtheeigenfunctionsofunitnormsuchthattheprojectionshavemaximumvariance.Thiscanbeformulatedintermsofthefollowingoptimizationproblem.Afunction(t)2L2(si(t);t2T)isaprincipalcomponentifitmaximizesthe 140

PAGE 141

costfunction J()=NXi=1hProj(~si)i2kk21=NXi=1D~si;E2L2kk21;(7{14)whereistheLagrangemultiplierconstrainingtohaveunitnorm.Itcanbeshownthat(t)liesinthesubspacespannedbytheintensityfunctionsf~si(t);i=1;:::;Ng.Therefore,thereexistcoecientsb1;:::;bN2Rsuchthat (t)=NXj=1bj~sj(t)=bT~r(t):(7{15)withbT=[b1;:::;bN]and~r(t)=h~s1(t);:::;~sN(t)iT.SubstitutinginEquation 7{4 yields J()=NXi=1NXj=1bjD~si;~sjE!NXk=1bkD~si;~skE!+1NXj=1NXk=1bjbkD~si;~skE!=bT~I2b+1bT~Ib:(7{16)where~Iisthegrammatrixofthecenteredintensityfunctions(i.e.,~Iij=D~si;~sjEL2).Therefore,thisderivationisonlyvalidforpointprocesskernelsforwhichtheinnerproductisexplicitlydenedinthespaceofintensityfunctions(ingeneral,conditionalintensityfunctions).Asexpected,sinceinthiscasetheRKHSandthespaceofintensityfunctionsarecongruentbecausetheinnerproductproducesthesameresult,thiscostfunctionyieldsthesamesolution.However,unliketheprevious,thispresentationhastheadvantagethatitshowstheroleoftheeigenvectorsofthegrammatrixand,mostimportantly,howtoobtaintheprincipalcomponentfunctionsinthespaceofintensityfunctions.FromEquation 7{15 ,thecoecientsoftheeigenvectorsofthegrammatrixprovideaweighting 141

PAGE 142

Figure7-1. SpiketrainsusedforevaluationoftheeigendecompositioncoecientsofPCAalgorithm(A),andfortestingoftheresult(B).Ineithercase,thersthalfofspiketrainscorrespondstothersttemplateandtheremainingtothesecondtemplate. fortheintensityfunctionsofeachspiketrainsandthereforeexpresseshowimportantaspiketrainistorepresentothers.Inadierentperspective,thissuggeststhattheprincipalcomponentfunctionsshouldrevealgeneraltrendsintheintensityfunctionsoftheinputspiketrains. 7.3Results 7.3.1ComparisonwithBinnedCross-CorrelationToillustratethealgorithmjustderived,andtocomparetheuseofthemCIkernelwithbinnedcross-correlation(CC)inthistask,weperformedasimpleexperiment.Wegeneratedtwotemplatespiketrainscomprisingof10spikesuniformlyrandomdistributedoveranintervalof0.25s.Inaspecicapplicationthesetemplatespiketrainscouldcorrespond,forexample,totheaverageresponseofacultureofneuronstotwodistinctbutxedinputstimuli.Forthecomputationofthecoecientsoftheeigendecomposition(\trainingset"),wegeneratedatotalof50spiketrains,halfforeachtemplate,by 142

PAGE 143

(a)Eigenvaluesindecreasingorder. (b)Firsttwoeigenvectorsoftheeigendecomposi-tionoftheGrammatrix.Figure7-2. EigendecompositionofthecenteredGrammatrix~I. randomlycopyingeachspikefromthetemplatewithprobability0.8andaddingzeromeanGaussiandistributedjitterwithstandarddeviation3ms.Fortestingoftheobtainedcoecients,200spiketrainsweregeneratedfollowingthesameprocedure.ThesimulatedspiketrainsareshowninFigure 7-1 .AccordingtothePCAalgorithmderivedpreviously,wecomputedtheeigendecompositionofthematrix~IasgivenbyEquation 7{8 sothatitsolvesEquation 7{10 .TheevaluationofthemCIkernelwasestimatedfromthespiketrainsaccordingtoEquation 3{27 ,andcomputedwithaGaussiankernelwithsize2ms.Theeigenvaluesfl;l=1;:::;100gandrsttwoeigenvectorsareshowninFigure 7-2 .Thersteigenvaluealoneaccountsformorethan26%ofthevarianceofthedatasetintheRKHSspace.Althoughthisvalueisnotimpressive,itsimportanceisclearsinceitisnearly4timeshigherthanthesecondeigenvalue(6.6%).Furthermore,noticethatthersteigenvectorclearlyshowstheseparationbetweenspiketrainsgeneratedfromdierenttemplates(Fig. 7-2 (b)).Thisagaincanbeseenintherstprincipalcomponentfunction,showninFigure 7-3 ,whichrevealsthelocationofthespiketimesusedtogeneratethetemplateswhilediscriminatingbetweenthemwithoppositesigns.Aroundperiodsoftimewherethespikefromboth 143

PAGE 144

Figure7-3. Firsttwoprincipalcomponentfunctions(i.e.,eigenfunctions)inthespaceofintensityfunctions.TheyarecomputedbysubstitutingthecoecientsofthersttwoeigenvectorsoftheGrammatrixinEquation 7{15 (a)Projectionofthespiketrainsinthetrainingset. (b)Projectionofthespiketrainsinthetestingset.Figure7-4. ProjectionofspiketrainsontothersttwoprincipalcomponentsusingmCIkernel.Thedierentpointmarksdierentiatebetweenspiketrainscorrespondingtoeachoneoftheclasses. 144

PAGE 145

templatesoverlaptherstprincipalcomponentiszero.Ascanbeseenfromthesecondprincipalcomponentfunction,theroleofthesecondeigenvectoristoaccountforthedispersioninthedatacapableofdierentiatespiketrainsgeneratedfromdierenttemplates,especiallyaroundthetimeswheretheyoverlap.Bothdatasets,forevaluationandtesting,whereprojectedontothersttwoprincipalcomponents.Figure 7-4 showstheprojectedspiketrains.Asnotedfromthedierencebetweentherstandsecondeigenvalues,therstprincipalcomponentisthemainresponsibleforthedispersionbetweenclassesoftheprojectedspiketrains.ThishappensbecausethedirectionofmaximumvarianceistheonethatpassesthroughbothclustersofpointsintheRKHSduetothesmalldispersionwithinclass.Thesecondprincipalcomponentseemstoberesponsiblefordispersionduetothejitternoiseintroducedinthespiketrains,andsuggeststhatotherprincipalcomponentsmayplayasimilarrole.AmorespecicunderstandingcanbeobtainedfromtheconsiderationsdoneinSection 3.5.3 .There,thecongruencebetweentheRKHSinducedbythemCIkernel,HI,andtheRKHSinducedby,H,wasutilizedtoshowthatthemCIkernelisinverselyrelatedtothevarianceofthetransformedspiketimesinH.Inthisdatasetandforthekernelsizeutilized,thisguarantiesthatthevalueofthemCIkernelwithinclassisalwayssmallerthaninterclass.Thisisareasonwhyinthisscenariotherstprincipalcomponentalwayssucestoprojectthedatainawaythatdistinguishesbetweenspiketrainsgeneratedeachofthetemplates.PCAwasalsoappliedtothisdatasetusingbinnedspiketrains.Althoughcross-correlationisaninnerproductforspiketrainsandthereforetheabovealgorithmcouldhavebeenused,forcomparisontheconventionalapproachwasfollowed[ RichmondandOptican 1987 ; McClurkinetal. 1991 ].Thatis,tocomputethecovariancematrixwitheachbinnedspiketraintakenasadatavector.Thismeansthatthedimensionalityofthecovariancematrixisdeterminedbythenumberofbinsperspiketrain,whichmaybeproblematiciflongspiketrainsareusedorsmallbinsizesareneededforhightemporalresolution. 145

PAGE 146

(a)Eigenvaluesindecreasingorder. (b)Firsttwoeigenvectors/eigenfunctionsoftheeigendecompositionofthecovariancematrix.Figure7-5. Eigendecompositionofthecovariancematrix. (a)Projectionofthespiketrainsinthetrainingset. (b)Projectionofthespiketrainsinthetestingset.Figure7-6. Projectionofspiketrainsontothersttwoprincipalcomponentsofthecovariancematrixofbinnedspiketrains.Thedierentpointmarksdierentiatebetweenspiketrainscorrespondingtoeachoneofthetemplates. 146

PAGE 147

TheresultsofPCAusingbinsizeof5msareshowninFigure 7-5 andFigure 7-6 .Thebinsizewaschosentoprovideagoodcompromisebetweentemporalresolutionandsmoothnessoftheeigenfunctions(importantforinterpretability).ComparingtheseresultstheonesusingthemCIkernel,thedistributionoftheeigenvaluesisquitesimilarandthersteigenfunctiondoesrevealssomewhatofthesametrendasinFigure 7-3 .Thesameisnottrueforthesecondeigenfunction,however,whichlooksmuchmorejaggy.Infact,asFigure 7-6 shows,inthiscasetheprojectionsalongthersttwoprincipaldirectionsarenotorthogonal.Thismeansthatthecovariancematrixdoesnotfullyexpressthestructureofthespiketrains.Itisnoteworthythatthisisnotonlybecausethecovariancematrixisbeingestimatedwithasmallnumberofdatavectors.Infact,whenthebinnedcross-correlationwasutilizeddirectlyintheabovealgorithmastheinnerproductthesameeectwasobserved,meaningthatthebinnedcross-correlationdoesnotcharacterizethespiketrainstructureinsucientdetail.Sincethebinnedcross-correlationandthemCIkernelareconceptuallyequivalentapartfromthediscretizationintroducedbybinning,thisprovestheilleectsofthispreprocessingstepforanalysisandcomputationwithspiketrain,andpointprocessrealizationsingeneral. 7.3.2PCAofRenewalProcessesPCAis,inessence,alteringoperation.Therefore,themCIandnCIkernelsarenowcomparedforPCAofrenewalprocesses.Basically,aparadigmsimilartotheoneutilizedintheprevioussectionwasemployed.Twodatasetsweregenerated:forcomputationoftheeigendecomposition(i.e.,\training")with50spiketrains,andfortestingwith200spiketrains.Foreachdataset,thespiketrainsweregeneratedfromtworenewalpointprocessmodelswithgammadistributedinter-spikeintervals(shapeparameter=0:5and=3),onehalffromeachmodel.Allspiketrainswere1secondlongandwithmeanringrate20spk/s.ThesimulatedspiketrainsareshowninFigure 7-7 .Then,thealgorithmderivedinSection 7.1 wasappliedusingboththemCIandnCIkernel.RecallthatthePCAalgorithmisindependentofthepointprocesskernelused, 147

PAGE 148

Figure7-7. SpiketrainsfromrenewalpointprocessesforcomparisonofmCIwithnCIkernel.(A)\Training"spiketrainsforevaluationoftheeigendecompositioncoecientsofthePCAalgorithm,and(B)fortestingoftheresult(B).Eachdataset(trainingandtesting)isdividedintwohalves,eachcorrespondingtooneoftherenewalpointprocessmodels. theonlydierenceiswhichkernelisusedtocomputetheGrammatrixofthespiketrains.TheresultsoftheeigendecompositionareshowninFigure 7-8 .AlthoughthespiketrainvariabilityisconcentratedinasmallernumberofdimensionsforthemCIkernelthanthenCIkernel,thereacleardistinctionsbetweenthecontributionoftherstandsecondprincipalcomponentsinthelattercase(rstPCalmosttwiceasimportantassecondPC).Thiscanbejudgedmoreeasilyinthersteigenvectoroftheeigendecomposition,whichinthecaseofthenCIkernelshowsthattherstprincipalcomponentseparatesspiketrainsgeneratedfromdierentrenewalpointprocessmodel.Therelevanceofthisobservationcanassertedintheprojectionsofthedataset,showninFigure 7-9 .ForthemCIkernelcase,theprojectionsfromthetwopointprocessmodelsoverlapgreatly,beingonlynoticeablethehigherdispersionofspiketrainsfromtherstrenewalmodel(=0:5)duetotheirmoreirregularring.ForcaseusingthenCIkernel,however,the 148

PAGE 149

(a)EigenvaluesofthemCIGrammatrix. (b)FirsttwoeigenvectorsofthemCIGrammatrix. (c)EigenvaluesofthenCIGrammatrix. (d)FirsttwoeigenvectorsofthenCIGrammatrix.Figure7-8. EigendecompositionoftheGrammatrix,forthemCIandnCIkernels. rstprincipalcomponentaloneisresponsiblefortheseparationbetweenspiketrainsfromthetworenewalmodels,ashadbeennotedinFigure 7-8 (d).TheseresultsverifyoncemorethegeneralityofthenCIkernel,bybeingabletoquantifyanddiscriminatebetweenrenewalpointprocessmodels.Moreimportantly,theprojectionresultsinFigure 7-9 revealthattheuseofpointprocesskernelscapableofcopingwiththepointprocessmodelisveryimportantinensuringthatthedataistransformedintotheRKHSwhilepreservingthemodeldierences.Putdierently,aproperpointprocesskernelforagivenmodelcertiesthattheRKHSisrichenoughso 149

PAGE 150

(a)ProjectionofthetrainingsetspiketrainsusingmCIkernel. (b)ProjectionofthetestingsetspiketrainsusingmCIkernel. (c)ProjectionofthetrainingsetspiketrainsusingnCIkernel. (d)ProjectionofthetestingsetspiketrainsusingnCIkernel.Figure7-9. ProjectionofrenewalspiketrainsontothersttwoprincipalcomponentsusingmCIandnCIkernels.Thedierentpointmarksdierentiatebetweenspiketrainscorrespondingtoeachoneofthetemplates. 150

PAGE 151

thatlinearsuceinanalysingandprocessingthetransformeddatainthisspace.Because,inpracticethetrueunderlyingpointprocessmodelisunknownthesafestchoiceistowheneverpossibletotestusingthemostgeneralpointprocesskernelandcomparewithsimplerkernelstoinferaboutthecomplexityoftheunderlyingmodel. 151

PAGE 152

CHAPTER8CONCLUSIONANDTOPICSFORFUTUREDEVELOPMENTS 8.1ConclusionThepeculiarnatureofpointprocesshasmadetheapplicationofconventionalsignalprocessingmethodstotheirrealizationsdicultandimprecisetoapplyfromrstprinciples.Inthisrespect,binningiscurrentlythestandardapproachsinceittransformsthepointprocessintoadiscrete-timerandomprocesssuchthatconventionalsignalprocessingmethodscanbeused.However,binningisanimprecisemappingsinceinformationisirreversiblylostfromthepointprocessrealizations(see,forexample,Section 7.3.1 ).Themostpowerfulmethodologiestopointprocessanalysisarebasedonstatisticalapproachessincethedistributionsareestimateddirectly,thus,fullycharacterizingthepointprocess.Butsuchmethodologiesfaceseriousshortcomingswhenmultiplepointprocessesandtheircouplingsareconsideredsimultaneously,sincetheyareonlypracticalusinganassumptionofindependence.Nevertheless,processingofmultiplepointprocessesisveryimportantforpracticalapplications,suchasneuralactivityanalysis,withthewidespreaduseofmultielectrodearraytechniques.ThisdissertationpresentsareproducingkernelHilbertspace(RKHS)frameworkfortheanalysisofpointprocessesthathasthepotentialtoimprovethesetofmethodsandalgorithmsthatcanbedevelopedforpointprocessanalysis.Themaingoalofthisdissertationwastopresentthefundamentaltheoryinordertoestablishasolidfoundationandhopefullyenticefurtherworkalongthislineofreasoning.Indeed,thedualroleofthedissertationistoelucidatethesetofpossibilitiesthatareopenbytheRKHSformulationandtolinkthetheorytomethodsthatareincommonuse.SofurtherworkisneededtobringthepossibilitiesopenbyRKHStheorytofruitioninpointprocesssignalanalysis.ThecoreconceptofRKHStheoryistheconceptofinnerproductwhichisalsothefundamentaloperatorforsignalprocessingwithpointprocesses.Thereforemuchofthecontributionsofthisworkatthetheoreticallevelfocusedonshowinghowpoint 152

PAGE 153

processkernelscouldbedened,intermsofkernelsoneventcoordinatesorthestatisticaldescriptorsofthepointprocesses.Thelatterapproachis,inasense,anextensionoftheearlyworkof Parzen [ 1959 ]onstochasticprocessestopointprocessesbydeningbottom-upthestructureoftheRKHSonthestatisticsofthepointprocesses;thatis,theconditionalintensityfunctions(ingeneral).Thisresultprovidesasolidfoundationforfutureworkbothforpracticalalgorithmdevelopmentbutalsoonasimplewaytobringintotheanalysismorerealisticassumptionsaboutthestatisticsofpointprocesses.IndeedweshowthatthePoissonstatisticalmodelisbehindthesimplestdenitionoftheRKHS(thememorylesscross-intensitykernel)andthatthisRKHSprovidesalinearspacefordoingsignalprocessingwithpointprocesses.However,thesameframeworkcanbeappliedtoinhomogeneousMarkovintervalofevenmoregeneralpointprocessmodelswhichonlynowarebeginningtobeexplored.WewouldliketoemphasizethatbuildingaRKHSbottom-upisamuchmoreprincipledapproachthantheconventionalwaythatRKHSarederivedinmachinelearning,wherethelinktodatastatisticsisonlypossibleattheleveloftheestimatedquantities,notthestatisticaloperatorsthemselves.AnothertheoreticalcontributionistoshowtheexibilityoftheRKHSframework.Indeeditispossibletodenealternate,andyetunexplored,RKHSforpointprocessanalysisthatarenotlinearlyrelatedtotheintensityfunctions.Obviously,thiswillprovidemanypossibleavenuesforfutureresearchandthereisthehopethatitwillbepossibletoderivesystematicapproachestotailortheRKHSdenitiontothegoalofthedataanalysis.TherearebasicallytwodierenttypesofRKHSthatmimicexactlythetwomethodologiesbeingdevelopedinthemachinelearningandsignalprocessingliteratures:kernelsthataredataindependent()andkernelsthataredatadependent(CIkernels).Specicallyforpointprocesses,weshowinaspeciccasehowthattheformermaybeusedtocomposethelatter,buttheyworkwiththedatainverydierentways.ButwhatisinterestingisthatthesetwotypesofRKHSprovidedierentfeaturesinthetransformationtothespaceoffunctions.Theformerisamacroscopicdescriptorofthe 153

PAGE 154

spiketimeintervalsthatmaybeusableincoarseanalysisofthedata.Thelatterisafunctionaldescriptorofthedatabutitishardertocompute.Incurrentmethodsonlythelatterisbeingpursuedintheformofbinnedcross-correlation,butbyanalogywiththelargeimpactofkernelmethodsinstatisticallearning,anequallyimportantimpactoftheformermaybeexpected.Andyet,thetheoryandtheoperatorspresentedthisfarwillformthefoundationsforsuchfuturedevelopments.TherearealsopracticalimplicationsoftheRKHSmethodologypresentedinthisdissertation.SincetheRKHSisavectorspacewithaninnerproduct,alltheconventionalsignalprocessingalgorithmsthatinvolveinnerproductcomputationscanbeimmediatelyimplementedforpointprocessesintheRKHS.ThiswasillustratedinChapters 6 and 7 ,byderivingalgorithmsforclusteringandPCA,butmanyotherapplicationsarepossible,suchasltering.Notethattheclusteringalgorithmshowncouldalsobederivedusingcommondistancesmeasuresthathavebeendenedashasbeendonebefore[ Paivaetal. 2007 ].Butwestresstheeleganceoftheproposedformulationthatrstdenesthestructureofthespace(theinnerproduct)andthenleavesfortheusersthedesignoftheirintendedalgorithm,unliketheapproachespresentedsofarwhicharespecicfortheapplication.ThesamecanbeobservedinthederivationofthePCAalgorithmwherethederivationoccursintheRKHSandinawaythatisindependentoftheactualRKHSinducedbythepointprocesskernel.Thisisadvantageousasadvancesinpointprocesskernelsmaybeincorporatedinthederivedalgorithmsupontheiravailability,withouttheneedtorestructuretheimplementation.ThiswasdoneforbothclusteringandPCAforthecomparisonofthemCIandnCIkernels.Indeed,inbothcasesonlythenCIshowedsensitivitytotheparametersoftherenewalpointprocesses.Sinceinpracticethetruepointprocessmodelisunknown,thenCIkernelispreferableasitcanaccommodatepointprocessmodelsbeyondPoisson.Thetrade-oindoingsoisthat,inthecaseofthenonlinearCIkernelsdened,anotherkernelsizeparameter(ofK)needstobeselected,eventhoughinourexperimentstheresultsdependedonthisparameterquitecoarsely. 154

PAGE 155

TheRKHSframeworkisalsoofhighrelevancefordevelopmentofpointprocessanalysistools.ItwasshownthatthesimplestoftheCIkernelsconsideredisfundamentallyequaltothegeneralizedcross-correlation(GCC)whichextendsthemorecommonbinnedcross-correlation.ThisexposesthelimitationsofcurrentmethodologiesasitbringsforththeimplicitdependenceonthePoissonpointprocessmodel.Therefore,currentapproachescanaccuratelyquantifyatmostinteractionsintheratefunctions.Thegoodnewsarethatpointprocesskernelscapableofcopingwithmoregeneralpointprocessmodelswereshownhere.Thesekernelsareproperlydenedcovariancefunctions(Section 3.5.4 )whichcurrentanalysisoftenutilize.Hence,theycanreplacebinnedcross-correlation(orGCC)withoutmajorchangesincurrentparadigms.Therearestillothertopicsthatneedtoberesearchedforafullysystematicuseofthetechnique.Perhapsthemostimportantoneforpracticalapplicationsisthekernelsizeparameterofthekernelfunction.Thetheoryshowsclearlytheroleofthisfreeparameter;thatis,itsetsthescaleofthetransformationbychangingtheinnerproduct.Soitprovidesexibilitytotheresearcher,butalsosuggeststheneedtondtoolstohelpsetthisparameteraccordingtothedataandtheanalysisgoal.Fromaneurophysiologicalperspective,whichisparticularlyimportantinthiswork,thekernelsizehasabiologicalinterpretation.Becausethekernelfunctionutilizedintheestimationisassociatedwiththelteringofthepointprocess,andthesimilarityofthissteptothespike-to-membranepotentialconversion,thekernelsizecanbeinterpretedasthetimeconstantofthecellmembraneresistive-capacitivenetwork. 8.2TopicsforFutureDevelopmentsAssaidearlier,thisdissertationaimedprimarilytopresentthefundamentaltheoryandprovideexamplesforthereproducingkernelHilbertspace(RKHS)frameworkweproposeforprocessingofpointprocesses.However,therestillseveraltopicsthatneedworktofurthercompletethisresearchandbetterestablishthevalueoftheRKHSframework.Therearetwomaintopicsforfuturedevelopments: 155

PAGE 156

1. FilteringintheRKHS;and 2. DataecientCIkernelestimators.FilteringisthemostimportantsignalprocessingoperationforuseoftheRKHSframeworkinBMIs,whichrstmotivatedthiswork.AsreviewedinSection 2.4.3 ,thevastmajorityofcurrentBMIsapplytraditionallinear/nonlinearlteringmethodstobinnedspiketrainsbut,asshownforthePCAresultsinSection 7.3.1 ,thebinnedcross-correlationdoesnotfullycharacterizethestructureofspiketrains.Althoughconceptuallyitimplementsthesameidea,themCIkernelestimatoryieldedamoreconsistentoutcomeandthereforeitsuseinBMIshasthepotentialtoimprovecurrentresults.ThesemaybeimprovedfurtherbyutilizingthenCIkernel.ButevenifthemCIkernelisusedthereisasignicantimprovementovercurrentmethodologiesastheanalysiscanbeimplementedacrosstimescalesnaturallybyincorporatingtheestimatorwithmultiplekernelsizes.Putdierently,thekernelsizeinthepointprocesskernelestimatorcanbeutilizedasacontinuousparameterthatmeasurestheinteractionsoftheneuronsatvarioustimescales.Thedicultiesindevelopingaprocedureforlteringinthiscasearefundamentallythesameaswithotherkernelmethods,namelytheneedforregularization.Inthisregard,recentdevelopmentssuggestthatthisstepmayavoidedexplicitlyinonlineimplementationsifstochasticgradientsareutilized(sincethegradientregularizestheoptimization).Formally,PCAmaybeutilizedsincethedimensionallyreductionregularizestheGrammatrix.ThesecondtopicisthatofdevelopmentsoftheCIpointprocesskernels,andmostspecicallytheirestimators.InspiteofthesuccessfulresultsusingthenCIkernel,thiskernelwasnottrulydesignedforpointprocessesbeyondPoissonandhenceitssensitivityissomewhatlimitedespeciallyasthecomplexityofthepointprocessmodelincreases.Nevertheless,itishopedthattheseresultswillstemfurtherdevelopmentsandleadtothedesignofdataecientCIkernelestimators.Acrucialstepinderivinganestimator 156

PAGE 157

foraCIkernelistheestimationoftheconditionalintensityfunction,ascanbenoticedinSection 3.4 .Forestimationoftheratefunction,kernelsmoothingcanbeusedquiteeciently.ButcurrentmethodsforestimationoftheconditionalintensityfunctionaredataintensivewhichpreventsamorewidespreaduseofCIkernelscapableofcopingwithgeneralpointprocessmodels.Therefore,Ibelievethatthesolutionmightinvolvetheuseofsemi-parametricmodelsofthehistoryevolutioninconjunctionwiththenonlinearkernelsideastoenhancethedimensionalityofthepointprocesskernelmemory. 157

PAGE 158

APPENDIXABRIEFINTRODUCTIONTORKHSTHEORYInthisappendix,webrieyreviewsomebasicconceptsofkernelmethodsandRKHStheorynecessaryfortheunderstandingofthisdissertation.Thepresentationhereismeanttobeasgeneralandintroductoryaspossible,sothenotationwaspurposelychosentobedierentfromtheoneusedthroughoutthisdocument.ThefundamentalresultinRKHStheoryisthewell-knownMoore-Aronszajntheorem[ Aronszajn 1950 ; Moore 1916 ].LetKdenoteagenericsymmetricandpositivedenitefunctionoftwovariablesdenedonsomespaceE.Thatis,afunctionK(;):EE!Rwhichveries: (i) Symmetry:K(x;y)=K(y;x);8x;y2E. (ii) Positivedeniteness:foranynitenumberofl2Npointsx1;x2;:::;xl2E,andanycorrespondinglcoecientsc1;c2;:::;cl2R, lXm=1lXn=1cmcnK(xm;xn)0:(A{1)ThesearesometimescalledMercerconditions[ Mercer 1909 ].ThentheMoore-Aronszajntheorem[ Aronszajn 1950 ; Moore 1916 ]guarantiesthatthereexistsauniqueHilbertspaceHofrealvaluedfunctionsonEsuchthat,foreveryx2E, (i) K(x;)2H,and (ii) foranyf2H f(x)=hf();K(x;)iH:(A{2)TheidentityonEquation A{2 iscalledthereproducingpropertyofK,and,forthisreason,HissaidtobeanRKHSwithreproducingkernelK.Twoessentialcorollariesofthistheoremcanbeobserved.First,sincebothK(x;)andK(y;)areinH,wegetfromthereproducingpropertythat K(x;y)=hK(x;);K(y;)iH:(A{3) 158

PAGE 159

Hence,KevaluatestheinnerproductinthisRKHS.Thisidentityisthekerneltrick,wellknowninkernelmethods,andthemaintoolforcomputationinthisspace.Second,aconsequenceofthepreviouspropertieswhichcanbeexplicitlyseeninthekerneltrickisthat,givenanypointx2E,therepresenterofevaluationintheRKHSisx()=K(x;).NoticethatthefunctionaltransformationfromtheinputspaceEintotheRKHSHevaluatedforagivenx,andingeneralanyelementoftheRKHS,isarealfunctiondenedonE.Theseminalworkby Parzen [ 1959 ]providesaquiteinterestingperspectivetoRKHStheory(areviewispresentedin Wahba [ 1990 ,Chapter1]).Inhiswork,ParzenprovedthatforanysymmetricandpositivedenitefunctionthereexistsaspaceofGaussiandistributedrandomvariablesdenedonthesamedomainforwhichthisfunctionisthecovariancefunction.Assumingstationarityandergodicity,thisspacemightjustaswellbethoughtofasaspaceofrandomprocesses.Inotherwords,anykernelinducinganRKHSdenotessimultaneouslyaninnerproductintheRKHSandacovarianceoperatorinanotherspace.Furthermore,itisestablishedthatthereexistsanisometricisomorphism,thatis,aone-to-oneinnerproduct-preservingmapping,alsocalledacongruence,betweenthesetwospaceswhicharethussaidtobecongruent.ThisisanimportantresultasitsetsupacorrespondencebetweentheinnerproductduetoakernelintheRKHStoourintuitiveunderstandingofthecovariancefunctionandassociatedlinearstatistics.Simplyput,duetothecongruencebetweenthetwospacesanalgorithmcanbederivedandinterpretedinanyofthespaces. 159

PAGE 160

APPENDIXBACOMPARISONOFBINLESSSPIKETRAINMEASURES B.1IntroductionSpiketrainsimilaritymeasuresor,conversely,dissimilaritymeasuresareimportanttoolstoquantifytherelationshipamongpairsofspiketrains.Indeed,thedenitionofsuchameasurecanbeessentialforclassication,clusteringorotherformsofspiketrainanalysis.Forexample,justbyusingadistance(dissimilarity)measureitispossibletodecodetheappliedstimulusfromaspiketrain[ VictorandPurpura 1996 1997 ; WohlgemuthandRonacher 2007 ].Thisispossiblebecausethemeasureisusedtoquantifyhowmuchthespiketraindiersfroma\template"orsetsofreferencespiketrainsforwhichtheinputstimulusisknownand,hence,classiedaccordingly(Figure B-1 ).However,naturallythesuccessofthisclassicationisdependentofthediscriminativeabilityofthemeasure.Atraditionalmeasureofsimilaritybetweentwospiketrainsistomeasurethe(empirical)cross-correlationofthebinnedspiketrains[ Brownetal. 2004 ].However,toavoidthedicultiesassociatedwithbinningandtopreventestimationerrorsofinformationwhenbinningisdone,binlessspiketraindissimilaritymeasureshavebeenproposed.ThreewellknownsuchmeasureswhichweshallconsiderforcomparisonareVictor-Purpura's(VP)distance[ VictorandPurpura 1996 1997 ] 1 ,vanRossum'sdistance[ vanRossum 2001 ]andthecorrelation-basedmeasureproposedby Schreiberetal. [ 2003 ].Thesemeasureshavebeenutilizedindierentneurophysiologicalparadigms( Victor [ 2005 ]andreferenceswithin)andfordierenttasks,suchasclassication[ VictorandPurpura 1996 1997 ]andclusteringofspiketrains[ Fellousetal. 2004 ; Paivaetal. 2007 ; 1 Actually,intheirworks, VictorandPurpura [ 1996 1997 ]proposednotonebutseveralspiketraindistances.Namely,Dspike[q],Dinterval[q],Dcount[q]andDmotif[q].Inthisstudy,andasinmostreferencestotheirworks,VPdistancereferstoDspike[q]forafaircomparisontotheotherdistancesconsideredinthisstudy. 160

PAGE 161

FigureB-1. Typicalexperimentalsetupforclassicationusingspiketraindissimilarities.Inthissetupthemeasureisutilizedtoquantifythedissimilaritybetweenthenewspiketrainandthereferencespiketrainsforeachofthestimulus.Then,theunlabeledstimulusisinferredastheonecorrespondingtotheclassforwhichthenewspiketrainhassmalleraveragedissimilarity. ToupsandTiesinga 2006 ].However,wefeelthatinneitheroftheseworkswasthechoiceofthemeasureusedhavebeenproperlyarguedversusthecandidates.Thisisperhapsbecause,totheauthorsknowledge,asystematiccomparisonhasnotyetbeenattemptedintheliterature.Theworkby Kreuzetal. [ 2007 ]comparestheISIdistanceproposedinthatpaperwithseveralspiketrainmeasures,includingtheonesconsideredinthiswork.However,thisisdoneonlyforsynchronyofspiketrainsgeneratedunderaspecialmodelwithquitestrongcouplingsamongneurons.Thischapterllsthisvoidbycomparingtheabovementionedspiketrainmeasuresinmultipleparadigmsandunderrealisticscenarios.AswillbeshownfromthepresentationinSection B.2 ,eachmeasureimpliesagivenkernelfunctionthatmeasuressimilarityintermsofasinglepairofspiketimes.Anotherissueaddressedherewastowhatextentthiskernelaectstheperformanceofeachmeasure.Therefore,inspiredbytheideasintroducedinChapter 3 ,themeasuresarerstextendedtoasetoffourkernelsandcomparedforallofthese.Byevaluatingthemeasuresusingallofthesekernelsthecomparisonismadekernelindependentandshowstheconnectionandgeneralityoftheprinciplesusedindesigningthemeasures. 161

PAGE 162

B.2BinlessSpikeTrainDissimilarityMeasures B.2.1Victor-Purpura'sDistanceHistorically,Victor-Purpura's(VP)distance[ VictorandPurpura 1996 1997 ]wastherstbinlessdistancemeasureproposedintheliterature.TwokeydesignconsiderationsinthedenitionofthisdistancewerethatitneededtobesensitivetotheabsolutespiketimesandwouldnotcorrespondtoEuclideandistancesinavectorspace.Therstconsiderationwasduetothefactthatthedistancewasinitiallytobeutilizedtostudytemporalcodinganditsprecisioninthevisualcortex.Asstatedbytheauthors,thebasichypothesisisthataneuronisnotsimplyaratedetectorbutcanalsofunctionasacoincidencedetector.Withinthisrespectthedistanceiswellmotivatedbyneurophysiologicalideas.Thesecondconsiderationisbecause,inthiswayitis\notbasedonassumptionsabouthowresponsesshouldbescaledorcombined"[ VictorandPurpura 1996 ].TheVPdistancedenesthedistancebetweenspiketrainsasthecostintransformingonespiketrainintotheother.Threeelementaryoperationsintermsofsinglespikesareestablished:movingonespiketoperfectlysynchronizewiththeother,deletingaspike,andinsertingaspike.Onceasequenceofoperationsisset,thedistanceisgivenasthesumofthecostofeachoperation.Thecostinmovingaspikeattmtotnisqjtmtnj,whereqisaparameterexpressinghowcostlytheoperationis.Becauseahigherqmeansthatthedistanceincreasesmorewhenaspikeneedstobemoved,thedistanceasafunctionofqexpressestheprecisionofthespiketimes.Thecostofdeletingorinsertingaspikeissettoone.Sincethetransformationcostforthespiketrainsisnotunique,thedistanceisnotyetwelldened.Moreover,thiscriterionneedstoguaranteethefundamentalaxiomsofadistancemeasureforanyspiketrainssi,sjandsk: (i) Symmetry:d(si;sj)=d(sj;si) (ii) Positiveness:d(si;sj)0,withequalityholdingifandonlyifsi=sj (iii) Triangleinequality:d(si;sj)d(si;sk)+d(sk;sj): 162

PAGE 163

Toensurethetriangleinequalityanduniquenessofthedistancebetweenanytwospiketrains,thesequencewhichyieldstheminimumcostintermsoftheoperationsisused.Therefore,theVPdistancebetweenspiketrainssiandsjisdenedas dVP(si;sj),minC(si$sj)XlKqtici[l];tjcj[l];(B{1)whereC(si$sj)isthesetofallpossiblesequencesofelementaryoperationsthattransformsitosj,orvice-versa,andc()[]2C(si$sj).Thatis,ci[l]denotestheindexofthespiketimeofsimanipulatedinthelthstepofasequence.Kq(tici[l];tjcj[l])isthecostassociatedwiththestepofmappingtheci[l]thspikeofsiattici[l]totjcj[l],correspondingtothecj[l]thspikeofsj,orvice-versa.Inotherwords,Kqisadistancemetricbetweentwospikes.Supposetwospiketrainswithonlyonespikeeach,themappingbetweenthetwospiketrainsisachievedthroughthethreeabovementionedoperationsandthedistanceisgivenby Kq(tim;tjn)=minqjtimtjnj;2=8><>:qjtimtjnj;jtimtjnj<2=q2;otherwise:(B{2)Thismeansthatifthedierencebetweenthetwospiketimesissmallerthan2=qthecostislinearlyproportionaltotheirtimedierence.However,ifthespikesarefartherapartitislesscostlytosimplydeleteoneofthespikesandinsertitattheotherlocation.Showninthisway,Kqisnothingbutascaledandinvertedtriangularkernelappliedtothespiketimes.Thisperspectiveoftheelementarycostfunctioniskeytoextendthiscosttootherkernels,aswewillpresentlater.Atrstglanceitwouldseemthatthecomputationalcomplexitywouldbeunbearablebecausetheformulationofthealgorithmdescribesthedistanceintermsofafullsearchthroughallallowedsequencesofelementaryoperations.Luckily,ecientdynamic 163

PAGE 164

FigureB-2. Spiketrainandcorrespondinglteredspiketrainutilizingacausalexponentialfunction(Equation B{4 ). programmingalgorithmsweredevelopedwhichreduceittoamoremanageablelevelofO(NiNj)[ VictorandPurpura 1996 ],i.e.,thescaledproductofthenumberofspikesinthespiketrainswhosedistanceisbeingcomputed. B.2.2vanRossum'sDistanceSimilartotheVPdistance,thedistanceproposedby vanRossum [ 2001 ]utilizesthefullresolutionofthespiketimes.However,theapproachtakenisconceptuallysimplerandmoreintuitive.Simplyput,vanRossum'sdistance[ vanRossum 2001 ]istheEuclideandistancebetweentheexponentiallylteredspiketrains. 2 Aspiketrainsidenedonthetimeinterval[0;T]andspiketimesftim:m=1;:::;Nigcanbewrittenasacontinuous-timesignalasasumoftime-shiftedimpulses, si(t)=NiXm=1(ttim);(B{3)whereNiisthenumberofspikesintherecordinginterval.Inthisperspective,thelteredspiketrainisthesumofthetime-shiftedimpulseresponseofthesmoothinglter,h(t), 2 Filteredspiketrainscorrespondtowhatisoftenreferredtoas\shotnoise"inthepointprocessesliterature[ Papoulis 1965 ,Section16.3]. 164

PAGE 165

andcanbewrittenas fi(t)=NiXm=1h(ttim):(B{4)Forthesmoothinglter, vanRossum [ 2001 ]proposedtouseacausaldecayingexponentialfunction,writtenmathematicallyash(t)=exp(t=)u(t),withu(t)beingtheHeavisidestepfunction(illustratedinFigure B-2 ).TheparameterinvanRossum'sdistancecontrolsthedecayrateoftheexponentialfunctionand,hence,theamountofsmoothingthatisappliedtothespiketrain.Thus,itdetermineshowmuchvariabilityinthespiketimesisallowedandhowitiscombinedintotheevaluationofthedistance.Inessence,playsthereciprocalroleoftheqparameter(Equation B{2 )fortheVPdistance.Thechoicefortheexponentialfunctionwasduetobiologicalconsiderations.Theideaisthataninputspikewillevokeapost-synapticpotentialatthestimulatedneuronwhich,simplistically,canbeapproximatedthroughtheexponentialfunction[ DayanandAbbott 2001 ].Intermsoftheirlteredcounterparts,itiseasytodeneadistancebetweenthespiketrains.AnintuitivechoiceistheusualEuclideandistance,L2([0;T]),betweensquareintegrablefunctions.Thedistancebetweenspiketrainssiandsjisthereforedenedas dvR(si;sj),1 Z10[fi(t)fj(t)]2dt:(B{5)vanRossum'sdistancealsoseemsmotivatedbytheperspectiveofaneuronasacoincidencedetector.Thisperspectivemaybeinducedbythedenition.Whentwospiketrainsare\close"moreoftheirspikeswillbesynchronized,whichtranslatesintoasmallerdierenceofthelteredspiketrainsandthereforeyieldsasmallerdistance.Despitethisformulation,themulti-scalequanticationcapabilityofthedistancewasnoticedbeforeby vanRossum [ 2001 ].Thebehaviortransitionssmoothlyfromacountofnon-coincidencespikestoadierenceinspikecountasthekernelsizeisincreased.ThisperspectivecanbeobtainedfromEquation B{4 ifonenoticesthatitcorrespondstokernelintensityestimationwithfunctionh[ Reiss 1993 ].Inmorebroadtermsonecan 165

PAGE 166

thusthinkofvanRossum'sdistanceastheL2([0;1))distancebetweentheestimatedintensityfunctionsattimescale.Thus,vanRossum'sdistancecanbeusedtomeasurethedissimilaritybetweenspiketrainsatanytimescalesimplybyselectingappropriately.Evaluationofthedistanceisnumericallystraightforward,asitdirectlyimplementsEquation B{5 .Butexplicitcomputationofthelteredspiketrainsandintegralinadiscrete-timesimulationiscomputationallymoreintensivethanevaluatingtheVPdistancewhichdependsonlyonthenumberofspikesinthespiketrains.Furthermore,thecomputationburdenwouldincreaseproportionallytothelengthofthespiketrainsandinverselyproportionaltothesimulationstep.However,asshownby Paivaetal. andutilizedin Paivaetal. [ 2007 ],vanRossum'sdistancecanbeevaluatedintermsofacomputationallyeectiveestimatorwithorderO(NiNj),givenas dvR(si;sj)=1 224NiXm=1NiXn=1L(timtin)+NjXm=1NjXn=1L(tjmtjn)35+NiXm=1NjXn=1L(timtjn);(B{6)whereL()=exp(jj=)istheLaplaciankernel.Thus,thisdistancecanbecomputedwiththesamecomputationalcomplexityastheVPdistance. B.2.3Schreiberetal.InducedDivergenceThethirddissimilaritymeasureconsideredinthispaperisderivedfromthecorrelation-basedmeasureproposedby Schreiberetal. [ 2003 ].LikevanRossum'sdistance,thecorrelationmeasurewasalsodenedintermsofthelteredspiketrains.Insteadofusingthecausalexponentialfunction,however,SchreiberandcoworkersproposedtoutilizetheGaussiankernel.Thecoreideaofthiscorrelationmeasureistheconceptofdotproductbetweenthelteredspiketrains.Actually,inanyspacewithaninnerproducttwotypesofquadraticmeasuresarenaturallyinduced:theEuclideandistance,andacorrelationcoecient-likemeasure,duetotheCauchy-Schwarzinequality.TheformercorrespondstotheconceptutilizedbyvanRossum,whereasthelatterisconceptuallyequivalenttothedenitionproposedbySchreiberandassociates.So,inthissense,thetwomeasuresaredirectlyrelated.Nevertheless,itmustbeemphasizedthat,liketheVP 166

PAGE 167

distance,thismeasureisnon-Euclideansinceitisanangularmetricoflteredspiketrains[ Paivaetal. ].Indeningthemeasure,writethelteredspiketrainsas gi(t)=NiXm=1G=p 2(ttim);(B{7)whereG=p 2(t)=exp[(t)2=2]istheGaussiankernel.NoticethedependenceofthelteringonwhichplaysinthiscasethesameroleasintheexponentialfunctioninvanRossum'sdistance,andisinverselyrelatedtoqinVPdistance.Assumingadiscrete-timeimplementationofthemeasure,thenthelteredspiketrainscanbeseenasvectors,forwhichtheusualdotproductcanbeused.Basedonthis,theCauchy-Schwarz(CS)inequalityguarantiesthat j~gi~gjjk~gikk~gjk;(B{8)wheregi,gjarethelteredspiketrainsinvectornotation,and~gi~gjandk~gik,k~gikdenotesthelteredspiketrainsdotproductandnorm,respectively.Thenormisgivenasusualbyk~gik=p ~gi~gi.Becausebyconstructionthelteredspiketrainsarenon-negativefunctions,thedotproductisalsonon-negative.Consequently,rearrangingtheCauchy-Schwarzinequalityyieldsthecorrelationcoecient-likequantify, r(si;sj)=~gi~gj k~gikk~gjk;(B{9)proposedby Schreiberetal. [ 2003 ].Noticethatliketheabsolutevalueofthecorrelationcoecient,0r(si;sj)1.Equation B{9 ,however,takestheformofasimilaritymeasure.Utilizingtheupperbound,adissimilaritycanbeeasilyderived, dCS(si;sj)=1r(si;sj)=1~gi~gj k~gikk~gjk:(B{10)InlightoftheperspectivepresentedhereweshallhereafterrefertodCSastheCSdissimilaritymeasure. 167

PAGE 168

TheCSdissimilarity,liketheprevioustwomeasures,canalsobeutilizeddirectlytomeasuredissimilarityintheringratesofspiketrainsmerelybychoosingalarge.SimilartovanRossum'sdistance,thisisshownexplicitlyintheformulationofthemeasureintermsoftheinnerproductofintensityfunctions,withthetimescalespeciedby.AnimportantdierencewithregardstotheVPandvanRossum'sdistancesneedstobepointedout.dCSisnotadistancemeasure.Althoughitistrivialtoprovethatitveriesthesymmetryandpositivenessaxioms,themeasuredoesnotfulllthetriangleinequality.Nevertheless,sinceitguarantiesthersttwoaxiomsitiswhatiscalledintheliteratureapre-metric[ PekalskaandDuin 2005 ].Inthedenitionofthemeasureand,moreimportantly,intheutilizationoftheconceptofthedotproductthelteredspiketrainswereconsiderednite-dimensionalvectors[ Schreiberetal. 2003 ].Ifthisnaveapproachistaken,thenthecomputationalcomplexityinevaluatingthemeasurewouldsuerfromthesamelimitationsasthedirectimplementationofvanRossum'sdistance.But,likethelatter,adataeectivemethodcanbeobtainedinthesamewaytocomputethedistance[ Paivaetal. ], dCS(si;sj)=1PNim=1PNjn=1exph(timtjn)2 22i r PNim;n=1exph(timtin)2 22iPNjm;n=1exph(tjmtjn)2 22i:(B{11)EvaluatingthedistanceusingthisexpressionhasacomputationalcomplexityoforderO(NiNj),justlikethetwopreviouslypresentedmeasures. B.3ExtensionoftheMeasurestoMultipleKernelsFromthepreviouspresentationitshouldbeobservablethateachmeasurewasoriginallyassociatedwithaparticularkernelfunctionwhichmeasuresthesimilaritybetweentwospiketimes.Interestingly,thekernelfunctionisfoundtobedierentinallthreesituations.Inanycase,itisremarkablethatthemeasuresareconceptuallydierentirrespectiveofthedierencesinthekernelfunction.Tofurthercompleteour 168

PAGE 169

studywewerealsointerestedinverifyingtheimpactofdierentkernelfunctionsineachmeasure.Inthissectionwefurtherdeveloptheseideas.Inparticular,wepresentthedetailsinvolvedinreplacingthedefaultkernelforeachdissimilaritymeasureand,wheneverpertinent,intuitivelyexplainhowthisapproachrevealstheconnectionsbetweenthemeasures.Itshouldberemarkedthatsimilarconsiderationshavebeenpresentedpreviouslyby SchrauwenandCampenhout [ 2007 ],althoughunderadierentanalysisparadigm.InSection B.2.1 thedistancebetweentwospikesfortheVPdistanceisdenedthroughthefunctionKq.ThisdistancerepresentstheminimumcostintransformingaspikeintotheotherintermsoftheelementaryoperationsdenedbyVictorandPurpura.Asbrieypointedout,thisfunctionisequivalenttohaving Kq(tim;tjn)=211=q(timtjn);(B{12)whereisthetriangularkernelwithparameter, (x)=8><>:1jxj=(2);jxj<20;jxj2;(B{13)whichis,inessence,asimilaritymeasureofthespiketimes.Noticethatthisperspectivedoesnotchangethenon-EuclideanpropertiesoftheVPdistancesincethosepropertiesarearesultoftheconditioninEquation B{1 .Putinthisway,itseemsobviousthatotherkernelfunctionsmaybeusedinplaceofthetriangularkernel,asbrieyalludedby VictorandPurpura [ 1997 ].ThekernelintheVPdistanceisnotexplicitinthedenition.Rather,isthecostassociatedwiththethreeelementaryoperations.Similarly,invanRossum'sdistanceandCSdissimilaritymeasuretheperspectiveofakerneloperatingonspiketimesisnotexplicitinthedenition.Thedierencehoweveristhatthekernelarisesnaturallyasanimmediatebyproductofthelteringofthespiketrains.Thisresultisnoticeable 169

PAGE 170

FigureB-3. (a)Kernelsutilizedinthisstudyand(b)thecorrespondingKqfunctioninducedbyeachofthekernels. intheexpressionsforcomputationalecientevaluationgivenbyEquation B{6 andEquation B{11 .Again,andjustasproposedfortheVPdistance,alternativekernelfunctionscanbeutilizedintheevaluationofthedissimilaritymeasuresinsteadoftheproposedkernelbytheoriginalconstruction.Assaidearlier,eachofthespiketrainmeasuresconsideredherewasdenedwithadierentkernelfunction.Toprovideasystematiccomparison,eachmeasurewasevaluatedwithfourkernels:thetriangularkernelinEquation B{13 ,andtheLaplacian,Gaussian, 170

PAGE 171

andrectangularkernels,Laplacian:(x)=expjxj (B{14)Gaussian:(x)=expx2 22 (B{15)Rectangular:(x)=8><>:1;jxj<0;jxj; (B{16)Forreference,thesefourkernelsandinduceddistancefunctionKqintermsofeachofthekernelsaredepictedinFigure B-3 .Inthiswayeachmeasurewasevalutedforthekernelitwasoriginalydenedandtheotherkernelsforafaircomparison.Notethatifotherkernelswheretobechosenthesewouldhavetobesymmetric,maximumattheorigin,andalwayspositive,toensurethesymmetryandpositivenessofthemeasure.Additionally,fortheVPdistancetobewellposed,thekernelsneedtobeconcavesothattheoptimizationinEquation B{1 garantiesthetriangleinequality.However,theGaussianandrectangularkernelsarenotconcaveandthusforthesekernelstheVPmeasureisapre-metric.Thismeansthatwhenthesekernelsareusedtheresultingdissimilarityisnotawelldeneddistance.Nevertheless,weutilizethesekernelshereregardlesssinceouraimsaretostudytheeectofthethiskernelofthediscriminationability,andalsotocomparethemeasuresappartthisfactor.Itisinterestingtoconsidertheconsequencesintermsofthelteredspiketrainsassociatedwiththechoiceofeachofthefourkernelspresented.Asmotivatedby vanRossum [ 2001 ],thebiologicalinspirationbehindtheideainutilizinglteredspiketrainsisthattheycanbethoughtofaspost-synapticpotentialsevokedattheeerentneuron.Inthissense,kernelsaremathematicalrepresentationsoftheinteractionsinvolvedwiththisidea.Asshownbefore,theLaplacianfunctionresultsfromtheautocorrelationofaone-sidedexponentialfunction.Likewise,theGaussianfunction(withkernelsizescaledbyp 2)resultsfromitsownautocorrelation.Thetriangularresultsfromtheautocorrelationoftherectangularfunction.Thesmoothingfunctionassociatedwiththerectangular 171

PAGE 172

FigureB-4. Estimatedpdfofthemeasuresforeachkernelconsidered(green)andcorrespondingttedGaussianpdf(blue).Thepdfwasestimatedbyanormalizedhistogramoftheevaluationofthemeasurewithkernel/binsize2msfor1000pairsofuncorrelatedspiketrainswithmeanringrate20spk/sandjitternoiseof3ms.(DetailscanbefoundinSection B.4.3 .) functioncorrespondstotheinverseofthesquarerootofasincfunction.BasedontheseobservationsitseemstousthattheLaplaciankernelis,fromthefourkernelsconsidered,themostbiologicallyplausible. B.4ResultsInthissectionresultsareshownforthethreedissimilaritymeasuresintroducedintermsofanumberofparameters:kernelfunction,ringrate,kernelsize,and,inthelastparadigmpresented,synchronyandjitteroftheabsolutespiketimes.Threesimulationparadigmsarestudied.Ineachparadigmwewillbeinterestedinverifyinghowwellcanthedissimilaritymeasurementsdiscriminatedierencesinspiketrainswithregardstoaspecicfeature.Toquantifythediscriminationabilityofeachmeasureinascale-freemanner,theresultsshallbepresentedandanalyzedintermsofadiscriminantindexdenedas (A;B)=d(A;B)d(A;A) p 2d(A;B)+2d(A;A);(B{17)whered(A;A),d(A;B)denotesthemeanofthedissimilaritymeasureevaluatedbetweenspiketrainsfromthesameanddierentsituations,respectively,and2d(A;A),2d(A;B)denotesthecorrespondingvariances.Theuseofadiscriminantindexwaschoseninstead 172

PAGE 173

of,forexample,ROCplotsforeaseofdisplayandanalysis,andbecauseinthiswaytheconclusionsdrawnhereareclassier-free.(A;B)quantieshowwelltheoutcomeofthemeasurecanbeusedtodierentiatethesituationAfromthesituationB.IntermsofFigure B-1 ,thinkthatd(A;A);2d(A;A)characterizesthedistributionofthedissimilaritymeasureevaluationforspiketrainsinresponsetostimulusA,andd(A;B);2d(A;B)characterizesasimilardistributionbutinwhichthedissimilaritiesareevaluatedbetweenaspiketrainevokedbystimulusAandaspiketrainevokedbystimulusB.ThisissupportedbythefactthatthedistributionoftheevaluationofthemeasurescanbereasonablyttedtoaGaussionpdf(Figure B-4 ).Therefore,thediscriminantindexisutilizedinthesimulatedexperimentalparadigmstocomparehowwellthedissimilaritydistinguishesspiketrainsgeneratedunderthesameversusdierentconditions,withregardstoaparameterspecifyinghowdierentspiketrainsfromdierentstimulusare.ThediscriminantindexisconceptuallysimilartothatoftheFisherlineardiscriminantcost[ Dudaetal. 2000 ].Akeydierencehoweveristhattheabsolutevalueisnotused.Thisisbecausenegativevaluesoftheindexcorrespondtounreasonablebehaviorofthemeasure;thatis,thedissimilaritymeasureyieldssmallervaluesbetweenspiketrainsgeneratedunderdierenceconditionsthanspiketrainsgeneratedforthesamecondition.Obviously,intuitivelythedesiredbehavioristhatthedissimilaritymeasureyieldsaminimumforspiketrainsgeneratedsimilarly.Forcontrasttothebinlessdissimilaritymeasuresconsidered,resultsarealsopresentedforabinnedcross-correlationbaseddissimilaritymeasure,denoteddCC.ThismeasureisdenedjustliketheCSdissimilaritythroughEquation B{10 .Thedierenceisthatnow~giand~gjarenitedimensionalvectorscorrespondingtothebinnedspiketrainsand,thus,~gi~gjistheusualEuclideandotproductbetweentwovectors.NoticethatdCCisinessenceequivalenttoquantizethespiketimes(withquantizationstepequaltothebinsize)andevaluatingdCSusingtherectangularkernel,withkernelsizeequaltohalfthebinsize.Hence,dCCcanbealternativelycomputedutilizingEquation B{11 .Theformer 173

PAGE 174

FigureB-5. Valueofthedissimilaritymeasuresforeachkernelconsideredasafunctionofthemodulatingspiketrainringrate.Alldissimilarityevaluationarewithregardstoanhomogeneousspiketrainwithaveragerate20spk/s.Foreachmeasureandkernel,resultsaregivenforfourdierentkernelsizes(showninthelegend)intermsofthemeasureaveragevalueplusorminusonestandarddeviation.Thestatisticsofthemeasureswereestimatedover1000randomlygeneratedpairsofspiketrains. approachismoreadvantageousforlargebinsizewhereasthelatteriscomputationallymoreeectiveforsmallerbinsize(largernumberofbins). B.4.1DiscriminationofDierenceinFiringRateTherstparadigmconsideredwasintendedtoanalyzethecharacteristicsofeachmeasurewithregardstotheringrateofonespiketrainrelativelytoanotherofxedringrate.Thekeypointwastounderstandifthemeasurescouldbeusedtodierentiatetwospiketrainsofdierentringrates.Thisisimportantbecauseneuronshavebeenfoundtooftenencodeinformationinthespiketrainringrates[ Adrian 1928 ; Dayan 174

PAGE 175

FigureB-6. Discriminantindexofthedissimilaritymeasuresforeachkernelasafunctionofthemodulatingspiketrainringrate.SeetheresultsinFigure B-5 forreference.Thedierentcurvesarefordierentkernelsizes(showninthelegend). andAbbott 2001 ; Riekeetal. 1999 ].Tosimplifymatters,allspiketrainsweresimulatedasonesecondlonghomogeneousPoissonprocesses.Althoughthissimplicationisunrealistic,itallowsarstanalysiswithouttheintroductionofadditionaleectsduetomodulationofringratesinthespiketrains.Thescenariowheretheringratesaremodulatedovertimeisconsideredinthenextsection.Anotherimportantfactorintheanalysisisthespiketrainlength.Naturally,inthisscenario,thediscriminationofthemeasuresisexpectedtoimproveasthespiketrainlengthisincreasedsincemoreinformationisavailable.Inpracticehoweverthisvalueisoftensmallerthanonesecond.Thus,thevaluewaschosenasacompromisebetweenareasonablevalueforactualdataanalysisandgoodstatisticalillustrationofthepropertiesofeachmeasure. 175

PAGE 176

Inourstudy,simulationsweremadeforeachdissimilaritymeasureutilizingeachofthefourdescribedkernels.Ineachcase,theanalysiswasrepeatedforfourkernelsizes,10,25,50and100milliseconds.Thekernelsizesusedwerepurposelychosenrelativelylargesinceringrateinformationcanonlybeextractedataslowertimescale.TheresultsareshowninFigure B-5 intermsofmeanvaluesplusorminusonestandarddeviation,asestimatedfrom1000randomlygeneratedspiketrainpairs.Foreachpair,oneofthespiketrainswasgeneratedatareferenceringrateof20spk/s,whereastheringrateoftheotherwasoneof2.5to40spk/s,instepsof2.5spk/s.Utilizingtheestimatedstatistics,thediscriminationprovidedbythemeasureswasevaluatedintermsofthediscriminationindex(Equation B{17 )withregardstotheresultswhenbothspiketrainshaveringrate20spk/s.TheresultsareshowninFigure B-6 .TheresultsforVPandvanRossum'sdistancesreecttheimportanceofthechoiceoftimescale,materializedintheformofthekernelsizeselection.Onlyforthelargestkernelsize(100ms)didthesetwodistancesbehaveasweintuitivelyexpected.Thisisnotsurprisingsincediscriminationcanonlyoccurifthedissimilaritycanincorporateanestimationoftheringrateinitsevaluation.Evenforthiskernelsizethediscriminantindexcurveshowsasmallbiastowardssmallerringrates.Thisisnaturalsincetheoptimalkernelsizeisinnity,andsmallerkernelsizetendstoresultinbiasrelatedtothetotalnumberofspikes.ThediscriminationbehavioroftheCSdissimilarityhoweverseemsnearlyinsensitivetothechoiceofthekernelsize.Ontheotherhand,whentheringrateisabovethereferencetheoutcomeisnotthedesired.Forlowerringrates,thepositivediscriminationindexisduetothepresenceofthenormofthespiketraininthedenominatorofthedenition.Oneofthemostremarkableobservationsistheconsistencyoftheresultsforeachmeasurethroughoutthefourkernels.Althoughtherearesubtledierencesinvaluestheyseemtobeofimportanceonlyforsmallkernelsizesforwhich,aspointedout,theresultsarenotsignicantanyway.ComparingwiththeresultsfortheCCdissimilarityweverifytheresemblancewiththeCSdissimilarity.Likethelatter,theCC 176

PAGE 177

FigureB-7. Valueofthedissimilaritymeasuresforeachkernelintermsofthephasedierenceoftheringratemodulation.Likeinthepreviousparadigm,resultsareshownforeachmeasure,kernel,andfourdierentkernelsizes(showninthelegend)intermsofthemeasureaveragevalueplusorminusonestandarddeviation.Thestatisticswereestimatedover1000randomlygeneratedpairsofspiketrains. dissimilarityalsoisunabletocorrectlydistinguishincreasesinringrateofonespiketrainwithrespecttotheother. B.4.2DiscriminationofPhaseinFiringRateModulationThescenariodepictedinthepreviousparadigmisobviouslysimplistic.Inthiscasestudy,analternativesituationisconsideredinwhichspiketrainsmustbediscriminatedthroughdierencesintheirinstantaneousringrates.SpiketrainsweregeneratedasonesecondlonginhomogeneousPoissonprocesseswithinstantaneousringrategivenbysinusoidalwaveformsofmean20spk/s,amplitude10spk/sandfrequency1Hz.Apairofspiketrainswasgeneratedatatimeandthephasedierenceofthesinusoidalwaveforms 177

PAGE 178

FigureB-8. DiscriminantindexofthedissimilaritymeasuresforeachkernelintermsofthephaseoftheringratemodulationasgivenbyFigure B-7 .Thedierentcurvesarefordierentkernelsizes(showninthelegend). usedtomodulatetheringrateofeachspiketrainvariedfrom0to360degrees.Thegoalwastoverifyifthemeasuresweresensitivetoinstantaneousdierencesintheringrateascharacterizedbythemodulationphasedierence.Thistooisasimplicationofwhatisoftenfoundinpracticewhereringrateschangeabruptlyandinannon-periodmanner.Nevertheless,theparadigmaimsatrepresentingageneralsituationwhilesimultaneouslybeingrestrictedtoallowforatractableanalysis.Obviously,theresultsaresomewhatdependentonourchoiceofsimulationparameters.Forexample,lowermeanringrateswouldmeanthatthedissimilaritymeasureswouldbelessreliableand,hence,havehighervariance.Thiscouldbepartiallycompensatedbyincreasingthespiketrainlength.However,theabovevaluesareanattempttoapproximaterealdata. 178

PAGE 179

Thesimulationprotocolissimilartothatofthecaseanalyzedintheprevioussection.Foreachphasedierence,werandomlygenerated1000spiketrainpairssuchthattheringratemodulationofthetwospiketrainsdieredbythephasedierenceandappliedthedissimilaritymeasuresusingeachofthefourdescribedkernels.Asbefore,theanalysiswasrepeatedforfourkernelsizes,10,25,50and100milliseconds.Again,thekernelsizesusedwerechosenlargesinceringrateinformationcanonlybeextractedataslowertimescale.ThestatisticsofthedissimilaritymeasuresareshowninFigure B-7 .TheanalysisoftheseresultswiththediscriminationindexwithrespecttothestatisticsofeachmeasureatzerophaseisdepictedinFigure B-8 .Inthisparadigm,themaximumvalueofthemeasureswasdesiredtooccurat180,withamonotonicallyincreasingbehaviorforphasedierencessmallerandmonotonicallydecreasingforphasedierencesgreater.AsFigure B-8 shows,allmeasuresperformedsatisfactorilyusinganyofthefourkernelsandatanykernelsize.TheCSdissimilarityhasthebestdiscriminationwiththediscriminationindexreaching0.8,comparedtoamaximumvalueof0.65forthesecondbest.Ontheotherend,overalltheCC-baseddissimilarityperformedtheworse.ComparingwiththeCSdissimilarity(whichdiersonlybecausethespiketimesarenotquantized)weverifyonceagainthedisadvantagesofdoingbinning.Withregardstotheeectofeachkernel,theGaussiankernelconsistentlyyieldsthebestdiscriminationforthesamekernelsize.Conversely,theLaplacianandrectangularkernelsseemtoperformtheworst,althoughthisobservationislargelymeasuredependent.Asexpected,andsimilarlytothepreviousparadigm,thebestdiscriminationisobtainedforthelargestkernelsizesinceityieldsabetterestimationoftheintensityfunction.Itisnoteworthyhoweverthatinthisparadigmthekernelsizecannotbechosentoolarge,otherwisetheintensityfunctionwouldbeoversmoothed,thusreducingthedierentiationbetweenphasesanddecreasingthediscriminationperformance.Thisphenomenonwasobservedwhenweattemptedakernelsizeof250ms(notshown). 179

PAGE 180

FigureB-9. Valueofthedissimilaritymeasureforeachkernelasafunctionofthesynchronyamongspiketrains.Thestatisticswereestimatedover1000randomlygeneratedpairsofspiketrainssimulatedwithMIPmodelandaverageringrate20spk/s.Thekernelsizewas2ms.Thedierentcurvesshowresultunderdierentlevelsofjitterstandarddeviation,withthevalueinthelegend. B.4.3DiscriminationofSynchronousFiringsInthisscenarioweconsiderthatspiketrainsaretobedierentiatedbasedonthesynchronyofneuronrings.Moreprecisely,spiketrainsaredeemeddistant(ordissimilar)withregardstotherelativenumberofsynchronousspikes.Thatis,dissimilaritymeasuresareexpectedtobeinverselyproportionaltotheprobabilityofaspikeco-occurwithaspikeinanotherspiketrain.Thismeansthat,unliketheprevioustwocasestudieswheredierencesinringratewereanalyzed,thiscaseputstheemphasisofanalysisintheroleofeachspike.Thus,sincethetimescaleofanalysisismuchmorene,theprecisionofaspiketimehasincreasedrelevance. 180

PAGE 181

FigureB-10. DiscriminantindexofthedissimilaritymeasuresforeachkernelintermsofthesynchronybetweenthespiketrainsasgivenbyFigure B-9 .Thedierentcurvesarefordierentstandarddeviations(showninthelegend)ofthejitteraddedtothesynchronousspikes. Togeneratespiketrainswithagivensynchronythemultipleinteractionprocess(MIP)modelwasused[ Kuhnetal. 2003 2002 ].IntheMIPmodelareferencespiketrainisrstgeneratedasarealizationofaPoissonprocess.Thespiketrainsarethenderivedfromthisonebycopyingspikeswithprobability".Theoperationisperformedindependentlyforeachspikeandforeachspiketrain.Putdierently,"istheprobabilityofaspikeco-occurringinanotherspiketrain,andthereforecontrolswhatwerefertoassynchrony.Itcanalsobeshownthat"isthecountcorrelationcoecient[ Kuhnetal. 2003 ].TheresultingspiketrainsarePoissonprocesses.Bygeneratingthereferencespiketrainwithringrate"itisensuredthatthederivedspikestrainshaveringrate.Tomakethesimulationmorerealistic,jitternoisewasaddedtoeachspiketimetorecreate 181

PAGE 182

thevariabilityinspiketimesoftenencounteredinpractice,thusmakingthetaskofndingspikesthataresynchronousmorechallenging.Jitternoisewasgeneratedasindependentandidenticallydistributedzero-meanGaussiannoise.Foreachcombinationofsynchronyandjitterstandarddeviation,1000spiketrainspairsweregenerated,andthedissimilaritymeasuresintermsofthefourdierentkernelswereevaluated.Allspiketrainswereonesecondlongandtheringrate20spk/s,forsimilarreasonsasinthepreviousparadigms.Thekernelsizefortheresultsshownwas2ms.Thekernelsizewaschosensmallsinceinthisscenariothecharacterizingfeatureissynchronousrings.TheresultsareshowninFigure B-9 ,andintermsofthediscriminationindexinFigure B-10 .FromFigure B-10 ,theCSandCCdissimilaritieshavenotablybetterdiscriminationabilitythanVPandvanRossum'sdistance.TheresultsalsorevealthattheCSdissimilarityismoreconsistentthantheCCdissimilaritysinceitsdiscriminationdecreasesinamoregradedmannerwiththepresenceofvariabilityinthesynchronousspiketimes(evenforthesamekernelfunction).TheVPandvanRossum'sdistanceshavecomparablediscriminationability.Comparingthemeasurementsintermsofthekernelfunctions,itwasfoundthattheLaplaciankernelprovidesthebestresults,followedbythetriangularkernel.Nevertheless,theadvantagebetweendierentkernelsissmall. B.5FinalRemarksWecomparedbinlessspiketrainmeasurespresentedintheliteraturefortheirdiscriminationability.Giventhewideuseofthesemeasuresinspiketrainsanalysis,classicationandclustering,webelievethisstudyisfundamentalforunderstandingthebehaviorofeachmeasureanddecidingwhichmightbemoreappropriatetakingtheintendedaimintoconsideration.Nevertheless,theaimwasnotjusttodirectlycomparethepublishedmeasures.Here,weextendedthesemeasuresandprovidedabroaderperspectivewhich,inouropinion,waslackinginthepreviouspresentations.Inthereviewofthemeasures,itwasshownthatthe 182

PAGE 183

measurescanbereformulatedintermsofelementarykernelsondierencesofsinglepairsofspiketimes.Hence,thiskernelcanbereplacedbyanyotherfunctionabletosimilarlycapturethe\closeness"ofthespiketimes.Thispointofviewisimportantinshowingthegeneralityandindependenceofthemeasureswithregardstothekernel.Moreover,itallowsforthecomparisonstobedonewithoutkernelspeciceects.Another,moreimportantperspectivepresentedwasthatanyofthemeasuresconsideredisamulti-scalequantierofdissimilaritybetweenspiketrains,withscalecontrolledbythekernelsize.Thisisbecause,asexplicitlyveriedforthevanRossum'sdistanceandCSdissimilarity,themeasuresimplicitlydointensityfunctionestimation.Thisobservationiskeyfortheunderstandingofhowandwhythemeasurescanbeutilizedtoquantifydissimilarityininstantaneousringrates,despitetheirformulationaimedatspiketiming-basedparadigms.Themeasureswerecomparedinthreeexperimentswiththeinformationfordiscriminationcontainedinaverageringrates,instantaneousringratesandsynchrony.Thesewereselectedtoillustratetheconceptsdiscussedandbecausetheywerethoughttorepresentthehypothesistobetestedwithdataanalysis.Ofcourse,thesimulatedparadigmsaresimpliedapproximationsofthemorecomplexscenariosthatmaybeobservedinpractice.Unfortunately,theresultsrevealsthatnosinglemeasureperformsthebestorconsistentlythroughoutallthreeparadigms.Forinstance,iftheVPandvanRossum'sdistanceshaveconsistentdiscriminationintheconstantringrateparadigmtheyareclearlyoutperformedinthesynchrony-baseddiscriminationtaskbytheCSandCCdissimilarities,buttheresultsoftheselatteronesarenotatallusableintherstparadigm,mostlybecausetheirunstabilityforsmallnumberofspikes.Nevertheless,allmeasuresareconsistentandcomparablyperforminthesecondparadigm,intermsofmodulationoftheinstantaneousringrates.Anintriguingbutnotentirelysurprisingresultisthat,althoughtheVPdistanceandvanRossum'sdistanceyieldsquitedierent 183

PAGE 184

resultsasnoticedclearlyinFigure B-5 andFigure B-7 ,theirdiscriminationisthesameinallparadigms(Figure B-6 ,Figure B-8 andFigure B-10 ).Theresultsalsosuggestthatthedependenceofthemeasuresonaspecickernelisminor.Aconsiderablymorerelevantissueisthekernelsize,asemphasizedintheringrateparadigms.Thisisbecause,asmentioned,themeasuresquantifyrelationsintermsof(implicit)intensityfunctions.Hence,ifthekernelsizeisnotproperlyselectedtheestimateoftheintensityfunctionsdoesnotaccountforthedesiredfeatureinthespiketrains.Finally,theresultsdepicttheimportanceofbinlessspiketrainmeasures.Asstatedearlier,theonlydierencebetweentheCSdissimilarityevaluatedwiththerectangularkernelandtheCCdissimilarityisthetimequantizationincurredwithbinning.ComparingtheresultsinthesetwosituationsinFigure B-8 andFigure B-10 showsthatsmallimprovementsindiscriminationandrobustnesstojitternoisewereachievedintherstandsecondcases,respectively,byutilizingthespiketimesdirectly. 184

PAGE 185

REFERENCES O.O.Abbe.Uberblutkorper-zahlung.JenaZ.Med.Naturwiss.,13:98{105,1879. E.D.Adrian.TheBasisofSensation:theactionofthesenseorgans.W.W.Norton&Co.,NewYork,1928. A.M.Aertsen,G.L.Gerstein,M.K.Habib,andG.Palm.Dynamicsofneuronalringcorrelation:modulationof\eectiveconnectivity".JournalofNeurophysiology,61(5):900{917,1989. N.Aronszajn.Theoryofreproducingkernels.TransactionsoftheAmericanMathematicalSociety,68(3):337{404,May1950. L.A.BaccalaandK.Sameshima.Directedcoherence:atoolforexploringfunctionalinteractionsamongbrainstructures.InM.A.L.Nicolelis,editor,MethodsforNeuralEnsembleRecordings,pages179{192.CRCPress,1999. M.S.Bartlett.Thespectralanalysisofpointprocesses.J.R.Stat.Soc.B,25(2):264{296,1963. A.J.BellandT.J.Sejnowski.Aninformation-maximizationapproachtoblindseparationandblinddeconvolution.NeuralComputation,7(6):1129{1159,Nov.1995. C.Berg,J.P.R.Christensen,andP.Ressel.HarmonicAnalysisonSemigroups:TheoryofPositiveDeniteandRelatedFunctions.Springer-Verlag,NewYork,NY,1984. D.R.Brillinger.Estimationofthesecond-orderintensitiesofabivariatestationarypointprocess.JournaloftheRoyalStatisticalSociety{SeriesB,38:60{66,1976. D.R.Brillinger.Nervecellspiketraindataanalysis:aprogressionoftechnique.JournaloftheAmericanStatisticalAssociation,87(418):260{271,June1992. E.N.Brown,R.Barbieri,V.Ventura,R.E.Kass,andL.M.Frank.Thetime-rescalingtheoremanditsapplicationtoneuralspiketraindataanalysis.NeuralComputation,14(2):325{346,2001a.doi:10.1162/08997660252741149. E.N.Brown,D.P.Nguyen,L.M.Frank,M.A.Wilson,andV.Solo.Ananlysisofneuralreceptiveeldplasticitybypointprocessadaptiveltering.ProceedingsoftheNationalAcademyofScience,USA,98:12261{12266,2001b. E.N.Brown,R.E.Kass,andP.P.Mitra.Multipleneuralspiketraindataanalysis:state-of-the-artandfuturechallenges.NatureNeuroscience,7:456{461,2004.doi:10.1038/nn1228. J.M.Carmena,M.A.Lebedev,R.E.Crist,J.E.O'Doherty,D.M.Santucci,D.F.Dimitrov,P.G.Patil,C.S.Henriquez,andM.A.L.Nicolelis.Learningtocontrolabrain-machineinterfaceforreachingandgraspingbyprimates.PLoSBiology,1(2),Nov.2003.doi:10.1371/journal.pbio.0000042. 185

PAGE 186

C.E.CarrandM.Konishi.Acircuitfordetectionofinterauraltimedierencesinthebrainstemofthebarnowl.JournalofNeuroscience,10(10):3227{3246,1990. J.K.Chapin,K.A.Moxon,R.S.Markowitz,andM.A.L.Nicolelis.Real-timecontrolofarobotarmusingsimultaneouslyrecordedneuronsinthemotorcortex.NatureNeuroscience,2(7):664{670,July1999. Z.ChiandD.Margoliash.Temporalprecisionandtemporaldriftinbrainandbehaviorofzebranchsong.Neuron,32(1{20):899{910,Dec.2001. Z.Chi,W.Wu,Z.Haga,N.G.Hatsopoulos,andD.Margoliash.Template-basedspikepatternidenticationwithlinearconvolutionanddynamictimewarping.JournalofNeurophysiology,97(2):1221{1235,Feb.2007.doi:10.1152/jn.00448.2006. D.R.Cox.Somestatisticalmethodsconnectedwithseriesofevents.J.R.Stat.Soc.B,17:129{164,1955. D.R.CoxandV.Isham.PointProcesses.ChapmanandHall,1980. D.J.DaleyandD.Vere-Jones.AnIntroductiontotheTheoryofPointProcesses.Springer-Verlag,NewYork,NY,1988. S.Darmanjian,A.R.C.Paiva,andJ.C.Prncipe.HierarchaldecompositionofneuraldatausingboostedmixturesofHiddenMarkovChainsanditsapplicationtoaBMI.InProceedingsoftheIEEEInternationalJointConferenceonNeuralNetworks,IJCNN-2007,Orlando,FL,USA,Aug.2007. P.DayanandL.F.Abbott.TheoreticalNeuroscience:ComputationalandMathematicalModelingofNeuralSystems.MITPress,Cambridge,MA,USA,2001. P.DiggleandJ.S.Marron.Equivalenceofsmoothingparameterselectorsindensityandintensityestimation.JournaloftheAmericanStatisticalAssociation,83(403):793{800,Sept.1988. J.P.DonoghueandS.P.Wise.Themotorcortexoftherat:cytoarchitectureandmicrostimulationmapping.JournalofComparativeNeurology,212(12):76{88,1982. R.O.Duda,P.E.Hart,andD.G.Stork.Patternclassication.WileyInterscience,2ndedition,2000. U.R.Eden,L.M.Frank,R.Barbieri,V.Solo,andE.N.Brown.Dynamicanalysisofneuralencodingbypointprocessadaptiveltering.Neurocomputing,16(5):971{998,May2004. J.J.Eggermont.Propertiesofcorrelatedneuralactivityclustersincatauditorycortexresemblethoseofneuralassemblies.JournalofNeurophysiology,96(2):746{764,July2006.doi:10.1152/jn.00059.2006. 186

PAGE 187

A.K.Erlang.Thetheoryofprobabilitiesandtelephoneconversations.NytTidsskriftforMatematikB,20,1909.Reprintedat http://oldwww.com.dtu.dk/teletraffic/Elang.html A.K.Erlang.Solutionofsomeproblemsinthetheoryofprobabilitiesofsignicanceinautomatictelephoneexchanges.Elektroteknikeren,13,1917.Reprintedat http://oldwww.com.dtu.dk/teletraffic/Elang.html J.-M.Fellous,P.H.E.Tiesinga,P.J.Thomas,andT.J.Sejnowski.Discoveringspikepatternsinneuronalresponses.JournalofNeuroscience,24(12):2989{3001,Mar.2004.doi:10.1523/JNEUROSCI.4649-03.2004. W.A.Freiwald,A.K.Kreiter,andW.Singer.Synchronizationandassemblyformationinthevisualcortex.InM.A.L.Nicolelis,editor,ProgressinBrainResearch,pages111{140.ElsevierScience,2001. A.P.Georgopoulos,J.F.Kalaska,R.Caminiti,andJ.T.Massey.Ontherelationsbetweenthedirectionoftwo-dimensionalarmmovementsandcelldischargeinprimatemotorcortex.JournalofNeuroscience,2:1527{1537,Nov.1982. A.P.Georgopoulos,R.E.Kettner,andA.B.Schwartz.Primatemotorcortexandfreearmmovementstovisualtargetsinthree-dimensionalspace.II.codingofthedirectionofmovementbyaneuronalpopulation.JournalofNeuroscience,8(8):2928{2937,Aug.1988. G.L.GersteinandA.M.Aertsen.Representationofcooperativeringactivityamongsimultaneouslyrecordedneurons.JournalofNeurophysiology,54(6):1513{1528,1985. G.L.GersteinandD.H.Perkel.Simultaneouslyrecordedtrainsofactionpotentials:Analysisandfunctionalinterpretation.Science,164(3881):828{830,May1969.doi:10.1126/science.164.3881.828. G.L.GersteinandD.H.Perkel.Mutualtemporalrelationshipsamongneuronalspiketrains.statisticaltechniquesfordisplayandanalysis.BiophysicalJournal,12(5):453{473,May1972. G.L.Gerstein,D.H.Perkel,andJ.E.Dayho.Cooperativeringactivityinsimultaneouslyrecordedpopulationsofneurons:detectionandmeasurement.Jour-nalofNeuroscience,5(4):881{889,1985. W.GerstnerandW.Kistler.SpikingNeuronModels.MITPress,2002. J.Graunt.ObservationoftheLondonBillsofMortality. http://www.ac.wwu.edu/~stephan/Graunt/graunt.html ,1662. M.GreenwoodandG.U.Yule.Aninquiryintothenatureoffrequencydistributionsrepresentativeofmultiplehappeningswithparticularreferencetotheoccurrenceofmultipleattacksofdiseaseorofrepeatedaccidents.J.R.Statist.Soc.A,83:255{279,1920. 187

PAGE 188

S.Grun,M.Diesmann,andA.Aertsen.UnitaryEventsinmultiplesingle-neuronactivity.I.detectionandsignicance.NeuralComputation,14(1):43{80,2002a. S.Grun,M.Diesmann,andA.Aertsen.UnitaryEventsinmultiplesingle-neuronactivity.II.nonstationarydata.NeuralComputation,14(1):43{80,2002b. R.H.R.Hahnloser.Cross-intensityfunctionsandtheestimateofspike-timejitter.BiologicalCybernetics,96(5):497{506,May2007.doi:10.1007/s00422-007-0143-7. D.A.Harville.Matrixalgebrafromastatistician'sperspective.Springer,1997. N.G.Hatsopoulos,C.L.Ojakangas,L.Paninski,andJ.P.Donoghue.Informationaboutmovementdirectionobtainedfromsynchronousactivityofmotorcorticalneurons.ProceedingsoftheNationalAcademyofScience,USA,95(26):15706{15711,Dec.1998. S.Haykin.AdaptiveFilterProcessing.Prentice-Hall,4ndedition,2002. J.M.Hurtado,L.L.Rubchinsky,andK.A.Sigvardt.Statisticalmethodfordetectionofphase-lockingepisodesinneuraloscillations.JournalofNeurophysiology,91(4):1883{1898,Apr.2004. E.M.Izhikevich.Polychronization:Computationwithspikes.NeuralComputation,18(2):245{282,Feb.2006. R.E.KassandV.Ventura.Aspike-trainprobabilitymodel.NeuralComputation,13(8):1713{1720,Aug.2001. R.E.Kass,V.Ventura,andC.Cai.Statisticalsmoothingofneuronaldata.Network:ComputationinNeuralSystems,14:5{15,2003. A.Y.Khinchin.Mathematicalmethodsinthetheoryofqueueing.Grin,London,1960.EnglishtranslationoftheRussianoriginal,originallypublishedin1955. S.-P.Kim.DesignandanalysisofoptimaldecodingmodelsforBrain-MachineInterfaces.PhDthesis,UniversityofFlorida,May2005. S.P.Kim,J.C.Sanchez,Y.N.Rao,D.Erdogmus,J.M.Carmena,M.A.Lebedev,M.A.L.Nicolelis,andJ.C.Principe.AcomparisonofoptimalMIMOlinearandnonlinearmodelsforbrain-machineinterfaces.JournalofNeuralEngineering,3(2):145{161,2006.doi:10.1088/1741-2560/3/2/009. T.Kreuz,J.S.Haas,A.Morelli,H.D.I.Abarbanel,andA.Politi.Measuringspiketrainsynchrony.JournalofNeurosciencemethods,165(1):151{161,Sept.2007.doi:10.1016/j.jneumeth.2007.05.031. A.Kuhn,S.Rotter,andA.Aertsen.Correlatedinputspiketrainsandtheireectsontheresponseoftheleakyintegrate-and-reneuron.Neurocomputing,44{46:121{126,June2002.doi:10.1016/S0925-2312(02)00372-7. 188

PAGE 189

A.Kuhn,A.Aertsen,andS.Rotter.Higher-orderstatisticsofinputensemblesandtheresponseofsimplemodelneurons.NeuralComputation,15(1):67{101,2003. B.G.LindseyandG.L.Gerstein.Twoenhancementsofthegravityalgorithmformultiplespiketrainanalysis.JournalofNeurosciencemethods,150(1):116{127,2006.doi:10.1016/j.jneumeth.2005.06.019. W.MaassandC.M.Bishop,editors.PulsedNeuralNetworks.MITPress,1998. W.Maass,T.Natschlager,andH.Markram.Real-timecomputingwithoutstablestates:Anewframeworkforneuralcomputationbasedonperturbations.NeuralComputation,14(11):2531{2560,2002.doi:10.1162/089976602760407955. Z.F.MainenandT.J.Sejnowski.Reliabilityofspiketiminginneocorticalneurons.Science,268(5216):1503{1506,1995.doi:10.1126/science.7770778. K.V.MardiaandP.E.Jupp.DirectionalStatistics.JohnWiley&Sons,WestSussex,England,2000.ISBN0-471-95333-4. V.Z.Marmarelis.NonlinearDynamicModellingofPhysiologicalSystems.IEEEPressSeriesinBiomedicalEngineering.JohnWiley&Sons,2004. V.Z.Marmarelis.IdenticationofnonlinearbiologicalsystemsusingLaguerreexpansionsofkernels.AnnalsofBiomedicalEngineering,21:573{589,1993. J.W.McClurkin,T.J.Gawne,L.M.Optican,andB.J.Richmond.Lateralgeniculateneuronsinbehavingprimates.II.Encodingofvisualinformationinthetemporalshapeoftheresponse.JournalofNeurophysiology,66(3):794{808,Sept.1991. J.Mercer.Functionsofpositiveandnegativetype,andtheirconnectionwiththetheoryofintegralequations.PhilosophicalTransactionsoftheRoyalSocietyofLondon{SeriesA,209:415{446,1909. R.E.MirolloandS.H.Strogatz.Synchronizationofpulse-coupledbiologicaloscillators.SIAMJournalonAppliedMathematics,50(6):1645{1662,Dec.1990. E.H.Moore.OnproperlypositiveHermitianmatrices.BulletinoftheAmericanMathematicalSociety,23:59,1916. J.E.Moyal.Thegeneraltheoryofstochasticpopulationprocesses.ActaMathematica,108(1):1{31,dec1962. A.Y.Ng,M.I.Jordan,andY.Weiss.Onspectralclustering:Analysisandanalgorithm.InAdvancesinNeuralInformationProcessingSystems,volume14,2001. M.A.L.Nicolelis.Brain-machineinterfacestorestoremotorfunctionandprobeneuralcircuits.NatureReviewsNeuroscience,4:417{422,2003.doi:10.1038/nrn1105. A.R.C.Paiva,I.Park,andJ.C.Prncipe.AreproducingkernelHilbertspaceframeworkforspiketrains.Resubmitted. 189

PAGE 190

A.R.C.Paiva,J.-W.Xu,andJ.C.Prncipe.Kernelprincipalcomponentsaremaximumentropyprojections.InProceedingsoftheInternationalConferenceonIndepen-dentComponentAnalysisandBlindSourceSeparation,ICA-2006,pages846{853,Charleston,SC,Mar.2006.doi:10.1007/11679363 105. A.R.C.Paiva,S.Rao,I.Park,andJ.C.Prncipe.Spectralclusteringofsynchronousspiketrains.InProceedingsoftheIEEEInternationalJointConferenceonNeuralNetworks,IJCNN-2007,Orlando,FL,USA,Aug.2007. A.R.C.Paiva,I.Park,andJ.C.Prncipe.Reproducingkernelhilbertspacesforspiketrainanalysis.InProceedingsoftheIEEEInternationalConferenceonAcoustics,Speech,andSignalProcessing,ICASSP-2008,LasVegas,NV,USA,Apr.2008. C.Palm.Intensityvariationsintelephonetrac.NorthHolland,Amsterdam,1988.Englishtranslationoftheauthor'sthesis,originallypresentedin1943. A.Papoulis.Probability,randomvariables,andstochasticprocesses.McGraw-Hill,NewYork,1965. I.Park,A.R.C.Paiva,T.B.DeMarse,andJ.C.Prncipe.Anecientalgorithmforcontinuous-timecrosscorrelationofspiketrains.JournalofNeurosciencemethods,128(2):514{523,Mar.2008.doi:10.1016/j.jneumeth.2007.10.005. E.Parzen.TimeSeriesAnalysisPapers.Holden-Day,SanFrancisco,CA,1967. E.Parzen.Ontheestimationofaprobabilitydensityfunctionandthemode.AnnalsofMathematicalStatistics,33(2):1065{1076,Sept.1962. E.Parzen.StatisticalinferenceontimeseriesbyHilbertspacemethods.TechnicalReport23,AppliedMathematicsandStatisticsLaboratory,StanfordUniversity,Stanford,California,Jan.1959. E.PekalskaandR.P.W.Duin.TheDissimilarityRepresentationforPatternRecognition:FoundationsAndApplications.WorldScientic,2005.ISBN9-812-56530-3. D.H.Perkel,G.L.Gerstein,andG.P.Moore.Neuronalspiketrainsandstochasticpointprocesses.I.thesinglespiketrain.BiophysicalJournal,7(4):391{418,July1967a. D.H.Perkel,G.L.Gerstein,andG.P.Moore.Neuronalspiketrainsandstochasticpointprocesses.II.simultaneousspiketrains.BiophysicalJournal,7(4):419{440,July1967b. J.C.Prncipe,D.Xu,andJ.W.Fisher.Informationtheoreticlearning.InS.Haykin,editor,UnsupervisedAdaptiveFiltering,volume2,pages265{319.JohnWiley&Sons,2000. A.Ramakrishnan.Stochasticprocessesrelatingtoparticlesdistributedinacontinousinnityofstates.ProceedingsoftheCambridgePhilosophicalSociety,46(4):595{602,1950. 190

PAGE 191

J.O.RamsayandB.W.Silverman.FunctionalDataAnalysis.Springer-Verlag,1997.ISBN0-387-94956-9. R.-D.Reiss.ACourseonPointProcesses.Springer-Verlag,NewYork,NY,1993. B.J.RichmondandL.M.Optican.Temporalencodingoftwo-dimensionalpatternsbysingleunitsinprimateinferiortemporalcortex.II.Quanticationofresponsewaveform.JournalofNeurophysiology,51(1):147{161,Jan.1987. B.J.Richmond,L.M.Optican,andH.Spitzer.Temporalencodingoftwo-dimensionalpatternsbysingleunitsinprimateprimaryvisualcortex.I.Stimulus-responserelations.JournalofNeurophysiology,64(2):351{369,Aug.1990. A.Riehle,S.Grun,M.Diesmann,andA.Aertsen.Spikesynchronizationandratemodulationdierentiallyinvolvedinmotorcorticalfunction.Science,278(5345):1950{1953,Dec.1997.doi:10.1126/science.278.5345.1950. F.Rieke,D.Warland,R.deRuytervanSteveninck,andW.Bialek.Spikes:exploringtheneuralcode.MITPress,Cambridge,MA,USA,1999.ISBN0-262-18174-6. R.W.Rodieck,N.Y.-S.Kiang,andG.L.Gerstein.Somequantitativemethodsforthestudyofspontaneousactivityofsingleneurons.BiophysicalJournal,2(4):351{368,July1962. K.SameshimaandL.A.Baccala.Usingpartialdirectedcoherencetodescribeneuronalensembleinteractions.JournalofNeurosciencemethods,94(1):93{103,Dec.1999. J.C.Sanchez.Fromcorticalneuralspiketrainstobehavior:modelingandanalysis.PhDthesis,UniversityofFlorida,May2004. J.C.Sanchez,D.Erdogmus,Y.Rao,S.-P.Kim,M.Nicolelis,J.Wessberg,andJ.C.Principe.Interpretingneuralactivitythroughlinearandnonlinearmodelsforbrainmachineinterfaces.InProceedingsoftheInternationalConferenceIEEEEngineeringinMedicineandBiologySociety,pages2160{2163,Cancun,Mexico,Sept.2003. J.C.Sanchez,J.C.Prncipe,andP.R.Carney.Isneurondiscriminationpreprocessingnecessaryforlinearandnonlinearbrainmachineinterfacemodels?InProceedingsoftheInternationalConferenceonHuman-ComputerInteraction,2005. B.Scholkopf,A.Smola,andK.-R.Muller.Nonlinearcomponentanalysisasakerneleigenvalueproblem.NeuralComputation,10(5):1299{1319,1998. B.Scholkopf,C.J.C.Burges,andA.J.Smola,editors.AdvancesinKernelMethods:SupportVectorLearning.MITPress,1999. B.SchrauwenandJ.V.Campenhout.Linkingnon-binnedspiketrainkernelstoseveralexistingspiketraindistances.Neurocomputing,70(7{8):1247{1253,Mar.2007.doi:10.1016/j.neucom.2006.11.017. 191

PAGE 192

S.Schreiber,J.M.Fellous,D.Whitmer,P.Tiesinga,andT.J.Sejnowski.Anewcorrelation-basedmeasureofspiketimingreliability.Neurocomputing,52{54:925{931,June2003.doi:10.1016/S0925-2312(02)00838-X. H.Seidel.Uberdieprobabilitatensolcherereignissewelchenurseitenvorkommen,obgleichsieunbeschranktoftmoglichsind.Sitzungsber.Math.Phys.Cl.Akad.Wiss.Munchen,6:44{50,1876. K.V.Shenoy,D.Meeker,S.Cao,S.A.Kureshi,B.Pesaran,C.A.Buneo,A.P.Batista,P.P.Mitra,J.W.Burdick,andR.A.Andersen.Neuralprostheticcontrolsignalsfromplanactivity.NeuroReport,14(4):591{596,Mar.2003. D.L.Snyder.RandomPointProcessinTimeandSpace.JohnViley&Sons,NewYork,1975. D.Song,R.H.M.Chan,V.Z.Marmarelis,R.E.Hampson,S.A.Deadwyler,andT.W.Berger.Nonlineardynamicmodelingofspiketraintransformationsforhippocampal-corticalprostheses.IEEETransactionsonBiomedicalEngineering,54(6):1053{1066,June2007.doi:10.1109/TBME.2007.891948. S.K.SrinivasanandA.Vijayakumar.PointProcessesandProductDensities.NarosaPublishingHouse,2003.ISBN81-7319-558-7. J.V.ToupsandP.H.E.Tiesinga.Methodsforndingandvalidatingneuralspikepatterns.Neurocomputing,69(10{12),June2006. W.Truccolo,U.T.Eden,M.R.Fellows,J.P.Donoghue,andE.N.Brown.Apointprocessframeworkforrelatingneuralspikingactivitytospikinghistory,neuralensemble,andextrinsiccovariateeects.JournalofNeurophysiology,93:1074{1089,Feb.2005.doi:10.1152/jn.00697.2004. E.Vaadia,I.Haalman,M.Abeles,H.Bergman,Y.Prut,H.Slovin,andA.Aertsen.Dynamicsofneuronalinteractionsinmonkeycortexinrelationtobehaviouralevents.Nature,373(6514):515{518,Feb.1995.doi:10.1038/373515a0. M.C.W.vanRossum.Anovelspikedistance.NeuralComputation,13(4):751{764,2001. V.Ventura,R.Carla,R.E.Kass,S.N.Gettner,andC.R.Olson.Statisticalanalysisoftemporalevolutioninsingle-neuronringrates.Biostatistics,3(1):1{20,2002. J.D.Victor.Spiketrainmetrics.CurrentOpinioninNeurobiology,15(5):585{592,Sept.2005.doi:10.1016/j.conb.2005.08.002. J.D.VictorandK.P.Purpura.Natureandprecisionoftemporalcodinginvisualcortex:Ametric-spaceanalysis.JournalofNeurophysiology,76(2):1310{1326,Aug.1996. J.D.VictorandK.P.Purpura.Metric-spaceanalysisofspiketrains:theory,algorithms,andapplication.Network:ComputationinNeuralSystems,8:127{164,Oct.1997. 192

PAGE 193

H.Wagner,S.Brill,R.Kempter,andC.E.Carr.Microsecondprecisionofphasedelayintheauditorysystemofthebarnowl.JournalofNeurophysiology,94(2):1655{1658,2005.doi:10.1152/jn.01226.2004. G.Wahba.SplineModelsforObservationalData,volume59ofCBMS-NSFRegionalConferenceSeriesinAppliedMathematics.SIAM,1990. J.Wessberg,C.R.Stambaugh,J.D.Kralik,P.D.Beck,M.Laubach,J.K.Chapin,J.Kim,S.J.Biggs,M.A.Srinivasan,andM.A.L.Nicolelis.Real-timepredictionofhandtrajectorybyensemblesofcorticalneuronsinprimates.Nature,408(6810):361{365,Nov.2000.doi:10.1038/35042582. S.WohlgemuthandB.Ronacher.Auditorydiscriminationofamplitudemodulationsbasedonmetricdistancesofspiketrains.JournalofNeurophysiology,97:3082{3092,Feb.2007.doi:10.1152/jn.01235.2006. H.Wold.OnstationarypointprocessesandMarkovchains.Skand.Aktuar.,31:229{240,1948. W.Wu,M.J.Black,D.Mumford,Y.Gao,E.Bienenstock,andJ.P.Donoghue.Modelinganddecodingmotorcorticalactivityusingaswitchingkalmanlter.IEEETransactionsonBiomedicalEngineering,51(6):933{942,July2004. J.Yvon.Latheoriestatistiquedesuidesetl'equationd'etat.Herman,Paris,1935. 193

PAGE 194

BIOGRAPHICALSKETCH AntonioR.C.PaivawasborninOvar,Portugal,in1980.In2003,hereceivedhisB.S.degreeinelectronicsandtelecommunicationsengineeringfromtheUniversityofAveiro,Portugal.Duringhisundergraduatestudies,hereceivedfourmeritscholarships,aDr.ValeGuimar~aesawardforbestdistrictstudentattheUniversityofAveiro,andanEng.FerreiraPintoBastoawardfromAlcatelPortugalfortopgraduatingstudentinthemajor.Aftercompletinghisundergraduatestudies,hedidresearchinimagecompressionasaresearchassistantofDr.ArmandoPinhoforalmostayear.Inthefallof2004,hejoinedtheComputationalNeuroEngineeringLaboratoryundersupervisionofDr.JoseC.Prncipeforhisgraduatestudies,havingobtainedtheM.S.andPh.D.degreesinelectricalandcomputerengineeringinthefallof2005andthesummerof2008,respectively.HisdoctoralresearchfocusedonthedevelopmentofareproducingkernelHilbertspacesframeworkforanalysisandprocessingofpointprocesses,withapplicationsonsingle-unitneuralspiketrains.Hisresearchinterestsare,broadly,signalandimageprocessing,withspecialinterestinbiomedicalandbiologicalapplications.Inparticular,theseinclude:kernelmethodsandinformationtheoreticlearning,imageprocessing,brain-inspiredcomputation,principlesofinformationrepresentationandprocessinginthebrain,sensoryandmotorsystems,anddevelopmentofbiologicalandbiomedicaldataanalysismethods. 194