Citation
Development of a personalized diagnostic model for kidney stone disease tailored to acute care by integrating large clinical, demographics and laboratory data: the diagnostic acute care algorithm - kidney stones (DACA-KS)

Material Information

Title:
Development of a personalized diagnostic model for kidney stone disease tailored to acute care by integrating large clinical, demographics and laboratory data: the diagnostic acute care algorithm - kidney stones (DACA-KS)
Creator:
Zhaoyi Chen
Victoria Y. Bird
Rupam Ruchi
Mark S. Segal
Jiang Bian
Saeed R. Khan
Marie-Carmelle Elie
Mattia Prosperi
Publisher:
BMC, BMC Medical Informatics and Decision Making
Publication Date:
Language:
English

Subjects

Subjects / Keywords:
Diagnostic algorithm ( fast )
Kidney stones ( fast )
Big data analysis ( fast )

Notes

Abstract:
Background: Kidney stone (KS) disease has high, increasing prevalence in the United States and poses a massive economic burden. Diagnostics algorithms of KS only use a few variables with a limited sensitivity and specificity. In this study, we tested a big data approach to infer and validate a ‘multi-domain’ personalized diagnostic acute care algorithm for KS (DACA-KS), merging demographic, vital signs, clinical, and laboratory information. Methods: We utilized a large, single-center database of patients admitted to acute care units in a large tertiary care hospital. Patients diagnosed with KS were compared to groups of patients with acute abdominal/flank/groin pain, genitourinary diseases, and other conditions. We analyzed multiple information domains (several thousands of variables) using a collection of statistical and machine learning models with feature selectors. We compared sensitivity, specificity and area under the receiver operating characteristic (AUROC) of our approach with the STONE score, using cross-validation. Results: Thirty eight thousand five hundred and ninety-seven distinct adult patients were admitted to critical care between 2001 and 2012, of which 217 were diagnosed with KS, and 7446 with acute pain (non-KS). The multi-domain approach using logistic regression yielded an AUROC of 0.86 and a sensitivity/specificity of 0.81/0.82 in cross-validation. Increase in performance was obtained by fitting a super-learner, at the price of lower interpretability. We discussed in detail comorbidity and lab marker variables independently associated with KS (e.g. blood chloride, candidiasis, sleep disorders). Conclusions: Although external validation is warranted, DACA-KS could be integrated into electronic health systems; the algorithm has the potential used as an effective tool to help nurses and healthcare personnel during triage or clinicians making a diagnosis, streamlining patients’ management in acute care. Keywords: Diagnostic algorithm, Kidney stones, Big data analysis.
General Note:
Chen et al. BMC Medical Informatics and Decision Making (2018) 18:72 https://doi.org/10.1186/s12911-018-0652-4; Pages 1-14

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Downloads

This item is only available as the following downloads:


Full Text

PAGE 1

RESEARCHARTICLEOpenAccess Developmentofapersonalizeddiagnosticmodelforkidneystonediseasetailoredtoacutecarebyintegratinglargeclinical,demographicsandlaboratorydata:thediagnosticacutecarealgorithm-kidneystones(DACA-KS)ZhaoyiChen1* ,VictoriaY.Bird2,RupamRuchi3,MarkS.Segal3,JiangBian4,SaeedR.Khan5,Marie-CarmelleElie6andMattiaProsperi1AbstractBackground:Kidneystone(KS)diseasehashigh,increasingprevalenceintheUnitedStatesandposesamassiveeconomicburden.DiagnosticsalgorithmsofKSonlyuseafewvariableswithalimitedsensitivityandspecificity.Inthisstudy,wetestedabigdataapproachtoinferandvalidatea‘multi-domain’personalizeddiagnosticacutecarealgorithmforKS(DACA-KS),mergingdemographic,vitalsigns,clinical,andlaboratoryinformation.Methods:Weutilizedalarge,single-centerdatabaseofpatientsadmittedtoacutecareunitsinalargetertiarycarehospital.PatientsdiagnosedwithKSwerecomparedtogroupsofpatientswithacuteabdominal/flank/groinpain,genitourinarydiseases,andotherconditions.Weanalyzedmultipleinformationdomains(severalthousandsofvariables)usingacollectionofstatisticalandmachinelearningmodelswithfeatureselectors.Wecomparedsensitivity,specificityandareaunderthereceiveroperatingcharacteristic(AUROC)ofourapproachwiththeSTONEscore,usingcross-validation.Results:Thirtyeightthousandfivehundredandninety-sevendistinctadultpatientswereadmittedtocriticalcarebetween2001and2012,ofwhich217werediagnosedwithKS,and7446withacutepain(non-KS).Themulti-domainapproachusinglogisticregressionyieldedanAUROCof0.86andasensitivity/specificityof0.81/0.82incross-validation.Increaseinperformancewasobtainedbyfittingasuper-learner,atthepriceoflowerinterpretability.WediscussedindetailcomorbidityandlabmarkervariablesindependentlyassociatedwithKS(e.g.bloodchloride,candidiasis,sleepdisorders).Conclusions:Althoughexternalvalidationiswarranted,DACA-KScouldbeintegratedintoelectronichealthsystems;thealgorithmhasthepotentialusedasaneffectivetooltohelpnursesandhealthcarepersonnelduringtriageorcliniciansmakingadiagnosis,streamliningpatients’managementinacutecare.Keywords:Diagnosticalgorithm,Kidneystones,Bigdataanalysis *Correspondence:zhaoyi@ufl.edu1DepartmentofEpidemiology,CollegeofPublicHealthandHealthProfessions&CollegeofMedicine,UniversityofFlorida,2004MowryRoad,POBox100231,Gainesville,Florida32610-0231,USAFulllistofauthorinformationisavailableattheendofthearticle TheAuthor(s).2018OpenAccessThisarticleisdistributedunderthetermsoftheCreativeCommonsAttribution4.0InternationalLicense(http://creativecommons.org/licenses/by/4.0/),whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktotheCreativeCommonslicense,andindicateifchangesweremade.TheCreativeCommonsPublicDomainDedicationwaiver(http://creativecommons.org/publicdomain/zero/1.0/)appliestothedatamadeavailableinthisarticle,unlessotherwisestated.Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 https://doi.org/10.1186/s12911-018-0652-4

PAGE 2

BackgroundKidneystone(KS)diseaseprevalencehasincreasedintheUnitedStatesfrom5.2%(6.3%malesand4.1%females)in1994to8.8%(10.6%malesand7.1%females)in2012[1].Sinceitisoneofthecostliesturologicdis-easesintheUnitedStates,anincreaseinprevalenceposesahugeeconomicburdenonsociety.Thecostofdiagnosis,treatmentandpreventionofKSdiseasein2007wasestimatedtobe~$4billionand,duetopopu-lationgrowthalone,isprojectedtoincreasebymorethan$780millionby2030[2,3].ThepresenceofKSalsoplacestheindividualsatincreasedriskofdevelop-mentofchronickidneydisease.Inaprospectivecohortstudy,thosewhohadKSwasassociatedwitha50–67%higherriskofdevelopingchronickidneydiseaseascom-paredtothosewhodidnothave,KSgroupalsohadtwicetheriskofdevelopingend-stagerenaldisease[4].Theemergencydepartment(ED)isacommonplacewherepatientwithKSareevaluatedanddiagnosed.Duringthepasttwodecades,asignificantincreaseinEDvisitswithstone-relatedsymptomshasbeenobserved[5],withover1.3millionindividualsperyearpresentingtotheEDwithKSintheUnitedStates.Theclinicalpresen-tationtotheEDwithKScommonlyinvolvesacuteback,flankorgroinpain,nausea,vomitingandsometimesbloodinurine.Theworkupmayincludeinitiallabtestssuchascompletebloodcountwithdifferential,compre-hensivemetabolicpanel,andurineanalysis;butoftenthesetestsarenotpromptlymeasuredorareinappropri-atelyinterpreted[5].Across-sectionalanalysisofthe2007–2010NationalHealthandNutritionExaminationSurvey(NHANES)datasetsuggeststhatobesity,diabetes,andgoutallhaveasignificantpositiveassociationwithkidneystonehistory[1].ResultsfromtheNurses’HealthStudy,alargepopulation-basedlongitudinalstudy(years2001–2012)demonstratedthathighbody-massindex(BMI),choleli-thiasis,diabetesandspecificdietaryfactorsareassociatedwithahigherriskofKSformationinfemales[6].In2014,aclinicalpredictionscore-namedSTONE-wasderivedandvalidatedinretrospectiveandprospectivecohorts[7].TheSTONEscoreincludesfivevariables:malesex,shortdurationofpain,non-blackrace,presenceofnauseaorvomiting,andmicroscopichematuria.TheSTONEscorewasalsoexternallyvalidatedandshowedgoodvalidityinpatientswithflankpain[8].AnupdatedSTONE-PLUSscore,augmentedbypoint-of-carelimitedultrasonog-raphyassessinghydronephrosis,wasrecentlyreleasedandtestedprospectivelyonanEDpopulationsample,withonlyamoderateimprovementinriskstratification[9].AsKSdiseaseismultifactorialinnature,wehypothesizedthatanapproachincorporatinglaboratorydataandadd-itionalclinicalcharacteristicswoulddramaticallyimproveaKSdiagnosticmodel,leadingtoearlierdiagnosisandabetterunderstandingofitscomplexetiology.Inaddition,thisapproachcouldreducethenumberofunnecessaryradiographictestingi.e.CTscans,intheacutecaresetting.Inthisstudy,wetestedabigdataapproach,mergingdemographic,vitalsigns,clinical,andlaboratoryinfor-mation,toinferandvalidatea‘multi-domain’personal-izeddiagnosticscoreforKS.Weutilizedalarge,single-centerdatabaseofpatientsadmittedtoEDandotherintensive/acutecareunitsinalargetertiarycarehospital(over58,000admissionswithmajorityadmittedthroughED).Weanalyzedtheinformationdomainsindividually(e.g.onlycomorbidities,oronlylabtests),together,andcomparedourapproachwiththeSTONEscore.Anumberofstatisticalandmachinelearningmodelswerefitandcomparedtooptimizeperformance.Usingthismulti-domainintegrationapproachourgoalwastosignificantlyimprovethesensitivityandspecifi-cityofKSdiagnosisinacutesettings.MethodsStudypopulationThestudypopulationcomprisedindividualsadmittedtocriticalcareunitsattheBethIsraelDeaconessMedicalCenterinBoston,Massachusetts,UnitedStates,between2001and2012.DataarestoredelectronicallyintheMedicalInformationMartforIntensiveCare(MIMI-C-III)database,whichisavailabletothepublicuponrequest,uponCollaborativeInstitutionalTrainingInitia-tive(CITI)training,andlicenseagreementforfulldownloadandresearch[10].MIMIC-IIIincludesinfor-mationon:demographics;clinicaldiagnosesandproce-duresencodedwiththeInternationalClassificationofDiseasesver.9(ICD-9)ontology;vitalsignmeasure-mentsmadeatthebedside(~1datapointperhour);laboratorytestresults;medications;caregivernotes;im-agingreports;mortality(bothin-andout-of-hospital).Thisisasecondarydataanalysis.WeusedtheMIMIC-IIIver.1.4,releasedonSeptember2nd,2016.Ourstudyincludedpatientsaged18yearsandolder,di-videdintofourgroupsbasedontheICD-9diagnosesduringhospitalization:(a)KScases(ICD-9592,includingsub-codes592.0,592.1,592.9);(b)patientsdiagnosedwithgenitourinarydiseases(GUD)exceptKS(anyICD-9codeintheintervals580–591or593–599),e.g.patientswithnephritis,nephroticsyndrome,nephrosis;(c)patientsadmittedtoacutecarewithotherconditions(OTH)whodidnothaveanyKSorGUDdiagnosed(anyICD-9codenotincluding580–599)torepresentageneralpatientpopulation;(d)patientsadmittedwithacutelocalizedpain(ALP)ofabdominal(ICD-9code:789.0),back(ICD-9code:724.2),flank,orgroin(identi-fiedthroughpatients’electronicchartrecord).InadditiontoICD-9codes,wealsoexaminedrecordedChenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page2of14

PAGE 3

chartedeventsonALPfromthedataset.PatientswithbothKSandGUDcodeswereputintotheKSgroup.Eachpatientwasassociatedtoacovariatevectorofdemographicinfo,vitalsigns,clinicaldiagnoses,proce-dures,medicaments,andlaboratorytestsperformeddur-inghospitalization.StatisticalanalysisDescriptiveanalysiswasusedtoassessdemographicchar-acteristics(e.g.gender,age,insurancestatus,andreligion),vitalsigns(e.g.BMI,bloodpressure),laboratorytests(e.g.creatinine),anddistributionofICD-9diagnosesatadmis-sionandduringhospitalization.WealsocalculatedtheCharlsonComorbidityIndex(CCI)usingDeyo’salgorithm[11],andtheestimatedglomerularfiltrationrate(eGFR)usingtheCKD-EPI(ChronicKidneyDiseaseEpidemi-ologyCollaboration)equationequation[12].DuetoalowfrequencyofKS,weincludedonlyICD-9diagnosticcodesthatwereoccurredinlessthan5countsoftheKSgroup,andlabteststhatperformedinatleast50%oftheKSformers.Missingvalueswereim-putedviapopulationmedian/mode.UnivariateanalysiswasconductedtoassessdifferencesbetweenKSandGUD/OTH/ALPgroupsondemographics,ICD-9diag-noses,andlabtests,usingStudent’st-test,Wilcoxonranktest,orchi-squaretest,whereappropriate.Signifi-cancep-valueswereadjustedusingFalseDiscoverRate(FDR)correction[13].InordertoinferaKSdiagnosticscore,wefittedacol-lectionofmultivariablelogisticregressionmodelswiththeGUD,OTHorALPasnegativeexamples,usingdif-ferentinputcovariatedomains.Specifically,weevaluatedsevenmodels:(a)demographicvariablesandvitalsigns(includingbloodpressure,heartrateandbodytemperature)(b)CCI,plusdemographicvariables;(c)eGFRalone;(d)ICD-9diagnosis(top-25asselectedbytheunivariatefilter,i.e.thetop-25variablesthatweredifferentlydistributedbetweenKSandothergroups),plusdemographicvariables;(e)laboratorytests(top-25asselectedbytheunivariatefilter),plusdemographicvariables;(f)ICD-9diagnosisandlaboratorytests(top-50asselectedbytheunivariatefilter),plusallothervariablesincludedinmodels(a)to(e);(g)stepwise(for-ward-backward)selectionofmodel(f);(h)STONEmodel.NotethatICD-9codesusedtodefinetheGUDwerenotusedasinputcovariatestoanyofthemodels,exceptfortheSTONEmodelwherehematuria(ICD-9code599.7)isacovariate.Also,thedurationofpaintopresentationintheSTONEscorecouldnotbepreciselyascertainedfromourdata;weusedICD-9codesinthe338sfamilypluscodes780.96and789.0,excludingchronicpainentries,usingaweightof2(theSTONEscorea<6hpainisweighted3and6–24hpainisweighted1,butdurationofpainwasnotavailableinourdataset).InadditiontoICD-9codes,wealsousedchartedeventstoidentifypainevents.Fornausea/vomit-ingweusedICD-9787.0codes.Inasensitivityanalysis,wealsoevaluatedthecontributionofGUDcodestooverallperformanceofmodels(d)to(g).Modelcomparison,evaluation,andselectionwerecarriedoutusinga10-foldcross-validationframework[14],comparingperformanceindex(seebelow)distributionsfromtherepeatedsamplingfoldsusingBengioandNadeau’scorrectiontotheStudent’st-test[15].however,th.Inadditiontologisticregression,wealsofitanumberofmachinelearningtechniquesonthefullvariablesetasinmodel(f).Indetails:(i)adecisiontreebymeansoftheC4.5algorithms[16];(ii)LogitBoostalgorithminconjunctiontologisticregression[17];(iii)arandomfor-est(optimizingnumberoftreesupto1000)[18];(iv)asuperlearnerstackingalltheabovemethodsplusasingle-rulelinearmodel,internallyoptimizedvia5-foldcross-validation[19].Giventhehighclassimbalance,inadditiontothestandardmodelfit,wealsousedthesyntheticminorityover-samplingtechnique(SMOTE)internallytothecross-validation[20].Theunivariatefeatureselectionforthesemachinelearningalgorithmswasdoneinternallywithinthecross-validationsetting.Theperformanceanddiscriminativeabilityofmodelswasassessedusingsensitivity(truepositiverate),specifi-city(truenegativerate),andtheareaunderthereceiveroperatingcharacteristic(AUROC),whichistheexpect-ationthatauniformlydrawnrandompositivecaseisrankedbeforeauniformlydrawnrandomnegative(anareaof100%representsaperfecttest;anareaof50%representsaworthlesstest)[21].Theoptimalsensitivity/specificitycutoffwaschosenbasedonthemaximaloftheYouden’sJstatistic[22].AllstatisticalanalyseswereconductedusingSASsoftwarever.9.4(SASInstituteInc.,Cary,NC,USA)andWekaver.3.9[23].ResultsTherewere38,597distinctadultpatients(>18-year-old)intheMIMIC-IIIdatabaseadmittedtocriticalcareunitsbetweenJune2001toOctober2012(90%fromemergencyroomadmission,8%electivesurgery,and2%urgentcareservices),ofwhich217werediagnosedwithKS,14,391withGUD,23,931asOTHwhodidnothaveanyGUDnorKS,and7446asALPwithabdominal,back,flank,groinpain.Table1summarizespopulationcharacteristicsamongthethreegroups.TherewasanexcessoffemalesintheKSgroupascomparedtootherthreegroups(45.2%vs.54.3%,58.1%and52.4%,respectively,p<0.05).Mostsamplepopulationwereadmittedthroughemergencyorurgent(84.2%).Thedistributionofracewassimilarbe-tweenKSandGUD,butcomparingtoOTHandALP,KShadahigherproportionofwhite(76.5%vs.71.1%Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page3of14

PAGE 4

and72.7%)andblackAfricanAmerican(10.6%vs6.0%and7.4%,p=0.008).ThemedianeGFRinKSwas65.3,lowerthaninOTH(93.1,p=0.0013)andALP(77.3,p<0.0001),buthigherthanGUD(49.3,p<0.0001).Themedian(IQR)STONEscoreinKSformerswas4,higherthaninGUD(2,p<0.0001)orinOTH(2,p<0.0001),butnotdifferentfromALP(4,p=0.46).Figure1showsthecomparisonofthedistributionsofagecategoriesbygender,CCIandBMIinthethreegroupsofKS,GUDandOTH.ThehighestratesofKSwereseenintheagegroup71–80forbothmales(30%)andfemales(23%),andtheratesofKSincreasedsignificantlyafter50years-of-ageinmales,whileinfe-malesasteadyincreasewasobservedafter30years-of-agewithalevelingoffafter70years.AsforBMI,KShadthehighestoveralldistribution(median29.1)amongallfourgroups(medianofGUD,OTHandALP:27.5,27.2,27.0),italsohadthehighestproportionofobese(17%vs11%inGUD,9%inOTHand2%inALP,allp-values<0.05).Figure2showsthemostfrequentICD-9diagnosesinallfourgroupsofKS,GUD,OTHandALP,collatingthetop-10frequenciesofeachgroup.Essentialhypertension(45.8%),disordersoffluid,electrolyte,andacid-basebal-ance(44%),andsepticemia(41.7%)weremostfrequentlydiagnosedconditionsamongKSpatients.Someofthesehighfrequencycomorbiditiesalsohaddifferentdistribu-tioninKScomparedtoothergroups.Forexample,ratesofsepticemiaandcertainadverseeffects(includinganaphylaxis,unspecifiedmedicationadverseeffects,un-specifiedallergy,etc.)inKSwerehigherthaninGUD,OTHorALP(18%,36%and23%higherrespectively).Theproportionofessentialhypertensionwas10%higherinKSthanGUDorALPbutwassimilartotherateinOTH;heartfailureandhypertensiverenaldiseasehadmuchlowerrates(14%and16%lessrespectively)inKSthaninGUD,buttherateswerehigherinKSFcompa-ringtoOTH(8%and10%higher).WhenlookingattheSTONEvariables,wefoundthathematuriawaspositivelyassociatedwithKS(7.4%vs.4.6%inGUD,p=0.051,andvs.1.1%inOTH,p<0.0001,andvs.1.5%inALP,p<0.0001);98.6%ofKSformershadexperiencedpainwhile53.1%ofGUDand57.6%ofOTHhadpainevents(bothp<0.0001);0.92%ofKSformershadvomiting Table1Characteristicsofthestudypopulation(n=38,597),stratifiedbyoutcomegroupkidneystones(KS)othergenitourinarydiseases(GUD)otherconditions(OTH)acutelocalizedpain(ALP)%(N)%(N)p-value%(N)p-value%(N)p-valueTotal21714,39123,9317446GenderMale45.2%(98)54.3%(7816)0.007258.1%(13895)0.000152.4%(3902)0.04EthnicityWhite76.5%(166)72.0%(10359)0.3471.1%(17007)0.000872.7%(5413)0.04Black10.6%(23)10.4%(1493)6.0%(1445)7.4%(549)Hispanic3.2%(7)2.9%(416)3.5%(833)3.1%(231)Asian1.4%(3)2.5%(358)2.3%(555)1.7%(124)Other8.3%(18)12.2%(1765)17.1%(4091)15.2%(1129)InsuranceMedicare/Medicaid/Government67.3%(146)77.6%(11164)0.000257.8%(13837)0.009464.7%(4814)0.48Selfpay/Private32.3%(70)21.7%(3130)40.4%(9671)34.4%(2560)Missing0.5%(1)0.7%(97)1.8%(423)1.0%(72)AdmissiontypeElective9.2%(20)8.2%(1176)0.9220.2%(4841)<0.000116.0%(1193)0.01Emergency89.4%(194)89.6%(12895)76.9%(18406)80.7%(6005)Urgent1.4%(3)2.2%(320)2.9%(684)3.3%(248)Median(IQR)Median(IQR)Median(IQR)Median(IQR)Age67(56–77)72(59–82)<0.000162(50–75)0.004964(51–77)0.083BMI29.1(28.3–30.5)27.5(26.9–27.6)0.1527.2(27.2–27.2)0.002327.0(26.9–27.1)<0.0001CharlsonIndex1(0–2)2(1–4)<0.00011(0–2)0.741(0–3)0.12eGFR50.8(33.4,81.9)38.9(22.5,62.3)0.001382.5(59.1,98.7)<0.000165.1(37.3,93.9)<0.0001STONE4(2–4)2(2–4)<0.00012(2–4)<0.00014(2–4)0.46Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page4of14

PAGE 5

and0.46%hadnausearecorded,andthepercentagesofvomitingandnauseainKSwereslightlyhigherthaninotherthreegroups.Hydronephrosis(variablefromSTONE-PLUS)wasalsopositivelyassociatedwithKS(35.94%vs.1.54%inGUD,p<0.0001andvs.0%inOTH,p<0.0001).Next,weperformedunivariateanalysisofICD-9diag-nosisandlabtestscomparingKSwithGUD/OTH/ALP.Atotalof940distinctthree-letterICD-9codeswereidentifiedinthewholestudypopulation;aftercodefilteringbasedonlowfrequency(<5casesinKS),83variablesremained.Forlaboratorytests,atotalof754entrieswerefound,furthercondensedto637bymanualinspectionofphysicians,andreducedto69afterfre-quencyfiltering.Thefrequenciesofmissingvaluesoftheseincludedlabtestsrangesfrom0to45%,66.0%and45.2%inGUD,OTHandALPrespectively,withthemajorityofthemhavelessthan50%ofmissing.Table2showsfrequenciesofthetopICD-9diagnosisidentifiedthroughunivariateanalysis,selectingthosewithanFDR-adjustedp-valuebelow0.1(uptothetop-25).Overall,7ICD-9weredifferentiallydistributed Fig.2Prevalenceofthetop-10mostfrequentICD-9diagnosesinKS,GUD,OTHandALPgroups Fig.1Distributionsofagecategoriesbygender,CCIandBMIinKS,GUD,OTHandALPgroupsChenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page5of14

PAGE 6

betweenKSandGUDatthe5%FDRlevel,while25ofthemwerefounddifferentbetweenKSandOTHorALPatthesamesignificancelevel.Outofthe69labtestsperformedinmorethanhalfofKSpatients,43,50,and25showedasignificant(5%FDRlevel)meanordistributionlocationshiftbetweenKSvs.GUD,KSFvs.OTH,andKSFvs.ALP,respectively.Thetop-25labtestsrankisshowninTable3.Inordertoderiveamulti-domaindiagnosticmodelofKSdiagnosis,wefitteddifferentlogisticmodelsonse-lectedcovariateinputdomains,asspecifiedintheMethodssection,andcomparedagainsttheSTONE.Table4summarizestheperformanceindicesformodels(a)through(h),showingaverage(st.dev.)AUROC,sensi-tivity,specificityacross10-foldcross-validationruns(i.e.resultsobtainedonthetestdata),alongwiththebestYoudenÂ’sJ.Figure3(toppanels)showstheROCcurvesforeachmodel,alsoobtainedbyaveragingthe10testssets,fortheKSvs.)GUD,KSvs.OTH,andKSvs.ALPdatasamples.Overall,model(f),i.e.thetop-rankedICD-9diagnosisandlaboratorytestsplusdemographicvariables,andmodel(g),i.e.thestepwiseselectionoffeaturesincludedinmodel(f),showedthebestperfor-mance,withAUROCs~80%.Allothermodelsweresig-nificantlylessperformant(adjustedp<0.05)thanthesetwo.Followingcross-validatedAUROCranking,thesec-ondbest-performingmodelswerethosewithtop-rankedICD-9codes(d),laboratorytests(e),CCI(b),eGFR(c),anddemographicsalone(a).Notably,modelsusingtop-rankedICD-9diagnosticcodesshowedhighsensitivityandmoderatespecificity,whilemodelsusingtoplabtestsshowedmoderatesensi-tivityandhighspecificity,whilebothhighsensitivityandhighspecificitywereachievedinthemulti-domainmodels.TheSTONEmodel(h)yieldedrelativelylowAUROC(62%forKSvs.GUD,64%forKSvs.OTH,and61%forKSvs.ALP).WhenweaddedtheICD-9codeforhematuriaandotherGUDcodestothesetofinputvariablesformodels(f)and(g),performanceincreasedsignificantly:ForKS Table2Top-rankedICD-9diagnosesdifferentiallyassociatedwithKSvs.GUD/OTH/ALPICD-9codeConditionfrequencyinkidneystones(KS)othergenitourinarydiseases(GUD)otherconditions(OTH)acutelocalizedpain(ALP)frequencyp-value*frequencyp-value*frequencyp-value*401Essentialhypertension45.6%(99)35.4%(5101)0.0248.5%(11447)0.6337.6%(2797)0.0224276Disordersoffluid,electrolyte,andacid-basebalance43.8%(95)45.6%(6565)0.7818.3%(4375)<0.000127.9%(2047)<0.000138Septicaemia41.5%(90)23.4%(3361)<0.00015.4%(1297)<0.000116.3%(1216)<0.0001995Certainadverseeffects39.6%(86)21.4%(3078)<0.00014.8%(1159)<0.000113.2%(983)<0.0001785Symptomsinvolvingcardiovascularsystem24.9%(54)18.5%(2664)0.095.7%(1353)<0.000111.1%(827)<0.0001428Heartfailure24.9%(54)38.3%(5508)0.0016.2%(3867)0.00229.3%(2185)0.194541Otherbacteriainfections21.2%(46)16.3%(2350)0.143.1%(752)<0.00017.5%(559)<0.0001287Purpuraandotherhemorrhagicconditions12.9%(28)11.7%(1680)0.765.1%(1211)<0.00017.9%(589)0.0293790Nonspecificfindingsonexaminationofblood12.0%(26)9.8%(1413)0.545.2%(1243)<0.00016.0%(449)0.0019403Hypertensiverenaldisease11.1%(24)27.0%(3892)<0.00011.2%(278)<0.000110.8%(805)0.9642278Obesityandotherhyperalimentation10.1%(22)6.0%(863)0.084.7%(1121)0.0014.2%(315)0.0003311Depressivedisorder,notelsewhereclassified10.1%(22)7.7%(1104)0.775.9%(1403)0.014.7%(351)0.0015300Neuroticdisorders9.2%(20)5.68%(817)0.085.71%(1366)0.034.0%(295)0.0003327Sleepdisorders9.2%(20)5.4%(778)0.093.5%(843)<0.00012.9%(212)<0.0001112Candidiasis8.8%(19)4.3%(619)0.022.0%(479)<0.00014.0%(298)0.0023416Chronicpulmonaryheartdisease7.8%(17)6.9%(989)0.773.4%(809)0.0014.3%(317)0.0365799Decreasedlibidoandotherill-definedconditions7.4%(16)4.7%(668)0.192.4%(569)<0.00012.5%(186)<0.0001788Symptomsinvolvingurinarysystem7.4%(16)5.7%(820)0.773.0%(712)0.0013.1%(229)0.0019288Diseasesofwhitebloodcells6.0%(13)4.7%(674)0.772.6%(629)0.0012.0%(148)0.0004574Cholelithiasis5.5%(12)2.9%(419)0.111.6%(392)<0.00013.1%(230)0.0558570Acuteandsubacutenecrosisofliver4.1%(9)5.0%(717)0.760.8%(184)<0.00013.1%(228)0.4404345Epilepsy3.7%(8)3.4%(491)0.872.9%(688)0.481.5%(111)0.0345346Migraine2.8%(6)0.79%(113)0.021.32%(316)0.110.8%(57)0.0052*first25ICD-9codesorthosewithafalsediscoveryrate-adjustedp-value<0.05areshownChenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page6of14

PAGE 7

Table3Top-rankedlaboratorytestsdifferentiallyassociatedwithKSvs.GUD/OTH/ALPLabtestitemkidneystones(KS)othergenitourinarydiseases(GUD)otherconditions(OTH)acutelocalizedpain(ALP)meanMedian(IQR)MeanMedian(IQR)p-value*MeanMedian(IQR)p-value*MeanMedian(IQR)p-value*UrineWhiteBloodCells(WBC)(#/hpf)5.54.9(3.1,6.2)3.02.2(1.7,2.9)<0.00011.51.5(1.5,1.5)<0.00012.11.2(1.2,2.2)<0.0001RedBloodCells(RBC)(#hpf)5.84.5(3.3,6.2)3.42.4(1.7,3.3)<0.00011.91.4(1.4,1.4)<0.00012.71.4(1.4,2.4)<0.0001Creatinine(mg/dL)80.073.0(73.0,73.0)84.778.0(70.5,86.0)0.0377.975.0(75.0,75.0)0.0381.975.0(75.0,78.0)0.03Protein(mg/dL)4.04.0(3.4,4.2)3.93.8(3.4,4.0)0.033.53.4(3.4,3.4)<0.00013.73.4(3.4,3.8)<0.0001pH6.06.0(5.5,6.5)5.85.7(5.2,6.2)<0.00016.06.0(5.5,6.5)0.866.06.0(5.4,6.5)0.17BloodBands4.53.0(2.0,3.8)2.51.3(1.0,1.9)<0.00011.50.8(0.8,0.8)<0.00012.30.8(0.5,2.0)<0.0001Potassium(mEq/L)4.14.0(3.9,4.1)4.34.2(4.0,4.4)<0.00014.14.1(3.9,4.2)0.094.24.1(3.9,4.3)<0.0001Lipase(U/L)67.128.0(26.0,33.0)62.635.0(30.0,42.0)0.0344.931.0(31.0,31.0)0.0565.431.0(27.0,45.0)0.03Magnesium(mEq/L)2.02.0(1.8,2.1)2.12.0(1.9,2.2)<0.00012.02.0(1.9,2.1)0.032.02.0(1.9,2.1)0.01Glucose(mg/dl)146.0137.3(137.3,137.3)143.6134.0(127.2,141.0)0.03139.5133.3(127.6,139.7)0.03141.0133.3(124.2,145.3)0.03CreatineKinase,MBIsoenzyme(ng/mL)8.54.4(4.0,5.0)0.05(4.0,6.8)0.0311.55.0(5.0,5.0)0.3211.95(4.7,6)0.03RedCellDistributionWidth(RDW)(%)14.914.5(13.7,15.7)15.715.3(14.2,16.7)<0.000114.514.1(13.4,15.2)0.00115.315.0(13.92,16.3)0.005TotalCO2(mEq/L)24.024.0(22.0,25.5)24.824.8(22.7,26.9)0.0325.925.8(24.7,27.0)<0.000125.725.8(23.8,27.4)<0.0001RedBloodCells(cells/mcL)3.73.6(3.3,4.0)3.53.4(3.2,3.8)<0.00013.73.7(3.3,4.1)0.093.63.5(3.3,3.8)0.01Chloride(mEq/L)102.6102.4(101.3,103.7)101.9102.0(100.5,103.3)0.001101.9102.0(100.5,103.0)<0.0001101.8101.9(100.7,103.0)<0.0001UreaNitrogen(mg/dL)25.720.8(14.1,30.7)34.529.5(19.5,44.4)<0.000117.815.8(12.0,21.0)<0.000125.720.1(13.8,32.3)0.95Creatinine(mg/dL)1.41.1(0.8,1.6)1.91.4(1.0,2.1)0.0030.90.8(0.7,1.0)<0.00011.30.9(0.7,1.4)0.80Albumin(g/dL)3.33.2(2.9,3.6)3.13.1(2.8,3.5)0.0023.53.5(3.3,3.7)<0.00013.23.3(2.8,2.6)0.31Phosphate(mg/dL)3.33.2(2.8,3.7)3.73.5(3.1,4.1)<0.00013.33.3(2.9,3.6)0.603.53.4(3.0,3.9)<0.0001OxygenSaturation(mmHg)90.192.0(92.0,92.0)90.392.1(90.4,93.8)0.7893.495.3(95.3,95.3)<0.000191.595.0(90.3,95.6)0.02BaseExcess(mEq)1.61.4(2.7,0.3)1.00.6(2.5,0.9)0.150.10.1(0.6,1.0)<0.00010.20.1(1.6,1.4)<0.0001pH7.47.4(7.4,7.4)7.47.4(0.3,7.4)0.447.47.4(7.4,7.4)<0.00017.47.4(7.3,7.4)0.001pO2(kPa)4.84.8(4.6,4.9)4.84.8(4.6,5.0)0.795.15.1(4.9,5.3)<0.00014.94.9(4.7,5.1)0.001Bicarbonate(mEq/L)24.624.8(22.6,26.5)24.724.8(22.5,26.9)0.7825.825.8(24.2,27.4)<0.000125.525.5(23.6,27.2)0.001LactateDehydrogenase(LD)(IU/L)291.9237.3(230.9,250.0)399.4268.0(230.3,320.0)0.08272.4235.7(235.7,235.7)0.03330.4235.7(226,301)0.29*first25labitemsorthosewithafalsediscoveryrate-adjustedp-value<0.1areshownChenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page7of14

PAGE 8

vs.GUD,model(g)achievedAUROCof88%(p<0.0001w.r.t.modelswithnonGUD-specificICD-9codes)withsensitivityof77%andspecificityof87%;forKSvs.OTH,model(g)achievedAUROCof98%(p<0.0001),withsensitivityof88%andspecificityof98%;forKSvs.ALP,model(g)achievedAUROCof87%(p<0.0001),withsensitivityof81%andspecificityof82%.Model(f)hadverysimilarperformance(notshown).However,theseGUDvariablesweremeasuredconcurrentlywithKS,sowedidnotincludetheminourfinalpredictionmodel,butitcouldbeusedasinputiftheseGUDvariableshappenedinoneÂ’shistorytoimprovethepre-dictivityandperformanceofthemodels.Whenweappliedthemachinelearningtechniques,usingthesamecross-validationsettings,forthecomparisonbetweenKSandGUDorOTH,wedidnotobserveasubstantialincreaseinperformanceindiceswiththeusageoftheLogitBoostselectorinalternativetothestepwise,butanincreasedperformancewasobservedforKSvs.ALP(p<0.0001).ThevariablesselectedbytheLogitBoostwereconcordantwiththevariablesselectedfromstepwiselogisticregressionmodel(g),althoughtheLogitBoosttendedtoselectafewmore.Thedecisiontreeshowedapeculiarbehaviorascomparedtothelogisticregression,withincreasedsensitivityathigherspecificitybutthenlowerplateau.Therandomforestshowedhigher(almostperfect)AUROCandsensitivity/specificity(significantbelowthe0.0001levelwithrespecttothelogisticregressionanddecisiontree)andthesuperlearnerwascomparabletotherandomforest.Infact,thehighestweightofthesuperlearnerwasthatoftherandomforest,followedbythedecisiontree,asinglerule,andtheLogitBoost.ThebottompanelsofFig.3showthecross-validatedROCcurvescorrespondingtoKSvs.GUD,KSvs.OTH,andKSvs.ALP.ThedecisiontreeforKSvs.ALPisdepictedinFig.4.UsingtheSMOTE,performanceresultsforallmodelswerelowerbutrankingsimilar(notshown). Table4Comparisonofpredictionperformanceofdifferentmodels,using10-foldcrossvalidationModel/OutcomeAUCSensitivitySpecificityJkidneystones(KS)vs.othergenitourinarydiseases(GUD)(a)Demographic0.63(0.02)0.690.510.20(b)CharlsonÂ’scomorbidityindex0.69(0.02)0.690.620.31(c)eGFR0.62(0.02)0.650.560.21(d)ICD0.74(0.02)0.750.630.38(e)Labs0.76(0.02)0.670.740.41(f)All0.81(0.02)0.750.760.51(g)All(Stepwise)0.80(0.02)0.760.710.47(h)STONE0.62(0.02)0.550.640.19KSvs.otherconditions(OTH)(a)Demographic0.65(0.02)0.640.620.27(b)CCI0.65(0.02)0.680.570.25(c)eGFR0.71(0.02)0.590.750.35(d)ICD0.82(0.02)0.680.870.55(e)Labs0.90(0.01)0.810.870.68(f)All0.92(0.01)0.900.800.70(g)All(Stepwise)0.92(0.01)0.900.810.71(h)STONE0.64(0.02)0.620.650.27KSvs.acutelocalizedpain(ALP)(a)Demographic0.60(0.02)0.710.470.18(b)CharlsonÂ’scomorbidityindex0.62(0.02)0.600.610.21(c)eGFR0.57(0.02)0.590.440.15(d)ICD0.77(0.02)0.740.690.43(e)Labs0.85(0.02)0.660.900.56(f)All0.88(0.01)0.780.850.63(g)All(Stepwise)0.86(0.01)0.810.820.63(h)STONE0.61(0.02)0.580.600.18Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page8of14

PAGE 9

Thefinalmodelofchoicewasthestepwise-selectedmodel(g),becauseinconjunctionwithoptimalper-formance,itincludedfewervariablesthanmodel(f)(15variablesforeachcomparisonvs.50variablesinmodel(f)).Table5displaysthefinalmodel(withoddsratiosandconfidenceintervals)whichwenameastheDiagnosticAcuteCareAlgorithmforKidneyStones(DACA-KS).ThestepwiseregressionforKSvs.GUDyieldedafewnonspecificpredictors(e.g.nonspecificfindingsonexaminationofblood(ICD-9:790),Othercomplicationsofprocedures(ICD-9:998))whichwereremovedwithoutlossinperformance.Inaddition,althoughrandomforestandsuperlearnershowedbetterperformance,giventhehighclassim-balancewehaveinthesamplepopulation,wecannotsureaboutthegeneralizabilityofthesemodelsindifferentdataset,sowefocusedmoreoninterpretabi-lityespeciallywhenthelogisticregressionmodelhadgoodperformanceaswell.Infact,theSMOTEperformanceestimatesofthesuperlearneraswellasoftherandomforestarelower.DiscussionInthislargesampleofindividualsadmittedtoacutecarebetween2000and2012,weaimedtoinferamulti-domain,personalized,diagnosticalgorithmriskassessmentforKSdisease.Witharobustmodelcollectionandselectionframework,undercross-validationsettings,wedemon-stratedthattheintegratedmodelimprovesbothspecificityandsensitivityascomparedtoasingledomainmodel.Also,itincludesmoreextensiveparameterscomparedtotheSTONEscore.TheSTONEscoreutilizespresentationsofKS-relatedsymptoms(pain,hematuria,nausea/vomiting)andtwodemographicpredictors(genderandrace).Inoursamplepopulation,onlyasmallproportionof(KS)patientshadhematuriaandnausea/vomitingpresentorrecorded.Ourstudyevaluatedthousandsofpotentialpredictorsamongthedifferentdomains,comparingrelativepropor-tionsandshiftsindistributionsbetweenKSformersandtheGUD,OTHandALPgroups,ourmodelcanmakeper-sonalizedpredictionforeachindividualbasedonhis/herparametersfromdifferentdomains.Thefeaturesusedinourfinalmodelsareusuallyroutinelytestedincriticalcare Fig.3ModelcomparisonviaAUROC.Legend:Leftpanels:kidneystone(KS)formersvs.othergenitourinarydiseases(GUD);middlepanels:KSvs.othernon-genitourinary(OTH)conditions;rightpanels:KSvs.acutelocalizedpain(ALP)intheabdomen,back,flank,orgroin.Toppanels:logisticregressionmodelsuponstepwisefeatureselection,fitonselectedinputdomains;Bottompanels:comparisonofmachinelearningtechniquesonthefullinputset.Curvesshownareaveragedover10-foldcross-validation,i.e.usingthetestsetsChenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page9of14

PAGE 10

unit,ortestedatadmission,therefore,allinformationtoimplementourmodelshouldbeavailableinanICUsetting,andcanbeeasilyadaptedtodifferentclinicalsettingsbyaddingorremovingfeatures.WereportaseriesofnovelfindingsinKSthataresignificantlydifferentthanGUD,OTHandALPpopulationsandwhichcouldaidinthetri-ageofpatientswhentheypresenttotheEDorareadmit-ted/transferredintocriticalcare.Anumberofthesevariablesareworthofdiscussionindetail.Inourstudycohort,wefoundthatKSpeakedatthe7thdecadeofage;withvariationofprevalenceatdifferentagegroupsbetweenbothgenders,overall,wefoundahigherprevalenceoffemalesinthiscohort.KSprevalencewasthehighestinnon-Hispanicwhites,similarlytootherstudies[1].LowerratesofprivateinsurancecoveragewerefoundinKS(comparingwithOTH),whichsuggeststhatsocio-economicstatusmaycontributetoriskfactorsassoci-atedwithKS.Previousstudiesshowedthatlowerincome[1]andlowercoverageofprivateinsurance[24]areassoci-atedwithhigherriskofKS[25].Inourpopulation,KSformershadthehighestpreva-lenceofobesitywhencomparedtotheGUD/OTH/ALPgroups,andourfinalmultivariatemodelsuggestedthatpatientswithobesityaretwotimesmorelikelytobedi-agnosedwithKScomparingwithGUDorALPpatients.WefoundthatKS,OTHandALPwereahealthierco-hortwithlowerCCIandhighereGFRwhencomparedtotheGUD.PreviousstudieshavedemonstratedthatKSformershavehigherriskofdevelopingchronickidneydiseases[4,26];infact,inourstudywefoundatendencytoadecreasedeGFRinKSwithrespecttoOTH/ALPgroups,andthispointstothenecessityofmonitoringandmanagementofKStopreventprogres-sionintochronickidneydisease.ThemostcommondiagnosisassociatedwithEDvisitswashypertension,anditsprevalencewashigherinpa-tientswithKScomparingtoGUDandALP.Disordersoffluids,electrolytes,andacid-basebalancewasalsofre-quentlyfoundinKSandGUD,butnotinthediagnosisintheOTH/ALPgroup.Ameta-analysisfoundthat Fig.4DecisiontreeforthediagnosisofKSpatientsvs.ALPpatientsLegend:Eachleafnodecontainsthepredictedclass(1ifKS,0ifALP)andthenumbersbetweenparenthesesindicatetotalnumberofinstances(firstnumber)reachingtheleaf,andthenumberofthoseinstancesthataremisclassified(secondnumber).Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page10of14

PAGE 11

increasingwaterintakewasassociatedwithsignificantlyreducedriskofkidneystonesanditwasdosedependentforeachincreaseof500mlofwater[27].ForKSfor-mers,thesinglemostsignificantpreventivemeasureisincreasingfluidintake.IntheGUDpopulation,disordersoffluidsandelectrolytesareawell-knownentity.Inaddition,diseasesofacidbaseandelectrolytessuchasrenaltubularacidosis(RTA)andpartialRTA,whichmaypresentwithhyperchloremicacidosis,hypokalemia,andnormalorminimallyreducedGFR[28],alsohaveahigherprevalenceofKS[29].Inter-estingly,inourKScohortwefoundhigherlevelsofserumchlorideandlowerlevelsofserumbicarbonate,lowerserumpotassiumlevels,andelevatedurineproteincomparingtotheGUD/OTH/ALPgroups,Additionalresearcheffortsmaybeabletofullyeluci-datethesignificanceofthesefindings.Wefoundthatpurpuraandotherhemorrhagiccondi-tionswerehigherintheKSpopulationwhencomparedtotheOTH/ALPpopulationbuttherewasnosignificantdifferencewhencomparedtotheGUDgroup.ThedistributionofserumlipaseandcreatininekinaseMBisoenzymeweresignificantlylowerinKSascomparedtotheGUDandOTH/ALPgroups.Renalhandlingoflip-aseinvolvesremovaloflipasefromserumbyglomerularfiltrationoflipasewithnearlycompleteabsorptionoffreeoxalateinthebowellumen[30].Disordersoflipidmeta-bolismhavebeenassociatedwiththemetabolicsyndromeandobesity[31].LowerlevelsoflipaseintheKSgroupneedstobefurtherelucidatedastherehavenotbeenpre-viousreportsofthisfinding.CreatininekinaseMB(CK-MB)isanenzymethatiselevatedinrenaldiseaseanditmaybeelevatedevenintheabsenceofmyocardialinjury;however,thesignificanceofitselevationis Table5TheDiagnosticAcuteCareAlgorithm-KidneyStones(DACA-KS)ItemsKidneyStonesvs.OtherGenitourinaryDiseasesKidneyStonesvs.OtherConditionsKidneyStonesvs.AcuteLocalizedPainOR(95%CI)p-valueOR(95%CI)p-valueOR(95%CI)p-valueInsurance1.70(1.25,2.32)0.001CharlsonComorbidityIndex0.81(0.75,0.88)<0.00010.91(0.84,0.97)0.010.88(0.82,0.94)0.001Certainadverseeffects(ICD-9:995)2.93(2.14,4.00)<0.00016.33(4.47,8.96)<0.0001Hypertensiverenaldisease(ICD-9:403)0.55(0.34,0.89)0.015.37(2.96,9.77)<0.0001Obesityandotherhyperalimentation(ICD-9:278)2.06(1.28,3.32)0.0032.18(1.25,3.82)0.04Candidiasis(ICD-9:112)2.37(1.39,4.06)0.002Decreasedlibidoandotherill-definedconditions(ICD-9:799)2.30(1.35,3.93)0.0021.98(1.08,3.64)0.033.40(1.85,6.27)0.001urinewhitebloodcells(WBC)1.13(1.10,1.16)<0.00011.26(1.22,1.30)<0.00011.15(1.11,1.19)<0.0001urinepH1.45(1.20,1.74)<0.0001urineprotein1.26(1.01,1.57)0.042.19(1.78,2.69)<0.00011.99(1.61,2.45)<0.0001bloodMagnesium0.17(0.08,0.35)<0.0001bloodChloride3.97(2.06,7.65)<0.00015.05(2.02,12.59)0.0012.67(1.01,7.03)0.05bloodAlbumin1.78(1.37,2.32)<0.0001bloodredbloodcells(RBC)1.60(1.22,2.10)0.0011.49(1.11,1.99)0.011.77(1.24,2.52)0.002Disordersoffluid,electrolyte,andacid-basebalance(ICD-9:276)1.70(1.23,2.37)0.002Otherbacteriainfections(ICD-9:041)5.47(3.60,8.31)<0.00012.67(1.81,3.93)0.01Sleepdisorders(ICD-9:327)3.06(1.76,5.31)<0.00013.29(1.80,6.02)0.001Chronicpulmonaryheartdisease(ICD-9:416)2.07(1.13,3.79)0.02Acuteandsubacutenecrosisofliver(ICD-9:570)3.14(1.38,7.14)0.01bloodpO20.52(0.36,0.76)0.001Neuroticdisorders(ICD-9:300)2.28(1.34,3.88)0.03Hyperplasiaofprostate(ICD-9:600)2.88(1.48,5.60)0.02bloodCO20.70(0.61,0.80)<0.0001bloodphosphate0.60(0.47,0.76)<0.0001urinebands1.15(1.07,1.24)0.002urineRDW0.79(0.71,0.88)<0.0001Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page11of14

PAGE 12

controversial[32].FurtherinvestigationiswarrantedtounveilboththeroleoflowlipaseandCK-MBisoenzymeinKSformers.AsetofneurologicfindingsinourstudydemonstratedthatmigraineheadacheswerehigherinKSandOTHcomparedtoGUD/ALP.Sleepdisorder,neuroticdis-order,anddepressiondisorderwerealsohigherinKSpatients.MigraineheadachemedicationssuchasTopa-maxpromotean(RTA)-likephenomenon[33].Sleepdisordersandfatiguehavebeenassociatedwithmigraineheadaches[34].Inouranalysis,sleepdisturbancesandlowlibidowerecorrelatedwiththediagnosisofKSwhencomparedtoGUD,OTHandALP.Lowlibidoduetolowtestosteronecouldbecorrelatedwithpoorsleepquality,sinceanormalcircadianrhythm/cycleisneces-saryforcentraleffectsonnormaltestosteroneproduc-tion[35].LowtestosteronelevelsnotonlyassociatedwithlowlibidobutalsohavebeenrelatedwithKS,Otunctermuretal.showedthatmaleKSpatientshadlowertestosteronelevels,althoughthepotentialcausalrelationshipwerenotconfirmed[36].Perhapsthemostimportantfindingandamonghighmorbidityandmortalityconditions,septicemiaandcan-didiasiswerefoundtohaveahighcorrelationwithKSformersonly.Reyneretal.[37]reportedthatofpatientspresentingtotheEDwithurosepsis,one-tenthpre-sentedwithanatomicurinaryobstruction,andthatmor-talitywashigherinthisgroup,occurringinalmostone-thirdofcases.Earlyimagingissuggestedinthisgroupofpatients,duetosuspectedanatomicobstruc-tionandneedforimmediateinterventiontoavoidmor-tality.OurdataconfirmsthisfindingofahigherrateofurosepsisinKSpatientswhencomparedtoothergroups.Thissuggeststhat,aspartofanalgorithmtoidentifypatientswithKS,ahighindexofsuspicionshouldtriggerimmediateactionwithearlyimagingtoidentifyanatomicurinaryobstructioninsepticpatientstopreventmortalities.Inaddition,thepresenceofcan-didiasiswasfoundtohaveahigherassociationaKSdiagnosis.Candidiasisisafungalinfectionthatcanvaryinpresentation-fromlocaltosystemicandinvasive,itmaybefoundamongdebilitated,elderlyandinpatientswithindwellingurethralcatheters[38],combiningwithourfindings,patientspresentingtotheEDwithcandi-duriamaybeconsideredforimmediateimagingtoiden-tifyanypotentialanatomicobstructionoftheurinarytract.Interestingly,somevariablesinthemodelwerenotdirectlyassociatedwithriskofkidneystone:com-paringtoOTHpatients,KSpatientsweremorelikelytohavechronicpulmonaryheartdiseaseoracuteandsub-acutenecrosisofliver.Theseconditionsmightbeassoci-atedwithcertainKSprognosticoutcomes.Futurestudiesthehelpfurthertheunderstandingsoftheseas-sociationsareneeded.Thereareseverallimitationsofourstudy.First,weanalyzedasamplefromasinglesite,withoutexternalvalidation;thecharacteristicsofpatientsintheKS,GUD,OTHandALParedifferentandtheremaybeaselectionbiaswhichwedidnotadjustfor.Inaddition,manypotentiallyusefullabtestsweredroppedbecauseoflowfrequencyintheKSgroup;otherrelevantlabpredictorsforKSmaybefoundoutsidethoseroutinelymeasuredinpeoplebeingtriagedattheEDbasedonadmissionÂ’ssymptoms.Second,therewasahigh-classimbalance,forwhichthepowerofthestudycanbeaffected,aswellasthederivationofadiagnosticmodel,eventhoughwetriedtoaddressinpartthisissueusingtheSMOTEtechnique.Third,whenusinglogisticregression,wedidnotconsiderinteractionsamongvariables(consid-eringonlytwo-waysinteractionswouldhaveproducedn2variables,andwewouldhaveneededtousemoreefficientlibraries,withparallelorcloudcomputing),thereforethemodelassumedalinearrelationship.En-semblemethods,i.e.therandomforestandthesuperlearner,achievedalmostperfectperformance,buttheresultwasnotconfirmedwiththeSMOTEclassrebalancing,andthiswarrantsfurtherexternalvalidationusingtheTRIPODprotocol[39].Eventhoughweusednestedcross-validationforparameteroptimization,theremayhavebeenoverfitting.Fourth,weacknowledgeasubparcalculationoftheSTONEscorebecausewecouldnotassessthedurationofpain,andthesmallnumberofsubjectswithvomitingandnauseainoursampleindicatingtheremaybeunder-reportingduringdatacollection.Duetothecross-sectionalnatureofthisstudy,wecannotdeter-minethecausalityofthepredictorsforKSformation,butevenusinglongitudinaldatabasewithvariablesonlyfromearlierdata,thecausalityofthepredictorsisstillunabletobeconfirmed.Futurestudiesmayhelpaddresstheselimitationsandhelpdesigningearly-riskdiagnosticmodelsapplicabletothegeneralpopulation.Despitetheselimitations,ourstudyprovidedacom-pactandhigh-performancediagnosticmodelfordiagno-sisofKS.ConclusionsDACA-KScouldbeintegratedintoelectronichealthsystems;thealgorithmhasthepotentialusedasaneffectivetooltohelpnursesandhealthcarepersonnelduringtriageorcliniciansmakingadiagnosis,streamlin-ingpatientsÂ’managementinacutecare.Asweentertheeraofprecisionmedicine,weenvisionafamilyofDACA-modelsformanyotherconditionsinadditiontoKS,derivedinthesamewayfrombigintegratedbiomedicaldatabases.Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page12of14

PAGE 13

AbbreviationsALP:Acutelocalizedpain;AUROC:Areaunderthereceiveroperatingcharacteristics;BMI:Bodymassindex;CCI:CharlsonComorbidityIndex;DACA:Diagnosticacutecarealgorithm;ED:Emergencydepartment;eGFR:Estimatedglomerularfiltrationrate;GUD:Genitourinarydiseases;KS:Kidneystones;OTH:Otherconditions;SMOTE:SyntheticMinorityOver-samplingTechniqueAcknowledgementsWewouldliketothankMITLabforComputationalPhysiologyformakingtheMIMICdatabaseavailableforresearchuse.FundingThisworkwassupportedbytheUFHCC/IOACancer-AgingCollaborativeGrantProgram,thefundingbodyhasnorolesinthedesignofthestudyandcollection,analysis,andinterpretationofdataandinwritingthemanuscript.AvailabilityofdataandmaterialsThedatasetsgeneratedandanalysedduringthecurrentstudyareavailableintheMIMIC-IIIdatabases,https://mimic.physionet.org/[10].Authors’contributionsZCdesignedthestudy,carriedouttheanalysis,revisedtheresults,draftedandrevisedthemanuscript.MPdesignedthestudy,carriedouttheanalysis,revisedtheresults,draftedandrevisedthemanuscriptandgavefinalapprovalforsubmission.VYB,RR,MSS,JB,SRKandMCErevisedthemanuscriptandgaverelevantintellectualcontribution.Allauthorsreadandapprovedthefinalmanuscript.EthicsapprovalandconsenttoparticipateTheuseofMIMIC-IIIdatabasewasapprovedbythedataprovider(BethIsraelDeaconessMedicalCenterandtheMassachusettsInstituteofTechnology)aftercompletionofrequiredCollaborativeInstitutionalTrainingInitiative(CITI)training,andaDataUseAgreementwassigned.Therequirementforindividualpatientconsentwaswaivedbecausethestudydidnotimpactclinicalcareandallprotectedhealthinformationwasdeidentified.De-identificationwasperformedincompliancewithHealthInsurancePortabilityandAccountabilityAct(HIPAA)standardsinordertofacilitatepublicaccesstoMIMIC-III,andprotectedhealthinformation(PHI)wereremoved.ThestudyprotocolofthisspecificanalysiswasapprovedbytheUniversityofFloridaIn-stitutionalReviewBoard.ConsenttopublicationNotapplicable.CompetinginterestsJiangBianandMattiaProsperiareAssociateEditorsforBMCMedicalInformaticsandDecisionMaking.Publisher’sNoteSpringerNatureremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.Authordetails1DepartmentofEpidemiology,CollegeofPublicHealthandHealthProfessions&CollegeofMedicine,UniversityofFlorida,2004MowryRoad,POBox100231,Gainesville,Florida32610-0231,USA.2DepartmentofUrology,UniversityofFlorida,Gainesville,Florida,USA.3DivisionofNephrology,Hypertension,&RenalTransplantation,UniversityofFlorida,Gainesville,Florida,USA.4DepartmentofHealthOutcomesandBiomedicalInformatics,UniversityofFlorida,Gainesville,Florida,USA.5DepartmentofPathology,Immunology,andLaboratoryMedicine,Gainesville,Florida,USA.6DepartmentofEmergencyMedicine,UniversityofFlorida,Gainesville,Florida,USA.Received:31October2017Accepted:6August2018 References1.ScalesCDJr,SmithAC,HanleyJM,SaigalCS.urologicdiseasesinAmericaproject.PrevalenceofkidneystonesintheUnitedStates.EurUrol.2012Jul;62(1):160–5.2.AntonelliJA,MaaloufNM,PearleMS,LotanY.UseoftheNationalHealthandnutritionexaminationsurveytocalculatetheimpactofobesityanddiabetesoncostandprevalenceofurolithiasisin2030.EurUrol.2014Oct;66(4):724–9.3.MorganMS,PearleMS.Medicalmanagementofrenalstones.BMJ.2016Mar14;352:i52.4.RuleAD,BergstralhEJ,MeltonLJ3rd,LiX,WeaverAL,LieskeJC.Kidneystonesandtheriskforchronickidneydisease.ClinJAmSocNephrol.2009Apr;4(4):804–11.5.GrahamA,LuberS,WolfsonAB.Urolithiasisintheemergencydepartment.EmergMedClinNorthAm.2011Aug;29(3):519–38.6.ProchaskaML,TaylorEN,CurhanGC.Insightsintonephrolithiasisfromthenurses’healthstudies.AmJPublicHealth.2016Sep;106(9):1638–43.7.MooreCL,BomannS,DanielsB,LutyS,MolinaroA,SinghD,GrossCP.DerivationandvalidationofaclinicalpredictionruleforuncomplicatedureteralSTONE--theSTONEscore:retrospectiveandprospectiveobservationalcohortstudies.BMJ.2014Mar26;348:g2191.8.HernandezN,SongY,NobleVE,EisnerBH.Predictingureteralstonesinemergencydepartmentpatientswithflankpain:anexternalvalidationoftheSTONEscore.WorldJUrol.2016Oct;34(10):1443–6.9.DanielsB,GrossCP,MolinaroA,SinghD,LutyS,JesseyR,MooreCL.STONEPLUS:Evaluationofemergencydepartmentpatientswithsuspectedrenalcolic,usingaclinicalpredictiontoolcombinedwithpoint-of-carelimitedultrasonography.AnnEmergMed.2016Apr;67(4):439–48.10.JohnsonAE,PollardTJ,ShenL,LehmanLW,FengM,GhassemiM,MoodyB,SzolovitsP,CeliLA,MarkRG.MIMIC-III,afreelyaccessiblecriticalcaredatabase.SciData.2016May24;3:160035.11.DeyoRA,CherkinDC,CiolMA.AdaptingaclinicalcomorbidityindexforusewithICD-9-CMadministrativedatabases.JClinEpidemiol.1992Jun;45(6):613–9.12.LeveyAS,StevensLA,SchmidCH,ZhangYL,CastroAF3rd,FeldmanHI,etal.Anewequationtoestimateglomerularfiltrationrate.AnnInternMed.2009;150(9):604–12.13.BenjaminiY,HellerR.Falsediscoveryratesforspatialsignals.JASA.2007Dec;102(480):1272–81.14.HastieT,TibshiraniR,FriedmanJ.Theelementsofstatisticallearning:datamining,inference,andprediction.Secondedition.NewYork:Springer;2009.15.NadeauC,BengioY.Inferenceforthegeneralizationerror.MachLearn.2003;52:239.16.Quinlan,J.R.C4.5:ProgramsforMachineLearning.MorganKaufmannPublishers,1993.17.FriedmanJ,HastieT,TibshiraniR.Additivelogisticregression:astatisticalviewofboosting.AnnStat.2000;28(2):337–407.18.Breiman,L.RandomForestsMachineLearning2001,45:5–32.19.vanderLaanMJ,PolleyEC,HubbardAE.Superlearner.StatApplGenetMolBiol.2007;6:Article25.20.ChawlaNV,BowyerKW,HallLO,KegelmeyerW.SMOTE:syntheticminorityover-samplingtechnique.JArtifIntellRes.2002;16:321–57.21.FawcettT.AnintroductiontoROCanalysis.PatternRecognLett.2006;27:861–74.22.SchistermanEF,PerkinsNJ,LiuA,BondellH.Optimalcut-pointanditscorrespondingYoudenindextodiscriminateindividualsusingpooledbloodsamples.Epidemiology.2005Jan;16(1):73–81.23.FrankE,HallMA,WittenIH.TheWEKAWorkbench.OnlineAppendixfor"DataMining:PracticalMachineLearningToolsandTechniques",MorganKaufmann,FourthEdition,2016.24.WatnickS,WeinerDE,ShafferR,InrigJ,MoeS,MehrotraR.DialysisadvisoryGroupoftheAmericanSocietyofnephrology.Comparingmandatedhealthcarereforms:theaffordablecareact,accountablecareorganizations,andtheMedicareESRDprogram.ClinJAmSocNephrol.2012Sep;7(9):1535–43.25.ScalesCDJr,LinL,SaigalCS,BennettCJ,PonceNA,MangioneCM,LitwinMS.NIDDKurologicdiseasesinAmericaproject.EmergencydepartmentrevisitsforpatientswithkidneystonesinCalifornia.AcadEmergMed.2015Apr;22(4):468–74.Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page13of14

PAGE 14

26.SandsJM,LaytonHE.Thephysiologyofurinaryconcentration:anupdate.SeminNephrol.2009May;29(3):178–95.27.ShangW,LiL,RenY,GeQ,KuM,GeS,XuG.Historyofkidneystonesandriskofchronickidneydisease:ameta-analysis.PeerJ.2017;5:e2907.28.XuC,ZhangC,WangXL,LiuTZ,ZengXT,LiS,DuanXW.Self-fluidManagementinPreventionofkidneystones:aPRISMA-compliantsystematicreviewanddose-responsemeta-analysisofobservationalstudies.Medicine(Baltimore).2015Jul;94(27):e1042.29.BuckalewVMJr.Nephrolithiasisinrenaltubularacidosis.JUrol.1989Mar;141(3Pt2):731–7.30.JungeW,MlyuszM,EhrensHJ.Theroleofthekidneyintheeliminationofpancreaticlipaseandamylasefromblood.JClinChemClinBiochem.1985Jul;23(7):387–92.31.MeadJR,IrvineSA,RamjiDP.Lipoproteinlipase:structure,function,regulation,androleindisease.JMolMed(Berl).2002Dec;80(12):753–69.32.JeremiasA,AlbiriniA,ZiadaKM,ChewDP,BrenerSJ,TopolEJ,EllisSG.Prognosticsignificanceofcreatinekinase-MBelevationafterpercutaneouscoronaryinterventioninpatientswithchronicrenaldysfunction.AmHeartJ.2002Jun;143(6):1040–5.33.WelchBJ,GraybealD,MoeOW,MaaloufNM,SakhaeeK.Biochemicalandstone-riskprofileswithtopiramatetreatment.AmJKidneyDis.2006Oct;48(4):555–63.34.LucchesiC,BaldacciF,CafalliM,DiniE,GiampietriL,SicilianoG,GoriS.Fatigue,sleep-wakepattern,depressiveandanxietysymptomsandbody-massindex:analysisinasampleofepisodicandchronicmigrainepatients.NeurolSci.2016Jun;37(6):987–9.35.PakzadR,SafiriS.PoorsleepqualitypredictsHypogonadalsymptomsandsexualdysfunctioninmalenon-standardshiftworkers:methodologicalissuestoavoidpredictionfallacy.Urology.2017Jan;2336.OtunctemurA,OzbekE,CakirSS,DursunM,PolatEC,OzcanL,BesirogluH.Urolithiasisisassociatedwithlowserumtestosteronelevelsinmen.ArchItalUrolAndrol.2015Mar31;87(1):83–6.37.ReynerK,HeffnerAC,KarvetskiCH.Urinaryobstructionisanimportantcomplicatingfactorinpatientswithsepticshockduetourinaryinfection.AmJEmergMed.2016Apr;34(4):694–6.38.PfallerMA,DiekemaDJ.Epidemiologyofinvasivecandidiasis:apersistentpublichealthproblem.ClinMicrobiolRev.2007Jan;20(1):133–63.39.MoonsKG,AltmanDG,ReitsmaJB,IoannidisJP,MacaskillP,SteyerbergEW,VickersAJ,RansohoffDF,CollinsGS.Transparentreportingofamultivariablepredictionmodelforindividualprognosisordiagnosis(TRIPOD):explanationandelaboration.AnnInternMed.2015Jan6;162(1):W1–73. Chenetal.BMCMedicalInformaticsandDecisionMaking (2018) 18:72 Page14of14


xml version 1.0 encoding UTF-8 standalone no
fcla fda yes
!-- Development of a personalized diagnostic model for kidney stone disease tailored to acute care by integrating large clinical, demographics and laboratory data: the algorithm stones (DACA KS) ( Mixed Material ) --
METS:mets OBJID AA00064852_00001
xmlns:METS http:www.loc.govMETS
xmlns:xlink http:www.w3.org1999xlink
xmlns:xsi http:www.w3.org2001XMLSchema-instance
xmlns:daitss http:www.fcla.edudlsmddaitss
xmlns:mods http:www.loc.govmodsv3
xmlns:sobekcm http:digital.uflib.ufl.edumetadatasobekcm
xmlns:lom http:digital.uflib.ufl.edumetadatasobekcm_lom
xsi:schemaLocation
http:www.loc.govstandardsmetsmets.xsd
http:www.fcla.edudlsmddaitssdaitss.xsd
http:www.loc.govmodsv3mods-3-4.xsd
http:digital.uflib.ufl.edumetadatasobekcmsobekcm.xsd
METS:metsHdr CREATEDATE 2019-02-22T10:10:34Z ID LASTMODDATE 2018-12-07T08:07:57Z RECORDSTATUS COMPLETE
METS:agent ROLE CREATOR TYPE ORGANIZATION
METS:name UF,University of Florida
METS:note Created using CompleteTemplate 'INTERNAL' and project 'NONE'.
OTHERTYPE SOFTWARE OTHER
Go UFDC FDA Preparation Tool
INDIVIDUAL
UFAD\renner
METS:dmdSec DMD1
METS:mdWrap MDTYPE MODS MIMETYPE textxml LABEL Metadata
METS:xmlData
mods:mods
mods:abstract displayLabel Abstract Background: Kidney stone (KS) disease has high, increasing prevalence in the United States and poses a massive
economic burden. Diagnostics algorithms of KS only use a few variables with a limited sensitivity and specificity. In
this study, we tested a big data approach to infer and validate a multi-domain personalized diagnostic acute care
algorithm for KS (DACA-KS), merging demographic, vital signs, clinical, and laboratory information.
Methods: We utilized a large, single-center database of patients admitted to acute care units in a large tertiary care
hospital. Patients diagnosed with KS were compared to groups of patients with acute abdominal/flank/groin pain,
genitourinary diseases, and other conditions. We analyzed multiple information domains (several thousands of variables)
using a collection of statistical and machine learning models with feature selectors. We compared sensitivity, specificity
and area under the receiver operating characteristic (AUROC) of our approach with the STONE score, using cross-validation.
Results: Thirty eight thousand five hundred and ninety-seven distinct adult patients were admitted to critical care between
2001 and 2012, of which 217 were diagnosed with KS, and 7446 with acute pain (non-KS). The multi-domain approach
using logistic regression yielded an AUROC of 0.86 and a sensitivity/specificity of 0.81/0.82 in cross-validation. Increase in
performance was obtained by fitting a super-learner, at the price of lower interpretability. We discussed in detail
comorbidity and lab marker variables independently associated with KS (e.g. blood chloride, candidiasis, sleep disorders).
Conclusions: Although external validation is warranted, DACA-KS could be integrated into electronic health systems; the
algorithm has the potential used as an effective tool to help nurses and healthcare personnel during triage or clinicians
making a diagnosis, streamlining patients management in acute care.
Keywords: Diagnostic algorithm, Kidney stones, Big data analysis.
mods:accessCondition type restrictions on use Rights The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
mods:language
mods:languageTerm text English
code authority iso639-2b eng
mods:location
mods:physicalLocation University of Florida
UF
mods:url access object in context http://ufdc.ufl.edu/AA00064852/00001
mods:name
mods:namePart Zhaoyi Chen
Victoria Y. Bird
Rupam Ruchi
Mark S. Segal
Jiang Bian
Saeed R. Khan
Marie-Carmelle Elie
Mattia Prosperi
mods:note Chen et al. BMC Medical Informatics and Decision Making (2018) 18:72
https://doi.org/10.1186/s12911-018-0652-4; Pages 1-14
mods:originInfo
mods:publisher BMC, BMC Medical Informatics and Decision Making
mods:dateIssued 2018
mods:recordInfo
mods:recordIdentifier source sobekcm AA00064852_00001
mods:recordContentSource University of Florida
mods:subject SUBJ650_1 fast
mods:topic Diagnostic algorithm
SUBJ650_2
Kidney stones
SUBJ650_3
Big data analysis
mods:titleInfo
mods:title Development of a personalized diagnostic model for kidney stone disease tailored to acute care by integrating large clinical, demographics and laboratory data: the diagnostic acute care algorithm kidney stones (DACA-KS)
mods:typeOfResource mixed material
DMD2
OTHERMDTYPE SOBEKCM SobekCM Custom
sobekcm:procParam
sobekcm:Aggregation ALL
UFIRG
UFIR
IUF
sobekcm:Wordmark UFIR
sobekcm:bibDesc
sobekcm:BibID AA00064852
sobekcm:VID 00001
sobekcm:Publisher
sobekcm:Name BMC, BMC Medical Informatics and Decision Making
sobekcm:Source
sobekcm:statement UF University of Florida
sobekcm:SortDate 736694
METS:amdSec
METS:digiprovMD DIGIPROV1
DAITSS Archiving Information
daitss:daitss
daitss:AGREEMENT_INFO ACCOUNT PROJECT UFDC
METS:techMD TECH1
File Technical Details
sobekcm:FileInfo
METS:fileSec
METS:fileGrp USE reference
METS:file GROUPID G1 METS1 unknownx-mets CHECKSUM b47002e6cc5cb4014ffebcff02b93b35 CHECKSUMTYPE MD5 SIZE 7624
METS:FLocat LOCTYPE OTHERLOCTYPE SYSTEM xlink:href AA00064852_00001.mets
G2 PDF2 applicationpdf 16a202b7cf756bfaf8ec77b623921215 2948326
12911_2018_Article_652.pdf
METS:structMap STRUCT2 other
METS:div DMDID ADMID (DACA-KS) ORDER 0 main
ODIV1 1 Main
FILES1 Page
METS:fptr FILEID
FILES2 2