<%BANNER%>

Minimal Detectable Change and Patient Reported Outcomes in Falls Rehabilitation

Permanent Link: http://ufdc.ufl.edu/UFE0022620/00001

Material Information

Title: Minimal Detectable Change and Patient Reported Outcomes in Falls Rehabilitation
Physical Description: 1 online resource (98 p.)
Language: english
Creator: Romero, Sergio
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: Rehabilitation Science -- Dissertations, Academic -- UF
Genre: Rehabilitation Science thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The overall aim of this project was to investigate the reliability of two instruments used for the assessment of balance and to explore patient's expectations and success criteria for the rehabilitation of falls. The first experiment investigated minimal detectable change (MDC) for two common instruments used to assess gait and balance. The results of this study indicated that for the Berg Balance Scale and the Dynamic Gait Index, 6.6 and 3.1 points respectively were required to be 95% confident that 'genuine' change had occurred. These results suggest that a significant amount of error is associated with these instruments. In addition, the results suggested that MDC values are not a constant feature of the instruments. MDC values for the high function group were 6.3 BBS points, as compared to 7.3 points for the low function group. That is, the values of MDC change based on the ability level of the persons assessed. The second experiment investigated patient's success criteria and expectations with treatment. Participants reported considerable initial levels of impairment in energy and drive, mobility, and pain. Lower scores were seen in interaction with people and community and social life. These findings suggest that domains with a strong social component were not as affected as domains with a strong physical component. Participants in this study required significant improvement to consider their treatment successful. Domains such as mobility; and energy and drive, required significantly larger reductions than the community and social life; and interactions with people domains. This provides information about what is important to patients receiving this intervention. Participants expected mobility to change the most. However, similar finding was reported in the domain of energy and drive. An interesting finding was that, participant's expectation was that the treatment would not meet their success criteria, indicating that residual levels of impairment were expected. Collectively, this series of studies promotes our understanding of significant change in patients receiving rehabilitation services related to falls. The results obtained indicate that current rehabilitation programs must consider the limitations of available instruments and take into consideration the needs and expectations of patients.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Sergio Romero.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Velozo, Craig A.
Local: Co-adviser: Light, Kathy E.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2010-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022620:00001

Permanent Link: http://ufdc.ufl.edu/UFE0022620/00001

Material Information

Title: Minimal Detectable Change and Patient Reported Outcomes in Falls Rehabilitation
Physical Description: 1 online resource (98 p.)
Language: english
Creator: Romero, Sergio
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: Rehabilitation Science -- Dissertations, Academic -- UF
Genre: Rehabilitation Science thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The overall aim of this project was to investigate the reliability of two instruments used for the assessment of balance and to explore patient's expectations and success criteria for the rehabilitation of falls. The first experiment investigated minimal detectable change (MDC) for two common instruments used to assess gait and balance. The results of this study indicated that for the Berg Balance Scale and the Dynamic Gait Index, 6.6 and 3.1 points respectively were required to be 95% confident that 'genuine' change had occurred. These results suggest that a significant amount of error is associated with these instruments. In addition, the results suggested that MDC values are not a constant feature of the instruments. MDC values for the high function group were 6.3 BBS points, as compared to 7.3 points for the low function group. That is, the values of MDC change based on the ability level of the persons assessed. The second experiment investigated patient's success criteria and expectations with treatment. Participants reported considerable initial levels of impairment in energy and drive, mobility, and pain. Lower scores were seen in interaction with people and community and social life. These findings suggest that domains with a strong social component were not as affected as domains with a strong physical component. Participants in this study required significant improvement to consider their treatment successful. Domains such as mobility; and energy and drive, required significantly larger reductions than the community and social life; and interactions with people domains. This provides information about what is important to patients receiving this intervention. Participants expected mobility to change the most. However, similar finding was reported in the domain of energy and drive. An interesting finding was that, participant's expectation was that the treatment would not meet their success criteria, indicating that residual levels of impairment were expected. Collectively, this series of studies promotes our understanding of significant change in patients receiving rehabilitation services related to falls. The results obtained indicate that current rehabilitation programs must consider the limitations of available instruments and take into consideration the needs and expectations of patients.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Sergio Romero.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Velozo, Craig A.
Local: Co-adviser: Light, Kathy E.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2010-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022620:00001


This item has the following downloads:


Full Text
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101222_AAAABU INGEST_TIME 2010-12-22T18:27:18Z PACKAGE UFE0022620_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 951195 DFID F20101222_AABFNC ORIGIN DEPOSITOR PATH romero_s_Page_08.jp2 GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
c66caf52349753fcf88a5ef0d186d9b6
SHA-1
931bdf2dd13c8379317d01ff3fee409ab018ef3e
110732 F20101222_AABFMN romero_s_Page_81.jpg
60889c5ca5e17732a2d5652a7bee82b2
6b98af2ebe1eb3f2135e3a381beaa428fc9a386a
107960 F20101222_AABFLZ romero_s_Page_62.jpg
f8b17289eafc094f96c53d6a599da19a
e2c65f34ca1de7a05928de043cc71d8533e67c8e
676555 F20101222_AABFND romero_s_Page_09.jp2
6019e9f33863a95b17cce02adafb5c64
4d15d8459636a57d95bc77d5bb74c12d170606e7
106052 F20101222_AABFMO romero_s_Page_82.jpg
eb9ffdd04db4d6f0ac9cdf8ecf2dc4a9
32f3530c2d77837f148a3c47ab14e571555e7552
1051849 F20101222_AABFNE romero_s_Page_10.jp2
c495f70865ab136158d2bb3ac1603f01
5fa9a334a28e519078af094f7930be14995c4963
98845 F20101222_AABFMP romero_s_Page_84.jpg
befe83a0ac791c48373780bbf802f671
3dbce83a7ed19b4c8c9edf62e6e92373e9c35b50
1051901 F20101222_AABFNF romero_s_Page_11.jp2
5b52eedd0417e6cc185c64580395b4d4
625a43efd880a8712d4d41f7f54c8922c0e436a6
101755 F20101222_AABFMQ romero_s_Page_85.jpg
451f3f70638b10932cf9485793d66606
952c1ee2125e336795f7bc6990ef7b7f6393ed82
1051925 F20101222_AABFNG romero_s_Page_12.jp2
373bc3dca6d990140f1c30473842b7a7
f807c9627538b85a8d68b91ae1de600825088a95
85714 F20101222_AABFMR romero_s_Page_87.jpg
9ef4bde3b3741a9ddc7d4573bbc6c3e5
b5a99ee4a49a1d40330110f09d5cfcb632757d9c
1051910 F20101222_AABFNH romero_s_Page_13.jp2
3125bb1087bed01978354524aefcc39d
fd6ed1a5f5cc163bf963a8e9701b291256c49b4a
112639 F20101222_AABFMS romero_s_Page_88.jpg
74d955feb3db945cb0f309db21a4374f
02d12963a2a9c67656ed39d2175a0a4958ce5335
1051985 F20101222_AABFNI romero_s_Page_14.jp2
17b0a808f4c3b3494cbb71301984d108
e5911363fe534ff1ef867aa71947bd3358fb8bf1
130802 F20101222_AABFMT romero_s_Page_89.jpg
e050db3fe62e794e0824b6f4663646d2
9225f655d1574b83f392c5447e96655103ba8c6b
1051974 F20101222_AABFNJ romero_s_Page_15.jp2
c1c772e2c68aab449369774ca5ca331a
a6d5bc873cc5b0fb60c52a3090852faa606dffca
139940 F20101222_AABFMU romero_s_Page_90.jpg
bcf4f7a5f619fea0c0b7ffb63f2b7ce6
3350c89e06f34fe72d244cb0624b04cbf70d2b6e
1051986 F20101222_AABFNK romero_s_Page_16.jp2
aa0eaaec1a919d44002a918e23bd6874
87f31012ad7962b3ad6744e4fb6134ae2a759de4
112335 F20101222_AABFMV romero_s_Page_91.jpg
55e2e19470ede2f59575b0c3c8c5e7e1
9f0fd08439ec7cf7a0cd1acd8342df2c3c689312
1051969 F20101222_AABFNL romero_s_Page_17.jp2
56b9d3a06e0efb397c307ce3dc3f7f34
3bd3b31d3e325fbec7c1aa7808dffd6ad538904b
138668 F20101222_AABFMW romero_s_Page_92.jpg
a9090d1c1e032250f6eeee6bbc44d5b6
65d3cb56059e414eae7cf7e7a737c127fb3b1a16
1051920 F20101222_AABFNM romero_s_Page_18.jp2
3bc1f81ec873b66045556b9dd14052e2
bc95991e986ce6dbf8b9fc131336437a958b746b
119192 F20101222_AABFMX romero_s_Page_94.jpg
bea8c7b1f5113bf10ba12bfb7015c722
03bdc50f3d61ffa2aa56593c87f0644b748ce42b
1051983 F20101222_AABFOA romero_s_Page_38.jp2
cbc99f18955bc58b211015105b29a454
38aa245ff1775d81edb750445d0804b3ebc3511d
F20101222_AABFNN romero_s_Page_19.jp2
1945c89081f276a246c78538b510123c
22476796b9193f65adb8eb289e47f3af9c9e8052
72486 F20101222_AABFMY romero_s_Page_98.jpg
1dbc54de1254226c1dd17cbed75f5dd8
08371f62e688ce134ad4598e1cd71328833828d7
246504 F20101222_AABFOB romero_s_Page_40.jp2
5d8fa805d38015db018be02eb9e351a6
b2499caddf8cf779b0b2e843f0ee8eed6b449021
246556 F20101222_AABFMZ romero_s_Page_01.jp2
11c856f6d4989c8c5b586e4cddf1723b
edb3f5f7df14e47958af1413c964bc7293a8346c
1051917 F20101222_AABFOC romero_s_Page_41.jp2
47bd1951c9a20bd5e616ac87dd80ffb5
c0e220a1c00aa247dca3d3c5e26875048c8f50d3
1051977 F20101222_AABFNO romero_s_Page_20.jp2
170d01c404068be2b77e1f4880066e1a
b175d32bd8677079aa78818dd931cf30258cb10c
1051982 F20101222_AABFOD romero_s_Page_42.jp2
90f68f2dad5549dce90a0d61373fc507
9c570c9a7845b7cf388fec8941200bc45023d0c2
1051961 F20101222_AABFNP romero_s_Page_21.jp2
3aa3245faa93b0f45cc5596fb3cc01b3
fbaaccbd5c06f9b2c8f4fd312e3c2a282a2c4cff
1051948 F20101222_AABFOE romero_s_Page_43.jp2
54969e6e8c0b6e7455964f093442539a
d2dc26b7476b3428e2efd497dbe3815a03941eec
1051966 F20101222_AABFNQ romero_s_Page_23.jp2
8f1d8766e6dd386aa50ade09aca6e2ed
7d4b1324b124e91c78e4160fd0e123d546cb605d
1051932 F20101222_AABFOF romero_s_Page_44.jp2
76acfbfdf274a12f163c5bf9d8874020
11b4b389f9629057723f129d06eaaec009e8f876
1051981 F20101222_AABFNR romero_s_Page_26.jp2
b3a2ecb3d777b57a289727732599b80a
6d7d1fd04fd981e666c4e2bb96426a3266fda116
1051980 F20101222_AABFOG romero_s_Page_45.jp2
85a3b3b2478123093a2281f15dfbab24
99d782567cbbbcd47aa5ab7027364ed70fc4f37a
1051926 F20101222_AABFNS romero_s_Page_27.jp2
2bbcd8cbd00b80ce751e971669d2f323
6362f09409347bb0d65314ca4718025309c54c5d
1051984 F20101222_AABFOH romero_s_Page_46.jp2
bc413da2658900691df4c869fdd00131
68f8afcb2a07044c6aaed47b5a718f244ea77c6b
1051968 F20101222_AABFNT romero_s_Page_29.jp2
f4df8b634cb8b38f0ceb9ee786bf3367
fe08c0a1b84805cb0e3556e576a74ab18e4a54f9
1051949 F20101222_AABFOI romero_s_Page_47.jp2
c68c10bdb6ca05c1928a298ecc8221bd
cd6de3733f94388ab221712ee3aeb3a5cda64564
F20101222_AABFNU romero_s_Page_30.jp2
bb72ef04b04e000d093fc95b07aae813
6e270b1f38dc50c78ab4818ae020424eac71b321
F20101222_AABFOJ romero_s_Page_48.jp2
90382783b47b5faa4380ff458c54aea5
d9b6a5c42cbdd9dfd761012289d3473241f0ffbb
F20101222_AABFNV romero_s_Page_32.jp2
93b9f53cc95cf5a9401d7f4370b9eea9
5781c69d770a95794e04b83c7d39557e291a5f46
1051963 F20101222_AABFOK romero_s_Page_49.jp2
a2d233189503a60d777b58a8b625ad33
ec8d048f4493d4ed8e2bfea72cbf39492361d07b
F20101222_AABFNW romero_s_Page_33.jp2
a67b436f912ec54429140022c1f4650c
3682cc06416a4cbcf3758dac8dd20069df33a3d8
1051956 F20101222_AABFOL romero_s_Page_50.jp2
c769066c29d2b09aff4c66bf3f35a34e
1930fe681215f2da28979750df8f00281f67942d
F20101222_AABFNX romero_s_Page_34.jp2
57ca7db131083efb8de24e46093c6a0a
74b0d691ccc931a7e5f88b7d57658b8b7b6e486e
F20101222_AABFPA romero_s_Page_68.jp2
01b2c0958da72da40f7a09fce8a80e2f
388e6b5e9d42774868cf6d4a55dbef00ced9ebe4
1051947 F20101222_AABFOM romero_s_Page_51.jp2
f2d44dcbe6770633dc51105a531d7dce
1b39b140e624feb9f62384d5a340977848de9a7f
F20101222_AABFNY romero_s_Page_35.jp2
4c71da661f915e7537d0142850840c77
6f7c12517aec4e12fb36d4c8eabaf013c643da9a
F20101222_AABFPB romero_s_Page_69.jp2
9aac6dccc195e3c23488005ec354f69d
8dabd4d47746f21263cdf1e19c0f58ecc8233450
1051967 F20101222_AABFON romero_s_Page_52.jp2
4935b5fcc8b106ff0fcbfb483a5df430
9b8f22c2a6bd555cbb9c2fc26b06bd251d959308
F20101222_AABFNZ romero_s_Page_37.jp2
f7bbafb7a0679c22f14baeb931e340c4
7e638e3b34c99d905e772ab885128eb4ebe54dc0
F20101222_AABFPC romero_s_Page_70.jp2
d2ee8bcd4184eb5738e0224903642d76
d290a74466c9e76ef2aaeae6c3dcb854f41d7dc0
F20101222_AABFOO romero_s_Page_53.jp2
9a67a681a55e83f4b5b4c35ef080c862
83c8353a2f1007aae491043d1f1210938479d896
F20101222_AABFPD romero_s_Page_71.jp2
5be1f0cb8a80c794c88916ca572af44c
46876f81d5c74b20ffa7d67a429d9647bccdb8cc
1051975 F20101222_AABFPE romero_s_Page_72.jp2
9cc6348947b12a83e304468e974cbddb
624ea64df481aa4789cd98bc6e0ee5e3890a6160
F20101222_AABFOP romero_s_Page_54.jp2
2601514edef421575da6d3ac99b31452
bccbeeda1a798157ff175068da005bb82a180ff2
F20101222_AABFPF romero_s_Page_73.jp2
8ab379020af8323d85e2734b93f24f87
5874eac87c218a83924c064ed9c1ef34a2b17b51
172596 F20101222_AABFOQ romero_s_Page_57.jp2
26f8e001767e49423c152702939b9ff0
016e34ea485e3f36900163cc6ba24cdfa722ba86
1051957 F20101222_AABFPG romero_s_Page_75.jp2
9eea06c5a154ea8cd362fee6892119ac
beb6794d3f765a39187347b99e4dc23780c0bfae
299217 F20101222_AABFOR romero_s_Page_59.jp2
059ecf77b6ab03c0ea792ae8b6fa6e59
c86b734134bc0116d4baa50a3e3ea0b76040fc83
295009 F20101222_AABFPH romero_s_Page_76.jp2
7cb734f46c8046e3d4a49cba3a169d47
2c6670438b051a792a1e7b7be7dde0d5844ab661
284873 F20101222_AABFOS romero_s_Page_60.jp2
8cb7380b1069ccdfdeae5a18b5089b71
e2502eb05f027e3ebcd30396ecb48c988a444169
1051960 F20101222_AABFPI romero_s_Page_79.jp2
367de7c0484bf81c7c081bc1230977f4
0677e5230f6c11726e489efe4c784563c6d9f240
F20101222_AABFOT romero_s_Page_61.jp2
9b5705c1d78c95dfca952cd63908eaca
6a73c8ea24945689f5b445ad529567ecad812fe4
1051979 F20101222_AABFPJ romero_s_Page_80.jp2
c8a35d08e3f56fd7793de74d6fd45749
6f18ea0c25bae743c472ba35d5f8a91e90e0d2d4
F20101222_AABFOU romero_s_Page_62.jp2
4dee9200ffd5f4a902d5dcbc751c8f4e
06a6ae2c8def2705102c72ab7a84ea4475cedda5
F20101222_AABFPK romero_s_Page_81.jp2
91b9c4f02569ebb0d6db1e71dbe3e748
034806bc0f9e5d91a5a59e387b79d91942c4ba72
1051931 F20101222_AABFOV romero_s_Page_63.jp2
11c34184e64a02907b0e2f779bc0d810
93c701a2332fd25ccc21406ba0068df872edf04c
F20101222_AABFPL romero_s_Page_82.jp2
a799564fbec888e62e4cb5d09fa56ce4
0b14a04ee3f915285d1a1bb44f1073e4cb08c014
F20101222_AABFOW romero_s_Page_64.jp2
899e6e6781ba992ef85fdf701174b1b9
58b6beb68a243880112d19973ac9f9e3f536f595
25271604 F20101222_AABFQA romero_s_Page_02.tif
52568014c3ec127055f54cc62f4afaf5
8bcb85947fa3d3637c9db1b0d7cef7f588e63b70
1051971 F20101222_AABFPM romero_s_Page_83.jp2
e4ebd421023cd478e0b568a1d8ef4fd4
ebda5677c656ee6fed17781356f8bee91f115f54
1051937 F20101222_AABFOX romero_s_Page_65.jp2
b10d182451694589970565687a9e32f9
8752bec2124dc92f85deac80db15cf91eca22dc9
F20101222_AABFQB romero_s_Page_03.tif
d842e54db8c0f772a227f4a59fe02389
26f9b80edb1fce6e90a7715b47f9f04684098df0
F20101222_AABFPN romero_s_Page_84.jp2
b17918b06bdcf513ff62eb922bd85fa2
3abd10d498d9565ed9e8d071e9ed263f5e3fc7e2
1051918 F20101222_AABFOY romero_s_Page_66.jp2
d86a6f1b3a7cd26af4176b91a2eb11bd
758024210c97276ead78734f6a92ab8d12f9d4ed
F20101222_AABFQC romero_s_Page_04.tif
b3641f34c3be776272d1b20f283e95e1
c514012677cfe96a7d6b4c0897eafd49b61a9a95
F20101222_AABFPO romero_s_Page_85.jp2
77916d51863e9189c3f8151895ebe9b9
2745c18e70691e9ab8b78c673a8934d42ce1ee53
F20101222_AABFOZ romero_s_Page_67.jp2
b5c427e2f9ed2e59670d6b55b922ba15
72078f0583397f0b76d7d5fcaf6d1dfa2a227ee4
F20101222_AABFQD romero_s_Page_05.tif
2bb328ed1deaed73861b8815e1352319
22407a83c251b20b5d23e0df21513dad05df9b8d
899898 F20101222_AABFPP romero_s_Page_87.jp2
370316e1e87de8c9158501d205da4834
d0cffafccc621f829a52bba987b89f96066a52a6
F20101222_AABFQE romero_s_Page_07.tif
7804f4abd4945e23f7585e14ae37ec32
b66573c6cedc83c96e7b9fc615b66a7e55430139
F20101222_AABFQF romero_s_Page_09.tif
470981f2cede6595454cf2ccc15d67fb
6365f83a3bafc443fef6735911dc98bc3f20f088
1051950 F20101222_AABFPQ romero_s_Page_88.jp2
b949c0765b8072c88a3417d46ecdee25
5f23107a5af8dcccc1c52498afde942dce6d6b93
F20101222_AABFQG romero_s_Page_12.tif
1b5f37e09d8c3805962aba86706db58f
a414cfc7869feaa2c088ca9001b92d3e85fc4f84
1051900 F20101222_AABFPR romero_s_Page_89.jp2
d249a17cf60ca81018fa93a942356499
117c260db262e7eb6ae83ee63652ebe99adcef86
F20101222_AABFQH romero_s_Page_13.tif
fb15cf9b37c498201f0b5883144251f0
8c367d9db8c722b1196999eaa2bb13341666ab19
F20101222_AABFPS romero_s_Page_90.jp2
a37d25290633957f81324ebf1ff1c461
660a9429e29e7f0b2fcc5b6dfcbb8d462dfbb8b9
F20101222_AABFQI romero_s_Page_14.tif
a32ff55ed4f82d85886ec64f2e627f26
02f262c09d688dfc3235e92759a0416d8a33370e
1051921 F20101222_AABFPT romero_s_Page_91.jp2
34da166daba618bb436d4347735dafc5
d84bb452ee9ad1ece189002cc8ce2899eb14dc93
F20101222_AABFQJ romero_s_Page_16.tif
b59a945651b752856ac10346a8bc8b7c
3ec1ef216479f206704211ce85c12112700a9559
1051954 F20101222_AABFPU romero_s_Page_93.jp2
339a6b9bd688f500b4f877c649a0ab1e
fa7facb8cb3d6259c452387fe0951be0faa773c5
F20101222_AABFQK romero_s_Page_17.tif
d075ac090195908f8fed420c8b8e3e33
3eb0aeb013786d322c36caa928a9b41010c02d2b
F20101222_AABFPV romero_s_Page_94.jp2
ab66005582de673bf13885555bd81342
65ec262353fdf144e7489a24964b488ca7f068d9
F20101222_AABFQL romero_s_Page_19.tif
0dcf52a515f50d22f5607a652b768434
2df693e53df8b7e8b85553c0d8e06912f748d1a2
F20101222_AABFPW romero_s_Page_95.jp2
ef17e6bdfedcbd8359c3c3fe3b6f30f8
49b1b84de40e51cc26212959c7c274287138b83e
F20101222_AABFQM romero_s_Page_20.tif
5715f3159236598742529e18295373c2
1c4c3a0dfc1121d0720abb2dca8ea36dc80f5358
F20101222_AABFPX romero_s_Page_96.jp2
46160e7e4b08d1daaa661d0281f89e2d
469074d3058d6d1c24773e2edf7414402ce2b11b
F20101222_AABFRA romero_s_Page_40.tif
de666a00fe277ac7eed3d0b81cb833cf
0ee58e483125643408c7149058647a607a222f54
677867 F20101222_AABFPY romero_s_Page_97.jp2
41680e2b50e72286223a414dd628ba99
9520dd19547c84e3afd627d0432548b30678ae98
F20101222_AABFRB romero_s_Page_41.tif
f873ffff8cc93b37fc3f7a3ddaef93f4
a8dc3a07bf5109118f885d2d12de85051a016765
F20101222_AABFQN romero_s_Page_22.tif
3593e7b502d711a8a454b4fa99418ca4
46c2df00bd7aa51f56ec19dd48b3c9f6f211cf29
769707 F20101222_AABFPZ romero_s_Page_98.jp2
2bae5807d62eeee22197b62bbc6e8db5
2f166dffa99da29d12a3a0dba40b8d45322257c4
F20101222_AABFRC romero_s_Page_42.tif
af468a7878b195b63b6b11a1f86fafff
7df285bb54c2fd97c279e1d5623c4e8a1940da37
F20101222_AABFQO romero_s_Page_23.tif
f4b89c377be4c2afb78e4c2b858be2c7
83606b193a0d1ccab1ca4996cd9d723b08fefd72
F20101222_AABFRD romero_s_Page_43.tif
e858c9861efa8c7d9397845c0e0b1cd3
ee7fcbcc6c2a96356c3efd2688f6554e94946bec
F20101222_AABFQP romero_s_Page_25.tif
ec58cecd3d93c3258e26844de6d2bb36
a108df17e867416f66854f65f5a5cbe8cc06b331
F20101222_AABFRE romero_s_Page_44.tif
e8eb0583aade9767e4cec7dc079ea477
299c88ce0e92b6772fa8ee487e52e621afef9fbe
F20101222_AABFQQ romero_s_Page_26.tif
0888531ac6228cad13da8e8718ba3235
e5cdeb07350136d0b6cc246e4c84051f98235273
F20101222_AABFRF romero_s_Page_45.tif
896939102481a348071aa3206e01d6d5
eaf256a806d9cb11bb59e756cf66d9d764299c73
F20101222_AABFRG romero_s_Page_46.tif
79e76fcd7f375ad8a2d06d61b365de1f
e6a9f9e103e0dcb98708840388e409640fe763d5
F20101222_AABFQR romero_s_Page_28.tif
5537a625963df769e515cfd4c788371b
cd613e121b97f7f180b8de4b3b9f43902cf0c74b
F20101222_AABFRH romero_s_Page_47.tif
8e46b3f64551496b9f3f1a72572917ac
bef54564e100952cfd7c0846f18adfb994fea316
F20101222_AABFQS romero_s_Page_31.tif
4eb0290f49f2262a03ba2d278839f928
51321115f64cdc79821b327e6cc7044d9fa776a9
F20101222_AABFRI romero_s_Page_48.tif
5ccaccf5bcacfee501d1a1f7a5b98212
c028c960b927e1623d1dd6c29245decd4915a9d5
F20101222_AABFQT romero_s_Page_32.tif
adebd052aadb996cd231006a59924a7b
18fc0023d6e4b0a41abc502b3181fc414049a071
F20101222_AABFRJ romero_s_Page_50.tif
108c7fbc69157e62831e606e273218b8
b0ee43b06f28671adac1dc8a47b98ae7ba49fc99
F20101222_AABFQU romero_s_Page_33.tif
66954d4fd5cf17dea647431fbc40ab44
7c87656246abadfcea1e139869693f7d6179b8d6
F20101222_AABFRK romero_s_Page_52.tif
ce96c65dbc9f51bd5fe5e2dd1e3ac37a
f44b3b7c78c56352609d5806167ef7bb93ce7926
F20101222_AABFQV romero_s_Page_34.tif
9d2aebf965b1485277b8e4045fdb82a1
7ff49b77b3ac64a3f4d15b96c2bd136082d3ca67
F20101222_AABFRL romero_s_Page_53.tif
210cd3e4c9bd17ccaced3feb52e2ff0a
31217d870c85bc6c3d1dedff1a25237bfd29e447
F20101222_AABFQW romero_s_Page_35.tif
7a1e2e94c8950e61f3ff9821b94debad
aa5368e718230cf0ba46da0f82eb6cae34820a37
F20101222_AABFSA romero_s_Page_70.tif
be4c50f99a1413f63c4c34b9713c9296
c07b40b84f1e08ec0a3aec184dda2403cfdf9070
F20101222_AABFRM romero_s_Page_54.tif
2ec6f8aa75e9cd294e0171bfbae70e3e
b052f9fe0ed2b250e59b47f64b8285859871f0a2
F20101222_AABFQX romero_s_Page_36.tif
9deea72f716bbfc8282f051e4e057a3c
1b7478b839932099314775ae4c105ff2743e3298
F20101222_AABFSB romero_s_Page_72.tif
58cc8b14a99ade05636f9aac5511c953
05721528acac0213ec2e6abc537c0e31547af79a
F20101222_AABFRN romero_s_Page_56.tif
bd9e8cef578caa340bf151df6e83a2ef
a9e1eb6b5d47fb23ef7ec27ae3d8f85f67cb3eeb
F20101222_AABFQY romero_s_Page_38.tif
c55cf3bfe0217a9e296bda6b1504a887
04240cf37c427f22c8069bedaf232a0e07aab594
F20101222_AABFSC romero_s_Page_73.tif
2f39b1847472218b75bfbac4e68f0483
2e26d70421bf93da59127652eaf5e1286414f66a
F20101222_AABFRO romero_s_Page_57.tif
f0faa90e05c8a63e1af93b40caef61fc
6d6c3cfe6dcf63af11c4a2526b4e0add6f92b21c
F20101222_AABFQZ romero_s_Page_39.tif
6acbf026950cea9f13703b4a71ecb5e7
db22f2b86bd527bb0b1ad18e87c1aa2d483eb36e
F20101222_AABFSD romero_s_Page_74.tif
6b6722c8de1fce53963e634e275676b3
48060add0f6a11d59881b1addbb129624a3b770c
F20101222_AABFRP romero_s_Page_58.tif
3bacf473e02d7b4ce0ead18667a0a7ad
0ed82eff950bca01c344ff4c692615265510c1a1
F20101222_AABFSE romero_s_Page_75.tif
d428ccf36d7fe934917b2700a03c6ab5
f9f033517237e63090ca9f81fc8022d5f5d32a0e
F20101222_AABFRQ romero_s_Page_60.tif
8e84b0b9ef5600af0c9d72437cc21641
c4adce58ce69d3b8b049bd3710cc572a3b7a0213
F20101222_AABFSF romero_s_Page_76.tif
f10d14ff5ad2a718433d1465d6aad20e
25fcc2f24510c60a5357dba0f03f2665154f2887
F20101222_AABFRR romero_s_Page_61.tif
9459be0739258c5dfde825bcc6a23624
4cc8348f38a1cde97da0e22b2874e3ecc6de1f77
F20101222_AABFSG romero_s_Page_77.tif
c77576d8ca10c7eb1819b1b05e071e96
07d5d5652789b7e7aee609b0d8fc2ba692eafaef
F20101222_AABFSH romero_s_Page_80.tif
090ed3290f085bd699e30a34e093bbb3
9770d4f94b87d56fbe527e9b22c5c1d697db725f
F20101222_AABFRS romero_s_Page_62.tif
f6c6fb61187adec4066ce82c800fc1fe
142af264a97e248ae88cb79303c11219351f7b00
F20101222_AABFSI romero_s_Page_81.tif
ef1fea2af1783e33b99fbf1f6effa655
0a360658a858d928461cb47cc59b1606d5c35c53
F20101222_AABFRT romero_s_Page_63.tif
65fa9c6442267925b6b58c7a4049e64d
c6c1732a90a9396ed78a1aa8dcc5e301bcc3b5da
F20101222_AABFSJ romero_s_Page_82.tif
5d9f99beecb1cf61ceb103357c3b1e30
c41d306fb39424ad64636864e60d2a0ec6871513
F20101222_AABFRU romero_s_Page_64.tif
7e71eba5f1c1c3bbbda52119f079188e
da367dddcd7fa764c85ddcffc7b5df0ff792a8e2
F20101222_AABFSK romero_s_Page_83.tif
763c627a1b2f7cb0978b6a6ede2a381c
329b980444e4b5f9012a9e2d69f905e82e9ec994
F20101222_AABFRV romero_s_Page_65.tif
50bddad39dc04144e4fc3a41e338f4fb
173a3d1f81b179990f554a0d32b2ef32ff0be91c
F20101222_AABFSL romero_s_Page_84.tif
cae962dbe5b73d78a0c5eb890d7bcf21
0ec1e1a34a0894c24c7a0b1415ce23e031ba76fd
F20101222_AABFRW romero_s_Page_66.tif
f332591e76d04c5f4ff50469a296a382
eab9a6eb9865c95f0b045caa3448a6bd0e9e64d6
F20101222_AABFSM romero_s_Page_85.tif
31a3559fc3cecffd8a8fa37c094e7862
a5ccc3e08191da41d36fbe69780076b0daf43ecc
F20101222_AABFRX romero_s_Page_67.tif
edbbe832132b0c807544e5f34585576a
240d35661611d780967034973aaaf42a5255f323
23925 F20101222_AABFTA romero_s_Page_07.pro
2ce6d9a7aac5b8e10e7a42375f7b9a8d
73485fb165b41e35b601f8c4472d815980fb529b
F20101222_AABFSN romero_s_Page_86.tif
ab1c0daf97674e8f4c271d220d9e7757
0834b36a30eed2a40f4bf0c2b71e11a7241c86ca
F20101222_AABFRY romero_s_Page_68.tif
63f3696fdc5e6800dd841b0c7486fe3d
29d25cf513eeb1b7e81cb0805b353156dd864c2a
42007 F20101222_AABFTB romero_s_Page_08.pro
fdee562e24d3dc8d5f065852c6384377
95dc69fe6187184d2db867f45da98ae35640562b
F20101222_AABFSO romero_s_Page_87.tif
cc37430fa48b8d37ef5d13974c33c4f1
ced796a27de773626d9d1073ac23ea5a3cdb5020
F20101222_AABFRZ romero_s_Page_69.tif
4fbd122acee55f8263202b274ef50caf
3dce8673965b446eade8c054b6f7ce910ad150ff
30089 F20101222_AABFTC romero_s_Page_09.pro
f2388e1dde7e5c5f40a24488a1be69e4
bd774a53838358b344c131a0096ac1147dda1ade
F20101222_AABFSP romero_s_Page_88.tif
a90a17f416016383d62d4bcbd35930e1
4a845caa12f3df37603df396f150758ef7444ddb
51164 F20101222_AABFTD romero_s_Page_10.pro
2107ab3c84f06972038e0574be41512e
95ffa745378d32c906f9702524217f8c8b60f847
F20101222_AABFSQ romero_s_Page_89.tif
ed195de95aa502e1471c8e4eff1b1f27
2f7ad401095a87003d5861266a91d0230aa6d8a8
52863 F20101222_AABFTE romero_s_Page_11.pro
f911fe4c2163af94a3078a751589961b
f51123b3161def3ef6339461a50854d505517fc3
F20101222_AABFSR romero_s_Page_90.tif
0f06569fc506358a26f6e8aea89de052
a6bb401cf31dd27f44a7506d8a8365f9fbed36b6
55958 F20101222_AABFTF romero_s_Page_14.pro
f64d6f25dd081f82eedcadf75bc91cd4
f249f3b1a0c445b4388094e7a43d7460bece1410
F20101222_AABFSS romero_s_Page_91.tif
08bb28e977137cfac0f3e3b0dfb2c9ab
79e18612b6151551369fd15a38d47a5317b3c8d2
51959 F20101222_AABFTG romero_s_Page_15.pro
94788bc1cfd2097a7ec4f79dc9d1923f
5c2dba38f6e9fc9f01eb7eafe6c9e316b08faf3a
56142 F20101222_AABFTH romero_s_Page_17.pro
7af058423c89da0e32b302c7f89c0920
54b0adbf88f9b664998576c437493ce910be0ccb
F20101222_AABFST romero_s_Page_93.tif
6c0e76efa2a33c7c80fea188181bc9fc
6a1f0a13af65f10863692dfb86ef34dddf8783d0
55226 F20101222_AABFTI romero_s_Page_18.pro
9116126d6f13d0556d3a7cd404f7b136
774c80584a7e1279b2e68bcfc0f26fbd5671a75b
F20101222_AABFSU romero_s_Page_94.tif
ca9829fdb4e652721c54c9bab9053024
5242cf28b5b5d8e85d81c9a164de9c1fa917af48
50968 F20101222_AABFTJ romero_s_Page_19.pro
5d5e42a504dc67c6b53c8e5af7098181
50c238ff54c6d404f8ef6ce7194b6c72189af135
F20101222_AABFSV romero_s_Page_95.tif
555ffff1f67a85356427c836b8d71277
ebeab6578ce476ced26c9b166155588811ea2cf3
53399 F20101222_AABFTK romero_s_Page_21.pro
a9c523314d1ac3cba439c87da34b8cb8
a2442582102de0ff99a64f2738cf31e38b609f12
F20101222_AABFSW romero_s_Page_96.tif
96bd44333fa72035c45b101fcaa71481
914781f869d2fcb931f47a9346af517160a4a864
53288 F20101222_AABFTL romero_s_Page_22.pro
7c052d35ea6d1bc4df24e23688062517
84e59fc068f82fc5018979f671feef35a190c934
F20101222_AABFSX romero_s_Page_98.tif
98867dad482181ebe15973f358054505
7cd9f0772b64bb3507eae6f7a79f4b5c23d27672
55888 F20101222_AABFUA romero_s_Page_45.pro
97be0e7e098476923fbf26d7568d9239
b7e63b23ff13c9ffe9ae99d4ade03b50c984cbee
55065 F20101222_AABFTM romero_s_Page_23.pro
5c6e172d73859899390d0a7cac32b68d
7b8ba7149f3456b7d5cd7caf9599fdcea4076a74
803 F20101222_AABFSY romero_s_Page_02.pro
595056df9767c82aa1eab6c4e39de57b
afd1191fbb7dd1009a0cda8e58c3b66447894d72
51803 F20101222_AABFUB romero_s_Page_46.pro
ac41b95823c33f4ffd6e04f9b4d08c8b
7cdaf717828577d9a2822e129d0de99b8e650081
48108 F20101222_AABFTN romero_s_Page_25.pro
7e4442d605954acb0ba7998609e6bd30
8c84c615b92172d01b1a822108541195d9f5680a
49360 F20101222_AABFSZ romero_s_Page_05.pro
ab818447c9b83d5c04e3b5ded8ac61c2
04e22c08e34d47a5da190224a0c53171e47c4683
53958 F20101222_AABFUC romero_s_Page_47.pro
33b1ebfdd1dc9f248e3cb283268c01b3
6720500b531c793a4d8c781c427eea59e0574a35
53173 F20101222_AABFTO romero_s_Page_26.pro
daa22af1a7312f76ca8e1fee0f9b49f5
fd3cf55dc175aeb318a44d7ec0fb5bb042e987d2
49870 F20101222_AABFUD romero_s_Page_48.pro
ecbc8a28e274e991a72bf0a1271ab29a
51a0b24263f4aaeddb71d71eee2452a1b28b1e96
53985 F20101222_AABFTP romero_s_Page_27.pro
97674bd7d3bafef4ae7c92713e950ebe
db5085ac75908afac2e59453a1f7f61ec83eca50
51260 F20101222_AABFUE romero_s_Page_49.pro
c90bce2b9515fb44490aa8c207444cfa
9f8a9c0d4162bf35025f16cb4967b8ffb66d3d84
52641 F20101222_AABFTQ romero_s_Page_29.pro
378a167494a3c95560f2395275af6f6d
ff32cf39dd9488dc33327b22b9b5fbb50b8cc082
51503 F20101222_AABFUF romero_s_Page_51.pro
da28b2584d0d7eaffe9fe741952caefd
85e27005b57c99b76fb950ff336cf75889b2dbec
54642 F20101222_AABFTR romero_s_Page_30.pro
079ae46aca8dfefa13358a840b75c024
f7fd517033cfd8b1f819ec5187abbbb3c42277a6
52523 F20101222_AABFUG romero_s_Page_53.pro
3a1f4aba013469f8be526da27727edd2
e7f61ae51e6971d97129d5363c9f8a424f778fee
53981 F20101222_AABFTS romero_s_Page_31.pro
a5c417a8d7826aa3dcc1b7b3430975fa
e024173124db93d7264ac8800d7aa2ffc4b94d5d
12411 F20101222_AABGAA romero_s_Page_06.QC.jpg
c49d8449f35be44ddded01a8ae93226c
277b9ec28058ab93927c071db103c25959f4fbd0
56011 F20101222_AABFUH romero_s_Page_55.pro
83cafc816904682edeb47dc0b811447e
64568043ffa456fee0fe80befa5541ed62be8a97
56331 F20101222_AABFTT romero_s_Page_32.pro
9aa96adad5273dab8e860c698dda3aae
0317481ace9dba6f6d3c220a3cb5096fe49314c6
29016 F20101222_AABGAB romero_s_Page_85.QC.jpg
3b748540ba6e345de89e8b793f1d146e
649793fe3ee775efaeb32f6ffd11f718cfb232b6
47803 F20101222_AABFUI romero_s_Page_56.pro
591fb62d66e239be8793da69b70dfa1e
fe11835976aef6bc35ee72f681d9f35a59ef6215
8794 F20101222_AABGAC romero_s_Page_16thm.jpg
6762045e3987957323393bf16b31613d
668226c8f3b80937bf6858f173fe629993182934
4381 F20101222_AABFUJ romero_s_Page_57.pro
706aff914db7bea1851e4768d0f6fa9e
2ff462b0ef1bd9bd735c7f3965b58c0d313f12ff
53421 F20101222_AABFTU romero_s_Page_36.pro
f4a94cb4fb40bef25bc7dcdaf609d8bd
ad3d28ebff5ad05efc7f98b926838e2692c6c0ba
35868 F20101222_AABGAD romero_s_Page_55.QC.jpg
12800e3589857c3e13a6f6a59d8fb2f2
2feeba258b308ec5426e9d4e19e7b5a8e3472aad
12850 F20101222_AABFUK romero_s_Page_59.pro
6a112b34f79f22c7cf4da383d360fc91
32ba78df92a0a25281bac07366fb6598a75f35c8
54366 F20101222_AABFTV romero_s_Page_38.pro
80e0ffe8197d01f86fae0093b09794f9
ebf8f8e0f1e7d30cdab1ec4689b985920d24ab0a
8554 F20101222_AABGAE romero_s_Page_14thm.jpg
9c56b06df4ee9703f7eecba6e64c0780
1c567ab996c3da309021ea4f55e74e07ed5bfbae
51333 F20101222_AABFUL romero_s_Page_61.pro
6871031f599ddea5da70e12d4f9b8174
b79ccb2bd480720cf4ffc197ed814bd87f5d5e39
33211 F20101222_AABFTW romero_s_Page_39.pro
734dc2fe5f243d739e17848560e79bff
5fc3675670feefa1f7c4ad7e388d88c66e01c0b0
35373 F20101222_AABGAF romero_s_Page_53.QC.jpg
7eb83578a61be53de4d4bf343bd51ca8
2ca7bebbb96834edd010f596b0147957efc305d8
14545 F20101222_AABFVA romero_s_Page_78.pro
d24afb248554379ce49620cd6c0a01bf
d083d38d5bd5286b2dee742c4b10bde6f2307782
52854 F20101222_AABFUM romero_s_Page_63.pro
94b235a728625a0070e6e77c8168f602
896d316b7ef76808f74285690fb225c49da78584
5149 F20101222_AABFTX romero_s_Page_40.pro
2077e964747d5c7866b0be4daec658b2
5a4d1ea3950a7f2f241d59bef3a2a6fd2588d67b
8779 F20101222_AABGAG romero_s_Page_38thm.jpg
519b1256dd6da41ca103b89d3331d0f5
64cea44e11b3c3a8aeb6283315e1c6f75d7b4121
55865 F20101222_AABFVB romero_s_Page_79.pro
49606b17e468f40abf482a56bae56443
c6295f887f336a60df7e37f4b068fcf2c11a8748
49847 F20101222_AABFUN romero_s_Page_64.pro
8f08a3976666690c8163f0885f0e5115
6e4aff2e5dfe47bd064151f800e3abf41dfa590a
50471 F20101222_AABFTY romero_s_Page_41.pro
4db47d0fd325e92d2ace5591d4c72427
5eda50099aa5c73d298aadada9e75b52e96f9d9d
36651 F20101222_AABGAH romero_s_Page_74.QC.jpg
f6e84e785028b0efc22a6d5088741d87
c201cd0b35576c854723cf78e9c8185b33386fde
54499 F20101222_AABFVC romero_s_Page_80.pro
55ddae641833beb1d4cc4df1904c670b
0eb04fe0e29dd569c89f5b52a228e4a3e84545b5
55660 F20101222_AABFUO romero_s_Page_65.pro
f1b77d10b81230dc5fcb3cd43ee1d8f5
d1885882a1f4e95eb6baa1c5fa8dac3e839e459d
51114 F20101222_AABFTZ romero_s_Page_44.pro
7cc12bdf0d3d817b0115a637edef18fd
35e2f52bf30b5e0a7206b37b5df677e7bea5b175
6642 F20101222_AABGAI romero_s_Page_87thm.jpg
0686648d67879a805a9c7ee65831ded6
a16e2280386ec7ffcd8f696824757f409819b0de
54523 F20101222_AABFVD romero_s_Page_81.pro
a64c1f93b37a062935136f5c60a6fe2a
c9b177fe55417ff289b74026a443cbeeede184b7
52021 F20101222_AABFUP romero_s_Page_66.pro
91055cd11b81a92a234f5e9f160d056f
05c1bdf66e9b792a457d433e5f942b3fffddecbe
35100 F20101222_AABGAJ romero_s_Page_20.QC.jpg
5dd6345887fa01823857ba7b4b7fe929
4085b87ddd5a8e45d20ca78438da93a31d20c628
54854 F20101222_AABFVE romero_s_Page_83.pro
83e06eb56eb65eb7b9c803ed11ea5461
f3270ca6aefa6aa8574b053b84cd3c29ab5704b5
55536 F20101222_AABFUQ romero_s_Page_67.pro
fa0aab91965a8bf01d99145cd9995ac2
d9cf745d66cfa7bffed75c597b29ada5f9a442ab
8328 F20101222_AABGAK romero_s_Page_11thm.jpg
38afabebfbad6c78237ca39024d3948a
61b4a67e7e2ce688c5446b922dbd40c9e75362c4
49182 F20101222_AABFVF romero_s_Page_84.pro
eb99df28f9acb4c7097601d0bbb6c622
440479acb6bbfafadf5589a86ea14b85859520d0
53379 F20101222_AABFUR romero_s_Page_69.pro
fa525d6204c8235977ac44fc295267e2
574a5068bb122bf29d560f27e65837dd368f6f56
6910 F20101222_AABGBA romero_s_Page_04thm.jpg
ffb1405b3d19ab02e58914aa8cc02a30
a81ca0a87fed1cb6a626ac5ec1ee63557095c227
4732 F20101222_AABGAL romero_s_Page_97thm.jpg
aaa54a91b06bb38c17a01ba10c64eaf5
13aaddb6f387a9198e2302290e58e094e5ffa925
44588 F20101222_AABFVG romero_s_Page_85.pro
4b4f29063e0554794f9a3d3c2c087ba9
85653e42a6f14669c2ae39906fb6a5e6222e1cb4
56411 F20101222_AABFUS romero_s_Page_70.pro
d88f7aa3e55c044453dc40439ef2aae1
9a0066e7146088f88d7990752475948ae25b46bd
8482 F20101222_AABGAM romero_s_Page_23thm.jpg
4586765e4e414b440447c0c91c9b3c2f
7795e1d52513ec92df279fbbf7dd2762d311435e
25117 F20101222_AABFVH romero_s_Page_86.pro
06a2b0afe70ec0b2f4b68470c8a5f65c
d79a5243a1788baffe1bd7c89a36410eabe475e7
F20101222_AABFUT romero_s_Page_71.pro
680a327acb688bbd04e6b4ef0fe9b4ba
d0f8ff6fd39c95bc10f78bed2ada4998f23cdb39
8540 F20101222_AABGBB romero_s_Page_50thm.jpg
3b58906ebd55f53b964c67871aa63051
346bba6b1553742b2aef508995b33a4303a273ad
8412 F20101222_AABGAN romero_s_Page_88thm.jpg
b22c99a3e069b8fd4419f10aa2d1b4e0
e6391358ccaaaa3b591e65f9b74842636df4d836
37603 F20101222_AABFVI romero_s_Page_87.pro
6536e3e2cdbeca72da6249f3b07a4e43
4b33d5db6baf6c4aecba8866e9fc2d5e79925247
53669 F20101222_AABFUU romero_s_Page_72.pro
25ab2290ef61f90a84b99b45433daffc
83adc2941dc18381b1e262af1eea10060a26b88a
908 F20101222_AABGBC romero_s_Page_03.QC.jpg
ebfb2fa831a9ddcfb78e81def7f488c3
c9e8aed113548e0a402846da793c8d768f7bfdd0
30192 F20101222_AABGAO romero_s_Page_04.QC.jpg
abae60130cb8373e03fbf594310cdecc
af05fff6ad9aa333e04c04ff2f9e60e5c92859b7
55747 F20101222_AABFVJ romero_s_Page_88.pro
d874f6cd59aa7d809b070a237e295de9
618875303b8fe453fe6a2b992c0426caf3399d3a
34496 F20101222_AABGBD romero_s_Page_69.QC.jpg
9982979aa31727a25eb931d77ad122ec
0f24e7f77b1d8a2a06ae30400e1482ca10fc6e87
8518 F20101222_AABGAP romero_s_Page_72thm.jpg
7313c34996ae9828583cd567af39e319
6edcad58e511707dc9ce6d12b7a0fde618c7b341
52573 F20101222_AABFUV romero_s_Page_73.pro
35ae6a80462ce0d01c09d8d22a9552fd
cbc014ff8099444327eb04c48ae1ab1269caf86f
33385 F20101222_AABGBE romero_s_Page_48.QC.jpg
855087c8815fbfb753c530f9a40f928a
cedfa5fb28241814202b59d77e6a1882223b9082
3789 F20101222_AABGAQ romero_s_Page_58thm.jpg
fd66c2bbd238c3cae8f98248996fc85f
c5ce768e5d096c64d2241a8cb5569653b38eedb3
65754 F20101222_AABFVK romero_s_Page_89.pro
bed45302d6d0b335edfa2ca946a55325
164f6f5dfd51a4073645d371b6c7daadde89596e
56943 F20101222_AABFUW romero_s_Page_74.pro
6c3263279a9069021f237f0e687a5d53
1fb1b838943d8d3e3ca17704d4168871b1cc5add
8675 F20101222_AABGBF romero_s_Page_21thm.jpg
c80e40846f68a82ceaab51da00528910
0a25d3f53ef40e045210145901efc8479c4e3e45
34076 F20101222_AABGAR romero_s_Page_63.QC.jpg
4983562434260ad0406865972a2e218d
7a6f9a43489e2426e289a9bf63d489a91bc53efc
66445 F20101222_AABFVL romero_s_Page_90.pro
3247a035a977844f594ef94edcf3d2b5
dee9c04122fef8030f91af1b39073a0422d3892c
56146 F20101222_AABFUX romero_s_Page_75.pro
6d40e1eb314e216a98b8ccae28b9633c
fadcc6fc26fa5c0f253d781dd0062ae2facd1f46
8848 F20101222_AABGBG romero_s_Page_43thm.jpg
d4f44c482f09ec0858e29312550935a9
b6f060b6e4e0916c5be6983e70bb581bbbe1d460
2123 F20101222_AABFWA romero_s_Page_10.txt
bbbcc47648e8536b18fcf3b5a559dcba
1e91270c8e256b6261e6ba32cbdca5f9559891db
34492 F20101222_AABGAS romero_s_Page_16.QC.jpg
c07202de4dd7f939cd16c065f0bca2fe
9e47e472a4f297a7d08f8f8297cc66f9b6cb0857
57648 F20101222_AABFVM romero_s_Page_91.pro
a9af59710e91e8f0b8e441e9fb93d07a
2952c3537e973813b7354059b9f912a329ef1912
13200 F20101222_AABFUY romero_s_Page_76.pro
48c5740263764001481882b2d2c5a797
6b519ffd4ef6d80795014840230b8b8c368d18f6
36276 F20101222_AABGBH romero_s_Page_43.QC.jpg
93ad73e36f7ac23df98b6bc75da1f5a6
d85a2ecdc91a830ec9337edb629ec7a64111a8ba
2085 F20101222_AABFWB romero_s_Page_11.txt
5367ca29341ec43c4622665c01ae4b48
6bd5e2d52b0171033b824bad63ec3a42d45972aa
33399 F20101222_AABGAT romero_s_Page_15.QC.jpg
98a5f7173365a53fae220a3ebf311f24
62ef55c25789cd2116693389f8ffa43fbd34a575
67165 F20101222_AABFVN romero_s_Page_92.pro
c6890b73d24d4f9f763f951afb2504e9
a79c8bdb8bc6c72a548497c34fe93edb3c4e0efe
29174 F20101222_AABFUZ romero_s_Page_77.pro
a293366492e71fa76df1d22c100a4488
27b79f246cb84585ca61298ddf06b4e1eece283d
9001 F20101222_AABGBI romero_s_Page_40.QC.jpg
83b30100900d75f341b0e8a9d81cfc0c
2d817ee400bb502ce557e0fa8d69fc131f2262df
2213 F20101222_AABFWC romero_s_Page_12.txt
77c208f8f1fd6117a1ede228cd115884
e22d30d3c0cd5cfb9451895c4efa2c27d0beb2f7
9007 F20101222_AABGAU romero_s_Page_53thm.jpg
66540d9c3ffcde85e04a5b56e5361197
136701702f50fc47f48a84ad2b7d0159c8e84746
62482 F20101222_AABFVO romero_s_Page_93.pro
c120d69fc1898eaeed6c8c84bbc30836
1dbc27f32fc111f556ec882a79f4ab3760d92348
8776 F20101222_AABGBJ romero_s_Page_28thm.jpg
f814b73c58efab5036178604ebdeab0c
d858d25ff9794a71d58ffc3e160cb622de4d901b
2064 F20101222_AABFWD romero_s_Page_13.txt
0c5b2889917ec6dfa5fbc48f96821f24
de398afbd38c85e80b9ea7a39e8417094943f27c
33645 F20101222_AABGAV romero_s_Page_96.QC.jpg
205cae398f32068a246efbf9aa816fe7
f68facfc8d21e8ff8fb5dbbfcf3d57aa96e9dca1
62319 F20101222_AABFVP romero_s_Page_94.pro
d6856ec12d8a2dba05018e1e3d6ced79
6610d573b891960d5d68d3dc481d20b41baa59f2
35167 F20101222_AABGBK romero_s_Page_54.QC.jpg
b490c5d7fc9b7b8aa9dff719cd08e6e2
d6a24cf1e0456ae5fc17b6dd63964ec2b43c1653
2240 F20101222_AABFWE romero_s_Page_14.txt
75148c0291fb90337c9ae4ab5b62fbaf
69f680ae1feac21caede78520ca867ae4716eb3a
32052 F20101222_AABGAW romero_s_Page_84.QC.jpg
7dafb5c69e5211bcf4e1b1613a059b65
f675ae25094a1419e13c565f276dcda12c3d7d0e
56251 F20101222_AABFVQ romero_s_Page_96.pro
94296537f8e773efe6e86578f1d6f398
3b5f9bd2519ff99f6708c8346cc046b85890e537
3257 F20101222_AABGCA romero_s_Page_06thm.jpg
8a1d1331e063210b53b9ce9e7dd6326b
ef7605e9d9bd997788ad234e3243b6e557910c48
25956 F20101222_AABGBL romero_s_Page_87.QC.jpg
08d36edd494130ef99547b8cf8998cd0
54350369646026821976a84222ce417d2dca9deb
2173 F20101222_AABFWF romero_s_Page_16.txt
646838d62d38704773da5bba33db3c87
ea74edde789d970de5053b6e391a13ecfe20adb5
36439 F20101222_AABGAX romero_s_Page_18.QC.jpg
681121ada853e8465af01d78ab938f3d
e95d19a67fa52c4ed3dae919d554fe8147c1224f
34316 F20101222_AABFVR romero_s_Page_98.pro
6dede1f7a08159fc709052fc78d3a845
48da4ec48d87bb849b4e2534158a4d368351eef3
8810 F20101222_AABGCB romero_s_Page_75thm.jpg
579d9b63440e2b580957bb8dfc7b038c
a675834e6f719c6ce804a6bfc58ca218fc912273
35372 F20101222_AABGBM romero_s_Page_71.QC.jpg
6ef5d4189f13889858cd6deb245381d1
758f5c74aaeb7ea79c031825b6c10a65488e3176
2205 F20101222_AABFWG romero_s_Page_17.txt
21e633269c6d5c574fd718f8d6b5dae2
d2d1ac8ee8cf8289f948ffa44e83b7428a4142ee
36641 F20101222_AABGAY romero_s_Page_93.QC.jpg
49d77d4ee3a26c6acba5c55a94ba70ac
1f8bef239eb073ea249064923d10edb4ac89bc3a
492 F20101222_AABFVS romero_s_Page_01.txt
72816d04f0e07995de43ab5659b1fa8d
598a97f279611ab200a0f7b6d0541923770e29b1
9116 F20101222_AABGBN romero_s_Page_45thm.jpg
b59110613ebd99f38ed900bd28cbf703
adf23bcc1613db37d48ad128eb1262abd44d6b89
2039 F20101222_AABFWH romero_s_Page_19.txt
c0778c0f55ffe30ce6dffc8ebaef0ed8
8ab7a57c53e34d7cc5dec3b9c4fe1de0f9566504
2689 F20101222_AABGAZ romero_s_Page_78thm.jpg
0a0f3311c3dbf5fcbbb442ceb243ff60
6a17a8efc933234d3538b5b36c1da60bd2601809
92 F20101222_AABFVT romero_s_Page_03.txt
51d0c7138ccaed3867c0150b539b9252
8eb9d29402813da571251e95e224e01e20ce0c24
9299 F20101222_AABGCC romero_s_Page_90thm.jpg
504f901d6236330381aff28173832e2e
3343f9b0a811cf966d3ff503995c71c17c908e76
33000 F20101222_AABGBO romero_s_Page_88.QC.jpg
acb1adeb034d65687e0e999942d0a320
62cc69e9c533edd5857e64785c777314f7db2ef0
2088 F20101222_AABFWI romero_s_Page_20.txt
5479fbc115d99ecad2809e44373c5cc2
920f3ebb4dc1fefe120c116842955304fbe83bd6
3433 F20101222_AABFVU romero_s_Page_04.txt
9f3e725e8613dc8b7f637f9f5f02285c
d2d0b02afbab1c3b32f37e67882e367c8047c4f5
35092 F20101222_AABGCD romero_s_Page_94.QC.jpg
114c308f228711e5fad81c8e24b95512
f84d59410f11511715befcf2dd89816e9c21f8db
7920 F20101222_AABGBP romero_s_Page_56thm.jpg
71a2a4ac71247125b95b9d89df55dece
7c818220c2cffcba1b09673705cccd9d7399e761
2115 F20101222_AABFWJ romero_s_Page_21.txt
190588d9f71cfdda141517ada7f89db7
b10cfe562570aa889d7f281d1dd7ef93344cac23
2065 F20101222_AABFVV romero_s_Page_05.txt
a42f6974797b686952498a8c0b9f2341
11bf7624a8fb152a2e2dc4e39813bac17dbc4f7d
35865 F20101222_AABGCE romero_s_Page_47.QC.jpg
52c583f98884b3d756016b63b9a38ce5
3de3511f71e6d72525c83a0a9b2cefb52921fdae
8471 F20101222_AABGBQ romero_s_Page_51thm.jpg
c6048ca2ef1734f0cc37987294d67d0b
172bd25fd3766d99691563b5f9f615de31521606
2102 F20101222_AABFWK romero_s_Page_22.txt
22692f6cdb93ac06cd611f87fe9633cd
316a49d2a7ff7173b58750dcb7d9760b9834c31e
33014 F20101222_AABGCF romero_s_Page_91.QC.jpg
b22448dc9788ac70d699d60ca2461d11
04db43492b471d73ce40ef5a305154aef3095699
8818 F20101222_AABGBR romero_s_Page_34thm.jpg
06c503d90b35a8ef63cff0e97f25bb17
558f5ef5e5a7358eb3dcc118c78a4443b298d50a
2163 F20101222_AABFWL romero_s_Page_23.txt
da5dbb675b237df40d839c74689e080d
9a9c4b9e018136c88a1278f2499dd2d487ef05a6
847 F20101222_AABFVW romero_s_Page_06.txt
ef130f1d2e4bafc73732e1963812af20
d70a387cc476b4d61d0763dff1ba620b4c682011
9089 F20101222_AABGCG romero_s_Page_89thm.jpg
854e133ec4895fa020c7d7108b72b74d
7824100e7ee602e918ae88608535cebcf5053741
2242 F20101222_AABFXA romero_s_Page_42.txt
026f0165a6a6233df4be312e2c8d3120
54794df67ea2264a6f2bc9f37f6721a05bdb1cf0
36062 F20101222_AABGBS romero_s_Page_37.QC.jpg
15db5fb49ed33784dda281fec380f109
673f794614c67df1ec995e6ff32c772b4a72c30c
1904 F20101222_AABFWM romero_s_Page_25.txt
50fedbd3d781a0408ea5db3b2eddf1a1
0cfc0d896feeac8b8d45966f47427cb8322cb901
1032 F20101222_AABFVX romero_s_Page_07.txt
be0f26b51992870a2e41de9048a523a3
ec76ab336323c97b21af543f9074c8f437a12e48
2166 F20101222_AABFXB romero_s_Page_43.txt
7c65ac87367e131c4d3926c32af7a27d
b25b75df50c91ec6414fddf177bc4b3d46617747
8522 F20101222_AABGBT romero_s_Page_22thm.jpg
8ff7de89510b3ef54a8a3bcb9df88262
3764a06aa5ce59713bc8e6887d5508036331d9d7
2089 F20101222_AABFWN romero_s_Page_26.txt
ef2cc4c098f9a70e31cdc25b98d82cc4
6c36563565c92c35d146a127b373151542fa1e0d
1873 F20101222_AABFVY romero_s_Page_08.txt
5f2523bee1f695febbe9b00111aff06a
3ecb723bb799b20697f75db810cecbb48e645885
31416 F20101222_AABGCH romero_s_Page_56.QC.jpg
6fd5f234e0d5a918ca3eff6a0d54b021
bf07a33db213afa06ee8cd598732e6e0fe9ce446
2016 F20101222_AABFXC romero_s_Page_44.txt
47fc65ddeae0bb21ed281563bd738b68
cb3935869585aaad9c5c9aef79e5434c6e291b19
4489 F20101222_AABGBU romero_s_Page_05thm.jpg
24cbaf224e264257eba7b92b8662d3e7
4a114c4c09d8de3d5aeaa9b93fdca56479844b4e
F20101222_AABFWO romero_s_Page_27.txt
a5d512ec1f6d81a9339c947ecb94de70
5c435bfbe4b5bf5fb98fb8ea66e07c65fabe99ab
1207 F20101222_AABFVZ romero_s_Page_09.txt
4e4286aa83fad925432cecd644a0e2dd
d21492ea877bd288a12d30ad65630aa0e35cfd58
14500 F20101222_AABGCI romero_s_Page_58.QC.jpg
dd5ed22571e47b64588bb15f4485205c
3d08a042e4dd3227242da27697b0937435b97f99
2204 F20101222_AABFXD romero_s_Page_45.txt
e0a072c146b15b4a051bb037c8ce2a51
1887af89ef436d0b5ec3bab0ff4a079daee31be3
34765 F20101222_AABGBV romero_s_Page_72.QC.jpg
038e85c6fee4e6cd7e71010f5f7fb807
26f578b8d6fb505c32e56a3953a628b15c66bfa1
2207 F20101222_AABFWP romero_s_Page_28.txt
ac622114276a14274118918894ca032a
1c0b2e9431b75a5773f17ed55c86f6325cefed94
37258 F20101222_AABGCJ romero_s_Page_70.QC.jpg
aadd7da8a67aa388231c78ef975300ad
a8679dd13a70d845a05b84601a97c7d9f815bd6c
F20101222_AABFXE romero_s_Page_46.txt
c677477ed9618d744e2c825e696a4b88
02b3473a49413a80b7449440af08f2f82b84e908
8174 F20101222_AABGBW romero_s_Page_64thm.jpg
a9669faed207cef7a4e655c7156ec528
22bb11289d51e02a4f6326d52e9a108daf2f666e
2074 F20101222_AABFWQ romero_s_Page_29.txt
ba280a741e1eaadcaf9b6c68c25fe810
57b332013e7a9fedcf6df7280e6547e44307d3fc
35857 F20101222_AABGCK romero_s_Page_12.QC.jpg
c752a540272e12d6ffb2c05ad3b95d76
711701cd9b3979d843355ecb6daf67f36f238b74
2127 F20101222_AABFXF romero_s_Page_47.txt
51dd92be4c013a52b2ce8b2225ca41e1
9c3333d89a84e1cec03ee11a74efc3fb5a492afe
34072 F20101222_AABGBX romero_s_Page_22.QC.jpg
3d511fafa8904cc91f1f508aa2041530
aa07ef9c20e6b309b4aabdb6ca852771014a2f78
2153 F20101222_AABFWR romero_s_Page_30.txt
2378a4fb9a4b2219557f4885215d8a95
df00b537b3012587a235a288ecc0eac4d2330352
33651 F20101222_AABGDA romero_s_Page_29.QC.jpg
75aa153d461072a571a6322cd29355ed
6b323b58b4207d71202da2b20893d05ca1520f95
33814 F20101222_AABGCL romero_s_Page_10.QC.jpg
9012765b969ef839303bc5cc21374a60
f3ecbb6e1d50b7d4c6f64cedbb2bd733424b868f
1976 F20101222_AABFXG romero_s_Page_48.txt
8768b518498670fd9a93a862c45b0a6a
17ed13ce4abb636e403cb2c305cef8218e9734f1
8203 F20101222_AABGBY romero_s_Page_25thm.jpg
d284142661e85ed2afbd9ad9addb5a68
5881751e801954c08be7dd0e1630c84a8f0bb409
2120 F20101222_AABFWS romero_s_Page_31.txt
9f7270c8d611b6b6c180248bcbc7d6e4
4b095ed48bcae3795c9ce24eee30403877f624a3
2347 F20101222_AABGDB romero_s_Page_76thm.jpg
c0e2b2b0f2c224638e83e4ef7bae7dd2
3dc24a14944f697371acd178738b35175696a946
34654 F20101222_AABGCM romero_s_Page_61.QC.jpg
0170c3e49109d38121432efaa5519889
6597fb3792ad1ab5bd713d9d8feb98bc35a24884
2015 F20101222_AABFXH romero_s_Page_49.txt
6627ead24a93a482c30bca57800ca0f6
d4675a6bf1b437659a23cc7e6451fa7cbb04b1c3
34339 F20101222_AABGBZ romero_s_Page_41.QC.jpg
664ae3066a7fe6bdd3f3f428426d9f2a
bf61ca04612429a64437ac6fc5a4087aa48146b9
2169 F20101222_AABFWT romero_s_Page_34.txt
724a3ba03b9df9965aa1af1ee4844bb8
86ceaa52688eeb1eb2653b6b6d8c3d55022ad915
8815 F20101222_AABGDC romero_s_Page_62thm.jpg
eec8c63aa74f59a698bc77116110b354
fd43076cd33dbf63b9954d41d73870766a87675c
8969 F20101222_AABGCN romero_s_Page_74thm.jpg
ba0571e138ac5bce3da525952ee8cfc2
cd25d1ef3cfe4400e0ce5f47ca9d47adf198fba6
2037 F20101222_AABFXI romero_s_Page_51.txt
91d6c987898e6c76467bdb30b30daa2d
ec09278cf3fd93c801d00082781e46ec966a3e5e
2139 F20101222_AABFWU romero_s_Page_35.txt
335b8787d707a398120118248437aa38
25b5def9701ad258ef8a1d44035530661c67c654
5187 F20101222_AABGCO romero_s_Page_77thm.jpg
e802fe3852e67077ba63354679ee7b08
697f8e64dca67ea20a9ee1eff6b0e9593225c754
2175 F20101222_AABFXJ romero_s_Page_52.txt
31ca7424a89638165780400832913370
77fc4819603c6fa7877907ce26adc6595ca842d6
2114 F20101222_AABFWV romero_s_Page_36.txt
498a757c9cf6844b2b491a3ef373ce1c
e726f64b5e4c5d55e43bae22bcb3eb5c9736b884
8200 F20101222_AABGDD romero_s_Page_44thm.jpg
12f5c1ad31044a89c5839f49a6e12d25
179ac478e1fbf308663d7a0d2dd6c224eacc9c8d
35757 F20101222_AABGCP romero_s_Page_75.QC.jpg
63dfd7047b75f6115a8ffff000e724da
6f06fb41eb230859a369e2913a3550b0da6a4533
F20101222_AABFXK romero_s_Page_53.txt
6c2408602b0648a9e155155edc8d0bb0
b6e4f1e54d20f6fac10e5da81f8f5c556a05a061
F20101222_AABFWW romero_s_Page_37.txt
8d3dd2753114e278c3202fa33e3ac9b7
d4d218cb55fd0263ba144c6914786a2ca75092c9
19975 F20101222_AABGDE romero_s_Page_77.QC.jpg
e4e462fe3261ce7fc396419dbc67f39e
11fa83ecd383663217ef86797aa3f858034b775f
34279 F20101222_AABGCQ romero_s_Page_33.QC.jpg
7b9a5c00e24fa840d206f61e3f96cd16
d13b742969fe4ed496dc1b169b25cdb46677fd43
2136 F20101222_AABFXL romero_s_Page_54.txt
77d536e6be0f269f68d4231d4c8d88a8
2f9b1c82ba4f65e0c4f831be2f206448eb443972
147313 F20101222_AABGDF UFE0022620_00001.xml FULL
839f20dd634b07e4b35a965bd1ab7862
0ea1c77564f34a4818eb03c6e2fbf4875d2de340
19443 F20101222_AABGCR romero_s_Page_86.QC.jpg
0fa7c3b04610d0ce5d5441dc666b9bec
540709e5ec8e21b7758b32ca1444a4e708eb9d30
2194 F20101222_AABFXM romero_s_Page_55.txt
2ed80da582361e8836e5c2729ac9e36b
1be06e062256805c3e569e3205682dfaca8e800c
2183 F20101222_AABFWX romero_s_Page_38.txt
9ef6382ed0c4c59879d4be2ef88765f2
ce8b65034fdc23606593e2f7442e22cbec45ff76
1142 F20101222_AABGDG romero_s_Page_02.QC.jpg
b6c6e3d7472ea50f393c8c9620d42c02
7d4aa7a0359be21421a07f03496ab8ae564e0629
F20101222_AABFYA romero_s_Page_72.txt
4cc215811b275832730f6906be5ed4a4
a13d6401dbf0ff316de76d1e328057cef3e9ab4d
8878 F20101222_AABGCS romero_s_Page_24thm.jpg
b2277ea6e6078e12e4c1aabb6fdd82ca
04ce9c8ed712f132db1e3e1ccc1a7dc650408abd
1941 F20101222_AABFXN romero_s_Page_56.txt
d3b9552d1352961ed718d83bf95aabf9
3684a538a84ac175313e2192d9ee20c623a859eb
1322 F20101222_AABFWY romero_s_Page_39.txt
a55e928c88297518cb62bd0e98cddbef
8ac0658ad263c961a3637ee747a70eeb0204dc32
15345 F20101222_AABGDH romero_s_Page_07.QC.jpg
de21dffdc6adad08df3df8448ace43a7
88f3907465a793deff1af518e9170e32fc2188fa
2075 F20101222_AABFYB romero_s_Page_73.txt
2624166550ce4f891f68bb2b494d82b0
66d839e8df43a6e2126ad13b9c90e3029a548dca
36561 F20101222_AABGCT romero_s_Page_89.QC.jpg
d2991576a9e6c2ee8cbffb7d572e90e7
287d57556aa5da2efc75101227ba519b0cb3240c
234 F20101222_AABFXO romero_s_Page_57.txt
168c169f60880233d70392dc5fce345b
0e886e2cfb96f888597aeef07e92d34404b2b7d3
2118 F20101222_AABFWZ romero_s_Page_41.txt
c27c8fef92a809aaf3b5e3fad009a508
d11e8ab69d77e99035a3303846aa9d1a67cef020
7228 F20101222_AABGDI romero_s_Page_08thm.jpg
c80f5ae33b440049a326fae9e2454f01
19c5aeec75dbbf62bfff245581e2c746ffc3574e
2236 F20101222_AABFYC romero_s_Page_74.txt
8418234cf1146058d0caf489714b98b2
196a48684488f8c76c45c622d00c87b84f69a8ac
8215 F20101222_AABGCU romero_s_Page_01.QC.jpg
e2317e4b832c1ee5f2da89d0ca772b78
1efeec796cd746ac7c555fd73a7ea5620ab54de0
852 F20101222_AABFXP romero_s_Page_58.txt
26de7b93eaa96cd1bf4bf10cc917ad62
21edfb908b8485ec6b2baf013386bdbc332ef3eb
19785 F20101222_AABGDJ romero_s_Page_09.QC.jpg
d8f837b78dc70d5fd50de9eea6276c53
5f66cc41c1ec0895e83f015fedac4ffc01e715c7
F20101222_AABFYD romero_s_Page_75.txt
8a4b1bfc4dd806ed7261b2594e176384
246ba3344f607fac833b724d354e84cec8d55456
8707 F20101222_AABGCV romero_s_Page_26thm.jpg
8f9597ef4639725d1abb775c15bb0b27
a790a8b04566ca7d4e1a004602df15662d7ee1a3
938 F20101222_AABFXQ romero_s_Page_59.txt
ab1bc95f75094f3b75a5b61ee98485cf
88e5ca2c4719298ae9cfe1145fefbadbea918303
5184 F20101222_AABGDK romero_s_Page_09thm.jpg
5816a0e6dd75da2540afb624a2501d59
e79848666d4f034966c10c973af6b94d63500e65
528 F20101222_AABFYE romero_s_Page_76.txt
8b6c5abc847de9d3a81eca1e3c76c5a5
a2eaf50b4927cd4873ca7fd0bd61e70027320a8e
8567 F20101222_AABGCW romero_s_Page_96thm.jpg
62f2cbfc36ecd1ad7094daa220fd3381
40262cc5c4e4163d0d61f0886454c00d5da97e7b
873 F20101222_AABFXR romero_s_Page_60.txt
091c5375251bf6327becb53d1d7b694c
ede71978b3e562e32bd6725191324cf37232646b
7496 F20101222_AABGEA romero_s_Page_57.QC.jpg
db1e0ecf7820d57f30c2d943fbef0046
1ae41dbc8c901b60cc30b44fef77bf2c4bb95f2f
8209 F20101222_AABGDL romero_s_Page_10thm.jpg
f12d496ad1926da703610baff2371ebc
4208dea7b3a992ebb4839d5a609e2bc91e51d004
2260 F20101222_AABFYF romero_s_Page_79.txt
051be266650eeb83dba7770c97156f7a
9db9d6acd92b2554b0c5ccd74d9ad63cc8a3b68d
8576 F20101222_AABGCX romero_s_Page_66thm.jpg
76753670879129fdfd4579c5ec591eed
ed3ae301ed41d26c99ae2c07d5dd3d1c5413b23b
2101 F20101222_AABFXS romero_s_Page_61.txt
9ab3f648f237823ba2028a302b51dbca
583c70039c1241f5e7eb634165c156b0f17a7800
12190 F20101222_AABGEB romero_s_Page_59.QC.jpg
42ad1e15f4aeeb8cb00d211cd3e8845d
365e518cab38d46aab46fe5d0e00985e2a575b5d
37349 F20101222_AABGDM romero_s_Page_24.QC.jpg
866c7096e17410eddf0fb6bfb41eff6f
c9aac6a82a0d01e47338bba7cd7bffd4a0c9a8fa
2172 F20101222_AABFYG romero_s_Page_80.txt
4b420494eec72e082fa6917e253f0b08
ee792e1f91d93db5fb13ad2455f3194904100346
2941 F20101222_AABGCY romero_s_Page_57thm.jpg
a8d977e424f94d8b53ba1d85c7c27c6f
a69fdade189f7d453c8e20c434d880807ceacb3b
2161 F20101222_AABFXT romero_s_Page_62.txt
13963c2d3dce313fe5ab2c1e106efb51
a05524a57eb1a402f787bcb3f090c5a6e3b63843
4445 F20101222_AABGEC romero_s_Page_59thm.jpg
a4bff94f90fe2438b0f4b6847346c997
6cf3174d20e227a3ac55db2ae189a81174e17799
35455 F20101222_AABGDN romero_s_Page_26.QC.jpg
2d44e8576171d27842f4f5a4c30c44ba
d7e75ebd97068f1c3cbedb2ec9c4da7a0deba323
2091 F20101222_AABFYH romero_s_Page_82.txt
618758fe219945aa62147ac86184f23a
1d385c36f9ed9875b78b5b6873db94264998d9a9
36515 F20101222_AABGCZ romero_s_Page_14.QC.jpg
6d7217965cfc5254b368397b5e1ceef7
e62952682a3b8bc72984602537145ee80e77f81f
2082 F20101222_AABFXU romero_s_Page_63.txt
d662a894757a72ac3c12781ec739a6f4
cac4240a33f7674aa12745a7769dc8da2a182c65
8407 F20101222_AABGED romero_s_Page_61thm.jpg
295022cdae7b9067b2a05fcbda62e12c
492e0533c943381cb625e8c3c30720eaf5b501eb
34864 F20101222_AABGDO romero_s_Page_34.QC.jpg
d5359af273254e1579b1a3f8051dcbf7
ea2a8f6a5129a28025a5baf58cdc11e14df1f1ad
2187 F20101222_AABFYI romero_s_Page_83.txt
5a2815596e8a56cb68e0f311ef232809
eb9aa967ccd79c5d71bab8c6c972efcb8d3f988e
2185 F20101222_AABFXV romero_s_Page_65.txt
452721e03b8e340ca8dc25b324bfb269
ae36390f87ef9dab7a1a9e8bd87b5cf506f6c82e
8517 F20101222_AABGDP romero_s_Page_35thm.jpg
55b54d33a28b7471a57a6bbc5a6bc0ae
b816a0f9e50091dc7ca2df467a21932150e058a7
1949 F20101222_AABFYJ romero_s_Page_84.txt
71018e8701d94f91f1d01d242e97d09b
2884f0d990bf0b8435ce5ba4c2d8f3ad2ba0f3d9
2051 F20101222_AABFXW romero_s_Page_66.txt
b26eed1ad03a9bf5daa37e5c4b3aeafe
5004a302080f9adf6c62c310a990b50b4c73cdd4
32796 F20101222_AABGEE romero_s_Page_64.QC.jpg
e3b4adf15c60356e4be93397ea3b914d
32f40bb1a692d61f62fc49b536e09790051774ae
35169 F20101222_AABGDQ romero_s_Page_38.QC.jpg
245543554a9be4d9555514b891864b40
fbc89ca4f15823367535731490036234a13f6e3a
1052 F20101222_AABFYK romero_s_Page_86.txt
70da21c8778b9ab34e58be63bb778086
054b7d194b0c65a3730672692d7b0a0b515b5090
2141 F20101222_AABFXX romero_s_Page_69.txt
d57e896dcbd4148ca3788732952fdfde
687fddd4e6879dbf450ccf5406e26385f693a3a5
9028 F20101222_AABGEF romero_s_Page_65thm.jpg
801de73335527ba5b261a3259df5d083
b74aae8326665d7bcd7973f19dead94c4802f6ec
22397 F20101222_AABGDR romero_s_Page_39.QC.jpg
46f8f079166ec681fdff59ecf612c81f
c0bae2d26b872f6d873375f44a627cb7ac7de453
1479 F20101222_AABFYL romero_s_Page_87.txt
3331a1c860583f19d97d919051dca0ea
b961beae3f36a1f538a89be5c731495723a58cac
8401 F20101222_AABGEG romero_s_Page_68thm.jpg
3f5c1794ff76b940fee1436c22f2ffe9
3a1e7db97516875aee863d253dd8e438c0255bef
7374 F20101222_AABFZA romero_s_Page_85thm.jpg
373e27c8a7f20a6d50a5583dc7b6f60e
f96637115d54a45916f31cd6460b5cfc02900eb6
5664 F20101222_AABGDS romero_s_Page_39thm.jpg
81c7b518c1c6ddc93040022b8aa6a70b
32741ef5f6fde17a827afdbadf7cf974bc49b348
2230 F20101222_AABFYM romero_s_Page_88.txt
f15aa97c80b85420cf58685c775711ae
14dcfb7a7bd2e5ab8c73981b1f251e4425a9c809
F20101222_AABFXY romero_s_Page_70.txt
af47926563932fa60cc782442a856383
e742753f2f0eec5c3c8b4fbdbe86afa3e68104b1
9441 F20101222_AABGEH romero_s_Page_76.QC.jpg
a9e8ac03f123afd2fb79d51a2942300f
abf6c62549e7a25c7eacb86d3eebd1f671d6c55b
4059 F20101222_AABFZB romero_s_Page_07thm.jpg
653dca0f24d43ff512f2817edde6b8e3
7937f864ce0cce8ae149c46bbdc7cc560c51df80
38703 F20101222_AABGDT romero_s_Page_42.QC.jpg
a05cfe754b9351d021793f4ac8f54232
ebf073018b95b6a417fdaa20d595e5f7ea85d3fc
2622 F20101222_AABFYN romero_s_Page_89.txt
e4d2348f81d0f1fcd6fe4301e7dfcde4
e46701c4d7d30ebf15b219b52049282e59efae5c
2182 F20101222_AABFXZ romero_s_Page_71.txt
7e472e2eba4dae8449c9b75babf4de1c
c323fb33388b577e8e41018965fe33d5924f7202
36014 F20101222_AABGEI romero_s_Page_79.QC.jpg
3d3d2424ec434135aeca88be533597e7
c7c604e0df71763a4b873a26444d5c9dab2747b7
8822 F20101222_AABFZC romero_s_Page_71thm.jpg
e03dd98ed2a9cdc886c9d48335898fa7
2b9ed3702ba3b7292602982867830838d6935ccf
9087 F20101222_AABGDU romero_s_Page_42thm.jpg
381276fc3c43958dbc42213b04433b61
3f697a554515b82a27e8d54e3efd433bee91bdab
2643 F20101222_AABFYO romero_s_Page_90.txt
dcba61b9dae9c0c90b2844051bcfeade
b200652fab6abfdac169ddfcd816c27a6cbf870e
8811 F20101222_AABGEJ romero_s_Page_79thm.jpg
d87d68ab6f592fa9d61e140c4cdf3648
75883f27432bfeef150c57c8cff0dcb1ea7e3d92
2962 F20101222_AABFZD romero_s_Page_40thm.jpg
06dc2c5b32482c3ff118803ce26aa5c3
cd8effe0ad112c8b175c45d82f1f76dbd1dc9b87
35168 F20101222_AABGDV romero_s_Page_46.QC.jpg
2529bd3a762ee9477bf6f507ccddc51f
280c12527349b190db1df63b123cdfd96fe120d8
2302 F20101222_AABFYP romero_s_Page_91.txt
0a2a02b84406b14fe92f42985d784901
e11466ce69f93f8d71fc621fc3be6782a7b629f4
8347 F20101222_AABGEK romero_s_Page_82thm.jpg
d5648da0f2a9c885c535d4e481d9d475
0ea3b6d38f25bb1698f24baeb67703c3220ec614
9063 F20101222_AABFZE romero_s_Page_18thm.jpg
b1276b0ce280fdcadfb75f2d6c31bca4
df7455acc8c6ed3620b9ada7c2aee839d2469db3
8645 F20101222_AABGDW romero_s_Page_47thm.jpg
90be6732393064a076abe3f630c054a3
d9c955b31d427bd0784ca37084d5ce2cc617c822
2655 F20101222_AABFYQ romero_s_Page_92.txt
bf4c9aa1f7592a1a368e84bd70d0141e
1d2b2b806d4f32348069875cd22c16312dccbbab
35672 F20101222_AABGEL romero_s_Page_83.QC.jpg
9aac8eaebd0cc980eb85b027ca789e57
2e1ba5da622d477aafcb70529c34a539579a08a9
8665 F20101222_AABFZF romero_s_Page_69thm.jpg
b5c3832e60147957e269c85c8730dd7a
6a7bcece5a1b20b753672cc0b0831a86e4631a69
34656 F20101222_AABGDX romero_s_Page_49.QC.jpg
57ff0661165988c5d2f5bb96d210f3f6
92a2206967bedd155a23df45b16cc9f510fe8576
2479 F20101222_AABFYR romero_s_Page_93.txt
b00c06730d0be6db7f69ceb5b9f6e6e4
3ceddc142f5bf91a1086624fb804c0f5352aa413
7776 F20101222_AABGEM romero_s_Page_84thm.jpg
3f6db955c29212b5aa600a92fc4499c4
53ab9b0737b0d85494784ed9121467e062de4457
35006 F20101222_AABFZG romero_s_Page_66.QC.jpg
69c984098feee308ad7d945d9be68ead
d0481aa1a2e9aa98a058afe41e940b259f6f340b
8773 F20101222_AABGDY romero_s_Page_49thm.jpg
4f4af88e1a3919bdb9c9b567a8e15e2e
92645637b249d625a442f9f76cc0da3b3f335dc4
2237 F20101222_AABFYS romero_s_Page_96.txt
15882964a24f833d51229bd00ae33462
42187e62ad37e8917e47609b0a1f1c3117b5a3ab
38451 F20101222_AABGEN romero_s_Page_90.QC.jpg
6353666feb8036ca908c9052bb095647
12a38982fb54c9b4b5db6476a389534a34ac5b94
8488 F20101222_AABFZH romero_s_Page_41thm.jpg
430788e95f4104904ee5e8d60a0296f3
0e0ce74d60f432dbeaf151e6ed3c501161f643dd
8544 F20101222_AABGDZ romero_s_Page_52thm.jpg
1062ef3a02f4a7a0d2315946f5143c81
bc2992b810ee6e5b68233fe8280b54593c6007dc
2113 F20101222_AABFYT romero_s_Page_01thm.jpg
f495b331d3017de78759cb1a22d6b58a
6a86538d6f01b502944178eeb1ac9e0b426ed32e
8948 F20101222_AABGEO romero_s_Page_93thm.jpg
02bf5a1e9a898435d699227369c662ac
3ea3295658a3a6f3dd5ec5c3d8428e6a6f2b4a57
8507 F20101222_AABFZI romero_s_Page_20thm.jpg
296a2626b1b1db1ba0156c6c6d7e1d2c
436c1dd77ed792c1370452656c8a65d78e5b23e1
8415 F20101222_AABFYU romero_s_Page_48thm.jpg
15345edf3b6fa6baf13ca98c7dc38c42
73742aa4be5ecd186381c4f0f6c697d57f1a6dc8
8984 F20101222_AABGEP romero_s_Page_95thm.jpg
26a262b48f6fe8c3a67b012916347806
1ac5daffafe2e72fe89e6372a35b6fd11e55062a
8981 F20101222_AABFZJ romero_s_Page_37thm.jpg
d6220ef3287caa43ac3a7bd2c4165034
4d082e7fab1283501458bcf586d5ca1a34870c29
8609 F20101222_AABFYV romero_s_Page_29thm.jpg
78a45ea668c4b9480861fa276dea6d28
cce63cfba3f6bf6b5c48bf5de83e68857015fafb
23495 F20101222_AABGEQ romero_s_Page_98.QC.jpg
988bcf46478847443a15cf3898c360d6
58131f3483785d26c5d915ded779a51a4426d429
430 F20101222_AABFZK romero_s_Page_03thm.jpg
2c87fe1fd4b3be7ff1bb9b18beba309a
7114c079a0b91551fe23dad2da7ab409cef45cf4
34594 F20101222_AABFYW romero_s_Page_36.QC.jpg
8306ff47047fd438a30fa88b32502f38
8bf899d8ffe1e690ea660eca8b6cbe68ae19b833
35864 F20101222_AABFZL romero_s_Page_17.QC.jpg
670bbe68dec962cc48aa22552873e2d0
5ed6f6d02fbb56a8317c45d454f72ab7a31047ed
32929 F20101222_AABFYX romero_s_Page_19.QC.jpg
534f1db49ed26726e4d7f1ada54bb731
d7e707d70f67146281fd3f1fd2254c7fb6226a24
8639 F20101222_AABFZM romero_s_Page_33thm.jpg
43c2ca5122222e1592464f125e04145e
aca53a3623b459e80134c1e7ac3e9acea275da34
33052 F20101222_AABFYY romero_s_Page_25.QC.jpg
0f991f7edd456bb62aa2294a42f9edc7
3b1665523d93f336a7e7d4a7c83b33834753ca4a
37020 F20101222_AABFZN romero_s_Page_45.QC.jpg
158bc22d785e730b797c10790d513f70
a8786edf214f0e454fa6f63e984c5615fab0cc0b
37331 F20101222_AABFZO romero_s_Page_67.QC.jpg
e136dca626d39a4ac3cacb56990cab98
86b349859f54da79343dc4350f32295fdf59f1e1
35005 F20101222_AABFYZ romero_s_Page_51.QC.jpg
95a677475d609ba741c057e93e04c13b
59c0fbddcf751de9dbbb129a8d5dc039e439efdf
8585 F20101222_AABFZP romero_s_Page_30thm.jpg
a4b39836c66679297a4ab13ab63a896c
4c8b4d739f614f966e84cfddb7d7c5e5b27b415c
8913 F20101222_AABFZQ romero_s_Page_67thm.jpg
a1433f473f4dfb4470016c0c1816d09e
67f0ebfee49d9909a45a47ce641fcf5440708619
8549 F20101222_AABFZR romero_s_Page_46thm.jpg
7ac590f9dd4a3397f6ecb235046ee115
d83bbe841bbc3370416dcef1e5ab4c6e7e5739b5
33700 F20101222_AABFZS romero_s_Page_13.QC.jpg
fd08db0a04959db8e5f98a9f4e064547
eea1dfe42623c88b7afa302aac572c2a5580fa15
8959 F20101222_AABFZT romero_s_Page_91thm.jpg
f5ac6f6c2e39aa0aed7f65ad5cdd7c37
cf6b79e1249cfef4c89da774129157fa8bacf3a3
33588 F20101222_AABFZU romero_s_Page_82.QC.jpg
c2c075fe16556476bfba47ba745327bb
ecdd78c7a7692dcb73330b964a7bd8345957e091
532 F20101222_AABFZV romero_s_Page_02thm.jpg
e8cd673b7e2522fa28c9e65f5233373e
814a685210f4bc6e0dce5a5a30a31fec9c7232c0
8451 F20101222_AABFZW romero_s_Page_36thm.jpg
80156c6912d6162a328364b8a598c2c4
004784e9ad03dfd14eb69f87a2a915656984e16b
34435 F20101222_AABFZX romero_s_Page_62.QC.jpg
efc686912d0d9d2ba9bbdcb2051886a3
f5013be15a69858ebde2151fddfe8e53e6c6d04b
8753 F20101222_AABFZY romero_s_Page_80thm.jpg
c0452539f517260fdecea90c8dc58ad3
76484d27a7a79265d277bbd636a28596d15d036c
8873 F20101222_AABFZZ romero_s_Page_12thm.jpg
96e09a322a7c876ad89d3b2d44816e05
fb802a29e9ac9698b612e9846486156d3c300e6c
33511 F20101222_AABFEC romero_s_Page_11.QC.jpg
d5013155ffd1eef7b0168673b9ad70a8
e929b1d0bd63c82abda826e8cc7e0e44f90a6e21
2221 F20101222_AABFED romero_s_Page_24.txt
dfdebef2d63fca1508d63bbcd31ebf0a
ac2627af0cd298ea4ce9a1536cc00427783ee9fc
68498 F20101222_AABFEE romero_s_Page_97.jpg
15aab9e4f473abfc610722bd20075777
b11aabbebfc7f0c6036c1556e16b3aa2a9f4a489
1051958 F20101222_AABFEF romero_s_Page_74.jp2
c97af640db654ef1e95c3b29d9e06a60
18b0aadc08feb3641b96e64c6f7a4cb0703b37be
56591 F20101222_AABFEG romero_s_Page_24.pro
cfac4b255c64c93ecc1418025a92e5ce
61bbe719530600ec7ebaa6178df0bd0c00f048cb
8687 F20101222_AABFEH romero_s_Page_27thm.jpg
3297da846c80b8b17d2d97dd01153e54
dbf588c05be1a0eddb6f9a376dcfa216b6b460eb
F20101222_AABFEI romero_s_Page_92.tif
75a951aca978f44934aaa1b25fe4b466
8c0e0e7a1afd04af5aa22c610ae1a20c7216bc6d
5005 F20101222_AABFEJ romero_s_Page_86thm.jpg
d32d82b959819ed867609178520935a8
ae2db3ea130140511f31c5a63ed9dc9a516b5ed0
320792 F20101222_AABFEK romero_s_Page_78.jp2
1ba57dfa7d2902a368d67dfb86522908
2712a1d989076eefe3d4a28d1092f0b60ead5d68
F20101222_AABFFA romero_s_Page_18.tif
3a76ba76d4dea882be875016c339cc10
886c862b4d624815d0f4ff1aa0986d8d22ac04a8
112094 F20101222_AABFEL romero_s_Page_96.jpg
d33c7748c592ceed0a339cc6b39b123f
16d610dd0efd8a1e932c9131615b6563435c21de
32770 F20101222_AABFFB romero_s_Page_78.jpg
aff88b6107d0ab1b6337590df9292e33
0f3655e8d791e905b41f35bb59712785de2b4b99
F20101222_AABFEM romero_s_Page_10.tif
5cbfb62a1f4850cbeae79e4430168737
5b73c29c9a6847efd1f0fdfc0f54896c42181711
113524 F20101222_AABFFC romero_s_Page_45.jpg
7b9c776e55bc07d3ce83e5e5830a7428
d3a3b13eac1782b945d90f54dd800cacd6bb4a34
54976 F20101222_AABFEN romero_s_Page_62.pro
4eb1220c6363371b68706eb1e20579c6
f0f9cb9f2d088419404e32e89d882f21ac4c9952
38438 F20101222_AABFFD romero_s_Page_95.QC.jpg
8a63edd4bf8d437250cd9a5a012cb6e3
a2531e3ebf88d9cd21de889f6ff34f9e13aaf538
8568 F20101222_AABFEO romero_s_Page_63thm.jpg
0b0155635a3aec64c038b1d93f444f66
472be7b83579b00993be325550498d7bd85c2702
34292 F20101222_AABFFE romero_s_Page_50.QC.jpg
6e74442642fe1047110a8c3040e5bb26
46a6147ec28f815a3799279386e69c7b18c01611
8403 F20101222_AABFEP romero_s_Page_13thm.jpg
77bcc1f5140126666d61f3b5d7bfbf07
2e251249f80e101ac49c8d411d48987c9e84dff4
F20101222_AABFFF romero_s_Page_59.tif
965cfca69dce43004dbf97391773c1d2
1dd9633bf88210ff97115ccf56ffd0c319b2333d
F20101222_AABFEQ romero_s_Page_27.tif
822ec9ceb694fbfc3d23e1c54d8c97c4
5d9149ac7e7cd44f27e164937ba63d0152c62901
F20101222_AABFER romero_s_Page_51.tif
3a1768eb846a485fdbd8a4f29d17d910
3341f76c61eca36099f896272eafcf7f2e150e7b
53306 F20101222_AABFFG romero_s_Page_33.pro
8a9e571f64af4872c5a914cd744c5662
78e2d4db489dd343a679799ad3f682144e7dd51f
F20101222_AABFES romero_s_Page_11.tif
ddd3ba6dff80d641924e741a43ae8c47
d54f39d2bfc289ef11c3fe6b1de3a915875d82d6
19194 F20101222_AABFFH romero_s_Page_97.QC.jpg
8a169671dadc1b44daed8668a78b7eda
71ee7218ab756aca3bb9d3d9863f437efc1b7c9f
214 F20101222_AABFET romero_s_Page_40.txt
d934334fb02d8ef0fbd166386c7bd6a8
3594a7608b88b8f24477133ad3f3443f362ca2f5
9017 F20101222_AABFFI romero_s_Page_17thm.jpg
3deb2d3728f4cf6230af15cc10014e6e
9ba37854381f61b1afb419ff8cc37fa3316d591e
F20101222_AABFEU romero_s_Page_55.tif
71b7d8dd911e04191518e060823e4fa1
7d9453cc871fbf335182760ed4cbce09e671d71c
8284 F20101222_AABFFJ romero_s_Page_19thm.jpg
f8c5459645b4699e20865bf8b484038e
e095073192121c74397884dad4782187b704f06e
35234 F20101222_AABFEV romero_s_Page_31.QC.jpg
1182fc7909529a5f65a8508e452ab9a2
6507ff4c9f9dc362a37770d4b4af4c9ffd87d2ab
52956 F20101222_AABFFK romero_s_Page_82.pro
bcf212bb623b2a46bac213e29d274e20
c163aa917fd38271fb43898fe6d6a4f251600ca8
F20101222_AABFEW romero_s_Page_32thm.jpg
36c90a6bbac017e9eb0b4b938af07b1f
2c87592f90d9c4fd23a90ce304f6097214f55345
5712 F20101222_AABFGA romero_s_Page_98thm.jpg
ebffab7173a8222d8e71ca058002ca21
481b22f107b17757cae93bed84107020186ccfdf
1051927 F20101222_AABFFL romero_s_Page_28.jp2
504d08375db8f42c16f954052b092437
f3094eae5e1170d832d7c2685f97757494e74ee8
37505 F20101222_AABFEX romero_s_Page_32.QC.jpg
101258fe7366424898bd0de3f11a4eaa
b6c36e1dbb51d2fa3ddd6ff29d6eba493f6cc3d4
104987 F20101222_AABFGB romero_s_Page_61.jpg
479c1c9a6952ce636cc21045572bc2fd
e84903473948d3f12fc8d70ec385c793c7b20259
F20101222_AABFFM romero_s_Page_78.tif
ed5a8d18efa2ee877b00ee82d1e4c073
0be5694aa9ba121279a1f994f1f275af117f3892
55412 F20101222_AABFEY romero_s_Page_12.pro
62f4ea242f04f287d03c8f7a9da2df74
e8fc62327d18209f3d195714e7da4192fa23c75b
57051 F20101222_AABFGC romero_s_Page_42.pro
0c6592e721d4314ad968f62bd61e11c4
b28d07c515cd0dcfc74b61f54c5ecf9be4e84ee2
773090 F20101222_AABFFN romero_s_Page_06.jp2
cbd8ec3e4f1e4d2820a065c47f0b723b
e2e8bcfbfccb62340c56d119ea42cbfaf8b87549
33532 F20101222_AABFEZ romero_s_Page_68.QC.jpg
d7f9372ef4a2294572174a582d833c6b
405f63f4ecd2f87ee6fcae3fd61e41ab1643fd64
8888 F20101222_AABFGD romero_s_Page_55thm.jpg
28127d159a71a67040484dc2a4600ee2
e4861e7725506b57238d480570a32f7ce50357ed
106809 F20101222_AABFFO romero_s_Page_20.jpg
e4ae9384ade92fdfea3e1081f3a5dba7
a9ea8f3465c46b9c677495ce0a1bde3d4dbaca38
F20101222_AABFGE romero_s_Page_01.tif
6d20c4453784a30543e4e071704db018
c1cea63eb703e82902f2d9a21e9c06ec041ee8f4
54378 F20101222_AABFFP romero_s_Page_54.pro
e89e36e2f7c44dd5000c6582ac7719f7
859f11cd2be17384be7293b8f125bfef12c69dc3
36416 F20101222_AABFGF romero_s_Page_27.QC.jpg
56cb4c0d60dd671f20d9726bc50d4c2d
2594fae46065bba04f1e8ae1e71dcf65dd3bf73a
51272 F20101222_AABFFQ romero_s_Page_50.pro
cf2194d3f5af204ea2fb2b0c2d72d3ff
9dcbc4fe5a575570483b82d73df7fafd43a7a7c6
128902 F20101222_AABFGG romero_s_Page_93.jpg
47fbd667e21721cb0f234609c17ed933
b128f10bcdcbe5f0bc30388560d96f6c3d657717
68142 F20101222_AABFFR romero_s_Page_95.pro
18a4c4676e1f91432038330607d76f08
a2a2526fda7a4f2051522e9eaa04b3841d12c8a6
F20101222_AABFFS romero_s_Page_49.tif
88ebd70420126cffd68130f56f12f519
4f1b6b7bf74312357c95960964cfe55e35a6e54d
F20101222_AABFGH romero_s_Page_30.tif
7314e2fe5ba26309d4e28717a54287a6
09d7e720130aa2b1eb8a2bbc82e74c604a83f9bf
28069 F20101222_AABFFT romero_s_Page_08.QC.jpg
500893a44ee361cc5d0240ba740cf079
542d5344b30f5947d3e274dcb4750e776f1c9c03
2186 F20101222_AABFGI romero_s_Page_67.txt
c73762dab05750a2f3b0aa3d5b67b2b2
73cc9f94fdcb2f73e638c97730c4e9cdcee39b07
641589 F20101222_AABFFU romero_s_Page_77.jp2
5a003877d80c9075499beb7eedd5779c
2046b775fd53c3d776c70a37c67b85f2d0b53f9a
78158 F20101222_AABFGJ romero_s_Page_04.pro
eefe1ff3dcb908be86c44c7c7654c08e
1951398916fb4623d6fd4e9f835404514f797e61
F20101222_AABFFV romero_s_Page_71.tif
bc9d109c750440d7e8298ecdca7f06ec
025153f860ca7884826d1dbd7e0383ab564b1f75
734152 F20101222_AABFGK romero_s_Page_39.jp2
2ada8561b12abbbb2c070e1c15c01c81
78a7e2f4737b40615234a396142c747a01b9ced4
8893 F20101222_AABFFW romero_s_Page_70thm.jpg
7752f7daedbf52b5eb8b4242d69cba67
94db9106558d938db6073d666141d3e2d215a085
38357 F20101222_AABFGL romero_s_Page_92.QC.jpg
c7eb26a166f53ff2e5f49f294bdbe56b
a2249156a2d7af1e02d1f0137de2fed98e69d915
465 F20101222_AABFFX romero_s_Page_03.pro
705cc557c74d60f35d1c872149ca830d
81511517931bae3413271d517f2bec465c226ae3
1051965 F20101222_AABFHA romero_s_Page_36.jp2
b295c7cad51c1c2af169bce5b5608193
60e4ae76ef936600cd3ec83d6ba62ebfe913de68
F20101222_AABFGM romero_s_Page_68.txt
c190bc23b6cc565d282f089514ebb569
fa157ba9b925dd57a3114761f5b5072df642c729
1051972 F20101222_AABFFY romero_s_Page_92.jp2
642b3dd25d9d8548def48db3e53dcd34
13e0dd6b66e251ae19241924ccd641fc49e2942f
54208 F20101222_AABFHB romero_s_Page_52.pro
bc1c578240cba7ee2adb5c88c9099295
cd46119675cc0f7a7ca3df08acf006085f724d8e
F20101222_AABFGN romero_s_Page_29.tif
d82117d0762a322cbb05700a8f8ffe4e
6080e53ab9d51c6b5567f6f243955661e03fc49f
105392 F20101222_AABFFZ romero_s_Page_29.jpg
fcb9abd6c6709e13608bf77d7e0cd3c4
13413c49c032be2ee2733109faa45f794cb3c802
109515 F20101222_AABFHC romero_s_Page_31.jpg
9acf50b8f7ce581a1c0838f707ecd9e8
177a7de5efa6b1030251af743fc1cfbf1dadfce2
F20101222_AABFGO romero_s_Page_04.jp2
988da9549cc8f2293b1cd859df26f1f2
5557079fc57e56919a270dd3e5c463d5aa2ad518
55020 F20101222_AABFHD romero_s_Page_16.pro
55d2ee3659989e7a153375d0feba0f6c
82423b30359c6338a378ac55e663195a269fbede
F20101222_AABFGP romero_s_Page_24.jp2
a50e9e42ffe117df174eb1123906eded
6e0ed3e381f7c2142d34f0ba042e4e8835cb3077
F20101222_AABFHE romero_s_Page_06.tif
b43f1b050acfc4786d6423c563fd4a7d
b944d67303ffe2711938dbb584771e7ef398f768
8688 F20101222_AABFGQ romero_s_Page_31thm.jpg
bb01edf3cdeed7f819d2510cf83b3857
3f6c0cf071b016d7541c788561d7482bc26a5cbf
110714 F20101222_AABFHF romero_s_Page_75.jpg
7e98077b3ef90c3b7717e2d114155e8c
855b7454645a83a3532fdf78e303e3b15dcba6c6
51531 F20101222_AABFGR romero_s_Page_68.pro
973f26037117a11842a78bec5e6aeb3c
6d21f88aa65eb722e425c17d6449622608eac119
54327 F20101222_AABFHG romero_s_Page_35.pro
9bcee9940a70cb24e4d1088f37e8c027
e6a6f82d22dbfc07a0ab321dde94415a5acf62e3
1842 F20101222_AABFHH romero_s_Page_85.txt
fb1087e590b4f78116c6eeaec07d1ebd
175714563f40ac51f3479c6927835edd1a8bc45e
F20101222_AABFGS romero_s_Page_37.tif
482c34db57ae7b64dff148533b16abef
17ac5e2fc15055e8dcabf820b81e88c5b82a842a
1392 F20101222_AABFGT romero_s_Page_77.txt
7364b045c08a8361996512eabafa344b
14d6f9b45d6faa7d66c078957dd83c7a8d09f899
2096 F20101222_AABFHI romero_s_Page_33.txt
c310941f4a15ac41c0056f515df3be81
0acecaf638b370229e93ed201d1cb16e77ebaff1
F20101222_AABFGU romero_s_Page_81.txt
3a60811c412d16a4f9a44887d3e59c2f
0431e237a93e342d17c95fc7c462cd86c1fdf4d2
2011 F20101222_AABFHJ romero_s_Page_64.txt
937e4a1f90bd9f3f8439077cfb7a41c4
8750204b899b017017518170524e0188c8b82bab
33898 F20101222_AABFGV romero_s_Page_73.QC.jpg
6c5b5ac8a558ce282a93416aad2749ee
2177741d36e637dbf431c3f85b9f1e363b98f157
8450 F20101222_AABFHK romero_s_Page_73thm.jpg
d82a68c2503cf19db28251eb2b5440ae
5111b079e654d360f4078f2ca0def7b2dbf9bbdb
1398 F20101222_AABFGW romero_s_Page_98.txt
3a92733bba163dcd6aee138bef8f057b
febfb11fc094fa1d5b51350ee7bb8c3a74b876f6
F20101222_AABFIA romero_s_Page_15thm.jpg
5ddae14008e05a67a94b1f1752b02be6
be659f82761179cb0aea964cb2b9d28f18c93ac4
106202 F20101222_AABFHL romero_s_Page_22.jpg
0fd1b63153aa085acbf597551ce3c4ac
76babeac9b020f098f81fcb9f7f18888fab9012f
110868 F20101222_AABFGX romero_s_Page_14.jpg
4ebc281075ad9c2f0128c9180b84a4e1
6c328aa60022875fdd0b680712b1ec5a2629741c
F20101222_AABFIB romero_s_Page_21.tif
0118f09716d3fde7b148d357d0728f94
53903ab1b5f39196f6aa33c4e93524facbe118c3
377267 F20101222_AABFHM romero_s.pdf
3ba668830793478699147dddec9c75f3
2b1a1918b55b4d10de77f27ff2d5193581aae2d2
14455 F20101222_AABFGY romero_s_Page_03.jp2
b872b66d2900f29d12ccae86f117bdec
0d72f54b4488aa0150fde158ad63210c5329eed3
27931 F20101222_AABFIC romero_s_Page_76.jpg
7819d626e2dcd772e5dd1e18cc6c4802
ac60f503288802b456fe303734d42526d0b5eca8
8970 F20101222_AABFHN romero_s_Page_94thm.jpg
9c7f929b4a13c5a7eff0ee3dc1c3f035
bb10939f668e19d86eb41c705b5381b5e42eb433
F20101222_AABFGZ romero_s_Page_79.tif
b4b79c8d3d7625b99db575346b64a5dd
89c408a762cdde296d5ed5130c6075b2325dedc4
9145 F20101222_AABFID romero_s_Page_92thm.jpg
3715edf863b9ad5a19fe0edf1f8764aa
4083bb0b76cf6e4c076f18f6ad3b01de90313898
11890 F20101222_AABFHO romero_s_Page_60.QC.jpg
35c67bbb2a3b3c4faf1ced274d74ca38
ff6e3c1d6334e4bea91145c468b4e1421c0158a4
36850 F20101222_AABFIE romero_s_Page_80.QC.jpg
4be70b5953fc1412af82a00139e8a50b
51a5f5b44e4d1e467ef7fed79f9961d7f41afc00
54941 F20101222_AABFHP romero_s_Page_34.pro
00e01592504cb887ed64b6b623834c9b
00205f8ef7385ccb2fa642b468d5b19311454672
19724 F20101222_AABFIF romero_s_Page_58.pro
f53ff269e2707237c36f1d817a1d68a9
fb308964232afa0652c37f9993707e8a8a7e5448
64977 F20101222_AABFHQ romero_s_Page_86.jpg
4650cdeb9738762fd0b13b08b06a76d9
8ae19346f9a699b0c476df149504b2d58ca91a1a
55160 F20101222_AABFIG romero_s_Page_43.pro
10de91dd3c9137f1c69a4e18ad3de52f
40b8f0e4fe2cd41e351e86cf0d1d46cdc947f93b
2209 F20101222_AABFHR romero_s_Page_32.txt
8e80e64016800373742aa3a1239f6e68
1175b4054e241a563e06ebc565002d950efb9bfa
640518 F20101222_AABFIH romero_s_Page_86.jp2
3bbf004fdaa5133c25c242c74970332d
ebb9eef170c10c1242053327a702a1e059371a06
18242 F20101222_AABFHS romero_s_Page_05.QC.jpg
6f9c557a47299573ce354a3a58fcf9f5
bded090a2286a5f147e6515ab7d38feb9f31d18f
35398 F20101222_AABFII romero_s_Page_35.QC.jpg
597131d173a8fddcd94e0228a50ed26d
31914225cc2e5f9e715437b8a324fd1a62f52df2
F20101222_AABFHT romero_s_Page_55.jp2
66dbfe304169af66086e90af74098bfd
43c1509d5efd43fc3c80d8faf674cbfaf95e21be
34415 F20101222_AABFHU romero_s_Page_21.QC.jpg
f6fbb1a265d4bedebe84002b3a0a87b8
11cf397b251d4f34520c3483f19da6253a728aa0
2044 F20101222_AABFIJ romero_s_Page_15.txt
b88fe5ee1c1a67dc3575ff7979653b24
c020bd8353a914f88b94c8fb750e198cbd756c44
2474 F20101222_AABFHV romero_s_Page_94.txt
3bd0a5c86267fd5cdc4227b3342bf526
6bdb620e5f78d0e972194784abd137fb537c4bbd
2199 F20101222_AABFIK romero_s_Page_18.txt
3e8cccaf5ddaa374c0de79dc6be2c13b
48a0a161f1050d2b906133beda5924484f217102
F20101222_AABFHW romero_s_Page_15.tif
0519a551e4c41881f64f17732f7080e6
17ab66bed83028a5b39e27f88ee4e301dc831644
1189 F20101222_AABFIL romero_s_Page_97.txt
bd0a2fb34d403759641a70ab03f3dc3a
e2d495f189442ab357ba8f793b5d5298d2b6276e
F20101222_AABFHX romero_s_Page_08.tif
3b9f244a7cb18731bba63dec0b486972
36af5ba7621a77763d3ef7afd2256308613afb7b
36620 F20101222_AABFJA romero_s_Page_52.QC.jpg
70cf91154171c5b6411e1a1b180297b2
a00286af94c04a7a0e7270af8acc2cd9b2e9eb5e
2704 F20101222_AABFIM romero_s_Page_95.txt
c19ce2f680eac11bf7ab20cf745e101a
500056c19ba5a16e388c2ba2931bc6d051fe6812
55444 F20101222_AABFHY romero_s_Page_28.pro
d228e3b0c38729624ee5bd254cf98ebb
0d94ccb55797889ae04691417257f03aed565dda
33357 F20101222_AABFJB romero_s_Page_44.QC.jpg
3bafd81d8d0de69c7079572621df69ac
a9b52255fdf778572d4f25e3b13524670ec69311
111935 F20101222_AABFIN romero_s_Page_79.jpg
b71f0235c768f4d5504b95767eb5de33
8fcb621d94db2e01b4a7bfef83dcf4d94145d5fe
8211 F20101222_AABFHZ romero_s_Page_01.pro
8ca072b5d2f641ccb53beef0b1559a09
a4e168eefecfab05afb2ae59c3331b4d75d04166
10715 F20101222_AABFJC romero_s_Page_78.QC.jpg
c3c1ddd163f7342a08e77eafe176b723
fdde80e96cfb81101233b08eb0ad2b06771e219d
109368 F20101222_AABFIO romero_s_Page_83.jpg
339d75cfa343dc64466eb19a5a4a0eaa
bf8e24fe30c6efd97dd4c3357bcc9b59ad43c31a
52887 F20101222_AABFJD romero_s_Page_20.pro
5fef24b8b3faeb188723a834c4bad0dd
1a00e5a1b9c8d58db04cd31d70d490706a1e154e
34990 F20101222_AABFIP romero_s_Page_81.QC.jpg
ea2e1125b0add39376bf6dc6f1fbf152
17e7415e5938be70aa7ab0e2e1f038de8ce532b8
712 F20101222_AABFJE romero_s_Page_78.txt
df940f76ebeec1f9712dd81dd684b71e
4dba17f55aa01ea12722b0669ad1e5bdd413c3fc
64070 F20101222_AABFIQ romero_s_Page_77.jpg
c3ac08b4dba4c843b91d96a064b830a5
45c6c8d374f5847d77a5e530c6ee8ada44a15675
12673 F20101222_AABFJF romero_s_Page_60.pro
312cf9f9db5fd135f90779e1cddb0b9f
b4a6237e051c3c01de2240966e454c383c78fbe0
36415 F20101222_AABFIR romero_s_Page_28.QC.jpg
d0073fb4a27070aada2a3ae31b523200
24a35bb6ebce4da6163a08247eddf50532fad5fc
52458 F20101222_AABFJG romero_s_Page_13.pro
e54499c413bb885c70724085a92b90e6
ca249c282b5031917550285b81ed3240549e8f39
974108 F20101222_AABFIS romero_s_Page_07.jp2
94b53970fcb940ce1c06004574f916a0
3a08b701374d40e6c5bdbf3f044f2239820acb54
8804 F20101222_AABFJH romero_s_Page_83thm.jpg
14928e3a12f8d81a0c9a1a68a07bd405
265613074d0f9d6fe1c2da62c4f07908836c28c1
55100 F20101222_AABFIT romero_s_Page_37.pro
465bc99f01a776244e57698a6f6964ea
5bc2692370782753d3ef705bcb90476349320dfd
F20101222_AABFJI romero_s_Page_24.tif
11f81f5616966bda2539f0473026395d
363bd59b7ad76b043b050e05910e430a45698cda
F20101222_AABFIU romero_s_Page_97.tif
b9f2fe079a02cf34c857c25f013e1122
e28d7319b8fa3a73050bddf0308bee31f6b8235d
29487 F20101222_AABFJJ romero_s_Page_97.pro
3e139ac181ca98a34de27b190ba9b795
ff6474fdcca0cc032f6251ff491152dda3a1462d
8676 F20101222_AABFIV romero_s_Page_54thm.jpg
b2f1faf21b0514bceb2f39ef1fc10ab8
03dfee9948c6ca4c39f27fc9f44a645c3a8a317b
1051928 F20101222_AABFIW romero_s_Page_56.jp2
42b44106055e5b1c999ae2ba0e5a1d95
a591e961555eeb510f5a5abb0a2cf1da91756efe
35729 F20101222_AABFJK romero_s_Page_65.QC.jpg
ad5a8f5adcfbc4ed5d997647892b362f
497f573d4d471872d9c491b5e19d7d10d41588e4
F20101222_AABFIX romero_s_Page_31.jp2
26a16cc518a69b7512d9fb7bf89d4ea5
d99045ad968027805d4afaeb42c82d1f93ba191b
133278 F20101222_AABFKA romero_s_Page_04.jpg
4d6ad9cb42808be86ad39971e05ce668
addfe28e27ba195d39d2eac52cf118dfd5865972
4046 F20101222_AABFJL romero_s_Page_60thm.jpg
7e9e482e130f06f0a349fc1ca02575f4
0ccf3dc3d00a3206578980bca0d2ed8ebc143941
20342 F20101222_AABFIY romero_s_Page_06.pro
002d4aaa2cdecac2bb219e448d7925c2
2f691b8a71342b11b2d81860747b0c5598db884e
80674 F20101222_AABFKB romero_s_Page_05.jpg
41a6963597575f8713f7895e331db845
0b9c5177a5f2b6005d4cb9c2994749419d766bac
87 F20101222_AABFJM romero_s_Page_02.txt
7c7b6d1c5ae49f7062d763361fac59b3
004b274c6dfd4dfda550f5bb42bdf82b879e6436
141888 F20101222_AABFIZ romero_s_Page_95.jpg
63b1295e00f8d23244f9cd1ee7dfdc8c
7d022024e88587521043ea54743f46a746a27fdf
43464 F20101222_AABFKC romero_s_Page_06.jpg
79b1e93a347fc0283ad62410cbd856c3
dcd454b8100520ae9d97094aa5186bde40926bf0
8901 F20101222_AABFJN romero_s_Page_81thm.jpg
41b582d0ae8b04936e175c11fe17482f
47903eaa2894725fe772daee5ea07b33b5e2e3d2
53554 F20101222_AABFKD romero_s_Page_07.jpg
2a9b945a6c3c4f4a854d44c76074c82d
ef4f294b21f512a37b725afc506224a867ac3baa
35859 F20101222_AABFJO romero_s_Page_30.QC.jpg
dca84d91c43e6782aa2681a977833cba
de5607f0dbb6aff076a1d8db7305e2d010bb89f5
87956 F20101222_AABFKE romero_s_Page_08.jpg
90d0c41846ad4fca978ae9e29698b5b4
26c0985b82064efd2e26083e746ed71130f0e587
F20101222_AABFJP romero_s_Page_25.jp2
e45c8d281e8266f2c832de45d9907ec0
f42fc617b97ec3b5066dd2014491c340f50797d8
61820 F20101222_AABFKF romero_s_Page_09.jpg
9daea6e0cf825efa0baaef008b81b6bb
deddb44bca29838532457991381071690e8d14d1
2070 F20101222_AABFJQ romero_s_Page_50.txt
2713559e75c92b02f6877d3006c683ed
7aa4dcbcd7fc3acc2132e29cde308a90e9ceba4d
103693 F20101222_AABFKG romero_s_Page_10.jpg
ce8c8253981e661fe8b3672e0ae8ead0
b8c0b8b71692fb5ef68c8dfb736de8aa7a3a4f13
36421 F20101222_AABFJR romero_s_Page_23.QC.jpg
86708a7d0cddf908f40ed598dbcc9c2f
5e79fb0299f8980685423c49e02c2ac2832a1353
105061 F20101222_AABFKH romero_s_Page_11.jpg
3afaa102a385c6e6602e8d3153cf0b09
15f77448d12c09fa7094906b885c5d3fba18818a
438382 F20101222_AABFJS romero_s_Page_58.jp2
e4d8a2a60852d75e5513cfcd88abb7bc
5e8104dfa2fae53fefee61264e3216463f0db1dc
109228 F20101222_AABFKI romero_s_Page_12.jpg
d69174abd4faaed8b29741be803e6008
69e65e89fbfa96c0beade33d644493bc599c26fd
F20101222_AABFJT romero_s_Page_22.jp2
82a882f43fa9dcb06e083d96e26fb7cf
7923a0c71b1fcc1fed0572f9d79b1871a6151aa7
105429 F20101222_AABFKJ romero_s_Page_13.jpg
a5c0e55d17fccc9335c34ce698007c5c
aad782a142eb06aaa8d0b838f70596e90accf013
113957 F20101222_AABFJU UFE0022620_00001.mets
0db2b18ec8ed1f030a8b6d0ddf57ddbe
c66aa99137362a8ca6472198bd707b2732755df2
103624 F20101222_AABFKK romero_s_Page_15.jpg
5c1522fc98d5ce4a0f12d23305eae08a
458d640150651710fde978ef96a66e4c632e2db8
108184 F20101222_AABFLA romero_s_Page_35.jpg
e4c22aa16b213d82830da2eb877e8ee6
047ddb8455cdfb5b7d54c459f22843fe2a86466a
107989 F20101222_AABFKL romero_s_Page_16.jpg
eb74b83a90fbf38986796a2b32b49601
9859fb487134b51b0bf051af7ef7ac494ca43e71
26150 F20101222_AABFJX romero_s_Page_01.jpg
c3ef952cfb032d654669cd408ac24105
a15e4acec0f64868087115b4da96e4ac7dc6978d
106837 F20101222_AABFLB romero_s_Page_36.jpg
0fe98491f560c47bfc8ec51ce2ad4951
a2b0adcf7b86573dd51f8607b0accd74d18f629e
111753 F20101222_AABFKM romero_s_Page_17.jpg
b1f6b6389f005573e0fb771bba8d6305
d33e551ac1b8291a37f11ab94c78ff6984073484
3860 F20101222_AABFJY romero_s_Page_02.jpg
a9b3e6447d1d32d90767d719189eb0ad
7433fc401899433c2af0b9a60583d007140c8663
110679 F20101222_AABFLC romero_s_Page_37.jpg
728e82f322c3429e1a0cc745d86642d0
700d5b6103c5c96a801e79b2fdcd474570fb6e12
109214 F20101222_AABFKN romero_s_Page_18.jpg
fb7426aa7f99436b4452c06de17181f9
415c4fdb679699c711c642526aa111c52d545bb5
2863 F20101222_AABFJZ romero_s_Page_03.jpg
50e5e97daf1a8f0537f44e8f3af95b01
351c593b415c7553b187be8f3ed4c894c0499aaa
109432 F20101222_AABFLD romero_s_Page_38.jpg
469ab685e3ad088ea1d9f8c30f94c671
e27d2b687ac9823fd3e298988d16591ff98124b0
102706 F20101222_AABFKO romero_s_Page_19.jpg
c0df3225fb2582288f929bb10b1dbe2e
900be0da6b9a5a30434263f21aa2487d79e8d396
68472 F20101222_AABFLE romero_s_Page_39.jpg
f24c38b7c61cee91e743b3fab99cf550
1e37f5a2801733e918e1685f950494bbccc4b693
106707 F20101222_AABFKP romero_s_Page_21.jpg
6331f51e8d0519ec48a839d7e08b6807
4df10c592d08b77f1bda9c229366b36438868ad3
26243 F20101222_AABFLF romero_s_Page_40.jpg
86016601a9d518acd76d7c2c55e5867f
c12e0da81841f34e4af9a3de17345bf5d43e4237
109926 F20101222_AABFKQ romero_s_Page_23.jpg
e437f7c9d4f6852049a0598cf906666b
e6576456e86dd0bc00c2a6b22c6cf69882bb2b0e
105073 F20101222_AABFLG romero_s_Page_41.jpg
2857d9182bc927318c4addc06a590e00
3a80ae33085677d8a3fa86c532dcbd8f95cf1e39
111304 F20101222_AABFKR romero_s_Page_24.jpg
1e3aa8ee67665a073f30a7fd374dad33
40db54e7cf4f49c2f38ed8338a4447b64a679fc6
115365 F20101222_AABFLH romero_s_Page_42.jpg
ef730dbf45af8791544862360a1b0c87
65c6b486839a7b8ba430fb190211e529aef2b81b
98697 F20101222_AABFKS romero_s_Page_25.jpg
84145cedc62a563c67093a0c78ea09af
b942f36f48e796242edf0ca74cac313a774a5364
109467 F20101222_AABFLI romero_s_Page_43.jpg
2b9554ccc23b83eba57d9330c4efa728
614c564caebd0a6a0297b7cc7fb29556b289edcf
107477 F20101222_AABFKT romero_s_Page_26.jpg
6b1edf0a310fdfb8181390d50d0b3dec
a5d488d7018afcb4b1bb4b6ad3747719a2f60a74
100858 F20101222_AABFLJ romero_s_Page_44.jpg
aeabce0014eba32cd778a3e597601561
6fb3273b2133cd19281bf36a3052af3db2ccf56d
109900 F20101222_AABFKU romero_s_Page_27.jpg
7e2cdfa05ea4e12964deaa1e26de038a
e5ca62880e579209e58d8f2ad16b1c94ccb08504
104946 F20101222_AABFLK romero_s_Page_46.jpg
a6d964e84bbc677d1b5677f3d583d9ef
b2ae8e797c835e80c1b8ea6faee271c8b4ae17c7
110465 F20101222_AABFKV romero_s_Page_28.jpg
ec8a9e410c9b89e6326994a141135f6b
7bed2c2793b262e5ae0ab06df065f93c31752e77
109928 F20101222_AABFLL romero_s_Page_47.jpg
0300a261939dd65196229e80228f8862
09a162b35e6b4051634bdf9f74ba91e39d9360c1
109492 F20101222_AABFKW romero_s_Page_30.jpg
770208f3cd3bc2cefc94fba727226c44
3b9fdd69fb5e2af5fdddfc0fcbdd3eb61b9e5fe7
112689 F20101222_AABFKX romero_s_Page_32.jpg
395ff6f96fbbfce213c0f35c8b5dfaac
14c56247ab7dd8fbda5bd62046de8bf618e60993
105757 F20101222_AABFMA romero_s_Page_63.jpg
862bd1534dde28e0724eed4a39874f93
ccee1c19cb91399ebef16d4093c6189d1c8ec0e1
100365 F20101222_AABFLM romero_s_Page_48.jpg
0b779e944602dba5827233e0a7dde188
7118c4ad4b8d15e6636bbffd8b26ed18817de1a6
106655 F20101222_AABFKY romero_s_Page_33.jpg
ec64cd85caf4a68d67ace0006d947fb7
a555788c01509114c351b399ab5201a71efbb19a
99031 F20101222_AABFMB romero_s_Page_64.jpg
593e479682b261e084fb366f2b0a0846
e2b65b69f5b0f13857af84f26d5900ed0f1bc727
107010 F20101222_AABFLN romero_s_Page_49.jpg
cd7ebdd48dc403f508d6359dc82758b0
25d85c4f2ead76a9512a2f561165d2cc9f90dae4
108486 F20101222_AABFKZ romero_s_Page_34.jpg
6146ecfb755e2ffd8a5fa9fbb3a1d288
1743fd21e9fa8a4ebed92097705ec7ee76f08f46
111886 F20101222_AABFMC romero_s_Page_65.jpg
ec86c5021e3d3fccedf207dcead0fa6a
ed11997a2f812705a3481d4ea1b8b8de73fce8f3
103908 F20101222_AABFLO romero_s_Page_50.jpg
e8d020dfdc1e3841adb6ab834d20b561
c907dd4572eb4a48f427bed07dd1134f6ce44237
105656 F20101222_AABFMD romero_s_Page_66.jpg
073feddc8a9628f80d573f38fd43061f
ce4b74e6fd1e6e461a12c3caf4115faaa46b4d43
111502 F20101222_AABFME romero_s_Page_67.jpg
703023c4728c402ac9394ff3c2ac1fb1
b1d71de88936b8b186022ebd7f41cd2303616265
104422 F20101222_AABFLP romero_s_Page_51.jpg
382588141cf54771b75bfaa36e928131
afb8885cc864de6c65ead6087096012f02ed46ca
105413 F20101222_AABFMF romero_s_Page_68.jpg
b61f1a80d788435d5320a670de8fccc9
d0a2d3130f1e9c98a52265e407f188e5e8c57756
111080 F20101222_AABFLQ romero_s_Page_52.jpg
4bf9f84d5e990189df6bb96999668596
1948960c8a041d8bdf66f47eed22f368b3eda8fa
106327 F20101222_AABFMG romero_s_Page_69.jpg
98513f9d0c26318a67001a4e26c74cc6
ab861d43def18a33bdc99f30cc147268665e1589
105843 F20101222_AABFLR romero_s_Page_53.jpg
80044d66026ff0be796aef1c739aad10
fbb42f6574e3d015feed1bb6d271bccbb5cf972a
112583 F20101222_AABFMH romero_s_Page_70.jpg
7815675af87b84f0cabba5f11d066c2a
6ce2f75f475f1b94ae678c6f0975e8924b88917e
110250 F20101222_AABFLS romero_s_Page_54.jpg
b151c03e6efbb16fed511b13902a82c4
db4e4ce0f57734f2987569f160d91f96eb0fe016
110317 F20101222_AABFMI romero_s_Page_71.jpg
4a1ad3c92ff3d8fe9edbdf22d96ed100
8b53e1e13f4006629d9adcdc324a4ba03541693b
111331 F20101222_AABFLT romero_s_Page_55.jpg
74ee4cd290ec3f8c132e88ebba27b4df
6ac72024d558a6108858f801e5f2dc86aeac7617
105700 F20101222_AABFMJ romero_s_Page_72.jpg
bfdfe7d521cfdb455933c18c0a6eb76b
0a488fdbe94ec9244ecc9c0dca3770fb10266714
98911 F20101222_AABFLU romero_s_Page_56.jpg
f10cdf13a091578e085ea0dc25751684
5890de875aebc61ce8f78a341136ced133a3f19a
104004 F20101222_AABFMK romero_s_Page_73.jpg
cec08f9694e1d0f3098b2fc47e010f8c
125d72f9f849161308d5911d9da74b78780b2070
22535 F20101222_AABFLV romero_s_Page_57.jpg
65160cc963356223a2cb32b084e54990
cd31e4e6c06c30ab9f3746091baad4eae13e8a03
113541 F20101222_AABFML romero_s_Page_74.jpg
4e0efa2fb4f4897686b4e2d8c58c0bdb
143b422dbf1c700765ee051c844ee162f2eb4fed
45299 F20101222_AABFLW romero_s_Page_58.jpg
46e8c5f45e0d7d529685ea89cd9bc0fd
e739c5f0f62d9207bd56160811399b1b822f3de2
25158 F20101222_AABFNA romero_s_Page_02.jp2
863d4b6e24d62d2d01e7aac64ffa43cf
3137c39dddf64cffd4082d4199b843b51f3381d0
109054 F20101222_AABFMM romero_s_Page_80.jpg
1e566928cde61925805310fea429f306
a260e262b2952f89a7d9f4140ba21ea4521a3235
34824 F20101222_AABFLX romero_s_Page_59.jpg
29b421fec32a4f2ffa17435318f4930e
f875a9efb6d885d05ee0e748e5c15034aa7255df
F20101222_AABFNB romero_s_Page_05.jp2
5d5cd3f7ceb64fa3fff8ce998e115ab2
d8f41d69d7b750e94076db3c264bc4550c3bc489
33018 F20101222_AABFLY romero_s_Page_60.jpg
7d35ff25042419189508d9e39a11ca69
956e5df150e53c9f559c828ba8b4059cd557b30b







MINIMAL DETECTABLE CHANGE AND PATIENT REPORTED OUTCOMES INT FALLS
REHABILITATION





















By

SERGIO ROMERO


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2008


































O 2008 Sergio Romero











































To "Cu"












TABLE OF CONTENTS


page

LI ST OF T ABLE S ................. ...............6................


LI ST OF FIGURE S .............. ...............7.....


AB S TRAC T ......_ ................. ............_........8


CHAPTER


1 INTRODUCTION AND LITERATURE REVIEW .............. ...............10....


Introducti on .................. ...............10.................
Theoretical Framework............... ...............1
Clinical Change .............. ...............14....
Clinical Important Differences .............. ...............18....
Calculating Clinically Meaningful Change .............. ...............19....
Anchor-Based Methods ................. ...............20.................
Cross-sectional methods............... ...............20

Longitudinal methods............... ...............21
Distribution-Based Approaches............... ...............2
Paired t-Statistic .............. ...............24....
Growth curve analysis ................. ...............24........... ....
Methods Based on Sample Variation .............. ...............25....
Effect size ................ .... ......... .. ......... .............2
Standardized Response Mean (SRM) .............. ...............25....
Methods Based on Measurement Precision ................. ...............26...............
Standard Error of Mean (SEM) ................. ...............26........... ...
Reliable Change (RC) .............. ...............26....
Choosing an Appropriate Method .............. ...............27....
Patient Reported Outcomes (PRO) ................. ...............28................
PRO Instrum ents ............... .. .. .. ..... ......... ...............2
PROs and the International Classification of Function (ICF) ................. ................ ..30
Advantages of Using PROs ................... ............ ...............32......
Issues and Concerns about the Use of PRO ................. ...............35..............
Sum m ary ................. ...............3.. 8..............


2 ESTABLISHING THE MINIMUM DETECTABLE CHANGE (MDC) FOR THE
BERG BALANCE SCALE AND DYNAMIC GAIT INDEX .............. .....................4


Introducti on ................. ........... ...............41.......
Gait and Balance Assessment ................. ...............41........... ...
The Berg Balance Scale (BB S) ................. ...............41..............
The Dynamic Gait Index (DGI) ............... ... ........ ... ........ ... .......4
The Intraclass Correlation Coefficient (ICC), the Standard Error of Measurement
(SEM) and the Minimal Detectable Change (MDC) .................. ................4













M ethod s .............. ...............46....

Subj ects .........__... ..... .__ ...............46....
Testing Procedure ........._.__....... .__. ...............47....

Analy si s ........._.__........_. ...............48....
Re sults........._.__....... .__ ...............50....
Discussion ........._.__....... .__ ...............52....
Conclusion ........._.__........_. ...............56....


3 PATIENTS SUCCESS CRITERIA AND EXPECTATIONS INT FALLS
REHABILITATION ................. ...............61.................


Introducti on ................. ...............61.................
M ethod s .............. ...............64....

Subj ects ................. ...............64.................
Testing Procedure ................. ...............66.................

A naly si s ................. ...............67.......... .....
Re sults ................ ...............69.................
Discussion ................. ...............71.................


4 GENERAL SUMMARY AND CONCLUSIONS .............. ...............79....


Experiment I Summary ................. ...............80........... ....
Experiment II Summary ................. ... ...............8 1
General Conclusions and Future Directions ................. ...............83........... ...


APPENDIX Patient' s Perspective Outcome Questionnaire (PPOQ) ................. ................ ..85


LIST OF REFERENCES ........._.__....... .__. ...............88...


BIOGRAPHICAL SKETCH .............. ...............98....











LIST OF TABLES


Table page

2-1 Absolute difference in BB S scores between initial and re-scored assessments. ...............58

2-2 Absolute difference in DGI scores between initial and re-scored assessments ...............58

2-3 MDC values for the BBS and DGI .............. ...............58....

3-1 Demographics .............. ...............77....

3-2 Initial levels descriptive statistics .............. ...............77....

3-3 Success criteria (initial levels-success levels) descriptive statistics .............. .................77

3-4 Treatment expectations criteria (initial levels-expectations levels) descriptive
stati sti cs .........._.... ......._..._ ........_......78.










LIST OF FIGURES


FiMr page

1-1 International Classification of Functioning Disability and Health (ICF) model ...............40

2-1 Distribution of BBS difference scores (Initial BBS- re-scored BBS). .............. ..... .........._57

2-2 Distribution of DGI difference scores (Initial DGI- re-scored DGI) ................. ...............57

2-3 BBS results for all participants. Mean value and difference between initial and re-
scored values. .............. ...............59....

2-4 DGI results for all participants. Mean value and difference between initial and re-
scored values. .............. ...............59....

2-2 Mean BBS score for different absolute differences in BB S between testing occasions....60

2-3 Mean DGI score for different absolute differences in DGI between testing occasions.....60









Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

MINIMAL DETECTABLE CHANGE AND PATIENT REPORTED OUTCOMES INT FALLS
REHABILITATION


By

Sergio Romero

August 2008

Chair: Craig A. Velozo
Co-chair: Kathye E. Light
Major: Rehabilitation Science


The overall aim of this proj ect was to investigate the reliability of two instruments used for

the assessment of balance and to explore patient' s expectations and success criteria for the

rehabilitation of falls.

The first experiment investigated minimal detectable change (MDC) for two common

instruments used to assess gait and balance. The results of this study indicated that for the Berg

Balance Scale and the Dynamic Gait Index, 6.6 and 3.1 points respectively were required to be

95% confident that "genuine" change had occurred. These results suggest that a significant

amount of error is associated with these instruments. In addition, the results suggested that MDC

values are not a constant feature of the instruments. MDC values for the high function group

were 6.3 BBS points, as compared to 7.3 points for the low function group. That is, the values of

MDC change based on the ability level of the persons assessed.

The second experiment investigated patient' s success criteria and expectations with

treatment. Participants reported considerable initial levels of impairment in energy and drive,

mobility, and pain. Lower scores were seen in interaction with people and community and social









life. These findings suggest that domains with a strong social component were not as affected as

domains with a strong physical component.

Participants in this study required significant improvement to consider their treatment

successful. Domains such as mobility; and energy and drive, required significantly larger

reductions than the community and social life; and interactions with people domains. This

provides information about what is important to patients receiving this intervention. Participants

expected mobility to change the most. However, similar finding was reported in the domain of

energy and drive. An interesting finding was that, participant' s expectation was that the

treatment would not meet their success criteria, indicating that residual levels of impairment

were expected.

Collectively, this series of studies promotes our understanding of significant change in

patients receiving rehabilitation services related to falls. The results obtained indicate that current

rehabilitation programs must consider the limitations of available instruments and take into

consideration the needs and expectations of patients.









CHAPTER 1
INTRODUCTION AND LITERATURE REVIEW

Introduction

The elderly population is the fastest growing group in the USA [1]. By the year 2030, it is

expected that one in five Americans will be 65 or older. This represents an increase in the elderly

population that will double that of the total population growth [2]. This epidemiological profile is

particularly relevant to the national healthcare system, as the patient population will also

experience similar growth. This growth in the number and proportion of older people will place

increasing health and economic demands on the national healthcare system.

Although aging is a highly individual-dependent phenomenon, a progressive loss of

biological function is expected with age. Sometimes, this can lead to functional decline, loss of

independence, and, ultimately, a decrease in quality of life. In a number of elders, this biological

decline can result in compromised mobility. This is one of the most disabling conditions elders

can experience, since mobility limitations restrict the individual from fully participating in

everyday activities and can contribute to further functional decline [3]. Individuals with mobility

problems are prone to falls. The consequences of falls are far reaching and affect not only the

individual, but also caregivers and the healthcare system in general.

Falls are a serious health problem in the older population. They are the leading cause of

injury deaths among people 65 and older [4]. In 2002, nearly 13,000 people ages 65 and older

died from fall-related injuries [4]. Falls affect more than one-third of adults aged 65 years and

older each year [5]. As a result of the rapid growth in this segment of the population, fall-related

medical care is imposing an enormous demand on the US healthcare systems. In 1994, the cost

of fall-related injuries was $27.3 billion. This number is expected to reach $43.8 billion by the










year 2020 [6]. Preventive and rehabilitative strategies are needed to stop, or at least slow down,

this trend.

Several important areas need to be addressed relating to falls in the elder population.

First, there is a need to accurately identify the population at risk. Next, assessment tools used to

identify individuals at risk and measure performance change must be reliable and convey

information that is clinically relevant and easy to interpret. Finally, effective and feasible

rehabilitation strategies must be developed to specifically target this population. To this end,

rehabilitation goals must include the patients' criteria to ensure adherence to treatment and

accurately determine treatment success. The overall goals of this dissertation are to investigate

meaningful change in two assessment tools commonly used in falls rehabilitation programs, and

to explore patients' expectations and success criteria in the rehabilitation of falls.

In the subsequent literature review I will develop the background to address the study's

purpose. First, I will introduce the conceptual model that has guided this dissertation, the World

Health Organization' s International Classification of Function and Disability (ICF). The ICF

provides a theoretical framework to encompass the multiple dimensions of the rehabilitation

process and the factors that affect this process. Next, I will review the concept of clinical change

and all of its variations (e.g. minimal detectable change, clinically important change, etc.).

Rehabilitation decisions are based on assessing and reassessing patients to determine the

effectiveness of treatment. A clear understanding of the properties of our assessment instruments

and how to interpret the results obtained with these instruments is of utmost importance to

clinicians and researchers. Third, I will review the literature pertaining to Patient Reported

Outcomes (PRO), since understanding what the patient values in the rehabilitation process is









critical. Finally, I will provide a summary of this literature review and how it supports the

overall purpose of this dissertation.

Theoretical Framework

Several theoretical models have been developed and explored over the years. In the field

of rehabilitation, there has been a shift from what was previously known as "disabling" models,

to a more proactive theoretical framework termed "enablement models". In rehabilitation

sciences, an "enablement" model frequently used is the International Classification of

Functioning, Disability, and Health (ICF), proposed by the World Health Organization [7]. This

model was used to guide the research questions and analyses for this dissertation.

The ICF provides a theoretical framework for the analysis of health conditions, body

structure and function, activity, and participation, and environmental and personal factors. There

are two main components in the ICF model. The first is concerned with functioning and

disability and includes the areas of body structure, body functions, activity, and participation.

The second part includes the components of contextual factors, and includes environmental and

personal factors (see figure 1.1).

In this dissertation, the health condition of interest includes a variety of disorders and

diseases that influence gait and balance in the elder population and can lead to falls. The ICF

model offers a holistic approach that considers the dynamic interaction of different aspects of a

health condition from a biological, individual, and social perspective. In individuals with gait and

balance disorders, a number of health conditions contribute to a decline in biological function

that can lead to falls. In the ICF model, this is represented at the body function and structure side

of the model (Figure 1.1). This can result in restriction and inability to perform certain activities

and can also influence the individual's social participation. This transition from body function

and structure to activity and participation limitations can also work in the opposite direction. For










example, a decrease in participation and activities can lead to deterioration at the body function

and structure level. In addition, personal and environmental factors are also taken into account by

ICF model to draw a comprehensive picture of how an individual, within his or her limitations,

and conditioned by the environment and his or her own personal characteristics, functions in

society.

This dissertation focuses in two distinct, although directly connected, aspects of the

rehabilitation of individuals with gait and balance disorders. First, two assessment instruments

used in clinical practice to evaluate gait and balance will be studied. These instruments are used

to test individual's ability to perform certain tasks related to gait and balance. Therefore, the first

experiment in this dissertation will be investigating the activity component of the ICF model.

Secondly, a patient reported outcome questionnaire will be used to investigate individual's

expectations and success criteria when participating in a gait and balance intervention. Since the

ICF is not only a theoretical model but also a classification system, using the coding system of

the ICF, a comprehensive picture of an individual's experience when going through a gait and

balance intervention can be drawn. The ICF categories facilitate the description and

classification of all aspects of function and health in individuals, independent of a specific

assessment instrument. The current ICF lists 1,424 categories referring to body functions and

structure, activities and participation, and environmental factors [7]. The ICF model's

classification system was used to produce a patient reported outcome questionnaire to identify

specific areas of the model that are important to individuals with gait and balance problems

including: mobility, self-care, interactions with people, community and social life, energy and

drive, mental function, emotional distress, sensory function, and pain.










Figure 1.1 shows the areas of the model tested in the two experiments included in this

dissertation. The ICF model emphasizes interconnectivity of all the areas of the model. This can

lead to some overlapping, with some concepts being represented in more than one area of the

model. Sensory function and pain, energy and drive, mental function, and emotional distress are

included at the level of body function and structure. Some of these concepts are highly connected

to personal factors and could also be represented in this area of the model. Balance, self care, and

mobility are included under the area of activity. Again, one can argue that some of these

concepts could be included under participation. Finally, interactions with people, and community

and social life have been included under the participation area of the model. Although contextual

factors are not specifically addressed in this dissertation, they must be considered when

interpreting the results of this investigation. A combination of functioning and disability,

together with contextual factors contributes to the ultimate goal of addressing quality of life and

life satisfaction.

Clinical Change

Selecting outcome measures: Health research is based on scientist's ability to measure

the different dimensions of the health construct. In the past few decades, the medical community

has shown an increased interest for the use of evidence-based clinical practice. This interest has

been supported by governing agencies, insurance companies, and consumers. These groups

recognize the need for interventions that are scientifically driven, produce measurable and

meaningful results, and are based on theoretical models that cover the entire spectrum of health.

To this end, many measurement instruments have been developed to assess different

aspects of health. The selection of a suitable instrument generally depends on the goals and

characteristics of the population being assessed. In rehabilitation, most assessment tools aim at

evaluating functional aspects related to health. In general, the relevance of measurement









instruments can be optimized by incorporating a theoretical framework of health and disability,

establishing the purpose of the measurement, and assuring that these instruments are

psychometrically sound.

A theoretical framework of health and disability provides the conceptual basis for

developing and using an assessment instrument. For example, under the International

Classifieation of Function Disability and Health (ICF) [7], a researcher can develop

measurement instruments to assess the areas of body function and structure, activities, and

participation, and the influence of personal and environmental factors [8]. Using a theoretical

model helps researchers identify the domains of interest and provide a complete view of how the

different domains interact with each other to affect the overall health status of the individuals

under study.

Another essential factor in the development and selection of outcome measures is to

establish the purpose of the measurement. In general, assessment instruments are used to

discriminate among individuals, predict future outcomes, and evaluate interventions [9].

Discriminative instruments are used to differentiate between individuals based on specific

criteria when no external gold standard exists for validating these measures [10]. These

instruments are used for investigating between-subj ect differences where groups of individuals

are assigned to separate treatment conditions. For example, a researcher interested in

investigating differences in balance among two groups of elders with Parkinson's disease should

use a discriminative assessment instrument. Predictive instruments are used to categorize

individuals into predetermined category when a gold standard is available [10]. This gold

standard is used to determine whether individuals have been classified correctly. For example, a

shorter version of an instrument can be used to assess a particular condition. Later, the results










can be compared to the original or gold standard instrument. Finally, evaluative instruments are

used to measure the magnitude of longitudinal change in an individual or group (within-subj ects

experimental design) [10]. In this type of investigation, two measurements are obtained from the

same sample. Changes in performance within each participant across treatments are used to

determine a treatment effect.

Measurement instruments used in rehabilitation must have sound psychometric

properties. In classical test theory, these properties assure that the instruments we use can

provide information that is meaningful, valid, and consistent with the construct we are

measuring. The key traditional psychometric properties of an instrument are validity, reliability,

responsiveness to change, and minimal clinically important difference (MCID) [12].

Validity refers to the ability of an instrument to measure what it is intended and presumed

to measure [13]. A valid measure must be reliable (consistent). However, a reliable measure does

not have to be valid. Validity can be investigated and defined in a number of ways. First, it is

possible to correlate measures with a criterion measure known to be valid. This is considered the

criterion validity. If the criterion measure is collected at the same time as the measure being

validated, the concurrent validity is obtained. When the criterion is collected later, the validity

obtained is the predictive validity. A different type of validity is based on the construct of the

instrument been used. Construct validity refers to whether an instrument measures the construct

it is supposed to measure. Finally, content validity, or face validity, is simply the extent to which

a measure represents all facets of a given construct [14].

Instruments used in rehabilitation must also be reliable. A measurement is reliable when

repeatedly testing a particular subj ect under the same conditions produces the same results [15].

A measurement can reliably measure the wrong attribute. Such a measurement will be reliable,









but not valid. There are a number of ways to determine reliability. First, inter-rater reliability is

used to assess the degree to which different raters give consistent measurements of the same

phenomenon. Second, test-retest reliability is used to assess the consistency of a measure from

one time to another. And third, internal consistency reliability is used to assess the consistency of

results across items within an instrument [16].

Responsiveness is also important to determine changes over time, which may be

indicative of therapeutic effects. Instrument responsiveness is the ability of the instrument to

precisely detect meaningful changes [17]. The measurements of an instrument used in

rehabilitation must be able to identify clinically significant differences between and within

patients over time [18]. In addition, the instrument should only be responsive to changes in the

variable being assessed, and should not be influenced by changes in other variables [19]. Related

to the responsiveness of an instrument are the concepts of sensitivity and specifieity. Sensitivity

is the ability of an instrument to detect changes in the variable under study, when they occur. It is

a measure of the probability of correctly identifying a change. Specifieity is the ability of an

instrument to correctly identify when no changes in the variable under study occurs. It is a

measure of the probability of correctly identifying no change [1 l].

Finally, related to the concept of responsiveness, is the ability of an instrument to detect

minimal clinically important differences (MCID). The minimal important difference has been

defined as "the smallest difference in score in the domain of interest which patients perceive as

beneficial and which would mandate, in the absence of troublesome side-effects and excessive

cost, a change in the patient's management" [21]. The concept of clinically important change

introduces a new variable in measurement; the patient' s perspective. In addition, clinically

important change can have consequences at the clinicians, researchers and societal level. Out of










all psychometric properties desirable in any assessment instrument, none has more practical

implications than the ability of an instrument to detect MCID. At the end, the ultimate goal of an

intervention is to produce results that are important to the persons receiving the treatment. An in

depth analysis of MCID follows.

Clinical Important Differences

Interpretation of clinical assessments is often difficult. This is especially true when the

variables measured are based on some abstract construct. For example, interpreting a change in

blood glucose levels of 20 mg/dL is easy, but interpreting a change of 4 points in a Quality of

Life instrument can prove to be a difficult task. Therefore, establishing clinical importance is

more difficult as the concept being assessed becomes more abstract. In addition, clinically

important differences may be different across groups of patients defined by diseases, levels of

severity, cultural background, socioeconomic status, and nationality [22]. Determining clinically

meaningful differences is important because, sometimes, studies are based on small differences

in mean scores between groups, which can lead to a statistically significant difference when the

sample sizes are large. However, statistical significance is not equivalent to clinical significance

[23].

Different perspectives in clinically important differences: There are several important

issues to consider when looking at clinically important changes. First, from the point of view of

the patient, a meaningful change may be one that results in a meaningful reduction in symptoms

or improvement in functional status. On the other hand, clinicians can consider a meaningful

change when the patient' s improvement results in a change in treatment or disease prognosis.

These two perspectives may not coincide all the time. In addition, societal and institutional

perspectives for determining what constitutes a clinically important change can also differ from

the clinicians' and patients' view [23]. From societies' perspective small changes can be









meaningful if the condition of interest affects a large number of people. Institutions may be more

concerned with changes that influence health care policies [23].

Another important issue to consider is whether clinically meaningful changes are based

on individual or group differences. When looking at group differences, it is important to take into

consideration that mean changes do not provide information about individual scores. It is

possible that even in groups with small mean differences a number of subj ects could exhibit

significant changes. On the other hand, these differences could be attributed to measurement

error associated with the measurement instrument. Therefore, when investigating issues related

to public health, where even small differences can have great impact, reporting group differences

is appropriate [22]. Conversely, differences at the individual level are more relevant when

individual decisions about a particular treatment must be made. Furthermore, the amount of

change necessary to be considered clinically meaningful is also influenced by whether it is

applied to an individual or a group. Relatively small improvements at the individual level may be

considered clinically important when looking at the group level [22].

Calculating Clinically Meaningful Change

To date, two broad strategies have been suggested for calculating MCID: 1) anchor-based

measures and 2) distribution based approaches [24]. Anchor-based methods examine the

relationship between an instrument' s measure and an independent measure (or anchor) to explain

the meaning of a particular degree of change. Therefore, anchor-based approaches need an

independent standard or anchor that is itself interpretable and at least moderately correlated with

the instrument being explored [25]. On the other hand, distribution based approaches rely on the

statistical distribution of scores in a given instrument [25].









Anchor-Based Methods

Anchor-based methods have been used to determine clinically meaningful change via

cross-sectional and longitudinal approaches.

Cross-sectional methods

A cross-sectional approach is used when comparing groups that are different in terms of

some disease-related criterion [26]. The difference in mean values across groups is used to

estimate the minimal clinically important difference. For example, in well known medical

conditions where severity stages have been determined, such as in Parkinson's disease (Hoehn

and Yahr scale), a difference equivalent of moving from one stage to the next can be used as

MCID. Cross-sectional methods can also be used to compare individuals with and without a

particular diagnosis. For example, Johnson et al. [27] investigated differences in SF-36 scores in

patients with hypertension. They found that hypertensive patients scored on average 4. 1 points

lower on the SF-36 compared to those without hypertension. They determined that this

difference (4. 1 points) could be used to establish MCID. One disadvantage of this approach is

that generalizing the results to other samples can be misleading because it is difficult to control

for other variables that can cause the group differences. In addition, as with all cross-sectional

designs, differences in mean scores may not accurately reflect true change [25].

MCID can also be inferred by linking the results to some external, non-disease related

criteria. For example, Testa and Simonson [28] suggested that a 0. 1 standard deviation decrease

in the General Perceived Health scale was comparable in importance to the stress associated with

experiencing the death of a close friend. These external criteria can provide a useful, easy to

understand, anchor for comparison; however, interpretation of the results can be difficult in some

cases. Again, as in other anchor-based approaches, there must be an assumption that all other

variables remain stable (do not change).









Another cross-sectional approach consists of dichotomizing patients based on functioning

level after treatment (functional vs. non-functional group). Based on the principle that a patient

should be in the normal range of functioning after clinical intervention, Jacobson and Truax [29]

propose three possibilities for identifying recovery status. First, a patient's score after clinical

intervention is 2 standard deviations better compared to the dysfunctional group. Second, the

patient's score after clinical intervention is within 2 standard deviations of the functional

population. Finally, the patient's score after clinical intervention is closer to the mean of the

functional population than the mean of the dysfunctional population. This method bases the

identification of MCID on the ability of patient to move from one category to the next.

The use of cross-sectional comparisons has received two maj or criticisms. First, it is

likely that when comparing two groups, more than one variable may be responsible for the

differences between groups. Some researchers have suggested the use of regression models to

control for other possible variables [30]. Second, some researchers argue that cross-sectional

differences are not always equal to longitudinal changes using the same groups [31].

Longitudinal methods

This approach is used when comparing group changes across time. One of the most

commonly used anchor-based approaches for establishing clinically meaningful change in

longitudinal studies is the use of global ratings of change.

Jaeschke et al. [32] used this approach to investigate MCID in patients with respiratory

problems. They used the Chronic Respiratory Questionnaire and the Chronic Heart Failure

Questionnaire to address change in dyspnea, fatigue, and emotion. After treatment, patients were

asked about their global rating of change. Based on their responses, they were classified into four

groups: "no change," "minimum," "moderate," and "largest," for each domain. After









investigating the mean change in the three domains, the authors concluded that a difference of

half a point constituted an MCID.

Another longitudinal method for establishing clinically meaningful change involves the

prognosis of future events. This method looks at individuals who experience a particular event

such as mortality, use of medical care, cost of interventions or time to discharge [33].

Differences in individuals that experience and do not experience the event are used to determine

MCID.

A final method for investigating MCID is the use of receiver operating characteristic

(ROC) curve. The ROC curve method attempts to discriminate between patients who do and do

not achieve clinically significant change using a single cutoff point [34]. Sensitivity, probability

that a test result will be positive when the disease is present (true positive rate, expressed as a

percentage), is plotted against specificity, probability that a test result will be negative when the

disease is not present (true negative rate, expressed as a percentage). Each point on the curve

represents a different cutoff. Usually, the point where sensitivity and specificity have the highest

value is chosen as a MCIF cutoff point. Some studies choose the point where sensitivity equals

specificity. Both of these cutoff choices are arbitrary because they do not consider the

differences in importance between false positives, false negatives, and correct identification.

MCID estimates based on arbitrary ROC curve cutoffs could be very different from estimates

from cutoffs that compare correct and incorrect classifications [34].

In general, anchor-based techniques offer the advantage of linking changes in the variable

of interest to outside meaningful anchors. This is particularly useful when investigating results

for which the patient is the maj or source of information, such as in pain research and quality of

life investigations. However, anchor-based approaches have several limitations. When using










global ratings, it is possible to obtain results that are affected by recall bias, especially when the

time delay is long. Patients may simply forget or be affected by their current life situation. In

addition, this method offers no information about the reliability and validity of the responses

obtained. Another potential limitation is the generalizability of results obtained with different

anchors. The use of different anchors may lead to different conclusions about the amount

necessary to determine MCID. Some studies have found that conclusions obtained from anchor-

based methods may vary depending on whether the anchors were obtained prospectively or

retrospectively [35].

Most importantly, anchor-based methods do not take into consideration the precision of

the instrument used. It is possible that MCID established by this method are within the range of

error in the instrument. Any change within this range cannot be attributed to the treatment or

intervention. Furthermore, interpretation of results may be difficult if there is not a linear

relationship between the scores and the anchor chosen [35].

Distribution-Based Approaches

Distribution-based approaches to determine MCID are based on the statistical

characteristics of the obtained sample. Three categories of distribution-based measures have been

proposed [23]. These are: methods based on statistical significance, methods based on sample

variation, and methods based on measurement precision. Methods based on statistical

significance evaluate change taking into consideration the probability that this change occurred

by random variation. These methods are affected by sample size. Therefore, other things being

equal, increasing the sample size may yield results that are statistically significant. Two

approaches that use these methods include the paired t-statistic and growth curve analysis [36].

The second category includes methods based on sample variation. Different types of variation

used include baseline variation of the sample (effect size), variation of change scores










(standardized response mean), and variation of change scores in a stable group (responsiveness

statistic) [37]. These methods are independent of sample size, because variation is expressed as

an average variation around a mean value. The last method is based on the measurement

precision of the instrument. These methods evaluate change in relation to variation of the

instrument instead of variation of the sample. They include the standard error of the mean (SEM)

and the responsiveness statistic. These methods are also sample-independent.

Paired t-Statistic

The t-statistic is used to test the hypothesis that there is no change in the average response

on a measure over two time points. The paired t-statistic has been commonly used in a one-group

repeated measures design [38]. It is calculated as the difference between pre-test and post-test

scores divided by the standard error of measure change [39]. A concern with the use of this

method to measure individual change is the fact that it only accounts for the statistical

significance of the difference. This difference depends not only on the amount of change, but

also on the sample size and the variability of the measure [39]. If used to establish a cutoff point,

increasing the sample size will reduce the amount of difference necessary to reach this threshold.

Statistical significance is not appropriate to establish the clinical importance of a change in score.

Growth curve analysis

Individual growth curve coefficients can be estimated using hierarchical linear modeling

[40]. Improvement rates are calculated by dividing the empirical Bayes estimated linear slope by

the empirical Bayes estimated posterior standard error of the slope [40]. This method, like the t-

statistic, is influenced by sample size. Speer and Greenbaum [41] claim that this method

performs better than other distribution-based methods because it uses all data points to establish

rates of change. They also report that a limitation of this method is the fact that it requires large

samples to provide stable estimates of change [41]. Another limitation of this method is the









assumption of not having any missing data points or that missing data points are randomly

missing. Violations of this assumption can result in biased conclusions [42].

Methods Based on Sample Variation

Effect size

Effect size is a broad name given to a number of indices that measure the magnitude of a

treatment effect. Unlike previously presented significance tests, these indices are independent of

sample size. Cohen [43] defined the standardized difference between two groups as the

difference between the means, divided by the standard deviation of either group. Cohen

concluded that the standard deviation of either group could be used when the variances of the

two groups are homogeneous. He also established guidelines for the interpretation of effect size;

for example, .20 for "small" effects, .50 for "moderate" effects, and .80 for "large" effects. Some

researchers have investigated MCID based on effect size. Samsa and colleagues [22] propose an

effect size of .20 as an appropriate definition of a MCID. Some limitations of using the effect

size include the need for a homogeneous distribution; the size of the standard deviation, either at

base line or after treatment, will have an inverse effect on effect size; and with large standard

deviations producing smaller effect sizes [44].

Standardized Response Mean (SRM)

SRM is defined as mean score change divided by the standard deviation of that score

change [38]. A large SRM indicates that the change is large relative to the background

variability in the measurements [45]. The SRM also uses cutoff points of .20, .50, and .80 to

define small, moderate, and large effect sizes [46]. One limitation of the SRM is that comparable

individual changes that have different SRM values depending on the variability of change in the

sample [45].









Methods Based on Measurement Precision

Standard Error of Mean (SEM)

The SEM is a measure of the precision of an instrument. Therefore it is closely related to

the concept of minimal detectable change. The SEM is the standard error in an observed score

when the true score is not captured by the instrument used. In other words, it indicates how close

a person's score is to their true score; the score that they would get if a test could be completely

error-free [47]. It is calculated by using the sample standard deviation and the sample reliability

coefficient. The exact formula estimates the SEM as the standard deviation of the instrument

multiplied by the square root of one minus its reliability coefficient [47]. The SEM is considered

to be an attribute of the measure and not a characteristic of the sample [47]. It is possible that the

SEM of a particular measure would vary based on the method used to estimate the reliability

coefficient and the presence of extreme scores. For MCID, SEM values of 1 SEM, 1.96 SEM,

and 2.77 SEM have been suggested [48-49]. The SEM is expressed in the original metric of the

measure it describes. This is important because it can help with interpretation of the results.

Moreover, the SEM is a theoretically fixed parameter of a measure [49]. This means that for

nearly all true scores, the deviation around the true score from repeated measurements is about

the same [50].

Reliable Change (RC)

A reliable change index is based on the amount of change that indicates the extent to

which the observed change exceeds measurement error [50]. This index is referred to as the

standard error of measurement difference (SEMD). The SEMD is directly related to the SEM,

but produces smaller values. Therefore, this method is more conservative for a given cutoff value

than the SEM approach, classifying fewer individuals as improved or deteriorated. A cutoff

value of 1.96 has been suggested to determine whether an observed change in scores over time









should be categorized as unchanged, improved, or deteriorated [50]. A disadvantage of this

method is that it assumes that measurement error is constant across the range of possible scores

[50].

Choosing an Appropriate Method

A number of methods have been proposed to investigate MCID. Anchor-based

approaches have the advantage of linking changes to a meaningful external anchor. In addition,

some of these methods include the most important measure of the significance of change; the

patient' s perspective. However, these methods do not consider the possible range of error

associated with all instruments. In addition, interpretability of results is difficult when comparing

investigations that use different external anchors. On the other hand, distribution based

approaches provide a way to establish amount of change outside the limits of the instrument' s

error. In addition, these approaches provide a common metric that has equivalent meaning across

measures, populations, and studies [50]. The distribution-based approaches that are better suited

for establishing MCID are those based on the measurement precision of the instrument (SEM,

RC). These measures establish the amount of error that is inherent to the instrument and the

amount of random error that can be expected in repeated measures. In addition, they are not

influenced by variability in the sample at baseline (as is the effect size), variability of the

observed change (as is the Responsiveness Statistic), or the sample size (as are the t-statistic and

growth curve analysis). Finally, these measures can be used to establish cutoff points based on a

desired confidence level [50].

The first experiment in this dissertation uses the SEM method to investigate differences

in balance scores (Berg Balance Scale and Dynamic Gait Index) that represent a minimal

detectable change. The SEM is a measure of responsiveness and can also imply reliability of an

instrument. The SEM expresses measurement error in the same units of the original tool and is









not influenced by variability among subjects. This technique is well suited for clinical practice. It

provides an intuitive score, in the same unit of the original instrument, which can be used across

measurements and populations. In addition, because the SEM looks at within individual

variability, this technique is more appropriate than other responsiveness approaches when

therapists need to make a decision about an individual [51].

Patient Reported Outcomes (PRO)

Health professionals' approach to patients and their problems is greatly influenced by the

conceptual models around which their knowledge is organized. Traditionally, the effects of a

particular disease or medical condition have been assessed by using methods such as

physiological exams, performance tests, and clinical observations. These methods have been

employed, and still are, to fulfill the requirements of a conceptual model described as the

"Biomedical Model". In this model, a disease or medical condition is the result of a

pathophysiological event, intrinsic to the individual, resulting in a reduction of the individual's

quality of life. As a result, curing or managing a disease or medical condition revolves around

identifying the disease, understanding it, and learning to control and alter its course [52].

"Biopsychosocial Models", such as the one used to guide this dissertation, the ICF, expands the

concept of disease to include not only the pathophysiological event, but also the psychological

and societal consequences of the disease process [53]. With the acceptance of this new

conceptual model, comes a need to develop tools that can effectively measure individual's social

and personal factors that affects the disease process and can play an important role in the

diagnostic and rehabilitation process.

Measuring the impact of social and psychological factors in the disease process requires

participation of the patient. For some medical conditions the patient is the only source of

information. For example, depression is a condition that, often, has no observable or measurable










physical symptoms. Clinicians must rely on information provided by the patient to correctly

diagnose and treat patients with this medical condition. In addition, to establish endpoints in the

rehabilitation of these patients, clinicians must use tools that take into consideration the patient' s

perspective. Patient reported outcomes (PRO) instruments become indispensable in these

situations.

PRO Instruments

PRO has been defined by the Federal Drug Administrationas: "Any report coming

directly from patients (i.e., study subjects) about a health condition and its treatment" [54]. PRO

instruments are used to measure treatment benefits by capturing concepts related to how a patient

feels or functions with respect to his or her health or condition. The ideas, activities, behaviors,

or feelings measured by PRO instruments can be either verifiable in nature, such as walking, or

can be non-observable, known only to the patient, such as pain, depression etc. Although these

symptoms are highly dependent on the patient perception, historically, these assessments were

often made by clinicians who observed and interacted with patients. Recently, these kinds of

assessments are increasingly performed with PRO instruments.

The idea of asking patients about their feelings and symptoms is not new. In fact, doctors

have used this technique throughout the history of the medical profession [55]. What makes PRO

instruments different is the fact that information about symptoms and performance is being

obtained directly from patients. This is done without interpretation from clinicians, using

structured questionnaires that are shown to give reproducible, meaningful, and quantitative

assessments of how patients feel and how they function [56]. Therefore, the adequacy of a PRO

instrument is based on its ability to capture the patient' s evaluation of the impact of disease on

their functioning and well-being [57-58]. For this reason, PRO instruments can be categorized










under a new set of instruments that uses a conceptual framework where the patient is the focus of

the assessment.

PROs and the International Classification of Function (ICF)

The ICF is a thorough framework that can be used as a reference to evaluate, compare

and classify instruments used in the rehabilitation field. In addition, the ICF is a classification

system and can be used to develop new instruments, using the coding system it provides.

Because the ICF was developed to include social, personal and environmental aspects of health

and disability, it is an ideal framework for the evaluation of PRO instruments.

PRO instruments provide information about how patients feel or function with respect to

their health or condition. Information generated by a PRO instrument can serve to measure

treatment benefit, from the patient perspective. To arrive at this conclusion, there must be

evidence that the PRO instrument is based on a valid theoretical construct. In fact, one of the

most important psychometric properties of a tool is its construct validity. Construct validity

refers to the degree to which inferences can legitimately be made from the measures in a study to

the theoretical constructs on which those measures were based [59]. PRO instruments can be

used to measure simple constructs such as the ability to perform activities of daily living or more

complex constructs such as quality of life, which includes physical, psychological and social

components .

The ICF model can be used as a reference to investigate what areas of the rehabilitation

spectrum are covered by PRO instruments. Activities and participation (right side of the model)

are the areas of the ICF model measured by PRO instruments. For example, one of the most

widely used PRO instruments is the SF-36 [60]. This instrument measures generic health status

in the general population. Patients are asked to evaluate their general health and limitations in

activities as a result of their physical health or emotional problems. Although this instrument was









not developed using the ICF as a reference, it covers many areas of the model. Questions such

as: "Does your health now limit you in these activities? If so, how much?" or "During the past 4

weeks, to what extent has your physical health or emotional problems interfered with your

normal social activities with family, friends, neighbors, or groups?" assess the ICF areas of

activity and participation. Although personal factors are not directly assessed with this

instrument, this must be considered when interpreting the results. Individuals' perception of their

mental and physical function is heavily influenced by personal factors such as their personality,

cultural background, and societal roles. Except for measuring the influence of the environment,

the SF-36 covers important psycho-social areas of the ICF model.

Other PRO instruments are specifically designed using the ICF as a theoretical model and

the code it provides as the basis for classification. For example, the PAR-PRO, a measure of

home and community participation, was developed as a broad measure of home and community

involvement for persons with disabilities [60]. This instrument is based on the domains of

activities and participation in the ICF, including learning and applying knowledge,

communication, mobility, self-care, domestic life, interpersonal interactions and relationships,

maj or life areas, and community social and civic life. A particular challenge of the ICF is to be

able to distinguish between activities and participation when using the coding in the activities

and participation domains. Often, the only possible indicator of participation is coding through

performance, which might also have to be coded as frequency of participation in a particular

activity [60]. However, instruments such as the PAR-PRO demonstrate the potential for using

the extensive classification system of the ICF for the development of PRO instruments. In this

dissertation, a questionnaire to assess the patients' perspective was developed using the ICF









classification system to assess areas of the model of particular importance to individuals

undergoing gait and balance rehabilitation related to falls.

Advantages of Using PROs

Many chronic conditions have a debilitating effect that progressively deteriorates the

patients' quality of life. Chronic diseases have social, personal and mental implications that can

lead to fatigue, depression, pain, and isolation. Traditional measurement tools (e.g. blood

pressure instruments, or blood sugar level counts), are very accurate at measuring physiological

function. However, a new set of sensitive and well-validated tools are needed to improve and

standardize measurements of symptoms related to social, personal, and environmental factors

associated with health. In general, the use of the patient' s perspective becomes more important as

the variable being measured becomes more abstract. Concepts that do not have a well define

physiological component such as pain, quality of life, life satisfaction, or self-efficacy, can be

better understood by considering the patients' perspective.

PRO instruments are ideal and, sometimes, the only way to measure ICF concepts of

activity, participation and the influence of personal and environmental factors in health and

disability. For example, when measuring quality of life (QOL), a concept that is highly

individual and context dependent, we have no choice but to use PRO instruments. In fact, some

researchers and clinicians use these two terms indistinctively [61]. QOL or health related quality

of life (HQRL) questionnaires are PRO instruments that explicitly include the patient' s

perception of the broad impact of disease on their functioning and overall wellbeing [62].

Ultimately, the goal of a therapeutic intervention is to impact health and increase function and

quality of life. Consequently, the use of PRO instruments in clinical practice to determine end

points and evaluate treatment effectiveness is critical. This has been recognized by many national

and international agencies (FDA [54], NIH [63], European Agency for the Evaluation of Medical









Products [10], WHO [65]) that identify the need to use PRO instruments to improve health and

quality of health care.

There is enough evidence that support the use of PRO instruments, especially those that

assess QOL, as a treatment outcome. In fact, QOL has been a strong prognostic variable for

survival in several cancer related studies [66-67]. QOL data can be especially important when

two treatment options with similar survival outcomes are available. In these cases, QOL

outcomes can be the deciding factor for choosing a particular treatment. For example, a woman

with breast cancer might face a decision of whether to opt for a mastectomy or conservation of

the breast. Both treatment options produce similar survival outcomes, but each has implications

for QOL that will determine the woman's ultimate decision. For some women avoiding the

radiation therapy required to conserve the breast is important. For others, conservation of the

breast is of most importance. Even when survival outcomes are different, some patients may

select a less effective treatment because of the effect the treatment may have on their QOL [68].

It is clear that the availability of QOL data is essential for making a balanced and informed

decision about treatment options.

PRO instrument can also play a role in the diagnose and treatment effect of health issues

related to the ICF domain of body function. Some symptoms and treatment effects are not

measurable and only known to the patient. Concepts such as pain intensity or pain relieve are not

observable and have no direct physical manifestation. Again, PRO instruments are needed to

assess these areas. This is important because, sometimes, improvements in a particular clinical

measurement may not correlate with how the patient functions or feels. For instance, a patient

can demonstrate an improvement in a test of muscle strength, but this may not correlate with

improvements in walking or impact the patient' s ability to perform daily activities.










The pain literature provides valuable information on the importance of patient' s

perceptions and perspectives. Hodgkins and Daltroy [69] investigated the assessment of pain by

physicians and patients. They found that physicians' rating of pain is generally lower than that of

the patients. In addition, male physicians tended to rate female patients' pain lower than that of

male patients. The patients' perception of their pain can also be variable. When distracted, a

patient may provide a lower rating of their pain than when they focus on the pain sensation.

Inconsistency in how the patients evaluate a particular condition does not change the value of the

information obtained when using PRO instruments. The patients' experience is what the patient

says it is at any given time [69]. This information is valuable for diagnosing and treating the

condition. It can lead to a better understanding of the nature of the experience and how patients'

personal factors can affect their perception.

PRO instruments are often developed to measure what patients want and expect from their

treatment and what is most important to them [54]. The patient' s perspective is critical to

evaluate treatment effect and patient satisfaction with treatment. Ideally, a treatment intervention

or health strategy should be aimed at addressing all aspects of the construct of health. PRO

instruments allow clinicians and researchers to assess an area of the model that was previously

not well understood because of its abstract nature. Adding these instruments to physiological

clinical measures will help to obtain a more complete picture of the patient' s health and how it is

affected by their personal and societal circumstances.

As previously mentioned, seeking information from the patients about their health

condition and how it affects their function and participation is not new. However, PRO

instruments provide a formal assessment that may be more reliable than the traditionally used

informal patient interview. PRO instruments use a predetermined format to minimize









measurement error and ensure consistency [54]. Instruments can be self-reported or clinician

administered. In the first case, PRO instruments avoid possible clinicians' bias and offer an

unfiltered response that reflects more closely how the patients rate their health. Well-developed

and adequately validated PRO instruments have been shown to provide information that matches

the results obtained by experts in the particular field of interest. In fact, often, this is the method

used to study the validity of PRO instruments [54].

Issues and Concerns about the Use of PRO

The use of PRO instruments to evaluate health and disability is widely accepted in the

medical and research community [70]. They add value to traditional clinical assessments and

offer a unique perspective of the patient experience. There are few disadvantages for the use of

PRO instruments; however several methodological, theoretical, and practical considerations must

be critically reviewed to ensure the information obtained is accurate and useful.

One maj or concern with the use of PRO is that of definition. PRO instruments are

commonly used to assess concepts of activity, participation, and social and personal interactions.

These concepts are abstract in nature; therefore generalization of Eindings obtained with the use

of PRO instruments must be done with caution. For example, the concept of quality of life

(QOL) is receiving increased attention in the clinical and research arena. QOL is an abstract

concept for which it is impossible to create a single instrument that assesses it. Researchers have

attempted to solve this issue by using a narrower definition of QOL. The idea of health related

quality of life comes from these efforts of having a more precise definition of the concept being

measured. Another approach to this issue is to create specific instruments that relate to a

particular health condition. This serves a double purpose because it also helps to reduce the

ceiling and floor effect experienced when instruments are too general.










In the past few decades researchers have adopted this strategy and developed a number of

instruments aiming at particular health conditions. The list of instruments is so extensive that

some organizations, realizing the difficulty of having access to these instruments, have created

databases where clinicians and researchers can search using criteria such as disease, patient

population, or type of instruments. For example, the ProQolid database, developed by Mapi

Research Institute, contains over 500 instruments and aims at identifying and describing PRO

and QOL instruments to help researchers and clinicians choose appropriate instruments and

facilitate access to them.

Another concern relates to the administration of PRO instruments. The use of self or

clinician administered questionnaires is difficult in certain patients with communication and

cognitive impairments. For example, patients with stroke often have speech or cognitive deficits

that make the use of PRO instruments difficult. Similar difficulties are encountered when using

these instruments with those who have a low education level, do not speak the language, or come

from a different cultural background. Of particular interest is the issue of cultural relevance,

especially when people from different cultural backgrounds are compared using the same

instrument. PRO instruments require an internal evaluation of several aspects of one's life and

how these aspects are influenced by the health condition of interest. These values are influenced

by the patient' s culture and previous experiences. For example, in the pain literature there is

enough evidence to support differences in the pain experience based on ethnic, social, gender,

and geographical factors [71].

Another issue to consider is the psychometric characteristics of some PRO measures.

There are concerns in the research literature about some PRO measures being inadequately

conceptualized, lacking psychometric rigor, and having inconsistently applied psychometric









methods [72]. To solve this issue, some national and international organizations have released

guidelines for the creation and evaluation of PRO instruments. Recently, the Scientific Advisory

Committee (SAC) of the Medical Outcomes Trust created a document to guide in the evaluation

of PRO instruments. In this guide the SAC states that PROs should be evaluated on the following

seven dimensions; 1) the use of pre-specified conceptual and measurement models; 2) the

strength of empirical support for the reliability and validity of the scale(s); 3) the responsiveness

of PRO to clinical change; 4) the methods) for interpreting scores; 5) the level of respondent and

administrative burden; 6) the equivalence of alternative forms of administration; and 7) the rigor

with which translations are adapted for use in specific cultural contexts [73]. This comprehensive

list is useful for researchers interested in evaluating PRO instruments, but may have little use for

clinicians. There is a need to create a way of translating these guidelines efforts into clinical

practice.

Patient reported outcomes (PROs) are a necessary and valid way of including the

patients' perspective into research and clinical practice. Using the International Classification of

Functioning, Disability, and Health (ICF) model clinicians and researchers can evaluate existing

PRO instruments, and propose new instruments to assess particular areas of the model. PRO

instruments must be carefully selected to meet the needs of the specific population of interest. No

single instrument has universal application in health assessment.

There are some concerns about the psychometric properties of PRO instruments. These

instruments are often used to explore abstract concepts that are difficult to interpret and

conceptualize. Assigning a single score value to broad concepts such as quality of life may be an

oversimplification of a very complex component of life. However, researchers have found

strategies to minimize this problem. Using instruments that are disease and population specific,










based on sound theoretical concepts, reliable, responsive to clinically important change, and

culturally sensitive may ameliorate some of the concerns expressed by members of the research

community .

PRO measurements, such as participation, satisfaction with care, and quality of life are

increasingly being required by accreditation agencies, patients, policy-makers, and medical

insurers as quality indicators of best practice [73]. These groups recognize the need to empower

patients by giving them the opportunity to include their perspective in the selection of treatments

and evaluation of treatment outcomes. To my knowledge, the second study in this dissertation is

an attempt at using an ICF-based PRO instrument to assess expectations and patient satisfaction

in the rehabilitation of falls.

Summary

The number of people 65 and older is likely to continue rising in the future. Falls are

already a maj or health concern in this population and will likely continue to be a burden to elders

and the health system. To offer the best possible, most efficient medical care to the increasing

number of elders, clinicians should use the best assessment instruments available and include

evidence base information in their clinical practice. In addition, to provide patients the treatment

they deserve, clinicians need to understand the patient' s expectations and goals. Patient

satisfaction is the ultimate goal in any service provider, including medical services.

In research, most rehabilitation interventions are evaluated by using some test of

statistical significance. Assessment instruments used during these interventions provide a

measure of performance that can be used to assess disease severity or monitor improvement.

Tests of statistical significance offer important information about groups of patients, but fail to

capture differences at the individual level. In recent times, a number of investigators are looking

at other statistical methods that can provide information about clinically relevant change [29, 41,









47, 50]. This dissertation is a first attempt at investigating clinically meaningful changes in

measures of gait and balance used to evaluate geriatric patients who have fallen or are at risk of

falling.

In addition, a new set of instruments are also receiving a great deal of attention from the

research community. Recognizing the need to empower patients and understand their personal

needs, patient reported outcomes are been included in clinical trials to assess the relevance of the

intervention, from the patient' s perspective. In fact, during the inaugural ceremony of the new

NIH PRO initiative Director Elias A. Zerhouni said 'There is a pressing need to better quantify

clinically important symptoms and outcomes that are now difficult to measure. Clinical measures

of outcome such as x rays and lab tests have minimal relevance to the day-to-day functioning of

patients with such chronic diseases as arthritis, multiple sclerosis, and asthma, as well as chronic

pain conditions' (NIH, 2005). This dissertation is a first attempt at using a PRO instrument to

assess expectations and success criteria, from the patient' s perspective, of a rehabilitation

program for patients with gait and balance problems who have fallen or are at risk of falling.










Health condition

(disorder or d sease)


Interactions
with people,
Community
& Social life.


Figure 1-1. International Classification of Functioning Disability and Health (ICF) model









CHAPTER 2
ESTABLISHING THE MINIMUM DETECTABLE CHANGE (MDC) FOR THE BERG
BALANCE SCALE AND DYNAMIC GAIT INDEX

Introduction

Most rehabilitation efforts to treat falls include a component of physical rehabilitation.

These treatment plants are based on assessments performed with instruments specifically

developed to obtain information related to the condition of interest. Therapists assess and

reassess patients to identify specific problem areas and establish improvement criteria. Two

assessment tools commonly used in physical therapy, and specifically in the evaluation of elder

individuals who have fallen or are at risk of falling, are the Berg Balance Scale (BB S) and the

Dynamic Gait index (DGI) [74].

Gait and Balance Assessment

The Berg Balance Scale (BBS)

The BBS is a frequently used performance-based scale that assesses postural balance [75].

The test consists of 14 commonly used tasks: Sitting to standing, standing unsupported, sitting

unsupported, standing to sitting, transfers, standing with eyes closed, standing with feet together,

reaching forward with outstretched arm, retrieving an obj ect from floor, turning to look behind,

turning 3600, placing alternate foot on stool, standing with 1 foot in front, and standing on 1 foot.

The scoring method is based on a 5-point ordinal scale of 0 (indicates the lowest level of

function) to 4 (indicates the highest level of function), with the total score ranging from 0 to 56.

The BBS was specifically designed to be used at the clinic. It requires minimal equipment

(stopwatch, chair, stool and ruler) and can be applied in under 15 minutes.

A considerable amount of evidence suggests that the BBS is a valid measure of standing

balance. Initially, Berg et al. [75] correlated BBS scores with a general rating of balance made by

therapist (Pearson r=.81). Other studies by the same author have also demonstrated high









correlation values between the BBS and other measures of balance. For instance, the Pearson r

correlations between the BBS and the balance sub scale of the Tinetti Performance-Oriented

Mobility Assessment and the Barthel Index mobility subscale were .91 and .67 respectively [76].

Other researchers have also found high correlations between BB S scores and other motor and

functional measurements: Fugl-Meyer Test motor and balance subscales (Pearson r=.62-.94),

Timed Up & Go Test (TUG) scores (Pearson r=-.76), Emory Functional Ambulation Profile

(Pearson r--.60), and Gait Speed (Pearson r- .81) [77- 78]. The BBS scores also correlated

moderately with data obtained for the Dynamic Gait Index (Spearman coefficient=.67), and

center-of-pressure measures (-.40 to -.67 [Kendall coefficient of variance]) [78].

Several studies have also reported high intra- and inter-rater reliability for the BBS. Berg et

al. [79] used videotaped evaluations of the BB S to obtain inter-rater reliability (ICC= .98 for

total BBS scores). The same researchers replicated these results in a test-retest format, producing

a within rater ICC= .97 and between rater ICC=.98. The large maj ority of studies investigating

reliability of the BB S have used some form of correlation coefficient such as the Pearson's

Product-Moment Correlation Coefficient (r) and the intra-class correlation coefficient [ICC],

with the latest becoming more popular in recent times. A fundamental problem of ratio indexes

such as the ICC is that the error of measurement and true variability are expressed in relative

terms. An ICC score is a ratio of within subj ect and between subj ect variability. Thus, the range

of genuine differences in any attribute is sample dependent. Therefore, previously reported high

ICC values for the BBS must be considered with caution.

The Dynamic Gait Index (DGI)

The DGI was developed by Shumway-Cook and Woollacott [80] to assess balance in the

older adult at risk for falling. This functional gait scale consists of 8 common gait tasks: walking

at different speeds on a level surface, walking with horizontal and vertical head turns, ambulating









over and around obstacles, ascending and descending stairs, and making quick turns. Each item

is scored on a 4-level ordinal scale, where 3= "normal", 2= minimal impairment, 1= moderate

impairment and 0 = severe impairment. The maximum possible score is 24 pints. The DGI can

be administered in 10 minutes and requires minimal equipment.

The psychometric properties of the DGI have not been extensively investigated. Validity

of the scale has been supported by moderate correlation with the BBS (Spearman rank order

correlation, r = 0.71), [81]. Sensitivity and specificity to identify individuals with a history of

falls has been established at 59% and 64% respectively [82]. The test developers investigated the

inter-rater and test-retest reliability of the scale using a small sample of 5 older adults and 5

raters. They found ICC values of .96 (inter-rater) and .98 when subjects were re-tested a week

later by 2 therapists. Intra-rater reliability has not been reported in the literature. Despite being

widely used in the clinic, the psychometric properties of the DGI have not been investigated

suffi ci ently.

The BBS and DGI are used to assess different dimensions of balance (i.e. static and

dynamic balance). Since the assessment of balance control is a crucial step to identify individuals

who are at risk of falling, these two instruments are often the central components of the physical

therapy evaluation. Scores from these instruments are regularly used to determine a particular

treatment and to monitor improvement. Moreover, low scores in these instruments have been

associated with an increased risk of falling in the elder population (i.e., a BBS score of <45 and

DGI of <19) [76, 82-84]. Therapists use these cut-off points as a reference to guide treatment and

monitor progress. Cut-off scores are taken as absolute values to determine the success of a

particular intervention and are often used to report patient progress. For example, a patient who

improves from an initial BBS score of 40 to a final score of 46 can be reported as being outside









of the range associated with a high risk of falling. This anchor-based method must be used with

caution, because assessment instruments are not always responsive enough to detect small

changes in performance, and these changes can be masked by the multiple sources of error

associated with the instrument.

Unfortunately, when results differ from one assessment to the next, it cannot be assumed

that true change has occurred; some or all of the change could be attributed to measurement

error. Error can be inherent to the test used or represent the naturally existing fluctuation in

patients' performance. The amount of error across measurements of the same test is related to the

reliability of the test. Reliability is the ability of a particular test to consistently provide the same

value when no change has occurred [85]. It is also a measurement of the obj activity of the test.

There are several statistical methods that have been used to measure reliability. They can be

divided in two maj or groups: measures of relative and absolute reliability. Relative reliability

refers to the degree of association between repeated measurements. In other words, relative

reliability measures the strength of the correlation between repeated measures. It takes into

account the total group variability (between subj ect/measurement) and the individual

measurements variability (within subj ect/measurement) to obtain a correlation coeffcient, for

example, the Pearson correlation coeffcient or the Intra-class correlation coeffcient (ICC) [86].

Absolute reliability refers to the variability of the scores from measurement to measurement

(within subject/measurement). This approach does not take into account the range of individual

scores and is not sample-dependent [87]. Some of the tools used to calculate absolute reliability

include: the coeffcient of variation (CV) and the standard error of the measurement (SEM), [88-

91].









While correlation methods used to calculate relative reliability are excellent sources of

information to compare groups of patients, the SEM is more appropriate for clinical practice, that

is, when making decisions about individual patients [90]. However, to date, only one published

study has used a measurement of absolute reliability to look at the psychometric properties of the

BBS and no published study has addressed this issue with the DGI. Stevenson [91] used the

SEM to investigate error associated with the use of the BBS in stroke patients. He found a SEM

(in BB S units) of 2.49 in patients with stroke receiving inpatient rehabilitation. In addition, he

calculated a confidence interval around the SEM and found that a change of 6 BBS points was

needed to be 90% confident of genuine change. This finding is somewhat surprising and

questions previously reported high BBS reliability scores. Further investigation in different

populations is needed.

The Intraclass Correlation Coefficient (ICC), the Standard Error of Measurement (SEM)
and the Minimal Detectable Change (MDC)

The ICC, or intraclass correlation coefficient, is the most commonly reported reliability

measure in the literature. The ICC provides information about the measure's ability to

differentiate among subj ects. The ICC, as with other correlation coefficients, provides values

between 1.0 and -1.0, with high absolute values indicating less variability between scores. The

ICC incorporates total variability (between subj ect/measurement), and error associated with it,

and the individual variability (within subj ect/measurement) to obtain a ratio. This technique is

most appropriate for investigating differences between groups of patients.

A less frequently used reliability index is the standard error of measure (SEM). While the

ICC expresses the proportion of variance of an observation due to between-subj ect variability in

the true scores, the SEM is a measure of within subj ect variability. The SEM expresses

measurement error in the same unit of the original tool and is not influenced by variability










among subj ects. The SEM is closely related to the concept of minimal detectable change (MDC)

expressed by Stratford et al [92]. The MDC is the amount of change in a given measure that must

be obtained for a clinician to determine that true change has occurred. The MDC is expressed as

a confidence interval around the SEM, indicating the values that are within the range of error

attributable to the measuring instrument. The MDC provides the clinician useful and easy to

understand criterion for change in patients' performance.

Therefore, investigations are needed to determine the amount of change in the Berg

Balance Scale and Dynamic Gait Index necessary for a therapist to conclude that "true" change

has occurred (MDC). In addition, understanding how MDC values change at different score

levels of the BBS and DGI (cut-off points, <45 and <19 respectively) may prove to be valuable

for clinicians that are faced with making treatment decisions based on these values.

Thus, the purpose of this study was to use the standard error of measurement (SEM) to

investigate the Minimal Detectable Change associated with the Berg Balance Scale and Dynamic

Gait Index. In this study, I attempted to improve on previous reliability investigations by

providing clinicians with estimates of measurement error that are easy to interpret and can be

used to make clinical decisions.

Methods

Subj ects

The sample consisted of 42 subj ects (26 Males and 16 Females, age 55 and older)

participating in a larger, funded, research study looking at the link between smoking and

recovery from frailty in older Floridians. This study was supported by a grant from the Florida

Department of Health and received approval of the Institutional Review Board for the University

of Florida and the Research and Development Committee at the North Florida/South Georgia

VA Medical Center. Inclusion criteria included: community dwellers with a history of falling









twice in the past 12 months, the ability to walk 20 feet (with or without an assistive device), and

a score of 24 or higher on the Mini-Mental State Exam [97].

Participants were recruited in two different ways. First, patients from a Gait and Balance

disorders clinic at the North Florida/South Georgia VA Medical Center were approached by their

therapists and asked if they were interested in participating in a research study. If the patients

agreed, a research coordinator explained the study and, if still interested, consented the patients.

Secondly, letters were sent to doctor's offices explaining the study and flyers were distributed

throughout the community. Participants responded to the advertisement and, if they met the

study's inclusion criteria, were enrolled in the study.

Testing Procedure

Enrolled participants were tested with an extensive battery of tools including: Berg

Balance Scale (BBS), Dynamic Gait Index (DGI), isometric lower extremity strength, physical

function domain of the MOS36, Falls Efficacy Scale, Geriatric Depression Score, pain

experience VAS, timed tests of gait, Frenchay IADL scale, Pleasant Event Schedule, and

spontaneous self-selected gross motor activity as measured by accelerometry. After the initial

assessment, a home exercise program was prescribed. Participants were instructed to keep

exercise logs to record adherence to the program. Participants were monitored for 3 months.

During this time, a total of 3 evaluations were performed at 4, 8 and 12 weeks. In each of these

evaluations, participants received the previously mentioned battery of assessment tools. In

addition, exercise logs were collected.

Evaluations were conducted by two experienced physical therapists specialized in gait and

balance disorders in the elder population (>7 years experience in geriatric Physical Therapy).

From each evaluation, the BBS and DGI tests were videotaped by a research assistant with a

Sony DCR-VX2100 digital camcorder. Videotapes were converted to DVD format for later









view. Recorded sessions were re-scored at a later time (time between initial and re-scores > two

weeks) by the same therapists. Therapists used a TV screen or computer monitor to view the

recorded evaluations. During the re-scoring of the videotaped evaluations, therapists were

allowed to pause, play in slow motion, and/or replay any portion of the evaluation they were

unsure about. Therapists were blinded to previous score and whether the recordings were from

an initial, 4 weeks, 8 weeks, or 12 weeks evaluation. For the purpose of this study, only initial

evaluations were used. All participants were assessed and re-assessed by the same therapist. Data

was recorded by the physical therapists and research assistant. All data was later entered into a

central database.

Analysis

All statistical analysis and graphical representations were performed with SPSS 13.0

software for Windows (SPSS Inc., Chicago, IL, USA) and Microsoft Office Excel software for

Windows (Microsoft Corporation, Redmond, Washington, USA).

Box plots were used to investigate the presence of outliers in the data (figures 2-1, 2-2).

The distribution of the absolute differences between tests (initial BBS and DGI and re-scored

BBS and DGI) was plotted. Cases with values between 1.5 and 3 box lengths (interquartile

range) from the upper or lower edge of the box were consider mild outliers. Cases with values

more than 3 box lengths from the upper or lower edge of the box were considered extreme

outliers. For the BBS data, 3 outliers were identified. The DGI data did not present any outliers.

Since no extreme outliers were observed in either data-set, all scores were considered valid for

subsequent analysis.

The procedure suggested by Stratford [86, 92] was used to calculate the standard error of

measurement (SEM), also referred as the absolute reliability. The equation for the SEM is:










SEM= SD 1-ICC where SD= sample standard deviation, and ICC= intraclass correlation

coefficient. However, Stratford stated that SEM can also be calculated from the square root of

the mean square error term in a repeated measure ANOVA. In addition, the SEM was used to

calculate the Minimal Detectable Change (MDC). The MDC is the product of the SEM, the

tabled z-score for a desired confidence interval and the -92. The -92 term acknowledges two

measurements are being compared. For a 95% confidence interval the MDC= SEM 1.96* -\2

(1.96= Z-value associated with a two-sided 95% confidence interval). Confidence intervals were

also calculated with a 90% and 80% (MDC (90%) = SEM 1.645* -\2, and MDC (80%)= SEM

* 1.28* -\2). The use of a parametric test (ANOVA) requires that the data meet the normality

assumption. Normality was visually explored with Normal Q-Q plots and tested with the

Kolmogorov-Smirnov normality test. In addition, the use of the SEM, because it assumes a

normal distribution of error, requires that the measurement error is not related to the magnitude

of the measured variable. This is referred as heteroscedasticity. Heteroscedastic data shows a

relationship between the amount of measurement error and the magnitude of the measurement.

Heteroscedasticity was formally examined by plotting the absolute differences between initial

value and re-scored value, against the mean score. Additionally, Spearman's rho correlation was

used to rule out a relationship between each individual's absolute score difference and his or her

mean.

The SEM and MDC procedures described above were also used to investigate the amount

of error associated with individuals at different levels of the BBS and DGI rating scale. Because

the "true" score in these two assessments is unknown, the mean value between the initial and re-

scored values of the BBS and DGI was used to dichotomize the participants in two groups.










Commonly used cut-off points (<45 for the BBS and <19 for the DGI) were used to form the

groups. The SEM and MDC were calculated for the four resulting groups.

Finally, a correlation analysis was performed to investigate a possible relationship between

individual's difference (BB S initial BB S re-scored and DGI initial -DGI re-scored) and

absolute difference in the BBS and the DGI. To acknowledge the ordinal nature of the BBS and

DGI, a non-parametric Spearman's rho correlation analysis was selected.

Results

A total of 42 participants were assessed with the BBS and the DGI. The average age was

75.6 years (range 59 to 88 years). The ratio of males to females was 26 males (62 %) and 16

females (3 8%). The participant' s mean initial BBS score was 40.7 points (SD=7.3, range 18-53).

The re-scored mean value was 41.8 points (SD=7.5, range 24-55). For the DGI, the mean initial

value was 13.4 (SD=4.2, range 3-21), and the re-scored mean was 13.1 (SD=4.3, range 4-22). A

distribution of the absolute difference between initial and re-scored values is found in table 2. 1

for the BBS and table 2.2 for the DGI. The mean absolute difference was 2.57 (SD=2.4, range 0-

11) for the BBS, and 1.29 (SD=.99, range 0-3).

The distribution of absolute values of the difference between initial BBS and re-scored

BBS was investigated with a box plot (Figure 2.1). For the BBS, three participants' scores were

identified as outliers. Their absolute values were 8, 11, and 6 respectively. These scores were

considered mild outliers because their values laid between 1.5 times and 3.0 times the

interquartile range below the first quartile or above the third quartile. Therefore these 3 scores

were included in all subsequent analysis. The distribution of absolute values of the difference

between initial DGI and re-scored scores for the DGI was also explored with a Box plot (Figure

2.2). No outliers were identified in this case.









The absolute differences in the BBS ranged from 0 to 11 points (Table 2.1i). Fifty seven

percent of the participants had a BBS absolute difference of 2 BBS points or less. The mean

absolute difference was 2.45 BBS points. A graphical representation of the distribution of score

differences between initial and re-scored BBS is presented in Eigure 2.3. For the DGI, the

absolute differences in scores ranged from 0 to 3. Seventy four percent of participants had a

difference in score of 1 or less DGI points. The mean absolute difference was 1.13 DGI points.

Figure 2.4 shows a graphical representation of the differences in DGI score between the 2 testing

scenarios.

The distribution of scores from the BBS and DGI were visually inspected for normality

using normal Q-Q plots. Both distributions showed no significant departure of the data from

normality. Formal analysis of the data with a Kolmogorov-Smirnov test showed no significant

departures from normality (D42= .143, p>.05, D42= .09, p>.05, for the BBS and DGI

respectively).

Visual inspection of the distribution of absolute values of the difference between the two

test conditions plotted against their mean, show that the data was reasonably homoscedastic for

both the BB S (figure 2-3) and DGI (Higure 2-4). Spearman' s rho correlations (BB S, rs = -.09

p>.05, and DGI, rs = .02 p>.05) confirmed the lack of relationship between the mean score and

the difference in scores (initial minus re-scored values).

Repeated measures ANOVA were performed to calculate the mean square error term, as

described by Stratford [86, 92]. The BBS provided a mean square error of 5.2 and the DGI 1.1.

For a 95% confidence interval, the minimal detectable change (MDC) is -\5.2 2.77= 6.4 for the

BBS and -\1.1* 2.77= 2.95 for the DGI. Therefore, a change greater than 6.4 BBS points and 2.9

DGI points is necessary to reveal a change that exceeds the measurement error associated with









this instruments and show "genuine" change. Additional confidence intervals of 90% and 80%

were performed for the BBS and DGI (table 2.3). This table also includes BBS MDC results

when dividing the participants in functional groups (above and below 45 BBS points). This

grouping resulted in 30 cases classified as low function (>45) and 12 cases classified as high

function (<45). The MDC95% WAS 7.3 BBS points for the low function group, and 6.3 BBS points

for the high function group.

Additional correlation analysis was performed to investigate a possible relationship

between absolute change in the BBS and DGI. A Spearman' s rho correlation value of rs = .15

p>.05 suggest that there was no relationship between participants' score differences in both

instruments. Therefore, differences in scores in the BBS (initial BBS minus re-scored BBS) were

not correlated with differences in the DGI (initial DGI minus re-scored DGI).

Discussion

The Berg Balance Scale and the Dynamic Gait index are two instruments widely used in

clinical practice to measure individual's gait and balance ability, and monitor improvement in

these areas. High reliability values have been reported for both instruments [79, 82]. However,

previous investigations have used a form of correlation, such as the Pearson's product-moment

correlation or the Intraclass Correlation Coefficient (ICC), to investigate the reliability of these

instruments. While correlational investigations are suitable for investigating the degree of

agreement between groups of subj ects in repeated measures, they offer little information about

the amount of change an individual needs to achieve "genuine" change. That is, the amount of

change beyond the error associated with the instrument.

Absolute reliability is a more appropriate way of investigating the reliability of an

instrument intended for use in a clinical setting, where clinicians are more concerned about

individual change. In this investigation, the results of the absolute reliability of the Berg Balance









Scale and the Dynamic Gait Index indicate that 6.4 and 2.9 points respectively are required to be

95% confident genuine change has occur between 2 testing occasions. This information is

valuable for clinicians that can apply these numbers, at the individual level, to assess

improvement in function over time.

Although the 95% confident level is widely accepted in the research community, one could

argue that, in clinical practice, a lower confidence level could be of practical use to make

appropriate clinical decisions. In this investigation, confidence levels of 90% and 80% were

calculated. The BBS showed MDC90% Of 5.4 and MDCsosn of 4.2 points. The DGI presented

MI/DC90% Of 2.5 and MDCsosn of 1.9 points. Even at the lowest confidence level both instruments

demonstrated an estimated amount of error that should be considered when making clinical

judgments. It is worth noting that, in this investigation, the DGI demonstrated half the amount of

estimated error as compared to the BBS. However, this comparison does not take into account

the range of values of both instruments. That is, the MDC of 6.4 points in the BB S is equal to

1 1.5% of the total possible score of the BB S (56 points) while the MDC of 2.9 DGI points is

equal to 12% of the total possible score for the DGI (24 points). Therefore, based on these

results, both instruments present similar amounts of variability between the two testing

occasions.

This investigation is not without limitations. The main disadvantage of using a

distribution-based approach to assess change is that the results offer no indication of the

importance of this change. That is, minimal detectable change (MDC) is not equal to minimally

important change (MIC). In fact, is possible that MIC change is smaller than MID. In this case,

the instrument would not be able to detect the desired change, since the MIC would be smaller

then the measurement error of the instrument. Identifying when genuine change (beyond









measurement error) occurs is a necessary but incomplete step in the process of judging the

importance of an observed clinical change.

Distribution based approaches such as the standard error of measurement (SEM) used in

this investigation, assumes that the measurement error is constant across the range of possible

sores. In this investigation, individuals were dichotomized into two functional groups to

investigate the possible fluctuation of the SEM at two different levels of the scale. For the Berg

Balance Scale, individuals with lower performance (BB S=<45) demonstrated higher SEM

values. Their MDC95% WAS 7.3 BBS points. In contrast, participants with higher performance

scores (BBS=>45) showed lower SEM values. Their MDC95% WAS 6.3 BBS points. However,

dichotomizing the initial pool of participants reduced the sample size of the groups. Therefore,

these results should be interpreted with caution. Further investigation is needed to substantiate

the possibility of different MDC levels based on performance. The same approach was not used

with scores from the DGI, because dichotomizing this group resulted in a sample size for the

high level group of only 10 participants.

A methodological issue worth considering is the use of videotaped evaluations to establish

the reliability of an instrument, especially when the scores of the initial live evaluation are

compared with scores obtained by evaluating videotaped performances. This method has been

widely used and published. In fact, the initial reliability study conducted by Berg et al [79] used

videotaped assessments to investigate the intra-rater reliability of the BB S. A clear disadvantage

of this design is that it does not take into account the natural fluctuation of the participant' s

performance when tested in two separate occasions. Therefore, clinical decisions based on this

must be made with caution, since not all sources of error are considered. However, in a recent

publication Stevenson [91] found a MDC95% Value Of 6.9 BBS points when assessing stroke










patients in a test re-test design. The fact that Stevenson's results are comparable to what was

found in the present experiment, suggest that the variation seen in both experiments is mostly

due to the instrument's reliability and not within patient reliability. Interestingly, Stevenson' s test

re-test experiment used the best performance of 3 trials as their value for each item. With this

approach, it seems plausible to conclude that the "true" score is more easily captured, and the

within subj ect variability decreased. In addition, Stevenson used the data reported by Berg and

colleagues [93] in their reliability study to calculate the MDC95%. He found a MDC95% Of 6.2.

Again, the investigation by Berg employed a test re-test design with stroke subj ects and

produced similar results to the present study. To address the possibility of increased variability

when using two distinct methods of evaluation (live performance vs. videotaped evaluation)

future research should consider using only one of the methods to evaluate MDC in these

instruments.

The therapists participating in this experiment offered feedback about the appropriateness

of using videotaped sessions to investigate MDC. In general, they agreed there were some

limitations in terms of having the right camera perspective to accurately assess a particular task.

For example, in some static items of the BB S where body sway is an important factor, the

therapists found the camera was not stable enough to judge the participant' s sway. However, the

therapists also indicated the use of the video allowed them to pause, slow down, or rewind and

play again the tape, if they felt unsure about a particular performance. In addition, an advantage

of using videotaped sessions is that it eliminates the possibility of a learning effect when

assessing participants at two separate occasions. When testing and re-testing subjects within a

short period of time, it is plausible to assume that subj ects could perform better after being

familiar with the test and the testing environment. From this experience, it is clear that a more









standardized way of filming the assessments, possibly using multiple fixed cameras, would result

in a more accurate assessment of participants' performance.

While often time researchers focus on significant group mean changes in the variable of

interest to draw conclusions about the effectiveness of a particular intervention, clinicians face

the need to assess individual patients to judge a particular condition or monitor improvement. In

this study, the BBS and DGI demonstrated mean values between test occasions of less than one

point, suggesting that, as a group, both testing occasions provided almost indistinguishable

results. However, at the individual level, these two instruments demonstrate an important amount

of variability. Therefore, clinicians must be aware of this issue and consider the minimal

detectable change values when making individual decisions based on these instruments.

Conclusion

The procedure outlined by Stratford [86, 92] take reliability assessment to a more

sophisticated, yet much more user friendly level. This statistical approach allows clinicians to

simply determine when genuine change occurs among two testing occasions.

This experiment is a first attempt at investigating the Minimal Detectable Change (MDC)

of the Berg Balance Scale (BB S) and the Dynamic Gait Index (DGI) in elder community

dwellers participating in a rehabilitation program. The results from this investigation demonstrate

that a change of 6.4 point in the BBS and 2.9 points in the DGI is necessary to be 95% confident

genuine change in function has occurred between 2 assessments. These guidelines are important

assessing individuals' performance to monitor progress and guide treatment in clinical practice.

Future investigations are needed to explore MDC at different functional levels.


























5.00-if


0.00e -1 itiui oB dfeec cr (nta B r-ce BS





-5.00-


DGI Ditf


Figure 2-2. Distribution of DGI difference scores (Initial DGI- re-scored DGI)














Table 2-1. Absolute difference in BBS scores between initial and re-scored assessments
Difference in Number of Cumulative
BBS points subj ects Percent percent
0 9 21.4 21.4
1 8 19.0 40.5
2 7 16.7 57.1
3 8 19.0 76.2
4 5 11.9 88.1
5 1 2.4 90.5
6 2 4.8 95.2
8 1 2.4 97.6
11 1 2.4 100.0
Total 42 100.0

Table 2-2. Absolute difference in DGI scores between initial and re-scored assessments
Difference in Number of Cumulative
DGI points subj ects Percent percent
0 13 31.0 31.0
1 18 42.9 73.8
2 5 11.9 85.7
3 6 14.3 100.0
Total 42 100.0


Table 2-3. MDC values for the BBS and DGI
MDC values for the BBS and DGI
Low function High function DGI
BBS BBS (n=30) BBS (n=12)
MDC95% 6.4 7.3 6.3 2.9
MDC90% 5.4 5.8 4.2 2.5
MDCsosn 4.2 4.5 3.2 1.9













10-




C a

r 0 o
oo o
0~ 0

c o o a 0
u, I0 00 0
O 00 o o o


5-r O





-10-




20 30 40 50
BBS combined mean (initial and re-scored)



Figure 2-3. BBS results for all participants. Mean value and difference between initial and re-
scored values.



3- O O O




t 2- a o





a~ 00000 coo 0 a


e0


S-1 a o a o o










5 10 15 20
DGI combined mean (initial and re-scored)



Figure 2-4. DGI results for all participants. Mean value and difference between initial and re-
scored values.





O


o o
So~o
o oo

s o o
'40.00 O O
O




U> 30.00-
m a o




20.00
O O




0 2 4 6 8 10 12
Absolute Difference BBS initial BBS re-scored




Figure 2-2. Mean BBS score for different absolute differences in BBS between testing occasions










20.00

O O
o a
o c

a o
o 0
a
O
oi
-0 o o
o o

co a


oo








0 0.5 1 1.5 2 2.5 3
Absolute difference DGI initial DGI re-scored




Figure 2-3. Mean DGI score for different absolute differences in DGI between testing occasions









CHAPTER 3
PATIENTS SUCCESS CRITERIA AND EXPECTATIONS INT FALLS REHABILITATION

Introduction

Despite the efforts of researchers and funding agencies to understand the mobility

problems many older adults endure, falls and its consequences are taking epidemic proportions in

the western world. In the US, falls are the leading cause of injury deaths among persons over 65.

Falls among elderly persons account for approximately 16,000 deaths and 1.8 million emergency

room visits annually, and fall-related injuries for people 65 and older cost a total of $27.3 billion

in 2003 alone [105]. These alarming statistics have prompted the US senate to recently pass a

new legislation to reduce and prevent elder falls through public education campaigns and

research [117]. This new effort acknowledges the need for the research community to continue

working towards finding solutions to prevent the incidence of falls, improve the treatments

available and ameliorate the consequences of falling.

The causes of falls have been heavily studied and multiple factors have been identified.

Many of these studies indicate that fall prevention programs that include a multidisciplinary

approach with a component of physical rehabilitation can significantly reduce the incidence of

falls [99-101]. However, although these programs may offer significant improvements, reducing

the rates of falling may be dependent on adherence to the programs. Adherence rates for

participation in fall prevention programs have been historically low [102-103]. To maximize

acceptability and adherence among older people, we need to have a better understanding of the

factors that contribute or detract from patients adhering to fall prevention programs. Therefore, it

seems intuitive to turn to the patient to identify factors that are important to them, and determine

the type of treatment that will most satisfy their needs. In addition, to evaluate treatment,










therapists should be sensitive to the amount of improvement patients need to experience to

consider their treatment successful.

Most rehabilitation strategies include end-points based on clinical assessment tools and

conclusions drawn from standard statistical methods of significance. Statistical significance is

important, but not sufficient to establish clinically relevant conclusions [104]. A better approach

is to consider the number of patients that reach a clinically important end-point. To date, few

investigations have used the patient' s perspective to arrive at these end-points. In the falls

literature, this issue has not been explored. Determining endpoints in balance rehabilitation

interventions that reflect the patient's view and contribute to their satisfaction would appear to be

a valuable empirical endeavor.

There is an increased interest in the research community about establishing what

constitutes a clinically important change for healthcare interventions [106-108]. The previous

study in this dissertation demonstrated that available assessment instruments are not always

accurate at detecting change. Regardless of whether a change can be measured or observed, the

interpretation of this change will depend on whose perspective we consider. From the clinician' s

perspective, a change that results in a modification of treatment or patient' s prognosis is certainly

considered a significant change. Researchers consider a change that achieves statistical

significance as a relevant change. In addition, most researchers are concerned with differences in

group of patients. However, group differences could reflect large change in some patients and

modest change in others, or modest change in many patients. Therefore, group changes are

difficult to interpret and apply, since there is no indication of the likelihood of a positive change

in a single patient. A better approach, but seldom reported in the literature, is to turn to the

patient to identify relevant change. Patient reported outcomes are crucial to identify relevant









domains that impact the quality of life of the patient. It is the patient who experiences their

quality of life, and only they are in a position to ultimately judge whether a change is important

[109].

Rehabilitation strategies need to focus on improvements that are perceived by the patient

as being beneficial. This will ensure higher compliance with treatments and a more positive

rehabilitation experience. Empowering the patient can lead to positive outcomes and influence

patient' s satisfaction with treatment. Most of the patient satisfaction literature has concentrated

on treatment of non-chronic conditions. In these cases, patients who experience amelioration of

their symptoms report high levels of satisfaction. Inversely, patients with chronic conditions are

thought to experience less satisfaction with treatment. There is some challenge to this

expectation in the chronic pain literature. Chronic pain patients have reported moderate to high

levels of satisfaction even with small reductions of pain [1 10-1 11]. These patients report that,

while pain was still present, their therapy had helped them reduce some of the collateral

consequences of chronic pain, such as mood disturbances, sleeping problems, etc.

Most elder individuals who have fallen or are at risk of falling, can be classified as chronic

patients. In fact, Calkins and colleagues [1 12] reported that almost 75 percent of the elderly (age

65 and over) have at least one chronic illness. Furthermore, having one or more chronic

condition has been associated with an increased risk of falling by several investigators [1 13-1 15].

Therefore, it is possible that individuals who fall will share some of characteristics of patients

with other chronic illnesses, and base their satisfaction with treatment due to collateral

consequences. Other domains important to this group, such as energy and drive, emotional

distress or social interactions, are areas that may be impacted by fall treatment interventions.










Surprisingly, to date, no published study has examined patient satisfaction in falls rehabilitation

programs.

After identifying relevant domains, investigators need to assess how much improvement

in each domain represents treatment success. Traditionally, this has been accomplished by using

clinical assessment tools that mimic "real life" situations. However, these tools often fail to cover

the domains that are important to patients. Thus, a comprehensive approach that takes into

account the multiple areas of concern to patients is warranted. After establishing the domains of

relevance to this population, these criteria can be used to compare satisfaction across varied

studies and treatment options.

Therefore, the primary aim of this study is to investigate the patient' s success criteria

across several domains including: mobility, self-care, interactions with people, community and

social life, energy and drive, mental function, emotional distress, sensory function, and pain. In

addition, a secondary aim is to investigate patient expectations for treatment across above

mentioned domains.

Methods

Subj ects

A total of 50 participants (age 55 and older) were enrolled in this study. Twenty of these

participants were also part of a larger, funded, research study looking at the link between

smoking and recovery from frailty in older Floridians (DOH- 04NIR-15). This larger study was

supported by a grant from the Florida Department of Health. The remaining 30 participants were

only participating in the present study. Both studies were individually approved by the

Institutional Review Board for the University of Florida and the Research and Development

Committee at the North Florida/South Georgia VA Medical Center. Inclusion criteria for both

studies included: community dwellers with a history of falling, the ability to walk 20ft (with or









without an assistive device), and a score of 24 or higher on the Mini-Mental State Exam [97].

Participants were not screened for any particular condition. Therefore, the participants' pool

consisted of individual with an extensive number of medical conditions including: diabetes,

hypertension, neuropathies, orthopedic problems, general dizziness, history of stroke, general

frailty, etc.

Participants in the DOH- 04NIR-15 study were recruited in two different ways. First,

patients from a Gait and Balance disorders clinic, at the North Florida/South Georgia VA

Medical Center, who were receiving outpatient physical therapy services related to their history

of falling or being at risk of falling, were approached by their therapists and asked if they were

interested in participating in a research study. If the patients agreed, a research coordinator

explained the study and, if still interested, consented the patients. Secondly, letters were sent to

doctor' s offices explaining the study and flyers were distributed throughout the community.

Participants responded to the advertisement and, if they met the study's inclusion criteria, were

enrolled in the study. The remaining participants, only enrolled in the present study, were also

recruited at the North Florida/South Georgia VA Medical Center Gait and Balance disorders

clinic during their scheduled appointments at the clinic. These participants were also receiving

outpatient physical therapy services related to their history of falling or being at risk of falling.

All participants received identical initial evaluations, consisting of a battery of assessment

instruments including: Patient's Perspective Outcome Questionnaire (PPOQ), Berg Balance

Scale (BBS), Dynamic Gait Index (DGI), isometric lower extremity strength, physical function

domain of the MOS36, Falls Efficacy Scale, Geriatric Depression Score, pain experience VAS,

timed tests of gait, Frenchay IADL scale, Pleasant Event Schedule, and spontaneous self-selected

gross motor activity as measured by accelerometry. All physical assessment instruments were









administered by licensed Physical Therapist specialized in the rehabilitation of falls.

Questionnaires were administered by a research assistant. A home exercise program, based on

the initial assessment, was then prescribed by the therapist. Subsequent re-evaluations at 4

weeks, 8 weeks and 12 weeks, were completed to record progress and compliance with the

program.

Testing Procedure

The same procedure was used for both groups of participants. In brief, during the initial

evaluation, participants were administered the Patient' s Perspective Outcomes Questionnaire

(PPOQ, Appendix 1). A research assistant read the questionnaire out loud and recorded the

answers in paper forms. Participants were allowed to ask questions and the research assistant

offered clarification for any questionnaire items that participants felt unsure about.

The Patient's Perspective Outcome Questionnaire (PPOQ): Currently, there are no

instruments in the literature to assess success criteria and expectations for treatment of falls. The

PPOQ is an adaptation of the Patient Centered Outcomes Questionnaire developed by Robinson

and colleagues [104]. These researchers used this questionnaire to assess success criteria and

expectations for treatment of chronic pain. Dr. Robinson participated in the development of the

PPOQ.

The PPOQ is identical to the Patient Centered Outcomes Questionnaire except for the

domains it measures. It consists of 4 questions that address 9 different health domains. These

four questions include: 1) current levels of involvement for each domain, 2) changes in each

domain that will represent a successful treatment, 3) treatment outcome expectations, and 4)

importance of each of the domains. These domains are based on common problem-areas

associated with elder individuals who have fallen or are at risk of falling. The PPOQ uses the

language of the World Health Organization (WHO) International Classification of Functioning









Disability and Health (ICF) classification system [116]. Domains used in this questionnaire refer

to specific domains within the ICF classification system and include: mobility; self-care;

interactions with people; community and social life; energy and drive; mental function;

emotional distress, sensory function; and pain. Participants rate their perception on a scale of 0

(none/not affected/not important) to 100 (worst imaginable/most affected/most important).

Clarifieation about each of the domains is included in the PPOQ. To ensure uniformity and

standardization of the instrument, the ICF definition for each of the domains in the PPOQ is

included. For instance, the ICF defines mobility as: "this term refers to the ability to change

location or transfer from one place to another. It also includes actions such as carrying, moving

or manipulating objects and capacity to walk, run or climb. Lastly, mobility also refers to the

ability to use various forms of transportation" [116]. This exact definition of mobility is included

in the PPOQ and was used by the research assistant to explain the different domains. For

example, when asking the question: On a scale of 0 (not at all important) to 100 (most

important), please indicate how important it is for you to see improvement in you mobility", the

research assistant provides the above mentioned definition of mobility. The same procedure was

used to explain all domain definitions in the questionnaire.

Analysis

All statistical analysis and graphical representations were performed with SPSS 13.0

software for Windows (SPSS Inc., Chicago, IL, USA) and Microsoft Office Excel software for

windows (Microsoft Corporation, Redmond, Washington, USA).

First, descriptive statistics were generated for each of the domains. A repeated measures

ANOVA was performed to determine whether differences existed across domains in the usual

levels of involvement. Follow up paired t-tests, corrected for multiple comparisons (Bonferroni

correction) were performed to investigate differences between mobility and all other domains.










Then, the success criteria were determined by subtracting the usual level across domains

from their expected level. For example, an individual with usual levels of mobility of 60/100 and

success criteria of 40/100 requires 20 point change in mobility to consider their treatment

successful. Success criteria were transformed to percentage change. Therefore, in the previous

example, 20 point change represents 33.3% of the initial 60/100. Repeated measures ANOVA

were performed on the percentage change success criteria scores to determine whether

differences existed across domains in the amount of change necessary for participants to consider

their treatment successful. Then, paired-t-tests, corrected for multiple comparisons (Bonferroni

corrections) were performed to investigate possible differences between domains.

Next, usual levels across domains were subtracted from their expected levels to obtain

treatment expectations criteria. Again, these scores were transformed to percentages to represent

the percentage amount of change participants expected after treatment. A repeated measures

ANOVA was performed to determine whether differences existed across domains in the

percentage amount of change participants expected after treatment. Comparisons between

mobility and all other domains were performed with paired t-tests corrected for multiple

comparisons (Bonferroni correction).

Next, participants were dichotomized to form two groups (compliant vs. non-compliant).

Compliance was defined as a participant who completed the standard 12 weeks program.

Participants were re-evaluated 3 times after the initial evaluation, at 4, 8, and 12 weeks.

Participants who attended the last re-evaluation (12 weeks) were considered compliant.

Participants who missed some of the intermediate evaluations (4 or 8 weeks), but attended the

last re-evaluation (12 weeks) were still considered compliant. A multivariate analysis of variance










(MANOVA) was used to explore possible differences between compliant and non-compliant

groups in treatment expectations across domains.

Results

Demographic characteristics of the sample are presented in table 3-1. Table 3-2 contains

descriptive statistics from the PPOQ initial levels. Participants reported low to moderate initial

levels of restriction in mobility, self-care, interactions with people, community and social life,

energy and drive, mental function, emotional distress, sensory function, and pain associated with

their conditions (higher scores, worse the condition) Energy and drive; mobility; and pain

received the highest scores (53, 47, and 44 respectively), while interactions with people and self-

care received the lowest (21 and 24 respectively). Differences of initial levels across domains

were explored with repeated measures ANOVA. Mauchly's test indicated that the assumption of

sphericity had been violated (X2 (35)= 52.02, p<.05); therefore degrees of freedom were

corrected using Greenhouse-Geisser estimates of sphericity (e = .79). Results indicated the

existence of significant differences of initial levels among domains F= (6.35, 310.98)= 8.6,

p<.05. Paired t-tests, adjusted for multiple comparisons (Bonferroni correction), were used for

posttests. The main analysis comparing mobility to other domains resulted in 8 comparisons.

Therefore, for 8 domain comparisons a .006 level of significance was selected. Participants

reported higher levels of impairment in the mobility domain, compared to self-care and

interactions with people (P<.006). The community and social life domain showed similar trend

but the analysis failed to reach statistical significance t(49)= 2.79, P= 008.

To determine the amount of change necessary for participants to deem their treatment

successful, their initial levels across domains were subtracted from their success criteria (table 3-

3). Participants considered a mean reduction of 52% in mobility as a successful outcome. Lower









results were found for the domains of sensory function (34%), self-care (35%), interactions with

people (22%), and, community and social life. Energy and drive required the largest reduction

(59%). A repeated measures ANOVA was used to investigate whether differences existed in the

amount of change necessary for participants to consider their treatment successful. Mauchly's

test indicated that the assumption of sphericity had not been violated (X2 (35)= 46.9, p>.05);

therefore degrees of freedom were not corrected. The amount of change required for participants

to consider their treatment successful was significantly different among domains F= (8, 368)=

7.04, p<.05. Paired t-tests were used for posttests. A Bonferroni correction was applied and so all

effects are reported at .006 level of significance. It appeared that the reduction in mobility

necessary for successful treatment was significantly greater than the reductions necessary for

successful treatment of self-care, interactions with people and community and social life

(P<.006). Therefore, in this group of participants, treatment success criteria was significantly

different across domains, suggesting participants require different amounts of change across

domains to consider their treatment successful.

Next, initial levels across domains were subtracted from their expected levels to obtain

treatment expectations criteria. Subj ects expected similar levels of mobility and energy and drive

(both 42%). Interactions with people received the lowest score (18%) (Table 3-4). A Repeated

measures ANOVA was used to investigate whether differences existed in the amount of change

participants expected after treatment. Sphericity assumption was tested with Mauchly's test.

Results indicated that the assumption had been violated (X2 (35)= 76.02, p<.05); therefore

degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity (e = .74).

Results indicated the existence of significant differences among domains in the percentage

amount of change participants expected after treatment F= (5.91, 277.78)= 3.94, p<.05. Paired t-









tests, adjusted for multiple comparisons (Bonferroni correction), were used for posttests. For this

group, the percentage change participants expected for the mobility domain was significantly

greater than the percentage change expected for the domains of self-care, interactions with

people and pain (P<.006). Lastly, the results from multivariate analysis of variance (MANOVA)

indicated there were no significant differences between compliant and non-compliant groups in

treatment expectations across domains. The Wilks Lambda multivariate test of overall

differences among groups was not significant (p= 0.934). An exploratory descriptive

investigation of groups of participants, based on compliance, revealed that VA participants were

40% compliant, while community participants presented 80% compliance.

Discussion

Consistent with the theoretical model that guided this research, participants with mobility

problems leading to falls demonstrated significant levels of interference across several health

domains, representing activity and participation, and body function. In this group, participants

reported considerable initial levels of interference in domains such as energy and drive (54/100),

and pain (45/100). Not surprisingly, the mobility domain also received high scores (48/100).

Lower scores were seen in domains such as interaction with people (21/100), community and

social life (33/100), and self-care (24/100). These findings suggest that rehabilitation strategies

should take into consideration the complex and multidimensional nature of falls and provide

interventions that target the different domains that affect this population. A number of

publications have investigated this issue. For example, Gillespie and colleagues [117] conducted

a systematic literature review of randomized controlled trial programs designed to reduce the

number of falls in community-dwelling, institutionalized, or hospitalized elderly people. The

authors concluded that, interventions targeting only the physical aspects related to falls produced










lower outcomes when compared to multidimensional interventions that took into account the

intrinsic and environmental risk factors of patients.

The present study is a first attempt at using patient reported outcomes to investigate several

health domains related to individuals receiving rehabilitation services related to falling. More

specifically, this investigation focused on three fundamental aspects of rehabilitation: how

participants perceived their levels of impairment across several health domains, how much

change was necessary across domains to consider their treatment successful, and what were their

treatment expectations.

As mentioned before, significant levels of impairment were reported across domains.

Interestingly, several differences were found between domains. The energy and drive domain

received the highest score, suggesting that, for this group, issues such as feelings of fatigue,

motivation and energy level are commonly present. There is support in the literature for this

finding. Fatigue has been associated with a number of conditions in the elderly population,

including diabetes, heart failure, Parkinson's disease, cancer, sleep disorders, and hormonal

changes [1 18-121]. In addition, participants reported considerable levels of pain. There is

supporting evidence that suggest pain is a risk factor for falls and also that pain can lead to

activity avoidance [122-123]. The findings from the present investigation suggest that pain and

fatigue should be considered when assessing this population.

Patient reported outcomes provide a unique opportunity to evaluate clinical practice, from

the patient's perspective. In the present study, participants required significant changes in a

number of health domains to consider their treatment successful. The mobility domain required

significantly larger reductions than the community and social life; and interactions with people

domains. The clinical implication of this finding is that success criteria differ across domains,










suggesting that rehabilitation intervention should be guided towards areas of greatest concern to

the patient.

This study also explored participants' expectations with treatment outcomes. Not

surprisingly, participants expected mobility to change more than in other domains (42%). An

unanticipated finding was that participants expected the same amount of change in the domain of

energy and drive. This finding, suggest that perhaps participants view this domain as an

extension of the mobility domain. That is, participants may connect improvements in mobility

with increased energy and drive and, therefore, expect that energy and drive will increase as

mobility improves.

In this study, participants expected different amounts of change across domains. Again,

this finding must be interpreted with caution, since percentage change are heavily influenced by

initial scores. For example, a participant reporting a 10 point change from an initial level of

90/100 and an expected level of 80/100 produces a percentage change of 11%, while a

participant reporting a 10 point change from an initial level of 30/100 and an expected level of

20/100 produces a percentage change of 33%. In this group, participants reported low initial

levels of interference with self-care and their expected level was also low. A similar trend was

apparent in other domains, where high initial levels also resulted in high expectations, while low

initial levels resulted in low expectations. It is plausible to speculate that participants were

influenced by their initial scores and based their subsequent answers proportionally.

In the present study, participants had reasonable treatment expectations. Their expectations

were lowest in domains related to participation, such as community and social life, and

interactions with people. Perhaps, participants found it difficult to see the connection between

improvement in physical function and improvement in social roles. Across all domains,










participants reported lower expected scores compared to their success criteria. This indicates

that, in this population, participants did not expect their treatment to meet their success criteria.

That is, participants expected residual levels across domains following treatment.

Results of the multivariate analysis of variance indicated no differences in expectations

existed between groups of participants based on compliance with the treatment. However, these

results must be interpreted with caution, due to the lack of statistical power resulting from

dividing the participant pool into two groups. In addition, for a number of participants

compliance was unknown, resulting in an even smaller sample size (compliant group N= 14,

non-compliant N= 7). Further investigation is warranted in this area. A larger sample size that

represents the general geriatric population should be used to investigate compliance. The veteran

population presents certain characteristics that can have an effect on compliance. In the present

study, although the sample size was too small to draw statistical conclusions, there were marked

differences between VA participants and community participants. VA participants were 40%

compliant, while community participants presented 80% compliance. In the larger DOH-04NIR-

15 study, VA compliance was 33%. Compliance with rehabilitation interventions is essential for

individuals who fall, and should be further investigated. For researchers, lack of compliance

represents an additional problem, because of the bias sample selection. Drawing conclusions

about the effectiveness of a particular rehabilitation strategy when samples are only

representative of a small group of compliant participants is highly suspect. There could be

confounding factors that make this group different and affect how this group reacts to treatment.

This study presents a number of limitations. The results obtained are based on a

questionnaire specifically design for this investigation. Although this questionnaire is based on a

previously published instrument [104], the psychometric properties of the former and the current










questionnaire have not been investigated. Further work is required to ascertain the reliability,

validity, and generalizability of findings obtained with this questionnaire. A methodological

strength of this questionnaire is that it uses the language of the ICF. This allows for comparison

with other instruments that use the same language. The ICF classification system has been

successfully used to develop a number of instruments [124]. Using the ICF classification system,

a standardized set of relevant categories can be investigated for specific populations.

Another limitation of this study concerns the limited information about participants'

characteristics. More specifically, the medical diagnosis of the pool of participants was

unknown. In addition, two distinct populations were included in this study, veterans and

community participants. Although both groups were community dwellers, there are certain

characteristics of the veteran population that could influence their perception, expectation, and

success criteria. Veterans participating in this study were recruited during their initial visit to

physical therapy services. In contrast, the community dwellers participating in this study

responded to advertisements distributed by the research team. The difference in recruitment

approaches could produce bias and have an effect on participants' expectations.

The present study is a preliminary attempt at exploring patient reported outcomes and

expectations in the treatment of falls. The results of this investigation suggest that patients

participating in falls rehabilitation present a number of limitations that far exceed the mobility

domain. Participants' success criteria varied across domains, suggesting that, for this population,

some domains are more important than others. In addition, participants had reasonable

expectations, but considered change in the most affected domains most important.

Future work should be guided towards refining and validating the PPOQ. In addition, this

instrument could be used to link clinical performance measures with patients' expectations and










success criteria. This way, the effectiveness of rehabilitation interventions can be determined

using a mixed clinical/patient centered approach. To provide patients the treatment they deserve,

clinicians need to understand the patient' s expectations and goals. Patient satisfaction is an

important goal for all services, including medical services. This investigation provides a first

look at quantifying some of the areas needed to ensure patient' s satisfaction with the treatment of

falls.










Table 3-1. Demographics
Number of
participants
UF 12
VA 38
Total 50
Non-Compliant group 7
Compliant group 14
Age 74.1 (SD 10.9)
Male 38
Gender
Female 12


Table 3-2. Initial levels descriptive statistics
N Minimum Maximum Mean Std. Deviation
Mobility 50 0 95 47.64* 28.56
Self-care 50 0 95 23.68* 32.17
Interactions with people 50 0 100 20.80* 29.05
Community and social life 50 0 100 33.50 37.70
Energy and drive 50 0 100 53.60 29.67
Mental function 50 0 100 37.60 32.38
Emotional distress 50 0 100 40. 14 34.41
Sensory 50 0 95 37.38 28.72
Pain 50 0 100 43.92 34.45
P= .006

Table 3-3. Success criteria (initial levels-success levels) descriptive statistics
% Std.
N Minimum Maximum Mean Change Deviation
Mobility 50 0 70 24.87 51.86* 18.94
Self-care 50 0 90 14.66 35.52* 25.14
Interactions with people 50 0 75 9.15 22.27* 18.27
Community and social 50 27.96*
0 95 16.91 27.05
life
Energy and drive 50 0 80 31.60 58.84* 23.82
Mental function 50 0 80 21.23 44.99 24.55
Emotional distress 50 0 85 25.13 48.24 26.11
Sensory 50 0 80 15.40 34.33 19.40
Pain 50 0 85 24.53 44.81 24.27
P= .006










Table 3-4. Treatment expectations criteria (initial levels-expectations levels)
descriptive statistics
% Std.
N Minimum Maximum Mean Change Deviation
Mobility 50 0 70 19.98 42.30* 18.34
Self-care 50 0 60 8.02 23.80* 13.35
Interactions with people 50 0 75 6.28 17.87* 14.00
Community and social 50
0 95 15.43 26.78* 25.55
life
Energy and drive 50 0 70 23.38 42.49* 23.10
Mental function 50 0 65 14.51 34.44 17.90
Emotional distress 50 0 85 18.74 38.94 22.83
Sensory function 50 0 80 11.28 32.01 15.86
pain 50 0 80 13.98 27.92 19.19
P= .006









CHAPTER 4
GENERAL SUMMARY AND CONCLUSIONS

Overall, the obj ectives of this dissertation were to investigate minimum detectable change

in outcome measures used to assess individuals receiving rehabilitations services related to falls,

and to explore treatment success criteria and expectations, in the same population. Recently,

there is considerable debate in the medical and scientific community about what treatment

outcomes constitute a meaningful change, and how to measure this change [125-127]. Clinical

performance measures do not always capture all aspects of the patients' experience and often

produce results that are difficult to interpret. In an attempt to simplify the interpretation of

scores, single numbers are assigned to complex processes. In the name of obj activity, statistical

analysis are performed on these scores, and the results are use to evaluate performance and

assess change. Still, statistical significance does not necessarily equal meaningful change.

Patient reported outcomes (PRO) are receiving attention as legitimate outcome measures

for clinical research. PRO instruments are used to measure treatment benefits by capturing

concepts related to how a patient feels or functions with respect to his or her health or condition.

This approach turns to the patient to investigate what is important to then. The ideas, activities,

behaviors, or feelings measured by PRO instruments can be either verifiable in nature, such as

walking, or can be non-observable, known only to the patient, such as pain, depression etc.

Although these symptoms are highly dependent on the patients' perception, historically, these

assessments were made by clinicians who observed and interacted with patients. Recently, these

kinds of assessments are increasingly performed with PRO instruments. It seems intuitive to

consider the patient's opinion when investigating what constitutes meaningful change.

The two experiments described in this dissertation are aimed at investigating change in

geriatric patients. Specifically, the group of patients investigated received rehabilitation










interventions related to mobility problems. The overall aim of this dissertation was to explore

change in this group of patients employing two different approaches. First, minimum detectable

change was investigated in clinical instruments used with this population. Second, a newly-

developed PRO instrument was used to determine what constitutes successful treatment

outcomes and expectations when participating in a rehabilitation program. A general conclusion

for both studies in this dissertation follows.

Experiment I Summary

The goal of this study was to investigate minimal detectable change (MDC) for two

common instruments used to assess gait and balance in the elder population. The Berg Balance

Scale [75] and the Dynamic Gait Index [82] were explored in this experiment. The procedure

outlined by Stratford [86, 92] was used to calculate the MDC. Stratford proposed to use the

standard error of measure (SEM) to calculate the amount of change in a given measure that must

be obtained for a clinician to determine that true change has occurred. The MDC is expressed as

a confidence interval around the SEM, indicating the values that are within the range of error

attributable to the measuring instrument. The MDC is expressed in the same unit of the original

instrument, providing clinicians useful and easy to understand criterion for change in patients'

performance .

The results of this study indicated that for the Berg Balance Scale and the Dynamic Gait

Index, 6.4 and 2.9 points respectively were required to be 95% confident that "genuine" change

had occurred between 2 testing occasions. These results suggest that a significant amount of

error is associated with these instruments. In addition, the results suggested that MDC values are

not a constant feature of the instruments. MDC values for the high functional level group were

6.3 BBS points. In contrast, participants in the lower functional group presented MDC values of

7.3. That is, the values of MDC change based on the ability level of the persons assessed.









The above findings are relevant to present day clinical practice. Clinicians use the Berg

Balance Scale and Dynamic Gait Index routinely to assess patients with mobility problems. An

advantage of using the standard error of measure to determine minimal detectable change is that

this method provides information about individual scores. Traditional methods of statistical

significance rely on group differences to investigate the properties of the instruments. Group

differences are relevant for researchers, but must be considered with caution when decisions

must be made about an individual patient. The results of this investigation can be applied at the

individual level. Knowing the amount of error associated with these instruments can help

clinicians make decisions about individual's performance and monitor change overtime.

Experiment II Summary

The primary aim of this study was to use a PRO questionnaire to investigate patient' s

success criteria and expectations when receiving rehabilitation services related to falls. More

specifically, the patient's success criteria was assessed across several health domains including:

mobility, self-care, interactions with people, community and social life, energy and drive, mental

function, emotional distress, sensory function, and pain. In addition, a secondary aim was to

investigate patient expectations for treatment across above mentioned domains.

In this study, participants with mobility problems leading to falls demonstrated significant

levels of interference across several of the health domains measured. Participants reported

considerable initial levels of impairment in domains such as energy and drive (53/100), and pain

(44/100). The mobility domain also received high scores (indicating impairment) (47/100).

Lower scores were seen in domains such as interaction with people (21/100) and community and

social life (33/100). These findings suggest that, in this population, domains with a strong social

component were not as affected as domains with a strong physical component.









Participants in this study required significant improvement in health domains to consider

their treatment successful. Domains such as mobility; and energy and drive, required

significantly larger reductions than the community and social life; and interactions with people

domains. This provides information about what is important to patients receiving this

intervention. Furthermore, these Eindings could lead to developing rehabilitation strategies

guided towards areas of greatest concern to the patient.

This study also explored participants' expectations with treatment outcomes. Participants

expected mobility to change the most as a result of this intervention (52%). However, similar

Ending was reported in the domain of energy and drive. An interesting finding is that across

domains, participant' s expectation was that the treatment would not meet their success criteria.

This indicates that, for this population of elder individuals with mobility problems, residual

levels of impairment in the measured domains are expected after treatment. Compliance was also

investigated in this group. No differences were found between compliant and non-compliant

groups based on treatment expectations.

The results of this study point out that a number of health domains are significantly

affected in this population. Patients receiving rehabilitation services related to falls have

treatment expectations that far exceeds the mobility problems for which they are been treated. In

exploring meaningful change in patients receiving rehabilitation interventions, the patient's

expectations and success criteria must be considered. By linking existing clinical instruments

with patient reported outcomes researchers and clinicians can be sure that therapies used achieve

a meaningful change. Physical rehabilitation strategies must take into consideration the complex

and multidimensional nature of falls and provide interventions that target the different domains

that affect this population.









General Conclusions and Future Directions

The question of what constitutes a meaningful change in the population investigated in this

dissertation remains unanswered. However, several conclusions can be drawn from the work

presented. First, clinical instruments provide valuable information about patients' performance

and can help clinicians and researchers evaluate patient' s ability and monitor improvement.

However, when these instruments are used at the individual level, clinicians must be aware that

all change is not genuine change. That is, instrument error must be considered when reporting

patient change at the individual level. Second, elder patients receiving mobility-related

rehabilitative services expect their treatment to produce changes in a number of health domains

that extend beyond mobility improvement.

Patient reported outcomes could serve as a bridge to link clinical practice with meaningful

patient-centered treatment results. There is a growing movement for patients to take an active

role in their medical care and be involved in making decision about treatment options [128].

Patients demand services that meet their needs and expect treatments to address nonclinical

aspects that affect their day to day life. This view is receiving attention from the research

community and regulatory agencies. A number of randomized clinical trials are now including

patient reported outcomes as important endpoints in addition to traditional clinical measures

[129-130]. Regulatory agencies are also recognizing this need and establishing criteria for the

use of patient reported outcomes [126]. Incorporating the patient' s view in the rehabilitation

process ensures that interventions meet the patient' s needs and therefore play a role at

empowering the patient and making him or her responsible for actively participating in the

rehabilitative process.

A number of interesting questions have arisen as a result of this dissertation. Further work

is warranted at investigating minimal detectable change levels associated with different









functional levels of the patients investigated. This is particularly important for high risk patients

for which small change in performance, indicating improvement or deterioration, can have

serious consequences. In this functional group, knowing the exact functional level of the patient

can help clinicians take appropriate measures to avoid possible injuries, for example prescribe an

assistive or protective device.

Future work should also be conducted to investigate issues of compliance in this

population. In this dissertation, possibly due to small sample size, no significant differences

between compliant and non-compliant groups were found. Still, low compliance levels were seen

in the investigated group. The traditional model of care, where clinicians tell patients what to do

and try to motivate them to change, may not be the most effective method of intervention.

Empowering the patient by considering their specific needs and addressing their expectations

might result in better compliance with treatment. PRO instruments can be used to incorporate the

patient' s perspective in the rehabilitative process. These instruments can serve as an anchor to

compare clinical instruments against meaningful patient-centered outcomes.

Collectively, this series of studies promotes our understanding of significant change in

patients receiving rehabilitation services related to falls. The results obtained indicate that current

rehabilitation programs must consider the limitations of available instruments and take into

consideration the needs and expectations of patients. Ultimately, this research aims to influence

treatment by providing information to help clinicians select the best tools available for the

rehabilitation of falls, and suggests the inclusion of the patient' s perspective as one of the outcome

measures in their treatment plans.









APPENDIX
PATIENT' S PERSPECTIVE OUTCOME QUESTIONNAIRE (PPOQ)

We have identified some common problems people can experience as they get older.
Specifically, older people who have balance problems and may have suffered a fall often face
problems of mobility, self-care, interactions with people, community and social life, energy
and drive, mental function, emotional distress, sensory function, and pain (see the last page
for an extended explanation of these problem-areas). We would like to ask you a few
questions to see how important these problems are to you.

FIRST, WE WOULD LIKE TO KNOW YOUR USUAL LEVELS OF MOBILITY, SELF-
CARE, INTERACTION WITH PEOPLE, COMMUNITY AND SOCIAL LIFE, ENERGY
AND DRIVE, MENTAL FUNCTION, EMOTIONAL DISTRESS, SENSORY FUNCTION,
AND PAIN.

On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate your
usual level (during the past week) of
mobility
self-care
interactions with people
community and social life
energy and drive
mental function
emotional distress
sensory function
pain

PATIENTS UNDERSTANDABLY WANT THEIR TREATMENT TO RESULT IN DESIRED
OR IDEAL OUTCOMES. UNFORTUNATELY, AVAILABLE TREATMENTS DO NOT
ALWAYS PRODUCE DESIRED OUTCOMES. THEREFORE, IT IS IMPORTANT FOR US
TO UNDERSTAND WHAT TREATMENT OUTCOMES YOU WOULD CONSIDER
SUCCESSFUL.

On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate the
level each of these areas would have to be at to consider treatment successful.

mobility
self-care
interactions with people
community and social life
energy and drive
mental function
emotional distress
sensory function
pain











NOW, WE WOULD LIKE TO KNOW WHAT YOU EXPECT YOUR TREATMENT TO DO
FOR YOU.

On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate the
levels you expect following treatment.

mobility
self-care
interactions with people
community and social life
energy and drive
mental function
emotional distress
sensory function
pain

FINALLY, WE WOULD LIKE TO UNDERSTAND HOW IMPORTANT IT IS FOR YOU TO
SEE IMPROVEMENT IN YOUR MOBILITY, SELF-CARE, INTERACTIONS WITH
PEOPLE, COMMUNITY AND SOCIAL LIFE, ENERGY AND DRIVE, MENTAL
FUNCTION, EMOTIONAL DISTRESS, SENSORY FUNCTION, AND PAIN FOLLOWING
TREATMENT.

On a scale of 0 (not at all important) to 100 (most important), please indicate how important it is
for you to see improvement in your...

mobility
self-care
interactions with people
community and social life
energy and drive
mental function
emotional distress
sensory function
pain









Problem areas explained:


Mobility: This term refers to the ability to change location or transfer from one place to
another. It also includes actions such as carrying, moving or manipulating objects and
capacity to walk, run or climb. Lastly, mobility also refers to the ability to use various
forms of transportation.

Self-care: This problem-area is about caring for oneself, washing and drying oneself,
caring for one's body and body parts, dressing, eating and drinking, and looking after
one's health.

Interactions with people: This area is about the ability to socially interact with
strangers, friends, relatives, family members and significant others.

Community and social life: This area is about the actions and tasks required to engage
in organized social life outside the family, in community, social and civic areas of life.
Examples include: participating in religious or spiritual activities and participation in
leisure or recreational activities.

Energy and drive: This problem area refers to feelings of fatigue, motivation and energy
level.

Mental function: This problem area includes issues related to memory, attention,
concentration, and decision making.

Emotional distress: This problem area includes feelings of depression, anger, anxiety,
and frustration.

Sensory function: This area includes problems of vision and hearing.

Pain: This area includes all types of pain, including chronic and acute.









LIST OF REFERENCES


1. Miaskowski C. The impact of age on a patient's perception of pain and ways it can be
managed. Pain Manag Nurs 2000; 1:2-7.

2. Centers for Disease Control and Prevention, Department of Health and Human Services.
Healthy Aging for Older Adults [online]. (2007) [cited 2008 Apr 14]. Available from
URL: http://www. cdc. gov/aging/

3. Seematter-Bagnoud L, Wietlisbach V, Yersin B, Bidla CJ. Healthcare Utilization of
Elderly Persons Hospitalized After a Noninjurious Fall in a Swiss Academic Medical
Center. Journal of the American Geriatrics Society, 2006; 54(6),891-897.

4. Centers for Disease Control and Prevention, National Center for Injury Prevention and
Control. Web-based Injury Statistics Query and Reporting System (WISQARS) [online].
(2006) [cited 2008 Apr 14]. Available from URL: www.cdc .gov/ncipc/wi sqars

5. Hausdorff JM, Rios DA, Edelber HK. Gait variability and fall risk in community-living
older adults: a 1-year prospective study. Archives of Physical Medicine and
Rehabilitation 2001; 82(8):1050-6.

6. Englander F, Hodson TJ, Terregrossa RA. Economic dimensions of slip and fall injuries.
Journal of Forensic Science 1996; 41(5):733-46.

7. World Health Organization. International Classification of Functioning (ICF), Disability
and Health. Geneva: WHO; 2001.

8. Engel GL. The need for a new medical model: a challenge for biomedicine. Science
1977; 196(4286):129-36.

9. Kirshner B, Guyatt GH. A methodological framework for assessing health indices. J
Chronic Dis 985; 38:27-36.

10. Guyatt GH, Jaeschke R, Feeny DH, Patrick DL. Measurements in clinical trials:
choosing the right approach. In: B. Spilker, Editor, Quality of life and
pharmacoeconomics in clinical trials, Lippincott-Raven Publishers, Philadelphia; 1996;
41-49.

11. Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Ann Intern
Medl993; 118:622-629.

12. Wyrwich KW, Metz S, Babu AN. The reliability of retrospective change assessments.
Qual Life Res 2002; 11:636.

13. Sim J, Arnell P. Measurement validity in Physical Therapy research. Phys Ther 1993;
73: 102-15.










14. Juniper EF, Guyatt GH, Epstein RS, Ferrie PJ, Jaeschke R, Hiller TK. Evaluation of
impairment of health related quality of life in asthma: development of a questionnaire for
use in clinical trials. Thorax 1992; 47:76-83.

15. Portney LG, Watkins MP. Reliability. In: Portney LG, Watkins MP. Foundations of
Clinical Research: applications to practice. 2nd ed. New Jersey: Prentice Hall Health;
2000: 53-68.

16. Walmsley RP, Amell TK. The application and interpretation of intraclass correlation in
the assessment of reliability in Isokinetic dynamometry. Isokinet Exerc Sci 1996; 6: 117-
24.

17. Juniper EF, Guyatt GH, Epstein RS, Ferrie PJ, Jaeschke R, Hiller TK. Evaluation of
impairment of health related quality of life in asthma: development of a questionnaire for
use in clinical trials. Thorax 1992; 47:76-83.

18. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J
Chronic Dis 1985; 38: 27-36.

19. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical
change: an analogy to diagnostic test performance J Chronic Dis. 1986; 39:897-906.

20. Salter K, Jutai JW, Teasell R, Foley NC, BItensky J, Bayley M. Issues for selection of
outcome measures in stroke rehabilitation: ICF activity. Disabil Rehabil 2005; 27:315-
340.

21. Jaeschke R, Singer J, Guyatt GH. Ascertaining the minimal clinically important
difference. Control Clin Trials 1989; 10:407-415.

22. Samsa G, Edelman D, Rothman ML, Williams GR, Lipscomb J, Matchar D.
Determining clinically important differences in health status measures: a general
approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics
1995; 15:141-155.

23. Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes
in health-related quality-of-life scores. J Clin Oncol 1998; 16:139-144.

24. Norman GR, Sridhar FG, Guyatt GH, Walter SD. The Relation of Distribution- and
Anchor-Based Approaches in Interpretation of Changes in Health Related Quality of
Life. Medical Care 2001; 39:1039-1047.

25. Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, and the Clinical
Significance Consensus Meeting Group. Methods to explain the clinical significance of
health status measures. Mayo Clinic Proceedings 2002; 77:371-383.

26. Deyo RA, Inui TS, Leininger J, Overman S. Physical and psychosocial function in
rheumatoid arthritis: clinical use of a self-administered health status instrument. Arch
Intern Med 1982; 142:879.









27. Johnson PA, Goldman L, Orav EJ, Garcia T, Steven D, Pearson, et al. Comparison of the
medical outcomes study short-form 36-item health survey in black patients and white
patients with acute chest pain. Med Care 1995; 33:145-160.

28. Testa MA, Simonson DC. Assessment of quality-of-life outcomes. New Engl J Med
1996; 28:835-840.

29. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining
meaningful change in psychotherapy research. J Consult Clin Psychol 1991; 59:12-19.

30. Samsa G, Edelman D, Rothman ML, Williams GR, Lipscomb J, Matchar D.
Determining clinically important differences in health status measures: a general
approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics
1995; 15:141-155.

31. Revicki DA, Allen H, Bungay K, Williams GH, Weinstein MC. Responsiveness and
calibration of the general well-being adjustment scale in patients with hypertension. J
Clin Epidemiol 1994; 437:1333-1342.

32. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the
minimal clinically important difference. Control Clin Trials 1989; 10:407-415.

33. Idler EL, Angel RJ. Self-rate health and mortality in the NHANES-I epidemiology
follow-up study. Am J Public Health 1990; 80:446-452.

34. Farar JT. What is clinically meaningful: outcome measures in pain clinical trials. Clin J
Pain 200; 16(2 Suppl):S106-12.

35. Lydick F, Yawn BP. Clinical interpretation of health-related quality of life data. In: M.J.
Staquet, R.D. Hays and P.M. Fayers, Editors, Quality of life assessment in clinical trials:
methods and practice. Oxford University Press, Oxford 1998:299-314.

36. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness:
a critical review and recommendations. J Clin Epidemiol 2000; 53:459-468.

37. G.R. Norman, P. Stratford and G. Regehr. Methodological problems in the retrospective
computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;
50:869-879.

38. Liang NH, Larson MG, Cullen KE, Schwartz JA. Comparative measurement efficiency
and sensitivity of five health status instruments for arthritis research. Arthritis Rheum
1985; 28:542-547.

39. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status
measures: statistics and strategies for evaluation. Control Clin Trial 1991; 12:142S-158S.

40. Byrk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis
methods. Newbury Park (CA), Sage; 1992.









41. Speer C, Greenbaum PD. Five methods for computing significant individual client
change and improvement rates: support for an individual growth curve approach. J
Consult Clin Psychol 1995, 63:1044-1048.

42. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods
7; 2002:147-177.

43. Cohen, J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ:
Lawrence Earlbaum Associates 1988:34.

44. Kazis LE, Anderson JJ, Meenan RS. Effect sizes for interpreting changes in health
status. Med Care 1989; 27: Suppl 3:S178-S189.

45. Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. John
Wiley&Sons, Chichester; 2000.

46. Clancy CM, Eisenberg JM. Outcomes research: measuring the end results of health care.
Science 1998; 282(5387):245-6.

47. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based
criterion for identifying meaningful intra-individual changes in health-related quality of
life. J Clin Epidemiol 1999; 52:861-873.

48. McHorney CA, Tarlov A. Individual-patient monitoring in clinical practice: are
available health status surveys adequate. Qual Life Res 1995; 4:293-307.

49. Anastasi A, Urbina S. Psychological Testing. 7th ed. Upper Saddle River, NJ: Prentice-
Hall; 1997.

50. Hageman WJ, Arrindell WA. Establishing clinically significant change: increment of
precision and distinction between individual and group level of analysis. Behav Res Ther
1999; 37:1169-1193.

51. Roebroeck ME, Harlaar J, Lankhorst GJ. The application of generalizability theory to
reliability assessment: an illustration using isometric force measurements. Phys Ther
1993; 73 (6): 386-95.

52. Engel GL. The need for a new medical model: a challenge for biomedicine. Science
1977; 196(4286):129-36.

53. Engel GL. The clinical application of the biopsychosocial model. Am J Psychiatryl1980;
137: 535-544.

54. U.S. Food and Drug Administration. Guidance for Industry. Patient-Reported Outcome
measures : use in medical product development to support labeling claims. Draft
guidance. [cited 2008 June 20]. Available from:
http://www.fda. gov/cder/guidance/5460dft. pdf









55. Hippocrates. The Book of Prognostics, Part 2 (approxi. 400 B.C.). As translated by
Francis Adams (Great Books Index, 1977-99). [cited February 2008]. Available from:
http://classics. mit. edu/Hippocrates/prognost.mb .txt

56. FDA Consumer Magazine. The Importance of Patient-Reported Outcomes ... It' s All
About the Patients 2006; Vol 40: 6. Available from:
http://www.fda.gov/fdac/606_toc.html.

57. Revicki DA, Osoba D, Fairclough D, Barofsky I, Berzon R, Leidy NK, Rothman M.
Recommendations on health-related quality of life research to support labeling and
promotional claims in the United States. Qual. Life Res. 2000; 9:887-900.

58. Patrick DL, Erickson P. Health status and health policy: quality of life in health care
evaluation and resource allocation. Oxford University Press, New York; 1993.

59. Carmines E G, Zeller RA. Reliability and validity assessment. Newbury Park: Sage
Publications; 1991.

60. Ostir G, Granger C, Black T, Roberts P, Burgos L, Martinkewiz P, et al. Preliminary
Results for the PAR-PRO: A Measure of Home and Community Participation. Arch Phy
Med and Rehab 2006; 87(8): 1043-51.

61. Willkea RJ, Burkeb LB, Ericksonc P. Measuring treatment impact: a review of patient-
reported outcomes and other efficacy endpoints in approved product labels. Controlled
Clinical Trials 2004; 25(6):535-52.

62. Revicki DA, Osoba D, Fairclough D, Barofsky I, Berzon R, Leidy NK, Rothman M.
Recommendations on health-related quality of life research to support labeling and
promotional claims in the United States. Qual Life Res. 2000; 9(8):887-900.

63. Zerhouni E. NIH Roadmap. Science 2003; 302(5642):63-72.

64. Stucki G. International classification of functioning, disability and health (ICF): A
promising framework and classification for rehabilitation medicine. Am J Phys Med
Rehabil 2005; 84:733-740.

65. World Health Organization. International Classification of Diseases, Disorders and
Injuries, 10th revision. Geneva: World Health Organization, 1994.

66. Degner L, Sloan J. Symptom distress in newly diagnosed ambulatory cancer patients and
as a predictor of survival in lung cancer. J Pain Symptom Manage 1995; 10: 1-8.

67. Sloan JA, Loprinzi CL, Kuross SA, Miser AW, O'Fallon JR, Mahoney MR, et al.
Randomized comparison of four tools measuring overall quality of life in patient with
advanced cancer. J Clin Oncol 1998; 16:3662-3673.

68. Frost MH, Huschka M. Quality of Life from a Patient' s Perspective: Can We Believe the
Patient? Current Problems in Cancer. 2005; 29(6):326-331.









69. Hodgkins M, Albert D, Daltroy L. Comparing patients' and their physicians' assessment
of pains. Pain 1985; 23: 273-277.

70. Patel KK, Veenstra DL, Patrick DL. A Review of Selected Patient-Generated Outcome
Measures and Their Application in Clinical Trials. Value in Health 2003; 6(5):595-
603(9).

71. Davidhizar R, Giger JN. A review of the literature on care of clients in pain who are
culturally diverse. International Nursing Review 2004; 51(1):47-55.

72. Atkinson MJ, Lennox RD. Extending basic principles of measurement models to the
design and validation of Patient Reported Outcomes. Health Qual Life Outcomes 2006;
4: 65.

73. Lohr KN. Assessing health status and quality-of-life instruments: attributes and review
criteria. Qual Life Res. 2002; 11:193-205.

74. Medley A. Predicting the probability of falls in community dwelling persons with brain
injury: a pilot study. Brain Inj. 2006; 20(13-14): 1403-8.

75. Berg K, Maki B, Williams JI, Holliday PJ, Wood-Dauphinee SL. Clinical and laboratory
measures of postural balance in an elderly population. Archives Of Physical Medicine
and Rehabilitation 1992a; 73:1073-1080.

76. Berg KO, Wood-Dauphinee SL, Williams JI, Maki B. Measuring balance in the elderly:
validation of an instrument. Can J Public Health. 1992b; 83(suppl 2):S7-S 11.

77. Wolf SL, Catlin PA, Gage K, Gurucharri K, Robertson R, Stephen K. Establishing the
reliability and validity of measurements of walking time using the Emory Functional
Ambulation Profile. Phys Ther. 1999; 79: 1122-1133.

78. Listen RA, Brouwer BJ. Reliability and validity of measures obtained from stroke
patients using the Balance Master. Arch Phys Med Rehabil.1996; 77:425-430.

79. Berg K, Wood-Dauphinee S, Williams JI, Gayton D. Measuring balance in the elderly:
preliminary development of an instrument. Physiother Can. 1989c; 41:304-31 1.

80. Shumway-Cook A, Woolhicoii MH. Motor Control: Theory and Practical Applications,
Philadelphia, Pa: Lippincott Williams & Wilkins; 2001:401, 405-406,

81. Whitney S, Wrisley D, Furman J. Concurrent validity of the Berg Balance Scale and the
Dynamic Gait Index in people with vestibular dysfunction. Physiother Res Int 2003;
8(4):178-86.

82. Shumway-Cook A, Baldwin M, Polissar NL, Gruber W. Predicting the probability for
falls in community-dwelling older adults. Physical Therapy 1997; 77(8):812-9.









83. Bogle Thorbahn LD, Newton RA. Use of the Berg Balance Test to predict falls in
elderly persons. Phys Ther 1996 Jun;76(6):576-83; discussion 584-5.

84. Whitney SL, Hudak MT, Marchetti GF. The dynamic gait index relates to self-reported
fall history in individuals with vestibular dysfunction. J Vestib Res. 2000; 10:99-105.

85. Guyatt GH, Kirshner B, Jaeschke R. Measuring health status: what are the necessary
measurement properties? J Clin Epidemiol. 1992; 45:1341-1345.

86. Stratford PW. Reliability: consistency or differentiating among subj ects (editorial). Phys
Ther 1999: 69: 299-300.

87. Shikiar R, Harding G, Leahy M, Lennox RD. Minimal important difference (MID) of the
Dermatology Life Quality Index (DLQI): results from patients with chronic idiopathic
urticaria. Health Qual Life Outcomes. 2005; 20;3:36.

88. Birmingham TB, Kramer JF, Speechley M, Chesworth BM, MacDermid J. Measurement
variability and sincerity of effort: clinical utility of isokinetic strength coefficient of
variation scores. Ergonomics 1998: 41: 853-863.

89. Bland JM, Altman DG. Measurement error. BMJ 1996; 313:744.

90. Roebroeck ME, Harlaar J, Lankhorst GJ. The application of generalizability theory to
reliability assessment: an illustration using isometric force measurements. Phys Ther
1993; 73(6):386-95.

91. Stevenson TJ. Detecting change in patients with stroke using the Berg Balance Scale.
Australian Journal of Physiotherapy 2001; 47:29-38.

92. Stratford P, Finch W, Solomon E, Binkley P, Gill J, Moreland C. Using the Roland-
Morris questionnaire to make decisions about individual patients. Physiotherapy Can
1996, 48:107-110.

93. Berg K, Wood-Dauphinee S, Williams JI. The Balance Scale: reliability assessment with
elderly residents and patients with an acute stroke. Scandinavian Journal of Rehabilitation
Medicine 1995; 27:27-36.

94. Tyson SF, DeSouza LH. Reliability and validity of functional balance tests post stroke.
Clin Rehabil 2004; 18: 916-923.

95. Mao HF, Hsueh IP, Tang PF, Sheu CF, Hsieh CL. Analysis and comparison of the
psychometric properties of three balance measures for stroke patients. Stroke 2002; 33:
1022-1027.

96. Ottonello M, Ferriero G, Benevolo E, Sessarego P, Dughi D. Psychometric evaluation of
the Italian version of the Berg balance scale in rehabilitation inpatients. Eur Med Phys
2003; 39:181-189.










97. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state": a practical method for
grading the cognitive state of patients for the clinician. Psychiat. Res 1975; 12(3): 189-
198.

98. Guyatt G, V Montori PJ, Devereaux H, Schuinemann H, Bhandari M. Patients at the
centre: In our practice and in our use of language. ACP Journal Club 2004; 140: A-11.

99. Chang JT, Morton SC, Rubenstein LZ, Mojica W, Maglione M, Suttorp M, et al.
Interventions for the prevention of falls in older adults: Systematic review and meta-
analysis of randomized controlled trials. British Medical Journal 2004; 328:680-683.

100.Gillespie LD, Gillespie W J, Robertson MC, Lamb SE, Cumming RG, Rowe BH.
Interventions for preventing falls in elderly people (Cochrane Review). London: Wiley
2001.

101. Skelton D, Todd C. What are the main risk factors for falls amongst older people and
what are the most effective interventions to prevent these falls? How should interventions
to prevent falls be implemented? Health Evidence Network Synthesis. Copenhagen,
Denmark: World Health Organization 2004.

102.Robertson M C, Devlin N, Gardner MM, Campbell A J. Effectiveness and economic
evaluation of a nurse delivered home exercise program to prevent falls: Randomized
controlled trial. British Medical Journal 2001; 322:697-701.

103. Stevens M, Holman CD, Bennett N, de Klerk N. Preventing falls in older people:
Outcome evaluation of a randomized controlled trial. Journal of the American Geriatrics
Society 2001;49: 1448-1455.

104.Robinson M, Brown J, George S, Edwards P, Atchison J, Hirsh A, Waxenberg L,
Wittmer V, Fillingim R. Multidimensional success criteria and expectations for treatment
of chronic pain: The patient perspective. Pain Medicine 2005; 6(5):336-45.

105.Centers for Disease Control and Prevention, National Center for Injury Prevention and
Control. Web-based Injury Statistics Query and Reporting System (WISQARS). [cited
2008 Jan 15]. Available from URL: www.cdc.gov/ncipc/wisqars.

106.Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in
health-related quality of life. J Clin Epidemiol. 2003; 56:395-407.

107.Beaton DE, Boers M, Wells GA. Many faces of the minimal clinically important
difference (MCID): a literature review and directions for future research. Curr Opin
Rheumatol. 2002; 14:109-14.

108.De Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM. Minimal
changes in health status questionnaires: distinction between minimally detectable change
and minimally important change. Health Qual Life Outcomes 2006; 4:54-59.









109. Schimnemann HJ, Guyatt GH. Goodbye MCID! Hello MID, Where Do You Come From?
Health Serv Res 2005; 40(2):593-597.

110.Ward SE, Gordon DB. Patient satisfaction and pain severity as outcomes in pain
management: a longitudinal view of one setting's experience. J Pain Symptom Manage.
1996; 11:242-251.

111.Pellino TA, Ward SE. Perceived control mediates the relationship between pain severity
and patient satisfaction. J Pain Symptom Manage 1998; 15:110-116.

112.Calkins E, Boult C, Wagner E, Pacala JT. New ways to care for older people. Building
systems based on evidence. New York: Springer; 1999.

113.Bergland A, Wyller TB. Risk factors for serious fall-related injury in elderly women
living at home. Injury Prevention 2004; 10:308-313.

114. Sattin RW. Falls among older persons: A public health perspective. Annual Review of
Public Health 1992; 13:489-508.

115. Tinetti ME, Doucette J, Claus E, Marottoli R. Risk factors for serious injury during falls
by older persons in the community. Journal of the American Geriatrics Society 1995;
43:1214-1221.

1 16.International Classification of Functioning (ICF), Disability and Health. World Health
Organization. Vol. 2002, 2001.

117.Gillespie LD, Gillespie WJ, Cumming R, Lamb SE, Rowe BH. Interventions to reduce
the incidence of falling in the elderly. Cochrane Library 1997; 4: 1-29.

118.Lichstein KL, Means MK, Noe SL, Aguillard RN. Fatigue and sleep disorders.
Behaviour Research and Therapy 1997; 35:733-740.

119.Karlsen K, Larsen JP, Tandberg E, Jorgensen K. Fatigue in patients with Parkinson's
disease. Movement Disorders, 1999; 14:237-241.

120.Nail LM. Fatigue in patients with cancer. Oncol Nurs Forum 2002; 29:537-546.

121.Swain MG. Fatigue in chronic disease. Clin Sci 2000; 99: 1-8.

122.Peat TG, Harris LH, Wilkie R, Croft PR. The prevalence of pain and pain interference in
a general population of older adults: cross-sectional findings from the North Staffordshire
Osteoarthritis Project (NorSTOP). Pain 2004; 110:361-368.

123.Blyth FM, Cumming R, Mitchell P, Wang JJ. Pain and falls in older people. European
Journal of Pain 2007; 11(5):564-571.









124.Cieza A, Brockow T, Ewert T, Amman E, Kollerits B, Chatterji S, et al. Linking health-
status measurements to the International Classification of Functioning; Disability and
Health. J Rehabil Med 2002; 34: 205-10.

125.Middel B, van Sonderen E. Statistically significant change versus relevant or important
change in (quasi) experimental design. Int J Integr Care 2002; 2: 1-22.

126.Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, O'Neill R, Kennedy
DL. Patient-Reported Outcomes to Support Medical Product Labeling Claims: FDA
Perspective. Value in Health 2007; 10(s2):S125-S137.

127.Rothman ML, Beltran P, Pharm D, Cappelleri JC, Lipscomb J, Teschendorf B. The
Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group Value in Health 2007;
10(s2):S66-S75.

128.Wagner EH, Austin BT, Von Koroff M. Improving outcomes in chronic illness.
Managed Care Quarterly 1996; 4(2):12-25.

129.Revicki. FDA draft guidance and health-outcomes research. The Lancet 2007;
369(9561):540-542 D.

130.Fairclough DL. Patient reported outcomes as endpoints in medical research. Stat
Methods Med Res 2004; 13:115-38.









BIOGRAPHICAL SKETCH

Sergio Romero was born and raised in the South of Spain. He moved to the USA in 1991

to pursue his studies in the area of exercise and sport science. In 1996, he received a bachelor' s

degree in exercise and sport science from the University of Florida. He continued his studies at

University of Florida and received a master' s degree in exercise and sport science in 1998. The

next few years, he worked in a variety of departments at this university and developed an interest

in geriatrics, and more specifically in falls prevention and falls rehabilitation. After taking a few

courses in geriatrics during 2002 and 2003, he was officially accepted in the rehabilitation

science doctoral program in the college of Public Health and Health Professions in spring 2004.

He specialized in movement disorders and continued working in the area of falls in the geriatric

population. He was involved in a variety of research proj ects. In 2007, he received a pre-doctoral

national fellowship from the Veterans Administration (VA) to work on his dissertation proj ect.

The financial support he received from the VA allowed him to dedicate a full year to the

completion of this dissertation proj ect. He received a Doctor of Philosophy degree from the

University of Florida in August 2008.





PAGE 1

1 MINIMAL DETECTABLE CHANGE AND PATI ENT REPORTED OUTCOMES IN FALLS REHABILITATION By SERGIO ROMERO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008

PAGE 2

2 2008 Sergio Romero

PAGE 3

3 To Cu

PAGE 4

4 TABLE OF CONTENTS page LIST OF TABLES................................................................................................................. ..........6 LIST OF FIGURES.........................................................................................................................7 ABSTRACT.....................................................................................................................................8 CHAP TER 1 INTRODUCTION AND LITERATURE REVIEW..............................................................10 Introduction................................................................................................................... ..........10 Theoretical Framework.......................................................................................................... .12 Clinical Change......................................................................................................................14 Clinical Important Differences............................................................................................... 18 Calculating Clinically Meaningful Change ............................................................................19 Anchor-Based Methods...................................................................................................20 Cross-sectional methods........................................................................................... 20 Longitudinal methods............................................................................................... 21 Distribution-Based Approaches.......................................................................................23 Paired t-Statistic....................................................................................................... 24 Growth curve analysis..............................................................................................24 Methods Based on Sample Variation.............................................................................. 25 Effect size.................................................................................................................25 Standardized Response Mean (SRM)......................................................................25 Methods Based on Measurement Precision..................................................................... 26 Standard Error of Mean (SEM)................................................................................26 Reliable Change (RC)..............................................................................................26 Choosing an Appropriate Method...................................................................................27 Patient Reported Outcomes (PRO).........................................................................................28 PRO Instruments.............................................................................................................29 PROs and the International Classification of Function (ICF).........................................30 Advantages of Using PROs............................................................................................. 32 Issues and Concerns about the Use of PRO.................................................................... 35 Summary.................................................................................................................................38 2 ESTABLISHING THE MINIMUM DETEC TABLE CHANGE (MDC) FOR THE BERG BAL ANCE SCALE AND DYNAMIC GAIT INDEX.............................................. 41 Introduction................................................................................................................... ..........41 Gait and Balance Assessment.................................................................................................41 The Berg Balance Scale (BBS).......................................................................................41 The Dynamic Gait Index (DGI)......................................................................................42 The Intraclass Correlation Coefficient (I CC), the Standard Error of Measurem ent (SEM) and the Minimal Detectable Change (MDC)................................................... 45

PAGE 5

5 Methods..................................................................................................................................46 Subjects............................................................................................................................46 Testing Procedure............................................................................................................47 Analysis....................................................................................................................... ....48 Results.....................................................................................................................................50 Discussion...............................................................................................................................52 Conclusion..............................................................................................................................56 3 PATIENTS SUCCESS CRITERIA AND EXPECTATIONS IN FALLS REHABILITATION ............................................................................................................... 61 Introduction................................................................................................................... ..........61 Methods..................................................................................................................................64 Subjects............................................................................................................................64 Testing Procedure............................................................................................................66 Analysis....................................................................................................................... ....67 Results.....................................................................................................................................69 Discussion...............................................................................................................................71 4 GENERAL SUMMARY AN D CONCLUSIONS .................................................................79 Experiment I Summary........................................................................................................... 80 Experiment II Summary......................................................................................................... 81 General Conclusions and Future Directions........................................................................... 83 APPENDIX Patients Perspective Outcome Questionnaire (PPOQ)........................................ 85 LIST OF REFERENCES...............................................................................................................88 BIOGRAPHICAL SKETCH.........................................................................................................98

PAGE 6

6 LIST OF TABLES Table page 2-1 Absolute difference in BBS scores be tween initial and re -scored assessments ................. 58 2-2 Absolute difference in DGI scores be tween initial and re-scored assessm ents................. 58 2-3 MDC values for the BBS and DGI....................................................................................58 3-1 Demographics............................................................................................................... .....77 3-2 Initial levels de scriptive statistics ...................................................................................... 77 3-3 Success criteria (initial levels-su ccess levels) descriptive statistics .................................. 77 3-4 Treatment expectations cr iter ia (initial levels-expectations levels) descriptive statistics..............................................................................................................................78

PAGE 7

7 LIST OF FIGURES Figure page 1-1 International Classification of Functi oning Disability and He alth (ICF) model ............... 40 2-1 Distribution of BBS diffe rence scores (Initial BBSre-scored BBS)................................ 57 2-2 Distribution of DGI difference sc ores (Initial DGIre-scored DGI) ................................. 57 2-3 BBS results for all participants. Mean value and difference between initial and rescored values. .....................................................................................................................59 2-4 DGI results for all participants. Mean value and difference between initial and rescored values. .....................................................................................................................59 2-2 Mean BBS score for different absolute di fferences in BBS between testing occasions .... 60 2-3 Mean DGI score for different absolute differences in DGI between testing occasions..... 60

PAGE 8

8 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy MINIMAL DETECTABLE CHANGE AND PATI ENT REPORTED OUTCOMES IN FALLS REHABILITATION By Sergio Romero August 2008 Chair: Craig A. Velozo Co-chair: Kathye E. Light Major: Rehabilitation Science The overall aim of this project was to investigate the reliabilit y of two instruments used for the assessment of balance and to explore patients expectations and success criteria for the rehabilitation of falls. The first experiment investigated minima l detectable change (MDC) for two common instruments used to assess gait and balance. The re sults of this study indicated that for the Berg Balance Scale and the Dynamic Gait Index, 6.6 and 3.1 points respectively we re required to be 95% confident that genuine change had occu rred. These results suggest that a significant amount of error is associated w ith these instruments. In additi on, the results suggested that MDC values are not a constant feature of the inst ruments. MDC values for the high function group were 6.3 BBS points, as compared to 7.3 points for the low function group. Th at is, the values of MDC change based on the ability level of the persons assessed. The second experiment investigated patients success criteria a nd expectations with treatment. Participants reported considerable initial levels of impairment in energy and drive, mobility, and pain. Lower scores were seen in interaction with people and community and social

PAGE 9

9 life. These findings suggest that domains with a strong social com ponent were not as affected as domains with a strong physical component. Participants in this study required signifi cant improvement to consider their treatment successful. Domains such as mobility; and energy and drive, required significantly larger reductions than the community and social life; and interactions with people domains. This provides information about what is important to patients receiving this intervention. Participants expected mobility to change the most. However, similar finding was reported in the domain of energy and drive. An interesting finding was that, participants expectation was that the treatment would not meet their success criteri a, indicating that residua l levels of impairment were expected. Collectively, this series of studies promotes our understand ing of significant change in patients receiving rehabilitation services related to falls. The results obtained indicate that current rehabilitation programs must consider the limita tions of available instruments and take into consideration the needs and e xpectations of patients.

PAGE 10

10 CHAPTER 1 INTRODUCTION AND LITERATURE REVIEW Introduction The elderly population is the fast est growing group in the USA [1]. By the year 2030, it is expected tha t one in five Americans will be 65 or ol der. This represents an increase in the elderly population that will double that of the total population growth [2]. This epidemiological profile is particularly relevant to the national healthcar e system, as the patien t population will also experience similar growth. This growth in the number and proportion of older people will place increasing health and economic demands on the national healthcare system. Although aging is a highly individual-de pendent phenomenon, a progressive loss of biological function is expected wi th age. Sometimes, this can lead to functional decline, loss of independence, and, ultimately, a d ecrease in quality of life. In a number of elders, this biological decline can result in compromised mobility. This is one of the most disabling conditions elders can experience, since mobility limitations restri ct the individual from fully participating in everyday activities and can contribute to further functional decline [3]. Individuals with mobility problems are prone to falls. The consequences of falls are far reaching and affect not only the individual, but also caregivers and the healthcare system in general. Falls are a serious health problem in the ol der population. They are the leading cause of injury deaths among people 65 and older [4]. In 2002, nearly 13,000 people ages 65 and older died from fall-related injuries [4]. Falls affect more than one-third of adults aged 65 years and older each year [5]. As a result of the rapid gr owth in this segment of the population, fall-related medical care is imposing an enormous demand on the US healthcare systems. In 1994, the cost of fall-related injuries was $27.3 billion. This nu mber is expected to reach $43.8 billion by the

PAGE 11

11 year 2020 [6]. Preventive and rehabilitative strate gies are needed to stop, or at least slow down, this trend. Several important areas need to be addressed relating to falls in the elder population. First, there is a need to accurately identify the population at risk. Next, assessment tools used to identify individuals at risk and measure perf ormance change must be reliable and convey information that is clinically relevant and eas y to interpret. Finally, effective and feasible rehabilitation strategies must be developed to specifically targ et this population. To this end, rehabilitation goals must include the patients criteria to ensu re adherence to treatment and accurately determine treatment success. The overall goals of this dissertation are to investigate meaningful change in two assessment tools commonly used in falls rehabilitation programs, and to explore patients expectati ons and success criteria in th e rehabilitation of falls. In the subsequent literature review I will develop the b ackground to address the studys purpose. First, I will introduce the conceptual mo del that has guided this dissertation, the World Health Organizations International Classifica tion of Function and Disability (ICF). The ICF provides a theoretical framework to encompass the multiple dimensions of the rehabilitation process and the factors that affect this process. Next, I will review the concept of clinical change and all of its variations (e.g. minimal detectable change, clinically important change, etc.). Rehabilitation decisions are based on assessi ng and reassessing patients to determine the effectiveness of treatment. A clear understanding of the properties of our assessment instruments and how to interpret the result s obtained with these instruments is of utmost importance to clinicians and researchers. Third, I will review the literature pertaining to Patient Reported Outcomes (PRO), since understanding what the pa tient values in the rehabilitation process is

PAGE 12

12 critical. Finally, I will provide a summary of this literature review and how it supports the overall purpose of this dissertation. Theoretical Framework Several theoretical m odels have been develope d and explored over the years. In the field of rehabilitation, there has been a shift from what was previously known as disabling models, to a more proactive theoretical framework te rmed enablement models. In rehabilitation sciences, an enablement model frequently us ed is the Internati onal Classification of Functioning, Disability, and Hea lth (ICF), proposed by the World Health Organization [7]. This model was used to guide the research ques tions and analyses for this dissertation. The ICF provides a theoretical framework for the analysis of health conditions, body structure and function, activity, a nd participation, and environmen tal and personal factors. There are two main components in the ICF model. The first is concerned with functioning and disability and includes the areas of body structure, body func tions, activity, and participation. The second part includes the components of contex tual factors, and includes environmental and personal factors (see figure 1.1). In this dissertation, the hea lth condition of interest include s a variety of disorders and diseases that influence gait and balance in the elder population and can lead to falls. The ICF model offers a holistic approach that considers the dyna mic interaction of different aspects of a health condition from a biological, i ndividual, and social perspective. In individuals with gait and balance disorders, a number of health conditions contribute to a decline in biological function that can lead to falls. In the IC F model, this is represented at the body function and structure side of the model (Figure 1.1). This can result in rest riction and inability to perform certain activities and can also influence the individuals social participation. This tran sition from body function and structure to activity and part icipation limitations can also wo rk in the opposite direction. For

PAGE 13

13 example, a decrease in particip ation and activities can lead to deterioration at the body function and structure level. In addition, personal and envi ronmental factors are also taken into account by ICF model to draw a comprehensive picture of how an individual, within his or her limitations, and conditioned by the environment and his or her own personal characteristics, functions in society. This dissertation focuses in two distinct, although directly connected, aspects of the rehabilitation of individuals w ith gait and balance disorders. First, two assessment instruments used in clinical practice to evaluate gait and ba lance will be studied. These instruments are used to test individuals ability to perform certain task s related to gait and balance. Therefore, the first experiment in this dissertation will be investig ating the activity compone nt of the ICF model. Secondly, a patient reported outcome questionnaire will be used to investigate individuals expectations and success criteria when participati ng in a gait and balance intervention. Since the ICF is not only a theoretical model but also a classification system, usi ng the coding system of the ICF, a comprehensive picture of an indi viduals experience when going through a gait and balance intervention can be drawn. The ICF categories facilitate the description and classification of all aspects of function and health in individuals, independent of a specific assessment instrument. The current ICF lists 1,424 categories referri ng to body functions and structure, activities and participation, and environmental factors [7]. The ICF models classification system was used to produce a pa tient reported outcome questionnaire to identify specific areas of the model that are important to individuals with ga it and balance problems including: mobility, self-care, interactions w ith people, community and social life, energy and drive, mental function, emotional di stress, sensory function, and pain.

PAGE 14

14 Figure 1.1 shows the areas of the model tested in the two experime nts included in this dissertation. The ICF model emphasizes interconnectivity of all the areas of the model. This can lead to some overlapping, with some concepts being represented in more than one area of the model. Sensory function and pain, energy and driv e, mental function, and emotional distress are included at the level of body function and structure. Some of these concepts are highly connected to personal factors and could also be represented in this area of the model. Balance, self care, and mobility are included under the area of activit y. Again, one can argue that some of these concepts could be included under participation. Fi nally, interactions with people, and community and social life have been included under the partic ipation area of the mode l. Although contextual factors are not specifically addressed in this dissertation, they must be considered when interpreting the resu lts of this investigation. A combin ation of functioning and disability, together with contextual factors contributes to the ultimate goal of addr essing quality of life and life satisfaction. Clinical Change Selecting o utcome measures: Health research is based on scientists ability to measure the different dimensions of the health construct. In the past few decades, the medical community has shown an increased interest for the use of ev idence-based clinical practice. This interest has been supported by governing agencies, insuran ce companies, and consumers. These groups recognize the need for in terventions that are scientifically driven, produce measurable and meaningful results, and are based on theoretical models that cover the entire spectrum of health. To this end, many measurement instruments have been developed to assess different aspects of health. The selecti on of a suitable instrument gene rally depends on the goals and characteristics of the population being assessed. In rehabilitation, most assessment tools aim at evaluating functional aspects rela ted to health. In general, the relevance of measurement

PAGE 15

15 instruments can be optimized by incorporating a theoretical framework of health and disability, establishing the purpose of the measurement, and assuring that th ese instruments are psychometrically sound. A theoretical framework of health and di sability provides the conceptual basis for developing and using an assessment instrume nt. For example, under the International Classification of Function Disability and Health (ICF) [7], a researcher can develop measurement instruments to assess the areas of body function and structure, activities, and participation, and the influence of personal and environmental factors [8]. Using a theoretical model helps researchers identify the domains of in terest and provide a complete view of how the different domains interact with e ach other to affect the overall h ealth status of the individuals under study. Another essential factor in the developmen t and selection of outcome measures is to establish the purpose of the measurement. In ge neral, assessment instruments are used to discriminate among individuals, predict future outcomes, and evaluate interventions [9]. Discriminative instruments are used to differentiate between individu als based on specific criteria when no external gold standard exis ts for validating these measures [10]. These instruments are used for invest igating between-subject differences where groups of individuals are assigned to separate treatment conditions. For example, a researcher interested in investigating differences in balance among two groups of elders with Park insons disease should use a discriminative assessment instrument. Pred ictive instruments are used to categorize individuals into predetermined category when a gold standard is available [10]. This gold standard is used to determine whether individuals have been classified correctly. For example, a shorter version of an instrument can be used to assess a particular condi tion. Later, the results

PAGE 16

16 can be compared to the original or gold standa rd instrument. Finally, evaluative instruments are used to measure the magnitude of longitudinal ch ange in an individual or group (within-subjects experimental design) [10]. In this type of inve stigation, two measurements are obtained from the same sample. Changes in performance within ea ch participant across treatments are used to determine a treatment effect. Measurement instruments used in reha bilitation must have sound psychometric properties. In classical test th eory, these properties assure th at the instruments we use can provide information that is m eaningful, valid, and consistent with the construct we are measuring. The key traditional psychometric propertie s of an instrument are validity, reliability, responsiveness to change, and minimal clinic ally important difference (MCID) [12]. Validity refers to the ability of an instrume nt to measure what it is intended and presumed to measure [13]. A valid measure must be reliable (consistent). However, a reliable measure does not have to be valid. Validity can be investigated and defined in a number of ways. First, it is possible to correlate measures with a criterion measure known to be valid. This is considered the criterion validity. If the criter ion measure is collected at the same time as the measure being validated, the concurrent validity is obtained. When the criterion is collected later, the validity obtained is the predictive validity. A different type of validity is based on the construct of the instrument been used. Construct validity refers to whether an instrument measures the construct it is supposed to measure. Finally, content validity, or face validity, is simply the extent to which a measure represents all facets of a given construct [14]. Instruments used in rehabilitation must also be reliable. A measurement is reliable when repeatedly testing a particular subject under the same conditions produces the same results [15]. A measurement can reliably measure the wrong attribute. Such a measurement will be reliable,

PAGE 17

17 but not valid. There are a number of ways to determ ine reliability. First, inter-rater reliability is used to assess the degree to which different raters give consistent measurements of the same phenomenon. Second, test-retest reliability is used to assess the consistency of a measure from one time to another. And third, inte rnal consistency reliability is us ed to assess the consistency of results across items within an instrument [16]. Responsiveness is also important to de termine changes over time, which may be indicative of therapeutic effects. Instrument responsiveness is th e ability of the instrument to precisely detect meaningful ch anges [17]. The measurements of an instrument used in rehabilitation must be able to identify clin ically significant differences between and within patients over time [18]. In addition, the instrument should only be responsi ve to changes in the variable being assessed, and should not be influenced by changes in other variables [19]. Related to the responsiveness of an instrument are the concepts of sensitivity and specificity. Sensitivity is the ability of an instrument to detect changes in the variab le under study, when they occur. It is a measure of the probability of correctly identifying a change. Speci ficity is the ability of an instrument to correctly identify when no changes in the variable under study occurs. It is a measure of the probability of corr ectly identifying no change [11]. Finally, related to the concept of responsiveness, is the ability of an in strument to detect minimal clinically important differences (MCI D). The minimal important difference has been defined as the smallest differen ce in score in the domain of inte rest which patients perceive as beneficial and which would mandate, in the abse nce of troublesome side-effects and excessive cost, a change in the patient's management [21]. The concept of clinically important change introduces a new variable in measurement; the pa tients perspective. In addition, clinically important change can have consequences at the c linicians, researchers and societal level. Out of

PAGE 18

18 all psychometric properties desirable in any a ssessment instrument, none has more practical implications than the ability of an instrument to detect MCID. At the end, the ultimate goal of an intervention is to produce results that are important to the persons receiving the treatment. An in depth analysis of MCID follows. Clinical Important Differences Interpretation of clinical assessments is ofte n difficult. This is especially true when the variables measured are based on some abstract co nstruct. For example, in terpreting a change in blood glucose levels of 20 mg/dL is easy, but inte rpreting a change of 4 points in a Quality of Life instrument can prove to be a difficult tas k. Therefore, establishing clinical importance is more difficult as the concept being assessed be comes more abstract. In addition, clinically important differences may be different across gro ups of patients defined by diseases, levels of severity, cultural background, socioeconomic status, and nationality [22]. Determining clinically meaningful differences is important because, sometimes, studies are based on small differences in mean scores between groups, which can lead to a statistically significant difference when the sample sizes are large. However, statistical signifi cance is not equivalent to clinical significance [23]. Different perspectives in clin ically important differences: There are several important issues to consider when looking at clinically impor tant changes. First, from the point of view of the patient, a meaningful change may be one that results in a meaningful reduction in symptoms or improvement in functional status. On the other hand, clinicians can consider a meaningful change when the patients improvement results in a change in treatment or disease prognosis. These two perspectives may not coincide all th e time. In addition, societal and institutional perspectives for determining what constitutes a c linically important change can also differ from the clinicians and patients vi ew [23]. From societies perspective small changes can be

PAGE 19

19 meaningful if the condition of interest affects a large number of people. Institutions may be more concerned with changes that influe nce health care policies [23]. Another important issue to consider is whet her clinically meaningful changes are based on individual or group differences. When looking at group differences, it is important to take into consideration that mean changes do not provide information about individual scores. It is possible that even in groups w ith small mean differences a number of subjects could exhibit significant changes. On the other hand, these di fferences could be attributed to measurement error associated with the measur ement instrument. Therefore, when investigating issues related to public health, where even small differences ca n have great impact, reporting group differences is appropriate [22]. Conversely, differences at the individual level are more relevant when individual decisions about a par ticular treatment must be made Furthermore, the amount of change necessary to be consider ed clinically meaningful is al so influenced by whether it is applied to an individual or a group. Relatively small improvements at the individual level may be considered clinically important when looking at the group level [22]. Calculating Clinically Meaningful Change To date, two broad strategies have been s uggested for calculating MCID: 1) anchor-based measures and 2) distribution based approaches [24]. Anchor-based methods examine the relationship between an instruments measure and an independent measure (or anchor) to explain the meaning of a particular degree of change. Therefore, anchor-based approaches need an independent standard or anchor th at is itself interpretable and at least moderately correlated with the instrument being explored [25]. On the othe r hand, distribution based approaches rely on the statistical distribution of scores in a given instrument [25].

PAGE 20

20 Anchor-Based Methods Anchor-based methods have been used to determine clinic ally meaningful change via cross-sectional and l ongitudinal approaches. Cross-sectional methods A cross-sectional approach is used when comparing groups that are different in terms of some disease-related criterion [26]. The differe nce in mean values across groups is used to estimate the minimal clinically important diffe rence. For example, in well known medical conditions where severity stages have been determined, such as in Parkinsons disease (Hoehn and Yahr scale), a difference equi valent of moving from one stage to the next can be used as MCID. Cross-sectional methods can also be used to compare individuals with and without a particular diagnosis. For example, Johnson et al. [27] investigated differences in SF-36 scores in patients with hypertension. They found that hyperte nsive patients scored on average 4.1 points lower on the SF-36 compared to those without hypertension. They determined that this difference (4.1 points) could be used to establish MCID. One disa dvantage of this approach is that generalizing the results to other samples can be misleading because it is difficult to control for other variables that can cause the group differe nces. In addition, as with all cross-sectional designs, differences in mean scores may not accurately reflect true change [25]. MCID can also be inferred by linking the re sults to some extern al, non-disease related criteria. For example, Testa and Simonson [28] suggested that a 0.1 standard deviation decrease in the General Perceived Health scale was comparab le in importance to the stress associated with experiencing the death of a close friend. These external criteria can provide a useful, easy to understand, anchor for comparison; however, interpretation of the results can be difficult in some cases. Again, as in other anchor-based approach es, there must be an assumption that all other variables remain stable (do not change).

PAGE 21

21 Another cross-sectional appr oach consists of dichotomizing patients based on functioning level after treatment (functional vs. non-functional group). Based on the principle that a patient should be in the normal range of functioning afte r clinical intervention, Jacobson and Truax [29] propose three possibilities for identifying recovery status. First, a patient' s score after clinical intervention is 2 standard devi ations better compared to th e dysfunctional group. Second, the patient's score after clinical intervention is wi thin 2 standard deviations of the functional population. Finally, the patient's scor e after clinical intervention is closer to the mean of the functional population than the mean of the dysfunctional popul ation. This method bases the identification of MCID on the ab ility of patient to move from one category to the next. The use of cross-sectional comparisons has received two major criticisms. First, it is likely that when comparing two groups, more th an one variable may be responsible for the differences between groups. Some researchers have suggested th e use of regression models to control for other possible variab les [30]. Second, some research ers argue that cross-sectional differences are not always e qual to longitudinal changes us ing the same groups [31]. Longitudinal methods This approach is used when comparing group changes across time. One of the most commonly used anchor-based approaches for establishing clinically meaningful change in longitudinal studies is the use of global ratings of change. Jaeschke et al. [32] used this approach to investigate MCID in patients with respiratory problems. They used the Chronic Respiratory Questionnaire and the Chronic Heart Failure Questionnaire to address change in dyspnea, fatig ue, and emotion. After treatment, patients were asked about their global rating of change. Based on their responses, th ey were classified into four groups: "no change," "minimum," "moderate," and "largest," for each domain. After

PAGE 22

22 investigating the mean change in the three domai ns, the authors concluded that a difference of half a point constituted an MCID. Another longitudinal method for establishing clinically meaningful change involves the prognosis of future events. This method looks at individuals who experi ence a particular event such as mortality, use of medical care, cost of interventions or tim e to discharge [33]. Differences in individuals that experience and do not experience the event are used to determine MCID. A final method for investigating MCID is the use of receiver op erating characteristic (ROC) curve. The ROC curve method attempts to discriminate between patients who do and do not achieve clinically significant change using a single cutoff point [34]. Sensitivity, probability that a test result will be positive when the diseas e is present (true positive rate, expressed as a percentage), is plotted against sp ecificity, probability th at a test result will be negative when the disease is not present (true nega tive rate, expressed as a percentage). Each point on the curve represents a different cutoff. Us ually, the point where sensitivity and specificity have the highest value is chosen as a MCIF cutoff point. Some st udies choose the point where sensitivity equals specificity. Both of these cutoff choices are arbitrary because they do not consider the differences in importance between false positiv es, false negatives, and correct identification. MCID estimates based on arbitrary ROC curve cutoffs could be very different from estimates from cutoffs that compare correct and incorrect classifications [34]. In general, anchor-based techniques offer the advantage of linking changes in the variable of interest to outside meaningful anchors. This is particularly useful wh en investigating results for which the patient is the major source of inform ation, such as in pain research and quality of life investigations. However, anchor-based appr oaches have several limitations. When using

PAGE 23

23 global ratings, it is possible to ob tain results that are affected by recall bias, especially when the time delay is long. Patients may simply forget or be affected by their current life situation. In addition, this method offers no information about the reliability and validity of the responses obtained. Another potential limitation is the generalizability of results obtained with different anchors. The use of different anchors may lead to different conclusions about the amount necessary to determine MCID. So me studies have found that conc lusions obtained from anchorbased methods may vary depending on whether the anchors were obtaine d prospectively or retrospectively [35]. Most importantly, anchor-based methods do not take into considera tion the precision of the instrument used. It is possible that MCID es tablished by this method are within the range of error in the instrument. Any cha nge within this range cannot be attributed to the treatment or intervention. Furthermore, interp retation of results may be diff icult if there is not a linear relationship between the scores and the anchor chosen [35]. Distribution-Based Approaches Distribution-based approaches to dete rmine MCID are based on the statistical characteristics of the obtained sample. Three cate gories of distribution-based measures have been proposed [23]. These are: methods based on stat istical significance, methods based on sample variation, and methods based on measuremen t precision. Methods based on statistical significance evaluate change taking into consideration the probability that this change occurred by random variation. These methods are affected by sample size. Therefore, other things being equal, increasing the sample size may yield resu lts that are statistically significant. Two approaches that use these methods include the pair ed t-statistic and growth curve analysis [36]. The second category includes methods based on samp le variation. Different types of variation used include baseline variation of the sample (effect size), variation of change scores

PAGE 24

24 (standardized response mean), and variation of change scores in a stable group (responsiveness statistic) [37]. These methods are independent of sample size, because variation is expressed as an average variation around a mean value. The last method is based on the measurement precision of the instrument. These methods evalua te change in relation to variation of the instrument instead of variation of the sample. They include the st andard error of the mean (SEM) and the responsiveness statistic. These methods are also sample-independent. Paired t-Statistic The t-statistic is used to test the hypothesis th at there is no change in the average response on a measure over two time points. The paired t-statistic has been commonly used in a one-group repeated measures design [38]. It is calculated as the difference between pre-test and post-test scores divided by the standard error of measure change [39]. A concern with the use of this method to measure individual change is the fa ct that it only accounts for the statistical significance of the difference. This difference depends not only on the amount of change, but also on the sample size and the variability of the measure [39]. If used to establish a cutoff point, increasing the sample size will reduce the amount of difference necessary to reach this threshold. Statistical significance is not appropriate to establish the clinical importance of a change in score. Growth curve analysis Individual growth curve coefficients can be estimated using hierarchical linear modeling [40]. Improvement rates are calculated by dividi ng the empirical Bayes estimated linear slope by the empirical Bayes estimated poste rior standard error of the slope [40]. Th is method, like the tstatistic, is influenced by sample size. Sp eer and Greenbaum [41] claim that this method performs better than other distribution-based met hods because it uses all data points to establish rates of change. They also report that a limitation of this method is the fact that it requires large samples to provide stable estimates of change [41]. Another limitation of this method is the

PAGE 25

25 assumption of not having any missing data point s or that missing data points are randomly missing. Violations of this assumption can result in biased co nclusions [42]. Methods Based on Sample Variation Effect size Effect size is a broad name given to a numbe r of indices that measure the magnitude of a treatment effect. Unlike previously presented sign ificance tests, these indi ces are independent of sample size. Cohen [43] defined the standa rdized difference betw een two groups as the difference between the means, divided by the standard deviation of either group. Cohen concluded that the standard devi ation of either group could be used when the variances of the two groups are homogeneous. He also established gui delines for the interpre tation of effect size; for example, .20 for small effects, .50 for moder ate effects, and .80 for large effects. Some researchers have investigated MCID based on e ffect size. Samsa and colleagues [22] propose an effect size of .20 as an appropriate definition of a MCID. Some limitations of using the effect size include the need for a homogeneous distribution; the size of the standard deviation, either at base line or after treatment, will have an invers e effect on effect size; and with large standard deviations producing smaller effect sizes [44]. Standardized Response Mean (SRM) SRM is defined as mean score change divide d by the standard deviation of that score change [38]. A large SRM indicates that the change is large relati ve to the background variability in the measurements [45]. The SR M also uses cutoff points of .20, .50, and .80 to define small, moderate, and large effect sizes [4 6]. One limitation of the SRM is that comparable individual changes that have di fferent SRM values depending on the variability of change in the sample [45].

PAGE 26

26 Methods Based on Measurement Precision Standard Error of Mean (SEM) The SEM is a measure of the precision of an instrument. Therefore it is closely related to the concept of minimal detectable change. The SEM is the standard error in an observed score when the true score is not captured by the instru ment used. In other words, it indicates how close a persons score is to their true sc ore; the score that they would ge t if a test could be completely error-free [47]. It is calculated by using the sample standard deviation and the sample reliability coefficient. The exact formula estimates the SEM as the standard deviation of the instrument multiplied by the square root of one minus its re liability coefficient [47]. The SEM is considered to be an attribute of the measure and not a characteristic of the samp le [47]. It is possible that the SEM of a particular measure would vary based on the method used to estimate the reliability coefficient and the presence of extreme scores For MCID, SEM values of 1 SEM, 1.96 SEM, and 2.77 SEM have been suggested [48-49]. The SEM is expressed in the original metric of the measure it describes. This is important because it can help with interpretation of the results. Moreover, the SEM is a theoretic ally fixed parameter of a measure [49]. This means that for nearly all true scores, the devi ation around the true score from repeated measurements is about the same [50]. Reliable Change (RC) A reliable change index is based on the amount of change that indicates the extent to which the observed change exceeds measurement e rror [50]. This index is referred to as the standard error of measurement difference (SEMD) The SEMD is directly related to the SEM, but produces smaller values. Therefore, this met hod is more conservative for a given cutoff value than the SEM approach, classifying fewer individuals as improved or deteriorated. A cutoff value of 1.96 has been suggested to determine whet her an observed change in scores over time

PAGE 27

27 should be categorized as unchanged, improved, or deteriorated [50]. A disadvantage of this method is that it assumes that measurement error is constant across the range of possible scores [50]. Choosing an Appropriate Method A number of methods have been proposed to investigate MCID. Anchor-based approaches have the advantage of linking changes to a meaningful external anchor. In addition, some of these methods include the most important measure of the signif icance of change; the patients perspective. However, these methods do not consider the possible range of error associated with all instruments. In addition, interpretability of re sults is difficult when comparing investigations that use differe nt external anchors. On the other hand, distribution based approaches provide a way to establish amount of change outside the limits of the instruments error. In addition, these approach es provide a common metric that has equivalent meaning across measures, populations, and studies [50]. The distri bution-based approaches that are better suited for establishing MCID are those based on the measurement precision of the instrument (SEM, RC). These measures establish the amount of error that is inherent to the instrument and the amount of random error that can be expected in repeated measures. In addition, they are not influenced by variability in the sample at base line (as is the effect si ze), variability of the observed change (as is the Responsiveness Statistic), or the sample size (as are the t-statistic and growth curve analysis). Finally, these measures can be used to establish cutoff points based on a desired confidence level [50]. The first experiment in this dissertation uses the SEM method to investigate differences in balance scores (Berg Balance Scale and D ynamic Gait Index) that represent a minimal detectable change. The SEM is a measure of res ponsiveness and can also im ply reliability of an instrument. The SEM expresses measurement error in the same units of the original tool and is

PAGE 28

28 not influenced by variability among s ubjects. This technique is well su ited for clinical practice. It provides an intuitive score, in th e same unit of the original instru ment, which can be used across measurements and populations. In addition, beca use the SEM looks at within individual variability, this technique is more appropriate than other re sponsiveness approaches when therapists need to make a deci sion about an individual [51]. Patient Reported Outcomes (PRO) Health professionals approach to patients and their problem s is grea tly influenced by the conceptual models around which their knowledge is organized. Traditionally, the effects of a particular disease or medical condition have been assessed by using methods such as physiological exams, performance tests, and c linical observations. These methods have been employed, and still are, to fulfill the requirements of a conceptual model described as the Biomedical Model. In this model, a diseas e or medical condition is the result of a pathophysiological event, intrinsic to the individual, re sulting in a reductio n of the individual's quality of life. As a result, curing or mana ging a disease or medical condition revolves around identifying the disease, unders tanding it, and learning to cont rol and alter its course [52]. Biopsychosocial Models, such as the one used to guide this dissertation, the ICF, expands the concept of disease to include not only the pat hophysiological event, but also the psychological and societal consequences of the disease pr ocess [53]. With the acceptance of this new conceptual model, comes a need to develop tools that can effectively measure individuals social and personal factors that affects the disease pr ocess and can play an important role in the diagnostic and rehabi litation process. Measuring the impact of social and psychologi cal factors in the disease process requires participation of the patient. Fo r some medical conditions the pa tient is the only source of information. For example, depression is a condit ion that, often, has no observable or measurable

PAGE 29

29 physical symptoms. Clinicians must rely on in formation provided by the patient to correctly diagnose and treat patients with this medical co ndition. In addition, to establish endpoints in the rehabilitation of these patients, cl inicians must use tools that take into consideration the patients perspective. Patient reported outcomes (PRO) instruments b ecome indispensable in these situations. PRO Instruments PRO has been defined by the Federal Drug Adm inistrationas: "Any report coming directly from patients (i.e., study subjects) abou t a health condition and its treatment" [54]. PRO instruments are used to measure treatment benef its by capturing concepts re lated to how a patient feels or functions with re spect to his or her hea lth or condition. The ideas activities, behaviors, or feelings measured by PRO instruments can be e ither verifiable in natu re, such as walking, or can be non-observable, known only to the patient, su ch as pain, depression etc. Although these symptoms are highly dependent on the patient perception, historically, these assessments were often made by clinicians who observed and inter acted with patients. Recently, these kinds of assessments are increasingly performed with PRO instruments. The idea of asking patients about their feelings and symptoms is not new. In fact, doctors have used this technique thr oughout the history of the medical profession [55]. What makes PRO instruments different is the f act that information about symp toms and performance is being obtained directly from patients. This is done without interpretation from clinicians, using structured questionnaires that are shown to give reproducib le, meaningful, and quantitative assessments of how patients feel and how they f unction [56]. Therefore, the adequacy of a PRO instrument is based on its ability to capture the patients evaluation of the impact of disease on their functioning and well-being [57-58]. For this reason, PRO in struments can be categorized

PAGE 30

30 under a new set of instruments that uses a conceptual framework where the patient is the focus of the assessment. PROs and the International Classification of Function (ICF) The ICF is a thorough fram ework that can be used as a reference to evaluate, compare and classify instruments used in the rehabilita tion field. In addition, the ICF is a classification system and can be used to develop new inst ruments, using the coding system it provides. Because the ICF was developed to include social, personal and environmental aspects of health and disability, it is an id eal framework for the evaluation of PRO instruments. PRO instruments provide information about how patients feel or function with respect to their health or condition. Info rmation generated by a PRO instrument can serve to measure treatment benefit, from the patient perspective. To arrive at this conclusion, there must be evidence that the PRO instrument is based on a va lid theoretical construct. In fact, one of the most important psychometric properties of a tool is its construct vali dity. Construct validity refers to the degree to which inferences can legiti mately be made from the measures in a study to the theoretical constructs on wh ich those measures were based [59]. PRO instruments can be used to measure simple constructs such as the ab ility to perform activities of daily living or more complex constructs such as quality of life, which includes physical, psychological and social components. The ICF model can be used as a reference to investigate what areas of the rehabilitation spectrum are covered by PRO instruments. Activities and participation (right side of the model) are the areas of the ICF model measured by PRO instruments. For example, one of the most widely used PRO instruments is the SF-36 [60]. Th is instrument measures generic health status in the general population. Patients are asked to eval uate their general health and limitations in activities as a result of their physical health or emotional problems. Although this instrument was

PAGE 31

31 not developed using the ICF as a reference, it co vers many areas of the model. Questions such as: Does your health now limit you in these activities? If so, how much? or During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups? as sess the ICF areas of activity and participation. Although personal fact ors are not directly assessed with this instrument, this must be considered when interp reting the results. Individu als perception of their mental and physical function is h eavily influenced by personal factors such as their personality, cultural background, and societal ro les. Except for measuring the in fluence of the environment, the SF-36 covers important psycho-so cial areas of the ICF model. Other PRO instruments are specifically design ed using the ICF as a theoretical model and the code it provides as the basis for classifica tion. For example, the PAR-PRO, a measure of home and community participation, was develope d as a broad measure of home and community involvement for persons with disabilities [60]. This instrument is based on the domains of activities and participation in the ICF, including learning an d applying knowledge, communication, mobility, self-care, domestic life, interpersonal interactions and relationships, major life areas, and community soci al and civic life. A particular challenge of the ICF is to be able to distinguish between activities and partic ipation when using the coding in the activities and participation domains. Often, the only possible indicator of participation is coding through performance, which might also have to be coded as frequency of participation in a particular activity [60]. However, instruments such as th e PAR-PRO demonstrate the potential for using the extensive classification system of the ICF fo r the development of PRO instruments. In this dissertation, a questionnaire to a ssess the patients perspectiv e was developed using the ICF

PAGE 32

32 classification system to assess areas of the m odel of particular importance to individuals undergoing gait and balance reha bilitation related to falls. Advantages of Using PROs Many chronic conditions have a debilitating e ffect that progressively deteriorates the patien ts' quality of life. Chronic diseases have social, personal and mental implications that can lead to fatigue, depression, pain, and isol ation. Traditional measurement tools (e.g. blood pressure instruments, or blood sugar level counts), are very accura te at measuring physiological function. However, a new set of sensitive and well -validated tools are needed to improve and standardize measurements of symptoms related to social, personal, and environmental factors associated with health. In genera l, the use of the patients persp ective becomes more important as the variable being measured becomes more abstract. Concepts that do not have a well define physiological component such as pain, quality of life, life satisfaction, or self-efficacy, can be better understood by considering the patients perspective. PRO instruments are ideal and, sometimes, th e only way to measure ICF concepts of activity, participation and the in fluence of personal and environmental factors in health and disability. For example, when measuring quality of life (QOL), a concept that is highly individual and context dependent, we have no choi ce but to use PRO instruments. In fact, some researchers and clinicians use these two terms i ndistinctively [61]. QOL or health related quality of life (HQRL) questionnaires are PRO instru ments that explicitly include the patients perception of the broad impact of disease on their functioning and overall wellbeing [62]. Ultimately, the goal of a therapeutic intervention is to impact health and increase function and quality of life. Consequently, th e use of PRO instruments in clin ical practice to determine end points and evaluate treatment effectiveness is cr itical. This has been recognized by many national and international agencies (FDA [54], NIH [63], European Agency for the Evaluation of Medical

PAGE 33

33 Products [10], WHO [65]) th at identify the need to use PRO instruments to improve health and quality of health care. There is enough evidence that support the use of PRO instru ments, especially those that assess QOL, as a treatment outcome. In fact, QOL has been a strong prognostic variable for survival in several cancer related studies [66-67] QOL data can be especially important when two treatment options with similar survival outcomes are available. In these cases, QOL outcomes can be the deciding factor for choosing a particular treatment. For example, a woman with breast cancer might face a decision of whet her to opt for a mastectomy or conservation of the breast. Both treatment options produce simila r survival outcomes, but each has implications for QOL that will determine the womans ul timate decision. For some women avoiding the radiation therapy required to conserve the breas t is important. For others, conservation of the breast is of most importance. Even when surv ival outcomes are different, some patients may select a less effective treatment because of the effect the treatment may have on their QOL [68]. It is clear that the availability of QOL data is essential for making a balanced and informed decision about treatment options. PRO instrument can also play a role in the di agnose and treatment effect of health issues related to the ICF domain of body function. So me symptoms and treatment effects are not measurable and only known to the patient. Concepts such as pain intensity or pain relieve are not observable and have no direct physical manifestation. Again, PRO instruments are needed to assess these areas. This is important because, some times, improvements in a particular clinical measurement may not correlate with how the patien t functions or feels. For instance, a patient can demonstrate an improvement in a test of mu scle strength, but this may not correlate with improvements in walking or impact the pati ents ability to perform daily activities.

PAGE 34

34 The pain literature provid es valuable information on the importance of patients perceptions and perspectives. H odgkins and Daltroy [69] investigat ed the assessment of pain by physicians and patients. They found that physicians rating of pain is generally lower than that of the patients. In addition, male physicians tended to rate female patients pa in lower than that of male patients. The patients per ception of their pain can also be variable. When distracted, a patient may provide a lower rating of their pain than when they focus on the pain sensation. Inconsistency in how the patients evaluate a part icular condition does not ch ange the value of the information obtained when using PRO instruments. The patients experience is what the patient says it is at any given time [69]. This inform ation is valuable for di agnosing and treating the condition. It can lead to a better understanding of the nature of the experience and how patients personal factors can affect their perception. PRO instruments are often developed to measure what patients want and expect from their treatment and what is most important to them [54]. The patients perspective is critical to evaluate treatment effect and pa tient satisfaction with treatment. Ideally, a treatment intervention or health strategy should be aimed at addressi ng all aspects of the c onstruct of health. PRO instruments allow clinicians and researchers to assess an area of the model that was previously not well understood because of its abstract nature. Adding thes e instruments to physiological clinical measures will help to obtain a more complete picture of the patients health and how it is affected by their personal and societal circumstances. As previously mentioned, seeking inform ation from the patients about their health condition and how it affects their function and participation is not new. However, PRO instruments provide a formal assessment that may be more reliable than the traditionally used informal patient interview. PRO instrument s use a predetermined format to minimize

PAGE 35

35 measurement error and ensure c onsistency [54]. Instruments can be self-reported or clinician administered. In the first case, PRO instrument s avoid possible clinicians bias and offer an unfiltered response that reflects more closely how the patients rate their health. Well-developed and adequately validated PRO instruments have been shown to provide information that matches the results obtained by experts in the particular field of interest. In fact, often, this is the method used to study the validity of PRO instruments [54]. Issues and Concerns about the Use of PRO The use of PRO instruments to evaluate health and disability is widely accepted in the m edical and research community [70]. They add value to traditional clinical assessments and offer a unique perspective of the patient experien ce. There are few disadvantages for the use of PRO instruments; however several methodological, th eoretical, and practical considerations must be critically reviewed to ensure the information obtaine d is accurate and useful. One major concern with the use of PRO is that of definition. PRO instruments are commonly used to assess concepts of activity, participation, and social and personal interactions. These concepts are abstract in nature; therefor e generalization of findings obtained with the use of PRO instruments must be done with caution. For example, the concept of quality of life (QOL) is receiving increased attention in the clin ical and research arena. QOL is an abstract concept for which it is impossible to create a single instrument that assesses it. Researchers have attempted to solve this issue by using a narrowe r definition of QOL. The idea of health related quality of life comes from these efforts of having a more precise definition of the concept being measured. Another approach to this issue is to create specific instruments that relate to a particular health condition. Th is serves a double purpose because it also helps to reduce the ceiling and floor effect e xperienced when instrume nts are too general.

PAGE 36

36 In the past few decades researchers have adopt ed this strategy and developed a number of instruments aiming at particular health conditions. The list of instruments is so extensive that some organizations, realizing the difficulty of having access to these instruments, have created databases where clinicians and researchers can s earch using criteria such as disease, patient population, or type of instruments. For example, the ProQolid database, developed by Mapi Research Institute, contains over 500 instruments and aims at identifying and describing PRO and QOL instruments to help researchers and clinicians choose appropr iate instruments and facilitate access to them. Another concern relates to the administrati on of PRO instruments. The use of self or clinician administered questionnaires is difficu lt in certain patients with communication and cognitive impairments. For example, patients with stroke often have speech or cognitive deficits that make the use of PRO instruments difficult. Similar difficulties are encountered when using these instruments with those who have a low edu cation level, do not speak the language, or come from a different cultural background. Of particular interest is the issue of cultural relevance, especially when people from different cultural backgrounds are compared using the same instrument. PRO instruments require an internal evaluation of several as pects of ones life and how these aspects are influenced by the health co ndition of interest. These values are influenced by the patients culture and previo us experiences. For example, in the pain literature there is enough evidence to support differences in the pain experience based on ethnic, social, gender, and geographical factors [71]. Another issue to consider is the psychome tric characteristics of some PRO measures. There are concerns in the research literature about some PRO measures being inadequately conceptualized, lacking psychom etric rigor, and having inconsis tently applied psychometric

PAGE 37

37 methods [72]. To solve this issue, some national and international organizations have released guidelines for the creation and evaluation of PRO in struments. Recently, the Scientific Advisory Committee (SAC) of the Medical Outcomes Trust cr eated a document to guide in the evaluation of PRO instruments. In this gui de the SAC states that PROs s hould be evaluated on the following seven dimensions; 1) the use of pre-specified conceptual and measurement models; 2) the strength of empirical support for the reliability and validity of the scale(s); 3) the responsiveness of PRO to clinical change; 4) the method(s) for interpreting scores; 5) th e level of respondent and administrative burden; 6) the equivalence of altern ative forms of administration; and 7) the rigor with which translations are adapted for use in spec ific cultural contexts [73]. This comprehensive list is useful for researchers interested in eval uating PRO instruments, but may have little use for clinicians. There is a need to create a way of translating these guidelines efforts into clinical practice. Patient reported outcomes (PROs) are a necessary and valid way of including the patients perspective into research and clinical practice. Using the International Classification of Functioning, Disability, and Health (ICF) model clinicians and re searchers can evaluate existing PRO instruments, and propose new instruments to assess particular areas of the model. PRO instruments must be carefully selected to meet th e needs of the specific po pulation of interest. No single instrument has universal appl ication in health assessment. There are some concerns about the psychom etric properties of PRO instruments. These instruments are often used to explore abstract concepts that are difficult to interpret and conceptualize. Assigning a single score value to broad concepts such as quality of life may be an oversimplification of a very complex component of life. However, researchers have found strategies to minimize this problem. Using inst ruments that are diseas e and population specific,

PAGE 38

38 based on sound theoretical concepts reliable, responsive to clin ically important change, and culturally sensitive may ameliorate some of the concerns expressed by members of the research community. PRO measurements, such as participation, satisfaction with care, and quality of life are increasingly being required by accreditation agencies, patients, policy-makers, and medical insurers as quality indicators of best practice [73]. These groups recognize the need to empower patients by giving them the opportunity to include th eir perspective in the selection of treatments and evaluation of treatment outcomes. To my know ledge, the second study in this dissertation is an attempt at using an ICF-base d PRO instrument to assess expectations and patient satisfaction in the rehabilitation of falls. Summary The num ber of people 65 and older is likely to continue rising in the future. Falls are already a major health concern in this population and will likely cont inue to be a burden to elders and the health system. To offer the best possible, most efficient medical care to the increasing number of elders, clinicians should use the be st assessment instruments available and include evidence base information in their clinical practi ce. In addition, to provide patients the treatment they deserve, clinicians need to understand th e patients expectations and goals. Patient satisfaction is the ultimate goal in any serv ice provider, including medical services. In research, most rehabil itation interventions are evalua ted by using some test of statistical significance. Assessm ent instruments used during th ese interventions provide a measure of performance that can be used to assess disease severity or monitor improvement. Tests of statistical significance offer important in formation about groups of patients, but fail to capture differences at the individu al level. In recent times, a numb er of investigators are looking at other statistical methods that can provide information about cl inically relevant change [29, 41,

PAGE 39

39 47, 50]. This dissertation is a firs t attempt at investig ating clinically meaningful changes in measures of gait and balance used to evaluate geriatric patients who have fa llen or are at risk of falling. In addition, a new set of instruments are also receiving a great deal of attention from the research community. Recognizing the need to em power patients and understand their personal needs, patient reported outcomes are been included in clinical trials to assess the relevance of the intervention, from the patients perspective. In fact, during the inaugural ceremony of the new NIH PRO initiative Director Elias A. Zerhouni said There is a pressing need to better quantify clinically important symptoms and outcomes that are now difficult to measure. Clinical measures of outcome such as x rays and lab tests have mi nimal relevance to the da y-to-day functioning of patients with such chronic diseases as arthritis, mu ltiple sclerosis, and asthma, as well as chronic pain conditions ( NIH, 2005). This dissertation is a first attem pt at using a PRO instrument to assess expectations and success criteria, from th e patients perspective, of a rehabilitation program for patients with gait and balance problem s who have fallen or are at risk of falling.

PAGE 40

40 Figure 1-1. International Cla ssification of Functioning Disab ility and Health (ICF) model Interactions with people, Community & Social life. Gait & Balance, Selfcare & Mobility. Sensory function, and pain, Energy & Drive, Mental function, Emotional Distress,

PAGE 41

41 CHAPTER 2 ESTABLISHING THE MINIMUM DETECTAB LE CHANGE (MDC) FOR THE BERG BALANCE SCALE AND DYNAMI C GAIT INDEX Introduction Most rehabilitation efforts to treat falls in clude a com ponent of physical rehabilitation. These treatment plants are based on assessments performed with instruments specifically developed to obtain information related to th e condition of interest. Therapists assess and reassess patients to identify specific problem areas and establish improvement criteria. Two assessment tools commonly used in physical ther apy, and specifically in the evaluation of elder individuals who have fallen or ar e at risk of falling, are the Be rg Balance Scale (BBS) and the Dynamic Gait index (DGI) [74]. Gait and Balance Assessment The Berg Balance Scale (BBS) The BBS is a frequently used perform ance-based scale that assesses postural balance [75]. The test consists of 14 commonly used tasks: Sitting to standing, standing unsupported, sitting unsupported, standing to sitting, tran sfers, standing with eyes closed, standing with feet together, reaching forward with outstretched arm, retrieving an object from floor, turning to look behind, turning 360, placing alternate foot on stool, standing with 1 foot in front, and standing on 1 foot. The scoring method is based on a 5-point ordina l scale of 0 (indicates the lowest level of function) to 4 (indicates the highe st level of function), with the total score ranging from 0 to 56. The BBS was specifically designed to be used at the clinic. It requires minimal equipment (stopwatch, chair, stool and ruler) an d can be applied in under 15 minutes. A considerable amount of evidence suggests th at the BBS is a valid measure of standing balance. Initially, Berg et al. [75] correlated BBS scores with a general rating of balance made by therapist (Pearson r=.81). Othe r studies by the same author have also demonstrated high

PAGE 42

42 correlation values between the BBS and other meas ures of balance. For instance, the Pearson r correlations between the BBS and the balance subscale of the Tinetti Performance-Oriented Mobility Assessment and the Barthel Index mobility subscale were .91 and .67 respectively [76]. Other researchers have also found high correlati ons between BBS scores and other motor and functional measurements: Fugl-Meyer Test mo tor and balance subscales (Pearson r=.62.94), Timed Up & Go Test (TUG) scores (Pearson r=.76), Emory Functional Ambulation Profile (Pearson r=.60), and Gait Speed (Pearson r= .81) [7778]. The BBS scores also correlated moderately with data obtained for the Dyna mic Gait Index (Spearman coefficient=.67), and center-of-pressure measures (.40 to .67 [K endall coefficient of variance]) [78]. Several studies have also reporte d high intraand inter-rater reliability for the BBS. Berg et al. [79] used videotaped evaluations of the BBS to obtain in ter-rater reliability (ICC= .98 for total BBS scores). The same researchers replicated these results in a test -retest format, producing a within rater ICC= .97 and betw een rater ICC=.98. The large majo rity of studies investigating reliability of the BBS have used some form of correlation coefficient such as the Pearson's Product-Moment Correlation Coefficient (r) and th e intra-class correlatio n coefficient [ICC], with the latest becoming more popular in recent times. A fundamental problem of ratio indexes such as the ICC is that the error of measuremen t and true variability are expressed in relative terms. An ICC score is a ratio of within subject and between subject variability. Thus, the range of genuine differences in any attribute is samp le dependent. Therefore, previously reported high ICC values for the BBS must be considered with caution. The Dynamic Gait Index (DGI) The DGI wa s developed by Shumway-Cook and W oollacott [80] to assess balance in the older adult at risk for falling. This functional ga it scale consists of 8 common gait tasks: walking at different speeds on a level surface, walking with horizontal and vertical head turns, ambulating

PAGE 43

43 over and around obstacles, ascending and descendi ng stairs, and making quick turns. Each item is scored on a 4-level ordinal scale, where 3= normal, 2= minimal impairment, 1= moderate impairment and 0 = severe impairment. The maxi mum possible score is 24 pints. The DGI can be administered in 10 minutes and requires minimal equipment. The psychometric properties of the DGI have not been extensively investigated. Validity of the scale has been supported by moderate correlation with the BBS (Spearman rank order correlation, r = 0.71), [81]. Sensitivity and specificity to identify individua ls with a history of falls has been established at 59% and 64% respectiv ely [82]. The test devel opers investigated the inter-rater and test-retest reliability of the scal e using a small sample of 5 older adults and 5 raters. They found ICC values of .96 (inter-rater) and .98 when s ubjects were re-tested a week later by 2 therapists. Intra-rater reliability has no t been reported in the literature. Despite being widely used in the clinic, the psychometric pr operties of the DGI have not been investigated sufficiently. The BBS and DGI are used to assess different dimensions of balance (i.e. static and dynamic balance). Since the assessment of balance c ontrol is a crucial step to identify individuals who are at risk of falling, thes e two instruments are often the ce ntral components of the physical therapy evaluation. Scores from these instruments are regularly used to determine a particular treatment and to monitor improvement. Moreover, low scores in these instruments have been associated with an increased risk of falling in the elder population (i.e., a BBS score of <45 and DGI of <19) [76, 82-84]. Therapists use these cut-off points as a re ference to guide treatment and monitor progress. Cut-off scores are taken as ab solute values to determine the success of a particular intervention and are of ten used to report patient progr ess. For example, a patient who improves from an initial BBS score of 40 to a final score of 46 can be reported as being outside

PAGE 44

44 of the range associated with a high risk of falli ng. This anchor-based method must be used with caution, because assessment instruments are not always responsive enough to detect small changes in performance, and these changes can be masked by the multiple sources of error associated with the instrument. Unfortunately, when results differ from one a ssessment to the next, it cannot be assumed that true change has occurred; some or all of the change coul d be attributed to measurement error. Error can be inherent to the test used or represent the naturally existing fluctuation in patients performance. The amount of error across measurements of the same test is related to the reliability of the test. Reliability is the ability of a particular test to consistently provide the same value when no change has occurred [85]. It is al so a measurement of the objectivity of the test. There are several statistical methods that have been used to measure reliability. They can be divided in two major groups: measures of relative and absolute reliabil ity. Relative reliability refers to the degree of associa tion between repeated measurements. In other words, relative reliability measures the strength of the correlation between repeated measures. It takes into account the total group variabil ity (between subject/measurement) and the individual measurements variability (within subject/measurem ent) to obtain a correlation coefficient, for example, the Pearson correlation coefficient or the Intra-class correlation coefficient (ICC) [86]. Absolute reliability refers to the variability of the scores from measurement to measurement (within subject/measurement). This approach does not take into account th e range of individual scores and is not sample-dependent [87]. Some of the tools used to calculate absolute reliability include: the coefficient of variation (CV) and th e standard error of the measurement (SEM), [8891].

PAGE 45

45 While correlation methods used to calculate relative reliability are excellent sources of information to compare groups of patients, the SEM is more appropriate for clinical practice, that is, when making decisions about individual patients [90]. However, to date, only one published study has used a measurement of absolute reliability to look at the psychome tric properties of the BBS and no published study has addr essed this issue w ith the DGI. Stevenson [91] used the SEM to investigate error associated with the use of the BBS in stroke patients. He found a SEM (in BBS units) of 2.49 in patients with stroke r eceiving inpatient rehabilitation. In addition, he calculated a confidence interval around the SEM and found that a change of 6 BBS points was needed to be 90% confident of genuine cha nge. This finding is somewhat surprising and questions previously reported high BBS reliabil ity scores. Further investigation in different populations is needed. The Intraclass Correlation Coefficient (ICC), the Standard Error of Measureme nt (SEM) and the Minimal Detectable Change (MDC) The ICC, or intraclass correlation coefficient, is the most commonl y reported reliability measure in the literature. The ICC provides information about the measures ability to differentiate among subjects. The ICC, as with other correlation coefficients, provides values between 1.0 and -1.0, with high absolute values in dicating less variability between scores. The ICC incorporates total variability (between subject/measurement), and erro r associated with it, and the individual variability (w ithin subject/measurement) to obt ain a ratio. This technique is most appropriate for investigating differences between gr oups of patients. A less frequently used reliability index is the standard error of measure (SEM). While the ICC expresses the proportion of variance of an observation due to betweensubject variability in the true scores, the SEM is a measure of w ithin subject variability. The SEM expresses measurement error in the same unit of the origin al tool and is not in fluenced by variability

PAGE 46

46 among subjects. The SEM is closely related to the concept of minimal de tectable change (MDC) expressed by Stratford et al [92]. The MDC is th e amount of change in a given measure that must be obtained for a clinician to determine that true change has occurred. The MDC is expressed as a confidence interval around the SEM, indicating the values that are within the range of error attributable to the measuring instrument. The MDC provides the clinician useful and easy to understand criterion for change in patients performance. Therefore, investigations are needed to determine the am ount of change in the Berg Balance Scale and Dynamic Gait Index necessary for a therapist to conclude that true change has occurred (MDC). In additi on, understanding how MDC values change at different score levels of the BBS and DGI (cut-off points, <45 a nd <19 respectively) may prove to be valuable for clinicians that are faced with making treatment decisions based on these values. Thus, the purpose of this study was to use the standard error of measurement (SEM) to investigate the Minimal Detectable Change associ ated with the Berg Balance Scale and Dynamic Gait Index. In this study, I attempted to improve on previous reliability investigations by providing clinicians with estimates of measurement error that ar e easy to interpret and can be used to make clinical decisions. Methods Subjects The sam ple consisted of 42 subjects ( 26 Males and 16 Females, age 55 and older) participating in a larger, funded, research study looking at the link between smoking and recovery from frailty in older Floridians. This study was supported by a grant from the Florida Department of Health and received approval of the In stitutional Review Board for the University of Florida and the Research and Development Committee at the North Florida/South Georgia VA Medical Center. Inclusion criteria included: community dwellers with a history of falling

PAGE 47

47 twice in the past 12 months, the ab ility to walk 20 feet (with or without an assistive device), and a score of 24 or higher on the Mini-Mental State Exam [97]. Participants were recruited in two different ways. First, patients from a Gait and Balance disorders clinic at the North Florida/South Geor gia VA Medical Center were approached by their therapists and asked if they were interested in partic ipating in a research study. If the patients agreed, a research coordinator ex plained the study and, if still in terested, consented the patients. Secondly, letters were sent to doctors offices explaining the study and fl yers were distributed throughout the community. Participants responded to the advertisement and, if they met the studys inclusion criteria, were enrolled in the study. Testing Procedure Enrolled p articipants were tested with an extensive battery of t ools including: Berg Balance Scale (BBS), Dynamic Gait Index (DGI), isometric lower extrem ity strength, physical function domain of the MOS36, Falls Efficacy Scale, Geriatric Depression Score, pain experience VAS, timed tests of gait, Frenchay IADL scale, Pleasant Event Schedule, and spontaneous self-selected gross motor activity as measured by accelerometry. After the initial assessment, a home exercise program was prescrib ed. Participants were instructed to keep exercise logs to record adherence to the progr am. Participants were monitored for 3 months. During this time, a total of 3 evaluations were pe rformed at 4, 8 and 12 weeks. In each of these evaluations, participants receiv ed the previously mentioned ba ttery of assessment tools. In addition, exercise l ogs were collected. Evaluations were conducted by tw o experienced physical therapis ts specialized in gait and balance disorders in the elder population (>7 years experience in geriatric Physical Therapy). From each evaluation, the BBS and DGI tests were videotaped by a research assistant with a Sony DCR-VX2100 digital camcorder. Videotapes were converted to DVD format for later

PAGE 48

48 view. Recorded sessions were re-scored at a late r time (time between initial and re-scores > two weeks) by the same therapists. Therapists used a TV screen or computer monitor to view the recorded evaluations. During the re-scoring of the videotaped evaluations, therapists were allowed to pause, play in slow motion, and/or replay any portion of the evaluation they were unsure about. Therapists were b linded to previous score and whet her the recordings were from an initial, 4 weeks, 8 weeks, or 12 weeks eval uation. For the purpose of this study, only initial evaluations were used. All particip ants were assessed and re-assessed by the same therapist. Data was recorded by the physical therapists and research assistant. All data was later entered into a central database. Analysis All statistical analysis and graphical representations were perf ormed with SPSS 13.0 software for Windows (SPSS Inc., Chicago, IL, USA) and Microsoft Office Excel software for Windows (Microsoft Corporat ion, Redmond, Washington, USA). Box plots were used to investigate the presence of outliers in the data (figures 2-1, 2-2). The distribution of the absolute differences be tween tests (initial BBS and DGI and re-scored BBS and DGI) was plotted. Cases with values between 1.5 and 3 box lengths (interquartile range) from the upper or lower edge of the box we re consider mild outliers. Cases with values more than 3 box lengths from the upper or lowe r edge of the box were considered extreme outliers. For the BBS data, 3 outliers were identif ied. The DGI data did not present any outliers. Since no extreme outliers were observed in either data-set, all scores were considered valid for subsequent analysis. The procedure suggested by Stratf ord [86, 92] was used to calc ulate the standard error of measurement (SEM), also referred as the abso lute reliability. The equation for the SEM is:

PAGE 49

49 SEM= SD ICC 1 where SD= sample standard deviat ion, and ICC= intraclass correlation coefficient. However, Stratford stated that SEM can also be calculated fro m the square root of the mean square error term in a repeated m easure ANOVA. In addition, the SEM was used to calculate the Minimal Detectable Change (MDC ). The MDC is the product of the SEM, the tabled z-score for a desired confidence interval and the 2. The 2 term acknowledges two measurements are being compared. For a 95% confidence interval the MDC= SEM 1.96* 2 (1.96= Z-value associated with a two-sided 95% confidence interval). Confidence intervals were also calculated with a 90% and 80% (MDC (90%) = SEM 1.645* 2, and MDC (80%)= SEM 1.28* 2). The use of a parametric test (ANOVA) requires that the data meet the normality assumption. Normality was visually explored wi th Normal Q-Q plots and tested with the Kolmogorov-Smirnov normality test. In addition, the use of the SEM, because it assumes a normal distribution of error, requ ires that the measurement error is not related to the magnitude of the measured variable. This is referred as heteroscedasticity. Hetero scedastic data shows a relationship between the amount of measurement error and the magnitude of the measurement. Heteroscedasticity was formally examined by plo tting the absolute differences between initial value and re-scored value, against the mean scor e. Additionally, Spearman's rho correlation was used to rule out a relationship between each indi viduals absolute score difference and his or her mean. The SEM and MDC procedures de scribed above were also used to investigate the amount of error associated with individua ls at different levels of the BBS and DGI rating scale. Because the true score in these two assessments is unknown, the mean va lue between the initial and rescored values of the BBS and DGI was used to dichotomize the participants in two groups.

PAGE 50

50 Commonly used cut-off points (<45 for the BBS a nd <19 for the DGI) were used to form the groups. The SEM and MDC were calculate d for the four resulting groups. Finally, a correlation analysis was performed to investigate a possible relationship between individuals difference (BBS initial BBS re-s cored and DGI initial DGI re-scored) and absolute difference in the BBS and the DGI. To acknowledge the ordinal nature of the BBS and DGI, a non-parametric Spearman's rho correlation analysis was selected. Results A total of 42 participants were assessed w ith the BBS and the DGI. The average age was 75.6 years (range 59 to 88 years). The ratio of males to females was 26 males (62 %) and 16 females (38%). The participants mean ini tial BBS score was 40.7 points (SD=7.3, range 18-53). The re-scored mean value was 41.8 points (SD=7 .5, range 24-55). For the DGI, the mean initial value was 13.4 (SD=4.2, range 3-21), and the re-scored mean was 13.1 (SD=4.3, range 4-22). A distribution of the absolute difference between in itial and re-scored valu es is found in table 2.1 for the BBS and table 2.2 for the DGI. The m ean absolute difference was 2.57 (SD=2.4, range 011) for the BBS, and 1.29 (SD=.99, range 0-3). The distribution of absolute values of the difference betw een initial BBS and re-scored BBS was investigated with a box plot (Figure 2.1). For the BBS, three participants scores were identified as outliers. Their ab solute values were 8, 11, and 6 re spectively. These scores were considered mild outliers because their valu es laid between 1.5 times and 3.0 times the interquartile range below the first quartile or above the third quartile. Therefore these 3 scores were included in all subsequent analysis. The di stribution of absolute values of the difference between initial DGI and re-scored scores for the DGI was also explored with a Box plot (Figure 2.2). No outliers were identified in this case.

PAGE 51

51 The absolute differences in the BBS ranged from 0 to 11 points (Table 2.1). Fifty seven percent of the participants had a BBS absolute difference of 2 BBS points or less. The mean absolute difference was 2.45 BBS points. A graphical representation of the distribution of score differences between initial and re-scored BBS is presented in figure 2.3. For the DGI, the absolute differences in scores ranged from 0 to 3. Seventy four percent of participants had a difference in score of 1 or less DGI points. Th e mean absolute difference was 1.13 DGI points. Figure 2.4 shows a graphical representation of the differences in DGI score between the 2 testing scenarios. The distribution of scores from the BBS and DGI were visually inspected for normality using normal Q-Q plots. Both di stributions showed no significant departure of the data from normality. Formal analysis of the data with a Kolmogorov-Smirnov test showed no significant departures from normality (D42= .143, p>.05, D42= .09, p>.05, for the BBS and DGI respectively). Visual inspection of the distri bution of absolute values of the difference between the two test conditions plotted against th eir mean, show that the data wa s reasonably homoscedastic for both the BBS (figure 2-3) and DGI (figure 2-4) Spearmans rho correlations (BBS, rs = -.09 p>.05, and DGI, rs = .02 p>.05) confirmed the lack of relationship between the mean score and the difference in scores (initial minus re-scored values). Repeated measures ANOVA were performed to calculate the mean square error term, as described by Stratford [86, 92]. The BBS provide d a mean square error of 5.2 and the DGI 1.1. For a 95% confidence interval, the mi nimal detectable change (MDC) is 5.2 2.77= 6.4 for the BBS and 1.1* 2.77= 2.95 for the DGI. Therefore, a ch ange greater than 6.4 BBS points and 2.9 DGI points is necessary to reveal a change that exceeds the measurement error associated with

PAGE 52

52 this instruments and show genuine change. Additional confidence intervals of 90% and 80% were performed for the BBS and DGI (table 2.3 ). This table also includes BBS MDC results when dividing the participants in functional groups (above and below 45 BBS points). This grouping resulted in 30 cases classified as low function (>45) and 12 cases classified as high function (<45). The MDC95% was 7.3 BBS points for the low function group, and 6.3 BBS points for the high function group. Additional correlation analysis was performe d to investigate a possible relationship between absolute change in the BBS and DGI. A Spearmans rho correlation value of rs = .15 p>.05 suggest that there was no relationship betw een participants score differences in both instruments. Therefore, differen ces in scores in the BBS (initi al BBS minus re-scored BBS) were not correlated with differences in the DGI (initial DGI minus re-scored DGI). Discussion The Berg Balance Scale and the Dynam ic Gait index are two instruments widely used in clinical practice to measure individuals gait and balance ability, and monitor improvement in these areas. High reliability values have been reported for both instruments [79, 82]. However, previous investigations have used a form of correlation, such as the Pearson's product-moment correlation or the Intraclass Correlation Coefficien t (ICC), to investigate the reliability of these instruments. While correlational investigations are suitable for inves tigating the degree of agreement between groups of subjects in repeated measures, they offer little information about the amount of change an individual needs to achieve genuine change. That is, the amount of change beyond the error associ ated with the instrument. Absolute reliability is a more appropriate way of investigating the reliability of an instrument intended for use in a clinical sett ing, where clinicians are more concerned about individual change. In this investigation, the results of the absolute reliability of the Berg Balance

PAGE 53

53 Scale and the Dynamic Gait Index indicate that 6.4 and 2.9 points re spectively are required to be 95% confident genuine change has occur betwee n 2 testing occasions. This information is valuable for clinicians that can apply these numbers, at the individual level, to assess improvement in function over time. Although the 95% confident level is widely ac cepted in the research community, one could argue that, in clinical practice, a lower confid ence level could be of practical use to make appropriate clinical decisions. In this investigation, confidence levels of 90% and 80% were calculated. The BBS showed MDC90% of 5.4 and MDC80% of 4.2 points. The DGI presented MDC90% of 2.5 and MDC80% of 1.9 points. Even at the lowest confidence level both instruments demonstrated an estimated amount of error that should be considered when making clinical judgments. It is worth noting that, in this inves tigation, the DGI demonstrat ed half the amount of estimated error as compared to the BBS. Howeve r, this comparison does not take into account the range of values of both instruments. That is, the MDC of 6.4 points in the BBS is equal to 11.5% of the total possible score of the BBS (56 points) while the MDC of 2.9 DGI points is equal to 12% of the total possible score for th e DGI (24 points). Therefore, based on these results, both instruments present similar am ounts of variability be tween the two testing occasions. This investigation is not without limita tions. The main disadvantage of using a distribution-based approach to assess change is that the results o ffer no indication of the importance of this change. That is, minimal det ectable change (MDC) is not equal to minimally important change (MIC). In fact, is possible that MIC change is sm aller than MID. In this case, the instrument would not be able to detect th e desired change, since the MIC would be smaller then the measurement error of the instrume nt. Identifying when genuine change (beyond

PAGE 54

54 measurement error) occurs is a necessary but in complete step in the process of judging the importance of an observed clinical change. Distribution based approaches such as the st andard error of measurement (SEM) used in this investigation, assumes that the measurement error is constant across the range of possible sores. In this investigation, individuals were dichotomized into two functional groups to investigate the possible fluctuation of the SEM at two different levels of the scale. For the Berg Balance Scale, individuals with lower perf ormance (BBS=<45) demonstrated higher SEM values. Their MDC95% was 7.3 BBS points. In contrast, pa rticipants with higher performance scores (BBS=>45) showed lower SEM values. Their MDC95% was 6.3 BBS points. However, dichotomizing the initial pool of participants reduced the sample size of the groups. Therefore, these results should be interpreted with caution. Fu rther investigation is n eeded to substantiate the possibility of different MDC levels based on performance. The same approach was not used with scores from the DGI, because dichotomizing this group resulted in a sample size for the high level group of only 10 participants. A methodological issue worth consid ering is the use of videotap ed evaluations to establish the reliability of an instrument, especially when the scores of the initial live evaluation are compared with scores obtained by evaluating videotaped performances. This method has been widely used and published. In fact the initial relia bility study conducted by Berg et al [79] used videotaped assessments to investigate the intra -rater reliability of the BBS. A clear disadvantage of this design is that it does not take into account the natural fluctua tion of the participants performance when tested in two separate occasio ns. Therefore, clinical decisions based on this must be made with caution, since not all sources of error are considere d. However, in a recent publication Stevenson [91] found a MDC95% value of 6.9 BBS points when assessing stroke

PAGE 55

55 patients in a test re-test design. The fact that Stevensons results are comparable to what was found in the present experiment, s uggest that the varia tion seen in both experiments is mostly due to the instruments reliability and not within patient reliability. Intere stingly, Stevensons test re-test experiment used the best performance of 3 trials as thei r value for each item. With this approach, it seems plausible to conclude that th e true score is more easily captured, and the within subject variability decreased. In additi on, Stevenson used the data reported by Berg and colleagues [93] in their reliabil ity study to calculate the MDC95%. He found a MDC95% of 6.2. Again, the investigation by Berg employed a test re-test design with stroke subjects and produced similar results to the present study. To a ddress the possibility of increased variability when using two distinct methods of evaluation (live performance vs. videotaped evaluation) future research should consider using only one of the methods to evaluate MDC in these instruments. The therapists particip ating in this experiment offered feedback about the appropriateness of using videotaped sessions to investigate MD C. In general, they agreed there were some limitations in terms of having the right camera pers pective to accurately a ssess a particular task. For example, in some static items of the BBS where body sway is an important factor, the therapists found the camera was not stable enough to judge the participants sway. However, the therapists also indicated the us e of the video allowed them to pause, slow down, or rewind and play again the tape, if they felt unsure about a pa rticular performance. In addition, an advantage of using videotaped sessions is that it eliminates the possibi lity of a learning effect when assessing participants at two se parate occasions. When testing and re-testing subjects within a short period of time, it is plausible to assume that subjects could pe rform better after being familiar with the test and the testing environment. From this experience, it is clear that a more

PAGE 56

56 standardized way of filming the assessments, possibly using multiple fixed cameras, would result in a more accurate assessment of participants performance. While often time researchers focus on significant group mean changes in the variable of interest to draw conclusions a bout the effectiveness of a partic ular intervention, clinicians face the need to assess individual patients to judge a particular condition or monitor improvement. In this study, the BBS and DGI demonstrated mean valu es between test occasions of less than one point, suggesting that, as a group, both testing oc casions provided almost indistinguishable results. However, at the individual level, thes e two instruments demonstrate an important amount of variability. Therefore, clinic ians must be aware of this issue and consider the minimal detectable change values when making individual decisions based on these instruments. Conclusion The procedure outlined by Stratford [86, 92] take reliability assessment to a more sophisticated, yet much more user friendly level. This st atistical approach a llows clinicians to simply determine when genuine change occurs among two testing occasions. This experiment is a first attempt at investigating the Minimal Detectable Change (MDC) of the Berg Balance Scale (BBS) and the D ynamic Gait Index (DGI) in elder community dwellers participating in a reha bilitation program. The re sults from this investigation demonstrate that a change of 6.4 point in the BBS and 2.9 point s in the DGI is necessary to be 95% confident genuine change in function has occurred between 2 assessments. These guidelines are important assessing individuals performance to monitor progress and guide treatment in clinical practice. Future investigations are needed to expl ore MDC at different functional levels.

PAGE 57

57 Figure 2-1. Distribution of BBS difference scores (Initial BBSre-scored BBS) Figure 2-2. Distribution of DGI difference scores (Initial DGIre-scored DGI)

PAGE 58

58 Table 2-1. Absolute difference in BBS scores between initial and re-scored assessments Difference in BBS points Number of subjects Percent Cumulative percent 0 9 21.4 21.4 1 8 19.0 40.5 2 7 16.7 57.1 3 8 19.0 76.2 4 5 11.9 88.1 5 1 2.4 90.5 6 2 4.8 95.2 8 1 2.4 97.6 11 1 2.4 100.0 Total 42 100.0 Table 2-2. Absolute difference in DGI scores between initial and re-scored assessments Difference in DGI points Number of subjects Percent Cumulative percent 0 13 31.0 31.0 1 18 42.9 73.8 2 5 11.9 85.7 3 6 14.3 100.0 Total 42 100.0 Table 2-3. MDC values for the BBS and DGI MDC values for the BBS and DGI BBS Low function BBS (n=30) High function BBS (n=12) DGI MDC95% 6.4 7.3 6.3 2.9 MDC90% 5.4 5.8 4.2 2.5 MDC80% 4.2 4.5 3.2 1.9

PAGE 59

59 Figure 2-3. BBS results for all participants. Me an value and difference between initial and rescored values. Figure 2-4. DGI results for all participants. Mean value and di fference between initial and rescored values.

PAGE 60

60 Figure 2-2. Mean BBS score for different absolute differences in BBS between testing occasions Figure 2-3. Mean DGI score for different absolute differences in DGI between testing occasions

PAGE 61

61 CHAPTER 3 PATIENTS SUCCESS CRITERIA AND EXPECT ATIONS IN FALLS REHABILITATION Introduction Despite the efforts of researchers and f unding agencies to understand the mobility problems many older adults endure falls and its consequences ar e taking epidemic proportions in the western world. In the US, falls are the lead ing cause of injury deaths among persons over 65. Falls among elderly persons account for approxi mately 16,000 deaths and 1.8 million emergency room visits annually, and fall-related injuries for people 65 and ol der cost a tota l of $27.3 billion in 2003 alone [105]. These alarming statistics have prompted the US senate to recently pass a new legislation to reduce and prevent elder falls through public education campaigns and research [117]. This new effort acknowledges the need for the research community to continue working towards finding solutions to prevent th e incidence of falls, improve the treatments available and ameliorate the consequences of falling. The causes of falls have been heavily studied and multiple factors have been identified. Many of these studies indicate that fall preven tion programs that include a multidisciplinary approach with a component of physical rehabil itation can significantly reduce the incidence of falls [99-101]. However, although these programs may offer signi ficant improvements, reducing the rates of falling may be dependent on adhe rence to the programs Adherence rates for participation in fall prevention programs have been historically low [102-103]. To maximize acceptability and adherence among older people, we need to have a better understanding of the factors that contribute or detract from patients adhering to fall pr evention programs. Therefore, it seems intuitive to turn to the patient to identify factors that are important to them, and determine the type of treatment that will most satisfy th eir needs. In addition, to evaluate treatment,

PAGE 62

62 therapists should be sensitive to the amount of improvement patients need to experience to consider their treatment successful. Most rehabilitation strategi es include end-points based on clinical assessment tools and conclusions drawn from standard statistical met hods of significance. Statistical significance is important, but not sufficient to establish clinically relevant conclusions [1 04]. A better approach is to consider the number of patients that reach a clinically important end-point. To date, few investigations have used the pa tients perspective to arrive at these end-points. In the falls literature, this issue has not been explored. Determining endpoi nts in balance rehabilitation interventions that reflect the patients view and c ontribute to their satisfac tion would appear to be a valuable empirical endeavor. There is an increased interest in the re search community about establishing what constitutes a clinically important change for healthcare interv entions [106-108]. The previous study in this dissertation demonstrated that av ailable assessment instruments are not always accurate at detecting change. Regardless of whet her a change can be measured or observed, the interpretation of this change will depend on whose perspective we consider. From the clinicians perspective, a change that results in a modification of treatment or patients prognosis is certainly considered a significant change. Researchers co nsider a change that achieves statistical significance as a relevant change. In addition, most researchers are concerned with differences in group of patients. However, group differences coul d reflect large change in some patients and modest change in others, or modest change in many patients. Therefore, group changes are difficult to interpret and apply, si nce there is no indicat ion of the likelihood of a positive change in a single patient. A better approach, but seldom reported in the literature, is to turn to the patient to identify relevant cha nge. Patient reported outcomes are crucial to identify relevant

PAGE 63

63 domains that impact the quality of life of the patient. It is the patient who experiences their quality of life, and only they are in a position to ultimately judge whether a change is important [109]. Rehabilitation strategies need to focus on im provements that are perceived by the patient as being beneficial. This will ensure higher compliance with treatments and a more positive rehabilitation experience. Empowering the patient can lead to positive outcomes and influence patients satisfaction with treatment. Most of th e patient satisfaction literature has concentrated on treatment of non-chronic condi tions. In these cases, patients who experience amelioration of their symptoms report high levels of satisfacti on. Inversely, patients with chronic conditions are thought to experience less satisf action with treatment. There is some challenge to this expectation in the chronic pain l iterature. Chronic pain patients have reported moderate to high levels of satisfaction even with small reductions of pain [110-111]. Thes e patients report that, while pain was still present, their therapy ha d helped them reduce some of the collateral consequences of chronic pain, such as m ood disturbances, sleeping problems, etc. Most elder individuals who have fallen or are at risk of falling, can be classified as chronic patients. In fact, Calkins and colleagues [112] repor ted that almost 75 percent of the elderly (age 65 and over) have at least one chronic illness. Furthermore, having one or more chronic condition has been associated with an increased risk of falling by severa l investigators [113-115]. Therefore, it is possible that individuals who fa ll will share some of characteristics of patients with other chronic illnesses, and base their sa tisfaction with treatment due to collateral consequences. Other domains important to this group, such as energy and drive, emotional distress or social interactions, are areas that may be impacted by fall treatment interventions.

PAGE 64

64 Surprisingly, to date, no published study has examin ed patient satisfaction in falls rehabilitation programs. After identifying relevant domains, investigators need to assess how much improvement in each domain represents treatment success. Tr aditionally, this has been accomplished by using clinical assessment tools that mi mic real life situations. However, these tools often fail to cover the domains that are important to patients. T hus, a comprehensive approach that takes into account the multiple areas of concern to patients is warranted. After esta blishing the domains of relevance to this population, these criteria can be used to compare satisfaction across varied studies and treatment options. Therefore, the primary aim of this study is to investigate the patients success criteria across several domains including: mobility, self-care, interacti ons with people, community and social life, energy and drive, mental function, em otional distress, sensor y function, and pain. In addition, a secondary aim is to investigate pa tient expectations for treatment across above mentioned domains. Methods Subjects A total of 50 particip ants (age 55 and older) were enroll ed in this study. Twenty of these participants were also part of a larger, funded, research study looking at the link between smoking and recovery from frailty in older Floridians (DOH04NIR-15). This larger study was supported by a grant from the Florida Department of Health. The remaining 30 participants were only participating in the pres ent study. Both studies were individually approved by the Institutional Review Board for the University of Florida and the Res earch and Development Committee at the North Florida/ South Georgia VA Medical Center Inclusion criteria for both studies included: community dwellers with a hist ory of falling, the ability to walk 20ft (with or

PAGE 65

65 without an assistive device), a nd a score of 24 or higher on the Mini-Mental State Exam [97]. Participants were not screened for any particular condition. Th erefore, the participants pool consisted of individual with an extensive num ber of medical conditions including: diabetes, hypertension, neuropathies, orthope dic problems, general dizziness, history of stroke, general frailty, etc. Participants in the DOH04NIR-15 study were recruited in two diffe rent ways. First, patients from a Gait and Balance disorders c linic, at the North Florida/South Georgia VA Medical Center, who were receiving outpatient physical therapy services related to their history of falling or being at ri sk of falling, were approached by their therapists and asked if they were interested in participa ting in a research study. If the patien ts agreed, a research coordinator explained the study and, if still in terested, consented the patients. Secondly, letters were sent to doctors offices explaining the study and flyers were distributed throughout the community. Participants responded to the advertisement and, if they met the studys in clusion criteria, were enrolled in the study. The remaining participants only enrolled in the present study, were also recruited at the North Florida/ South Georgia VA Medical Center Gait and Balance disorders clinic during their scheduled appointments at the clinic. These participan ts were also receiving outpatient physical therapy services related to th eir history of falling or being at risk of falling. All participants received identical initial ev aluations, consisting of a battery of assessment instruments including: Patients Perspective Outcome Questionnaire (PPOQ), Berg Balance Scale (BBS), Dynamic Gait Index (DGI), isomet ric lower extremity strength, physical function domain of the MOS36, Falls Efficacy Scale, Geri atric Depression Score, pain experience VAS, timed tests of gait, Frenchay IADL scale, Pleas ant Event Schedule, and spontaneous self-selected gross motor activity as measured by acceleromet ry. All physical assessment instruments were

PAGE 66

66 administered by licensed Physical Therapist specialized in the rehabilitation of falls. Questionnaires were administered by a research assistant. A home ex ercise program, based on the initial assessment, was then prescribed by th e therapist. Subsequent re-evaluations at 4 weeks, 8 weeks and 12 weeks, were completed to record progress and compliance with the program. Testing Procedure The same procedure was used for both groups of participants. In brie f, during the initial evaluation, participants were administered th e Patients Perspective Outcomes Questionnaire (PPOQ, Appendix 1). A research assistant read the questionnaire out loud and recorded the answers in paper forms. Participants were allo wed to ask questions and the research assistant offered clarification for any questionnaire items that participants felt unsure about. The Patients Perspective Outcome Questionnaire (PPOQ): Currently, there are no instruments in the literature to assess success criteria and expectations for treatment of falls. The PPOQ is an adaptation of the Patient Centered Outcomes Questionnaire developed by Robinson and colleagues [104]. These resear chers used this ques tionnaire to assess success criteria and expectations for treatment of chronic pain. Dr. Robinson participated in the development of the PPOQ. The PPOQ is identical to the Patient Cent ered Outcomes Questionnaire except for the domains it measures. It consists of 4 questions that address 9 different health domains. These four questions include: 1) current levels of involvement for each domain, 2) changes in each domain that will represent a successful treatment, 3) treatment outcome expectations, and 4) importance of each of the domains. These domains are based on common problem-areas associated with elder individuals who have falle n or are at risk of falling. The PPOQ uses the language of the World Health Organization (WHO ) International Classification of Functioning

PAGE 67

67 Disability and Health (ICF) classification system [116]. Domains used in th is questionnaire refer to specific domains within the ICF classifica tion system and include: mobility; self-care; interactions with people; community and soci al life; energy and drive; mental function; emotional distress, sensory functi on; and pain. Participants rate their perception on a scale of 0 (none/not affected/not important ) to 100 (worst imaginable/most affected/most important). Clarification about each of the domains is in cluded in the PPOQ. To ensure uniformity and standardization of the instrument, the ICF defi nition for each of the domains in the PPOQ is included. For instance, the ICF defines mobility as: this term refers to the ability to change location or transfer from one place to another. It also includes actions such as carrying, moving or manipulating objects and capacity to walk, run or climb. Lastly, mobility also refers to the ability to use various forms of transportation [1 16]. This exact definition of mobility is included in the PPOQ and was used by the research assi stant to explain the di fferent domains. For example, when asking the quest ion: On a scale of 0 (not at all important) to 100 (most important), please indicate how important it is for you to see improvement in you mobility, the research assistant provides the above mentioned definition of mobility. The same procedure was used to explain all domain defi nitions in the questionnaire. Analysis All statistical analysis and graphical representations were perf ormed with SPSS 13.0 software for Windows (SPSS Inc., Chicago, IL, USA) and Microsoft Office Excel software for windows (Microsoft Corporation, Redmond, Washington, USA). First, descriptive statistics were generated for each of the domains. A repeated measures ANOVA was performed to determine whether diff erences existed across domains in the usual levels of involvement. Follow up paired t-tests, corrected for multiple comparisons (Bonferroni correction) were performed to investigate differences between mobility and all other domains.

PAGE 68

68 Then, the success criteria were determined by subtracting the usual level across domains from their expected level. For example, an individual with usual levels of mobility of 60/100 and success criteria of 40/100 requires 20 point change in mobility to consider their treatment successful. Success criteria were tr ansformed to percentage change Therefore, in the previous example, 20 point change represents 33.3% of the initial 60/100. Repeated measures ANOVA were performed on the percentage change su ccess criteria scores to determine whether differences existed across domains in the amount of change necessary for part icipants to consider their treatment successful. Then, paired-t-tests, corrected for multiple comparisons (Bonferroni corrections) were performed to investigate possible differences between domains. Next, usual levels across domain s were subtracted from their expected levels to obtain treatment expectations criteria. Again, these scor es were transformed to percentages to represent the percentage amount of change participants expected after treatment. A repeated measures ANOVA was performed to determine whether di fferences existed across domains in the percentage amount of change participants e xpected after treatment. Comparisons between mobility and all other domains were performed with paired t-tests corrected for multiple comparisons (Bonferroni correction). Next, participants were dichotomized to fo rm two groups (compliant vs. non-compliant). Compliance was defined as a participant w ho completed the standard 12 weeks program. Participants were re-evaluated 3 times afte r the initial evaluation, at 4, 8, and 12 weeks. Participants who attended the last re-evalua tion (12 weeks) were considered compliant. Participants who missed some of the intermediate evaluations (4 or 8 w eeks), but attended the last re-evaluation (12 weeks) were still consider ed compliant. A multivariate analysis of variance

PAGE 69

69 (MANOVA) was used to explore possible differences between compliant and non-compliant groups in treatment expectations across domains. Results Dem ographic characteristics of the sample ar e presented in table 31. Table 3-2 contains descriptive statistics from the PPOQ initial levels Participants reported low to moderate initial levels of restriction in mobility, self-care, inte ractions with people, co mmunity and social life, energy and drive, mental function, emotional distre ss, sensory function, and pain associated with their conditions (higher scores, worse the condition) Energy and drive; mobility; and pain received the highest scores (53, 47, and 44 respectively), while interactions with people and selfcare received the lowest (21 and 24 respectively). Differences of initial levels across domains were explored with repeated measures ANOVA. Ma uchlys test indicated that the assumption of sphericity had been violated 2((35)= 52.02, p<.05); therefore degrees of freedom were corrected using Greenhouse-Geisse r estimates of sphericity ( = .79). Results indicated the existence of significant differences of initial levels among domains F = (6.35, 310.98)= 8.6, p<.05. Paired t-tests, adjusted for multiple comp arisons (Bonferroni correction), were used for posttests. The main analysis comparing mobility to other domains resulted in 8 comparisons. Therefore, for 8 domain comparisons a .006 level of significance was selected. Participants reported higher levels of impairment in th e mobility domain, compared to self-care and interactions with people (P<.006) The community and social life domain showed similar trend but the analysis failed to reach statistical significance t (49)= 2.79, P = 008. To determine the amount of change necessary for participants to deem their treatment successful, their initial levels across domains were subtracted from their success criteria (table 33). Participants considered a mean reduction of 52% in mobility as a successful outcome. Lower

PAGE 70

70 results were found for the domains of sensory function (34%), self-care (35 %), interactions with people (22%), and, community and social life. Energy and drive required the largest reduction (59%). A repeated measures ANOVA was used to investigate whether diffe rences existed in the amount of change necessary for participants to consider their treatm ent successful. Mauchlys test indicated that the assumption of sphericity had not been violated 2((35)= 46.9, p>.05); therefore degrees of freedom were not corrected. The amount of change required for participants to consider their treatment successful was significantly different among domains F = (8, 368)= 7.04, p<.05. Paired t-tests were used for posttests. A Bonferroni co rrection was applied and so all effects are reported at .006 level of significance. It a ppeared that the reduction in mobility necessary for successful treatment was significan tly greater than the reductions necessary for successful treatment of self-car e, interactions with people and community and social life (P<.006). Therefore, in this group of participan ts, treatment success criteria was significantly different across domains, suggesting participants require different amounts of change across domains to consider their treatment successful. Next, initial levels across domains were subtra cted from their expected levels to obtain treatment expectations criteria. Subjects expected similar levels of mobility and energy and drive (both 42%). Interactions with pe ople received the lowest score (18%) (Table 3-4). A Repeated measures ANOVA was used to investigate whethe r differences existed in the amount of change participants expected after treatment. Spheric ity assumption was tested with Mauchlys test. Results indicated that the assumption had been violated 2((35)= 76.02, p<.05); therefore degrees of freedom were corrected using Gr eenhouse-Geisser estimates of sphericity ( = .74). Results indicated the existence of significant differences among domains in the percentage amount of change participants expected after treatment F = (5.91, 277.78)= 3.94, p<.05. Paired t-

PAGE 71

71 tests, adjusted for multiple comparisons (Bonferroni correction), were used for posttests. For this group, the percentage change participants expected for the mob ility domain was significantly greater than the percentage cha nge expected for the domains of self-care, interactions with people and pain (P<.006). Lastly, the results from multivariate analysis of variance (MANOVA) indicated there were no significan t differences between compliant and non-compliant groups in treatment expectations across domains. The Wilks Lambda multivariate test of overall differences among groups wa s not significant (p= 0.934). An exploratory descriptive investigation of groups of partic ipants, based on compliance, revealed that VA participants were 40% compliant, while community partic ipants presented 80% compliance. Discussion Consisten t with the theoretical model that guide d this research, participants with mobility problems leading to falls demonstrated significan t levels of interference across several health domains, representing activity and participati on, and body function. In th is group, participants reported considerable initial leve ls of interference in domains such as energy and drive (54/100), and pain (45/100). Not surprisingly, the mobility domain also received high scores (48/100). Lower scores were seen in domains such as interaction with people (21/100), community and social life (33/100), and self-car e (24/100). These findings suggest that rehabilita tion strategies should take into consideration the complex and multidimensional nature of falls and provide interventions that target the different domains that affect this population. A number of publications have investigated this issue. For example, Gillespie and colleagues [117] conducted a systematic literature review of randomized controlled trial programs designed to reduce the number of falls in community-dwelling, institu tionalized, or hospitalized elderly people. The authors concluded that, interventions targeting on ly the physical aspects re lated to falls produced

PAGE 72

72 lower outcomes when compared to multidimensiona l interventions that took into account the intrinsic and environmental risk factors of patients. The present study is a first attempt at using pa tient reported outcomes to investigate several health domains related to individuals receiving rehabilitation services related to falling. More specifically, this investigation focused on thr ee fundamental aspects of rehabilitation: how participants perceived their le vels of impairment across seve ral health domains, how much change was necessary across domains to consider their treatment su ccessful, and what were their treatment expectations. As mentioned before, significant levels of impairment were reported across domains. Interestingly, several differences were found between domains. The energy and drive domain received the highest score, sugge sting that, for this group, issues such as feelings of fatigue, motivation and energy level are commonly present. There is support in the literature for this finding. Fatigue has been associated with a number of conditions in the elderly population, including diabetes, heart failure, Parkinsons disease, cancer, sleep disorders, and hormonal changes [118-121]. In addition, participants reported considerable levels of pain. There is supporting evidence that suggest pain is a risk factor for falls a nd also that pain can lead to activity avoidance [122-123]. The fi ndings from the present investig ation suggest that pain and fatigue should be considered when assessing this population. Patient reported outcomes provide a unique opportunity to evalua te clinical practice, from the patients perspective. In the present study, participants required significant changes in a number of health domains to consider their treatment successful. The mobility domain required significantly larger reduc tions than the community and social life; and interactions with people domains. The clinical implication of this finding is that success criteria differ across domains,

PAGE 73

73 suggesting that rehabilitation inte rvention should be guided toward s areas of greatest concern to the patient. This study also explored participants e xpectations with treat ment outcomes. Not surprisingly, participants expected mobility to change more than in other domains (42%). An unanticipated finding was that participants expected the same amount of change in the domain of energy and drive. This finding, suggest that perhaps pa rticipants view this domain as an extension of the mobility domain. That is, participants may connect improvements in mobility with increased energy and drive and, therefore, expect that energy and drive will increase as mobility improves. In this study, participants expected different amounts of change across domains. Again, this finding must be interpreted with caution, since per centage change are heavily influenced by initial scores. For example, a participant report ing a 10 point change from an initial level of 90/100 and an expected level of 80/100 produ ces a percentage change of 11%, while a participant reporting a 10 point change from an initial level of 30/100 and an expected level of 20/100 produces a percentage change of 33%. In this group, participants reported low initial levels of interference with self -care and their expected level was also low. A similar trend was apparent in other domains, where high initial levels also resulted in high expectations, while low initial levels resulted in low expectations. It is plausible to speculate that participants were influenced by their initial scores and ba sed their subsequent answers proportionally. In the present study, participants had reasonabl e treatment expectations Their expectations were lowest in domains related to participat ion, such as community and social life, and interactions with people. Perh aps, participants found it difficu lt to see the connection between improvement in physical function and improveme nt in social roles. Across all domains,

PAGE 74

74 participants reported lower expected scores comp ared to their success criteria. This indicates that, in this population, participants did not expect their treatment to meet their success criteria. That is, participants expected residual levels across domains following treatment. Results of the multivariate analysis of variance indicated no differences in expectations existed between groups of participants based on compliance with the trea tment. However, these results must be interpreted with caution, due to the lack of statistical power resulting from dividing the participant pool into two groups In addition, for a number of participants compliance was unknown, resulting in an even smaller sample size (compliant group N= 14, non-compliant N= 7). Further invest igation is warranted in this ar ea. A larger sample size that represents the general ge riatric population should be used to investigate compliance. The veteran population presents certain character istics that can have an effect on compliance. In the present study, although the sample size was too small to draw statistical conclusions, there were marked differences between VA participants and commun ity participants. VA participants were 40% compliant, while community participants presen ted 80% compliance. In the larger DOH-04NIR15 study, VA compliance was 33%. Compliance with re habilitation interventi ons is essential for individuals who fall, and should be further investigated. For re searchers, lack of compliance represents an additional problem, because of th e bias sample selection. Drawing conclusions about the effectiveness of a particular rehabilitation strategy when samples are only representative of a small group of compliant pa rticipants is highly suspect. There could be confounding factors that make this gr oup different and affect how this gr oup reacts to treatment. This study presents a number of limitat ions. The results obtained are based on a questionnaire specifically design for this invest igation. Although this que stionnaire is based on a previously published instrument [ 104], the psychometric properties of the former and the current

PAGE 75

75 questionnaire have not been investigated. Further work is requir ed to ascertain the reliability, validity, and generalizability of findings obtained with this questionnaire. A methodological strength of this questionnaire is that it uses the langua ge of the ICF. This allows for comparison with other instruments that us e the same language. The ICF cl assification system has been successfully used to develop a number of instruments [124]. Using the ICF classification system, a standardized set of relevant categories can be investig ated for specific populations. Another limitation of this study concerns the limited information about participants characteristics. More specifi cally, the medical diagnosis of the pool of participants was unknown. In addition, two distinct populations were included in this study, veterans and community participants. Although both groups we re community dwellers, there are certain characteristics of the veteran population that could influence their perception, expectation, and success criteria. Veterans participating in this study were recruited during their initial visit to physical therapy services. In contrast, the co mmunity dwellers partic ipating in this study responded to advertisements distributed by the research team. The difference in recruitment approaches could produce bias and have an effect on pa rticipants expectations. The present study is a preliminary attempt at exploring patient reported outcomes and expectations in the treatment of falls. The results of this investigation suggest that patients participating in falls rehabilitation present a number of limitations that far exceed the mobility domain. Participants suc cess criteria varied across domains, suggesting that, for this population, some domains are more important than othe rs. In addition, partic ipants had reasonable expectations, but considered change in th e most affected domains most important. Future work should be guided towards refining and validating the PP OQ. In addition, this instrument could be used to li nk clinical performance measures with patients expectations and

PAGE 76

76 success criteria. This way, the effectiveness of rehabilitation interventi ons can be determined using a mixed clinical/patient centered approach. To provide patients the treatment they deserve, clinicians need to understand the patients expectations and goal s. Patient satisfaction is an important goal for all services, including medical services. This investig ation provides a first look at quantifying some of the areas needed to en sure patients satisfactio n with the treatment of falls.

PAGE 77

77 Table 3-1. Demographics Number of participants UF 12 VA 38 Total 50 Non-Compliant group 7 Compliant group 14 Age 74.1 (SD 10.9) Male 38 Gender Female 12 Table 3-2. Initial levels descriptive statistics N Minimum Maximum Mean Std. Deviation Mobility 50 0 95 47.64* 28.56 Self-care 50 0 95 23.68* 32.17 Interactions with people 50 0 100 20.80* 29.05 Community and social life 50 0 100 33.50 37.70 Energy and drive 50 0 100 53.60 29.67 Mental function 50 0 100 37.60 32.38 Emotional distress 50 0 100 40.14 34.41 Sensory 50 0 95 37.38 28.72 Pain 50 0 100 43.92 34.45 P= .006 Table 3-3. Success criteria (initial leve ls-success levels) descriptive statistics N Minimum Maximum Mean % Change Std. Deviation Mobility 50 0 70 24.87 51.86* 18.94 Self-care 50 0 90 14.66 35.52* 25.14 Interactions with people 50 0 75 9.15 22.27* 18.27 Community and social life 50 0 95 16.91 27.96* 27.05 Energy and drive 50 0 80 31.60 58.84* 23.82 Mental function 50 0 80 21.23 44.99 24.55 Emotional distress 50 0 85 25.13 48.24 26.11 Sensory 50 0 80 15.40 34.33 19.40 Pain 50 0 85 24.53 44.81 24.27 P= .006

PAGE 78

78 Table 3-4. Treatment expecta tions criteria (initial levels-expectations levels) descriptive statistics N Minimum Maximum Mean % Change Std. Deviation Mobility 50 0 70 19.98 42.30* 18.34 Self-care 50 0 60 8.02 23.80* 13.35 Interactions with people 50 0 75 6.28 17.87* 14.00 Community and social life 50 0 95 15.43 26.78* 25.55 Energy and drive 50 0 70 23.38 42.49* 23.10 Mental function 50 0 65 14.51 34.44 17.90 Emotional distress 50 0 85 18.74 38.94 22.83 Sensory function 50 0 80 11.28 32.01 15.86 pain 50 0 80 13.98 27.92 19.19 P= .006

PAGE 79

79 CHAPTER 4 GENERAL SUMMARY AND CONCLUSIONS Overall, the objectives o f this dissertation we re to investigate mini mum detectable change in outcome measures used to assess individuals r eceiving rehabilitations services related to falls, and to explore treatment success criteria and e xpectations, in the same population. Recently, there is considerable debate in the medical and scientific community about what treatment outcomes constitute a meaningful change, and how to measure this change [125-127]. Clinical performance measures do not always capture al l aspects of the patien ts experien ce and often produce results that are difficult to interpret. In an attempt to simplify the interpretation of scores, single numbers are assigned to complex pro cesses. In the name of objectivity, statistical analysis are performed on these scores, and th e results are use to evaluate performance and assess change. Still, statistical significance doe s not necessarily equal meaningful change. Patient reported outcomes (PRO) are receivi ng attention as legitimate outcome measures for clinical research. PRO instruments are used to measure treatment benefits by capturing concepts related to how a patient feels or functions with respect to his or her health or condition. This approach turns to the patient to investigate what is important to then. The ideas, activities, behaviors, or feelings measured by PRO instrument s can be either verifiable in nature, such as walking, or can be non-observable, known only to the patient, such as pain, depression etc. Although these symptoms are highly dependent on the patients perception, historically, these assessments were made by clinicians who observe d and interacted with patients. Recently, these kinds of assessments are increasingly performed with PRO instruments. It seems intuitive to consider the patients opinion when investig ating what constitutes meaningful change. The two experiments described in this dissert ation are aimed at inve stigating change in geriatric patients. Specifically, the group of patients investigated re ceived rehabilitation

PAGE 80

80 interventions related to mobility problems. The ove rall aim of this dissertation was to explore change in this group of patients employing two di fferent approaches. Firs t, minimum detectable change was investigated in clinical instruments used with this population. Second, a newlydeveloped PRO instrument was used to determ ine what constitutes successful treatment outcomes and expectations when participating in a rehabilitati on program. A general conclusion for both studies in this dissertation follows. Experiment I Summary The goal of this study was to investigate m inimal detectable change (MDC) for two common instruments used to assess gait and ba lance in the elder popula tion. The Berg Balance Scale [75] and the Dynamic Gait Index [82] were explored in this ex periment. The procedure outlined by Stratford [86, 92] was used to calculate the MDC. Stratford proposed to use the standard error of measure (SEM) to calculate the amount of change in a given measure that must be obtained for a clinician to determine that true change has occurred. The MDC is expressed as a confidence interval around the SEM, indicating the values that are within the range of error attributable to the measuring instrument. The MDC is expressed in the same unit of the original instrument, providing clinicians useful and easy to understand criterion fo r change in patients performance. The results of this study indicated that for the Berg Bala nce Scale and the Dynamic Gait Index, 6.4 and 2.9 points respectively were required to be 95% conf ident that genuine change had occurred between 2 testing occasions. These results suggest that a significant amount of error is associated with these instruments. In addition, the results suggested that MDC values are not a constant feature of the instruments. MD C values for the high functional level group were 6.3 BBS points. In contrast, participants in the lower functional group presented MDC values of 7.3. That is, the values of MDC change based on the ability level of the persons assessed.

PAGE 81

81 The above findings are relevant to present day clinical practice. Clinicians use the Berg Balance Scale and Dynamic Gait Index routinely to assess patients with mobility problems. An advantage of using the standard error of measure to determine minimal detectable change is that this method provides information about individual scores. Traditional methods of statistical significance rely on group differenc es to investigate the properti es of the instruments. Group differences are relevant for researchers, but mu st be considered with caution when decisions must be made about an individual patient. The resu lts of this investigatio n can be applied at the individual level. Knowing the amount of error associated with these instruments can help clinicians make decisions about individuals performance and monitor change overtime. Experiment II Summary The prim ary aim of this study was to use a PRO questionnaire to investigate patients success criteria and expectations when receiving rehabilitation services related to falls. More specifically, the patients success criteria was assessed across seve ral health domains including: mobility, self-care, interactions with people, comm unity and social life, energy and drive, mental function, emotional distress, sensory function, an d pain. In addition, a secondary aim was to investigate patient expectations for tr eatment across above mentioned domains. In this study, participants with mobility prob lems leading to falls demonstrated significant levels of interference across se veral of the health domains me asured. Participants reported considerable initial leve ls of impairment in domains such as energy and drive (53/100), and pain (44/100). The mobility domain also received hi gh scores (indicating impairment) (47/100). Lower scores were seen in domains such as in teraction with people (21/100) and community and social life (33/100). These findings suggest that, in this populati on, domains with a strong social component were not as affected as domains with a strong physical component.

PAGE 82

82 Participants in this study required significan t improvement in health domains to consider their treatment successful. Domains such as mobility; and energy and drive, required significantly larger reduc tions than the community and social life; and interactions with people domains. This provides information about wh at is important to patients receiving this intervention. Furthermore, these findings coul d lead to developing re habilitation strategies guided towards areas of greates t concern to the patient. This study also explored participants expectat ions with treatment outcomes. Participants expected mobility to change the most as a result of this intervention (52%). However, similar finding was reported in the domain of energy and dr ive. An interesting finding is that across domains, participants expectati on was that the treatment would not meet their success criteria. This indicates that, for this population of elde r individuals with mob ility problems, residual levels of impairment in the measured domains are expected after treatment. Compliance was also investigated in this group. No differences were found between compliant and non-compliant groups based on treatment expectations. The results of this study poi nt out that a number of health domains are significantly affected in this population. Patie nts receiving rehabilitation serv ices related to falls have treatment expectations that far exceeds the mob ility problems for which they are been treated. In exploring meaningful change in patients receiving re habilitation interven tions, the patients expectations and success criteria must be cons idered. By linking existing clinical instruments with patient reported outcomes researchers and clinic ians can be sure that therapies used achieve a meaningful change. Physical re habilitation strategies must take into consideration the complex and multidimensional nature of falls and provide in terventions that target the different domains that affect this population.

PAGE 83

83 General Conclusions and Future Directions The questio n of what constitutes a meaningful ch ange in the population investigated in this dissertation remains unanswered. However, severa l conclusions can be drawn from the work presented. First, clinical instru ments provide valuable informa tion about patients performance and can help clinicians and researchers evalua te patients ability and monitor improvement. However, when these instruments are used at the i ndividual level, clinicians must be aware that all change is not genuine change That is, instrument error must be considered when reporting patient change at the individual level. Sec ond, elder patients receiving mobility-related rehabilitative services expect their treatment to produce changes in a number of health domains that extend beyond mobility improvement. Patient reported outcomes could serve as a bridge to link clin ical practice with meaningful patient-centered treatment results. There is a gr owing movement for patients to take an active role in their medical care and be involved in making decision about tr eatment options [128]. Patients demand services that meet their needs and expect treatments to address nonclinical aspects that affect their day to day life. This view is receiving attention from the research community and regulatory agencies. A number of randomized clinical trials are now including patient reported outcomes as important endpoints in addition to traditional clinical measures [129-130]. Regulatory agencies are also recognizing this need a nd establishing criteria for the use of patient reported outcomes [126]. Incorpor ating the patients view in the rehabilitation process ensures that interventions meet the patients needs and therefore play a role at empowering the patient and making him or her re sponsible for actively participating in the rehabilitative process. A number of interesting questions have arisen as a result of this dissertation. Further work is warranted at investigating minimal detectab le change levels asso ciated with different

PAGE 84

84 functional levels of the patients investigated. This is particularly important for high risk patients for which small change in performance, indi cating improvement or deterioration, can have serious consequences. In this f unctional group, knowing the exact functional level of the patient can help clinicians take appropriate measures to avoid possible injuries, for example prescribe an assistive or protective device. Future work should also be conducted to i nvestigate issues of compliance in this population. In this dissertation, possibly due to small sample size, no significant differences between compliant and non-compliant groups were found. Still, low compliance levels were seen in the investigated group. The trad itional model of care, where clinic ians tell patients what to do and try to motivate them to change, may not be the most effective method of intervention. Empowering the patient by considering their specif ic needs and addressing their expectations might result in better compliance with treatment. PRO instruments can be used to incorporate the patients perspective in the rehabilitative process. These instruments can serve as an anchor to compare clinical instruments against m eaningful patient-centered outcomes. Collectively, this series of studies promotes our understand ing of significant change in patients receiving rehabilitation services related to falls. The results obtained indicate that current rehabilitation programs must consider the limita tions of available instruments and take into consideration the needs and expectat ions of patients. Ultimately, this research aims to influence treatment by providing information to help clinicians select th e best tools available for the rehabilitation of falls, and suggests the inclusion of the patients pe rspective as one of the outcome measures in their treatment plans.

PAGE 85

85 APPENDIX PATIENTS PERSPECTIVE OUTCOME QUESTIONNAIRE (PPOQ) We have identified some common problems people can experience as they get older. Specifically, older people who have balance prob lems and may have suffered a fall often face problems of mobility, self-care, interactions with people, community and social life, energy and drive, mental function, emot ional distress, sensory function and pain (see the last page for an extended explanation of these problem-areas). We would like to ask you a few questions to see how import ant these problems are to you. FIRST, WE WOULD LIKE TO KNOW YOU R USUAL LEVELS OF MOBILITY, SELFCARE, INTERACTION WITH PEOPLE, CO MMUNITY AND SOCIAL LIFE, ENERGY AND DRIVE, MENTAL FUNCTION, EMOTI ONAL DISTRESS, SENSORY FUNCTION, AND PAIN. On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate your usual level (during the past week) of mobility __________ self-care __________ interactions with people __________ community and social life __________ energy and drive __________ mental function __________ emotional distress __________ sensory function ___________ pain __________ PATIENTS UNDERSTANDABLY WANT THEIR TREATMENT TO RESULT IN DESIRED OR IDEAL OUTCOMES. UNFORTUNATELY, AVAILABLE TREATMENTS DO NOT ALWAYS PRODUCE DESIRED OUTCOMES. TH EREFORE, IT IS IMPORTANT FOR US TO UNDERSTAND WHAT TREATMEN T OUTCOMES YOU WOULD CONSIDER SUCCESSFUL. On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate the level each of these areas would have to be at to consider treatment successful. mobility __________ self-care __________ interactions with people __________ community and social life __________ energy and drive __________ mental function __________ emotional distress __________ sensory function __________ pain __________

PAGE 86

86 NOW, WE WOULD LIKE TO KNOW WHAT YOU EXPECT YOUR TREATMENT TO DO FOR YOU On a scale of 0 (none/not affected) to 100 (worst imaginable/most affected), please indicate the levels you expect following treatment. mobility __________ self-care __________ interactions with people __________ community and social life __________ energy and drive __________ mental function __________ emotional distress __________ sensory function ___________ pain __________ FINALLY, WE WOULD LIKE TO UNDERSTAND HOW IMPORTANT IT IS FOR YOU TO SEE IMPROVEMENT IN YOUR MOBILITY SELF-CARE, INTERACTIONS WITH PEOPLE, COMMUNITY AND SOCIAL LIFE, ENERGY AND DRIVE, MENTAL FUNCTION, EMOTIONAL DISTRESS, SENS ORY FUNCTION, AND PAIN FOLLOWING TREATMENT. On a scale of 0 (not at all important) to 100 (mos t important), please indica te how important it is for you to see improvement in your mobility __________ self-care __________ interactions with people __________ community and social life __________ energy and drive __________ mental function __________ emotional distress __________ sensory function ___________ pain __________

PAGE 87

87 Problem areas explained: Mobility : This term refers to the ability to change location or transfer from one place to another. It also includes actions such as carrying, moving or ma nipulating objects and capacity to walk, run or climb. Lastly, mobility also refers to the ability to use various forms of transportation. Self-care: This problem-area is about caring for oneself, washing and drying oneself, caring for one's body and body parts, dressi ng, eating and drinking, and looking after one's health. Interactions with people: This area is about the ability to socially interact with strangers, friends, relatives, family members and significant others. Community and social life: This area is about the actions and tasks required to engage in organized social life outside the family, in community, social and civic areas of life. Examples include: participating in religious or spiritual activities and participation in leisure or recreational activities. Energy and drive : This problem area refers to feelings of fatigue, motivation and energy level. Mental function: This problem area includes issu es related to memory, attention, concentration, and decision making. Emotional distress : This problem area includes feelin gs of depression, anger, anxiety, and frustration. Sensory function : This area includes problems of vision and hearing. Pain : This area includes all types of pain, including chronic and acute.

PAGE 88

88 LIST OF REFERENCES 1. Miaskowski C. The im pact of age on a patien t's perception of pain and ways it can be managed. Pain Manag Nurs 2000; 1:2-7. 2. Centers for Disease Control a nd Prevention, Department of Health and Human Services. Healthy Aging for Older Adults [online] (2007) [cited 2008 Apr 14]. Available from URL: http://www.cdc.gov/aging/ 3. Seematter-Bagnoud L, Wietlisbach V, Yersin B, Bla CJ. Healthcare Utilization of Elderly Persons Hospitalized After a Noninj urious Fall in a Swiss Academic Medical Center. Journal of the American Geriatrics Society, 2006; 54(6),891. 4. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Web-based Injury Statistics Quer y and Reporting System (WISQARS) [online]. (2006) [cited 2008 Apr 14]. Available from URL: www.cdc.gov/ncipc/wisqars 5. Hausdorff JM, Rios DA, Edelber HK. Gait vari ability and fall risk in community-living older adults: a 1-year prospective stu dy. Archives of Physical Medicine and Rehabilitation 2001; 82(8):1050. 6. Englander F, Hodson TJ, Terregrossa RA. Econo mic dimensions of slip and fall injuries. Journal of Forensic Science 1996; 41(5):733. 7. World Health Organization. International Clas sification of Functioning (ICF), Disability and Health. Geneva: WHO; 2001. 8. Engel GL. The need for a new medical model: a challenge for biomedicine. Science 1977; 196(4286):129-36. 9. Kirshner B, Guyatt GH. A methodological fram ework for assessing health indices. J Chronic Dis 985; 38:27. 10. Guyatt GH, Jaeschke R, Feeny DH, Patrick DL. Measurements in clinical trials: choosing the right approach. In: B. Sp ilker, Editor, Quality of life and pharmacoeconomics in clinical trials, Lippi ncott-Raven Publishers, Philadelphia;1996; 41. 11. Guyatt GH, Feeny DH, Patrick DL. Measuring h ealth-related quality of life. Ann Intern Med1993; 118:622. 12. Wyrwich KW, Metz S, Babu AN. The reliability of retrospective change assessments. Qual Life Res 2002; 11:636. 13. Sim J, Arnell P. Measurement validity in P hysical Therapy research. Phys Ther 1993; 73: 102-15.

PAGE 89

89 14. Juniper EF, Guyatt GH, Epstein RS, Ferrie PJ Jaeschke R, Hiller TK. Evaluation of impairment of health related quality of life in asthma: development of a questionnaire for use in clinical trials. Thorax 1992; 47:76. 15. Portney LG, Watkins MP. Reliability. In: Portney LG, Watkins MP. Foundations of Clinical Research: applications to practi ce. 2nd ed. New Jersey: Prentice Hall Health; 2000: 53-68. 16. Walmsley RP, Amell TK. The application and interpretation of intraclass correlation in the assessment of reliability in Isokinetic dynamometry. Isokinet Exerc Sci 1996; 6:11724. 17. Juniper EF, Guyatt GH, Epstein RS, Ferrie PJ Jaeschke R, Hiller TK. Evaluation of impairment of health related quality of life in asthma: development of a questionnaire for use in clinical trials. Thorax 1992; 47:76. 18. Kirshner B, Guyatt G. A methodological fram ework for assessing health indices. J Chronic Dis 1985; 38: 27-36. 19. Deyo RA, Centor RM. Assessing the responsiven ess of functional scales to clinical change: an analogy to diagnostic test performance J Chronic Dis. 1986; 39:897. 20. Salter K, Jutai JW, Teasell R, Foley NC, BIte nsky J, Bayley M. Issues for selection of outcome measures in stroke rehabilitatio n: ICF activity. Disab il Rehabil 2005; 27:315 340. 21. Jaeschke R, Singer J, Guyatt GH. Ascerta ining the minimal clinically important difference. Control Clin Trials 1989; 10:407. 22. Samsa G, Edelman D, Rothman ML, Williams GR, Lipscomb J, Matchar D. Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1995; 15:141. 23. Osoba D, Rodrigues G, Myles J, Zee B, Pate r J. Interpreting the si gnificance of changes in health-related quality-of-life scores. J Clin Oncol 1998; 16:139. 24. Norman GR, Sridhar FG, Guyatt GH, Walter SD. The Relation of Distributionand Anchor-Based Approaches in Interpretation of Changes in Health Related Quality of Life. Medical Care 2001; 39:1039. 25. Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, and the Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings 2002; 77:371. 26. Deyo RA, Inui TS, Leininger J, Overman S. Physical and psychosocial function in rheumatoid arthritis: clinical use of a self -administered health status instrument. Arch Intern Med 1982; 142:879.

PAGE 90

90 27. Johnson PA, Goldman L, Orav EJ, Garcia T, Steven D, Pearson, et al. Comparison of the medical outcomes study short-form 36-item h ealth survey in black patients and white patients with acute chest pain. Med Care 1995; 33:145. 28. Testa MA, Simonson DC. Assessment of qua lity-of-life outcomes. New Engl J Med 1996; 28:835. 29. Jacobson NS, Truax P. Clinical significan ce: a statistical approach to defining meaningful change in psychotherapy rese arch. J Consult Clin Psychol 1991; 59:12. 30. Samsa G, Edelman D, Rothman ML, Williams GR, Lipscomb J, Matchar D. Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1995; 15:141. 31. Revicki DA, Allen H, Bungay K, Williams GH, Weinstein MC. Responsiveness and calibration of the general wellbeing adjustment scale in pa tients with hypertension. J Clin Epidemiol 1994; 437:1333. 32. Jaeschke R, Singer J, Guyatt GH. Measurem ent of health status. Ascertaining the minimal clinically important difference. Control Clin Tria ls 1989; 10:407-415. 33. Idler EL, Angel RJ. Self-rate health a nd mortality in the NHANES-I epidemiology follow-up study. Am J Public Health 1990; 80:446. 34. Farar JT. What is clinically meaningful: outcome measures in pain clinical trials. Clin J Pain 200; 16(2 Suppl):S106-12. 35. Lydick F, Yawn BP. Clinical interpretation of health-related quality of life data. In: M.J. Staquet, R.D. Hays and P.M. Fayers, Editors, Quality of life assessment in clinical trials: methods and practice. Oxford Univ ersity Press, Oxford 1998:299. 36. Husted JA, Cook RJ, Farewell VT, Gladman DD Methods for assessing responsiveness: a critical review and recommendati ons. J Clin Epidemiol 2000; 53:459. 37. G.R. Norman, P. Stratford and G. Regehr. Methodological problems in the retrospective computation of responsiveness to change: th e lesson of Cronbach. J Clin Epidemiol 1997; 50:869. 38. Liang NH, Larson MG, Cullen KE, Schwartz JA. Comparative measurement efficiency and sensitivity of five health status instru ments for arthritis research. Arthritis Rheum 1985; 28:542. 39. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures: statistics and strategies for ev aluation. Control Clin Trial 1991; 12:142SS. 40. Byrk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis methods. Newbury Park (CA), Sage; 1992.

PAGE 91

91 41. Speer C, Greenbaum PD. Five methods for computing significan t individual client change and improvement rates: support fo r an individual growth curve approach. J Consult Clin Psychol 1995, 63:1044. 42. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods 7; 2002:147. 43. Cohen, J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates 1988:34. 44. Kazis LE, Anderson JJ, Meenan RS. Effect sizes for interpreting changes in health status. Med Care 1989; 27: Suppl 3:S178S189. 45. Fayers PM, Machin D. Quality of life: a ssessment, analysis and interpretation. John Wiley&Sons, Chichester; 2000. 46. Clancy CM, Eisenberg JM. Outcomes research: measuring the end results of health care. Science 1998; 282(5387):245-6. 47. Wyrwich KW, Tierney WM, Wolinsky FD. Fu rther evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol 1999; 52:861. 48. McHorney CA, Tarlov A. Individual-patient monitoring in clinical practice: are available health status surveys ad equate. Qual Life Res 1995; 4:293. 49. Anastasi A, Urbina S. Psyc hological Testing. 7th ed. Upper Saddle River, NJ: PrenticeHall; 1997. 50. Hageman WJ, Arrindell WA. Establishing clinic ally significant change: increment of precision and distinction between individual and group level of analysis. Behav Res Ther 1999; 37:1169. 51. Roebroeck ME, Harlaar J, Lankhorst GJ. The a pplication of generali zability theory to reliability assessment: an illustration using isometric force measurements. Phys Ther 1993; 73 (6): 386-95. 52. Engel GL. The need for a new medical model: a challenge for biomedicine. Science 1977; 196(4286):129-36. 53. Engel GL. The clinical application of the biopsychosocial model. Am J Psychiatry1980; 137: 535-544. 54. U.S. Food and Drug Administration. Guidan ce for Industry. Patient-Reported Outcome measures : use in medical product development to support labeling claims. Draft guidance. [cited 2008 June 20]. Available from: http://www.fda.gov/cder/guidance/5460dft.pdf

PAGE 92

92 55. Hippocrates. The Book of Prognostics, Part 2 (approxi. 400 B.C.). As translated by Francis Adams (Great Books Index, 1977-99) [cited February 2008]. Available from: http://classics.mit.edu/Hippocrates/prognost.mb.txt 56. FDA Consumer Magazine. The Importance of Patient-Reported Outcomes Its All About the Patients 2006; Vol 40: 6. Available from: http://www.fda.gov/fdac/606_toc.html. 57. Revicki DA, Osoba D, Fairclough D, Barofs ky I, Berzon R, Leidy NK, Rothman M. Recommendations on health-rela ted quality of life resear ch to support labeling and promotional claims in the United States. Qual. Life Res. 2000; 9:887. 58. Patrick DL, Erickson P. Health status and health policy: quality of life in health care evaluation and resource allocation. Oxfo rd University Press, New York; 1993. 59. Carmines E G, Zeller RA. Reliability and validity assessment. Newbury Park: Sage Publications; 1991. 60. Ostir G, Granger C, Black T, Roberts P, Bu rgos L, Martinkewiz P, et al. Preliminary Results for the PAR-PRO: A Measure of Ho me and Community Participation. Arch Phy Med and Rehab 2006; 87(8):1043-51. 61. Willkea RJ, Burkeb LB, Ericksonc P. Measuring treatment impact: a review of patientreported outcomes and other efficacy endpoint s in approved product labels. Controlled Clinical Trials 2004; 25(6):535-52. 62. Revicki DA, Osoba D, Fairclough D, Barofs ky I, Berzon R, Leidy NK, Rothman M. Recommendations on health-rela ted quality of life resear ch to support labeling and promotional claims in the United States Qual Life Res. 2000; 9(8):887-900. 63. Zerhouni E. NIH Roadmap. Sc ience 2003; 302(5642):63-72. 64. Stucki G. International cla ssification of functioning, disa bility and health (ICF): A promising framework and classification for rehabilitation medicine. Am J Phys Med Rehabil 2005; 84:733. 65. World Health Organization. International Cl assification of Diseases, Disorders and Injuries, 10th revision. Geneva : World Health Organization, 1994. 66. Degner L, Sloan J. Symptom distress in newly diagnosed ambulator y cancer patients and as a predictor of survival in lung cancer. J Pain Symptom Manage 1995; 10:1. 67. Sloan JA, Loprinzi CL, Kuross SA, Miser AW, O'Fallon JR, Mahoney MR, et al. Randomized comparison of four tools measuri ng overall quality of life in patient with advanced cancer. J Clin Oncol 1998; 16:3662. 68. Frost MH, Huschka M. Quality of Life from a Patients Perspective: Can We Believe the Patient? Current Problems in Cancer. 2005; 29(6):326-331.

PAGE 93

93 69. Hodgkins M, Albert D, Daltroy L. Comparing patients and their physicians assessment of pains. Pain 1985; 23: 273. 70. Patel KK, Veenstra DL, Patrick DL. A Review of Selected Patient-Generated Outcome Measures and Their Applicati on in Clinical Tria ls. Value in Hea lth 2003; 6(5):595603(9). 71. Davidhizar R, Giger JN. A review of the literature on care of clients in pain who are culturally diverse. In ternational Nursing Re view 2004; 51(1):47-55. 72. Atkinson MJ, Lennox RD. Extending basic prin ciples of measurement models to the design and validation of Patie nt Reported Outcomes. Health Qual Life Outcomes 2006; 4: 65. 73. Lohr KN. Assessing health stat us and quality-of-life instrume nts: attributes and review criteria. Qual Life Res. 2002; 11:193. 74. Medley A. Predicting the probability of fa lls in community dwelling persons with brain injury: a pilot study. Brain Inj. 2006; 20(13-14):1403-8. 75. Berg K, Maki B, Williams JI, Holliday PJ, Wood-Dauphinee SL. Clinical and laboratory measures of postural balance in an elderl y population. Archives Of Physical Medicine and Rehabilitation 1992a; 73:1073-1080. 76. Berg KO, Wood-Dauphine SL, Williams JI, Maki B. Measuring balance in the elderly: validation of an instrument. Can J Public Health.1992b; 83(suppl 2):S7S11. 77. Wolf SL, Catlin PA, Gage K, Gurucharri K, Robertson R, Stephen K. Establishing the reliability and validity of measurements of walking time using the Emory Functional Ambulation Profile. Phys Ther.1999; 79:1122. 78. Liston RA, Brouwer BJ. Reliability and valid ity of measures obtained from stroke patients using the Balance Master. Ar ch Phys Med Rehabil.1996; 77:425. 79. Berg K, Wood-Dauphinee S, Williams JI, Gayton D. Measuring balance in the elderly: preliminary development of an inst rument. Physiother Can. 1989c; 41:304. 80. Shumway-Cook A, Woolhicoii MH. Motor Cont rol: Theory and Pract ical Applications, Philadelphia, Pa: Lippincott Williams & Wilkins; 2001:401, 405-406, 81. Whitney S, Wrisley D, Furman J. Concurrent validity of the Berg Balance Scale and the Dynamic Gait Index in people with vestib ular dysfunction. Physiother Res Int 2003; 8(4):178-86. 82. Shumway-Cook A, Baldwin M, Polissar NL, Gruber W. Predicting the probability for falls in community-dwelling older adu lts. Physical Therapy 1997; 77(8):812.

PAGE 94

94 83. Bogle Thorbahn LD, Newton RA. Use of the Be rg Balance Test to predict falls in elderly persons. Phys Ther 1996 Jun;76(6):576-83; discussion 584-5. 84. Whitney SL, Hudak MT, Marchetti GF. The dyna mic gait index relate s to self-reported fall history in individuals with vestibular dysfunction. J Vestib Res. 2000; 10:99-105. 85. Guyatt GH, Kirshner B, Jaeschke R. Measuri ng health status: what are the necessary measurement properties? J Clin Epidemiol. 1992; 45:1341. 86. Stratford PW. Reliability: consistency or diffe rentiating among subjects (editorial). Phys Ther 1999: 69: 299. 87. Shikiar R, Harding G, Leahy M, Lennox RD. Minimal important difference (MID) of the Derm atology Life Quality Index (DLQI): result s from patients with chronic idiopathic urticaria. Health Qual Life Outcomes. 2005; 20;3:36. 88. Birm ingham TB, Kramer JF, Speechley M, Chesworth BM, MacDermid J. Measurement variability and sincerity of effort: clinical utility of isokinetic st rength coefficient of variation scores. Ergonomics 1998: 41: 853. 89. Bland JM, Altman DG. Measur ement error. BMJ 1996; 313:744. 90. Roebroeck ME, Harlaar J, Lankhorst GJ. The a pplication of generali zability theory to reliability assessment: an illustration using isometric force measurements. Phys Ther 1993; 73(6):386-95. 91. Stevenson TJ. Detecting change in patients w ith stroke using the Berg Balance Scale. Australian Journal of P hysiotherapy 2001; 47:29-38. 92. Stratford P, Finch W, Solom on E, Binkley P, Gill J, Moreland C. Using the RolandMorris questionnaire to make decisions about individual patients. Physiotherapy Can 1996, 48:107. 93. Berg K, Wood-Dauphinee S, Williams JI. The Ba lance Scale: reliability assessment with elderly residents and patients with an acute stroke. Scandinavian Journal of Rehabilitation Medicine 1995; 27:27-36. 94. Tyson SF, DeSouza LH. Reliability and validity of functional balance tests post stroke. Clin Rehabil 2004; 18: 916. 95. Mao HF, Hsueh IP, Tang PF, Sheu CF, Hsieh CL. Analysis and comparison of the psychometric properties of three balance measures for stroke patients. Stroke 2002; 33: 1022. 96. Ottonello M, Ferriero G, Benevolo E, Sessare go P, Dughi D. Psychometric evaluation of the Italian version of the Berg balance scal e in rehabilitation inpa tients. Eur Med Phys 2003; 39:181.

PAGE 95

95 97. Folstein MF, Folstein SE, McHugh PR. "Min i-mental state": a practical method for grading the cognitive state of patients for the clinicia n. Psychiat. Res 1975; 12(3):189198. 98. Guyatt G, V Montori PJ, Devereaux H, Schne mann H, Bhandari M. Patients at the centre: In our prac tice and in our use of language ACP Journal Club 2004; 140: A-11. 99. Chang JT, Morton SC, Rubenstein LZ, Moji ca W, Maglione M, Suttorp M, et al. Interventions for the prevention of falls in older adults: Systematic review and metaanalysis of randomized controlled trials British Medical Jo urnal 2004; 328:680-683. 100. Gillespie LD, Gillespie W J, Roberts on MC, Lamb SE, Cumming RG, Rowe BH. Interventions for preventing falls in elderly people (Cochrane Review). London: Wiley 2001. 101. Skelton D, Todd C. What are the main risk factors for falls amongst older people and what are the most effective interventions to prevent these falls? How should interventions to prevent falls be implemented? Health Evidence Network Synthesis. Copenhagen, Denmark: World Health Organization 2004. 102. Robertson M C, Devlin N, Gardner MM, Campbell A J. Effectiveness and economic evaluation of a nurse delivered home exerci se program to prevent falls: Randomized controlled trial. British Me dical Journal 2001; 322:697-701. 103. Stevens M, Holman CD, Bennett N, de Kler k N. Preventing falls in older people: Outcome evaluation of a randomized controlled tr ial. Journal of the American Geriatrics Society 2001;49:1448-1455. 104. Robinson M, Brown J, George S, Edwards P, Atchison J, Hirsh A, Waxenberg L, Wittmer V, Fillingim R. Multidimensional succe ss criteria and expectations for treatment of chronic pain: The patient perspectiv e. Pain Medicine 2005; 6(5):336-45. 105. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Web-based Injury Statistics Qu ery and Reporting System (WISQARS). [cited 2008 Jan 15]. Available from URL: www.cdc.gov/ncipc/wisqars. 106. Crosby RD, Kolotkin RL, Williams GR. Defi ning clinically meaningful change in health-related quality of life. J Clin Epidemiol. 2003; 56:395. 107. Beaton DE, Boers M, Wells GA. Many faces of the minimal clinically important difference (MCID): a literature review and directions for future research. Curr Opin Rheumatol. 2002; 14:109. 108. De Vet HC, Terwee CB, Ostelo RW, Becker man H, Knol DL, Bouter LM. Minimal changes in health status ques tionnaires: distinctio n between minimally detectable change and minimally important change. Health Qual Life Outcomes 2006; 4:54.

PAGE 96

96 109. Schnemann HJ, Guyatt GH. Goodbye MCID! Hello MID, Where Do You Come From? Health Serv Res 2005; 40(2):593. 110. Ward SE, Gordon DB. Patient satisfaction a nd pain severity as outcomes in pain management: a longitudinal view of one set ting's experience. J Pain Symptom Manage. 1996; 11:242-251. 111. Pellino TA, Ward SE. Perceived control mediat es the relationship between pain severity and patient satisfaction. J Pain Symptom Manage 1998; 15:110-116. 112. Calkins E, Boult C, Wagner E, Pacala JT. New ways to care for older people. Building systems based on evidence. New York: Springer; 1999. 113. Bergland A, Wyller TB. Risk factors for seri ous fall-related injury in elderly women living at home. Injury Prevention 2004; 10:308-313. 114. Sattin RW. Falls among older persons: A public health perspective. Annual Review of Public Health 1992; 13:489-508. 115. Tinetti ME, Doucette J, Claus E, Marottoli R. Risk factors for serious injury during falls by older persons in the community. Journal of the American Geri atrics Society 1995; 43:1214-1221. 116. International Classification of Functioning (IC F), Disability and H ealth. World Health Organization. Vol. 2002, 2001. 117. Gillespie LD, Gillespie WJ, Cumming R, Lamb SE Rowe BH. Interventions to reduce the incidence of falling in the elde rly. Cochrane Library 1997; 4:1-29. 118. Lichstein KL, Means MK, Noe SL, Aguillard RN. Fatigue and sleep disorders. Behaviour Research and Therapy 1997; 35:733. 119. Karlsen K, Larsen JP, Tandberg E, Jorgense n K. Fatigue in patients with Parkinsons disease. Movement Disorders, 1999; 14:237. 120. Nail LM. Fatigue in patients with ca ncer. Oncol Nurs Forum 2002; 29:537. 121. Swain MG. Fatigue in chronic disease. Clin Sci 2000; 99:1. 122. Peat TG, Harris LH, Wilkie R, Croft PR. The pr evalence of pain and pa in interference in a general population of older ad ults: cross-sectional findings from the North Staffordshire Osteoarthritis Project (NorSTOP). Pain 2004; 110:361. 123. Blyth FM, Cumming R, Mitchell P, Wang JJ. Pain and falls in older people. European Journal of Pain 2007; 11(5):564-571.

PAGE 97

97 124. Cieza A, Brockow T, Ewert T, Amman E, Kolleri ts B, Chatterji S, et al. Linking healthstatus measurements to the International Classification of Functioning; Disability and Health. J Rehabil Med 2002; 34: 205-10. 125. Middel B, van Sonderen E. Statistically signif icant change versus relevant or important change in (quasi) experimental de sign. Int J Integr Care 2002; 2: 1. 126. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, O'Neill R, Kennedy DL. Patient-Reported Outcomes to Support Medical Product Labeling Claims: FDA Perspective. Value in Health 2007; 10(s2):S125S137. 127. Rothman ML, Beltran P, Pharm D, Cappeller i JC, Lipscomb J, Teschendorf B. The Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group Value in Health 2007; 10(s2):S66S75. 128. Wagner EH, Austin BT, Von Koroff M. Improving outcomes in chronic illness. Managed Care Quarterly 1996; 4(2):12-25. 129. Revicki. FDA draft guidance and health -outcomes research. The Lancet 2007; 369(9561):540-542 D. 130. Fairclough DL. Patient reported outcomes as endpoints in medical research. Stat Methods Med Res 2004; 13:115-38.

PAGE 98

98 BIOGRAPHICAL SKETCH Sergio Romero was born and raised in the South of Spain. He m oved to the USA in 1991 to pursue his studies in the area of exercise and sport science. In 1996, he received a bachelors degree in exercise and sport science from the Univ ersity of Florida. He continued his studies at University of Florida and receiv ed a masters degree in exercise and sport science in 1998. The next few years, he worked in a variety of departme nts at this university and developed an interest in geriatrics, and more specifically in falls pr evention and falls rehabilitation. After taking a few courses in geriatrics during 2002 and 2003, he wa s officially accepted in the rehabilitation science doctoral program in the college of Publ ic Health and Health Professions in spring 2004. He specialized in movement disorders and continue d working in the area of falls in the geriatric population. He was involved in a va riety of research projects. In 2007, he received a pre-doctoral national fellowship from the Veterans Administra tion (VA) to work on hi s dissertation project. The financial support he received from the VA allowed him to dedicate a full year to the completion of this dissertation project. He r eceived a Doctor of Philosophy degree from the University of Florida in August 2008.