<%BANNER%>

Improving Problem Solving Efficiency

Permanent Link: http://ufdc.ufl.edu/UFE0021617/00001

Material Information

Title: Improving Problem Solving Efficiency The What and How of Caution
Physical Description: 1 online resource (106 p.)
Language: english
Creator: Knowles, Martin
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: beaker, cannibals, caution, crossing, efficiency, error, hobbit, illegal, learning, missionaries, moves, orcs, penalty, problem, river, solving, titration, transfer
Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: A three-stage framework was proposed describing the reduction of illegal moves in a problem solving situation. The Caution Induction Framework (CIF) proposed that consequences for making or avoiding illegal moves in the first stage of the framework influenced the problem solver to increase the amount of attention devoted to the problem in the second stage. This increase in attention subsequently increased checking or evaluation behavior in the third stage resulting in the reduction of illegal moves. Three experiments were presented to test specific aspects of the framework. The first experiment tested whether a punishment was necessary to reduce illegal moves or if threat alone, without punishment, was sufficient to yield the same result. The second experiment tested whether an aversive stimulus was necessary or if a positive consequence would yield illegal move reductions. Specifically, participants in the critical condition in the second experiment received candy for avoiding illegal moves and for solving the problem efficiently. The third experiment tested whether an instruction to check each move for legality would result in illegal move reductions. In addition, participants in the critical condition in the third experiment were asked to perform this behavior aloud while their voice was recorded. The results from the three experiments mostly supported the proposed framework of CIF; however, future research is required to further test the assumptions of the framework.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Martin Knowles.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Berg, William K.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0021617:00001

Permanent Link: http://ufdc.ufl.edu/UFE0021617/00001

Material Information

Title: Improving Problem Solving Efficiency The What and How of Caution
Physical Description: 1 online resource (106 p.)
Language: english
Creator: Knowles, Martin
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: beaker, cannibals, caution, crossing, efficiency, error, hobbit, illegal, learning, missionaries, moves, orcs, penalty, problem, river, solving, titration, transfer
Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: A three-stage framework was proposed describing the reduction of illegal moves in a problem solving situation. The Caution Induction Framework (CIF) proposed that consequences for making or avoiding illegal moves in the first stage of the framework influenced the problem solver to increase the amount of attention devoted to the problem in the second stage. This increase in attention subsequently increased checking or evaluation behavior in the third stage resulting in the reduction of illegal moves. Three experiments were presented to test specific aspects of the framework. The first experiment tested whether a punishment was necessary to reduce illegal moves or if threat alone, without punishment, was sufficient to yield the same result. The second experiment tested whether an aversive stimulus was necessary or if a positive consequence would yield illegal move reductions. Specifically, participants in the critical condition in the second experiment received candy for avoiding illegal moves and for solving the problem efficiently. The third experiment tested whether an instruction to check each move for legality would result in illegal move reductions. In addition, participants in the critical condition in the third experiment were asked to perform this behavior aloud while their voice was recorded. The results from the three experiments mostly supported the proposed framework of CIF; however, future research is required to further test the assumptions of the framework.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Martin Knowles.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Berg, William K.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0021617:00001


This item has the following downloads:


Full Text
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101108_AAAAAB INGEST_TIME 2010-11-08T05:25:55Z PACKAGE UFE0021617_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 25271604 DFID F20101108_AAABFG ORIGIN DEPOSITOR PATH knowles_m_Page_040.tif GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
0eeb6f3edcd49914afd13db987062938
SHA-1
77ef650b7d1a1f3e32b83e5527910da9d13facbc
F20101108_AAABER knowles_m_Page_024.tif
1282f4355931af0cb2f7f5d4c6aea8e2
ae4f67988429f0d5854f6cc63f26b613562bbb30
111190 F20101108_AAAAZL knowles_m_Page_084.jpg
c9c18ff17b1cea21624b3c5349ece3b6
34a9a2b01973402a8a7fbe44245e5429bc4d5d3a
110026 F20101108_AAAAYX knowles_m_Page_070.jpg
44a2bb9576a563e5d46ebe3f46735cfb
9773a2890df022ef50265cc68b191e51842e1567
F20101108_AAABFH knowles_m_Page_041.tif
8435907cfaf28cf5b291f3e9edfc0a8f
d8269901664893d7f7c35888f697f322daa0d6ac
F20101108_AAABES knowles_m_Page_025.tif
1a926e1e1ef7fab5819541760a616605
87a639d5dd4f61e1478cfc5037054ae7e43934d0
117695 F20101108_AAAAZM knowles_m_Page_085.jpg
a8c9df522f460beb6e5d82660109294b
7ffe74101d3c38a8356f28a480b82467b1be2ef4
93829 F20101108_AAAAYY knowles_m_Page_071.jpg
a9aa6b2c6cbe09935f0b517c601576f2
7509f0cc3b19cdff8700ccceb2f05c744f1d08ec
F20101108_AAABFI knowles_m_Page_042.tif
8e531a09f44ddb3e7edb8f03d0dd981f
35d132a8bbc8efb8f8b9d12de7eae51458a814b6
F20101108_AAABET knowles_m_Page_026.tif
dab835078f4df751b7b2d676eada92c8
ab1f206acebe03dbf77b5d2dc6940334588b0d7f
107123 F20101108_AAAAZN knowles_m_Page_086.jpg
deddccc1c52b376922c71423bf40b17d
a6ab34710a89cdc9780095d6441987d36207af5c
108614 F20101108_AAAAYZ knowles_m_Page_072.jpg
f24b213f7dba6f48b3a3ab30c17857ab
1721da60c60d80cb26471745008eda21832ab874
F20101108_AAABEU knowles_m_Page_027.tif
f29d5d919c74db1feb5e4a998f3a382b
a1d072f8aed3f2ddfd7d8c2f679feceb50ecea1d
107090 F20101108_AAAAZO knowles_m_Page_087.jpg
0af1ef95a4ffb6a0d6296f39ddd7d6ad
16346286c7c14162b80f9b8b78df7ff386af1953
F20101108_AAABFJ knowles_m_Page_043.tif
713bb9a7d0f74df0e71b97abf9c1089c
2c5e41fe8b86d50bd365ebb4b6c12d66bb24ed26
F20101108_AAABEV knowles_m_Page_028.tif
be20802827cc8bbae4e45e52ecc34aad
1ce466f12c53c43702f493df5aa8ab3ccca4d0fe
106749 F20101108_AAAAZP knowles_m_Page_088.jpg
2077239436a0b761c28662f6d09635a4
b1234b0176904d9fbc37fb6b61b8d3d6fda815a7
F20101108_AAABFK knowles_m_Page_044.tif
acd5b01739d4b3f53150d195a45bdee0
98bff626fce60862429849a5e571c6142bb96e0a
F20101108_AAABEW knowles_m_Page_029.tif
a58c1db87bd566e601479a3c49beaa10
fe3e3c115829d2a2f593f857cca7119f2473f63c
102115 F20101108_AAAAZQ knowles_m_Page_089.jpg
9d5bc9f8ced3de458b6760674cac238a
98aaa366eb1a86d743b73cccd7993283c94f5704
F20101108_AAABFL knowles_m_Page_047.tif
c53dca15178d22befce5600a5a8a7789
d042b8d3b41af826e8d8648a8eff1ebcce9417d3
F20101108_AAABEX knowles_m_Page_030.tif
0c3b52ba53c75fe6ecb4e0a3a5ccf447
f636744d24adaa6bbf2b66ca11d0be1131b293f5
109527 F20101108_AAAAZR knowles_m_Page_090.jpg
55de5e2f5cce10132c819cd89a87f932
c060244f0fe62c0691e8a326b120a63ac5535ce4
F20101108_AAABGA knowles_m_Page_063.tif
3e7ca4cd0e5829e693678ae1506cff2d
de24240bc22b60cd9563209205d12de7ad61053c
F20101108_AAABFM knowles_m_Page_048.tif
037fbbc2ad57221be55771f2e5e78906
02d651eea0b59b8d5c38172f501e0d3b21b2849f
F20101108_AAABEY knowles_m_Page_032.tif
44a8cd0c60e80bcd8f3f8bdc7a7b5cf6
0f36b0b41db72a9ac8e9362714fffaf4aa07df20
110707 F20101108_AAAAZS knowles_m_Page_091.jpg
b93973639fb79fdbcce3c0aa2f831247
b0953720c7b602cbe2e2d0b63648cf932915767f
F20101108_AAABGB knowles_m_Page_064.tif
2d5e680863ba1196ce40d7dde349c2f1
5378e889a341fd54f26b2015828db9d256e0830e
F20101108_AAABFN knowles_m_Page_049.tif
31a32ed542d0b162c388845ff7e9892a
f3d134c650022baa645a758af127c8456bc2b5d7
F20101108_AAABEZ knowles_m_Page_033.tif
0d8c9f314f7d8520cf52d9c083906d70
b23e1d4777364accc895260c268ad59233a2441a
107363 F20101108_AAAAZT knowles_m_Page_093.jpg
08b92600c6060909fb5797fb127274c1
3888a19be73dd8fe1556b42af60051662dbb722b
F20101108_AAABGC knowles_m_Page_065.tif
e0e38d91a939d61d66e8a0556a708d98
3864a515fa8883b5a557fb3e83eaf44e86d1de6b
F20101108_AAABFO knowles_m_Page_050.tif
d2580b88b1a6a93c3651de083716ec74
c5f98af183a712ce283ee6575d45f22d692b479f
112034 F20101108_AAAAZU knowles_m_Page_094.jpg
237ab2909129148e76401f154b4127c5
720a5b0d132a9d1c93619551bff642b4ff750e21
F20101108_AAABGD knowles_m_Page_066.tif
b5d0f493c886a4f87d65b8547190bddd
adf9dd06b19a71e6a0159ef1414f53ab29254731
F20101108_AAABFP knowles_m_Page_051.tif
d144c237eb73d762403a49264a0299e4
9b116092bcb572c3204b4aa82ef19ae4cb331e76
103436 F20101108_AAAAZV knowles_m_Page_095.jpg
12fa1c03f659338b14068c16585edfdd
d446a2721aa399676b7ac22b9d53defa06f17958
F20101108_AAABGE knowles_m_Page_067.tif
7ca9df0e887babfe15d02fa5c51f4706
4d70fca72ce1ae97cb6c3e07f54f72fae406ae5f
110688 F20101108_AAAAZW knowles_m_Page_097.jpg
58e3c5c0c1a0da8fab507f8d7a3566e0
fb795c0564407f936684a2cc20dd78478e2dc5b8
F20101108_AAABGF knowles_m_Page_068.tif
40afdaefb5408e820122953ce5fd59dd
076c0bb1e1fa6a13817c9581639679884e3b1419
F20101108_AAABFQ knowles_m_Page_052.tif
18f5847ae52d397174d67fe29e904a48
ea0ebbcff2cbd26c57a6613e8bf89b351b28dd88
112349 F20101108_AAAAZX knowles_m_Page_098.jpg
0300f9d7435541a556a49e3e729cca93
142856666f9d4eed7d04e2b343d95d872b334fce
F20101108_AAABGG knowles_m_Page_069.tif
08f0e0859a9371c128623423828f4424
b4661676812bfc4ff20a4207e17273ef57ad4e23
F20101108_AAABFR knowles_m_Page_053.tif
74fc0276a0b0de4845cc199172e44553
a7130c73fc6d5216d1d89343db6ecdcbe52c4e59
32866 F20101108_AAAAZY knowles_m_Page_099.jpg
83836ba3580ce2348ac13915b0eda03c
2ba7a43dfd7044495500d68d63f74afbfe6e810e
F20101108_AAABGH knowles_m_Page_070.tif
70542bc15d0d54994c0a542a895f051a
7336322b7ea7b12091b6600cf64bcfb295a05015
F20101108_AAABFS knowles_m_Page_054.tif
eb5cc4dc3b5048b290406cf3a491d3be
280faeee6c158d423cee9837fca3e19db80aee33
102118 F20101108_AAAAZZ knowles_m_Page_100.jpg
1393f04cdf3355b9e0271016e1c2aae1
4678335a73522aa1e4db4419fe10d36b7485f45b
F20101108_AAABGI knowles_m_Page_071.tif
8ed607339749af6a053d0b2bb2afe009
f177e59f7b2df37392bee5e597d048896956fc02
F20101108_AAABFT knowles_m_Page_055.tif
0dc2d1bef5714ffbe0f25e040f801ab0
81213498ea4c474e919134002b4a9e89d5a111c9
F20101108_AAABGJ knowles_m_Page_072.tif
0481b9de628c2629ab16d4ecc74da481
f2cc5433b74f8c3e391fa7f811105cc818f37c76
F20101108_AAABFU knowles_m_Page_056.tif
d860434a03bd6599657a60c63d76123f
3f5ab3f7e9b12c0c6183891ea0b23ed8fec23206
F20101108_AAABGK knowles_m_Page_073.tif
620ff5899e289b5492a8e5f6d8554383
3165a51962fdfa69bc2264ebcedc8c3da990adbb
F20101108_AAABFV knowles_m_Page_057.tif
6549ffefbf41dd1dd88a4182cde28ca4
9499a42a8432442e65331e65d16c1209c1df86b5
F20101108_AAABGL knowles_m_Page_074.tif
29a38ea1b9df852fa7909c4f34c3b200
e7e796d0add102e2ede409b22759e89fdc40572c
F20101108_AAABFW knowles_m_Page_058.tif
21b3d1a85e8e49ff7861ab9c3c687eac
35c70815e08669c0f887ccabae2710dec097b86f
F20101108_AAABHA knowles_m_Page_089.tif
83ba340b0ee2cb59822731a2f9bd6867
2352429814ba749efa1c3b1bc2cd63fe55d6a37b
F20101108_AAABGM knowles_m_Page_075.tif
602c31d94a2c1b17b40cf1e5c660d04f
fc7e021dd909c6cf393971cd7a9373d89a02b926
F20101108_AAABFX knowles_m_Page_059.tif
5ae6e1be4a226c68765bbf9cc181e0e5
dcdf2b587222c51ff103c79aebd8bfea10bcad2d
F20101108_AAABHB knowles_m_Page_090.tif
23e820001cf971895b38968871fd01ee
93263d89dcf9dcf141a7de87f2aedf4fd896956c
F20101108_AAABGN knowles_m_Page_076.tif
85b5e53d8d267509cea799defe37f053
347a7815a2bd8c451a1ddff65bdca2d192bcedd9
F20101108_AAABFY knowles_m_Page_060.tif
32ec4b74412131fab5cd7f11e9da3876
7dd14274662cbc519d4d4d22c3af902dee201d1b
F20101108_AAABHC knowles_m_Page_091.tif
d3da09509e0f519d9fb3b7c790cae858
fc8dfbd422a7a1bb8cbfa3894192806b696e8542
F20101108_AAABGO knowles_m_Page_077.tif
c16bea2a652c1d0ec6e568877b58243c
8d8352bbefbbfc210576e7aa112cb5e91a615736
F20101108_AAABFZ knowles_m_Page_062.tif
91d64003cd7d2be14e63e76e26927c98
487ffefc61b750828619fd9ae6e1041d21e12f86
F20101108_AAABHD knowles_m_Page_093.tif
f8145bf05ec8c8ee453e266d340a135e
8f5391080367191a0d92b6b680e541557dcc5519
F20101108_AAABGP knowles_m_Page_078.tif
db9826f7533f320cd705cf585489f9a2
f2d10eb2094a889d884a65c88bff5bee85836fa0
F20101108_AAABHE knowles_m_Page_094.tif
9a06d7fc93bc690c18c7a7d95bb61558
2f2333704c0999971ed39b7fd0d45a97acc09a0c
F20101108_AAABGQ knowles_m_Page_079.tif
4316720bc7a1f2adc6d34b788a492395
ee8f762b34333a6dd347b8757fe266ac6d2d6025
F20101108_AAABHF knowles_m_Page_095.tif
c0ad2f9407b7daf84bd57e45640d41ad
8841c57cde5d21e8e935713b9f241e8bc4c00e6f
F20101108_AAABHG knowles_m_Page_096.tif
750debfca08f888ed5726c68270eb5f6
7787682012df7ecd347eee8040f662ed43038497
F20101108_AAABGR knowles_m_Page_080.tif
cdce573c0344667b641b883e63b88830
791e2d1e3b8acbd3d185d3c6235983492fe71a6f
F20101108_AAABHH knowles_m_Page_098.tif
7aef7ee3174cc5e74d6cca517573b19e
37eb923a53449776bafc725659a1718ee72f6ea2
F20101108_AAABGS knowles_m_Page_081.tif
271995f14480c80317b2cbe53ec38ba0
583994a4f7b47bca14501b5d94ad9a53956590a8
F20101108_AAABHI knowles_m_Page_099.tif
88d811bde0ce8a0f69c79ab164e981dd
795ba1b9e58575870d25f4aa0599dd522802e266
F20101108_AAABGT knowles_m_Page_082.tif
83c4f9a7d05e40eac13f8cfab13424b3
08859acdea9297376df7af131a1b8f9f311ae869
F20101108_AAABHJ knowles_m_Page_100.tif
ca9bfe4a3c78db8a66c8121fc69bc821
699f65b2322f627e83efb5b7411dc5df5ac66e37
F20101108_AAABGU knowles_m_Page_083.tif
9fbae211ca51c98d6e3f7f4bfa8589e4
9375cb9d06c6271164f69b953764096907af34d6
F20101108_AAABHK knowles_m_Page_101.tif
280227ec51f650960514fb0d97d04a62
293603f0be8c0856e6318e1424f2a57bb6eb0621
F20101108_AAABGV knowles_m_Page_084.tif
4b5e16c3b3d9dd56475bb318089c9ae2
7ad329cb2d7f42edcbdf97fddbb86e61065fc740
F20101108_AAABHL knowles_m_Page_102.tif
f48b82fd7a834bc8ebc65f5a97669f1b
b4499dfc038182b19cbf0bea0ae5676c420a648d
F20101108_AAABGW knowles_m_Page_085.tif
44cbf55b2ea75d66dd3ccaf7461ceed0
fd977ea3cd3c66c09a13692db403de03d2ca18a3
F20101108_AAABHM knowles_m_Page_103.tif
1eaaec2ab0e7b4a1ab497840792697d4
bb8ef9b31596459f51b9195f199c91ff183d3e8c
F20101108_AAABGX knowles_m_Page_086.tif
68cb1a538a8f93397930789f1eb2e34d
a93838d0b4790b0bc586b67695953a62841fb459
53896 F20101108_AAABIA knowles_m_Page_015.pro
5da35141bd8282d0706a9152a7690f7f
23c822b2ff1d24419e6f97dd746824a7f57c965a
F20101108_AAABHN knowles_m_Page_104.tif
995712e9ee7d38bf3b71afca4052d97f
594d71d6ec4afc966d3fd37aa92ce1714752c175
F20101108_AAABGY knowles_m_Page_087.tif
b68fcfbb5f6ad17f10ca717ba46fd391
00bc1e435364414abd16571cfa13386df97de9df
46883 F20101108_AAABIB knowles_m_Page_016.pro
279ba85e7069a59324420ee0f33cc5a6
e6b451b4f1792e42447aa1c99eacd5cbc20a7d4a
F20101108_AAABHO knowles_m_Page_106.tif
ef72b32039ffdad70559a18257409606
f8b7e015cfd64aea8eb3974d99c022fc0c653429
F20101108_AAABGZ knowles_m_Page_088.tif
f2e649f3facbfd702e834ce2d022c4c7
74bb6e69343fe88ed5b57667dfc38dac6ec7748c
56497 F20101108_AAABIC knowles_m_Page_017.pro
b3344ec63d1b7e8ca4c8a15d294a378e
bc12e0d7f614cff602dda85f599d02ef16e43499
7911 F20101108_AAABHP knowles_m_Page_001.pro
6561c2673b4cc65c53f71051afa58b6b
65cd10e521a6926d4c3ef9b02a161aa6f0c92f99
52318 F20101108_AAABID knowles_m_Page_018.pro
bd227609f35e8693c900ead0f5adbe89
7f57f9d495aea1132733c361dc21733399eb45e0
909 F20101108_AAABHQ knowles_m_Page_002.pro
a0f12a112bdd7962a0e760276390cc1b
e4bf5842ffd563458659c19b2643d7659695c2be
53530 F20101108_AAABIE knowles_m_Page_019.pro
1ee16c43d53883480642a653f114520b
41fe8f510ec30db99e7d041630ea46822b78a957
2703 F20101108_AAABHR knowles_m_Page_003.pro
06c02fc00e1bbdcf4731c60d729746e3
2bd949d5d1c11efc7596537a0e2df39c82d45e34
55436 F20101108_AAABIF knowles_m_Page_020.pro
7d34ba4a366ca72ae04987910a432b7c
44acd98b280602bfd598e43b1e45fa6fcba1526b
53975 F20101108_AAABIG knowles_m_Page_022.pro
2535f38d32a1035a89fc4dfa52dd99d8
5b7b0668261c8c1edbf67bfbdac28a123e2f48e2
18811 F20101108_AAABHS knowles_m_Page_004.pro
2cfcb6413d3b4024039749d9578eed98
87257e89e41f5b31b50f0381f742f65f43fea6c6
53836 F20101108_AAABIH knowles_m_Page_023.pro
e1913e16f2aae45588f31d01d6fce690
f05c87b67631b04ce4ed5a1feaf448584399f658
71190 F20101108_AAABHT knowles_m_Page_005.pro
c8e51b15cecd80b58451965d8c7cdc2e
302a37a9178bb5ebab26111121980b9fea76871c
52223 F20101108_AAABII knowles_m_Page_024.pro
6c753251351e28eba80b7fef6637c37e
8b3619b4282194cb69ba0c3f71161301bcf6db81
40013 F20101108_AAABHU knowles_m_Page_006.pro
336d124a4e5ce849760e5e010ec2d94e
b140dd587037abd1b8260d0aa6b0fa1c3dc1d20e
31100 F20101108_AAABIJ knowles_m_Page_025.pro
deca77af096cf7a66b4d73bc68a320c6
040deee268fb22e71396b37adfbc3202974a0ba1
11767 F20101108_AAABHV knowles_m_Page_007.pro
fcf29c7d1dd20cb3fd6c06074f76267f
f0482611c1ea2f943cb7bf9d6490d8c33f144980
54379 F20101108_AAABIK knowles_m_Page_027.pro
a87429b8827d4bb72b580da648cd4876
b194ef253545294f68eff105389ed524e76d4c30
18797 F20101108_AAABHW knowles_m_Page_008.pro
4b37546e364b991feca1d3e6e01b18d3
6a5117e6299ef088ed314889f83691ee3e90bba3
56768 F20101108_AAABIL knowles_m_Page_028.pro
12fbc6d3f6f2562f9e4781d4233e7e84
29ce3faa61bde4fa989aa975ca9755307f7e4b00
53054 F20101108_AAABHX knowles_m_Page_011.pro
11e0135acfad9a059b4f95008b8c301d
2ed0cd4b3c691ea126e829fc02a80d4b41cf6a72
41404 F20101108_AAABJA knowles_m_Page_046.pro
9dd92df9351391434093e166253524ef
e2526f66920a29e23047c9b972c55c214243ca6a
53988 F20101108_AAABIM knowles_m_Page_030.pro
5f487353328e60a439884e99c64aa68d
9bbb3b77c0b5d5a96621ee35b74cf3f3b5918592
54337 F20101108_AAABHY knowles_m_Page_012.pro
b5b149cd13ceb13755c2cc976b42e19a
06cf757be361d802b832fa4826b43ad4a50e895f
55674 F20101108_AAABJB knowles_m_Page_047.pro
4d8b5f8715fd5e673c28ae7c03d5b23e
1aa7730947c7adb48d3b013e3fa11e733254c468
56436 F20101108_AAABIN knowles_m_Page_031.pro
e6be1bb86052ed72f87accb2b882e68b
ab3de11a61359957b47b8e3f988bfa81af508a91
50351 F20101108_AAABHZ knowles_m_Page_014.pro
7774496a3a04e9cab22dd618604121f2
bb2aac25dc18825232ea61bda92c386356723806
56156 F20101108_AAABJC knowles_m_Page_048.pro
3a92fd3aa094e9d90099e6319d2fbd83
5538f88866ee627f23d1e07adc444fb6a1053853
45293 F20101108_AAABIO knowles_m_Page_032.pro
f8f66b57d1425ef4b52052def55b70d3
bb91d823735ca9954e5a86d3dd88262d1da1003c
53846 F20101108_AAABJD knowles_m_Page_049.pro
4f5b288d3e3749f987136ffd08d1d22b
c0c1a58fdfed0e959b5c7bfa28621ea22c834283
54531 F20101108_AAABIP knowles_m_Page_033.pro
02be38fa080f0ad33c0957ebc69a354c
81e0377f9e65b15b08b272e567808d4a51fcc405
57047 F20101108_AAABJE knowles_m_Page_050.pro
c211f84d030826c179c7a51334537d97
253bfc6635645d4cee664e50b34e915e0c68db5c
54896 F20101108_AAABIQ knowles_m_Page_034.pro
bafb543b404e7371b6852f0db7a32014
9924ecdae9e9b83ee0fdcd55018864b414d7447d
56063 F20101108_AAABJF knowles_m_Page_051.pro
a68b3631387847a3c8d3ede0fe13f65b
8257af9c1afd133ed5ded8667208d69abfb29e4c
48732 F20101108_AAABIR knowles_m_Page_035.pro
c4b54311c3d708c342bf6f2f38c71f6f
f82facbf3a6379057ebcca7e93c006ae26ab2199
3316 F20101108_AAABJG knowles_m_Page_052.pro
bbedcf72b03678e848e589cdfb50899b
40204e7a761bf1cea3e104812ffdcf3261fa4651
49559 F20101108_AAABIS knowles_m_Page_036.pro
f2ed504bd6d4c06ed4449aece69cb872
86557c1608805f070ee8a8d7da004e9b1ad46e45
54181 F20101108_AAABJH knowles_m_Page_053.pro
032f26549edec23d096daf8c2d3d4a59
0799b4094245eadbd62b0b47cbb65efd34b2b67e
49737 F20101108_AAABJI knowles_m_Page_054.pro
2f5e827444e3ca21eae88abed08270bf
78ba6db176739d41550658896b01e6838b78bff7
29996 F20101108_AAABIT knowles_m_Page_037.pro
377b14d2895fbb77bff71837c1499c80
7414deff9d2048004c8030905ba0a120956d4f0d
54943 F20101108_AAABJJ knowles_m_Page_055.pro
672037fb9ab807b61402c79e46f03f9c
c36d7723f5398f20ceae4313be2b67169c84d306
52943 F20101108_AAABIU knowles_m_Page_038.pro
b765fede4c5816289a5d33c3c756b2a6
1bf11b9653bcb775d7df5d7d0879f1ac27b7d840
54542 F20101108_AAABJK knowles_m_Page_056.pro
ed4cb7bfe291bb7f84ef00a5587377fb
c9dad2f746f6e5fb29b6675a1e68a3f5c465b8d3
32646 F20101108_AAABIV knowles_m_Page_039.pro
78b819e00a6c2d280fb042f9dc14f872
9858243d401b20651fef45caab1240aea4966da3
56029 F20101108_AAABJL knowles_m_Page_057.pro
473c31a7305f8a2b90fc160a5ab84e44
8adf333c4ae66864440f8f8a8585eb3674fc5ad7
56065 F20101108_AAABIW knowles_m_Page_040.pro
a9745a0c46e9d5e0d91b4610d3e0936c
12ef62c9d118c72b6063182d6fe0b48ea3a12014
38832 F20101108_AAABKA knowles_m_Page_073.pro
9dbc05572419c8a0fd9cbda67800f95f
4b5990f91a495b127c6495d70c0057193e375bc7
54762 F20101108_AAABJM knowles_m_Page_058.pro
c6ea56556714de48afaeeeec853c4b6c
dc6f05479d34ed34a0f87e995f61b40ce1571f0c
56361 F20101108_AAABIX knowles_m_Page_041.pro
84aa995d5c097081f042996031926c93
84d86e8c9a34ab89bb123ed24ca10317357e769e
55041 F20101108_AAABKB knowles_m_Page_074.pro
0e5c2c00868462587ab70c2b0f510d5f
40cc3a8e78764b830938d09454a1bd4d20ae7c76
44545 F20101108_AAABJN knowles_m_Page_059.pro
5b2080bea0bfef20ae37f272429cb789
5b2fdb249c5744e23933c13571a0f6537ceae4c8
50071 F20101108_AAABIY knowles_m_Page_044.pro
539365f3d662ee30f1f3b6b472b7f30e
06bcc956939c282c3f8b9b57c986d17d42502914
54101 F20101108_AAABKC knowles_m_Page_075.pro
f28d571d3c4bbd87b836ec7d575863c9
29db7dd0b5d85eff56a1ab039233e9bea9066134
53215 F20101108_AAABJO knowles_m_Page_060.pro
4bf1d8c399acbab325d1fcbbce969e0f
749fdf6df0c2b098d58daed4fdeabf0d4aedb36d
53733 F20101108_AAABIZ knowles_m_Page_045.pro
eaf4a3befc75ae6d61380b9cc4f7c4ec
183f6ad813a0b5585318f00a9c49fce3bfdf44bb
31662 F20101108_AAABKD knowles_m_Page_076.pro
c2c6f35d8474ec6073fad86befb379c8
c24dcfc7d2e48c9b982ed4a38bca8ea991e5f66d
50574 F20101108_AAABJP knowles_m_Page_061.pro
9be9fa4597fd135c9a09c9889a13bb96
c382011ae0c58a027805d54cb45324df7a134567
54312 F20101108_AAABKE knowles_m_Page_078.pro
afde1e228150de9264176153557873e8
65552f0b6210fa6b7402016685c18f0f416628b8
26965 F20101108_AAABJQ knowles_m_Page_062.pro
e14c11f6b0719150f16adae942e3a033
5d1ca7ca737cba8f7d64b293c3ad4f3ac4f93e65
55933 F20101108_AAABKF knowles_m_Page_079.pro
2a0d23767bc2a8b33f5a215c38de8c52
ae5eb02379bf45323fafbc231fba0c39a3680f8a
53942 F20101108_AAABJR knowles_m_Page_063.pro
03e55db3fd1123dbf353441d78f8ffb6
a07db1960a55bbec6e1674ca3863f7e16c15d22f
54758 F20101108_AAABJS knowles_m_Page_064.pro
3a52d73082c5d0861a3bd391830a5c36
2ad73e80bb2ff25eb4f65313218a8790a4a680e1
54949 F20101108_AAABKG knowles_m_Page_080.pro
18e2d5856b06325948501e4d406e867a
845fda3890b74d79497e3eca4a915053393eb2b2
47237 F20101108_AAABJT knowles_m_Page_065.pro
3e29e6a7a072b6b79ea30379d1b0f698
14ce940e3a49c66ca31e57ff0aababf85aeacfb7
56245 F20101108_AAABKH knowles_m_Page_081.pro
e7f10ae7fadba961f930e1a18da5847b
de5171538cc81d4a9c1c66d22e4c45eb766a8ed0
53858 F20101108_AAABKI knowles_m_Page_082.pro
f605afdc9b24fe4b15c647cbc0953217
7e964ac26bfcd0484365d10ac0f664432817ee75
50713 F20101108_AAABJU knowles_m_Page_066.pro
1dc095a23dfb6d94a5cfe13e02e469e2
2486eb61301d83b20812a95faea86258cb1bbcb8
56364 F20101108_AAABKJ knowles_m_Page_083.pro
677d12d0311ead1c571fdc92daa74128
b25b46019cb1089a6b0593d40e8bf89913697870
55566 F20101108_AAABJV knowles_m_Page_068.pro
20962b653f80909d65c4eb72df6fb47d
12a27d860d716e41cbba22575544516e92e1e789
55531 F20101108_AAABKK knowles_m_Page_084.pro
a613d68cdc577fc14b231b8f1c1ebd29
d7f66a16429173319a2899cb56d9141f8b8fb114
54916 F20101108_AAABJW knowles_m_Page_069.pro
7a8fedf3e0d0999ea0dac5930e94781b
676f3142f963b17ef93cdc354821b7c60bc7ed85
58111 F20101108_AAABKL knowles_m_Page_085.pro
23b6e4925af42e1a8255251fb6fc3d1c
5f476b47deeb1b223b1ceb19d9a3d6dc64ee01ee
55159 F20101108_AAABJX knowles_m_Page_070.pro
33a91ef4702661e096497e21677c7183
0aab2cd20ffd7b2a5e4ad4decbc781ec5f90079f
28890 F20101108_AAABLA knowles_m_Page_102.pro
79bfb8989de1f4105c9df5e4e88980bd
c863038b358f4eab0ecb6fd593a9e079838d7087
53052 F20101108_AAABKM knowles_m_Page_086.pro
2214a1b453354a4480ef3a97d5c8d59a
82cff44dd89c1352c79446997f3c47b9fa37950a
41184 F20101108_AAABJY knowles_m_Page_071.pro
34c1af1cc18ba14c4ea44aa9934314e6
afe615c2f0311a0619d755d731e83eeab878ba47
20023 F20101108_AAABLB knowles_m_Page_103.pro
b95cf96dde23b1408632713fb57c0755
91b767c02ee8dba76406e50a389e00edc438bcbf
52912 F20101108_AAABKN knowles_m_Page_087.pro
b4a4a4e70bc5a699b913a6961ebb4262
3d372bcca69a7271e4a669bd3c2ed589cf5da833
54123 F20101108_AAABJZ knowles_m_Page_072.pro
ee53261a82948534c818c309b3be6458
39db9c5106cefc26696f514e938bf0114b0aa8f8
61296 F20101108_AAABLC knowles_m_Page_104.pro
31a0e7a17e1690d9c1d93cf667390b90
6cb08a8a79d6b2b96f668a78e7061f3702f5e3fa
53315 F20101108_AAABKO knowles_m_Page_088.pro
0fb255263ef3dbbaf58e20372cdc7397
fc067e806f3372eab951fa42adbd4687b270bebe
56973 F20101108_AAABLD knowles_m_Page_105.pro
a293f2a90275fd3b84daaeb228039e19
bfc00faaa6e0b080c1625bae2debce8119d4f43f
50999 F20101108_AAABKP knowles_m_Page_089.pro
7fa9b442ef38d7097e839adc2e279ae7
629fe3f17c6138353d2937842a08f3b59135e7a9
430 F20101108_AAABLE knowles_m_Page_001.txt
c375b030230ee5af149331677f78b511
34954925c405cb1b81e188bb2b9b153df408d1a4
54414 F20101108_AAABKQ knowles_m_Page_090.pro
ee43246cdd50c11d9a94ed133a5a8269
a801ed8e5c98fc648dff918eef5d737218ab6117
101 F20101108_AAABLF knowles_m_Page_002.txt
e27b806ec8ce5e46f450b65bfd91629a
794a7af90e764284e51b3ce872f24a8109e5a1ca
55124 F20101108_AAABKR knowles_m_Page_091.pro
725fc246ba331623b08c0df2cca3a85b
1e855714245441a774438f7ee0c412154dfa29ee
162 F20101108_AAABLG knowles_m_Page_003.txt
78ad9a500bec285dd720432a5b2bcb83
1b300699d06421c896c23caa6016281f6d6d29fd
53590 F20101108_AAABKS knowles_m_Page_092.pro
4ba02806d2b32764bbd35d06577e816a
2f09d61883e13594deacedf2aaf4fbebb090d29f
785 F20101108_AAABLH knowles_m_Page_004.txt
4e69a68c9840ca40124c8010bd130325
7362ab97bca2b4edf82df93549c40df77ff84158
53548 F20101108_AAABKT knowles_m_Page_093.pro
d6de9e9f6df01d8aee5825a0f26b5558
d55b61cc26ba1c3aa421271cbee5e8504d3aa895
3144 F20101108_AAABLI knowles_m_Page_005.txt
971a452f8c3554ab66f600abf2d3a30c
da31aa238a524bcf45f3ceaf02ea5bdc9a1d61b2
55175 F20101108_AAABKU knowles_m_Page_096.pro
7323245e2a71fd1e1a452c5e1699a176
7a634e8859c140b0b470bcb8181163fe9d1929e1
1718 F20101108_AAABLJ knowles_m_Page_006.txt
b1f5ca8ebc0b1ce2a64dfcc24ab541cd
45f5f7d4753b86d557e0c505e00813d2e58ae66c
469 F20101108_AAABLK knowles_m_Page_007.txt
cf3cafc4f6a85129f7c94debe8f34829
d7f98d8ee6d364056a8044887e1d92fad23f26bd
54714 F20101108_AAABKV knowles_m_Page_097.pro
9194a37a17f097bdcf9bb04d9034186f
83331ad5142e7f2ac81ff372c4997a3886e6e1d5
804 F20101108_AAABLL knowles_m_Page_008.txt
88e305b6c91b686a69979526ae97c22c
c38de80ada5e8dc8a3b350768d53a4b3210f3af4
56132 F20101108_AAABKW knowles_m_Page_098.pro
0490aca528f6a0a488252e1f89be1531
a94e195e3fcf67709037287d40e11f3b66ad7bf9
1565 F20101108_AAABMA knowles_m_Page_025.txt
13b1e4a4d9781d76e95880fa4ad4e483
ef85726c6d41c4f997867af5d0d115260f78ee8e
1929 F20101108_AAABLM knowles_m_Page_009.txt
bb6d17d7c8b23b459f34ed1cb3d20b0e
d1a4548c199176afceec784fe6a12443643e2eca
15037 F20101108_AAABKX knowles_m_Page_099.pro
5c53f705712ec1b6f66484963731d3eb
33e7dd5444435899b3e81229a131ec5be4342a58
2109 F20101108_AAABMB knowles_m_Page_026.txt
766f350d2b0282e6172c060bca734a08
f2b8f7196b10e77220edbd5a8c268a5a9af4f9c2
168 F20101108_AAABLN knowles_m_Page_010.txt
fb55d0401d346aeec7eccb87a4b9bf3a
877267e287da1048e00bc2dc8ae8e8c5c5114766
50812 F20101108_AAABKY knowles_m_Page_100.pro
640d7ef9dd435ed632e10bc5d0253028
838f6e0852f001369305e1ea60294964768d7af0
2138 F20101108_AAABMC knowles_m_Page_027.txt
70ef58e62bd56e805d62b1c45fb18912
e5f3410e1dbaedd6e807bde1add4167181857c4d
2167 F20101108_AAABLO knowles_m_Page_011.txt
829bc961681d4707ede1559b5081308d
87d799961fbfc4d0db40e12bc8a9a4c0fbea2922
52827 F20101108_AAABKZ knowles_m_Page_101.pro
a21811b5a395d7891120d3cfd13a466f
4ed293d575aa833d07b29c8467cbebc1c9d13d9a
2224 F20101108_AAABMD knowles_m_Page_028.txt
bc0fc7682435faf81da76b2b4e7dad3d
548f1174d7d8f74dd8c2f539ccbef89e4399881d
1927 F20101108_AAABLP knowles_m_Page_013.txt
e5232c969b7c56acd1684983ede4b63a
70c937ce26c9acfda694a6d6ecdae651d2715a40
2103 F20101108_AAABME knowles_m_Page_029.txt
60b89f891a86a32b53adf9858e2b3396
6a19c0391e9d9f7502a295424a760ade81f91e67
2026 F20101108_AAABLQ knowles_m_Page_014.txt
ea9f8d71b53c69a508d00e69fda6bf41
482ae20627676a890aaa496f2226ff0f0fa0287b
2118 F20101108_AAABMF knowles_m_Page_030.txt
d244d1a468bbd68e1076ddc05c145ba7
b9bc3fd7531a8e949061616855d64762932aea34
2119 F20101108_AAABLR knowles_m_Page_015.txt
faec5745643b6358185f545f2c43a693
7bcc8b1ef53ec10c965deb5c7239816a5733a75f
2214 F20101108_AAABMG knowles_m_Page_031.txt
3fb433742c6855e7a726db19f960b1b9
f4e7801ed61cda456f7bd665c2c427aa0e1b1d46
1897 F20101108_AAABLS knowles_m_Page_016.txt
f6ac42ae09011061e4669f6dc1901090
1da044160593500783e96a397be7f36d53b87806
1836 F20101108_AAABMH knowles_m_Page_032.txt
8f6b186fc654efea076f9abb4da8141d
3ce173683a574d6ee657d9226387f92c38780f7a
2211 F20101108_AAABLT knowles_m_Page_017.txt
e66fc1bd79520286a380e6264f6577f2
7d9660feae413bec309276c663ee1bfd163a6d9d
2221 F20101108_AAABMI knowles_m_Page_033.txt
aec7918a9650c673fdd5849a5142b30d
df99abd66b8859621698274cee48baf2fa4b2e09
2065 F20101108_AAABLU knowles_m_Page_018.txt
45298c1cab825d41cb01a7364fc5470b
9f8ae7051f55ee81ba505a75595c0a3310571ac4
2151 F20101108_AAABMJ knowles_m_Page_034.txt
a11aa07e50d982982f0697096bd7f68f
c7c65da88bf62515b7680a53f6110df5ccc98c39
2175 F20101108_AAABLV knowles_m_Page_020.txt
6e2f41fad0e8b488ad7041304ff1da29
acf300b854883ea796b852d09cf2624042406fec
1968 F20101108_AAABMK knowles_m_Page_035.txt
d696baf1718a7dc681bba61e09ec1da8
70a797a9e81af9934b56958a652196ed6063e82e
2089 F20101108_AAABML knowles_m_Page_038.txt
5878b92d2668a93b406d3c2740165d76
709a31ee243c36a256e28f61a426f32f6c92b56d
2237 F20101108_AAABLW knowles_m_Page_021.txt
f46feb4b05345cb49aff6e5de92aec75
fd2df83128531b9b227fb0a2c20db1b60468eef2
1333 F20101108_AAABMM knowles_m_Page_039.txt
90331b0f95d657ca119dc6737bf17e72
3a7a25cc8683f313319a8e3879e82cd848009d5b
2120 F20101108_AAABLX knowles_m_Page_022.txt
a552080b5956004cf729212a4db7a85c
7afde50dfdcf276f649c30cf041b4ddcb79c4a00
2147 F20101108_AAABNA knowles_m_Page_056.txt
c143bdecf8cf3d2ed570e09aecf32af0
f90fb24df450bc54993bbb6759e07d2502f60f9d
2212 F20101108_AAABMN knowles_m_Page_040.txt
89992d7fd71aa30c44b06ab98246f62b
f0f846272cd318da7f8b7de4fdb58af72cf836f4
2129 F20101108_AAABLY knowles_m_Page_023.txt
0f58e0f0bf618d9496fd0d9747e14f13
667c7f3c12c4c9c3d9ef353385884958cbe134f7
2210 F20101108_AAABNB knowles_m_Page_057.txt
01957222443c02ca68c0469522111f59
b1823b7de73b70f92268987ba1189049205bba2b
2202 F20101108_AAABMO knowles_m_Page_042.txt
3fcef08903e42970e0efd9136214be4b
b6c9fac3cdd09b0096705c85472834b8fd0a7d50
2058 F20101108_AAABLZ knowles_m_Page_024.txt
8818b9e62204bd50dd0e65fe40a415ab
16c074b2a484401209650601b4eef5f65000201d
2370 F20101108_AAABNC knowles_m_Page_058.txt
db5170a76409d42958acaefdc620bf18
a6853d9e7a6d8bba50eaa027bd15cc641953eaca
2036 F20101108_AAABMP knowles_m_Page_043.txt
ea66ea2948a59c387e08d623ac3b98e2
45536dc6c77a6b2f085e36d0d488f13391626e04
2104 F20101108_AAABND knowles_m_Page_060.txt
87de388332cb064a006355efbc333198
e75cc54dbc03a28c28e0a18b469f1acce68edc20
2196 F20101108_AAABMQ knowles_m_Page_044.txt
f424dbbf5140a5a81818c96ab80b7306
f0ba158334bdd23f8eb55389846bd7b09d2c1dad
F20101108_AAABNE knowles_m_Page_061.txt
8d941634d89a78cb355a39259042c297
24b5efd81a2704d9d235636a0ff46677693da334
1863 F20101108_AAABMR knowles_m_Page_046.txt
c3a081fa26918a3fa4975b4f57388137
3334b5574faf956cf61d3edb1b1fd8dc1ad0cf5a
1076 F20101108_AAABNF knowles_m_Page_062.txt
e95bc3b068d7e1202489678d1f494a75
b9fbd21dfa6d3a9f5710c2c26f84438c894ee61f
2184 F20101108_AAABMS knowles_m_Page_047.txt
90b11083bd2661ca3a6bb89936434c1d
dd81154b9522329c7843d36daf35f8b71af6abf5
2203 F20101108_AAABNG knowles_m_Page_063.txt
f687a62b242aa4224da24f3a17bb1341
95cf38c9bf0d885fc37264b77237d6c34b930db5
2206 F20101108_AAABMT knowles_m_Page_048.txt
f2db105a03fded7e556427cfd2c74600
15e7889a9fad5530c1acbcc9ba62d534f8494046
2154 F20101108_AAABNH knowles_m_Page_064.txt
72491be6ddcae9c2175172d693f5d106
02ca44003c73ad9a98f660567dde74d627f19489
2371 F20101108_AAABMU knowles_m_Page_049.txt
1d2f1ad8fc79bb049a9822e89b675cf5
e1068f951388b02415b17017da6e87ef3dc6ef02
1909 F20101108_AAABNI knowles_m_Page_065.txt
a29c51ad672f96cba01714cf91caffbd
990074eec6ab7380e8bc972deb205ad1659b6bf2
2241 F20101108_AAABMV knowles_m_Page_050.txt
fd21a6b04f9cab3ecd5f77358603cd74
6d3ba09dd2d150e75e2d40c6e08d9517f834e916
2004 F20101108_AAABNJ knowles_m_Page_066.txt
408ba4054f3e1dfe9bc6256ea3a33a98
81ad67293d983c3521c1fcd2797f78322168dbcb
F20101108_AAABMW knowles_m_Page_051.txt
18a11ebc855fc9f86bfbbde772e43c79
bff72eb967f69affafa75edd347c7b966e1cfb23
2225 F20101108_AAABNK knowles_m_Page_067.txt
357638bbaaa03e6616a5cc679caeaeba
fdf6ead5b770f6de0a2e3d7b51ef9bc65f5380e5
2177 F20101108_AAABNL knowles_m_Page_068.txt
952c58628c867fe25e3324de96298337
a423c338be4cc96d514cc1eeb30bd2cf6f25fc7a
F20101108_AAABMX knowles_m_Page_053.txt
0b3d32b3ddf21dd21a8c1aaf9122a34b
2b720910ab15e4c9b9003d68923a22db81459786
F20101108_AAABOA knowles_m_Page_083.txt
940aaa23042b745cffdde44f7a520f1f
163163fa24b25f8eaf459e5b6b897d986009359b
2160 F20101108_AAABNM knowles_m_Page_069.txt
7e4570c61826e4eecc160521d3a90f2f
1cff1e3ad6230f2879fd6f36d5d8c9bacbab4838
1972 F20101108_AAABMY knowles_m_Page_054.txt
28f41a17114565b6f257db38b9d7fff6
d5e2b12bc713ab7fbfff7d8594863c1badb26615
2178 F20101108_AAABOB knowles_m_Page_084.txt
c6af88f79a3b33ad24ace1f34ce5f642
79ee018c3eadc6925b9623e26aece2b5d25d83a1
2166 F20101108_AAABNN knowles_m_Page_070.txt
aa212c912bcbfa7ba98b35fa398469b8
6389717bdd5857bf58129610a24d243d6f9a922a
2156 F20101108_AAABMZ knowles_m_Page_055.txt
eed317521eb10f773603f099190e8789
698a2bcda9abaa74b71c785c9288913090649d4f
2293 F20101108_AAABOC knowles_m_Page_085.txt
823b8037deb8487642c0dbd48982a733
118db36d7a4b1ee0a3395838969332d185fefde4
1879 F20101108_AAABNO knowles_m_Page_071.txt
8ab3febc34c6c3783d8683a96631e4a2
aaf8769076bd5f79f7a7cba1db0ef4a7cd76a899
2086 F20101108_AAABOD knowles_m_Page_086.txt
c94d531e175434edf4024b7223426dea
9bb87fca20606642f025a717d10084ced0c3b56f
F20101108_AAABNP knowles_m_Page_072.txt
99dadb2a8e9757e47fcefbc506eee655
c37d75bd056c5424ec7dc67688adf9b8972edfc1
2080 F20101108_AAABOE knowles_m_Page_087.txt
15afe9c77c57af071ab43847be6ea038
0ffa355bb6f07024976a96bb2f718cbdecffa469
1734 F20101108_AAABNQ knowles_m_Page_073.txt
b513fb84ca8baf6026e4d7997c1b69a7
5112dd0b9d2e4a0d400d4acf22b9d09d57df90a5
2106 F20101108_AAABOF knowles_m_Page_088.txt
e6a700dbd47684f4f5421b0880487d22
0d95b849cdae77fe6884c199334998fd880125f2
2161 F20101108_AAABNR knowles_m_Page_074.txt
f04f9e409f3e87e34db1ef7e9d3601be
dd20825fdd8aae9aa00eccaa0b17ca0fd810ddc2
2012 F20101108_AAABOG knowles_m_Page_089.txt
d8d8d31485661e37b9ea99c81447d967
fa9e99c94012b5d4c58af36d2c4ecf1956d302c2
2127 F20101108_AAABNS knowles_m_Page_075.txt
cf27b898946eb46ce937ce679b0f7485
d7474aea14b274e0775613892632a56a902824f4
2137 F20101108_AAABOH knowles_m_Page_090.txt
19b01111d931df59777e5b3689eb81a2
2c01f080a05712a44a743a97d3b0d07f6354f015
1259 F20101108_AAABNT knowles_m_Page_076.txt
98c691d77d582b94626d1c7056150c97
fd46ef24189f6808b84b146ae684963026b8c192
2194 F20101108_AAABOI knowles_m_Page_091.txt
31d2a1ff43a2c74d4f7f12a49633bfe4
1e4c868636d6de92e16a273d0a6d9f20f801dd79
2108 F20101108_AAABNU knowles_m_Page_077.txt
15e62c80d7aedd8b69ddc2c4253d2a1c
cccc75320b4526e733f575e22a431bf649dadba8
F20101108_AAABOJ knowles_m_Page_092.txt
436e141e2ec5063ce77d4e26a437a744
51fef14fe4f96f2a777e94acb15051536aa0be7e
2135 F20101108_AAABNV knowles_m_Page_078.txt
1278dae569120fafea9b065915bb0b7b
67d2204969e2ffedb15b00b52f39d76bccd01688
2105 F20101108_AAABOK knowles_m_Page_093.txt
d567094b8b9b5ff90885e52fbdb72abe
d741585513943f67d560f749b2c633a52343e358
2195 F20101108_AAABNW knowles_m_Page_079.txt
41f3a2f14c96a840a1403ceaa9229202
c583079781816cff7a4885ac221f54cff9800e0a
F20101108_AAABOL knowles_m_Page_094.txt
85170b7bbf83bd3e4834fdd96fff6ab2
203ef4ae9898827327b5fc60cfffe07260f3896f
2158 F20101108_AAABNX knowles_m_Page_080.txt
986adbc71dc917ff8b36e1ff87fa2989
d7aab7060723470f73b1a9d8cd13de5c0027ba49
7979 F20101108_AAABPA knowles_m_Page_044thm.jpg
847806b050582093f635e8fd5d30b721
c157a629acabbab5dc7a241f652588867502e17b
2063 F20101108_AAABOM knowles_m_Page_095.txt
e7231d20e194ad47e199bd285eed2f89
f91a0d553c3007f0c623e75e249582c8932ffb0e
8212 F20101108_AAABPB knowles_m_Page_036thm.jpg
e97bbba25fa41ae4c950fc8780b0047f
a75b56e7dd6bb6854c3e11b4a05bdf0c7d950c73
2165 F20101108_AAABON knowles_m_Page_096.txt
c9ab0b57c2fca5d1b9917efc0c13e4f1
44ffb5dcdf05276e37da547d59047558ae53c147
2207 F20101108_AAABNY knowles_m_Page_081.txt
70c01eccd82048d1dcc28911e8dd3bf4
ede3f6c95c8465efb0f2d055c21e4e14767987d4
7862 F20101108_AAABPC knowles_m_Page_016thm.jpg
88aac542598e66ff8b6d1fff6bd82a4a
7e0db0352c6b13f361d23cbee7bc3d7e69fea97b
2172 F20101108_AAABOO knowles_m_Page_097.txt
eb0542fdc444dc0ac77630eed671b668
4d7f79a6df9e2c38621f83869c0d2386067785a8
2114 F20101108_AAABNZ knowles_m_Page_082.txt
0cbd8d5ed39afac76a5959bfd82609b4
3639bc38670b3b11d22e189464893bf2c410e48c
F20101108_AAABOP knowles_m_Page_098.txt
d664e8d1d7cfe5727d1216c7f725a283
2a271b6cc9ef13a832bdbeb8c7ef800368d3b460
4470 F20101108_AAABPD knowles_m_Page_062thm.jpg
e692fa88db2f1234abd170efab9bf5d7
fbdbfaf6d2525bf7c420c714bfe506703e291efb
599 F20101108_AAABOQ knowles_m_Page_099.txt
0483ffc54458884e74a03be74131dc7d
e2c809ad13cf5e7879e7c3896c77c6eb8828add1
9057 F20101108_AAABPE knowles_m_Page_104thm.jpg
0de90ab56eb514c55ab998727aa6cac8
b8c0a11f18dbf016a467d9f7248c124856ab8861
F20101108_AAABOR knowles_m_Page_100.txt
6aa336b28cb53a13e030e7619f9d9e96
3f8e4e8c2f51b10587266566b8a42774c56bc068
26858 F20101108_AAABPF knowles_m_Page_025.QC.jpg
f6d5b1797dfb453763357577526dd840
385075775255a333d9526676267d2f3b95d5c465
2082 F20101108_AAABOS knowles_m_Page_101.txt
f6cbcc0e4906fec386d0bac0f02b64f9
f496e89caf55feba5228e8f931dbfc045b6f0774
9080 F20101108_AAABPG knowles_m_Page_041thm.jpg
15745370cd9dc401ebd35fca2eb588fb
6091f2de515dc818676e14492ade5e7af1ecc916
1150 F20101108_AAABOT knowles_m_Page_102.txt
ccf5ce65e81463925cb46977a32b62d3
310ede9e2aa389742bca0eb08e73f4979a81373f
7284 F20101108_AAABPH knowles_m_Page_007.QC.jpg
8443d46b083938b9c936608951e05b6b
391321167a75b864c1e2ddfd5e609efba516ce55
855 F20101108_AAABOU knowles_m_Page_103.txt
ba9b50435e0f01a52d35927315413059
bdaf5854d45411041fc042651ba37880d959a5b8
36725 F20101108_AAABPI knowles_m_Page_068.QC.jpg
de4ad1ddf06cc5290ce8d241c873b5a6
0a480bae7c03afb06d2e5d7d07fd179d5da57637
2497 F20101108_AAABOV knowles_m_Page_104.txt
ece38c53c42455ce9c9d87b1cd740ad3
5bf225004941454bdcb38c8d189f9905e517b241
8970 F20101108_AAABPJ knowles_m_Page_090thm.jpg
19e1cdcbcc817b464a4b95b6069fa920
18b80d2789c7d6b06269df5f972d56e111fd87d9
2330 F20101108_AAABOW knowles_m_Page_105.txt
2826338d888b6dec4630abe8df5a28d0
5b1e3e83cd90ea13d7829763ce0314289db92cc6
33155 F20101108_AAABPK knowles_m_Page_014.QC.jpg
babc447f70fae7465ad19c5baf34fe5a
4d0eed7be3b3f37815a240345bd60ecd0a22b373
1163 F20101108_AAABOX knowles_m_Page_106.txt
4364654151cd5051ad308e1b6d64e151
1ff1b83301ea64a9e0037d530360f25dbc541d89
9199 F20101108_AAABPL knowles_m_Page_050thm.jpg
07f7accdb763db1fb6bf9eca88fde51b
de639d801a6aaa4045a447c2c9f54e9a496a3865
F20101108_AAABOY knowles_m_Page_001thm.jpg
8d2587bbb10340a747895096fb1df12c
ced9be9a922da4c8f228174123ac6658b173482f
4937 F20101108_AAABQA knowles_m_Page_106thm.jpg
a34bea9b41bb8e83cd71bdcc6c0d3b91
07974b1fb9d17881f12fdf441b521533121319d3
7567 F20101108_AAABPM knowles_m_Page_039thm.jpg
7590f7b150b6b4ce20cf9330e6c462d9
bca25a7c3eba3be11f4996c6be1503f580269f7e
33019 F20101108_AAABQB knowles_m_Page_095.QC.jpg
a0934fe9ab219c855ea4fa9d90cbdbee
efd4c32bc6e59ab5d5cea5a79d100186fa122d68
8978 F20101108_AAABPN knowles_m_Page_020thm.jpg
901e9a286e4a470e0dcf152a402049ea
31d308a515e2f0f17831c0eb1369649ea0b31f43
491258 F20101108_AAABOZ knowles_m.pdf
dd7b7e6be5397a22d29a507f68fbf8e9
dc394747fce5d1f90f6f201de9361d7eab9ec943
8495 F20101108_AAABQC knowles_m_Page_054thm.jpg
8fb24b4703651788ce0f2e3e443e6433
1d3f2564ed5bc6d795f6a245bba5561d96f7dda8
31481 F20101108_AAABPO knowles_m_Page_071.QC.jpg
77cb1f2df833dc1e682cbc08a1af1284
fbdfdcc2a07e0436dd74338dab97b86a3182fd4d
8793 F20101108_AAABQD knowles_m_Page_029thm.jpg
5a9a4ad1ad12027e54524285286a8687
2df87d5cecd1946ffa3264be1fd2ccf8d32fc7ab
29745 F20101108_AAABPP knowles_m_Page_073.QC.jpg
b64c96590311374f5bbdd4f4bc470c36
b9d3b700afdf2f622955e513b23f44dc6719d2a6
11385 F20101108_AAABQE knowles_m_Page_099.QC.jpg
ab849fb46d4235784f10d3a64815469f
6dd54207603ab074c47711ecb7cb79f2e659ed70
8121 F20101108_AAABPQ knowles_m_Page_089thm.jpg
c797f98eb18aa5a49eb88176c840c10e
049a6c0914eadae45624214730d2ddac9a76a514
35707 F20101108_AAABQF knowles_m_Page_033.QC.jpg
74647a90a4abd146d496210ef90fc7ef
d38596157a40e03c2bf5e5c46847e9551e09399e
8172 F20101108_AAABPR knowles_m_Page_059thm.jpg
a37028938d0a0371e1a501a57b0ef5e5
62fbaed37c52e454fc83cdbabdd3ef3308580066
8695 F20101108_AAABQG knowles_m_Page_063thm.jpg
7e00a8e0dbcc7de2a81598b2d7df77e4
2c7f7201922e7d11ebe850f52f2a7c159dae0acb
35679 F20101108_AAABPS knowles_m_Page_063.QC.jpg
b9e64a0d37ef89eadbe5cde5a80b1994
ba5ce834609be717682ea8c2cdc104407fd34304
9104 F20101108_AAABQH knowles_m_Page_098thm.jpg
d8a0d86147f5cbec655a982c48c5588d
3a71084d52beabac4d2f6053bcb718c72d23bb1a
35294 F20101108_AAABPT knowles_m_Page_024.QC.jpg
2d9c357204b10dce4f127b4f4c00b4db
557b8e7a49b2ec9aef9b07b73cf04e4a7bbe22e1
8783 F20101108_AAABQI knowles_m_Page_092thm.jpg
7b6c69b750113e22be9423d0912232f7
cc5cca6f0e35c1f2c612df2181b055dde50e950b
33460 F20101108_AAABPU knowles_m_Page_100.QC.jpg
30fecb6ed82b9185c4fe54443f298388
380bdd2bbcabc6c90a44cd5a6c41049dc9ab3ac1
35820 F20101108_AAABQJ knowles_m_Page_074.QC.jpg
be2c311db8aa9cf23c79f1ad284bfb35
17899b9895b1f8d3c4f436bac410400f4187e197
7987 F20101108_AAABPV knowles_m_Page_013thm.jpg
75538fbd610748060aaebe9b89e2b6f7
648ccb94f90336f35dfa9c2b735f1710608a387c
9209 F20101108_AAABQK knowles_m_Page_048thm.jpg
49a35f009e4870933a8594bd05d69484
e62af824ec3f089f35dcb622551a52545174d9ab
8639 F20101108_AAABPW knowles_m_Page_101thm.jpg
6c3989d369e24ead1493834a2afadaac
42ec7283ff7d9500ebaed97ee1c220c881b3dbc9
851 F20101108_AAABQL knowles_m_Page_003thm.jpg
938bd08fb7861af24dfdd7363d3d3b6f
e76b032665f07fe107e90ea56e7f9efd0891dd51
38158 F20101108_AAABPX knowles_m_Page_067.QC.jpg
2715949aae4055468c5ac549b5b0ff40
98c64f45ae121a9c1623bb2815675a5fed957dda
8742 F20101108_AAABRA knowles_m_Page_055thm.jpg
5260e8800298e7af9925da68b66727d0
52b1b25f39276b0c45c4e6cd2f07e99a58faba7a
F20101108_AAABQM knowles_m_Page_007thm.jpg
d759b0fdee9b259ba68ac6552767c2c2
e8f41677d2451a4dea7ea0901762ef6340f499a8
35479 F20101108_AAABPY knowles_m_Page_082.QC.jpg
6ea4847e749e29d1d8fb530ad033f22a
7386d5aad3e673c26d2e8be4078818c2f7266c0e
8886 F20101108_AAABRB knowles_m_Page_072thm.jpg
bbee0ed2406869d87cdfa9f1e013c538
d21ea31c9fe65fefebcca5739685e313fd2a5c26
18646 F20101108_AAABQN knowles_m_Page_062.QC.jpg
59da600d834fff857979b7fb2eff6db6
70da62ecd8f2586c7f8038744c02292d0a30bcbf
4246 F20101108_AAABPZ knowles_m_Page_010.QC.jpg
c6539452e52c0e7ce12775e15a2ca984
7ae95a6b649498201b04b84a1dcd4d94fec5bf49
8699 F20101108_AAABRC knowles_m_Page_095thm.jpg
7615fcab639290d3794125c0df9b598a
83f41d73eded517063934514760db7298cdee0be
8768 F20101108_AAABQO knowles_m_Page_040thm.jpg
a05e75f740097d275a58e820b5c0b951
81b2f12714b7a3aee3c077f948e994b2cd24d5e4
8857 F20101108_AAABRD knowles_m_Page_064thm.jpg
2504d6f2533e485177ef4c76dbda4eda
7f0a84249db3c8522aa809dcdf7f9065c1164456
11180 F20101108_AAABQP knowles_m_Page_008.QC.jpg
77cdbecd8d8099a15d9283027e834fb1
5848e347c59c41061e3fdd128f69c1625b925343
1195 F20101108_AAABRE knowles_m_Page_002.QC.jpg
83226c9f395dbab15aa3325f6929cb3f
6ab6fc5790980610476ea5cb0fd2f8f5249aa440
35017 F20101108_AAABQQ knowles_m_Page_093.QC.jpg
8e7ec404e7a6f90f07b3540750a5f09c
085ea789bfa7d0e9b8de93c472809a672057d646
36224 F20101108_AAABRF knowles_m_Page_020.QC.jpg
818297408109807fc6bd4b13858e467c
2736b444366e00eeaf7e5ea3e61fabc94868d2f7
7920 F20101108_AAABQR knowles_m_Page_046thm.jpg
725cca0178b6b0745c78e846927d6c96
2e36650f196956bb538d6badf459ee1f73ef762a
8746 F20101108_AAABRG knowles_m_Page_015thm.jpg
23c89bbc380da6d10d546d716e790149
b24c9c0e8e45eed086d0756dc665abf15861f17e
8550 F20101108_AAABQS knowles_m_Page_026thm.jpg
b57b2e34905c51db4ba918a08aa632f7
246f7f19e5d0bde2be92a9fe9c2fcddd6609dfd0
3491 F20101108_AAABRH knowles_m_Page_004thm.jpg
b3109e2973cfbf54addb7f4a44ddc0c6
e14a7065bc53bd40009fa370ee2891796fc90129
32131 F20101108_AAABQT knowles_m_Page_035.QC.jpg
2452b0921315b83e78d6b4c95bfe0443
abbb28f34a5602bc9c81c58824b41a075a5eb15c
36486 F20101108_AAABRI knowles_m_Page_056.QC.jpg
0edcb78d9f306361bf3e12a285919fcf
8e960e06f88176279cb27dece70fb153d24a591f
8435 F20101108_AAABQU knowles_m_Page_061thm.jpg
535c2b4a24b1469b9573697d00e933ee
e165bfcbd12e29eb4a271eb5da4f0a9195752f0e
28164 F20101108_AAABRJ knowles_m_Page_039.QC.jpg
5e78085066535cf73ca574c286ff6aef
3c56478e30a3c3cd2c928b1c2d7262c2f4f663e4
9124 F20101108_AAABQV knowles_m_Page_094thm.jpg
d9f55ce2079493e0e4c164afa0a0afed
d823e4dbb4c830c973775212c3a2779d6855452f
4053 F20101108_AAABRK knowles_m_Page_006thm.jpg
107a7988ded6181f7f03d5f7acb5e1af
5150c838d42a40f1e906853e8929af0229c941cc
8803 F20101108_AAABQW knowles_m_Page_096thm.jpg
fb2932d23c9b6da19ad71b3ca4d7cada
9f5d356ca22d64c5747d953533e8788128cebd6c
8785 F20101108_AAABRL knowles_m_Page_081thm.jpg
c9480b97e006f89a6b733339d00f7d9d
3ce8fb2401592c61c43bfe7a6ca38e2b7415634c
37084 F20101108_AAABQX knowles_m_Page_083.QC.jpg
a6344bcde98110bee1e878ed3a443154
b24c7a9f19b77f3f42ed257779fc6262797de14d
36007 F20101108_AAABRM knowles_m_Page_078.QC.jpg
477367710c5eba858f374d646d7db17f
2541e19f31239279724aa58e3710671adfe061f6
8553 F20101108_AAABQY knowles_m_Page_087thm.jpg
ead51f9150c3f7bbc2004ce86cab409e
3cc805d78e701c16d6a7263e26aace8b86a8fd03
35723 F20101108_AAABSA knowles_m_Page_012.QC.jpg
3346d4f7478b8383dd359a63fd020c9b
fd7e5cef4faea7c60fad11297403fb485d98d90f
7053 F20101108_AAABRN knowles_m_Page_037thm.jpg
be07ccd1685505d320a7fd047587d9c5
c3c8fa84b83388149d5d99a57d583f591657e7ab
8954 F20101108_AAABQZ knowles_m_Page_051thm.jpg
4a74f854f8f8719c7e170d13d28c1cfe
2e80f1b0ad9cc828b9d39ad183c56e49018ab31d
34840 F20101108_AAABSB knowles_m_Page_029.QC.jpg
88ff0d64ad0d2ded043a17126095e258
d5a44c855025b7581baf46bdb18454fbd7df23b5
8573 F20101108_AAABRO knowles_m_Page_060thm.jpg
4cbdeb88ea101166526d4316095606f5
315930c6eff11ee1122b4dda25e7c164fa81c21a
4754 F20101108_AAABSC knowles_m_Page_102thm.jpg
3d5ef2757c6eebec5f1792b87b9817c7
87481b0ae2a64632f51504808bbceaabd21a4991
33926 F20101108_AAABRP knowles_m_Page_101.QC.jpg
8240059f71cff7656a09da2c4aa91911
a3895371058a100d0d9a1d5c4a05baea0ff3de66
33485 F20101108_AAABSD knowles_m_Page_061.QC.jpg
533a8f3c34b149edf68b14875c0a053e
4fc802b85a3d7950507654e02b817457f66e1edc
8731 F20101108_AAABRQ knowles_m_Page_067thm.jpg
8a74704e1ec2859255d9373db107ef8e
c3dc6f1f58a8ea301ed6ff21f8c060225abaddf6
8972 F20101108_AAABSE knowles_m_Page_047thm.jpg
d0864f53fff3d3cab6379d08c6a234db
e9abb394c7e1e06aaf46228ec7bf99cc6097d756
36543 F20101108_AAABRR knowles_m_Page_041.QC.jpg
811adc0ffcb2d6adab74b65d3307245b
1db0d9a86cc307266c0dd4f2b24ce43835c1079a
20321 F20101108_AAABSF knowles_m_Page_102.QC.jpg
54902088bd25ae1adb313a08fc745bc8
835eaabff668d966e64ceda3d310ebeea262b72f
31449 F20101108_AAABRS knowles_m_Page_016.QC.jpg
dcd36b44592549babd4251d2292dd491
2178e4f973d27f22d5e73ab2603682e088fa2bc7
5557 F20101108_AAABSG knowles_m_Page_005thm.jpg
93608cc92821c147e89aebd7800905fa
ef3e114e2f65b78639a64d0670ff6eacf902e342
9007 F20101108_AAABRT knowles_m_Page_068thm.jpg
3b0cf3eb8252a8495d86b872591329fe
fddf141c6efec1df493a72913dd92bbf46a0c195
33914 F20101108_AAABSH knowles_m_Page_089.QC.jpg
8989b2e40b0a0799d7480b0c65040e03
3c636df952408aa2c8c56539305c42c4c506a40c
8150 F20101108_AAABRU knowles_m_Page_071thm.jpg
02098add80602983cd7b9bfdc97b24cd
db8f1ba295e26233ff1067445fc8b8ed3e936c8e
9265 F20101108_AAABSI knowles_m_Page_074thm.jpg
2585002d15c460c76ec41bee9f260d74
b633ee247ed9a4b1e1e21d8cd3263ff70e6c453c
35119 F20101108_AAABRV knowles_m_Page_092.QC.jpg
1ccc72e785a48fa84adf310d3251a020
509d9f3a90b0ac3d7b1ae45d683ef15d38d09e61
8349 F20101108_AAABSJ knowles_m_Page_024thm.jpg
9b3fa591d20b9d3865926424215c0d26
18710695c167bc3f8411c4a8c949d8f5a33a40c8
35081 F20101108_AAABRW knowles_m_Page_026.QC.jpg
ea94ce33fb2137c84e65d0eda76634f6
165b2f8b1410ec0619e5960a90e441278e598119
36611 F20101108_AAABSK knowles_m_Page_031.QC.jpg
9fead727dabec048fc7e5ab57a8fe258
d5f2303c5f4b5803730e3f1bf019f03078b670bd
37852 F20101108_AAABRX knowles_m_Page_017.QC.jpg
c952a3ddef3df87687a2589c4a071298
ea1e2e9142b5698722d6f62d23ee68599876ffd8
8862 F20101108_AAABSL knowles_m_Page_019thm.jpg
e3f59314f8e066d2d9debaf42452a5b8
9c1f092d6c0f077cc82ee43d601f3d52277ce5be
8582 F20101108_AAABRY knowles_m_Page_058thm.jpg
4231fd13182f963f92836d266f93ba30
202491f9b3cec96e243744ccdfb5852ce99e717f
37195 F20101108_AAABTA knowles_m_Page_040.QC.jpg
b326bde3488ce6d336d88147cb41b68a
6515ff3da07a13144060ec9f7af7348fa1160cd9
2367 F20101108_AAABSM knowles_m_Page_003.QC.jpg
9a6c95d446368f6a402f18b9dcf60ded
f5112b6856853a7dd7a062474178b2c7bec6bf25
36368 F20101108_AAABRZ knowles_m_Page_079.QC.jpg
70c9ee675406a76d5d561d402317576f
b6bf3fb10fb5dff3f7f17917975a351c81d5fdb6
8585 F20101108_AAABTB knowles_m_Page_082thm.jpg
05e5063a121726501623cd90c3039005
db5c25096813d9e83a3cba3db6a569d977c8e2f8
13228 F20101108_AAABSN knowles_m_Page_004.QC.jpg
88200d2ce0d823169624201e9f99ff62
89c883b9071c0bad042db238464e0ee64db86caa
35045 F20101108_AAABTC knowles_m_Page_053.QC.jpg
f07e383461f36e5c92c19e7db26019fb
65cecc04b89e753b136c883e814dd085e84da67e
30504 F20101108_AAABSO knowles_m_Page_032.QC.jpg
4f564781e8a756d086b84b6741ba756c
9146177d9264b7de34aeaa20a5ea68fa9127a164
31348 F20101108_AAABTD knowles_m_Page_059.QC.jpg
adaf5028e1176c819491200e2b8ce4f8
19ab1991d6e2a4e022dc330234f96eef9b7e671e
36179 F20101108_AAABSP knowles_m_Page_084.QC.jpg
bc3d97bda5c301fe01c4239787a360b8
ee467a1f16c244090767fdb50a4dc3239993935a
15714 F20101108_AAABTE knowles_m_Page_006.QC.jpg
53cfe867544305b08c4eb943d8d385a8
df25e5877a66c0c6fc86107f5187c85f5a0fce22
30464 F20101108_AAABSQ knowles_m_Page_046.QC.jpg
02c067293daf911969675ddff6e92ae5
1cddbb7e079a21bfc8b0f0a36690dfe14ecb32a4
8036 F20101108_AAABTF knowles_m_Page_065thm.jpg
62ebdf199ffa3f28ba64c638b0abef5f
1de5c9327a7cf579779a2d2a56a16f76526bbf68
36268 F20101108_AAABSR knowles_m_Page_070.QC.jpg
e8039160320270721fcc1e03b153fca0
cd3e3b2c329950355f54b4aa756e1d5d0ef0e69d
8891 F20101108_AAABTG knowles_m_Page_028thm.jpg
59c7378fc2014dfa1489e06bc6c6d435
57d5ffa2ddae2f0e17b49f67aa09acf6b1a87885
F20101108_AAABSS knowles_m_Page_033thm.jpg
689958306f9160fd9602b04ce5e32112
e497758bcdc65a5b4d0441beade87f627b4809da
34549 F20101108_AAABTH knowles_m_Page_011.QC.jpg
8c06ae63503d8d184f4cbab4d1be8545
c6753f8f23f7603159f95e38e0a13aa9bbefc056
8551 F20101108_AAABST knowles_m_Page_077thm.jpg
1622f1571b04a91863d8380a8d494fe5
0e352406608e0daecda4b90c3c2f2087eb94d168
19929 F20101108_AAABTI knowles_m_Page_106.QC.jpg
e56f8a6a0485df180543e7cd2bda4001
eb4f48dcb62ccd9ecb39ad3edf13a6b524ffd224
8806 F20101108_AAABSU knowles_m_Page_027thm.jpg
0624f036878e50ab9ee1f45c0ad51e7d
08405aa5f9f7b2ea357c4a275ea0b12dfe1bbf15
8654 F20101108_AAABTJ knowles_m_Page_023thm.jpg
07e7bf80173dbdc7e6e74b583367d38d
edcfe496693dff872598710a5fd32527625b5694
35668 F20101108_AAABSV knowles_m_Page_022.QC.jpg
869ef81570b61cdf630384c499ebe4da
54c53503d8611da27b6f0f91e1f281e8c408b190
33297 F20101108_AAABTK knowles_m_Page_066.QC.jpg
0846cd193afee829ff09b569522d4b5a
3fa8cca2a4b6e8690ad8c6513d9908319254af61
8782 F20101108_AAABSW knowles_m_Page_017thm.jpg
e60de74f85bb3129aa3281a242c619ec
487ff45d554e24c4b3061668964fb2bef211e3c9
9139 F20101108_AAABTL knowles_m_Page_021thm.jpg
07ffaae0941d22f44eaf356cff898188
00eb7c0c0a647b4acedee9260a8ff8244fb08585
8576 F20101108_AAABSX knowles_m_Page_053thm.jpg
b9ac82d23521fb12e6a53c986209ffdd
c93d2e63d2a66515439b35fcf26cf8151006deb4
32662 F20101108_AAABTM knowles_m_Page_054.QC.jpg
b09e930f1251d508271aeb5efb8d8e89
da3906b1c4213ff0e2851938e3990189bda9a0ec
36352 F20101108_AAABSY knowles_m_Page_064.QC.jpg
ee819126ce054bd4939f34117c4943fd
dd1632a54c2e05e49f478e4106d163903218c535
5289 F20101108_AAABUA knowles_m_Page_076thm.jpg
0c1859d4224f4abcc42485edc6e99175
483b485e9d921d5ca11aaac56d669d1f9456aa66
8915 F20101108_AAABTN knowles_m_Page_069thm.jpg
44a4969f6054aa1d8ed73d84b46fd458
8d5b384492ca9802478247bb7c29e5206eb5a69e
8741 F20101108_AAABSZ knowles_m_Page_097thm.jpg
2d43c0f17708d82ea9c94576aedaf98d
120857268687928489a25fa88bb8dc5134e48580
36461 F20101108_AAABUB knowles_m_Page_048.QC.jpg
872ceed329f1e02f48503dd4f84bee6c
8b4ca1d1f7ddfe37229503e9fa744284a1122ba1
35708 F20101108_AAABTO knowles_m_Page_023.QC.jpg
543ebb1b5523f5ecf45322120d4e2a29
3f40c46b05b5004fd9f7bc38b012a801051d9012
33184 F20101108_AAABUC knowles_m_Page_043.QC.jpg
2d588e29064d8ae051344a7360610847
80bed78d5196e1f425a21c575423319f45226475
32326 F20101108_AAABTP knowles_m_Page_036.QC.jpg
4f81ce3c09af8847e6a4351de4641a8e
93111c6071c47f2d109310e10f07b317ab817e31
34892 F20101108_AAABUD knowles_m_Page_104.QC.jpg
b5931c7b16d2ea60c102cff33f36ffc1
3253592bee2190cb014c25591b15753a04a24307
1125 F20101108_AAABTQ knowles_m_Page_010thm.jpg
ff49d2aa3848baddf473fa90ec86361d
08182b77fa8a762c155f1628df1e585b22302e0b
35429 F20101108_AAABUE knowles_m_Page_060.QC.jpg
fc11cc7b1a8554f0c079a1db51427bbb
b14f174298673e26c1846ec71d3608011e76182f
36185 F20101108_AAABTR knowles_m_Page_090.QC.jpg
9145be2cf5a2e30b905ead905bdd3a93
f4e8023bd9541e71840f76a8d3ce4079e977f9b2
36261 F20101108_AAABUF knowles_m_Page_015.QC.jpg
0b9c76d033e49b9981d2bb3c35d87678
31ab949c0877d396abd3d5d413f430c8f7c548e4
35522 F20101108_AAABTS knowles_m_Page_055.QC.jpg
7d0188f6d929344d9cccfccffa0b8d2c
9f0eb5d3b11c6033a5207f12e7e784513df02154
8008 F20101108_AAABUG knowles_m_Page_001.QC.jpg
14f3bd7642e21628fa7fda90dd3fa682
387416fbfbddddb01f2b186a0eefd32b6096008e
38326 F20101108_AAABTT knowles_m_Page_028.QC.jpg
e20fdbf82b0d8600a623f4f014bc3f04
8285b51d04b2fd237cc2f6194d3f5d1799e9ea00
8185 F20101108_AAABUH knowles_m_Page_066thm.jpg
c3057d87e4b9ec1b9d1bbc77563c60ea
71946c9eac01eed220288db635840f1beca788b1
31529 F20101108_AAABTU knowles_m_Page_044.QC.jpg
47e4c1fcad3f327c65b1166b8d9a2286
10dc8b59f754ccff952d4f5a35315eb9bb4d65ae
36140 F20101108_AAABUI knowles_m_Page_047.QC.jpg
d63505d6469f816544f07afe2300b98e
1add8c9c1826a25c5f8d9da883d12c94cea234ff
7304 F20101108_AAABTV knowles_m_Page_025thm.jpg
61dd1387e2d07bab8b66367c84218b30
ac35e418fa37efdea725dd698bc95396fa4f96f8
9098 F20101108_AAABUJ knowles_m_Page_031thm.jpg
d10147796f3c3790a358545a40d02a3a
10fab7a7ba708ed19d96e1ca7176a23dba675d88
8336 F20101108_AAABTW knowles_m_Page_105thm.jpg
9f957a476a3aa520690a64c9a5625853
ba808262a72e7b4551ff43bdcd771e5ccd85d3bd
8703 F20101108_AAABUK knowles_m_Page_086thm.jpg
2295463fb8e122d6211812da804f416a
6452b2ef8ebd49312b2c6056f1c2a8c15b01d1b1
32968 F20101108_AAABTX knowles_m_Page_013.QC.jpg
14533a570ac49252890fdcc5af63b547
128843e850ed5230eb2486b15bd399ece947356d
35008 F20101108_AAABUL knowles_m_Page_058.QC.jpg
f5eefd4efb85ccff6436b262be72352c
cf0924b5c1e17732e4a8973c280e4fc50de87c58
554 F20101108_AAABTY knowles_m_Page_002thm.jpg
5518f8ee9679dc0d872380d639677414
669cf44a3781ec16cfdcd2babce7caadf546db0a
9157 F20101108_AAABVA knowles_m_Page_078thm.jpg
b443174b34cde36ed93fcd37f0615e1d
9a9e11e9f819dfa33064a0ab19fec0ab7e04377d
33931 F20101108_AAABUM knowles_m_Page_088.QC.jpg
1b1490d115e309da7f8be7523d94ce69
17484d2b5319d9b66a8503e2306f25588d28d53b
35193 F20101108_AAABTZ knowles_m_Page_038.QC.jpg
0f383cb6c9d3ce7c56cc7116e0217640
6f96a59988b3609bfea825eeadc902b784952514
7811 F20101108_AAABVB knowles_m_Page_073thm.jpg
9c6ca4eefcf3c7561793ecdb18217426
859d8ceaf95f6d7c9dcb3f6bab1973d80c283870
8998 F20101108_AAABUN knowles_m_Page_045thm.jpg
294b4af1644f9f9d70c7a87b9e78ee75
d284ded94bab2ece196d7e53b2f0cf097432d396
9140 F20101108_AAABVC knowles_m_Page_057thm.jpg
dddb8c76e5d5e592088ab776d98041bd
255e1024a06add97860305db37db3761795f0827
7831 F20101108_AAABUO knowles_m_Page_035thm.jpg
b6dca5ceb7d753692c6e900118d9309e
81690ab5847fc53343e75160558b0813be2129af
22145 F20101108_AAABVD knowles_m_Page_005.QC.jpg
c9af0e79d10fbfa6f7feca2b24a4c954
8fe73fd6d9b67421ae23e7e61190c022062f2dde
8382 F20101108_AAABUP knowles_m_Page_100thm.jpg
3cbfe4914a79e2fcd302e2dee2680421
8f38e6a9967643d8afc88910c45c5657dd7790d1
36690 F20101108_AAABVE knowles_m_Page_097.QC.jpg
e28333dbb32c585e82d93f179c372b0f
c2e515fee75596539bd287d68dbc85ab0345b27e
36142 F20101108_AAABUQ knowles_m_Page_072.QC.jpg
7b06c1fc74c493150b6fba13f297b88e
b9c95ff98a44455281eb4f8d253c57b27ca54356
36762 F20101108_AAABVF knowles_m_Page_098.QC.jpg
e7a39732d0c5a67a957d866d7f12781c
3b1b5b1927b266dca74b1281d4b5e8afbbb1a307
3516 F20101108_AAABUR knowles_m_Page_052.QC.jpg
547fb892c820aa3448190b882e44d6de
2f918a5cb4af799ffba7c472df3c060a8427ec5b
8707 F20101108_AAABVG knowles_m_Page_049thm.jpg
93e2bcfcb899ed74c262abded02bd1f7
f1373f76fc701f6d3b81acded03a45517c909a0c
36738 F20101108_AAABUS knowles_m_Page_094.QC.jpg
67edd9b5252cb03aab2e8d8fbd817601
f5eaa56c34407f59f5ce94913b9268e4e98dace0
8521 F20101108_AAABVH knowles_m_Page_038thm.jpg
73fbcc5821dc5e5b97ec31e06999e501
52132d8af4378b565aacf71d0206018cea542c72
36402 F20101108_AAABUT knowles_m_Page_051.QC.jpg
d69d4d6d9f2cce0a2e21d2423bbc5f64
6ac6147f5346696a1dfcb0195b2a9cb87dde1d01
8914 F20101108_AAABVI knowles_m_Page_083thm.jpg
be5e6f3a88cbfc18c093846fa15199e4
16752d0a642be2d77ba36eb56d7f70411040d605
F20101108_AAABUU knowles_m_Page_030.QC.jpg
e01b888e3afe9a59e6cfbdf470dcc679
a4e44923dd3aea95cd7d96b2b1efcb021cf7673c
8902 F20101108_AAABVJ knowles_m_Page_084thm.jpg
ce424fc064a56594f842657a2085e33c
212a3680b006a15ab59cf498caad9af1751c7812
34494 F20101108_AAABUV knowles_m_Page_087.QC.jpg
ccf3e7bee24908ee389de93b6340d7e1
6c66c02813a1265206b898086c2229c7fa575a3f
8539 F20101108_AAABVK knowles_m_Page_043thm.jpg
4069c7426a15847de9f8c103fbee6891
0c1542b77ca2650583037fb4c8d1ced741a7b64e
9005 F20101108_AAABUW knowles_m_Page_079thm.jpg
a0dcead262e6fdab83297df632ade7b5
68d9b4d42ea8b335444af82ab4c31bb1f02ae272
1002 F20101108_AAABWA knowles_m_Page_052thm.jpg
39ee3e98030710c77cbcc657704acfbc
c815695f6cd63743b992842a4f013dd1c0a86380
36269 F20101108_AAABVL knowles_m_Page_091.QC.jpg
c1c16b1ae5e3daa0868c04a6554fa1b2
c49737161f6fda10842c951a98bfdba5ebc9307e
34170 F20101108_AAABUX knowles_m_Page_077.QC.jpg
1d3a12389de09702dc833ab7b91c85df
d6314be3e0f6b70063595bafdca890878a4ed2fa
35331 F20101108_AAABVM knowles_m_Page_019.QC.jpg
84c9e85a0e2b3b36c9949fe88f2d8dd7
1577ae06c72dfdc802c5771c703521d35295b6b9
34445 F20101108_AAABUY knowles_m_Page_049.QC.jpg
518279aff4ba05454fdd6cf50f3329b9
486f59156f2531e4194d1a5c96937fe6b29362df
F20101108_AAABWB knowles_m_Page_069.QC.jpg
dba68ae59a8224a3a04a3e8e22968b9e
5fff4094f022f505b52c48d522e8046fc4b8a91b
36227 F20101108_AAABVN knowles_m_Page_096.QC.jpg
08c63d59cc72dd94f6347543c42343e9
2f9ccd7f86eb51584d020da813b75be0e15c1165
37211 F20101108_AAABUZ knowles_m_Page_042.QC.jpg
2457a4af86186df4a773ef67284a6f9a
9b409c26853629a44fb007eff7bb88a89b975619
36077 F20101108_AAABWC knowles_m_Page_075.QC.jpg
5f7fefc759211e83d42298e465d228c2
1ad8cf52cc5a45278194a2fee2da6f2f4c2303e9
160467 F20101108_AAABVO UFE0021617_00001.xml FULL
5b1faaa53416b3b4254d1817a5ec6edd
e3836c16da1967b79dbb36ffdaadc3a8af98de3d
8747 F20101108_AAABWD knowles_m_Page_075thm.jpg
194922914c56c8d115e0b0afac266b0a
7eb992936be2ab00577350e573d63171693fba10
2914 F20101108_AAABVP knowles_m_Page_008thm.jpg
0cc75b50199da34524cf5c7cca51410e
49ab9047a3eef5133706ae79b9acf7e3ef67986a
21519 F20101108_AAABWE knowles_m_Page_076.QC.jpg
b433a7b277dd1cd47a96682e1f0264aa
130602b74c16777425c485423abf6d37125a0e70
8640 F20101108_AAABVQ knowles_m_Page_012thm.jpg
57543f5ebe89b0e4bd801c69574023ae
448952018b2714e956d2087d510fb856a2abae97
38736 F20101108_AAABWF knowles_m_Page_085.QC.jpg
1007080857799e8f19e68f40db25e3df
d822ff959df4c73eda866c77d1c45bfa37df9756
33854 F20101108_AAABVR knowles_m_Page_018.QC.jpg
f6226d6b72107880862a64c51302ef54
f9824d4e3758047efa47d4651747efd6ea440075
9111 F20101108_AAABWG knowles_m_Page_085thm.jpg
cc0a64020fe89bdca61f97ebafcf6550
43fd29ad11051f7bf7e446a50ab2d7d99428ab47
8472 F20101108_AAABVS knowles_m_Page_018thm.jpg
71834f25a85131c1e0959d152c4adc64
3fa342afa461f120684878c10cc4218d0e86eea6
34645 F20101108_AAABWH knowles_m_Page_086.QC.jpg
8ac4b3f641121e19840083bf0907b24c
fcaf707e0a4aeee934269e6cb06159331351a8ff
36456 F20101108_AAABVT knowles_m_Page_027.QC.jpg
86212136cc939121aea5f6a3488cb5e5
0d6bba0dd619bbbd494f9989e6ebe358a00ab8a9
8648 F20101108_AAABWI knowles_m_Page_088thm.jpg
7e72290ff8dc30b89d81f10483ee868e
45ea25cb9ca25a1bb34941a3bfd46e6aaee51c73
8749 F20101108_AAABVU knowles_m_Page_030thm.jpg
76de4cc7b7aa026d7c80947ea25c91f1
0ef847c3617a0e4c7d47a09d01f51eb621b297c7
8771 F20101108_AAABWJ knowles_m_Page_091thm.jpg
b7b3328e37110ad40f4a6f50ea34063d
38450b7e6ab048b6a5591d6a3cea598a7ff6e23d
7480 F20101108_AAABVV knowles_m_Page_032thm.jpg
09dc6fd7bac6b16dc4814fc1f598c8d5
5e91ebeac77bdb91afc10ddea6818b34a22b321a
8516 F20101108_AAABWK knowles_m_Page_093thm.jpg
652cebe8be5ce2ca38adc0bea5d2c460
95911087d6855663096e2766d465faa241cbf4d5
36382 F20101108_AAABVW knowles_m_Page_034.QC.jpg
f5f681ee633f3d3c93d4f28a7a75c3b2
f90ccee9a883126feb830dcbb9c68cf6db49f595
2782 F20101108_AAABWL knowles_m_Page_099thm.jpg
53359aa92b5cca5ec3f104dd54219787
238dc8bef839fe38a3be64538068c96d23ff0dc8
8801 F20101108_AAABVX knowles_m_Page_034thm.jpg
848af576d55ecec3cdd6f4047fb05212
22a6e177c685d07dfb0963c2e6f97dcb182c6fd6
3626 F20101108_AAABWM knowles_m_Page_103thm.jpg
b1b95f726404decee6c24478c9dbab7c
592695ccb8c18515a38cc6fca1170bd466873b12
8872 F20101108_AAABVY knowles_m_Page_042thm.jpg
f6d19238f0b1364b402161548d3b1353
d2541df5467ee34112778a2a866cef03062024b1
37237 F20101108_AAABVZ knowles_m_Page_050.QC.jpg
244b2919446449263e444d54783676c7
deb1bcac6cd061bc0d419771fc883839ad64f59a
85603 F20101108_AAAAUA knowles_m_Page_025.jpg
19bd30ef4b0dd89ded670a21a597ab5b
a5b15f299dd825ded5b3417057daac6cc1f97f52
51507 F20101108_AAAAUB knowles_m_Page_095.pro
95b18f9be586f1bf0ba3f54160fff851
2479684d9a31d05f9a2ded9ad0098c132da6e7d2
F20101108_AAAAUC knowles_m_Page_004.tif
7e4a716477aa4a82d23d5ffdb4b50a96
c56ea42be0cb2db1c09288295e0bebe0f058b768
9101 F20101108_AAAAUD knowles_m_Page_070thm.jpg
edaa85506eeb6d99172e5d1219e876e4
d21cb1c1ab56f3c78f5f708af216405430eabedf
108194 F20101108_AAAAUE knowles_m_Page_092.jpg
036a08713335c08ab7ebb902bfa8db76
960f532115c8488131a87eae6cb2066347de5562
106532 F20101108_AAABAA knowles_m_Page_101.jpg
34f13c0ac8703ca6d2e10a5791256550
99fd4bc59e6566617ed7747f8d084d48b5ab315e
F20101108_AAAAUF knowles_m_Page_019.txt
a5a75bb558b41b865e57d6ecc8b0164a
5d933c107d95575580e4f2446dd6ed9870009145
61065 F20101108_AAABAB knowles_m_Page_102.jpg
62e00c7c263b78b7b8f9e26fcaf8b3c9
ba6b0af0c9d7d3893e44bc313105034e77d995ed
F20101108_AAAAUG knowles_m_Page_092.tif
804cfb6f370934e04e357a8d0888f51e
63c619538cf7a17253c06c42d217206c23acdde7
44366 F20101108_AAABAC knowles_m_Page_103.jpg
ae747de2b2d70aa2b7a3616c887026ab
246f5874e3edbff058ca4742b8724f3c5c4c4b02
F20101108_AAAAUH knowles_m_Page_022.tif
07a26e4fd432f71fd4d360b70626ec6b
f04db70be3ad956195c238f4f099910f2f6e2992
116320 F20101108_AAABAD knowles_m_Page_104.jpg
60f7ba435abcf4ffc3c03523a6c75e7f
b88c2886dce2baa35d506317b36408c6a2d83add
56311 F20101108_AAAAUI knowles_m_Page_094.pro
fb830ffb705ac2d0744bc30169dcb431
d4e956f8060578851ffe9bb4573a39bbb6f6ed85
56038 F20101108_AAAATU knowles_m_Page_042.pro
740ee511bdd230f478a5eb53a998d6bc
534f6c9124989fbe94d42e6f7d0b76b0bcfed2ec
114014 F20101108_AAABAE knowles_m_Page_105.jpg
03e1dbf86e3380203e7351f4aacd9974
c6a5f9f25608bbe2977f877295792727f24abd94
31770 F20101108_AAAAUJ knowles_m_Page_065.QC.jpg
093498fd2a96ffbb957ded148ecfc0a2
6f1205d424213f0d05001496403f8cf975717dfc
28389 F20101108_AAAATV knowles_m_Page_106.pro
95350deb32497b367c99fb4577f7be29
48f8abcfaa55c9b14da33f34185a08d8a544f665
62226 F20101108_AAABAF knowles_m_Page_106.jpg
0a0d8ce1de69f601f90cb2b501f1b54d
e7fb55fa59872e7ec4f351c56364f40a98788826
1004704 F20101108_AAAAUK knowles_m_Page_009.jp2
9e27f9a3ed566a2ce67b0a4f7cdba3ad
f03a5fab783f4e0d89c5774d295ba92973b5af4a
44476 F20101108_AAAATW knowles_m_Page_009.pro
6028e47fc1b6c0a962f2660416a95f55
8ac5152c15a916f8c06aa530b66202da692aeeef
238078 F20101108_AAABAG knowles_m_Page_001.jp2
dcb7b05f385ce8c43d8adbf300b9e4d4
b49db0aa9113c541b315d48eda5df1669fbbbf00
1051969 F20101108_AAAAVA knowles_m_Page_019.jp2
822f87c6bd430bf8dfaeb5db5fe64f7b
3eff34169e32bd778941dc5e9504986c372e7213
108723 F20101108_AAAAUL knowles_m_Page_053.jpg
7f627c1c41aeb7d089e2131da0b556a7
93c82bc258e08d1113578af32e8783dc21646747
51870 F20101108_AAAATX knowles_m_Page_077.pro
7320797637205956171685d00f37f718
de301e1786f0cdccf6a8caa093b8d963699ea284
27285 F20101108_AAABAH knowles_m_Page_002.jp2
68f2ddc3b9dd97cbe987ef034a8347b4
57b8e1d7aa4af63595e13bb27ac0c7a447398afa
33713 F20101108_AAAAUM knowles_m_Page_105.QC.jpg
130fe37ab6f465bfe322cb9ebf3290bd
de8778a26291502e42627219d9764452e17b297d
1051944 F20101108_AAAATY knowles_m_Page_043.jp2
2e78a2be38cfb6f815adb56c2686a9e2
0dd5630cd1bc0afc8a750870e3bb1601e124d115
65817 F20101108_AAABAI knowles_m_Page_003.jp2
35fdec4d98b1270e5aaeb79c737a5ba5
7e408c01a5311e6d24b194d16a5e876a04733c28
651308 F20101108_AAAAVB knowles_m_Page_102.jp2
7a2c38c1b0472d272a4884215fab7982
ddb5ce2e6bcb482ece8aec6abdc25b7153487c10
91938 F20101108_AAAAUN knowles_m_Page_046.jpg
fd0f67972f355a560f42bdc7ce95d771
34d7689c4788c55a82805cdb637e1f0b4797ac63
106475 F20101108_AAAATZ knowles_m_Page_024.jpg
f41c1c77e7bb5a101c0bde6f80464dca
769b9e02add9bc1a265a02166410963ed154541e
426948 F20101108_AAABAJ knowles_m_Page_004.jp2
cf282255b6263d86843399ae741ce9d4
f6a16e1d3875cbd7f21b0e8fa9ec08fd045130ff
37311 F20101108_AAAAVC knowles_m_Page_021.QC.jpg
026d69583ad9bfaf129b0bd59b4864db
6d4a0fc413831e75acda010bf6b9cd5f2d3db2cd
25406 F20101108_AAAAUO knowles_m_Page_037.QC.jpg
e9a7d6fa22c933eabdbd063afd1f1165
e442c0b591c7a6a4bfeeb1644c41d79eb9c238db
1051971 F20101108_AAABAK knowles_m_Page_005.jp2
d8d547f379cc746b97c8ac922b638def
54afaca5cf447350f48efe7e44b171e680ec4e76
53611 F20101108_AAAAVD knowles_m_Page_026.pro
57927379a41f04a038ce6cb575c0e90f
574e489a47a095c0397b5be76695b38b646ef1f9
8951 F20101108_AAAAUP knowles_m_Page_022thm.jpg
865b6399ea519949362e98c27cbfeb6b
a86b6ec147c158ece17dced174a48c8bca6c52d9
F20101108_AAAAVE knowles_m_Page_097.tif
fb1055b15645d6b893f63ae522fd983d
e4df94ff4ed2a45333d33d8f67b81d73e2453b60
56953 F20101108_AAAAUQ knowles_m_Page_021.pro
3378ff314d29d6d6e85c1eb04567c1a6
4a9965dcc2bd22cf13e79cd3b23005d36b336f0c
1186 F20101108_AAAAVF knowles_m_Page_037.txt
a79f9fc00726b888b126ab7e9b4cbfe0
47d37b52e326e23bfd333b8805527f939ded93b6
1051978 F20101108_AAAAUR knowles_m_Page_077.jp2
19e0496018330f797b8489d77915ddb9
c28ef0e68219e5042d378afa9a8f9eb725d3f893
984468 F20101108_AAABBA knowles_m_Page_025.jp2
30d1189a07541748be03a01c1a22418f
742efe2e04fb0ad1e9664656da9577f111dc3d95
F20101108_AAABAL knowles_m_Page_006.jp2
0d370b52c0447b3ea29b4685584d3089
a895b635c0c00bcd55b3f85e125d64c3f583c402
4095 F20101108_AAAAVG knowles_m_Page_010.pro
8ae2e6c0d3a801c4e03f43867fbca4e4
38a73202748b3e4e9944f428d2c83eeaaa219a49
109606 F20101108_AAAAUS knowles_m_Page_096.jpg
6241e7eb31d313036f09db27471716c5
3109908085b384ca4b3de544823df8b62da76055
1051981 F20101108_AAABBB knowles_m_Page_026.jp2
8193685342907d5313adbd2e1d97ef5d
74dcc2ed4527a82c7b957dab6ed02f7bea08f328
391104 F20101108_AAABAM knowles_m_Page_007.jp2
5fcd1f88164bbfa3a339597ac3b5be3e
6ae2abf6644190c4da0c8af652f1727c19242f8e
1051947 F20101108_AAAAVH knowles_m_Page_064.jp2
c838a734b7d042defec07b9c4f9fc832
84ecb8ce652fb9df0f6dfc94b471a6c3f3ae1d0a
8589 F20101108_AAAAUT knowles_m_Page_056thm.jpg
ea5d9091a891d2ab7dfe29b170440eb2
5d77b2b443a828143baf086e5ec6c10dd421193e
F20101108_AAABBC knowles_m_Page_028.jp2
2206ee5aaa9c0322e545e634a34f230a
da4a4e8e151b69e6c28740b0a474a8a8ca875368
651818 F20101108_AAABAN knowles_m_Page_008.jp2
dcbd79c6b8fdf4bd1d6a67ad52c66f42
1fbc2a21e020fb573f840c4e000865a7a1a573cf
1051952 F20101108_AAAAVI knowles_m_Page_012.jp2
8d2a1daa06dee2c381f53de1eb6b1c89
c89522934125beea452a41904d8ba4c3e4508b06
1051977 F20101108_AAAAUU knowles_m_Page_014.jp2
186b9aa0d127417636fab2df3fc08433
0348036f9372d253abca7e0ca1583e51bbf7c8b1
1051980 F20101108_AAABBD knowles_m_Page_029.jp2
3c5bc38ea9ed7299ec3aad9bbc6c8a69
6eb0f9974e621625a68387c92586cc78a557de7b
99544 F20101108_AAABAO knowles_m_Page_010.jp2
d0c716a74c7a92a7d95f599bcd9391d4
a76439da511bb231b76592ec59fa2509dc913dd2
F20101108_AAAAVJ knowles_m_Page_045.txt
cb75d772da07c3d7a4c239a2b282517d
1c073550b1f211266f7224a3de4dfb67ba0422ac
1051986 F20101108_AAAAUV knowles_m_Page_027.jp2
2512893fd7c864559a87a8cbb06934f9
72adf7d12164e157a3bcc189afd3b2c6c46c71e4
1051974 F20101108_AAABBE knowles_m_Page_030.jp2
a6497d10d49cf094a876ce08c4b3a4fc
15351e2e36f0b1b9fc8048c03c0f143f7cf3b87b
1051967 F20101108_AAABAP knowles_m_Page_011.jp2
4d58d0c7eb1f3661ca965a6015f6dc13
7aa72defba4b700fe6465d66e7669fe77a442815
F20101108_AAAAVK knowles_m_Page_080thm.jpg
f9d0c2dfc7516f1900b0348f82bd8e8a
853bc3e7423cde2d272c72ee65f43af4a50d0f40
82124 F20101108_AAAAUW knowles_m_Page_039.jpg
0a74ca926fcc3d6a07c5f7e25298ca01
92425c795c3f3e418d02bdb805b181f6af914d5d
1051943 F20101108_AAABBF knowles_m_Page_031.jp2
82355192c801aba29b4ec48011d6daed
39b2d33e6d5d36b2fa6318b1848da2183952dc48
1051982 F20101108_AAABAQ knowles_m_Page_013.jp2
147db0bd3d8672f0b61f4e291471a76c
6ab7619778d9fe91b862227e4576b8d6d6b90a9b
53409 F20101108_AAAAVL knowles_m_Page_029.pro
37c76f362be1833ffdf9424dce74414a
400c405c7d561af216291b54abc8a4d7219facfb
35802 F20101108_AAAAUX knowles_m_Page_080.QC.jpg
2e6411843a7cb58c009eace695a65295
e2c85ef727e9ef558d3ac93aad46e7897c465ae4
1025759 F20101108_AAABBG knowles_m_Page_032.jp2
07caa8e36d6e085e68a08f6b21fdbf2e
2208d9552d83c1d666ece73bfefd81067da332e6
36913 F20101108_AAAAWA knowles_m_Page_057.QC.jpg
8f51fc25eec9dda0727eca1d1ba15fd5
055d22d6ccdff1a26acdda17975f61ab9190a3c0
F20101108_AAABAR knowles_m_Page_015.jp2
9dddcbe3d0cc15f79e0d338a6722d5d4
f901fd69a7d8c3ba00302ef0369cfb9e6d30f3de
4040 F20101108_AAAAVM knowles_m_Page_002.jpg
133191b9081f21a84090ac8c7b93249e
ec2baea200888197ccfb19a20f35390976b63423
2155 F20101108_AAAAUY knowles_m_Page_012.txt
6503e68f239fff687355401305fbdbd9
be44c832d9b702d77d7f431882612379df89ea37
F20101108_AAABBH knowles_m_Page_033.jp2
cf91a95531d2ee0bc2ffb39ac91b2fe7
130c2687edbfa2708a82a5b6432e55eade7b719b
F20101108_AAAAWB knowles_m_Page_046.tif
c2ad2889c07b72c871ed7642487188bc
92171992e17e1d7ea8ca8661d6b79068b60dc1f1
1045848 F20101108_AAABAS knowles_m_Page_016.jp2
27a80525702667d3faf33c3e09eb3974
0214ba3bd35271cb30b878f59610d11ba56a689c
8223 F20101108_AAAAVN knowles_m_Page_014thm.jpg
bcee4b46ca44bf30b7f191ef34106857
88f2491b90b96ad955ce1419ea51cf1d95ce3cbb
35628 F20101108_AAAAUZ knowles_m_Page_045.QC.jpg
fe85b2a638d120facc637db126e8ecf4
70df0a50178478f60ed6269c649fdfb969fd3f50
F20101108_AAABBI knowles_m_Page_034.jp2
7157e44fe0f5db45ef76b9a8c04c6ac2
b9284cb8c161116302df67b8e23bb8f266f61326
1051976 F20101108_AAABAT knowles_m_Page_017.jp2
459a71ce655597639451b5ea7c30a7c4
28b97d1d13a55e9ddc7a01943854d65772f6e1e9
56728 F20101108_AAAAVO knowles_m_Page_067.pro
75217871b03a8a3ec1c7e18bbf142461
6b618f8e34fc0b8527f138211b6276956276fa1a
1051946 F20101108_AAABBJ knowles_m_Page_035.jp2
e833b45686ceb8a14ea3fe161680b250
daf065a5fc2f29b4f448634af54c467219ff795d
1051940 F20101108_AAAAWC knowles_m_Page_042.jp2
c355e6bbbb9b67de0dae26b8fc2612f1
510bc21322608faf7eae1f8d63307c02b3d3774b
1051959 F20101108_AAABAU knowles_m_Page_018.jp2
517ff95a68de8a0d968ff07c13f25606
eb4f706478bbedbe4b6b51c746ef2c8020089367
F20101108_AAAAVP knowles_m_Page_105.tif
244e0da9bd6bb41ed0621dba7663ac7d
330a2913ae314441ce42d141dccf260285ee9ef3
1051970 F20101108_AAABBK knowles_m_Page_036.jp2
83950196a85b99e8fa845ae412b5ccc3
a6f5dcdf9c30a29493c6fe6e4d11ffb7042be51b
F20101108_AAAAWD knowles_m_Page_061.tif
a822c08c1ffde07f46d9f5e10b0d59a8
abb6640b8a585ee011649f421f1085ffb16059df
1051966 F20101108_AAABAV knowles_m_Page_020.jp2
5b93beca91a8b72e3b49358dc746af57
0e4a49a0729fb55eee8ddee7de59b5d8d8c316b2
F20101108_AAAAVQ knowles_m_Page_041.txt
34c9971ecce3750769e7c1c97a81116b
c6dd9a74232541d9b696142fc6e462b986b6e8d4
853831 F20101108_AAABBL knowles_m_Page_037.jp2
973711e81a06388d0d07534a610204d9
ff338d0bee2fce4437eb6cdcff20d939282aee59
2015 F20101108_AAAAWE knowles_m_Page_059.txt
0789b9ad36a4cc8fa11944fd81761568
d3586e55d53b7e661f48d7aad4f5257a8796139b
1051929 F20101108_AAABAW knowles_m_Page_021.jp2
c5cb3be7f4f9cd2a2f205da99acd1484
4afe1eceee4ed847589f3745185e66b9974bce3f
51672 F20101108_AAAAVR knowles_m_Page_043.pro
9540481c87d3cc187cae539b9715a058
459565f779d10a59b9e76c186a5c1eb7efe88ac4
1051964 F20101108_AAABCA knowles_m_Page_055.jp2
0fdf4f1a0e884302494e22d562bad441
363564b78f22dd36450077f07f7cabaf1e23e3e7
1000357 F20101108_AAAAWF knowles_m_Page_046.jp2
7f28f0bff1c947abf73bfd7f20302863
c7b3905cb73900a83903323658ff648ea892afca
F20101108_AAABAX knowles_m_Page_022.jp2
6622c8135aaaf494224adcdba79ff010
71dc5ff687a3662ef588e4934c1d0903594cddda
8535 F20101108_AAAAVS knowles_m_Page_011thm.jpg
38040d124e908a8782aa6754f7fe9199
ed863c92e82840ddc6780491512a9d9d901cac71
1051958 F20101108_AAABCB knowles_m_Page_056.jp2
d934379248d84bf69a7323fd479cc29e
782aaa26a72079ccc67286fcb0daafcabf51daaa
F20101108_AAABBM knowles_m_Page_038.jp2
6215abe0486e08585dd04d886fd70e44
3e91353331c132b2319f6a36e26145c499ee7fab
7281 F20101108_AAAAWG knowles_m_Page_009thm.jpg
bb478a19223b156f073b2a1ee70b8290
4b4078cb3bd9b1eae2231adfff7cd6d81c239426
1051965 F20101108_AAABAY knowles_m_Page_023.jp2
edfbb979d4dbefbd973e8ff9ad770198
08bf64345d9497c1d97bacb5f03bcbfc02acc21d
136 F20101108_AAAAVT knowles_m_Page_052.txt
b853e728a2328bcdb9d7a9d14161e837
14091bdf1f494f72d80f3a023dbccb21431cacf2
1051968 F20101108_AAABCC knowles_m_Page_057.jp2
15a28272145cee8ecc1e66e017aa3fb3
d4b8e7b26e064b6c9ea166c7fc369cd149772de1
899394 F20101108_AAABBN knowles_m_Page_039.jp2
ab4ad6195644fe05145036632f2b8cfb
f7d71b937a64372ee65cdbb97851040f417c5b20
36942 F20101108_AAAAWH knowles_m_Page_081.QC.jpg
d340702896619abee422a1703f22b5e2
8ae6389b2aba377b52a5daccc1c237859182f3b8
F20101108_AAABAZ knowles_m_Page_024.jp2
ab09bc1bdc4f2f428061890db875f36f
866278027c102fb87b342b7d26bfafe4cbef395a
29291 F20101108_AAAAVU knowles_m_Page_009.QC.jpg
067cb7e6d010f4138a6d7fa24fdfcbb2
5a010aa07fe54902e5192e502aa0ceb85f648d9a
1051948 F20101108_AAABCD knowles_m_Page_058.jp2
47ba46052efdf8e2f45776fdff4d8b1e
835c86885bc3cbd95b0259e6f298fe34b85aba2c
1051956 F20101108_AAABBO knowles_m_Page_040.jp2
990658213a61c448de97bba93dbc8ef0
21bd92b695e46bc7c7d850a70eac6ab6ae2549b7
F20101108_AAAAWI knowles_m_Page_031.tif
bc98d97810cbf1754ff50ead2349ac89
a5bb0f26bbc3c060d8d9b5bd19253e415f115028
1963 F20101108_AAAAVV knowles_m_Page_036.txt
4c2985bda888e253d23edb0f7c35ce20
bf5359d105e8eeed83379231aa1d2b0f0da5e048
1004740 F20101108_AAABCE knowles_m_Page_059.jp2
60d59043d0923068edf2d101787ee124
f31ee5900d0e53b8a4b5528656d03223a628eab9
1051913 F20101108_AAABBP knowles_m_Page_041.jp2
c655f986504602a637a8dcced8f4cde2
f95bfa2c9f9ab34b65f5d8cde83788af7fc0f67b
124009 F20101108_AAAAWJ UFE0021617_00001.mets
7f5e0ca5311aea9b2d1ca1814ccb99d2
bfcf9e1cfefa84dd2cb2dccda4007d66ecaa246b
F20101108_AAAAVW knowles_m_Page_016.tif
69cb5aaa3ddaafeff491ddb186954f2c
6d3cf24a6f46a2ba66f60bee1f20a79c018fca0f
1051951 F20101108_AAABCF knowles_m_Page_060.jp2
9a73a1b3a519884f1671dc0ee03e7e4b
408bdf34b2fa97f6cd5b6e6dc2b687bea8295e18
1051972 F20101108_AAABBQ knowles_m_Page_044.jp2
5072d6ee343fe5c8cdd4004eef361e6a
44bf5c0dc43ed3da5530eca48e651ada6be4a339
14302 F20101108_AAAAVX knowles_m_Page_103.QC.jpg
0ffd32720ef008c4417dd3aa6b6980f2
0ff78c8b53f281b0f46a0dee175642cacf166903
F20101108_AAABCG knowles_m_Page_061.jp2
efa68c06ba7813132d6440722ff197be
e795ec3f9d46cb85b0d06a8dce419ae20e704f8a
94771 F20101108_AAAAXA knowles_m_Page_016.jpg
2b8ccb726b6dde246763be1167aad3c9
e0fe7c1643066bb63c6e7a4bffe8fc837ce8417e
1051973 F20101108_AAABBR knowles_m_Page_045.jp2
202bfaa355ebfb2d26092efa44b9448f
4b4f3b5e30b692e3c6ba37194f52acd76c2ff924
48785 F20101108_AAAAVY knowles_m_Page_013.pro
c5a4523168bbe3bfe67558240259dc1e
2c1ff170d8349a87563f3e8629054e433d2b1f55
599978 F20101108_AAABCH knowles_m_Page_062.jp2
1aab3905d52d3ad28b5a7fd70e25c379
73a0929f20e8b26e7f595a3661f992fd06652286
114327 F20101108_AAAAXB knowles_m_Page_017.jpg
86cd5a8be833a4fc9c9c7d915abfc061
769efe5d231e89decc06157d646f281d15304ec7
F20101108_AAABBS knowles_m_Page_047.jp2
3befcb20ae1e5cc5cd577be84dcf37ba
c503cbb600316cf003430c62dac661bb66c79106
25239 F20101108_AAAAWM knowles_m_Page_001.jpg
6804ec75a09ccff0dc118b7d7d655ef4
1488acbe98e0a06830bbad05efe796d1f8a65bc3
F20101108_AAAAVZ knowles_m_Page_045.tif
d450281078bc78a4f6a317633c9c20a9
eb5805c53ecb4f1ca9078fd1ae21370f21b1aaad
F20101108_AAABCI knowles_m_Page_063.jp2
ea35f3d889ea3a063837c154358eef5b
e24223e5ba16d1874cca2a034f3377c9d6d39eb3
104837 F20101108_AAAAXC knowles_m_Page_018.jpg
9a2bb3f244282d222e33e8f7c0ba23d5
ca7d1c8a1d537ff2ca4551b9e31a7daffea674fe
1051957 F20101108_AAABBT knowles_m_Page_048.jp2
22cce911c5ff0051dd30234ceb7dadb1
77ffc13db8c3db5efe9d49ae190d61f338d090c7
6895 F20101108_AAAAWN knowles_m_Page_003.jpg
2e3fcaf2b6b9b36e88d8851e95f94985
d7374e5462a3248de0d12f08b67dae1cd407a584
1051921 F20101108_AAABCJ knowles_m_Page_065.jp2
3e74936e203783f14ca33677d60c1a8c
a2f2a828a8f299848c7222cd16a3865373cae382
1051960 F20101108_AAABBU knowles_m_Page_049.jp2
fa11a56fb922e588cedce517e0cdfe68
18c316046eba0bfb01550793fca7156bec8cae5e
41677 F20101108_AAAAWO knowles_m_Page_004.jpg
c70c1286fecc20342e35f5b319ed5a0d
50c84d84aaa2f4b92217432b9e37f76a61dd16eb
F20101108_AAABCK knowles_m_Page_066.jp2
abf3ffd3b6fb053f48400975665ae39f
b79a231feac79c5b5bcab8f32c04b671b9aa4e1e
108301 F20101108_AAAAXD knowles_m_Page_019.jpg
a05bf2f9040a6df78f5f5540bff13c99
a0f5b466e245c0f922d51790929301535a8fbee1
F20101108_AAABBV knowles_m_Page_050.jp2
479e9d0697f28b1bc414d7ab17219f69
5b6d89934e460153e446358aea248e6e0ca308bc
105805 F20101108_AAAAWP knowles_m_Page_005.jpg
3074876eeac4ee496ca2eadd75f4fb4c
5abc9fbb696b3c745a79b47f05e59355a977cdd8
1051886 F20101108_AAABCL knowles_m_Page_067.jp2
f53ad23274c4652a991dfc1f9961d635
27128647cb1fe6ea330bf59362894e913f0a9d03
111682 F20101108_AAAAXE knowles_m_Page_020.jpg
c8724784f86afead2988f84f221f99af
c3824f612e14a245e72feabbeae532a7deaa9bfb
F20101108_AAABBW knowles_m_Page_051.jp2
90e1fed7153aad588fafa35d251ada8c
fb9c65182631b898dbe007249998930f26fa6e55
66926 F20101108_AAAAWQ knowles_m_Page_006.jpg
19c6ac6037ad9582bde07679802f6d99
8e42bbf94764800f447f5dbf40c63f026863ca08
F20101108_AAABCM knowles_m_Page_068.jp2
27b84e94bc24b3d4f96d860eb24cf52d
c4f985b10de641ac960946e9969b6933bde85f68
114534 F20101108_AAAAXF knowles_m_Page_021.jpg
a65dc54170061420d1a5037237b9e563
ae3dbc85c05dc3f74ccfe9d89382c11259f6c3c2
81605 F20101108_AAABBX knowles_m_Page_052.jp2
4623f77c00a03459086a35a852c1f617
ce6ced717edd192b586c971e44393343c992b46e
24624 F20101108_AAAAWR knowles_m_Page_007.jpg
4c871373cef18a0cdf088bd2b400823a
5fac02622c784a8e14db46213859c8409691d45a
F20101108_AAABDA knowles_m_Page_083.jp2
b6bc7807b7364423041b72fc0fdff246
e86e7a1264556667c7bea1ec4f8d465329b36d6c
109055 F20101108_AAAAXG knowles_m_Page_022.jpg
edabe10600d4e7bf14497ad9cbbc878c
ac1b35e88656eb42070b5479d3492fc7b208e5df
1051942 F20101108_AAABBY knowles_m_Page_053.jp2
ec9d4c46856df9307319a1bddf0c0fbf
066c9509d66982429dd7e0edeb0bcfbd167ed13c
38204 F20101108_AAAAWS knowles_m_Page_008.jpg
69a15226caa00fd952d2f46df5ee040d
a15536fb14cd59ad9b89df50806619325613ffe9
F20101108_AAABDB knowles_m_Page_084.jp2
182542c719a10a9b0e26d91613d5f877
abdf5d13d901c3ef6c5d21af032928645ec7ba6a
F20101108_AAABCN knowles_m_Page_069.jp2
8e3be740d6af068605fbf70f4da83da3
60a2b6992342dae92df084845415bb100311bd26
108280 F20101108_AAAAXH knowles_m_Page_023.jpg
fbb453e712b949eb6d76518da765cac0
e06c8e7c019b0e3069033180a3cae05eac7c7054
1051934 F20101108_AAABBZ knowles_m_Page_054.jp2
2a531c9cce8456bf204e21c0579accae
ebdbbb7d74a1c496bc6f0c2d8f9d30d6d4ea316c
92343 F20101108_AAAAWT knowles_m_Page_009.jpg
8e85ffa408d0d9dc18503b57c47f16d8
a8a6f0360d0c9f6ab9b70fd5e2a39317a5d2c429
F20101108_AAABDC knowles_m_Page_085.jp2
f6f30547b6dbabf18c7df3a29c6fe474
df3eff067bdf9c3d9ac7d0656e53cffbbbbcb8ec
F20101108_AAABCO knowles_m_Page_070.jp2
5b623a7bd5af8ff1a478391b31cdc295
7287519899ab0333943fbcdd7712b37d093d3bd8
107722 F20101108_AAAAXI knowles_m_Page_026.jpg
c496414b44f498972ed40f3d7e2d6cdb
a46f86b80310dc4bc941e1c52500f89c44f92a21
11321 F20101108_AAAAWU knowles_m_Page_010.jpg
0aa11dd5ab744c8c2b0c55ee73bd4798
7e764a3db41750876ed5013d7e47180be82fbd0e
1051983 F20101108_AAABDD knowles_m_Page_086.jp2
a1ca0d27621b620363e26cbdc948c7f8
944420e1d9b711ffab9beea92c44046c313b0f7c
965083 F20101108_AAABCP knowles_m_Page_071.jp2
4d7436a62effe7a3a9946ee775158bab
0de1850c8c7a950516d37f7713f48c1e0c981ae9
109374 F20101108_AAAAXJ knowles_m_Page_027.jpg
712e49ec0b594bcd0d174c9f5257f8b1
95ef309fb3b064a62e9b23469f4a3d3dc4c4f181
106473 F20101108_AAAAWV knowles_m_Page_011.jpg
3978c4cb0f0588f48ad283890b9f4876
f912f7ea7aa49a66d5fd2bf3df56becc28dbf7e3
F20101108_AAABDE knowles_m_Page_087.jp2
fd087c81517521e41b5843a638a55e2d
f087576621eb27c4e9e318b91cc2ec932cc6e2dc
1051985 F20101108_AAABCQ knowles_m_Page_072.jp2
96c17ff1b0c131c7cb7083cbc243162c
ae711c10c29f49d12436bd5d831a8257897e8580
114861 F20101108_AAAAXK knowles_m_Page_028.jpg
57e1e7a6471dcf046a4a6cf2fb6154d0
c50c629bb088d80efdfeb3836eeacde75a07c05a
109466 F20101108_AAAAWW knowles_m_Page_012.jpg
610692c74dc4e60dbcdc08d4e59e09a0
3baf4d4a3987ad8bb2d6a54991e9488cf6ef2b86
F20101108_AAABDF knowles_m_Page_088.jp2
d275c526e0b000d565212fc325979839
7779ba0db1775bba2fdf12343180feaee5815d48
108880 F20101108_AAAAYA knowles_m_Page_045.jpg
c5133c258d2011ec5c509eae0efcae9d
552a766f07a4f4bb4f600a216ba21d12cb22a18d
951196 F20101108_AAABCR knowles_m_Page_073.jp2
00888e9e3668074a44aa92919e96391a
d23c904d290b5b869058dbb8ba52e449a729af34
106128 F20101108_AAAAXL knowles_m_Page_029.jpg
12552b37343d4e9243b9669e88d7a222
1a76ba4b5ac6e568a8336526db85e25d97182290
100135 F20101108_AAAAWX knowles_m_Page_013.jpg
6eefad5a248d3ef751451509acfdf940
80c3b3d8552045e4dbd52b542f8a032b8d4ebd45
F20101108_AAABDG knowles_m_Page_089.jp2
8299f228d9839f11234cb85331e48eaa
03c8687a31f334e7fc169c8e90c9bcca5c453759
111090 F20101108_AAAAYB knowles_m_Page_047.jpg
7da661c0c80cf5d8ab7545367fb200c2
16cdb5e0a6c5d88832f32a5189aaef643de1fcb3
F20101108_AAABCS knowles_m_Page_074.jp2
2b3f6c09b75b2fc9399b0bc09356c47b
08d34448e307925f21c8f182678297d2cddfda2c
107128 F20101108_AAAAXM knowles_m_Page_030.jpg
68b32be106563080e5564747f34390b0
2e2bd45903a5ac0c8735bc908932bbfd068efe03
101369 F20101108_AAAAWY knowles_m_Page_014.jpg
2b617481b46ce07f6d7e0e048404469e
e1081f2b773a4e3763c49114ef296d9e8d6d308b
F20101108_AAABDH knowles_m_Page_090.jp2
2f54a59038db150fc85b8ca81dff5faa
011851fb00540c1e09251889de7177a5962e1513
111829 F20101108_AAAAYC knowles_m_Page_048.jpg
73677d01457bc013ee787fd5329ed2e2
aba1c469a50e84c47afd89f92e88a0db942e3792
F20101108_AAABCT knowles_m_Page_075.jp2
8e78cc647e4eaf88a85a3a3fd2fe37a3
f6753e31ef32e82a42bf8202f4c45d12c65bc5a5
112101 F20101108_AAAAXN knowles_m_Page_031.jpg
85c70317bf96a2fba8fda5b2939a6238
7742aacf5fe45278aa3583445ac45523c436cb34
106516 F20101108_AAAAWZ knowles_m_Page_015.jpg
cda4491ee8f97f3f8a13423a48913009
7e6646e5667e63483da484ec1f0c3a601cf3f99b
1051984 F20101108_AAABDI knowles_m_Page_091.jp2
ede266d1336505709963e8de8da5613e
9ebc8dd7963c6093cbdc8ae931e81d06aba64c19
105913 F20101108_AAAAYD knowles_m_Page_049.jpg
6de905ab3fef6d0ca75545de7d1b3bb5
e0edd987be47c2e96b0a6c8f59a1b045ba9f34c4
719367 F20101108_AAABCU knowles_m_Page_076.jp2
44ccdd8166f664a72d426e0e8ba78731
39d4947488f1bababf18937373ec08469bf4c3ee
94445 F20101108_AAAAXO knowles_m_Page_032.jpg
bb7802f33522ca6eb0d5a1bf5b9e49d3
522950a756b94d2de8c4b1160e2544e3ad83f4fa
1051955 F20101108_AAABDJ knowles_m_Page_092.jp2
0edc192f2ee1e3f5c0e53d376184e027
8e5faf27f317943a66b3be0622ae417e3e2f0584
F20101108_AAABCV knowles_m_Page_078.jp2
5c3ec78769b97699ed6e112a9d3fa762
6e19a44812a66f268bc3a370818d686a5da34b88
111023 F20101108_AAAAXP knowles_m_Page_033.jpg
98b33ccd9f8fa88f4a2f39c1fd859c0a
d3cd9c5ccd83e6284402e8308ae0fe8dc53a2d4a
1051979 F20101108_AAABDK knowles_m_Page_093.jp2
2e2114abe02810815999b0e4bb07ab24
259e728a64c80aed0f1d6bd44834e489def21a79
114205 F20101108_AAAAYE knowles_m_Page_050.jpg
8ce87ccba70547cf1ae1164bd0b44214
6468562a4cc2cdc3831cdbe5ce897999bc6d3556
F20101108_AAABCW knowles_m_Page_079.jp2
f93d408fd6797cf8cf4ed2c1b43736f9
ed141de9526702dfa28430faf276336836adab8a
109729 F20101108_AAAAXQ knowles_m_Page_034.jpg
cbcdc567decbd9d22237f1d80c4ae70a
ef5eb721345d9bc0529634672319943497df8076
1051924 F20101108_AAABDL knowles_m_Page_094.jp2
15779c49596a9c10ccd3e749d38b0713
b58a43b30b1847a9f3a76deb3d149df792bce724
112189 F20101108_AAAAYF knowles_m_Page_051.jpg
8e8729f73be022c973759d76d4fdd705
16bc79cae012e0a7c63d939606179355961127a4
F20101108_AAABCX knowles_m_Page_080.jp2
48c22001b12f5d486dc32f97cb5c3d1c
609fd5e482362e389fb9cf7ea58e5ced7d6c2c10
98889 F20101108_AAAAXR knowles_m_Page_035.jpg
a5628324d76b06c69d7dd07d22fdf030
fd77beb07e9d88e87e7f20dad7fe1122e9781cbb
F20101108_AAABEA knowles_m_Page_005.tif
7b92797b67198790066521d09875f5fd
796062eb7ddf7d5b20dfa88bf6cf22b360c4df92
1051975 F20101108_AAABDM knowles_m_Page_095.jp2
9617242257a364efa8364bf19a863e63
f30b87fb4fdec4c9b31398bfd03026ffcb369498
9333 F20101108_AAAAYG knowles_m_Page_052.jpg
6de691461f06b7d389008f6bbb815bef
3af028e8c0fbae67363f2186de0f1396b754f2ee
1051954 F20101108_AAABCY knowles_m_Page_081.jp2
3cf426c8c4b0634b73f919c4ed90502b
33291f2047c185c6f3dd633f825e727ab012186e
99985 F20101108_AAAAXS knowles_m_Page_036.jpg
e75751d123824db4e3b0bdf7de6abd87
01d84da4ec82896cd7a6eef4c9cf1391def4a1b6
F20101108_AAABEB knowles_m_Page_006.tif
235cb6bd33024295998efca026ea1604
c335fb0185d2063aadea96950f0e45faea33e145
F20101108_AAABDN knowles_m_Page_096.jp2
a92c1a64264e76cf86bd9ca53000abee
509d7b56f482c4f52544c233dd3ddc201300d320
99735 F20101108_AAAAYH knowles_m_Page_054.jpg
212b0d3087c0f6e02e9289e8d37a4b53
8067fb0347e5a565138b43a83fa3ac47b461fd16
F20101108_AAABCZ knowles_m_Page_082.jp2
ffb5cd5bc3474eb8fa678ab7db1d5727
091563263ae9ed3d4a4e6e2c30e5149e085b4326
74952 F20101108_AAAAXT knowles_m_Page_037.jpg
61bcf296af0abecf5c97500cc0144c80
b905a4bed32faae949adc16695885023adbda230
F20101108_AAABEC knowles_m_Page_007.tif
007f50b99dd3cde5c1093070ecc07823
785a788ab6f92222f32f452ee62f228fd75af164
F20101108_AAAAYI knowles_m_Page_055.jpg
3eead07fa5d9c3220081383febdf0984
0312f84bb21293a8266777c00e1787c4211a97e0
107736 F20101108_AAAAXU knowles_m_Page_038.jpg
cbb10187c09876a85276bf735fae6f5d
3bff740d07fa527a910e75c6a53424a6a4b0cc1d
F20101108_AAABED knowles_m_Page_008.tif
653d9d5892f632b6eac36325725fd5d4
1e91225e01b0037ed2bfe1dabf6582bcd70e2200
F20101108_AAABDO knowles_m_Page_097.jp2
a1188b185cbd0e389acd5ee74b1e134d
bc6aa407f3e38e73ee3b1409ad69ead544009bc5
107913 F20101108_AAAAYJ knowles_m_Page_056.jpg
745f50788834a2418159ee89f90c8d35
c51dc0226dc974316f0a6328ebdaa139451dae1b
112079 F20101108_AAAAXV knowles_m_Page_040.jpg
48f3d6dabdf5ee9d1482b318315e4bba
ebdf5e9e6c7a49eba06627b964ac21d6d0dd5d2d
F20101108_AAABEE knowles_m_Page_009.tif
e1a4b932014cf7533069babba5158e94
1c46be4f9a35b8a614b60eebce9a56f594c5545a
F20101108_AAABDP knowles_m_Page_098.jp2
73867ef2ca6c84167a459ca437b5dd2e
54fa3e69406fba5bad45407f1a7026a17fc55036
112907 F20101108_AAAAYK knowles_m_Page_057.jpg
9e8c64abf98ff64cdc67c449f7063771
164bd9c5c2a23f789707dd680f3ccfec07989c2c
111899 F20101108_AAAAXW knowles_m_Page_041.jpg
de2cd07f470c959074034bb2b4e2fda0
a8782fe5d55e0b92be9e320f1993b6a1593426de
F20101108_AAABEF knowles_m_Page_010.tif
de4480d85790980950eb86d9b833bd0f
17d90624fe60aa90c543b3ac7700341c07f17b38
339476 F20101108_AAABDQ knowles_m_Page_099.jp2
528120f56f42bbaf8e3f9f4f9c540eaa
0120d2fd46fbcfe8d348e2157db9a8a4359b00ec
112600 F20101108_AAAAXX knowles_m_Page_042.jpg
7684fb018feb2b69379bbe3f139161ae
890d533dc333305c4935dbc524370944034a2b8b
F20101108_AAABEG knowles_m_Page_011.tif
454707d574115d333af1f46c45c07b5f
b8cc4ff7f1c62c04116eb03ce4ca66bedc2e3e50
91064 F20101108_AAAAZA knowles_m_Page_073.jpg
ec4e0cccca90767e6eb84409fb28d57f
112ee275bb97a1d6d247b7ccf574838ddd3e0f2b
F20101108_AAABDR knowles_m_Page_100.jp2
d1ceaf53db7156baa1a5034659ad5473
775a9dd1febd213cb2fb175e9261c4bdda08a78d
109928 F20101108_AAAAYL knowles_m_Page_058.jpg
d403ab8f198a69d3375af793a35d2ffc
3cdc905e80dc0a79c3b6fab9fff2d4f18e92eb50
104022 F20101108_AAAAXY knowles_m_Page_043.jpg
67dbab4a1c7f60b57e4324af04ba6a49
233e9d4ae9953b1b9e3d54743f857ef982417b3b
F20101108_AAABEH knowles_m_Page_012.tif
2c9fde191908facc23105a6f0b0f87c9
ff9d32707adffb754f23045e3a9282d616f73d9e
110494 F20101108_AAAAZB knowles_m_Page_074.jpg
df1709d4641e152c219af960e29715c4
c0d61bf57aa3740b2ee29262d02b6a72dae65653
F20101108_AAABDS knowles_m_Page_101.jp2
37f3236bdccb4f24213868c04eb6600e
5306b84fa957059993683243499b83677e0eb1fd
98102 F20101108_AAAAYM knowles_m_Page_059.jpg
f86711b5da4e85dedfc0fb425b5d0d7d
cdf9e9ba416b1aa516ba0c4aa7f52282686d97a1
101217 F20101108_AAAAXZ knowles_m_Page_044.jpg
9d0e1fd9ae079ff67ad1b8a40982a3bb
58f4a09b8267f865aa8c31f8202c75d1d7bc97c6
F20101108_AAABEI knowles_m_Page_013.tif
33829f184bd58f44d2bebef7f614df0b
e2989827bf2f8ed8b45903339dbaf7cbd19cb4df
109420 F20101108_AAAAZC knowles_m_Page_075.jpg
10171efb3d80c207a022d258af7dc079
b31bf0e2abecacaede78d5463e1d327ad0360d9c
460427 F20101108_AAABDT knowles_m_Page_103.jp2
a3d5231bba564ff160160081c7425199
4ce3d1d2229bac0bbfb9eccba5b656e8354ddea1
108711 F20101108_AAAAYN knowles_m_Page_060.jpg
3c4d4b7acbc1d1738e5edd2d51630fce
a8240466f04e22229700494dcf070b379b71efb9
F20101108_AAABEJ knowles_m_Page_014.tif
50f6fbccb774748497901b0fd675ab10
66eab1edf046bbd91d593600448df5dab6e1af8d
66923 F20101108_AAAAZD knowles_m_Page_076.jpg
22ea46499f897c8c99444cd053db1669
896b14605e7160e3c197f2ea6a973b00dfb82328
F20101108_AAABDU knowles_m_Page_104.jp2
2cfca920d73698601f7bb824212e39d8
2153d9104dd92372cf80f9ee9513a9fe5d7bb402
103016 F20101108_AAAAYO knowles_m_Page_061.jpg
2de963f9c5838816c1acbc7f053a6a9c
27c38d5a8c8b6c6b7ca8104243667bca0d254fcc
F20101108_AAABEK knowles_m_Page_015.tif
b2ab31fa6e77e24c877e525c143c4e3c
b2e2ddb3d0157d976cdf3fe2578b2ccc2a7789b1
104229 F20101108_AAAAZE knowles_m_Page_077.jpg
88970e6953cb8087e18c44eb91bc6fa1
188aff35ed1536ad687d057c94d53da871698f1d
F20101108_AAABDV knowles_m_Page_105.jp2
23f2502869e415a2bf876a5b089ee047
9b40c00fe7f5e6fd25979a6d2a8d1dd68238560b
55775 F20101108_AAAAYP knowles_m_Page_062.jpg
3d1686200f3cfa0032abf8fd70d8dda0
a28fbe40415a631fa78ef41c9c1657fc3853e584
F20101108_AAABEL knowles_m_Page_017.tif
d8c1dc0b4b480a3b4b6aa98a8a866d7c
fe73de910f5140b3a0c5e262d992201c40bfa3d3
656813 F20101108_AAABDW knowles_m_Page_106.jp2
1f910efc3758b5b219928bf881f1d189
01f00b6cb22fd749f4087818867c05a68c38fc74
109693 F20101108_AAAAYQ knowles_m_Page_063.jpg
76531f39bfe2c142859ba83dd8728c50
5d399e84c6008f46481a7b97597bdfc721369170
F20101108_AAABFA knowles_m_Page_034.tif
de83a8fcec717bd7020fed02af7315cc
9c4765147db44b80165646f8c7ef4c277f14ee45
F20101108_AAABEM knowles_m_Page_018.tif
35491a3cd84c8bd3fd05ba5163856e02
799574636bee67dfd30c04e27d9300458a687b58
108663 F20101108_AAAAZF knowles_m_Page_078.jpg
cf1fab8f54022d3b9d0c2c6dcc49b8d1
20ed524e2bbe501bb63cd403ba74e55d6f89ea53
F20101108_AAABDX knowles_m_Page_001.tif
0ef972ee8bc975ffc420a8fb2c6f9b97
0f1378597fa6367bbf2834c8b7816f48b2b903b7
111393 F20101108_AAAAYR knowles_m_Page_064.jpg
4526ef04a4fb18a60dde69195de9d902
07c3d5e44a6530056b5c1b02da2686c94b1d8566
F20101108_AAABFB knowles_m_Page_035.tif
c0c457778603f16b220cb011145b473d
4414dfc972c2af2ae7d7b55ca624baecd13fb7e7
F20101108_AAABEN knowles_m_Page_019.tif
2b6cd8bd4b5131132b29d8198645bce6
aa89c501134044cc08b4f770e022952e03e48c23
111627 F20101108_AAAAZG knowles_m_Page_079.jpg
6241193bcc8fbba9a5a4b49b3e93dbde
b29b4f9384fce45dc9f02752c7d05d62bc4ae39a
F20101108_AAABDY knowles_m_Page_002.tif
09acebe646fcf6aa7344288eed314c10
6983a89e7ad7cd7b23b67efa01e05fccbd904ff3
96538 F20101108_AAAAYS knowles_m_Page_065.jpg
1a8330f383c6161440260ef9d6597178
6459bd28dd3323927a192d06d9a62cf349a46ae8
F20101108_AAABFC knowles_m_Page_036.tif
dd83891b4e885d8227d977e4a3122fc2
f787b94556d462c8142929d03ef82fcb7e521fa0
F20101108_AAABEO knowles_m_Page_020.tif
e9a4dccf423b6908f9333aad0d9bebc9
231114ad04a37bdad46e0a62562c010cfb4627f8
111185 F20101108_AAAAZH knowles_m_Page_080.jpg
f52174e5f94f221ab25ea9b1b4200d76
26cfac634fe77b2b7017336756f78cc836cf35fd
F20101108_AAABDZ knowles_m_Page_003.tif
5e52e7c0b4335a99effd3ec4bc0910fe
9b497cb7401dcefcc6192237fd0f8dff06484303
101674 F20101108_AAAAYT knowles_m_Page_066.jpg
80d71a5f9c246f5cceaa8a3808d60621
909d5303f23c4bac2e6c19347af092109c544bae
F20101108_AAABFD knowles_m_Page_037.tif
4d4f97cba959ef084429fdacac25b4ea
7c73586f1720ff132cf644bedc5e399936a6d1c0
113365 F20101108_AAAAZI knowles_m_Page_081.jpg
3ff01b0a91e64f222b9d5b317aa6a15f
337c5dd9823f126fc23d6505043f1fa1f3e45c44
114284 F20101108_AAAAYU knowles_m_Page_067.jpg
6746922f51b36b1f16dc4ad1d16e98a0
c2737cd407c05de74c623027849b8870ffa9fc1d
F20101108_AAABFE knowles_m_Page_038.tif
725f031de29e92e1d74feba3fa188fab
4cdeffe8618ad559f51da0c966ffea8c29f1e0d8
F20101108_AAABEP knowles_m_Page_021.tif
22c90f97bbfdc106043bd0b5edd4f2c8
a182b75fd15d26c0d5d4a4051e997ca2af5854b2
109528 F20101108_AAAAZJ knowles_m_Page_082.jpg
59abf3a9b7e1cdb21e6725e67bac47c0
18b8865a057181def9b146ee2a91d27c827028a9
111209 F20101108_AAAAYV knowles_m_Page_068.jpg
3c32f3af69bda194cc64efe13ffed5dc
adba6ba714a35a8c08d73b104e33794058dffddb
F20101108_AAABFF knowles_m_Page_039.tif
f97eab94f384e8bc12ea55a3c9e8ea00
e3807ba93fd1d3a59b575423600abe22d019f415
F20101108_AAABEQ knowles_m_Page_023.tif
28b60bcc4323120d5469d8fca05c722c
3640f8c2e1e9a395c454aed25cc6ffb5e222f742
111650 F20101108_AAAAZK knowles_m_Page_083.jpg
3c0e053397d7e6c713c3fbf47454f462
6ad9c56fe451a92e46963139342e99cc7b5edf10
109864 F20101108_AAAAYW knowles_m_Page_069.jpg
133b348895aff2095ac1d8530151bfd2
daa3dd9cf1ad87a72f88ef68fd4ea3c3320ac2b1







IMPROVING PROBLEM SOLVING EFFICIENCY: THE WHAT AND HOW OF CAUTION


By

MARTIN E. KNOWLES
















A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2008







































O 2008 Martin E. Knowles



































To my wife, son, and parents, you have made me who I am and I live for you and because of you









ACKNOWLEDGMENTS

I thank my mom and dad, who provided me and my brothers with everything we ever

needed. They sacrificed so that I could receive an excellent education and always supported me

in everything I did. I thank my wife for putting up with me, I know it' s not an easy job. She has

done a phenomenal job raising our son, and she gives me strength every day. I thank my advisor,

mentor, and friend, Peter. He has provided me with many lessons in and outside of the

classroom, and I have learned so much from him. Last, but definitely not least, I thank my

committee, Drs. Abrams, Berg, Brenner, and Fischler. Their flexibility, unselfishness, and

willingness to help are something I will never forget.














TABLE OF CONTENTS


page


ACKNOWLEDGMENTS .............. ...............4.....


LIST OF TABLES ................ ...............7............ ....


LIST OF FIGURES .............. ...............8.....


AB S TRAC T ......_ ................. ............_........9


CHAPTER


1 INTRODUCTION ................. ...............11.......... ......


Illegal Move Commissions: Frequency and Proposed Causes ................ ............ .........12
Proposed Causes: Move Selection ................. ...............14........... ..
Early Research ................... ........... ...............17.......
Knowles and Delaney (2005) ................. .................. ...............18. ....
Caution Induction Framework (CIF (pronounced "chef")) ................. ................ ...._..23

Stage 1 (Consequence) .............. ...............24....
Stage 2 (Involvement) .............. ...............25....
Stage 3 (Facilitation) .............. ...............26....
Che cki ng ................. ...............27.____ ......
Evaluation............... ...............2

CIF Summary .............. ...............30....

Experim ents .............. ...............3 2....


2 EXPERIMENT 1 .............. ...............33....


M ethods .............. ...............35....

Participants .............. ...............3 5....
Problem s ........._... ......___ ...............36....
Hobbits & orcs .............. ...............36....
Titration ................. ...............37.................

Design ................. ...............38.................
Procedure ................. ...............40.................
Results and Discussion ................. ...............42................


3 EXPERIMENT 2 ................ ...............53........... ....


M ethods .............. ...............54....

Participants .............. ...............54....
Problems ................ ...............54.................

Design ................. ...............54.................
Procedure ................. ...............56.................













Results and Discussion ................. ...............56........... ....


4 EXPERIMENT 3 .............. ...............63....


M ethods .............. ...............65....

Participants .............. ...............65....
Problems ................ ...............66.................

Design ................. ...............66.................
Procedure ................. ...............67.................
Results and Di scussion ................. ...............69................


5 GENERAL DI SCUS SSION ................. ...............77.......... ....


Illegal Moves ................. ...............77.................
Transfer Effects ................ ... ......................81

Untested Assumptions: Illegal Move Reduction ................. ............... ......... ...83

Integration With Existing Theories of Illegal Move Selection ................. ......................85
Real World Application and Future Research ................ ...............97...............


6 CONCLUSION................ ..............10


APPENDIX THINKING ALOUD INSTRUCTIONS .............. ...............103....


LIST OF REFERENCES ................. ...............104................


BIOGRAPHICAL SKETCH ................. ...............106......... ......












Table page

2-1 Legal moves made on the first and second. ............. ...............44.....

2-2 Illegal moves made on the first and second. .............. ...............49....

3-1 Means and standard deviations (SD) by group. ............. ...............58.....

3-2 Means and standard deviations (SD) of illegal. ................ ................................61


LIST OF TABLES










LIST OF FIGURES

Figure page

1-1 The three-stage Caution Induction Framework. ............. ...............25.....

2-1 The display seen by the participant for the. ............. ...............37.....

2-2 The display seen by the participant in the ................. ...............39.............

2-3 Average moves times in milliseconds by group. ............. ...............46.....

3-1 Average time per move for each of the four ................. ...............59.............

4-1 Average time per move for each of the four ................. ...............71.............

4-2 Illegal moves for each group were presented. ............. ...............73.....









Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy


IMPROVING PROBLEM SOLVING EFFICIENCY: THE WHAT AND HOW OF CAUTION


By

Martin E. Knowles

December 2008

Chair: W. Keith Berg
Major: Psychology

A three-stage framework was proposed describing the reduction of illegal moves in a

problem solving situation. The Caution Induction Framework (CIF) proposed that consequences

for making or avoiding illegal moves in the first stage of the framework influenced the problem

solver to increase the amount of attention devoted to the problem in the second stage. This

increase in attention subsequently increased checking or evaluation behavior in the third stage

resulting in the reduction of illegal moves.

Three experiments were presented to test specific aspects of the framework. The first

experiment tested whether a punishment was necessary to reduce illegal moves or if threat alone,

without punishment, was sufficient to yield the same result. The second experiment tested

whether an aversive stimulus was necessary or if a positive consequence would yield illegal

move reductions. Specifically, participants in the critical condition in the second experiment

received candy for avoiding illegal moves and for solving the problem efficiently. The third

experiment tested whether an instruction to check each move for legality would result in illegal

move reductions. In addition, participants in the critical condition in the third experiment were

asked to perform this behavior aloud while their voice was recorded. The results from the three










experiments mostly supported the proposed framework of CIF; however, future research is

required to further test the assumptions of the framework.









CHAPTER 1
INTTRODUCTION

People solve problems every day in a variety of settings. Unfortunately, these problems are

not often handled as well as one would like. A person's mistakes on a problem may waste both

time and effort. An important goal for psychologists is therefore to (1) determine what

difficulties problem-solvers may encounter when working on such problems, and (2) develop

techniques for overcoming these difficulties. Kotovsky and colleagues extensively studied the

factors that contribute to problem difficulty (Kotovsky, Hayes, & Simon, 1985; Kotovsky &

Simon, 1990). However, less attention has been devoted to overcoming these difficulties and

improving problem-solving performance.

What makes a task a problem is that the solution is not immediately available to the

problem-solver. If the solution were immediately evident then it would not be a problem; it

would be a task that needed to be completed, but not solved. The solution is unknown, so the

problem solver must take action to find the solution and this requires that the problem solver

interact with the problem physically and/or mentally. Since knowledge of the problem is limited,

the problem solver will often make several incorrect moves. These incorrect moves may include

legal moves that take the problem-solver down the wrong path (e.g., turning south down a street

when the final destination is north does not violate any laws, but is incorrect because it takes us

away from our goal). Incorrect moves may also be illegal because they violate one of the rules of

the problem (e.g., turning the wrong way down a one-way street violates a rule because it breaks

the law, even if it would bring us closer to our final destination).

An illegal move, as the author has referred to them, may carry different consequences

depending on the problem. It may have severe consequences, such as the termination of the

problem or in the most extreme cases even injury or death (e.g., running a red traffic light









and causing an accident). However, the rule violation may have less severe consequences that

result in a minor penalty (traffic fine) or a simple warning (verbal warning to stop at all red

traffic lights in the future). This dissertation was focused on two goals first, to understand the

consequences that aid a problem-solver in reducing illegal moves, and second to understand how

the problem solver's behavior was modified by those consequences.

The prevalence of illegal moves and potential theories of how and why illegal moves are

made will be discussed first. Next a literature review providing insight into illegal move

reductions will be reviewed. Then an in-depth overview of Knowles and Delaney's (2005)

problem-solving research will be provided as it acted as a basis for much of the work that was

presented here. This will be followed by an overview and in depth presentation of the CIF

framework for the reductions of illegal moves proposed in this dissertation. Finally, this chapter

will be completed with a brief preview of the three experiments that were conducted.

[Illegal Move Commissions: Frequency and Proposed Causes

Illegal moves may often account for a significant portion of a problem-solving episode.

Knowles and Delaney (2005) reported that upwards of 20% of the total moves in the hobbits and

orcs problem were illegal and Jeffries, Polson, Razran, and Atwood (1977) reported illegal

moves rate averages up to 32.8% on similar problems. Therefore, techniques that reduce illegal

moves would likely reduce solution lengths and possibly the time required to complete the

problem.

To develop techniques for reducing illegal moves it would first be beneficial to understand

the reasons why problem-solvers select illegal moves. Research suggests three maj or causes for

illegal moves. The first cause is related to understanding of the problem, the second to

limitations on our mental resources, and the third is related to the specifics of people' s move-

selection heuristics. I will consider each in turn.










Proposed Causes: Understanding

One reason for the selection of illegal moves could be that the problem-solver does not

understand the problem fully or the rules that result in an illegal move. Understanding the

problem refers to knowledge of the goal of the task and how to reach that goal. Understanding

the rules refers to knowledge of the parameters in which a problem-solver must work in to solve

the problem. The rules are the restrictions set up by the environment or those explicitly stated.

Kotovsky and Simon (1990) found illegal moves when the problem solver had trouble

understanding the problem or more specifically trouble understanding what constituted an actual

move. However, additional research has found that problem-solvers select illegal moves even

when the problem and the rules of the problem are well understood (Jeffries et al., 1977; Zhang

& Norman, 1994; Knowles & Delaney, 2005). People often make illegal moves, but the reasons

why remain unclear.

Proposed Causes: Resource Limits

One maj or explanation for illegal moves was that problem-solvers have resource

limitations. Exactly what these resource limitations are has not always been specified, but

usually the term resources refers to working or short-term memory limits, attentional capacity,

and thinking speed. Jeffries et al. (1977) proposed that illegal moves were selected due to

resource limitations that prevented problem-solvers from correctly calculating future states or

from checking moves for legality at all. Resource limitations have also been proposed for

performance deficits, such as a lack of planning, in other tasks as well. Atwood and Polson

(1976) attributed a lack of planning in water jugs problems to short-term memory limitations. A

lack of planning was also obtained by O'Hara and Payne (1998) in research involving the 8-










puzzle. This lack of initial planning seems to support the claim that problem solvers do not have

the resources to plan complete solutions.

Recent research, however, indicates that problem solvers often do have the resources

necessary to plan complete solutions and to avoid illegal moves. Delaney, Ericsson, and Knowles

(2004) found that when instructed, participants were able to plan complete solutions to difficult

water jugs problems without any feedback. O'Hara and Payne (1998) also found that when the

cost of making a move increased on the 8-puzzle, participants engaged in planning more

frequently. Furthermore, Knowles and Delaney (2005) found that increasing the cost of

selecting illegal moves subsequently reduced the number of illegal moves made, even though the

resources necessary for calculating future states and for remembering to check each move for

legality were not altered. These findings indicated that participants have the resources necessary

to perform more efficiently on these tasks. It is still unclear why illegal moves are made if the

resources are available to avoid them.

Proposed Causes: Move Selection

One additional area of previous research that has received attention in the problem-solving

literature is modeling how a problem-solver selects each move. Understanding what was done by

the problem-solver in selecting a move would be of great value to understanding how and why

illegal moves are made. However, how a problem-solver assesses a move as acceptable or not is

not entirely understood. Researchers have proposed models to explain how moves were

evaluated and selected. One model that can be used to explain the evaluation process that

problem-solvers may have engaged in on such tasks as the hobbits and orcs problem was

proposed by Jeffries, Polson, Razran, and Atwood (1977). In this model the evaluation function:

e, = aMl + bC, + cP, (1-1)









was used to determine if a move was acceptable and exceeded the required criterion. In Jeffries

et al.'s evaluation function (Equation 1-1) e, was the value of the evaluation function for state i,

in which there were M, hobbits, C, orcs, and P, hobbit-ore pairs on the right bank and a, b, and c

were constant positive weighting factors.

This model consisted of a three-stage process and in the first stage the problem-solver

evaluated potential moves according to a specific noticing order. Noticing order referred to the

idea that the problem-solver would always consider specific combinations of travels first as these

combinations would likely have a higher probability of making greater advances towards the

Einal goal state. If a move was evaluated and had a higher value than the current state and had not

been visited before then it would have been selected with a specific probability d +, where i was

the number of moves that had been considered and rejected during this episode. If the move was

recognized as being previously visited then it was selected with probability P. If no move was

selected in Stage 1 then the model attempted to find a move to a novel state in Stage 2. If all

moves were identified as previously visited then the model attempted to find the optimal move in

Stage 3 or it randomly selected a move in this stage.

Once a move was selected in one of the three stages it was then checked for legality with

the model's illegal move filter. The resulting states of moves were checked for legality. A move

was rej ected if it was determined that the move was illegal and executed if it was determined that

the move was legal. Jeffries et al.'s (1977) illegal move filter described the probability of

checking a move for legality as fixed probability E 1 and the probability of correctly rej ecting an

illegal move as fixed probability E2. JeffrieS et al. also proposed that there were hard-to-detect

and easy-to-detect illegal moves and that easy-to-detect illegal moves were always rej ected. This









was one of the first models the author found that specifically addressed how a problem-solver

may identify and rej ect illegal moves.

Another model that was designed to provide an understanding of how moves were selected

was Lovett' s (1998) ACT-R model of choice. Lovett & Anderson (1996) have also used ACT-R

to understand how the prior experiences of success and failure affected future decisions on the

Building Sticks Task (BST). Lovett's model assumed that problem-solvers selected their next

move based on the highest expected gain according to the following equation:

E = PG -C (1-2)

where E was the expected gain of the selected move, P was the estimated probability of

achieving the production' s goal, G was the value of the goal, and C was the estimated cost to be

expended in reaching the goal.

Lovett and Anderson (1996) presented participants with the building sticks task (BST)

where participants had to add and subtract three different size sticks to create a stick equal to a

specified target length (similar to the water jugs task used by Luchins, 1942). In this task the

participants solved several of these puzzles and the authors were particularly interested in

participants' tendency to select one stick over the others as their first move and how previous

successes and failures of beginning with different sticks affected this decision. Lovett' s model of

choice, as applied to BST, focused on a problem-solvers' ability to reflect on these past

successes and failures of operators to calculate E from applying an operator. This model was

designed for BST, but could be modified to model move selections on other tasks as well. Many

other move selection models exist for various problems, but to review them all here would not be

practical .









Next I will turn to a discussion of studies that attempted to reduce illegal move rates. The

amount of research in the problem solving literature that specifically looked at reducing illegal

moves during a problem solving episode was limited. However, there were several works that

used the number of illegal moves committed as a dependent measure. Such research provides

insight into how problem-solvers reacted to specific manipulations and allowed for speculation

on how problem-solvers approached and solved problems.

Early Research

Two relevant studies which presented manipulations that reduced illegal moves provided

insight into increasing problem solving efficiency. Zhang and Norman (1994) changed the

problem constraints in isomorphs of the Tower of Hanoi problem from internal rules (verbal or

written constraints that were to-be-remembered: e.g. the tall peg cannot be placed in the middle

hole even though it fits) to external rules (constraints not explicitly stated because they were

embedded/implied by the physical environment: e.g. the pen cannot be pushed through the eye of

the needle). Not surprisingly, changing the problem so that the environment prevented

participants from making illegal moves resulted in illegal move reductions. Kotovsky and Simon

(1990) took a very challenging problem where the available move options were not easily

understood and presented participants with all the legal options at every problem state. With the

legal options available participants reduced the number of illegal moves they made on isomorphs

of the Chinese Ring puzzle. These studies presented manipulations that reduced illegal moves

and provided insight into problem solving performance. However, in both cases the problem

was altered and could have potentially reduced the resources required to avoid illegal moves.

Therefore it is difficult to determine whether either study lent support to the resource limitation

hypothesis or not (Jeffries, Polson, Razran, & Atwood, 1977). In addition, changing the

environment (making internal rules external) and presenting additional information (providing










participants with all potential legal moves), respectively, to reduce resource requirements will

not often be possible in the real-world and such manipulations may alter the problem itself.

Ultimately the limitations of these works to address the illegal move reductions made these

results difficult to apply to other problems in the laboratory or real world.

Although the occurrence of illegal moves accounted for a significant portion of total

moves, in at least some problems, the reduction of illegal moves in problem-solving does not

appear to be widely studied. This may have been due to the idea that the resource limitation

hypothesis (illegal moves occurred because we have a limited number of resources available

while working on a problem, thus illegal moves were inevitable) was so widely accepted and

therefore there was little that could be done to improve illegal move commissions. Another

possibility was that it may have been overlooked that illegal moves played such a significant role

in problem solving episodes. A third possibility was that the link between illegal move

reductions on laboratory controlled experiments and the real world application of these findings

were not obvious and thus such research did not seem beneficial.

Knowles and Delaney (2005)

The most extensive research on illegal move reduction was conducted by Knowles and

Delaney (2005). An overview of that paper was offered here for two main reasons; (1) they

intentionally tested manipulations that reduced illegal moves and (2) their work acted as a basis

for much of the work presented in this dissertation.

Specifically, Knowles and Delaney proposed a 3-stage framework in an attempt to

understand where and how different manipulations affected a problem-solvers selection and

rej section of an illegal move. They presented three experiments demonstrating ways to reduce

illegal moves on isomorphs of the missionaries and cannibals or river crossing problems. They









looked at what knowledge was transferred when solving the same problem twice and knowledge

that was transferred to a novel isomorph. Finally, they introduced the idea that caution (this term

is subsequently defined) was a result of their manipulations and that this caution subsequently

led to the reduction of illegal moves.

Knowles and Delaney (2005) presented a hierarchical 3-stage framework in an attempt to

understand how and where manipulations would affect a problem-solver in their selection and

rej section of illegal moves. The first stage asked whether or not an illegal move came to mind

(Generation-Rate Hypothesis). If an illegal move did not then a legal move would be selected. If

an illegal move did come to mind then the problem-solver moved to the second stage which

asked whether or not the problem-solver checked the rules to see if the move was legal (Caution

Hypothesis). Caution, which described this stage, referred to the increasing likelihood that a

candidate move would be checked for legality. If the rules were not checked then the illegal

move was committed. If the rules were checked then the problem-solver moved to the third stage

which asked whether the rules were correctly checked (Rule-Verification Hypothesis). If the

rules were not checked correctly then an illegal move was committed. However, if the rules were

correctly checked then the illegal move would have been rej ected and avoided and another move

would have been considered. This framework described illegal move rej sections and allowed their

experimental manipulations to address different stages/hypothesis of the framework (the

Generation, Caution, Verification framework of Knowles and Delaney was referred to as GCV

herein).

In Experiment 1, Knowles and Delaney (2005) punished participants for illegal moves on

the hobbits and orcs version of the river crossing problem to assess participant' s ability to avoid

illegal moves. In addition, participants were instructed to think aloud while working on the









problem in an attempt to discover when and how participants selected and rej ected illegal moves.

When compared to a control group Knowles and Delaney found that those participants that were

punished made fewer illegal moves while the number of legal moves made did not differ. When

compared to a silent control group the legal and illegal moves for participants that were

instructed to think aloud while solving the problem did not differ. These findings indicated that

participants were able to avoid illegal moves when their attention was directed to do so and that

thinking allowed had minimal to no effect on problem solving performance.

Experiment 2 was similar to Experiment 1 in that participants solved the same problem and

the experimental group was penalized for making illegal moves while the control group was not

penalized. However, there were two main differences; (1) there was no think aloud condition, all

participants solved the problems silently, (2) each participant solved the same problem twice

where the second solution attempt was solved with no penalties regardless of the instructions on

the first problem. The results replicated those of Experiment 1 in that the penalty group made

fewer illegal moves while the number of legal moves did not differ on the first solution attempt.

The control group made fewer illegal moves on the second solution compared to the first, but

there was no change in illegal moves for the penalty group. One of the more surprising results

was that the benefits from being penalized on the first problem were sustained and carried over

to the second solution attempt as the illegal moves made did not differ between solution

attempts. The lack of a decrease from the first solution attempt to the second may have indicated

a floor affect. Having each participant solve the same problem twice allowed the authors to

assess if there were any sustained benefits from solving the first problem under penalty

instructions. This also allowed for the comparison of the change in legal and illegal moves with

practice on the same problem.










Experiment 3 was similar to Experiment 2 except that the second solution was replaced

with an isomorph of the first problem. This second problem had the same underlying structure as

the first problem, but did not have any outwardly obvious similarities. As in Experiment 2

regardless of the instructions on the first solution attempt all participants solved the second

problem without penalty. The results largely replicated those of the first two experiments where

the number of legal moves did not significantly differ between the penalty and control groups,

but the penalty group made fewer illegal moves compared to the control. The results revealed

that those in the penalty group continued to make fewer illegal moves on the second solution

even when they were presented with a novel isomorph and no penalty. In addition, practice on

the first problem did not reduce legal or illegal moves on the second problem and indicated that

little or no learning was transferred from one isomorph to the other. These findings indicated that

something that was not problem specific, potentially some type of general problem solving

knowledge, was learned and transferred to the second solution attempt in the penalty group.

Experiments 2 and 3 asked participants to solve two problems and provided information as

to what was learned and transferred. In Experiment 2, participants solved the same problem

twice and made fewer illegal moves on the second solution attempt. In the third experiment

participants solved one problem and then solved a novel isomorph of that problem, but this did

not significantly reduce illegal moves on the second solution attempt. These findings lend

support to the idea that problem specific learning gained from practice, and not general problem

solving knowledge, was transferred to the second solution in Experiment 2 as there was lack of

transfer of leaming to a novel isomorph from practice alone in Experiment 3. However, in

Experiment 3 the illegal move reductions, which followed a penalty, were transferred to a novel

isomorph indicating that some type of general problem solving learning was acquired and









transferred to a novel problem as a result of being penalized. This general problem solving

knowledge was of great interest to Knowles and Delaney.

One main focus of the work by Knowles and Delaney was on understanding how the

penalty manipulation affected participants in illegal move reductions. The second stage of their

framework gave rise to the Caution Hypothesis, which stated that over time a problem-solver

would potentially be more likely to check candidate moves for legality, thus reducing the number

of illegal moves committed. The authors stated that this hypothesis seemed the most intuitively

valuable to test because it likely allowed for general learning of a tendency to check moves for

legality that could potentially be transferred to a novel problem. Knowles and Delaney conducted

a regression analysis on legal moves, illegal moves, and illegal moves considered (but not made).

The analysis revealed that legal moves accounted for significant variance (the more legal moves

made the more illegal moves made), lending support to the Generation Rate Hypothesis. The

analysis also revealed that once legal moves were statistically controlled the number of illegal

moves considered and correctly rej ected were inversely related to the number of illegal moves

made (the more illegal moves correctly rej ected the fewer illegal moves were made), lending

support to the Caution Hypothesis. Based on these results caution may have played a large role

in increasing problem solving efficiency and received additional attention in this dissertation

work.

In summary, Knowles and Delaney (2005) provided invaluable groundwork for the current

work presented here. Their work demonstrated that illegal moves could be reduced during a

problem solving episode when the cost of making a move was increased (penalty for making

illegal moves). Their results also indicated that problem specific learning was transferred to the

same problem (Experiment 2) and that general problem-solving learning gained through penalty









enforcement was transferred to a novel isomorph (Experiment 3). The questions that this

research led to, which were highlighted in the current work included: (1) could other aversive

manipulations besides penalty lead to illegal move reductions, (2) was an aversive manipulation

required for reductions or could positive manipulations yield the same results, (3) were

participants checking moves for legality more often or was their accuracy of those they checked

improving over time?

Caution Induction Framework (CIF (pronounced "chef"))

Knowles and Delaney's (2005) GCV framework proposed that a punishment for illegal

moves induced what they termed "caution," which was defined as an increased frequency of

checking moves for legality. However, caution may be only one of the factors that explained

why fewer illegal moves occurred after punishing them. I will therefore present the Caution

Induction Framnework (CIF) as a heuristic tool for laying out what behaviors might have

contributed to the effects of punishment on illegal moves. The CIF framework will provide a

basis for the empirical tests to follow, and lay the groundwork for a future program of research

on illegal move reduction.

A brief overview of the CIF framework will be provided first to facilitate a deeper base

understanding of the framework. The framework has three stages (Consequence, Involvement,

and Facilitation), as shown in Figure 1-1. The first stage (consequence) referred to the external

manipulations that affected problem-solving performance. The second stage (involvement)

referred to the result of the manipulations that occurred in the first stage. The assumption of the

framework was that the consequences in Stage 1 resulted in increased attention and thus the

problem-solver became increasingly involved in the problem. The third stage (facilitation)

specified what behaviors were altered to decrease illegal move commissions and that these

changes were a result of increased attention devoted to the problem in Stage 2.









The third stage of CIF begins with the indication that increased attention may have resulted

in increases in either the frequency or precision of certain behaviors. That is, problem-solvers

may have performed these behaviors more often or more accurately to reduce illegal moves,

respectively. The two main behaviors specified in CIF are evaluation and checking. These are

conceptually similar to the evaluation functions and illegal move filter from Jeffries et al. (1977).

Evaluation refers to the consideration of moves to determine which move has the highest

potential of assisting the problem-solver in advancing towards the final goal state. Evaluation

occurs prior to the selection of a candidate move and once a move is selected as the best potential

move it might or might not then be submitted to be checked using GCV. Checking refers to

testing a potential move for legality after the move has been selected, but prior to being executed.

The checking behavior used by problem-solvers incorporates the GCV framework from Knowles

and Delaney (2005).

Stage 1 (Consequence)

In Stage 1 (the consequence stage), the experimenter provided additional instructions or

consequences for performance on the problem. Previous research has demonstrated that a minor

punishment (rating words for pleasantness for 30 s) occurring immediately after every illegal

move subsequently reduced the rate of illegal moves (Knowles & Delaney, 2005). However, it

may be possible to obtain similar effects with less intrusive methods such as a threat or reward.

Threat and reward are included in CIF, but it was not yet known whether these factors had the

same effect on illegal move commissions as punishment. For now, threat and reward are

assumed to have the same effect as punishment, but this is further addressed after completion of

the experiments. CIF makes the assumption that any consequence of illegal moves in Stage 1 act

upon the problem-solver to influence Stage 2.










Stage 2 (Involvement)

In Stage 2 (the involvement phase), CIF assumes that problem-solvers have the ability to

perform better than they do and this is a result of the problem-solver not becoming fully

involved/engaged in the problem or the problem-solver not allotting enough attention to the

problem. The amount of attentional resources possessed by a problem-solver and how these

resources are distributed across the problem were of importance to understanding human

problem-solving. However, understanding the amount of available attention and how it may be

allotted to a problem remained outside the scope of this project. The assumption of CIF is that

the attention in Stage 2 is increased and that this results in increased activity in Stage 3 and

ultimately a reduction in illegal moves.


1Punishmen eiar

Stage 2
Increased Attention)






Checking r Evaluation



Stage 3 Generation Caution Verlfication se ece d?




Checked I Illegal
Yes No
(Knowles &. Delaney,




(Reduced Illegal Moves




Figure 1-1. The three-stage Caution Induction Framework (CIF) demonstrating the process of
reducing illegal moves in a problem solving episode.










Stage 3 (Facilitation)

Stage 3 (the facilitation phase) is a more complex stage compared to the rest of the

framework and relies on increased attention passed on from Stage 2. Due to the complexity of

Stage 3 the terms accuracy and frequency are first defined followed by subsections that explain

checking and evaluation. The checking subsection discusses the Generation, Caution,

Verifieation Framework from Knowles and Delaney (2005) and how increases in accuracy

and/or frequency may affect problem-solving performance. The evaluation subsection does not

focus on specific evaluation functions used by problem-solvers to reduce illegal moves-- it only

assumes that evaluation of moves occurs and that increasing the frequency and quality of these

evaluations indirectly reduces illegal moves. If people select moves randomly (with minimal

evaluation), then one might expect a greater frequency of illegal moves compared to people who

carefully consider what moves to select.

With the increased attention passed on from Stage 2 the problem-solver either increases the

accuracy or the frequency of either checking or evaluation behaviors, which reduce illegal

moves. An increase in accuracy means that the problem-solver performed this behavior with

fewer mistakes compared to previous attempts resulting in an increased likelihood of not

committing an illegal move. For example, if you check an illegal move for legality ten times, but

only correctly rej ect the illegal move three times (a rate of 30%) then increasing correct

rej sections to five out of the ten (50%) would increase the number of illegal moves rej ected from

three to five. An increase in frequency simply means that the problem-solver performed this

behavior more often. For example, if you only check three out of every ten moves for legality --

and you always correctly rej ect illegal checked moves -- then increasing frequency to checking

five out of every ten moves would increase the number of correct rej sections from three to five.









Either checking or evaluation behaviors can be increased in their accuracy or their

frequency, resulting in fewer illegal moves. It is also possible that any combination of increasing

the accuracy and/or frequency of either checking and/or evaluation simultaneously would result

in reducing illegal moves.

Checking

Checking refers to a problem-solvers' verifying whether the resulting state of a potential

move is valid or invalid according to the rules of the problem. If checking never occurs then the

consideration and selection of an illegal move would likely always result in the commission of

an illegal move. Knowles and Delaney (2005) provided us with a framework addressing how an

illegal move was correctly rej ected. To correctly rej ect an illegal move there were three steps that

must have occurred according to Knowles and Delaney. An illegal move must have come to

mind and been selected as a potential move, the problem-solver must have remembered to check

the move for legality, and the problem-solver must have accurately checked the resulting state of

the move and correctly rej ected the illegal move. This framework, which is referred to here as

the Generation, Caution, Verification (GCV) framework, was taken from Knowles and Delaney

(2005) and has been applied to CIF in an attempt to understand how illegal moves were reduced

following a penalty or another type of consequence (GCV is displayed in the box under the

"Checking" oval in CIF in Figure 1).

Illegal moves can be reduced if the problem-solver becomes more accurate during the

checking process. This increase in accuracy would have its effect in the third colored box of

GCV. Even though a problem-solver checks a move for legality that does not guarantee that the

move will be correctly rej ected. The problem-solver may miscalculate the future state and

erroneously think that the move is legal. Jeffries et al. (1977) proposed that calculation errors

were due to resource limitations. However, the author believed that this was more likely due to









insufficient resources being allotted to the process. With increased attention the problem-solver

may allot additional resources to the checking process to correctly calculate the resulting state,

resulting in more correct rej sections of illegal moves.

Problem-solvers may not only become more accurate at checking moves, but they may

check moves for legality more frequently. This increase in the frequency of checking would have

its effect in the second colored box of GCV. With minimal attention allotted to the problem,

problem-solvers may not bother to check the legality of moves at all. They may select a move

and then execute it because there is little cost if the move proved to be illegal. However, when

the cost of making illegal moves increases the problem-solver may attempt to verify the legality

of every move before it is executed. Even if the accuracy of checking does not increase the

number of illegal moves should still be reduced by the increase in the frequency of checking.

Evaluation

Stage 3 of CIF indicates that illegal moves could be reduced through evaluation; however,

the current work does not explore evaluation and made no assumptions as to how a problem-

solver evaluate moves. Determining the specific evaluation method used seems premature at this

point, but it has been included for framework completeness. Evaluation refers to problem-

solvers' assessment of a move to determine if the move they have selected reaches a selection

criterion specified by the problem-solver. How the criterion is set and how each move is

calculated is not fully known and is discussed in more detail below. Previously, in the Proposed

Causes: Move Selection section a simple move evaluation model was described (Jeffries et al.,

1977). CIF did not currently make any specific assumptions or proposals, other than those

previously discussed in the GCV framework or those otherwise explicitly stated herein, as to

how moves are evaluated. It assumes only that evaluation occurs and that the problem-solver's

goal is to select the best candidate move that will advance towards the goal. If the criterion is met









then the move is considered to be acceptable. If more than one of the considered moves meets or

exceeds the criterion then the potential move that exceeded the criterion by the greatest margin is

selected. The selected move is then either executed or checked for legality prior to being

committed.

As with checking, illegal move reductions could potentially occur through either

increasing the frequency or the accuracy of evaluation or both. With minimal attention devoted

to the problem, problem-solvers may select moves without first evaluating them. This would

likely result in the selection of several poor moves, which should be rej ected, but are executed.

Poor moves are defined as moves that do not bring the problem-solver closer to the goal state

and this includes moves that are illegal because they would never advance the problem-solver

towards the goal state. However, several illegal moves in the problem space may "appear" to

advance the problem-solver closer to the goal; so would increased frequency of evaluation result

in more illegal moves? This is a fair point and describes a real possibility; however without a

specific evaluation function being used it is not possible to determine the probability of this

occurring. However, although a specific move evaluation function is not assumed it would likely

be the case that prior experience would affect subsequent moves. Prior experience would allow

the problem-solver to rej ect illegal moves based on the premise that they have been previously

committed or previously considered and correctly rej ected. Therefore, with an increase in the

frequency of evaluation the problem-solver would likely learn from previous mistakes and avoid

additional illegal moves.

Problem-solvers may not only evaluate more frequently, but they may increase their

accuracy at evaluation. To increase the accuracy of evaluation the problem-solver may increase

the criterion a potential move must meet before it is selected. With a more stringent assessment









of candidate moves the problem-solver would likely reduce not only the rate of illegal moves,

but legal moves as well. One way illegal moves could be reduced would be if the increased

criterion required that a move be legal. Such a requirement would likely involve the checking of

candidate moves for legality as part of the evaluation process. Legal moves would be reduced

because the increased criterion should avoid, in most cases, revisiting previously visited states

and legal moves that take the problem-solver away from the goal. CIF's evaluation portion is not

specifically tested therefore these claims can not be supported and additional research is required

to determine the true effect of increasing the frequency and/or accuracy of evaluation on such

problems.

Stage 3 of CIF received the most attention as it represents the place in the problem-solving

episode where illegal moves are rej ected. As is displayed in the framework, these rejections

could occur through several different behaviors. The two primary behaviors are checking and

evaluation and increasing the accuracy or frequency of either of these behaviors would likely

result in illegal move rej sections. In addition, increases in evaluation could likely also influence

checking behaviors as they may be engaged in following the evaluation of a candidate move.

Although Stage 3 received the most attention and appears to be the most important it also proved

to be the most difficult to experimentally test as it is based upon untested assumptions of human

problem-solving.

CIF Summary

In summary, CIF is a 3-stage framework that attempts to provide a better understanding of

how consequences influence behavior to reduce illegal moves and increase problem solving

performance. In Stage 1 the framework is restricted to only a few potential testable

manipulations that may help to reduce illegal moves. Stage 2 is built upon the previous findings

that problem-solvers may have greater abilities and additional resources to avoid illegal moves









and plan ahead (Knowles & Delaney, 2005; Delaney, Ericsson, & Knowles, 2004; O'Hara &

Payne, 1998). The third stage looks at two primary behaviors that would likely result in illegal

move reductions, checking and evaluation. In addition, the third stage built upon the previous

work of Knowles and Delaney and incorporated their GCV framework.

According to CIF the evaluation of a move may involve checking, but checking is not

required for evaluation to result in a reduction in illegal moves and checking may occur and

reduce illegal moves independent of evaluation. If a participant evaluates a move, but does not

check the move for legality they may still benefit from the evaluation and select better moves

that are more likely legal, resulting in the reduction of illegal moves. A participant may also

select a move without evaluating that move, but still check the legality of the move and increase

the probability of identifying the move as illegal, ultimately rej ecting that move.

In conclusion, there are four questions of importance that needed to be addressed according

to CIF: (1) if threat and reward yield the same result as punishment in Stage 1 (reduced illegal

moves) (2) the amount of attention a problem-solver devotes to the problem after punishment,

threat, and reward in Stage 2 (also how this attention may be devoted to different areas of the

framework) (3) what actual behavior(s) problem-solvers engage in to reduce illegal moves in

Stage 3 and (4) determining the evaluation process a problem-solver engages in when

considering a move in Stage 3. This proj ect attempted to answer the first question to determine if

less aversive, less intrusive techniques could be substituted for punishment and yield similar

results (Experiments 1 & 2). The finding that threat and reward have the same effect as

punishment would support the idea that an increase in attention and not avoidance behavior was

responsible for illegal move reduction. The second question was outside the scope of this proj ect

and should only be considered after it is determined that an increase in attention occurred after









the manipulations in Stage 1. Attempting to determine the amount of attention devoted to the

problem without first verifying that increased attention was responsible for the change in

behavior would have been pointless. The third question was also addressed in this proj ect to

determine what behavior(s) problem-solvers engaged in after the manipulations in Stage 1 of CIF

(Experiment 3). Understanding how problem-solvers were able to reduce illegal moves would

enable us to discover better ways of increasing this behavior to improve problem solving

efficiency. The fourth question was also outside the scope of this proj ect and should only be

considered after it is determined that problem-solvers engaged in evaluation processes.

Attempting to determine the evaluation process used to select moves without verifying that an

evaluation process took place would have also been pointless.

Experiments

Three experiments were presented in an attempt to obtain a deeper understanding of the

role of caution in a problem-solving episode and how it changed based on the interaction

between the problem solver and the problem. These experiments aided in determining what was

required to induce caution and also helped determine what behavior was observed as a result of

being cautious. The first experiment examined whether a threat of punishment without any

punishment was sufficient to induce caution. The second experiment examined whether an

instruction to be cautious or a reward for being cautious without any penalty or threat of penalty

was sufficient to induce caution. Finally, the third experiment examined how the frequency and

accuracy of legality checking was altered as a result of increased cautiousness.









CHAPTER 2
EXPERIMENT 1

Since illegal moves have accounted for a significant portion of total moves in at least some

problems, knowing and understanding ways to reduce such moves would offer great benefits. In

previous studies participants reduced the number of illegal moves they made after being

penalized for illegal moves (Knowles & Delaney, 2005). However, these findings inspired the

question of what other manipulations may have also reduced illegal moves. In particular, what

other less aversive, less intrusive manipulations could have been implemented that would have

reduced illegal moves? Less aversive manipulations would likely minimize unnecessary effects

on the problem-solver and less intrusive manipulations would likely be easier to implement due

to minimizing interference. Such advantages would potentially have real world applications for

improving performance in business climates, the military, school, etc.

In the first experiment, participants in the two critical conditions were threatened that upon

completion of the problem they would be penalized for the illegal moves they committed during

their solution. One of the groups experienced the punishment prior to beginning the problem

while the other group was simply told of the penalty without experiencing the punishment.

Results from these two groups were compared to two additional groups, one that was penalized

after every illegal move and one that was not penalized or threatened for making illegal moves.

In addition, participants in all four conditions solved a second problem that had no penalty or

threat of penalty to determine what learned information was transferred to a novel isomorph.

Threat, punishment, and reward were manipulations made on the problem and this was

accounted for in Stage 1 of CIF (Figure 1-1). These manipulations also had a direct effect on

Stage 2 of the framework where they increased the amount of attention devoted to solving the

problem to deal with the so-called threat. People could have often been lackadaisical when they










approached a problem that had little or no consequence for performance that was not optimal.

This could have been thought of as having the motivation to do well being turned "off".

However, increasing the consequence or perceived consequence, as with threat, may have

increased the amount of attention a person decided to devote to solving the problem as was

shown in Stage 2 of CIF and this could have been thought of as turning motivation to the "on"

position. This increase in attention or turning "on" motivation would have in turn had an effect

on Stage 3 of the framework and would have ultimately resulted in the reduction of illegal

moves.

The author believed that if the threat was perceived as real it should have been enough to

influence the participant to engage in the problem more fully and should have resulted in an

increase in behavior that would have reduced illegal moves. How the different manipulations of

threat were perceived would determine how much attention was devoted to the problem and to

what degree illegal moves were reduced. If the participant was told the threat, but they did not

experience it before beginning the problem, they may have overestimated or underestimated the

severity of the threat. If the threat was overestimated then the effect in Stage 2 would have been

maximized and the participant may have reduced illegal moves to the degree that they were

reduced in the penalty condition or even to a greater degree. If the threat was underestimated

then the effect in Stage 2 would have been minimal and the reduction of illegal moves would

have been less than that in the penalty condition and may have been similar to the no-cost control

condition. Experiencing the threat before beginning the problem could have also displayed

similar effects compared to the threat group that did not experience the penalty. After completing

the initial penalty before beginning the problem the participant may have not perceived the

penalty as very costly and this would have likely minimized the effect on Stage 2 (little or no









increase in attention). In contrast, the participant may have perceived the penalty as very costly

and this likely would have maximized the effect on Stage 2 (large increase in attention).

The author predicted that a participant would have been accurate at predicting the cost of

the threat even when the threat was not experienced prior to beginning the problem. This would

have resulted in illegal move reductions that were greater than the no penalty condition and

similar to those of the penalty condition. However, it seemed likely that after performing the

penalty, but prior to beginning the problem, the participant may have become even more

attentive and may have showed illegal move reductions that were greater than those obtained in

the no experience threat condition and closer to those in the penalty condition. If illegal move

reductions for either of the critical conditions were greater than those in the no-cost condition

then this would have provided evidence supporting the claim that less intrusive methods were

capable of inducing caution.

Methods

Participants

Participants were recruited from General Psychology courses through the human subj ect

pool at the University of Florida. Participation was voluntary and each participant received

course credit for their participation. The duration of the experiment was approximately one hour

and each participant was awarded two experimental credits. One hundred participants

participated in this experiment; they were randomly assigned to one of the four groups. Twenty

participants were unable to solve both problems correctly within the time limit and their data was

therefore dropped from the analysis. Participants that were unable to solve both problems

correctly in the time allowed were replaced until there were 20 participants in each of the four

groups.










Problems

Hobbits & ores

The hobbits & orcs problem (which was also referred to as the missionaries and cannibals

or river-crossing problem) consisted of one boat and six travelers, three of which were hobbits

and three of which were orcs. All travelers began on the left bank of a river, which was near the

middle of the computer screen, and the boat began on the left bank at the bottom. The goal was

to move all six travelers to the right bank of the river using the boat. However, the rules stated

that the boat could only hold a maximum of two travelers at one time, and at least one traveler

was required in the boat for it to cross the river. The rules also stated that at no time could the

orcs outnumber the hobbits on either bank of the river because the orcs would then kill the

hobbits. A button was located on the display to reveal the rules. If at any time a participant forgot

one of the rules they could click on the button to look up any of the three rules. An example of

the display seen by participants for this problem was presented in Figure 2-1.

The problem was written in Microsoft Visual Basic and presented on a Gateway desktop

computer. Participants used the mouse to select icons representing the travelers to be moved and

then selected the boat to send the selected travelers to the other bank of the river. After a traveler

was selected it appeared at the bottom of the screen next to the boat. Clicking on a traveler a

second time removed him from the boat and placed him back on the bank in the middle of the

screen. If the participant added too many travelers to the boat, allowed the orcs to outnumber the

hobbits, attempted to move the boat with no travelers selected, or violate the rules in any other

way he/she was then notified via a message box and the move did not occur (problem description

taken from Knowles and Delaney, 2005).









I


Forget A Ruleo


Figure 2-1. The display seen by the participant for the hobbits and orcs problem where two
hobbits had been selected in the beginning state.

Titration

The titration problem was an isomorph of the hobbits and orcs problem in which the

participant was asked to remove unstable isotopes from a white beaker. The beaker contained six

isotopes, three blue and three orange. The goal of the problem was to remove all six isotopes

from the beaker using a dropper to extract them. The participant began by removing isotopes and

then alternated between adding and removing isotopes thereafter. However, the rules stated that

the dropper would hold only two isotopes at one time, and at least one isotope was required in

the dropper for the participant to add or remove isotopes. The rules also stated that if the number

of blue isotopes in the white beaker did not equal the number of orange isotopes in the white

beaker then the number of blue isotopes had to equal either zero (the minimum number possible)

or three (the maximum number possible) or the isotopes would become unstable and explode. If










at any time a participant forgot one of the rules, he or she could click on the button to display

three additional buttons, one for each of the three rules.

The problem was written in Microsoft Visual Basic and presented on a Gateway desktop.

A red "Remove" or "Add" button appeared on the screen indicating to the participant whether

he/she was to be removing or adding isotopes to or from the white beaker at each step. When the

"Remove" button was present the participant would use the computer mouse to click on the

isotopes in the white beaker, which would then appear in the dropper. Clicking on the "Remove"

button completed the move and made the "Remove" button disappear. The "Add" button then

appeared along with a large blue and orange beaker. Clicking the blue or orange beaker added

the appropriate color isotope to the dropper. Clicking the "Add" button completed the move and

emptied the isotopes from the dropper into the white beaker. The "Add" button and the two

colored beakers disappeared, and the "Remove" button appeared again.

The rules stated that it was impossible to have more than three blue or orange isotopes on

the screen at any time and this included the contents of the white beaker, the dropper and the

colored beakers. If a participant added too many isotopes to the dropper, attempted to remove or

add isotopes with no isotopes in the dropper, allowed the isotopes to become unstable, or

violated the rules in any other way he/she was notified via a message box, and the move did not

occur. An example of the display seen by participants for this problem was presented in Figure

2-2.

Design

Four groups of participants solved two problems. The major difference between the

groups occurred on the first problem when participants were instructed as to the consequences of

making an illegal move. Different instructions were provided to each group informing them of

the consequences of making illegal moves on the first problem. However, for the second










problem all four groups received the same instructions specifically, that there were no

consequences of making illegal moves. The second problem was included to examine transfer

effects. The four conditions differed on the first problem as follows:
























Figure 2-2. The display seen by the participant in the titration problem where one orange isotope
had been selected in the beginning state.

In the no-cost group, a message box appeared after each illegal move indicating that an

illegal move was made. The participant clicked an "OK" button to continue working on the

problem from the last legal state. A participant was not penalized or threatened in any way. This

group was identical to the no-cost group used in Knowles and Delaney (2005).

In the punishment group, a participant was informed before beginning the first problem

that after each illegal move the problem would be paused and they would have to rate the

pleasantness of words for 45 s. After every illegal move the participant was informed of the

illegal move and the screen then turned gray. Words appeared in the center of the screen and the

participant had to rate the words for pleasantness on a scale from 1 to 5 (1 being very unpleasant,









3 neutral and 5 very pleasant). After 45 s the screen returned to the problem and the participant

was allowed to continue working on the problem from the last legal state they had reached. This

group was identical to the punishment group used in Knowles and Delaney (2005).

In the threat group, a participant was instructed prior to beginning the first problem that for

every illegal move he/she made the penalty of rating words for pleasantness would accumulate

and he/she would have to complete the penalty after solving the problem. That is, for every

illegal move the participant would have to complete 45 s of rating words for pleasantness. A

participant in this condition was not penalized for making illegal moves and did not receive any

additional information after making an illegal move. After making an illegal move a message

box appeared stating that an illegal move was made and the participant was instructed to click

"OK" and was then allowed to continue working on the problem from the last legal state.

Finally, the experience condition was identical to the threat condition except that prior to

beginning the problem the participant completed the 45 s penalty phase once.

Procedure

Participants were tested individually and were randomly assigned to one of the four

conditions. Half the participants solved the hobbits and orcs problem followed by the titration

and the other half first solved the titration problem followed by the hobbits and orcs. To begin,

the participant read the cover story for the first problem aloud and then read the three problem

rules. The participant was asked to review the cover story and the rules and was instructed to

study and memorize the three rules. Once the participant was able to recite the three problem

rules to the experimenter without error the tutorial phase began.

During the tutorial phase, the participant was shown an example problem on the computer

with instructions explaining how the problem worked and how he/she was to navigate the

problem space. The participant was instructed to click on the "Forget a Rule?" button and then to









click on each individual rule button and asked to recite each rule aloud. The participant was

informed that the "Forget a Rule?" button was on the screen while he/she worked on the

problems so that the participant could check the rules at anytime if he/she so desire. Next, the

participant was instructed to move one of each character to the next position and then he/she was

instructed to move both characters back to the starting place so that the participant could

understand how the characters were moved. Then, the participant received instructions on

violating each of the three rules in succession and was asked to violate each rule within the

display to ensure that he/she fully understood what constituted an illegal move. The experimenter

checked to make sure that the participant had done this correctly.

Finally, the participant was instructed to make three legal moves on the practice problem.

The practice problem consisted of two hobbits and two orcs or two blue and two orange isotopes

(instead of three of each as in the experimental problem), depending on which problem he/she

solved first. At the end of the tutorial phase the participant was given the opportunity to ask

questions that he/she may have had. The main goal of the tutorial phase was to ensure that the

participant understood the problem, including the rules, the goal, and how to move the travelers.

The participant in the punishment condition was notified after the tutorial phase of the

penalty for violating Rule 3 and received instructions on how to complete the word-rating task.

The participant was penalized for Rule 3 violations only, there were no consequences for

violations of Rules 1 or 2. Participants in both the threat and experience conditions were

informed that they would have to complete the punishment upon completion of the problem,

once for every illegal move. Those in the experience condition completed a sample of the

punishment for 45 s before beginning the problem. In all conditions, testing began when the

participant clicked on the mouse to initiate the program. If a participant was unable to solve the










problem within the 20-minute time limit, that participant was then assisted in finishing the

problem and his or her data was not included in the analyses. The maximum solution time of 20

minutes was chosen to restrict the session length to one hour.

After completion of the first problem the participant was asked to judge how many illegal

moves he/she believed he/she had committed on the completed problem. These estimates were

used to calculate a difference score between the participant' s estimation and the actual number of

illegal moves. The difference scores were compared between the groups to determine if accuracy

differed. Participants in the threat and experience conditions were also informed that they would

not have to complete the penalty and would be asked if they believed that they were going to

receive the delayed punishment for making illegal moves. This question was asked to determine

if participants believed the instructions of a delayed punishment. The participant was then

instructed that he/she would now attempt to solve a different problem. There was no indication

or mention during the experiment that the two problems were similar in any way. The participant

was given a cover story for the second problem where he/she learned the rules and completed a

tutorial phase, just as the participant did for the first problem. Before beginning the second

problem all participants were instructed that there would be no penalty for illegal moves on the

second problem. If a participant was unable to solve the second problem within the 20-minute

time limit, that participant was then assisted in finishing the problem and his or her data from

both problems was not included in the analyses.

Results and Discussion

Move data. For this experiment a 4 x 2 x 2 mixed factorial Analysis of Variance

(ANOVA) was conducted: Group (No-Cost vs. Punishment vs. Threat vs. Experience) x Order

(Hobbits first vs. Titration first) x Solution (First vs. Second). Group and order were between-

subjects factors and solution was a within-subj ects factor. Illegal moves committed on the first









and second problem were of primary importance; however the number of legal moves and

average time to complete each move were also analyzed.

Based on previous findings obtained by Knowles and Delaney (2005) no main effect of

order, no main effect of solution, and no interaction for any of the factors for illegal moves, legal

moves, and time were predicted. This would support the assumption that the isomorphs did not

differ in difficulty, the order of presentation did not influence the results, and the results for the

first solution did not differ significantly from the second solution. It was possible that a

participant could transfer some general learning to the second problem and show a main effect of

solution with improved performance on the second solution, but such a result was not found in

Knowles and Delaney so such a result was not expected here either. It was also predicted that

legal moves and total time would not differ significantly between the four groups. However, it

was anticipated that a main effect of group would be obtained for illegal moves. A 4 x 2 x 2

mixed factorial ANOVA was conducted on legal moves made, average time to complete a move,

and illegal moves committed with Group and Order as between-subj ects factors and Solution as a

within subjects factor.

Legal move data. The results for legal moves made revealed no main effect of solution, F;

< 1, indicating that the number of legal moves on the first and second problem did not differ

statistically. There was no main effect of order, F(1,72) = 1.27, p > .05, M~SE = 244.42,

indicating that the number of legal moves made did not statistically differ based on which

problem the participant solved first. There was also no main effect of group, F < 1, indicating

that the number of legal moves made between the groups was not statistically different. These

results lend support to the prediction that legal moves remain relatively unaffected by the










manipulations made following illegal moves. Legal move averages and standard deviations for

each group by solution were presented in Table 2-1.

Table 2-1. Legal moves made on the first and second solutions were displayed by group and
solution and included mean and standard deviation (SD). All independent and
pairwise comparisons were not statistically significant between groups.
Group Mean SD
Solution 1 No-Cost 25.50 12.60
Threat 20.85 11.43
Experience 24.30 20.98
Punishment 22.40 8.63

Solution 2 No-Cost 23.05 22.56
Threat 20.70 10.04
Experience 25.50 16.95
Punishment 18.35 6.65
Move time data. The total time to complete the problem was divided by the total

number of moves for both the first and second solution for each participant in order to derive

average time per move in milliseconds (ms). The timer for the program that presented the

problems began in the experience group when participants first experienced the penalty and did

not pause or stop during the penalty in the punishment group. As a result of these continuous

timers adjustments were necessary to account for the timer being engaged while the participant

was not working on the problem.

The presentation of words in the penalty phase took 45 s -- that is, 3 s for each of the 15

words displayed. To account for the continuous timer 55 s was deducted from the total time to

complete the first problem in the experience group prior to calculating move time averages. The

55 s includes 45 s for the presentation of the words during the penalty phase and 10 s for the

brief instructions presented prior to the words during the penalty phase. In addition, 55 s was

deducted from the total time for the first problem in the punishment group for the first illegal

move and 45 s for each additional move because it was not required that the instructions be

repeated after they were initially understood.










The results revealed no main effect for order F < 1, indicating that average time per move

was not statistically different regardless of which problem was solved first. The main effect of

solution F(1,72) = 40.68, p < .001, M~SE = 21,099,533.55 was significant and the main effect of

group F(3,72) = 2.52, p = .064, M~SE = 85,590,859.42, approached significance. The Order x

Group interaction was not significant F < 1, the three-way interaction F(3,72) = 1.73, p > .05,

M~SE = 21,099,533.55 and the Solution x Order interaction were not significant F(3,72) = 2.94, p

=.091, M~SE = 21,099,533.55, and the Solution x Group interaction was significant F(3,72) =

3.06, p = .034, M~SE = 21,099,533.55. One potential explanation for the group and trend not

yielding significant results could have been attributed to a lack of power, therefore potential

explanations and follow-up analysis were discussed.

Independent samples t-tests conducted on the first solution revealed that those in the no-

cost group (M~= 14,256.50 milliseconds (ms)) took significantly less time to make moves when

compared to the experience (M~= 21,665.85 ms) and punishment groups (M~= 20,078.65 ms)

with t(3 8) = 2.99, p =.005 and t(3 8) = 2. 13, p < .05, respectively. Independent samples t-test also

revealed that the difference between the threat group (M~= 16,344.68 ms) and the experience

group, t(38) = 1.93, p = .062, approached significance. No additional differences were significant

on the first or second solution. Figure 2-3 displays average move times for each group and

solution.

The observed differences in move times may have indicated increased planning and/or

attention in the experience and punishment groups, which could have potentially supported the

CIF framework because increased checking and/or evaluation would have likely taken more

time. Participants in the threat condition may have made moves quicker than those in the

experience condition, although not statistically, which may have been due to a lack of power. In









addition, the average times for the threat group did not differ significantly from those in the

punishment group. These Eindings may have indicated that experiencing the penalty prior to

beginning the problem had a greater effect of increasing attention and resulted in either more

checking or greater evaluation of future moves. The lack of a significant difference between the

no-cost and threat groups may have indicated that because the threat group did not experience

the penalty the amount of increased attention generated from the threat may have been less than

that generated in the experience condition.


Experiment 1: Average Mlove Times

250001 -o-No-Cost
-m- Threat
20000 _Experience

15000 ---iPunishment



10000
Solution 1 Solution 2
Solution



Figure 2-3. Average moves times in milliseconds by group for the first and second solution.
Error bars represent standard errors.

An additional explanation for the greater move times in the punishment group could be

attributed to the penalties, which occurred during the problem solving episode. After completing

the penalty phase the participant may have had an adjustment period of re-engaging in the

problem once he/she was returned to the last legal move. This re-engaging may have included

recreating their previous move so that that specific illegal move could be avoided in the future or

it may have involved recalculating all moves from that state. However, the greater times in the

punishment group were more likely due to an increase in attention because similar increased









move times were found in the experience group where an adjustment period explanation was not

possible because the participant was not interrupted during the problem solution. In addition, the

punishment group did not statistically differ from the threat group. The threat group may have

also realized some increased attention, though not as much as the experience group. The lack of

significant differences on the second solution could have been attributed to a floor effect where

the participants were moving at a fairly efficient pace as a result of experience in completing the

first solution.

Another explanation for the time differences could have been attributed to an error in the

design of the computer program that was used. Because the timer was not initially set up

properly to automatically "pause" during the penalty phase the generalized time adjustment (55 s

for the initial penalty phase and 45 s for each subsequent penalty phase) may have led to

significant variance compared to the actual times the participant took for penalties. The

participant may have taken more or less than 10 s to learn the penalty rules in the experience and

punishment conditions during the first penalty phase. The participant may have also taken

additional time during each subsequent penalty in the punishment group before beginning the

word rating task (words were not generated during this task until the participant clicked a button,

a 45 s allotment for the penalty phase was based upon the assumption that the participant

immediately clicked the word generation button as soon as the penalty screen appeared).

Paired samples t-tests for each group comparing average times between Solution 1 and

Solution 2 were also conducted. This analysis revealed no difference between solutions for the

no-cost group, t (19) = 1.08, p > .05, but significant differences for all other groups. In the threat

group the participants were significantly faster on Solution 2 (M~= 11,777.38 ms) than on the

first solution (M~= 16,344.68 ms), t(19) = 2.95, p < .01. In the experience group participants were










quicker on Solution 2 (M~= 15,614.67 ms) compared to Solution 1 (M~= 21,665.86 ms), t(19) =

4. 13, p = .001. Finally, in the punishment group the second solutions (M~= 13,257.43 ms) were

also faster than the first solutions (M~= 20,078.65 ms), t(19) = 3.74, p = .001.

These results indicated that those participants that were penalized or were concerned with

future penalty took longer to make moves on the first solution compared to the second, which

would support the predictions of CIF that manipulations in Stage I would result in increased

attention in Stage 2. Participants in the no-cost group, however, performed with similar speed on

both problems, most likely because there was no penalty or concern of future penalties,

consistent with CIF's prediction that a lack of consequence in Stage I would result in little or no

increase in attention in Stage 2. Whether these results indicate increased planning, cautiousness,

or some other mechanism was not something that could have been concluded by these results

alone and required additional research. However, these results were promising and indicated that

the threat of punishment, whether experienced or not, was sufficient to produce results similar to

those observed when a penalty was administered after ever illegal move. These results lent

support to the CIF framework that penalty and the threat of a penalty resulted in similar move

times, potentially indicating an increase in attention.

[Illegal move data. The illegal move data revealed no main effect of solution, F < 1,

indicating that the participant made a similar number of illegal moves on the first and second

problem. There was no main effect of order, F < 1, indicating that the participant made a similar

number of illegal moves on each problem regardless of which problem was presented first. There

was, however, a main effect of group F(3,72) = 3.36, p < .05, M~SE = 19.96, indicating that

participants in the four groups did not make the same number of illegal moves. Independent

samples t-tests for the first solution revealed that the no-cost group made significantly more










illegal moves on the first solution than the threat t(3 8) = 2.37, p < .05, experience t(3 8) = 2.3 1, p

< .05, and punishment groups t(3 8) = 3.15, p < .005, which did not significantly differ from each

other. Independent samples t-tests conducted on the second solution revealed that the groups did

not differ from each other. Results are displayed below in Table 2-2 for illegal moves.

Although none of the two-way and three-way interactions were statistically significant it

appeared that the trends indicated a potential 2-way Solution x Group interaction. On Solution 1,

the no-cost group made significantly more illegal moves compared to the rest of the groups

which did not differ. Participants in the punishment group (M~= 2.60) made approximately one

fewer illegal moves compared to the threat (M~= 3.40) and experience (M~= 3.65) groups, whose

participants made a similar number of illegal moves. On the second solution, the number of

illegal moves made by participants did not differ significantly between the four groups.

However, participants in the threat (M~= 2.70) and punishment (M~= 2.65) groups made

approximately two fewer illegal moves compared to participants in the no-cost (M~= 4.20) and

experience (M~= 4.65) groups. An obvious explanation for these differences, which were not

statistically significant, was that the experiment lacked the power to detect the effects with 20

participants in each of the four groups. Because these effects were potentially not detected due to

a lack of power it did not seem unreasonable to expand upon these results.

Table 2-2. Illegal moves made on the first and second solutions were displayed by group and
solution and included the means and standard deviations (SD). Superscript 3iS
significant at p < .005, and all others significant at p < .05.
Group Mean SD
Solution 1 No-Cost 6.85 1235.82
Threat 3.40 2.95
Experience 3.65 2 2.16
Punishment 2.60 3 1.60

Solution 2 No-Cost 4.20 4.41
Threat 2.70 2.25
Experience 4.65 7.65
Punishment 2.65 2.08











A potential explanation for the trend that the punishment group made fewer illegal moves

compared to the threat and experience groups on the first solution was that as these two groups

made illegal moves and were not penalized the threat of penalty could have "worn off' over the

duration of the solution. The participant could have started off avoiding illegal moves, but as

there was no immediate penalty illegal moves could have increased as time increased. A second

explanation for the trend could have been that the threat in these two groups was not as impactful

as the immediate penalty in the punishment group and thus illegal moves rates were higher over

the total duration of the problem. Trends on the second solution attempt were less clear with

fewer potential explanations. The threat and experience groups were very similar except that

those in the experience group completed the penalty phase once before beginning. Despite the

overwhelming similarities between the two groups, including the similar number of illegal

moves made on the first solution, participants in the experience group made approximately two

additional illegal moves on the second solution. Due to the fact that the only difference between

the groups was the completion of the penalty prior to beginning in the experience group, this was

likely responsible for the trend. However, it remained unclear why completing the penalty once

before beginning the problem on the first solution attempt would result in approximately two

additional illegal moves on the second solution. Trends on the first and second solution were not

significant and additional research would be required to determine if these trends would be

significant with additional power and the cause of such results.

Paired samples t-tests revealed no differences between the first and second solution for the

threat, experience, and punishment groups, all t 's < 1. The paired samples t-test for the no-cost

group was also not significant, but showed a trend towards a decrease in illegal moves on the

second problem, t(19) = 1.82, p = .084. As was previously stated a lack of power may have










potentially been responsible for some trends not being statistically significant. On the second

solution (M~= 4.20) participants in the no-cost group made more than two fewer illegal moves on

average when compared to first solution (M~= 6.85) performance. This appeared to be a large

reduction in illegal moves, relatively speaking, but the trend was not statistically significant. This

trend could have potentially been due to some type of general problem-solving learning on the

first solution that allowed for transfer and illegal move reductions on the second solution. The

illegal move changes from the first to second solution for the other three groups was also not

significant, but the changes were smaller compared the no-cost group and seemed to be more in

line with the statistical results. Because the t-tests between the first and second solutions for the

other three groups were not significant this lent support to the idea that illegal move reductions

on the second solution were sustained even after the penalty or threat of penalty had been

removed.

The results in this section largely replicated and supported the results found in Knowles &

Delaney (2005). The main effect of both solution and order were not significant and the no-cost

group made significantly more illegal moves on the first solution compared to the punishment

group. The punishment group, along with the threat group, did not show increases of illegal

moves from the first solution to the second, potentially indicating sustained benefits through

continued reductions in illegal moves. The experience group did not statistically show a change

in illegal moves from the first to second solution, but the average did increase by one illegal

move, 3.65 to 4.65, potentially indicating a lack of power to detect the change. In addition,

participants in the threat and experience groups made significantly fewer illegal moves compared

to the no-cost group and they did not significantly differ from the punishment group. These

results provided support for CIF's assumptions: less intrusive methods of increasing attention









and inducing caution were possible and resulted in similar illegal move reductions compared to

the punishment group.









CHAPTER 3
EXPERIMENT 2

The purpose of the second experiment was to determine whether or not a punishment or

threat of punishment was necessary to induce cautiousness. The two critical groups in this

experiment received no negative consequences for illegal moves. Instead one group was

instructed to "do their best" and to concentrate on avoiding illegal moves and the other group

was rewarded for avoiding illegal moves and performing efficiently (performing efficiently was

explained later in this chapter). Similar to Experiment 1, both of these groups were compared to

a group that was punished for illegal moves and a group that received no punishment. In this

experiment there were no negative consequences or threat of negative consequences for the

illegal moves in the two critical conditions. Therefore, the results should have provided insight

into whether or not negative consequences were necessary to induce caution.

As in Experiment 1, Stage I was manipulated, but this time the consequences were a

reward or motivating instruction (as opposed to the threats used in Experiment 1). This

manipulation would have then affected Stage 2, where motivation would have been switched

"on" and attention would have been increased according to CIF. This increase in attention would

have ultimately resulted in a reduction in illegal moves. The author predicted that there would

have been fewer illegal moves in the motivation and reward groups compared with the no-cost

condition that was not punished and did not receive additional instructions or reward. It was also

predicted that the reward condition would likely reduce illegal moves to a greater degree than the

motivating instruction. If a reward or motivating instruction was able to induce cautiousness then

this would have supported the claim of the framework; manipulations in Stage 1 influenced

people to increase their attention and this was what caused a reduction in illegal moves, not

punishment or fear of punishment, but an increase in attention to the problem.









Methods


Participants

The process for recruiting participants was the same as the first experiment. In Experiment

2, 108 participants took part in the study; each was randomly assigned to one of the four groups.

A total of 28 participants were unable to solve both problems correctly within the time limit and

their data was therefore dropped from the analysis. Participants were that were unable to solve

the problems were replaced until there were 20 participants in each of the four groups.

Problems

The same problems were used as in Experiment 1 -- hobbits & orcs and titration.

Design

Four groups were used in this experiment: no-cost, punishment, motivation, and a reward

group. As in Experiment 1, the groups differed in the consequences of making illegal moves on

the first of two problems. All groups received identical instructions when they solved the second

problem specifically, they were informed that there was no consequence or reward for avoiding

illegal moves. The purpose of Eixing the instructions for all four groups on the second problem

was to examine transfer effects. The manipulations on the first problem were as follows:

The no-cost group was identical to that used in Experiment 1, and received no special

instructions, rewards, or penalties for making illegal moves.

The punishment group differed only slightly from the Experiment 1 punishment group.

The punishment (rating the words for pleasantness) occurred on a Eixed interval schedule rather

than immediately after every illegal move. The participant had to complete the penalty every 30 s

if a penalty was made in that time frame. When an illegal move was made, the participant saw a

message box stating that an illegal move was made and then they were allowed to continue

working on the problem from the last legal state. Once the 30 s time interval was complete the









screen then turned gray and the participant was instructed that he/she recently committed an

illegal move and he/she would be instructed to complete the penalty. If an illegal move was not

made in the 30 s interval then the participant received no information and continued working on

the problem until the end of an interval in which an illegal move was committed, the problem

was solved, or time expired. Administering the penalty at a fixed interval instead of immediately

after every illegal move was done to maximize the similarity between the punishment and reward

conditions.

In the motivation condition, prior to starting the first problem, the participant was

instructed that the main focus of the experiment was to assess his/her ability to avoid illegal

moves. The participant was informed to "do their best" to avoid illegal moves in an attempt to

solve the problem with the fewest number of illegal moves possible. No reward, penalty, or

additional instructions were given to the participant in this group.

Finally, in the reward condition, prior to starting the first problem the participant was

asked for his/her preference of Skittles or M&M candies. The participant was then instructed that

he/she would be rewarded with the candy of choice for avoiding illegal moves and solving the

problem efficiently. The participant was told that the reward would be based primarily on the

avoidance of illegal moves, but to maximize the reward he/she should try to complete the

problem in a timely manner and in the minimum number of moves possible. A dish was next to

the participant and the experimenter deposited one piece of candy into the dish every 30 s if no

illegal moves were made in that time frame and the participant reached a new legal state in that

time frame that had not been previously visited during the problem solving episode. If in the 30 s

time frame the participant reached a new legal state and then back-tracked to a previously visited

state the participant was still rewarded with candy as long as an illegal move was not made in









that interval. In addition, the participant received an additional candy for completion of the

problem and an additional piece of candy for every 30 s interval remaining before six minutes

(e.g. three extra candies for completing the problem before 4:30 because there were three full 30

s intervals remaining before six minutes was reached). Six minutes was chosen based on the

average time to complete the problem obtained in Knowles and Delaney (2005).

Procedure

The procedures followed that of the first experiment with the exception that those in the

motivation and reward condition received additional instructions before beginning the first

problem. After completing the first problem, participants in the motivation and reward conditions

were asked if they believed they were going to receive punishment for making illegal moves. As

in Experiment 1, participants completed the second problem under no-cost instructions.

Results and Discussion

Move data. A 4 x 2 x 2 mixed factorial ANOVA was conducted: Group (No-Cost vs.

Punishment vs. Motivation vs. Reward) x Order (Hobbits first vs. Titration first) x Solution (First

vs. Second). Group and order were between-subj ects factors and solution was a within-subj ects

factor. As in the first experiment, illegal moves committed on the first and second problem were

of primary importance, but number of legal moves and time per move were also analyzed.

Based on previous findings obtained by Knowles and Delaney (2005) it was anticipated

that there would be no main effect of order, no main effect of solution, and no interaction for any

of the factors. This would indicate that the isomorphs did not differ in difficulty, the order of

presentation did not influence the results, and the results for the first solution did not differ

significantly from the second solution. It was also anticipated that legal moves and average time

per move would not differ significantly between the four groups. However, it was predicted that

a main effect of group would be obtained for illegal moves.










Legal move data. The results for legal moves revealed no main effect for solution F < 1 or

order F(1,72) =1.20, p > .05, M~SE = 286. 19. Although the result of the analysis on group

revealed no main effect F(3,72) = 2.42, p = .073, M~SE = 286. 19, it did approach significance.

The trend for this analysis was for participants in the reward group to make the most legal moves

(M~= 32.03) and for participants in the no-cost (M~= 24.63), motivation (M~= 23.3 5), and

punishment (M~= 23.40) groups to make similar numbers of legal moves. This trend, though not

significant, could have potentially been due to participants adapting a strategy of trying to

maximize rewards by navigating through legal states while avoiding illegal moves, thus

increasing legal moves. Previous results have indicated that legal moves have remained fairly

stable during experimental manipulations so this trend was somewhat surprising.

Move time data. The procedures for calculating average times per move were the same as

those used in Experiment 1. The punishment group was adjusted 55 s for the first penalty and

45 s for each subsequent penalty. Penalties occurred at a fixed interval so the number of penalties

may have been fewer than the number of illegal moves. If more than one illegal move occurred

in a given interval, time was adjusted per penalty phase. The results for move time data revealed

no main effect for solution or order, both F 's < 1. However, the main effect of group was

significant, F(3,72) = 2.77, p < .05, M~SE = 29,883,813.92, the Solution x Order interaction was

significant, F(1,72) = 4.70, p < .05, M~SE = 12,203,979. 10, and the Order x Group interaction

approached significance, F(3,72) = 2.66, p = .054, M~SE = 29,883,813.92. The means and

standard deviations for each group by solution were displayed in Table 3-1.

Independent samples t-tests revealed that the punishment group took longer to solve the

first problem compared to the reward and motivation groups, as displayed in Table 3-1. These

results could have been attributed to a more "carefree" mentality in the reward and motivation










condition due to the motivating instructions or potential reward of candy and potentially

increased planning in the punishment group as participants attempted to avoid illegal moves. In

addition, the penalty in the punishment group occurred at a fixed interval after an illegal move

was made in that interval. So, an alternative explanation could have been that participants were

waiting or preparing as they knew the penalty phase could be initiated at any time and this

resulted in increased move times. A final alternative proposed in the first experiment was that

after completing a penalty phase participants may have required a re-adjustment period when

returning to the problem and this would have increased move times.

Table 3-1. Means and standard deviations (SD) by group and solution for average times per
move were presented in milliseconds. Superscripts 1, 2, 3 WeTO Significant at p < .05.
Group Mean SD
Solution 1 No-Cost 14124.72 5055.92
Motivation 11593.931 3812.97
Rewa rd 12083.44 2 3480.25
Punishment 14938.61 1,2 4186.00

Solution 2 No-Cost 13086.21 4252.88
Motivation 12892.55 5652.86
Rewa rd 1 0961 .40 3 3026.99
Punishment 14578.37 3 6916.07
On the second solution, the reward group remained significantly quicker than the

punishment group, but the difference between motivation and punishment was no longer

significant. It was possible that participants in the punishment group were still being cautious

and taking additional time planning or preparing for a penalty that could have appeared at any

moment. Participants in the reward condition may have taken the problem less seriously and

been less cautious due to the fact that they were administered candy on the first solution. The

distribution of candy on the first solution may have made the problem seem like more of a game

than an experiment resulting in less attention, planning, and/or caution.

The order by group interaction approached significance and post hoc analysis were

conducted as the study design may have lacked the power to detect the effect. An independent










samples t-test analysis revealed that participants in the no-cost group may have had longer

average move times when solving the titration problem first (M~= 15,325.77) compared to

participants that solved the hobbits and orcs problem first followed by the titration problem (M~=

11,885.17), t(18) = 2.07, p = .053 as this analysis approached significance. Results from the

independent samples t-tests for the motivation t(18) = 1.18, p > .05, reward t(18) = 1.67, p > .05,

and punishment groups t(18) = 1.15, p > .05, revealed non-significant results. One potential

explanation for this trend could was the idea that those in the no-cost group had no consequences

for performance and therefore no reason to devote additional attention towards the problem. This

lack of additional resources toward the problem may have resulted in more difficulty encoding

the titration problem when it was presented first, as it may have appeared more abstract and

provided less information to the participants (participants were unable to see the contents of the

destination beakers in the titration problem whereas participants could always see both banks of

the river in the hobbits and orcs problem). Average move times for the different groups by order

were presented in Figure 3-1.


Experiment 2: Average Move Times

18000
16000-

S14000 O-Titration
S12000 Titration-HO
10000-
8000
No-Cost Motivation Reward Punishment
Group


Figure 3-1. Average time per move for each of the four groups was displayed for participants
that solved the hobbits and orcs (HO) problem first and those that solved the titration
problem first. The error bars represented standard error.










[Illegal move data. The analysis of the illegal moves made revealed no main effect of

solution, F(1,72) = 2.77, p = .099, M~SE = 10.99, no main effect of order, F < 1, and no main

effect of group, F(3,72) = 1.86, p = .321, M~SE = 19.02. In addition, the Solution x Order and

Group x Order interactions were not significant, both Fs < 1. Neither the Group x Solution

interaction, F(3,72) = 2.23, p = .092, M~SE = 10.99, nor the three-way interaction were

significant, F(3,72) = 1.57, p = .205, M~SE = 10.99. These results were surprising given the

previous findings from Knowles and Delaney (2005) and Experiment 1, in which the punishment

group made significantly fewer illegal moves compared to the no-cost group on the first solution.

Table 3-2 provides the means and standard deviations for illegal moves by group and solution.

One potential explanation for the absence of significant results could have been due to a

lack of power. For this reason, independent samples t-tests were conducted between the groups

for illegal moves on the first solution attempt. The independent samples t-tests comparing no-

cost and motivation, t(38) = 1.98, p = .056, no-cost and reward, t < 1, no-cost and punishment,

t(3 8) = 1.24, p = .222, motivation and reward, t(3 8) = 2.00, p = .053, motivation and punishment,

t(38) = 1.01, p = .320, and reward and punishment, t(38) = 1.55, p = .129, were not significant.

The punishment group has typically made fewer illegal moves compared to the no-cost group,

but this comparison was not significant. Also of interest were the two experimental groups

compared to the no-cost group, which revealed a trend approaching significance comparing the

motivation and no-cost groups. Although not significantly, the motivation group made the fewest

number of illegal moves (M~= 3.40) out of all four groups. It may have been possible that with

additional power both the punishment and motivation groups would have made significantly

fewer illegal moves compared to the no-cost group.










The most unexpected Einding in this experiment was the lack of a significant difference in

illegal moves between the no-cost and punishment group. One potential explanation could have

been that since the penalty was set at a Eixed interval, the number of penalties could have been

(and often were) fewer than the number of illegal moves committed. For example, if the

participant committed an illegal move at the beginning, the middle and the end of a 30 s interval

then three illegal moves were made, but only one penalty was administered at the end of the 30 s

time interval. Being penalized less often compared to previous experiments could have reduced

the effect, resulting in non-signifieant differences between the groups for illegal moves.

Table 3-2. Means and standard deviations (SD) of illegal moves by group and solution for illegal
moves were presented.
Group Mean SD
Solution 1 No-Cost 5.40 3.42
Motivation 3.40 2.96
Rewa rd 6.85 7.13
Punishment 4.25 2.34

Solution 2 No-Cost 3.75 2.55
Motivation 3.95 2.63
Rewa rd 4.15 3.10
Punishment 4.55 4.43
Also of interest and importance was the lack of a significant difference between the no-

cost group and the two experimental conditions. One explanation for the lack of a reduction in

illegal moves in the reward condition could have been attributed to the reward itself. It was

possible that candy was not very rewarding to the participants and significant improvements may

have been obtained with a more valuable reward, like money. An additional possibility was that

the reward rules awarded additional or bonus candy at the completion of the problem for

efficient performance and this may have had little or no effect on problem solving behavior.

Awarding candy after the problem had been completed may not have influenced participants to

avoid illegal moves during the problem. In addition, the lack of a reduction of illegal moves in









the motivation condition could have indicated that the instructions were not effective in

increasing attention.

Given that this experiment was run last out of the three experiments and was run at the end

of the semester this may have resulted in less motivated students, which may have affected the

results. Students who had waited until the end of the semester to volunteer and complete their

experimental credits were likely less motivated to complete the credits and may have been more

resistant to the experimental manipulations. Another explanation for the results could have been

that an adverse stimulus or threat of an adverse stimulus was required to increase attention and

reduce illegal moves. Thus, participants in the motivation and reward conditions would not have

been influenced to increase attention and reduce illegal moves. Unfortunately, the lack of typical

effects in this experiment could not be clearly understood from the collected data and further

research is required to answer these questions.









CHAPTER 4
EXPERIMENT 3

The previous two experiments concentrated on finding ways to aid participants in

becoming more attentive, through threat or reward, so that they could perform more efficiently

on the problem. Once a participant had become motivated and decided to allot additional

attention to the problem, what did this increased attention do to reduce illegal moves? In

previous research Knowles and Delaney (2005) proposed that punishment induced caution and

that caution was checking a move against the rules for legality so that illegal moves could be

rej ected and avoided. Experiment 3 instructed participants to engage in this checking behavior so

that illegal moves could be avoided and so that the result of such behavior could be assessed.

In the critical conditions of Experiment 3, participants were asked to check the legality of

each move after selecting it, but before executing it. One group was asked to perform this check

while thinking aloud so that it could be monitored by the experimenter and the other group was

asked to perform the check silently. Participants received training in thinking aloud and were

asked to verbalize any thoughts they had while working on the problem. The training and

instructions were similar to those proposed by Ericsson and Simon (1993). The silent check

group was used for comparison to the aloud group to ensure that the aloud task did not affect

performance. Based on previous research, the illegal move rates of the silent and aloud check

groups should not have differed; both should have shown significant reductions in illegal moves

and all other data between the two groups, except for time per move, should have been similar

(Ericsson and Simon, 1993; Knowles and Delaney, 2005).

Instructing a participant to check a move for legality would have likely increased the

frequency of this checking behavior. This would have ultimately influenced participants to rej ect

more illegal moves compared to those not instructed to check. Asking participants to check









moves for legality would have affected Stage 2 of CIF because participants would have become

more attentive with the requirement of checking each move for legality. Platt and Griggs (1993)

found that when participants were asked to explain their choices in Wason's 4-card selection task

they obtained the highest success rates every reported for this difficult task. These results

appeared to support the idea that increasing engagement in a task, whether through requiring

explanations for behavior or through possibly a check of legality, can yield significant

improvements in performance. Instructing participants to check moves would also have a direct

effect on Stage 3 of the framework where checking behavior was directly manipulated.

Instructing participants to check the legality of each move before it was made allowed CIF

to make clear predictions. CIF predicted that the instruction to check would have increased

attention and decreased the number of illegal moves made compared to the no-cost group. The

prediction was that illegal move reductions would have achieved rates similar to the punishment

group in the first experiment and previous findings of Knowles and Delaney (2005).

In addition, a measure of working memory using the operation span (OSPAN) task was

also administered at the end of the experiment (Kane, Bleckley, Conway, & Engle, 2001; Turner

& Engle, 1989). The reason for obtaining working memory scores was to determine the extent to

which resources play a role in human problem solving. If the resource limitation hypothesis

(Jeffries et al., 1977) was correct then working memory should significantly correlate with the

number of illegal moves a participant makes. The OSPAN task assessed working memory by

presenting equations and words on a computer screen. The participant had to verify the equations

while maintaining the words in memory. To assess the participant' s working memory span the

words had to be recalled in the order that they were presented. This task was described in more

detail in the procedure section.










The purpose of measuring OSPAN was to examine the role played by working memory

resources in problem solving, which has been proposed as a cause for the selection of illegal

moves (Jeffries et al., 1977). Those with low working memory spans may have had difficulty

calculating future states or remembering to check each move for legality contributing to the

difficulty of the problem. Previous research on these types of problems did not reveal a

significant relationship between illegal or legal moves made and working memory (Knowles and

Delaney, 2005). However, the evidence here may prove otherwise since participants were

instructed to check aloud. This manipulation allowed the experimenter to verify that the

participant actually engaged in this behavior and enabled the experimenter to assess the rate at

which they checked and the accuracy of this checking. The role of working memory in problem

solving was outside the scope of CIF; therefore the framework was unable to make a prediction

as to the relationship between problem solving and working memory. The author believed that

such factors contributed to problem solving and predicted a negative correlation between illegal

moves and working memory and a negative correlation between legal moves and working

memory.

Methods

Participants

The process for recruiting participants was the same as the first two experiments. In

Experiment 3, 103 participants took part in the study; each was randomly assigned to one of the

four conditions. Of these, 23 participants were unable to solve both problems correctly within the

time limit and their data was therefore dropped from the analysis. Participants that were unable

to solve the problems were replaced until there were 20 participants in each of the four groups.









Problems

The problems used in Experiment 3 were the same hobbits & orcs and titration problems as

those used in the first two experiments.

Design

In this experiment there were four groups: no-cost group, a punishment group, a check

group, and an aloud group. As in Experiments 1 and 2, the groups differed as to the instructions

they received on the first problem and the instructions on the second problem did not differ

between the groups so that transfer effects could be examined. All groups solved the second

problem silently without punishment or instructions to check each move.

The punishment and no-cost groups were identical to those in Experiment 1. The no-cost

group received no special instructions, rewards, or penalties for making or avoiding illegal

moves. The punishment group penalized participants immediately after every illegal move by

having them rate the pleasantness of words for 45 s just as participants did in the first

experiment.

The check condition instructed the participant that after selecting a move he/she would

check the move for legality before executing that move. A button was located on the display so

that after checking the move but before executing it, the participant would click the button to

indicate that they had checked the move and that he/she believed it to be a legal move. This

button was primarily on the display so the experimenter could verify that those in the silent

condition were performing the check. In the aloud condition the button remained on the display

so that the similarity between the two conditions was maximized. Participants in the aloud

condition were also instructed to click on the check button before making a move to indicate that

the move had been checked. In addition, the computer program tracked when and how often a

participant forgot to click the button to indicate that they had checked the move for legality.










A participant in the aloud condition received instructions and training on thinking aloud

and was then asked to verbalize his/her thoughts as they worked on the problem (instructions for

thinking aloud were provided in the Appendix). These procedures were similar to the instructions

proposed by Ericsson & Simon (1993). If a participant was silent for more than 3 s or speaking

too softly he/she was reminded to keep talking. The experimenter attended to the participant' s

verbal protocols in an attempt to ensure that the participant checked each move for legality

before executing that move. As in the check condition the participant was asked to click on the

"Check" button before finishing his/her move to indicate that he/she had checked the move and

believed it to be a legal move. If the participant did not check a move for legality he/she was

reminded to continue doing so and the computer tracked each checked and non-checked move.

The participant was asked to speak into a microphone while solving the first problem and verbal

protocols were recorded on a mini-disc recorder.

Procedure

The procedures followed those of Experiment 1, except that participants in the aloud

condition received instructions and training at the beginning of the tutorial phase, to think aloud

as they solved the problem. The participant was given instructions for thinking aloud similar to

those suggested by Ericsson and Simon (1993) and was asked to verbalize his/her thoughts by

saying whatever came into his/her head. The participant was then instructed to think aloud as

he/she imagined him/herself walking through a house that he/she was very familiar with and to

count the number of windows in the house. The experimenter stopped this task when the

participant was able to provide descriptors of the interior of the house, which demonstrated

understanding of thinking aloud. If the participant was too vague or did not understand the task

then the experimenter provided an example of describing the interior of a house while counting

the windows and then asked the participant to complete the task. This task was very brief and










typically did not take more than 30 s; every participant in the think aloud group was able to

complete this task. During the tutorial phase, participants in the aloud group were asked to work

on the practice problem while thinking aloud and to check each move for legality. The other

three groups completed the tutorial phase in a fashion similar to the participants in Experiment 1.

As in the other experiments, all participants solved the second problem silently and without

checking. After completion of both problems, participants in all groups completed the operation

span (OSPAN) task to assess working memory capacity (Kane, Bleckley, Conway, & Engle,

2001; Turner & Engle, 1989).

The OSPAN task was a task used to assess a person's ability to maintain information in

memory while completing another task, which evaluated a person's working memory span. The

task presented a simple mathematical equation followed by a word (e.g., (9 / 3) 1 = 2 CONE)

in a Power Point presentation on a computer screen. The participant was to read the equation

aloud, verify whether or not the equation was correct (some equations were true and some were

false), and then say aloud the word that followed the equation. Once this was completed the

experimenter immediately presented the next slide on the computer screen with a new equation

and word. Once the new slide appeared the participant was to begin reading the equation aloud

immediately, verify it and then read the word. Anywhere between two and five equations/words

were presented and then the participant saw a colored screen with "???" at the completion of a

series (all the words presented since the last colored screen). This was the participant's cue to

recall all the words he/she had seen in that series on a piece of paper in the order in which they

were presented. There were a total of 12 series (three series each with two words, three words,

four words, and five words) and they were randomly presented so that the participant did not

know how many words would be in that series until that series was over. The participant









received one point for every correct word, although the participant only received points if all the

words in the series were recalled correctly in the correct order (e.g., no points given if all four

words recalled, but the order of Words 1 and 2 were switched; four points given if all words

recalled correctly in the correct order).

Results and Discussion

Move data. For this experiment a 4 x 2 x 2 mixed factorial ANOVA was conducted:

Group (No-Cost vs. Punishment vs. Check vs. Aloud) x Order (Hobbits first vs. Titration first) x

Solution (First vs. Second). Group and order were between-subjects factors and solution was a

within-subj ects factor. As in the first two experiments, illegal moves committed on the first and

second problem were of primary importance, but number of legal moves and time per move were

also analyzed.

As in the first two experiments it was anticipated that there would be no main effect of

order, no main effect of solution, and no interaction for any of the factors indicating that the

isomorphs did not differ in difficulty, the order of presentation did not influence the results, and

the results for the first solution did not differ significantly from the second solution. It was also

anticipated that legal moves and average time per move would not differ significantly between

the four groups. Those participants in the check and aloud conditions could have had slightly

higher times as a result of the instruction to check and the process of verbalizing thoughts in the

aloud condition, but this should not have affected performance otherwise. However, it was

anticipated that a main effect of group would be obtained for illegal moves.

Legal move data. An analysis of legal moves revealed no main effect of solution, F < 1,

no main effect of order, F(1,72) = 1.3 1, p > .05, M~SE = 228.21, and no main effect of group,

F(3,72) = 1.07, p > .05, M~SE = 228.21. These results indicated that, regardless of which problem

was solved first, the number of legal moves did not differ between the first and second solution










or between the four groups. These results replicated the Eindings of the first two experiments and

support the claim that the manipulations have little or no effect on the legal moves within the

problems.

Move time data. Procedures for calculating average time per move were the same as

those used in the first two experiments, and the adjustments made to the punishment group for

illegal move penalties were the same as those imposed on data from Experiment 1. Analysis

revealed no main effect of order, F(1,72) = 2.60, p > .05, M~SE = 50,735,853.35. However, a

main effect of solution was found, F(1,72) = 42. 10, p < .001, M~SE = 37,739,536.72, which

indicated that participants took less time making moves on the second solution (M~= 12,645.75

ms) compared to the first solution (M~= 18,948.23). A main effect of group, F(3,72) = 3.97, p =

.011i, MSE = 50,73 5,853.3 5 was present, indicating that the groups did not make moves at the

same rate. The results also revealed a two-way Solution x Group interaction, F(3,72) = 5.64, p =

.002, = 37,739,536.72. As indicated in Figure 4-1 some of the times decreased from the first

solution to the second and some did not. Paired samples t-tests revealed no change for the no-

cost group from the first solution to the second, t(19) = 1.65, p > .05. The difference for the

check group, t(19) = 2.27, p < .05, the aloud group, t(19) = 9.20, p < .001, and the punishment

group, t(19) = 2. 19, p < .05, showed significant decreases in time from the first solution to the

second. The means for the groups by solution attempt were displayed in Figure 4-1.

Independent samples t-tests were also conducted on the groups for the first and second

solutions. There were no significant differences between the groups for the second solution

attempt. On the first solution attempt the no-cost group took less time on moves compared to the

check, t(3 8) = 2.84, p < .01 and aloud groups, t(3 8) = 5.86, p < .001, and approached significance

for the punishment group, t(3 8) = 1.70, p = .098. The check group did not significantly differ










from the aloud, t(3 8) = 1.60, p > .05, or the punishment groups, t < 1. The punishment group

took significantly less time making moves on the first solution when compared to the aloud

group, t(3 8) = 2.50, p < .05. Average move times for the groups were displayed in Figure 4-1 by

solution attempt.

The check, aloud, and punishment groups took significantly less time on the second

solution compared to the first. The results may have been due to the increased activities such as

checking and/or thinking aloud in the experimental conditions, but may have also been due to

increased attention as proposed in Experiment 1. It was also not surprising that the check and

aloud groups took significantly longer to make moves on the first solution when compared to the

no-cost group because the two experimental groups had to take additional actions in checking

moves for legality and/or speaking aloud. During the second solution when the manipulations

were removed all groups had similar move times.


Experiment 3: Average Mlove Times

35000
30000-
25000-
S20000 -1 -*- Solution 1
15000 --e- Solution 2
S150000
5 000 -
500

No-Cost Check Aloud Punishment
Group



Figure 4-1. Average time per move for each of the four groups was displayed for the first and
second solutions in milliseconds. The error bars represented standard error.

[Illegal move data. An analysis of illegal moves revealed that there was neither a

significant main effect of solution, F < 1, nor a significant main effect of order, F < 1. The main









effect of group was significant, F(3,72) = 3.12, p < .05, M~SE = 18.24, and was followed up with

independent and paired samples t-tests. Independent samples t-tests revealed that the no-cost

group made significantly more illegal moves when compared to the check, t(3 8) = 2.59, p < .05,

aloud, t(38) = 3.41, p < .005, and punishment group, t(38) = 3.03, p < .005. The paired samples t-

tests revealed no significant differences between solutions for the no-cost, check, and

punishment groups. The analysis of the aloud group indicated that participants made more illegal

moves on the second solution when compared to the first. Figure 4-2 displayed the group means

for both the first and second solution. In addition, the Solution x Order and 3-way interactions

were not significant, Fs < 1. The Solution x Group interaction, F(3,72) = 1.94, p =. 132, M~SE =

12.20, and Order x Group interaction, F(3,72) = 1.27, p =.291, M~SE = 18.24, were also not

significant. Illegal moves for each group for the first and second solutions were displayed in

Figure 4-2.

The analysis of illegal moves replicated previous findings that when participants were

penalized for illegal moves they made significantly fewer illegal moves compared to participants

that were not penalized for making illegal moves. In addition, participants in the two

experimental conditions made significantly fewer illegal moves compared to the no-cost control

group. These findings indicated that when participants were instructed to be and were

accountable for checking moves for legality this resulted in the reduction of illegal moves. These

results directly addressed and supported Stage 3 of CIF where an increase in checking behavior

resulted in a decrease in illegal moves. This reduction occurred even when participants were

instructed to think aloud while solving the problem.

One very interesting result was that participants in the aloud group made significantly

more illegal moves on the second solution (M~= 4.60) than they did on the first (M~= 2.00), t(19)










= 2.73, p = .013. Even though the check and punishment groups showed sustained benefits on

the second solution with no significant increases in illegal moves, the aloud group did not and

showed a release, making more illegal moves on the second solution than the first.

An increase of illegal moves from the first solution to the second for the aloud group was

an unexpected result. Knowles and Delaney (2005) performed a similar experiment where

participants were instructed to think aloud under no-cost instructions or to think aloud under

punishment instructions. Neither group was instructed to perform a check of moves for legality.

Both groups in Knowles and Delaney's experiments showed no significant effects of thinking

aloud and showed typical no-cost and punishment results, but neither group solved a second

problem to assess transfer effects. In addition, the check group in the current experiment showed

sustained benefits with no significant difference between illegal moves on the first and second

solution. Since transfer effects were not previously explored with think aloud instructions, these

results could be due to the think aloud instructions or to some interaction between thinking aloud

and checking.


Experiment 3: Illegal Mloves




a, 6-
65
14
li -m Solution 2




No-Cost Check Aloud Punishment
Group



Figure 4-2. Illegal moves for each group were presented by condition with standard errors
indicated as error bars.









Checking data. The checking data provided insight into the relationship between checking

behavior and the reduction of illegal moves, a direct indication of how checking affected illegal

moves. Participants in the check and aloud groups were instructed to click on a "check" button

on the computer display before executing a move to indicate that they had checked that move for

legality. The computer program tracked when a move was not checked prior to making that

move. This information allowed for a physical indication of when a participant did not check a

candidate move for legality. These values were compared with the number of illegal moves

committed by each participant in the check and aloud groups. Illegal moves were not reliably

correlated with checking in the check group, r(19) = .244, p = .300. In the aloud group, as the

number of illegal moves decreased the frequency of missed checks decreased, r(19) = .657, p =

.002.

The results from the correlation analysis revealed some unexpected results in that checking

in the check group was not significantly related to illegal moves, but checking in the aloud group

was related to illegal moves. One potential explanation for this finding was that those in the

check group did not verbalize their thoughts and therefore were not held accountable to the same

degree as those in the aloud group. A participant in the check group could have simply clicked

the "check" button prior to each move without actually engaging in any checking behavior

because there was no way for the experimenter to actually verify what the participant was doing

"in his/her head." This explanation seemed unlikely because participants in the check group

made a similar number of illegal moves compared to participants in the aloud group and fewer

than those in the no-cost group. In contrast, a participant in the aloud group was reminded to

continue verbalizing if he/she was silent for too long, which may have influenced the participant

to be more compliant in checking moves. An additional explanation could have been that










participants in the check group could have checked moves as often as those in the aloud group,

but forgot to click the check button more often as a result of not verbalizing. The results did not

support this explanation -- the number of times the check button was not clicked prior to making

a move did not statistically differ between the check (M~= 1.75) and aloud (M~= 2.40) groups, t <

1. The check and aloud groups both made fewer illegal moves compared to the no-cost group

and did not differ from each other, which may have indicated that participants in the check group

were likely checking and rej ecting moves to the same degree as the aloud group.

OSPAN data. After completion of the two problems, participants were asked to complete

the OSPAN task to assess their working memory span. Out of the 80 total participants that

solved both problems within the 20 min time limit in Experiment 3, 10 participants were unable

to complete the OSPAN task in the time allowed. The other 70 completed the OSPAN task and

their data were submitted to a correlation analysis. Nineteen participants in the check group

completed the OSPAN task, but their scores were not significantly related to either the number of

illegal moves made r(18) = -.067, p = .785 or the number of missed opportunities to check a

move for legality r(18) = -.252, p = .297. Eighteen participants in the aloud group completed the

OSPAN task, their scores were not significantly related to the number of illegal moves made

r(17) = -.275, p = .269, but they were significantly related to the number of missed opportunities

to check a move for legality r(17) = -.479, p = .044. The measure obtained by the computer

program was the number of times a participant made a move without first clicking the check

button, in other words, the number of missed opportunities to check for legality. This negative

correlation indicated that as a participant's working memory span score increased the probability

of that participant forgetting to check a move for legality decreased.









The correlation between missed opportunities to check a move for legality and memory

span score in the aloud group appeared to support the resource limitation hypothesis. That is,

those participants with more available memory were more able to remember to check moves for

legality. However, a similar finding was not found in the check group and the correlations

between memory span scores and the number of illegal moves made were not significant, which

did not support a resource limitation hypothesis. One potential explanation could have been that

with the instruction to think aloud and check moves for legality simultaneously in the aloud

group this could have taxed enough resources that those with larger working memory spans were

more able to perform the check consistently when compared to those with smaller working

memory spans. This explanation did not seem to explain why similar benefits were not seen in

correlations between working memory and illegal moves for the aloud group. Taken as a whole

these results seemed to indicate that a resource limitation hypothesis could not be entirely ruled

out, but that it likely played a smaller role than has been assumed in previous research.









CHAPTER 5
GENERAL DISCUSSION

The three experiments presented here addressed specific assumptions of the CIF

framework for the reduction of illegal moves. Experiment 1 tested whether an intrusive penalty

was necessary to reduce illegal moves or if a threat without penalty for each illegal move would

be sufficient. Experiment 2 set out to determine if an aversive stimulus was necessary to reduce

illegal moves or if a more positive consequence could help to improve problem solving

efficiency. Experiment 3 instructed participants to engage in checking behaviors to directly

assess the ability of these actions to reduce illegal moves. Experiments 1 and 3 seemed to

support aspects of the framework, while Experiment 2 produced non-significant results. As noted

previously, several of the trends seemed intriguing, but low statistical power to detect some of

these effects was likely at fault. Therefore, some of the trends that were not significant were

expanded upon further in this section.

[Illegal Moves

The first experiment demonstrated that the threat of a penalty, whether it was experienced

or not, was able to reduce illegal moves compared to a group that was not penalized or

threatened. Participants in the no-cost group made the most illegal moves (M~= 6.85) and the

threat and experience groups made fewer illegal moves (M~s = 3.40 and 3.65, respectively), but

similar numbers compared to each other. Because the threat and experience groups made similar

numbers of illegal moves, verbal instructions alone without any corporeal consequences were

apparently sufficient to reduce illegal moves. Participants in the punishment group made slightly

fewer illegal moves (M~= 2.60) compared to the two experimental groups on the first solution,

although not significantly. One potential explanation for the trend of fewer illegal moves in the

punishment group could be attributed to the idea that the punishment manipulation was more










powerful and thus had the potential to reduce illegal moves to a greater degree compared to a

threat alone. The lack of significance may have also been attributable to a floor effect, whereby

significance may have been obtained in problems with a greater opportunity to make more illegal

moves.

Surprisingly, the second experiment did not reveal any significant illegal move reductions

on the first solution. The intent of the motivation and reward manipulations was to determine if

positive consequences or instructions to avoid illegal moves would result in reductions. The

motivation (M~= 3.40) and punishment (M~= 4.25) groups, although not significantly, made

fewer illegal moves compared to the no-cost group (M~= 5.40). It is possible that with increased

power to detect the effects the motivation and punishment groups would have made significantly

fewer illegal moves compared to the no-cost group. The trend indicated that participants in the

motivation group made the fewest illegal moves. Why they potentially made fewer than the

punishment group is not well understood. One possibility was that the extra motivating

instructions may have seemed suspicious to participants and caused them to be even more

cautious for fear of the "unknown." Alternatively, they may have taken the instructions as a hint

that avoiding illegal moves was the secret to solving the problem.

Candy in the reward group was set on a Eixed interval schedule to limit the possibility of

participants making additional legal moves to receive more candy. Even though the reward group

received candy for avoiding illegal moves these participants made the most illegal moves (M~=

6.85). Such findings were unexpected and did not seem to be explainable by CIF. Some potential

explanations could have been that the candy was not motivating enough or that the interval

schedule was confusing or detrimental in some way. Participants only received one piece of

candy for each successful 30 s time interval. It is possible that with more candy offered or a more









substantial reward performance could have been improved. In addition, because the reward was

administered on a fixed interval schedule participants could have had optimal performance that

occurred early in the 30 s interval, but did not receive a reward until the end of the interval or

potentially not at all if an illegal move was subsequently made in that interval. The lack of a

significant reduction in illegal moves was surprising here, but the absence of such findings may

have been due to a fault in the experimental design. One alternative explanation for the lack of

significant reductions in illegal moves for the motivation and reward groups could have been that

aversive consequences are necessary to reduce illegal moves.

The most surprising finding in Experiment 2 was the lack of a significant difference

between the punishment and no-cost group. The punishment group was also set to administer a

penalty at a fixed interval to maximize the similarity between the punishment and reward groups.

However, this allowed participants to make illegal moves that did not result in a penalty because

multiple illegal moves could have been made in one 30 s interval that only resulted in one

penalty. This design may have reduced the effect of the punishment and was likely responsible

for the absence of significant reductions in illegal moves.

Experiments 1 and 2 specifically looked at Stage 1 of CIF where consequences for making

or avoiding illegal moves were manipulated in an attempt to reduce illegal moves. Experiment 1

lent support to CIF's claim that punishment and threat without punishment resulted in illegal

move reductions. Experiment 2 indicated that the reward manipulation was not able to reduce

illegal moves, but that a motivational manipulation showed promise. Further research is required

to explore what other consequences, if any, are able to assist problem-solvers in avoiding illegal

moves. According to CIF, manipulations in the first stage of the framework reduced illegal

moves by acting on Stage 2 to increase a problem-solvers attention towards the problem. CIF









made no specific assumptions about the attentional resources available and how they could be

distributed to reduce illegal moves, only that problem-solvers do not often perform at optimal

levels and that the consequences in Stage I would result in additional resources attributed to the

task. Stage 2 of CIF was not specifically tested here and further research is required to determine

how and where attention would be increased.

Stage 3 of CIF proposed that increased attention from Stage 2 would allow for increased

accuracy and/or frequency of checking and/or evaluation. The checking portion of the

framework utilized the Generation, Caution, Verifieation Framework (GCV) from Knowles and

Delaney (2005). Experiment 3 specifically looked at checking behavior and its ability to reduce

illegal moves. Participants in the experimental conditions were instructed to check each move for

legality prior to executing that move, approximately half of those participants did this silently

and the other half did so aloud. The check (M~= 3.00), aloud (M~= 2.00), and punishment (M~=

2.65) groups made significantly fewer illegal moves compared to the no-cost group (M~= 5.90).

These results indicated that increased frequency of checking resulted in reduced illegal moves,

lending support to that concept of the framework.

The check and aloud groups were both instructed to check moves for legality and to click a

button on the display prior to each move to indicate that they had checked that move, this was

done silently or aloud, respectively. There was a significant relationship between checking and

illegal moves for the aloud group, but not for the check group, indicating that those in the aloud

group reduced the number of illegal moves they made as checking increased. It was surprising

that data from the check group did not yield a significant relationship, especially since

performance in this group was similar to the aloud group where there was a significant

relationship. One potential explanation could have been that since these participants did not









check aloud they could have not been checking at all and could have just simply clicked the

button prior to making a move. This explanation did not seem very plausible considering that

participants in the threat group made a similar number of illegal moves compared to the aloud

group, t(3 8) = 1.24, p > .05. Another, plausible explanation was that participants in the threat

group were checking moves for legality in their head, but they were less likely to click the check

button. This explanation also did not seem likely because an independent samples t-test between

the two groups for number of moves not checked revealed no significant difference, t < 1. One

final explanation could have been that both groups were checking and rej ecting illegal moves,

but the aloud group did so with more accuracy. Although this final explanation appears to be the

most likely it is not possible to determine the true cause based on current findings.

Taken together the three experiments provide valuable information about the framework

presented and about problem-solving behaviors more generally. Experiment 1 lent support to

CIF's assumption that a physical penalty was not required to reduce illegal moves and that threat

alone helped to induce caution. Experiment 2 presented somewhat confusing results and further

research is required to determine what other types of consequences could reduce illegal moves.

Experiment 3 directly looked at increasing checking behaviors and lent support to the claim that

increasing the frequency of checking reduced illegal moves. These findings support CIF's claim

that less intrusive manipulations can induce caution and that checking behaviors directly reduce

illegal moves.

Transfer Effects

In all three experiments, each participant solved an isomorph of the first problem where the

second problem had the same underlying structure and solution as the first problem, but did not

share any outwardly similar features. This isomorph was solved after the first solution and was

always solved under no-cost instructions. Having each participant solve the second problem with









the same instructions and manipulations allowed for the observation of transfer effects from

learning obtained on the first problem. For all comparisons, except one, the participants made a

similar number of illegal moves on the second problems when compared with the first, which

largely supported previous findings from Knowles and Delaney (2005). These Eindings seemed

to indicate that the manipulations on the first solution, where illegal moves were reduced in most

groups compared to the no-cost group, provided participants with sustained benefits that carried

over to the second problem where illegal moves remained relatively low. However, there was

one exception and some trends that should be addressed.

In Experiments 1 and 2, participants in the no-cost conditions demonstrated trends toward

making fewer illegal moves on the second solution compared to the first. These trends may have

potentially indicated that participants learned something that could be transferred to a novel

isomorph or maybe it indicated that participants were "settling down" and were less nervous so

performance improved, although not significantly. In Experiment 2, participants in the reward

condition yielded numbers that approached significance, t(19) = 2.08, p = .051, potentially

indicating that fewer illegal moves were made on the second problem (M~= 4. 15) compared to

the first (M~= 6.85). Why participants in the reward group made so many illegal moves on the

first solution is unknown. These illegal moves commissions on the first solution were likely

driving the trend toward fewer illegal moves on the second solution, but these results seemed

counterintuitive given previous research where incentives helped to improve performance. Wieth

and Burns (2006) found that the incentive to leave the experiment early resulted in improved

performance on both incremental and insight problems. In addition, they found that the incentive

resulted in further processing of the problem and increased participants' memory of the problem










they solved. These findings support the transfer effects seen where participants showed a trend

for reduced illegal moves on the second solution.

Finally, participants in the aloud group of Experiment 3 made significantly more illegal

moves on the second solution (M~= 4.60) compared to the first (M~= 2.00), t(19) = 2.73, p = .013.

This finding was unexpected and could have potentially been due to the aloud instructions

causing the participants to take more time on the first solution. The participants may have felt the

need to "make-up time" on the second solution, which could have led to carelessness and an

increase in the number of illegal moves on the second solution. Another potential explanation

could be that the aloud instructions prevented some necessary encoding or learning on the first

solution, as transfer effects with aloud instructions were not explored in previous studies. The

check group made a similar number of illegal moves on the first and second solution, so the

checking instructions were not likely the cause of the increase. However, it was possible that the

interaction between the think aloud and checking instructions could have been responsible for

participants making more illegal moves on the second problem when compared to the first. This

was a very interesting finding and the author believes that this warrants further exploration as the

potential for think aloud instructions to affect, not performance, but learning could indicate a

negative side effect of this manipulation not seen in previous research (Delaney, Ericsson, &

Knowles, 2004; Knowles & Delaney, 2005).

Untested Assumptions: Illegal Move Reduction

CIF was proposed as a novel framework and therefore made several assumptions as to how

a problem solver is able to reduce illegal moves. In the first stage of the framework the

assumption was that various consequences for making illegal moves influenced the problem-

solver to alter some aspect of his/her behavior. In the second stage the assumption was that the

consequences in Stage 1 influenced the problem solver to increase the amount of attention









devoted to the problem. With this increased attention the problem-solver then either increased

the frequency and/or accuracy of either checking or evaluating behavior in Stage 3 of the

framework. Because CIF was a novel framework it was not possible to test every assumption or

every aspect of the framework here. Several assumptions remain untested and require future

research to assess the ability of the framework to describe human problem-solver performance.

Experiments 1 and 2 addressed Stage 1 of the framework and attempted to assess what

consequences would invoke caution and reduce illegal moves. Threat, punishment, motivation,

and reward were all tested and threat and punishment successfully resulted in illegal move

reductions while motivation showed promise. The reward consequence did not prove to reduce

illegal moves. In addition, there may be other consequences not mentioned here that have the

ability to reduce illegal moves and further research would be necessary to determine what, if any,

other consequences would reduce illegal moves.

The assumption of CIF was that consequences in Stage 1 of the framework would motivate

the problem-solver to devote more attention to the problem-solving episode. Previous research

has indicated that illegal moves may occur due to resource limitations (Jeffries, Polson, Razran,

& Atwood, 1977), recent research seemed to indicate otherwise (Knowles & Delaney, 2005).

Because a resource limitation hypothesis cannot fully explain illegal moves, it made sense to

assume that we do not always perform to our full potential and that we may have additional

resources available to perform more efficiently. The framework assumed that these additional

resources were represented as attentional focus and that consequences in Stage 1 tap into these

additional attentional resources to improve performance. However, the framework made no

assumptions as to how this additional attention was evoked or to how it was distributed, only that

it exists and that it can be utilized, the current results seemed to support these assumptions.










Experiment 3 specifically looked at the third stage of the framework and how increased

checking behavior would influence illegal moves. This increased checking resulted in reduced

illegal moves as the framework predicted. However, changes in evaluation behavior and

increasing the accuracy of checking were not assessed. It seemed intuitive that as the accuracy of

checking a move for legality increased the likelihood of correctly rej ecting illegal moves would

also increase. However, further exploration is necessary to determine when and to what degree a

problem-solver may increase the accuracy of checking candidate moves. CIF also assumed that

as attention towards the problem increased a problem-solver may begin evaluating candidate

moves more frequently or evaluating moves to a higher criterion.

Integration With Existing Theories of Illegal Move Selection

One of the main goals of this work was to address why illegal moves are made. One

potential explanation for this, proposed by Jeffries and colleagues (1977), was that illegal moves

are made due to resource limitations. Although the term "resources" was not well-defined, it

usually refers to working or short-term memory limits, attentional capacity, and thinking speed.

This explanation proposed that on problems such as those used in these experiments participants

do not have the capacity to either remember to check a move for legality or to calculate the

resulting state properly to assess legality. However, recent research has seemed to indicate

otherwise. Knowles and Delaney (2005) found that when penalized for illegal moves participants

were able to avoid illegal moves. The current work found similar results involving not only

penalty, but the mere threat of a penalty indicating that problem-solvers do have the resources

necessary to avoid illegal moves. In addition, work on various types of problems such as the 8-

puzzle, the Tower of Hanoi, and water jugs problems has demonstrated that problem-solvers

actually have the ability to plan out solutions to difficult problem states and entire problems

themselves, again indicating that we have the resources necessary to perform better on these










tasks (O'Hara & Payne, 1998; Welsh, Cicerello, Cuneo, & Brennan, 2001; Delaney, Ericsson, &

Knowles, 2004).

Just because we can perform better on certain tasks than the resource theorists predicted

does not imply that we have no limits on our cognitive resources. In fact, a central tenet of

cognitive psychology is that many of our cognitive abilities are measurable and have

documented limits. For example, there are significant limits on our short-term memory (Miller,

1956). Furthermore, Anderson and colleagues have successfully used 185 ms as an estimate of

time to shift visual attention in versions of ACT-R (Anderson & Lebiere, 1998; Gray, Sims, Fu,

& Schoelles, 2006). Because we have measurable cognitive limits, a resource limitation

hypothesis cannot be entirely ruled out. However, on the tasks described above they likely play a

smaller role than initially assumed, and resource theorists need to be more clear about exactly

what resource is taxed and how.

Another possible explanation is that mental resources may not place a hard limit on our

ability to check for illegal moves, but that we are "cognitive misers" who try to conserve

resources as much as possible. Previous and current research has demonstrated through strategy

changes, planning, and improved performance that we are often much more capable than our

initial performance would indicate (for example Simon & Reed, 1976; O'Hara & Payne, 1998;

Delaney, Ericsson, & Knowles, 2004; Knowles & Delaney, 2005). In Wilson's (2002) review

article of embodied cognition, she stated that one of the views of embodied cognition was an off-

loading of cognitive work onto the environment. This off-loading referred to the strategy of

accessing information in the environment as needed instead of using resources, like committing

such information to memory. In tasks such as the hobbits and orcs, problem-solvers may have

used trial and error strategies instead of planning deeply or checking moves for legality, thereby









conserving resources. A trial and error strategy would likely result in many illegal moves.

However, with no penalty for illegal moves, such a strategy was not very costly in terms of

resources, and was usually successful in advancing the problem-solver to the goal. Such an

explanation seems to fit well with participants' performance in the no-cost groups, but it does not

explain why a threat or penalty would change such behavior.

Gray and colleagues referred to this off-loading concept as the minimum memory

hypothesis and compared this to the soft constraints hypothesis to describe participants' behavior

on the Blocks World task (Gray et al., 2006). The soft constraints hypothesis, in contrast to the

minimum memory hypothesis, stated that optimal performance was based on the currency of

time and not on conserving memory resources. Gray et al. (2006) found that as the time cost of a

task increased participants began to utilize their memory more to avoid procedures that took

additional time, ultimately reducing the total time spent on the task. The idea of participants

adjusting behavior to reduce the total time on a task actually fits well with the results observed in

the first two experiments. Participants in the punishment group avoided illegal moves because

the penalty phase automatically added an additional 45 s. Those in the threat and experience

groups may have avoided illegal moves because they were told that illegal moves would have

added additional time at the end of the task. It may have been possible that participants in the

motivation group demonstrated a trend towards reduced illegal moves because they saw the

motivating instructions as a hint to avoid illegal moves to reduce total solution time. Finally,

participants in the reward group may not have shown illegal move reductions because doing so

would not have decreased the time to solve the task. In fact, attempting to maximize the reward

would have potentially taken longer to solve the problem.









The soft constraints hypothesis seemed to offer a valid explanation for many of the results

obtained, in at least Experiments 1 and 2. Such findings offered a novel explanation for some of

the unaddressed aspects of CIF. Stage 2 of CIF indicated that the consequences in Stage 1

resulted in an increase in attention. However, the soft constraints hypothesis may have indicated

that increased attention was actually a reevaluation or redistribution of attention to reduce the

total time a problem-solver spends on the task.

A reevaluation or redistribution of attention in the CIF framework could have been

explained as a shift in strategy. The author believed that participants initially engaged in a trial

and error strategy because there was little or no cost for illegal moves and this strategy was often

successful at eventually reaching the goal state. Increasing the cost of making illegal moves may

have lead to a representational change of the problem and a shift in strategy. Lovett and

Schunn' s (1999) RCCL model predicted that unsuccessful strategies would lead to a

representation change of the problem and through learning unsuccessful strategies were

abandoned and successful strategies were adopted. Simon and Reed (1976) also found a strategy

shift towards means-ends analysis when participants were given a hint of a state they would

encounter during their solution. Because illegal moves were highlighted in the threat and

punishment groups the execution of such a move may have seemed like a failed strategy to the

participants and prompted a representational change or a strategy shift. This idea seemed

plausible as adopting a more successful strategy could have likely resulted in illegal move

reductions.

An additional explanation for the avoidance of illegal moves could have been attributed to

problem-solvers "learning" from their illegal moves. Grobe and Renkl (2007) had participants

begin by working on pretest probability problems to assess their prior topic knowledge. Next










they presented participants with worked examples of probability problems. Some of the

participants were presented with worked solutions that were all correct and some participants

were presented with a mixture of correct and incorrect solutions (participants were told when a

solution was incorrect). Following this exercise, participants completed a post-test of problems

that varied in their similarity to the worked examples they had seen. The post-test results

indicated that those participants with low prior knowledge benefited from seeing the correct

solutions only, but those participants with high prior knowledge benefited from seeing both the

correct and incorrect solutions.

In the current experiments participants were not assessed on their prior problem-solving

knowledge, but it may have been possible that at least some of the participants were able to learn

from the illegal moves they made. Then why were illegal moves reduced in the experimental

groups compared to the no-cost control? Threatening or penalizing participants for illegal moves

may have increased their attention to the task and motivated them to learn from their illegal

moves so that they could be avoided in the future. With little or no cost to making illegal moves

learning may have seemed like a waste of time or waste of resources. After participants in the

experimental groups made illegal moves they may have been more likely to assess why that

move was illegal and this would have aided in avoiding illegal moves in the future. This would

explain why transfer effects to novel isomorphs were seen where participants continued to make

a reduced number of illegal moves. A problem-solver' s ability to learn from his/her illegal

moves seems like a plausible explanation not only for transfer effects, but for illegal move

reductions on the initial problem where the first illegal move has the potential to provide a

valuable lesson.









Learning may have occurred from reflecting on an illegal move, but it may have also been

possible that learning took place during the penalty phase. Moss, Kotovsky, and Cagan (2007)

presented participants with a Remote Associates Test (RAT) where participants had to find a

word that was associated to a group of words presented. After several RAT problems,

participants completed a lexical decision task that provided hints to the RAT problems that were

not solved correctly. Participants were then returned to the previously solved and unsolved RAT

problems they had attempted. The results indicated that the hint appeared to help participants

solve the uncompleted RAT problems. One explanation Moss et al. considered was that of an

incubation period. What actually occurs during an incubation period is not fully known, but it is

essentially a "break" from the problem where the problem-solver may continue working on the

problem subconsciously or the break may influence a strategy change. One example of where an

incubation period is often helpful is with insight problems where the problem-solver can often

return to the problem with greater success on a subsequent attempt. It could be that in the current

experiments the 45 s penalty phase acted as an incubation period where participants either

continued working on the problem or this may have prompted a strategy change. This

explanation seemed unlikely as their was no incubation period in the threat or experience groups

in Experiment 1, which both demonstrated illegal move reductions compared to the no-cost

group.

Previous theories from the problem-solving literature and other areas of research help to

provide alternative explanations and insight to the current findings. Although resource

limitations may play a role in the current findings their influence likely plays a smaller role than

initially assumed. Resource limitations may have seemed like a valid explanation for many

results because problem-solvers may be "cognitive misers" and conserve resources or they may









have an alternative goal of conserving time. It is through experimental manipulations that we are

able to gain insight into the true potential of the human problem-solver. When there are changes

to the problem-solving environment, as with punishment or threat, we see that the problem-

solver adapts. This adaptation could be defined as an attentional shift or strategy shift, but no

matter what the term problem-solvers are able to alter performance and improve. The current

Endings contribute to the problem-solving literature and provide additional insight into problem-

solvers' ability to adapt and improve.

How Do People Evaluate Moves?

Several move evaluation functions have been proposed in the problem-solving literature.

However, it did not seem wise to adopt one of these functions to CIF without first testing to see

which one had the best fit. At the same time, illegal moves are most likely selected during the

move selection phase and only rej ected during evaluation or checking. Therefore, it is worth

considering how earlier theories have accounted for move selection, with an eye towards

understanding why illegal moves are chosen according to those theories. One of the very first

theories of problem-solving that looked specifically at how problem-solvers solved a problem

and what they were doing was Newell and Simon's (1988)' General Problem Solver (GPS)

program. GPS was a computer program that attempted to mimic human problem-solving

performance by creating and completing subgoals to reach the ultimate goal of advancing to the

final goal state. Newell and Simon had a participant think aloud as he/she worked on a symbolic

logic problem and then used the verbal protocol to compare performance to the computer

program' s attempt at solving the problem. Although, GPS did not map the participant's moves

exactly the findings proved promising and have been influential to problem-solving research.


SNewell and Simon's 1988 work was read by the author as a chapter reprint. GPS was previously
introduced by Newell, Shaw, and Simon in 1959.









Since the introduction of GPS back in the late 1950's models of human problem-solving

have evolved and become more specified. In fact, a model of human problem-solving that was

discussed in the first chapter was proposed specifically for problems like the hobbits and orcs.

The model proposed by Jeffries et al. (1977) included the evaluation function:

e, = aMl + bC, + cP, (-)

The assumption of this evaluation function, as with most evaluation functions, was that a

problem-solver was able to assign candidate moves a value and determine which move had the

highest probability of advancing the problem-solver towards the goal state. Due to the design of

the evaluation function those moves that placed more travelers on the goal side (the right bank)

of the river and did so by maintaining a balance of missionaries and cannibals (missionaries are

interchangeable with hobbits and cannibals with orcs) to reduce the probability of having a

missionary eaten would be more likely to be selected. Such a strategy seemed intuitive and

potentially a good fit for CIF.

The assumptions of CIF stated that illegal moves may be reduced through increasing the

frequency and/or accuracy of evaluating moves. Increasing the frequency of evaluating moves

would simply entail using an evaluation function, like this one, more often instead of selecting

moves at random. One argument was that by increasing the frequency of evaluation the

probability of selecting an illegal move actually increased because many illegal moves actually

bring the problem-solver closer to the goal and would therefore be evaluated higher than legal

moves. The evaluation function proposed by Jeffries et al. indicated that problem-solvers attempt

to keep the travelers in missionary-cannibal pairs to minimize the chance of the missionaries

being eaten. This assumption of the evaluation function would help to prevent illegal moves

sometimes, but the problem cannot be solved unless the pairs are split. Since a split of the pairs









was inevitable the threat of evaluation increasing illegal moves remained. Jeffries et al.'s model

also stated that there were two types of illegal moves, easy-to-detect and hard-to-detect and that

easy-to-detect illegal moves were always rej ected. Hard-to-detect illegal moves were checked

and rejected based on fixed probabilities. CIF assumptions differed here in that these

probabilities were not fixed and that checking could occur independent or as a results of

evaluation. If the evaluation of moves can occur in CIF without checking behavior then the

possibility of illegal moves increasing with increased evaluation cannot be ruled out.

The second alternative for evaluation to aid in avoiding illegal moves according to CIF

was by increasing the accuracy of evaluation, which can be done in two different ways. One way

was by including an illegal move filter as part of the evaluation function to rej ect illegal moves

as noted previously. The second assumed that problem-solvers are affected by different

consequences that alter performance. These consequences increase attention towards the

problem, which may potentially increase learning and the integrating of past experiences to

increase the probability of selecting legal moves more often. In their model, Jeffries et al.

included a memory process that allowed for previously visited states to be avoided to prevent

backtracking and to facilitate progression through the problem, but since it was based on the

resource limitation hypothesis little or no learning or adaptation to the problem states were

incorporated. Based on this it would not be likely or maybe not possible that a problem-solver

could solve the problem in the minimum number of moves without any illegal moves. However,

out of the 240 participants that completed both problems 9 were able to solve the problem in the

minimum number of moves with no illegal moves in the current experiments. These participants

for the most part were in various groups and in different experiments.









To help explain how a problem-solver is able to efficiently solve this problem with no

wasted moves a brief summary of the solution of the hobbits and orcs problem is provided here

(the solution is almost completely linear so all participants solved the problem in this order).

First the problem-solver had to position all of the orcs on the right bank of the river. Next the

hobbits were moved over in a balanced fashion so that there were two hobbits and two orcs on

the right bank. Then for the first and only time in the problem the problem-solver moved two

travelers to the left bank, one hobbit and one orc. At this point the problem solver could safely

move all the hobbits to the right (goal) bank where they remained until the rest of the travelers

were moved to the goal bank. To utilize the current evaluation function to explain this

performance the constant weighting factors a, b, and c would need to be adjusted at different

states of the problem. For instance, b, which accompanied the cannibals' value on the right bank

would need to be weighted heavier towards the beginning of the problem to ensure that the

cannibals were moved first. Once this phase was completed the weighting factors again would

need to be adjusted so that a, which accompanied the missionaries, was weighted more heavily

to ensure that they would be moved. Increasing the accuracy of the evaluation function would

require that participants adjusted these weighting factors at each state and this increased accuracy

would likely aid in the avoidance of illegal moves.

The evaluation function and model proposed by Jeffries et al. (1977) appeared to provide

a good description of participants' problem-solving behavior presented here with minor

revisions. The maj or assumptions of their model were based on a resource limitation hypothesis,

but since the current and recent research seemed to indicate otherwise such adjustments seemed

warranted. Because this evaluation function was based on the same types of problems used here

it had a natural fit into the CIF framework. However, it would a gross oversight to think that









research and models taken from other types of problems could not provide valuable insight as

well.

An additional model that could be used to explain the results obtained here was based on

research that has utilized John Anderson's ACT* theory (often referred to now as ACT-R).

ACT* was based on the firing of productions, which were condition-action pairs (Anderson,

1987). If a condition was satisfied then the corresponding action would be executed, e.g. if the

traffic signal is red, then I will stop my vehicle. Lovett's (1998) work on the building sticks task

(BST) has utilized the ACT-R theory to model problem-solvers' ability to choose moves that will

likely reach the goal state. Lovett' s ACT-R based model assumed that problem-solvers selected

their next move based on the highest expected gain according to the following equation:

E = PG -C (1-2)

where E was the expected gain of the selected move, P was the estimated probability of

achieving the production' s goal, G was the value of the goal, and C was the estimated cost to be

expended in reaching the goal.

Since previous research indicated that problem-solvers have the resources to calculate

future states and to plan then they should have the ability to calculate expected gains (E) that are

equal to true gains in tasks such as the hobbits and orcs. Unlike BST, the hobbits and orcs task

was not solved several times and it had a Eixed solution and fixed goal state that was not

manipulated by the experimenter. The problem solver had to advance through the problem space

encountering many novel states along the solution path. At any given state subsequent states

could have been calculated and evaluated as to their ability to advance the participant to a novel

legal state and the success of an operator was independent of its history of success. In addition,

the problem space for the hobbits and orcs problem was almost completely linear. This enabled










participants to solve the problem and completely avoid all illegal moves by correctly calculating

future states and only selecting novel legal moves. In contrast, it was not possible to accurately

calculate the validity of a move in BST, influencing participants to rely primarily on history of

successes and failures of operators in these types of problems. Lovett' s model, as applied to

BST, focused primarily on participants' ability to make decisions based on previous successes

and failures. However, with some modification it may be possible to apply Lovett' s model to the

hobbits and orcs problems to account for illegal move reductions obtained in previous and

current research.

In Lovett' s model P was the estimated probability of achieving the production' s goal and

was made up of the product of qr. The probability that the production would have achieved the

desired state q was fixed to 1 and the probability of achieving the production goal given arrival at

the intended state r was based on previous successes and failures. However, the hobbits and orcs

task differs greatly from the BST in that previous successes and failures have less of an impact

and problem-solvers could have calculated future states and illegal moves based on the rules. To

adapt this model to the hobbits and orcs problem r would have been fixed to 1 because successes

and failures contribute little to the subsequent selection of operators and q would = the problem-

solvers' ability to correctly calculate the future state. It was possible that q would also contain

the 3-stage framework GCV proposed by Knowles and Delaney (2005) if the move were to be

assessed for legality. However, previous research seemed to indicate that when the cost for

making illegal moves was low problem-solvers often did not engage in this activity. With an

increase in cost for an illegal move problem-solvers would have likely increased the frequency

and/or accuracy with which they engaged in calculating q resulting in P yielding higher values

for legal moves and the reduction of illegal moves.










As indicated above, move evaluation functions from other problems or from similar

problems may have the potential of being updated and integrated into the CIF framework.

However, additional research is required to determine if a move evaluation function can be

adapted from another problem, from similar problems, or if a novel move evaluation function

would provide the best fit for CIF.

Real World Application and Future Research

Problems are something that we encounter every day. Whether it is balancing our

checkbooks, driving to work, or cooking a meal, these daily tasks consume a large portion of our

lives and understanding how to solve these problems more efficiently would offer great benefits.

Laboratory results have revealed that one way to improve performance may be through

punishment (Knowles & Delaney, 2005). However, being punished for every mistake or illegal

move we make does not sound like a lot of fun. Therefore finding alternative ways to improve

performance seemed desirable.

The current work looked at alternative ways for improving problem solving and discovered

that a threat without punishment reduced illegal moves. Motivation and reward did not result in

significant reductions, but motivation showed promise. These findings seemed to support some

things that we already do or experience today. You threaten your child not to touch the stove for

he will get burned or your boss threatens you that you will get fired if you show up late for work

again. The findings presented here indicated that a credible threat can yield improvements as

significant as punishment, at least on these types of problems. The motivation and reward results

were somewhat surprising based on common practices in the world today. Motivational seminars

and tapes are utilized by companies and individuals in an attempt to enhance performance and

rewards are offered in the form of money or other incentives for meeting or exceeding expected

goals. Because motivation and rewards are successfully used in our society to drive performance









it strengthens the claim that the design lacked the power to detect these effects or that the

motivating instructions or rewards were not adequate. So then what has this work offered?

The CIF framework made several assumptions about how we solve problems and operate

in a given problem space. The main assumption of CIF was that we do not perform optimally in

certain situations and that we have the ability to do better. The results presented here supported

that assumption by demonstrating that with consequences for illegal moves or by increasing

cautiousness through checking we can reduce illegal moves and increase problem solving

efficiency. One potential method for avoiding illegal moves is by being more cautious and

checking moves for legality. We can liken this to an airline pilot who has a checklist of items he

must verify and complete before he is able to take-off. Of course, we do not need a checklist for

everything we do in life, but being more cautious in certain situations would serve us well. Some

examples of areas in life where we could apply this information and benefit include: driving

(checking and abiding by the speed limit and other laws would likely prevent us from getting a

ticket or getting into an accident), Einances (checking our bank balance before writing a check to

ensure that we do not overdraw our account), and military training (ensuring that a soldier knows

the proper procedures for a critical situation so that no one gets injured). The real world

applications and future avenues for such findings seem limitless as we are constantly striving to

become more efficient to conserve time and/or resources.

The results obtained here seemed to support the CIF framework that certain consequences

of illegal moves can reduce illegal moves and that checking moves for legality may be one

method for reducing those moves. However, several questions still remain and future research

should focus on these questions. First, what other types of consequences, including different

types of motivators and rewards, are able to reduce illegal moves? Second, Stage 2 of the









framework should be further studied to determine if an attentional shift is responsible for

increased caution and if so how this attentional shift reallocates these additional resources. Third,

future research should also focus on the evaluation function used by problem-solvers in

determining which move to select next in the problem-solving episode. Finally, how the

accuracy of checking is altered over the duration of a problem should be explored to determine

the likelihood of errors being made after a problem-solver has decided to check a move for

legality.









CHAPTER 6
CONCLUSION

Three experiments were presented to assess the ability of the CIF framework to describe

and predict problem solving performance and the reduction of illegal moves. The main

assumption of the framework was that consequences of illegal moves increased attention to

reduce illegal moves. The first experiment tested whether a penalty was required during the

problem solving phase to increase attention or whether the threat of penalty alone would be

sufficient to increase attention and reduce illegal moves. The second experiment addressed

whether an adverse stimulus or threat of an adverse stimulus was required or whether motivation

or reward could be effective at inducing caution and increasing attention. Finally, the third

experiment directly addressed a portion of the framework by instructing participants to check

moves for legality and also collected verbal protocols to assess what participants were actually

doing/thinking.

Experiment 1 results indicated that punishment was not required to increase attention as

significant results were found in both the threat and experience conditions. These Eindings

demonstrated that less intrusive interventions could potentially be as effective in increasing

attention when compared to punishment that occurred during the problem solving episode. The

results of Experiment 1 have significant real world applications that could be helpful when

punishment can have serious negative effects or when punishment is not feasible during a

problem solving episode. However, further research is required to determine the true

applicability of these results to real world settings.

Experiment 2 did not achieve the predicted results and the motivation and reward

conditions were not effective at reducing illegal moves. The lack of illegal move reductions

could be attributed to one of several explanations. Experiment 2 utilized undergraduate










participants and was run at the end of the semester. Thus, the participants in this experiment were

likely those that waited until the end of the semester and may have been less motivated and

resistant to the experimental manipulations. Alternatively, the motivating instructions and the

candy reward may have not been sufficient to increase attention and a different motivation or

more valuable reward may have achieved significant reductions. Finally, it was also possible that

there were no "end of semester" effects and there was nothing wrong with the motivating

instructions or reward and that an adverse stimulus or threat of adverse stimulus was necessary to

increase attention. Experiment 1 demonstrated that a punishment was not necessary to reduce

illegal moves, but the concern of future punishment was able to reduce illegal moves. The

concern of future punishment still had the belief of something negative and may be a necessity in

inducing caution.

Experiment 3 directly manipulated and tested the third stage of the CIF framework.

Instructing participants to check moves for legality reduced the number of illegal moves made

whether participants were instructed to think aloud or not. This manipulation was the first that

did not focus on the consequences of making an illegal move and attempted to reduce illegal

moves by avoided them before they were made. This direct manipulation revealed that increasing

the frequency of checking resulted in a decrease in illegal moves. This finding also has practical

applications in that recommending or requiring a person to check for legality in problem solving

episodes may help increase problem solving efficiency by reducing illegal moves.

One unexpected result obtained in Experiment 3 that was not addressed was the increase in

illegal moves from the first to the second problem for the aloud group. This release from benefits

was likely due to an interaction between the thinking aloud and checking as there was no release









found in the check group and no release in aloud groups in previous research (Knowles &

Delaney, 2005). Further investigation is required to determine the specific cause of the release.

In conclusion two experiments lent support to the CIF framework and indicated that

increased attention was a likely cause of reductions in illegal moves. The threat of punishment

and instructions to check moves for legality both reduced illegal moves and provided support to

the framework. However, reductions in illegal moves were not discovered with motivating

instructions or reward, possibly indicating that an adverse stimulus or threat of adverse stimulus

was required to induce caution and increase attention. In conclusion, these experiments provided

support for CIF, with the exception of Experiment 2, and indicated that the framework may be a

valuable tool in determining potential behaviors on such tasks. Additional research is required to

further explore the ability of CIF to predict problem solving behavior and to determine if rewards

and motivation have the ability to reduce illegal moves.









APPENDIX
THINKING ALOUD INSTRUCTIONS: EXPERIMENT 3

The thinking aloud instructions presented to a participant in Experiment 3 were as follows:

"In this experiment you will be asked to solve problems on the computer using the mouse.

However, as you work on the problem you will be asked to think out loud. That is, as you work

on the problem I would like you to say whatever comes into you head. It doesn't matter if it

makes sense or not, I just want you to say whatever you are thinking. For practice, try this simple

task: Think of as house that you are very familiar with, it could be your' s or a friend's. Think out

loud as you imagine yourself walking through the house counting the number of windows you

see. You may begin whenever you are ready."










LIST OF REFERENCES

Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solutions.
Psychological Review, 94(2), 192-210.

Anderson, J. R., & Lebiere, C. (Eds.). (1998). Atomic components of thought. Hillsdale, NJ:
Erlbaum.

Atwood, M. E., & Polson, P. G. (1976). A process model for water jug problems. Cognitive
Psychology, 8, 191-216.

Delaney, P. F., Ericsson, K. A., & Knowles, M. E. (2004). Immediate and sustained effects of
planning in problem solving. Journal ofExperintental Psychology: Lea-l~rning. Mentory, and'
Cognition, 30, 1219-1234.

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge,
MA: MIT Press.

Gray, W. D., Sims, C. R., Fu, W. T., & Schoelles, M. J. (2006). The soft constraints hypothesis:
A rational analysis approach to resource allocation for interactive behavior. Psychological
Review, 113 (3), 461-482.

Grobe, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: Can this foster
learning outcomes? Learning andlnnstruction, 17, 612-634.

Jeffries, R., Polson, P. G., Razran, L., & Atwood, M. E. (1977). A process model for
missionaries-cannibals and other river-crossing problems. Cognitive Psychology, 9, 412-
440.

Kane, M. J., Bleckley, M. K., Conway, A. R. A., & Engle, R. W. (2001). A controlled attention
view of working memory capacity. Journal ofExperintental Psychology: General, 130,
169-183.

Knowles, M. E., and Delaney, P. F. (2005). Lasting reductions in illegal moves following an
increase in their cost: Evidence from river-crossing problems. Journal ofExperintental
Psychology: Lea-l~rning. Mentory, and' Cognition, 31, 670-682.

Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence
from Tower of Hanoi. Cognitive Psychology, 1 7, 248-294.

Kotovsky, K., & Simon, H. A. (1990). What makes some problems really hard: Explorations in
the problem space of difficulty. Cognitive Psychology, 22, 143-183.

Lovett, M. C. (1998). Choice. In J. R. Anderson & C. Lebiere (Eds.), The atomic components of
thought (pp. 255-296). Mahwah: Lawrence Erlbaum Associates.

Lovett, M. C., & Anderson, J. R. (1996). History of success and current context in problem
solving: Combined influences on operator selection. Cognitive Psychology, 31, 168-217.










Lovett, M. C., & Schunn, C. D. (1999). Task representations, strategy variability, and base-rate
neglect. Journal ofExperintental Psychology: General, 128(2), 107-130.

Luchins, A. S. (1942). Mechanization in problem solving. Psychological M~onographs, 54
(Whole No. 248).

Miller, G. A. (1956). The magical number 7, plus or minus 2: Some limits on our capacity for
processing information. Psychological Review, 63(2), 81-97.

Moss, J., Kotovsky, K., & Cagan, J. (2007). The influence of open goals on the acquisition of
problem-relevant information. Journal ofExperintental Psychology: Human Learning and'
M~entoy, 33(5), 876-891.

Newell, A., & Simon, H. A. (1988). GPS, a program that simulates human thought. In A. M.
Collins, E. E. Smith, (Eds.), Read'ings in cognitive science: A perspective fions psychology
and'artificial intelligence (pp. 453-460). San Mateo, CA: Morgan Kaufmann, Inc.

O'Hara, K. P., & Payne, S. J. (1998). The effects of operator implementation cost on planfulness
of problem solving and learning. Cogritive Psychology, 35, 34-70.

Platt, R. D., & Griggs, R. A. (1993). Facilitation in the abstract selection task: The effects of
attentional and instructional factors. Th2e Quarterly Journal ofExperintental Psychology A:
Human ExperimentalPsychology, 46A(4), 591-613.

Simon, H. A., & Reed, S. K. (1976). Modeling strategy shifts in a problem-solving task.
Cogritive Psychology, 8, 86-97.

Turner, M. L., & Engle, R. W. (1989). Is working-memory capacity task dependent? Journal of
M~entoy and Language, 28, 127-154.

Welsch, M., Cicerello, A., Cuneo, K., & Brennan, M. (2001). Error and temporal patterns in
Tower of Hanoi performance: Cognitive mechanisms and individual differences. Thze
Journal of General Psychology, 122(1, 69-8 1 .

Wieth, M., & Burns, B. D. (2006). Incentives improve performance on both incremental and
insight problem solving. The Quarterly Journal ofExperintental Psychology, 59(8), 1378-
1394

Wilson, M. (2002). Six views of embodied cognition. Psychonontic Bulletin & Review, 9 (4),
625-636.

Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cogritive
Science, 18, 87-122.









BIOGRAPHICAL SKETCH

Martin E. Knowles was born in Huntington, New York, to James and Joyce Knowles in

1978. At the age of seven, he and his family moved to Tampa, Florida, where he lived until he

was 18. At this time, he began attending Florida State University in Tallahassee, Florida, where

he completed a Bachelor of Science degree in the fall of 2000. In 2001, Martin moved to

Gainesville, Florida and began his graduate studies in cognitive psychology at the University of

Florida, under the supervision of Dr. Peter F. Delaney. He married his wife Stephanie in the

spring of 2004 and completed his Master of Science degree that same year. His master' s work

was subsequently accepted for publication in the Journal of Experimental Psychology: Learning,

Memory, and Cognition. In March of 2006, Martin became a father to a healthy baby boy and

subsequently moved to Jacksonville, Florida with his wife and son. He successfully defended his

dissertation in August of 2008, and received his Doctor of Philosophy degree in December of

2008, from the University of Florida.





PAGE 1

1 IMPROVING PROBLEM SOLVING EFFICIEN CY: THE WHAT AND HOW OF CAUTION By MARTIN E. KNOWLES A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008

PAGE 2

2 2008 Martin E. Knowles

PAGE 3

3 To my wife, son, and parents, you have made me who I am and I live fo r you and because of you

PAGE 4

4 ACKNOWLEDGMENTS I thank m y mom and dad, who provided me a nd my brothers with everything we ever needed. They sacrificed so that I could receive an excellent education a nd always supported me in everything I did. I thank my wife for putting up with me, I know its not an easy job. She has done a phenomenal job raising our son, and she give s me strength every day. I thank my advisor, mentor, and friend, Peter. He has provided me with many lessons in and outside of the classroom, and I have learned so much from him. Last, but definitely not least, I thank my committee, Drs. Abrams, Berg, Brenner, and Fi schler. Their flexibility, unselfishness, and willingness to help are something I will never forget.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS...............................................................................................................4 LIST OF TABLES................................................................................................................. ..........7 LIST OF FIGURES.........................................................................................................................8 ABSTRACT.....................................................................................................................................9 CHAP TER 1 INTRODUCTION..................................................................................................................11 Illegal Move Commissions: Frequency and Proposed Causes ............................................... 12 Proposed Causes: Move Selection................................................................................... 14 Early Research................................................................................................................. 17 Knowles and Delaney (2005)..........................................................................................18 Caution Induction Framework (CIF (pronounced ch f)) ....................................................23 Stage 1 (Consequence)....................................................................................................24 Stage 2 (Involvement)..................................................................................................... 25 Stage 3 (Facilitation)....................................................................................................... 26 Checking...................................................................................................................27 Evaluation.................................................................................................................28 CIF Summary..................................................................................................................30 Experiments.................................................................................................................... ........32 2 EXPERIMENT 1....................................................................................................................33 Methods..................................................................................................................................35 Participants......................................................................................................................35 Problems..........................................................................................................................36 Hobbits & orcs......................................................................................................... 36 Titration.................................................................................................................... 37 Design..............................................................................................................................38 Procedure.........................................................................................................................40 Results and Discussion.................................................................................................... 42 3 EXPERIMENT 2....................................................................................................................53 Methods..................................................................................................................................54 Participants......................................................................................................................54 Problems..........................................................................................................................54 Design..............................................................................................................................54 Procedure.........................................................................................................................56

PAGE 6

6 Results and Discussion.................................................................................................... 56 4 EXPERIMENT 3....................................................................................................................63 Methods..................................................................................................................................65 Participants......................................................................................................................65 Problems..........................................................................................................................66 Design..............................................................................................................................66 Procedure.........................................................................................................................67 Results and Discussion.................................................................................................... 69 5 GENERAL DISCUSSION..................................................................................................... 77 Illegal Moves...................................................................................................................77 Transfer Effects...............................................................................................................81 Untested Assumptions: Illegal Move Reduction............................................................ 83 Integration With Existing Theories of Illegal Move Selection ...............................................85 Real World Application and Future Research........................................................................ 97 6 CONCLUSION..................................................................................................................... 100 APPENDIX THINKING ALOUD INSTRUCTIONS............................................................. 103 LIST OF REFERENCES.............................................................................................................104 BIOGRAPHICAL SKETCH.......................................................................................................106

PAGE 7

7 LIST OF TABLES Table page 2-1 Legal moves made on the first and second........................................................................ 44 2-2 Illegal moves made on the first and second....................................................................... 49 3-1 Means and standard de viations (SD) by group. ................................................................. 58 3-2 Means and standard deviations (SD) of illegal.................................................................. 61

PAGE 8

8 LIST OF FIGURES Figure page 1-1 The three-stage Caution Induction Framework................................................................. 25 2-1 The display seen by th e participant for the. ....................................................................... 37 2-2 The display seen by the participant in the.......................................................................... 39 2-3 Average moves times in milliseconds by group................................................................ 46 3-1 Average time per move for each of the four...................................................................... 59 4-1 Average time per move for each of the four...................................................................... 71 4-2 Illegal moves for each group were presented.................................................................... 73

PAGE 9

9 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy IMPROVING PROBLEM SOLVING EFFICIEN CY: THE WHAT AND HOW OF CAUTION By Martin E. Knowles December 2008 Chair: W. Keith Berg Major: Psychology A three-stage framework was proposed descri bing the reduction of illegal moves in a problem solving situation. The Caution Induction Framework (CIF) proposed that consequences for making or avoiding illegal moves in the first stage of the framework influenced the problem solver to increase the amount of attention devoted to the problem in the second stage. This increase in attention subsequently increased ch ecking or evaluation behavior in the third stage resulting in the reduction of illegal moves. Three experiments were presented to test sp ecific aspects of the framework. The first experiment tested whether a punishment was necessa ry to reduce illegal move s or if threat alone, without punishment, was sufficient to yield the same result. The second experiment tested whether an aversive stimulus was necessary or if a positive consequence would yield illegal move reductions. Specifically, participants in th e critical condition in the second experiment received candy for avoiding illegal moves and fo r solving the problem efficiently. The third experiment tested whether an instruction to chec k each move for legality would result in illegal move reductions. In addition, participants in the critical condition in the third experiment were asked to perform this behavior aloud while thei r voice was recorded. The results from the three

PAGE 10

10 experiments mostly supported the proposed fram ework of CIF; however, future research is required to further test the assumptions of the framework.

PAGE 11

11 CHAPTER 1 INTRODUCTION People solve problem s every day in a variety of settings. Unfortunately, these problems are not often handled as well as one would like. A persons mistakes on a problem may waste both time and effort. An important goal for psychologists is therefore to (1) determine what difficulties problem-solvers may encounter when working on such problems, and (2) develop techniques for overcoming these difficulties. Kotovsky and colleagues extensively studied the factors that contribute to pr oblem difficulty (Kotovsky, Hayes, & Simon, 1985; Kotovsky & Simon, 1990). However, less attention has been devoted to overcoming these difficulties and improving problem-solving performance. What makes a task a problem is that the so lution is not immediately available to the problem-solver. If the soluti on were immediately evident then it would not be a problem; it would be a task that needed to be complete d, but not solved. The solution is unknown, so the problem solver must take action to find the solu tion and this requires that the problem solver interact with the problem physica lly and/or mentally. Since know ledge of the problem is limited, the problem solver will often make several inco rrect moves. These incorrect moves may include legal moves that take the problem -solver down the wrong path (e.g., turning south down a street when the final destination is north does not violate any laws, but is incorrect because it takes us away from our goal). Incorrect moves may also be illegal because they violate one of the rules of the problem (e.g., turning the wrong way down a one-w ay street violates a rule because it breaks the law, even if it would bring us closer to our final destination). An illegal move, as the author has referred to them, may carry different consequences depending on the problem. It may have severe consequences, such as the termination of the problem or in the most extreme cases even injury or death (e.g., runn ing a red traffic light

PAGE 12

12 and causing an accident). However, the rule violation may have less severe consequences that result in a minor penalty (traffic fine) or a si mple warning (verbal warning to stop at all red traffic lights in the future). This dissertation was focused on two goals first, to understand the consequences that aid a problem -solver in reducing illegal moves, and second to understand how the problem solvers behavior wa s modified by those consequences. The prevalence of illegal moves and potential theories of how and why illegal moves are made will be discussed first. Next a literature review providing insight into illegal move reductions will be reviewed. Then an in-depth overview of Knowles and Delaneys (2005) problem-solving research will be provided as it ac ted as a basis for much of the work that was presented here. This will be followed by an ove rview and in depth presentation of the CIF framework for the reductions of illegal moves propos ed in this dissertatio n. Finally, this chapter will be completed with a brief preview of the three experiments that were conducted. Illegal Move Commissions: Frequency and Proposed Causes Illegal m oves may often account for a signifi cant portion of a problem-solving episode. Knowles and Delaney (2005) reported that upwards of 20% of the total moves in the hobbits and orcs problem were illegal and Jeffries, Pols on, Razran, and Atwood (1977) reported illegal moves rate averages up to 32.8% on similar proble ms. Therefore, techniques that reduce illegal moves would likely reduce solution lengths and possibly the time required to complete the problem. To develop techniques for reducing illegal moves it would first be beneficial to understand the reasons why problem-solvers select illegal moves. Research suggests three major causes for illegal moves. The first cau se is related to understanding of the problem, the second to limitations on our mental resources, and the thir d is related to the spec ifics of peoples moveselection heuristics. I will consider each in turn.

PAGE 13

13 Proposed Causes: Understanding One reason for the selection of illegal moves could be that the problem-solver does not understand the problem fully or the rules that result in an illegal m ove. Understanding the problem refers to knowledge of the goal of the ta sk and how to reach that goal. Understanding the rules refers to knowledge of the parameters in which a problem-solver must work in to solve the problem. The rules are the re strictions set up by the environm ent or those explicitly stated. Kotovsky and Simon (1990) found illegal moves when the problem solver had trouble understanding the problem or more specifically trouble understanding what constituted an actual move. However, additional research has found that problem-solvers select illegal moves even when the problem and the rules of the problem are well understood (Jeffr ies et al., 1977; Zhang & Norman, 1994; Knowles & Delaney, 2005). People often make illegal moves, but the reasons why remain unclear. Proposed Causes: Resource Limits One major explanation for il legal moves was that problem-solvers have resource limitations. Exactly what these resource limitations are has not always been specified, but usually the term resources refers to working or short-term memo ry limits, attentional capacity, and thinking speed. Jeffries et al. (1977) proposed that illegal moves were selected due to resource limitations that preven ted problem-solvers from correctly calculating future states or from checking moves for legality at all. Reso urce limitations have also been proposed for performance deficits, such as a lack of pla nning, in other tasks as well. Atwood and Polson (1976) attributed a lack of pla nning in water jugs problems to short-term memory limitations. A lack of planning was also obtained by OHara and Payne (1998) in re search involving the 8-

PAGE 14

14 puzzle. This lack of initial planning seems to support the claim that problem solvers do not have the resources to plan complete solutions. Recent research, however, indicates that pr oblem solvers often do have the resources necessary to plan complete solu tions and to avoid illegal move s. Delaney, Ericsson, and Knowles (2004) found that when instructed, participants were able to plan complete solutions to difficult water jugs problems without any feedback. OHara and Payne (1998) also found that when the cost of making a move increased on the 8-puzzl e, participants engaged in planning more frequently. Furthermore, Knowles and Dela ney (2005) found that in creasing the cost of selecting illegal moves subsequently reduced th e number of illegal moves made, even though the resources necessary for calculating future states and for remembering to check each move for legality were not altered. These findings indicate d that participants have the resources necessary to perform more efficiently on these tasks. It is still unclear why illegal moves are made if the resources are available to avoid them. Proposed Causes: Move Selection One additio nal area of previous research that has received attention in the problem-solving literature is modeling how a pr oblem-solver selects each move. Understanding what was done by the problem-solver in selecting a move would be of great value to understanding how and why illegal moves are made. However, how a problem-s olver assesses a move as acceptable or not is not entirely understood. Researchers have pr oposed models to explain how moves were evaluated and selected. One model that can be used to explain the evaluation process that problem-solvers may have engaged in on such tasks as the hobbits and orcs problem was proposed by Jeffries, Polson, Razran, and Atwood (1977). In this model the evaluation function: ei = aMi + bCi + cPi (1-1)

PAGE 15

15 was used to determine if a move was acceptable and exceeded the required criterion. In Jeffries et al.s evaluation f unction (Equation 1-1) ei was the value of the evaluation function for state i in which there were Mi hobbits, Ci orcs, and Pi hobbit-orc pairs on the right bank and a b, and c were constant positive weighting factors. This model consisted of a three-stage process and in the first stage the problem-solver evaluated potential moves according to a specific noticing order. Noticing order referred to the idea that the problem-solver would al ways consider specific combinations of travels first as these combinations would likely have a higher probability of making greater advances towards the final goal state. If a move was evaluated and had a higher value than the cu rrent state and had not been visited before then it would have been selected with a specific probability i + 1, where i was the number of moves that had been considered a nd rejected during this ep isode. If the move was recognized as being previously visited then it was selected with probability If no move was selected in Stage 1 then the model attempted to find a move to a novel state in Stage 2. If all moves were identified as previously visited then the model attempted to find the optimal move in Stage 3 or it randomly selected a move in this stage. Once a move was selected in one of the three stages it was then checked for legality with the models illegal move filter. The resulting st ates of moves were checked for legality. A move was rejected if it was determined that the move was illegal and executed if it was determined that the move was legal. Jeffries et al.s (1977) il legal move filter descri bed the probability of checking a move for legali ty as fixed probability 1 and the probability of correctly rejecting an illegal move as fixed probability 2. Jeffries et al. also proposed that there were hard-to-detect and easy-to-detect illegal moves and that easy-to-de tect illegal moves were always rejected. This

PAGE 16

16 was one of the first models the author found th at specifically addresse d how a problem-solver may identify and reject illegal moves. Another model that was designed to provide an understanding of how moves were selected was Lovetts (1998) ACT-R model of choice. L ovett & Anderson (1996) have also used ACT-R to understand how the prior expe riences of success and failure affected future decisions on the Building Sticks Task (BST). Lovetts model assu med that problem-solvers selected their next move based on the highest expected gain according to the following equation: E = PG C (1-2) where E was the expected gain of the selected move, P was the estimated probability of achieving the productions goal, G was the value of the goal, and C was the estimated cost to be expended in reaching the goal. Lovett and Anderson (1996) presen ted participants with the building sticks task (BST) where participants had to add and subtract three different size sticks to create a stick equal to a specified target length (similar to the water jugs task used by Luchins, 1942). In this task the participants solved several of these puzzles and the authors were particularly interested in participants tendency to select one stick over the others as thei r first move and how previous successes and failures of beginning w ith different sticks affected th is decision. Lovetts model of choice, as applied to BST, focused on a probl em-solvers ability to reflect on these past successes and failures of operators to calculate E from applying an operator. This model was designed for BST, but could be modified to mode l move selections on other tasks as well. Many other move selection models exist for various prob lems, but to review them all here would not be practical.

PAGE 17

17 Next I will turn to a discussion of studies that attempted to reduce illegal move rates. The amount of research in the problem solving literatu re that specifically l ooked at reducing illegal moves during a problem solving episode was limite d. However, there were several works that used the number of illegal moves committed as a dependent measure. Such research provides insight into how problem-solvers reacted to specific manipulat ions and allowed for speculation on how problem-solvers appro ached and solved problems. Early Research Two relevant studies which pr esented m anipulations that re duced illegal moves provided insight into increasing problem solving efficiency. Zhang and Norman (1994) changed the problem constraints in isomorphs of the Tower of Hanoi problem from internal rules (verbal or written constraints that were to-be-remembered : e.g. the tall peg cannot be placed in the middle hole even though it fits) to external rules (constraints not expl icitly stated because they were embedded/implied by the physical environment: e.g. the pen cannot be pushed through the eye of the needle). Not surprisingly, changing the pr oblem so that the environment prevented participants from making illegal moves resulte d in illegal move reductions. Kotovsky and Simon (1990) took a very challenging problem where th e available move options were not easily understood and presented part icipants with all the le gal options at every problem state. With the legal options available participants reduced the number of illegal moves they made on isomorphs of the Chinese Ring puzzle. These studies presen ted manipulations that reduced illegal moves and provided insight into problem solving perfor mance. However, in both cases the problem was altered and could have potentially reduced the resources required to avoid illegal moves. Therefore it is difficult to determine whether e ither study lent support to the resource limitation hypothesis or not (Jeffries, Polson, Razra n, & Atwood, 1977). In a ddition, changing the environment (making internal rules external) and presenting additional information (providing

PAGE 18

18 participants with all potential legal moves), respectively, to reduce resource requirements will not often be possible in the re al-world and such manipulations may alter the problem itself. Ultimately the limitations of these works to address the illegal move reductions made these results difficult to apply to other proble ms in the laboratory or real world. Although the occurrence of illegal moves accoun ted for a significant portion of total moves, in at least some problems, the reducti on of illegal moves in problem-solving does not appear to be widely studied. This may have b een due to the idea that the resource limitation hypothesis (illegal moves occurred because we ha ve a limited number of resources available while working on a problem, thus illegal moves were inevitable) was so widely accepted and therefore there was little that could be done to improve ille gal move commissions. Another possibility was that it may have been overlooked that illegal move s played such a significant role in problem solving episodes. A third possibi lity was that the link between illegal move reductions on laboratory controlle d experiments and the real world application of these findings were not obvious and thus such re search did not seem beneficial. Knowles and Delaney (2005) The most extensive research on illegal m ove reduction was conducted by Knowles and Delaney (2005). An overview of that paper wa s offered here for two main reasons; (1) they intentionally tested manipulations that reduced illegal moves and (2) their work acted as a basis for much of the work presen ted in this dissertation. Specifically, Knowles and Delaney proposed a 3-stage framework in an attempt to understand where and how different manipulations affected a pr oblem-solvers selection and rejection of an illegal move. They presented three experiments demonstrating ways to reduce illegal moves on isomorphs of the missionaries a nd cannibals or river cr ossing problems. They

PAGE 19

19 looked at what knowledge was transferred when solving the same problem twice and knowledge that was transferred to a novel isomorph. Finally, they intr oduced the idea that caution (this term is subsequently defined) was a result of their manipulations a nd that this caution subsequently led to the reduction of illegal moves. Knowles and Delaney (2005) presented a hierarch ical 3-stage framework in an attempt to understand how and where manipula tions would affect a problem-s olver in their selection and rejection of illegal moves. The first stage aske d whether or not an illegal move came to mind ( Generation-Rate Hypothesis). If an illegal move did not then a legal move would be selected. If an illegal move did come to mind then the pr oblem-solver moved to the second stage which asked whether or not the problem-solver checked the rules to see if the move was legal ( Caution Hypothesis ). Caution which described this stage, referre d to the increasin g likelihood that a candidate move would be checked for legality. If the rules were not checked then the illegal move was committed. If the rules were checked then the problem-solver moved to the third stage which asked whether the rules were correctly checked ( Rule-Verification Hypothesis ). If the rules were not checked correctly then an illega l move was committed. However, if the rules were correctly checked then the illega l move would have been rejected and avoided and another move would have been considered. This framework desc ribed illegal move rejec tions and allowed their experimental manipulations to address differe nt stages/hypothesis of the framework (the Generation, Caution, Verification framework of Knowles and Delaney was referred to as GCV herein). In Experiment 1, Knowles and Delaney (2005) punished participants for illegal moves on the hobbits and orcs version of the river crossing problem to assess particip ants ability to avoid illegal moves. In addition, part icipants were instructed to think aloud while working on the

PAGE 20

20 problem in an attempt to discover when and how pa rticipants selected and rejected illegal moves. When compared to a control group Knowles and De laney found that those pa rticipants that were punished made fewer illegal moves while the numbe r of legal moves made did not differ. When compared to a silent control group the legal a nd illegal moves for participants that were instructed to think aloud while so lving the problem did not differ. These findings indicated that participants were able to avoid illegal moves when their attention was dire cted to do so and that thinking allowed had minimal to no eff ect on problem solving performance. Experiment 2 was similar to Experiment 1 in th at participants solved the same problem and the experimental group was penali zed for making illegal moves wh ile the control group was not penalized. However, there were two main diff erences; (1) there was no think aloud condition, all participants solved the problems silently, (2) each participant solved the same problem twice where the second solution attempt was solved with no penalties regardless of the instructions on the first problem. The results replicated those of Experiment 1 in that the penalty group made fewer illegal moves while the number of legal moves did not differ on the first solution attempt. The control group made fewer illegal moves on the second solution compared to the first, but there was no change in illegal moves for the penalty group. One of the more surprising results was that the benefits from being penalized on th e first problem were sustained and carried over to the second solution attempt as the illegal moves made did not differ between solution attempts. The lack of a decrease from the first solution attempt to the s econd may have indicated a floor affect. Having each participant solve the same problem twice allowed the authors to assess if there were any sustained benefits from solving the first problem under penalty instructions. This also allowed for the comparison of the change in legal and illegal moves with practice on the same problem.

PAGE 21

21 Experiment 3 was similar to Experiment 2 except that the second solution was replaced with an isomorph of the first problem. This s econd problem had the same underlying structure as the first problem, but did not have any outward ly obvious similarities. As in Experiment 2 regardless of the instructions on the first solution attempt all participants solved the second problem without penalty. The resu lts largely replicated those of the first two experiments where the number of legal moves did not significantly differ between the penalty and control groups, but the penalty group made fewer illegal moves co mpared to the control. The results revealed that those in the penalty group continued to make fewer illegal moves on the second solution even when they were presented with a novel isomorph and no penalty. In addition, practice on the first problem did not reduce legal or illega l moves on the second problem and indicated that little or no learning was transferred from one isom orph to the other. These findings indicated that something that was not problem specific, potenti ally some type of general problem solving knowledge, was learned and transferred to the se cond solution attempt in the penalty group. Experiments 2 and 3 asked participants to solve two problems and provided information as to what was learned and transferred. In Experi ment 2, participants solved the same problem twice and made fewer illegal moves on the second solution attempt. In the third experiment participants solved one problem and then solved a novel isomorph of that problem, but this did not significantly reduce illegal moves on the second solution attempt. These findings lend support to the idea that problem specific learning gained from pr actice, and not general problem solving knowledge, was transferred to the second solution in Experiment 2 as there was lack of transfer of learning to a novel isomorph from practice alone in Experiment 3. However, in Experiment 3 the illegal move reductions, which followed a penalty, were transferred to a novel isomorph indicating that some type of gene ral problem solving learning was acquired and

PAGE 22

22 transferred to a novel problem as a result of being penalized. This general problem solving knowledge was of great intere st to Knowles and Delaney. One main focus of the work by Knowles and Delaney was on understanding how the penalty manipulation affected part icipants in illegal move reductions. The second stage of their framework gave rise to the Caution Hypothesis which stated that over time a problem-solver would potentially be more likely to check candida te moves for legality, thus reducing the number of illegal moves committed. The authors stated that this hypothesis seemed the most intuitively valuable to test because it likely allowed for ge neral learning of a tendency to check moves for legality that could potentially be transferred to a novel proble m. Knowles and Delaney conducted a regression analysis on legal moves, illegal moves, and illegal moves considered (but not made). The analysis revealed that legal moves accounted for significant variance (the more legal moves made the more illegal moves made), lending support to the Generation Rate Hypothesis. The analysis also revealed that once legal moves we re statistically controlle d the number of illegal moves considered and correctly re jected were inversely related to the number of illegal moves made (the more illegal moves correctly rejected the fewer illegal moves were made), lending support to the Caution Hypothesis. Based on these results caution may have played a large role in increasing problem solving efficiency and received additional attention in this dissertation work. In summary, Knowles and Delaney (2005) pr ovided invaluable groundwork for the current work presented here. Their work demonstrated that illegal moves could be reduced during a problem solving episode when the cost of ma king a move was increased (penalty for making illegal moves). Their results also indicated that problem specific learning was transferred to the same problem (Experiment 2) and that general problem-solving learning ga ined through penalty

PAGE 23

23 enforcement was transferred to a novel isomor ph (Experiment 3). The questions that this research led to, which were highlighted in the current work included: (1) could other aversive manipulations besides penalty lead to illegal move reductions, (2 ) was an aversive manipulation required for reductions or could positive mani pulations yield the same results, (3) were participants checking moves for legality more ofte n or was their accuracy of those they checked improving over time? Caution Induction Framework (CIF (pronounced ch f )) Knowles and Delaneys (2005) GCV framewor k proposed that a punishment for illegal moves induced what they termed caution which was defined as an increased frequency of checking moves for legality. However, caution ma y be only one of the factors that explained why fewer illegal moves occurred after punish ing them. I will therefore present the Caution Induction Framework (CIF) as a heuristic tool for laying out what behaviors might have contributed to the effects of punishment on illegal moves. The CIF framework will provide a basis for the empirical tests to follow, and lay the groundwork for a future program of research on illegal move reduction. A brief overview of the CIF framework will be provided first to facilitate a deeper base understanding of the framework. Th e framework has three stages ( C onsequence, I nvolvement, and F acilitation), as shown in Figure 1-1. The first stage ( consequence ) referred to the external manipulations that affected problemsolving performance. The second stage ( involvement ) referred to the result of the manipulations that occurred in the first stage. The assumption of the framework was that the consequences in Stage 1 resulted in increased attention and thus the problem-solver became increasingly involve d in the problem. The third stage ( facilitation ) specified what behaviors were altered to decr ease illegal move commissions and that these changes were a result of increased atten tion devoted to the problem in Stage 2.

PAGE 24

24 The third stage of CIF begins with the indication that increased attention may have resulted in increases in either the fre quency or precision of certain beha viors. That is, problem-solvers may have performed these behaviors more often or more accurately to reduce illegal moves, respectively. The two main behaviors specified in CIF are evaluation and checking. These are conceptually similar to the evalua tion functions and illegal move f ilter from Jeffries et al. (1977). Evaluation refers to the considerat ion of moves to determine which move has the highest potential of assisting the problem -solver in advancing towards the final goal state. Evaluation occurs prior to the selection of a candidate move and once a move is selected as the best potential move it might or might not then be submitted to be checked using GCV. Checking refers to testing a potential move for legality after the move has been selected, but prior to being executed. The checking behavior used by problem-solvers incorporates the GCV framework from Knowles and Delaney (2005). Stage 1 (Consequence) In Stage 1 (the consequence stage), the experim enter provid ed additional instructions or consequences for performance on the problem. Prev ious research has dem onstrated that a minor punishment (rating words for pleasantness for 30 s) occurring immediately after every illegal move subsequently reduced the rate of illega l moves (Knowles & Delaney, 2005). However, it may be possible to obtain similar effects with less intrusive methods such as a threat or reward. Threat and reward are included in CIF, but it was not yet known whether these factors had the same effect on illegal move commissions as punishment. For now, threat and reward are assumed to have the same effect as punishment, but this is further addre ssed after completion of the experiments. CIF makes the assumption that a ny consequence of illegal moves in Stage 1 act upon the problem-solver to influence Stage 2.

PAGE 25

25 Stage 2 (Involvement) In Stage 2 (the involvement phase), CIF assum es that problem-solvers have the ability to perform better than they do and this is a re sult of the problem-solver not becoming fully involved/engaged in the problem or the problemsolver not allotting e nough attention to the problem. The amount of attentional resources possessed by a problem-solver and how these resources are distributed across the problem were of importance to understanding human problem-solving. However, understanding the amount of available attention and how it may be allotted to a problem remained outside the scope of this project. The assumption of CIF is that the attention in Stage 2 is increased and that th is results in increased activity in Stage 3 and ultimately a reduction in illegal moves. Figure 1-1. The three-stage Ca ution Induction Framework (CIF) demonstrating the process of reducing illegal moves in a problem solving episode.

PAGE 26

26 Stage 3 (Facilitation) Stage 3 (the facilita tion phase) is a more complex stage compared to the rest of the framework and relies on increased attention passed on from Stage 2. Due to the complexity of Stage 3 the terms accuracy and frequency are firs t defined followed by subsections that explain checking and evaluation. The checking subs ection discusses the Generation, Caution, Verification Framework from Knowles and Dela ney (2005) and how increases in accuracy and/or frequency may affect problem-solving performance. The evaluation subsection does not focus on specific evaluation functions used by probl em-solvers to reduce illegal moves-it only assumes that evaluation of moves occurs and that increasing the frequency and quality of these evaluations indirectly reduces illegal moves. If pe ople select moves randomly (with minimal evaluation), then one might expect a greater frequency of illegal moves compared to people who carefully consider what moves to select. With the increased attention passed on from St age 2 the problem-solver either increases the accuracy or the frequency of either checking or evaluation behaviors, which reduce illegal moves. An increase in accuracy means that the problem-solver performed this behavior with fewer mistakes compared to previous attemp ts resulting in an increased likelihood of not committing an illegal move. For example, if you check an illegal move for legality ten times, but only correctly reject the illegal move three tim es (a rate of 30%) th en increasing correct rejections to five out of the ten (50%) would increase the number of illegal moves rejected from three to five. An increase in frequency simply means that the problem-solver performed this behavior more often. For example, if you only ch eck three out of every te n moves for legality -and you always correctly reject illegal checked moves -then increasing frequency to checking five out of every ten moves would increase the numb er of correct rejections from three to five.

PAGE 27

27 Either checking or evaluation behaviors can be increased in their accuracy or their frequency, resulting in fewer illegal moves. It is also possible that any combination of increasing the accuracy and/or frequency of either checking and/or evaluation simultaneously would result in reducing illegal moves. Checking Checking refers to a problem -solvers verify ing whether the resulti ng state of a potential move is valid or invalid according to the rules of the problem. If checking never occurs then the consideration and selection of an illegal move would likely always result in the commission of an illegal move. Knowles and Delaney (2005) pr ovided us with a framework addressing how an illegal move was correctly rejected. To correctly re ject an illegal move there were three steps that must have occurred according to Knowles and Delaney. An illegal move must have come to mind and been selected as a pot ential move, the problem -solver must have remembered to check the move for legality, and the problem-solver must have accurately checked the resulting state of the move and correctly rejected the illegal move This framework, which is referred to here as the Generation, Caution, Verification (GCV) fr amework, was taken from Knowles and Delaney (2005) and has been applied to CIF in an attempt to understand how illegal moves were reduced following a penalty or another t ype of consequence (GCV is displayed in the box under the Checking oval in CIF in Figure 1). Illegal moves can be reduced if the problem-solver becomes more accurate during the checking process. This increase in accuracy woul d have its effect in the third colored box of GCV. Even though a problem-solver checks a move for legality that does not guarantee that the move will be correctly rejected. The problem-s olver may miscalculate the future state and erroneously think that the move is legal. Jeffries et al. (1977) proposed that calculation errors were due to resource limitations. However, the author believed that this was more likely due to

PAGE 28

28 insufficient resources being allott ed to the process. With increas ed attention the problem-solver may allot additional resources to the checking pro cess to correctly calculate the resulting state, resulting in more correct re jections of illegal moves. Problem-solvers may not only become more accurate at checking moves, but they may check moves for legality more frequently. This increase in the frequency of checking would have its effect in the second colored box of GCV. Wi th minimal attention allotted to the problem, problem-solvers may not bother to check the legali ty of moves at all. They may select a move and then execute it because there is little cost if the move proved to be illegal. However, when the cost of making illegal moves increases the problem-solver may attempt to verify the legality of every move before it is executed. Even if the accuracy of checking does not increase the number of illegal moves should still be reduced by the increase in the frequency of checking. Evaluation Stage 3 of CIF indicates that illegal moves could be redu ced through evaluation; however, the current work does not explore evaluation an d m ade no assumptions as to how a problemsolver evaluate moves. Determining the specific evaluation method used seems premature at this point, but it has been included for framework completeness. Ev aluation refers to problemsolvers assessment of a move to determine if th e move they have selected reaches a selection criterion specified by the problem-solver. How the criterion is set and how each move is calculated is not fully kn own and is discussed in more detail below. Previously, in the Proposed Causes: Move Selection section a simple move evaluation model was desc ribed (Jeffries et al., 1977). CIF did not currently make any specific assumptions or proposals, other than those previously discussed in the GCV framework or t hose otherwise explicitly stated herein, as to how moves are evaluated. It assu mes only that evaluation occurs and that the problem-solvers goal is to select the best candida te move that will advance toward s the goal. If the criterion is met

PAGE 29

29 then the move is considered to be acceptable. If more than one of the considered moves meets or exceeds the criterion then the potential move that exceeded the criterion by the greatest margin is selected. The selected move is then either ex ecuted or checked for legality prior to being committed. As with checking, illegal move reductions could potentially o ccur through either increasing the frequency or the accuracy of eval uation or both. With minimal attention devoted to the problem, problem-solvers may select move s without first evaluating them. This would likely result in the selection of several poor moves, which should be rejected, but are executed. Poor moves are defined as moves that do not bring the problem-solver closer to the goal state and this includes moves that are illegal because they would never advance the problem-solver towards the goal state. However, several ille gal moves in the problem space may appear to advance the problem-solver closer to the goal; so would increased frequency of evaluation result in more illegal moves? This is a fair point a nd describes a real possibility; however without a specific evaluation function being used it is not possible to determine the probability of this occurring. However, although a specific move eval uation function is not assumed it would likely be the case that prior experience would affect subsequent moves. Prior experience would allow the problem-solver to reject illegal moves based on the premise that they have been previously committed or previously considered and correctly rejected. Therefore, with an increase in the frequency of evaluation the problem-solver would likely learn from previous mistakes and avoid additional illegal moves. Problem-solvers may not only evaluate more frequently, but they may increase their accuracy at evaluation. To increa se the accuracy of evaluation the problem-solver may increase the criterion a potential m ove must meet before it is selected With a more stringent assessment

PAGE 30

30 of candidate moves the problem-s olver would likely reduce not onl y the rate of illegal moves, but legal moves as well. One way illegal moves could be reduced would be if the increased criterion required that a move be legal. Such a requirement w ould likely involve the checking of candidate moves for legality as part of the evaluation process. Legal moves would be reduced because the increased criterion s hould avoid, in most cases, revisi ting previously visited states and legal moves that take the pr oblem-solver away from the goal. CIFs evaluation portion is not specifically tested therefore thes e claims can not be supported and additional research is required to determine the true effect of increasing the frequency and/or accuracy of evaluation on such problems. Stage 3 of CIF received the most attention as it represents the place in the problem-solving episode where illegal moves are rejected. As is displayed in the framew ork, these rejections could occur through several diffe rent behaviors. The two prim ary behaviors are checking and evaluation and increasing the accuracy or freque ncy of either of these behaviors would likely result in illegal move rejections. In addition, in creases in evaluation coul d likely also influence checking behaviors as they may be engaged in following the evaluation of a candidate move. Although Stage 3 received the most attention and appears to be th e most important it also proved to be the most difficult to experimentally test as it is based upon untes ted assumptions of human problem-solving. CIF Summary In summ ary, CIF is a 3-stage framework that attempts to provide a better understa nding of how consequences influence behavior to redu ce illegal moves and increase problem solving performance. In Stage 1 the framework is re stricted to only a few potential testable manipulations that may help to reduce illegal moves. Stage 2 is built upon the previous findings that problem-solvers may have greater abilities and additional resources to avoid illegal moves

PAGE 31

31 and plan ahead (Knowles & Delaney, 2005; Delaney, Ericsson, & Knowles, 2004; OHara & Payne, 1998). The third stage looks at two primary behaviors that w ould likely result in illegal move reductions, checking and evaluation. In ad dition, the third stage built upon the previous work of Knowles and Delaney and incorporated their GCV framework. According to CIF the evaluation of a move may involve checking, but checking is not required for evaluation to result in a reduction in illegal moves and checking may occur and reduce illegal moves independent of evaluation. If a participant evaluates a move, but does not check the move for legality they may still benefit from the ev aluation and select better moves that are more likely legal, resulting in the re duction of illegal moves. A participant may also select a move without evaluating that move but still check the legality of the move and increase the probability of identifying the move as illegal, ultimately rejecting that move. In conclusion, there are four questions of impor tance that needed to be addressed according to CIF: (1) if threat and rewa rd yield the same result as punishment in Stage 1 (reduced illegal moves) (2) the amount of attention a problem-s olver devotes to the pr oblem after punishment, threat, and reward in Stage 2 (also how this attention may be devoted to different areas of the framework) (3) what actual behavior(s) problemsolvers engage in to reduce illegal moves in Stage 3 and (4) determining the evaluation process a problem-solver engages in when considering a move in Stage 3. This project attempted to answer th e first question to determine if less aversive, less intrusive techniques could be substituted for punishment and yield similar results (Experiments 1 & 2). The finding that th reat and reward have the same effect as punishment would support the idea th at an increase in attention and not avoidance behavior was responsible for illegal move reduction. The second question was outside the scope of this project and should only be considered after it is determin ed that an increase in attention occurred after

PAGE 32

32 the manipulations in Stage 1. Attempting to dete rmine the amount of attention devoted to the problem without first verifying that increased attention was responsible for the change in behavior would have been pointle ss. The third question was also addressed in this project to determine what behavior(s) problem-solvers engage d in after the manipulations in Stage 1 of CIF (Experiment 3). Understanding how problem-solvers were able to reduce illegal moves would enable us to discover better ways of increasing this behavior to improve problem solving efficiency. The fourth question was also outside the scope of this project and should only be considered after it is determined that probl em-solvers engaged in evaluation processes. Attempting to determine the evaluation process used to select moves without verifying that an evaluation process took place woul d have also been pointless. Experiments Three experim ents were presented in an at tempt to obtain a deeper understanding of the role of caution in a problem-solving episode and how it changed base d on the interaction between the problem solver and the problem. Th ese experiments aided in determining what was required to induce caution and also helped determ ine what behavior was observed as a result of being cautious. The first experiment examined whether a threat of punishment without any punishment was sufficient to induce caution. Th e second experiment examined whether an instruction to be cautious or a reward for being ca utious without any penalty or threat of penalty was sufficient to induce caution. Fi nally, the third experiment examined how the frequency and accuracy of legality checking was altere d as a result of increased cautiousness.

PAGE 33

33 CHAPTER 2 EXPERIMENT 1 Since illegal m oves have accounted for a significant portion of to tal moves in at least some problems, knowing and understanding ways to redu ce such moves would offer great benefits. In previous studies participants reduced the number of illegal moves they made after being penalized for illegal moves (Knowles & Delane y, 2005). However, these findings inspired the question of what other manipulations may have al so reduced illegal moves. In particular, what other less aversive, less intrusiv e manipulations could have been implemented that would have reduced illegal moves? Less aversive manipulat ions would likely minimize unnecessary effects on the problem-solver and less intrusive manipulat ions would likely be easier to implement due to minimizing interference. Such advantages woul d potentially have real world applications for improving performance in business climates, the military, school, etc. In the first experiment, particip ants in the two critical condi tions were threatened that upon completion of the problem they would be penalized for the illegal moves they committed during their solution. One of the groups experienced the punishment prior to beginning the problem while the other group was simply told of the penalty without experiencing the punishment. Results from these two groups were compared to two additional groups, one that was penalized after every illegal move and one that was not penalized or threatened for making illegal moves. In addition, participants in all four conditions solved a second problem that had no penalty or threat of penalty to determine what learned information was transferred to a novel isomorph. Threat, punishment, and reward were manipul ations made on the problem and this was accounted for in Stage 1 of CIF (Figure 1-1). Th ese manipulations also had a direct effect on Stage 2 of the framework where they increased the amount of attention devoted to solving the problem to deal with the so-cal led threat. People could have ofte n been lackadaisical when they

PAGE 34

34 approached a problem that had little or no c onsequence for performance that was not optimal. This could have been thought of as having the motivation to do well being turned off. However, increasing the consequence or perceived consequence, as with threat, may have increased the amount of attention a person deci ded to devote to solving the problem as was shown in Stage 2 of CIF and this could have b een thought of as turning motivation to the on position. This increase in attenti on or turning on motivation woul d have in turn had an effect on Stage 3 of the framework and would have ulti mately resulted in the reduction of illegal moves. The author believed that if the threat was perc eived as real it shoul d have been enough to influence the participant to engage in the prob lem more fully and should have resulted in an increase in behavior that woul d have reduced illegal moves. Ho w the different manipulations of threat were perceived would determine how much attention was devoted to the problem and to what degree illegal moves were reduced. If the pa rticipant was told the threat, but they did not experience it before beginning the problem, they may have overestimated or underestimated the severity of the threat. If the th reat was overestimated then the effect in Stage 2 would have been maximized and the participant may have reduced illegal moves to the de gree that they were reduced in the penalty condition or even to a gr eater degree. If the threat was underestimated then the effect in Stage 2 would have been minimal and the reduction of illegal moves would have been less than that in the penalty condition and may have been similar to the no-cost control condition. Experiencing the threat before begi nning the problem could have also displayed similar effects compared to the threat group that did not experience the penalty. After completing the initial penalty before begi nning the problem the participan t may have not perceived the penalty as very costly and this would have likely minimized the effect on Stage 2 (little or no

PAGE 35

35 increase in attention). In contrast, the participan t may have perceived the penalty as very costly and this likely would have maximized the eff ect on Stage 2 (large increase in attention). The author predicted that a participant would have been accurate at predicting the cost of the threat even when the threat was not experi enced prior to beginning the problem. This would have resulted in illega l move reductions that were greate r than the no penalty condition and similar to those of the penalty condition. However, it seemed likely that after performing the penalty, but prior to beginning the problem, the participant may have become even more attentive and may have showed ill egal move reductions that were greater than those obtained in the no experience threat condition and closer to those in the pe nalty condition. If illegal move reductions for either of the critical conditions were greater than those in the no-cost condition then this would have provided evidence supporti ng the claim that less intrusive methods were capable of inducing caution. Methods Participants Participants were recruited from General Psychology courses thr ough the human subject pool at the University of Florida. Participa tion was voluntary and each participant received course credit for their particip ation. The duration of the experi ment was approximately one hour and each participant was awar ded two experimental credits. One hundred participants participated in this experiment; they were ra ndomly assigned to one of the four groups. Twenty participants were unable to solv e both problems correctly within the time limit and their data was therefore dropped from the analysis. Participants that were unable to solve both problems correctly in the time allowed were replaced until ther e were 20 participants in each of the four groups.

PAGE 36

36 Problems Hobbits & orcs The hobbits & orcs problem (which was also referred to as the missionaries and cannibals or river-crossing problem ) consisted of one boat and six trav elers, three of which were hobbits and three of which were orcs. A ll travelers began on the left bank of a river, which was near the middle of the computer screen, and the boat bega n on the left bank at the bottom. The goal was to move all six travelers to the right bank of the river using the boat. However, the rules stated that the boat could only hold a maxi mum of two travelers at one tim e, and at least one traveler was required in the boat for it to cross the river. The rules also stated that at no time could the orcs outnumber the hobbits on either bank of the river because the orcs would then kill the hobbits. A button was located on the di splay to reveal the ru les. If at any time a participant forgot one of the rules they could clic k on the button to look up any of th e three rules. An example of the display seen by participants for th is problem was presented in Figure 2-1. The problem was written in Microsoft Visual Basic and presented on a Gateway desktop computer. Participants used the mouse to select icons representing the trav elers to be moved and then selected the boat to send the selected travelers to the other ba nk of the river. After a traveler was selected it appeared at the bottom of the sc reen next to the boat. Clicking on a traveler a second time removed him from the boat and placed him back on the bank in the middle of the screen. If the partic ipant added too many travelers to the bo at, allowed the orcs to outnumber the hobbits, attempted to move the boat with no travelers sel ected, or violate the rules in any other way he/she was then notified via a message box a nd the move did not occur (problem description taken from Knowles and Delaney, 2005).

PAGE 37

37 Figure 2-1. The display seen by the participan t for the hobbits and orcs problem where two hobbits had been selected in the beginning state. Titration The titration problem was an isomorph of the hobbits and orcs problem in which the participant was asked to remove unstable isotopes from a white b eaker. The beaker contained six isotopes, three blue and three orange. The goal of the problem was to remove all six isotopes from the beaker using a dropper to extract them The participant began by removing isotopes and then alternated between adding and removing isotope s thereafter. However, the rules stated that the dropper would hold only two isot opes at one time, and at least one isotope was required in the dropper for the participant to ad d or remove isotopes. The rules also stated that if the number of blue isotopes in the white beaker did not equal the number of orange isotopes in the white beaker then the number of blue isotopes had to equal either zero (the minimum number possible) or three (the maximum number possible) or the isotopes would become unstable and explode. If

PAGE 38

38 at any time a participant forgot one of the rule s, he or she could clic k on the button to display three additional buttons, one fo r each of the three rules. The problem was written in Microsoft Visual Basic and presented on a Gateway desktop. A red Remove or Add button appeared on the screen indicating to the participant whether he/she was to be removing or addi ng isotopes to or from the white beaker at each step. When the Remove button was present the participant would use the computer mouse to click on the isotopes in the white beaker, which would then appear in the dropper. Clicking on the Remove button completed the move and made the Rem ove button disappear. The Add button then appeared along with a large blue and orange beaker. Clicking the blue or orange beaker added the appropriate color isotope to the dropper. Cl icking the Add button completed the move and emptied the isotopes from the dropper into th e white beaker. The Add button and the two colored beakers disappeared, and th e Remove button appeared again. The rules stated that it was impossible to have more than three blue or orange isotopes on the screen at any time and this included the co ntents of the white be aker, the dropper and the colored beakers. If a participant added too many isotopes to the dropper, attempted to remove or add isotopes with no isotopes in the dropper, allowed the isotopes to become unstable, or violated the rules in any other way he/she was notified via a message bo x, and the move did not occur. An example of the display seen by partic ipants for this problem was presented in Figure 2-2. Design Four groups of participants solved two problem s. The major difference between the groups occurred on the first problem when participants were instructed as to the consequences of making an illegal move. Different instructions were provided to each group informing them of the consequences of making illegal moves on th e first problem. However, for the second

PAGE 39

39 problem all four groups received the same inst ructions specificall y, that there were no consequences of making illegal moves. The s econd problem was included to examine transfer effects. The four conditions diffe red on the first problem as follows: Figure 2-2. The display seen by th e participant in the titration pr oblem where one orange isotope had been selected in the beginning state. In the no-cost group, a message box appeared after each illegal move indicating that an illegal move was made. The pa rticipant clicked an OK butt on to continue working on the problem from the last legal state. A participant was not penalized or threatened in any way. This group was identical to the no-cost group used in Knowles and Delaney (2005). In the punishment group, a participant was informed before beginning the first problem that after each illegal move th e problem would be paused and they would have to rate the pleasantness of words for 45 s. After every ille gal move the participant was informed of the illegal move and the screen then turned gray. Words appeared in th e center of the screen and the participant had to rate the words for pleasantness on a scale from 1 to 5 (1 being very unpleasant,

PAGE 40

40 3 neutral and 5 very pleasant). After 45 s the scr een returned to the problem and the participant was allowed to continue working on the problem fro m the last legal state they had reached. This group was identical to the punishment group used in Knowles and Delaney (2005). In the threat group, a participant was inst ructed prior to beginning the first problem that for every illegal move he/she made the penalty of rating words for pleasan tness would accumulate and he/she would have to complete the penalty after solving the problem. That is, for every illegal move the participant w ould have to complete 45 s of rating words for pleasantness. A participant in this condition was not penalized for making illegal moves and did not receive any additional information after making an illegal move. After making an illegal move a message box appeared stating that an illegal move was ma de and the participant wa s instructed to click OK and was then allowed to continue working on the problem from the last legal state. Finally, the experience condition was identical to the threat condition except that prior to beginning the problem the participant co mpleted the 45 s penalty phase once. Procedure Participan ts were tested i ndividually and were randomly a ssigned to one of the four conditions. Half the participants solved the hobbits and orcs pr oblem followed by the titration and the other half first solved the titration problem followed by the hobbits and orcs. To begin, the participant read the cover story for the firs t problem aloud and then read the three problem rules. The participant was asked to review the cover story and the rules and was instructed to study and memorize the three rules. Once the participant was able to recite the three problem rules to the experimenter without error the tutorial phase began. During the tutorial phase, the participant wa s shown an example problem on the computer with instructions explaining how the problem worked and how he/she was to navigate the problem space. The participant was instructed to click on the Forget a Rule? button and then to

PAGE 41

41 click on each individual rule button and asked to recite each rule aloud. The participant was informed that the Forget a Rule? button was on the screen while he/she worked on the problems so that the participant could check the ru les at anytime if he/she so desire. Next, the participant was instructed to move one of each ch aracter to the next position and then he/she was instructed to move both characters back to th e starting place so that the participant could understand how the characters were moved. Then, the participant recei ved instructions on violating each of the three rules in succession and was asked to violate each rule within the display to ensure that he/she fully understood wh at constituted an illegal move. The experimenter checked to make sure that the part icipant had done this correctly. Finally, the participant was instructed to ma ke three legal moves on the practice problem. The practice problem consisted of two hobbits and tw o orcs or two blue and two orange isotopes (instead of three of each as in the experimental problem), depending on which problem he/she solved first. At the end of the tutorial phase the participant was give n the opportunity to ask questions that he/she may have ha d. The main goal of the tutorial phase was to ensure that the participant underst ood the problem, including the rules, the go al, and how to move the travelers. The participant in the punishment condition wa s notified after the tutorial phase of the penalty for violating Rule 3 and received instru ctions on how to complete the word-rating task. The participant was penalized for Rule 3 viol ations only, there were no consequences for violations of Rules 1 or 2. Participants in both the threat and experience conditions were informed that they would have to complete the punishment upon completion of the problem, once for every illegal move. Those in the e xperience condition comple ted a sample of the punishment for 45 s before beginning the problem. In all conditions, testing began when the participant clicked on the mouse to initiate the pr ogram. If a participant was unable to solve the

PAGE 42

42 problem within the 20-minute time limit, that pa rticipant was then assisted in finishing the problem and his or her data was not included in the analyses. The maximum solution time of 20 minutes was chosen to restrict the session length to one hour. After completion of the first problem the pa rticipant was asked to judge how many illegal moves he/she believed he/she had committed on the completed problem. These estimates were used to calculate a difference score between the pa rticipants estimation and the actual number of illegal moves. The difference scores were compar ed between the groups to determine if accuracy differed. Participants in the threat and experience conditions were also informed that they would not have to complete the penalty and would be asked if they believed that they were going to receive the delayed punishment fo r making illegal moves. This question was asked to determine if participants believed the instructions of a delayed punish ment. The participant was then instructed that he/she would now attempt to solve a different problem. There was no indication or mention during the experiment that the two pr oblems were similar in any way. The participant was given a cover story for the second problem where he/she learned the rules and completed a tutorial phase, just as the pa rticipant did for the first prob lem. Before beginning the second problem all participants were instructed that there would be no penalty for illegal moves on the second problem. If a participant was unable to solve the second problem within the 20-minute time limit, that participant was then assisted in finishing the problem and his or her data from both problems was not incl uded in the analyses. Results and Discussion Move data. For this experim ent a 4 x 2 x 2 mixed factorial Analysis of Variance (ANOVA) was conducted: Group (NoCost vs. Punishment vs. Threat vs. Experience) x Order (Hobbits first vs. Titration first) x Solution (F irst vs. Second). Group a nd order were betweensubjects factors and solution was a within-subjects factor. Illegal moves committed on the first

PAGE 43

43 and second problem were of primary importan ce; however the number of legal moves and average time to complete each move were also analyzed. Based on previous findings obtained by Knowle s and Delaney (2005) no main effect of order, no main effect of solution, and no interaction for a ny of the factors for illegal moves, legal moves, and time were predicted. This would s upport the assumption that the isomorphs did not differ in difficulty, the order of presentation did not influence the results, and the results for the first solution did not differ significantly from the second solution. It was possible that a participant could transfer some general learning to the second problem and show a main effect of solution with improved performance on the seco nd solution, but such a result was not found in Knowles and Delaney so such a result was not exp ected here either. It was also predicted that legal moves and total time would not differ signi ficantly between the four groups. However, it was anticipated that a main effect of group w ould be obtained for illegal moves. A 4 x 2 x 2 mixed factorial ANOVA was conducted on legal moves made, average time to complete a move, and illegal moves committed with Group and Order as between-subjects factors and Solution as a within subjects factor. Legal move data. The results for legal moves made re vealed no main ef fect of solution, F < 1, indicating that the number of legal move s on the first and second problem did not differ statistically. There was no main effect of order, F (1,72) = 1.27, p > .05, MSE = 244.42, indicating that the number of legal moves made did not statistically differ based on which problem the participant solved first. There was also no main effect of group, F < 1, indicating that the number of legal moves made between the groups was not statistically different. These results lend support to the pred iction that legal moves remain relatively unaffected by the

PAGE 44

44 manipulations made following illegal moves. Lega l move averages and standard deviations for each group by solution were presented in Table 2-1. Table 2-1. Legal moves made on the first and second solutions were displayed by group and solution and included mean and standard deviation (SD). All independent and pairwise comparisons we re not statistically significant between groups. Group Mean SD Solution 1 No-Cost 25.50 12.60 Threat 20.85 11.43 Experience 24.30 20.98 Punishment 22.40 8.63 Solution 2 No-Cost 23.05 22.56 Threat 20.70 10.04 Experience 25.50 16.95 Punishment 18.35 6.65 Move time data. The total time to complete the problem was divided by the total number of moves for both the first and second solu tion for each participan t in order to derive average time per move in milliseconds (ms). Th e timer for the program that presented the problems began in the experience group when partic ipants first experienced the penalty and did not pause or stop during the penalty in the puni shment group. As a result of these continuous timers adjustments were necessary to account for the timer being engaged while the participant was not working on the problem. The presentation of words in the penalty phase took 45 s -that is, 3 s for each of the 15 words displayed. To account for the continuous ti mer 55 s was deducted from the total time to complete the first problem in the experience grou p prior to calculating move time averages. The 55 s includes 45 s for the presentation of the words during the penalty phase and 10 s for the brief instructions presented prior to the word s during the penalty phase. In addition, 55 s was deducted from the total time for the first proble m in the punishment group for the first illegal move and 45 s for each additional move because it was not required that the instructions be repeated after they were initially understood.

PAGE 45

45 The results revealed no main effect for order F < 1, indicating that average time per move was not statistically different re gardless of which problem was solv ed first. The main effect of solution F (1,72) = 40.68, p < .001, MSE = 21,099,533.55 was significant and the main effect of group F (3,72) = 2.52, p = .064, MSE = 85,590,859.42, approached significance. The Order x Group interaction was not significant F < 1, the three-way in teraction F(3,72) = 1.73, p > .05, MSE = 21,099,533.55 and the Solution x Order interaction were not significant F (3,72) = 2.94, p = .091, MSE = 21,099,533.55, and the Solution x Group interaction was significant F (3,72) = 3.06, p = .034, MSE = 21,099,533.55. One potential explanation for the group and trend not yielding significant results could have been attr ibuted to a lack of power, therefore potential explanations and follow-up an alysis were discussed. Independent samples t -tests conducted on the first solution revealed that th ose in the nocost group ( M = 14,256.50 milliseconds (ms)) took significantly less time to make moves when compared to the experience ( M = 21,665.85 ms) and punishment groups ( M = 20,078.65 ms) with t (38) = 2.99, p =.005 and t (38) = 2.13, p < .05, respectively. Independent samples t -test also revealed that the difference between the threat group ( M = 16,344.68 ms) and the experience group, t (38) = 1.93, p = .062, approached significance. No additional differences were significant on the first or second solution. Figure 2-3 displays average move times for each group and solution. The observed differences in move times may have indicated increased planning and/or attention in the experience and punishment gr oups, which could have potentially supported the CIF framework because increased checking and/ or evaluation would have likely taken more time. Participants in the threat condition may have made moves quicker than those in the experience condition, although not sta tistically, which may have been due to a lack of power. In

PAGE 46

46 addition, the average times for the threat group did not differ significantly from those in the punishment group. These findings may have indicat ed that experiencing the penalty prior to beginning the problem had a greate r effect of increasing attenti on and resulted in either more checking or greater evaluation of future moves. The lack of a significant difference between the no-cost and threat groups may have indicated that because the threat group did not experience the penalty the amount of increased attention generated from the threat may have been less than that generated in the experience condition. Experiment 1: Average Move Times10000 15000 20000 25000 Solution 1 Solution 2 SolutionTime (ms) No-Cost Threat Experience Punishment Figure 2-3. Average moves times in milliseconds by group for the first and second solution. Error bars represent standard errors. An additional explanation for the greater move times in the punishment group could be attributed to the penalties, wh ich occurred during the problem so lving episode. After completing the penalty phase the participant may have ha d an adjustment period of re-engaging in the problem once he/she was returned to the last legal move. This re-engaging may have included recreating their previous m ove so that that specific illegal move could be avoided in the future or it may have involved recalculating all moves from that state. However, th e greater times in the punishment group were more likely due to an in crease in attention becau se similar increased

PAGE 47

47 move times were found in the e xperience group where an adjustme nt period explanation was not possible because the participant was not interrupt ed during the problem solution. In addition, the punishment group did not statistic ally differ from the threat group. The threat group may have also realized some increased attention, though not as much as the experience group. The lack of significant differences on the sec ond solution could have been attr ibuted to a floor effect where the participants were moving at a fairly efficien t pace as a result of experience in completing the first solution. Another explanation for the time differences coul d have been attributed to an error in the design of the computer program that was use d. Because the timer wa s not initially set up properly to automatically pause during the pena lty phase the generalized time adjustment (55 s for the initial penalty phase and 45 s for each subsequent penalty phase) may have led to significant variance compared to the actual times th e participant took for penalties. The participant may have taken more or less than 10 s to learn the penalty rules in the experience and punishment conditions during the first penalty ph ase. The participant may have also taken additional time during each subsequent penalty in the punishment group before beginning the word rating task (words were not generated duri ng this task until the part icipant clicked a button, a 45 s allotment for the penalty phase was based upon the assumption that the participant immediately clicked the word generation button as soon as the penalty screen appeared). Paired samples t -tests for each group comparing average times between Solution 1 and Solution 2 were also conducted. This analysis revealed no difference between solutions for the no-cost group, t (19) = 1.08, p > .05, but significant differences fo r all other groups. In the threat group the participants were significantly faster on Solution 2 ( M = 11,777.38 ms) than on the first solution ( M = 16,344.68 ms), t (19) = 2.95, p < .01. In the experience group participants were

PAGE 48

48 quicker on Solution 2 ( M = 15,614.67 ms) compared to Solution 1 ( M = 21,665.86 ms), t (19) = 4.13, p = .001. Finally, in the punishment group the second solutions ( M = 13,257.43 ms) were also faster than the first solutions ( M = 20,078.65 ms), t (19) = 3.74, p = .001. These results indicated that those participants that were penalized or were concerned with future penalty took longer to make moves on the first solution compared to the second, which would support the predictions of CIF that manipul ations in Stage 1 woul d result in increased attention in Stage 2. Participan ts in the no-cost group, however, performed with similar speed on both problems, most likely because there was no penalty or concern of future penalties, consistent with CIFs prediction th at a lack of consequence in Stag e 1 would result in little or no increase in attention in Stage 2. Whether these results indicate increased planning, cautiousness, or some other mechanism was not something that could have been concluded by these results alone and required additional resear ch. However, these results were promising and indicated that the threat of punishment, whether experienced or not, was sufficien t to produce results similar to those observed when a penalty wa s administered after ever illegal move. These results lent support to the CIF framework that penalty and the threat of a penalty resulted in similar move times, potentially indicating an increase in attention. Illegal move data. The illegal move data revealed no main effect of solution, F < 1, indicating that the part icipant made a similar number of illegal moves on the first and second problem. There was no main effect of order, F < 1, indicating that the participant made a similar number of illegal moves on each problem regardless of which problem was presented first. There was, however, a main effect of group F (3,72) = 3.36, p < .05, MSE = 19.96, indicating that participants in the four groups did not make the same number of illegal moves. Independent samples t -tests for the first solution revealed that the no-cost group made significantly more

PAGE 49

49 illegal moves on the first solution than the threat t (38) = 2.37, p < .05, experience t (38) = 2.31, p < .05, and punishment groups t (38) = 3.15, p < .005, which did not significantly differ from each other. Independent samples t -tests conducted on the second soluti on revealed that the groups did not differ from each other. Results are displa yed below in Table 2-2 for illegal moves. Although none of the two-way and three-way inte ractions were statistically significant it appeared that the trends indicated a potentia l 2-way Solution x Group interaction. On Solution 1, the no-cost group made significantly more illegal moves compared to the rest of the groups which did not differ. Participants in the punishment group ( M = 2.60) made approximately one fewer illegal moves compared to the threat ( M = 3.40) and experience ( M = 3.65) groups, whose participants made a similar number of illegal moves. On the second solution, the number of illegal moves made by participants did not differ significantly between the four groups. However, participants in the threat ( M = 2.70) and punishment (M = 2.65) groups made approximately two fewer illegal moves compared to participants in the no-cost ( M = 4.20) and experience ( M = 4.65) groups. An obvious explanation fo r these differences, which were not statistically significant, was that the experiment lacked the power to detect the effects with 20 participants in each of the four groups. Because these effects were potentia lly not detected due to a lack of power it did not seem unreas onable to expand upon these results. Table 2-2. Illegal moves made on the first and second solutions were displayed by group and solution and included the means and sta ndard deviations (SD). Superscript 3 is significant at p < .005, and all others significant at p < .05. Group Mean SD Solution 1 No-Cost 6.85 1,2,3 5.82 Threat 3.40 1 2.95 Experience 3.65 2 2.16 Punishment 2.60 3 1.60 Solution 2 No-Cost 4.20 4.41 Threat 2.70 2.25 Experience 4.65 7.65 Punishment 2.65 2.08

PAGE 50

50 A potential explanation for the trend that the punishment gr oup made fewer illegal moves compared to the threat and expe rience groups on the first soluti on was that as these two groups made illegal moves and were not penalized the threat of penalty could have worn off over the duration of the solution. The participant could ha ve started off avoiding illegal moves, but as there was no immediate penalty illegal moves c ould have increased as time increased. A second explanation for the trend could have been that the threat in these two groups was not as impactful as the immediate penalty in the punishment group and thus illegal moves rates were higher over the total duration of the problem. Trends on th e second solution attempt were less clear with fewer potential explanations. The threat and expe rience groups were very similar except that those in the experience group completed the pe nalty phase once before beginning. Despite the overwhelming similarities between the two grou ps, including the similar number of illegal moves made on the first solution, participants in the experience group made approximately two additional illegal moves on the second solution. Du e to the fact that the only difference between the groups was the completion of the penalty prio r to beginning in the ex perience group, this was likely responsible for the trend. However, it remained unclear w hy completing the penalty once before beginning the problem on the first solu tion attempt would result in approximately two additional illegal moves on the second solution. Tr ends on the first and second solution were not significant and additional research would be requ ired to determine if these trends would be significant with additio nal power and the cause of such results. Paired samples t -tests revealed no differences between the first and second solution for the threat, experience, and punishment groups, all t s < 1. The paired samples t -test for the no-cost group was also not significant, but showed a trend to wards a decrease in illegal moves on the second problem, t (19) = 1.82, p = .084. As was previously stated a lack of power may have

PAGE 51

51 potentially been responsible for some trends no t being statistically significant. On the second solution ( M = 4.20) participants in the no-cost group made more than two fewer illegal moves on average when compared to first solution ( M = 6.85) performance. This appeared to be a large reduction in illegal moves, relativ ely speaking, but the trend was not statistically significant. This trend could have potentially been due to some type of general problem-solving learning on the first solution that allowed for transfer and i llegal move reductions on the second solution. The illegal move changes from the first to second solution for the other three groups was also not significant, but the changes were smaller compared the no-cost group and seemed to be more in line with the statistical results. Because the t-tests between the first and second solutions for the other three groups were not significant this lent support to th e idea that illegal move reductions on the second solution were sustained even afte r the penalty or threat of penalty had been removed. The results in this section largely replicat ed and supported the results found in Knowles & Delaney (2005). The main effect of both solution and order were not significant and the no-cost group made significantly more ill egal moves on the first soluti on compared to the punishment group. The punishment group, along with the threat group, did not show increases of illegal moves from the first solution to the second, potentially indica ting sustained benefits through continued reductions in illegal moves. The experience group did not statis tically show a change in illegal moves from the first to second solu tion, but the average did increase by one illegal move, 3.65 to 4.65, potentially indicating a lack of power to detect the change. In addition, participants in the threat and experience groups made significantly fewer illegal moves compared to the no-cost group and they did not signifi cantly differ from the punishment group. These results provided support for CIFs assumptions: less intrusive methods of increasing attention

PAGE 52

52 and inducing caution were possible and resulted in similar illegal move reductions compared to the punishment group.

PAGE 53

53 CHAPTER 3 EXPERIMENT 2 The purpose of the second experim ent was to determine whether or not a punishment or threat of punishment was necessary to induce cautiousness. The two critical groups in this experiment received no negative consequences for illegal moves. Instead one group was instructed to do their best and to concentr ate on avoiding illegal moves and the other group was rewarded for avoiding illegal moves and perfo rming efficiently (performing efficiently was explained later in this chapter). Similar to Expe riment 1, both of these groups were compared to a group that was punished for illegal moves and a group that received no punishment. In this experiment there were no negative consequences or threat of negative consequences for the illegal moves in the two critical conditions. Therefore, the results should have provided insight into whether or not negative conseque nces were necessary to induce caution. As in Experiment 1, Stage 1 was manipulated, but this time the consequences were a reward or motivating instruction (as opposed to the threats used in Experiment 1). This manipulation would have then affected Stage 2, where motivation would have been switched on and attention would have been increased acco rding to CIF. This increase in attention would have ultimately resulted in a reduction in ille gal moves. The author pr edicted that there would have been fewer illegal moves in the motivati on and reward groups compared with the no-cost condition that was not punished and did not receive additional instructions or reward. It was also predicted that the reward conditi on would likely reduce illegal move s to a greater degree than the motivating instruction. If a reward or motivating in struction was able to induce cautiousness then this would have supported the claim of the fram ework; manipulations in Stage 1 influenced people to increase their attention and this was what caused a reduction in illegal moves, not punishment or fear of punishment, but an increase in attention to the problem.

PAGE 54

54 Methods Participants The process f or recruiting participants was the same as the first experiment. In Experiment 2, 108 participants took part in the study; each was randomly assigne d to one of the four groups. A total of 28 participants were unable to solve both problems correctly within the time limit and their data was therefore dropped fr om the analysis. Participants were that were unable to solve the problems were replaced until there were 20 participants in each of the four groups. Problems The sam e problems were used as in Experiment 1 -hobbits & orcs and titration. Design Four groups were used in this experim en t: no-cost, punishment, motivation, and a reward group. As in Experiment 1, the groups differed in the consequences of making illegal moves on the first of two problems. All group s received identical instructions when they solved the second problem specifically, they were informed that there was no consequence or reward for avoiding illegal moves. The purpose of fixing the instruct ions for all four groups on the second problem was to examine transfer effects. The manipul ations on the first probl em were as follows: The no-cost group was identical to that used in Experiment 1, and received no special instructions, rewards, or penalt ies for making illegal moves. The punishment group differed only slightly from th e Experiment 1 punishment group. The punishment (rating the words for pleasantness) occurred on a fixed interval schedule rather than immediately after every ille gal move. The particip ant had to complete the penalty every 30 s if a penalty was made in that time frame. When an illegal move was made, the participant saw a message box stating that an illegal move was made and then th ey were allowed to continue working on the problem from the last legal state. Once the 30 s time interval was complete the

PAGE 55

55 screen then turned gray and the participant wa s instructed that he/she recently committed an illegal move and he/she would be instructed to complete the penalty. If an illegal move was not made in the 30 s interval then the participant received no information and continued working on the problem until the end of an interval in which an illegal move was committed, the problem was solved, or time expired. Administering the pena lty at a fixed interval instead of immediately after every illegal move was done to maximize th e similarity between the punishment and reward conditions. In the motivation condition, prior to star ting the first problem, the participant was instructed that the main focus of the experiment was to assess his/her ability to avoid illegal moves. The participant was informed to do their best to avoid illegal moves in an attempt to solve the problem with the fewest number of illegal moves possible. No reward, penalty, or additional instructions were given to the participan t in this group. Finally, in the reward condition, prior to starting the fi rst problem the participant was asked for his/her preference of Skittles or M&M ca ndies. The participant was then instructed that he/she would be rewarded with the candy of choice for avoiding illegal moves and solving the problem efficiently. The participant was told th at the reward would be based primarily on the avoidance of illegal moves, but to maximize the reward he/she should try to complete the problem in a timely manner and in the minimum nu mber of moves possible. A dish was next to the participant and the experiment er deposited one piece of candy in to the dish every 30 s if no illegal moves were made in that time frame and the participant reached a new legal state in that time frame that had not been prev iously visited during the problem solving episode. If in the 30 s time frame the participant reached a new legal state and then back-tracked to a previously visited state the participant was still rewarded with candy as long as an illegal move was not made in

PAGE 56

56 that interval. In addition, the participant received an additi onal candy for completion of the problem and an additional piece of candy for ev ery 30 s interval remaining before six minutes (e.g. three extra candies for completing the proble m before 4:30 because there were three full 30 s intervals remaining before six minutes was re ached). Six minutes was chosen based on the average time to complete the problem obtained in Knowles and Delaney (2005). Procedure The procedu res followed that of the first expe riment with the exception that those in the motivation and reward condition received additional instructi ons before beginning the first problem. After completing the first problem, partic ipants in the motivation and reward conditions were asked if they believed they were going to receive punishment for ma king illegal moves. As in Experiment 1, participants completed th e second problem under no-cost instructions. Results and Discussion Move data A 4 x 2 x 2 m ixed factorial ANOVA was conducted: Group (No-Cost vs. Punishment vs. Motivation vs. Reward) x Order (Hobb its first vs. Titration first) x Solution (First vs. Second). Group and order were between-subject s factors and solution was a within-subjects factor. As in the first experiment, illegal moves committed on the first and second problem were of primary importance, but number of legal move s and time per move were also analyzed. Based on previous findings obtained by Knowle s and Delaney (2005) it was anticipated that there would be no main effect of order, no main effect of solution, and no interaction for any of the factors. This would indicate that the isomorphs did not differ in difficulty, the order of presentation did not influence the results, and the results for the first solution did not differ significantly from the second solution. It was also anticipated that lega l moves and average time per move would not differ signifi cantly between the four groups. However, it was predicted that a main effect of group would be obtained for illegal moves.

PAGE 57

57 Legal move data. The results for legal moves revealed no main effect for solution F < 1 or order F (1,72) =1.20, p > .05, MSE = 286.19. Although the result of the analysis on group revealed no main effect F (3,72) = 2.42, p = .073, MSE = 286.19, it did approach significance. The trend for this analysis was for participants in the reward group to make the most legal moves ( M = 32.03) and for participants in the no-cost ( M = 24.63), motivation ( M = 23.35), and punishment ( M = 23.40) groups to make similar numbers of legal moves. This trend, though not significant, could have potentiall y been due to participants ad apting a strategy of trying to maximize rewards by navigating through legal st ates while avoiding illegal moves, thus increasing legal moves. Previous results have in dicated that legal moves have remained fairly stable during experimental manipulations so this trend was somewhat surprising. Move time data. The procedures for calculating averag e times per move were the same as those used in Experiment 1. The punishment group was adjusted 55 s for the first penalty and 45 s for each subsequent penalty. Penalties occurred at a fixed interval so the number of penalties may have been fewer than the number of illegal move s. If more than one illegal move occurred in a given interval, time was adjusted per penalt y phase. The results for move time data revealed no main effect for solution or order, both Fs < 1. However, the main effect of group was significant, F (3,72) = 2.77, p < .05, MSE = 29,883,813.92, the Solution x Order interaction was significant, F (1,72) = 4.70, p < .05, MSE = 12,203,979.10, and the Order x Group interaction approached significance, F (3,72) = 2.66, p = .054, MSE = 29,883,813.92. The means and standard deviations for each group by solution were displayed in Table 3-1. Independent samples t -tests revealed that the punishme nt group took longer to solve the first problem compared to the reward and motiv ation groups, as displayed in Table 3-1. These results could have been attributed to a more carefree mentality in the reward and motivation

PAGE 58

58 condition due to the motivating instructions or potential rewa rd of candy and potentially increased planning in the punishment group as partic ipants attempted to a void illegal moves. In addition, the penalty in the punishment group occurre d at a fixed interval after an illegal move was made in that interval. So, an alternative explanation could have been that participants were waiting or preparing as they kne w the penalty phase could be initiated at any time and this resulted in increased move times. A final alternative proposed in the first experiment was that after completing a penalty phase participants ma y have required a re-adjustment period when returning to the problem and this would have increased move times. Table 3-1. Means and standard deviations (SD) by group and solution for average times per move were presented in milliseconds. Superscripts 1, 2, 3 were significant at p < .05. Group Mean SD Solution 1 No-Cost 14124.72 5055.92 Motivation 11593.931 3812.97 Reward 12083.44 2 3480.25 Punishment 14938.61 1,2 4186.00 Solution 2 No-Cost 13086.21 4252.88 Motivation 12892.55 5652.86 Reward 10961.40 3 3026.99 Punishment 14578.37 3 6916.07 On the second solution, the reward group re mained significantly quicker than the punishment group, but the difference between motivation and punishment was no longer significant. It was possible that participants in the punishment group were still being cautious and taking additional time planning or preparing fo r a penalty that could have appeared at any moment. Participants in the reward condition may have taken the problem less seriously and been less cautious due to the f act that they were administered candy on the first solution. The distribution of candy on the first solution may have made the problem seem like more of a game than an experiment resulting in le ss attention, planning, and/or caution. The order by group interaction approached significance and post hoc analysis were conducted as the study design may have lacked the power to detect the effect. An independent

PAGE 59

59 samples t -test analysis revealed that participants in the no-cost group may have had longer average move times when solving the titration problem first ( M = 15,325.77) compared to participants that solved the hobbits and orcs problem first followed by the titration problem (M = 11,885.17), t (18) = 2.07, p = .053 as this analysis approached significance. Results from the independent samples t-tests for the motivation t (18) = 1.18, p > .05, reward t (18) = 1.67, p > .05, and punishment groups t (18) = 1.15, p > .05, revealed non-signifi cant results. One potential explanation for this trend could was the idea that those in the no-cost gr oup had no consequences for performance and therefore no reason to devote additional attention towa rds the problem. This lack of additional resources toward the problem may have resulted in more difficulty encoding the titration problem when it was presented first, as it may have appeared more abstract and provided less information to the participants (par ticipants were unable to see the contents of the destination beakers in the titra tion problem whereas participants could always see both banks of the river in the hobbits and orcs problem). Aver age move times for the different groups by order were presented in Figure 3-1. Experiment 2: Average Move Times8000 10000 12000 14000 16000 18000 No-CostMotivationRewardPunishment GroupTime (MS) HO-Titration Titration-HO Figure 3-1. Average time per move for each of th e four groups was displayed for participants that solved the hobbits and orcs (HO) problem first and those that solved the titration problem first. The error bars represented standard error.

PAGE 60

60 Illegal move data. The analysis of the illegal moves made revealed no main effect of solution, F (1,72) = 2.77, p = .099, MSE = 10.99, no main effect of order, F < 1, and no main effect of group, F (3,72) = 1.86, p = .321, MSE = 19.02. In addition, the Solution x Order and Group x Order interactions were not significant, both F s < 1. Neither the Group x Solution interaction, F (3,72) = 2.23, p = .092, MSE = 10.99, nor the three-wa y interaction were significant, F (3,72) = 1.57, p = .205, MSE = 10.99. These results were surprising given the previous findings from Knowles and Delaney ( 2005) and Experiment 1, in which the punishment group made significantly fewer illegal moves compar ed to the no-cost group on the first solution. Table 3-2 provides the means and standard devi ations for illegal moves by group and solution. One potential explanation for the absence of si gnificant results could have been due to a lack of power. For this reason, independent samples t -tests were conducted between the groups for illegal moves on the first solution attempt. The independent samples t -tests comparing nocost and motivation, t (38) = 1.98, p = .056, no-cost and reward, t < 1, no-cost and punishment, t (38) = 1.24, p = .222, motivation and reward, t (38) = 2.00, p = .053, motivation and punishment, t (38) = 1.01, p = .320, and reward and punishment, t (38) = 1.55, p = .129, were not significant. The punishment group has typically made fewer illegal moves compared to the no-cost group, but this comparison was not signi ficant. Also of interest were the two experimental groups compared to the no-cost group, which revealed a trend approaching sign ificance comparing the motivation and no-cost groups. A lthough not significantly, the mo tivation group made the fewest number of illegal moves ( M = 3.40) out of all four groups. It may have been possible that with additional power both the punishment and motiv ation groups would have made significantly fewer illegal moves compared to the no-cost group.

PAGE 61

61 The most unexpected finding in this experiment was the lack of a significant difference in illegal moves between the no-co st and punishment group. One poten tial explanation could have been that since the penalty was set at a fixed in terval, the number of penalties could have been (and often were) fewer than the number of ill egal moves committed. For example, if the participant committed an illegal move at the be ginning, the middle and the end of a 30 s interval then three illegal moves were made, but only one penalty was administered at the end of the 30 s time interval. Being penalized less often compared to previous experiments could have reduced the effect, resulting in non-significant differen ces between the groups for illegal moves. Table 3-2. Means and standard deviations (SD) of illegal moves by group and solution for illegal moves were presented. Group Mean SD Solution 1 No-Cost 5.40 3.42 Motivation 3.40 2.96 Reward 6.85 7.13 Punishment 4.25 2.34 Solution 2 No-Cost 3.75 2.55 Motivation 3.95 2.63 Reward 4.15 3.10 Punishment 4.55 4.43 Also of interest and importance was the l ack of a significant diffe rence between the nocost group and the two experimental conditions. One explanation fo r the lack of a reduction in illegal moves in the reward condi tion could have been attributed to the reward itself. It was possible that candy was not very rewarding to th e participants and signi ficant improvements may have been obtained with a more valuable reward, like money. An additional possibility was that the reward rules awarded additional or bonus candy at the completion of the problem for efficient performance and this may have had lit tle or no effect on problem solving behavior. Awarding candy after the problem had been completed may not have influenced participants to avoid illegal moves during the problem. In add ition, the lack of a reduc tion of illegal moves in

PAGE 62

62 the motivation condition could have indicated th at the instructions were not effective in increasing attention. Given that this experiment was run last out of the three experiments and was run at the end of the semester this may have resulted in less motivated students, which may have affected the results. Students who had waite d until the end of the semester to volunteer and complete their experimental credits were likely less motivated to complete the credits and may have been more resistant to the experimental manipulations. Anot her explanation for the results could have been that an adverse stimulus or thre at of an adverse stimulus was required to increase attention and reduce illegal moves. Thus, partic ipants in the motivation and re ward conditions would not have been influenced to increase atte ntion and reduce illegal moves. Un fortunately, the lack of typical effects in this experiment could not be clearl y understood from the coll ected data and further research is required to answer these questions.

PAGE 63

63 CHAPTER 4 EXPERIMENT 3 The previous two experim ents concentrated on finding ways to aid participants in becoming more attentive, through threat or rewar d, so that they could perform more efficiently on the problem. Once a participant had become motivated and decide d to allot additional attention to the problem, what did this increased attention do to reduce illegal moves? In previous research Knowles and Delaney (2005) proposed that punishment induced caution and that caution was checking a move against the rule s for legality so that illegal moves could be rejected and avoided. Experiment 3 instructed participants to engage in this checking behavior so that illegal moves could be avoide d and so that the result of such behavior could be assessed. In the critical conditions of Experiment 3, par ticipants were asked to check the legality of each move after selecting it, but before executing it. One group was asked to perform this check while thinking aloud so that it could be monito red by the experimenter and the other group was asked to perform the check silently. Participan ts received training in thinking aloud and were asked to verbalize any thoughts they had while working on the problem. The training and instructions were similar to those proposed by Ericsson and Simon (1993). The silent check group was used for comparison to the aloud group to ensure that the aloud task did not affect performance. Based on previous re search, the illegal move rates of the silent and aloud check groups should not have differed; both should have shown significan t reductions in illegal moves and all other data between the two groups, excep t for time per move, should have been similar (Ericsson and Simon, 1993; Knowles and Delaney, 2005). Instructing a participant to check a move for legality would have likely increased the frequency of this checking behavior This would have ultimately infl uenced participants to reject more illegal moves compared to those not instructed to check. Asking participants to check

PAGE 64

64 moves for legality would have affected Stage 2 of CIF because participan ts would have become more attentive with the requirement of checking each move for legality. Platt and Griggs (1993) found that when participants were asked to explain their choices in Wasons 4-card selection task they obtained the highest success rates every re ported for this difficult task. These results appeared to support the idea that increasing en gagement in a task, whether through requiring explanations for behavior or through possibl y a check of legality, can yield significant improvements in performance. Instructing participants to check m oves would also have a direct effect on Stage 3 of the framework where chec king behavior was di rectly manipulated. Instructing participants to check the legality of each move before it was made allowed CIF to make clear predictions. CIF predicted that the instruction to check would have increased attention and decreased the number of illegal moves made compared to the no-cost group. The prediction was that illegal move reductions woul d have achieved rates similar to the punishment group in the first experiment and previous findings of Knowles and Delaney (2005). In addition, a measure of working memory using the operation span (OSPAN) task was also administered at the end of the experiment (Kane, Bleckley, Conway, & Engle, 2001; Turner & Engle, 1989). The reason for obtaining working memory scores was to determine the extent to which resources play a role in human problem solving. If the resource limitation hypothesis (Jeffries et al., 1977) was correct then working memory should si gnificantly correlate with the number of illegal moves a participant makes. The OSPAN task assessed working memory by presenting equations and words on a computer scree n. The participant had to verify the equations while maintaining the words in memory. To asse ss the participants working memory span the words had to be recalled in the order that they we re presented. This task was described in more detail in the procedure section.

PAGE 65

65 The purpose of measuring OSPAN was to exam ine the role played by working memory resources in problem solving, which has been pro posed as a cause for the selection of illegal moves (Jeffries et al., 1977). Those with low working memory spans may have had difficulty calculating future states or remembering to ch eck each move for legality contributing to the difficulty of the problem. Previous research on these types of probl ems did not reveal a significant relationship between i llegal or legal moves made a nd working memory (Knowles and Delaney, 2005). However, the evidence here ma y prove otherwise sin ce participants were instructed to check aloud. This manipulation allowed the experi menter to verify that the participant actually engaged in th is behavior and enabled the expe rimenter to assess the rate at which they checked and the accuracy of this checking. The role of working memory in problem solving was outside the scope of CIF; therefore the framework was unable to make a prediction as to the relationship between problem solving and working memory. The author believed that such factors contributed to probl em solving and predicted a nega tive correlation between illegal moves and working memory and a negative co rrelation between legal moves and working memory. Methods Participants The process f or recruiting participants was the same as the first two experiments. In Experiment 3, 103 participants t ook part in the study; each was ra ndomly assigned to one of the four conditions. Of these, 23 participants were un able to solve both proble ms correctly within the time limit and their data was therefore dropped from the analysis. Participants that were unable to solve the problems were replac ed until there were 20 participants in each of the four groups.

PAGE 66

66 Problems The problem s used in Experiment 3 were the sa me hobbits & orcs and titration problems as those used in the first two experiments. Design In this experim ent there were four gr oups: no-cost group, a punishment group, a check group, and an aloud group. As in Experiments 1 and 2, the groups differed as to the instructions they received on the first problem and the inst ructions on the second problem did not differ between the groups so that tran sfer effects could be examine d. All groups solved the second problem silently without punishment or instructions to check each move. The punishment and no-cost groups were iden tical to those in Experiment 1. The no-cost group received no special instructions, rewards, or penalties for making or avoiding illegal moves. The punishment group penalized particip ants immediately after every illegal move by having them rate the pleasantness of words fo r 45 s just as participants did in the first experiment. The check condition instructed the participant that after selecting a move he/she would check the move for legality before executing th at move. A button was located on the display so that after checking the move but before executi ng it, the participant would click the button to indicate that they had checked the move and that he/she believed it to be a legal move. This button was primarily on the display so the experime nter could verify that those in the silent condition were performing the check. In the alou d condition the button remained on the display so that the similarity between the two condi tions was maximized. Participants in the aloud condition were also instructed to click on the check button before making a move to indicate that the move had been checked. In addition, the co mputer program tracked when and how often a participant forgot to click the button to indicate that they had checked the move for legality.

PAGE 67

67 A participant in the aloud condition received instructions and training on thinking aloud and was then asked to verbalize his/her thoughts as they worked on the problem (instructions for thinking aloud were provided in the Appendix). These procedures we re similar to the instructions proposed by Ericsson & Simon (1993) If a participant was silent for more than 3 s or speaking too softly he/she was reminded to keep talking. The experimenter attended to the participants verbal protocols in an attempt to ensure that the participant checked each move for legality before executing that move. As in the check co ndition the participant was asked to click on the Check button before finishing his/her move to indicate that he/she ha d checked the move and believed it to be a legal move. If the participant did not check a move for legality he/she was reminded to continue doing so and the computer tracked each checked and non-checked move. The participant was asked to sp eak into a microphone while solvi ng the first problem and verbal protocols were recorded on a mini-disc recorder. Procedure The procedures followed those of Experim ent 1, except that participants in the aloud condition received instructions a nd training at the beginning of the tutorial phase, to think aloud as they solved the problem. Th e participant was given instructi ons for thinking aloud similar to those suggested by Ericsson and Simon (1993) and was asked to verbalize his/her thoughts by saying whatever cam e into his/her head. The par ticipant was then instructed to think aloud as he/she imagined him/herself walking through a hous e that he/she was very familiar with and to count the number of windows in the house. Th e experimenter stopped this task when the participant was able to provide descriptors of the interior of the house, which demonstrated understanding of thinking aloud. If the participant was too vague or did not understand the task then the experimenter provided an example of describing the interior of a house while counting the windows and then asked the part icipant to complete the task. This task was very brief and

PAGE 68

68 typically did not take more than 30 s; every participant in the think aloud group was able to complete this task. During the tutorial phase, pa rticipants in the aloud group were asked to work on the practice problem while th inking aloud and to check each move for legality. The other three groups completed the tutorial phase in a fash ion similar to the participants in Experiment 1. As in the other experiments, a ll participants so lved the second problem silently and without checking. After completion of both problems, part icipants in all groups completed the operation span (OSPAN) task to assess working memory capacity (Kane, Bleckley, Conway, & Engle, 2001; Turner & Engle, 1989). The OSPAN task was a task used to assess a persons ability to main tain information in memory while completing another task, which ev aluated a persons working memory span. The task presented a simple mathematical equation followed by a word (e.g., (9 / 3) 1 = 2 CONE) in a Power Point presentation on a computer scr een. The participant was to read the equation aloud, verify whether or not the equation was corr ect (some equations were true and some were false), and then say aloud the word that fo llowed the equation. Once this was completed the experimenter immediately presented the next s lide on the computer screen with a new equation and word. Once the new slide appeared the part icipant was to begin re ading the equation aloud immediately, verify it and then read the word. Anywhere between two and five equations/words were presented and then the participant saw a colored screen with ??? at the completion of a series (all the words presented since the last colore d screen). This was the participants cue to recall all the words he/she had seen in that seri es on a piece of paper in the order in which they were presented. There were a total of 12 series (t hree series each with two words, three words, four words, and five words) and they were ra ndomly presented so that the participant did not know how many words would be in that series until that series was over. The participant

PAGE 69

69 received one point for every corr ect word, although the participant only received points if all the words in the series were recalle d correctly in the corre ct order (e.g., no points given if all four words recalled, but the order of Words 1 and 2 were switched; four points given if all words recalled correctly in the correct order). Results and Discussion Move data. For this experim ent a 4 x 2 x 2 mixed factorial ANOVA was conducted: Group (No-Cost vs. Punishment vs. Check vs. Aloud) x Order (Hobbits first vs. Titration first) x Solution (First vs. Second). Group and order were between-subjects factors and solution was a within-subjects factor. As in the first two expe riments, illegal moves committed on the first and second problem were of primary importance, but number of legal moves and time per move were also analyzed. As in the first two experiments it was anticipa ted that there would be no main effect of order, no main effect of solution, and no interaction for any of the factors indicating that the isomorphs did not differ in difficulty, the order of presentation did not influence the results, and the results for the first solution did not differ si gnificantly from the seco nd solution. It was also anticipated that legal moves and average time pe r move would not differ significantly between the four groups. Those participants in the ch eck and aloud conditions could have had slightly higher times as a result of the instruction to chec k and the process of verb alizing thoughts in the aloud condition, but this should not have affected performance otherwise. However, it was anticipated that a main effect of group would be obtained for illegal moves. Legal move data. An analysis of legal moves revealed no main effect of solution, F < 1, no main effect of order, F (1,72) = 1.31, p > .05, MSE = 228.21, and no main effect of group, F (3,72) = 1.07, p > .05, MSE = 228.21. These results indicated th at, regardless of which problem was solved first, the number of legal moves did not differ between the first and second solution

PAGE 70

70 or between the four groups. These results replicat ed the findings of the first two experiments and support the claim that the manipulat ions have little or no effect on the legal moves within the problems. Move time data. Procedures for calculating average time per move were the same as those used in the first two experiments, and the adjustments made to the punishment group for illegal move penalties were the same as thos e imposed on data from Experiment 1. Analysis revealed no main effect of order, F (1,72) = 2.60, p > .05, MSE = 50,735,853.35. However, a main effect of solution was found, F (1,72) = 42.10, p < .001, MSE = 37,739,536.72, which indicated that participan ts took less time making moves on the second solution ( M = 12,645.75 ms) compared to the first solution ( M = 18,948.23). A main effect of group, F (3,72) = 3.97, p = .011, MSE = 50,735,853.35 was present, indicating that the groups did not make moves at the same rate. The results also revealed a two-way Solution x Group interaction, F (3,72) = 5.64, p = .002, = 37,739,536.72. As indicated in Figure 4-1 some of the times decreased from the first solution to the second and some did not. Paired samples t -tests revealed no change for the nocost group from the first solution to the second, t (19) = 1.65, p > .05. The difference for the check group, t (19) = 2.27, p < .05, the aloud group, t (19) = 9.20, p < .001, and the punishment group, t (19) = 2.19, p < .05, showed significant decreases in time from the first solution to the second. The means for the groups by solution attempt were displayed in Figure 4-1. Independent samples t -tests were also conducted on the groups for the first and second solutions. There were no significant differen ces between the groups for the second solution attempt. On the first solution attempt the no-co st group took less time on moves compared to the check, t (38) = 2.84, p < .01 and aloud groups, t (38) = 5.86, p < .001, and approached significance for the punishment group, t (38) = 1.70, p = .098. The check group did not significantly differ

PAGE 71

71 from the aloud, t (38) = 1.60, p > .05, or the punishment groups, t < 1. The punishment group took significantly less time making moves on th e first solution when compared to the aloud group, t (38) = 2.50, p < .05. Average move times for the groups were displayed in Figure 4-1 by solution attempt. The check, aloud, and punishment groups took significantly less time on the second solution compared to the first. The results may ha ve been due to the increased activities such as checking and/or thinking aloud in the experimental conditions, but may have also been due to increased attention as proposed in Experiment 1. It was also not surprising that the check and aloud groups took significantly longer to make m oves on the first solution when compared to the no-cost group because the two experimental groups had to take additional actions in checking moves for legality and/or speaking aloud. Duri ng the second solution when the manipulations were removed all groups had similar move times. Experiment 3: Average Move Times0 5000 10000 15000 20000 25000 30000 35000 No-CostCheckAloudPunishment GroupTime (MS) Solution 1 Solution 2 Figure 4-1. Average time per move for each of th e four groups was displayed for the first and second solutions in milliseconds. The error bars represented standard error. Illegal move data. An analysis of illegal moves re vealed that there was neither a significant main effect of solution, F < 1, nor a significant ma in effect of order, F < 1. The main

PAGE 72

72 effect of group was significant, F (3,72) = 3.12, p < .05, MSE = 18.24, and was followed up with independent and paired samples t -tests. Independent samples t -tests revealed that the no-cost group made significantly mo re illegal moves when compared to the check, t (38) = 2.59, p < .05, aloud, t (38) = 3.41, p < .005, and punishment group, t (38) = 3.03, p < .005. The paired samples t tests revealed no significant differences between solutions for the no-cost, check, and punishment groups. The analysis of the aloud group i ndicated that participants made more illegal moves on the second solution when compared to the first. Figure 4-2 displayed the group means for both the first and second soluti on. In addition, the Solution x Order and 3-way interactions were not significant, F s < 1. The Solution x Group interaction, F (3,72) = 1.94, p =.132, MSE = 12.20, and Order x Group interaction, F (3,72) = 1.27, p =.291, MSE = 18.24, were also not significant. Illegal moves for each group for the first and second solutions were displayed in Figure 4-2. The analysis of illegal moves replicated previous findings that when participants were penalized for illegal moves they made significantly fewer illegal moves compared to participants that were not penalized for making illegal m oves. In addition, part icipants in the two experimental conditions made si gnificantly fewer illegal moves compared to the no-cost control group. These findings indicated th at when participants were instructed to be and were accountable for checking moves for legality this resulted in the reduction of illegal moves. These results directly addressed and supported Stage 3 of CIF where an increase in checking behavior resulted in a decrease in illegal moves. This reduction occurred even when participants were instructed to think aloud wh ile solving the problem. One very interesting result was that partic ipants in the aloud group made significantly more illegal moves on the second solution ( M = 4.60) than they did on the first ( M = 2.00), t (19)

PAGE 73

73 = 2.73, p = .013. Even though the check and punishment groups showed sustained benefits on the second solution with no signifi cant increases in illegal move s, the aloud group did not and showed a release, making more illegal move s on the second solution than the first. An increase of illegal moves from the first solution to the second for the aloud group was an unexpected result. Knowles and Delaney (2 005) performed a similar experiment where participants were instructed to think aloud unde r no-cost instructions or to think aloud under punishment instructions. Neither group was instruct ed to perform a check of moves for legality. Both groups in Knowles and Delaneys experiments showed no significant effects of thinking aloud and showed typical no-cost and punishment results, but neither group solved a second problem to assess transfer effect s. In addition, the check group in the current experiment showed sustained benefits with no si gnificant difference between illega l moves on the first and second solution. Since transfer effects we re not previously explored with think aloud instructions, these results could be due to the think aloud instructio ns or to some interaction between thinking aloud and checking. Experiment 3: Illegal Moves0 1 2 3 4 5 6 7 8 No-CostCheckAloudPunishment GroupIllegal Moves Solution 1 Solution 2 Figure 4-2. Illegal moves for each group were presented by condition with standard errors indicated as error bars.

PAGE 74

74 Checking data. The checking data provided insight in to the relationship between checking behavior and the reduction of illegal moves, a di rect indication of how checking affected illegal moves. Participants in the check and aloud group s were instructed to click on a check button on the computer display before executing a move to indicate that they had checked that move for legality. The computer program tracked when a move was not checked prior to making that move. This information allowed for a physical in dication of when a participant did not check a candidate move for legality. These values were compared with the number of illegal moves committed by each participant in the check and aloud groups. Illegal moves were not reliably correlated with checking in the check group, r (19) = .244, p = .300. In the aloud group, as the number of illegal moves decreased the frequency of missed checks decreased, r (19) = .657, p = .002. The results from the correlation analysis rev ealed some unexpected results in that checking in the check group was not signifi cantly related to illegal moves, but checking in the aloud group was related to illegal moves. One potential expl anation for this finding was that those in the check group did not verbalize thei r thoughts and therefore were not held accountable to the same degree as those in the aloud group. A participant in the check gr oup could have simply clicked the check button prior to each move without actually engaging in any checking behavior because there was no way for the experimenter to actually verify what the participant was doing in his/her head. This explanation seemed un likely because participants in the check group made a similar number of illegal moves compared to participants in the aloud group and fewer than those in the no-cost group. In contrast, a participant in the aloud group was reminded to continue verbalizing if he/she was silent for too long, which may have influenced the participant to be more compliant in checking moves. An additional explanation could have been that

PAGE 75

75 participants in the check group could have checked moves as of ten as those in the aloud group, but forgot to click the check button more often as a result of not verbalizing. The results did not support this explanation -the num ber of times the check button wa s not clicked prior to making a move did not statistically differ between the check ( M = 1.75) and aloud ( M = 2.40) groups, t < 1. The check and aloud groups both made fewer ill egal moves compared to the no-cost group and did not differ from each other, which may have indicated that particip ants in the check group were likely checking and reje cting moves to the same degree as the aloud group. OSPAN data. After completion of the two problems, pa rticipants were asked to complete the OSPAN task to assess their working memory span. Out of the 80 total participants that solved both problems within the 20 min time limit in Experiment 3, 10 participants were unable to complete the OSPAN task in the time allowed. The other 70 completed the OSPAN task and their data were submitted to a correlation anal ysis. Nineteen participants in the check group completed the OSPAN task, but their scores were not significantly related to either the number of illegal moves made r (18) = -.067, p = .785 or the number of missed opportunities to check a move for legality r (18) = -.252, p = .297. Eighteen participants in the aloud group completed the OSPAN task, their scores were not significantly related to the number of illegal moves made r (17) = -.275, p = .269, but they were significantly rela ted to the number of missed opportunities to check a move for legality r (17) = -.479, p = .044. The measure obtained by the computer program was the number of times a participant made a move without first clicking the check button, in other words, the number of missed oppor tunities to check for le gality. This negative correlation indicated that as a participants working memory span score increased the probability of that participant forgetting to ch eck a move for legality decreased.

PAGE 76

76 The correlation between missed opportunities to check a move for legality and memory span score in the aloud group appeared to suppo rt the resource limitation hypothesis. That is, those participants with more available memory we re more able to remember to check moves for legality. However, a similar finding was not found in the check group and the correlations between memory span scores and the number of illegal moves made were not significant, which did not support a resource limita tion hypothesis. One potential expl anation could have been that with the instruction to think aloud and check moves for le gality simultaneously in the aloud group this could have taxed enough resources that those with larg er working memory spans were more able to perform the check consistently when compared to those with smaller working memory spans. This explanation did not seem to explain why similar benefits were not seen in correlations between working memory and illega l moves for the aloud group. Taken as a whole these results seemed to indicate that a resource limitation hypothesis could not be entirely ruled out, but that it likely played a smaller role than has been assumed in previous research.

PAGE 77

77 CHAPTER 5 GENERAL DISCUSSION The three ex periments presented here a ddressed specific assumptions of the CIF framework for the reduction of illegal moves. Experiment 1 tested whether an intrusive penalty was necessary to reduce illegal m oves or if a threat without pena lty for each illegal move would be sufficient. Experiment 2 set out to determine if an aversive stimulus was necessary to reduce illegal moves or if a more positive conseque nce could help to improve problem solving efficiency. Experiment 3 instructed participants to engage in checking behaviors to directly assess the ability of these actions to reduce i llegal moves. Experiments 1 and 3 seemed to support aspects of the framework, while Experiment 2 produced non-significant results. As noted previously, several of the trends seemed intrigui ng, but low statistical pow er to detect some of these effects was likely at fault. Therefore, some of the trends that we re not significant were expanded upon further in this section. Illegal Moves The first experim ent demonstrated that the th reat of a penalty, whet her it was experienced or not, was able to reduce illegal moves comp ared to a group that was not penalized or threatened. Participants in the no-cos t group made the most illegal moves ( M = 6.85) and the threat and experience groups made fewer illegal moves ( M s = 3.40 and 3.65, respectively), but similar numbers compared to each other. Because the threat and experience groups made similar numbers of illegal moves, verbal instructions alone without any corporeal consequences were apparently sufficient to reduce illegal moves. Pa rticipants in the punishment group made slightly fewer illegal moves ( M = 2.60) compared to the two experi mental groups on the first solution, although not significantly. One potential explanation for the trend of fewer illegal moves in the punishment group could be attributed to the id ea that the punishment manipulation was more

PAGE 78

78 powerful and thus had the potential to reduce illegal moves to a greater degree compared to a threat alone. The lack of signifi cance may have also been attribut able to a floor effect, whereby significance may have been obtaine d in problems with a greater opportunity to make more illegal moves. Surprisingly, the second experi ment did not reveal any signi ficant illegal move reductions on the first solution. The intent of the motivation and reward manipulations was to determine if positive consequences or instructions to avoid illegal moves would result in reductions. The motivation ( M = 3.40) and punishment (M = 4.25) groups, although not significantly, made fewer illegal moves compared to the no-cost group (M = 5.40). It is possible that with increased power to detect the effects the motivation and punishment groups would have made significantly fewer illegal moves compared to the no-cost group. The trend indicated that participants in the motivation group made the fewest illegal moves. Why they potentially made fewer than the punishment group is not well understood. One po ssibility was that the extra motivating instructions may have seemed suspicious to participants and caused them to be even more cautious for fear of the unknown. Alternatively, they may have take n the instructions as a hint that avoiding illegal moves was the secret to solving the problem. Candy in the reward group was set on a fixed in terval schedule to lim it the possibility of participants making additional legal moves to receive more candy. Even though the reward group received candy for avoiding illega l moves these participants ma de the most illegal moves ( M = 6.85). Such findings were unexpected and did not s eem to be explainable by CIF. Some potential explanations could have been that the candy was not motivating enough or that the interval schedule was confusing or detrimental in some way. Participants only received one piece of candy for each successful 30 s time interval. It is possible that with more candy offered or a more

PAGE 79

79 substantial reward performance could have been improved. In addition, because the reward was administered on a fixed interval schedule particip ants could have had optimal performance that occurred early in the 30 s interval but did not receive a reward unt il the end of the interval or potentially not at all if an illeg al move was subsequently made in that interval. The lack of a significant reduction in illegal m oves was surprising here, but the absence of such findings may have been due to a fault in the experimental de sign. One alternative expl anation for the lack of significant reductions in illegal moves for the motiv ation and reward groups could have been that aversive consequences are necessary to reduce illegal moves. The most surprising finding in Experiment 2 was the lack of a significant difference between the punishment and no-cost group. The puni shment group was also set to administer a penalty at a fixed interval to maximize the sim ilarity between the punishment and reward groups. However, this allowed participants to make illega l moves that did not result in a penalty because multiple illegal moves could have been made in one 30 s interval that only resulted in one penalty. This design may have reduced the eff ect of the punishment and was likely responsible for the absence of significant re ductions in illegal moves. Experiments 1 and 2 specifically looked at Stage 1 of CIF where consequences for making or avoiding illegal moves were manipulated in an attempt to reduce illega l moves. Experiment 1 lent support to CIFs claim th at punishment and threat withou t punishment resulted in illegal move reductions. Experiment 2 indicated that th e reward manipulation was not able to reduce illegal moves, but that a motivational manipulation showed promise. Further research is required to explore what other consequences, if any, are ab le to assist problem-sol vers in avoiding illegal moves. According to CIF, manipulations in th e first stage of the framework reduced illegal moves by acting on Stage 2 to increase a problem -solvers attention towa rds the problem. CIF

PAGE 80

80 made no specific assumptions about the attentio nal resources available and how they could be distributed to reduce illegal moves, only that problem-solvers do not of ten perform at optimal levels and that the consequences in Stage 1 would result in additional resources attributed to the task. Stage 2 of CIF was not specif ically tested here and further re search is required to determine how and where attention would be increased. Stage 3 of CIF proposed that increased attention from Stage 2 would allow for increased accuracy and/or frequency of checking and/or evaluation. The checking portion of the framework utilized the Generation, Caution, Ve rification Framework (G CV) from Knowles and Delaney (2005). Experiment 3 specifically looked at checking behavior a nd its ability to reduce illegal moves. Participants in the experimental co nditions were instructed to check each move for legality prior to executing that move, approximately half of those particip ants did this silently and the other half did so aloud. The check ( M = 3.00), aloud ( M = 2.00), and punishment (M = 2.65) groups made significantly fewer illega l moves compared to the no-cost group ( M = 5.90). These results indicated that incr eased frequency of checking result ed in reduced illegal moves, lending support to that c oncept of the framework. The check and aloud groups were both instructed to check moves for legality and to click a button on the display prior to each move to indicate that they had checked that move, this was done silently or aloud, respectively. There was a significant relationship between checking and illegal moves for the aloud group, but not for the check group, indicating th at those in the aloud group reduced the number of illegal moves they made as checking increased. It was surprising that data from the check group did not yield a significant relations hip, especially since performance in this group was similar to the aloud group where there was a significant relationship. One potential explan ation could have been that si nce these participants did not

PAGE 81

81 check aloud they could have not been checking at all and could have just simply clicked the button prior to making a move. This explanation did not seem very plausible considering that participants in the threat group made a simila r number of illegal moves compared to the aloud group, t (38) = 1.24, p > .05. Another, plausible explanation wa s that participants in the threat group were checking moves for legality in their hea d, but they were less likely to click the check button. This explanation also did not seem likely because an independent samples t -test between the two groups for number of moves not checked revealed no significant difference, t < 1. One final explanation could have been that both groups were checking and rejecting illegal moves, but the aloud group did so with more accuracy. A lthough this final explanation appears to be the most likely it is not possible to determin e the true cause based on current findings. Taken together the three experiments provide valuable information about the framework presented and about problem-solving behaviors mo re generally. Experiment 1 lent support to CIFs assumption that a physical pe nalty was not required to reduce illegal moves and that threat alone helped to induce caution. Experiment 2 presented somewhat confusing results and further research is required to determine what other t ypes of consequences coul d reduce illegal moves. Experiment 3 directly looked at increasing checking behaviors and lent support to the claim that increasing the frequency of check ing reduced illegal moves. These findings support CIFs claim that less intrusive manipulations can induce cauti on and that checking beha viors directly reduce illegal moves. Transfer Effects In all three experim ents, each participant solved an isomorph of the first problem where the second problem had the same underlying structure and solution as the first problem, but did not share any outwardly similar features. This isom orph was solved after th e first solution and was always solved under no-cost instructions. Having each participant solve the second problem with

PAGE 82

82 the same instructions and manipulations allowe d for the observation of transfer effects from learning obtained on the first problem. For all co mparisons, except one, the participants made a similar number of illegal moves on the second pr oblems when compared with the first, which largely supported previous findings from Know les and Delaney (2005). These findings seemed to indicate that the manipulations on the first solution, where illegal moves were reduced in most groups compared to the no-cost gr oup, provided participan ts with sustained benefits that carried over to the second problem where illegal moves remained relatively low. However, there was one exception and some trends that should be addressed. In Experiments 1 and 2, participants in the no-cost conditions demonstrated trends toward making fewer illegal moves on the second solution compared to the first. These trends may have potentially indicated that participants learned something that could be transferred to a novel isomorph or maybe it indicated that participants were settling down and were less nervous so performance improved, although not significantly. In Experiment 2, participants in the reward condition yielded numbers that approached significance, t (19) = 2.08, p = .051, potentially indicating that fewer il legal moves were made on the second problem (M = 4.15) compared to the first ( M = 6.85). Why participants in the reward group made so many illegal moves on the first solution is unknown. These illegal moves commissions on the first solution were likely driving the trend toward fewer illegal moves on the second solution, but these results seemed counterintuitive given previous re search where incentives helped to improve performance. Wieth and Burns (2006) found that the incentive to leav e the experiment early resulted in improved performance on both incremental and insight problem s. In addition, they found that the incentive resulted in further processing of the problem a nd increased participants memory of the problem

PAGE 83

83 they solved. These findings support the transfer e ffects seen where participants showed a trend for reduced illegal moves on the second solution. Finally, participants in the aloud group of E xperiment 3 made significantly more illegal moves on the second solution ( M = 4.60) compared to the first ( M = 2.00), t (19) = 2.73, p = .013. This finding was unexpected and could have pot entially been due to the aloud instructions causing the participants to take more time on the fi rst solution. The participants may have felt the need to make-up time on the second solution, which could have led to carelessness and an increase in the number of illegal moves on the second solution. Another potential explanation could be that the aloud instructions prevented so me necessary encoding or learning on the first solution, as transfer effects with aloud instructions were not expl ored in previous studies. The check group made a similar number of illegal moves on the first and second solution, so the checking instructions were not lik ely the cause of the increase. Ho wever, it was possible that the interaction between the think al oud and checking instructions coul d have been responsible for participants making more illegal moves on the second problem when compared to the first. This was a very interesting finding and the author believ es that this warrants further exploration as the potential for think aloud instruct ions to affect, not performanc e, but learning could indicate a negative side effect of this manipulation not se en in previous research (Delaney, Ericsson, & Knowles, 2004; Knowles & Delaney, 2005). Untested Assumptions: Illegal Move Reduction CIF was proposed as a novel fram ework and ther efore made several assumptions as to how a problem solver is able to reduce illegal m oves. In the first stage of the framework the assumption was that various consequences for making illegal moves influenced the problemsolver to alter some aspect of his/her behavior In the second stage the assumption was that the consequences in Stage 1 influenced the problem solver to increase the amount of attention

PAGE 84

84 devoted to the problem. With this increased atte ntion the problem-solver then either increased the frequency and/or accuracy of either check ing or evaluating behavior in Stage 3 of the framework. Because CIF was a novel framework it wa s not possible to test every assumption or every aspect of the framework here. Several assumptions remain untested and require future research to assess the ability of the framework to describe human problem-solver performance. Experiments 1 and 2 addressed Stage 1 of the framework and attempted to assess what consequences would invoke caution and reduce i llegal moves. Threat, punishment, motivation, and reward were all tested and threat and puni shment successfully resu lted in illegal move reductions while motivation showed promise. The reward consequence did not prove to reduce illegal moves. In addition, there may be other co nsequences not mentioned here that have the ability to reduce illegal moves and further research would be necessary to determine what, if any, other consequences would reduce illegal moves. The assumption of CIF was that consequences in Stage 1 of the framework would motivate the problem-solver to devote more attention to the problem-solving episode. Previous research has indicated that illegal moves may occur due to resource limita tions (Jeffries, Polson, Razran, & Atwood, 1977), recent research seemed to indicate otherwise (Knowles & Delaney, 2005). Because a resource limitation hypothesis cannot fu lly explain illegal moves, it made sense to assume that we do not always perform to our fu ll potential and that we may have additional resources available to perform more efficiently. The framework assumed that these additional resources were represented as attentional focus a nd that consequences in Stage 1 tap into these additional attentional resources to improve performance. Ho wever, the framework made no assumptions as to how this addi tional attention was evoked or to how it was distributed, only that it exists and that it can be uti lized, the current results seemed to support these assumptions.

PAGE 85

85 Experiment 3 specifically looked at the thir d stage of the framework and how increased checking behavior would influence illegal moves. This increased checking resulted in reduced illegal moves as the framework predicted. However, changes in evaluation behavior and increasing the accuracy of checking were not assesse d. It seemed intuitive that as the accuracy of checking a move for legality increased the like lihood of correctly rejec ting illegal moves would also increase. However, further exploration is ne cessary to determine when and to what degree a problem-solver may increase the accuracy of chec king candidate moves. CIF also assumed that as attention towards the problem increased a problem-solver may begin evaluating candidate moves more frequently or evalua ting moves to a higher criterion. Integration With Existing Theori es of Illegal Move Selection One of the main goals of this work was to address why illegal moves are made. One potential explanation for this, proposed by Jeffrie s and colleagues (1977), was that illegal moves are made due to resource limitations. Although th e term resources was not well-defined, it usually refers to working or short-term memory limits, attentional capacity, and thinking speed. This explanation proposed that on problems such as those used in these ex periments participants do not have the capacity to either remember to check a move for legality or to calculate the resulting state properly to asse ss legality. However, recent rese arch has seemed to indicate otherwise. Knowles and Delaney (2005) found that when penalized for illegal moves participants were able to avoid illegal moves. The curr ent work found similar re sults involving not only penalty, but the mere threat of a penalty indicating that problem -solvers do have the resources necessary to avoid illegal moves. In addition, wo rk on various types of problems such as the 8puzzle, the Tower of Hanoi, and water jugs pr oblems has demonstrated that problem-solvers actually have the ability to plan out solutions to difficult problem states and entire problems themselves, again indicating that we have the resources necessary to perform better on these

PAGE 86

86 tasks (OHara & Payne, 1998; Welsh, Cicerello, Cuneo, & Brennan, 2001; Delaney, Ericsson, & Knowles, 2004). Just because we can perform better on certain tasks than the resource theorists predicted does not imply that we have no limits on our cognitive resources. In fact, a central tenet of cognitive psychology is that many of our c ognitive abilities are measurable and have documented limits. For example, there are significan t limits on our short-term memory (Miller, 1956). Furthermore, Anderson and colleagues have successfully used 185 ms as an estimate of time to shift visual attention in versions of ACT-R (Anderson & Lebiere, 1998; Gray, Sims, Fu, & Schoelles, 2006). Because we have measurab le cognitive limits, a resource limitation hypothesis cannot be entirely ruled out. However, on the tasks described above they likely play a smaller role than initial ly assumed, and resource theorists ne ed to be more clear about exactly what resource is taxed and how. Another possible explanation is that mental resources may not place a hard limit on our ability to check for illegal moves, but that we are cognitive misers who try to conserve resources as much as possible. Previous and cu rrent research has demons trated through strategy changes, planning, and improved performance that we are often much more capable than our initial performance would indicate (for exam ple Simon & Reed, 1976; OHara & Payne, 1998; Delaney, Ericsson, & Knowles, 2004; Knowles & Delaney, 2005). In Wilsons (2002) review article of embodied cognition, she stated that one of the views of embodied cognition was an offloading of cognitive work onto the environment. This off-loading referred to the strategy of accessing information in the environment as need ed instead of using resources, like committing such information to memory. In tasks such as the hobbits and orcs, problem-solvers may have used trial and error strategies instead of planning deeply or ch ecking moves for legality, thereby

PAGE 87

87 conserving resources. A trial and error strategy would likely result in many illegal moves. However, with no penalty for illegal moves, such a strategy was not very costly in terms of resources, and was usually successful in adva ncing the problem-solver to the goal. Such an explanation seems to fit well with participants performance in the no-cost groups, but it does not explain why a threat or penalty would change such behavior. Gray and colleagues referred to this offloading concept as the minimum memory hypothesis and compared this to th e soft constraints hypothesis to de scribe participants behavior on the Blocks World task (Gray et al., 2006). The soft constraints hypothesis in contrast to the minimum memory hypothesis, stated that optimal performance was based on the currency of time and not on conserving memory resources. Gray et al. (2006) found that as the time cost of a task increased participants bega n to utilize their memory more to avoid procedures that took additional time, ultimately reducing the total ti me spent on the task. The idea of participants adjusting behavior to reduce the total time on a task actually fits well with the results observed in the first two experiments. Participants in th e punishment group avoided illegal moves because the penalty phase automatically added an additi onal 45 s. Those in the threat and experience groups may have avoided illegal moves because th ey were told that illegal moves would have added additional time at the end of the task. It may have been possible th at participants in the motivation group demonstrated a trend towards reduced illegal moves because they saw the motivating instructions as a hint to avoid illegal moves to re duce total solution time. Finally, participants in the reward group may not have shown illegal move reductions because doing so would not have decreased the time to solve the ta sk. In fact, attempting to maximize the reward would have potentially taken longer to solve the problem.

PAGE 88

88 The soft constraints hypothesis seemed to offe r a valid explanation for many of the results obtained, in at least Experiments 1 and 2. Such findings offered a novel explanation for some of the unaddressed aspects of CIF. Stage 2 of CIF indicated that the consequences in Stage 1 resulted in an increase in attention. However, the soft constraints hypothe sis may have indicated that increased attention was actually a reevaluati on or redistribution of attention to reduce the total time a problem-solver spends on the task. A reevaluation or redistribution of attenti on in the CIF framework could have been explained as a shift in strategy. The author believed that participants initially engaged in a trial and error strategy because there was little or no cost for illegal moves and this strategy was often successful at eventually reaching the goal state. Increasing the co st of making illegal moves may have lead to a representationa l change of the problem and a shift in strategy. Lovett and Schunns (1999) RCCL model predicted that uns uccessful strategies would lead to a representation change of the problem and th rough learning unsuccessf ul strategies were abandoned and successful strategies were adopte d. Simon and Reed (1976) also found a strategy shift towards means-ends analysis when particip ants were given a hint of a state they would encounter during their solution. Because illegal moves were highlighted in the threat and punishment groups the execution of such a move may have seemed like a failed strategy to the participants and prompted a re presentational change or a stra tegy shift. This idea seemed plausible as adopting a more successful strategy could have likely resu lted in illegal move reductions. An additional explanation for the avoidance of illegal moves could have been attributed to problem-solvers learning from their illegal mo ves. Grobe and Renkl (2007) had participants begin by working on pretest probability problems to assess their prio r topic knowledge. Next

PAGE 89

89 they presented participants with worked exam ples of probability problems. Some of the participants were presented with worked solutions that were all correct and some participants were presented with a mixture of correct and inco rrect solutions (participants were told when a solution was incorrect). Following this exercise, participants completed a post-test of problems that varied in their similarity to the worked examples they had seen. The post-test results indicated that those participan ts with low prior knowledge be nefited from seeing the correct solutions only, but those participants with hi gh prior knowledge benefited from seeing both the correct and incorrect solutions. In the current experiments participants we re not assessed on their prior problem-solving knowledge, but it may have been possible that at leas t some of the participants were able to learn from the illegal moves they made. Then why were illegal moves reduced in the experimental groups compared to the no-cost control? Threaten ing or penalizing participants for illegal moves may have increased their attention to the task and motivated them to learn from their illegal moves so that they could be avoided in the future With little or no cost to making illegal moves learning may have seemed like a waste of time or waste of resources. Afte r participants in the experimental groups made illegal moves they ma y have been more likely to assess why that move was illegal and this would have aided in avoiding illegal moves in the future. This would explain why transfer effects to novel isomorphs were seen where participants continued to make a reduced number of illegal moves. A problem-solvers ability to learn from his/her illegal moves seems like a plausible explanation not onl y for transfer effects, but for illegal move reductions on the initial problem where the first illegal move has the potential to provide a valuable lesson.

PAGE 90

90 Learning may have occurred from reflecting on an illegal move, but it may have also been possible that learning took place during the penalty phase. Moss, Kotovsky, and Cagan (2007) presented participants with a Remote Associates Test (RAT) where participants had to find a word that was associated to a group of wo rds presented. After several RAT problems, participants completed a lexical decision task th at provided hints to the RAT problems that were not solved correctly. Participants were then retu rned to the previously solved and unsolved RAT problems they had attempted. The results indicated that the hint appeared to help participants solve the uncompleted RAT problems. One explana tion Moss et al. considered was that of an incubation period. What actually occurs during an incubation period is not fully known, but it is essentially a break from the problem where the problem-solver may continue working on the problem subconsciously or the break may influen ce a strategy change. One example of where an incubation period is often helpful is with insi ght problems where the problem-solver can often return to the problem with greater success on a subsequent attempt. It could be that in the current experiments the 45 s penalty phase acted as an incubation period where participants either continued working on the problem or this may have prompted a strategy change. This explanation seemed unlikely as th eir was no incubation period in the threat or experience groups in Experiment 1, which both demonstrated ille gal move reductions compared to the no-cost group. Previous theories from the problem-solving lite rature and other areas of research help to provide alternative explanati ons and insight to the current findings. Although resource limitations may play a role in the current findings their influence likely plays a smaller role than initially assumed. Resource limitations may have seemed like a valid explanation for many results because problem-solvers may be cognitive misers and conserve resources or they may

PAGE 91

91 have an alternative goal of conserving time. It is through experimental ma nipulations that we are able to gain insight into the true potential of the human problem-solver. When there are changes to the problem-solving environment, as with punishment or threat, we see that the problemsolver adapts. This adaptation could be defined as an attentional shift or strategy shift, but no matter what the term problem-solvers are able to alter performance and improve. The current findings contribute to the problem -solving literature and provide additional insigh t into problemsolvers ability to adapt and improve. How Do People Evaluate Moves? Several move evaluation functions have been proposed in the probl em-solving literature. However, it did not seem wise to adopt one of th ese functions to CIF without first testing to see which one had the best fit. At the same time, illegal moves are most likely selected during the move selection phase and only rejected during ev aluation or checking. Ther efore, it is worth considering how earlier theories have accounted for move selection, with an eye towards understanding why illegal moves are chosen according to those theories. One of the very first theories of problem-solving that looked specifi cally at how problem-solvers solved a problem and what they were doing was Newell and Simons (1988)1 General Problem Solver (GPS) program. GPS was a computer program that atte mpted to mimic human problem-solving performance by creating and completing subgoals to reach the ultimate goal of advancing to the final goal state. Newell and Simon had a participant think aloud as he/she worked on a symbolic logic problem and then used the verbal protoc ol to compare performance to the computer programs attempt at solving the problem. Although, GPS did not map the participants moves exactly the findings proved promising and have been influential to problem-solving research. 1 Newell and Simons 1988 work was read by the auth or as a chapter reprint. GPS was previously introduced by Newell, Shaw, and Simon in 1959.

PAGE 92

92 Since the introduction of GPS back in the la te 1950s models of human problem-solving have evolved and become more specified. In f act, a model of human problem-solving that was discussed in the first chapter was proposed sp ecifically for problems like the hobbits and orcs. The model proposed by Jeffries et al. ( 1977) included the evaluation function: ei = aMi + bCi + cPi (1-1). The assumption of this evaluation function, as with most evaluation functions, was that a problem-solver was able to assign candidate m oves a value and determine which move had the highest probability of advancing the problem-solver towards the goa l state. Due to the design of the evaluation function those moves that placed more travelers on the goal side (the right bank) of the river and did so by main taining a balance of missionaries and cannibals (missionaries are interchangeable with hobbits a nd cannibals with orcs) to redu ce the probability of having a missionary eaten would be more likely to be selected. Such a strategy seemed intuitive and potentially a good fit for CIF. The assumptions of CIF stated that illega l moves may be reduced through increasing the frequency and/or accuracy of evaluating moves. Increasing the frequency of evaluating moves would simply entail using an evaluation function, like this one, more ofte n instead of selecting moves at random. One argument was that by increasing the frequency of evaluation the probability of selecting an illegal move actually increased because many illegal moves actually bring the problem-solver closer to the goal and would therefore be evaluated higher than legal moves. The evaluation function proposed by Jeffries et al. indicated that problem-solvers attempt to keep the travelers in missi onary-cannibal pairs to minimize the chance of the missionaries being eaten. This assumption of the evaluation function would help to prevent illegal moves sometimes, but the problem cannot be solved unless the pairs are split. Since a split of the pairs

PAGE 93

93 was inevitable the threat of evaluation increasi ng illegal moves remained. Jeffries et al.s model also stated that there were two types of illegal moves, easy-to-detect and hard-to-detect and that easy-to-detect illegal moves were always reject ed. Hard-to-detect illegal moves were checked and rejected based on fixed pr obabilities. CIF assumptions di ffered here in that these probabilities were not fixed and that checking could occur independent or as a results of evaluation. If the evaluation of moves can occur in CIF without checking behavior then the possibility of illegal moves in creasing with increased evaluation cannot be ruled out. The second alternative for evaluation to aid in avoiding illegal moves according to CIF was by increasing the accuracy of evaluation, whic h can be done in two different ways. One way was by including an illegal move filter as part of the evaluation function to reject illegal moves as noted previously. The second assumed that problem-solvers are affected by different consequences that alter performance. These consequences increase attention towards the problem, which may potentially increase learning and the integrating of past experiences to increase the probability of selecting legal move s more often. In their model, Jeffries et al. included a memory process that allowed for previ ously visited states to be avoided to prevent backtracking and to facilitate progression through th e problem, but since it was based on the resource limitation hypothesis little or no learning or adaptation to the problem states were incorporated. Based on this it would not be likely or maybe not possible that a problem-solver could solve the problem in the minimum number of moves without any illegal moves. However, out of the 240 participants that completed both pr oblems 9 were able to solve the problem in the minimum number of moves with no illegal moves in the current experiments. These participants for the most part were in various gr oups and in different experiments.

PAGE 94

94 To help explain how a problem-solver is able to efficiently solve this problem with no wasted moves a brief summary of the solution of the hobbits and orcs problem is provided here (the solution is almost completely linear so all participants solved the problem in this order). First the problem-solver had to position all of th e orcs on the right bank of the river. Next the hobbits were moved over in a balanced fashion so that there were two hobbits and two orcs on the right bank. Then for the first and only time in the problem the problem-solver moved two travelers to the left bank, one hobbit and one orc. At this point the problem solver could safely move all the hobbits to the right (goal) bank where they remained until the rest of the travelers were moved to the goal bank. To utilize the current evaluation function to explain this performance the constant weighting factors a, b, and c would need to be adjusted at different states of the problem. For instance, b, which accompanied the cannibals value on the right bank would need to be weighted heavier towards the beginning of the problem to ensure that the cannibals were moved first. Once this phase wa s completed the weighting factors again would need to be adjusted so that a, which accompanied the missionaries, was weighted more heavily to ensure that they would be moved. Increasing the accuracy of the evaluation function would require that participants adjusted these weighting factors at each state and this increased accuracy would likely aid in the avoi dance of illegal moves. The evaluation function and model proposed by Je ffries et al. (1977) appeared to provide a good description of participants problem-sol ving behavior presented here with minor revisions. The major assumptions of their mode l were based on a resource limitation hypothesis, but since the current and recent research seemed to indicate otherwise such adjustments seemed warranted. Because this evaluati on function was based on the same types of problems used here it had a natural fit into the CIF framework. Howeve r, it would a gross oversight to think that

PAGE 95

95 research and models taken from other types of problems could not provide valuable insight as well. An additional model that could be used to explain the results obtained here was based on research that has utilized J ohn Andersons ACT* theory (often referred to now as ACT-R). ACT* was based on the firing of productions which were condition-action pairs (Anderson, 1987). If a condition was satisfied then the corresponding action would be executed, e.g. if the traffic signal is red, then I will stop my vehicle. Lovetts (1998) work on the building sticks task (BST) has utilized the ACT-R theory to model pr oblem-solvers ability to choose moves that will likely reach the goal state. Love tts ACT-R based model assumed that problem-solvers selected their next move based on the highest expect ed gain according to the following equation: E = PG C (1-2) where E was the expected gain of the selected move, P was the estimated probability of achieving the productions goal, G was the value of the goal, and C was the estimated cost to be expended in reaching the goal. Since previous research indicat ed that problem-solvers have the resources to calculate future states and to plan then they should have the ability to calc ulate expected gains ( E ) that are equal to true gains in tasks such as the hobbits and orcs. Unlike BST, the hobbits and orcs task was not solved several times and it had a fixe d solution and fixed goal state that was not manipulated by the experimenter. The problem so lver had to advance through the problem space encountering many novel states along the solution path. At any gi ven state subsequent states could have been calculated and evaluated as to th eir ability to advance th e participant to a novel legal state and the success of an operator was inde pendent of its history of success. In addition, the problem space for the hobbits and orcs problem was almost completely linear. This enabled

PAGE 96

96 participants to solve the problem and completely avoid all illegal moves by correctly calculating future states and only selecting novel legal moves. In contrast, it was not possible to accurately calculate the validity of a move in BST, influenc ing participants to rely primarily on history of successes and failures of operators in these types of problems. Lovetts model, as applied to BST, focused primarily on participants ability to make decisions based on previous successes and failures. However, with some modification it may be possible to apply Lovetts model to the hobbits and orcs problems to account for illegal move reductions obtained in previous and current research. In Lovetts model P was the estimated probability of achieving the productions goal and was made up of the product of qr The probability that the pr oduction would have achieved the desired state q was fixed to 1 and the probability of ach ieving the production goal given arrival at the intended state r was based on previous successes and fa ilures. However, the hobbits and orcs task differs greatly from the BST in that previous successes and failures have less of an impact and problem-solvers could have ca lculated future states and illegal moves based on the rules. To adapt this model to the hobbits and orcs problem r would have been fixed to 1 because successes and failures contribute little to the su bsequent selection of operators and q would = the problemsolvers ability to correct ly calculate the future state. It was possible that q would also contain the 3-stage framework GCV proposed by Knowles a nd Delaney (2005) if the move were to be assessed for legality. However, pr evious research seemed to indicate that when the cost for making illegal moves was low problem-solvers often did not engage in this activity. With an increase in cost for an illegal move problem-s olvers would have likely increased the frequency and/or accuracy with which they engaged in calculating q resulting in P yielding higher values for legal moves and the reduction of illegal moves.

PAGE 97

97 As indicated above, move evaluation functi ons from other problems or from similar problems may have the potential of being updated and integrat ed into the CIF framework. However, additional research is required to determine if a move evaluation function can be adapted from another problem, from similar pr oblems, or if a novel m ove evaluation function would provide the best fit for CIF. Real World Application and Future Research Problem s are something that we encounter every day. Whether it is balancing our checkbooks, driving to work, or co oking a meal, these daily tasks consume a large portion of our lives and understanding how to so lve these problems more efficien tly would offer great benefits. Laboratory results have revealed that one way to improve performance may be through punishment (Knowles & Delaney, 2005). However, being punished for every mistake or illegal move we make does not sound like a lot of fun. Therefore finding alternative ways to improve performance seemed desirable. The current work looked at alternative ways for improving problem solving and discovered that a threat without punishment reduced illegal moves. Motivati on and reward did not result in significant reductions, but motivation showed prom ise. These findings seemed to support some things that we already do or experience today. You threaten your child not to touch the stove for he will get burned or your boss threatens you that you will get fired if you show up late for work again. The findings presented here indicated that a credible threat can yield improvements as significant as punishment, at leas t on these types of problems. Th e motivation and reward results were somewhat surprising based on common practic es in the world today. Motivational seminars and tapes are utilized by companies and individu als in an attempt to enhance performance and rewards are offered in the form of money or ot her incentives for meeting or exceeding expected goals. Because motivation and rewards are successfu lly used in our society to drive performance

PAGE 98

98 it strengthens the claim that the design lacked the power to detect these effects or that the motivating instructions or reward s were not adequate. So then what has this work offered? The CIF framework made several assumptions about how we solve problems and operate in a given problem space. The main assumption of CIF was that we do not perform optimally in certain situations and that we have the ability to do better. Th e results presented here supported that assumption by demonstrating that with c onsequences for illegal moves or by increasing cautiousness through checking we can reduce ill egal moves and increase problem solving efficiency. One potential met hod for avoiding illegal moves is by being more cautious and checking moves for legality. We can liken this to an airline pilot who has a checklist of items he must verify and complete before he is able to take-off. Of course, we do not need a checklist for everything we do in life, but being more cautious in certain situa tions would serve us well. Some examples of areas in life where we could appl y this information and benefit include: driving (checking and abiding by the speed limit and othe r laws would likely prevent us from getting a ticket or getting into an accident), finances (c hecking our bank balance before writing a check to ensure that we do not overdraw our account), and military training (ensuri ng that a soldier knows the proper procedures for a crit ical situation so that no one gets injured). The real world applications and future avenues for such findings seem limitless as we are constantly striving to become more efficient to conserve time and/or resources. The results obtained here seemed to support th e CIF framework that certain consequences of illegal moves can reduce illegal moves and th at checking moves for legality may be one method for reducing those moves. However, severa l questions still remain and future research should focus on these questions. First, what ot her types of consequen ces, including different types of motivators and rewards, are able to reduce illegal moves? Second, Stage 2 of the

PAGE 99

99 framework should be further studied to determin e if an attentional sh ift is responsible for increased caution and if so how this attentional shift reallocates these additional resources. Third, future research should also focus on the evaluation function used by problem-solvers in determining which move to select next in the problem-solving episode. Finally, how the accuracy of checking is altered over the duration of a problem should be explored to determine the likelihood of errors being made after a pr oblem-solver has decided to check a move for legality.

PAGE 100

100 CHAPTER 6 CONCLUSION Three experim ents were presented to assess th e ability of the CIF framework to describe and predict problem solving performance and the reduction of illegal moves. The main assumption of the framework was that consequen ces of illegal moves increased attention to reduce illegal moves. The first experiment te sted whether a penalty was required during the problem solving phase to increase attention or whether the thr eat of penalty alone would be sufficient to increase attenti on and reduce illegal moves. Th e second experiment addressed whether an adverse stimulus or threat of an adverse stimulus was required or whether motivation or reward could be effective at inducing cauti on and increasing atten tion. Finally, the third experiment directly addressed a portion of the framework by instructing participants to check moves for legality and also collected verbal protocols to assess what participants were actually doing/thinking. Experiment 1 results indicated that punishment was not required to increase attention as significant results were found in both the thr eat and experience conditions. These findings demonstrated that less intrusive interventions could potentially be as effective in increasing attention when compared to punishment that occurred during the problem solving episode. The results of Experiment 1 have significant real world applications that could be helpful when punishment can have serious negative effects or when punishment is not feasible during a problem solving episode. However, further re search is required to determine the true applicability of these results to real world settings. Experiment 2 did not achieve the predicte d results and the motivation and reward conditions were not effective at reducing illegal moves. The lack of illegal move reductions could be attributed to one of several explanations. Experime nt 2 utilized undergraduate

PAGE 101

101 participants and was run at the e nd of the semester. Thus, the partic ipants in this experiment were likely those that waited until the end of the se mester and may have been less motivated and resistant to the experimental manipulations. Al ternatively, the motivating instructions and the candy reward may have not been sufficient to in crease attention and a different motivation or more valuable reward may have achieved significant reductions. Fina lly, it was also possible that there were no end of semester effects a nd there was nothing wrong with the motivating instructions or reward and that an adverse stimulus or threat of adverse stim ulus was necessary to increase attention. Experiment 1 demonstrated that a punishment was not necessary to reduce illegal moves, but the concern of future punish ment was able to reduce illegal moves. The concern of future punishment still had the belief of something negative and may be a necessity in inducing caution. Experiment 3 directly manipulated and test ed the third stage of the CIF framework. Instructing participants to check moves for le gality reduced the number of illegal moves made whether participants were instru cted to think aloud or not. This manipulation was the first that did not focus on the consequences of making an illegal move and attempted to reduce illegal moves by avoided them before they were made. Th is direct manipulation re vealed that increasing the frequency of checking resulted in a decrease in illegal moves. This finding also has practical applications in that recommending or requiring a person to check for legality in problem solving episodes may help increase problem solvi ng efficiency by reducing illegal moves. One unexpected result obtained in Experiment 3 that was not addressed was the increase in illegal moves from the first to the second problem for the aloud group. This release from benefits was likely due to an interaction between the th inking aloud and checking as there was no release

PAGE 102

102 found in the check group and no release in al oud groups in previous research (Knowles & Delaney, 2005). Further inve stigation is required to determine the specific cause of the release. In conclusion two experiments lent support to the CIF framework and indicated that increased attention was a likely cause of reductions in illegal moves. The threat of punishment and instructions to check move s for legality both reduced illeg al moves and provided support to the framework. However, reductions in illega l moves were not discovered with motivating instructions or reward, possibly i ndicating that an adverse stimulus or threat of adverse stimulus was required to induce caution and increase atte ntion. In conclusion, these experiments provided support for CIF, with the excepti on of Experiment 2, and indicated that the framework may be a valuable tool in determining potential behaviors on such tasks. Additional research is required to further explore the ability of CIF to predict problem solving behavior and to determine if rewards and motivation have the ability to reduce illegal moves.

PAGE 103

103 APPENDIX THINKING ALOUD INSTRUCTIONS: EXPERIMENT 3 The thinking aloud instructions presented to a participant in Experim ent 3 were as follows: In this experiment you will be asked to solve problems on the computer using the mouse. However, as you work on the problem you will be asked to think out loud. That is, as you work on the problem I would like you to say whatever comes into you head. It doesnt matter if it makes sense or not, I just want you to say whatev er you are thinking. For pract ice, try this simple task: Think of as house that you are very familiar w ith, it could be yours or a friends. Think out loud as you imagine yourself walking through th e house counting the number of windows you see. You may begin whenever you are ready.

PAGE 104

104 LIST OF REFERENCES Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solutions. Psychological Review, 94(2) 192-210. Anderson, J. R., & Lebiere, C. (Eds.). (1998). Atomic components of thought. Hillsdale, NJ: Erlbaum. Atwood, M. E., & Polson, P. G. (1976). A process model for water jug problems. Cognitive Psychology, 8, 191-216. Delaney, P. F., Ericsson, K. A., & Knowles, M. E. (2004). Immediate a nd sustained effects of planning in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition 30 1219-1234. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press. Gray, W. D., Sims, C. R. Fu, W. T., & Schoelles, M. J. (2006 ). The soft constraints hypothesis: A rational analysis approach to resour ce allocation for interactive behavior. Psychological Review 113 (3), 461-482. Grobe, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: Can this foster learning outcomes? Learning and Instruction, 17 612-634. Jeffries, R., Polson, P. G., Razran, L., & Atwood, M. E. (1977). A process model for missionaries-cannibals and ot her river-crossing problems. Cognitive Psychology 9, 412440. Kane, M. J., Bleckley, M. K., Conway, A. R. A., & Engle, R. W. (2001) A controlled attention view of working memory capacity. Journal of Experimental Psychology: General 130 169-183. Knowles, M. E., and Delaney, P. F. (2005). Lasting reductions in ille gal moves following an increase in their cost: Evidence from river-crossing problems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31 670-682. Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from Tower of Hanoi. Cognitive Psychology 17, 248-294. Kotovsky, K., & Simon, H. A. (1990) What makes some problems really hard: Explorations in the problem space of difficulty. Cognitive Psychology 22, 143-183. Lovett, M. C. (1998). Choice. In J. R. Anderson & C. Lebiere (Eds.), The atomic components of thought (pp. 255-296). Mahwah: La wrence Erlbaum Associates. Lovett, M. C., & Anderson, J. R. (1996). Histor y of success and curren t context in problem solving: Combined influences on operator selection. Cognitive Psychology, 31 168-217.

PAGE 105

105 Lovett, M. C., & Schunn, C. D. (1999). Task repres entations, strategy vari ability, and base-rate neglect. Journal of Experimental Psychology: General, 128(2) 107-130. Luchins, A. S. (1942). Mechanization in problem solving. Psychological Monographs, 54 (Whole No. 248). Miller, G. A. (1956). The magical number 7, plus or minus 2: Some limits on our capacity for processing information. Psychological Review, 63(2), 81-97. Moss, J., Kotovsky, K., & Cagan, J. (2007). The in fluence of open goals on the acquisition of problem-relevant information. Journal of Experimental Psychology: Human Learning and Memory, 33(5) 876-891. Newell, A., & Simon, H. A. (1988). GPS, a progr am that simulates human thought. In A. M. Collins, E. E. Smith, (Eds.), Readings in cognitive science: A perspective from psychology and artificial intelligence (pp. 453-460). San Mateo, CA: Morgan Kaufmann, Inc. O'Hara, K. P., & Payne, S. J. (1998). The effect s of operator implementation cost on planfulness of problem solving and learning. Cognitive Psychology 35, 34-70. Platt, R. D., & Griggs, R. A. (1993). Facilitation in the abstract selection task: The effects of attentional and instructional factors. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 46A(4), 591-613. Simon, H. A., & Reed, S. K. (1976). Modeling strategy shifts in a problem-solving task. Cognitive Psychology, 8 86-97. Turner, M. L., & Engle, R. W. (1989). Is working-memory capacity task dependent? Journal of Memory and Language 28, 127-154. Welsch, M., Cicerello, A., Cuneo, K., & Brennan, M. (2001). Error and temporal patterns in Tower of Hanoi performance: Cognitive mechanisms and individual differences. The Journal of General Psychology, 122(1) 69-81. Wieth, M., & Burns, B. D. (2006). Incentives improve performance on both incremental and insight problem solving. The Quarterly Journal of Experimental Psychology, 59(8) 13781394 Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9 (4), 625-636. Zhang, J., & Norman, D. A. (1994). Represen tations in distributed cognitive tasks. Cognitive Science 18, 87-122.

PAGE 106

106 BIOGRAPHICAL SKETCH Martin E. Knowles was born in Huntington, New York, to Jam es and Joyce Knowles in 1978. At the age of seven, he and his family move d to Tampa, Florida, where he lived until he was 18. At this time, he began attending Florida State University in Tallahassee, Florida, where he completed a Bachelor of Science degree in the fall of 2000. In 2001, Martin moved to Gainesville, Florida and began his graduate studies in cognitive psychology at the University of Florida, under the supervision of Dr. Peter F. Delaney. He married his wife Stephanie in the spring of 2004 and completed his Master of Scien ce degree that same year. His masters work was subsequently accepted for publication in th e Journal of Experiment al Psychology: Learning, Memory, and Cognition. In March of 2006, Martin became a father to a healthy baby boy and subsequently moved to Jacksonville, Florida with his wife and son. He successfully defended his dissertation in August of 2008, and received his Doctor of Philosophy degree in December of 2008, from the University of Florida.