<%BANNER%>

Implementing a Global Anti-DoS Service Based on Random Overlay Network

xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20110115_AAAAAB INGEST_TIME 2011-01-15T05:11:22Z PACKAGE UFE0007880_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 69714 DFID F20110115_AAABOD ORIGIN DEPOSITOR PATH shen_w_Page_73.QC.jpg GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
e81b9b30a9ec8634f62b2e8560f86981
SHA-1
702e64a2f73a55cd1da4ddc9e64ff2cc621a7172
22364 F20110115_AAABJG shen_w_Page_73thm.jpg
f0dc201985e53efa316f0965515c5b4a
80f4e6122f6e44980273c306cf89283f16df29e2
78094 F20110115_AAAASO shen_w_Page_30.QC.jpg
01332e8182a468863f323a3897c4638e
1711ad228f1a08a92ddfd9332cc734f1c58c0f00
28029 F20110115_AAABEJ shen_w_Page_19.pro
3665a19a7e071280ac3bc781388368ce
9f906a4ffdbd01306c1676eb0778950dafd94367
172058 F20110115_AAAAXL shen_w_Page_66.jpg
87c26d42935d1f8b73a83fc7242cb652
e5942ebf365230227558389719cbc2176b134dd0
42651 F20110115_AAABOE shen_w_Page_74thm.jpg
485a784dbd251c96a84adf0b8b024f4f
f3379598d73ae54c5395067f23a3a43fc742718c
12611 F20110115_AAABJH shen_w_Page_84thm.jpg
14f7a32154bc9358805e5c79dc3654c5
30a02daf3b56b7522557aece51edceddd36d9539
27023 F20110115_AAAASP shen_w_Page_49.pro
ea5b2793178f41376a3f651e87f70090
3d13d5e22756443eb1178897effa578e3649c77e
51044 F20110115_AAABEK shen_w_Page_20.pro
b21ff55bda8d329f693c760405a427fd
ac4c3ba9596015448ef73cc1065e4c9c602f88fa
214144 F20110115_AAAAXM shen_w_Page_68.jpg
7e0e3c93ff1f3429f60069c9b866d1e6
b9b34e25b95933f6e64e78591173ca29ad2f847a
37236 F20110115_AAABOF shen_w_Page_75thm.jpg
7ccdb53d55d6a43a3817d60804488da0
463c19004cb4f288f7a95f9c31f4ae72fd9939e0
78368 F20110115_AAABJI shen_w_Page_16.QC.jpg
bafd7d08881a1dda4a32ca50bc7a853a
f5e9a74ca7bf8d1ab8ce37ba777e5ce07d49b3c7
196532 F20110115_AAAASQ shen_w_Page_36.jpg
84ab73bca52316338fcbf290048a1514
92943ae254c7ee142997018f824df3e39219c372
43767 F20110115_AAABEL shen_w_Page_22.pro
8a1e7cab8a344e52e33b6c48458622bf
b28661ef24b8473a87bb9e6c3ce68df02c96e809
201031 F20110115_AAAAXN shen_w_Page_69.jpg
0078bbc0ff4ab1793a5d4f49eb24e545
517189ddc95bc76ae65a05f8039bec8fd4364ced
9722 F20110115_AAABOG shen_w_Page_78thm.jpg
a77f77bf7762295cdbc1522ee1018940
8198b35d1ae9c00fdea532e6cd4f4ecb7e1b62dd
71849 F20110115_AAABJJ shen_w_Page_36.QC.jpg
a6052cd5b95fc3f36bceacc2cebd9632
7488ea9c538dfb58573c17c5d0e894dfa81b3d47
1051984 F20110115_AAAASR shen_w_Page_09.jp2
fa0cc0f7c3a1f183dab6310c3db2cc35
8b86e2922a976917129f39f62b7904498d83883b
46506 F20110115_AAABEM shen_w_Page_23.pro
a1e0bfcc0f95b9d0b8bbc66813f8f259
6f92959bed65258596ccee2d6fac4bacc2a08e93
218922 F20110115_AAAAXO shen_w_Page_70.jpg
9edaab1fde9cd2c67cdbf10172d3b5f0
fbfdd81370e8b27eeb6ec8dc3e2a6f1b2163933b
30068 F20110115_AAABOH shen_w_Page_78.QC.jpg
bc8f5ab62382e08404b375299a8e498f
84262fe51ad2404965d4633b34e5107cb6a1de7d
22499 F20110115_AAABJK shen_w_Page_62thm.jpg
05a8c853e1a8c22d64fcd37e2e58b6e5
f9816f8bfca52d0eb0c675fc6579f5ccf863a44e
95712 F20110115_AAAASS shen_w_Page_38.jp2
ad7c2065aa5dc8d9cc4070b02e60c850
ef5f828eabab1fd318d342589bef31313ba969d1
215541 F20110115_AAAAXP shen_w_Page_71.jpg
e477fafe272dbdce5993f8f72ccc1432
955a14997265783b155e1b40f60e5a3791d720f5
11752 F20110115_AAABOI shen_w_Page_79thm.jpg
1df0f5bdb2ea1728611d0a85aacf208a
0e29509837e3b00b6f88c78a90713d84fa53ba6c
48697 F20110115_AAABJL shen_w_Page_14thm.jpg
bff77ada4884878c0ca1d76ab09a4dff
4e119f8263dbe56d13152329651d087480cb0919
49793 F20110115_AAAAST shen_w_Page_43.QC.jpg
53c600e538d634e2e983426e76569479
801ce07162faf8f84ec20d97840f20c77671c897
41457 F20110115_AAABEN shen_w_Page_24.pro
425202b3e4d73da0a5622fd8f2147225
257555734c37c39a9f00aefdcc8e307aeb7b3111
230819 F20110115_AAAAXQ shen_w_Page_72.jpg
e28720e58794aacc525b411429606591
85ab89394439feb2b458afc726d6ca016a541bd5
32577 F20110115_AAABOJ shen_w_Page_79.QC.jpg
d835c853c15ca6acd385d7ab8ff62f3a
f3013100d9cbda0cdc2754fe82a234d59ed66223
77633 F20110115_AAABJM shen_w_Page_74.QC.jpg
456f10b41a6cf7280b0aa55939c3afdc
2ba9e9b5d5a49d8fc9dc2950df233568afab4fb2
2045 F20110115_AAAASU shen_w_Page_58.txt
9000c729a51cd5256173d507d484427c
ed3cd6260d67d7806c4948a0b9c37542654652a8
36897 F20110115_AAABEO shen_w_Page_25.pro
5b30c3b8e23a30ddb0f5007d92e12d51
79e4365147d55c83032944b315b40dba6db905ac
189762 F20110115_AAAAXR shen_w_Page_73.jpg
08f628dbe0a0f271635a3306f87d5eea
8571c11ff1ba97c642bd47e508d062840988bb61
21022 F20110115_AAABOK shen_w_Page_80thm.jpg
01b5306b2e84b26d5789a01ec55b9515
be0b663f1c2474137efe77c06df3ffa53abb78d5
68196 F20110115_AAABJN shen_w_Page_22.QC.jpg
a20ba202f02f3677f28bc1721a845b4f
c6ab36e994ec7e0b11ab58ebd05a13700bc98dfb
68348 F20110115_AAAASV shen_w_Page_62.QC.jpg
19f095c15fd8ae5d267d94e0ca83d938
0b1931f70b22c581eca1678e35bf340990874b2a
25133 F20110115_AAABEP shen_w_Page_26.pro
f1f6f08830abf2a67ab18bd182a3b568
9667304be7c31a4e46c19a685ca368180f30ae78
183635 F20110115_AAAAXS shen_w_Page_74.jpg
cd68d42968acd217d1b59fa99aa4c15d
efda8efac78916fa46478c6ec9034c50f71ccb5f
69459 F20110115_AAABOL shen_w_Page_80.QC.jpg
72ef69f7df0214b22168c2357e7e8f57
be0b5108b567011919314a2bd8f3d10a581997bf
16612 F20110115_AAABJO shen_w_Page_27thm.jpg
f17253d89da886bf097221e4f37c6755
d07e2a141fd6b166465a5471372e654189fab374
1053954 F20110115_AAAASW shen_w_Page_22.tif
f49edbb973168ca7836137ec4d5a98fa
707399f5f3b1a5a1cbed2da4a3cf825d8dd1e5a4
31674 F20110115_AAABEQ shen_w_Page_27.pro
eb8fb7cfec38a554518e368330857258
381ff4cb77e9d36891d8cffb68316d8485914261
169606 F20110115_AAAAXT shen_w_Page_76.jpg
c7752b253b8c4407fea6e445c269e598
73347d6ee4ae1cbc7fb66f2f53a6be44fe799755
11454 F20110115_AAABOM shen_w_Page_81thm.jpg
b7bf9426e8499920cecfabeff8f04614
6d4f0b8f949fe563cbb5d572e9603e7ea8e53837
49721 F20110115_AAABJP shen_w_Page_53thm.jpg
0c1264683aeefa43cc3be6029b69ffd8
b34c0519ced7edb43e4973d3c9f35a56ef4310a7
315943 F20110115_AAAASX shen_w_Page_06.jpg
84e277644f8ac2d79a269accc30f4af7
8ec1ca2394fedf8f7a8fc03f90590c4d7dbb1a06
39161 F20110115_AAABER shen_w_Page_28.pro
a03c7f2fbda651a80cc2bcafed5c6367
48aa9bc232f8ffaa54a24efbf142bdbd99a607f0
174758 F20110115_AAAAXU shen_w_Page_77.jpg
07e9845075f94c054aa0771cd432e91f
918e625e75b49351d452d7d7e6cfe5361d91f83d
37744 F20110115_AAABON shen_w_Page_81.QC.jpg
a906631fce672e1943b0976c0c078e16
3ab222d4fc051d45b02ab1291e23c65b5a96fcc3
22167 F20110115_AAABJQ shen_w_Page_22thm.jpg
54417e98ef73fb337d02d9a515cd4eb8
69dbb75b23bc08c317315ad322090ac238ccde78
34309 F20110115_AAAASY shen_w_Page_08thm.jpg
62c188347cf7b4d20d309bb9146d2f86
f8a00dcdd21a30722edb2e9939c57661b0b107af
44726 F20110115_AAABES shen_w_Page_29.pro
e7fe5dddfd15019d1392a071ba877279
95406c575f333a92c2f0fa0f5838cc7adb92a33e
76965 F20110115_AAAAXV shen_w_Page_78.jpg
d959b0e4e24b47f7ca0a177ae9194d6a
82007a06901b74fba303c0aa54316257fea01aee
20828 F20110115_AAABOO shen_w_Page_82thm.jpg
e666de97efc743b48341093dfe93788f
f126395b2798370804ed03914c4934f7b19cf204
100682 F20110115_AAABJR shen_w_Page_59.QC.jpg
169b0646b43e6545486aa86006acd33e
1a84bfe635ad3dc714df0af320e6ef756581b994
45712 F20110115_AAABET shen_w_Page_32.pro
2127db2ca37f8d26ca8873b8e70263ac
ed8984ce5d70baa767b3cc86754e59d2d62d6aeb
83971 F20110115_AAAAXW shen_w_Page_79.jpg
5a7c66f7cc6696255ef2a9c8f1934bad
6a0d30490a1fb8211fe9786fa67b8d9d605b649e
200540 F20110115_AAAASZ shen_w_Page_67.jpg
80eb87791dba5eb85fe9725b778a9334
38966a763c01bc4c57c4f8a4a32d01ccfac9c543
69992 F20110115_AAABOP shen_w_Page_82.QC.jpg
c5001bbb81b8fa8b994388d31e4e384a
0ddeb3603ce31e8d484bc97d146db9e0699dfc35
49194 F20110115_AAABEU shen_w_Page_33.pro
6fd65d0fb3e80e657c01589b8b592dc9
9310517c2d72cd2c4300b9905fed544e820d357b
196714 F20110115_AAAAXX shen_w_Page_80.jpg
98f2912a64a28d53b97e0966225e5a26
ace4904e0d991e2d541b75bfc3276f77e560cd20
39193 F20110115_AAABOQ shen_w_Page_84.QC.jpg
9e88f5514cfe1765bbe36168b9ec33a0
0eca908ca024ef1b8d1510df147cd83d56669087
17782 F20110115_AAABJS shen_w_Page_83.QC.jpg
f29afd547a039657bd951453c4aeba3a
349ead3416309ea8e26f0f521dddb67be0bdebb2
45796 F20110115_AAABEV shen_w_Page_34.pro
f1dfa2e56f120a57e37eacd55faac206
13123000edd52fd8f52685f65e5f4bfd266e9737
217161 F20110115_AAAAXY shen_w_Page_82.jpg
a53b1924d570f4473e6264250f38321b
4294158a48d4dc599154b3020673982fe7d05259
103784 F20110115_AAABJT shen_w_Page_53.QC.jpg
bdb930f3785a3a6c0286780ede5da69e
3534de27db1ed909115d4a590e27341a5bf5a154
41886 F20110115_AAABEW shen_w_Page_35.pro
83fa0a763ea2033be4a0c715a10fa5aa
91f6a9be6c2fc9bc0f326c13956c63f1109a408e
72675 F20110115_AAAAVA shen_w_Page_57.jp2
f446ce76004fa9990b9cd0ef58504652
fef8d94e8157b45582c9c76a41ad85d07db97ffd
55426 F20110115_AAAAXZ shen_w_Page_83.jpg
22160c734ec116bd92aa385174436887
bdd85eb2cecef8990d79293244cdfc318890c253
25257 F20110115_AAABJU shen_w_Page_42thm.jpg
98a5cb11d7b665f35ab60ef8f4fa2f69
1faa1bb534c303e392f223602c381ef02ea9f78a
45140 F20110115_AAABEX shen_w_Page_36.pro
56062a20cfa5817a78db58a94c0b8f46
8de6d1c75c50c5ee127019d264e7c33cbb39507e
1051980 F20110115_AAAAVB shen_w_Page_25.jp2
e4230e564293c329e4fd36b1ec61fff7
b724c4a4667be8085f56dca95b5cfe4916ca6c9f
48145 F20110115_AAABJV shen_w_Page_59thm.jpg
35a5a07e1983659ddaee65cac1bb7b01
8741bdb53b651ab4950dd0a368a999f5dab3e3b2
35678 F20110115_AAABEY shen_w_Page_37.pro
9e51110684ab5684bbdb3dac3e0cf3b1
ef357e149399d063eea6b451c8316daa8b80a864
35209 F20110115_AAAAVC shen_w_Page_21.pro
2a5ae79faa847ae4392ababcf2b48456
4a99733d9c604edd29e6b2d324994c2afd8ecbe3
25271604 F20110115_AAABCA shen_w_Page_37.tif
eccae6c87bd1c820a9d51985814dd271
8347fde38be9ff9b70a7377a4d25df9db9ae371b
125815 F20110115_AAABJW UFE0007880_00001.xml FULL
973adb56d70a1965a8a3d3ca499343ea
d6a65cceb311ef561ded9bf3135c2d3b2a0a8f6b
43349 F20110115_AAABEZ shen_w_Page_38.pro
cff2798d4ef3c6c3f2cd13dde270101d
e7bc162a2f98fc1bd8e9a1ad225952182352e491
179184 F20110115_AAAAVD shen_w_Page_56.jpg
582bc36f06e61df3fc8fb44ef06dd6be
f2b7f48a06ebd26146c45b6f11543200aa1f676c
F20110115_AAABCB shen_w_Page_38.tif
4e331fa2c3b24de70f9cb89051b21ee9
b484a67d02a6f1142e297d83a3c4320cfbeb48f5
6745 F20110115_AAABJX shen_w_Page_01thm.jpg
897bb7d509f86999d64d337860b17726
41365bb67d726dd898f8d1234ca19a092743abb1
38478 F20110115_AAAAVE shen_w_Page_76.pro
715e64e136c0b45608863a735eabd4e4
ac1f0e3a76af12c110bc7c0960149cef10ad1fcc
F20110115_AAABCC shen_w_Page_39.tif
350bab2cb5b614a1dd3bdefbd20991a3
30be47b0a38c996d0b1ed2f58f8cd48f072d68ad
19095 F20110115_AAABJY shen_w_Page_01.QC.jpg
1653cd27d9b75cf1d254a7861cb2517b
41a7b95ab3419e90d8f96bc07aa9274085083a8f
20695 F20110115_AAAAVF shen_w_Page_77thm.jpg
f9623a94ed72773c3357d4d4f5262a81
8e18b37d7fc2cbc8f587c26e981b7b2bdcbe08e2
2020 F20110115_AAABHA shen_w_Page_20.txt
d0dc23f5ffcb8578f517e9d6a12ef3c4
af47979149830401b2c312603424fdd5b59e5d44
F20110115_AAABCD shen_w_Page_40.tif
aaef133b0fec1b1031eba435ffd39cc2
312a2428a3db0ebc2b67cdbc0d380aa5731513e2
3175 F20110115_AAABJZ shen_w_Page_02thm.jpg
2938f8e8fbc0e7fc2c580c18d8968a73
150cda5d839de7826d540ebacb3912c00884f860
89069 F20110115_AAAAVG shen_w_Page_40.jp2
f92f40dcb58f4ab7879c6777394c679f
59cf23b1043b18a818bf0a052d600970ada6883b
1839 F20110115_AAABHB shen_w_Page_22.txt
3dcedaa8521d3e7715827cdb8b2f454a
822962c052c694708b688d2d5774dde69f3a061c
F20110115_AAABCE shen_w_Page_42.tif
28b465eb428ee51f6c35583bfc1ab10f
8641799a6682edeabbf5b3f1961d47aa95840bef
97229 F20110115_AAAAVH UFE0007880_00001.mets
27dd4dfdcf015487f6fed8e42be6b0fc
a0b478d174918fa778cfb988dad45be34aa1e663
1899 F20110115_AAABHC shen_w_Page_23.txt
236011148c02643c790f2c4be35d087c
e6c85300e927ccb7bc92b1d2f6818edad52fc153
F20110115_AAABCF shen_w_Page_43.tif
6e454902f2af7285a217332fcf7a3b02
4e88c00f967f2d2e7db06b65bf01bd761bb643dd
21424 F20110115_AAABMA shen_w_Page_35thm.jpg
829ae521150c04cdecd7875544ef8d04
59a3e738d980ff4dc3a16eafcde170c9d4ec03d4
1739 F20110115_AAABHD shen_w_Page_24.txt
5deb110ff97470aae4af20146e04eb45
48d22c9dc3dae8ec2849189d73040818cf42d2eb
F20110115_AAABCG shen_w_Page_44.tif
89436ab746eb79ea4a43c69bc2270965
1e9312ea47039d7d048a4cfccba1bcf4904bf93c
65307 F20110115_AAABMB shen_w_Page_35.QC.jpg
b50b12a1c599013321dd05bc959a0344
8c14794488025a943a29f46680ff7da7324654e6
1545 F20110115_AAABHE shen_w_Page_25.txt
aa911e17f98015abfa13795c942baa16
53f81c6ac9f50d744f6b527e78d376d32e83152a
F20110115_AAABCH shen_w_Page_45.tif
2e1f59ce067c3c23ed68dbbe251bde82
86ae78efd3bd156a19b1cf1ab0c27d74978817a2
23114 F20110115_AAABMC shen_w_Page_36thm.jpg
4d7290cb7b22d53e6cfe9b55c986b0f2
a649d39e8d0dd37b4be4dcdcd668d86080942b36
14782 F20110115_AAAAVK shen_w_Page_02.jpg
6c1ebe19fd51fa1fa669c1af19bda789
5dce73bdd3360edb9e58f362c59320b2f26cd607
1322 F20110115_AAABHF shen_w_Page_27.txt
7a97ffd16c7ab4b7dc8908d8b7a96712
af510d841cc8eb822ddfab554f2df0519282f094
F20110115_AAABCI shen_w_Page_46.tif
bc6fdc286f27c30e8a3ca4cdcfda6557
81d01f682b6529a21b1572105aed6d43e9e2e352
87394 F20110115_AAABMD shen_w_Page_37.QC.jpg
6ae8ddbef0eb01a95c1d62aae4a3832a
5b289216b3aac7d962276a685b13be813dd088c9
18883 F20110115_AAAAVL shen_w_Page_03.jpg
1e6fa34ef0b1a08c5e1240fd8b07b5a6
6c387a9cf54baf03d0f6f05567cc16e79e6d0d28
1674 F20110115_AAABHG shen_w_Page_28.txt
ef23c249e2a1cf96dc68627fe1e2b1bc
744260f2c0386ab8684c61a0ab47d6fda471e4a2
F20110115_AAABCJ shen_w_Page_47.tif
c15440d6fc63336cd35b5d3d63b053c9
b32ec2856e899fe4ac4f9650f692a1e8c537f3a3
22029 F20110115_AAABME shen_w_Page_38thm.jpg
ce94d688737f586d33747d5f963fc3f8
9f37d2d86a7266f347115d58bd7229696b16bdb7
248849 F20110115_AAAAVM shen_w_Page_04.jpg
99d2f98bfbebcac9ac5623c11d659fbf
30dc1dff5e7ee0ecac1f2cb2e0e2e3cf3b919c78
1801 F20110115_AAABHH shen_w_Page_29.txt
64fa36c7ab0cfb25f17fd730633fa230
d052fd1492f1cc0855dac7c3487f9f25c819a03b
F20110115_AAABCK shen_w_Page_48.tif
351215d55cdb7acc81ceb865b01a746f
ff7c0eb923bac26a8de25e681b8e22a67f65a97b
62076 F20110115_AAABMF shen_w_Page_39.QC.jpg
b6fad4d3bce259219c7387580ddab252
6a81846756e326ce371fae49fea609bfb7d38d25
1978 F20110115_AAABHI shen_w_Page_30.txt
c638440d19e9931a89ddcd0886c0af88
ce89ddea3283cad5ce9e7212a99fee945926c0a9
344394 F20110115_AAAAVN shen_w_Page_05.jpg
ed23ff1d4d20349f2f35d1f57025a873
f11123df02193b79abb7ce456c5b72e6df700131
20431 F20110115_AAABMG shen_w_Page_40thm.jpg
fda70a1d019241fbac1050d44e772b4a
b6051ad468ce6d95d6de933b256917ac71f20306
1404 F20110115_AAABHJ shen_w_Page_31.txt
04ae59af3c3d2420f4f5a670d1baac14
ef72412122a4418f30fcdb4a6a853c8beac38ae0
100828 F20110115_AAAAVO shen_w_Page_08.jpg
a8f11c9557a29d09a6029cf428b977f8
38d0c603de6dd4aa9501d569090971cd7e306365
F20110115_AAABCL shen_w_Page_49.tif
93868b3fec72f7a5789b6a0030072f62
56958f28ae60ed388f61d7b200892ccdc0b0d09a
62610 F20110115_AAABMH shen_w_Page_40.QC.jpg
3c2effc40d346a72872bb5153a9a250c
f1eecd9d948cf4597c6d26164aead3fa7f7c0fd0
1842 F20110115_AAABHK shen_w_Page_32.txt
e09dd4c093f7ac319ad8cd4ae440aae4
43e8daea9211a7b08cc906508383aba6fe9021fb
243830 F20110115_AAAAVP shen_w_Page_09.jpg
a3896eb150ff8a190f037f7a1df0de9e
85bb47eb93bb5b059d7b96297d88775fac2c2e58
F20110115_AAABCM shen_w_Page_50.tif
672194f4d67f947a934d3b3e9de41257
f0c5e6e0e00bbd21769ac941f231a571b14e48da
22023 F20110115_AAABMI shen_w_Page_41thm.jpg
e0f6470f0cc2980b8ec6c1d3b9e32b70
d06c87e631c58c16946dbeecde8089562d0df0da
1942 F20110115_AAABHL shen_w_Page_33.txt
898b268b66b8bc09d7071d3f4708d831
d309c50a157e5a1b7743954d449615312ff919bf
57255 F20110115_AAAAVQ shen_w_Page_10.jpg
8db3844fdf84a84b815cd2b70e164980
437529af57318b8ab361c9754c22201dfd48229a
F20110115_AAABCN shen_w_Page_51.tif
cf5a977d6d73228cadeda8df7033c793
bb620d208aaa8e271360c6c4fdc61d220cdf8dd4
72081 F20110115_AAABMJ shen_w_Page_41.QC.jpg
9dbf467eb56a74a8ccce909fa11fbd8f
2c877b9f8bb611d2d7bf566dec23be7136d9f549
1817 F20110115_AAABHM shen_w_Page_34.txt
2dc12221410339a987e786b2e510d376
96c6ab3b17b5f90ed55990e1ee7948c0483a9ad2
164453 F20110115_AAAAVR shen_w_Page_11.jpg
ef549fede9ac453f7195ae2569b39573
50e2bb673cb9f487f44cff63319c9143bed246ac
F20110115_AAABCO shen_w_Page_52.tif
9200a27bbb71a0cc02056cb20bd10b86
d572f20c00eb048841a68030dcf495a4cef6e107
85570 F20110115_AAABMK shen_w_Page_42.QC.jpg
30770e36e59277fa921c8172ec1e4abb
37bcfaaa456c68e9a9dc4c87ddc3307843106363
1823 F20110115_AAABHN shen_w_Page_36.txt
7a9b29d3e40477e7bb15fa278c0708f5
4618606aea18b4ae5dd9a6a8a6ffe1d5900de56e
66102 F20110115_AAAAVS shen_w_Page_12.jpg
ee23d6b9e5c7d5fb0b5963e1509b8b55
6046a60c179b553455679d14a6ff53c5630b0c65
F20110115_AAABCP shen_w_Page_53.tif
265cb6af107fbccc866c061ad66a7e39
b5444d4a31674608aada318f028f2411c9a6f46c
16316 F20110115_AAABML shen_w_Page_43thm.jpg
a483a53c5cecd03c343e834552e1febd
76f53ef1ceb59c913a3448cd7fdd4a77ab4dc71f
1532 F20110115_AAABHO shen_w_Page_37.txt
39bad9f0cf7c6af31063f19e26cd00da
303739cab85db8b0f46c28cac687b53337d7eeb9
201593 F20110115_AAAAVT shen_w_Page_13.jpg
e05c5d7b010481340f00559c15616206
f5bcf27685c8185fca21aed3821261dc713594c6
F20110115_AAABCQ shen_w_Page_54.tif
840df1f3d92e3ccb28b67ef70d612ae6
b7cc3adde81816a7a8c0471b8c4fcb732027111b
46915 F20110115_AAABMM shen_w_Page_44thm.jpg
496b3def5f8618249c2be2925bd77af6
4477acd559f79b07bc616665ad79f757234d9ed8
1627 F20110115_AAABHP shen_w_Page_39.txt
8ed5c0a1a226248a8a6fb3e7c5b3a506
376f08565bdb9ebd5ce0424a8396483c7760df29
218901 F20110115_AAAAVU shen_w_Page_14.jpg
0032d03c4f621e6507a1705b60e48f33
ab58d26949c99f6b049efa10188589302c422b7e
F20110115_AAABCR shen_w_Page_55.tif
c36ac5d16fdb1eb226759c53b639ea09
6b78239021d8c8d0615921760ebbee87b617abbe
45612 F20110115_AAABMN shen_w_Page_45thm.jpg
bbe7385ac0201f0f392390064b3d174b
6645511fe3dc6cb3049ac25a85a803905fd0a3fe
F20110115_AAABCS shen_w_Page_56.tif
00805161abc1208a2e9471b49fd55526
5e6ddbb69bb04b3cf0775bfd97374e3baff9a6ba
203466 F20110115_AAAAVV shen_w_Page_15.jpg
21f5d9defa109cc6a6d385756a926c3e
a9359c30fa6d9d2a01525f9d924b2a975817211a
86277 F20110115_AAABMO shen_w_Page_45.QC.jpg
e9dc16c1a120b3073057b725650b7e7d
4943ab5eaebf3c276bdce36424f2ede5d74d9264
F20110115_AAABHQ shen_w_Page_40.txt
e087023a6bc2fbeb7d75031b1b2c26d7
4af96092a41dca4eb6be99d67b01637bebd77b29
F20110115_AAABCT shen_w_Page_57.tif
3fcb93adb5c3f1296e87a9277fb03c5a
3f8617d376bbc23e5ef26f3e08d920917a6a88cf
175414 F20110115_AAAAVW shen_w_Page_16.jpg
2186e091908b0c0a41f90194b2b312c6
2fe2416a50de298e967301d48060be4a0373a067
24186 F20110115_AAABMP shen_w_Page_46thm.jpg
7d6dcea760be6cfde0d908aa88b69861
71f392fe7c3437e95946e72269cc1509936b828d
2663 F20110115_AAABHR shen_w_Page_42.txt
a0ffb0011761f328f16f2363f9b7e1bd
a100285ea9ab12b671dcaa06757ed19547bab3be
F20110115_AAABCU shen_w_Page_58.tif
e38a4d751edd0de0a9299c75d1701be8
eab77ea2f3bda87caddc98a16ffe13da93fe57c4
226021 F20110115_AAAAVX shen_w_Page_18.jpg
d059e23896747d49ae87f3a0cfc325f9
74d689f2cd9024e667465532dfca9d32003ba8c1
24920 F20110115_AAABMQ shen_w_Page_47thm.jpg
daaa5e3517ac4311aa86157a25519ee5
c49d79914e32a3c630cb20828b9f13c606b54b60
1191 F20110115_AAABHS shen_w_Page_43.txt
960b168342355d80310267828b4a0b52
0b02ef57248e6a8e56f1eb4fa612ff80a200a881
F20110115_AAABCV shen_w_Page_59.tif
44022620a141f68952140d1c8f63a60b
cf50cad2a0790ef6db39e3281fbc20e296e8547c
161660 F20110115_AAAAVY shen_w_Page_19.jpg
7f2c82557afdbfb4739027dc6b065878
4d9d9f69ebeb02676fa8e094468abc61a598de24
74233 F20110115_AAABMR shen_w_Page_47.QC.jpg
6aebe062d4fb0ef888f8a0f14655739c
54df15b591aaf5d2c902cb34cb8454782dc6ee34
1933 F20110115_AAABHT shen_w_Page_44.txt
d2c4de5d3462f7e3840a83632c11e19f
900636cb47a721e027073537f0d784bc3ee7a722
F20110115_AAABCW shen_w_Page_61.tif
8056a59b59cf0dc737c6e7c3905a41e2
6858d85d7b0177b1338165ff9edbe62367b274e1
22765 F20110115_AAAATA shen_w_Page_52thm.jpg
04098d0e2f0372d1bbe98e09bd94cb1b
df4e5a926c401515a55ea8e1f1e528636e5c6e10
208481 F20110115_AAAAVZ shen_w_Page_20.jpg
3317a784427eea6137ac58b116018095
692723086996148f2100084755ccc423780925c8
23707 F20110115_AAABMS shen_w_Page_48thm.jpg
d6614b7c32390c24bbde8d8cd23f9c13
18f41094dbc6b63091bdb1360e18aa2e1ed403aa
1441 F20110115_AAABHU shen_w_Page_45.txt
838cebe8360a6a9f34fabcfafe06c90c
8db695d6e8f7bf6aeccf10980314f5d31c8ecb4e
F20110115_AAABCX shen_w_Page_62.tif
2812bc490de1f52d56e8d92738956209
4ebb611f47dee86ac4da5bb5ff12e14a88eb4362
176827 F20110115_AAAATB shen_w_Page_35.jpg
a20b14379e07fa5c0220b956978d6eda
b77ebe6f0cb3f7c1174875d41c2248671ded7c37
70225 F20110115_AAABMT shen_w_Page_48.QC.jpg
3cbd8ed825a0439b24f4c64b17ccadb0
c9f6a411cb5cec4917a35d12df4de81ff46ee6be
1989 F20110115_AAABHV shen_w_Page_46.txt
1497dbaf9d4f1f7ac630d546fed02921
a980bc74785f874c05faccbbd1f5218c13e7d8dd
100435 F20110115_AAAAYA shen_w_Page_84.jpg
6b17e7bf175c01f4b41088c476930910
73c28f26fd9bcba0f9909ff4e8f1fddff99ba05d
83021 F20110115_AAABAA shen_w_Page_63.jp2
3103c33d8bf9c99590ba65be30ec775d
4919acb927556fc74ba94815c5828db1392f9f51
F20110115_AAABCY shen_w_Page_63.tif
e2d55da02ec47ffe44de21df18a31f93
3f6104417e1dda24145cb2a756f0d7d49e8859af
50110 F20110115_AAAATC shen_w_Page_30.pro
8811df2edc0ee4edc830560bfcb0e896
c8ff558ecd4c8adeee5e9af5072919a15b4cfd8c
14600 F20110115_AAABMU shen_w_Page_49thm.jpg
05b4b1657fd8a660bc6231cdd79834af
5df839be3742cc22f2cd6dce7aada40e4475fb53
1925 F20110115_AAABHW shen_w_Page_47.txt
eabfb4969ef5c530a8a357a3ed241015
3af1183e1ff2ea5a0895eded9e8bec8b1e84eb9d
23072 F20110115_AAAAYB shen_w_Page_01.jp2
62d05f1896aa4ea197e527bcd918b5d4
ffaf6969124a53032b65dee0ae3d1a77aa7c78d3
110575 F20110115_AAABAB shen_w_Page_64.jp2
0ca3ffca0ed343a22d18ecd1859af509
d7b43867518d28d8103548082b1b3616246e6f03
F20110115_AAABCZ shen_w_Page_65.tif
17c246a91972bd8a46b91ad8fccc097d
b5fa23143be1554877285ce97acfec9688799d62
37582 F20110115_AAAATD shen_w_Page_63.pro
3cecdf2e32ab897afe540e08297c1bc2
7d83032cd5d29b4b72c419f1186bb3f467e852bd
2058 F20110115_AAABHX shen_w_Page_48.txt
9802dfa31b5618cfbbe6b3603c475645
6fdc99b8a23508946b2ad9f0181b91b5e40b1529
F20110115_AAABAC shen_w_Page_65.jp2
ccf1c059ecf6d5705edd34f6f5515d42
04b1e53a9cd459955af7b6f8cceed271ad8c959b
F20110115_AAAATE shen_w_Page_27.tif
d902e8243ad08e521f672639345414bd
2423fbb652574817c22abc9fdfc62230059790b9
23220 F20110115_AAABMV shen_w_Page_50thm.jpg
437950ca5cb9c533412fdfec840f737f
dba88f78218c0d0b41bee1e519b16c043378d93d
1576 F20110115_AAABHY shen_w_Page_49.txt
45efcae694a6bbdb8ceacd9aef2b9fc1
31947e46a262d763c813c3580aed5412847b850b
5606 F20110115_AAAAYC shen_w_Page_02.jp2
e60eb0292a4e8d1b56298d8af72b808d
36bdc571a166d4d77bad2795d0937d345d79db3b
87332 F20110115_AAABAD shen_w_Page_66.jp2
ddcec3e62a60fb6d8c7eea14f9aa09a6
e6f327f5c4f2f7ff89b23d54c30ca0707b746a17
F20110115_AAAATF shen_w_Page_25.tif
d52a449a8507464297b7d6241abfe31d
2d06521efea98bd4fb27fa34d8e649c0318897e1
40514 F20110115_AAABFA shen_w_Page_39.pro
65a937293dca3edbb1b7f73f11fa2d9c
73f684ac93620cdfdab12160295b22914b06cdd8
76567 F20110115_AAABMW shen_w_Page_50.QC.jpg
f4b3656849d2b57d31e9b687817f40e0
6b8d4a3f08bf7e77aecadfbf76030683cc2a0dcf
F20110115_AAABHZ shen_w_Page_50.txt
409fb9714270482788b9b4cffbfd8b5d
67b2c4a331892b6920c2737a7e2056156d3560c9
1051970 F20110115_AAAAYD shen_w_Page_04.jp2
9e07fdc353b8097ad5ae7aaa05aa8164
581d9e0ab0d7357a256d7e9bd87c794ea2b07f25
1051981 F20110115_AAABAE shen_w_Page_67.jp2
49f9801a6ace58ab46012bd70c759064
7088ea00bb7d033332f0f83aeef57403c7170b8d
45222 F20110115_AAAATG shen_w_Page_80.pro
2ce46d9ad2d0226cd80831227c71cbd3
0ad89b875fedb389926e3a5f0d194a1b1f33c784
40657 F20110115_AAABFB shen_w_Page_40.pro
5e66dcab53d94ea8335c65fd833caefc
39390a193eb19309b5c0271b6f134f03688cdc2c
50072 F20110115_AAABMX shen_w_Page_51thm.jpg
6beb0c08c15789f3cd207f45dee08c38
016f7a1855a449cd08acebc483c3a1f8a63f293c
F20110115_AAAAYE shen_w_Page_05.jp2
624bee8284ede2ec16bf68af4f5e9458
c1007783f1992e3e0fe50f3ff1d26c0f0f59c1a9
108135 F20110115_AAABAF shen_w_Page_68.jp2
e27cbf0b41a2e77856a57ef447fda19c
0f1dae2c91b9c73311a3743e4325d17b6f03e3df
212126 F20110115_AAAATH shen_w_Page_46.jpg
927b47121020e42f108e7a8a7c86c0d1
6f23d68b01d9019350f4e3e99251a3d5e5b669ca
45278 F20110115_AAABFC shen_w_Page_41.pro
3897a9dbda34a762c2847122cd89446f
b47be5870b22b7037b7f9c82b721ff9448184b5a
100364 F20110115_AAABMY shen_w_Page_51.QC.jpg
70bd57f857da8182f2f5343624f81408
57d68105f4fcb9f3d711de455f77766e7748d15e
5851 F20110115_AAABKA shen_w_Page_02.QC.jpg
da5d028a0548cfe4bde0c582d8bd37cc
bf7bd1f2b6d803769b6ca2ce24b88ead59016f0d
1051979 F20110115_AAAAYF shen_w_Page_06.jp2
219bbff5511ce988c662974a112927ad
e302dc4348192254706929a5c6a9a24a2d4a2360
1051965 F20110115_AAABAG shen_w_Page_69.jp2
87b3898e90b1f72f48b44e54f3f12622
8e06973158eb461faf678a6880e62cd174a46b1e
105597 F20110115_AAAATI shen_w_Page_81.jpg
bc83074170f1bcc67343a9da073c8609
dd4b433582918620d1d3b6031211f3db02e0eacb
29615 F20110115_AAABFD shen_w_Page_43.pro
bdcb5134590924553df7f4aed7011802
0abbea1dcaa4e30ea6a533ff617f6ee620e37017
75425 F20110115_AAABMZ shen_w_Page_52.QC.jpg
8035855179c8963e80e7a79ddd23e190
aa7c5bfdb6cccbcf32f9dc5810d449f213e9fcdc
3411 F20110115_AAABKB shen_w_Page_03thm.jpg
2cce10469f12eb669e1db65ae587ca85
e7068a08eeef6d25832f514e4a97a118e2059e2a
153926 F20110115_AAAAYG shen_w_Page_07.jp2
d443c02c48f2dcc48327e006e1a2775c
0fe656486fd59326069a964afb53b314a12763db
1034613 F20110115_AAABAH shen_w_Page_70.jp2
b7503a38c296608a9f8cb41e3644254a
dbbcfa589ce7108cc76be9b3e58584a0b401d59d
94765 F20110115_AAAATJ shen_w_Page_22.jp2
3b54ec770b9c3b5818b8f9bf92143972
5dc16e2d6800d1e75a180663b366e9f8633013fd
45614 F20110115_AAABFE shen_w_Page_44.pro
d210391d40f876ececad28b8c7f1f9d2
d734c2aa74739394aa0c3f1a6b5014fa8ffa160e
6221 F20110115_AAABKC shen_w_Page_03.QC.jpg
9bc16b1d3fc7c935f1bf80d77920104d
8ebde9c58c7d567fd4f7996a3b07a539444b5a83
561506 F20110115_AAAAYH shen_w_Page_08.jp2
a836b0512f71809f64ced7eadd00f4d1
f1b1684ae2a4526d482a5ac3e0f84e69234f44db
1051985 F20110115_AAABAI shen_w_Page_71.jp2
6c2fb03e8e553a67c8d59c9bc9bc864b
63b8871a185a819316140dda5bf6569e1f2c699a
53760 F20110115_AAAATK shen_w_Page_07.jpg
b0d3065b443bc40d5a271f13bafc67c9
27c2855e6ff13ddb73edcb5c5c09b0ad25fb96f2
34680 F20110115_AAABFF shen_w_Page_45.pro
f69c39d4afc8a0024e3fec12664fa960
57a376b0de1a57deee49782e580e68b761921de0
45513 F20110115_AAABKD shen_w_Page_04thm.jpg
45ea79e6a5b31a6c1f6044dc1f40866b
086314a40e6ff42936a5ff5ba33c488a8e781a43
214119 F20110115_AAAAYI shen_w_Page_10.jp2
aa0f5f5d7636d5e8fcf6b9e094bed316
a3624d06a6f3846b053029162befe44152d27a66
47557 F20110115_AAAATL shen_w_Page_59.pro
16e79b7deba76cf456a114ac54315ef2
cf428c6cbd19d77548346c6f65aee72f825e53f6
50516 F20110115_AAABFG shen_w_Page_46.pro
11fb8e3503a26b7e3b07eeb2d0475e91
906ec0e975cba6ce5830260391017acc568fd04f
95686 F20110115_AAABKE shen_w_Page_04.QC.jpg
593d399c7ae1f502f2f71b30b359dcd4
4284016233904b9bbd383dfa315d7cf435905dd6
82063 F20110115_AAAAYJ shen_w_Page_11.jp2
d6e69480a99ac740213310b2dbf8ad09
511388334b7ec9ddff1f0eebd8d0058b208c9f34
1051964 F20110115_AAABAJ shen_w_Page_72.jp2
87511fe9eb3c9bba6cb7546c2993f5ec
b6df5dd0955e93a3eb90ba38254d5946585d55aa
131039 F20110115_AAAATM shen_w_Page_42.jp2
19e2b93b778399fdd46c071de23019d4
6f8e96ddc5a19d9d49936576cf79a90ce813b476
48510 F20110115_AAABFH shen_w_Page_47.pro
d6584e21fa1e6d3345a999417fd5384a
97c5d2a92eb6535186966b9b53ad82b64bc5b25a
50245 F20110115_AAABKF shen_w_Page_05thm.jpg
3d44e364ce282877b95cfe22f339bb02
01c2a6363d56d7523cf37f3b4d8bc9085a55bebd
34252 F20110115_AAAAYK shen_w_Page_12.jp2
b8e69b9da38044b309481b20e00ad008
fe3704f0b47831e0e2c31d1b6299d7846d27df80
98901 F20110115_AAABAK shen_w_Page_73.jp2
62d438e201a83cfc0fa99870ddfe5f79
db9422c215628de92cecb1214c42f495906b86ce
980921 F20110115_AAAATN shen_w_Page_44.jp2
3cdf92a6ddb63a29e32ad576be44d7e1
39477ffa9a6c243a70fbae3b67cc1552231c1530
46103 F20110115_AAABFI shen_w_Page_48.pro
20733f8c174047f8fad762555ee72e37
af4b2247e93ef15940f606e752c2673beb8ac272
115187 F20110115_AAABKG shen_w_Page_05.QC.jpg
7d0afb4b31f21f83e18bef8a4d559788
51bede24789b10fd762cffb49dc7218fa32cf6d3
932078 F20110115_AAAAYL shen_w_Page_13.jp2
990b3f40a2b0e8bb884c4cb4becb1794
98d45451285adb109d1fbe0af42b34c6c9a4220e
1051973 F20110115_AAABAL shen_w_Page_74.jp2
eb99a31ada5f75332aee70c6dbd00f0a
84f5b8a6ea4bc50e5f8fcc0b81b13a9ff006eeb7
864 F20110115_AAAATO shen_w_Page_78.txt
cf248e8af40eff4af70a6a6e35258788
2d6e185a765ae6d6e66b717317d36ff1c7e0b729
47054 F20110115_AAABFJ shen_w_Page_51.pro
dcfe57987e5f7aea6e27c1a94ca2d36b
8e3ebd81c8e2436dadebe6a652c1b5a099385753
49116 F20110115_AAABKH shen_w_Page_06thm.jpg
1ea4f793dddba1f13a9f05ae4b07bb37
c90c8946a500e214324631c82e470ee3cbc64c4e
1020385 F20110115_AAAAYM shen_w_Page_14.jp2
8e4edd869d97265ad448e2e9b32ba5dd
a289d6b853d069fb6a459b3a633ce17a8fee6100
501478 F20110115_AAABAM shen_w_Page_75.jp2
594b174a3578a73753ff3db040e62592
77d5aafbbb62347afe1eb92aba557938b27b70db
F20110115_AAAATP shen_w_Page_64.tif
f8d4e4945150b362a142b6fd99f0374f
1a355fc27d28ca16853fbb05edb720305a4b9573
49243 F20110115_AAABFK shen_w_Page_52.pro
f19ce04d4c9a8bd51e4f5be70b9ddaf1
8d058b3a0ca586c016b7c602dda9ffc5d10101c9
106948 F20110115_AAABKI shen_w_Page_06.QC.jpg
0b424fdbc88913d8b8c7f68a059187a0
457ef3092dc5e70e9fb246d0e886290085c33ccf
105871 F20110115_AAAAYN shen_w_Page_15.jp2
35f505540d4c9ed899eb201998ede97c
365a63351ea2ef615e7bd79ddd8ac176dc3cd689
85549 F20110115_AAABAN shen_w_Page_76.jp2
6d07d68ea6ac95c791d964b46a93affc
e4769c0d31f41982a7df98264f5d0de6aab0fa46
F20110115_AAAATQ shen_w_Page_60.tif
600beed81803c1a26db1639172fb4e99
e667d73b1b95a5b19cab9e97f5a8426021772b64
53485 F20110115_AAABFL shen_w_Page_53.pro
235ee14be059f88a61f5faf756c517b9
5adc8413d740622dbe73d6d3832e455591d62b4f
29375 F20110115_AAABKJ shen_w_Page_07thm.jpg
05548b76dba9c06ae2f48c52f299463c
b3bb97a1576f16207fffb2f2862543dddf466767
F20110115_AAAAYO shen_w_Page_17.jp2
6eb5a0f76c29ed28c330058e18bc850f
2302564c80adc6e70c02c9d68c2d5d77e8cf0e17
91554 F20110115_AAABAO shen_w_Page_77.jp2
2e9a0268643624c787a0e901562ec5f3
f485456824663a44cbd90c497028c099c0cc6bfc
46800 F20110115_AAAATR shen_w_Page_71thm.jpg
badd6f97eadbaf18a3a3639719601289
cc76ce8ed09945164c9e52f1bef72db1078eb499
62944 F20110115_AAABFM shen_w_Page_54.pro
2d042981292c9c60d0b37b5d4b1371a9
5ad99dccacbb9220f61263c04ea9a35487fc9717
33092 F20110115_AAABKK shen_w_Page_07.QC.jpg
cceaa6ae4bde85ef27928413f47af936
4a921bd60f4d24ab6efa2be7f83cda6aef07032d
1051967 F20110115_AAAAYP shen_w_Page_18.jp2
a691437064b4718bc2a6a2b468a0f571
60469ce647111cf52548941e54db584135eb851a
39169 F20110115_AAABAP shen_w_Page_78.jp2
a57facc5796fd3422b65c2d065263172
632c70d7fca05013099695a40746dabc2791dc77
196368 F20110115_AAAATS shen_w_Page_34.jpg
891ed80835a93a4e26e2abef4cce59b2
b3541dfb8757530a87c7ac772470f2c8173c4a76
40598 F20110115_AAABFN shen_w_Page_55.pro
97747031827756c013efb574a94a7329
79666872a93cfa6c2a59e85c3e8ba060f79b3acd
53534 F20110115_AAABKL shen_w_Page_08.QC.jpg
996218f3d29fce43284aefd28ba651a1
6021bad771db69fcff92dda59613cfd8ff626cb4
680964 F20110115_AAAAYQ shen_w_Page_19.jp2
041db157c483317b3e78026017135030
8cee0196f9144ba0a0cdde6f82ec8c1b47e5403c
43962 F20110115_AAABAQ shen_w_Page_79.jp2
f0d4b76509941c33bb8dcd2c3524ae27
cddf3568903b2d199609a5dd1cf7613af750d4e0
803 F20110115_AAAATT shen_w_Page_08.txt
029276e5fa29e0fe995cfa3658cc58de
cd4144d57b1c0c20fa388aa303f5854e29f56b0c
46548 F20110115_AAABKM shen_w_Page_09thm.jpg
29520edf2e33f198a244bab34e5ea67e
aae6aa1a5b0902f106b45d874e17811edf7e93dd
97247 F20110115_AAABAR shen_w_Page_80.jp2
08ff3dd648beefd897678b56fbe48115
6c2d121dd8560457d30365db2fa9a1f9c20147d7
1411 F20110115_AAAATU shen_w_Page_21.txt
85f1332b450ea74fb9a50b6247561225
2e2574b1bfcafa7c106d04c3e154361d310db59e
40586 F20110115_AAABFO shen_w_Page_56.pro
671f489546d22363372a0027528fc5b9
7616eae2507000f95ad0b82f54eeb3f0dda89561
107864 F20110115_AAAAYR shen_w_Page_20.jp2
a369e9bf5ff8b72318df32a547729aaa
eed67025987aabf1d536a8caa8035a87b4db098c
100255 F20110115_AAABKN shen_w_Page_09.QC.jpg
b78aa7098cf78d62dbd7585338ae036d
2fd1a145f57bfadf303fc249538ce1136835f3bb
49504 F20110115_AAABAS shen_w_Page_81.jp2
17be1f9a2249bdefd9f2cd5ddcef846e
a2c6d3f5adf3715cb1af8a4285b79dbf495f0e68
60173 F20110115_AAAATV shen_w_Page_75.QC.jpg
c9be86480a6479919bfb10bc2bd70e0f
3130e54cc3aa1425b37509d192f295d3243cd5cb
51225 F20110115_AAABFP shen_w_Page_58.pro
51c4a854b6c1d989b78fee5666ff22c9
3966460acc3590e10e5ab4e4172c60030040f4c4
77698 F20110115_AAAAYS shen_w_Page_21.jp2
fd9113fc1586cebbef1e8c5df919abc8
6f80f7c4a0d48cc0ce2fbd2b0a9ba70fc09bdaf1
30453 F20110115_AAABKO shen_w_Page_10thm.jpg
8f172784e2d0fb6ed64da96a1a47f5cf
97bce044b3d31b583a614428520f247e44d5a09c
109749 F20110115_AAABAT shen_w_Page_82.jp2
0c9053e6923b10a350e94704e70698ab
af278eed0d1e22082ba434c11a43dc1f097bd0a9
446 F20110115_AAAATW shen_w_Page_01.txt
6261d6ebc6ed53d3eada05447a1a7275
10ed5d9249c339d19419f33d34f1323ec55202d0
2777 F20110115_AAABFQ shen_w_Page_60.pro
3bb921f672bc130bdd1d7781c847b150
b1f040e318623b4c933bfca4c813953bf5c7af5f
1051907 F20110115_AAAAYT shen_w_Page_23.jp2
3386cc5d5ebb7822cc7c745cfe8e3dab
d2aaa374061e8eb6216a1baf4dd5bcd0ae9677c0
36452 F20110115_AAABKP shen_w_Page_10.QC.jpg
46aea9b035ef58a2523f8351f1efc9f7
4e202d02386378f72cfbf747a1b41d90d47d3cf2
25643 F20110115_AAABAU shen_w_Page_83.jp2
465e840bfe13c8d80b916572a602ee96
9308f9746676148e82af41831748c61b2a5f560e
94054 F20110115_AAAATX shen_w_Page_44.QC.jpg
4abbbb6922cd46d69571e6ed2e42ff74
1defd8dbc62f4f8bdb657495b55faa522eef8c9c
57098 F20110115_AAABFR shen_w_Page_61.pro
634581fce8880a1d00d308a3e9b5926d
752d7fdffd5a67e5c90b74a3c73272652ffe9100
90207 F20110115_AAAAYU shen_w_Page_24.jp2
b778ddc6ee80ca0530abb0ae68b7e33d
cc7fa39415bcdd62ae2de227f3b7ebb1f96aed3e
57154 F20110115_AAABKQ shen_w_Page_11.QC.jpg
6fdb8e2e59b7b2fdb679b12f35fffe98
95134d7a7fb85e51287be6ff5e80978681c1bc15
F20110115_AAABAV shen_w_Page_01.tif
9b690cba7d5e820e3eb1c3146085b603
04a570a488c645c6f6596e856a726c0689db62b0
5710 F20110115_AAAATY shen_w_Page_83thm.jpg
234167a5e451db83da327d6afbcc624f
036d47faf2dc50fecea5a8fa968919cfb4e1c690
44053 F20110115_AAABFS shen_w_Page_62.pro
887f26fb779bfa3cbb3b8977796d2cf3
bb128f5a3d6698f6d7c0b419dc4920224ddd497c
1020334 F20110115_AAAAYV shen_w_Page_26.jp2
19f17578c1ab247af542345913d11085
852419547ca6ee47648c6ff2c50c921678108d74
26780 F20110115_AAABKR shen_w_Page_12.QC.jpg
981bc03e564153a6488e73d479a5c992
753891d89bf854b0481864b4af66a6d3ee92f663
F20110115_AAABAW shen_w_Page_02.tif
ae96d8fd07372f63d42d2cbc8de20bea
776c17e0b3a805664d83efbe6b62bfde8c4aff34
232941 F20110115_AAAATZ shen_w_Page_17.jpg
9f700561047a4e26ca315115a0a7619d
f1f556f1962306c552ce2a3534a74b41a576ba01
52795 F20110115_AAABFT shen_w_Page_64.pro
e1dc650cddb4910df07ce8c79c2af3d4
4d4c138da29655ec6a02ae32da678b95d71fb381
71297 F20110115_AAAAYW shen_w_Page_27.jp2
a3cf3e3d9504fb793952d981ad80eace
db187672f8b473fea584bc28b7131f07370ca366
46140 F20110115_AAABKS shen_w_Page_13thm.jpg
2c0ff3a9867c91ccdcca6b1f5c728396
c2ffee2ae22cfd4b6e5eb78dcfef078c873a196f
F20110115_AAABAX shen_w_Page_03.tif
4f9e5b8c9696cb0c8cfd34abb302b55e
78b617c734f732cb60985b93921c709ad3ae1122
22699 F20110115_AAABFU shen_w_Page_65.pro
d318cf794ec2e778174f61ed01c8bb3a
10ce558399b7ba2184e63eb96e0faabc41a14b74
84784 F20110115_AAAAYX shen_w_Page_28.jp2
e26435343d6a3232624872770d4f2058
c169de404711fcff3166303c59dae0546a3fba95
41145 F20110115_AAABFV shen_w_Page_66.pro
523f9fa61ba484f6983761e274f06c2b
f49f432cf267579400dc5c8436d59dc4aee8d781
97517 F20110115_AAAAYY shen_w_Page_29.jp2
9fb65b220d28c6f813d56f755a866ed7
63bffc9e152febeb69ef34e9f626f49fbd014b89
F20110115_AAABAY shen_w_Page_04.tif
07c6742aeaea064bf43b85fa61a56a2e
4811994527d67017a92e1f11a66cb8e5cc4e130b
90015 F20110115_AAABKT shen_w_Page_13.QC.jpg
9dff78c77b7de7b4bb098077992acf0f
fc252862f3ab7f2e1b73a5713503b781e5271e7c
20759 F20110115_AAABFW shen_w_Page_67.pro
35c0e6bcb20590e1ed2f54c648c02ea8
059369369cc776c94bda3241681575bb058878ac
149658 F20110115_AAAAWA shen_w_Page_21.jpg
ae02eb958a9c455bdc6b1648a552ca09
854b3dd3dba75f9a0f428ec49343fe84dca95042
105913 F20110115_AAAAYZ shen_w_Page_30.jp2
eda2948f7fcb7938e23867a8280c878a
279d7a6cb8c601492caa26521b042dfd59b36f60
F20110115_AAABAZ shen_w_Page_05.tif
8d34d78f353cb1554a21050a1405ebcc
b8bf506006543732cc07750dd7d61bf8e856f50a
95664 F20110115_AAABKU shen_w_Page_14.QC.jpg
102a6d1474a0c34b872a23c12a2d0e91
33347d3ea8972e74d461d22011ae84927f06b860
53655 F20110115_AAABFX shen_w_Page_68.pro
4aa2fea25d57d41c9772562b197daf21
fc9d76acdccafd4d43478a3f10898140f19d5eec
186859 F20110115_AAAAWB shen_w_Page_22.jpg
97ef657b1362d820b9a1452b5669e1ba
297fbb00dac34fab05331010ecabc43ce81fdca9
21253 F20110115_AAABKV shen_w_Page_15thm.jpg
170295bbefc3ef84b4fcdc558a8f033e
23943816201d72280f150202d4f467fe2644aa93
F20110115_AAABDA shen_w_Page_66.tif
e383d987652c39d419b3df4476f8284b
a5ee2c83fd87711f5f143742eb8328c48995194a
28674 F20110115_AAABFY shen_w_Page_69.pro
8e66d363eec82e0ee983af062c4ea1a4
c071af4eaf35f9e9ca10545d7a8bca523d85c789
238062 F20110115_AAAAWC shen_w_Page_23.jpg
4d912b59a41077cfbb26814eca2647ff
bfdc2f3be6cc38824eda78ef35d145a9289364f7
72418 F20110115_AAABKW shen_w_Page_15.QC.jpg
3e922e4792209ec59f5a4c905eadbf69
6298fc61b1927c5057690cf3b5b257a459092c42
F20110115_AAABDB shen_w_Page_67.tif
a8b3df8ed96c8924c0a7099da3a9cf5a
1b6e0187aba09a2d0089dd92241c7b53af0741b3
39765 F20110115_AAABFZ shen_w_Page_70.pro
9348a55e57717437f7bf3c4877ad24e5
f3f7c64bf1fec635420a06d3662617134d43a725
172513 F20110115_AAAAWD shen_w_Page_24.jpg
e93cc69b0f3a475c0900638fbacd7afe
40a4b321a53bfb6c0765577f6c2743cc50f4a8ff
43297 F20110115_AAABKX shen_w_Page_16thm.jpg
6a0cc33d375c960d5dfefd2cf5b658a0
63a249dc0137d30406e731dc4f47c541d9d65b90
F20110115_AAABDC shen_w_Page_68.tif
32b91fb92dcf243b3206e98bafd0e371
89bb39b3a4915e2ebf4ac8a80403101b6fd6e3a6
215894 F20110115_AAAAWE shen_w_Page_25.jpg
64968b678ade6e0ac5f2e121aea4fe18
65cadc587546d3ef230fdf0959aa71c63b2d5da5
49459 F20110115_AAABKY shen_w_Page_17thm.jpg
dc6744e0d30542ec736d8463fe89913c
dd4ad75f958ee03c63fe5b5aebc6b197c5292ae8
1883 F20110115_AAABIA shen_w_Page_51.txt
c3aac25726cbbfaccd0f4416c122bb10
4d99ae14745707bb01c4ee775f527c4f63e570b0
F20110115_AAABDD shen_w_Page_69.tif
3fff213a695752381f52fc4916b612bd
d5716496351c6befe74c29a3b581a733e38a663b
172929 F20110115_AAAAWF shen_w_Page_26.jpg
3c91e1c127b4b597f6550f03b83c6843
ff999506e4a79445de6bccf1fdfd171443158b75
103138 F20110115_AAABKZ shen_w_Page_17.QC.jpg
a294f32b9581d7658dcc9731f2c5bbff
f404129caf493d17921f089919f3b3deb94e9194
2017 F20110115_AAABIB shen_w_Page_52.txt
a6db4829d224b0e4dbf1242fb96dccb2
1196c7e9dad7eae60588bf352009350338542421
F20110115_AAABDE shen_w_Page_70.tif
5e8927f81b41a501628e0de1003afbea
ef6472a61ac8810f64da08a67278831c2ee50576
140842 F20110115_AAAAWG shen_w_Page_27.jpg
60eaf96741205c2a7ccbc809a6e325eb
78440999292c1a99d7dfe3caf0aaed10842f0dda
2147 F20110115_AAABIC shen_w_Page_53.txt
52892c7edd6e2dad36609dcd12ecbadf
d8e112e2341e432d739d17c0c4bcbd9a91f837d3
F20110115_AAABDF shen_w_Page_71.tif
940fe535ca53b9e4b00a2bc6c583c2f5
e6c3c31242ee96e79a23890051de9516c10afa54
170460 F20110115_AAAAWH shen_w_Page_28.jpg
eb380cae0fe2df8281f3b889f61156b4
dac0cc9b32b9b17c276eba6cbd38448e9c32a0d1
24982 F20110115_AAABNA shen_w_Page_54thm.jpg
c15b9b6bc20c6ed37143a5f92974f031
407c495ec21bdde12ea80da24e2beeb94b18d21b
2579 F20110115_AAABID shen_w_Page_54.txt
4226a00eb535c925fdad15f26828ecf9
bed622571a4ed0831a471737f01c3197eabdf741
F20110115_AAABDG shen_w_Page_72.tif
716bc94d03691fe3c621cffd37cff01e
67452f465f34ba8d64b72a0937c3ce4f99a25f30
194319 F20110115_AAAAWI shen_w_Page_29.jpg
f0c4ad986abb7eedffd5edc1e7345fe4
ff62b37e88e01c5bed357847e9546e9b6f122493
83741 F20110115_AAABNB shen_w_Page_54.QC.jpg
927f652afac751629c28cf8eca222e29
52b348b4e60022f3eb50eee799eef0bf705c8633
1624 F20110115_AAABIE shen_w_Page_56.txt
f6c60f59b1df16d399282cdd88d110b5
34b1ec89030d2ba52dd0ff00cf9bf31587900925
F20110115_AAABDH shen_w_Page_73.tif
5db667eaaca21aad8ae7ea2fda7e8ad6
8f01a2e08ff1f7b01a4f6f882d1408b8542ef0c9
208747 F20110115_AAAAWJ shen_w_Page_30.jpg
31f951eec8c97b273bc01b9d37fbadb3
3352cd85e4d2ec4bf92cc884679cc84556366b88
64771 F20110115_AAABNC shen_w_Page_55.QC.jpg
e78cc12c41e37a2e88a8666c67a4d852
34fe6e1db849fd229edc021372550d340f4a36f4
1254 F20110115_AAABIF shen_w_Page_57.txt
25277ca9ab4a02b8ba505d48135923ab
d10bcea9aedc2939061d0001fdcf07ed7cd059a3
F20110115_AAABDI shen_w_Page_74.tif
e547835c21c8a0baf7caa95591c302a1
b715328aef43344315eb7da2d2d02af9c469db72
196286 F20110115_AAAAWK shen_w_Page_32.jpg
1a9ba35ae9112562560952fae8923a6e
e6e1b204bb3cdfc79e69649bcfec2321ca211d6f
21230 F20110115_AAABND shen_w_Page_56thm.jpg
bfe4d7608e8e4ba46d02652bebd38f92
d1240e005b3dddf75186d6d573eaa9fc54e989bc
1981 F20110115_AAABIG shen_w_Page_59.txt
220c880cdc1bc61b3efe805af58ab22f
fc63d9c28ca1510174bccbcfafc6fa8d27a848d4
F20110115_AAABDJ shen_w_Page_75.tif
0d6a88080629bc84667c3f3f4e9e117d
ce5143392dda57a9e12099995b8e6de1e5f5e7c7
204888 F20110115_AAAAWL shen_w_Page_33.jpg
bd0bea8e85e9ee07a262c18fff3a2be5
d454991286fe88881bf73c4dd5caf6b3431a4b20
61544 F20110115_AAABNE shen_w_Page_56.QC.jpg
cfd67e8660f86d957585e6bdb8b25c59
958dd4ada039625f132501025e8087eb689568ca
153 F20110115_AAABIH shen_w_Page_60.txt
01464f96382a01d9a0318d4e3aa36e42
062521e1da520ac8ad61d199e5761043b60e2f77
F20110115_AAABDK shen_w_Page_76.tif
000a6a0a0c0da2426e59c038e9e5ccd2
a87a538941a41d62d21560a031f0261ff58a0daf
197766 F20110115_AAAAWM shen_w_Page_37.jpg
a0cf1abbd01f4ddab411f92f08adbea4
d6fa73894f83079c8178835b09a9ebc47bd67bfe
17925 F20110115_AAABNF shen_w_Page_57thm.jpg
53661c008aa1d841e24e19d7fe94394d
cc383626fe11f586d93026aef8bab409731277b2
2403 F20110115_AAABII shen_w_Page_61.txt
a00c3dfa9a0f2dce2620b22e728821eb
5c7d0f1ff4e02a481a890cd4afc64ef940c7e9e4
F20110115_AAABDL shen_w_Page_77.tif
abd0fe434752a1b08cf1f4b9e794adc5
540dadaabde7b57efb0f0ead10ee6b8fea20b7ed
184495 F20110115_AAAAWN shen_w_Page_38.jpg
e1a81b7038ceb8de5d096d2b19bd6bbb
545b56e6a7b2e0bf96335a2ccd3df6edfa0a7e23
57619 F20110115_AAABNG shen_w_Page_57.QC.jpg
7b173d06383b79e0928d131e50319a26
a96f8ae3b3a4fe2de5b2578d40c5b17d51aa6b77
1811 F20110115_AAABIJ shen_w_Page_62.txt
6e061343c2adf35e8ac746043eeac626
7c34a337342b71c524aed521853df711ded4b584
177686 F20110115_AAAAWO shen_w_Page_39.jpg
6408253ae0061a7624bf95768af59d36
0b3de8def7d1a29e6e87d100fd85792c5a2bca33
23271 F20110115_AAABNH shen_w_Page_58thm.jpg
fdd8cf0419a6ab757daa0a0ca0711277
40e0d7216629add51048dadc3225bd73ed94ff0d
1595 F20110115_AAABIK shen_w_Page_63.txt
982eaaae1822394f474f1a3644f99d9e
9d02eb8b1907196dcb654462e627341c5882ed4c
F20110115_AAABDM shen_w_Page_79.tif
988ac6ad27c4a90125e3f1aacf5762e7
ac1fe26d3fa98f8fbc2035a5f7af31301e89d31f
168310 F20110115_AAAAWP shen_w_Page_40.jpg
79b9e3d2988ca44b59ef855dc30cd97f
6f429bc02c47205b0d82c78c666a95b9de92b69f
79262 F20110115_AAABNI shen_w_Page_58.QC.jpg
4a2ee415e7bd0462d3133fc78b73eb2d
a46d3b7c6506981918bde81a561b64681635b87a
2301 F20110115_AAABIL shen_w_Page_64.txt
2f66b4151a4983f06beb902edcb4dac0
2e4e629be2b268523a5a981c7266192ee494f7f0
F20110115_AAABDN shen_w_Page_80.tif
bb8f25afed191036e44628943685df72
56d31e034e4529809cb02083151afada18d61848
253375 F20110115_AAAAWQ shen_w_Page_42.jpg
c3cf5fe862f57f6cc35d474c95ec1a6d
5c97385b57db043f84e754283e79ec31f4e7f059
3910 F20110115_AAABNJ shen_w_Page_60thm.jpg
550af02d55d32eec4c6290f2c6924f8b
20c6560fb9aa1e01ce60c4637ceb6fb849c82626
1052 F20110115_AAABIM shen_w_Page_65.txt
45513ce65a1d04a9acd16aed7595371e
c5ea4a5c8d3944f7732675c24b46b07dc27328a4
F20110115_AAABDO shen_w_Page_81.tif
fd4cb550690aebbab7b537e3f9a3a0ab
0424265a08b5f11323be09427fc6138a5502a922
126824 F20110115_AAAAWR shen_w_Page_43.jpg
1a997da29ac32e7cd3d3b76d10bfc677
1659e91ee5498efc1a59706b16fbc7300a0f82d7
8554 F20110115_AAABNK shen_w_Page_60.QC.jpg
8a15994aa536f89e744bc4d90daf9516
1e494c7d4f11e9f337c0ee654360caf70ab851a1
1810 F20110115_AAABIN shen_w_Page_66.txt
7a28bfb70f0aead8ffe1e2e6808ce45e
48668d77fa768d136a31ab2c1ffefd15dde09d20
F20110115_AAABDP shen_w_Page_82.tif
bb75dd90093bc9eed0650fafefb54bab
43480cb8132ddeb31e9792772cb09839356f65bb
212625 F20110115_AAAAWS shen_w_Page_44.jpg
41b228c88de30a7c6b602e43bdab312c
733386064297762c09653a88e64d4839cbbae9c6
21743 F20110115_AAABNL shen_w_Page_61thm.jpg
46cde6f5a24b4933d379c596f0e11302
6abbdf3724f8819ece11165478e502d7ecbeb7ae
1053 F20110115_AAABIO shen_w_Page_67.txt
4733bddba479e76f7a71a63f5e70b59c
4a04175977dacef8ef738e8d2562467a52a2f0d0
F20110115_AAABDQ shen_w_Page_83.tif
30c088640b115e842c084b608cd01b70
36e771ba4ff6169569468625f25e20cac53638ed
204329 F20110115_AAAAWT shen_w_Page_45.jpg
c968d6b9a57bce91154dc19c66d5e808
71853d3228d298ec9ab36cd2cda6294357c2efc5
64630 F20110115_AAAARW shen_w_Page_77.QC.jpg
563b1090a4dfc2aa5688d6c16f32dce4
748a10beb75a9e89725304695e308903c9bb01b4
19441 F20110115_AAABNM shen_w_Page_63thm.jpg
4c2d784776385c5a88c8a22fb516c00b
fb1a3365165adc12dff4b6cd4ecaf4af28843b68
2360 F20110115_AAABIP shen_w_Page_68.txt
da7be55f516e481555f96fcdb677f4fb
e6fca0a8893597e2368ce08df99be8e0fd5eb5b0
F20110115_AAABDR shen_w_Page_84.tif
8bcf13260d4d87ff24f8945731b993b2
c12437f9c539c14f9309b4b5480b9e282ee64d02
201734 F20110115_AAAAWU shen_w_Page_47.jpg
a6b9b8d1d2c53ab1c0a6725a713f701b
66ac592b57b35cfd3e1787c53c737ab6c9c3ac7a
30565 F20110115_AAAARX shen_w_Page_57.pro
2d953c08f2acd8e5c1d956cfb4f8bb9e
36489253ea7acda3e7e5040d0cacdce21f70c3b2
55957 F20110115_AAABNN shen_w_Page_63.QC.jpg
58b33ad52dbcc9ba472b21ced5b034b1
bbb644e44b11da5af4fc19cc3a417ea0cb0c7978
1486 F20110115_AAABIQ shen_w_Page_69.txt
ee31cc65e7a190dab70a2c74b59d0969
dc945b63ec13f5904ad8bca22744168be830d3a8
7575 F20110115_AAABDS shen_w_Page_01.pro
01f79cf05506b6ea3cd0b0e52b5a8035
891dd9d1227d4ea39b31749acd03e9efac4c0bf4
185723 F20110115_AAAAWV shen_w_Page_48.jpg
5a6833f7963a7905d758239a12257c77
d7b14157efd659ac7aed955a0850b37b4f2e0048
8709 F20110115_AAAARY shen_w_Page_12thm.jpg
2e9c2586a7de6c03e7dbfea749a80e9c
52277ef53ed573ca979ee55b35ae03e721a28202
24171 F20110115_AAABNO shen_w_Page_64thm.jpg
7bc958b2b2f8fbce0ecfadd694c101ed
74a97ef52b46635968c45e1b4846ceaf1dfb1263
1146 F20110115_AAABDT shen_w_Page_02.pro
d5a7d1aa6b85f65d2c55ddc75ae71d4f
e7c33ab1455e69ccac212e8553e0e686a4830fe2
109964 F20110115_AAAAWW shen_w_Page_49.jpg
5fa6f9feeab7e33ba9195dcf137917ba
cf59a2ba19017c189fc89f1f330dcd99f42926a3
53887 F20110115_AAAARZ shen_w_Page_49.jp2
10816f30c4a73035697191eb1babee0b
12b8843b738317f1ca2b294c698dd408ad44ceae
78808 F20110115_AAABNP shen_w_Page_64.QC.jpg
d7ee9b86c20a13893230a3abbaf6a477
d386181abca47c400008fdc15b65feb13f9100a6
2070 F20110115_AAABIR shen_w_Page_70.txt
66785dbc18b7b50aef92158778401f2b
6ff0578fc5d4e2b1425434f09b829f0ded8edb4c
206863 F20110115_AAAAWX shen_w_Page_50.jpg
8d0f8debea62ba3ca7d736a69408f165
c5c2cec250cb6e6ca2c3d9146f1b3c51ddb39acd
2130 F20110115_AAABDU shen_w_Page_03.pro
1b6737f1a3c3367b2969c83973aa3b48
d536589ab3665fe69b0a3ae089a8b4b78c123e4a
47192 F20110115_AAABNQ shen_w_Page_65thm.jpg
8f376bcf435784abce68827d34e23a84
8c02e7fab335199993693604337bfd256e944801
1878 F20110115_AAABIS shen_w_Page_71.txt
44104fbc0f3bea140bdb6350eaa7656d
ba278472758db3d078084a9c00f652e8f29a1a8a
222847 F20110115_AAAAWY shen_w_Page_51.jpg
bc0a85a6afc99d6713d441a34009bffd
25d9298026b7d0bbac854faf78245325da468b73
73834 F20110115_AAABDV shen_w_Page_04.pro
8ab63bc5285955d1ce4c1357eb1e8834
fbb696fa9c570b43ab71d9a4d0894f3520e15a0a
91583 F20110115_AAABNR shen_w_Page_65.QC.jpg
ac6a667c5dcf6e64b693a09653dd2228
7744fa96f0268ee7c3eaf2ca310a2f155392d4fb
2668 F20110115_AAABIT shen_w_Page_72.txt
5fcea36c5928b0b8c19dbd7e961701ae
f488a04dbf6b0f0f6f972b208503ce6d76444490
210667 F20110115_AAAAWZ shen_w_Page_52.jpg
043d4302457691b9f705c71eaec45ae1
8d64e5ea0a6fe4164aaa4c787ef3e280d5b93df0
105394 F20110115_AAABDW shen_w_Page_05.pro
ffd7dc9fffb06b9551a3df0c7d047391
c227104a9c4613bbf8712f7e399b76201a734b88
F20110115_AAAAUA shen_w_Page_41.tif
655050da159334a44ebeb94315b87d98
dbeaedce7619d2bf8418428bc04378e035bb00e4
19173 F20110115_AAABNS shen_w_Page_66thm.jpg
8e1fc9789c9673e767eccbd5d3fdb055
15354d7053e294db4b03972a13a4c15eaab88f15
1792 F20110115_AAABIU shen_w_Page_73.txt
2378672dd1fcd9b4bfcf488ef1fe724b
54c4a88a30b3ef0f6d5f094f1c400ebec2c0f7fa
104502 F20110115_AAABDX shen_w_Page_06.pro
0eee9c90b1ad1661c68274debe03ffda
66a24d76c3cd45d22f041fa4e2cc230ab789ad2b
53398 F20110115_AAAAUB shen_w_Page_27.QC.jpg
3c276a24afe74bde9ae07f5277670737
eb307c741e36cc8086f6f7196c1b32fa379b2c92
84824 F20110115_AAABNT shen_w_Page_67.QC.jpg
ca49117720027b6204c2284f1d474f68
fe8343f68296537857d4a9955eabfe3129fca54d
1027 F20110115_AAABIV shen_w_Page_74.txt
4965c2db8e410dd7c78a3ce2dd965aad
09932170befc2f1b7965f5aed6922d958a68fca5
1051971 F20110115_AAAAZA shen_w_Page_31.jp2
ec10cd5d9d5399e194649ea6b0f84cc2
422b3f7c8f62d14e82213f3ea74438eb9db25332
F20110115_AAABBA shen_w_Page_06.tif
15f5d3c5143b3f1f1a3955d46033475d
f2904861647207977555e77467d023fd81e9a15e
6869 F20110115_AAABDY shen_w_Page_07.pro
8aade4e2baefb42c5efbc0238f46d134
fe02d412745114b59d984b51655dd6ff48634925
21145 F20110115_AAAAUC shen_w_Page_55thm.jpg
5cb14252126ca68f4d17011612992f15
bac817678b327b961cf5d850ffcbe7c48434c7dd
23397 F20110115_AAABNU shen_w_Page_68thm.jpg
c7bb27345e5dae94181e395f29637ed7
f6daf849fc4fe643c17ccb3659f6d520cec04007
647 F20110115_AAABIW shen_w_Page_75.txt
4b619621524637449cac0ba4df4779f7
0172ec7e4fa99edb944919ff9db0e94f295245a9
97642 F20110115_AAAAZB shen_w_Page_32.jp2
bc46ded2c40272e15ca8464ef49e6d81
c4c9388ff885a3c23874fb39f6b9b7a76ea27118
F20110115_AAABBB shen_w_Page_07.tif
987f66d18e9977e3f00d3f9eec3af747
bc7bbc36e014f9090b0e39398ea80edfc4bb1946
18920 F20110115_AAABDZ shen_w_Page_08.pro
251e67f052ac40cc0027ba5b5ff9115d
77b19d6389c3b49575af9576969b19785edf3444
47902 F20110115_AAAAUD shen_w_Page_50.pro
34263d3a4d8e364d69bf954e174b6d34
a759f0451747e17a18f179a298c0d44089738045
72630 F20110115_AAABNV shen_w_Page_68.QC.jpg
af379e4bdaeb1907c788f428220e17e5
bf745da8e26614b991e0f6873eb7213d0373c0b4
1637 F20110115_AAABIX shen_w_Page_76.txt
d4262ded53a307e5f413e2689c806da5
fc76aded3bc4a310fcc54ca10f5c14d3e59e296c
105021 F20110115_AAAAZC shen_w_Page_33.jp2
fc95c455baf547890f9d3b1d2be72028
0253ed8695f7d5e69122cd9860bc0c91f9617350
F20110115_AAABBC shen_w_Page_08.tif
893250c4de276463da435029b1e0adf2
9ef5c70e507a1e7ee6f9a726585e0255a04bd15f
45298 F20110115_AAAAUE shen_w_Page_37thm.jpg
59c110c79d6580d4069c73615c023295
e7dca27030f136cdb64bb6a60ea168581ed7c749
848 F20110115_AAABIY shen_w_Page_79.txt
8c36cf58c946ceba9584549fd3816e41
b88b0054f4763153f99996d6389e321b6483d513
34271 F20110115_AAABGA shen_w_Page_71.pro
409b05cf960d81d1f76db6b3e0755ff6
32ee463bc8f966df97cd12c22c290d30f70d1cce
F20110115_AAABBD shen_w_Page_09.tif
e489925f6c70090ffe30a7daff21d3b2
9e7574c00dcff00ea11e403fba9c83b2cfe312c8
205694 F20110115_AAAAUF shen_w_Page_31.jpg
82dba35d347893868c1e1a78cda5860b
16133a8b4fd18bb13e934a2feac01eaac16febbd
45871 F20110115_AAABNW shen_w_Page_69thm.jpg
e66d09e64520170aa7dcb69d7edab87d
3ceca617e13b58db14573e855174ee9192713db6
1923 F20110115_AAABIZ shen_w_Page_80.txt
4b820ecd4adbb611c1cdf567cbcef1c6
9c1fd902a45468314b9f9c883b312e49892121f1
56352 F20110115_AAABGB shen_w_Page_72.pro
40ccf794305bab876fab92da39cd7d84
fd42be3d0a518f96fcb32c48345b1866f4ae5371
99292 F20110115_AAAAZD shen_w_Page_34.jp2
97c3756142d43e0f847975b68d2eb3a9
7af2253105bdb16e2cfc355b9b5c3582b8e1d010
F20110115_AAABBE shen_w_Page_10.tif
10aef18c5067a955124ba773d5ee28ee
0ee7492963fac31e93e3d5233c720b7e71de04f0
48840 F20110115_AAAAUG shen_w_Page_23thm.jpg
4ddfc67251c473eeb7cfbcfdf5a6774c
2894b02b53fd0ed80e680e34185448119b7ffe5a
88346 F20110115_AAABNX shen_w_Page_69.QC.jpg
17997b59b0b5d876c7e11bb678158279
6cc7b0c1e5c5c77d79eb775b06a321cb5d44b42f
44986 F20110115_AAABGC shen_w_Page_73.pro
113fb434fe7dd924f9b78d2e0d2c72ed
b263c533e317330b1c049c6ff9a04a5a2985dfc2
90854 F20110115_AAAAZE shen_w_Page_35.jp2
1b51e3a7e88d2731beba20a78985ba4c
17925537523e0e1b1033acbcdc0b0cfab4b1939d
F20110115_AAABBF shen_w_Page_11.tif
711d9564f26ab31cb0611b3da6fdc8ba
b2b35f223422694a28ce3d3115441745d078a0eb
80467 F20110115_AAAAUH shen_w_Page_46.QC.jpg
3d0b11bff8403607e1566a233b919336
e38d6c36ff8786e7325dbae4457521d25d956d9d
46623 F20110115_AAABNY shen_w_Page_70thm.jpg
4cc612244dcbd0e87cb46a86ffe1edcb
4addb9aaacb6425b91b6208abc58abdd22bdfa56
49100 F20110115_AAABLA shen_w_Page_18thm.jpg
ce1f2e0d17e26201ea2b30a06dc30e36
3e40b6b5289134039e00792e240d525befaae5bd
19777 F20110115_AAABGD shen_w_Page_74.pro
393a587364e345a7db588aee13e2cbc9
e5f4798423c133af87ebfe52afdde29a7f128780
96879 F20110115_AAAAZF shen_w_Page_36.jp2
412d43f2fdc08cd21f7fd5b521608a9b
9b8ba82000a231034ccfbfd7cd71fffa3189a5ed
F20110115_AAABBG shen_w_Page_13.tif
0981f9bd7a0c1c7bd69c10e679548edf
025c114d4d160a97695a4ce1a9d2e09c75c26819
F20110115_AAAAUI shen_w_Page_12.tif
9aac08deccac7aef2b401abd634c2e63
c17fa5a345e9179233377a0c5f43aa1e85377d7b
91777 F20110115_AAABNZ shen_w_Page_70.QC.jpg
52be59cc8df279bc61a35d3a0b7df3bd
57b7895ac9b20fdee3fef0820aba86dcdda19f5a
100157 F20110115_AAABLB shen_w_Page_18.QC.jpg
7fe90505d5f9a7e957552dc164a38311
9e27d694f47e00b4fd4e028af53dc7fc0f3bfcdf
12173 F20110115_AAABGE shen_w_Page_75.pro
8e7c7fd9722d83a179d4586459bd714d
a852cd5ca55c4c6b78cb0d3abbba44a969c85ae1
1023577 F20110115_AAAAZG shen_w_Page_37.jp2
42bb31e965635f0c1407c7b535652544
feb460990a3d95401b4f37b078b4d0759a97a0f7
F20110115_AAABBH shen_w_Page_14.tif
b3ebb98c6f0d02494ec9517537fb5df1
a0a02b1420cc996c70ad4554e24aa4da0f442248
61462 F20110115_AAAAUJ shen_w_Page_66.QC.jpg
9f6afcb4c1efa2be0ba91a1c4a2500f7
e74ebfd2140ab6bee6e364fe68c361d91265f588
42613 F20110115_AAABLC shen_w_Page_19thm.jpg
03eae63829eaa2a32129b45247075a89
6435333e857f5373b47159cba0e8f6b30f2e40a5
41704 F20110115_AAABGF shen_w_Page_77.pro
0a6171b30461549c6ef6ff5311c50c6a
73d7a7c21382fb6c49937ed63f77b9ea3371e11d
89871 F20110115_AAAAZH shen_w_Page_39.jp2
9cb961961dad060b42437556d2ff5547
c162f0093016ef318414711c1282837c7aef49c2
F20110115_AAABBI shen_w_Page_15.tif
47b963c748f87bb93af9af2ef3c7f624
8843da82361134b48fde7e38f2ce98751be096f2
1795 F20110115_AAAAUK shen_w_Page_41.txt
053659dd34eae607081a93784bc1c2fc
99ecbf18be44b1fe4810096b6b253078ec72040f
76993 F20110115_AAABLD shen_w_Page_19.QC.jpg
0e218f3b843893a3c974f811987e4d71
59a1dc4b8f7a68ff32794c23ac90057bd4f38fbf
18143 F20110115_AAABGG shen_w_Page_78.pro
f09e2d2d5d30d76cdbfd30662cb2dc7f
e4fb790015022ba84be0a60ee508458c47d3c057
97896 F20110115_AAAAZI shen_w_Page_41.jp2
22a509fa1ead67d0a3d28e93b622f33e
2f2c28bd6df558ebccc5ea1829fc68e1b85ad357
F20110115_AAABBJ shen_w_Page_16.tif
5469b5c7b2e883bdc6c65368af1a2ca2
b4aaa65317d7f08a0dbc43dc05784e201f848e81
23780 F20110115_AAAAUL shen_w_Page_33thm.jpg
e2219e02c0345ba7f7a9f6ed27139952
f278302e03ea39ae4d3493d1e068fc1f9b69dbd4
24068 F20110115_AAABLE shen_w_Page_20thm.jpg
5fcbdbc19cc405b0761e242c7651e866
efefe923de279c0f1e2670569c0988f7ddb8f523
19648 F20110115_AAABGH shen_w_Page_79.pro
5e4ed3d58168da2172168beb9f979546
e75ffdedf863ac2d5deaea7eb19321783576fd79
66019 F20110115_AAAAZJ shen_w_Page_43.jp2
f466d53cd50e10356c9bb682308f9933
98e5871589fac977422fca9583b6c3172408fbd1
1700 F20110115_AAAAUM shen_w_Page_77.txt
7bd6abab0b816af012041fde364ce48b
2b243f80c9bdcf635dfdbc2d52fd05f0636d9e37
78313 F20110115_AAABLF shen_w_Page_20.QC.jpg
9a3569b58d8db6558f3294285cb377b7
ca9bc7fc6ebe5b81ef0cdb8c84893b47dd14d7fc
21596 F20110115_AAABGI shen_w_Page_81.pro
e7d750866982caced4f59fadb9784560
b1927447c6a76b54c8b6b7101b5e6d60e9fbfff4
1016550 F20110115_AAAAZK shen_w_Page_45.jp2
722331fc525dc89b31031ac6eb9a6d0e
03f52435aa294bfd99c3dd9431edca93679482e2
F20110115_AAABBK shen_w_Page_17.tif
b594d81e6720d305e0087b9315f5f6e4
b1553374982db6905df66b5037f8dfe391912145
7825 F20110115_AAAAUN shen_w_Page_03.jp2
308d2868faaf99e3c4809891feae98f2
bea7b9c1d1b5b5a282c158081a36bb7677a7e93f
18973 F20110115_AAABLG shen_w_Page_21thm.jpg
1a4ded53cf4ae35e07e98c6987e1768b
70595410835f312451e2fd63053af119edaa7d82
10225 F20110115_AAABGJ shen_w_Page_83.pro
09df0156a7990d94cf355edd741db7a1
9957cfa51b697d485f31900518eea5c4fa148a3d
107400 F20110115_AAAAZL shen_w_Page_46.jp2
39c7c227ae57547401ede352ee8f9a56
00836683618d9daeb08cf96281002dc1a9013762
F20110115_AAABBL shen_w_Page_18.tif
43ee153db3d73414b110ed0900e79603
3973ddaa6c4439d7daacb385247de3d43d9665d8
51226 F20110115_AAAAUO shen_w_Page_84.jp2
f78e1aec8bc6b16257643ecb65b68f61
a21e62d1c5d0d8f3380ddeee0ff355f2619cb5b2
55936 F20110115_AAABLH shen_w_Page_21.QC.jpg
e56daefd091598d91be0fde2cc12fce2
52969bb517809e80baa43b29fa1f2b30a27f4f3a
21807 F20110115_AAABGK shen_w_Page_84.pro
7a768d985204f611cb46868312249076
10f14fe468be83aaab854739d755f7a4165433cd
101697 F20110115_AAAAZM shen_w_Page_47.jp2
207d69cb83161fd0fe8151f38ebe0ec6
9ba390b73ed4ae65b7231be0b69239be1fe9147c
F20110115_AAABBM shen_w_Page_19.tif
4db314e8caee8ce3c1a1faf17ee5d0e6
e593883cf9810e06de933567ba42b53d275fa966
1648 F20110115_AAAAUP shen_w_Page_55.txt
bd71a464f20722c6a13a09fdc9444ea9
674ef2bca9166eb2d116bc58562f705e615f6ed6
98364 F20110115_AAABLI shen_w_Page_23.QC.jpg
a5cedcea05fba6c507046f962f9b7a1a
40a38fd06071a7d40cc790079c1a42feb716e783
109 F20110115_AAABGL shen_w_Page_02.txt
7d6fc87202ee548baa22de8fed6a2b72
933b0c866a0d592c24e79f0ca07ecbc522bfc101
95282 F20110115_AAAAZN shen_w_Page_48.jp2
10ef6a85cbc8574f251f612cce33bb76
9a43188989d7448140e7458b60488c9c0323858b
F20110115_AAABBN shen_w_Page_20.tif
0ce077dadefbfa7ae3b8d74bf8ab1321
baa9f4245de53fd0ae116d858cfa269c49f78514
118617 F20110115_AAAAUQ shen_w_Page_75.jpg
b5273814e162ecfe8d9bd9292d77a738
476e95a5b552b77ed36581949862bf6e05899ce5
20964 F20110115_AAABLJ shen_w_Page_24thm.jpg
0a71bd0ae36e81caf9b7d727d2fa648d
22ba5eeaab8dc9546363548efd1940fe02934918
136 F20110115_AAABGM shen_w_Page_03.txt
d97d99d12612959f025cf8283fbcb355
fdc4947e3e85ac29d5e39b729986d0b0e7e51d42
102642 F20110115_AAAAZO shen_w_Page_50.jp2
b8dd111e3c79540e5fc0d0b1228e7a1b
61ca53acb0995043af63101cfa9d428c36520fa6
F20110115_AAABBO shen_w_Page_21.tif
a7f860e10ddcc2873892a712da0112c4
23b144848f33b57df4634a046151daec8844ebcd
51837 F20110115_AAAAUR shen_w_Page_01.jpg
84b420757e8458b505d48a312fec831d
e7d343f7a910e197b81ed964e66eef05d840529c
65400 F20110115_AAABLK shen_w_Page_24.QC.jpg
e532c16c55d0d14c21860f5e7ac23c01
0760e36f437f6dc95028f2dc8f2750431fa19c65
3063 F20110115_AAABGN shen_w_Page_04.txt
8b7dd1269d8fac0c888c3e63889a9fbc
7cf946e3aaa9d57184a9fa480af8cbef05e1dab5
1032879 F20110115_AAAAZP shen_w_Page_51.jp2
dff7a4d097afb883e91aa845ebd7716c
5ff3f7df18d9f80d830d2cbf946453d218c1ff81
F20110115_AAABBP shen_w_Page_23.tif
a7f224dfc444b8ed8a939e18dbc7d0a1
93c71a2b64feb99aa12c574019548f2f472590e3
805934 F20110115_AAAAUS shen_w_Page_16.jp2
dcc80a62b0e5a6d064838bfa64160a31
ca5712f4092efeef602bc2333850cd264e5fe247
47378 F20110115_AAABLL shen_w_Page_25thm.jpg
4336367bcfe1f3535d37337d9f9d8aad
8296842a348d25e7274dbf1a64a91baa37b7c2e5
4369 F20110115_AAABGO shen_w_Page_05.txt
9da6ed44e22e661c83c5877015518d04
cce4430e331f8cfa6f17e42d0a738f5b02a8bc86
104448 F20110115_AAAAZQ shen_w_Page_52.jp2
155d651973af4d62e1cc4c7fa57deb77
2e5e6a97bac60d550e4e6bac4131e5f79c1c599e
F20110115_AAABBQ shen_w_Page_24.tif
c517d7bdd5db7cf742adbff4a58b57a8
eafc7cd095d5bb842b65ed0dd84352e7538471d1
2267 F20110115_AAAAUT shen_w_Page_09.txt
35afbcf36ee8ad75e96ffc40d7f476a1
2806b0b0f061946368746aa1a8c394423e1c26de
90098 F20110115_AAABLM shen_w_Page_25.QC.jpg
ed6d56041a227d2ba7bd856bbc0c1d0d
023de8b2886aba6db84f0e59895736144fc465b8
1051972 F20110115_AAAAZR shen_w_Page_53.jp2
7f03a6a9de39c924862e2c134d500c6f
354874c9c7dadc513d8019962ac8516898d1faf6
F20110115_AAABBR shen_w_Page_26.tif
c8803d9dbcb088dcf3b1c7d8c1a7a798
b37fdbd7e4f61037118d1a45ab4e6d2153323ca0
49515 F20110115_AAAAUU shen_w_Page_82.pro
0378830514834110c1b0396774545077
6e11d3c1814832f96da558c781ffbfdfba92fbf1
43647 F20110115_AAABLN shen_w_Page_26thm.jpg
dd87e55de66f6775aecdf3ca1b63d598
15b87f52a23148872458be3650eee6b60910758a
4282 F20110115_AAABGP shen_w_Page_06.txt
84544e3806ea8c28758979fa30fe0d11
762909c648b01c32942ef5cc6216ee39786b9d8b
127172 F20110115_AAAAZS shen_w_Page_54.jp2
be1e3c1fdfab96c31f9ea5b911bac758
fdabfb7f1f5e458c748cc0b7f5fc78ab2bf1ee38
F20110115_AAABBS shen_w_Page_28.tif
995c07df98ff7dcebe0568d7293f3a47
424aae004cb4653bcc4f83cca36aca17ff613826
220 F20110115_AAAAUV shen_w_Page_10.txt
883383fabeed790565af745e015a007f
8b7bfaa3891ee49d0fe02ac20c580407d1e74db6
79036 F20110115_AAABLO shen_w_Page_26.QC.jpg
3af8559fdf9da29656c16165eeda625b
ebfe14133821c81bbd4c4943f30241e07ee3d4a3
89907 F20110115_AAAAZT shen_w_Page_55.jp2
23d7689b11b0970b62ea667a21fbfb81
c6c1e424c6decc5e09fb0f3da67d49b3687d9fec
F20110115_AAABBT shen_w_Page_30.tif
23609cdbf5c76424cae9df34deacfa79
7301b2a22f941d4554f66651f4edf905e20185ae
61516 F20110115_AAAAUW shen_w_Page_76.QC.jpg
54b475897bc2a4632547236895773736
aba312dcba1f11d977ffa78f4c62220a70acbf76
274 F20110115_AAABGQ shen_w_Page_07.txt
55a26af4a356437db82cb233e8dac817
07153e772102edb073d751b3e7321d60472f2dbf
20951 F20110115_AAABLP shen_w_Page_28thm.jpg
7969fb26fe77ede68b59f2e8c06b6b95
6da5e7068348cf30348fb551f91315a671376669
89458 F20110115_AAAAZU shen_w_Page_56.jp2
08efefee6c3350f1d1ba83a832e15026
4f65b189fe497a55b3aa131a0d45179be38d3629
F20110115_AAABBU shen_w_Page_31.tif
11007d73406ee56ca40fa7a9e9da72f3
50034996b63fb99c2efaff34544cf0c7b808a6bf
1134 F20110115_AAAAUX shen_w_Page_26.txt
5e51f9eb5e1dfd35c8f15356b8490113
870648836a35b415e5f341bd95ffb58bd58cc8fb
1642 F20110115_AAABGR shen_w_Page_11.txt
eddfe6fc5993473211fcfc3bcc1a0d56
950dad68fe9dbeb4ba3342baa5985911a906e3e6
61659 F20110115_AAABLQ shen_w_Page_28.QC.jpg
cee3b783f6ce3d5d0ec809a4eeb7afd3
881a089d87e2bfdcb128864a99733bababdd5099
107676 F20110115_AAAAZV shen_w_Page_58.jp2
0ef1731c64f90e6a509f7a4e36189c26
deb67b2540b260075a49e4915f088be86fd4d3f9
F20110115_AAABBV shen_w_Page_32.tif
c1b84bc473f70cb1de22ce8c4cdd8fbc
e2f907e10bfff0bcd47146bdc2e97df7b836fbce
1748 F20110115_AAAAUY shen_w_Page_38.txt
f4980ed255e16a47da20fc3cc08931c8
6c8e731d855ac344b0a225d55360f4a7dfdca012
581 F20110115_AAABGS shen_w_Page_12.txt
08d404b367525462c985eb1085ba9ca9
177f61597953c98175313fce344e54386b1bacc4
24033 F20110115_AAABLR shen_w_Page_29thm.jpg
c6eed817f3f68c365f236d8862a344ca
7a2481291265d5f95bcac34cc39db0be9f8329d8
1027837 F20110115_AAAAZW shen_w_Page_59.jp2
923518d06960034d8cdd026836e3557d
46c1be6c78ecdc08679cecd68b4d8127edfaaf74
F20110115_AAABBW shen_w_Page_33.tif
765a41c1bc998eb9257d2b09e423ac86
e76d794f038fdbf9940f1de629a42fd957ed9176
33500 F20110115_AAAASA shen_w_Page_31.pro
840505efbd3213e565d31c46ef91ccd4
f00718058ba351cdc26caeea31b1a1e1a9c86f67
73022 F20110115_AAAAUZ shen_w_Page_61.QC.jpg
d11290591fa8e0f6ab845d32fed0ed63
8aecab521e3ceb7bbf523ee89ce1b317842b377a
1737 F20110115_AAABGT shen_w_Page_13.txt
4341d81a92db28a171895b2eb34c51a2
c20a48f19e9b8685f78f90c77d9c6b57ede616ca
71308 F20110115_AAABLS shen_w_Page_29.QC.jpg
d764e3b0239389e02712a14dc404243c
db86fb335e978548829fac2151c00b190a319945
9194 F20110115_AAAAZX shen_w_Page_60.jp2
ec191d2253a6e299958e9dd95aaaf608
4c2ca3d03c50384460035a5ce9f50347b3c071c0
F20110115_AAABBX shen_w_Page_34.tif
4268300f41bfa9ef2a85310cbe45e44e
a1ffaa9fde9fd214c13119f9cf58cfc6eb125846
24729 F20110115_AAAASB shen_w_Page_30thm.jpg
461d3c7d6c5fcf4d0de4b28c24bcd65e
508582acf690ad6b40e31cce2ca0c08513bd45b2
1838 F20110115_AAABGU shen_w_Page_14.txt
ecfbb043a64465b4363c9446c170309f
1dba7a33cca899ec53881d28bd7b356a91fa428e
46204 F20110115_AAABLT shen_w_Page_31thm.jpg
2eb8e87ee86df0565d78f3e2312bda3a
a55aa66273176aa613521dcea06dea03c16a4afa
115322 F20110115_AAAAZY shen_w_Page_61.jp2
ad4f8c1961033f61c23fc5b01ee50e9a
4b0c201415042eb4a3561b3354441a289688682e
F20110115_AAABBY shen_w_Page_35.tif
c86fcd88ccd3ffbe58912ebea605e069
2c7f1586c16cfe33920572767ddaf3668c1c746c
19789 F20110115_AAAASC shen_w_Page_76thm.jpg
986d8d761be2008cb0f1daa3ac64ef71
c265cd60f0c4c2936a3f9fb88693ceca44773ff0
2057 F20110115_AAABGV shen_w_Page_15.txt
10d11d92242a2b9b1f81783fad3c8d78
2eaca55f28a76a87d49709e42b1cff102afc95c0
246793 F20110115_AAAAXA shen_w_Page_53.jpg
03637ccffcfbd7418d75bde2549d1afe
b18bad1408e3771469001f2bfc7d27b60ac81162
97743 F20110115_AAAAZZ shen_w_Page_62.jp2
0243391e48a8106c5937eb700bef8d09
c033126cb10a52d3cc56180465ae932b5e8f9f19
F20110115_AAABBZ shen_w_Page_36.tif
ed16d7b961ae86a658da4babeaeda9bd
e4ee25aacab2673e12ac1a6df61cfd43b474f8d4
46399 F20110115_AAAASD shen_w_Page_67thm.jpg
2eef167e1eafa445a8c09e3c64f34c38
664afce6c8ec92425554f91423109d5ad6adcb62
1557 F20110115_AAABGW shen_w_Page_16.txt
ecacb44b957c9fd94b7a37d7dfeffc02
a7e49bbddb11b3d6f31b720b0678a8ef1cb7b536
89539 F20110115_AAABLU shen_w_Page_31.QC.jpg
130719298e685e028633962b0266099f
405cf65cd4b294161eab316ca49c291cc6a38f2e
195037 F20110115_AAAASE shen_w_Page_41.jpg
6acd2cc98996fcbee64caf609586a90a
7d0fa74e01054cd496e6f9ae1d00cf55231679ae
1962 F20110115_AAABGX shen_w_Page_17.txt
97e5c2c839741c5812533b6c4ac4dc34
4dde794c254eb7d65eb7a47f8766dd89c3b56ae0
176436 F20110115_AAAAXB shen_w_Page_55.jpg
6552129fd06e80e5d80be3dd51c88f34
195b6fb81162b4684557cddadd5222f9cff7aacd
22984 F20110115_AAABLV shen_w_Page_32thm.jpg
2f142f231d03a4a04d59dd39a3c26a70
18a104dfb49407ad0ef976b5dcd32541e46d5cea
F20110115_AAAASF shen_w_Page_78.tif
6d04364dea653dad3ae8c1d9c86051ca
b2bf2369bcc7ab441b9a031afa4ff6d9511b5d96
56605 F20110115_AAABEA shen_w_Page_09.pro
84b376333e73a8df1fa2d3796173f0ff
8ccb06b795d74f212e6776b6bb9830b04f1f8285
1875 F20110115_AAABGY shen_w_Page_18.txt
41902b78a58bcb186b6ed233e29f856a
3aec00696c8365cb674b7dd4333e0e7224e97d64
144255 F20110115_AAAAXC shen_w_Page_57.jpg
60636572426d09b0a7cd6cf29c7c1657
3d9df7fd5a123d3dcb1ad44b821f63ab0cbc7d9b
71899 F20110115_AAABLW shen_w_Page_32.QC.jpg
338daf2340b43482a0b10a5ecc3c8cf4
e6ee43131694bf637dce214c50e9f30414bc7513
1779 F20110115_AAAASG shen_w_Page_35.txt
bdc9d8ac70cd578736bba5be0edf517d
b2890ac9f460a9aa8e6d0cd9b59188d58d2dfdfc
36518 F20110115_AAABEB shen_w_Page_11.pro
89d293191e522da8b35ad2a37a510a77
a5d53f190424e99fc5972e1d62a1bb8014e5d4b2
1252 F20110115_AAABGZ shen_w_Page_19.txt
d1d5249445f3687fb055aa747f64e777
9960aa54429a4cc3a87d941c8c2e9f71c6a2e735
215784 F20110115_AAAAXD shen_w_Page_58.jpg
b7e745d2c9a738c51abfdc6437a4e49e
abc53186d4a895dc5ce9a7942dec2cf6287b91b1
78572 F20110115_AAABLX shen_w_Page_33.QC.jpg
9793fcdeb1220d2ecb48cc0dacf21ff3
920211c4b35561107a0db603a34678ee511ed0ce
69299 F20110115_AAAASH shen_w_Page_38.QC.jpg
2ce8662f7f6cbb300849f80b6f3ea986
d0c16c3e76cd8f9e6f722b07b64a26cd3b0b9828
14414 F20110115_AAABEC shen_w_Page_12.pro
19b31b239336444c39c095613b171969
c69b858135cb5152d390722a8060c5aff5b6cdf5
219168 F20110115_AAAAXE shen_w_Page_59.jpg
b50e1f7b4118546e50b30ac682ac695c
684f0d16753e52d72cb82615671307dfa4274000
22418 F20110115_AAABLY shen_w_Page_34thm.jpg
ba286760aff5aba1ec035a6506738910
adc4ffd880f91ddc9d13a41dfcfa871c196d0615
888 F20110115_AAABJA shen_w_Page_81.txt
cbfd38fb31ccec5d5098864ce9e5e0eb
01fcbc1458b9b0c45ba4c45ae7e6b276ec8fe7e2
5501 F20110115_AAAASI shen_w_Page_10.pro
04ce6a700059a2adf6c731d4621fbbab
2df110979ccb744ecf2a508860c0af6c6d1cabe4
41642 F20110115_AAABED shen_w_Page_13.pro
63e83167a6940890568450349009446c
f4aeea9085ac3690dcef659f40ffe997448c1276
19520 F20110115_AAAAXF shen_w_Page_60.jpg
0bf3441f8f793e8885228427f32fb960
fdca1d32d1038e1ed9c6d0d12e69f4c37ac697af
73040 F20110115_AAABLZ shen_w_Page_34.QC.jpg
c2e5cba8d8d9dfec6684a9c14cb8a048
5a087179759b19c7516149e612bf621a77d79d1d
2067 F20110115_AAABJB shen_w_Page_82.txt
07ef9df69b2529b55aa8f188f25694cc
b002b8345d944d5c3b06c345fb9af28f2c571d8b
254933 F20110115_AAAASJ shen_w_Page_54.jpg
9b794288a17f469f3826e01b09a328cd
0954bf847ffd09ba2eb2515ad6888fd07f02c94a
44681 F20110115_AAABEE shen_w_Page_14.pro
189fb21fa1dd576b0019cdb87dc2550c
22e2f47ee48844e9b366afd0cbec66b31cdda6b4
227809 F20110115_AAAAXG shen_w_Page_61.jpg
9f48485802db156d8d91f6a337136c3e
9c51aeda740d53f7d1a98be158fba52af5843de9
466 F20110115_AAABJC shen_w_Page_83.txt
e573a378a9e20a4e7b150ecfc40ff7e6
238bec6a7d6627bee4590c35ebe9331f24fecfed
65536 F20110115_AAAASK shen_w_Page_42.pro
4bc49ad3f08f12626c52dee69fb19559
159736c23e5eb685e0a06a7b2d8d28bed7cdef08
50351 F20110115_AAABEF shen_w_Page_15.pro
0a78394979e5370706abab01bd47e69c
6bca3dc6c1bf74d27cb244b2329b1e43e58af5dc
186615 F20110115_AAAAXH shen_w_Page_62.jpg
7f4b7fc93fafb068f09fbed5b33fbf66
7ce70145eb23f334dae89e5feffcf1e4db0939a7
91975 F20110115_AAABOA shen_w_Page_71.QC.jpg
f44b4b694ed9df1ea816f97d2ae862d9
fe4eec2a85b4b0eefebd88b5c6172f843c5e48c5
914 F20110115_AAABJD shen_w_Page_84.txt
eff2ac196d557036c3fcdc6c3ad68af5
9cc03a4d388c9b598be18c5749adc243d8658d5a
18823 F20110115_AAAASL shen_w_Page_11thm.jpg
6fab549623f69cd8ef8bf698792c7448
1060f77e98579b2ffd1e0b2ef428d0a8b0d0850e
32855 F20110115_AAABEG shen_w_Page_16.pro
94a29eec32df1dbd5af6f4a3a47cfe43
ffb3f6814beae7ee8b2eef7a18e737a4cd42601e
163459 F20110115_AAAAXI shen_w_Page_63.jpg
e2f2605152e7a9f31e9827f295c45e74
741ce437684041e468d0bfd4449d7021b1253ff4
47971 F20110115_AAABOB shen_w_Page_72thm.jpg
98dd5950ebfb34fa76b69d6167c141f2
49f3a87c0b69eda7ea9fe36bc2d6d1f6f3c66e75
1289857 F20110115_AAABJE shen_w.pdf
eab8bd8b3f2a5ea649479dec5b5bc51b
2d4b7c1baffb7d7913d26af00b35b4a7ce9216a3
F20110115_AAAASM shen_w_Page_29.tif
b3c21abf33fcd5d5deb44093577a0a3b
6178086c3d1423bef1b5ea694f9fe8b43104791d
49840 F20110115_AAABEH shen_w_Page_17.pro
3dc9a08b104badaa7914904f34f61a9e
7242fdca617209a7427e7cb79a5606568f3ebbd2
218700 F20110115_AAAAXJ shen_w_Page_64.jpg
023dc256aff1c3dcdd46482933007cff
f386255640fe66952e8399a465a8b50116b92094
95171 F20110115_AAABOC shen_w_Page_72.QC.jpg
3c5a999f24307719cee6cd615506b3b2
1240cce3b78ead1d82f5740d3d5ef7afae1a3d75
21648 F20110115_AAABJF shen_w_Page_39thm.jpg
7782b9283205c00f88b0a793365b6178
5ecbcdebfa98c156a09b721a2576cad1c1bf58a0
40994 F20110115_AAAASN shen_w_Page_49.QC.jpg
59551c9b09629437e6cadd297ce39c58
1b4e279b8af8277643cb5052fc5afaf04a66ed47
46263 F20110115_AAABEI shen_w_Page_18.pro
08b0f0fa903a5f3b8e4ead3fd8a53411
b8b2ac9bc9f9b7b1f65a5ec1155c3a1d7585c980
227406 F20110115_AAAAXK shen_w_Page_65.jpg
287e31358030ed11541b668ced40fa5e
db14f844d1a18c00f48e46a3a3d04d8c3f0a107e



PAGE 1

IMPLEMENTING A GLOBAL ANTI-DoS SERVICE BASED ON RANDOM OVERLAY NETWORK By WEN-CHUAN SHEN A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE UNIVERSITY OF FLORIDA 2004

PAGE 2

Copyright 2004 by Wen-Chuan Shen

PAGE 3

To my family and friends who supporte d me with their patience and love.

PAGE 4

TABLE OF CONTENTS page LIST OF TABLES...........................................................................................................viii LIST OF FIGURES...........................................................................................................ix ABSTRACT.......................................................................................................................xi CHAPTER 1 INTRODUCTION........................................................................................................1 What Is Distributed Denial of Service (DDoS) Attack................................................1 Related Work................................................................................................................2 Ingress Filtering Proposed by Ferguson and Senie...............................................2 Route-Based Packet Filtering Proposed by Park and Lee.....................................2 SYN-Dog Proposed by Wang, Zhang and Shin.....................................................2 Self-Complete Defense Systems...................................................................................2 2 AID SYSTEM OVERVIEW........................................................................................4 Fundamental Ideas........................................................................................................4 Who Is Protected?..................................................................................................4 What Is Random Overlay Network (RON) for?....................................................5 How Does AID Defense System Work?...............................................................5 Implementation Issues..................................................................................................6 Packets Intercepting Modules................................................................................8 Handling Queued Packets in Userspace................................................................8 Showing Statistics.................................................................................................9 3 AID LAYER...............................................................................................................10 AID Layer for TCP Traffic.........................................................................................10 AID Layer for UDP Traffic........................................................................................12 PUSH Message....................................................................................................12 PULL Message....................................................................................................13 PULLANS Message............................................................................................13 CTRLT Message.................................................................................................14 Implementation Issues................................................................................................15 iv

PAGE 5

4 CLIENT END.............................................................................................................16 Module ClientFilter.o.................................................................................................16 Program Client............................................................................................................17 Packets from NF_IP_PRE_ROUTING...............................................................17 Packets from NF_IP_LOCALOUT.....................................................................18 Packets from NF_IP_POST_ROUTING.............................................................18 Implementation Issues................................................................................................20 What Is ServList?................................................................................................20 Why Changes a Packet's Destination in Hook NF_IP_LOCAL_OUT?.............20 How Is the Registration Done?............................................................................20 Not Perfectly Isolated from Higher-Level Applications.....................................21 The Maximum Transmission Unit (MTU) Problems..........................................21 5 SERVER END............................................................................................................23 Module ServerFilter.o.................................................................................................23 Program Server...........................................................................................................24 Packets from NF_IP_PRE_ROUTING...............................................................24 Packets from NF_IP_POST_ROUTING.............................................................26 Implementation Issues................................................................................................26 No Threads..........................................................................................................26 Important Variables.............................................................................................27 PCKBUFSIZET...........................................................................................27 PCKBUFSIZEN...........................................................................................27 IPQREADTIME...........................................................................................27 READINTERVAL.......................................................................................27 SENDPCKBUFNO......................................................................................28 AVGINTERVAL.........................................................................................28 TOTALCAP.................................................................................................28 RESERVEDTIMES.....................................................................................29 How the Registration Is Done.............................................................................29 Not Perfectly Isolated from Higher-Level Applications.....................................29 Program Alert......................................................................................................31 6 AID STATION...........................................................................................................32 AID Tunnel Tree.........................................................................................................32 Push Phase...........................................................................................................32 Pull Phase............................................................................................................34 Routing................................................................................................................34 Why Does a Tunnel Tree Try to Include Every AID Station?............................35 Variables k and q.................................................................................................36 Advantages of Random Overlay Network (RON)..............................................38 Distributed Virtual-Clock Packet Scheduling............................................................39 Basic Idea............................................................................................................39 How to Adjust T..................................................................................................40 v

PAGE 6

Programs for an AID Station......................................................................................40 Module AIDFilter.o.............................................................................................40 Program AID........................................................................................................41 TCP packets from NF_IP_PRE_ROUTING................................................41 UDP packets from NF_IP_PRE_ROUTING...............................................42 TCP packets from NF_IP_POST_ROUTING.............................................42 UDP packets from NF_IP_POST_ROUTING.............................................43 Implementation Issues................................................................................................43 No Threads..........................................................................................................43 Registration for Clients and Servers....................................................................44 Important Variables.............................................................................................44 PCKBUFSIZE..............................................................................................44 IPQREADTIME...........................................................................................44 READINTERVAL.......................................................................................44 SENDPCKBUFNO......................................................................................45 NEARBYAID..............................................................................................45 SNEDTINTERVAL.....................................................................................45 SENDPULLINVAL.....................................................................................45 DECREASERATIO.....................................................................................45 MAXVCTSEXCEED...................................................................................45 Adding New AID Stations...................................................................................46 Diameter of a Tunnel Tree..................................................................................46 Forwarding Packets.............................................................................................47 7 TESTING RESULTS AND ANALYSIS...................................................................49 Important Issues about Testing...................................................................................49 Testing Elements........................................................................................................50 Program TestClient..............................................................................................50 Program TestServer.............................................................................................51 Setting of the AID System...................................................................................51 Case 1..........................................................................................................................51 Case 2..........................................................................................................................54 Case 3..........................................................................................................................56 Case 4..........................................................................................................................58 Case 5..........................................................................................................................60 8 FUTURE WORK AND CONCLUSION...................................................................64 Limitations and Future Work......................................................................................64 Conclusion..................................................................................................................65 APPENDIX A HOW TO RUN...........................................................................................................66 B FILE GLOBAL.H.......................................................................................................68 vi

PAGE 7

LIST OF REFERENCES...................................................................................................70 BIOGRAPHICAL SKETCH.............................................................................................72 vii

PAGE 8

LIST OF TABLES Table page 7-1. List of important factors of AID system for testing..................................................51 7-2. Case 1 packets statistics in the AID station...............................................................52 7-3. Case 2 packets statistics in the AID station...............................................................54 7-4. Case 3 packets statistics in the AID station...............................................................56 7-5. Case 4 packets statistics in the AID station...............................................................59 7-6. Case 5 packets statistics in the AID station...............................................................62 viii

PAGE 9

LIST OF FIGURES Figure page 2-1. AID system architecture..............................................................................................4 2-2. Netfilter hooks.............................................................................................................7 3-1. AID layer header for TCP packets............................................................................11 3-2. PUSH message..........................................................................................................13 3-3. PULL message...........................................................................................................13 3-4. PULLANS message...................................................................................................14 3-5. CTRLT message........................................................................................................14 4-1. Inserting an AID layer header to a packet that enters AID tunnels...........................19 4-2. Inserting an AID layer header to a packet not entering AID tunnels........................19 5-1. Removing the AID layer header in server end..........................................................25 6-1. Tunnel tree created in push phase..............................................................................33 6-2. Four possibilities a packet can be routed...................................................................47 7-1. Distribution of incoming packets in case 1 at the AID station..................................53 7-2. How did T and arrival rate interact with each other in case 1...................................53 7-3. Distribution of incoming packets in case 2 at the AID station..................................55 7-4. How did T and arrival rate interact with each other in case 2...................................55 7-5. Distribution of incoming packets in case 3 at the AID station..................................57 7-6. How did T and arrival rate interact with each other in case 3...................................58 7-7. Distribution of incoming packets in case 4 at the AID station..................................59 7-8. How did T and arrival rate interact with each other in case 4...................................60 ix

PAGE 10

7-9. Distribution of incoming packet s in case 5 at the AID station................................62 7-10. How did T and arrival rate interact with each other in case 5.................................63 x

PAGE 11

Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Science IMPLEMENTING A GLOBAL ANTI-DoS SERVICE BASED ON RANDOM OVERLAY NETWORK By Wen-Chuan Shen December 2004 Chair: Shigang Chen Major Department: Computer and Information Sciences and Engineering Distributed denial of service (DDoS) is a major threat to the Internet nowadays. Legitimate users have a hard time accessing the servers that are under DDoS attacks. What makes it worse is that the attacking tools are easy to get. Even people without enough professional knowledge can launch a DDoS attack. Obviously, automated anti-DDoS systems become more and more important. Many current available solutions to DDoS attacks require universal installation and configuration across different administrative realms, which are impossible or very difficult to do in many situations. This thesis studied and provided another solution to DDoS attacks. An anti-DoS service (called AID) is presented in this thesis, and no global deployment is required. The AID service protects general TCP traffic. It guarantees all registered clients can access a registered server fairly even when the server is under DDoS attacks. A domain, like a school or company, can get immediate protection after having the AID service. xi

PAGE 12

Two primary parts of the AID service are the random overlay network (RON) and the distributed virtual-clock packet scheduling algorithm. The former forms tunnel trees, which connect registered clients to a registered server. Packets from registered clients go through the tree to the server when the server is under DDoS attacks. It is adapted and easy to manage. The latter is a packet scheduling algorithm to simulate client puzzles. It confines the amount of data a registered client can send to a server through RON to achieve fairness. xii

PAGE 13

CHAPTER 1 INTRODUCTION What Is Distributed Denial of Service (DDoS) Attack A DoS attack intends to make a server out of its resource (which could be bandwidth, buffers, CPU time, etc.). Attacker s can send a lot of requests to exhaust a server's bandwidth. Other legitimate users will be unable to access the server. Another example is SYN flooding attack [1-2] To establish a connection, a client sends a SYN packet at first. The server is going to keep this information in a buffer for a period T in order to recognize the following incoming packets. If attackers send enough SYN packets to make the buffer overflow, the server has no way to process requests from other users. In some cases, like route table updating or softwares bugs, a simple request can make the server do a considerable amount of computation. In this case, normal users cannot access the server. The basic idea of DoS attacks is simple, using a small amount of resources to overwhelm the server. What makes a DDoS attack different from a traditional DoS attack? In a DDoS attack, the clients launching the attack might be victims as well. Hackers compromise and install DDoS attacking programs on these hosts first. Then, hackers can remotely control these victims to attack the servers cooperatively. With DDoS attacks, it is possible to flood a big commercial server in a brute-force way. Besides, it becomes very difficult to trace back the attacker because the compromised hosts are not the real attackers. Usually, there are hundreds or thousands of compromised hosts and they are 1

PAGE 14

2 around the whole world. It makes attackers feel safe to do this. Today, how to protect hosts against DDoS attacks is very important. Related Work Ingress Filtering Proposed by Ferguson and Senie In ingress filtering [3] before a packet is transmitted into another domain, the router checks the packets source address. If it does not match the ingress filter rule, probably a spoofed source address, the packet will be dropped. Ingress filtering helps to trace back the attacker, while it cannot prevent an attack originating from a valid source address. Route-Based Packet Filtering Proposed by Park and Lee With partial deployment (about 18% in Internet AS topologies) [4] spoofed IP packets can be prevented from reaching their intended targets effectively. SYN-Dog Proposed by Wang, Zhang and Shin SYN-dog [5] is a software agent which can be installed at leaf routers of stub networks. It is stateless and light-weighted. Therefore, SYN-dog itself is immune to any flooding attacks. It detects SYN flooding attack by monitoring behavior of SYN-SYN/ACK packets. SYN-dog can also trace back the attacking source. Self-Complete Defense Systems There are a huge number of hosts on the Internet. It is almost impossible to make every host join a specific defense system. Here comes the problem. If a server has the defense system installed, can it resist the DDoS attack from clients that do not participate in the same defense system? The answer is no for most existing DDoS defense systems. Suppose we have a networked system of S + C. C is a set of client networks, and S is a set of server networks. C' is a subset of C and S' is a subset of S. C' + S' has a defense

PAGE 15

3 system installed. A defense system is self-complete if any client in C' can still access any host in S' even when under DDoS attacks, as long as the client itself does not participate the attack. It does not care if the attack is from C or C C'. In other words, a self-complete defense system should be able to defeat attacks from either inside area C' or outside area C C'. A self-complete defense system makes itself a clean area in the Internet. The area does not have to cover the entire Internet, and hosts in it are protected. Let us review the DDoS defense systems mentioned above. Ingress filtering: Source addresses of packets from C C' can be spoofed, because C C' do not check them. Therefore, DDoS attacks can be launched against the S' from C C'. In this case, clients in C' have difficulty to access S' even though C' and S' both join the defense system. Therefore, Ingress filtering is not self-complete. Route-Based Packet Filtering: Packets with spoofed source are prevented from reaching their intended targets effectively as long as 18% of Internet AS's join the defense system. However 18% of Internet AS's is a huge number. It is not self-complete until C' is as large as 18% of Internet AS's. SYN-dog: Attackers can still do SYN-flooding to S' from C C', because C C' is not under protection. Similar to ingress filtering, unless C' is as big as C, attackers from C C' can make S' not accessible to C'. In consequence, SYN-dog is not self-complete. The benefits of a self-complete system are apparent. It suits normal companies, schools and organizations. They can set up a self-complete defense system in their realm and become under protection immediately, independent of others. A working self-complete defense system (called AID), the detail of its structure and how it was implemented are presented in the thesis. The AID systems overview is in chapter 2.

PAGE 16

CHAPTER 2 AID SYSTEM OVERVIEW Fundamental Ideas The idea of the anti-DoS system (called AID) is from Chen et al [6] The AID system contains clients, servers and AID stations physically. Another two important parts in the AID system are random overlay network (RON) and distributed virtual-clock packet scheduling algorithm [6] Random overlay network is formed on AID stations. We did not draw a clear line between DoS attacks and DDoS attacks in the thesis. Whenever a server senses an attack, like flooding packets beyond its capacity or unusual requests from clients, the AID defense system will be triggered. The AID system's architecture is shown in Fig. 2-1. Figure 2-1. AID system architecture. The AID circle and AID tunnels symbolize RON, which is composed by AID stations. A client point can mean a client network, not just a client host. Likewise, a server point can be a server network behind a router. Who Is Protected? Register clients and servers that we want to protect at their nearby AID stations. The registration brings a shared secret key between the AID station and the registered 4

PAGE 17

5 host. The secret key is used in AID tunnels for integrity checking when under DDoS attacks. Everyone can join the AID service by registering at an AID station. What Is Random Overlay Network (RON) for? RON consists of all AID stations. When a registered server is under DoS attacks and other registered clients try to access the server, packets from these clients will go through the RON instead of the Internet. We say that these packets are entering AID tunnels, which are tree structure. They go through AID tunnels all the way to the attacked server. Other packets from unregistered clients will reach the server via the Internet. AID tunnels are one-way path for packets from registered clients to attacked registered servers. We do not allow traffic from registered servers to registered clients entering RON to minimize the load of AID stations. It is transmitted via the Internet. Of course, traffic related to unregistered clients or unregistered servers cannot enter AID tunnels. How Does AID Defense System Work? When under attacks, packets from registered clients go through RON but packets from unregistered clients go through the Internet. We make packets from AID tunnels have higher priority. Servers process them first. Hence, the external traffic (from unregistered clients) cannot influence the internal traffic (from registered clients). How about if a registered client is an attacker itself? Well, that is why we have distributed virtual-clock packet scheduling algorithm, which simulates client puzzles [7-10] Every AID station manages a virtual clock for every tunnel hooking on a client network. If a client has behavior of flooding a registered server, its virtual clock will run fast. When virtual clock's value is too big, packets from that client will be dropped. By doing this, we can separate the attacking traffic out and block it.

PAGE 18

6 Traffic in AID tunnels has integrity protection. Remember the secret key a client or server got after registering at an AID station? The secret key and other important data in a packet are put together and digested by MD5 algorithm. The 128-bit-long packet digest is used for integrity checking. As a result, alteration of packets in AID tunnels will be detected. Because the third party cannot forge the packets, integrity checking is also authenticity checking, verifying that the packets are really from the hosts as they claimed. Now we know how AID stations interact with clients and servers. We also know what random overlay network and distributed virtual-clock packet scheduling algorithm are for. More details of the AID system and how it was implemented were revealed in the following chapters. Implementation Issues We chose Linux as our developing platform and our programs only work on IPv4. As mentioned in the section How Does AID Defense System Work ," a registered client should send it packets via RON instead of the Internet. Meanwhile, the attacked server should be able to tell where a packet is from and give the one from AID tunnels higher priority. We do not want people to recompile their Linux kernels or rewrite application codes if possible. So, we introduced another layer between application layer and transport layer, called AID layer. Extra information is added into a packet as an AID layer header for the AID service. Higher application layer programs should not notice the existing of AID programs. Netfilter is one tool we used in our AID system to intercept and modify packets. It was included in Linux 2.4. It supports five different hooks and they are NF_IP_PRE_ROUTING, NF_IP_LOCAL_IN, NF_IP_FORWARD,

PAGE 19

7 NF_IP_LOCAL_OUT and NF_IP_POST_ROUTING. Fig. 2-2 shows how a packet goes through these hooks. Figure 2-2. Netfilter hooks. Hook 1 is NF_IP_PRE_ROUTING, hook 2 is NF_IP_LOCAL_IN, hook 3 is NF_IP_FORWARD, hook 4 is NF_IP_POST_ROUTING and hook 5 is NF_IP_LOCAL_OUT. NF_IP_PRE_ROUTING: A packet hits the hook after reaching the host and sanity checks but before the routing decision. NF_IP_LOCAL_IN: A packet hits the hook after the routing decision and the packet's destination is this host. NF_IP_FORWARD: A packet hits the hook after the routing decision if the packet's destination is another interface. NF_IP_LOCAL_OUT: A packet hits the hook when going down the kernel after a process creates and sends out the packet. NF_IP_POST_ROUTING: A packet hits the hook right before it is put on the wire. We can inject our handling functions into any of these hooks. When a packet goes through hooks, their handling functions will be executed. That is where and how we can modify the packet.

PAGE 20

8 Packets Intercepting Modules Our first step is to write modules to intercept interested packets in proper hook positions. The interested packets are queued into userspace. Doing in this way may cause some performance penalty because of switching between kernelspace and userspace. However, there are also some advantages. First, it is easier to debug a program running in userspace. Second, we have more libraries handy. Third, misbehavior, if any, of a program will not crash the whole system. Three modules totally, clientFilter.o takes care of packets in/out clients. Likewise, AIDFilter.o is for AID stations, and serverFilter.o is for servers. After loading an appropriate module on the host, interested packets will be queued to userspace. To stop it, just unload the loaded module. These queued packets will be inspected or modified later. Handling Queued Packets in Userspace To deal with the queued packets in userspace we need the library libipq developed by James Morris. It can be found easily on the Internet and simple to install. With the library, we can grab one packet out of the queue every time. We can drop the packet, do nothing, or modify and send it back to the kernel. One thing should be noticed is that checksums in TCP/UDP and IP headers need to be recalculated if the packet is altered. In the client end, all outgoing TCP packets are queued to userspace. If a packet's destination server is under attacks, its destination IP will be converted to an AID station's IP. The AID station is the one this client registered at. The AID station will notify the registered client if a registered server is currently under attack. If no attacks, packets are routed as usual. A registered client executes the program client. In AID stations, AID tunnel trees are built up to route packets to their destination servers. Assume a server registered at an AID station named A s When the server is

PAGE 21

9 under DoS attacks, an AID tunnel tree rooted at A s is formed. Packets from registered client to this server are routed from tree leaves to the root A s and finally to the server. Besides routing, virtual clocks for every client are maintained by AID stations as well. An AID station executes the program AID. In the server end, packets from the Internet and AID tunnels are separated. Process the latter first. If under attacks, a server will send alert messages to its registering AID stations, A s Then, this AID stations broadcasts alert messages to other AID stations. AID tunnel trees are constructed. A server executes the program server. Showing Statistics Programs client, server and AID, all record statistic information of TCP traffic. Users can know how many packets got through or were dropped and the reasons of dropping. They all have a while loop in main() whose condition is always true. To keep programs simple, we did not use threads or fork a child process. Then how can the programs interact with users when they want to see the statistics of traffic? The answer is signal. The reaction of the signal SIGINT was redefined in these three programs. When Ctrl-c is typed, programs will not be terminated. Instead, statistic information is printed out. We can type Ctrl-\ to send the signal SIGQUIT to stop the programs.

PAGE 22

CHAPTER 3 AID LAYER AID layer is added between application layer and transport layer. In this chapter, we defined AID layer headers, which are inserted between TCP/UDP headers and application layer data in a packet. Since we do not want AID service users to recompile their Linux kernels, Linux kernels have no idea of this new layer. AID layer headers are treated as application data actually by Linux kernels. In clients, the program client adds an AID layer header before a packet is sent out. In servers, the program server takes the AID layer header off. Only the AID system can recognize AID layer headers. As for AID stations, AID layer headers contain data needed for routing, constructing AID tunnel trees, and etc. Consequently, application programs in the client end or server end do not know they are already in the AID service and protected. Currently, only TCP traffic is protected in the AID system because TCPs congestion control feature is needed. When a packet enters RON, if it is a TCP packet, it belongs to traffic from a registered client to a registered server. If it is a UDP packet, such as a PULL message, or PUSH message, the packet is used to control the AID system. We explained what these UDP messages are later in this chapter. We have different AID layer headers for TCP and UDP packets. Both TCP and UDP traffic have integrity guard. AID Layer for TCP Traffic We have two kinds of AID layer headers for TCP packets. One is for packets transmitted via the Internet and the other is for packets transmitted via the RON (AID tunnels). Fig. 3-1 shows the contents of the headers and where they are inserted. 10

PAGE 23

11 A B Figure 3-1. AID layer header for TCP packets. A) For normal TCP packets that do not enter AID tunnels. Recognizing field is 0. B) For TCP packets to attacked servers that enters AID tunnels. Notice that recognizing field in the first figure is at the same position as server IP in the second figure. Server IP field is used to save the IP address of destination server. To travel through AID tunnels, a packets destination is changed to the AID station where it is routed next. However, the final destination is still the server, so we need to keep this information. Md5 digest field is used to check integrity. If checking fails, the packet will be dropped. Virtual clock timestamp field is used in distributed virtual-clock packet scheduling algorithm. Why do we need recognizing field even in normal packets? The problem is that when a server gets a packet, it has no way to know if the packet is from the Internet or AID tunnels. Recognizing field of normal packets is at the same location as server IP field of packets to an attacked server, right after the TCP header. When a packet arrives, the server checks this location. If it is 0, the packet is from the Internet; otherwise, the packet is from the AID tunnels. We assume server IP cannot be 0.0.0.0, so no conflicts. What information is under integrity protection? Source IP (4 bytes) and destination IP (4 bytes) addresses in the IP header: Source IP is always a client's IP address. However, destination IP could be the destination

PAGE 24

12 servers IP address if transmitted via the Internet or an AID station's IP address if in the AID tunnels. Source port (2 bytes) and destination port (2 bytes) in the TCP header: Unlike destination IP, a packets destination port is not changed when entering AID tunnels. Sequence number (4 bytes) and acknowledgement number (4 bytes) in the TCP header. (a) Recognizing field (4 bytes) in the AID layer header. (b) Server IP (4 bytes) in the AID layer header. (a) and (b) are in the same position and have the same size. Its value is 0 for normal packets, or destination servers IP for packets entering RON. Virtual clock timestamp in the AID layer: Only packets go through AID tunnels have this field. Whole application layer data. AID Layer for UDP Traffic There are several different UDP messages used to control the AID system. They are distinguished by the packet type field in the AID layer header. All of these messages are integrity-protected. PUSH Message PUSH messages notify other AID stations a server is currently under attacks. Fig. 3-2 shows the content of a PUSH message. An AID station or registered server can send PUSH messages. Md5 digest is for integrity protection. Packet type is set to 2 for PUSH messages (defined in global.h). AIDNO means AID station number, which records how many AID stations a packet will pass before reaching the server. It is essential for establishing AID tunnels. More details are explained in later chapters. Service port and server IP are the attacked servers IP address and port number.

PAGE 25

13 Figure 3-2. PUSH message. The AID layer header is inserted between the UDP header and application layer data. PULL Message PULL messages ask other AID stations what servers are currently under attacks. Figure 3-3. PULL message. There is no application layer data in a PULL message. Fig. 3-3 shows the content of a PULL message. There is nothing behind the AID layer header. Md5 digest is for integrity protection. Packet type is set to 0 here (defined in global.h). PULLANS Message When an AID station gets a PULL message from another AID station, the former will return information of all currently attacked servers it knows to the latter. Sending PULLANS messages does it. Fig. 3-4 shows the content of a PULLANS message. Packet type is set to 1 for PULLANS messages (defined in globa.h). Every AID station maintains a service list, which stores information of attacked servers. Each attacked server is a service list node. If an AID station has information of N attacked servers, there will be N nodes in the service list to be sent out. In a service list node, Distance says that from this AID station

PAGE 26

14 how many AID stations a packet still needs to pass to achieve the server, excluding the first AID station. The distance information helps to construct AID tunnel trees. A B Figure 3-4. PULLANS message. A) The content of a PULLANS message. B) The structure of the service list node. It contains IP and port of an attacked server. CTRLT Message T, the waiting interval, is for adjusting the speed of a virtual clock. When the arrival rate is larger than a server's capacity, a bigger T will be sent to AID stations to accelerate virtual clocks. Figure 3-5. CTRLT message.

PAGE 27

15 Packet type is set to 3 (defined as in global.h). A CTRLT message is for a specific tunnel tree of the attacked server with server IP. T field contains the new value of T for that specific tunnel tree. After getting a CTRLT message, an AID station updates its T. Most of UDP messages are related to RON maintenance and distributed virtualclock packet scheduling. We have not talked about them so far. They would be pointed out in later chapters. What fields are under integrity protection? Source IP (4 bytes) and destination IP (4 bytes) addresses in the IP header. Source port (2 bytes) and destination port (2 bytes) in the UDP header. Packet type (1 byte) in AID layer header. Whole application layer data. Implementation Issues Most UDP messages in our AID system have fixed size, except for the PULLANS message. Its size depends on how many nodes in the service list. If there are many nodes, the message packet will be too big to be sent out. In the circumstance, it should be divided into two or more PULLANS messages. How big is too big? We defined a constant, UDPMAXSIZE, in global.h. When a UDP message is bigger than UDPMAXSIZE, it will be chopped up into several packets.

PAGE 28

CHAPTER 4 CLIENT END On the client end, we need to filter incoming and outgoing packets. For example, when a TCP packet is leaving the client end, its destination needs to be checked. If the destination server is under attacks, the packet will enter AID tunnels; otherwise, it is routed as usual. ClientFilter.c and client.c are two main source files for client ends. ClientFilter.c is compiled as a module, queuing interested packets into userspace. Then, client.c takes queued packets out and does whatever is necessary. After a client registers at an AID station, A c it can get a secret key. The key is used to verify that the third party did not modify the communication between the client and A c The client also keeps A c 's IP address. If it tries to access an attacked registered server, its packet will be forwarded to A c When an AID station is informed that a server is attacked, it will send PUSH messages to its registered clients. For instance, the client gets PUSH messages from A c which is the AID station it registered at. All UDP messages in the AID system are sent to port 4369. However, there is no program in application level listening on this port. UDP packets to port 4369 are handled by the program client. Module ClientFilter.o By compiling clientFilter.c, we can get the module clientFilter.o. It hooks on handling functions at hooks NF_IP_PRE_ROUTING, NF_IP_LOCAL_OUT and NF_IP_POST_ROUTING. At NF_IP_PRE_ROUTING, only UDP packets to the port 16

PAGE 29

17 4369 are queued. Other incoming traffic is not related to the AID system. At NF_IP_LOCAL_OUT, all outgoing TCP packets are queued, except for the local traffic. Local traffic goes from loopback interface, 127.0.0.1, to loopback interface. At NF_IP_POST_ROUTING, all outgoing TCP packets are queued, except for the local traffic. Only outgoing TCP packets are queued since currently only TCP traffic is protected. When the module clientFilter.o is loaded in a host, the host must run the program client as well. Otherwise interested packets keep getting into the queue, but no programs take them out of the queue. The traffic is blocked if this happens. A host should load the module clientFilter.o and run the program client at the same time. It is meaningless to do just one of them. Program Client By compiling client.c and linking other relative source files, we can get the executable program client. It has a while loop in main() whose condition is always true. The program client deals with packets queued at different hooks by the module clientFilter.o. Now, we discuss what the program client does to packets from different hooks. Packets from NF_IP_PRE_ROUTING All packets in the queue grabbed at the hook NF_IP_PRE_ROUTING are UDP traffic to port 4369. For a client, the only UDP message of the AID system (to the port 4369) is PUSH. PUSH messages are sent by A c to inform the client what servers are attacked. Every client has a servList recording attacked servers (servList.h/servList.c). When getting PUSH messages, the client is going to update its servList. PUSH messages have md5 digest in it, for integrity checking. Others cannot pretend A c to send PUSH

PAGE 30

18 messages or pretend the client to send packets into AID tunnels as long as they do not know the secret key shared between A c and the client. These UDP packets do not go further from here in the kernel. We mentioned no application level programs listening on port 4369 earlier. The program client tells the kernel just drop them after it gets the information of PUSH messages. Packets from NF_IP_LOCALOUT The program client processes all TCP packets leaving the client host. First, it examines a packet's destination IP. If it finds a match in the servList (the destination service is under attacks), it changes the packets destination IP to A c 's IP and copy the packet with new destination IP back to the kernel. The destination port is not compared when searching a match in the servList. It is not necessary to distinguish different service ports on an attacked server. With A c 's IP as destination, the packet is going to enter an AID tunnel tree. If no match in servList, the program client just tells the kernel it did not do anything to the packet and the kernel can continue passing on the packet. Packets from NF_IP_POST_ROUTING All TCP packets come here after passing the hook NF_IP_LOCAL_OUT. The program client inserts a bunch of data into every packet as AID layer header. If a packet's original destination is an attacked server, its destination IP we can see here is A c 's IP. It was modified in the hook NF_IP_LOCAL_OUT. If the server is not under attacks, we can see the server's IP as the packet's destination IP. The program client inserts different AID layer headers into packets. If destination IP was changed into A c 's IP, it means the packets are going to enter AID tunnels. As shown in Fig. 4-1, application layer data is moved back 28 bytes, and an AID layer header is put into that 28 bytes space. Virtual clock timestamp is initialized to the client's

PAGE 31

19 local time. Md5 digest ensures A c that the packets are really from the client and not altered. Figure 4-1. Inserting an AID layer header to a packet that enters AID tunnels. A 28-byte-long AID layer header is injected. For packets not entering AID tunnels, their destination IP is still the server's IP. These packets do not need md5 digest and virtual clock timestamp. Nevertheless, they do need the 4-byte-long recognizing field. Fig. 4-2 shows how this sort of packets is dealt with. Recognizing field is an unsigned integer with value 0. We explained why we need it clearly in the chapter AID LAYER. Figure 4-2. Inserting an AID layer header to a packet not entering AID tunnels. A 4-byte-long AID layer header is injected. Either packets going to AID tunnels or not, this hook is the final chance we can modify them. After copying the modified contents back to the kernel, these packets are sent out right away.

PAGE 32

20 Implementation Issues What Is ServList? ServList is a simple list structure. It uses sequential search going through every list node to find a match. A servList node contains an attacked server's IP and port, but currently the port is not checked for a match in our AID system. A servList is updated by PUSH messages from A c Why Changes a Packet's Destination in Hook NF_IP_LOCAL_OUT? The program client used to change a packet's destination IP in the hook NF_IP_POST_ROUTING, but it did not work as we expected sometimes. For example, we have a server, an AID station and a client. Their IP addresses are 192.168.1.100, 192.168.1.101 and 192.168.1.103 respectively. Assume the server is under attacks, and the client sends a packet to the server. Before getting in the hook NF_IP_POST_ROUTING, the packet's destination IP is server's IP, 192.168.1.100. After leaving the hook but before really sent out, the packet's destination IP is the AID station's IP, 192.168.1.101. Then the packet leaves the client. Unexpected things happen here. The AID station does not get the packet, but the server gets it. If the server has FORWARD chain in iptables set well, the packet may be forwarded to the AID station. Anyway, it is not what we want. The packet should go to the AID station directly since we changed the packet's destination IP. We concluded that we should not change the packet's destination IP right before it leaves the host. We should do this in the hook NF_IP_LOCAL_OUT instead, and everything goes well. How Is the Registration Done? It is lots of work and difficult to make a complete secure system. Registration may be a secure hole in the system. In our AID system, clients get the secret key shared with

PAGE 33

21 A c by registration. We just took the easiest step here. The shared key was pre-configured in both the client and AID station, A c If users want to change the key, they need to redefine it in both sides, with the same value. Then recompile the codes. It is not hard. We wrote a makefile compiling and linking object files to generate the program client. It can be done by just one command. Clients also need A c 's IP address. It is not pre-defined in the codes. It is given as a command argument when users run the program client. It is possible to introduce public key system here, a better but complex way. We do not consider it currently. Not Perfectly Isolated from Higher-Level Applications Our goal is to make the AID system completely independent of higher-level applications. It means application level programs do not observe the existence of the AID service. Unfortunately, considering AID layer is not handled in the kernel and it is treated like application layer data, our AID system cannot be perfectly isolated from higher-level application. All outgoing TCP packets are inserted an AID layer header. When servers, which did not join the AID service, get these packets, their application programs will find extra junk data, the AID layer header we appending. Network communication follows protocols. The AID layer header is useless to the application programs. Protocols may be violated and the communication will fail. Before connecting servers that did not join the AID service, the module clientFilter.o should be unloaded first. The Maximum Transmission Unit (MTU) Problems Akin to the last issue, some more problems are caused by inserting an AID layer header that is not handled by kernel. We add 28 bytes into packets entering the AID tunnels and 4 bytes into the other packets. Is it harmless to enlarge a packet like this?

PAGE 34

22 The answer is not always true. Usually, the OS tries to buffer enough data to form big packets to avoid small size packets by Nagle algorithm. Every packet has headers, if only a few data inside, the ratio of headers in a packet goes high and it is inefficient. If we telnet or ssh a server, it may be fine. If we ftp or sftp a server, the kernel will try to buffer as many data as possible. Usually, a packet's size is as large as MTU. If we add an AID layer header in a packet in this case, the packet's size will be larger than MTU. The packet will be just dropped. We noticed this problem when testing ftp service in the AID system. There are two ways to solve the problem. One is to disable the Nagle algorithm, and the other is to strict the size of packets from higher-level applications. After opening a TCP socket, we have a chance to set socket options by calling the function setsockopt() in C library. We can pass in the option TCP_NODELAY to disable the Nagle algorithm or the option TCP_MAXSEG to change the maximum segment size for outgoing TCP packets. We chose the second way to keep the efficiency brought by the Nagle algorithm. It is another cause that the AID system is not totally isolated from higher-level applications. Most application programs do not strict the size of outgoing TCP packets. They fully take the advantage of Nagle algorithm. If programs tend to buffer data used in the AID system, ftp clients for example, most of packets cannot be sent out because of the huge size. It is easy to fix by setting the socket option TCP_MAXSEG. However, the application programs need to be recompiled. It can be fixed as well if we handle the AID layer in the kernel, but we need to recompile kernel though.

PAGE 35

CHAPTER 5 SERVER END On the server end, it needs to tell where packets come from, the Internet or AID tunnels. The server end also has to alert AID stations if it is attacked. ServerFilter.c and server.c are two main source files for server ends. ServerFilter.c is compiled as a module, queuing interested packets into userspace. Then, server.c takes queued packets out and does whatever is necessary. After a server registers at an AID station, A s it can get a secret key. The key is used to verify the communication between the server and A s are not modified by the third party. The server also keeps A s 's IP address. If it is under attacks, A s will be informed. When a registered server is attacked, it will send PUSH messages to the AID station, which the server registered at. This AID station called A s All UDP messages in the AID system are sent to port 4369. However, there is no program listening on this port. UDP packets to port 4369 are handled by the program server. We can characterize occurrences of DoS attacks in several ways. In our AID system, we use arrival rates, average incoming bytes per second, to determine if a server is attacked. Each server has its capacity, 10000 bytes per second for example. When the arrival rate is higher than its capacity, the server is under attacks. We can include other definitions of DoS attacks into the AID system easily in our implementation. Module ServerFilter.o By compiling serverFilter.c, we can get the module serverFilter.o. It hooks on handling functions at NF_IP_PRE_ROUTING, and NF_IP_POST_ROUTING. At 23

PAGE 36

24 NF_IP_PRE_ROUTING, all incoming TCP packets are queued, except for the local traffic. Local traffic goes from loopback interface, 127.0.0.1, to loopback interface. At NF_IP_POST_ROUTING, only UDP packets to the port 4369 are queued. Other outgoing traffic is not related to the AID system. Only incoming TCP packets are queued since currently only TCP traffic is protected. When the module serverFilter.o is loaded in a host, the host must run the program server as well. Otherwise interested packets keep getting into the queue, but no program takes them out of the queue. The traffic is blocked if this happens. A host should load the module serverFilter.o and run the program server at the same time. It is meaningless to do just one of them. Program Server By compiling server.c and linking other relative source files, we can get the executable program server. It has a while loop in main() whose condition is always true. The program server has two packet buffers, one for packets from the Internet and the other for packets from AID tunnels. The former is called bufferN and the latter is called bufferT. Because packets from AID tunnels have higher priority, the program server intends to handle packets in bufferT first. Let us see what the program server does to packets from different hooks. Packets from NF_IP_PRE_ROUTING In this hook, the program server has to remove the AID layer header from every queued packet. Before doing this, program server needs to know if the packet is from the Internet or AID tunnels. The program server inspects the recognizing field. If it is 0, the packet is from the Internet. If it is the server's IP (IP of the host runs the program server),

PAGE 37

25 the packet is from the AID tunnels. If neither 0 nor the server's IP, the packet has wrong contents and is dropped. If the packet is from the Internet, it is put into the bufferN. It is dropped if bufferN is full. The 4-byte-long recognizing field is removed from the packet. If the packet is from AID tunnels, the whole AID layer header is 28 bytes long, including the recognizing field, md5 digest and virtual clock timestamp. First, the program server inspects the packet's integrity with md5 digest. If failing, drops the packet. Then, the packet is put into the bufferT. Drops the packet if bufferT is full. Similarly, the whole 28-byte-long AID layer header is removed from the packet in the program server. Fig. 5-1 shows how an AID layer header is removed for an incoming TCP packet. A B Figure 5-1. Removing the AID layer header in server end. A) All incoming TCP packets should have an AID layer header. Other parts of a packet are not tainted. B) A packet from AID tunnels has a 28-byte-long AID layer header; otherwise its AID layer header is 4 bytes long.

PAGE 38

26 The program server does exactly contrary things to what the program client dose to packets from hook NF_IP_POST_ROUTING. The program client adds AID layer headers on packets, and they are ripped out here. As a result, higher-level applications have no idea of the existence of AID layer. When they get a packet, they see no data of an AID layer header. Packets from NF_IP_POST_ROUTING All packets in the queue grabbed at hook NF_IP_POST_ROUTING are UDP traffic to port 4369. The only UDP message belongs the AID system (to the port 4369) that would be sent out by a server end is PUSH. PUSH messages are sent to A s to say the server is attacked. Before leaving a server, these queued packets will be appended 16-byte-long md5 digest. It prevents the third party from forging PUSH messages and send them to A s Implementation Issues No Threads In the chapter AID System Overview, we pointed out no threads or child processes in programs client, server and AID for printing out statistic information under users' requests. Unlike the program client, the program server has one more thing to handle, sending packets in bufferN and bufferT to higher-level applications. The program server also has a while loop with a consistent true condition in main(). In every iteration, the while loop examines two things. First, sees if there are packets in bufferT and bufferN. A part of them are passed to higher-level applications. Second, sees if any packet was queued by the module serverFilter.o and read in one packet. Each of them takes only little time, so they look like running simultaneously. Threads make a program harder to

PAGE 39

27 maintain and may cause serious problems like resource competition and deadlocks, if not used very carefully. Important Variables There are some important variables defined in the source file server.c. They decide the way the program server works. To make the program server work properly, they need to be assigned reasonable values. We discuss them below. PCKBUFSIZET BufferT's size, if too small, packets from AID tunnels will be dropped frequently with heavy incoming traffic. BufferT stores packets from AID tunnels. PCKBUFSIZEN BufferN's size, if too small, packets from the Internet will be dropped frequently with heavy incoming traffic. BufferN stores packets from the Internet. IPQREADTIME In the section No Threads, we said two things are done every iteration in the while loop of main(). One of them is to read in a packet queued by the module serverFilter.o if the queue is not empty. If the queue is empty, the program server is blocked until a packet is put into queue by the module serverFilter.o. If blocked, the packets in the bufferN and bufferT cannot be sent to their destination, higher-level applications. Consequently, we need to constrain the time of this reading behavior. Do not wait more than IPQREADTIME microseconds if the queue is empty. If it is larger than 500000, the client side may feel painful lags. READINTERVAL BufferT and bufferN are examined every iteration in the while loop of main(), and some packets in the two buffers are sent to higher level applications. We are not sure

PAGE 40

28 how long one iteration may take. We may want packets stay in the buffers longer than the time of one iteration because we need to balance the traffic from AID tunnels and from the Internet (packets from AID tunnels have higher priority). It can be done by this variable. It defines how often packets in the two buffers are sent out. It is in microseconds, too. It cannot be too small or the program server cannot control the traffic. If the variable is too big, apparent lags appear. SENDPCKBUFNO Every READINTERVAL microseconds, packets in two buffers are taken out, but how many? The variable is the answer. If it is 10, the 10 packets from the two buffers can be sent out. Notice it is a total number for packets from both bufferT and bufferN. Since bufferT has higher priority, if 10 packets are picked up this iteration from bufferT, packets in bufferN have to wait until next iteration. If it is too small, the two buffers gets full easily. Packets will be dropped frequently when traffic is heavy. AVGINTERVAL The program server calculates the arrival rate every AVGINTERVAL seconds. If it is too small, the arrival rate may not be representative. If it is too big, the program server may not be able to detect attacks in real time (not sensitive enough). TOTALCAP Capacity of the server end, in our AID system, was defined as how many bytes per second the server end can handle. Once the arrival rate is higher than TOTALCAP, the program server alerts the AID system to create a tunnel tree by sending PUSH messages to A s

PAGE 41

29 RESERVEDTIMES When under attacked, a server has traffic from both the Internet and AID tunnels. We said the latter has higher priority, but how? The server handles data in bufferT and in bufferN with the ratio RESERVEDTIMES: 1. For instance, if TOTALCAP is 1000 bytes per second and RESERVEDTIMES is 4, 1000 4/(1+4) = 800 bytes per second is reserved for the traffic from tunnel trees, and 1000 x 1/(1+4) = 200 bytes per second is reserved for the traffic from the Internet. How the Registration Is Done In our AID system, servers get the secret key shared with A s by registration. We just took the easiest step here. The shared key was pre-configured in both the server and AID station, A s If users want to change the key, they need to redefine it in both sides, with the same value. Then recompile the codes. It is not hard. We wrote a makefile compiling and linking object files to generate the program server. It can be done by just one command. Not Perfectly Isolated from Higher-Level Applications Same as the program client, the program server cannot be totally isolated from higher-level applications because the Linux kernel does not actually handle AID layer headers. AID layer headers are viewed as application layer data by the kernel. The program client inserts an AID layer header into a packet and the program server removes the AID layer header from the packet. It is a little confusing here. Is the program client different from other client programs like telnet and ssh? Our program client does not try to connect the server host. It takes care of queued packets in a client host instead. Likewise, the program server is not a server daemon program. It does not listen on a

PAGE 42

30 port. Its job is taking care of queued packets in a server host. When a client host tries to connect a server host, there are four possibilities: The client host has clientFilter.o loaded and is running the program client, and the server host has serverFilter.o loaded and is running the program server as well. It works just fine in this case because both sides can recognize the AID layer headers inserted in packets. The client host has clientFilter.o loaded and is running the program client, but the server host does not load serverFilter.o. It does not work in this case because the server host will get packets with AID layer headers from the client host, and the server host cannot recognize them. AID layer headers are junk data for the server host, which make communication fail. The client host does not load clientFilter.o, but the server host has serverFilter.o loaded and is running the program server. It does not work in this case either because the client host sends out packets without AID layer headers. When the server host gets the TCP packets from the client host, it tries to know if the packets are from AID tunnels or the Internet by inspecting the recognizing field in AID layer headers. Of course, these packets do not have the recognizing field and application layer data is used as recognizing field. Then, communication fails. The client host does not load clientFilter.o and the server host does not load serverFilter.o either. Both sides know nothing about AID layer headers. It works well. In this case, the AID service has nothing to do with both sides. Communication just goes as without the AID service protection as before. In conclusion, if a client host wants to connect a server host that has serverFilter.o loaded and is running the program server, the client host should load the module clientFilter.o and run the program client before making a connecting. On the other hand, if a client host wants to connect a server host that does not load serverFilter.o, the client host should unload the module clientFilter.o and terminate the program client before making a connection. The client host should match the server host to make everything go well. One thing worth a mention is that loading clientFilter.o and running the program client do not mean the client host already joined the AID service. It should register at an AID station to make the AID service effective first. Same thing applies to the server host

PAGE 43

31 as well. However, an unregistered client host can still connect to a registered server host by loading clientFilter.o and running the program client. All packets from that client host cannot enter AID tunnels because the client has no secret key. They can only be transmitted via the Internet.. Loading or unloading the module clientFilter.o can be done by one command. A client host can adapt itself to different server hosts dynamically. Program Alert By compiling the source file alert.c and linking other relative source files, we can get the executable program alert. A server host can send PUSH messages to AID stations by executing the program alert. We leave the flexibility of defining DDoS attacks to the users. Users can define the situations of being attacked to meet their need. All they need to do is to run the program alert when the server host detects attacks. It will send A s a PUSH message to trigger the AID system. Afterward, all packets from registered clients to that server go through the AID tunnels. The program server inserts Md5 digest into the PUSH message packets at hook NF_IP_POST_ROUTING.

PAGE 44

CHAPTER 6 AID STATION AID stations are the cores of our AID system [6] They form an AID tunnel tree for each attacked registered server. How is a tunnel tree created? How are packets routed in a tunnel tree? How to resist attacks from registered clients? How was it implemented? Answers to the above questions are in this chapter. AID Tunnel Tree We have seen AID tunnels many times in the previous chapters. We know AID tunnels are tree structures. Packets from registered clients to the attacked registered servers would enter AID tunnels. In this section, we explained how an AID tunnel tree is constructed and how packets are routed inside. The push-n-pull process [6] establishes a tunnel tree from the registered clients to an attacked server. Push Phase Assume a server S is attacked; it sends a PUSH message to A s the AID station that it registered at. An AID tunnel tree for the server is going to be built up, and the tree's root node is A s The scenario is as follows. 1. Server S senses an attack and sends PUSH messages to A s A s is the root node of the AID tunnel tree, which called the first level node. 2. A s is the only AID station that knows S is under attacks so far. A s picks up k other AID stations randomly, and sends a PUSH message to each of them. Any other AID station could be selected. Subsequently, k+1 AID stations know S is under attacks at the end of the step. We call these k AID stations the second level nodes in the tunnel tree. How big should k be? We have deep discussion about it later. 3. Every second level node randomly picks up k other AID stations, and sends a PUSH message to each of them. Any other AID stations could be selected except 32

PAGE 45

33 for A s So, the k second levels nodes selected k 2 nodes totally. We call these k 2 nodes the third level nodes in the tunnel tree. Notice that a node might be picked up more than once in the step 2 and in step 3 because both steps randomly select k AID stations. Fig. 6-1 shows how is a tunnel tree created in push phase. Figure 6-1. Tunnel tree created in push phase. Not all third level nodes are shown in the figure. Arrows from nodes to nodes indicates the direction packets would be routed. See the broken arrows in the figure. Node A got PUSH messages from both A s and B. A would choose A s as its parent node because of shorter routing path. Similarly, node D got PUSH messages from B and A. D could pick either of them to be its parent node, but not both. In push phase, there might be some AID stations not receiving PUSH messages, which are unconnected nodes in the figure. Suppose we have N AID stations. We want to notify every AID station when a server is attacked. In step 1, only A s is notified. In step 2, k+1 AID stations are notified. In step 3, ideally, 1+k+k 2 AID stations are notified. If k is the square root of N, we get 1+k+k 2 > N, which means the tunnel tree covers every AID station. However, some AID

PAGE 46

34 stations picked in step 2 may be picked again in step 3 and some second level nodes may select the same third level nodes. We cannot guarantee every AID station is included in the tunnel tree. That is why we need pull phase. In push phase, k(k+1) PUSH messages are sent out totally, because only the first level node and the second level nodes would send PUSH messages. If we allow the third level nodes to send PUSH message, push phase will be expensive. The majority of AID stations can be reached in push phase. Pull Phase When a server detects an attack, a tunnel tree rooted at A s is built up. Some nodes might not get the PUSH messages in push phase. These nodes did not connect to the tunnel tree yet. We try to include them into the tunnel tree in pull phase. In pull phase an AID station will ask other AID stations by sending PULL messages what servers are attacked. In push phase, an AID station gets information of attacked server passively. If an AID station B gets PULL messages from another AID station A, B will send PULLANS messages back to A. PULLANS messages contain information of all attacked servers B knows. When A gets these PULLANS messages from B, it will update its attacked servers recording. Actually, an AID station sends PULL messages to q other AID stations. Like the variable k in push phase, we should choose a proper q. We will discuss q and k later. Each AID station sends PULL messages out periodically. Routing A s is the first AID station that knows the register server S is attacked, and A s is also the root of the tunnel tree for the server S. When an AID station A tells another AID station B that server S is attacked by either PUSH or PULLANS messages, A becomes B's parent node in the tunnel tree for the server S. Hence, A s is the parent node of the second level nodes, and the second level nodes are the parent nodes of the third level nodes.

PAGE 47

35 The structure of a tunnel tree changes dynamically because of PULLANS messages. In pull phase, if a third level node gets PULLANS messages from A s the only first level node, it will switch its parent to A s and become the second level node. Then, the routing path becomes one AID station shorter. A s is the root node, all TCP packets to the server S are routed to A s finally. If an AID station gets a packet whose final destination is server S, where the packet is routed to next? The AID station routes the packet to its parent node. The third level nodes route packets to the second level nodes, and the second level nodes route packets to A s Packets go from leaf nodes to the root node in the tunnel tree. Finally, A s forwards packets to the server S. An AID station could be a tree node of more than one tunnel trees. If N servers are attacked, there will be N tunnel trees built up on the random overlay network (RON). One tree is independent from another. Why Does a Tunnel Tree Try to Include Every AID Station? Two reasons here, first, a client is free to register at an arbitrary AID station. Suppose server S is attacked and client C is trying to connect S. C registered at AID station A c A tunnel tree would be created for server S. If the tunnel tree does not embrace A c A c does not know S is under attacks. In this case, C would not be informed by A c that S is attacked. Thus, client C keeps sending packets to server S via the Internet. These packets are not protected even though C did register and joined the AID service. We do not constrain which AID station can have registered clients, so we tries to include all AID stations. Second, it is about routing. AID station A routes packets to AID station B if B is A's parent node in the tunnel tree. A got PUSH or PULLANS messages from B before. B

PAGE 48

36 also routes these packets to its parent node. If B crashes and then restores, it will lose information about its parent node of the tunnel tree. Now, B does not know where to route the packets from A. B sends PULL messages to others when getting a packet that it does not know where to route. If one PULLANS message B got contains information it needs, where to route packets to server S, B hooks on the tunnel tree again. However, it is possible that all PULLANS messages B got contain nothing about server S. If we choose right k and q, the chance that this happens is very low. Variables k and q Assume we have a set of n AID stations. AID stations A, B and A s are elements of the set. In step 2 of push phase, A s sends PUSH messages to other k AID stations. The probability that A does not get the PUSH message from A s is 11nkn Now, suppose AID station B got a PUSH message from A s and is the second level node. In step 3, B would not send PUSH messages to A s and itself, so B could send PUSH messages to other k nodes out of (n 2). The probability that A does not get the PUSH message from B in step 3 of push phase is 22nkn Since there are k second level nodes, the probability that A does not receive any PUSH messages in step 3 of push phase is knkn22 A does not connect to the tunnel tree right after push phase if it obtained no PSUH message in step 1, 2 and 3. Finally, we got the probability that A is not covered by the tunnel tree after push phase is knknnkn2211 So, the probability, called P InTree that an arbitrary AID station A could be in the tunnel tree after push phase is

PAGE 49

37 knknnkn221112>n The expectation number of AID stations included in the tunnel tree after push phase is nP InTree. 1+ e1 e1 < < n Theorem 1: if and n= k, then P InTree e11> Proof: enknnknk112211> 1 enknnknk12211< enknk111< nknk< nkk1< ennn11 enn111 nn11 is a monotonically-increasing function and is equal to e1 when n is approaching infinity. Hence, if and 2> nk= P InTree e11> We use n= k in our AID system. Let us talk about variable q now. =s nP InTree AID stations are expected to be covered in the AID tunnel tree right after push phase and we know nP InTree >en11 by Theorem 1. An AID station sends out PULL messages to q other AID stations. The probability that at least one of these q AID stations is in the tunnel tree is

PAGE 50

38 [] qqqsenennn1111111111>> (1) If q = 10, the probability is greater than 0.99995. It is high enough that we can almost say it will happen. When an unconnected AID station receives PULLANS from another AID station that is in the tree already, it connects to the tree. With push-n-pull process, the chance that all AID stations are included in the tree is very high. Advantages of Random Overlay Network (RON) First, small diameter and modest nodal degree: A good overlay network topology should have small diameter and nodal degree. Unfortunately, they conflict with each other. We made a tradeoff in our RON topology. We have a fixed small diameter that is three and modest nodal degree that is about the square root of the number of all AID stations. With small diameter, packets can arrive at servers by passing few AID stations. With modest nodal degree, we save some resource and keep the availability against node failure. We explain why the diameter is three in the end of this chapter. Second, easy to set and maintain: A tunnel tree is established by sending PUSH and PULL messages. They are sent randomly by an AID station to other AID stations. We do not need a complicated algorithm to create the tree. Besides, every tree node just has to know who is its parent to forward packets. Not many things need to be remembered by an AID station. Capacity of the AID system can be increased by adding more AID stations. Third, against node failure: If a tree node, an AID station, is down somehow, its children nodes are disconnected from the tunnel tree. However, the children nodes can hook on the tree again in next pull phase. Therefore, we can easily shutdown an AID station for maintenance without affecting the entire AID system. It is also true for adding

PAGE 51

39 an AID station. A new-added AID station can connect the tree in pull phase as well. An AID station can join and leave the AID service with little damage. Distributed Virtual-Clock Packet Scheduling Basic Idea Assume we have an attacked server S and a tunnel tree for S. Every AID station maintains a virtual clock VC u [11] (initialized to be the local system time) for every tunnel u connecting with a client network. When an AID station gets a packet from tunnel u, VC u is updated as follows [6] VCu = max {VCu, current_time} + T L (2) The AID station then marks the packet's virtual clock timestamp as VCu. All AID stations local clocks should be synchronized. L is the length of the packet. T is called waiting interval. A s broadcasts a new T to all AID stations periodically by sending CTRLT messages. We use T to control the speed of a virtual clock. Since T can be changed dynamically, we can adjust a virtual clocks speed dynamically as well. The maximum rate a client can send data to server S via RON is 1/T. If an AID station gets a packet from tunnel u connecting to another AID station, the packet has a timestamp on it already. An AID stations puts all incoming TCP packets into a buffer in ascending order based on their virtual clock timestamps. When the buffer is full, the packet with largest timestamp will be dropped. We call a packets virtual clock timestamp VCTS. If VCTS the AID stations local time > is true for an incoming packet, the packet is just dropped, not put into the buffer even though the buffer is not full. The value of can be configured in the program. If a registered client hosts an attacker, its virtual clock will run very fast because of huge amount of traffic. Most of packets from the client will be

PAGE 52

40 dropped since their virtual clock timestamps are too big. In this way, server's capacity is shared fairly among all clients. How to Adjust T Server S reserves part of its capacity, called C s for RON. T is set to 1/ C s at first and broadcasted to all AID stations by A s There are two phases to adjust T. In each phase, new T is broadcasted to every AID station. Exponential phase: In this phase, A s doubles the value of T to make virtual clocks run twice faster. When virtual clocks run twice faster, the maximum arrival rate of server S from RON is cut by half. A s keeps in exponential phase until the arrival rate is below C s Then, A s enters linear phase. Linear phase: Suppose T is changed from I to 2I by the last update of T in exponential phase. In linear phase, A s decrease T by I periodically to slowdown virtual clocks until arrival rate is above C s We call the system converges at the moment. Then, A s may enter exponential phase again. Programs for an AID Station AID stations route packets from registered clients, form a tunnel tree for an attacked registered server and control the traffic flows of tunnel trees. AIDFilter.c and AID.c are two main source files for AID station. AIDFilter.c is compiled as a module, queuing interested packets into userspace. Then, AID.c takes queued packets out and does whatever is necessary. An AID station keeps information about its registered clients and servers. An AID station routes TCP packets in a tunnel tree, and uses UDP messages to control the AID system. Module AIDFilter.o By compiling AIDFilter.c, we can get the module AIDFilter.o. It hooks on handling functions at NF_IP_PRE_ROUTING, and NF_IP_POST_ROUTING. At NF_IP_PRE_ROUTING, all incoming TCP packets and UDP packets to port 4369 are queued, except for the local traffic. Local traffic goes from loopback interface, 127.0.0.1,

PAGE 53

41 to loopback interface. At NF_IP_POST_ROUTING, all outgoing TCP packets and UDP packets to port 4369 are queued, except for the local traffic. Queued TCP packets are traffic inside tunnel trees. They come from registered clients and head for registered servers. When the module AIDFilter.o is loaded in a host, the host must run the program AID as well. Otherwise interested packets keep getting into the queue, but no program takes them out of the queue. The traffic is blocked if this happens. A host should load the module AIDFilter.o and run the program AID at the same time. It is meaningless to do just one of them. Program AID By compiling AID.c and linking other relative source files, we can get the executable program AID. It has a while loop in main() whose condition is always true. The program AID has a packet buffer, storing incoming TCP packets. TCP packets from NF_IP_PRE_ROUTING Every queued incoming TCP packet in an AID station will go through the following processes. Verify the third party did not alter the packet. The packet is dropped if md5 digest stored in the packet is not equal to the one calculated by the AID station. Packet's virtual clock timestamp is refreshed as described in the section Distributed Virtual-Clock Packet Scheduling Examine the packet's virtual clock timestamp. If VCTS the AID stations local time > the packet will be dropped. Constant was defined as MAXVCTSEXCEED in AID.c. Most of offending packets are filtered out here. The AID station look up its routeList to know where to route the packet. Every AID station has a routeList, which is a list structure. A routeListNode contains information for a tunnel tree, inclusive of server's IP, distance to A s and the parent node's IP. An AID station may be embraced in more than one tunnel trees, and its routeList will contain more than one routeListNode. If The AID station does not know where to route the packet, no information for the destination server stored in

PAGE 54

42 the routeList, the AID station will drop the packet and send PULL messages to other AID stations. If a packet passes all of the above and the packet buffer is not full, it can be put into the packet buffer. If the packet buffer is full, the packet with maximum virtual clock timestamp in the packet buffer is selected. The incoming packet and selected packet are compared in their virtual clock timestamps. If the incoming packet has smaller timestamp, it can replace the selected packet in the packet buffer; otherwise, it will be just dropped. The packet's destination IP is modified to the parent node's IP, since the packet will be routed to the parent node in the tunnel tree. UDP packets from NF_IP_PRE_ROUTING Every queued incoming UDP packet in an AID station will go through the following process. Verify the third party did not alter the packet. The packet is dropped if md5 digest stored in the packet is not equal to the one calculated by the AID station. Recognize what kind of UDP message the packet is. In the chapter AID Layer, we said there are a couple of different UDP messages in the AID system. We can tell it by the packet's packet type field in the AID layer header. If it is a PULL message, the AID station sends whole information of its routeList in PULLANS messages to the asking AID station. It happens in pull phase. If it is a PULLANS message, the AID station updates its routeList with the data of the packet. It happens in pull phase. If it is a PUSH message, the AID station updates its routeList with the data of the packet and sends PUSH message to other AID stations. It happens in push phase. If it is a CTRLT message, the AID station updates the variable T for the specific tunnel tree. A CTRLT message contains IP of the server that the tunnel tree is for. See the chapter AID Layer for more details about different UDP messages. A UDP message has md5 digest. It cannot be forged without knowing the secret key. TCP packets from NF_IP_POST_ROUTING The AID station is going to forward every queued packet to its parent node. The destination IP was changed to the parent node's IP in the hook NF_IP_PRE_ROUTING already. Here, the program AID recalculates md5 digest because the packet's destination IP and virtual clock timestamp were changed. Finally, the program AID computes the checksums in the TCP header and IP header.

PAGE 55

43 UDP packets from NF_IP_POST_ROUTING Every queued UDP packet in this hook heads to port 4369. Md5 digest need to be calculated and inserted into every queued UDP packet. The checksums in the TCP header and IP header are recomputed in this hook. Afterward, the packets are ready to leave the AID station. Implementation Issues No Threads In the chapter AID System Overview, we pointed out no threads or child processes in programs client, server and AID for printing out statistic information under users' requests. Besides getting a packet from the queue, the program AID has three more things to do, sending packets in the packet buffer to higher-level applications, sending out CTRLT messages and sending out PULL messages. The program AID also has a while loop with consistent true condition in main(). In every iteration of the while loop, four things are examined. First, sees if the packet buffer has packets waiting and passes some packets to higher-level applications. Second, sees if it is time to send out CTRLT messages. We can define how often an AID station can send out CTRLT messages. Third, sees if it is time to send out PULL messages. We can also define how often an AID station can send out PULL messages. Fourth, sees if any packet was queued by the module AIDFilter.o and read in one packet. Each of them takes only little time, so they look like running simultaneously. Threads make a program hard to maintain and may cause serious problems like resource competition and deadlocks, if not used very carefully.

PAGE 56

44 Registration for Clients and Servers As we said before, the secret keys shared with clients and servers are pre-configured in the codes. If an AID station wants to register a client, the client's information needs to be added into AID.c. It is also true for registering a server. The program AID has to be recompiled after registration, which can be done by one command. Important Variables There are some important variables defined in the source file AID.c. They decide how the program AID works. To make the program AID work properly, they need to be assigned reasonable values. We discuss them below. PCKBUFSIZE The packet buffers size, if too small, incoming TCP packets will be dropped frequently with heavy traffic. IPQREADTIME We mentioned four things are done every iteration in while loop of main(). One is to read in a queued packet. If the queue is empty, the program might be blocked until a packet is queued by the module AIDFilter.o. If blocked, the packets in the packet buffer cannot be got by higher-level applications. Consequently, we need to constrain the time of this reading behavior. Do not wait more than IPQREADTIME microseconds if the queue is empty. If it is larger than 500000, the client side may feel painful lags. READINTERVAL The packet buffer is examined every iteration in the while loop of main(), and some packets are sent to higher-level applications. We are not sure how long one iteration may take. We may want packets stay in the packet buffer longer than the time of one

PAGE 57

45 iteration. It can be done by this variable. It defines how often the packet buffer is checked. It is in microseconds, too. SENDPCKBUFNO Every READINTERVAL microseconds, packets in the packet buffer are taken out, but how many? The variable is the answer. If it is 10, the 10 packets from the packet buffer can be sent to higher-level applications. If it is too small, packets will be dropped frequently when heavy traffic. NEARBYAID The number of other AID stations that are known by this AID station. PUSH, PULL, and CTRLT messages are sent to these neighbors. SNEDTINTERVAL If the AID station is A s a root node of a tunnel tree, it sends CTRLT messages to all other AID stations every SENDTINTERVAL seconds. SENDPULLINVAL Every SENDPULLINVAL seconds, an AID station sends PULL messages to other PULLNO (defined in global.h) AID stations. Periodically sending out PULL messages can make sure every AID station connects tunnel trees. DECREASERATIO In linear phase, A s decreases T by I DECREASERATIO is MAXVCTSEXCEED If a packet's VCTS AID stations local time > MAXVCTSEXCEED, the packet is dropped. MAXVCTSEXCEED is the constant

PAGE 58

46 Adding New AID Stations A new added AID stations can be included into an AID tunnel tree by sending PULL messages to others. However, we still need to make the AID station known by all other AID station. Like handling registrations, an AID station pre-configures its neighbors in AID.c. When a new AID station is added into the AID system, its neighbors AID.c needs to be updated and recompiled. It can be improved in a better but complex way. Diameter of a Tunnel Tree Remember the AIDNO field of a PUSH message and distance field of a server list node of a PULLANS message? Actually, they two mean the same thing, the distance to A s of a tunnel tree. We explain why our random overlay network's diameter is three here. Every tree node, an AID station, records its distance to A s For A s itself, the distance is 0. For the second level nodes, it is 1. For the third level nodes, it is 2. Suppose we have tree node A with distance of 1, tree node B with distance of 2 and tree node C not included in the tree yet. C has two ways to join the tunnel tree. Another node sends a PUSH message to C. It could be root A s or node A. If C gets PUSH messages from A s C's distance to A s is 1. If from A, C's distance to A s is 2. Notice only the root node and the second level nodes can send PUSH messages or push phase becomes expensive (more than k(k+1) PUSH messages sent out). C gets PULLANS messages from another AID station. It could be root node A s node A or node B. If C gets PUSH messages from A s C's distance to A s is 1. If from A, C's distance to A s is 2. If from B, C's distance to A s is 3. Assume we have another node D not included in the tree. D can join the tree by PUSH or PULLANS from node A or node B, but not node C. D ignores PULLANS message from the nodes with distance of 3. If D joins the tree by C's messages, C becomes D's parent node. That means D has distance of 4 to A s which is not allowed.

PAGE 59

47 If C's distance is 3 and it gets PULLANS or PUSH messages from A, C will switch its parent node to A to have smaller distance, 2. C becomes the third level node after switching. An AID station's distance to A s can only go smaller. By doing this, no cycle appears in a tunnel tree. As a result, distance between A s and every other tree nodes is not larger than 3. Actually, we can reset the diameter by redefining PUSHDEEP in file global.h. A tunnel trees diameter is PUSHDEEP+1. There are four possibilities of a packet being routed from a registered client to a registered server, shown in Fig. 6-2. Figure 6-2. Four possibilities a packet can be routed. The numbers are the value of AIDNO field of PUSH messages or distance field of PULLANS messages sent by the host. Packets are routed toward A s in an AID tunnel tree. Forwarding Packets Pay attention to the word forwarding. An AID station works like a router somewhat. Packets in RON are forwarded to A s by AID stations. The difference is normal routers do not adjust md5 digest or virtual clock timestamp of a packet. Usually, the forwarding function is turned off in a Linux machine by default. It has to be on. After version 2.4, the tool iptables is available in Linux. We use it to set forwarding rules in an AID station. To be simple, no sophisticated rules are used. We just allow all kind of forwarding. Of course, we can set more secure and elaborate forwarding rules. An

PAGE 60

48 AID station is not allowed to send out its own TCP packets. It can only forward TCP packets.

PAGE 61

CHAPTER 7 TESTING RESULTS AND ANALYSIS After going through the previous chapters, we understand how the AID system works theoretically and practically. In this chapter we talk about how we tested our AID system and analyze testing results. When an idea is transformed into real programs, unexpected problems show up always. We already discussed some of them in Implement Issues sections of previous chapters. The other practical problems are illustrated in this chapter. Important Issues about Testing The root access is required to load a module. A client host, a server host or an AID station needs to load clientFilter.o, serverFilter.o and AIDFilter.o respectively. We do not have enough Linux machines with the root access for tests. As a consequence, we just tested our AID system on three Linux machines, for a client host, a server host and an AID station separately. We might not test some functionality well with such a simple model. Since only one Linux machine is for client hosts, we have to simulate n client hosts on it by running n client processes. The AID station treats TCP packets from different source ports as they are from different hosts, even though they have the same source IP. A client process uploading a huge file to the server symbolizes an attacker. However, normal ftp programs cannot be used because of MTU problems. Packets sent out by a normal ftp client could be as big as MTU. There is no space for the AID layer header. We discussed this problem in the chapter CLIENT END in detail. Hence, we wrote two programs for this purpose, testServer and testClient. The server runs the program testServer to accept connections, and the client runs the program testClient to dump data to the server. In the program testClient, we can restrict the size of packets sent to testServer by resetting the socket option. If the packet size is smaller, testServer will get more packets (but same amount of application layer data). What we want to see from the testing are how fast the AID system converges, how T (waiting interval) changes, how many packets from legitimate users are dropped, 49

PAGE 62

50 how many packets from attackers are dropped, how T affects the average arrival rate, and etc. Many factors can influence the above behaviors, for example the packet buffer's size. Most of these factors are malleable variables in the codes. Testing Elements We have five testing cases. Client Linux machine had several processes of the program testClient to simulate more than one client ends. Each process of the program testClient might be given different parameters. Given parameters decided if a client end was a legitimate user or an attacker. Program TestClient Usage of program testClient is testClient testServerIP blockSize blockNO MAXSEG sleepTime TestClient is the filename of the executable. TestServerIP is the IP address of the host that runs the program testServer. TestClient will dump data there. The remaining four parameters are more meaningful. There is a for loop, which runs blockNO iterations in the program testClient. In every iteration a block whose size is blockSize is sent to testServer. It indicates the size of application layer data, not the whole packet. Because of headers, more than blockSize bytes are sent out in an iteration. With blockSize and blockNO, testClient knows how many application layer data it needs to send out, which are blockSize blockNO bytes. The parameter MAXSEG defines the maximum size of TCP packets. Notice that the size of the whole packet, including IP header, will be a little bit bigger than MAXSEG. We need a suitable MAXSEG to save enough space for an AID layer header. The last parameter, sleepTime, defines how many seconds the program testClient sleeps before running the next iteration.

PAGE 63

51 Program TestServer TestServer accepts connection requests from testClient and prints out application layer data sent by testClient. Setting of the AID System We have three Linux machines, one client, one AID station and one server. Table 7-1 shows the basic settings we used for testing. We explained what these factors mean in previous chapters. Servers capacity was 2000 bytes per second. 1600 bytes per second of it was reserved for traffic from AID tunnels. The setting was fixed during testing but the number of attackers and legitimate users varied. The attacking modes changed in different testing cases too. Table 7-1. List of important factors of the AID system for testing Name Value PCKBUFSIZET 50 PCKBUFSIZEN 50 IPQREADTIME 300000 READINTERVAL 300000 SENDPCKBUFNO 3 AVGINTERVAL 10 TOTALCAP 2000.0 RESERVEDTIMES 4.0 MAXVCTSEXCEED 4 DECREASERATIO 0.1 Case 1 There were two registered attackers. Their parameters were: Attacker1: testClient 192.168.1.102 1000 500 1000 0 Attacker2: testClient 192.168.1.102 1000 500 1000 0 Fig. 7-1 shows how many packets were dropped because of big VCTS at the AID station. MAXVCTSEXCEED is 4. Fig. 7-2A shows how variables T, I and decrease changed their values. T was changed from I to 2I by the last update of T in exponential phase and then entered linear phase. In linear phase, A s decreased T by decrease

PAGE 64

52 periodically to slow down virtual clocks. Decrease is equal to DECREASERATIO times I. Fig 7-2B shows the arrival rate at the AID station and the server. AvgT is the arrival rate of the tunnel tree at the server. AvgN is the arrival rate of the Internet at the server. AvgAID is the arrival rate at the AID station. All of them are average received bytes per second in the 10-seconds period. AID cap is part of server's capacity that was reserved for the registered clients. Server cap is server's whole capacity. In the third 10-seconds, avgN exceeded server's capacity, and the AID system was triggered. Two attackers started to send packets via RON, instead of the Internet. We can see avgN went down and finally to 0. Now, we examine Fig. 7-2A and Fig. 7-2B together. The AID system converged after the twelfth 10-seconds. T ranged between 0.000875 and 0.00175 when converging. When T went down, avgAID went up. T kept decreasing to I, in linear phase, until AvgAID was bigger than AID cap. At the moment, the AID system entered exponential phase again. When T doubled, avgAID declined dramatically. When avgAID became smaller than AID cap, the AID system entered linear phase. The AID system prevented avgAID from exceeding AID cap by tuning T. If no new attackers joined, after the AID system converged, T would fall into a fixed range as we can see in Fig. 7-2A. Table 7-2 shows how many packets and why they were dropped. About 2/3 of incoming packets were dropped. Table 7-2. Case 1 packets statistics in the AID station. A packet is dropped when the AID stations packet buffer is full, VCTS AID stations local time > integrity checking fails or the AID station does not know where to route the packet. Attacker1 Attacker2 Packet# in 1294 1230 Packet# out 487 456 BufferFull 0 BufferFull 0 BigVCTS 807 BigVCTS 774 MD5Fail 0 MD5Fail 0 Packet# dropped CantRoute 0 CantRoute 0

PAGE 65

53 Virtual clock timestamp localTime012345671418112116120124128132136140144148152156160164168172176180184188192196110011041108111211161120112411281packet#VCTS attacker1 attacker2 MAXVCTSEXCEED Figure 7-1. Distribution of incoming packets in case 1 at the AID station. Packets lay above the straight line MAXVCTSEXCEED were dropped. controlT (Fig. 7-2A)00.00050.0010.00150.0021611162126313641465156616671768186919610110 secondsvariable's valu e I d arrival rate (Fig. 7-2B)0500100015002000250030001591317212529333741454953576165697377818589939710110510 secondsarrival rate in 10 secon d avgT avgN AID cap server cap avgAID Figure 7-2. How did T and arrival rate interact with each other in case 1. A) After system converged, T pulsed in a fixed range. B) When avgAID exceeded AID cap, T doubled. Then avgAID fell again. Most of time, avgAID was below the yellow straight line, showing the arrival rate was controlled well.

PAGE 66

54 Case 2 Similar to case1, however, we added one legitimate registered client: Attacker1: testClient 192.168.1.102 1000 500 1000 0 Attacker2: testClient 192.168.1.102 1000 500 1000 0 Normal_user: testClient 192.168.1.102 250 700 250 1 The normal user was distinguished from two attackers in three ways. First, it sent smaller amount of data. Second, the size of packets from it was smaller. Third, it slept 1 second every iteration. T ranged between 0.001125 and 0.00225 when converging. It was bigger than in case 1 because we had three registered clients here. In Fig. 7-3, in the same time period, normal_user sent about twice packets as many as an attacker did because TCP had flow control mechanism. After many packets were dropped; attackers would slow down their traffic. In Fig. 7-3, we know packets from the legitimate was really safe because VCTS localTime was in the range of {0.5, 1} approximately, which was far away from 4. Normal_user was influenced by T much less harshly than attackers were. Likewise, Fig. 7-4A and Fig. 7-4B show how the AID system quelled arrival rate by adjusting T. Table 7-3 shows packets statistics of case 2. Table 7-3. Case 2 packets statistics in the AID station. Normal_user had no packets dropped. Attacker1 Attacker2 Normal_user Packet# in 473 474 952 Packet# out 173 172 952 BufferFull 0 BufferFull 0 BufferFull 0 BigVCTS 300 BigVCTS 302 BigVCTS 0 MD5Fail 0 MD5Fail 0 MD5Fail 0 Packet# dropped CantRoute 0 CantRoute 0 CantRoute 0

PAGE 67

55 VCTS localTime0123456714181121161201241281321361401441481521561601641681721761801841881921packet#VCTS-localTime attacker1 attacker2 normal_user MAXVCTEXCEED Figure 7-3. Distribution of incoming packets in case 2 at the AID station. Traffic from normal_user was pretty stable (the lowest series). controlT (Fig. 7-4A)00.00050.0010.00150.0020.002514710131619222528313437404346495210 secondsvariable's value I decrease T arrival rate (Fig. 7-4B)05001000150020002500300014710131619222528313437404346495210 secondsarrival rate in 10 seconds avgT avgN AID cap server cap av g AID Figure 7-4. How did T and arrival rate interact with each other in case 2. When T doubled, avgAID and avgT fell.

PAGE 68

56 Case 3 There were four registered attackers, and two registered legitimate clients. Notice that two legitimate clients sent out different amount of data with different packet sizes. Their parameters were: Attacker1: testClient 192.168.1.102 1000 500 1000 0 Attacker2: testClient 192.168.1.102 1000 500 1000 0 Attacker3: testClient 192.168.1.102 1000 500 1000 0 Attacker4: testClient 192.168.1.102 1000 500 1000 0 Normal_user1: testClient 192.168.1.102 250 700 250 1 Normal_user2: testClient 192.168.1.102 100 700 100 1 T ranged between 0.00175 and 0.0035 when converging. It was even bigger than in case 2 because we had six registered clients here. Normal_user1, having exactly the same parameters as normal_user in case 2, had packets dropped. In case 2, normal_user had no packets dropped. What made the difference to the clients with the same parameters? When the traffic load in the AID system is heavier (T goes bigger), packets have higher chances to be dropped even though they are not from hosts intending to attack. That is because the AID system tries to make a fair share of resource among all registered clients. If a client sends more, it has to wait longer for next sending. Packets from normal_user2 had small enough VCTS, so none was dropped. Table 7-4. Case 3 packets statistics in the AID station. Normal_user1 had more packets dropped, but it also had more incoming packets. Attacker1 Attacker2 Attacker3 Attacker4 Normal_user1 Normal_user2 Packet# in 411 357 388 366 1106 1215 Packet# out 116 97 106 97 636 1215 BufferFull = 0 BufferFull = 0 BufferFull = 1 BufferFull = 0 BufferFull = 0 BufferFull = 0 BigVCTS = 295 BigVCTS = 260 BigVCTS = 281 BigVCTS = 269 BigVCTS = 470 BigVCTS = 0 MD5Fail = 0 MD5Fail = 0 MD5Fail = 0 MD5Fail = 0 MD5Fail = 0 MD5Fail = 0 Packet# dropped CantRoute = 0 CantRoute = 0 CantRoute = 0 CantRoute = 0 CantRoute = 0 CantRoute = 0

PAGE 69

57 Normal_user1 had dropped packets, but it was still distinguished from other true attackers. First, its traffic was not slowed down as much as attackers. In the same time period, an attacker just sent out about 375 packets, but normal_user1 sent out 1106 packets (normal_user2 sent out 1215). Second, in Fig. 7-5 we can see VCTS-localTime for packets from normal_user1 rippled around 4. However, it is 6 for packets from attackers. In conclusion, normal_user1 had packets dropped, but it still maintained its communication with the server. VCTS localTime01234567815110115120125130135140145150155160165170175180185190195110011051110111511201packet#VCTS localTime attacker1 attacker2 attacker3 attacker4 normal_user1 normal_user2 MAXVCTSEXCEED Figure 7-5. Distribution of incoming packets in case 3 at the AID station. Traffic from normal_user2 was pretty stable (the lowest series). Even though some of packets from normal_user1 were discarded, it was still different from real attackers.

PAGE 70

58 controlT (Fig. 7-6A)00.0010.0020.0030.0041611162126313641465156616671768110 secondsvariable's value I decrease T arrival rate (Fig. 7-6B)050010001500200025003000350040001611162126313641465156616671768110 secondsarrival rate in 10 seconds avgT avgN AID cap server cap avgAID Figure 7-6. How did T and arrival rate interact each with other in case 3. When T doubled, avgAID and avgT fell; otherwise avgAID and avgT rose. Case 4 There were two registered attackers, two registered normal users and two unregistered normal users. Their parameters were: Attacker1: testClient 192.168.1.102 1000 500 1000 0 Attacker2: testClient 192.168.1.102 1000 500 1000 0 Normal_user1: testClient 192.168.1.102 250 700 250 1 Normal_user2: testClient 192.168.1.102 250 700 250 1 Normal_user3: testClient 192.168.1.102 250 700 250 1 Normal_user4: testClient 192.168.1.102 250 700 250 1 T ranged between 0.00125 and 0.0025 when converging. In Fig. 7-7, we can see incoming packets distribution of two registered normal users and two registered attackers. Packets from the other two unregistered normal users did not enter tunnel tree. As a result, the AID station had no statistics data about them. In this testing case avgN did not become zero after the AID system was triggered. Only the two registered attackers had

PAGE 71

59 packets dropped. Ideally, avgT:avgN = 1600:400 = 4:1 should be true in the case 4 (this ratio can be changed by modifying RESERVEDTIMES in server.c and signing a new contract between the server and AID station). However, because attackers slowed down their traffic (less than half amount of packets sent out as other normal users), avgT went down too. Table 7-5. Case 4 packets statistics in the AID station. Normal_user1 and normal_user2 are registered had no packets dropped. Attacker1 Attacker2 Normal_user1 Normal_user2 Packet# in 493 453 1187 1194 Packet# out 172 154 1187 1194 BufferFull 0 BufferFull 0 BufferFull 0 BufferFull 0 BigVCTS 321 BigVCTS 299 BigVCTS 0 BigVCTS 0 MD5Fail 0 MD5Fail 0 MD5Fail 0 MD5Fail 0 Packet# dropped CantRoute 0 CantRoute 0 CantRoute 0 CantRoute 0 VCTS localTime012345671511011512012513013514014515015516016517017518018519019511001105111011151packet#VCTS localTime attacker1 attacker2 normal_user1 normal_user2 MAXVCTSEXCEED Figure 7-7. Distribution of incoming packets in case 4 at the AID station. Traffic from normal_user1 and normal_use2 were pretty stable (the lower two series). It is very similar to Fig 7-3 with the exception that series for two normal users are a little bit higher.

PAGE 72

60 controlT (Fig. 7-8A)00.00050.0010.00150.0020.00250.00315913172125293337414549535761656910 secondsvariable's value I decrease T arrival rate (Fig. 7-8B)0500100015002000250030003500400015913172125293337414549535761656910 secondsarrival rate in 10 seconds avgT avgN AID cap server ca p avgAID Figure 7-8. How did T and arrival rate interact with each other in case 4. AvgN is the servers arrival rate of the traffic from the Internet. T would not affect AvgN directly, since the traffic did not enter the AID tunnels. However, because the traffic from AID tunnels had higher priority, the unregistered attackers could not flood the server. Case 5 This is an interesting case. It is very analogous to case 2, two attackers and one normal user. However, attackers chopped same amount of data into smaller pieces here. Attacker1: testClient 192.168.1.102 1000 500 300 0 Attacker2: testClient 192.168.1.102 1000 500 300 0 Normal_user: testClient 192.168.1.102 250 700 250 1 In case 2, MAXSEG was 1000 for attackers, and it was 300 in case 5. In Fig. 7-10, we can see that after system converged, value of T varied between 0.001375 and 0.00275. It is bigger than in case 2. What made case 5 so special? Let us compare it with case 2. In case 2, attackers sent out 1000 bytes, exclusive of headers, every iteration, and MAXSEG was 1000. Here, attackers also sent out 1000 bytes per iteration, but MAXSEG

PAGE 73

61 was 300. That means an iteration needs to send packets as many as three times in case 5. Then, what happened? See Fig. 7-9. First, unlike in case 2, an attacker almost sent out as many packets as normal user did. A packet's VCTS is decided by its size and T of the tunnel tree. When a packet is small, VCTS will be small too. Consequently, the packet has a higher chance to be accepted by an AID station. Second, compared with case 2, T became bigger when system converged, but VCTS-localTime for packets from attackers became smaller. Smaller packets made smaller VCTS but more packets sent out (heavier traffic) in an iteration made bigger T. Third, Since T became bigger and packets from normal_user had the same MAXSEG as in case 2, 250, we can see VCTS-localTime for packets from normal_user twisted a lot, unlike in case 2. In our AID service, packets from either attackers or legitimate users might be dropped. Because a server's capacity is fixed, if more clients try to access the server at the same time, every client could get less resource from the server. If a client intends to use more than its share, its packets will be discarded. The AID system controls the arrival rate not to surpass a server's capacity by this policy. We can see in case 2 and case 3. A client is treated differently when the traffic load changes. A legitimate client slows down its outgoing TCP traffic when sensing its packets were dropped (no acknowledgement from the other end), if it implemented TCP correctly. For an attacker, if it implemented TCP right, it would slow down outgoing traffic. If it did not, VCTS for its packets would grow very fast, and most of its packets would be dropped. Damage

PAGE 74

62 from attackers is soothed in both cases, and at the same time, a server is still accessible to legitimate clients. Table 7-6. Case 5 packets statistics in the AID station. Normal_user had no packets dropped. Attacker1 Attacker2 Normal_user Packet# in 1229 1185 1215 Packet# out 723 659 1215 BufferFull 1 BufferFull 3 BufferFull 0 BigVCTS 505 BigVCTS 523 BigVCTS 0 MD5Fail 0 MD5Fail 0 MD5Fail 0 Packet# dropped CantRoute 0 CantRoute 0 CantRoute 0 VCTS localTime012345615110115120125130135140145150155160165170175180185190195110011051110111511201packet#VCTS localTime attacker1 attacker2 normal_user MAXVCTSEXCEED Figure 7-9. Distribution of incoming packets in case 5 at the AID station. Notice the big wave of normal_user.

PAGE 75

63 controlT (Fig 7-10A)00.00050.0010.00150.0020.00250.003147101316192225283134374043464952555861646710 secondsvariable's valu e I decrease T arrival rate (Fig 7-10B)0500100015002000250030003500147101316192225283134374043464952555861646710 seondsarrival rate in 1 0 seconds avgT avgN AID cap server cap avgAID Figure 7-10. How did T and arrival rate interact with each other in case 5.

PAGE 76

CHAPTER 8 FUTURE WORK AND CONCLUSION Limitations and Future Work Our AID system has some limitations theoretically and practically. Improving they is our goal in future work. Protecting UDP traffic: We need the self-adaptation feature based on TCP congestion control to separate legitimate users and attackers. That is why our AID system only protects TCP traffic at present. Future work is to include UDP traffic into our AID service. Robustness against the compromise of AID stations: In the current design of our AID system, we did not address how to deal with the case that AID stations are compromised. A compromised AID station can send forged UDP messages (PUSH, PULL, CTRLT and etc.), drop packets from legitimate users and adjust virtual clock maliciously. The good thing is an AID station can be removed or added into the AID system easily. We could disconnect a suspicious station for an inspection Traceback: The AID system can resist against DoS attacks but cannot trace back to the origin of attacks. Flooding traffic might be from registered clients that are zombies remotely controlled by real attackers. We may implement existing IP traceback mechanisms in the AID system. Independency of higher-level application: Our AID system is not perfectly independent of higher-level applications because we introduced AID layer and it is not processed by the Linux kernel. However, we also do not want users to recompile their 64

PAGE 77

65 Linux kernels to join the AID service. We may find some other way to program our AID system to avoid the dilemma. Flexibility of programs: Most controlling factors are defined as constants in the programs. We need to recompile them if we want to do registration, change the secret keys, adjust virtual clock setting and etc. These factors can be saved in files to make our programs more flexible. Conclusion Most existing defense systems for DoS attacks are not self-complete. They usually need universal deployment. In the thesis a self-complete anti-DoS service (AID) was implemented and tested. The AID service can be applied to Internet services, such as ssh, ftp, www and so on. Everyone can join the AID service by registration and get immediate protection. The AID service provides a fair share of a server's resource to all registered clients. It requires no modification of end systems and routing infrastructure to join. Random overlay network accommodates an efficient and scalable structure to route traffic from registered clients. It changes dynamically. An AID station can be removed or added easily. Distributed virtual-clock packet scheduling algorithm blocks the traffic from attackers and manages the arrival rate of a server. A registered client host, which is not an attacker, can access a registered server even when the server is attacked. Finally, we still have some problems need to be solved in the future, for example, including UPD traffic into the AID service, making AID station robust, tracing back attackers and having programs more flexible.

PAGE 78

APPENDIX A HOW TO RUN Make sure the library libipq was installed before continue. It can be found in iptables-1.2.9. Program server: make server cd src_module make serverFilter.o insmod ip_queue insmod serverFilter.o cd .. ./server AIDSIP SERVERIP AIDSIP is IP of the AID station A s SERVERIP is the servers IP. Program client: make client cd src_module make clientFilter.o insmod ip_queue insmod clientFilter.o cd .. ./client AIDIP AIDIP is IP of the AID station A c Program AID make AID cd src_module make AIDFilter.o insmod ip_queue insmod AIDFilter.o cd .. ./AID clientIP serverIP 66

PAGE 79

67 ClientIP is IP of the registered client and serverIP is IP of the registered server. In our testing cases, we had only one client machine, AID station and server machine. If an AID station wants to register more than one client or server, information of the client/server should be added into the function initialize() of the source file AID.c. Program alert make alert ./alert AIDIP serverIP serverPort AIDIP is IP of AID station A s ServerIP:serverPort identifies the attacked service. Remove loaded module: rmmod ip_queue rmmod clientFilter rmmod AIDFilter rmmod serverFilter Turn on forwarding in iptables at an AID station: su echo > /proc/sys/net/ipv4/ip_forward iptables I FORWARD j ACCEPT

PAGE 80

APPENDIX B FILE GLOBAL.H File global.h defined many important constants. Their names and values we used in testing are: #define MTU 1500: Max transfer unit. #define BUFSIZE 4096: The size of the buffer that a queued packet is copied to. #define TCPINFOSIZE 28: The length of an AID layer header, in byte long, for TCP packets entering a tunnel tree. The whole packet looks like: IP header | TCP header | new destination IP (32 bits) | packet digest for integrity (128 bits) | virtual clock timestamp 64 bits | application data. So, (32 + 128 + 64)/8 = 28 bytes. #define UDPINFOSIZE 17: The length of an AID layer header, in byte long, for UDP packets. The whole packet looks like: IP header | UDP header | packet digest for integrity (128 bits) | packetType 8 bits | application data. So, (128+8)/8 = 17 bytes. #define MD5DGSIZE 16: Bytes of md5 digest created by md5 library, md5.h and md5.c. #define RCZSIZE 4: Bytes of recognizing field of normal TCP packets. #define UDPMAXSIZE 256: The max size in bytes of a UDP packet in the AID system, inclusive of md5 digest (16 bytes), packetType (1 byes), service list (7*n bytes). It should be big enough for different UDP messages. #define UDPTYPELEN 1: Length of the packetType field in a UDP packet in byte long. #define PULLNO 0: Number of the nearby AID stations this AID station should send PULL messages to (variable q). #define PUSHNO 0: Number of the nearby AID stations this AID station should send PUSH messages to (variable k). Using square root of NEARBYAID (defined in AID.c) is ok. #define PUSHDEEP 2: how many AID station a PUSH message can go through, exclusive the first one which is A s 68

PAGE 81

69 #define PULL 0: Value of packetType field for a PULL message. #define PULLANS 1: Value of packetType field for a PULLANS message. #define PUSH 2: Value of packetType field for a PUSH message. #define CTRLT 3: Value of packetType field for a CTRLT message. #define PCKTYPEOFFSET 16: The offset of packetType in UDP packets, the location is: UDP header 8 bytes | md5 digest 16 bytes | packetType 1 byte =16 #define VCSTAMPOFFSET 20: The offset of virtual clock timestamp in TCP, the location is: TCP header (offset bytes) | serverIP 4 bytes | md5 digest 16 bytes | VCTStamp 4 bytes, 4+16 = 20 #define DATAOFFSET 28: The offset of application data in TCP packets, the location is: TCP header (offset bytes) | serverIP 4 bytes | md5 digest 16 bytes | VCTStamp 8 bytes | application data, 4+16+8 = 28

PAGE 82

LIST OF REFERENCES 1. C. Schuba, I. Krsul, M. Kuhn, E. Spafford, A. Sundaram, and D. Zamboni, Analysis of A Denial of Service Attack on TCP, Proc. of IEEE Symposium on Security and Privacy, pp. 208-223, IEEE Computer Society Press, Oakland, CA, May 1997. 2. J. Lemon, Resisting SYN Flood DoS Attacks with A SYN Cache, Proc. of USENIX BSDCON2002, pp. 89-97, USENIX Association, Berkeley, CA, February 2002. 3. P. Ferguson and D. Senie, Network Ingress Filtering: Defeating Denial of Service Attacks Which Employ IP Source Address Spoofing, IETF, RFC 2267, January 1998. 4. K. Park and H. Lee, On the Effectiveness of Route-Based Packet Filtering for Distributed DoS Attack Prevention in Power-Law Internets, Proc. of ACM SIGCOMM 2001, vol. 31, pp. 15-26, August 2001. 5. H. Wang, D. Zhang, and K. G. Shin, SYN-dog: Sniffing SYN Flooding Sources, Proc. of 22nd International Conference on Distributed Computing System (ICDCS), pp. 421-428, IEEE Computer Society, Washington D.C., July 2002. 6. S. Chen, R. Chow, Y. Xia and Y. Ling, A Global Anti-DoS Service Based on Random Overlay Network, unpublished paper, Department of Computer and Information Science and Engineering, University of Florida, September 2004. 7. A. Juel and J. Brainard, Client Puzzles: A Cryptographic Countermeasure Against Connection Depletion Attacks, Proc. of Network and Distributed System Security Symposium (NDSS), pp. 151-165, Networks and Distributed Security Systems, San Diego, CA, February 1999. 8. T. Aura, P. Nikander, and J. Leiwo, DoS-Resistant Authentication with Client Puzzles, Cambridge Security Protocols Workshop 2000. LNCS, Springer-Verlag, vol. 2133, pp. 170-177, 2001. 9. D. Dean and A. Stubblefield, Using Client Puzzles to Protect TLS, paper presented at 10th Annual USENIX Security Symposium, Washington D.C., August 2001. 70

PAGE 83

71 10. X. Wang and M. K. Reiter, Defending Against Denial-of-Service Attacks with Puzzle Auctions, Proc. of IEEE Symposium on Security and Privacy, pp. 78-92, IEEE Computer Society, Washington D.C, May 2003. 11. L. Zhang, VirtualClock: A New Traffic Control Algorithm for Packet Switching Networks, ACM Transactions on Computer Systems, vol. 9, no. 2, pp. 101-124, May 1991.

PAGE 84

BIOGRAPHICAL SKETCH I earned my BS degree in computer science and information engineering from National Chiao Tung University in Taiwan. As an undergraduate, I was like a sponge absorbing all kinds of knowledge of computer sc ience accessible to me. In my junior and senior years, I worked on a project of providing QoS (quality of service) in wireless ATM network. Integer programming is the key of the project. Through those 4 years, I finished many projects in different areas: network, security, compiler, windows programming, audio processing, graphics, database system, etc. I am seeking my MS degree at the University of Florida by writing the thesis. After about 2 years training ar the University of Florida, I polished my professional knowledge and skills better and became more confident to face new challenges. 72


Permanent Link: http://ufdc.ufl.edu/UFE0007880/00001

Material Information

Title: Implementing a Global Anti-DoS Service Based on Random Overlay Network
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0007880:00001

Permanent Link: http://ufdc.ufl.edu/UFE0007880/00001

Material Information

Title: Implementing a Global Anti-DoS Service Based on Random Overlay Network
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0007880:00001


This item has the following downloads:


Full Text












IMPLEMENTING A GLOBAL ANTI-DoS SERVICE BASED ON RANDOM
OVERLAY NETWORK















By

WEN-CHUAN SHEN


A THESIS PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE

UNIVERSITY OF FLORIDA


2004

































Copyright 2004

by

Wen-Chuan Shen

































To my family and friends who supported me with their patience and love.
















TABLE OF CONTENTS

page

L IST O F T A B L E S ........ .. ...... .. ............ .................................................... viii

LIST OF FIGURES ......... ......................... ...... ........ ............ ix

ABSTRACT .............. .................. .......... .............. xi

CHAPTER

1 IN TR OD U CTION ............................................... .. ......................... ..

What Is Distributed Denial of Service (DDoS) Attack .........................................1
R elated W ork ............................................................. ........... ... .. .. 2
Ingress Filtering Proposed by Ferguson and Senie....................................2
Route-Based Packet Filtering Proposed by Park and Lee.................................2
SYN-Dog Proposed by Wang, Zhang and Shin ............................................. 2
Self-C om plete D defense System s............................................ ........................... 2

2 A ID SY STEM O V ER V IEW ............................................................. .....................4

Fundam mental Ideas .................. ....................................... ................ .4
W ho Is Protected?...................................... ........ ...... ............ ....
What Is Random Overlay Network (RON) for?..................................................5
How Does AID Defense System Work? .................................... ...............
Implementation Issues ................... .. ........ ........ ..................6
Packets Intercepting M odules.......................................... ........... ............... 8
Handling Queued Packets in Userspace..... .......... .......................................8
Show ing Statistics ................. .... ...................... ............. ................ .9

3 A ID L A Y E R ............................................................................................10

A ID L ayer for T C P T traffic ............................................ ....................................... 10
A ID L ayer for U D P Traffic .......................................................................... .... ... 12
PU SH M message ........................................................ .. ........ .... 12
PU L L M message ............................................................... ..... ............ 13
PULLAN S M message ...................................................................... ................ 13
C TR L T M message .......................................... ................... .. ...... 14
Im plem entation Issues .......................................... ................... .. .. .... 15









4 CLIEN T END ....................................................... ............ .. ............ 16

M odule C lientFilter.o ....................... ............................................ .. .......... ..16
Program Client.........................................................17
Packets from NF IP PRE ROUTING...................................................... 17
Packets from NF IP LOCALOUT .........................................18
Packets from NF IP POST ROUTING....................................... ................18
Im p lem entation Issu es ................................................................................................2 0
W hat Is ServL ist? .................. ...... .......................... .. ............... ..... ........... .. 20
Why Changes a Packet's Destination in Hook NF IP LOCAL_OUT? .............20
H ow Is the Registration D one?...................................... .................. ........ 20
Not Perfectly Isolated from Higher-Level Applications ...................................21
The Maximum Transmission Unit (MTU) Problems................ ...................21

5 SERVER END ..................................................... ............ .... ..... 23

M odule ServerF ilter.o ....... .............................................................. ......... ..... 23
P program Server .................................................................... ................. 24
Packets from NF IP PRE ROUTING.......................................................24
Packets from NF IP POST ROUTING....................................... ................26
Im p lem entation Issu es ...................................................................... .... ..............2 6
N o T h read s ................................................................2 6
Im portant V ariables ............................................................... .... 27
PCK B U F SIZET ............................................ ............ ... ....... .... 27
PCKBUFSIZEN .................. ................................ .. ... .......... .... 27
IPQ REA D TIM E ........................................................................... 27
R E A D IN TER V A L ............................................... ............................ 27
SEN D PC K B U FN O ............................................... ............ ............... 28
A V G IN T E R V A L .............................................................. .....................2 8
T O T A L C A P ...................................................................... ....................2 8
RESERVED TIM ES .............................................................................. 29
H ow the R registration Is D one ................. ......................... ............... ... 29
Not Perfectly Isolated from Higher-Level Applications ...................................29
Program Alert ................. ................. .........31

6 AID STATION ............ ...................... ............... ........32

A ID Tunnel Tree............................................. .. .. ........... ..... ...... 32
Push Phase ............. ..... ......... ... ...............32
P u ll P h a se ................................................................3 4
R outing ........................................34
Why Does a Tunnel Tree Try to Include Every AID Station? .........................35
V ariables k and q ........................................36
Advantages of Random Overlay Network (RON) ............................................38
Distributed Virtual-Clock Packet Scheduling ................. ................. ..... 39
B a sic Id e a ................................................................3 9
H ow to A dju st T ............................................................40


v









P program s for an A ID Station ............................................................ .....................40
M odule A ID Filter.o .......... ..... .................................................. .. ........ ..... 40
P ro g ram A ID .............................................................................. ..................... 4 1
TCP packets from NF IP PRE_ROUTING.................................41
UDP packets from NF IP PREROUTING.............................................42
TCP packets from NF IP POSTROUTING ..........................................42
UDP packets from NFIPPOST_ROUTING............... ..................43
Im p lem en tatio n Issu e s ................................................................................................4 3
N o T hreads ................. .................................... ................. 43
R registration for Clients and Servers ................................................. .............. 44
Im p ortant V ariab les ....................................................................................4 4
PCKBUFSIZE ............... .................. ................................44
IPQ REA D TIM E ............................................................................ 44
R E A D IN T E R V A L ............................................................ .....................44
SEN D PC K B U FN O ............................................... ............ ............... 45
N EA RB Y A ID ...................... ................ ................. ..... ...... 45
SN ED TIN TERV A L .......................................................... ............... 45
SENDPULLINVAL ........................... ............................................ 45
DECREA SERA TIO ............................................................................... 45
M A X V CTSEX CEED .............................................................. ............... 45
A dding N ew A ID Stations ..................................................... ...................46
D iam eter of a Tunnel Tree ............................................................................46
F orw arding P ackets ......................... .. .................... .. .. ...... ........... 47

7 TESTING RESULTS AND ANALYSIS ...............................................................49

Im portant Issues about T testing ........................................................................ ... 49
T testing E lem ents .......................................................................50
Program TestClient ........................................................................ 50
Program TestServer .............. ............................. .. ...... ................. 51
Setting of the A ID Sy stem ....................................................................... ....5 1
C a se 1 ................................................................................5 1
C ase 2 ................................................................................54
C a se 3 ................................................................................5 6
C ase 4 ................................................................................5 8
C a se 5 ................................................................................6 0

8 FUTURE WORK AND CONCLUSION ........................................................64

L im stations and Future W ork........................................................................ ......64
C conclusion ............................................................... .... ..... ........ 65

APPENDIX

A HOW TO RUN .................. .................................... ......... .............66

B F IL E G L O B A L .H ........... .................................................................... ................68









L IST O F R E F E R E N C E S ...................................... .................................... ....................70

B IO G R A PH IC A L SK E TCH ...................................................................... ..................72
















LIST OF TABLES


Table p

7-1. List of important factors of AID system for testing .........................................51

7-2. Case 1 packets statistics in the AID station.........................................................52

7-3. Case 2 packets statistics in the AID station........................................ .................54

7-4. Case 3 packets statistics in the AID station.................................. ...............56

7-5. Case 4 packets statistics in the AID station......................................... ................59

7-6. Case 5 packets statistics in the AID station........................................ .................62
















LIST OF FIGURES

Figure page

2-1. A ID system architecture ............................................................. .................. 4

2-2. N etfilter hooks ................ ............... ........... .... ........ .......... .... ....

3-1. AID layer header for TCP packets ............................... ............... 11

3-2. PUSH message ...................................... .......... .. ........... 13

3-3 P U L L m essage.............................................................. ......... .. ..... 13

3-4. PULLAN S m essage... .................................................................................... 14

3-5. C TR L T m message ...... ... .......................................... .. .. .......... ........ 14

4-1. Inserting an AID layer header to a packet that enters AID tunnels.........................19

4-2. Inserting an AID layer header to a packet not entering AID tunnels ......................19

5-1. Removing the AID layer header in server end .............................................. 25

6-1. Tunnel tree created in push phase........................................ .......................... 33

6-2. Four possibilities a packet can be routed........................................ ............... 47

7-1. Distribution of incoming packets in case 1 at the AID station.............................53

7-2. How did T and arrival rate interact with each other in case 1 ................................53

7-3. Distribution of incoming packets in case 2 at the AID station..............................55

7-4. How did Tand arrival rate interact with each other in case 2 .............................55

7-5. Distribution of incoming packets in case 3 at the AID station.............................57

7-6. How did Tand arrival rate interact with each other in case 3 ................................58

7-7. Distribution of incoming packets in case 4 at the AID station..............................59

7-8. How did Tand arrival rate interact with each other in case 4 .............................60









7-9. Distribution of incoming packets in case 5 at the AID station.............................62

7-10. How did T and arrival rate interact with each other in case 5. ..............................63















Abstract of Thesis Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Master of Science

IMPLEMENTING A GLOBAL ANTI-DoS SERVICE BASED ON RANDOM
OVERLAY NETWORK

By

Wen-Chuan Shen

December 2004

Chair: Shigang Chen
Major Department: Computer and Information Sciences and Engineering

Distributed denial of service (DDoS) is a major threat to the Internet nowadays.

Legitimate users have a hard time accessing the servers that are under DDoS attacks.

What makes it worse is that the attacking tools are easy to get. Even people without

enough professional knowledge can launch a DDoS attack. Obviously, automated anti-

DDoS systems become more and more important.

Many current available solutions to DDoS attacks require universal installation and

configuration across different administrative realms, which are impossible or very

difficult to do in many situations. This thesis studied and provided another solution to

DDoS attacks. An anti-DoS service (called AID) is presented in this thesis, and no global

deployment is required. The AID service protects general TCP traffic. It guarantees all

registered clients can access a registered server fairly even when the server is under

DDoS attacks. A domain, like a school or company, can get immediate protection after

having the AID service.









Two primary parts of the AID service are the random overlay network (RON) and

the distributed virtual-clock packet scheduling algorithm. The former forms tunnel trees,

which connect registered clients to a registered server. Packets from registered clients go

through the tree to the server when the server is under DDoS attacks. It is adapted and

easy to manage. The latter is a packet scheduling algorithm to simulate client puzzles. It

confines the amount of data a registered client can send to a server through RON to

achieve fairness.














CHAPTER 1
INTRODUCTION

What Is Distributed Denial of Service (DDoS) Attack

A DoS attack intends to make a server out of its resource (which could be

bandwidth, buffers, CPU time, etc.). Attackers can send a lot of requests to exhaust a

server's bandwidth. Other legitimate users will be unable to access the server. Another

example is SYN flooding attack [1-2]. To establish a connection, a client sends a SYN

packet at first. The server is going to keep this information in a buffer for a period Tin

order to recognize the following incoming packets. If attackers send enough SYN

packets to make the buffer overflow, the server has no way to process requests from other

users. In some cases, like route table updating or software's bugs, a simple request can

make the server do a considerable amount of computation. In this case, normal users

cannot access the server. The basic idea of DoS attacks is simple, using a small amount

of resources to overwhelm the server.

What makes a DDoS attack different from a traditional DoS attack? In a DDoS

attack, the clients launching the attack might be victims as well. Hackers compromise

and install DDoS attacking programs on these hosts first. Then, hackers can remotely

control these victims to attack the servers cooperatively. With DDoS attacks, it is

possible to flood a big commercial server in a brute-force way. Besides, it becomes very

difficult to trace back the attacker because the compromised hosts are not the real

attackers. Usually, there are hundreds or thousands of compromised hosts and they are









around the whole world. It makes attackers feel safe to do this. Today, how to protect

hosts against DDoS attacks is very important.

Related Work

Ingress Filtering Proposed by Ferguson and Senie

In ingress filtering [3], before a packet is transmitted into another domain, the

router checks the packet's source address. If it does not match the ingress filter rule,

probably a spoofed source address, the packet will be dropped. Ingressfiltering helps to

trace back the attacker, while it cannot prevent an attack originating from a valid source

address.

Route-Based Packet Filtering Proposed by Park and Lee

With partial deployment (about 18% in Internet AS topologies) [4], spoofed IP

packets can be prevented from reaching their intended targets effectively.

SYN-Dog Proposed by Wang, Zhang and Shin

SYN-dog [5] is a software agent which can be installed at leaf routers of stub

networks. It is stateless and light-weighted. Therefore, SYN-dog itself is immune to any

flooding attacks. It detects SYN flooding attack by monitoring behavior of SYN-

SYN/ACK packets. SYN-dog can also trace back the attacking source.

Self-Complete Defense Systems

There are a huge number of hosts on the Internet. It is almost impossible to make

every host join a specific defense system. Here comes the problem. If a server has the

defense system installed, can it resist the DDoS attack from clients that do not participate

in the same defense system? The answer is no for most existing DDoS defense systems.

Suppose we have a networked system ofS + C. C is a set of client networks, and S is a

set of server networks. C'is a subset of C and S'is a subset of S. C'+ S'has a defense









system installed. A defense system is self-complete if any client in C' can still access any

host in S' even when under DDoS attacks, as long as the client itself does not participate

the attack. It does not care if the attack is from C or C C'. In other words, a self-

complete defense system should be able to defeat attacks from either inside area C' or

outside area C C'. A self-complete defense system makes itself a clean area in the

Internet. The area does not have to cover the entire Internet, and hosts in it are protected.

Let us review the DDoS defense systems mentioned above.

* Ingressfiltering: Source addresses of packets from C C' can be spoofed, because
C C' do not check them. Therefore, DDoS attacks can be launched against the S'
from C C'. In this case, clients in C' have difficulty to access S' even though C'
and S' both join the defense system. Therefore, Ingressfiltering is not self-
complete.

* Route-BasedPacket Filtering: Packets with spoofed source are prevented from
reaching their intended targets effectively as long as 18% of Internet AS's join the
defense system. However 18% of Internet AS's is a huge number. It is not self-
complete until C'is as large as 18% of Internet AS's.

* SYN-dog: Attackers can still do SYN-flooding to S' from C C', because C C' is
not under protection. Similar to ingressfiltering, unless C' is as big as C, attackers
from C C' can make S' not accessible to C'. In consequence, SYN-dog is not self-
complete.

The benefits of a self-complete system are apparent. It suits normal companies,

schools and organizations. They can set up a self-complete defense system in their realm

and become under protection immediately, independent of others. A working self-

complete defense system (called AID), the detail of its structure and how it was

implemented are presented in the thesis. The AID system's overview is in chapter 2.














CHAPTER 2
AID SYSTEM OVERVIEW

Fundamental Ideas

The idea of the anti-DoS system (called AID) is from Chen et al [6]. The AID

system contains clients, servers and AID stations physically. Another two important

parts in the AID system are random overlay network (RON) and distributed virtual-clock

packet scheduling algorithm [6]. Random overlay network is formed on AID stations.

We did not draw a clear line between DoS attacks and DDoS attacks in the thesis.

Whenever a server senses an attack, like flooding packets beyond its capacity or unusual

requests from clients, the AID defense system will be triggered. The AID system's

architecture is shown in Fig. 2-1.

client
erver server endpoint
Client endpoint


server --o


server AD client
AID tunnelslient

Figure 2-1. AID system architecture. The AID circle and AID tunnels symbolize RON,
which is composed by AID stations. A client point can mean a client network,
not just a client host. Likewise, a server point can be a server network behind
a router.

Who Is Protected?

Register clients and servers that we want to protect at their nearby AID stations.

The registration brings a shared secret key between the AID station and the registered









host. The secret key is used in AID tunnels for integrity checking when under DDoS

attacks. Everyone can join the AID service by registering at an AID station.

What Is Random Overlay Network (RON) for?

RON consists of all AID stations. When a registered server is under DoS attacks

and other registered clients try to access the server, packets from these clients will go

through the RON instead of the Internet. We say that these packets are entering AID

tunnels, which are tree structure. They go through AID tunnels all the way to the

attacked server. Other packets from unregistered clients will reach the server via the

Internet. AID tunnels are one-way path for packets from registered clients to attacked

registered servers. We do not allow traffic from registered servers to registered clients

entering RON to minimize the load of AID stations. It is transmitted via the Internet. Of

course, traffic related to unregistered clients or unregistered servers cannot enter AID

tunnels.

How Does AID Defense System Work?

When under attacks, packets from registered clients go through RON but packets

from unregistered clients go through the Internet. We make packets from AID tunnels

have higher priority. Servers process them first. Hence, the external traffic (from

unregistered clients) cannot influence the internal traffic (from registered clients). How

about if a registered client is an attacker itself? Well, that is why we have distributed

virtual-clock packet scheduling algorithm, which simulates client puzzles [7-10]. Every

AID station manages a virtual clock for every tunnel hooking on a client network. If a

client has behavior of flooding a registered server, its virtual clock will run fast. When

virtual clock's value is too big, packets from that client will be dropped. By doing this, we

can separate the attacking traffic out and block it.









Traffic in AID tunnels has integrity protection. Remember the secret key a client

or server got after registering at an AID station? The secret key and other important data

in a packet are put together and digested by MD5 algorithm. The 128-bit-long packet

digest is used for integrity checking. As a result, alteration of packets in AID tunnels will

be detected. Because the third party cannot forge the packets, integrity checking is also

authenticity checking, verifying that the packets are really from the hosts as they claimed.

Now we know how AID stations interact with clients and servers. We also know

what random overlay network and distributed virtual-clock packet scheduling algorithm

are for. More details of the AID system and how it was implemented were revealed in

the following chapters.

Implementation Issues

We chose Linux as our developing platform and our programs only work on IPv4.

As mentioned in the section "How Does AID Defense System Work," a registered client

should send it packets via RON instead of the Internet. Meanwhile, the attacked server

should be able to tell where a packet is from and give the one from AID tunnels higher

priority. We do not want people to recompile their Linux kernels or rewrite application

codes if possible. So, we introduced another layer between application layer and

transport layer, called AID layer. Extra information is added into a packet as an AID

layer header for the AID service. Higher application layer programs should not notice

the existing of AID programs.

Netfilter is one tool we used in our AID system to intercept and modify packets. It

was included in Linux 2.4. It supports five different hooks and they are

NF IP PREROUTING, NF IP LOCALIN, NF IP FORWARD,








NF IP LOCALOUT and NF IP POSTROUTING. Fig. 2-2 shows how a packet goes
through these hooks.

[1] i ROUTE [3] [4]

I t
[2] ROUTE
It
[5]



Figure 2-2. Netfilter hooks. Hook 1 is NF IP PREROUTING, hook 2 is
NF IP LOCAL IN, hook 3 is NF IP FORWARD, hook 4 is
NF IP POST ROUTING and hook 5 is NF IP LOCAL OUT.
NFIPPREROUTING: A packet hits the hook after reaching the host and sanity
checks but before the routing decision.
NFIPLOCALIN: A packet hits the hook after the routing decision and the
packet's destination is this host.
NFIPFORWARD: A packet hits the hook after the routing decision if the
packet's destination is another interface.
NFIPLOCAL_OUT: A packet hits the hook when going down the kernel after a
process creates and sends out the packet.
NFIPPOST_ROUTING: A packet hits the hook right before it is put on the wire.
We can inject our handling functions into any of these hooks. When a packet goes
through hooks, their handling functions will be executed. That is where and how we can
modify the packet.









Packets Intercepting Modules

Our first step is to write modules to intercept interested packets in proper hook

positions. The interested packets are queued into userspace. Doing in this way may

cause some performance penalty because of switching between kernelspace and

userspace. However, there are also some advantages. First, it is easier to debug a

program running in userspace. Second, we have more libraries handy. Third,

misbehavior, if any, of a program will not crash the whole system. Three modules

totally, clientFilter.o takes care of packets in/out clients. Likewise, AIDFilter.o is for

AID stations, and serverFilter.o is for servers. After loading an appropriate module on

the host, interested packets will be queued to userspace. To stop it, just unload the loaded

module. These queued packets will be inspected or modified later.

Handling Queued Packets in Userspace

To deal with the queued packets in userspace we need the library libipq developed

by James Morris. It can be found easily on the Internet and simple to install. With the

library, we can grab one packet out of the queue every time. We can drop the packet, do

nothing, or modify and send it back to the kernel. One thing should be noticed is that

checksums in TCP/UDP and IP headers need to be recalculated if the packet is altered.

In the client end, all outgoing TCP packets are queued to userspace. If a packet's

destination server is under attacks, its destination IP will be converted to an AID station's

IP. The AID station is the one this client registered at. The AID station will notify the

registered client if a registered server is currently under attack. If no attacks, packets are

routed as usual. A registered client executes the program client.

In AID stations, AID tunnel trees are built up to route packets to their destination

servers. Assume a server registered at an AID station named As. When the server is









under DoS attacks, an AID tunnel tree rooted at A, is formed. Packets from registered

client to this server are routed from tree leaves to the root A, and finally to the server.

Besides routing, virtual clocks for every client are maintained by AID stations as well.

An AID station executes the program AID.

In the server end, packets from the Internet and AID tunnels are separated. Process

the latter first. If under attacks, a server will send alert messages to its registering AID

stations, A,. Then, this AID stations broadcasts alert messages to other AID stations.

AID tunnel trees are constructed. A server executes the program server.

Showing Statistics

Programs client, server and AID, all record statistic information of TCP traffic.

Users can know how many packets got through or were dropped and the reasons of

dropping. They all have a while loop in main( whose condition is always true. To keep

programs simple, we did not use threads or fork a child process. Then how can the

programs interact with users when they want to see the statistics of traffic? The answer is

signal. The reaction of the signal SIGINT was redefined in these three programs. When

Ctrl-c is typed, programs will not be terminated. Instead, statistic information is printed

out. We can type Ctrl-\ to send the signal SIGQUIT to stop the programs.














CHAPTER 3
AID LAYER

AID layer is added between application layer and transport layer. In this chapter,

we defined AID layer headers, which are inserted between TCP/UDP headers and

application layer data in a packet. Since we do not want AID service users to recompile

their Linux kernels, Linux kernels have no idea of this new layer. AID layer headers are

treated as application data actually by Linux kernels. In clients, the program client adds

an AID layer header before a packet is sent out. In servers, the program server takes the

AID layer header off. Only the AID system can recognize AID layer headers. As for

AID stations, AID layer headers contain data needed for routing, constructing AID tunnel

trees, and etc. Consequently, application programs in the client end or server end do not

know they are already in the AID service and protected. Currently, only TCP traffic is

protected in the AID system because TCP's congestion control feature is needed.

When a packet enters RON, if it is a TCP packet, it belongs to traffic from a

registered client to a registered server. If it is a UDP packet, such as a PULL message, or

PUSH message, the packet is used to control the AID system. We explained what these

UDP messages are later in this chapter. We have different AID layer headers for TCP and

UDP packets. Both TCP and UDP traffic have integrity guard.

AID Layer for TCP Traffic

We have two kinds of AID layer headers for TCP packets. One is for packets

transmitted via the Internet and the other is for packets transmitted via the RON (AID

tunnels). Fig. 3-1 shows the contents of the headers and where they are inserted.









A
IP header TCP header AID layer header application layer data

recognizing field 32 bits
B
IP header TCP header AID layer header application layer data


server IP 32 bits (recognizing field) md5 digest 128 bits virtual clock timestamp 64 bit

Figure 3-1. AID layer header for TCP packets. A) For normal TCP packets that do not
enter AID tunnels. Recognizing field is 0. B) For TCP packets to attacked
servers that enters AID tunnels. Notice that recognizing field in the first
figure is at the same position as server IP in the second figure.

Server IP field is used to save the IP address of destination server. To travel

through AID tunnels, a packet's destination is changed to the AID station where it is

routed next. However, the final destination is still the server, so we need to keep this

information. Md5 digest field is used to check integrity. If checking fails, the packet will

be dropped. Virtual clock timestamp field is used in distributed virtual-clock packet

scheduling algorithm.

Why do we need recognizing field even in normal packets? The problem is that

when a server gets a packet, it has no way to know if the packet is from the Internet or

AID tunnels. Recognizingfield of normal packets is at the same location as server IP

field of packets to an attacked server, right after the TCP header. When a packet arrives,

the server checks this location. If it is 0, the packet is from the Internet; otherwise, the

packet is from the AID tunnels. We assume server IP cannot be 0.0.0.0, so no conflicts.

What information is under integrity protection?

* Source IP (4 bytes) and destination IP (4 bytes) addresses in the IP header: Source
IP is always a client's IP address. However, destination IP could be the destination









server's IP address if transmitted via the Internet or an AID station's IP address if in
the AID tunnels.

* Source port (2 bytes) and destination port (2 bytes) in the TCP header: Unlike
destination IP, a packet's destination port is not changed when entering AID
tunnels.

* Sequence number (4 bytes) and acknowledgement number (4 bytes) in the TCP
header.

* (a) Recognizingfield (4 bytes) in the AID layer header.
(b) Server IP (4 bytes) in the AID layer header.
(a) and (b) are in the same position and have the same size. Its value is 0 for
normal packets, or destination server's IP for packets entering RON.

* Virtual clock timestamp in the AID layer: Only packets go through AID tunnels
have this field.

* Whole application layer data.

AID Layer for UDP Traffic

There are several different UDP messages used to control the AID system. They

are distinguished by the packet type field in the AID layer header. All of these messages

are integrity-protected.

PUSH Message

PUSH messages notify other AID stations a server is currently under attacks. Fig.

3-2 shows the content of a PUSH message. An AID station or registered server can send

PUSH messages. Md5 digest is for integrity protection. Packet type is set to 2 for PUSH

messages (defined in global.h). AIDNO means AID station number, which records how

many AID stations a packet will pass before reaching the server. It is essential for

establishing AID tunnels. More details are explained in later chapters. Service port and

server IP are the attacked server's IP address and port number.









IP header UDP header AID layer header application layer data


md5 digest 128 bits packet type 8 bits


AIDNO 8 bitsI service port 16 bits server IP 32 bits

Figure 3-2. PUSH message. The AID layer header is inserted between the UDP header
and application layer data.

PULL Message

PULL messages ask other AID stations what servers are currently under attacks.

IP header UDP header AID layer header a&l:l:li.: -ition layer data

(empty)
md5 digest 128 bits packet tpe 8 bits

Figure 3-3. PULL message. There is no application layer data in a PULL message.

Fig. 3-3 shows the content of a PULL message. There is nothing behind the AID

layer header. Md5 digest is for integrity protection. Packet type is set to 0 here (defined

in global.h).

PULLANS Message

When an AID station gets a PULL message from another AID station, the former

will return information of all currently attacked servers it knows to the latter. Sending

PULLANS messages does it. Fig. 3-4 shows the content of a PULLANS message.

Packet type is set to 1 for PULLANS messages (defined in globa.h). Every AID station

maintains a service list, which stores information of attacked servers. Each attacked

server is a service list node.

If an AID station has information of N attacked servers, there will be N nodes in the

service list to be sent out. In a service list node, Distance says that from this AID station









how many AID stations a packet still needs to pass to achieve the server, excluding the

first AID station. The distance information helps to construct AID tunnel trees.



A
IP header UDP header AID layer header application layer data


md5 digest 128 bits packet tpe 8 bits


service list node service list node I ......
service list
B
service list node {
unsigned int ip
unsigned short port;
unsigned char distance;
I

Figure 3-4. PULLANS message. A) The content of a PULLANS message. B) The
structure of the service list node. It contains IP and port of an attacked server.

CTRLT Message

T, the waiting interval, is for adjusting the speed of a virtual clock. When the

arrival rate is larger than a server's capacity, a bigger Twill be sent to AID stations to

accelerate virtual clocks.

IP header UDP header AID layer header application layer data


mrd5 digest 128 bits packet ype 8 bits


server ip 32 bits IT 64 bits


Figure 3-5. CTRLT message.









Packet type is set to 3 (defined as in global.h). A CTRLT message is for a specific

tunnel tree of the attacked server with server IP. Tfield contains the new value of Tfor

that specific tunnel tree. After getting a CTRLT message, an AID station updates its T

Most of UDP messages are related to RON maintenance and distributed virtual-

clock packet scheduling. We have not talked about them so far. They would be pointed

out in later chapters.

What fields are under integrity protection?

* Source IP (4 bytes) and destination IP (4 bytes) addresses in the IP header.

* Source port (2 bytes) and destination port (2 bytes) in the UDP header.

* Packet type (1 byte) in AID layer header.

* Whole application layer data.

Implementation Issues

Most UDP messages in our AID system have fixed size, except for the PULLANS

message. Its size depends on how many nodes in the service list. If there are many

nodes, the message packet will be too big to be sent out. In the circumstance, it should be

divided into two or more PULLANS messages. How big is too big? We defined a

constant, UDPMAXSIZE, in global.h. When a UDP message is bigger than

UDPMAXSIZE, it will be chopped up into several packets.














CHAPTER 4
CLIENT END

On the client end, we need to filter incoming and outgoing packets. For example,

when a TCP packet is leaving the client end, its destination needs to be checked. If the

destination server is under attacks, the packet will enter AID tunnels; otherwise, it is

routed as usual.

ClientFilter.c and client.c are two main source files for client ends. ClientFilter.c is

compiled as a module, queuing interested packets into userspace. Then, client.c takes

queued packets out and does whatever is necessary. After a client registers at an AID

station, Ac, it can get a secret key. The key is used to verify that the third party did not

modify the communication between the client and Ac. The client also keeps Ac's IP

address. If it tries to access an attacked registered server, its packet will be forwarded to

Ac.

When an AID station is informed that a server is attacked, it will send PUSH

messages to its registered clients. For instance, the client gets PUSH messages from Ac,

which is the AID station it registered at. All UDP messages in the AID system are sent to

port 4369. However, there is no program in application level listening on this port. UDP

packets to port 4369 are handled by the program client.

Module ClientFilter.o

By compiling clientFilter.c, we can get the module clientFilter.o. It hooks on

handling functions at hooks NFIPPREROUTING, NF IP LOCAL_OUT and

NFIPPOST_ROUTING. At NFIPPRE_ROUTING, only UDP packets to the port









4369 are queued. Other incoming traffic is not related to the AID system. At

NFIPLOCAL_OUT, all outgoing TCP packets are queued, except for the local traffic.

Local traffic goes from loopback interface, 127.0.0.1, to loopback interface. At

NF IP POSTROUTING, all outgoing TCP packets are queued, except for the local

traffic.

Only outgoing TCP packets are queued since currently only TCP traffic is

protected. When the module clientFilter.o is loaded in a host, the host must run the

program client as well. Otherwise interested packets keep getting into the queue, but no

programs take them out of the queue. The traffic is blocked if this happens. A host

should load the module clientFilter.o and run the program client at the same time. It is

meaningless to do just one of them.

Program Client

By compiling client.c and linking other relative source files, we can get the

executable program client. It has a while loop in main( whose condition is always true.

The program client deals with packets queued at different hooks by the module

clientFilter.o. Now, we discuss what the program client does to packets from different

hooks.

Packets from NF IP PRE ROUTING

All packets in the queue grabbed at the hook NFIP PRE_ROUTING are UDP

traffic to port 4369. For a client, the only UDP message of the AID system (to the port

4369) is PUSH. PUSH messages are sent by Ac to inform the client what servers are

attacked. Every client has a servList recording attacked servers (servList.h/servList.c).

When getting PUSH messages, the client is going to update its servList. PUSH messages

have md5 digest in it, for integrity checking. Others cannot pretend Ac to send PUSH









messages or pretend the client to send packets into AID tunnels as long as they do not

know the secret key shared between Ac and the client. These UDP packets do not go

further from here in the kernel. We mentioned no application level programs listening on

port 4369 earlier. The program client tells the kernel just drop them after it gets the

information of PUSH messages.

Packets from NF IP LOCALOUT

The program client processes all TCP packets leaving the client host. First, it

examines a packet's destination IP. If it finds a match in the servList (the destination

service is under attacks), it changes the packet's destination IP to Ac's IP and copy the

packet with new destination IP back to the kernel. The destination port is not compared

when searching a match in the servList. It is not necessary to distinguish different service

ports on an attacked server. With Ac's IP as destination, the packet is going to enter an

AID tunnel tree. If no match in servList, the program client just tells the kernel it did not

do anything to the packet and the kernel can continue passing on the packet.

Packets from NF IP POST ROUTING

All TCP packets come here after passing the hook NFIPLOCAL_OUT. The

program client inserts a bunch of data into every packet as AID layer header. If a

packet's original destination is an attacked server, its destination IP we can see here is Ac's

IP. It was modified in the hook NF IP LOCAL OUT. If the server is not under attacks,

we can see the server's IP as the packet's destination IP.

The program client inserts different AID layer headers into packets. If destination

IP was changed into Ac's IP, it means the packets are going to enter AID tunnels. As

shown in Fig. 4-1, application layer data is moved back 28 bytes, and an AID layer

header is put into that 28 bytes space. Virtual clock timestamp is initialized to the client's









local time. Md5 digest ensures Ac that the packets are really from the client and not

altered.


IP header TCP headerapplication layer data




IP header TCP header AID layer header application I e- data


server IP 32 bits (recognizing field) md5 digest 128 bits virtual clock timestamp 64 bits


Figure 4-1. Inserting an AID layer header to a packet that enters AID tunnels. A 28-
byte-long AID layer header is injected.

For packets not entering AID tunnels, their destination IP is still the server's IP.

These packets do not need md5 digest and virtual clock timestamp. Nevertheless, they do

need the 4-byte-long recognizing field. Fig. 4-2 shows how this sort of packets is dealt

with. Recognizing field is an unsigned integer with value 0. We explained why we need

it clearly in the chapter AID LAYER.


IP header TCP header application layer data




IP header TCP header AID layer header application l:--r data


recognizing field 32 bits

Figure 4-2. Inserting an AID layer header to a packet not entering AID tunnels. A 4-
byte-long AID layer header is injected.

Either packets going to AID tunnels or not, this hook is the final chance we can

modify them. After copying the modified contents back to the kernel, these packets are

sent out right away.









Implementation Issues

What Is ServList?

ServList is a simple list structure. It uses sequential search going through every list

node to find a match. A servList node contains an attacked server's IP and port, but

currently the port is not checked for a match in our AID system. A servList is updated by

PUSH messages from Ac.

Why Changes a Packet's Destination in Hook NFIPLOCAL_OUT?

The program client used to change a packet's destination IP in the hook

NFIPPOST_ROUTING, but it did not work as we expected sometimes. For example,

we have a server, an AID station and a client. Their IP addresses are 192.168.1.100,

192.168.1.101 and 192.168.1.103 respectively. Assume the server is under attacks, and

the client sends a packet to the server. Before getting in the hook

NF IP POSTROUTING, the packet's destination IP is server's IP, 192.168.1.100. After

leaving the hook but before really sent out, the packet's destination IP is the AID station's

IP, 192.168.1.101. Then the packet leaves the client. Unexpected things happen here.

The AID station does not get the packet, but the server gets it. If the server has

FORWARD chain in iptables set well, the packet may be forwarded to the AID station.

Anyway, it is not what we want. The packet should go to the AID station directly since

we changed the packet's destination IP. We concluded that we should not change the

packet's destination IP right before it leaves the host. We should do this in the hook

NFIPLOCAL_OUT instead, and everything goes well.

How Is the Registration Done?

It is lots of work and difficult to make a complete secure system. Registration may

be a secure hole in the system. In our AID system, clients get the secret key shared with









Ac by registration. We just took the easiest step here. The shared key was pre-configured

in both the client and AID station, Ac. If users want to change the key, they need to

redefine it in both sides, with the same value. Then recompile the codes. It is not hard.

We wrote a makefile compiling and linking object files to generate the program client. It

can be done by just one command. Clients also need Ac's IP address. It is not pre-defined

in the codes. It is given as a command argument when users run the program client. It is

possible to introduce public key system here, a better but complex way. We do not

consider it currently.

Not Perfectly Isolated from Higher-Level Applications

Our goal is to make the AID system completely independent of higher-level

applications. It means application level programs do not observe the existence of the

AID service. Unfortunately, considering AID layer is not handled in the kernel and it is

treated like application layer data, our AID system cannot be perfectly isolated from

higher-level application. All outgoing TCP packets are inserted an AID layer header.

When servers, which did not join the AID service, get these packets, their application

programs will find extra junk data, the AID layer header we appending. Network

communication follows protocols. The AID layer header is useless to the application

programs. Protocols may be violated and the communication will fail. Before

connecting servers that did not join the AID service, the module clientFilter.o should be

unloaded first.

The Maximum Transmission Unit (MTU) Problems

Akin to the last issue, some more problems are caused by inserting an AID layer

header that is not handled by kernel. We add 28 bytes into packets entering the AID

tunnels and 4 bytes into the other packets. Is it harmless to enlarge a packet like this?









The answer is not always true. Usually, the OS tries to buffer enough data to form big

packets to avoid small size packets by Nagle algorithm. Every packet has headers, if

only a few data inside, the ratio of headers in a packet goes high and it is inefficient.

If we telnet or ssh a server, it may be fine. If we ftp or sftp a server, the kernel will

try to buffer as many data as possible. Usually, a packet's size is as large as MTU. If we

add an AID layer header in a packet in this case, the packet's size will be larger than

MTU. The packet will be just dropped. We noticed this problem when testing ftp service

in the AID system. There are two ways to solve the problem. One is to disable the Nagle

algorithm, and the other is to strict the size of packets from higher-level applications.

After opening a TCP socket, we have a chance to set socket options by calling the

function setsockopto in C library. We can pass in the option TCPNODELAY to disable

the Nagle algorithm or the option TCP_MAXSEG to change the maximum segment size

for outgoing TCP packets. We chose the second way to keep the efficiency brought by

the Nagle algorithm.

It is another cause that the AID system is not totally isolated from higher-level

applications. Most application programs do not strict the size of outgoing TCP packets.

They fully take the advantage of Nagle algorithm. If programs tend to buffer data used in

the AID system, ftp clients for example, most of packets cannot be sent out because of

the huge size. It is easy to fix by setting the socket option TCPMAXSEG. However,

the application programs need to be recompiled. It can be fixed as well if we handle the

AID layer in the kernel, but we need to recompile kernel though.














CHAPTER 5
SERVER END

On the server end, it needs to tell where packets come from, the Internet or AID

tunnels. The server end also has to alert AID stations if it is attacked.

ServerFilter.c and server.c are two main source files for server ends. ServerFilter.c

is compiled as a module, queuing interested packets into userspace. Then, server.c takes

queued packets out and does whatever is necessary. After a server registers at an AID

station, As, it can get a secret key. The key is used to verify the communication between

the server and A, are not modified by the third party. The server also keeps A,'s IP

address. If it is under attacks, A, will be informed.

When a registered server is attacked, it will send PUSH messages to the AID

station, which the server registered at. This AID station called A,. All UDP messages in

the AID system are sent to port 4369. However, there is no program listening on this

port. UDP packets to port 4369 are handled by the program server. We can characterize

occurrences of DoS attacks in several ways. In our AID system, we use arrival rates,

average incoming bytes per second, to determine if a server is attacked. Each server has

its capacity, 10000 bytes per second for example. When the arrival rate is higher than its

capacity, the server is under attacks. We can include other definitions of DoS attacks into

the AID system easily in our implementation.

Module ServerFilter.o

By compiling serverFilter.c, we can get the module serverFilter.o. It hooks on

handling functions at NFIPPRE_ROUTING, and NFIPPOST_ROUTING. At









NF IP PRE ROUTING, all incoming TCP packets are queued, except for the local

traffic. Local traffic goes from loopback interface, 127.0.0.1, to loopback interface. At

NF IP POSTROUTING, only UDP packets to the port 4369 are queued. Other

outgoing traffic is not related to the AID system.

Only incoming TCP packets are queued since currently only TCP traffic is

protected. When the module serverFilter.o is loaded in a host, the host must run the

program server as well. Otherwise interested packets keep getting into the queue, but no

program takes them out of the queue. The traffic is blocked if this happens. A host

should load the module serverFilter.o and run the program server at the same time. It is

meaningless to do just one of them.

Program Server

By compiling server.c and linking other relative source files, we can get the

executable program server. It has a while loop in main() whose condition is always true.

The program server has two packet buffers, one for packets from the Internet and the

other for packets from AID tunnels. The former is called bufferN and the latter is called

bufferT. Because packets from AID tunnels have higher priority, the program server

intends to handle packets in bufferT first. Let us see what the program server does to

packets from different hooks.

Packets from NF IP PRE ROUTING

In this hook, the program server has to remove the AID layer header from every

queued packet. Before doing this, program server needs to know if the packet is from the

Internet or AID tunnels. The program server inspects the recognizing field. If it is 0, the

packet is from the Internet. If it is the server's IP (IP of the host runs the program server),









the packet is from the AID tunnels. If neither 0 nor the server's IP, the packet has wrong

contents and is dropped.

If the packet is from the Internet, it is put into the bufferN. It is dropped if bufferN

is full. The 4-byte-long recognizing field is removed from the packet.

If the packet is from AID tunnels, the whole AID layer header is 28 bytes long,

including the recognizing field, md5 digest and virtual clock timestamp. First, the

program server inspects the packet's integrity with md5 digest. If failing, drops the

packet. Then, the packet is put into the bufferT. Drops the packet ifbufferT is full.

Similarly, the whole 28-byte-long AID layer header is removed from the packet in the

program server. Fig. 5-1 shows how an AID layer header is removed for an incoming

TCP packet.

A


AID layer header
IP header TCP header headerapplication layer data

(remove AID layer header)



IP header TCP header application layer data


B
server IP 32 bits (recognizing field) md5 digest 128 bits virtual clock timestamp 64 bits

recognizing field 32 bits


Figure 5-1. Removing the AID layer header in server end. A) All incoming TCP packets
should have an AID layer header. Other parts of a packet are not tainted. B)
A packet from AID tunnels has a 28-byte-long AID layer header; otherwise its
AID layer header is 4 bytes long.









The program server does exactly contrary things to what the program client dose to

packets from hook NF IP POSTROUTING. The program client adds AID layer

headers on packets, and they are ripped out here. As a result, higher-level applications

have no idea of the existence of AID layer. When they get a packet, they see no data of

an AID layer header.

Packets from NF IP POST ROUTING

All packets in the queue grabbed at hook NFIPPOST_ROUTING are UDP

traffic to port 4369. The only UDP message belongs the AID system (to the port 4369)

that would be sent out by a server end is PUSH. PUSH messages are sent to As to say the

server is attacked. Before leaving a server, these queued packets will be appended 16-

byte-long md5 digest. It prevents the third party from forging PUSH messages and send

them to As.

Implementation Issues

No Threads

In the chapter AID System Overview, we pointed out no threads or child processes

in programs client, server and AID for printing out statistic information under users'

requests. Unlike the program client, the program server has one more thing to handle,

sending packets in bufferN and bufferT to higher-level applications. The program server

also has a while loop with a consistent true condition in main(). In every iteration, the

while loop examines two things. First, sees if there are packets in bufferT and bufferN.

A part of them are passed to higher-level applications. Second, sees if any packet was

queued by the module serverFilter.o and read in one packet. Each of them takes only

little time, so they look like running simultaneously. Threads make a program harder to









maintain and may cause serious problems like resource competition and deadlocks, if not

used very carefully.

Important Variables

There are some important variables defined in the source file server.c. They decide

the way the program server works. To make the program server work properly, they

need to be assigned reasonable values. We discuss them below.

PCKBUFSIZET

BufferT's size, if too small, packets from AID tunnels will be dropped frequently

with heavy incoming traffic. BufferT stores packets from AID tunnels.

PCKBUFSIZEN

BufferN's size, if too small, packets from the Internet will be dropped frequently

with heavy incoming traffic. BufferN stores packets from the Internet.

IPQREADTIME

In the section No Threads, we said two things are done every iteration in the while

loop of main(. One of them is to read in a packet queued by the module serverFilter.o if

the queue is not empty. If the queue is empty, the program server is blocked until a

packet is put into queue by the module serverFilter.o. If blocked, the packets in the

bufferN and bufferT cannot be sent to their destination, higher-level applications.

Consequently, we need to constrain the time of this reading behavior. Do not wait more

than IPQREADTIME microseconds if the queue is empty. If it is larger than 500000, the

client side may feel painful lags.

READINTERVAL

BufferT and bufferN are examined every iteration in the while loop of main(, and

some packets in the two buffers are sent to higher level applications. We are not sure









how long one iteration may take. We may want packets stay in the buffers longer than

the time of one iteration because we need to balance the traffic from AID tunnels and

from the Internet (packets from AID tunnels have higher priority). It can be done by this

variable. It defines how often packets in the two buffers are sent out. It is in

microseconds, too. It cannot be too small or the program server cannot control the

traffic. If the variable is too big, apparent lags appear.

SENDPCKBUFNO

Every READINTERVAL microseconds, packets in two buffers are taken out, but

how many? The variable is the answer. If it is 10, the 10 packets from the two buffers

can be sent out. Notice it is a total number for packets from both bufferT and bufferN.

Since bufferT has higher priority, if 10 packets are picked up this iteration from bufferT,

packets in bufferN have to wait until next iteration. If it is too small, the two buffers gets

full easily. Packets will be dropped frequently when traffic is heavy.

AVGINTERVAL

The program server calculates the arrival rate every AVGINTERVAL seconds. If

it is too small, the arrival rate may not be representative. If it is too big, the program

server may not be able to detect attacks in real time (not sensitive enough).

TOTALCAP

Capacity of the server end, in our AID system, was defined as how many bytes per

second the server end can handle. Once the arrival rate is higher than TOTALCAP, the

program server alerts the AID system to create a tunnel tree by sending PUSH messages

to A,.









RESERVEDTIMES

When under attacked, a server has traffic from both the Internet and AID tunnels.

We said the latter has higher priority, but how? The server handles data in bufferT and in

bufferN with the ratio RESERVEDTIMES: 1. For instance, if TOTALCAP is 1000 bytes

per second and RESERVEDTIMES is 4, 1000 x 4/(1+4) = 800 bytes per second is

reserved for the traffic from tunnel trees, and 1000 x 1/(1+4) = 200 bytes per second is

reserved for the traffic from the Internet.

How the Registration Is Done

In our AID system, servers get the secret key shared with As by registration. We

just took the easiest step here. The shared key was pre-configured in both the server and

AID station, A,. If users want to change the key, they need to redefine it in both sides,

with the same value. Then recompile the codes. It is not hard. We wrote a makefile

compiling and linking object files to generate the program server. It can be done by just

one command.

Not Perfectly Isolated from Higher-Level Applications

Same as the program client, the program server cannot be totally isolated from

higher-level applications because the Linux kernel does not actually handle AID layer

headers. AID layer headers are viewed as application layer data by the kernel. The

program client inserts an AID layer header into a packet and the program server removes

the AID layer header from the packet. It is a little confusing here. Is the program client

different from other client programs like telnet and ssh? Our program client does not try

to connect the server host. It takes care of queued packets in a client host instead.

Likewise, the program server is not a server daemon program. It does not listen on a









port. Its job is taking care of queued packets in a server host. When a client host tries to

connect a server host, there are four possibilities:

* The client host has clientFilter.o loaded and is running the program client, and the
server host has serverFilter.o loaded and is running the program server as well. It
works just fine in this case because both sides can recognize the AID layer headers
inserted in packets.

* The client host has clientFilter.o loaded and is running the program client, but the
server host does not load serverFilter.o. It does not work in this case because the
server host will get packets with AID layer headers from the client host, and the
server host cannot recognize them. AID layer headers are junk data for the server
host, which make communication fail.

* The client host does not load clientFilter.o, but the server host has serverFilter.o
loaded and is running the program server. It does not work in this case either
because the client host sends out packets without AID layer headers. When the
server host gets the TCP packets from the client host, it tries to know if the packets
are from AID tunnels or the Internet by inspecting the recognizingfield in AID
layer headers. Of course, these packets do not have the recognizing field and
application layer data is used as recognizing field. Then, communication fails.

* The client host does not load clientFilter.o and the server host does not load
serverFilter.o either. Both sides know nothing about AID layer headers. It works
well. In this case, the AID service has nothing to do with both sides.
Communication just goes as without the AID service protection as before.

In conclusion, if a client host wants to connect a server host that has serverFilter.o

loaded and is running the program server, the client host should load the module

clientFilter.o and run the program client before making a connecting. On the other hand,

if a client host wants to connect a server host that does not load serverFilter.o, the client

host should unload the module clientFilter.o and terminate the program client before

making a connection. The client host should match the server host to make everything go

well.

One thing worth a mention is that loading clientFilter.o and running the program

client do not mean the client host already joined the AID service. It should register at an

AID station to make the AID service effective first. Same thing applies to the server host









as well. However, an unregistered client host can still connect to a registered server host

by loading clientFilter.o and running the program client. All packets from that client host

cannot enter AID tunnels because the client has no secret key. They can only be

transmitted via the Internet..

Loading or unloading the module clientFilter.o can be done by one command. A

client host can adapt itself to different server hosts dynamically.

Program Alert

By compiling the source file alert.c and linking other relative source files, we can

get the executable program alert. A server host can send PUSH messages to AID stations

by executing the program alert. We leave the flexibility of defining DDoS attacks to the

users. Users can define the situations of being attacked to meet their need. All they need

to do is to run the program alert when the server host detects attacks. It will send A, a

PUSH message to trigger the AID system. Afterward, all packets from registered clients

to that server go through the AID tunnels. The program server inserts Md5 digest into

the PUSH message packets at hook NF IP POSTROUTING.














CHAPTER 6
AID STATION

AID stations are the cores of our AID system [6]. They form an AID tunnel tree

for each attacked registered server. How is a tunnel tree created? How are packets

routed in a tunnel tree? How to resist attacks from registered clients? How was it

implemented? Answers to the above questions are in this chapter.

AID Tunnel Tree

We have seen AID tunnels many times in the previous chapters. We know AID

tunnels are tree structures. Packets from registered clients to the attacked registered

servers would enter AID tunnels. In this section, we explained how an AID tunnel tree is

constructed and how packets are routed inside. The push-n-pull process [6] establishes a

tunnel tree from the registered clients to an attacked server.

Push Phase

Assume a server S is attacked; it sends a PUSH message to As, the AID station that

it registered at. An AID tunnel tree for the server is going to be built up, and the tree's

root node is As. The scenario is as follows.

1. Server S senses an attack and sends PUSH messages to A,. A, is the root node of
the AID tunnel tree, which called the first level node.

2. A, is the only AID station that knows S is under attacks so far. A, picks up k other
AID stations randomly, and sends a PUSH message to each of them. Any other
AID station could be selected. Subsequently, k+1 AID stations know S is under
attacks at the end of the step. We call these k AID stations the second level nodes
in the tunnel tree. How big should k be? We have deep discussion about it later.

3. Every second level node randomly picks up k other AID stations, and sends a
PUSH message to each of them. Any other AID stations could be selected except









for As. So, the k second levels nodes selected k2 nodes totally. We call these k2
nodes the third level nodes in the tunnel tree.

Notice that a node might be picked up more than once in the step 2 and in step 3

because both steps randomly select k AID stations. Fig. 6-1 shows how is a tunnel tree

created in push phase.


second level node
third level node
0 unconnected node


Figure 6-1. Tunnel tree created in push phase. Not all third level nodes are shown in the
figure. Arrows from nodes to nodes indicates the direction packets would be
routed. See the broken arrows in the figure. Node A got PUSH messages
from both A, and B. A would choose As as its parent node because of shorter
routing path. Similarly, node D got PUSH messages from B and A. D could
pick either of them to be its parent node, but not both. In push phase, there
might be some AID stations not receiving PUSH messages, which are
unconnected nodes in the figure.

Suppose we have N AID stations. We want to notify every AID station when a

server is attacked. In step 1, only A, is notified. In step 2, k+1 AID stations are notified.

In step 3, ideally, 1+k+k2 AID stations are notified. If k is the square root of N, we get

1+k+k2 > N, which means the tunnel tree covers every AID station. However, some AID









stations picked in step 2 may be picked again in step 3 and some second level nodes may

select the same third level nodes. We cannot guarantee every AID station is included in

the tunnel tree. That is why we need pull phase. In push phase, k(k+l) PUSH messages

are sent out totally, because only the first level node and the second level nodes would

send PUSH messages. If we allow the third level nodes to send PUSH message, push

phase will be expensive. The majority of AID stations can be reached in push phase.

Pull Phase

When a server detects an attack, a tunnel tree rooted at A, is built up. Some nodes

might not get the PUSH messages in push phase. These nodes did not connect to the

tunnel tree yet. We try to include them into the tunnel tree in pull phase. In pull phase an

AID station will ask other AID stations by sending PULL messages what servers are

attacked. In push phase, an AID station gets information of attacked server passively.

If an AID station B gets PULL messages from another AID station A, B will send

PULLANS messages back to A. PULLANS messages contain information of all attacked

servers B knows. When A gets these PULLANS messages from B, it will update its

attacked servers recording. Actually, an AID station sends PULL messages to q other

AID stations. Like the variable k in push phase, we should choose a proper q. We will

discuss q and k later. Each AID station sends PULL messages out periodically.

Routing

A, is the first AID station that knows the register server S is attacked, and A, is also

the root of the tunnel tree for the server S. When an AID station A tells another AID

station B that server S is attacked by either PUSH or PULLANS messages, A becomes B's

parent node in the tunnel tree for the server S. Hence, A, is the parent node of the second

level nodes, and the second level nodes are the parent nodes of the third level nodes.









The structure of a tunnel tree changes dynamically because of PULLANS messages. In

pull phase, if a third level node gets PULLANS messages from As, the only first level

node, it will switch its parent to A, and become the second level node. Then, the routing

path becomes one AID station shorter. A, is the root node, all TCP packets to the server S

are routed to A, finally.

If an AID station gets a packet whose final destination is server S, where the packet

is routed to next? The AID station routes the packet to its parent node. The third level

nodes route packets to the second level nodes, and the second level nodes route packets

to A,. Packets go from leaf nodes to the root node in the tunnel tree. Finally, As forwards

packets to the server S.

An AID station could be a tree node of more than one tunnel trees. If N servers are

attacked, there will be N tunnel trees built up on the random overlay network (RON).

One tree is independent from another.

Why Does a Tunnel Tree Try to Include Every AID Station?

Two reasons here, first, a client is free to register at an arbitrary AID station.

Suppose server S is attacked and client C is trying to connect S. C registered at AID

station Ac. A tunnel tree would be created for server S. If the tunnel tree does not

embrace Ac, Ac does not know S is under attacks. In this case, C would not be informed

by Ac that S is attacked. Thus, client C keeps sending packets to server S via the Internet.

These packets are not protected even though C did register and joined the AID service.

We do not constrain which AID station can have registered clients, so we tries to include

all AID stations.

Second, it is about routing. AID station A routes packets to AID station B if B is

A's parent node in the tunnel tree. A got PUSH or PULLANS messages from B before. B









also routes these packets to its parent node. If B crashes and then restores, it will lose

information about its parent node of the tunnel tree. Now, B does not know where to

route the packets from A. B sends PULL messages to others when getting a packet that it

does not know where to route. If one PULLANS message B got contains information it

needs, where to route packets to server S, B hooks on the tunnel tree again. However, it

is possible that all PULLANS messages B got contain nothing about server S. If we

choose right k and q, the chance that this happens is very low.

Variables k and q

Assume we have a set of n AID stations. AID stations A, B and As are elements of

the set. In step 2 of push phase, A, sends PUSH messages to other k AID stations. The


probability thatA does not get the PUSH message from As is Now, suppose
in--1 "

AID station B got a PUSH message from A, and is the second level node. In step 3, B

would not send PUSH messages to A, and itself, so B could send PUSH messages to other

k nodes out of (n 2). The probability that A does not get the PUSH message from B in

n-2-k
step 3 of push phase is i Since there are k second level nodes, the probability


(n-2-k'
that A does not receive any PUSH messages in step 3 of push phase is n- A
n-2 "

does not connect to the tunnel tree right after push phase if it obtained no PSUH message

in step 1, 2 and 3. Finally, we got the probability that A is not covered by the tunnel tree

Ss s i n-1-kk n-2-k k_
after push phase is So, the probability, called PinTree, that an
arbitrary AID station A could be in the tunnel tree after push phase is
arbitrary AID station A could be in the tunnel tree after push phase is











n-l-k x n-2-k The expectation number of AID stations included in the
n-1l n-2

tunnel tree after push phase is nxPinTree.


Theorem 1: if n > 2 and k = Fn then PinTree > 1 1
e


Proof: 1- l-kxn-2- >1--
n-1 n-2 e

n-l-k1 (n-2-k k
.n l-k_<-
n-1 n-2 e

n-l-k k+1
n-1 e

n-k 1
n) e

1-- <-
n e

S 1-- <-I
n e

1 1
S 1- <-
Se


-1 is a monotonically-increasing function and is equal to when n is
n e


approaching infinity. Hence, if n > 2 and k = 4n, PInree > 1 -1 We use k = b in
e

our AID system.

Let us talk about variable q now. II, = nxPITree AID stations are expected to be


covered in the AID tunnel tree right after push phase and we know nXPinTree


> n 1 by Theorem 1. An AID station sends out PULL messages to q other AID
ee e ee Ie e ee

stations. The probability that at least one of these q AID stations is in the tunnel tree is









n -1-j q n le]q 1
n-1 n-1 ) e4

If q = 10, the probability is greater than 0.99995. It is high enough that we can

almost say it will happen. When an unconnected AID station receives PULLANS from

another AID station that is in the tree already, it connects to the tree. With push-n-pull

process, the chance that all AID stations are included in the tree is very high.

Advantages of Random Overlay Network (RON)

First, small diameter and modest nodal degree: A good overlay network topology

should have small diameter and nodal degree. Unfortunately, they conflict with each

other. We made a tradeoff in our RON topology. We have a fixed small diameter that is

three and modest nodal degree that is about the square root of the number of all AID

stations. With small diameter, packets can arrive at servers by passing few AID stations.

With modest nodal degree, we save some resource and keep the availability against node

failure. We explain why the diameter is three in the end of this chapter.

Second, easy to set and maintain: A tunnel tree is established by sending PUSH and

PULL messages. They are sent randomly by an AID station to other AID stations. We

do not need a complicated algorithm to create the tree. Besides, every tree node just has

to know who is its parent to forward packets. Not many things need to be remembered

by an AID station. Capacity of the AID system can be increased by adding more AID

stations.

Third, against node failure: If a tree node, an AID station, is down somehow, its

children nodes are disconnected from the tunnel tree. However, the children nodes can

hook on the tree again in next pull phase. Therefore, we can easily shutdown an AID

station for maintenance without affecting the entire AID system. It is also true for adding









an AID station. A new-added AID station can connect the tree in pull phase as well. An

AID station can join and leave the AID service with little damage.

Distributed Virtual-Clock Packet Scheduling

Basic Idea

Assume we have an attacked server S and a tunnel tree for S. Every AID station

maintains a virtual clock VC, [11] initializedd to be the local system time) for every

tunnel u connecting with a client network. When an AID station gets a packet from

tunnel u, VC, is updated as follows [6].

VCu = max {VCu, current time} + T x L (2)

The AID station then marks the packet's virtual clock timestamp as VCu. All AID

stations' local clocks should be synchronized. L is the length of the packet. Tis called

waiting interval. A, broadcasts a new Tto all AID stations periodically by sending

CTRLT messages. We use Tto control the speed of a virtual clock. Since Tcan be

changed dynamically, we can adjust a virtual clock's speed dynamically as well. The

maximum rate a client can send data to server S via RON is 1/T. If an AID station gets a

packet from tunnel u connecting to another AID station, the packet has a timestamp on it

already.

An AID stations puts all incoming TCP packets into a buffer in ascending order

based on their virtual clock timestamps. When the buffer is full, the packet with largest

timestamp will be dropped. We call a packet's virtual clock timestamp VCTS. If VCTS

-"the AID station's local time > a is true for an incoming packet, the packet is just

dropped, not put into the buffer even though the buffer is not full. The value of a can be

configured in the program. If a registered client hosts an attacker, its virtual clock will

run very fast because of huge amount of traffic. Most of packets from the client will be









dropped since their virtual clock timestamps are too big. In this way, server's capacity is

shared fairly among all clients.

How to Adjust T

Server S reserves part of its capacity, called Cs, for RON. Tis set to 1/ C, at first

and broadcasted to all AID stations by A,. There are two phases to adjust T. In each

phase, new Tis broadcasted to every AID station.

* Exponential phase: In this phase, A, doubles the value of Tto make virtual clocks
run twice faster. When virtual clocks run twice faster, the maximum arrival rate of
server S from RON is cut by half. A, keeps in exponential phase until the arrival
rate is below Cs. Then, A, enters linear phase.

* Linear phase: Suppose Tis changed from Ito 21by the last update of Tin
exponential phase. In linear phase, A, decrease Tby I periodically to slowdown
virtual clocks until arrival rate is above Cs. We call the system converges at the
moment. Then, A, may enter exponential phase again.

Programs for an AID Station

AID stations route packets from registered clients, form a tunnel tree for an

attacked registered server and control the traffic flows of tunnel trees.

AIDFilter.c and AID.c are two main source files for AID station. AIDFilter.c is

compiled as a module, queuing interested packets into userspace. Then, AID.c takes

queued packets out and does whatever is necessary. An AID station keeps information

about its registered clients and servers. An AID station routes TCP packets in a tunnel

tree, and uses UDP messages to control the AID system.

Module AIDFilter.o

By compiling AIDFilter.c, we can get the module AIDFilter.o. It hooks on

handling functions at NFIPPRE_ROUTING, and NFIPPOST_ROUTING. At

NFIPPREROUTING, all incoming TCP packets and UDP packets to port 4369 are

queued, except for the local traffic. Local traffic goes from loopback interface, 127.0.0.1,









to loopback interface. At NF IP POSTROUTING, all outgoing TCP packets and UDP

packets to port 4369 are queued, except for the local traffic.

Queued TCP packets are traffic inside tunnel trees. They come from registered

clients and head for registered servers. When the module AIDFilter.o is loaded in a host,

the host must run the program AID as well. Otherwise interested packets keep getting

into the queue, but no program takes them out of the queue. The traffic is blocked if this

happens. A host should load the module AIDFilter.o and run the program AID at the

same time. It is meaningless to do just one of them.

Program AID

By compiling AID.c and linking other relative source files, we can get the

executable program AID. It has a while loop in main( whose condition is always true.

The program AID has a packet buffer, storing incoming TCP packets.

TCP packets from NFIPPRE_ROUTING

Every queued incoming TCP packet in an AID station will go through the

following processes.

* Verify the third party did not alter the packet. The packet is dropped if md5 digest
stored in the packet is not equal to the one calculated by the AID station.

* Packet's virtual clock timestamp is refreshed as described in the section Distributed
Virtual-Clock Packet Scheduling.

* Examine the packet's virtual clock timestamp. If VCTS -"the AID station's local
time > a, the packet will be dropped. Constant a was defined as
MAXVCTSEXCEED in AID.c. Most of offending packets are filtered out here.

* The AID station look up its routeList to know where to route the packet. Every
AID station has a routeList, which is a list structure. A routeListNode contains
information for a tunnel tree, inclusive of server's IP, distance to As, and the parent
node's IP. An AID station may be embraced in more than one tunnel trees, and its
routeList will contain more than one routeListNode. If The AID station does not
know where to route the packet, no information for the destination server stored in









the routeList, the AID station will drop the packet and send PULL messages to
other AID stations.

* If a packet passes all of the above and the packet buffer is not full, it can be put into
the packet buffer. If the packet buffer is full, the packet with maximum virtual
clock timestamp in the packet buffer is selected. The incoming packet and selected
packet are compared in their virtual clock timestamps. If the incoming packet has
smaller timestamp, it can replace the selected packet in the packet buffer;
otherwise, it will be just dropped. The packet's destination IP is modified to the
parent node's IP, since the packet will be routed to the parent node in the tunnel
tree.

UDP packets from NF IP PRE_ROUTING

Every queued incoming UDP packet in an AID station will go through the

following process.

* Verify the third party did not alter the packet. The packet is dropped if md5 digest
stored in the packet is not equal to the one calculated by the AID station.

* Recognize what kind of UDP message the packet is. In the chapter AID Layer, we
said there are a couple of different UDP messages in the AID system. We can tell
it by the packet's packet type field in the AID layer header. If it is a PULL
message, the AID station sends whole information of its routeList in PULLANS
messages to the asking AID station. It happens in pull phase. If it is a PULLANS
message, the AID station updates its routeList with the data of the packet. It
happens in pull phase. If it is a PUSH message, the AID station updates its
routeList with the data of the packet and sends PUSH message to other AID
stations. It happens in push phase. If it is a CTRLT message, the AID station
updates the variable Tfor the specific tunnel tree. A CTRLT message contains IP
of the server that the tunnel tree is for. See the chapter AID Layer for more details
about different UDP messages. A UDP message has md5 digest. It cannot be
forged without knowing the secret key.

TCP packets from NFIPPOST_ROUTING

The AID station is going to forward every queued packet to its parent node. The

destination IP was changed to the parent node's IP in the hook NFIPPRE_ROUTING

already. Here, the program AID recalculates md5 digest because the packet's destination

IP and virtual clock timestamp were changed. Finally, the program AID computes the


checksums in the TCP header and IP header.









UDP packets from NFIPPOST_ROUTING

Every queued UDP packet in this hook heads to port 4369. Md5 digest need to be

calculated and inserted into every queued UDP packet. The checksums in the TCP

header and IP header are recomputed in this hook. Afterward, the packets are ready to

leave the AID station.

Implementation Issues

No Threads

In the chapter AID System Overview, we pointed out no threads or child processes

in programs client, server and AID for printing out statistic information under users'

requests. Besides getting a packet from the queue, the program AID has three more

things to do, sending packets in the packet buffer to higher-level applications, sending out

CTRLT messages and sending out PULL messages. The program AID also has a while

loop with consistent true condition in main(). In every iteration of the while loop, four

things are examined. First, sees if the packet buffer has packets waiting and passes some

packets to higher-level applications. Second, sees if it is time to send out CTRLT

messages. We can define how often an AID station can send out CTRLT messages.

Third, sees if it is time to send out PULL messages. We can also define how often an

AID station can send out PULL messages. Fourth, sees if any packet was queued by the

module AIDFilter.o and read in one packet. Each of them takes only little time, so they

look like running simultaneously. Threads make a program hard to maintain and may

cause serious problems like resource competition and deadlocks, if not used very

carefully.









Registration for Clients and Servers

As we said before, the secret keys shared with clients and servers are pre-

configured in the codes. If an AID station wants to register a client, the client's

information needs to be added into AID.c. It is also true for registering a server. The

program AID has to be recompiled after registration, which can be done by one

command.

Important Variables

There are some important variables defined in the source file AID.c. They decide

how the program AID works. To make the program AID work properly, they need to be

assigned reasonable values. We discuss them below.

PCKBUFSIZE

The packet buffer's size, if too small, incoming TCP packets will be dropped

frequently with heavy traffic.

IPQREADTIME

We mentioned four things are done every iteration in while loop of main(. One is

to read in a queued packet. If the queue is empty, the program might be blocked until a

packet is queued by the module AIDFilter.o. If blocked, the packets in the packet buffer

cannot be got by higher-level applications. Consequently, we need to constrain the time

of this reading behavior. Do not wait more than IPQREADTIVME microseconds if the

queue is empty. If it is larger than 500000, the client side may feel painful lags.

READINTERVAL

The packet buffer is examined every iteration in the while loop of main(, and some

packets are sent to higher-level applications. We are not sure how long one iteration may

take. We may want packets stay in the packet buffer longer than the time of one









iteration. It can be done by this variable. It defines how often the packet buffer is

checked. It is in microseconds, too.

SENDPCKBUFNO

Every READINTERVAL microseconds, packets in the packet buffer are taken out,

but how many? The variable is the answer. If it is 10, the 10 packets from the packet

buffer can be sent to higher-level applications. If it is too small, packets will be dropped

frequently when heavy traffic.

NEARBYAID

The number of other AID stations that are known by this AID station. PUSH,

PULL, and CTRLT messages are sent to these neighbors.

SNEDTINTERVAL

If the AID station is As, a root node of a tunnel tree, it sends CTRLT messages to

all other AID stations every SENDTINTERVAL seconds.

SENDPULLINVAL

Every SENDPULLINVAL seconds, an AID station sends PULL messages to other

PULLNO (defined in global.h) AID stations. Periodically sending out PULL messages

can make sure every AID station connects tunnel trees.

DECREASERATIO

In linear phase, A, decreases Tby I. DECREASERATIO is .

MAXVCTSEXCEED

If a packet's VCTS "AID station's local time" > MAXVCTSEXCEED, the packet

is dropped. MAXVCTSEXCEED is the constant a .









Adding New AID Stations

A new added AID stations can be included into an AID tunnel tree by sending

PULL messages to others. However, we still need to make the AID station known by all

other AID station. Like handling registrations, an AID station pre-configures its

neighbors in AID.c. When a new AID station is added into the AID system, its

neighbor's AID.c needs to be updated and recompiled. It can be improved in a better but

complex way.

Diameter of a Tunnel Tree

Remember the AIDNO field of a PUSH message and distance field of a server list

node of a PULLANS message? Actually, they two mean the same thing, the distance to

As of a tunnel tree. We explain why our random overlay network's diameter is three here.

Every tree node, an AID station, records its distance to As. For As itself, the distance is 0.

For the second level nodes, it is 1. For the third level nodes, it is 2. Suppose we have

tree node A with distance of 1, tree node B with distance of 2 and tree node C not

included in the tree yet. C has two ways to join the tunnel tree.

* Another node sends a PUSH message to C. It could be root As or node A. If C gets
PUSH messages from As, C's distance to As is 1. If from A, C's distance to As is 2.
Notice only the root node and the second level nodes can send PUSH messages or
push phase becomes expensive (more than k(k+l) PUSH messages sent out).

* C gets PULLANS messages from another AID station. It could be root node As,
node A or node B. If C gets PUSH messages from As, C's distance to As is 1. If
from A, C's distance to As is 2. If from B, C's distance to As is 3.

Assume we have another node D not included in the tree. D can join the tree by

PUSH or PULLANS from node A or node B, but not node C. D ignores PULLANS

message from the nodes with distance of 3. If D joins the tree by C's messages, C

becomes D's parent node. That means D has distance of 4 to As, which is not allowed.









If C's distance is 3 and it gets PULLANS or PUSH messages from A, C will switch

its parent node to A to have smaller distance, 2. C becomes the third level node after

switching. An AID station's distance to A, can only go smaller. By doing this, no cycle

appears in a tunnel tree. As a result, distance between A, and every other tree nodes is

not larger than 3. Actually, we can reset the diameter by redefining PUSHDEEP in file

global.h. A tunnel tree's diameter is PUSHDEEP+1. There are four possibilities of a

packet being routed from a registered client to a registered server, shown in Fig. 6-2.

1. server--As(Ac)<-- client
-1 0 1

2. server As ---2nd AID station (Ac) client
-1 0 1 2

3. server -As--2nd AID station -3rd AID station (Ac)---client
-1 0 1 2 3

4. server-- As+-- 2nd AID station +--3rd AID station 4th AID station(Ac)-client
-1 0 1 2 3 4

Figure 6-2. Four possibilities a packet can be routed. The numbers are the value of
AIDNO field of PUSH messages or distance field of PULLANS messages
sent by the host. Packets are routed toward As in an AID tunnel tree.

Forwarding Packets

Pay attention to the word "forwarding." An AID station works like a router

somewhat. Packets in RON are forwarded to As by AID stations. The difference is

normal routers do not adjust md5 digest or virtual clock timestamp of a packet. Usually,

the forwarding function is turned off in a Linux machine by default. It has to be on.

After version 2.4, the tool iptables is available in Linux. We use it to set forwarding rules

in an AID station. To be simple, no sophisticated rules are used. We just allow all kind

of forwarding. Of course, we can set more secure and elaborate forwarding rules. An






48


AID station is not allowed to send out its own TCP packets. It can only forward TCP

packets.














CHAPTER 7
TESTING RESULTS AND ANALYSIS

After going through the previous chapters, we understand how the AID system

works theoretically and practically. In this chapter we talk about how we tested our AID

system and analyze testing results. When an idea is transformed into real programs,

unexpected problems show up always. We already discussed some of them in

Implement Issues sections of previous chapters. The other practical problems are

illustrated in this chapter.

Important Issues about Testing

* The root access is required to load a module. A client host, a server host or an AID
station needs to load clientFilter.o, serverFilter.o and AIDFilter.o respectively. We
do not have enough Linux machines with the root access for tests. As a
consequence, we just tested our AID system on three Linux machines, for a client
host, a server host and an AID station separately. We might not test some
functionality well with such a simple model.

* Since only one Linux machine is for client hosts, we have to simulate n client hosts
on it by running n client processes. The AID station treats TCP packets from
different source ports as they are from different hosts, even though they have the
same source IP.

* A client process uploading a huge file to the server symbolizes an attacker.
However, normal ftp programs cannot be used because of MTU problems. Packets
sent out by a normal ftp client could be as big as MTU. There is no space for the
AID layer header. We discussed this problem in the chapter CLIENT END in
detail. Hence, we wrote two programs for this purpose, testServer and testClient.
The server runs the program testServer to accept connections, and the client runs
the program testClient to dump data to the server. In the program testClient, we
can restrict the size of packets sent to testServer by resetting the socket option. If
the packet size is smaller, testServer will get more packets (but same amount of
application layer data).

* What we want to see from the testing are how fast the AID system converges, how
T (waiting interval) changes, how many packets from legitimate users are dropped,









how many packets from attackers are dropped, how T affects the average arrival
rate, and etc. Many factors can influence the above behaviors, for example the
packet buffer's size. Most of these factors are malleable variables in the codes.

Testing Elements

We have five testing cases. Client Linux machine had several processes of the

program testClient to simulate more than one client ends. Each process of the program

testClient might be given different parameters. Given parameters decided if a client end

was a legitimate user or an attacker.

Program TestClient

Usage of program testClient is

testClient testServerlP blockSize blockNO MAXSEG sleepTime

TestClient is the filename of the executable. TestServerlP is the IP address of the

host that runs the program testServer. TestClient will dump data there. The remaining

four parameters are more meaningful. There is a for loop, which runs blockNO iterations

in the program testClient. In every iteration a block whose size is blockSize is sent to

testServer. It indicates the size of application layer data, not the whole packet. Because

of headers, more than blockSize bytes are sent out in an iteration. With blockSize and

blockNO, testClient knows how many application layer data it needs to send out, which

are blockSize x blockNO bytes. The parameter MAXSEG defines the maximum size of

TCP packets. Notice that the size of the whole packet, including IP header, will be a little

bit bigger than MAXSEG. We need a suitable MAXSEG to save enough space for an AID

layer header. The last parameter, sleepTime, defines how many seconds the program

testClient sleeps before running the next iteration.









Program TestServer

TestServer accepts connection requests from testClient and prints out application

layer data sent by testClient.

Setting of the AID System

We have three Linux machines, one client, one AID station and one server. Table

7-1 shows the basic settings we used for testing. We explained what these factors mean

in previous chapters. Server's capacity was 2000 bytes per second. 1600 bytes per

second of it was reserved for traffic from AID tunnels. The setting was fixed during

testing but the number of attackers and legitimate users varied. The attacking modes

changed in different testing cases too.

Table 7-1. List of important factors of the AID system for testing
Name Value
PCKBUFSIZET 50
PCKBUFSIZEN 50
IPQREADTIME 300000
READINTERVAL 300000
SENDPCKBUFNO 3
AVGINTERVAL 10
TOTALCAP 2000.0
RESERVEDTIMES 4.0
MAXVCTSEXCEED 4
DECREASERATIO 0.1

Case 1

There were two registered attackers. Their parameters were:

* Attackerl: testClient 192.168.1.102 1000 500 1000 0
* Attacker2: testClient 192.168.1.102 1000 500 1000 0
Fig. 7-1 shows how many packets were dropped because of big VCTS at the AID

station. MAXVCTSEXCEED is 4. Fig. 7-2A shows how variables T, I and decrease

changed their values. Twas changed from I to 21by the last update of T in exponential

phase and then entered linear phase. In linear phase, A, decreased Tby decrease









periodically to slow down virtual clocks. Decrease is equal to DECREASERATIO times

I. Fig 7-2B shows the arrival rate at the AID station and the server. AvgTis the arrival

rate of the tunnel tree at the server. AvgNis the arrival rate of the Internet at the server.

AvgAID is the arrival rate at the AID station. All of them are average received bytes per

second in the 10-seconds period. 'AID cap' is part of server's capacity that was reserved

for the registered clients. 'Server cap' is server's whole capacity. In the third 10-

seconds, avgN exceeded server's capacity, and the AID system was triggered. Two

attackers started to send packets via RON, instead of the Internet. We can see avgNwent

down and finally to 0. Now, we examine Fig. 7-2A and Fig. 7-2B together. The AID

system converged after the twelfth 10-seconds. Tranged between 0.000875 and 0.00175

when converging. When Twent down, avgAID went up. Tkept decreasing to I, in linear

phase, until AvgAID was bigger than 'AID cap'. At the moment, the AID system entered

exponential phase again. When T doubled, avgAID declined dramatically. When

avgAID became smaller than 'AID cap', the AID system entered linear phase. The AID

system prevented avgAID from exceeding 'AID cap' by tuning T. If no new attackers

joined, after the AID system converged, Twould fall into a fixed range as we can see in

Fig. 7-2A. Table 7-2 shows how many packets and why they were dropped. About 2/3

of incoming packets were dropped.

Table 7-2. Case 1 packets statistics in the AID station. A packet is dropped when the
AID station's packet buffer is full, VCTS- "AID station's local time > a ,
integrity checking fails or the AID station does not know where to route the
packet.
Attacker 1Attacker2
Packet# in 1294 1230
Packet# out 487 456
Packet# dropped BufferFull 0 BufferFull 0
BigVCTS 807 BigVCTS 774
MD5Fail 0 MD5Fail 0
CantRoute 0 CantRoute 0











Mutual clock timestamp localTime


- attackers
- attacker
MAXVCTSEXCEED


'1


packet#

Figure 7-1. Distribution of incoming packets in case 1 at the AID station. Packets lay
above the straight line MAXVCTSEXCEED were dropped.


controlT (Fig. 7-2A) 1


0 002


( (N CO CO 10 seconds D
10 seconds


0O CO CO
I- o o Y) 00) 0;


arrival rate (Fig. 7-2B)


LD m) CO LD m) CO D 10 seconds L ) CO -
10 seconds


Figure 7-2. How did T and arrival rate interact with each other in case 1. A) After
system converged, Tpulsed in a fixed range. B) When avgAID exceeded
'AID cap', T doubled. Then avgAID fell again. Most of time, avgAID was
below the yellow straight line, showing the arrival rate was controlled well.


. 0 0015
-
P 0 001
S00005
n-


0


E7









Case 2

Similar to case, however, we added one legitimate registered client:

* Attackerl: testClient 192.168.1.102 1000 500 1000 0
* Attacker2: testClient 192.168.1.102 1000 500 1000 0
* Normal user: testClient 192.168.1.102 250 700 250 1
The normal user was distinguished from two attackers in three ways. First, it sent

smaller amount of data. Second, the size of packets from it was smaller. Third, it slept 1

second every iteration. Tranged between 0.001125 and 0.00225 when converging. It

was bigger than in case 1 because we had three registered clients here. In Fig. 7-3, in the

same time period, normaluser sent about twice packets as many as an attacker did

because TCP had flow control mechanism. After many packets were dropped; attackers

would slow down their traffic. In Fig. 7-3, we know packets from the legitimate was

really safe because VCTS localTime was in the range of {0.5, 1} approximately, which

was far away from 4. Normaluser was influenced by T much less harshly than attackers

were. Likewise, Fig. 7-4A and Fig. 7-4B show how the AID system quelled arrival rate

by adjusting T. Table 7-3 shows packets statistics of case 2.

Table 7-3. Case 2 packets statistics in the AID station. Normaluser had no packets
dropped.
Attackerl Attacker2 Normal user
Packet# in 473 474 952
Packet# out 173 172 952
Packet# BufferFull 0 BufferFull 0 BufferFull 0
dropped BigVCTS 300 BigVCTS 302 BigVCTS 0
MD5Fail 0 MD5Fail 0 MD5Fail 0
CantRoute 0 CantRoute 0 CantRoute 0













VCTS localTime


6 -


-- attackers
-- attacker2
normal user
IAXVCTEXCEED


5-
0








1-



1 41 81 121161201241 281321361 401441481521 561 601 641 681 721 761 801 841 881 921

packet#


Figure 7-3. Distribution of incoming packets in case 2 at the AID station. Traffic from
normal_user was pretty stable (the lowest series).


controlT (Fig. 7-4A)


--I -decrease


0.0025
. 0.002
S0.0015
- 0.001
* 0.0005
0 -


3000
o 2500
. 2000
o 01500
> ( 1000
500
0


/


I 0 0a l tC 0) C O O0 a I 0 1 ts.c 0) nd
0 C4 CN CI C- CI- LO
10 seconds


I 0 0 0o t. C O 0") CN a) O0 rI 0 0o t. C O 0")

10 seconds


Figure 7-4. How did T and arrival rate interact with each other in case 2. When T
doubled, avgAID and avgTfell.


I I









Case 3

There were four registered attackers, and two registered legitimate clients. Notice

that two legitimate clients sent out different amount of data with different packet sizes.

Their parameters were:

* Attackerl: testClient 192.168.1.102 1000 500 1000 0
* Attacker2: testClient 192.168.1.102 1000 500 1000 0
* Attacker3: testClient 192.168.1.102 1000 500 1000 0
* Attacker4: testClient 192.168.1.102 1000 500 1000 0
* Normal userl: testClient 192.168.1.102 250 700 250 1
* Normal user2: testClient 192.168.1.102 100 700 100 1
Tranged between 0.00175 and 0.0035 when converging. It was even bigger than in

case 2 because we had six registered clients here. Normaluserl, having exactly the

same parameters as normaluser in case 2, had packets dropped. In case 2, normaluser

had no packets dropped. What made the difference to the clients with the same

parameters? When the traffic load in the AID system is heavier (Tgoes bigger), packets

have higher chances to be dropped even though they are not from hosts intending to

attack. That is because the AID system tries to make a fair share of resource among all

registered clients. If a client sends more, it has to wait longer for next sending. Packets

from normal_user2 had small enough VCTS, so none was dropped.

Table 7-4. Case 3 packets statistics in the AID station. Normaluserl had more packets
dropped, but it also had more incoming packets.
Attackerl Attacker2 Attacker3 Attacker4 Normal userl Normal user2
Packet# 411 357 388 366 1106 1215
in
Packet# 116 97 106 97 636 1215
out
Packet# BufferFull BufferFull BufferFull BufferFull BufferFull BufferFull
dropped =0 =0 =1 =0 =0 =0
BigVCTS BigVCTS BigVCTS BigVCTS BigVCTS BigVCTS
=295 =260 =281 =269 =470 =0
MD5Fail MD5Fail MD5Fail MD5Fail MD5Fail MD5Fail
=0 =0 =0 =0 =0 =0
CantRoute CantRoute CantRoute CantRoute CantRoute CantRoute
=0 =0 =0 =0 =0 =0







57




Normaluserl had dropped packets, but it was still distinguished from other true

attackers. First, its traffic was not slowed down as much as attackers. In the same time

period, an attacker just sent out about 375 packets, but normaluserl sent out 1106

packets (normaluser2 sent out 1215). Second, in Fig. 7-5 we can see VCTS-localTime

for packets from normaluserl rippled around 4. However, it is 6 for packets from

attackers. In conclusion, normaluserl had packets dropped, but it still maintained its

communication with the server.


VCTS localTime

-- attacker attacker2
8 attacker attacker4
-- n.:rmn31 uLrer -- n:rnm31l uLer-'
7 r,- E.iCT -:EED
7-

6I








1-
00 4 Al 011191 01 0 0-0-0

So no M o 00 'oM l l
packet#

Figure 7-5. Distribution of incoming packets in case 3 at the AID station. Traffic from
normaluser2 was pretty stable (the lowest series). Even though some of
packets from normaluserl were discarded, it was still different from real
attackers.







58



controlT (Fig. 7-6A)
0.004

0.003
>
--I
-f 0.002
/ decrease
0.001 T

0

10 seconds

arrival rate (Fig. 7-6B) avgT
400


vuuu
3500
D 3000
.E w rnn


7 I


--avgN
AID cap


0 2000 server cap
I 'I avgAID
'E 15001
r 1000
S 500
0
CN I- CL C coO Z 0 C
10 seconds



Figure 7-6. How did T and arrival rate interact each with other in case 3. When T
doubled, avgAID and avgTfell; otherwise avgAID and avgT rose.

Case 4

There were two registered attackers, two registered normal users and two


unregistered normal users. Their parameters were:

* Attackerl: testClient 192.168.1.102 1000 500 1000 0
* Attacker2: testClient 192.168.1.102 1000 500 1000 0
* Normal userl: testClient 192.168.1.102 250 700 250 1
* Normal user2: testClient 192.168.1.102 250 700 250 1
* Normal user3: testClient 192.168.1.102 250 700 250 1
* Normal user4: testClient 192.168.1.102 250 700 250 1
Tranged between 0.00125 and 0.0025 when converging. In Fig. 7-7, we can see

incoming packets distribution of two registered normal users and two registered attackers.

Packets from the other two unregistered normal users did not enter tunnel tree. As a

result, the AID station had no statistics data about them. In this testing case avgN did not

become zero after the AID system was triggered. Only the two registered attackers had










packets dropped. Ideally, avgT:avgN= 1600:400 = 4:1 should be true in the case 4 (this

ratio can be changed by modifying RESERVEDTIMES in server.c and signing a new

contract between the server and AID station). However, because attackers slowed down

their traffic (less than half amount of packets sent out as other normal users), avgTwent

down too.

Table 7-5. Case 4 packets statistics in the AID station. Normaluserl and normaluser2
are registered had no packets dropped.
Attackerl Attacker2 Normal userl Normal user2
Packet# 493 453 1187 1194
in
Packet# 172 154 1187 1194
out
Packet# BufferFull 0 BufferFull 0 BufferFull 0 BufferFull 0
dropped BigVCTS 321 BigVCTS 299 BigVCTS 0 BigVCTS 0
MD5Fail 0 MD5Fail 0 MD5Fail 0 MD5Fail 0
CantRoute 0 CantRoute 0 CantRoute 0 CantRoute 0

attackers
VCTS localTime
attacker
normal user
7 normal_user2
I- P.I" '.:. TSE C. EEL,
6

C,5



3-

> 2

1 1




packet#


Figure 7-7. Distribution of incoming packets in case 4 at the AID station. Traffic from
normaluserl and normaluse2 were pretty stable (the lower two series). It is
very similar to Fig 7-3 with the exception that series for two normal users are
a little bit higher.











controlT (Fig. 7-8A) I decrease T
0.003
i 0.0025
m 0.002
0.0015
| 0.001
m 0.0005

LO M M r0 LO M M r0- LO M M r0i- Z6 LO M
N 0 0 CN CM C M LO L O tO tC O
10 seconds

arrival rate (Fig. 7-8B) -- avgT avgN
4000 AID cap server cap
D 3500 -- avgAID
3000

0 2000 -





10 seconds



Figure 7-8. How did Tand arrival rate interact with each other in case 4. AvgNis the
server's arrival rate of the traffic from the Internet. T would not affect AvgN
S Attacker: testClient 92.68..02 000 500 300












Attacker2: testClient 192.1 68.1.102 1000 500 300 0
S Normal user: testClient 92.10068.02 250 700 250
In case 2, XSEG was 1000 for attackers, and it was 300 in case 5. In Fig. 7-10,
i- O 0oi IO 0' LO rO"i- uILO 0") M I-L O"M
N4 04 O3 OM M I LO LO tO tO













10we can see that after system converged, value of Tvaried between 0.00nds1375 and 0.00275.


It is bigger7-8. How an in case 2. Wharrival rat made case 5 so special? Let us compare t with case 2.

In case 2, attackers arrivalsent out 1000 bytes, exclusive of headers, every iteration, affectvgN

directly,AXSEG was 1000. Here, attackers also sent out 1000 bytes per iterationHowever, because the
traffic from AID tunnels had higher priority, the unregistered attackers could
not flood the server.

Case 5

This is an interesting case. It is very analogous to case 2, two attackers and one

normal user. However, attackers chopped same amount of data into smaller pieces here.

* Attackerl: testCient 192.168.1.102 1000 500 300 0
* Attacker2: testCient 192.168.1.102 1000 500 300 0
* Normal-user: testCient 192.168.1.102 250 700 250 1
In case 2, MAXSEG was 1000 for attackers, and it was 300 in case 5. In Fig. 7-10,

we can see that after system converged, value of Tvaried between 0.001375 and 0.00275.

It is bigger than in case 2. What made case 5 so special? Let us compare it with case 2.

In case 2, attackers sent out 1000 bytes, exclusive of headers, every iteration, and

MAXSEG was 1000. Here, attackers also sent out 1000 bytes per iteration, but MAXSEG









was 300. That means an iteration needs to send packets as many as three times in case 5.

Then, what happened? See Fig. 7-9.

First, unlike in case 2, an attacker almost sent out as many packets as normal user

did. A packet's VCTS is decided by its size and Tof the tunnel tree. When a packet is

small, VCTS will be small too. Consequently, the packet has a higher chance to be

accepted by an AID station.

Second, compared with case 2, Tbecame bigger when system converged, but

VCTS-localTime for packets from attackers became smaller. Smaller packets made

smaller VCTS but more packets sent out (heavier traffic) in an iteration made bigger T.

Third, Since Tbecame bigger and packets from normaluser had the same

MAXSEG as in case 2, 250, we can see VCTS-localTime for packets from normaluser

twisted a lot, unlike in case 2.

In our AID service, packets from either attackers or legitimate users might be

dropped. Because a server's capacity is fixed, if more clients try to access the server at

the same time, every client could get less resource from the server. If a client intends to

use more than its share, its packets will be discarded. The AID system controls the

arrival rate not to surpass a server's capacity by this policy. We can see in case 2 and

case 3. A client is treated differently when the traffic load changes. A legitimate client

slows down its outgoing TCP traffic when sensing its packets were dropped (no

acknowledgement from the other end), if it implemented TCP correctly. For an attacker,

if it implemented TCP right, it would slow down outgoing traffic. If it did not, VCTS for

its packets would grow very fast, and most of its packets would be dropped. Damage










from attackers is soothed in both cases, and at the same time, a server is still accessible to

legitimate clients.

Table 7-6. Case 5 packets statistics in the AID station. Normaluser had no packets
dropped.
Attackerl Attacker2 Normal user
Packet# in 1229 1185 1215
Packet# out 723 659 1215
Packet# BufferFull 1 BufferFull 3 BufferFull 0
dropped BigVCTS 505 BigVCTS 523 BigVCTS 0
MD5Fail 0 MD5Fail 0 MD5Fail 0
CantRoute 0 CantRoute 0 CantRoute 0


VCTS localTime


- attackers
- attacker2
normaluser
M/A YXVCTSEXCEED


5-









1 -




packet#



Figure 7-9. Distribution of incoming packets in case 5 at the AID station. Notice the big
"wave" of normal user.














n I decrease T


0.003

0.0025

0.002

0.0015

0.001

0.0005

0


T I- 0 0e (D 0( (4 LO U 0 Z V N-
C0 C m C Il I- I- LO LO LO (U) U) t(0 t0
10 seconds


N 0 (l' t0. 0) C" L O0 U)C N 0 ( t0. 0) C" U) 00 C) N0
(C" C" C Cf" Cfl Cfl 0 '3 't 't 't U O) U L) tU O t).O t(.O
10 seconds




Figure 7-10. How did T and arrival rate interact with each other in case 5.


'I C- C0 C


controlT (Fia 7-10A)














CHAPTER 8
FUTURE WORK AND CONCLUSION

Limitations and Future Work

Our AID system has some limitations theoretically and practically. Improving they

is our goal in future work.

Protecting UDP traffic: We need the self-adaptation feature based on TCP

congestion control to separate legitimate users and attackers. That is why our AID

system only protects TCP traffic at present. Future work is to include UDP traffic into

our AID service.

Robustness against the compromise of AID stations: In the current design of our

AID system, we did not address how to deal with the case that AID stations are

compromised. A compromised AID station can send forged UDP messages (PUSH,

PULL, CTRLT and etc.), drop packets from legitimate users and adjust virtual clock

maliciously. The good thing is an AID station can be removed or added into the AID

system easily. We could disconnect a suspicious station for an inspection

Traceback: The AID system can resist against DoS attacks but cannot trace back to

the origin of attacks. Flooding traffic might be from registered clients that are zombies

remotely controlled by real attackers. We may implement existing IP traceback

mechanisms in the AID system.

Independency of higher-level application: Our AID system is not perfectly

independent of higher-level applications because we introduced AID layer and it is not

processed by the Linux kernel. However, we also do not want users to recompile their









Linux kernels to join the AID service. We may find some other way to program our AID

system to avoid the dilemma.

Flexibility of programs: Most controlling factors are defined as constants in the

programs. We need to recompile them if we want to do registration, change the secret

keys, adjust virtual clock setting and etc. These factors can be saved in files to make our

programs more flexible.

Conclusion

Most existing defense systems for DoS attacks are not self-complete. They usually

need universal deployment. In the thesis a self-complete anti-DoS service (AID) was

implemented and tested. The AID service can be applied to Internet services, such as ssh,

ftp, www and so on. Everyone can join the AID service by registration and get

immediate protection. The AID service provides a fair share of a server's resource to all

registered clients. It requires no modification of end systems and routing infrastructure to

join. Random overlay network accommodates an efficient and scalable structure to route

traffic from registered clients. It changes dynamically. An AID station can be removed

or added easily. Distributed virtual-clock packet scheduling algorithm blocks the traffic

from attackers and manages the arrival rate of a server. A registered client host, which is

not an attacker, can access a registered server even when the server is attacked. Finally,

we still have some problems need to be solved in the future, for example, including UPD

traffic into the AID service, making AID station robust, tracing back attackers and having

programs more flexible.















APPENDIX A
HOW TO RUN

Make sure the library libipq was installed before continue. It can be found in

iptables-1.2.9.

Program server:

* make server
* cd src module
* make serverFilter.o
* insmod ip_queue
* insmod serverFilter.o
* cd ..
* ./server AIDSIP SERVERIP
AIDSIP is IP of the AID station As. SERVERIP is the server's IP.

Program client:

* make client
* cd src module
* make clientFilter.o
* insmod ip_queue
* insmod clientFilter.o
* cd ..
* ./client AIDIP
AIDIP is IP of the AID station Ac.



Program AID

* make AID
* cd src module
* make AIDFilter.o
* insmod ip_queue
* insmod AIDFilter.o
* cd ..
* ./AID clientIP serverIP









ClientIP is IP of the registered client and serverIP is IP of the registered server. In

our testing cases, we had only one client machine, AID station and server machine. If an

AID station wants to register more than one client or server, information of the

client/server should be added into the function initialize( of the source file AID.c.



Program alert

* make alert
* ./alert AIDIP serverIP serverPort
AIDIP is IP of AID station A,. ServerIP:serverPort identifies the attacked service.



Remove loaded module:

* rmmod ip_queue
* rmmod clientFilter
* rmmod AIDFilter
* rmmod serverFilter


Turn on forwarding in iptables at an AID station:

* su -
* echo "1" > /proc/sys/net/ipv4/ip forward
* iptables -I FORWARD -j ACCEPT














APPENDIX B
FILE GLOBAL.H

File global.h defined many important constants. Their names and values we used

in testing are:

* #define MTU 1500: Max transfer unit.

* #define BUFSIZE 4096: The size of the buffer that a queued packet is copied to.

* #define TCPINFOSIZE 28: The length of an AID layer header, in byte long, for
TCP packets entering a tunnel tree. The whole packet looks like: IP header I TCP
header I new destination IP (32 bits) I packet digest for integrity (128 bits) I virtual
clock timestamp 64 bits I application data. So, (32 + 128 + 64)/8 = 28 bytes.

* #define UDPINFOSIZE 17: The length of an AID layer header, in byte long, for
UDP packets. The whole packet looks like: IP header I UDP header I packet digest
for integrity (128 bits) IpacketType 8 bits | application data. So, (128+8)/8 = 17
bytes.

* #define MD5DGSIZE 16: Bytes of md5 digest created by md5 library, md5.h and
md5.c.

* #define RCZSIZE 4: Bytes of recognizingfield of normal TCP packets.

* #define UDPMAXSIZE 256: The max size in bytes of a UDP packet in the AID
system, inclusive of md5 digest (16 bytes), packetType (1 byes), service list (7*n
bytes). It should be big enough for different UDP messages.

* #define UDPTYPELEN 1: Length of the packetType field in a UDP packet in byte
long.

* #define PULLNO 0: Number of the nearby AID stations this AID station should
send PULL messages to (variable q).

* #define PUSHNO 0: Number of the nearby AID stations this AID station should
send PUSH messages to (variable k). Using square root ofNEARBYAID (defined
in AID.c) is ok.

* #define PUSHDEEP 2: how many AID station a PUSH message can go through,
exclusive the first one which is A,.









* #define PULL 0: Value ofpacketType field for a PULL message.

* #define PULLANS 1: Value ofpacketType field for a PULLANS message.

* #define PUSH 2: Value ofpacketType field for a PUSH message.

* #define CTRLT 3: Value ofpacketType field for a CTRLT message.

* #define PCKTYPEOFFSET 16: The offset ofpacketType in UDP packets, the
location is: UDP header 8 bytes I md5 digest 16 bytes I packetType 1 byte =16

* #define VCSTAMPOFFSET 20: The offset of virtual clock timestamp in TCP,
the location is: TCP header (offset bytes) I serverIP 4 bytes I md5 digest 16 bytes
VCTStamp 4 bytes, 4+16 = 20

* #define DATAOFFSET 28: The offset of application data in TCP packets, the
location is: TCP header (offset bytes) I serverIP 4 bytes I md5 digest 16 bytes |
VCTStamp 8 bytes I application data, 4+16+8 = 28















LIST OF REFERENCES


1. C. Schuba, I. Krsul, M. Kuhn, E. Spafford, A. Sundaram, and D. Zamboni,
"Analysis of A Denial of Service Attack on TCP," Proc. oflEEE Symposium on
Security and Privacy, pp. 208-223, IEEE Computer Society Press, Oakland, CA,
May 1997.

2. J. Lemon, "Resisting SYN Flood DoS Attacks with A SYN Cache," Proc. of
USENIXBSDCON2002, pp. 89-97, USENIX Association, Berkeley, CA, February
2002.

3. P. Ferguson and D. Senie, "Network Ingress Filtering: Defeating Denial of Service
Attacks Which Employ IP Source Address Spoofing," IETF, RFC 2267, January
1998.

4. K. Park and H. Lee, "On the Effectiveness of Route-Based Packet Filtering for
Distributed DoS Attack Prevention in Power-Law Internets," Proc. ofACM
SIGCOMAM' 2001, vol. 31, pp. 15-26, August 2001.

5. H. Wang, D. Zhang, and K. G. Shin, "SYN-dog: Sniffing SYN Flooding Sources,"
Proc. of 22nd International Conference on Distributed Computing System
(ICDCS'02), pp. 421-428, IEEE Computer Society, Washington D.C., July 2002.

6. S. Chen, R. Chow, Y. Xia and Y. Ling, "A Global Anti-DoS Service Based on
Random Overlay Network," unpublished paper, Department of Computer and
Information Science and Engineering, University of Florida, September 2004.

7. A. Juel and J. Brainard, "Client Puzzles: A Cryptographic Countermeasure Against
Connection Depletion Attacks," Proc. of Network and Distributed System Security
Symposium (NDSS'99), pp. 151-165, Networks and Distributed Security Systems,
San Diego, CA, February 1999.

8. T. Aura, P. Nikander, and J. Leiwo, "DoS-Resistant Authentication with Client
Puzzles," Cambridge Security Protocols Workshop 2000. LNCS, Springer-Verlag,
vol. 2133, pp. 170-177, 2001.

9. D. Dean and A. Stubblefield, "Using Client Puzzles to Protect TLS," paper
presented at 10th Annual USENIX Security Symposium, Washington D.C., August
2001.






71


10. X. Wang and M. K. Reiter, "Defending Against Denial-of-Service Attacks with
Puzzle Auctions," Proc. of EEE Symposium on Security and Privacy, pp. 78-92,
IEEE Computer Society, Washington D.C, May 2003.

11. L. Zhang, "VirtualClock: A New Traffic Control Algorithm for Packet Switching
Networks," ACM Transactions on Computer Systems, vol. 9, no. 2, pp. 101-124,
May 1991.















BIOGRAPHICAL SKETCH

I earned my BS degree in computer science and information engineering from

National Chiao Tung University in Taiwan. As an undergraduate, I was like a sponge

absorbing all kinds of knowledge of computer science accessible to me. In my junior and

senior years, I worked on a project of providing QoS (quality of service) in wireless ATM

network. Integer programming is the key of the project. Through those 4 years, I

finished many projects in different areas: network, security, compiler, windows

programming, audio processing, graphics, database system, etc. I am seeking my MS

degree at the University of Florida by writing the thesis. After about 2 years' training

ar the University of Florida, I polished my professional knowledge and skills better and

became more confident to face new challenges.