<%BANNER%>

Pervasive Computing Approach to Energy Management


PAGE 1

PERVASIVE COMPUTING APPROAC H TO ENERGY MANAGEMENT By AHMED AHED ABUKMAIL A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2005

PAGE 2

Copyright 2005 by Ahmed Ahed Abukmail

PAGE 3

To my wife Nermeen; and my two beau tiful daughters, Raneem and Haneen.

PAGE 4

iv ACKNOWLEDGMENTS I would like to express my greatest appreci ation and gratitude to my supervisory committee chair (Dr. Abdelsalam Helal) fo r his guidance during my doctoral studies. I also express my gratitude to my supervisory committee members (Dr. Gerhard Ritter, Dr. Sartaj Sahni, Dr. Joseph Wilson, and Dr. Louis Block) for their help. I thank Dr. David Whalley (of Florida Stat e University), and Dr. Chris Healy (of Furman University) for providing the software package we used to calculate the number of loop iterations (vpcc compiler).

PAGE 5

v TABLE OF CONTENTS page ACKNOWLEDGMENTS.................................................................................................iv LIST OF TABLES............................................................................................................vii LIST OF FIGURES.........................................................................................................viii ABSTRACT....................................................................................................................... ..x CHAPTER 1 INTRODUCTION........................................................................................................1 2 RELATED WORK.......................................................................................................4 Energy-Management Techniques at the Architecture Level........................................4 Smart Batteries......................................................................................................5 Battery basics.................................................................................................5 Intelligent power drainage..............................................................................5 Energy-Aware Processors.....................................................................................6 Reducing Power through Circuitry Components..................................................7 Power and energy consumption equations.....................................................7 Voltage and frequency scaling.......................................................................7 Capacitance load reduction............................................................................8 Power Reduction through Architectural Design....................................................9 Operating System Techniques......................................................................................9 Energy-Management Solutions.............................................................................9 Memory and I/O Management............................................................................10 Communication Techniques................................................................................10 Scheduling...........................................................................................................11 Software Application Techniques...............................................................................12 Compilation Techniques......................................................................................12 Reordering instructions................................................................................12 Reduction of memory operands...................................................................13 Code generation through pattern matching..................................................13 Code annotation............................................................................................14 Remote task execution.................................................................................14 Application-Level Techniques............................................................................14 Tools for Low-Energy Design and Measurement......................................................16 PowerScope.........................................................................................................17

PAGE 6

vi Derivatives of SimpleScalar................................................................................17 Other Power-Estimation Techniques...................................................................18 Recent Work...............................................................................................................18 Grid Computing..........................................................................................................19 3 STATEMENT OF THE PROBLEM..........................................................................22 4 OVERVIEW OF THE APPROACH..........................................................................28 Overview of the Compile-Time Solution...................................................................28 Overview of the Runtime Support..............................................................................30 Implementation Setup.................................................................................................31 5 COMPUTATION OUTSOURCING FRAMEWORK...............................................32 Overview.....................................................................................................................32 Formal Model.............................................................................................................35 6 COMPILE-TIME STRATEGY..................................................................................39 Overview.....................................................................................................................39 Energy-Optimization Process.....................................................................................41 Calculating the Number of Loop Iterations................................................................41 Loop Data and Iterations Acquisition.........................................................................44 Calculating the Size of Loop Data..............................................................................46 Identifying Loop Instructions and Total Loop Execution Cost..................................46 Insert Outsourcing Code.............................................................................................48 7 RUNTIME SUPPORT................................................................................................49 Battery Monitor..........................................................................................................49 Network Monitor and Surrogate Service Discovery Server.......................................50 8 EXPERIMENTAL VALIDATION............................................................................51 Setup.......................................................................................................................... .51 Instruction-Level Energy Cost Estimation.................................................................53 Measuring Communication Cost................................................................................54 Simple Experimental Validation.................................................................................55 Large-Scale Benchmark Validation............................................................................56 Other Measurements...................................................................................................61 Normalization of Results............................................................................................65 9 CONCLUSION AND FUTURE WORK...................................................................72 LIST OF REFERENCES...................................................................................................76 BIOGRAPHICAL SKETCH.............................................................................................80

PAGE 7

vii LIST OF TABLES Table page 8-1 Total energy saved by ou tsourcing the Fibonacci loop............................................56 8-2 Total energy saved by outsourci ng the matrix multiplication loop..........................56

PAGE 8

viii LIST OF FIGURES Figure page 3-1 Improvements in la ptop technology from 1990-2001..............................................24 5-1 Framework for computati on outsourcing at runtime................................................33 5-2 Steps for executing a client progr am under the outsourcing framework.................34 5-3 Resource selection rule.............................................................................................37 5-4 State transition rule...................................................................................................37 5-5 The send rules for the client and the server..............................................................38 5-6 The receive rules for th e client and the server..........................................................38 6-1 Overview of compilation and optimization process.................................................40 6-2 Example of C program passed as input to vpcc.......................................................43 6-3 Resulting INF file for the program in Figure 6-2.....................................................43 6-4 Algorithm to show implementation of the pseudo-parser........................................45 6-5 Algorithm to calculate loop data size on ce all of the variables in each loop have been collected...........................................................................................................47 8-1 The C program used to estimate the energy cost of executing an empty for loop 100M times...............................................................................................................54 8-2 The C program used to estimate the energy cost of the ldr instruction....................54 8-3 Local versus remote execution of 300 iterations of a Fibonacci loop executing 300,000 times...........................................................................................................57 8-4 Local versus remote execution of matrix multiplication of a 400x400 matrix........58 8-5 Local versus remote execution of a bubble sort loop which sorts 50,000 integers..59 8-6 Input file for the ray tracing application...................................................................60

PAGE 9

ix 8-7 The 2-D image representing the 3-D scene generated by the ray tracing application for a 200x200 image..............................................................................60 8-8 Local versus remote execution for ray tracing three spheres in space and generating a 50x50 image........................................................................................62 8-9 Local versus remote execution for ray tracing three spheres in space and generating a 100x100 image....................................................................................63 8-10 Local versus remote execution for ray tracing three spheres in space and generating a 200x200 image....................................................................................64 8-11 Energy cost for the battery and network monitors execution.................................65 8-12 Comparison between theoretical and e xperimental measurements of the total energy consumed......................................................................................................69 8-13 The amount of base energy consumed based on the server processor's speed.........70 8-14 The effect of changing the communica tion metrics on the total energy consumed.70

PAGE 10

x Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy PERVASIVE COMPUTING APPROAC H TO ENERGY MANAGEMENT By Ahmed Ahed Abukmail December 2005 Chair: Abdelsalam Helal Major Department: Computer and In formation Science and Engineering. We present a novel approach to energy mana gement. This approach is a pervasive computing solution to the energy problem in mobile devices. Its pervasiveness arises from the fact that communication is used a nd viewed as an opportunity to save energy, whereby certain candidate sections of code (represented by basic program blocks, most likely loops) are outsourced from the mobile device on which they are executing, to a surrogate server machine with an infini te power source. As can be concluded, outsourcing code implies that the data invol ved in the computation will have to be transferred to and from the surrogate. This approach deviates from the traditional view of communication as a drain on the battery of a mobile device. The solution presented here is a compile-time solution and optimization, augmented by the necessary run-time support. The high-level source code is augmented by additional high-level code to intelligently (at run-time) allow the application running on the mobile device to outsource basic program blocks to a server. Both client and server applications are a bypr oduct of our approach, as the original source code is

PAGE 11

xi transformed into a client/server application where the client is installed on the mobile device and the server is stor ed on a surrogate machine. This approach uses a methodology from the domain of real-time systems to determine the number of loop it erations, and that facilitate s a compile-time computing of executing each loop. We also gathered the n ecessary information a bout the size of the data involved in each loop. This allows for determining the cost for outsourcing each loop (in terms of the cost for communicating th e data to the surrogate). Once both metrics (computation energy cost, and communication energy cost) have been determined, we can easily generate a code that allows the application to make the run-time decision of whether outsourcing is benefici al. This approach is a fine -grain approach to energy management, because it looks at basic program blocks (as opposed to certain predefined tasks or functions). For this system to allo w the execution of the cl ient/server application, the necessary run-time support was built to f acilitate this behavior. Results showed significant levels of energy saving, using our benchmarks.

PAGE 12

1 CHAPTER 1 INTRODUCTION The emergence of mobile and then pe rvasive computing (as new computing domains) introduced new challenges and research opportunities: one was energy management. These challenges arose from the mobility of used hardware [30]. Such hardware includes devices such as cellular phones, PDAs, laptop computers, and even MP3 players. The mobility of these devices implies that they are powered by mobile power sources represented by the battery of each device; and that, in tu rn, implies that the power source is limited. As these devices become more popular, a nd their use becomes more apparent and frequent, the need to manage their energy consumption becomes more vital to their operation. This is because the more often a ba ttery needs to be charged, the more often the mobile device is rendered immobile (w hich reduces the pervasiveness of their applications). Managing the energy consumed by these mobile devices has been an important subject in research and industry communities of both mobile and the pervasive computing. Solutions have been presented at the various levels and layers of the computer system, and often these solutions to the energy problem i nvolve a certain type of tradeoff. One of the most attractive avenues to energy management are high-level energymanagement techniques. Such a good argument was made for handling power management at high level that Ellis [8] propos ed a power-based API to allow for synergy between the application and th e system. One of the most attractive high-level solutions

PAGE 13

2 to energy management is a compiler-based solution that alleviates or minimizes the need for programmer power-awareness. This is done via compiler optimization. Velluri et al. [36] studied he effect of the traditional compiler optimization techniques on system power (and therefore energy). Results show ed that (except for loop unrolling and function inlining) most optimizations increa sed the energy consumed by the core of the processor. These results were (at least for loop unrolling) confirmed by Kandemir et al. [20]. We developed a new pervasive computing approach to energy management in mobile devices. This approach views comm unication (wireless communication) as an opportunity to conserve energy as opposed to being a drain on the battery. Sections of code represented by basic program blocks th at are most likely to be loops have the opportunity (at run-time) to execute locally on the mobile device, or remotely on a surrogate, by communicating the data involved in their comput ation back and forth to the surrogate. The decision to chose local vs. remo te execution is based on whether it is more energy-beneficial to execute locally, or to communicate with the surrogate and let the surrogate do the computation. Communicating with the surrogate and letting it do the computation is known as computation outsour cing. So, this approach is based on a tradeoff: the tradeoff here is between com putation and communication. At runtime, based on the condition of the battery of the device (full, empty, or in between), th e application can opt to go into an energy-saving mode and communicate with the surrogate for the opportunity to outsource code. This depends also on the availabili ty of the surrogate device, and user preference, in addition to the condition of the battery. Hence, this approach will be able to discover services within a wireless network. Service discovery

PAGE 14

3 must be simple as the decision-making code is inserted at compile-time. This is accomplished by two simple programs: the battery and network monitors. While this approach allows the decisi on to outsource code at runtime, it is considered a compile-time solution. This is done via high-level compiler optimization, by preprocessing the original (non-energy-aware ) source code, and gene rating two versions of the code. The first version is the client which runs on the m obile device; and the second version is the server, which runs on the surrogate device. The client and the server are generated, by inserting the necessary high-lev el code, to be able (at runtime) to make the intelligent decision of either outsourcing or not. To generate intelligent code that determines whether to outsource the code, we used a methodology from real-time systems to calcula te the number of loop iterations. It is safe to assume that this methodology will not be able to calculate the number of iterations for every kind of loop. However, it is capable of calculating loop it erations when the loops are nested, and when the loop bounds need to be determined at runtime, and it also allows loops with multiple exits. Unlike previous work, our study examined energy consumption at finer granularity. We examined certain candidate sections of programs (basic program blocks), and not entire programs, or specific task s or functions only within a computer program. This finer granularity yields better en ergy saving, as our test results show. Our study targeted a subset of the C programming language to verify our algorithm and methodology. As a proof of concept for this approach, we used three different kinds benchmarks involving loops with different complexity. We also de veloped a 3-D graphics rendering application to further show the energy saving bene fits resulting from our approach.

PAGE 15

4 CHAPTER 2 RELATED WORK Prolonging battery life (often called ener gy management) has long been the focus of research. This problem has many facets, which can be faced by addressing the various components of a mobile computer system. Earlier [1], we authored a book chapter on power-management techniques. These technique s include reducing energy at the systemarchitecture level, by targeting (reducing) various components of the power equations (P=CV2f) where P is power, C is capacitance load, V is supply voltage, and f is switching frequency. Other techniques targeted the ope rating system by saving energy involved in communication, by caching, by process scheduling, and by having an energy manager. Additionally, we presented software techniqu es grouped into two categories: specific application techniques, and compiler-based t echniques. These software techniques (like those targeting the operating system) are considered higher-level power-management techniques. Solutions that ta rget energy management usua lly involve some kind of a tradeoff. This chapter presents the various power and energy management techniques, that we introduced in the book chapter [1], and other techniques and new methodologies related to our study. Energy-Management Techniques at the Architecture Level Hardware and architecture are among the first areas researchers investigate when looking into energy and power management. Ther efore, they target the CMOS circuitry and the batteries of mobile devices. They also look for ways to develop processors that are energy and power efficient.

PAGE 16

5 Smart Batteries Battery basics Every battery has two terminals: one positive and one negative. Electrons flow from the negative to the positive terminal, wh en the two are connected (when the battery is installed). Electrons are formed on the negative terminal by chemical reaction that occurs inside the battery. Th is reaction occurs when the electrons travel from one terminal to the other. Rechargeable batteries (the batteries that we are concerned about in mobile devices) suffer from the problem of memory effect: the battery does not fully recharge because it has not drained complete ly before recharging. Lithium ion batteries solved this problem recently. However, once the battery is built, and as long as it is connected, the reaction occurs draining the battery power, and ev entually rendering it useless until it gets either recharged or replaced. Intelligent power drainage The aforementioned shortcomings of regul ar batteries can be summed by saying: they do not lose power/energy intelligently. Once they are connect ed to a device, the power drainage is going to happen. This motiv ated the development of the smart battery system (SBS) (http://www.sbs-forum.org/specs/i ndex.html). This batter y is different from non-smart batteries because it provides informa tion and system status to the host machine through the System Management Bus (SMBus ), and it has its own re-charger. This information can and will be used by the system, to manage power and energy in the mobile device. Operating systems standards and specifications such as advanced power management (APM) (http://support.microsoft.com/default.aspx?scid=kb;en-us;310752), and advance configuration and power interf ace (ACPI) (http://www.acpi.info). Currently, most laptop computers are shipped with smart batteries.

PAGE 17

6 A few companies in the indus try produce products for smar t-battery systems. These companies include (but are not limited to) Samsung, Motorola, and Hitachi as battery vendors. Semiconductor vendors that produce chip s for smart batteries include companies like Powersmart, Acer, and Adaptec. Energy-Aware Processors Another way to tackle power consumpti on in a computer system is to produce processors that consume less energy than regular processors do. A number of microprocessor companies in the industry have products that are ener gy-efficient. Intels Xscale architecture (http://www.intel.com/desi gn/intelxscale) design optimizes energy consumption. Xscale represents an integral building block of Inte ls PCA architecture (http://www.intel.com/pca). In addition, Inte l has the SpeedStep t echnology in building the Mobile Pentium III processor. SpeedStep provides the advantage of dropping to a lower frequency, and lower voltage, when the device is powered by battery; which results in conserving battery life while maintaining a high level of program performance (http://www.intel.com/support/processors/mobile/pentiumiii/ss.htm). Transmeta Corporation (Santa Clara, California) has the Crusoe family of processors (http://www.transmeta.com/crusoe/index.html). The Crusoe is an x86 compatible family of processors that is ligh tweight, high performance and 70% less power-consuming than other compatible mobile processors. Additi onally, the ARM family of processors is widely popular, and is geared toward reducing power cons umption while maintaining a high level of performance (http://www.arm.com).

PAGE 18

7 Reducing Power through Circuitry Components Power and energy consumption equations The power equation is P = V I where P is power, V is voltage, and I is current. This equation can be re-written as P = CfV2, where C is capacitance load, and f is frequency. The energy equation in troduces the time factor: E = P T = CfV2 T where T is elapsed time. So, the energy e quation can also be re-written as E = V I T Reducing any of the variables involved in any of these equations will reduce the energy and power consumed. Capacitance loa d, frequency, and voltage can be managed at the hardware and architectural level. Voltage and frequency scaling If voltage or frequency is reduced, then energy and power consumption will be reduced. Therefore, researchers targeted vol tage and frequency s caling techniques to achieve that result. Smit and Havinga [31] argued that reducing voltage means reducing performance; and thus additional hardware is needed to balance that difference to keep the same throughput. Hsu et al. [19] attempted dynamic frequenc y and voltage scaling. Their results were simulated, and the claim was that their results are similar to those of Transmetas Crusoe TM5400. At compile-time, they assign a clockfrequency, and voltage levels for input loop executions. Their idea is to assign deadlines to tasks, and utilize these deadlines to allow for voltage and frequency scaling. Simply, if a task finishes at time 0.5t and its deadline is at time t then it will not hurt the execution if the task takes t time to complete. So, at compile-time they identify these tasks and generate code that will allow for lower voltage and lower frequency to be used in the execution. This work supports the

PAGE 19

8 argument that additional hardware will be needed to achieve voltage and frequency scaling. Capacitance load reduction Reducing the capacitance load is another method to reduce the energy consumed by a computer. Two main mechanisms can be uti lized to handle this goal. Smit and Havinga [31] suggested that memory capacitance is great er than that of the processor. Therefore a reduction in memory operations, to keep as much work as possibl e on the processor and in registers is beneficial. Ge botys [14] also has done this work by reducing the off-chip activity. The solution is basically to use a minimum cost network flow to find minimum energy solutions by simultaneous partitioning of variables into memory and possibly allocate them into registers. Another mechanism to reduce the capacitan ce load is to reduce the logic state transition when handling memory addresses. In any processor the program counters tend to change multiple bits when accessing the next instruction or data from memory locations and that depends on the size of the in struction and if that instruction was a jump instruction or a sequential instruction. That transition increases the capacitance load. To minimize the transition, a Gray code solution can be utilized to facilitate memory addressing. Su et al. [33] did this solution along with a compiler. Sequential Gray codes vary only in one bit at a time. Therefore, there will only be a va riation of one bit, which is the minimum that can be achieved when accessing memory addresses (excluding the jump instructions).

PAGE 20

9 Power Reduction through Architectural Design Architectural design always seems be an at tractive solutions to the energy problem, and that is due to the fact that you can always leave the architecture the same but add to it certain components which will result in re ducing energy and power consumption. Bellas et al. [4] introduced a compilation methodology. At the hardware level, a second level cache was added to support the new compilati on strategy. This cache is a smaller cache and is much simpler to use than that of the first level cache. Witchel et al. [37] introduced a direct a ddressed caches soluti on, which allows the software to access any cached data without th e hardware doing any tag check. Therefore, the energy consumed by the tag check will be saved. A compiler solution was introduced to augment their hardware solution. Operating System Techniques Operating system solutions can be a very useful complement to architectural solution to energy management. While most ope rating systems are distributed with their own energy management solutions, additional techniques and methodologies were also researched to handle this problem. Energy-Management Solutions User-configurable energy management techniques are now implemented in operating system. Advanced power management (APM) is one of these techniques. In APM, users can define their desirable power sc heme. Some of the vari ables that users can control are the screen, the hard drive, and suspend/sleep mode s. Users can also instruct the screen and the hard drive to stop/sleep when the system has been idle for a while. Additionally, they can also specify if the system is idle, or, in the case of a laptop, the lid

PAGE 21

10 is closed, at these points the whole computer can save its state on the hard drive, sleep, and when it wakes up it will restore itse lf back to the previous state. Intel discussed some shortcomings of APM such as not considering future capabilities of PCs that in clude their communication capabi lities because APM drops the communication link. So, they came up with thei r own solution in Instantly Available PC which uses the ACPI specification to manage power while keeping the system online via either the modem or a LAN card. So, In tel and other industry partners contributed to the ACPI specification. Memory and I/O Management Memory access is expensive when it comes to energy consumption. Additionally, secondary storage (disk) is even more expens ive. So, the more the computation is left on the processor, and the storage is done in regi ster the better it is for energy consumption. For instance, making fewer in correct file predictions is a good methodology to save energy [38]. This was accomplished by observing : the probability and repeated history of file access do not occur without a reason. They claim that programs access more or less the same files in the same order when they execute every time, giving themselves a good knowledge for determining what they call program-specific last-successors for each file. Communication Techniques Energy saving communication techniques are getting increasingly important. This is due to the wide use of the Internet by almost every user of a computer. So, finding ways to save power on communication will definitely be beneficial. Communication systems have three states, sending, receiving, and idle. Switching between these three states is energy consum ing [31]. Therefore, reducing the transition from one state to another will definitely reduce power loss. Therefore, instead of serving

PAGE 22

11 the communication request of sending, we can buffer the data to send it in large chunks. The same thing can be done on the receiving end. We can only accept data that is of a certain size optimized for our power needs. Ot herwise, our system needs to go into an idle mod. In addition, with the availabili ty of solutions that conform the ACPI specification, there is no need any more to modify network protocols to support full connectivity while the syst em is in sleep mode. Kravets and Krishnan [21] provided a method for managing the communications device on a host through suspend and resu me operations. They claim that a communication device will continue to draw pow er unless it is suspended. Their solution takes into account the need fo r the host to know the communications patterns, so when it is suspended there is no communication goi ng on, otherwise problems such as buffer overflow will occur on the host and other hosts. So, the functionality of the suspend and resume operations is adaptable such that it will avoid such overflow. Additionally, Loy and Helal [ 24] introduced an applicatio n-based solution that can easily be utilized in developing energy mana gement techniques in the operating system. Scheduling Energy-aware scheduling in the system requires prior knowledge of the activity inside that system [5]. Monito rs can be incorporated within an operating system to handle this. Bellosa [5] introduced an event-driven energy accounting as a way to manage power in an operating system. The way this was done is by online analysis of the energy-usage patterns that are fed back into a scheduler to control the CPUs clock speed. This work here was done with real hard ware as opposed to simulation.

PAGE 23

12 Software Application Techniques High-level or software energy management techniques are getti ng highly attractive as solutions to the energy problem. Ellis [8] argued a case for the need for higher-level power management. The gains from higher-le vel energy management are great at the application level, and this stems from peopl es interests. People can enhance on hardware advances on power management, but also wh en hardware solutions are not available software developers can handle this issue to either comple ment hardware solutions, or obtain better results without newer, more advanced hardware. At the software, and application techniques there are two important sections, compiler development, and application-specific techniques. Compilation Techniques Research and experiments have shown th at, with the exception of loop unrolling, and function inlining, compiling for performance does not imply compiling for energy [36]. So, optimization techni ques that target energy as opposed to performance are needed, and rightfully so, since compilation for power is extremely important. It is important because it is an attractive soluti on to programmers that are not energy-aware. In this section we will discuss some techni ques that showed promise and some that did not when dealing with compiler optimization. Reordering instructions The energy and power equations showed that their values are directly proportional to the frequency of switching the signal from 0 to 1, i.e., logical state transitions. Tiwari et al. [34] mentioned that switching is a func tion of present input and previous state. So, the previous instruction is a factor in the f unction. Therefore, reor dering instructions can be a factor in reducing switc hing activity, and therefore, reduce energy consumption. In

PAGE 24

13 utilizing this technique in the 486DX2 arch itecture, it did not show much favorable results, it only showed little improvement. Reduction of memory operands Tiwari et al. [34] also showed, via ex periments on the 486DX2 architecture, that instructions with memory operands have hi gher energy consumption than instructions with register operands. Pipeline stalls, misaligned accesses and cache misses add to the cost energy-wise. Compiler optimizations achieve reducing the number of memory operands. In their research, the authors claim that the most efficient way to reducing the memory operands is via register allocation for temporary and frequently used variables, and that also leads to poten tial reduction in pipeline stalls and cache misses. However there are some issues with register allocat ion optimization techniques, such as more complex compilers, longer compile time, and register allocation algorithms need to be modified to optimize for low power. Additiona lly, larger caches, which will result in lower miss rates, will result in this te chnique being a little less significant. Code generation through pattern matching Aho et al. [3] introduced code genera tion through pattern matching and dynamic programming, and was later utilized by Fraser et al. [13] for generating code that optimizes for performance. The idea is to find a cover for intermediate representation (DAG) for each basic block of code. Findi ng that cover is done using dynamic programming and tree matching so that the ov erall cost is minimized. The cost function introduced in Aho et al. [3] and Fraser et al. [13], take performance under consideration. Therefore, to optimize for power, the metric used is power usage as opposed to the number of clock cycles used for the inst ructions. It was sugge sted that further investigation of this te chnique needs to be done.

PAGE 25

14 Code annotation Marculescu [25] proposed a methodology to annotate the code at compile-time to adaptively select, at runtime, the optimal number of instructions to be fetched or executed in parallel to save energy. However, this solution would require a dditional architectural changes. Remote task execution This is believed to be one of the most attractive compiler optimization techniques because it takes into account th e idea that while communica tion might be a drain on the battery, it can also be an opportunity to sa ve power by shipping the execution elsewhere. The idea of compiler-based remote task execu tion has been proposed in other research. Kremer et al. [22] have done some work that utilizes compiler-based remote execution of certain tasks in the field of image processing applications. However, their work included only hand compilation of the code and not comp iler implementation or any automation of task identification. They basica lly utilized what they called checkpoints in the code to determine the tasks delimiters. Additionally, Rudenko et al. [29] introduced remote task execution mechanisms. However, this approach migrates entire processes to remote servers, and then waits for their results back to the client remote machine. As far as Java applications are concerned, the work done by Pa lm et al. [27] introduces the possibility of migrating compilation (both optimizing and non-optimizing) step of running a Java program to a server, and that is due to th e power consumption cost of compiling Java programs. Application-Level Techniques Attacking the power problem at the a pplication level has been done quite extensively. However, the solutions provided ar e only useful in certa in applications, and

PAGE 26

15 most of the times can not be generalized to allow for utilization in other applications. Most of the work seems to have been done in the signal and image processing applications due to the amount of drainage that they inflect on the battery, since they use computation power, and screen power. Loy and Helal have introduced an ac tive mode power management (AM/PM) where they increase the amount of informati on available through the use of power aware API [24]. They saved up to 62 times the pow er used when compared to using no power management. They examined the amount pow er used by a connected wireless PC card when the connection is not needed. At that poi nt, they take advantage of stealing back as much of the power as possible. The targeted applications in this work were an email client and a web browser. Flinn and Satyanarayanan [10], demonstrat ed a collaborative relationship between operating systems and applications to meet use r-specified goals for battery life. In this work, applications can dynamically modify their behavior to c onserve energy. The way this is done is by the operating system monito ring of energy supply and demand. It is as simple as: if the supply of energy is plentifu l applications perform best, otherwise they will be biased towards conserving energy. They used PowerScope [11] to validate the measurements of energy consumption for accurate estimation. MP3 players are some of the most popular devices in the market at the present time. Haid et al. [16] developed an excelle nt application with energy awareness in mind. This work presents designing an energy-awar e MP3 player. They not only tackle the hardware side of the devise, but rather the so ftware side as well. They talked about the two technologies that influenced the design of wearable computers and these are system-

PAGE 27

16 on-chip (SOC) and System-In-Package (SIP). These two technologies are the reason that both hardware and software solutions will need to be developed for energy-aware wearable devices. In this work, they analyze a single-chip multimedia system to be used in the wearable devise. They take into accoun t the entire computing environment such as hardware, packaging, and software design. Th ey achieve low energy consumption by the use of detailed statistical analysis of th e energy consumption. They use an in-house designed runtime energy estimation tool. Additionally, Yuan et al. [39] investigat ed another multimedia application with respect to power-awareness. They present a middleware framework for coordinating the adaptation of multimedia application to the hard ware resources. They have three goals in mind when designing their system, the first of which is a soft r eal-time guarantee of multimedia application deadline, the second is to have sufficient energy for multimedia applications to finish their task, and the thir d is to waist as little energy as possible. To meet the three goals the framework presented makes three useful contributions: the first is to make multimedia applications make energy aware processor reservation. Secondly, is to formally model adaptability of hardware, software, and user preference. Finally, deliver sufficient processor and energy resources to the application and operate the processor as slow as possible to save energy. Tools for Low-Energy Design and Measurement Power and energy estimation and measurem ent tools are an essential part to developing solutions to the energy problem As developing and later measuring the power consumed must be accurate to gain bene fits from any newly developed technique.

PAGE 28

17 PowerScope PowerScope allows the measuring and estimation of the energy consumed via profiling [11]. By mapping energy consump tion to program structures, it can be determined which sections of a program consume more power. Also, the energy consumed by procedures/subroutines within a process can be determined. By providing this fine granularity of feedback, PowerScope allows one to focus on system components that are responsible for the largest energy consumption. Expe riments by the developers of PowerScope yielded 46% reduction in total energy consumed. The architecture of this system is com posed of a data collection phase, and an offline analysis phase. The data collec tion phase uses an Agilent Technologies (http://www.agilent.com) multi-meter connected by a GPIB cable to a data collection computer running an energy monitor com ponent. A profiling computer running an application and a system monitor component is getting power by be ing connected to the multi-meter for measurement Offline analysis utilizes a third compone nt (the energy analyzer) which will take the results produced in the data collection phase by the system monitor, and the energy monitor and correlate them to produce the energy prof ile of the application. As in CPU profilers, where sections of c ode wasting CPU cycles can be discovered, this profiler will allow for pinpointing the sec tions of code that consume lots of energy, and therefore must be dealt with. Derivatives of SimpleScalar SimpleScalar is used for building and m odeling applications to analyze program performance, hardware and software co-verif ication (http://www.simplescalar.com). It is a toolset that is used to develop modeling ap plications to simulate real programs running

PAGE 29

18 on various modern microprocessors (such as Alpha, PISA, ARM, and x86) and also various operating systems. It has simulators with different granul arities (high and low level). SimpleScalar is widely used in the research community. The toolset also includes performance visualization tools, statistical analysis resources, and debug and verification infrastructure. The licensing model for SimpleScalar allows users to extend the tool set. As a result the power management community took adva ntage of that to extend the tool set to develop projects for measuring and estimati ng power consumed by applications. These projects include the Power Analyzer Project at University of Michigan (http://www.eecs.umich.edu/~tnm/power), and th e Wattch project at Princeton University (http://parapet. ee.princeton.edu). Other Power-Estimation Techniques In addition to previously me ntioned tools, other research has been done to estimate energy for certain applications, systems, a nd devices. Cignetti et al. [7] described an energy model for the Palm In this work they target challenges faced by the programmers when developing energy efficient code. Recent Work Recently additional work has been done in the area of remote execution for energy management. Flinn et al. [9] present a re mote execution system for mobile clients powered with a battery. The system they descri be (Spectra) is a self -tuning system where it monitors resources in the environment, a nd based on the resources the decision is made to where the components of the application ar e executed. This approach, while useful and attractive, it is a coarse-gra ined solution to energy mana gement where opportunities to execute code remotely are done at the appli cation level as opposed to the basic program

PAGE 30

19 block level. Additionally, it doe s add a certain level of overhead to the mobile device where various monitors need to be inst alled on the device to monitor the various resources. These monitors include, not only a battery monitor, and network monitors, but they also include, a CPU monito r, and a file-cache monitor. Grid Computing When dealing with research that target s computation outsourcing via a distributed computing system, you have to consider grid computing as it is a new model that has a great potential in benefiting th is type of research. In looki ng into grid computing [12, 23], it looks like it would be th e perfect match for this type of re search as it utilizes things like Java remote method invocation (RMI), remote procedure call (RPC). However, most grid computing systems differ in an important aspect and that is the size of the problem being solved. No matter how large a program that we are trying to optimize for energy is, it will never be as large as the programs investigat ed within the grid computing domain, which tend to be huge in nature and usually requi re longer time to execute. Additionally and more importantly, the size of the middleware used within a grid computing environment is quite large, and computationally intensive to be utilized on a mobile device for energy considerations, as it will consume additional energy. That is why investigating mobile agent (MA) systems is questionable and must be pursued with care, however RPC and RMI are worth investigating, and actually th ey are considered for future research. Another important difference between our work and grid computing is that in grid computing, the geographic location of the res ources is of no concern. However, in our research the geographic location is quite important, as if outso urcing is to take place, the mobile device must be located within a pe rvasive smart space in order to be able

PAGE 31

20 communicate with the surrogate via the wirele ss network, so proximity is important in our research. While grid computing, in general, is not applicable as a whole computing paradigm, a simplified version of its behavior can be adopted in this research. When looking at the formal models for grid computing, we were able to recognize a formal model from grid computing that was useful in defining our own model. Nemeth and Sunderam [26] presented a formal model that is based on an abstract state machine (ASM). Their model encompasses a genera l description of grid and distributed computing. Their definition contai ns universes (sets) of resour ces, processes, users, task, etc. It also contains a signature consisti ng of function names on these universes. Their universes were: a set of processes (PROCESS) as set of applications (APPLICATION), a set of users (USER), a a set of resources (RESOURCE), a set of nodes (NODE) in the case of distributed computing but not for grid computing, a set of ta sks within a process (TASK), a set attributes (ATTR) for tasks, resources, and nodes along with a subset describing architectures (ARCH), a set of lo cations (LOCATION), and a set of messages (MESSAGE) for interaction between processe s. Along with these sets, the symbols, true false undef =, and the Boolean operators are included. The formal model presented, also included the functions that compose the signature on the super-universe containing all the aforem entioned universes [26]. The signature is composed of the functions: app: PROCESS APPLICATION, user: PROCESS USER, request: PROCESS RESOURCE {true, false}, uses: PROCESS RESOURCE {true, false}, mapped: PROCESS NODE, BelongTo: RESOURCE NODE {true, false}, installed: TASK NODE {true, false}, attr: {RESOURCE, NODE, TASK}

PAGE 32

21 ATTR, arch: RESOURCE ARCH, location: RESOURCE LOCATION, compatible: ATTR ATTR {true, false}, CanLogin: USER NODE {true, false}, CanUse: USER RESOURCE {true, false}, state: PROCESS {running, waiting, receive_waiting}, from: MESSAGE PROCESS, to: MESSAGE PROCESS, expecting: PROCESS PROCESS + {any}, and event: TAST {req_res, spawn, send, receive, terminate}. Given the mentioned functions defined w ithin the signature, they proceeded to define the rules for conventiona l distributed systems. They also, showed the problems of mapping conventional distributed systems rules in to grid systems. Finally, they provided the necessary modifications to formulate the mode for grid systems.

PAGE 33

22 CHAPTER 3 STATEMENT OF THE PROBLEM The problem of energy management has gained a lot of attention in the mobile and pervasive computing research community. This is due to the increased reliance on mobile devices by a wide spectrum of users. This in creased reliance on mobile devices stemmed from the increased capability of these de vices. This argument was supported by Helal [18] who gave a clos e look at the market for Java-e nabled phones and PDAs from a commercial standpoint to show that the capab ilities of these devi ces are increasing, and will continue to increase over time. J-Phone Communications (Japan) offered handsets that are based on the J2ME technology, which in clude a built in camera, a color display, and a 3-D graphics engine that displays imag es from different perspectives [18]. They also presented plans to offer mobile phones capable of providing geographic information, multimedia service (JPEG, and PNG graphics synchronized with sound). Helal [18] also presented information about Nokias plans to double its shipment of Java-enabled phones between the years 2002, and 2003 which give s a direction of where the mobile technology is headed. Helal [18] also hinted to the increased capabilities in PDAs as well by indicating that the PDA Profile (PDAP) might be too limited for where PDAs are headed and might not be able to accurately estimate the capability of high-end PDAs such as the Compaq iPAQ. Enabling mobile phones an d PDAs with Java is an indication that the trend is to make these mobile devices capable of performing ta sks usually performed on desktop computers.

PAGE 34

23 As the capabilities of mobile devices incr eases, their energy usage will increase, and therefore will drain the battery quicke r. A cellular phone or a PDA drains more power when it is playing an MP3 song, taki ng a picture, decoding and image, or recognizing a voice to voice-dial a phone number. The problem has been that the battery technology has not kept up with the other technologies co mprising a mobile computer system such as PDAs and cellular phones. Starner [32] gave a discu ssion of how much slower a dvances in battery technology have been than those for the other mobile computer components (the discussion was given for laptop computer, but information fo r wearable computers, PDAs and cellular phones was deduced to be similar). Figure 3-1 provides a graph representing the improvement in laptop technology from 19902001 [32]. As the graph indicates, CPU speed has kept up with Moors Law, but ba ttery capacity has not. In fact the figure suggest that the battery capaci ty is lagging behind. It was al so stated that mobile phone companies sell more batteries than they do phones as the lifetime of the phone is much longer than that of the batter y, and consumers usually must buy more than one battery to be able make use of their mobile devices e ffectively. So, battery technology seems to be the slowest technology for the com ponents of mobile devices. The continued increase in reliance and capabi lity of mobile devices indicate that the energy-saving problem is ongoing, and it needs to continue to be addressed on the long run. Due to their capability, mobile device us ers are ranging from teenagers to the elderly and they span a wide range of backgrounds and mobile device util ization. One of the most widely used feature set of mobile devices is that dealing with multimedia applications, especially among the youth. Th e young generation uses these devices to

PAGE 35

24 play video games; take and edit pictures and videos and record and play sounds and music on them. All of these applications consume significant energy. Not only does the young generation use multimedia applications ; think of the medical and dental professions. They deal with X-rays, threedimensional (3-D) magnetic resonance images (MRI) and 3-D representation of the mouth (dental application) for determining the Figure 3-1. Improvements in laptop technology from 1990-2001. location of a tooth implant. Other users c ould include a graphics designer working to generate a 3-D model of a home, or any othe r scene for that matter. Voice and speech recognition applications are known to have been extremely difficult to perform on a mobile device due to the in tensive computation involved.

PAGE 36

25 Consider a fraction of the computation invol ved in one of the previously mentioned application such as displaying an image representing a 3-D scene. The data involved in describing a 3-D scene is small. The output of this computation is a 2-D image display of the 3-D scene. However in order to get from one to the other, the amount of computation is quite large. In our 3-D benchmark (experi mental validation chapter), we displayed 3 spheres of different colors in space. In order to generate a 200x200 image representing the scene, the PDA used ran for over 16 mi nutes to generate the scene, consuming approximately 0.0.47 Amps (Including the operati ng system and the te rminal application, 0.19 without. Given that the battery that we used is a 1700 mAh (milli-Amp Hour) or 1.7 Ah, it will take less than 4 executions of the a pplication to drain the battery of the device. Therefore, multimedia applications are ve ry energy consuming, and must be handled differently. The emergence of pervasive computing as a byproduct of mobile and distributed computing enabled researchers to envisi on and later develop new methodologies to solving the energy problem. We observed that mobile users are usually located within communication and energy rich surroundings. Th is is because of the presence of the mobile user within a wireless network, whic h usually has other machines that are not mobile (connected via a wire to the netw ork), and thus do not suffer from the energy dissipation problem. Additionally, we observed that there exists no link between the mobile devices located with in the wireless network and those non-mobile ones. We concluded that, if a link can be established between the mobile and the non-mobile devices, the mobile devices will be able to save energy by outsourcing some of their computation to the non-mobile devices. The mobile devices will however pay a

PAGE 37

26 communication cost to faci litation the outsourcing. Th is observation led to the development of our methodology that utili zes pervasive smart spaces to outsource computation to non-mobile devices that do not suffer from the energy dissipation problem. This methodology is considered pervasive due to the fact that at runtime, the application intelligently makes the decisi on between local and remote execution by comparing the energy cost of computation vs. the energy cost of communication. The ability of the application to make such a deci sion is quite a challenge. Hence, the problem that is being solved in this research is to make ordinary applications, developed by nonenergy-aware developers, able to make such an intelligent decision at runtime. Therefore, this approach takes as input a program in its high-level representa tion (source code), and transform it via a compilation process into tw o versions, a client version and a server version, where the client is able to send reque sts to the server to be able to do some computation for the client by receiving data and sending back results (only the modified data). To be able to generate these two vers ions of the program, a priori knowledge of the runtime behavior of the program needs to be analyzed. Therefore, th e area of real-time systems is a very suitable area for utilizing in making such an analysis and determination, especially the ability to calcul ate the number of loop iteration s. In addition, the ability to detect basic program blocks (most likely l oops) at both the high-le vel and the low-level must be accomplished to determine both the communication cost invo lved in outsourcing a basic program block, and the energy co st of executing a single loop iteration. To facilitate the runtime decision-maki ng ability for the client, additional functionality has to be added to the mobile device to allow for the outsourcing to take

PAGE 38

27 place when applicable in a pervasive envir onment. This functionality includes a network monitor and a battery monitor similar to t hose introduced by Flinn et al. [9], however their implementation is much simpler due to the compile-time code augmentation. The network monitor will be able to find and locate available surrogate machines that are running an instance of the server. Additio nally, a battery monitor needs to be implemented to monitor the status of the ba ttery. The simplicity of these two monitors results in a negligible additional power consumption by the system, and will be overcome by the energy saving from the modified application.

PAGE 39

28 CHAPTER 4 OVERVIEW OF THE APPROACH To solve the problem stated in the pr evious section, the solution has to be composed of two parts. The first part of the solution is done at compile-time as an optimization technique at the high-level s ource code. The second part of the solution must provide the necessary support to the outco me of the first phase. This is due to the fact that the outcome resulting from the comp ile-time phase is a different formation than that initially developed. Overview of the Compile-Time Solution We have introduced this work as part of our fine-grain approach to power-aware computing [2]. First at compile-time, an a ssumption has been made that the source code has been tested and verified in its original form. Although that is done, this solution still validates the source code syntactically to ma ke sure that no inadvertent errors were introduced along the way. In addition to s yntax checking, the source code is also disassembled and the outcome of this proce ss is an assembly representation using the mnemonic representation of each instruction of the target architecture. At this point, information about the high-level source code and the low-level instructions will become available for the optimization technique part of this contributed research. The next step is to recognize basic pr ogram blocks (mainly loops) in both the source code and the assembly code, and simp ly match them. Recognizing loops at the high-level representation of the source code will result in the ability to collect all the data involved in the computation of the loop, and that will yield the energy cost of

PAGE 40

29 communication for sending all the data involved in the calcul ation out, and receiving only the data that changes (L-Values). Also, us ing the technique men tioned by Healy et al. [17], the number of iterations for each loop is calculated. As for the assembly code, the loops are recognized to determine the instruct ions involved in each loop, which will yield the entire energy cost of executing a single iteration of the each loop. In addition to instructions, at compile-time, we recognized whole library functions such as those belonging to the math library, a nd we added the value of their energy cost to the cost of the loop in which they are executed. This, along with the metric calculated before to find out the number of each loops iteration constr uct a good estimate of the total cost of the local execution of each loop. Before calculating the total cost of comm unication and the total cost of each loops computation, experiments were done to find out the cost of comm unicating a single unit of data (a byte), and the co st for executing each machin e instruction for the target architecture. As for calculating the cost of communicating a single byte, a client/server application was tested with multiple sizes of data to communicate between two machines, and the measurement for this was recorded and averaged. As far as each instructions energy cost, a similar approach to that presen ted by Tiwari et al. [ 35] was utilized where each supported instruction is isolated via hi gh-level code implementation, and executed multiple times within a loop a nd the final result is averaged based on the number of instructions used (we used 100 instructions within a loop executing 100 million times). In addition to testing machine instructions and ve rifying their cost, we tested pre-existing library function and verified their energy co st in a similar manner to the individual machine instructions.

PAGE 41

30 Overview of the Runtime Support To support the ability to outsource code, the application must be able to run in one of two modes: normal mode, or energy-saving mode. So, when an application starts, it will have to get some information based on the resources that are available. If the battery is susceptible to be drained quickly, then th e application needs to run in energy-saving mode, the user also has control over this. Ho wever, if the user decides to run in normal mode, then the application should not worry about computation outsourcing. In order for the application to be able to make the right decision, it has to contact the battery monitor at startup. The battery monitor would have already determined if energy saving is available via outsourcing (t his decisions is based on user preference also). Additionally, the battery monitor will contact the network monitor to check if the devices is actually connected to a network and that network c ontains surrogate servers. If so, then it will run in energy-saving mode lis ting the appropriate surrogate available for the application to utilize. This monitor is also similar to, but much simpler than, those discussed by Flinn and Satyanarayanan [9] and by Gu et al. [15]. The work done by Flinn et al. [9] suggests th at the cost of these monitors is nonnegligible. This is true in their case, as a lo t of the intelligence to execute code remotely is done at runtime as opposed to compile-time, and that is why their approach is a coarsegrained approach to energy management. Howe ver in our approach, while may utilize an idea presented by Flinn and Satyanarayanan [9] and by Gu et al. [15], the solution is much simpler and that is because the batter y monitor is a straightforward inquiry to operating systems advance power management (APM). As far as the network monitor is concerned, it will only be invoked if an en ergy-saving mode of operation is decided (mainly as an outcome of the battery monitor). Therefore, the cost is negligible for these

PAGE 42

31 two monitors. Implementation of the battery m onitor was as easy as looking at a single file containing information about the battery at certain increments of time. As for the network monitor, several approaches can be investigated, the simplest of which was proposed by Gu et al. [15] and it is ba sed on wireless broadcast for discovering surrogates. Implementation Setup The implementation was done on an Intel Xscale which is an integral part of the Intel PCA. We utilized the Sharp Zaurus SL-5600 (http://www.sharpusa.com), which contains an Intel Xscale PXA-250 processor, and is running Linux, as the mobile device. Installed on the Zaur us, was a low-power Socket Communications (Newark, California) (http://www.socketcom.com) lo w-power wireless LAN card. The outsourcing server is an Intel x86 machine running RedHat Linux 7.2. Developing applications on the Zaurus was achieved using Metrowerks (Austin Texas) Codewarrior for the Sharp Zaurus (http://www.metrowerks.com). This software comes with a packaged executable to run on the mobile device only during development to be able to debug and/or execute the a pplication on the Zaurus from a Microsoft Windows where Codewarrior is installed. The software is called MetroTRK (Target Resident Kernel). MetroTRK is only used to transfer executables to the Zaurus via our wireless network at the Harris Mobile Computi ng Laboratory at the University of Florida. The implementation supported only a s ubset of the C programming language, and handled the most popular assembly instru ctions, which mainly access memory, and handle the arithmetic and logic operations within the architecture. Although limited, supporting the particular subset of C we chos e was adequate to develop meaningful and useful application to prove our concept.

PAGE 43

32 CHAPTER 5 COMPUTATION OUTSOURCING FRAMEWORK Outsourcing computation is not a new te rminology here. However, the motivation behind outsourcing the computation to a remote server, and the approach under which we are outsourcing the computation is the contribution here. Our goal from this research is to show that an intelligent runtime decision can be made to decide if it is better to execute a section of code locally on the mobile device, or would it be more energy-beneficial to send its data to a remote serv er, and get the results back. Overview The overall framework for outsourcing is desc ribed in Figure 5-1. The idea is that a server machine accessible via a wireless netw ork can serve as a surrogate server for a host of mobile devices such as handhelds, PDAs and laptop computers. This server at runtime will receive requests from client programs running on any of these devices for outsourcing code to the server. The code that is in charge of making this decision is completely transparent to the programmer. All the programmer is required to do is to compile the code to optimize for energy. This will result in two version of the program being generated which the programmer will eventually have to compile a nd install. We believe that this is not a burden on the programmer in any way, and it is not a requirement for the programmer to have any knowledge of energy requirements/constraints. Once an application is comp iled, and two versions have been generated (a server version and a client version), and they are in stalled on their respectiv e machines, the user

PAGE 44

33 can then execute a client application on th e mobile device. This client application executes normally until it reaches a section of code that has been designated as outsourceable (having the potential for outsourcing), this is what we call the outsourcing candidate. Once this section is reached, then the intelligen t code that was inserted at compile-time is executed to make the outsourcing decision. As a matter of fact, the candidate code will not be executed (locally or remotely) until the decision making code is executed. Remote server running server programs for the various mobile applications Hand held computer running an outsourceable mobile client application Wireless Network Hand held computer running an outsourceable mobile client application PDA running an outsourceable mobile client application C o m m L i n k Laptop computer running an outsourceable mobile client application Figure 5-1. Framework for com putation outsourcing at runtime In figure 5-2, an illustration of the outs ourcing mechanism at runtime is given. The client runs on the mobile device, and once it reaches an outsource-able section of code, it determines if it is more energy beneficial to outsource or is it more beneficial to execute locally. If the determination is made to outsour ce, then it will send the data to the server and wait for the results back, otherwise it will continue to execute locally until it reaches the next available outs ource-able section of code. At all times, the serv er running on the

PAGE 45

34 surrogate machine is waiting for requests fr om client programs. Once it services the clients request it goes back to waiting for client requests ag ain, which occur once a candidate section of code deci des to outsource its computation. ClientServer Execute Until You reach an outsourceable candidate section of code More energy efficient to outsource? NO Yes Execute Locally Wait for a request from the client Request Received? NO Send Request and Data to Server (1) Yes Execute the Section of code Send changed data back Wait for Results (2) Results Received? Yes NO (1) Send Results (2) Figure 5-2. Steps for executing a client program under the outsourcing framework The decision to outsource a section of code is not an arbitrary decision. The mobile machine user must configure it to determine if outsourcing is desirabl e in the first place. Therefore, there is a battery and a network monitor running on the client machine that

PAGE 46

35 will help in making this determination. The ba ttery monitor will run and ask the user to determine the outsourcing policy that the user chooses. Once that is determined, then the network monitor gets involved to determine th e feasibility of outsour cing (if there is no network connection to a surrogate, then outsour cing will not occur. Once the feasibility is determined, then applications can be run in energy-saving mode, otherwise, they will run in normal mode. Using our model, we envision the developm ent and creation of an entity called a computation service provider (CSP). Different mobile users would subscribe to the CSP in order to service their ener gy-needs. The subscription will be by registering a copy of the server of the energy-aware application with the CSP. Whenever the user is within the proximity of (in the pervasive smart space co ntaining) the surrogate machine containing the server code, it would be possi ble for the client to outsource code to the server located on the surrogate. The outsourcing takes place by the client communicating its data to the server, let the server process the data, and then the client will get the results back. Formal Model Nemeth and Sunderam [26] presented a formal approach for defining the functionality of a grid system Their approach started by de fining distributed systems and showed how a grid system differs from the classic distributed system environment. Our model is a much more simplistic model than that they presented. Our model has a limited number of resources, and a limited number of processes. The resources in our model are the wireless network (WiFi) and the surrogate device. Our two processes are represented by the client and by the server versions of the original code. The model presented is based on an abstract state machine (ASM).

PAGE 47

36 In looking at their model, we realized that their model encompasses a general description of grid and distributed systems. Our model is a simplified representation of theirs. In our model, we de fine the process universe as PROCESS = {client, server} the resource universe as RESOURCE = {wireless_net, surrogate} and the location universe as LOCATION = {within-range, out-of-range} We use the same functions used in the grid and distributed computing domain, and add two of our own functions which are: execCost: TASK VALUE, and comCost: TASK VALUE, where execCost is a function that produces the valu e of the energy consumed by a specific task of a process. Similarly, the comCost produces the value of the ener gy consumed by communicating the data for a specific task of a process. As far as the functions that we use from grid and distri buted computing are concerned, we use the same exact definiti on presented by Nemeth and Sunderam [26]. The following functions are defined: user: PROCESS USER request: PROCESS RESOURCE {true, false} uses: PROCESS RESOURCE {true, false} loc: RESOURCE LOCATION CanUse :USER RESOURCE {true, false} state: PROCESS {running-normal, running-energy -saving, receive-waiting} we modified this function to fit our execution framework. from: MESSAGE PROCESS to: MESSAGE PROCESS event: TASK {req-res, send, receive, terminate} Upon defining the above functions, and uni verse sets, the rules for defining our system as a simplified grid computing system can clearly be defined. We redefined the resource selection rule (figure 5-3), the state transition rule (figure 5-4), the send rule for both the client and the server (figure 5-5), and the receive ru le for both the client and the server (figure 5-6).

PAGE 48

37 The resource se lction rule: state(client) := running-normal state(server) := receive-waiting if loc(wireless_net) = within-range & request(client, wireless_net) = true) & CanUse(user(client), wireless_net) then request(client, wireless_net) := false uses(client, wireless_net) := true if request(client, surrogate) & CanUse(client, surrogate) then request(client, surrogate) := false uses(client, surrogate) := true else uses(client, surrogate) := false request(client, surrogate) := true endif else uses(client, wireless_net) := false uses(client, surrogate) := false endif Figure 5-3. Resource selection rule The state transition rule: if uses(client, surrogate) & uses(client, wireless_net) then state(client) := running-energy-saving else state(client) := running-normal Figure 5-4 State transition rule

PAGE 49

38 Client send: if state(client) = running-energy-saving & commCost(task(client)) < execCost(task(client)) then if event (task(client)) = send(server) then extend MESSAGE by msg with to(msg) := server from(msg) := client end extend state(client) := receive-waiting Server send: if event(task(server)) = send(client) then extend MESSAGE by msg with to(msg) := client from(msg) := server end extend state(server) := receive-waiting Figure 5-5. Send rules for th e client and the server Client receive: if state(client) = receive-waiting & event(task(client) = receive(server) then if to(msg) = client & from (msg) = server then MESSAGE(msg) := false state(client) := running-energy-saving Server receive: if state(server) = receive-waiting & event(task(server) = receive(client) then if to(msg) = server & from (msg) = client then MESSAGE(msg) := false state(server) := running Figure 5-6. Receive rules for the client and the server

PAGE 50

39 CHAPTER 6 COMPILE-TIME STRATEGY Our compiler optimization technique for low energy analyzes a source program at the three different levels of representation (high, intermediate, a nd low). At the highlevel, we collect information about the data involved in each loop. At the intermediate level we utilize an algorithm described By H ealy et al. [17] to find out the number of loop iterations. The reason this is an intermediate level analysis is because they analyze the register transfer list (RTL) [6] representati on of the source code. At the low level, we determine the machine instructions generated by an assembler to determine which instructions are getting executed within each loop. All of the three levels of source code analysis are embedde d in our algorithm. Overview This new compilation technique utilizes pre-existing utilities such as the gcc compiler and Metrowerks assembler and compiler. We first pass the code that needs to be compiled to the gcc compiler to make sure th at it is syntactically correct, once that is done, we remove the resulting machine code as it will not be needed. Then we pass the same source code through our optimization preprocessing, along with the assembly code generated from passing the original source code through the Metrowerks assembler, to generate the two versions of the code, the cl ient and the server. Once the client and the server codes are generated, then the server is compiled for the target server machine, and the client is compiled for the mobile device. In our environment, the client is compiled using Metrowerks Codewarrrior, and the server is compiled on a Linux machine using

PAGE 51

40 the gcc compiler. Figure 6-1 shows the pr ocess for optimizing a C program using our technique. Source Code Syntax Checking (GCC Compiler) Assembler Source Code Assembly Code Energy Optimization Preprocessing Client Source Code Server Source Code Preexisting Compiler for the Client Machine Preexisting Compiler for the Server Machine Client Executable to run on the mobile device Server Executable to run on the Server Machine Figure 6-1. Overview of compilation and optimization process

PAGE 52

41 Energy-Optimization Process Our optimization technique modifies the high -level code (the C source code). The input to this process is a file containing the source c ode, and a file containing the assembly representation of this source c ode. Using the source code program, we determine the number of loop iterations, the si ze of the data involved in the loop execution. Then we determine, using the assembly representation (low-level representation) of the program, the instruc tions involved, and we calculate the total energy cost for all of the instru ctions using the energy cost of each individual instruction. In addition to the machine instructions we ha ndle also library functions (such as the math and standard libraries) called within each loop. The energy cost of each individual instruction was calculated usi ng a methodology similar to that presented by Tiwari et al. [35], and we give an explanation of this in our experimental valid ation. We calculate the energy cost for library functions in a similar manner to that of the machine instructions. Also, we had already measured the cost of transferring one byte of data using our wireless card. Given all of these metrics, we were able to insert socket code within our source program and conditional statements to determine at runtime if it is more energybeneficial to outsource a candi date section of code (basic program block/loop) or to execute it locally on the mobile device. Calculating the Number of Loop Iterations Healy et al. [17] developed a useful utility for predicti ng the worst-case execution time (WCET) of a program. They provided us w ith the software that will accomplish this task for C programs. Their algorithm is part of implementing a static timing analyzer for analyzing real-time systems, as predicting th e number of loop iteration is essential for analyzing real-time systems. Their approach automatically bounds the number of loop

PAGE 53

42 iterations. They handle nested loops, and l oops with multiple exits. Their methodology is implemented by analyzing the register transfer list (RTL) [6], which is an intermediate representation of th e source program. First, they identify the branches that can affect the number of times the loop executes. Secondly, they calculate when each br anch can change its direction. Third, they determine when each iteration branch can be reached. Finally, they calculate the minimum and maximum number of each loops iteration. If the loop invariant is a nonconstant, for the purposes of the timing analyz er they are implementing, they allowed the user to input the minimum and maximum values for this variable, and that is not needed for our compiler optimization technique, as our methodology supports non-constant loop invariant as, at runtime, its value will be known, and we can use a formula involving the invariant to be multiplied by the cost of ex ecuting each loop once (the energy cost) which gives us as a formula that is easily evaluated at runtime to determine if a section of code should be outsourced. Their implementation is integrated in the implementation of the Very Portable C Compiler (vpcc) introduced by Benitez and Davi dson [6]. The input to the modified vpcc (we will refer to it as vpcc) is a source pr ogram with a .c extension, and the output is a set of files, only one of which is of interest to us, and that is the file with the same name as the source program, except with a .inf exte nsion (the INF file). The INF file contains information about the maximum and the minimum number of loop iteration, and we are only interested in the maximum number of ite ration in our research and that is because we do not want to under-estimate, we want to outsource with a high degree of certainty that a benefit will be gained from outsourci ng. In figure 6-2, we give a sample C program

PAGE 54

43 that can be compiled with vpcc, and by passing ce rtain switches to it an INF file (Figure 6-3) will be generated. main() { int i, j; for(i = 0; i < 100 -j ; i = i + 3) {;} } Figure 6-2. Example of C program passed as input to vpcc -3 main loop 0 0 1 1 -1 -1 1 2 3 4 -1 4 -1 loop 1 1 -4 r[10] 0 r[9] 3 s -2 (100-.1_j-2)/3 (100-.1_j-2)/3-1 -1 3 -1 3 -1 block 1 lines 5-5 preds -1 succs 2 4 -1 makes_unknown 3 -1 doms 1 -1 1 82 4 0 8 7 () 1024 7 (100) 8 4 (%o1) 1 90 4 0 8 4 (%o1) 8 4 (%o3) 8 4 (%o1) 1 90 7 1 1024 7 () 8 4 (%o1) 8 7 () 1 62 4 2 2048 4 () 0 0 () 0 0 () 1 82 4 0 8 7 () 1024 7 () 8 4 (%o2) block 2 lines 5-5 preds 1 -1 succs 3 -1 makes_unknown 3 -1 doms 1 2 -1 2 32 4 0 8 4 (%o2) 1024 7 (3) 8 4 (%o2) block 3 lines 5-5 preds 3 2 -1 succs 4 3 -1 doms 1 2 3 -1 3 90 4 1 8 4 (%o2) 8 4 (%o1) 8 7 () 3 74 4 2 2048 4 () 0 0 () 0 0 () 3 32 4 0 8 4 (%o2) 1024 7 (3) 8 4 (%o2) block 4 lines 5-5 preds 1 3 -1 succs -1 doms 1 4 -1 4 80 4 0 128 4 () 8 7 () 0 0 () 4 15 4 0 0 0 () 0 0 () 0 0 () Figure 6-3. Resulting INF file for the program in Figure 6-2. The boldfaced expression represents the maximum nu mber of loop iterations

PAGE 55

44 Loop Data and Iterations Acquisition The first stage of our technique is to recognize the maximal basic program blocks (most likely these blocks will be loops). These basic program blocks (loops) will constitute the opportunity for optimization ( candidate code for outsource-ability). Once these basic program blocks are recognized at the high-level, then we collect all the data elements associated with them, and determine the beginning and end file positions of these loops. Additionally, we pass the relevant s ections of the original source code to the program that calculates the numbe r of iterations for each loop. The first part of this stag e is to implement a parse r-like module (we call it the pseudo-parser) to recognize basi c program blocks, collect the data used within each loop, and identify what variables are R-valued (do not change), and what variables are Lvalued (change). We did not need to implem ent a full parser here as the syntax has already been checked before entering this stag e of the algorithm. In figure 6-4 we present a high-level algorithm for our ps eudo-parser. In addition to acquiring the data elements involved in the calculation of each loop, we also determine which C library functions have been called to determine the contribution of their energy cost to the execution of the loop. Additionally, we determine if certain l oops are not outsource-ab le. All loops that involve Input/Output (I/O) rou tines are designated as non-outs ource-able. This holds true also for those loops that include nested loops with I/O functions. The second part of this stage is to figu re out the number of loop iterations and associate each number with each loop calcula ted by the pseudo-parser. The algorithm to do this is a very simple one. This algorithm is implemented using a very simple parser that parses only the lines that contain th e minimum and maximum iterations for each loop in the INF file. Once it extracts the expr ession representing the maximum number of

PAGE 56

45 iterations, it cleans it up by removing any extra characters such as those in figure 6-3 where the expression is (100-.1_j-2)/3 This particular expressi on is unique also as it contains a / which could be problematic as if ev erything used in the expression is an integer then at runtime, integer division mi ght happen, and therefore after we remove the substring .1_ from the expression and if we rec ognize the division oper ator, we insert the typecasting (double) right before it. Hence, th e resulting expression is (100 j 2) / (double)3 Pseudo-Parser Input : A C program Output: An array of loop data structures containing: a. The beginning and ending file position of the loop b. The variables and their sizes c. The L-Valued variable and their sizes get a token nestLevel = 0 loopNumber = 0 initialize a stack of loops to empty while there are tokens in the program do if the token is a loop loopNumber = loopNumber + 1 nestLevel = nestLevel + 1 assign loopNumber to the loop just entered assign nestLevel to the loop just entered push the current loopNumber on the stack else if the token is part of an expression including functions and structures if nestLevel > 0 if you encounter a variable if the variable is an L-value insert the variable as an L-value in the loop on top of the stack else insert the variable as a variable in the loop on top of the stack end if end if if you encounter and I/O function then The loop becomes non-outsourceable. end if end if else if you reached an end of a loop nestLevel = nestLevel -1 pop the top of the stack end if get the next token Done. Figure 6-4. Algorithm to show implementation of the pseudo-parser

PAGE 57

46 Calculating the Size of Loop Data The next stage is to calculate the data si ze for each loop. In this stage, we examine each variable involved in the loop, and base d on the size of the variable in bytes (including arrays), we add the value to our sum to calculate the size in bytes. Additionally, if the variable is an L-value (cha nges), then we add the size of the variable to our sum for the L-valued variable. This is a very important aspect of this algorithm, as if the data does not change, we only need to communicate it to the server, and we do not need it back, but if it changes, then we will exp ect it to be sent back to the mobile device. This way, we can minimize the amount of communication needed. Additionally, we check if the loop contains other loops, and if so, then we collect the variables of the nested loops only if these variables have not already been collected by a parent loop. The algorithm in figure 6-5 illustrates this procedure. At this point, we have the data size for each loop which when multiplied by the cost to send one byte of data gives us the total cost to send all the data w ithin the loop added to the cost to receive all the Lvalued data within the same loop. Identifying Loop Instructions and Total Loop Execution Cost Using the assembly code representation of the source program, we can recognize loops within the assembly code. The target architecture (Xscale) has a unique way of identifying loops. Loops can be identified by three consecutive instru ctions, the first of which is the compare instruction cmp followed by a conditional branch ble, blt, bge, bgt, bne, beq followed by an unconditional branch. b The unconditional branch is quite useful in this regard as it sends the control outside of the loop, and all we have to do is to go to that bran ch location, and find the other unconditi onal branch that completes the loop and returns us back two instructions before the cmp This way we are able to

PAGE 58

47 Calculate Loop Data Size: Input : An array of loop data structures Output: The same array updated to include the data size of each loop for each loop in the program do set the totalDataSize of the current loop to zero set the totalLvaluesSize of the current loop to zero for each variable within the loop do add the size of the variable to totalDataSize done for each L-valued variable within the loop do add the size of the variable to totalLvaluesSize done for each loop nested within the current loop for each variable in each nested loop if the variable does exist in the upper loop add its size to totalDataSize end if done for each L-valued variable in each nested loop if the variable does exist in the upper loop add its size to totalLvaluesSize end if done done done Figure 6-5. Algorithm to calculate loop data si ze once all of the variables in each loop have been collected. This is only done for outsource-able loops. identify or rather delimit where the assemb ly code for each loop starts and where it end. However, when loops are nested, we need more information to be able to map loops at the assembly level with those at the high le vel. The additional information needed is available in the structure containing information about all the loops (we call it the loopdata data structure). The information needed here is which loop is nested with which loop, and that information wa s obtained via our pseudo-parser.

PAGE 59

48 Once each loop was delimited, then it was just a matter of going through the instructions that constitute the loop, and adding their premeasured energy cost. In addition to the cost of each instruction there is a cost for pipeline stalls. This cost was obtained experimentally using multiple instruction sequences once the cost per instruction was determined. Therefore, when we recognize that certain instructions precede others (e.g., str before an ldr ), we add the measured pipeline stall energy cost. This calculation gave us the cost of a single execution of the loop. At this point, we have all what we need to be able to produce th e resulting client and server. The total loop execution cost becomes a matter of multiplying the cost of a single execution by the formula representing the number of loop iterations calculated before. Insert Outsourcing Code The implementation of this code was very large, but it was not difficult. As our pseudo-parser generated for us information of where each loop begi ns and where it ends. The location of where we need to generate th e necessary C code to create a client/server based application becomes a matter of inserti ng the necessary include files, and variable declaration (we declared them globally). The outsourcing code is only inserted for those loops that are flagged outsource-able.

PAGE 60

49 CHAPTER 7 RUNTIME SUPPORT Here we present the two monitors used as runtime support for the outsourcing mechanism. The two monitors work together and they get executed based on the user preference and the battery condition. The attract ive property of these two monitors can be summed in the fact that most of the time th ey are not consuming any energy. In fact, they consume very little energy when they are doing any work. The way these two monitors work depends on the user preference in the fi rst place, and once that has been determined, the condition of the battery of the mobile device takes co ntrol of the decision making process. These two monitors will run on the cl ient mobile device. In addition to these two monitors, a server program w ill run on the surrogate device waiting for connections from the client. The battery monitor, network monito r, and the server will together establish the service detection within the wireless network. Battery Monitor The battery monitor gets execu ted either by the user or the operating system. This is also a decision to be configured by the user. The user chooses if he/s he desires to run in energy-saving mode or in normal mode. If norma l mode is selected, then nothing happens and the monitor exits. Otherwise, if energysaving mode is selected, then the battery monitor will ask the user if the energy savi ng is to take place immediately, or it should wait until the battery gets below a certai n limit. If energy saving is to take place immediately, then the battery monitor w ill immediately call the network monitor. Otherwise, the battery monitor will sleep (cons uming a very negligible amount of energy)

PAGE 61

50 and periodically check the status of the battery by c ontacting the operating system. Contacting the operating system is a very trivia l matter as it will only look at a file called /proc/apm, and extract the remaining percen tage of the battery. Once it reaches the limit specified by the user, then will contact th e network monitor that will complete the task of setting up the device in an energy-saving mode. Network Monitor and Surrogate Service Discovery Server Once the network monitor is called, it will send out a broadcast that will be only received by a network server th at is providing any service for the client. This server will be running on the surrogate machine that is to service the client. Once the server receives the broadcast it then will establish a handsha ke with the client device and inform the device of its name, and that it is ready for servicing the device and it supports outsourcing. At that point, the network monitor will create a configuration file that is to be opened by the application to determine if it would run in energy-saving mode, or normal mode. If any type of error occurs on th e way to creating this file, the file will not be created, and hence there wi ll be no energy-saving mode. At runtime the client application will star t running and checks if the energy-saving file exists, and if so, then extract the inform ation about the server from it, establish the connection, and execute in energy-saving m ode, otherwise, execute in normal mode.

PAGE 62

51 CHAPTER 8 EXPERIMENTAL VALIDATION Our measurements, and experiments were done in two stages with our platform setup. The first stage was to estimate as accura tely as possible the cost of each supported machine instruction (assembly instruction). Secondly, the second stage is to measure the cost of each benchmark, first without our op timization, and second with our optimization. Setup Our target architect ure is an Intel Xscale which is an integral part of the Intel PCA. We chose the Sharp Zaurus SL-5600, which contains an Intel Xscale PXA-250 processor, and is running Linux, as our mob ile device. Installed on the Zaurus, is a Sockets low-power wireless LAN card. The outsourcing server is an Intel x86 machine running RedHat Linux 7.2. Developing applications on the Zaur us was achieved using Metrowerks Codewarrior for the Sharp Zaurus. This software comes with a packaged executable to run on the mobile device only during developmen t to be able to debug and/or execute the application on the Zaurus from a Microsoft Windows where Codewarrior is installed. The software is called MetroTRK (Target Resident Kernel). We use MetroTRK to transfer executables to the Zaurus via our wireless network at the Harris Mobile Computing Laboratory at the Un iversity of Florida (http:// www.harris.cise.ufl.edu). This Microsoft Windows machine on which Codewarrior is installed also happened to be the same machine on which we record our measurements.

PAGE 63

52 For measuring the energy consumed, we used an Agilent 34401A multi-meter which was connected to a Microsoft Windows 2000 desktop computer via an IEEE488.1 General Purpose Interface Bus (GPI B) cable. Installed on the desktop is Agilents Intuilink plugin which works with Microsoft Word and Excel. We used Excel because its plugin allows for multiple readings as opposed to Words single reading. The multi-meter has two J-hooks that were placed in series between the AC adapter and the Zaurus to place the multi-mete r in series to measure the total current drawn by the Zaurus. The voltage coming from the AC adapter remained at a constant 5 volts. Therefore, the only two factors in the energy equation are the current drawn and the time in seconds as energy is given by the equation: E = V I T Where E is the energy consumed, V is the voltage, I is the current, and T is the elapsed time. To measure the energy consumed by either a running process on the Zaurus, or by data communication, we calcu late the difference between the current drawn when the Zaurus is idle, and when the Zaurus is eith er running a process or sending/receiving data. To make this as accurate as possible, the onl y application that we ran on the Zaurus was the Linux Terminal. We also turned off the light of the Zaurus LCD. These measures that we took, and as our experiments show, resulted in a constant current drawn by the Zaurus while idle, which gave us the ability to ge t good measurement with as little sampling noise as possible. The multi-meter only allowe d us to take samples at one tenth of a second. Therefore, we had to execute code that consumes enough time to allow for as accurate a measurement as can be obtained.

PAGE 64

53 Instruction-Level Energy Cost Estimation Before applying our optimization to source code, and besides knowing what machine instructions were used, we had to know the energy cost for each machine instruction involved. Our work targeted only a subset of instructions from the Xscale architecture, which was sufficient to testi ng our benchmarks, and by no means is that a limitation of our approach. We used a methodology similar to that desc ribed by Tiwari et al [35] to estimate the cost of each instruction. The methodology su ggested executing several instances of a single machine instruction within a loop and average the energy consumed to obtain the per instruction cost. Figure 8-1 shows a sm all C programs which when executed gives a good estimate of the energy consumed by an empty for loop. Figure 8-2 shows a second C program that gives a good estimate for th e load instruction (LDR), which loads a memory location into a register. Notice, to mi nimize the affect of the instructions that execute the loop header (as the loop body is the several occurrences of the load instruction) we used a regist er loop control variable. The ability to declare register variable in a language like C made it very possible for us to be able to isolate single instructions which enhanced th e accuracy of our estimation. In order to minimize the effects of the cache, and to better estimate inter-instruction energy consumption, we developed additional benchmarks that include multiple instances of a series of different instructions (e .g. ldr followed by str) and we observed their behavior, and recording the amount of additi onal energy consumption recorded, and used these metrics in our code to update the in struction-level energy consumption values calculated before. In addition to minimizing the effect of caching the instructions, our

PAGE 65

54 benchmarking of each instruction, also, estimate d the effect of the pipeline data hazards involved in the execution of each instruction. Measuring Communication Cost To measure communication cost per byte, we wrote small UNIX socket programs that sent data back and forth to a server, a nd our results confirmed th e card statistics that were mentioned on the datasheet of our wire less LAN card. The data sheet suggests that this card is active 90% of the time transmitting at 265 mA and receiving at 170 mA main() { register int i; for(i = 0; i < 100000000; i++) { ;} } Figure 8-1. The C program used to estimate the energy cost of executing an empty for loop 100M times main() { register int i; registerint x; int a, b, c, d, e, f, g, h, m, n; for(i = 0; i < 100000000; i++) { // repeat 10 time the following 10 statement // each line represents an ldr instruction x = a; x = f; x = b; x = g; x = c; x = h; x = d; x = m; x = e; x = n; } } Figure 8-2. The C program used to estimat e the energy cost of the ldr instruction

PAGE 66

55 Simple Experimental Validation To test the effect of our approach on en ergy, we implemented three different simple benchmarks that span three different forma tions of data and execution complexity. The first of which was the Fibonacci loop, which c ontains constant data, but it executes in O(n) time. In other words, the size of the data remains constant, while the execution changes with n where n is the number to which we are trying to calculate the Fibonacci number. Due to the sampling limitation of our multi-meter, we had to test this 3 times using 3 large numbers to get more accurate results of our measurements. We performed the testing using the numbers, 100000, 200000, and 300000. Another benchmark that we used was a rectangular version of the bubble-sort loop which executes in O(n2) and the data size is linear. So, as the data size grows so will the computation complexity. We sorted 10000, 20000, and 30000 integers. The last benchmark that we used was a square matrix multiplication loop, which runs in O(n3) where n is the number or rows and the number of columns of each matrix. For matrix multiplication we used a 200x200, a 300x300, and 400x400 matrices. Each one of the benchmarks was executed on the Zaurus before our optimization and after our optimization. Table 8-1 gives the percentage of total energy saved by outsourcing the loop that calcu lates the Fibonacci number. Table 8-2, shows the total energy saved from outsourcing the square matrix multiplication loop. As can be concluded from the tables, a significant amount of energy can be saved when computationally expensive code can be outsourced. One interesting observation in our testi ng was the bubble-sort loop. The smallest energy saving was 98% for sorting 10,000 integers. This result is well expected due to the fact that compared to the amount of computation involved in bubble-sort, the size of the

PAGE 67

56 data is very negligible. That is not the cas e in the matrix multiplication and the Fibonacci sequence calculation. Table 8-1. Total energy save d by outsourcing the Fibonacci loop Value of n Energy savings % 100000 60 200000 80 300000 87 Table 8-2. Total energy saved by outs ourcing the matrix multiplication loop Number of rows Energy savings % 200 73 300 80 400 88 Figures 8-3, 8-4, and 8-5 show comp arison histograms between the local executions of the benchmarks vs. their remote execution (outsourcing) The results were generated while executing each version on the Zaurus while the Intuilink software is recording the data into a spreadsheet. Large-Scale Benchmark Validation To show the benefits of our approach, a more realistic benchmark had to be developed to show that this approach ha s more meaningful and potentially industryutilizable benefits. Some of the most computationally intensive computations are those involved in generating an image representing a 3-D graphics scene. To generate a 3-D graphics scene, the input and output of the program are extremely non-expensive processes, as even more complex scenes can be described with a vi rtually small amount of data. Also, the output is always a 2-D Imag e. But to get from a 3-D description of the scene to a 2-D Image depicting the scene, a huge amount of calculation has to be done. In this benchmark, the size of the data is O(n2) and the order of the computation is also O(n2) However, the amount of constant calc ulation within each iteration, is huge

PAGE 68

57 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:01.1 00:00:02.1 00:00:03.1 00:00:04.1 00:00:05.1 00:00:06.0 00:00:07.0 00:00:08.0 00:00:09.1 00:00:10.1 00:00:11.1 00:00:12.1 00:00:13.0 00:00:14.0 00:00:15.0 00:00:16.0 00:00:17.0 00:00:18.0 00:00:19.0 00:00:19.9 00:00:20.9 00:00:21.9Sample Time Intervals (Seconds)Current Drawn (Amps ) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.500:00:00.0 00:00:01.4 00:00:02.8 00:00:04.3 00:00:05.7 00:00:07.1 00:00:08.5 00:00:10.0 00:00:11.4 00:00:12.9 00:00:14.3 00:00:15.8 00:00:17.2 00:00:18.6 00:00:20.2 00:00:21.8 00:00:23.4 00:00:25.1 00:00:26.7 00:00:28.3 00:00:30.0 00:00:31.6 00:00:33.0 00:00:34.5Sample Time Intervals (Seconds)Current Drawn (Amps ) B A Figure 8-3. Local versus remote execu tion of 300 iterations of a Fibonacci loop executing 300,000 times. A) The histogram showing the current drawn during the time intervals listed for local execution. B) The histogram showing the current drawn during the time intervals listed for remote execution (longer spikes are for sendi ng while the shorter ones are for receiving). Voltage remains a cons tant 5 volts for both histograms.

PAGE 69

58 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.500:00:00.0 00:00:02.8 00:00:05.7 00:00:08.6 00:00:11.4 00:00:14.3 00:00:17.1 00:00:20.0 00:00:22.8 00:00:25.7 00:00:28.6 00:00:31.5 00:00:34.4 00:00:37.2 00:00:40.1 00:00:42.9 00:00:45.8 00:00:48.6 00:00:51.7 00:00:54.5 00:00:57.4 00:01:00.2 00:01:03.1 00:01:06.0Sample Time Intervals (Seconds)Current Drawn (Amps) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.700:00:00.0 00:00:02.1 00:00:04.2 00:00:06.3 00:00:08.4 00:00:10.4 00:00:12.5 00:00:14.6 00:00:16.7 00:00:18.8 00:00:20.9 00:00:23.0 00:00:25.1 00:00:27.2 00:00:29.3 00:00:31.4 00:00:33.5 00:00:35.6 00:00:37.7 00:00:39.8 00:00:41.9 00:00:44.0 00:00:46.1Sample Time Intervals (Seconds)Current Drawn (Amp s A B Figure 8-4. Local versus remote execution of matrix multiplication of a 400x400 matrix. A) The histogram showing the current drawn during the time intervals listed for local execution. B) The histogram showing the current drawn during the time intervals listed for remote execu tion. Voltage remains a constant 5 volts for both histograms. The energy sa ving is obvious in this benchmark.

PAGE 70

59 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:14.5 00:00:29.0 00:00:43.7 00:00:58.5 00:01:13.4 00:01:28.6 00:01:44.0 00:01:59.6 00:02:15.3 00:02:31.3 00:02:47.3 00:03:03.6 00:03:20.0 00:03:36.7 00:03:53.7 00:04:10.9 00:04:28.4 00:04:45.9 00:05:04.0 00:05:23.0 00:05:42.8 00:06:02.7 00:06:23.5Sample Time Intervals (Seconds)Current Drawn (Amps) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.700:00:00.0 00:00:03.7 00:00:07.3 00:00:10.9 00:00:14.6 00:00:18.4 00:00:22.0 00:00:25.7 00:00:29.3 00:00:33.0 00:00:36.8 00:00:40.4 00:00:44.1 00:00:47.9 00:00:51.6 00:00:55.4 00:00:59.1 00:01:03.1 00:01:06.8 00:01:10.6 00:01:14.5 00:01:18.3Sample Time Intervals (Seconds)Current Drawn (Amps)A B Figure 8-5. Local versus remote executi on of a bubble sort loop which sorts 50,000 integers. A) The histogram showi ng the current drawn during the time intervals listed for local execution, a nd it took approximately 6 minutes. B) The histogram showing the current draw n during the time intervals listed for remote execution, and it took only 1.2 mi nutes. Voltage remains a constant 5 volts for both histograms.

PAGE 71

60 when compared to the amount of communi cating each unit of data involved in the computation. Our experimental results show a significant amount of energy saving for generating a scene by ray-tracing 3 spheres of different sizes and colors in space to generate 3 different images of 50x50, 100x100, and 200x200. Figure 8-6, shows the input data passed to the ray-tracing proces s and figure 8-7 shows the image produced. The input to the ray tracing application is quite a simple input composed of a few floating point numbers. These numbers represen t the description of the world composed of: the observer, the light source, and the para meters describing three spheres in space. 200 200 .1 200 200 4 4 4 .8 .8 .8 4 4 4 0 0 0 1 1000 5 5 3 0.1 0.1 0.1 0.3 .5 .4 .3 1 .8 1 10 0.5 0.5 0.8 0.2 .3 .4 .2 1 .8 1 10 0.2 0.8 0.5 0.1 .4 .3 .5 1 .8 1 10 Figure 8-6. Input file for the ray tracing application Figure 8-7. The 2-D image representing th e 3-D scene generated by the ray tracing application for a 200x200 image

PAGE 72

61 We did our experiments on various sizes of data and we generated a 50x50 image, a 100x100 image, and a 200x200 image for the same scene. The amount of computation was so large that in all three cases, the computation was outsource d. Figures 8-8, 8-9, and 8-10 show the results of comparing local exec ution vs. remote execution for the 3 sizes of images mentioned above for the same scene. Other Measurements To accurately, and comprehensively discu ss our results, other metrics and overhead costs had to be measured once our approach is applied and utilized. One of these metrics is represented by the size of the executab le. The size of the executable was impacted greatly when it involved our first three benchmarks as they are considered small programs. The increase in executable size was 7 times for sort and matrix multiplication routines, and 3.5 times for the Fibonacci calculation. However, it was more encouraging when we dealt with much bigger application (which is going to be the case in real life). The size of the executable increased by only 40%. In addition, we measured the total execu tion cost of the battery and network monitors to realize that they consume ve ry little energy when running, as the work involved in their execution is very sporadic and very minimal. Figure 8-11 illustrates the total energy consumed by the monitors (only when they are running) otherwise the cost is near zero. Compile time was another metric that we m easured. It can easily be concluded that the compile-time will at least doub le, as the result of our approach will generate two files instead of one. However, in the process of generating two files, our energy optimization process at compile-time consumed about 1-2 additional seconds for the simple benchmarks, and about 5-7 seconds for the more complex benchmark of ray tracing.

PAGE 73

62 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.500:00:00.0 00:00:05.2 00:00:10.3 00:00:15.5 00:00:20.8 00:00:26.0 00:00:31.4 00:00:36.5 00:00:41.7 00:00:46.8 00:00:52.0 00:00:57.1 00:01:02.4 00:01:07.6 00:01:13.0 00:01:18.2 00:01:23.4 00:01:28.7 00:01:34.0 00:01:39.3 00:01:44.6 00:01:49.9 00:01:55.3 00:02:00.7Sample Time Intervals (Seconds)Current Drawn (Amps ) 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:01.5 00:00:03.1 00:00:04.7 00:00:06.2 00:00:07.8 00:00:09.3 00:00:10.8 00:00:12.3 00:00:13.9 00:00:15.4 00:00:16.9 00:00:18.5 00:00:20.0 00:00:21.5 00:00:23.1 00:00:24.6 00:00:26.1 00:00:27.7 00:00:29.2 00:00:30.8 00:00:32.3Sample Time Intervals (Seconds)Current Drawn (Amps)A B Figure 8-8. Local versus remote execution fo r ray tracing three s pheres in space and generating a 50x50 image. A) The histogram showing the current drawn during the time intervals listed for lo cal execution (approximately 66 Joules in 1.2 minutes). B) The histogram s howing the current drawn during the time intervals listed for remote execution (approximately 2.5 Joules in 2 seconds). Voltage remains a consta nt 5 volts for both histograms.

PAGE 74

63 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:11.1 00:00:22.1 00:00:33.3 00:00:44.5 00:00:55.6 00:01:06.8 00:01:18.3 00:01:29.3 00:01:40.5 00:01:51.7 00:02:03.1 00:02:14.6 00:02:26.1 00:02:37.8 00:02:49.5 00:03:01.3 00:03:13.4 00:03:25.6 00:03:37.8 00:03:50.1 00:04:02.7 00:04:15.2 00:04:28.0Sample Time Intervals (Seconds)Current Drawn (Amps) 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:01.4 00:00:02.9 00:00:04.3 00:00:05.7 00:00:07.1 00:00:08.6 00:00:10.0 00:00:11.4 00:00:12.9 00:00:14.3 00:00:15.7 00:00:17.1 00:00:18.6 00:00:20.0 00:00:21.4 00:00:22.8 00:00:24.3 00:00:25.7 00:00:27.2 00:00:28.6 00:00:30.1 00:00:31.5 00:00:32.9Sample Time Intervals (Seconds)Current Drawn (Amps)A B Figure 8-9. Local versus remote execution fo r ray tracing three s pheres in space and generating a 100x100 image. A) The hist ogram showing the current drawn during the time intervals listed fo r local execution (approximately 240 Joules in 4 minutes). B) The histog ram showing the current drawn during the time intervals listed for remote execution (approximately 2.5 Joules in approximately 3 seconds). Voltage re mains a constant 5 volts for both histograms.

PAGE 75

64 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:32.4 00:01:04.9 00:01:37.8 00:02:11.6 00:02:46.9 00:03:23.7 00:04:01.3 00:04:40.9 00:05:22.8 00:06:05.1 00:06:47.3 00:07:30.5 00:08:15.8 00:09:04.5 00:09:55.4 00:10:48.8 00:11:47.5 00:12:45.4 00:13:38.8 00:14:34.6 00:15:32.7 00:16:33.6 00:17:39.9Sample Time Intervals (Seconds)Current Drawn (Amps)B A 0 0.1 0.2 0.3 0.4 0.5 0.600:00:00.0 00:00:01.4 00:00:02.9 00:00:04.3 00:00:05.7 00:00:07.1 00:00:08.6 00:00:10.0 00:00:11.4 00:00:12.8 00:00:14.3 00:00:15.7 00:00:17.1 00:00:18.6 00:00:20.0 00:00:21.4 00:00:22.8 00:00:24.3 00:00:25.7 00:00:27.1 00:00:28.5 00:00:30.1 00:00:31.5Sample Time Intervals (Seconds)Current Drawn (Amps) Figure 8-10. Local versus remote execution fo r ray tracing three spheres in space and generating a 200x200 image. A) The hist ogram showing the current drawn during the time intervals listed fo r local execution (approximately 1020 Joules in 17 minutes). B) The histog ram showing the current drawn during the time intervals listed for remote execution (approximately 2.5 Joules in approximately 6 seconds). Voltage re mains a constant 5 volts for both histograms.

PAGE 76

65 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.500 : 0 0 : 0 0.0 00 : 0 0 : 0 3.2 00 : 0 0 : 0 6.4 00 : 0 0 : 0 9.6 00 : 0 0 : 1 2.8 00 : 0 0 : 1 6.0 00 : 0 0 : 1 9.1 00 : 0 0 : 2 2.3 00 : 0 0 : 2 5.5 00 : 0 0 : 2 8.7 00 : 0 0 : 3 1.8 00 : 0 0 : 3 5.0 00 : 0 0 : 3 8.2 00 : 0 0 : 4 1.4 00 : 00 : 4 4.7 00 : 0 0 : 4 7.9 00 : 0 0 : 5 1.1 00 : 0 0 : 5 4.3Sample Time Intervals (Seconds)Current Drawn (Amps) Figure 8-11. Energy cost for the battery a nd network monitors execution. Only the current and the time are shown as th e voltage is a constant 5 volts throughout the experiments. Normalization of Results In order to show the normalization of ener gy cost for an outsourced application, the processor speed of the server has to be taken into consideration. This is due to the overhead energy (base energy) spent between the time it sends the data to the server and the time it receives the results back. This ener gy spent is taken into consideration as it lengthens the time the processor is active a nd therefore it can not go into sleep mode (which saves energy). The amount of energy spent during sleep mode is actually quite negligible and irrelevant to the calculation of the energy consumed. Therefore, the energy consumed by an application depends on th e base energy as well as the communication cost.

PAGE 77

66 In our measurements, the base energy consumed by the mobile device was calculated at 0.25 Amps 5 Volts (server execution time). Add to it the communication energy cost, the result will be the total energy consum ed by each loop executing remotely. First of all, a model has to be devised for the local execution of each program. Assume that we are consuming PowerBase watts without running any applications on the mobile device. Our mobile device executes MipsPower MIPS/watt. Therefore, if our application requires AppInst Million instructions to execute, and it takes TLocal seconds to execute. That means that the total en ergy consumed during the execution of the application is given by the following equation: MipsPower AppInst PowerBase Tlocal Elocal Given that mips AppInst Tlocal where the processor is capable of mips MIPS (should be a number very close to the processor sp eed). The equation for calculating the energy consumed locally can be rewritten as follows: MipsPower mips AppInst mips AppInst PowerBase Elocal2 Therefore, the Energy spent in the executi on of the application locally depends not only on the time, and power, but it also depe nds on the speed of the processor. So the larger the amount of mips is, the smaller the am ount of energy consumed. At this point we can calculate the en ergy consumed by executing the loop (or the entire application if it has only one loop) remotely. Prior to its execution, the mobile device is consuming PowerSleep watts, However, after the start of the execution, the

PAGE 78

67 device will spend PowerAwake watts as it will wake up. Therefore, the amount of PowerBase = PowerSleep + PowerAwake As the value of PowerSleep is not a factor in the energy consumed by the application (the mobile device is spending it regardless of what it does), the value of PowerBase will be equal to the value of PowerAwake The server device can execute in Tremote seconds. The energy cost of sending and receiving the data is a function of the time to communi cate the data, as well as the power cost for the communications. Therefore, we consider ts and tr as the time it takes to send and receive respectively, and ps and pr are the power cost for send ing and receiving the data. Therefore, the energy cost for outsourcing an application is given by the following equation: PowerBase Tremote pr tr ps ts Eremote Just like in the case of ca lculating the time it takes the client to execute a program, the same calculation can be applied to the server. The server is running RAppInst million instructions to execu te, and it is running on rmips MIPS (or MHz). Therefore Tremote can be given by: rmips RAppInst Tremote and as a result, the equation to calculate the energy consumed by executing the application remotely is as follows: rmips PowerBase RAppInst pr tr ps ts Eremote Therefore, the lower the values of ts, ps, tr, pr, and the higher the value rmips (the faster the server machine is), the less the amount of base energy consumed waiting for the results. As a result, fa ster communication (lower ts, and tr) as well as faster server machines makes a difference wh en outsourcing computations.

PAGE 79

68 In order to model our normalization of the measurements according to the processors speed, we applied th e above equation for calculating Eremote, both, theoretically, and experiment ally using our benchmarks. Our 3-D rendering loop was benchmarked using 3 different image sizes (50x50, 100x100, and 200x200). As a result, you can see that the execution time for the ap plications will quadruple from one run to the other. Also, we used the same size data when running each benchmark (200x200 image) to handle (using the same executable) all benchmarks. Therefore, in effect, we modeled slowing down the processor to of its speed with each run, while keeping the communication cost the same. The processor th at we used for the server was a 500 MHz Pentium III (this is also the value of the rmips), which means with the 50x50 image, it will have the same speed, with the 1 00x100 image will effectively model a 125 MHz processor, and with the 200x200 it will be a 31.5 MHz processor. We also estimated, the number of instructions executed for the 50x50 application, and they came out to be approximately 125 million instructions. The estimation was done based on the number of assembly instructions generated as well as the possible logic of th e program, so it is only an estimation as opposed to an actual c ount. The transmission time for sending and receiving was modeled to take 1 second each time with the sending consuming .265 Amps, and the receiving consuming .170 Amps a ccording to the specification of the WiFi card. The value of base current lost was esti mated at .25 Amps. The voltage is assumed to be a constant 5 Volts throughout the process Theoretically, the value of the energy w ould be calculated using the values given by the specifications of the network card (.265, and .170) for communication, and the processor speeds for each benchmark (500, 125, and 31.5). Which means that, given the

PAGE 80

69 125 million instructions application, the total amount of base energy consumed would be multiplied by 1 and 4 seconds respectively for each run and that means it should quadruple. When we measured the values of the af orementioned metrics, we came up with values that are very close to the ones me ntioned above. The run for the 50x50 image took approximately of a second, the 100x100 t ook approximately 1.2 seconds, and the 200x200 took approximately 4.2 seconds. The tr ansmission time was approximately 1 minute for each the send and receive. The current cost for the send was approximately 0.255 Amps, and for the receive it was approximately .170 Amps. Figure 8-12 shows a comparison between the theore tical and the experimental measurements of the total energy consumed for each processor speed. Figure 8-13 shows a comparison between the theoretical and experimental measurements of the amount of base energy consumed. Our results match those acquired by the normalized analysis for the remote execution of the application. 0 1 2 3 4 5 6 7 8 9 10 11 50012531.5 Processor Speed (MHz)Total Energy Consumed (Joules) Theoretical Experimental Figure 8-12. Comparison between theoretical and experimental measurements of the total energy consumed.

PAGE 81

70 0 1 2 3 4 5 6 50012531.5 Processor Speed (MHz)Base Energy Consumed (Joules) Theoretical Experimental Figure 8-13. The amount of base energy cons umed based on the server processor's speed 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 11.11.21.31.41.51.61.71.81.92 Time to Send to Server (seconds)Total Energy ConsumedA 0 0.5 1 1.5 2 2.5 3 3.5 4 11.11.21.31.41.51.61.71.81.92 Time to Receive from Server (seconds)Total Energy ConsumedB 0 0.5 1 1.5 2 2.5 3 3.5 11.11.21.31.41.51.61.71.81.92 Power to Send to Server (watts)Total Energy ConsumedC 0 0.5 1 1.5 2 2.5 3 3.5 4 11.11.21.31.41.51.61.71.81.92 Power to Receive from Server (watts)Total Energy ConsumedD Figure 8-14 The effect of changing the communication metrics on the total energy consumed. A) The effect of the send tim e. B) The effect of the receive time. C) The effect of the send power. D) The effect of the receive power.

PAGE 82

71 We have mentioned before that the value of Eremote also depends on the power as well as the time consumed during the communication (ts, ps, tr, pr). In this work, we model the theoretical behavior of the effect of these communication metrics on the energy cost of the equation. We will assume that the processor speed is 500 MHz (500 MIPS), the base current drawn is 0.25 Amps with the same application of an approximate 125 million instructions. Figure 8-14 shows the effect of changing the metrics ts, ps, tr and pr.

PAGE 83

72 CHAPTER 9 CONCLUSION AND FUTURE WORK Our experimental evaluations showed th at computation outsourcing within a pervasive computing smart space has a gr eat potential in energy management. By exporting CPU processing responsibilities into the network, the mobile device was able to deliver the expected functi onality while consuming less energy and lasting for a longer period of time. Therefore, communication shoul d not be viewed as a drain on the battery, but as an opportunity to save energy. We found that the research done in the domai n of real-time systems is quite useful as its aim is always knowing as much as po ssible about an application before runtime (e.g., compile-time). The reason for that in r eal-time systems is to execute programs to finish within a deadline. That means that the more we know about program behavior before execution the better the program can be tuned to either meet a deadline in the case of real-time systems, or consume less en ergy in our case of m obile and pervasive computing. However we believe that the li mitations of the utilized methodology which are represented in the inabil ity to handle non-counter-based loops, such as those loops that are pointer-based, logical -expression based, and some non -rectangular nested loops, raise the need to investigate other computer science disciplines, specifically automated software verification. While the work done in the area of au tomated verification for loop invariant generation is not quite mature enough, it has a great promise to be combined with the utilized real-time systems methodology to gene rate accurate estimates for the number of

PAGE 84

73 loop iterations. We will investig ate the research done in this area to determine the extent of its applicability in our wo rk. Specifically, we will look at the work done by Pasareanu and Visser [28]. We believe that, when combined with the methodology we utilized for determining the loop iterations, the area of loop invariant generation will be quite beneficial to our research. As part of our future work, we will continue the implementation of the client/server version that we have already started. Ho wever, this implementation will contain additional support for dynamic memory allo cation and subroutine-level computation outsourcing. In addition, additional work needs to be done to evaluate a more accurate communication cost. This part will be an ongoi ng part of the research as we utilize additional outsourcing techniques. In additi on, we will continue to identify specific applications that will benefit from our appro ach which will be a re finement of defining categories of application areas that we believe will benefit from this approach. To handle multiple platforms, targeting languages such as Java and C++ will be necessary. This will require porting a compiler to handle both languages while augmenting with the code that we obtained to calculate the number of loop iterations. Additionally, as we will support additional types of basic program blocks we will build the compiler support that will handle estima ting total execution cost of these basic program blocks such as additi onal types of loops than thos e supported so far, recursive functions (inherently these are loops), library linked functions whose energy consumption is predefined, and functions defined within the code. Additionally, we will investigate in the case of non-Java languages, the possibility to be able to cross-compile them for the most popular mobile devices.

PAGE 85

74 In this research, we assumed that if a basic program block (loop) contained an I/O operation, it is determined as non-outsource-able However, in our future work we will be looking at opportunities where I/O operations may be performed elsewhere. For input operations that involve files, if the file exists elsewhere, then it could be energybeneficial to read the file on a remote machine, and u tilize its contents remotely. Similarly, if producing the output elsewhere and basically all we are interested in is a display of this output which maybe cheaper th an displaying the output locally, then that is another opportunity that needs to be investigated. Identification of the applications, whic h contain the computationally expensive basic program blocks, is an essential part of this research. We will investigate and research the different types of basic program blocks that fall into this category and test them and provide them as benchmarks for our research. In addition, we will build the support for other useful applications that will benefit from our approach. Java remote method invocation (RMI), and remote procedure call (RPC) based systems will also be applicable in our rese arch where we will let the system make the necessary data communication according to its po licies and that will benefit our approach especially at the level of outsourcing sp ecific functions, and subroutines. These two systems are utilized in grid computing environments. We will weigh the benefits of every outs ourcing mechanism with each type of basic program block we investigate, and investigat e which mechanism allows us to save more energy with a basic program block. This will lead to a hybrid approach where a single application may contain one, two, or three outsourcing mechanisms.

PAGE 86

75 Also, as far as service discovery is con cerned, we will investigate a lighter version of some of the well-know service discove ry systems lik UPnP (http://www.upnp.org), and Jini (http://www.sun.com/software/jini). We believe that these systems can be utilized without spending a large amount of energy as the intelligence has already been built in the modified program.

PAGE 87

76 LIST OF REFERENCES [1] A. Abukmail, A. Helal, Power Awarene ss and Management Techniques, in: M. Ilyas, I. Mahgoub (Eds.), Mobile Computing Handbook, CRC Press, Boca Raton, FL, 2004. [2] A. Abukmail, A. Helal, A Pervasive Intern et Approach to Fine-Grain Power-Aware Computing in: IEEE/IPSJ International Sym posium on Applications and the Internet, Phoenix, Arizona, January 2006. [3] A. Aho, M. Ganapathi, S. Tjiang, Code Generation Using Tree Matching and Dynamic Programming, ACM Transactio ns on Programming Languages and Systems 11 (4) (1989) 491-516. [4] N. Bellas, I. Hajj, C. Polychronopoulos, G. Stamoulis, Architectural and Compiler Support for Energy Reduction in the Memo ry Hierarchy of High performance Microprocessors, in: International Sy mposium on Low Powe r Electronics and Design, Monterey, CA, February1998. [5] F. Bellosa, The Benefits of Event-Driven Energy Accounting in Power-Sensitive Systems, in: 9th ACM SIGOPS European Workshop, Kolding, Denmark, September 2000. [6] M. Benitez, J. Davidson, A Portable Global Optimizer and Linker, in: ACM Conference on Programming Language De sign and Implementation, Atlanta, Georgia, July 1988. [7] T. Cignetti, K. Komarov, C. Schlatter Ellis, Energy estimation tools for the Palm, in: 3rd ACM International Workshop on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Boston, MA, August 2000. [8] C. Ellis, The Case for Higher Level Power Management, in: 7th Workshop on Hot Topics in Operating Systems, Rio Rico, AZ, March 1999. [9] J. Flinn, S. Park, M. Satyanarayanna n, Balancing Performance, Energy, and Quality in Pervasive Computing, in: 22nd In ternational Conference on Distributed Computing Systems, Vienna, Austria, July 2002. [10] J. Flinn, M. Satyanarayanan, Energy-aware adaptation for mobile applications, in: 17th ACM Symposium on Operating Syst em Principles, Kiawah Island, SC, December 1999.

PAGE 88

77 [11] J. Flinn, M. Satyanarayanan, PowerScope: A Tool for Profiling the Energy Usage of Mobile Applications, in: 2nd IEEE Workshop on Mobile Computing Systems and Applications, New Orleans, LA, February 1999. [12] I. Foster, C. Kesselman, S. Tuecke, The anatomy of the Grid, in: T. Berma, G. Fox, A. Hey (Eds), Grid Computing: Making th e Global Infrastructu re a Reality, Wiley Series in Communication Networking a nd Distributed Systems, John Wiley and Sons Ltd, West Sussex, England, 2003. [13] C. Fraser, D. Hanson, T. Proebsting, Engin eering Efficient Code Generators using Tree Matching and Dynamic Programming, Technical Report No. CS-TR-386-92, Princeton University, August 1992. [14] C. Gebotys, Low Energy Memory and Register Allocation Using Network Flow, in: 34th Conference on Design Automation, Anaheim, California, June 1997. [15] X. Gu, A. Messer, I. Greenberg, D. Milo jicic, K. Nahrstedt, Adaptive Offloading for Pervasive Computing, IEEE Pervas ive Computing 3 (3) (2004) 66-73. [16] J. Haid, W. Schogler, M. Manninger, Desi gn of an Energy-Aware MP3-Player for Wearable Computing, in: Telecommunicati on and Mobile Computing Conference, Graz, Austria, March 2003. [17] C. Healy, M. Sjdin, V. Rustagi, D. Whalley, Bounding Loop Iterations for Timing Analysis, in: IEEE Real-Time Technology and Applications Symposium, Denver, CO, June 1998. [18] A. Helal, Pervasive Java Part II, IE EE Pervasive Computing 1 (2) (2002) 85-89. [19] C-H. Hsu, U. Kremer, M. Hsiao, Comp iler-Directed Dynamic Frequency and Voltage Scheduling, in: 1st Internati onal Workshop on Power-Aware Computer Systems, Cambridge, MA, November 2000. [20] M. Kandemir, N. Vijaykrishnan, M. Ir win, W. Ye, Influence of compiler optimizations on system power, in: 37th Conference on Design Automation, Los Angeles, CA, June 2000. [21] R. Kravets, P. Krishnan. Power Management Techniques for Mobile Communication, in: 4th ACM/IEEE Inte rnational Conference on Mobile Computing and Networking, Dallas, Texas, October 1998. [22] U. Kremer, J. Hicks, J. Rehg, Compile r-Directed Remote Task Execution for Power Management, in: Workshop on Comp ilers and Operating Systems for Low Power, Philadelphia, PA, October 2000.

PAGE 89

78 [23] C. Lee, D. Talia, Grid programming models: current tools, issues and directions, in: T. Berma, G. Fox, A. Hey (Eds), Grid Computing: Making the Global Infrastructure a Reality, Wiley Seri es in Communication Networking and Distributed Systems, John Wiley and Sons Ltd, West Sussex, England, 2003. [24] R. Loy, A. Helal, Active Mode Power Ma nagement in Mobile Devices, in: 5th World Multi-Conference on Systematics, Cybernetics, and Informatics, Orlando, FL, July 2001. [25] D. Marculescu, Profile-Driven Code Ex ecution for Low Power Dissipation, in: International Symposium on Low Power Electronics and Design, Rapallo, Italy, July 2000. [26] Z. Nemeth, V. Sunderam, A Formal Framework for Defining Grid Systems, in: 2nd IEEE/ACM International Symposium on Clus ter Computing and the Grid, Berlin, Germany, May 2002. [27] J. Palm, J. Eliot, B. Moss, When to use a compilation service?, in: ACM Joint Conference on Language Compilers and Tools for Embedded Systems and Software and Compilers for Embedded Systems, Berlin, Germany, June 2002. [28] C. Pasareanu, W. Visser, Verification of Java Programs Using Symbolic Execution and Invariant Generation, in: 11th In ternational SPIN Workshop on Model Checking of Software, Barcelona, Spain, April 2004. [29] A. Rudenko, P. Reiher, G. Popek, G. Kuenning, The Remote Processing Framework for Portable Computer Power Saving, in: ACM Symposium on Applied Computing, San Antonio, TX, February 1999. [30] M. Satyanarayanan, Pervasive Computing: Vision and Challenges, IEEE Personal Communication 8 (4) (2001) 10-17. [31] G. Smit, P. Havinga A Survey of energy saving techniques for mobile computers, Internal Technical Report, University of Twente, Enschede, Netherlands, 1997. [32] T. Starner, Powerful Change Part 1: Batteries and Possible Alternatives for the Mobile Market, IEEE Pervasive Computing 2 (4) (2003) 86-88. [33] C-L. Su, C-Y. Tsui, A. Despain, Low Power Architecture Design and Compilation Techniques for High-Performance Proce ssors, Technical Report No. ACAL-TR94-01, University of Southern California. February 1994. [34] V. Tiwari, S. Malik, A. Wolfe, Comp ilation Techniques for Low Energy: An Overview, in: International Symposium on Lo w Power Electronics and Design, San Diego, CA, October 1994. [35] V. Tiwari, S. Malik, A. Wolfe, T. Lee, Instruction Level Power Analysis and Optimization of Software, VLSI Signa l Processing Systems 13 (2) (1996) 1-18.

PAGE 90

79 [36] M. Velluri, L. John, Is Compiling for Pe rformance == Compiling for Power?, in: 5th Annual Workshop on Interaction between Compilers and Computer Architecture, Monterrey, Mexico, January 2001. [37] E. Witchel, S. Larsen, C. Ananian, K. Asanovic, Direct Addressed Caches for Reduced Power Consumption, in: 34th Annual International Symposium on Microarchitecture, Austin, Texas, December 2001. [38] T. Yeh, D. Long, S. Brandt. Conservi ng Battery Energy through Making Fewer Incorrect File Predictions, in: IEEE Workshop on Power Management for RealTime and Embedded Systems, Taipei, Taiwan, May 2001. [39] W. Yuan, K. Nahrstedt, X. Gu, C oordinating Energy-Aware Adaptation of Multimedia Applications and Hardware Resources, in: 9th ACM Multimedia Middleware Workshop, Ottawa, Canada, October 2003.

PAGE 91

80 BIOGRAPHICAL SKETCH Ahmed Abukmail received a Bachelor of Science degree in computer and information science from the Ohio State University (Columbus, Ohio) in 1993. In 1995, he received a Master of Science degree in computer and information science from the University of Florida. During the same year, he was admitted to the Ph.D. program. In 1997, Ahmed left the University of Florida for 5 years to pursue a career in industry. In 2003, he returned to the University of Florida to complete his Ph.D. degree. His major research is in the areas of mobile and pervasiv e computing with particular interest in energy and power-aware compu ting, programming languages, compilers, and distributed systems.


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20110331_AAAABC INGEST_TIME 2011-03-31T11:42:39Z PACKAGE UFE0013060_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 623 DFID F20110331_AABARE ORIGIN DEPOSITOR PATH abukmail_a_Page_45.txt GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
6f095b743dbfec70d0f613c5324dc3e0
SHA-1
0367be2c06dce0a0e797bab7ba5b77edc5c9a6b3
1053954 F20110331_AABARF abukmail_a_Page_79.tif
979b01eb0a5d079ca2c48627130c4daa
a0f18c9adf8dfe28c8d48b3fd98f7acba4d8dd96
1779 F20110331_AABARG abukmail_a_Page_43.txt
e4d81ab8a42a1bf845ba6210e71c2b03
8beca453b88fbf3801232dee1c8c1ec2e13e7fa6
6086 F20110331_AABARH abukmail_a_Page_59thm.jpg
794118734dbee30353de5bfe26cd2370
a83b976891ce31f7ef5440a23c9963f75bf7bfcd
104341 F20110331_AABARI abukmail_a_Page_46.jp2
e55695e8f6625789caeec1f28408b930
bbbfe8b031e90ac7f963a325addc7571ad3bb404
29299 F20110331_AABARJ abukmail_a_Page_33.QC.jpg
a1a62a4c33528d32a77bf45c9a25be32
1be3dd4de63d6d1cc6f87136f2cb0a62bc38afd0
78353 F20110331_AABARK abukmail_a_Page_58.jpg
2276ea36ffe0b27e02f24c6e7568730e
50ca8f4626f8fa01c2c1ba96dac704668794978c
95999 F20110331_AABARL abukmail_a_Page_85.jpg
632eeb94c4bb9797eb2345d940a90aac
9be01a376de1bc7f9d9d20e81bfe62cb8736371c
F20110331_AABASA abukmail_a_Page_11.tif
9580725de13fa6148cfb0ac91dde6f14
70d4cd590849ffc5e798174abd4e3fbf4d16ced1
1888 F20110331_AABARM abukmail_a_Page_85.txt
c204ffcb254d563dacab556b19eb309b
3a6c63e6d10366f18eefe65c51f6a5989a011c5c
F20110331_AABASB abukmail_a_Page_52.tif
91278409ae2dce25aadc453ba38b7151
87e23d99492ff980550810e21719db70cfb3664a
27646 F20110331_AABARN abukmail_a_Page_19.QC.jpg
22a01a11c07ecd7448232361d7239793
ecdd3370eb9008321cf8abb5b6fffd52c7db52c5
16867 F20110331_AABASC abukmail_a_Page_51.QC.jpg
e7edf136ff61ade0cbc41939b13e5223
5d9fb531edacb8e54504a50b6ec6820c29f9662a
78748 F20110331_AABARO abukmail_a_Page_61.jpg
ac0d2f95115c845bded6324eed3061b8
ab26b98648c88327dc92b6a4384ac703926bb3bb
90062 F20110331_AABASD abukmail_a_Page_74.jpg
8f2a7157dc93cda395b46ac269b39b59
4a104f2e8fce55351083810967bf4df456838cd6
F20110331_AABARP abukmail_a_Page_04.tif
c5823225bc32e58c68031b1a62465e51
2bc80422375528af79d6b15a03a677e00f618c14
26934 F20110331_AABASE abukmail_a_Page_75.QC.jpg
75018d032aed08ec203f3b71f1c66261
9ec5255d792044bbeb8ea96f9345839cc88cab9a
87082 F20110331_AABARQ abukmail_a_Page_43.jpg
061ff4e722a0078bec3edd58b081be94
a307ecb590b0c47149d2d617911dddf791f04e9d
1265 F20110331_AABASF abukmail_a_Page_35.txt
c65ea6870f82276581903d722b83a607
9a913714eeef26d35133d9a9b8187e541a3ed577
4929 F20110331_AABARR abukmail_a_Page_48thm.jpg
04bd3b18e94489aed0c44ed16b8900ed
a2f7f51d8ec221dd5f7c11584562d3ad62710c94
98132 F20110331_AABASG abukmail_a_Page_18.jp2
1412f0537455312dbac56bc77ebc8a03
eb6b79aee1ee40b2bb4070244807260a7e42b407
8358 F20110331_AABARS abukmail_a_Page_31thm.jpg
0a81ca3f7e4abb563df2bba424a906b2
dc610189346c23b9e04073878c08276c99a35343
7682 F20110331_AABASH abukmail_a_Page_78thm.jpg
7a8af417d47976ae3828bd108bc795a6
c2515ddd012d404ca9ba3b2c5732e6a6f1ec8bd7
45682 F20110331_AABART abukmail_a_Page_63.pro
001a535c573600083e1a8104e3b24176
8e42300f75c4fef31b1cae68df6b92eaee79913d
4632 F20110331_AABASI abukmail_a_Page_09thm.jpg
bedd6d867c92077be08f06feef9e6682
d93a7abb264a8015c741c0d97a9e8660d2bd9db0
1765 F20110331_AABARU abukmail_a_Page_77.txt
0d6507e42381a8e8ebf504ee3cd8abc7
704941aa47160b085a4254b53f9e1e23d2b16830
23629 F20110331_AABASJ abukmail_a_Page_59.QC.jpg
494ab22ee98ca23ca4069bf77471e5b1
3154ec4ef9547a60a9f043d7e0db9cd2d47beb22
33223 F20110331_AABARV abukmail_a_Page_21.QC.jpg
e9d64baceaa2881cd92bcba54f536fd1
19dfdc4a733afe85b0ba18e34ae60ed16ee58ba3
7054 F20110331_AABASK abukmail_a_Page_62thm.jpg
138d8acb6bbc47393d91906b89bed39d
fdefd4b4b43b36c36c5a0dca06d2cc583a995b50
63181 F20110331_AABARW abukmail_a_Page_48.jp2
09e9ed45d33b1ab4c22c3aefa471bcce
4d9e5a0240c0f4fcbd7b19e1fc8c9e1df252925c
1062 F20110331_AABASL abukmail_a_Page_76.txt
0483b207b50424e6e42ea234cb200bbc
73c7a338b2a605183cb031f9b6563312315d6e22
2066 F20110331_AABARX abukmail_a_Page_52.txt
b16d3e66037ef281a6b85935ed1ba99d
45758eb733f2a682c1f38ed56bfb3986fe726d8c
7026 F20110331_AABASM abukmail_a_Page_15thm.jpg
8be1a383d1ab0d2f35696d56bc75bfb9
d4c417dce7b71a30407e3bcbfebc9ad9a6426dd2
2109 F20110331_AABATA abukmail_a_Page_57.txt
60672ae9a4bb9984b7034a2620b4a6a5
1cc6a0455e38f99bd4f97ef5bb71ba6f27ad1d2b
1822 F20110331_AABASN abukmail_a_Page_20.txt
7cc95e6b3dc7a7e4f30a4b4ee8c8500e
289bb8a67000027836ed32f50a03ca8637f15538
F20110331_AABARY abukmail_a_Page_25.tif
f6d85fa6f0400f8121474b993ca2ca47
9345077f0283119762057dfd39603e32c0b58ecd
92271 F20110331_AABATB abukmail_a_Page_12.jp2
414d50e543f53c039cbad910a91cfddc
8864a3e869198e1a71649582187a8fecbe01b96c
49504 F20110331_AABASO abukmail_a_Page_21.pro
8e227c5ad47ab0979156854037dc689c
c937efc7dcaf675e1faa1c69020598ea51e2ddc9
7081 F20110331_AABARZ abukmail_a_Page_19thm.jpg
f736930cab8f265f2435aef270a1e58e
c9dbe681cb4e2ffdc870e88c270c7a8b2cceed45
48263 F20110331_AABATC abukmail_a_Page_31.pro
ed64643733c710e3dd6f4f0a520eba27
2171740a3c70d194fa9906784bc2e72fe7a683c8
8373 F20110331_AABASP abukmail_a_Page_30thm.jpg
e69e1a5ee3b0de3a5156aafe7fb2e04d
487ebc972ace071a22178b99a6c5cf28a889dcdf
F20110331_AABATD abukmail_a_Page_38.tif
c788ace038d37d75ff6b7edf4399e60e
977a37d418067a422ac6fb2b4142e91e2a34cc89
88683 F20110331_AABASQ abukmail_a_Page_15.jpg
e9dd02c22e96dde72828f1481d49b99a
14349242744449aeda9a711a2a93b5a81b7784a3
2859 F20110331_AABATE abukmail_a_Page_04thm.jpg
a0c5965a6e457f34eb931c10f22494fe
abfd78a765284f259b2579cc889670b7385ad75b
25271604 F20110331_AABASR abukmail_a_Page_68.tif
b9eae59ece66ebbd2b68efc5f5254ea7
57dcd54ac853cbe212f7804e07aaedff3bcb2ba4
48982 F20110331_AABATF abukmail_a_Page_64.pro
443a68480276c3edbc8310900b8ea190
84d2f38066217b27acf67640432cc9f5edab7982
7271 F20110331_AABASS abukmail_a_Page_43thm.jpg
9ffd35e518f900659e5ca7696086f93c
64f03cb7c07da51dd863bc7a86aa1aca14836811
1792 F20110331_AABATG abukmail_a_Page_83.txt
de904a97b7b1f0b65989b862b14864cd
6c5874a45e38cbf0e3a8f97902127661fbff12c2
F20110331_AABAST abukmail_a_Page_64.tif
4b4124f461b2af897f0539b2ea98fe8b
b4c01150b462d8e0bb5d97773db97156d20f1aa3
F20110331_AABATH abukmail_a_Page_67.tif
50386b55032676a01ab3cb4f6b8ac06c
29836de9560f926e40961b2233cc495f8b83cdb9
32471 F20110331_AABASU abukmail_a_Page_42.QC.jpg
7528f377f0c314c10bc34855dca9b0f3
ddfe9ef03ff7b56ef4c170ea79a033d8bac6f116
F20110331_AABATI abukmail_a_Page_20.tif
a767f36e136279da797a6db92803bf77
310f3897fdb3f29bd541b3f019bf3cf755f5c83b
F20110331_AABASV abukmail_a_Page_23.tif
0cd4cfc31594eb8b449b95b6b78f30cc
109c2b1053462c3706ac50cee7b2cc5583cc9872
658 F20110331_AABATJ abukmail_a_Page_02thm.jpg
4dc43fc5918d193434c92cd0e0b2368f
09bde3e858111293c68c96fd7c99147ef4bb681d
1899 F20110331_AABASW abukmail_a_Page_29.txt
fee132d155c23fb3e319890df52f5e53
8a40e88369f5f6bfca112d98d227284d07de0276
101889 F20110331_AABATK abukmail_a_Page_31.jpg
e0078626e5a5b068f5b3a1b87d5567c9
24dc986e1f441d66a01bfb30e626d18aad27efce
102237 F20110331_AABASX abukmail_a_Page_36.jpg
d92452cbba96779b3c9dcf186ac96e91
435927b6b778514580ee95fa2b6cc95fe3faca42
52645 F20110331_AABATL abukmail_a_Page_57.pro
dcc11b24f50a1be42e80f6400712ff36
6867d19b53d826c82de5998496f3ffc1d0397fd5
7681 F20110331_AABASY abukmail_a_Page_74thm.jpg
9b64811d5d8b5df842b9642d8341b366
0f5f3232227298e9e76a7f4b244e4945124cd620
7724 F20110331_AABAUA abukmail_a_Page_22thm.jpg
ffa1840463be4c5ea6684d1485136c08
59afe171bd3ecb8db64786c3c52e799cb3432ba9
50186 F20110331_AABATM abukmail_a_Page_34.pro
da14a0785e807a51e4a547a2077375eb
a32439cb65af5e2ca50818dab7624a80ba753ab5
30185 F20110331_AABAUB abukmail_a_Page_74.pro
555d1751fdd5c3bac0b8ae040220288f
6b0ca56abcfbfbbfb5282cd5c3d676b0da52b90d
18649 F20110331_AABATN abukmail_a_Page_91.pro
e8a5382333de4293d22701e284800613
edcf845b0fe97217c02bf14be54d9bae7eaf2513
8204 F20110331_AABASZ abukmail_a_Page_66thm.jpg
fc5283373dad516415d4e4d53d6ec6a3
acb4aef83d80b40779ddee8573b133fcb57c3a0d
51316 F20110331_AABAUC abukmail_a_Page_52.pro
ef89d70dbdc2b7601bcaa78a4f99b13f
e68b1ceb1d0ef5d151120660faf8fd295314b810
99535 F20110331_AABATO abukmail_a_Page_63.jp2
0995be4e82b77e2382621999f43e153d
b196820029834fe23f837af2fc57053082983015
103370 F20110331_AABAUD abukmail_a_Page_05.jpg
807ae18d25c3bbe7525800473b6e18d8
6b9903d173054abc9a05c455b2eee846f407f5c9
F20110331_AABATP abukmail_a_Page_80.tif
c6ae7f05baef933751c80ef40e8158da
1418f12601231aaa6943d1ab5aa9778ad692cc43
1750 F20110331_AABAUE abukmail_a_Page_33.txt
ea454093750c92e5a5a3e99998dd3af2
13bf56339199f7f12b604b731af8801d2b577572
1975 F20110331_AABATQ abukmail_a_Page_13.txt
671eb1478450143247e05c785a66d087
a68efb05a6b5555adadafc23bfe568e7a143958e
4534 F20110331_AABAUF abukmail_a_Page_51.pro
98635b5fe594f7853629cb2eb8d1ff7f
639c6a13569e6f9fd270d21ab6b8afaa33fa3348
31667 F20110331_AABATR abukmail_a_Page_16.QC.jpg
acb6e81eeccfdf5173891d489ab62095
bf34d198a3b053de514eb4de338dde4e08c068bd
1818 F20110331_AABBAA abukmail_a_Page_18.txt
9a5ec0a8d77cb0e58395b76581fe1d46
19942bd34a5ca799ff705c34db102bd9160e1e2e
58973 F20110331_AABAUG abukmail_a_Page_65.jp2
ee426d004a6cf560a9768b51fcd017c5
610802b645fe3b48f678a683b99ed9f56017ff46
93857 F20110331_AABATS abukmail_a_Page_18.jpg
b9725151e9c10c180dc2636345358236
c17c36b702422a1603c56ffd69a34a795659e5d1
1637 F20110331_AABBAB abukmail_a_Page_19.txt
57d72a793dfacd7be2e8ec0f7ce263fa
ae350ae7d7b8c6f21cb8410a2a64128494060beb
95363 F20110331_AABAUH abukmail_a_Page_78.jp2
5cb70a31c084015c3904b38c29bc2e71
63452c76e39e9d5974996fc192db7d1046467f2d
89838 F20110331_AABATT abukmail_a_Page_76.jpg
9a2b3d6a58e982f2a36080509535c9a9
6e87f42e93f6ec662ffbc6ebebb61d01cc5f511b
1961 F20110331_AABBAC abukmail_a_Page_21.txt
c116f6bc9051eb887599dc0b7348e76c
9d69b50442a2c34d61a1d9f7430265abd02c55a6
26663 F20110331_AABAUI abukmail_a_Page_68.QC.jpg
d1933b11cd3367a71e50c2977e1aa790
95c9b4e5f78e3310b58aa78ed5ade6f236a5c469
1051980 F20110331_AABATU abukmail_a_Page_08.jp2
8e737329c9de3271bd2a3b1f2a6400a8
45e791055322934f25013c0eb370378e5b1e15ce
1815 F20110331_AABBAD abukmail_a_Page_22.txt
395e71324018b01c48cdd046fbd3b427
dda31177fa00ad4bb800a90e990c43892b274d9b
352 F20110331_AABAUJ abukmail_a_Page_07.txt
cf98915d5522d87765d75c11f88121cb
d71d32d014e0a2c106db7c82bb35e6a43c1b5419
1749 F20110331_AABATV abukmail_a_Page_10.txt
7a3fee255949437cdd5f50b1db928bc5
3ad154baaa220af2a98d7213a7f24f9b266bc070
1918 F20110331_AABBAE abukmail_a_Page_23.txt
7933fbbae8451e9dfec5ef8866f3969f
bba5c93fdf557b051a80cfaf224dfe39f539da35
542292 F20110331_AABAUK abukmail_a_Page_81.jp2
dd3ac2839bfdc6c2dc4c964c1ee153ad
f744296e8e5f9ec40caa6d41d3952f50927346d8
630 F20110331_AABATW abukmail_a_Page_04.txt
c1aaab51e12125b50f8465b7440a72e6
b8c238c668dcac46291170db9f14fdbbfdc010bf
1949 F20110331_AABBAF abukmail_a_Page_24.txt
7797a63e7bbcc0f28f2ea058bd8e2262
33057068964b61139e93ea1fa49dd5f31498f0d0
31510 F20110331_AABAUL abukmail_a_Page_47.QC.jpg
da586150ea7fea0f5bbc774f8466c269
cdfd9e07b4c03aa664f2c7db70f663765b0b9274
2129 F20110331_AABATX abukmail_a_Page_08.txt
bc4e73008908acb3bf35177ca200e934
07e96bf8f4369d13f97737bf78dfe16a7e49dadd
8353 F20110331_AABAVA abukmail_a_Page_57thm.jpg
b64979759fe5afc5d5bd2dcf87db9dde
c977a52a81211dc55ec4448af4bf8b7b9d67fd99
919962 F20110331_AABAUM abukmail_a_Page_44.jp2
d2e91e7912fc395edc335eff5684cda2
a2015bf3ebf84f3b5fff55bb0748e49bde104991
1957 F20110331_AABATY abukmail_a_Page_64.txt
a3e01cda5af49dc5464658227c639f91
94434311d506f131c613467245be377f7601e4d6
1836 F20110331_AABBAG abukmail_a_Page_25.txt
96a8279e35f0184209c48af3c45f986b
b4a81b4dfafca30e179c87f72d32a14229c07422
106724 F20110331_AABAUN abukmail_a_Page_42.jp2
dcbe8ba82d1cef32642815a29d013bf1
6d75c099d2f931b15ba532a6b44addefbdc7d5e0
6699 F20110331_AABATZ abukmail_a_Page_80thm.jpg
90c1a9f769fdff3c7a9f4eafdec0a2f7
0cfe776e526fc021570051c20596f6b8ea6f9ba3
1863 F20110331_AABBAH abukmail_a_Page_27.txt
304272d18b336dc47d03aac07e26ec3d
958357d72453ed7e213fb2827f8061885c27a3d4
F20110331_AABAVB abukmail_a_Page_86.tif
f474f7bf79ff06f76cfa815c17c4418c
a43ade9faa63b20cc75d4b6a1acbeae98fe434c7
F20110331_AABAUO abukmail_a_Page_39.tif
60e02a4ffce1275cddae90805ae84882
6c2dc3503641e79d5cadade9959e256178855ed1
1849 F20110331_AABBAI abukmail_a_Page_28.txt
779473c8b4de9e278e7cbba0055aa7b8
aae1e4984d41c76cd8f6ccfcecb8bcc2e0b4d622
8391 F20110331_AABAVC abukmail_a_Page_37thm.jpg
ddb823926a8c38a4613b55a3e6d20a10
bc0aa07d3d388aa538c0f2121b5ec4db07046c23
48894 F20110331_AABAUP abukmail_a_Page_90.jp2
bf4f07278edb01e56cb1cbdc6cb3c5de
39039151cccb8083f1376997843dc353925e4f43
2024 F20110331_AABBAJ abukmail_a_Page_30.txt
afd0431d372c03465bb5518a54a4fb21
9f5dac8907f88f3435f57bf8bdda64a5c351b611
39283 F20110331_AABAVD abukmail_a_Page_61.pro
8d939ffeea01b24a652650b9f1578366
75d545f74580002176d0ac9fb46867308926a3fb
104152 F20110331_AABAUQ abukmail_a_Page_16.jp2
434b473d9f2a77c08f2bd1cec6deec5d
d756fcb083912ecfcb1e49b238ada2201570b4fc
1920 F20110331_AABBAK abukmail_a_Page_31.txt
b3257f74f5d65068a95c51a2e7b583f9
36154afddcb9b43eda6379ce0defcd7c60307a61
F20110331_AABAVE abukmail_a_Page_77.tif
ce2eb101bee594dff4cb7bb7aa00b171
8d69348500bce9c3e7e83a3e7217071800898351
2001 F20110331_AABAUR abukmail_a_Page_79.txt
4d542e3221be2eeb71c4e8f0cb6a1fca
41cb163e9a744b4576abef6935d4a925c120ce15
1192 F20110331_AABBBA abukmail_a_Page_54.txt
ea7c9776a4f190582aeb7a3a1e1b0e60
e1d3fecd5e947e03810dfd124f1ab8d61280d6b3
757 F20110331_AABBAL abukmail_a_Page_32.txt
4bef95ccd73170e9ea75c163733c201c
cb6dab14d80f957d32b40ecc618cd7f4fd6680be
106172 F20110331_AABAVF abukmail_a_Page_64.jp2
158a596e7938fbdae62a2cf61d784f1c
68b941c84d7d8f846e66ab69e954820d90c2d5de
28270 F20110331_AABAUS abukmail_a_Page_65.pro
5504bc3d99d0dcf70f1bce563e20b425
bf85042d0af85e271413f8dea4ebc966544b6b46
2119 F20110331_AABBBB abukmail_a_Page_55.txt
02690e466c64f57b4ca11b6f86d4538b
ed86e68681539be9ea97a0b5d166a7d2ac5faabe
1982 F20110331_AABBAM abukmail_a_Page_34.txt
4f3674c1a1eeb5ca122c86683ce40f96
708a4a05d62a535331223134bbdd91172b3b8a7f
7673 F20110331_AABAVG abukmail_a_Page_86.QC.jpg
3c93ae22638f3d3be7fdb24d6fa18136
6b05a398229bbe4ac29d57a0584a225256431dd7
7251 F20110331_AABAUT abukmail_a_Page_17thm.jpg
d15248f8d4b70aef1798024f581c3912
5f5b41d7f96087ca7f7e712fa8f1f49e6b4e6af1
1877 F20110331_AABBBC abukmail_a_Page_56.txt
c95e24a54eb1a908a63e79da0cbb4786
5f2fbc14306463983854fbeab60ec5c6f77db979
2003 F20110331_AABBAN abukmail_a_Page_37.txt
0417fe439d94c9142cb2df916bf5628f
5d2eb9065b54006b798a399a522cc9f0611660a6
1129 F20110331_AABAVH abukmail_a_Page_09.txt
99a6067662aae3b20281df7ea68bd468
6d34eabcc945fe6e97dde4be46c5cbb5dfb95bb4
58359 F20110331_AABAUU abukmail_a_Page_88.pro
8ec262720445070581e8ae86d53b194b
7e39e00eb975620c025e7a7d04d970449c5030af
1586 F20110331_AABBBD abukmail_a_Page_58.txt
381933d6b07cd7c604109f1b49a9abfd
615f1a30048b6b3a1f0e7df73a915f4f8d23563f
689 F20110331_AABBAO abukmail_a_Page_38.txt
094bcea75f245adf47cee058d21e9e5f
c87d7c6acd212ca4696d0cfe45eca7a9a578afb7
30684 F20110331_AABAVI abukmail_a_Page_27.QC.jpg
b3ba051d619a91543f12096644ce6519
1ba24a62a6b352c413f9afd5e72e7b53dc871cb7
F20110331_AABAUV abukmail_a_Page_51.tif
860dfe05d2e6ff74d270070f66e60a68
4cb192924bef88a84e2c8d9fd9d56374be68bf06
1446 F20110331_AABBBE abukmail_a_Page_59.txt
06f6e99b57388ed33345bb27e2fe8479
b301ab4e39dda32d630ad59d82bf2fa303c1473f
1812 F20110331_AABBAP abukmail_a_Page_39.txt
aa4907c9309478a8be0bcfa9fac82f68
a1e2d2b12c34a27c8f0e1b7cb1f3a0431696fa7e
34728 F20110331_AABAVJ abukmail_a_Page_41.QC.jpg
ae891aead3e159bc608a5c4f6c87bb2c
f815a97442047a94243e42eb7c5140890e0a2c82
1971 F20110331_AABAUW abukmail_a_Page_36.txt
344948b142f3a9d6c6a72e215194e13a
4779b96e9c7eed65939492c815718ac495c96d5f
1857 F20110331_AABBBF abukmail_a_Page_60.txt
0c4cf4806f54e8bee0979aaf058cb63f
992eb8c160455c7318b2ab0f5d549aa8fd0af27c
1992 F20110331_AABBAQ abukmail_a_Page_40.txt
cc145e5abc68dc000a7b5f3b3e55dd89
e5eaf95964eb5ecc2e2e9af8f057e3f3ba6ea19c
23299 F20110331_AABAVK abukmail_a_Page_76.pro
b913dfbc5b7974f8ce8a0406f2b155e2
3d20c256006f6faefc45e83d5dc4c0aa8bfe4243
F20110331_AABAUX abukmail_a_Page_22.tif
916963fba8499e5ebb32c397b435aa56
c2c333bdce9d402948cdfd261fa9064cbf6c18f9
1583 F20110331_AABBBG abukmail_a_Page_61.txt
385b38b7db73db33d3fa19ccaa619525
6ec2d7f72c60c2b8f66d52590fcaad32a3f61307
2145 F20110331_AABBAR abukmail_a_Page_41.txt
1035c3f256486b1e1e377d155b44cbcb
5cfef881cd587f6112e616be174101e991a100a3
93554 F20110331_AABAVL abukmail_a_Page_77.jp2
42724a930eff2da1b7deaadf8a65ba45
a12dc77f23ae72b561cb0728e6213b9cc6c9e9b9
8302 F20110331_AABAUY abukmail_a_Page_55thm.jpg
0fdb9cbc586f0da61fd135c4cb1b105c
29979199b65909629d895d11e4f855a9e45e48c2
109384 F20110331_AABAWA abukmail_a_Page_34.jp2
fc83b79706f18c31eb40275285a1a1f4
799b82b581f6c37a96142b79c92caf4cbc0c8e7f
1940 F20110331_AABBAS abukmail_a_Page_42.txt
53d672c46d22dd333bab2e0261db80ba
b49e65f6a874ee43e5401ac71795bf4df8adcd9e
63100 F20110331_AABAVM abukmail_a_Page_11.jpg
e71f506e01558740356bef3ebbb43829
850b05770daec4f71f59e70320d74a6a6089d2cc
22962 F20110331_AABAUZ abukmail_a_Page_69.pro
45efade0aa94fcaba13503cc50d4e78a
9c997023864bca30c9d20025cfae1acccb43b7be
1662 F20110331_AABBBH abukmail_a_Page_62.txt
9eae2deb2330134786e385279e173d48
4b2d7d3cc443fa5aaf8de5643e5503d18393db8e
107768 F20110331_AABAWB abukmail_a_Page_47.jp2
d54a46f0299d06af628814b53dc8afac
0dfb45973963223549f908ba5418ce8b0bc396c5
1950 F20110331_AABBAT abukmail_a_Page_46.txt
debc08cf9dd2693bd071efc58a28dba2
1b58629b507abbc1c63d6528d80d137683b28599
45169 F20110331_AABAVN abukmail_a_Page_20.pro
696ff99e7eeec92b2d133c044f7c18ec
931a932fe3223724b845a4563184bec288c0448a
1846 F20110331_AABBBI abukmail_a_Page_63.txt
9b9000f0256f7a2e66ed7b0365747cc4
8ec3df665d5a5bbc63ecbb0427204ccf40a8c8d0
6666 F20110331_AABAWC abukmail_a_Page_07.pro
43c398a4c096efeb63586c2e8eb2cc3e
2963308cbd4e0b24f4c0474fe3a4dbf727a68ee6
1923 F20110331_AABBAU abukmail_a_Page_47.txt
1b31f0a60eb65d13599f7ccd6ebfe601
b31d3b9a0a775aa84e84d51747d0efc2605ca7fe
107026 F20110331_AABAVO abukmail_a_Page_40.jp2
b01f936d54a2b84ac6e3d5483a784c90
1b3cf5246a77f2817478e527879390d6860faf03
1178 F20110331_AABBBJ abukmail_a_Page_65.txt
803dfd5c549384422699bacb2521c5bf
8648464428f42001d26b86216b7acc43594d301e
47967 F20110331_AABAWD abukmail_a_Page_23.pro
bc84aff35afc32f2f2eb2d98b7519a47
e866d6171b6b0c080f9c25008994d11ee9231d63
986 F20110331_AABBAV abukmail_a_Page_48.txt
10e762e2a97fc5b3a30c6010084540a6
fcdfbfc0c16a270bf048540972373edc1203d87a
109730 F20110331_AABAVP abukmail_a_Page_31.jp2
f7d48f4b7f5a970801c86e5b35114ced
22be12cf7a6bdb3b9f74dc814e37eae140a96196
2022 F20110331_AABBBK abukmail_a_Page_66.txt
55cfb688bc1d0d0dee37105de1dc5cf9
f7e0c1f1e4e476d0242ad03fd50e170afe5e3654
1067 F20110331_AABAWE abukmail_a_Page_69.txt
5aaaa4231ad605e037ea717f82bce97c
e6c875035e1bebdb0a626efd5cd6d0170842f380
1226 F20110331_AABBAW abukmail_a_Page_49.txt
7037ec677850864a60a2350d2d793719
d0d9c3deb6f365c66e5b92603c658bd3edebddc9
F20110331_AABAVQ abukmail_a_Page_56.tif
3a55eb99a3b0415f691384976ef553fb
c4d9d91843f04ea4006b816d68ccf6a607b3fa5f
1715 F20110331_AABBBL abukmail_a_Page_67.txt
5fe51a7f17ff7ebc2ca94ed7eb8917f5
ac7c9d7c405f7f593a8479f9b60226b9bdafd286
F20110331_AABAWF abukmail_a_Page_88.tif
e75de1cb3b121142b07951aa05398504
06f3effeb945e549cb22ddec46870993d2a90890
1850 F20110331_AABBAX abukmail_a_Page_50.txt
188cc4b9bb11901dade120ad927a0c3f
a7d07389290e14fc32b17f47afe33f107ccff54b
1916 F20110331_AABAVR abukmail_a_Page_26.txt
4600402d65780f0e6d937f27ce44a754
c406b7e62f4f3ee7c02efcccde9140d533d9e686
2427 F20110331_AABBCA abukmail_a_Page_88.txt
29c3f4b7cc7430fa78227de3ffccb23d
ee2c34023548bc28c8f777bb160d03e646306128
1236 F20110331_AABBBM abukmail_a_Page_68.txt
a273b9c839652cfcc9c994cd6849cf6a
0fa370daf8db2c27719d4e44a3d7ecb56445e96c
8258 F20110331_AABAWG abukmail_a_Page_36thm.jpg
a7cd22bd25ec84e6bb467c10d9647112
ced84de1b9dad8d20225ca45e5b02db5beea877f
180 F20110331_AABBAY abukmail_a_Page_51.txt
98041b7601e108923c8ff09dc7b7599d
642098885c4ce7338e931de73774186bf027285e
25405 F20110331_AABAVS abukmail_a_Page_35.QC.jpg
4dff9824f95107905d870df0cd1ea0e6
3a466b732ccdfcd587cb49624e4899d9ffd1bc80
2684 F20110331_AABBCB abukmail_a_Page_89.txt
d8d440608f930d14b4dc8fc43272593a
1d0e1451a9c42ba3fb3ae82e51bd928528799eb4
1173 F20110331_AABBBN abukmail_a_Page_70.txt
42a1ca1a2675d267c19e29b3dcf8961c
6ae16954ec127f437502e8fff37d0d3e3de6bf83
7426 F20110331_AABAWH abukmail_a_Page_60thm.jpg
ae0102f6d07e2035e533733696537a3f
d482f51ec6187c56f06ec19e1565c15b30bfdf79
1997 F20110331_AABBAZ abukmail_a_Page_53.txt
2d5596268f595a2a37accc9715b213e1
35f190ec42c57ab282a1a039df06979e978ddf65
3351 F20110331_AABAVT abukmail_a_Page_90thm.jpg
8cb15b5fc495265a45ac189c37bede56
81d4d020bcea5c600f41d651de79997e7263d14e
794 F20110331_AABBCC abukmail_a_Page_91.txt
520b8df5286d9191727d00e2879717fa
2b52ae37fb6f73df0284288b0bad820b8c13fe34
935 F20110331_AABBBO abukmail_a_Page_71.txt
743ba328828b1e89677f85b674c1df44
b6a8b403749e12b0443d95705325a9cfe164f543
6644 F20110331_AABAWI abukmail_a_Page_10thm.jpg
ab1c411b95ab0abe41037b6f7f4d5970
7588a839e9ead2e67ff7653927f1b9104f3433a0
92109 F20110331_AABAVU abukmail_a_Page_43.jp2
c298a15940bcb3ac337a7361ad2400e3
532bbee346e0b49b1f2991c2becb2d25f137d46c
7624 F20110331_AABBCD abukmail_a_Page_01.pro
fe3f1b00c7c7eb359e33168dd0f95c99
be7bb7d822a2d1bb9d33eb880cefc0389d824e6f
1976 F20110331_AABBBP abukmail_a_Page_72.txt
e0e4e519ffddc75532f95a723ca2d2ec
1ba50507323e74e2d2edbad6d7355f39030e6942
106276 F20110331_AABAWJ abukmail_a_Page_53.jp2
9fc1e9b1e621e4da1519fcc7840a552b
fa63bc0509f12546d2dc36f508f2fdfdfec63ad6
941 F20110331_AABAVV abukmail_a_Page_90.txt
23f91b7ec258565428d757dc13a75521
415038f057582ed398f7005e7c6c19bcc83163c7
1279 F20110331_AABBCE abukmail_a_Page_02.pro
ff1eb9665e722bdf17e7fe783a58aa2f
30828e476b915cf6e8fd084b3acdcb68821159de
1264 F20110331_AABBBQ abukmail_a_Page_73.txt
2f280e285c524386232bc0641ae3c6c8
1b832823a391884eb7e3f9a5429bb97cb372014e
F20110331_AABAWK abukmail_a_Page_28.tif
b5aeaf8d10efd3d6256a0a7aed39389e
76015f7999bfe4acd108d76ec1fcd5c8fb827037
74120 F20110331_AABAVW abukmail_a_Page_59.jpg
6a5f67f763618b5f3ba85999411e95dc
455bd75df87ebeb9d8f2bb57af9984bda6a33c40
2107 F20110331_AABBCF abukmail_a_Page_03.pro
0041f08c07514dcee99313491d29f17f
931dea29ea13d3c04b560b0a21c43089d0e9b552
1476 F20110331_AABBBR abukmail_a_Page_74.txt
fe8ab95bd37c0aa77c2b2a9f929dad87
15b51f212712e438276325e4be631287b607d4e1
95287 F20110331_AABAWL abukmail_a_Page_15.jp2
847ef114291d6bc4a61b9cf5126f70be
143326b9b4364ce34a8b7ca2841a74cc81481274
22041 F20110331_AABAVX abukmail_a_Page_81.pro
cc442ef7de78f27d649722fca5f4276a
ec4977d1b8448fb70786a1e9d599a7fc4ddc87a1
14484 F20110331_AABBCG abukmail_a_Page_04.pro
199046fccd12bd427b2d054937a70266
fce6e45847b8c61e3f1480c52f846b0a0c4a5359
F20110331_AABAXA abukmail_a_Page_06.tif
29587aee67e3e882fc70567b6bb0b435
3a13d11fb41420a767afdb484bbe2755be649319
1267 F20110331_AABBBS abukmail_a_Page_75.txt
9c2e282cc1c36aa9b4282fa75c5de88d
46d51a97b79a614da69daa2b0b1fde0b40914b9a
99581 F20110331_AABAWM abukmail_a_Page_42.jpg
7098869724dd5a1fd3a37212f9493f5c
16cf2a94b159c413dfbc9e98ec9be6c205792186
99154 F20110331_AABAVY abukmail_a_Page_22.jp2
c18689f059e42fdb8bddb17fec55ccc1
7fc7d675296781d93b7634fda9304d67d49ce111
74625 F20110331_AABBCH abukmail_a_Page_05.pro
1525cb28c68305b6d381c4c08a8f7c5f
45a1f8c0ee6e8009fa120ae70eb9ccde9faee091
F20110331_AABAXB abukmail_a_Page_07.tif
dd40b89c6be475ab9232b6ce464d3b48
6eb5ccea0675f30d9cc410d19ca816c251839e78
F20110331_AABBBT abukmail_a_Page_78.txt
e85116105181c3878f4ac2846bfec574
a956be2e2706c92e05043339d9670b4763ed086b
93445 F20110331_AABAWN abukmail_a_Page_63.jpg
cf7a6206739a73c4336f894e5768f362
f4333c009a07e608a72a261c6705f6b301220ab9
9121 F20110331_AABAVZ abukmail_a_Page_89thm.jpg
12efa5c670e20a4e16bdd189a93f0a56
6a9fef4c29dc7a0c5400cdf77d5025beb0804e41
F20110331_AABAXC abukmail_a_Page_08.tif
2303d8465120d58871cb07ed384e27ec
9707fc4bd30507bef7581c5df7ae7b7ceb26e681
1513 F20110331_AABBBU abukmail_a_Page_80.txt
a747c3b5b7152acc1984cb2924d99983
53c88f18d6b7340b9281caa1a6289295ca8db5d5
7446 F20110331_AABAWO abukmail_a_Page_83thm.jpg
f68e5525cf507f42be923cd1820709ac
2c0a4fd9dd4586f6d81dab51f01d51dcba85f699
82294 F20110331_AABBCI abukmail_a_Page_06.pro
824c6bbd3ece671dd85724f3c33d31dd
2e0f77e4abdf7eb5e49192d6a978b376a381a728
F20110331_AABAXD abukmail_a_Page_09.tif
db2b074f9692136250f8e7175d8a7dc9
e11b1b228768bc449d05dfef88f1ea1653ebfb14
997 F20110331_AABBBV abukmail_a_Page_81.txt
a9d853f9d4def171aa370297ece690a6
94a52439e19e3cb50a8368aef8987a59b654b01f
2570 F20110331_AABAWP abukmail_a_Page_44.txt
e2bc3b175d41d0d3f561d2e20f0ca36d
742d2c941908cc9d6b1caa7ad49c1b1af8338add
52118 F20110331_AABBCJ abukmail_a_Page_08.pro
d16d395d0def9c00118c8dc79fe65701
0e9be37440b694f409bf900a55d9f96d27d03898
F20110331_AABAXE abukmail_a_Page_10.tif
be2bfd5200d99883d3d975ef81e80aad
e6aebd71dfe7d48e403f5fd611375d8526dea15a
598 F20110331_AABBBW abukmail_a_Page_82.txt
88ca2630199abaea07e13c0bdd94bf07
7e4bc6bee62a371cd7ffdcbe1747fca1b6a55c2f
50330 F20110331_AABAWQ abukmail_a_Page_13.pro
500cb4a0030d6648ec16438babc72a54
67e5c5335387da595cc74498a7ef48e5e1700c21
28132 F20110331_AABBCK abukmail_a_Page_09.pro
ea48c3448407b4ab59863c28c72f1d8f
fee0dc60b2dd94178e604ac84b90de7e8f540173
F20110331_AABAXF abukmail_a_Page_12.tif
eee79e7c3265ddbb860a2ed12f63cc87
8b4c8f5e1399b16ac5763b1655ffd022182ee6a9
1952 F20110331_AABBBX abukmail_a_Page_84.txt
d425cfede207d39bc6353ca8c65eae1a
5cd446b9a2684b56dafdbea9979c03bffd74590a
1709 F20110331_AABAWR abukmail_a_Page_02.QC.jpg
a72792d1e4b992000bee1fc1ca846c8c
9fa525aea8d711294bdd6b62b0c5f29484057a9f
47236 F20110331_AABBDA abukmail_a_Page_29.pro
ac17ea7f8aba8630a37391d6e239e65b
cd7cd16a83ee0d9561712db7b35b2a93d98dbdb9
40305 F20110331_AABBCL abukmail_a_Page_10.pro
2c6e3df9d13520ad14edda1d04b879b2
a92195e68675a9a5d8262329afcdc6c45d7bfd22
F20110331_AABAXG abukmail_a_Page_13.tif
b826aea14ee0c1f91d957f01053db184
9995b610abc6a6a2f905b2e478b852d0997ebeac
446 F20110331_AABBBY abukmail_a_Page_86.txt
ba06bc4d39465354ae09c9628b5a02ee
7e680452958e7d4ba75bdaf84c1e66fe48375ff5
29431 F20110331_AABAWS abukmail_a_Page_77.QC.jpg
c23dd638e4edd39301bef1e4fd41d65e
02418abe058fca436e624ba4c98e0eb2b2c8f047
50708 F20110331_AABBDB abukmail_a_Page_30.pro
580b3c1506f3bceb87e607cf933f3dd3
e1599f0bac2bc7abdc38c1e5660567e9e0e61127
30357 F20110331_AABBCM abukmail_a_Page_11.pro
5460f32d68425ba4d7d4990fdffcf741
282b9cecf6a4f689986baf8262dcbf87ce33077d
F20110331_AABAXH abukmail_a_Page_14.tif
87350004958d9cb34245658c92d2c351
0e8368dd0f406dc6b45df704887446d8538477b8
2137 F20110331_AABBBZ abukmail_a_Page_87.txt
7626b1fc12d036ede28b8703e9a6e065
79a73778c9a079d8a31dc20178812797f3f2c26f
90497 F20110331_AABAWT abukmail_a_Page_50.jpg
ff899e54474c98acfe15183159b2432a
dcdbf015b08b2960aa2dd81e2161fe77bbeeb8d9
18859 F20110331_AABBDC abukmail_a_Page_32.pro
c9e90fe50d755fe416cb69032ff6c697
900629ded0f7cdb34d816cf0500fe3b3eb63c55c
41389 F20110331_AABBCN abukmail_a_Page_12.pro
2d050226b764fb50cc75898cdae56f2b
84c841a7d6d00d7bcd8f9fdcfc1cc973101a9ec6
F20110331_AABAXI abukmail_a_Page_15.tif
7d5c26381cf714a4c007f44590483435
ccd92fe883941782171c22e2d0dadb9a71b7b6ce
148388 F20110331_AABAWU UFE0013060_00001.xml FULL
9b9d4c026b604b69f801f0221bdeca37
d759211a03b663e7392c87bd674576f2096bd487
BROKEN_LINK
abukmail_a_Page_01.tif
42666 F20110331_AABBDD abukmail_a_Page_33.pro
fb4f5b76f6ac3c5d24f3686a1f2782c0
cf24ad4077c6d9411bdfe003ae279a68b2ddf830
50009 F20110331_AABBCO abukmail_a_Page_14.pro
4f8290c4be36a5ac788d5e27cacaf80a
4a36ca9d630b2683d6d62301289f9e03c167f726
F20110331_AABAXJ abukmail_a_Page_16.tif
a561cf5a8252a1499354fcfcecc7fba5
edb374f82520ed44bbd8fe92c6378ff87059fed5
28209 F20110331_AABBDE abukmail_a_Page_35.pro
4685c55c08c2e2ddeb3cb7293bb3e48b
19ed28f153aed50f2ea65ab92a579dba5594d384
42153 F20110331_AABBCP abukmail_a_Page_15.pro
595770892939a9cecd9937aab7c18bbc
0270cb6a226f73321c9bd459d215cb9cb7da78e1
F20110331_AABAXK abukmail_a_Page_17.tif
db46d0d75a348e4c8cccf46c4a1e28d5
7ab95689a3115c71a407129c68d76461fca86f1b
50050 F20110331_AABBDF abukmail_a_Page_36.pro
177bca1fe21de586f7a240561412ef13
2be46b5f6b375a9d7012143efc2b2497944c5b40
47856 F20110331_AABBCQ abukmail_a_Page_16.pro
da6c93b5fd9e4854d0e5baa6784ef993
a5a5d746e6814fd047b2546f136fbf7e1088b1c0
F20110331_AABAXL abukmail_a_Page_18.tif
dab5825ee2b15c11ee76c2820c5d50f9
50b54b33d4b787d757987665076a46f17fe05e63
F20110331_AABAWX abukmail_a_Page_02.tif
e1d8f85c81639f280d587976d31782e2
f7de2a04328852c46bfbb50e940967a9379b886e
50868 F20110331_AABBDG abukmail_a_Page_37.pro
3470d47cd861d4b203e9a0ff9978358f
415f12fd024c09f6e433b72242a50dfcbf73772b
42579 F20110331_AABBCR abukmail_a_Page_17.pro
c77fead89b92db7b131addf7fb5f17c4
6b383585803f0ce85079a4c5ef941d61ee579f1b
F20110331_AABAXM abukmail_a_Page_19.tif
8d3269f838cf2f9548e853e4d75e10e7
069a7e7a9d668c2c948d8c84585a8339e5a5bc49
F20110331_AABAWY abukmail_a_Page_03.tif
28cdb58560195bb43fdfde6e51523f98
82672822d78c858675879657469310327a2de219
17318 F20110331_AABBDH abukmail_a_Page_38.pro
6ac28b06e4c22b9a9eb21b24457ba530
f6105b0555846f644c89695c4d947561f16323c0
F20110331_AABAYA abukmail_a_Page_40.tif
9f9b28a3170e803bf03aa75571761a86
59a17b2400e6365cdcc02a47f9c15d81267e01f8
45774 F20110331_AABBCS abukmail_a_Page_18.pro
d67ebcc1ece6fa5b3be9f710d39540f5
1592be225b044816c67a1c9d484251df90dda2c8
F20110331_AABAXN abukmail_a_Page_21.tif
adbea5b18c40d7e51684ff51ed7acb58
2f81a84365a76ad66a31d72d45d19e1d3054ead3
F20110331_AABAWZ abukmail_a_Page_05.tif
f19709fcf143fd106a84b07fbac48999
b07937eb8b209de87b101acc7b6cc455eed5045b
43479 F20110331_AABBDI abukmail_a_Page_39.pro
3e701ddb6272b28b0971b57e2f583bf0
214429056ec7a2ca5246c353d0fe671c1aa1bd21
F20110331_AABAYB abukmail_a_Page_41.tif
d013ca5af8513e7b2eb2e113aa7a5533
133427a9277056b7f10c2a73bec1fe147ce63c16
41005 F20110331_AABBCT abukmail_a_Page_19.pro
e6adf061e512e501b0b5dfed87260fe2
36866bbce59e1c474fa8dfb6133b111ffc3ee677
F20110331_AABAXO abukmail_a_Page_24.tif
9cabb167a1b3489f3112857b207168ea
c3f3e441d097cfb27c0b7016d49a3f6e7c2d9f08
F20110331_AABAYC abukmail_a_Page_42.tif
6d35ea73fcf5b558e2f43c811a652988
cb634cc26f2232f0e82df55f4ac6160a6802b03a
45683 F20110331_AABBCU abukmail_a_Page_22.pro
e236d5375d3ff316168eaa88d5ec24ec
335d56c35cc1e1a056dcaa6bf15620a26797b108
F20110331_AABAXP abukmail_a_Page_26.tif
08fe32082fb622df3e18610d1bd0aef7
d21f26296ab4d2cd1c9b3de8f0de163e60b80ccc
50646 F20110331_AABBDJ abukmail_a_Page_40.pro
c282be9fda66a368862069bbb5101c54
89cfc7d6cabf704f66e8a6ad2bce4fc7398e9e68
F20110331_AABAYD abukmail_a_Page_43.tif
4f34e463c8cca79d8c8f69913a234615
f642ef4280141ffe132dcbeae4ce692f4c1b5114
49457 F20110331_AABBCV abukmail_a_Page_24.pro
6b084b20443e767c929975f9c9784631
003d75b3d77bb5ee8bfc60363f42dfcd77fe087f
F20110331_AABAXQ abukmail_a_Page_27.tif
21125ed2f6070e19f29fbb9f58663966
0df2f49c400ffd1c85811bca7f848afdf4ccf238
53868 F20110331_AABBDK abukmail_a_Page_41.pro
7a14d32573a6a4ee4730b485a2bce3cb
7ad9444d59bb9564364c8cd394263f7b84c7c116
F20110331_AABAYE abukmail_a_Page_44.tif
8f99bae3c26117b36e2654eb1462d9e6
909d597b69af487f7280d17606dc4ed79be0da28
46156 F20110331_AABBCW abukmail_a_Page_25.pro
074d2847fec48d11e73fb2086f7d6b4f
2ab70ba8035e03618f5944f81fd4a4c285019504
F20110331_AABAXR abukmail_a_Page_29.tif
bd1f4f9185f36a35f07bd40cbc05a3ae
f60bdfa600eabc1d3c6a27bfba7057cd3db83b9e
44411 F20110331_AABBEA abukmail_a_Page_60.pro
1b58dff97632d84d0210b0eea049c3bb
ccafd6a12a28aff234ce4c004d8dea404270e402
48077 F20110331_AABBDL abukmail_a_Page_42.pro
083be57b7319b46b3c2e765f7eaa6361
337e22ebb010011ca5cf23d2fc20b1d6040e06e6
F20110331_AABAYF abukmail_a_Page_45.tif
69e06da5dc6e5231522642c88e974a5c
9ce46005f5b8ad571bd49a1f297de95447384b79
48555 F20110331_AABBCX abukmail_a_Page_26.pro
43f2ae36e809fc004fdf3c2f3b6fc070
c4e5d659aaa5bcbba472344893a71015755d6e54
F20110331_AABAXS abukmail_a_Page_30.tif
4c6b976d14233c82c15e555fcc01a478
3f172400db66c748a3b175abcdb593a27ffe8ef0
39033 F20110331_AABBEB abukmail_a_Page_62.pro
db30ebace6c4d7049c03d0ba07583bf2
2633a9ab7041c914b79ea9862c78ae3770903e1b
42300 F20110331_AABBDM abukmail_a_Page_43.pro
510af638be4a0936e0b4dcda3ceb47c8
b214c5f2bf6de170dadc9e4501542be14d865be4
F20110331_AABAYG abukmail_a_Page_46.tif
3e21e518b57e7298b32d08b1324af1e1
fb97c68564526348bf14869aea7e03f69b99266a
46639 F20110331_AABBCY abukmail_a_Page_27.pro
28370ac8c0f7d563b8ce95c7b8be45cc
5dc14b0168f65e3eaac8c68f5df22eee813101fa
F20110331_AABAXT abukmail_a_Page_31.tif
28f0117b3f96aadce74fa7a559d2e158
61d7eeec7b5eaa8404e770f2c5e55b26880df621
50569 F20110331_AABBEC abukmail_a_Page_66.pro
3ac91d64f2463dec65a622cd1b51b799
a4eda0e4c1fc0d5a1e02b6f3fde3e43b4f09d274
44323 F20110331_AABBDN abukmail_a_Page_44.pro
c29c1c0faba748dd7d6a3bb6d1357223
1f7a76d72e461a1e9684d0067ec7a00ef20795f7
F20110331_AABAYH abukmail_a_Page_47.tif
8479dacba002e3a708d997353ffbe4fb
d9d3c60cff2ed70e580623ccc97ddad62745f959
46313 F20110331_AABBCZ abukmail_a_Page_28.pro
dc93c3a2df24ea5492037a3c60840229
187366b860713b17ee5737990513c8b6867048f6
F20110331_AABAXU abukmail_a_Page_32.tif
7d92c914c80dbc4295ed9347338b3733
5eff2168ec30c011613377b5f6636ab6ff98e0cb
41885 F20110331_AABBED abukmail_a_Page_67.pro
4c63594f0501c2e80fda8bfd69653887
37bfbba9f1803dba2e1a952143f9943a335d5e1f
16212 F20110331_AABBDO abukmail_a_Page_45.pro
eaca1154ef48ea7f04fe13a98a6898b8
2d8eeec92bbc15a5a427321dde5024939b241bbc
F20110331_AABAYI abukmail_a_Page_48.tif
3d9e7e717fe20a21a2feec8afee079e8
d0fc34f6f842e210336ad68fc88552694ce3f759
F20110331_AABAXV abukmail_a_Page_33.tif
437e16a4071bfc2faf2028f0e316c576
212dcbee9815af02ece8bae5b9b543ae5268a03a
24218 F20110331_AABBEE abukmail_a_Page_68.pro
f1f052c72b7c7ee9a322b3916b283ae3
0c689a903159e7b92f1809a20ecaddbbf07dae20
48513 F20110331_AABBDP abukmail_a_Page_46.pro
e2f2ed014c530e384a0a8008233571b9
f75b61bf0b6c3cb00b7e09b096e97cf00c2fa02f
F20110331_AABAYJ abukmail_a_Page_49.tif
89e2c607d2a526a4210eb0eb94b68bed
da633021e8ce01c6c98475b5e6dd2c81e68bab71
F20110331_AABAXW abukmail_a_Page_34.tif
0ce590d31022d065e8cc20e83d38fd28
e563298f7c91e5ca55658f2b75c5000c6a1b04f1
26546 F20110331_AABBEF abukmail_a_Page_70.pro
3237a34c5841664e8d35e080decc0688
4cb06500b9e8ee3a71e8b282709e2e1ee581aa2b
48549 F20110331_AABBDQ abukmail_a_Page_47.pro
29e92fbb8a03d999af5542e5851252cc
b9753de7c7d74ca7633fbe8b29f2f4d062fe1d6d
F20110331_AABAYK abukmail_a_Page_50.tif
4edc6c67a58a0f8a5dc8f15d2b0762fd
b7f2f77b3811f4b18e8ffaa271bfe7588fa19e4b
F20110331_AABAXX abukmail_a_Page_35.tif
5601ac2aa099dc3952fe12908384580a
4483dbf6ceb22e5c75e84554d7a3ad39331c2a64
22831 F20110331_AABBEG abukmail_a_Page_71.pro
b4703739d0ae4c08d317d8816f6cf220
5d0d4443a6fc92c432f23c259a7e86b9f95a6e65
21960 F20110331_AABBDR abukmail_a_Page_48.pro
06050c1ca26a4dc9badf6ad4121abf66
a658ff2715ea7a9722929679989d0a8a3ba4c4ff
F20110331_AABAYL abukmail_a_Page_53.tif
807f26eab9557744c9d3af9ba1c3ed84
0ff320b4108f360acd8e96291c08a288ed11bd13
F20110331_AABAXY abukmail_a_Page_36.tif
46ee744ba0edbceffff2740a59a9f14c
ea94f15cf059ff06bf4233665adc3ed5b0c81915
49177 F20110331_AABBEH abukmail_a_Page_72.pro
0723f7fc53e6d2bc8c28d254b365fc17
19062b6faae36ef76f52a3d7d5d778a99ea631fc
F20110331_AABAZA abukmail_a_Page_72.tif
72411ff22da636c0465c2277b0d3adaf
2258b6cf225da73b5ec3d032a4369fe48ec299bc
25224 F20110331_AABBDS abukmail_a_Page_49.pro
78d658f1630c8570efbc182f878285c4
e6e8cdbfa116c19228ea1a9af9a3bfc71d423c0d
F20110331_AABAYM abukmail_a_Page_54.tif
7b86503974179c3acec52747f4d28643
ad2026a2cd069261054fe5a879f67f98c1df06d4
F20110331_AABAXZ abukmail_a_Page_37.tif
63a32f6b1bb10f0063247147c0b06a07
cb1176369ee822ad1dedfa6dedddd5153c8a4bbd
24112 F20110331_AABBEI abukmail_a_Page_73.pro
8b22c4cc885cf9735b07f52db47daf7a
180073259af2a9bb211c77cb91234b8142801487
F20110331_AABAZB abukmail_a_Page_73.tif
49610c896056b3881f9eb0f6d9cf3487
1f1b1781862122ff53be9cfb17a10e1413563e3c
44257 F20110331_AABBDT abukmail_a_Page_50.pro
0ee70a38339119b4a2496c20ac8e7600
7ceac28ca466738605b58337b542f91571efc932
F20110331_AABAYN abukmail_a_Page_55.tif
e30cb536ec67a568cd584e95b8ea59d9
2e8b3e9a4b6e3816826ecc97d62485dfb27cdd65
24362 F20110331_AABBEJ abukmail_a_Page_75.pro
5e1fc03e8855698c089e34a866dd4be6
91c71e2f03ed5406aef24646d4637e366ee688db
F20110331_AABAZC abukmail_a_Page_74.tif
81db6fcfd50749955d4df23496c54428
0df3a1a9384770f5a1741244337f20cf8a63e153
50745 F20110331_AABBDU abukmail_a_Page_53.pro
9f4b7b7f53fb4c5891fffb6a69a14816
5eccb4f05efdf30d58b70a640f2928bcf6dada58
F20110331_AABAYO abukmail_a_Page_57.tif
94b052ab729522bf7c203c132d7b4d56
985972a55d0e988f7de13c7d19fb786cd4e3747e
F20110331_AABAZD abukmail_a_Page_75.tif
9f007ee12831a31cdd7c80d9efc2afee
21d2fe4ec05d79c9efbd1450f14fce1190a66ef5
31534 F20110331_AABBDV abukmail_a_Page_54.pro
98d29ca07f1a2ecb801cb96461b8575c
86de235a9fdeb8981831d2b1116262ac5a4aca1b
F20110331_AABAYP abukmail_a_Page_58.tif
27e32b17d5ed869245159d1b36956f07
debcb2ea2d224dd6a7749f8a743bc267a3fd9e4f
40587 F20110331_AABBEK abukmail_a_Page_77.pro
d077712adcf5ac11ff0b54e34f01a03d
1bbd141f241058594c058e1c3a922593ad024f95
F20110331_AABAZE abukmail_a_Page_76.tif
40fddbce9b35e245b09a8d1087bdb25f
1fd719e03b650daebf9d0bada8ca4d3c92527a72
53216 F20110331_AABBDW abukmail_a_Page_55.pro
e6cbe2ddad8c28732b48f0ba13614005
4d3d1e13d32dacb21feb00758adcc38d320127a3
F20110331_AABAYQ abukmail_a_Page_59.tif
2e5335ecde91d62c19fae3ec3079d4d8
dbe68e24f574fddacc71f320e7e5af014c983d2a
43834 F20110331_AABBEL abukmail_a_Page_78.pro
a1c0069ab5a1f2f11d02f1755fe652a8
a15c8c0ac3e955da7c9dc0603b5b97ae92171372
F20110331_AABAZF abukmail_a_Page_78.tif
e6ed4d22edb85327dc6b81397d88b1d2
991849880099310f8cb930cba7d6f025a99e48ac
46546 F20110331_AABBDX abukmail_a_Page_56.pro
a8e1499db5c1d7054bbfb4e133ddf2b5
e5fb6a75dd52aede287444480d9ac8cb92256941
F20110331_AABAYR abukmail_a_Page_60.tif
f017e769547dbf0841edf997ecbdc766
0956f12e40dbfb14d7bec126174851203692fdfb
1790 F20110331_AABBFA abukmail_a_Page_03.QC.jpg
496185faccff6cbda5df66122a9c917f
103b670b263280b82eac9daff62c48fa0e359c30
50818 F20110331_AABBEM abukmail_a_Page_79.pro
48f5ef06a0e1611daebc8e1c199a54a5
f7d5c4b429b03907dee2e349caf360f30a9eee1c
F20110331_AABAZG abukmail_a_Page_81.tif
0a1c8df3de1577e9876a510a361646e5
b06c1a0cc20fb58c5e01d60e9ca404a681be1146
40118 F20110331_AABBDY abukmail_a_Page_58.pro
aea54907fc25bbc47021d52299e9f895
ae3bf2f1cdb9a1b83e7cefdfbd7a2c24ee608021
F20110331_AABAYS abukmail_a_Page_61.tif
ef776bba073f3608aa5d39881aa319a0
bdddc7f9096f136682ca24914b9ea86436a39429
33719 F20110331_AABBFB abukmail_a_Page_04.jpg
b062312e04d967522dea4af04d30c60f
c24960b2788ef934679c71c3148fa84dc313cd34
34967 F20110331_AABBEN abukmail_a_Page_80.pro
2bc504790eb14d69d9dc7b788e9a78a5
0d1ef8080ce08e372e54e2932bc342764f21425d
F20110331_AABAZH abukmail_a_Page_82.tif
8bb3c433d71b428bcc19858ffb4f9339
e8812671de10c61c0e6c6b053cc7fdf7a826dd94
35342 F20110331_AABBDZ abukmail_a_Page_59.pro
4b0890f02d7bd33aa6f68f0e90584636
43f448f4c6591677e22dcafe831cf2ead8a19f6b
F20110331_AABAYT abukmail_a_Page_62.tif
424f27d726116f8c1469685bb03580bc
fc93ed21916d9b20bb53635e87849d422a9500c5
11049 F20110331_AABBFC abukmail_a_Page_04.QC.jpg
6c8366215a66ea5c27760cf0c36beee3
8ea99b14b1c6a4dcdb00bb3fc9ace67f8d2b7652
13905 F20110331_AABBEO abukmail_a_Page_82.pro
b66050eaf7b01509ce0c671467114271
664bb450f939b874c7003e0146337a83a069e193
F20110331_AABAZI abukmail_a_Page_83.tif
88e48feed16ae37191296ab23daebecf
6d3fa935ccd95854c7dc4cc284341bf3fd138276
F20110331_AABAYU abukmail_a_Page_63.tif
a7b5716a9e2f2e7360e3067f51d50fc9
1d4afcdc9bc5092861b85377da29e1ed514ea3bf
23112 F20110331_AABBFD abukmail_a_Page_05.QC.jpg
04e68c53aefed90ab066d7acf19872d8
3b646f30315f8199cdf33a0df2c304e05fee13f0
43623 F20110331_AABBEP abukmail_a_Page_83.pro
c3e2b7b70c659f41c3c03141ba92ad3c
ade86d463c1ed780d5a8b219fbdf1d80464c4e2d
F20110331_AABAZJ abukmail_a_Page_84.tif
e12ca4f51f1c0f18e47e8e3c37d5795e
efcb55429e4807c1934e4c24dd1f34b4c584c791
F20110331_AABAYV abukmail_a_Page_65.tif
1d80360c45efa01a33ce3a79a0ad7df1
ddf6ec83f90f841b314b8a0bb6565add96eb3362
121179 F20110331_AABBFE abukmail_a_Page_06.jpg
0842886f001aab75d9ffea7ebdf4043f
d8bbd5146c92b2d35454954e1532da70a6f013fc
49478 F20110331_AABBEQ abukmail_a_Page_84.pro
001f54b7b476f82cbc9213eb0e3d572b
5717b5133d2c01f5de536abf162342440e5b73e1
F20110331_AABAZK abukmail_a_Page_85.tif
66a33a47d3b6c99645bffa468247c173
f80396e0bcac5e6911b42565207a139ec35c7ec7
F20110331_AABAYW abukmail_a_Page_66.tif
4fff7336d62dc89319f5ae90d1f6abe2
707d0d85e3360057e6171bf122138e4e60309c47
28646 F20110331_AABBFF abukmail_a_Page_06.QC.jpg
b9cdade6ae7a79413591c8bc5f0a3bcb
7f8e9e263728a4744cf0f2e9679571e4577cf1fb
47445 F20110331_AABBER abukmail_a_Page_85.pro
f1b93b60f04dbd0b18a79e2e4bb6bef9
591cf18f2c744aedfee3199332c9fbe448e04aaa
F20110331_AABAZL abukmail_a_Page_87.tif
e71aee854bc602a91f684a32748edeb5
7c3fb192e8b3f17a63981f5b4c6dbf9283641a28
F20110331_AABAYX abukmail_a_Page_69.tif
b2221aea3079fbd31370d3bed75d62e4
5f99ad7a0d5983ea6b6a715ae4f60581f643f1a2
15771 F20110331_AABBFG abukmail_a_Page_07.jpg
463842a6460ff463cd2d04b07b76d386
179c5916d566528b3727ecc34fbfb1193f159b3f
10027 F20110331_AABBES abukmail_a_Page_86.pro
bc8391ce04a1cad064d1657e457879db
74b2e2eb614654f706a9887a37194308fe12db19
F20110331_AABAZM abukmail_a_Page_89.tif
57e14c6524f31c10645351fae89418b1
b03785c70da7ce435ffd625c68cf04e741b41eb8
F20110331_AABAYY abukmail_a_Page_70.tif
9137db429a8977a93dba32919b17110c
0719535ffc0b7e58733f35aeb94ac0e2b00bbc0e
5095 F20110331_AABBFH abukmail_a_Page_07.QC.jpg
e1ff0b6fe3d346ff88c51fd94169eef7
d99dd1d5a88b20c7f0b7268a4965af7718b9c7f7
50848 F20110331_AABBET abukmail_a_Page_87.pro
f9c75c5f849cd4440911dd3a8b39b092
9337d4a5a42566319763b398756c8346ef4f545d
F20110331_AABAZN abukmail_a_Page_90.tif
557c875d4562be002a8036cdebaeccaf
5e1e45eb2e9fb88dd49214d8c18fa38e4518eca7
F20110331_AABAYZ abukmail_a_Page_71.tif
46186abb4062699fd37f7a71a034fd4f
f2de1896ddf676c9e76916b73371f3dc51d0918c
98525 F20110331_AABBFI abukmail_a_Page_08.jpg
e2ffebceb8b4976e3169debb1e0494fc
16b81b15cc200aee27a6f79eee9238c502cffb31
64996 F20110331_AABBEU abukmail_a_Page_89.pro
2043984771e8d6f6723ee4d2a7a67978
ca61cf91bf2831a9e96025353938084dadb79f2d
F20110331_AABAZO abukmail_a_Page_91.tif
7daf58c73e0d7f250e25bc25476da353
0a27ba6015bfad054ae4392f4db1b89a2edbcdaa
28677 F20110331_AABBFJ abukmail_a_Page_08.QC.jpg
0fc1cadb573217f8dab99f3fe4f82740
85bffe05f53a69f280d63f898c75055f41ea31ab
21503 F20110331_AABBEV abukmail_a_Page_90.pro
dbac7a232b830b1b6ca3614df5d5c946
a0a870d85a20196b8669bdc28666a2a7dd8609d1
411 F20110331_AABAZP abukmail_a_Page_01.txt
0b50ab521149528188fb1ec6bdb60003
7c054016b83fb72f8f342d347d0d00a63c8aa27b
60372 F20110331_AABBFK abukmail_a_Page_09.jpg
b86a0da6fb68c26e7270eeac20095f00
90505306e827af5e742348a143f976a3a978bc16
23553 F20110331_AABBEW abukmail_a_Page_01.jpg
7f7369f132ea04e8f008766473fb51e8
6e52a416289a6f8fe6dd45d2479e3638a8ea4eec
111 F20110331_AABAZQ abukmail_a_Page_02.txt
46b899caf2b804a3a3f64068da0561e4
731dd5578c6608c2be6b6db6d1b461b4743f6f31
135 F20110331_AABAZR abukmail_a_Page_03.txt
a84cc69438ad4f06043cd3aea01e7597
9a486f8f8769f751363b68380dfdf58a6085ffe9
84091 F20110331_AABBGA abukmail_a_Page_19.jpg
348117d81c2e8422bdcb36a315f2c397
00832b0ecd5e524494d026cd6c0aa1c2b4efa7e3
17445 F20110331_AABBFL abukmail_a_Page_09.QC.jpg
742f8ae5e10f1260d0320ee0ad28cef2
cd9252cc71d4d62c9da1fe34b1dcaee97298892f
7486 F20110331_AABBEX abukmail_a_Page_01.QC.jpg
0612750de844293540bac1e05d45dd74
86ef4bb201bcea78f9412b2598cc873b32462171
3245 F20110331_AABAZS abukmail_a_Page_05.txt
ee412e1725dde1fb3511bf638e611c0e
6687a071b3c4763087e8d936de9f7909f2f79e1e
91802 F20110331_AABBGB abukmail_a_Page_20.jpg
2cd05bfd97ff85f1a696242c53f588f8
308fefd6ad96334bd361e871cb39fe2c5f3c1db0
84792 F20110331_AABBFM abukmail_a_Page_10.jpg
243badc289f9af6e36fe0e272d736a27
c13907ce5d2fa95ef890a66d15c6353b7fc7f09e
4840 F20110331_AABBEY abukmail_a_Page_02.jpg
594f76f5870a847e2dc3e43bc738ef1f
a3bc7ead1d44350dd550d0e00df9c81297d13d62
3281 F20110331_AABAZT abukmail_a_Page_06.txt
23a02c9832188ce86d3f458041774988
bf7f1a9134805de43cbabeca8f4f37939091958e
31033 F20110331_AABBGC abukmail_a_Page_20.QC.jpg
7b925e0f68dd7f66d4e98bf8b7559a33
73eed00b8fcea63ec9569d121e0e5460060c165c
27018 F20110331_AABBFN abukmail_a_Page_10.QC.jpg
fff97e7a59c7a7aec3aa87497cc7b7f8
81742cc12dc8b0330c67059d7ee9a738a705f4fe
7024 F20110331_AABBEZ abukmail_a_Page_03.jpg
b58fc53f5fefdb28ad0b0aa2d5e9abdc
921ebf7641ce013fae127850823181b8dfa70fba
1211 F20110331_AABAZU abukmail_a_Page_11.txt
54aad568b44a141258279e8791242f48
d88a5a16888eeb6a79921e7a7884fd9fe612de6b
100168 F20110331_AABBGD abukmail_a_Page_21.jpg
d4240f91a1ac91242d776d520b3f02cb
cdd99cd74bf18d1aa184fb21ae3def0514c9a240
20483 F20110331_AABBFO abukmail_a_Page_11.QC.jpg
4bc31f1b186870a333171fa5b37b7500
ed629d65953a9a7aa01d7c3ced82f3b010b65b9b
1716 F20110331_AABAZV abukmail_a_Page_12.txt
b0d30c67965b0951bab4756307354c71
f53229534e9025b38b47a4cd4ff74bf5add55211
92852 F20110331_AABBGE abukmail_a_Page_22.jpg
c0b506481e564fc55f7dc04231e95cfa
75e3f288131577f303d127ef1b056cf805826cf0
85263 F20110331_AABBFP abukmail_a_Page_12.jpg
c31a32b0c559cd9f9ce8f5ed50f9ae23
56c0b40b92f083c5336a5fd1bcb9e764a6184e99
F20110331_AABAZW abukmail_a_Page_14.txt
58521e7aa040b6180ef973e896bb0e29
5df8e5ae1090025d85042a3ba50526304692b292
30298 F20110331_AABBGF abukmail_a_Page_22.QC.jpg
6287151e3bab24446597f91f5cda647a
cd4ad5e6fc79fc88f99a29c782052a289963b2e1
28277 F20110331_AABBFQ abukmail_a_Page_12.QC.jpg
650bd8264416ac371c07d887cf5e9db4
a60156d0c0668e1767f189948d5c7a58dfc076a5
1751 F20110331_AABAZX abukmail_a_Page_15.txt
a6bc97052e075c2ceaea9c53e9ba9c90
ba1ff3ccb64240be0008bdadd42f88df7612c636
97548 F20110331_AABBGG abukmail_a_Page_23.jpg
2730caf92d552fd18f57db8f4f83211c
6281a49f3c328db68c3406aea7be5224fa6d712e
101129 F20110331_AABBFR abukmail_a_Page_13.jpg
44322da8cbb3b5a310dc381d46c3dcce
7a8791256eee8764dc16f54d5b0bb255c794846f
1887 F20110331_AABAZY abukmail_a_Page_16.txt
b812b293549c2feb8b73b6d3ba4f7f91
31a04658db92d9f0813f4d4807f755fc176e4db2
30971 F20110331_AABBGH abukmail_a_Page_23.QC.jpg
7c1ebefaf3de3ddb47a9469ece3a6c3c
ce239314641d3ca90da8fdf778ec049c5fb199a4
32677 F20110331_AABBFS abukmail_a_Page_13.QC.jpg
99cd088efcf953dacc9fb3fb94996cc2
ca96afd402aa8b006b2f4f6ee4b9cca9124bb5a4
1695 F20110331_AABAZZ abukmail_a_Page_17.txt
c9b081488080307f2343506e64770ea5
78353d31c8b196b01b4563c2b9539ab938a1d0e7
100144 F20110331_AABBGI abukmail_a_Page_24.jpg
b53e6c0be2b4f8d244c2cfbfbaadf5af
f875cd5ed64a4c010fdf01e47ae21fa2dfb6200d
100216 F20110331_AABBFT abukmail_a_Page_14.jpg
7db11fdb9536760e4d4b4d2a40307459
e16858070b7b43ea18968beebdfde1f9724e7f12
32673 F20110331_AABBGJ abukmail_a_Page_24.QC.jpg
cc23921220f83449f0dad1a00f555ca3
33f95b6196ca4b8245952a78b3edd9809ae2271e
31988 F20110331_AABBFU abukmail_a_Page_14.QC.jpg
3ee6d19c65617aaeab710c2523c37b27
2592c39680068bae6edccbe41566ba3bf5df36d9
95485 F20110331_AABBGK abukmail_a_Page_25.jpg
c63f76d5142c6239c89a12c16bdc7649
e048eef7e6a009b3cf02a14e048828d5a2a99e68
28331 F20110331_AABBFV abukmail_a_Page_15.QC.jpg
6db35cada5276bc0a9e396e9bc557982
88f7df35512c1e3a550f3974b25175c445f902bb
30147 F20110331_AABBGL abukmail_a_Page_25.QC.jpg
494ab0497250feae4744ac5690c1b434
f48e8fd449cc7fcea723f479baf0ffc9890a6156
97815 F20110331_AABBFW abukmail_a_Page_16.jpg
8c7a08cf13d6e3a97f4680e11a532c6b
1684577a8bedb93d365e8d111d7f25ef9476c667
88872 F20110331_AABBFX abukmail_a_Page_17.jpg
34c7439e549f3e1956b9cacab0011da1
17729d459fa86d63f5b0faefdcb94e4e69e35d56
33097 F20110331_AABBHA abukmail_a_Page_34.QC.jpg
dbd12a9b5d864120799d31295acb14db
a2c96a852bd5cc023a707117aebd01baa48fdd5a
98945 F20110331_AABBGM abukmail_a_Page_26.jpg
e6c17c0402f9a5448576a2f7cbff9afd
e7abe1d22a8bb8a1d3369db7ef56438e49241277
29025 F20110331_AABBFY abukmail_a_Page_17.QC.jpg
1ce4644085ae020a50bcb308f3182c14
2a5f522db54bd58798c4fa7777217a65ebe02d43
75945 F20110331_AABBHB abukmail_a_Page_35.jpg
36fd1bab1aa6801284460ddf48f35fbc
a4b9cd009c00c03701bc389b22d07103604bf9b6
32203 F20110331_AABBGN abukmail_a_Page_26.QC.jpg
fcffbfaa452201eea7fc8d18139fd233
0e21d5e259eb0191797d2c3b4dc84f0bf719575f
30836 F20110331_AABBFZ abukmail_a_Page_18.QC.jpg
d9b020f36555e66d4ad65fc688afc3c9
81a8e588840e5c719797b9f86fdbf8399f408c70
33183 F20110331_AABBHC abukmail_a_Page_36.QC.jpg
c7a69d8e3e76c717c59f3686ba2de82a
f1142216f13d35aeba378076c639286fb5b701ed
94997 F20110331_AABBGO abukmail_a_Page_27.jpg
49d510a4d4606d79f5a968be11a43adf
9709acac22e7a45b122a2974290a2ea4d69164c3
101072 F20110331_AABBHD abukmail_a_Page_37.jpg
d15da189460d4cc9cd8ab26d35e4bdc4
aa0853a67ba89d0f37ae53f0435abf81e14b7ea5
94615 F20110331_AABBGP abukmail_a_Page_28.jpg
648245ab1058de53c5bac05729c99d3d
a3b910f4dfdb2929c53b6553c151c49273ac1d99
33589 F20110331_AABBHE abukmail_a_Page_37.QC.jpg
09d2133359241373092b14be908509f6
113d2751c9c81f32dc7001ed6933bcfa42d149e2
31470 F20110331_AABBGQ abukmail_a_Page_28.QC.jpg
0b791245671684759679e9d6ead03666
a7742c07d0d722e04ed174f3ae2aef3f903060cd
37882 F20110331_AABBHF abukmail_a_Page_38.jpg
5ff64ae35b878d02617dd35a9f1cf3ac
b642afb70d3c7d4d8cb0a1f2e2f2ef5783da24ad
96539 F20110331_AABBGR abukmail_a_Page_29.jpg
733d1093d600748c69b2755a4023c9bc
79652eebe88588a31d97bf38d3334085bb69e180
12849 F20110331_AABBHG abukmail_a_Page_38.QC.jpg
980e4969c0c461c6bb6296a6fc903794
384e133e03104be6997bb9a3547b66c873cf4f9d
32279 F20110331_AABBGS abukmail_a_Page_29.QC.jpg
4bfa88d04853686b658a0f51bb6931e3
69f7ae7c5cf7bfe3c70323d830b941ab43f55624
89120 F20110331_AABBHH abukmail_a_Page_39.jpg
fb0629385382d671e8e110a81b2567ef
df78323a641a3e6a7614cc07d4198d857c75ae8b
102588 F20110331_AABBGT abukmail_a_Page_30.jpg
a3821946bd2500474cfcdbeab98a278d
64c1bc7d9d548d93ff425cb2e30b8a330c6ad023
28288 F20110331_AABBHI abukmail_a_Page_39.QC.jpg
b8600c7e4b5e1415efb211d7358e18fd
338fc25439487b439b1e461eae6ef2a447f2d43a
33403 F20110331_AABBGU abukmail_a_Page_30.QC.jpg
95afe18dca9a5d156346720ca593fdbf
3a9c45ad6473a0cc59bc47abb1c69f97279ac6df
100910 F20110331_AABBHJ abukmail_a_Page_40.jpg
4ac86ed150858d6cf2215034af663a8d
a03383fa4852df0d6bd0c235ce4dec391529eb71
32983 F20110331_AABBGV abukmail_a_Page_31.QC.jpg
3ca56eaadca2385b87b9bcfc261e9b3f
3a85918b10c9a4690087829d95a9be274a3a01f8
32369 F20110331_AABBHK abukmail_a_Page_40.QC.jpg
dd3edf53f463a09a53dee1993af15634
696233661d44cbe81a503bb2668ede1eb927477c
44355 F20110331_AABBGW abukmail_a_Page_32.jpg
55fa8c1ab34a0552171007091edf245f
8a25dcf2e15463f1bf0bb06c6fdd17533e0f1061
107370 F20110331_AABBHL abukmail_a_Page_41.jpg
56e464166c479b0186e2f7ccf7240fb4
35e45c8c681cfb48da1c46c6e97f0bfa7f5dc6b4
14564 F20110331_AABBGX abukmail_a_Page_32.QC.jpg
4c5f2e04bd9e3fb3ba9c7c2397a3d1e0
03ad042dd3400bbc909a3a582849a1c9d774fa3f
102482 F20110331_AABBIA abukmail_a_Page_52.jpg
2f8289151f43f52a3b882f0d22f7166f
923b1e117866a656d128c116e2deeeb414baa4f3
27987 F20110331_AABBHM abukmail_a_Page_43.QC.jpg
ef17d4fd4d919bbd4c898f57c8bcbe3a
5ae19bbdb7781cc268d09917f4d41c0365014c44
90524 F20110331_AABBGY abukmail_a_Page_33.jpg
ed3e115dc43cc67064a50f625e7f08ac
5308495fd13c49e60faabb204ddc679de1043e21
33085 F20110331_AABBIB abukmail_a_Page_52.QC.jpg
b1a382780a41d3dc9779aed516a05f04
e1ab6a76c58941edc665c3a3a24a0366e2ca956d
101988 F20110331_AABBGZ abukmail_a_Page_34.jpg
413d52f9eb582c257666c70fe2a26e3f
ee376fcfe87dcf4e3272f8292c787174781e2d87
102578 F20110331_AABBIC abukmail_a_Page_53.jpg
f5eb314c93d77bfcf37872ff2ef4d3bc
d21533538d9b7168525bd6e555fc430468f9b293
86947 F20110331_AABBHN abukmail_a_Page_44.jpg
46e9003d0d35a1e1fc3b6cad25df7bc2
36cc7de3e94a51c178a02edfffc6497c9e717ec3
33076 F20110331_AABBID abukmail_a_Page_53.QC.jpg
da8c950e04cbcf7b7baab6891b1e8100
c1610efde9642e430569d7629c3338fb59d509a6
28992 F20110331_AABBHO abukmail_a_Page_44.QC.jpg
d03e260a3890246c7b70d879dd9829dc
b630cb6916a1398d1fa0b923226bb6da5b757544
58951 F20110331_AABBIE abukmail_a_Page_54.jpg
52f1f6ce834881bc68afb4fc6c9e26ad
d5c21491536c883890ec2ca3fc4c05d74d9fd21c
64058 F20110331_AABBHP abukmail_a_Page_45.jpg
206167876cf30e03bdaf1f66b9143190
3fce1465aca99a420327969579b91698c32f1ba8
18115 F20110331_AABBIF abukmail_a_Page_54.QC.jpg
c1d2ac776db232e316098514a5a89251
0ce1a5c159ee30b43c89df484fcea6282bebbc56
22327 F20110331_AABBHQ abukmail_a_Page_45.QC.jpg
65d9870d9157d865e320170ea5fb06c5
9a3b4ed9955f011f62e22faeffc3eb01aaac67be
106883 F20110331_AABBIG abukmail_a_Page_55.jpg
52aa61f5b589c8a0f4ce5e2a85ca2c25
8f0c187daaa1074f3186447f24b87773a65ab66f
96773 F20110331_AABBHR abukmail_a_Page_46.jpg
ad9d6733a35a1b1c3197c338b3498982
c718c91cf8075fded3c6142ae17e4144caee7d73
34525 F20110331_AABBIH abukmail_a_Page_55.QC.jpg
a31a5ae28defee540baa075cd918ef42
8cc3f3f6a8b9a05946a8cb0e86e5cc2162bc8542
30928 F20110331_AABBHS abukmail_a_Page_46.QC.jpg
cc719152cb540d344f65978374a71e86
1879fe18c1f79d3b7c55bd7ec7acea2b52be773b
73825 F20110331_AABBII abukmail_a_Page_56.jpg
e99f85bcd732389716a60153692c2095
007c0ec6305355f7435426ef2460b51d1f931b6c
99174 F20110331_AABBHT abukmail_a_Page_47.jpg
d42e34210515aefdbbd847ef7aa47a58
01a0783f23acad68533750b458c74a343e52bb9a
22632 F20110331_AABBIJ abukmail_a_Page_56.QC.jpg
c7ae13ac362a012ea133e9e38e9f68e0
09dd576139cd29a0e1309dfade6b76a768bcd37c
59528 F20110331_AABBHU abukmail_a_Page_48.jpg
30951160e01c0f32e3166cfa61f4931e
1bba9065649ce5bf32156d6955cdd016dcc6f5a0
103607 F20110331_AABBIK abukmail_a_Page_57.jpg
f14c7e8539445d38b5f945b0eed3c939
5f8ab2a2f563449657835171e3b8b60e999122e6
18227 F20110331_AABBHV abukmail_a_Page_48.QC.jpg
f1c783415002c2194d2ae6a7f6f56a17
5f23d0117256ae732b16181bca55afabb1ebfea1
34656 F20110331_AABBIL abukmail_a_Page_57.QC.jpg
4345b8c13f3ee0e2aa2f4279dd2e684e
edd64e92ae6a1bc73c3cae6286c3171aaba3a29b
52890 F20110331_AABBHW abukmail_a_Page_49.jpg
2abe51a4b42b8346c81216ea6471f0c6
74b99c4e6c272d5c2d58f8aceca391f360e84312
28422 F20110331_AABBJA abukmail_a_Page_67.QC.jpg
c05e64ccf46d084e4da776746f4c7d60
c959660c72184f30c79e63c9ef5005c91f5f8b99
23922 F20110331_AABBIM abukmail_a_Page_58.QC.jpg
82030d4a9d2f43ec6ba7f9fb964de829
a8e5d92cbdc13c40d80aa3c84878b3a2b5d7e131
16289 F20110331_AABBHX abukmail_a_Page_49.QC.jpg
344d187e5a7a17ca85cb6f133942187f
14072a70ce65ec6e457474dfa8056b48bf588dd9
92438 F20110331_AABBJB abukmail_a_Page_68.jpg
66315635342b6e19dce864df6fabcd4f
5fa4b0aca10679f29737ecd977d1480ffd5b26a2
91069 F20110331_AABBIN abukmail_a_Page_60.jpg
01cbeaba7fa0e9d947c45e98ce0fe830
3258da8c664c96f13beb0e1cb0a5fea877a47c46
28933 F20110331_AABBHY abukmail_a_Page_50.QC.jpg
0c570c61355689e2eaa8a006a3605903
c2fd4a34735922d12ba7fa5728ca027503cd6fa1
92082 F20110331_AABBJC abukmail_a_Page_69.jpg
7ef2cd0e4b0283c67e34387e4dc91292
b6219246be569c22617a9239face9dbcdfa00928
48800 F20110331_AABBHZ abukmail_a_Page_51.jpg
bca53b1d3e1faba10e19b73224bdda6b
b4709b20d81271c30c68ca7a2996c978932280fb
26717 F20110331_AABBJD abukmail_a_Page_69.QC.jpg
9b716987ef70188743600f4f3700b59b
711a49272794174d0b2ee600c9c41c88dade535a
29033 F20110331_AABBIO abukmail_a_Page_60.QC.jpg
e69c7a362d0c579c6990ac6e5204575c
e97e999e988966a6d00b39942fb6a3138d255b8f
85880 F20110331_AABBJE abukmail_a_Page_70.jpg
20bb77ccdc9c674fa5f04f4fcbb10d26
f5fb26ed3dbe03abf8f6d9ba5244a89fec0a0aed
25638 F20110331_AABBIP abukmail_a_Page_61.QC.jpg
a1ab80ee3d3a7cdb8a897d19f335077d
12517fd44912b82f7d8f927dd10885133d614555
25552 F20110331_AABBJF abukmail_a_Page_70.QC.jpg
30c971d7731240239433e53383f5056f
9d5cecb06b0c12d64fe1795c029face633e9eafb
82965 F20110331_AABBIQ abukmail_a_Page_62.jpg
d4492664c3d45ef9242981a6f33da06e
4425409d3fda5cdab9257c3e0b174f63b6f31536
59814 F20110331_AABBJG abukmail_a_Page_71.jpg
009ff39acea02ed3efe4af9d4726706f
d7ab071d09ac9c31f7641f24bf1672725437a344
26636 F20110331_AABBIR abukmail_a_Page_62.QC.jpg
d8364b4036cdb921208a534749bbe0fa
13de918e71f60465ee8a5022ba2fcc947f906f4e
20307 F20110331_AABBJH abukmail_a_Page_71.QC.jpg
60b73ae0b602987949ebfde0a907815e
ca32040b671f01e7ccd38c98ac9e71e3bffca264
30098 F20110331_AABBIS abukmail_a_Page_63.QC.jpg
be9a224e4c539783c5189fe36d4c8fb5
1967494ff3fc6d39e866eeaaf26543ee4ddb9f0f
F20110331_AABBJI abukmail_a_Page_72.jpg
e7a080c55882b31b1ec4aab0014c57ef
b8f56a10b07659e5649fe1c7a393d28310fec088
99149 F20110331_AABBIT abukmail_a_Page_64.jpg
9361ac9d37d186696fb01387eb6665a4
ea52582c2e36227c530debf70cd20afc864f542f
31642 F20110331_AABBJJ abukmail_a_Page_72.QC.jpg
d621cf082a64143d863d48213e7a38d7
c32b18d0dac916b98121c09ef5a4017e5ccbc02a
31943 F20110331_AABBIU abukmail_a_Page_64.QC.jpg
41d347fe45b3c180d424bfefe6c2f0b7
3b8b1b6cebb7acce45f99aecbb3867bce01abc2c
90315 F20110331_AABBJK abukmail_a_Page_73.jpg
3a67cdb553105e4cb5d4e0e3992af308
cbd1fd03a25ac41bd5f2306eda1041af1d634b3c
58637 F20110331_AABBIV abukmail_a_Page_65.jpg
65a23b6f5654d7cdf5687b4917d92391
968d6061160daf4bd6fd39d1cb21fe23a548c5ad
26348 F20110331_AABBJL abukmail_a_Page_73.QC.jpg
d16593f919bc13341977e536f1158b4b
3e0f5770293f1ef4773ac48bdb673506d5f3fc1d
19690 F20110331_AABBIW abukmail_a_Page_65.QC.jpg
ae790dec5828d22629c0ca85f61a7253
83b18fef82e257cd0343dc02d464b4e3364f8c24
26472 F20110331_AABBJM abukmail_a_Page_74.QC.jpg
92356e2a0bb0b5a973ec0a1501aa3ce3
4cb06e7ec4641ac3187d49cf65a8cc5feda1e9a8
103343 F20110331_AABBIX abukmail_a_Page_66.jpg
23b81d2652208b8f04b41fe1740ca5ec
09a68429baff3fca25442e7d4b80a462cf79d783
90434 F20110331_AABBKA abukmail_a_Page_83.jpg
5fa782e50e9757dbd0784d5b3481f4c0
dd4ee861c0f18adc25c972d641f7463daeccf7ce
88517 F20110331_AABBJN abukmail_a_Page_75.jpg
d70f8f4cfad1b3c2e884a0ebcc29c9d6
f40aaf6273437d5c1850485ac97b23c8b27ed5e4
33547 F20110331_AABBIY abukmail_a_Page_66.QC.jpg
3b5845bedfe816f5c5bc1064f1fb39e2
9320957f03ba256b254630ac2be9c2f234e7221e
29343 F20110331_AABBKB abukmail_a_Page_83.QC.jpg
4bad2478373734e978e9fc3068f4c080
e5d77e42447397da629b9be849dfac09d3c1f330
27990 F20110331_AABBJO abukmail_a_Page_76.QC.jpg
5ff70889745a9348a5355eff3c642b13
4dadae33439c5803aa61e2ba8ed6abbc7bd62be4
86078 F20110331_AABBIZ abukmail_a_Page_67.jpg
b0fb44d166968f0d4a9f636cac82cd04
f2a3f9cc5ac4fd4d05eae78de95f2eed859b6674
99880 F20110331_AABBKC abukmail_a_Page_84.jpg
16f99e241da511fc857ceb6c1f8b8f83
3368b3cf09834ea813e27f05c042040171947937
31859 F20110331_AABBKD abukmail_a_Page_84.QC.jpg
3a5630e47221a9924c1bb8f63813e7ad
9f65abee0c30e26cb89c25ad85f6e807e4bb7204
87279 F20110331_AABBJP abukmail_a_Page_77.jpg
509eec5df404b28328148e116d687d15
6db065e30003a060bf50ac6d10596c8b91504053
31693 F20110331_AABBKE abukmail_a_Page_85.QC.jpg
afcf9a064ed275fd0fa033e7f9413457
170db9038bb2746a6bcbd886a5b010b17c1daffd
88032 F20110331_AABBJQ abukmail_a_Page_78.jpg
ced85a7120e6d93ea68bbafc8437ac22
c7e467b78368679b95b9079e89be2a8380f65434
22996 F20110331_AABBKF abukmail_a_Page_86.jpg
62a3d5aa7101dcb25f757ffb9b9b8be3
27355f007530dff94e18f1a1e9c8a8f9f75c29f5
28151 F20110331_AABBJR abukmail_a_Page_78.QC.jpg
4829d0ee80cb1837ee4c738d57105d9c
6c6935258054b4e1e0c4fe0cf67f4e8037c2441e
102113 F20110331_AABBKG abukmail_a_Page_87.jpg
a98e88459b1881c84fffa153a64b3467
f02f2be415c150f4aee3d91c1ba06ac899646d5a
103666 F20110331_AABBJS abukmail_a_Page_79.jpg
96d19e46b08761162d9b9b13be24d851
25fb926cca0a9fda3c3e1c0f4048482ca646a655
29895 F20110331_AABBKH abukmail_a_Page_87.QC.jpg
5fd38e1b41e988100b7284942a3c25e9
dd6caae2d0ee8ab02c90529bfc7515978dcb0ef4
33592 F20110331_AABBJT abukmail_a_Page_79.QC.jpg
51808acbdf66b73f7171e66fdf3fe369
2c9a7258daa09719e5da90f18ca56ca2c2abf248
117768 F20110331_AABBKI abukmail_a_Page_88.jpg
45ad23e4608be8db9ddb3f32ab397017
ece1e44397daecf25c23f6fe2295df8d71a5dc67
34106 F20110331_AABBKJ abukmail_a_Page_88.QC.jpg
a0c59f48b36e3a4698df2e980fa6b895
502b7cd6384c2f58048e8a74b83b87e134306e8c
73991 F20110331_AABBJU abukmail_a_Page_80.jpg
d796b6202abd726a2f2cdb2d782f45a0
7976d254b06ddb2e20d58e275f0638ba917ca370
135452 F20110331_AABBKK abukmail_a_Page_89.jpg
fdb414fa9d1ef04bb62d01716824f8a4
199a9d7c1274d4738d5b666c52850b5e3ba22298
24491 F20110331_AABBJV abukmail_a_Page_80.QC.jpg
8a22b3ffee29d816040ae8eba79fe0a5
4d07de4b2ca50fecb46774d8aa5100810db92270
37582 F20110331_AABBKL abukmail_a_Page_89.QC.jpg
264d7915fcaa2d74b3279b6c94b653d3
d99276d8335227e1dd5c3136f4dd57ff1a4c4eaa
57729 F20110331_AABBJW abukmail_a_Page_81.jpg
45012368dfd20f132ce98ccb172cc3e6
09e0978c8a9320c28a18aa105d83c962c5be4caa
108829 F20110331_AABBLA abukmail_a_Page_13.jp2
98dd69e2d1c65cdff8e6e26ef0be4dc5
4ceed44e3eb0c907e6e0339c334b8028586a0dbb
49225 F20110331_AABBKM abukmail_a_Page_90.jpg
bd461dcecae7eed0b70754811d489860
a1f24b47c2a64c1ec3da8159d009c47478afab9c
18755 F20110331_AABBJX abukmail_a_Page_81.QC.jpg
deb995fd42ca431dc8750fe5cb06b693
5e84e9cba5ad38c6271f67397844c2848a2a4cb2
107639 F20110331_AABBLB abukmail_a_Page_14.jp2
bf99c5606dab72284e4a3220e2d4c651
286edcc88fb3df4a9c1a7b452b0f16da9ed5e2e9
13580 F20110331_AABBKN abukmail_a_Page_90.QC.jpg
77589c8c821ab1b3233de4ef6b8c244e
fb72be1d360f5638c3e3232e3b2730f06f310865
30885 F20110331_AABBJY abukmail_a_Page_82.jpg
9b7f5d8c4823e8b17794acc2a67316c5
04f773ebfb2af2ccf05b7e4c314e076728e2ff5c
97018 F20110331_AABBLC abukmail_a_Page_17.jp2
37df855eaea0c08be22aea76775aada7
2da0a91626e1893a5e99435ad7c32739e54883e3
41914 F20110331_AABBKO abukmail_a_Page_91.jpg
5ef795d373527cb89ded93dc8552cc77
45f94b41aee0733310dd353a0a969443d47dcdd2
10723 F20110331_AABBJZ abukmail_a_Page_82.QC.jpg
11d7934bfb148f9ea65c75b6ed6b073e
54db557eb5d62544b1bd79acb4f8b6f0b51f636b
90008 F20110331_AABBLD abukmail_a_Page_19.jp2
d45b768c841763f6549f818b8c74b432
497fda2a42ac304f0c76e4d33dc9639c8b9807ec
13273 F20110331_AABBKP abukmail_a_Page_91.QC.jpg
48f36bd560b6a89f52602f7ec1df561d
71508df8896cdc6a7facc3534edbe21020e11d75
99740 F20110331_AABBLE abukmail_a_Page_20.jp2
6abdd292f0f59d95d63f07ef03fcb5e3
fdc8002dfe2e21615017396ba9362fd2fc956d1a
107588 F20110331_AABBLF abukmail_a_Page_21.jp2
61565f4ac1fc4ac61bc48262da50760f
f4e3d5dbe4256d43ee8c22f57994a50c41afc5d4
23246 F20110331_AABBKQ abukmail_a_Page_01.jp2
af4a762ba77a0f47c1b24574c08d1579
7b373799638d2b18f9cff1f0f81850fbfff3c5b7
103904 F20110331_AABBLG abukmail_a_Page_23.jp2
59f75bbfde2b922e60f8678ced5d38cb
4c84c307c2dd38a7f97892ca980d78bb7607bd24
6117 F20110331_AABBKR abukmail_a_Page_02.jp2
cb1519e960894e24db2310c4a7d5821f
2b578a7320296bb487fc3562f28fc38ca808f2a0
106384 F20110331_AABBLH abukmail_a_Page_24.jp2
a1c16dd0c0b27e04a8780ccd61dbf986
50ca9b9bf8ba1971f112cad8f134bd4bbcbe3c1d
7910 F20110331_AABBKS abukmail_a_Page_03.jp2
93c530ce6d36a0e2121dd1b12816d2c3
313d6e2345041e4ac3a9691608ce5f1eeeb92ca2
100067 F20110331_AABBLI abukmail_a_Page_25.jp2
157053742452534a56d84d521a19d867
497ba7b77725469eab6831aefe95d2390024c916
35152 F20110331_AABBKT abukmail_a_Page_04.jp2
89eacce37cc0835f09e919a6be7ab011
b91f07b9eced3ed052e11523187ed2d3348d59be
106347 F20110331_AABBLJ abukmail_a_Page_26.jp2
08c14f491d94d9f5e83df74f167baccf
ddd55511b363449d5728e5f32b726f480c0ec0ae
1051966 F20110331_AABBKU abukmail_a_Page_05.jp2
25d216c64c7253476c808c965aaf732a
8f12997ff198f8e294a8464cbfee815800f9e507
102974 F20110331_AABBLK abukmail_a_Page_27.jp2
8309133cd0a589504e42a914b2176ff7
d958f8b56050ffeef31e7ed2438be053ef0735c5
1051985 F20110331_AABBKV abukmail_a_Page_06.jp2
2daadb1384fc2a6ff5c7547c1d5b629c
2d74df2a6c59000207ae22187dac6305ad0e8a2a
103518 F20110331_AABBLL abukmail_a_Page_28.jp2
75b97cf340b3b91d9b3bfea50850649a
14ae4f5221ecf7620a0e0a410e70f3547389acfd
236101 F20110331_AABBKW abukmail_a_Page_07.jp2
ddf77b028a933ca08bbbd956f384a22f
b33fa47446d2e0335dc0177bea4ee2843f30881d
103603 F20110331_AABBLM abukmail_a_Page_29.jp2
84a0c39573c3f5f76629e0dbdd8d5bfb
8073a8c5af7b5eea1b7c2d1cab360c10581d8311
1051976 F20110331_AABBKX abukmail_a_Page_09.jp2
b3728a6ed09217991f8b331dd56df2c7
67999bdf4f8b258b24fba1bef7e50d762aa30bc5
108763 F20110331_AABBMA abukmail_a_Page_52.jp2
9a47467f823ddadde19163b9379ac11d
e399f4a80ab25f6847d6fd6aa1d6590b660dbbc7
109161 F20110331_AABBLN abukmail_a_Page_30.jp2
a958164e86b0b84454e04d5220eced93
e0ec6874dd4ab19c35d05c26af8d62346043683f
88821 F20110331_AABBKY abukmail_a_Page_10.jp2
e707056f005db6ae2c8c5e0acb6ae6eb
1488222b7579f96db6fc814ba3c417af06651cf0
57197 F20110331_AABBMB abukmail_a_Page_54.jp2
87c682cc2efc82d5af9c156d0594b506
1460c1b8d07c1350aad3e6dd5def637f40753272
48387 F20110331_AABBLO abukmail_a_Page_32.jp2
b8c5b0da95256cc4c09eb20b7360ab9a
1b4032c52aa5f70499c53b8d73e439f03db746f4
67987 F20110331_AABBKZ abukmail_a_Page_11.jp2
e3a3f3971b9897129b0bc2af2349be81
97fed3eaac17971724a19a0be1c3ef6d2bb78097
112138 F20110331_AABBMC abukmail_a_Page_55.jp2
edcef7c2940dbee5a09028e0c1704e56
70a31dee90c35ecba3969c5bd2b751b2d5f1c8a9
94284 F20110331_AABBLP abukmail_a_Page_33.jp2
ee143edc89bf85905644c75d49f0d109
6ccc386874e7d29372e57d23555c7efdf7ad018e
83438 F20110331_AABBMD abukmail_a_Page_56.jp2
eb85f0c8e5057cc999e2a920b1e6e1fb
a4790a9d0ea2fdfea3cef5a04ff6e722034482b5
750764 F20110331_AABBLQ abukmail_a_Page_35.jp2
14806d07cb45003b373c101b1c2b4aaa
c673eea0f4ee81c3e1cd07b68df85eac03c35760
109735 F20110331_AABBME abukmail_a_Page_57.jp2
39cd41d775cf76f0b413bf3fbbcf2ffc
044c33c10eafa42317515fc64625dbed69833465
77219 F20110331_AABBMF abukmail_a_Page_58.jp2
a2f52ac7fafbc62aeacab9ea76a58be3
08ce797725c062e2b0f662829be2e6a387cf5593
108178 F20110331_AABBLR abukmail_a_Page_36.jp2
dfbf349040874c2f517a5417cbb54876
c67cae50d3ab9c1e7a5f82a957b82bd560307571
78098 F20110331_AABBMG abukmail_a_Page_59.jp2
7d919dd10410ca5564518ee2b5ddbc2a
b93b9b4afa131231f52fd64d128654990a52a0f5
108045 F20110331_AABBLS abukmail_a_Page_37.jp2
61f14a71e8668574af9d2f588b80e42d
89a5ad65a257258c2c1650a9ca1137e65d52d9ae
94794 F20110331_AABBMH abukmail_a_Page_60.jp2
be3515e7dbcc55d2f655b8e449748213
574da9a5b33fa6ca7f649cf9fe95d668d54cbb70
40140 F20110331_AABBLT abukmail_a_Page_38.jp2
a21055e7831e029d92a50903cb1529d0
4358780aa7d6808c58c4641846ebcc2d0d5942ed
84537 F20110331_AABBMI abukmail_a_Page_61.jp2
48335a10eb6bf2ed9c7c2aec18339d74
f0afa8c9c709c851dd0803bffae7b10c5b7dc2f3
92539 F20110331_AABBLU abukmail_a_Page_39.jp2
0e299a92c5386d0ea868eb10ef46fb18
faf046185e10d971f8c63884ccd54076da8e8c3c
87916 F20110331_AABBMJ abukmail_a_Page_62.jp2
8ec2980b0b49f27db0420d49983a8dd3
7bbda00b3ca4ae141324b3273b80fc0f10365d96
112794 F20110331_AABBLV abukmail_a_Page_41.jp2
00d6b3c31edbfc67b8b32a531dd4655e
3ed6d1fef736bce398d63cdaf57fda12730779e9
109973 F20110331_AABBMK abukmail_a_Page_66.jp2
1427e573499c6f6a8d4ac44bdc48f83b
e2f4786ba9336b14a27f03fc0138d5c7d595aa33
65581 F20110331_AABBLW abukmail_a_Page_45.jp2
6de1b0184f443bb0f48ee68be75b4760
09baacee7c29385656aff980491daadc0c16c10c
91913 F20110331_AABBML abukmail_a_Page_67.jp2
81be13f5179a3a5616226364777b7c0e
48bf58eb387177fb5d42abc6b4316e4d10294b21
58367 F20110331_AABBLX abukmail_a_Page_49.jp2
48d81fee665490837d1d1823b9a18709
d5d12d62e64d65ca46aa336de09d51540dd5a0c9
102562 F20110331_AABBNA abukmail_a_Page_85.jp2
64423fb7eff73a8d0dce770b37479598
573aeacc3dee4b5c92e9d0ac3ea6bedbfad9be71
1051955 F20110331_AABBMM abukmail_a_Page_68.jp2
cc61e548fe046b74dd28c3d700ad176c
49568f74402c519344e8bba49baeb805c1531815
95350 F20110331_AABBLY abukmail_a_Page_50.jp2
eae32a5a956c6315f540da32d13d8d20
203e5b5befabbbde015da43310836269459ddf07
25013 F20110331_AABBNB abukmail_a_Page_86.jp2
6b8acc6f8ceb734df53d2cfd190db975
54d5f7663270820a217a1ff9f0ff99569b53fd7d
1051956 F20110331_AABBMN abukmail_a_Page_69.jp2
22a03cc2d92cce2713f88debd5374130
09394adc69aff8664de6dcbf3a11b3f84cff8888
478170 F20110331_AABBLZ abukmail_a_Page_51.jp2
dadce3c6beac3068b2653c452dbbc16c
d84ba59c35aa6e2387e1d906de5f90289c25cb25
109167 F20110331_AABBNC abukmail_a_Page_87.jp2
a0dc7efda57ce1d3f7fed2d738ef6d2a
2b760a092a1c2250fd60f6a60f8301a53f0243e1
977857 F20110331_AABBMO abukmail_a_Page_70.jp2
6b4660249500c11b104bdb269acabd74
2bacad65c31eff14fd4c4440c9d85b568ac19bea
125062 F20110331_AABBND abukmail_a_Page_88.jp2
dccb2ceb6a94447467a2ac535a427297
2b37fe9d61e14168d65cfe6169756d807d4d8238
621860 F20110331_AABBMP abukmail_a_Page_71.jp2
923b45304a6a67c12b0e30cd2f6b450e
ff50272f1c03602260ddf10aa37af9aa9840b84a
136586 F20110331_AABBNE abukmail_a_Page_89.jp2
b993ad1af9ab183ce65f2c32236086d9
914453b3dfbf75ca3ef7b2bfc0acf0971284bcbf
106432 F20110331_AABBMQ abukmail_a_Page_72.jp2
53bd574ca778ade6ce1c0e407c70a204
d6ad808b9487a177a182a4939ec8efa00e44b472
44018 F20110331_AABBNF abukmail_a_Page_91.jp2
9f9229461abb2f2c1c7073f588a813d5
17e40ba935d1c42375f5e76fe261aa29108e8081
1023481 F20110331_AABBMR abukmail_a_Page_73.jp2
184ce739e29e4f3702c5f13a42472131
bb02f1df15366b5d9eb2be44b98a526100ea0eda
2139 F20110331_AABBNG abukmail_a_Page_01thm.jpg
8c3f06fc439c2854d8918333d7433281
c4b8002cc03b920e7813d58cfa3d8d0bba2aed4d
744 F20110331_AABBNH abukmail_a_Page_03thm.jpg
a4d2089c91224bc98f8692244673358d
2960ef8d2f03a9d9af38c5ca48438e811237aff4
1030449 F20110331_AABBMS abukmail_a_Page_74.jp2
f4f605ca19a00a4a7a38c7c16868f7de
ae11a9e08b80012746ab3ca1ea70f31299a5872e
5696 F20110331_AABBNI abukmail_a_Page_05thm.jpg
aee7b7321f1159aa5966615d7e909e8d
6bd2d4e9da4de14e618180c68860f19e1754c601
1034733 F20110331_AABBMT abukmail_a_Page_75.jp2
7ae49dc7c89d77c4408372b229abbc2b
cd82b42aa0bb15ee4a7d0b35639e598b3d356a26
6799 F20110331_AABBNJ abukmail_a_Page_06thm.jpg
4dac874f66b3402cee0a277a9d5a552a
f657747c7664fa0cce910c5be452b6a6a588af00
967062 F20110331_AABBMU abukmail_a_Page_76.jp2
10096dc15b86cfa6b2efd11f6750bdef
fc4f8bc574e1f63c89159ae24047cfcd90d3501a
1704 F20110331_AABBNK abukmail_a_Page_07thm.jpg
8ce726ce43c239ffe241aac6c40f5625
c23e6c7d13fed2cc392309a6d9db0f578b8c27ac
111225 F20110331_AABBMV abukmail_a_Page_79.jp2
7070bc2ea320d97dd1aca461e6bd3d96
01a3fa362c2dbffa799c381b91b249a816a31f97
7209 F20110331_AABBNL abukmail_a_Page_08thm.jpg
4001ea6edfcdba389c09f89b5e628788
e76a6a7746b31a29e89fd66d2b3eb2bb646ff1d2
783569 F20110331_AABBMW abukmail_a_Page_80.jp2
1117f96716fd5f598aadcb80ea099834
839e46a0379ffc86b5593e84c8445c1dd87a5aab
8093 F20110331_AABBOA abukmail_a_Page_29thm.jpg
3cf2a57f0167dba6e6e0d9e7164a60f6
02a18c34ce7b6ea94669aa8032a1b98a394838b7
5102 F20110331_AABBNM abukmail_a_Page_11thm.jpg
35972cb3459ba6b0cd4db95537cab9b1
2d4cb816cb2250cba136c91bba32b1708dc1b4a8
33260 F20110331_AABBMX abukmail_a_Page_82.jp2
b85a8c8a91dfaa790551c3f39681257c
2132c72165574a733aeb050cac1b1f24ab811058
3727 F20110331_AABBOB abukmail_a_Page_32thm.jpg
62139b774cf8eff3d2e988b46c725857
c81a29ce2e0d66ad166660ff502cd05a1bc129cf
6931 F20110331_AABBNN abukmail_a_Page_12thm.jpg
5ef9043f03299a343acbd7b1296027ab
349d758cc3b963a2e13132e08e3995c231e3bb51
95205 F20110331_AABBMY abukmail_a_Page_83.jp2
61eb6d84b9fa4808dbb834c1580af152
d246a4a12515c92139ba876a2006cbbaa3a5f2c6
7283 F20110331_AABBOC abukmail_a_Page_33thm.jpg
559ad6d08a36d1a980ff986b5b988fd5
9fa619076c714260eb62fcdde72071124c5a2f70
8067 F20110331_AABBNO abukmail_a_Page_13thm.jpg
eea757bc72b92a8606616721d7294bdc
1dec8068d241482b993ebf3663f53c6138c8ac43
105496 F20110331_AABBMZ abukmail_a_Page_84.jp2
f7f63f2d7d3d1a9fe0e3005f2eecefe6
0bb5f73a9d9438fcd5acee166ccbd22bec1631d6
8372 F20110331_AABBOD abukmail_a_Page_34thm.jpg
0e34c58b003116d7f12191bbe8e31af2
d450bb91569bd5e1a7679d4600a597b03b51e48f
8270 F20110331_AABBNP abukmail_a_Page_14thm.jpg
3182292d7cdb52045d8606dcdda85222
f15c2d1f8928c97c0c2fee57a9d61da13b501612
7093 F20110331_AABBOE abukmail_a_Page_35thm.jpg
cae58bbe2e04bd42821ada94f4ce0afe
6a45e95658c0d36eaccee78c6741201cb33d45e3
7781 F20110331_AABBNQ abukmail_a_Page_16thm.jpg
87e6443f1a8d1118d8cd20de205782f0
bb8f0cb1e7dc9c0874d9d3cd7817bd82554aea3f
3170 F20110331_AABBOF abukmail_a_Page_38thm.jpg
a061d22273c11efadd61e66eb07147c1
35f8f0013182f702b37a89d38cab9d2ad10914ca
7942 F20110331_AABBNR abukmail_a_Page_18thm.jpg
a34187b83c7f281e2c8d7ee3e8277d64
cb44a77a557760aca93389c187572e48cb3e02c3
7218 F20110331_AABBOG abukmail_a_Page_39thm.jpg
ff4bdddae0f22c91a6c3106427b62fc3
e4bc7796a2c076ef70253f098566705d2b2d5a3a
7573 F20110331_AABBNS abukmail_a_Page_20thm.jpg
d137489984ff2dcd749bcdcb272fbeaf
0e679d9b70fc231caf3f701738d58bab122742cf
8327 F20110331_AABBOH abukmail_a_Page_40thm.jpg
55206956c85d48946508031695bc7bc1
c118037ab372642ebafba3880d1b4c49c322c4ef
8410 F20110331_AABBOI abukmail_a_Page_41thm.jpg
6f4d7ac6eb787c6e50ca0a7b8cfccd23
e0acaee73b5a50dcc659c235704c23e609e3947e
8189 F20110331_AABBNT abukmail_a_Page_21thm.jpg
e585b1c6023867f15a325437ffbedf54
7bad5461876ae6a574a16f5f586f162ac4ef2b04
8155 F20110331_AABBOJ abukmail_a_Page_42thm.jpg
11319a5fde0993144245fe5206f0660f
2b045538e156f1bde27f20e1c23666a62e577575
7692 F20110331_AABBNU abukmail_a_Page_23thm.jpg
d2414748d7d518e3585254fa2f48c514
87a44a74d6b353e11b5cd69fdf6718e58af4e8f8
7756 F20110331_AABBOK abukmail_a_Page_44thm.jpg
37c06d022284ca8d521ed388468e865f
fc9478ce19f57eb1f96c252480a56d6b0ec18929
8120 F20110331_AABBNV abukmail_a_Page_24thm.jpg
d73df62a0b7a3557da4f1229fcc399bf
601447149ad9c5fff3de2b72a62aaa9354456df0
6448 F20110331_AABBOL abukmail_a_Page_45thm.jpg
7290ff114874a1ca7f408f62f935f068
06d0c3abaf74aa7df14d489f0f181ba16cc22a70
7618 F20110331_AABBNW abukmail_a_Page_25thm.jpg
006955ed1763bf9765f68acb57e799bf
c67a0f870a52572c61a09193573e64a2f6a3e040
8063 F20110331_AABBOM abukmail_a_Page_46thm.jpg
945ab8ec6f95fac938ac4283d42a8a03
df4b89e5c3b4b7535b0645462376a8e1f2a82b58
8375 F20110331_AABBNX abukmail_a_Page_26thm.jpg
2649a665d38358ff63a464b05b350b06
537ff70b54056b4a38224bd5f2977309d072a46a
7101 F20110331_AABBPA abukmail_a_Page_67thm.jpg
70065f6cea72dfd9ea2de9325337f469
d917731af44774bac6b8431ddf8f62850024c17f
7693 F20110331_AABBON abukmail_a_Page_47thm.jpg
12ced45ced223ca4d686d6ee70f007d9
6d206bc655b401f50a4a90b1a31a0d7362908bbd
7813 F20110331_AABBNY abukmail_a_Page_27thm.jpg
1005c83dca435dae2a12578d60d9c8e6
d77d6f42afff264edef1200f8ac66e22c1b910f7
7233 F20110331_AABBPB abukmail_a_Page_68thm.jpg
e7b0829ccf2925b012bbea38187d25e6
5a47d6fbf03ffb79582da72cb2fa6855cc298904
4634 F20110331_AABBOO abukmail_a_Page_49thm.jpg
f547e94757eda5076370413b6db9bd2e
cc3c2dc728ce5b0f467784f3fb4501810be97c9f
7488 F20110331_AABBNZ abukmail_a_Page_28thm.jpg
b7ab143cd54903414902908cebedb42a
5b37e78ccc6b481305249512403e8dd806bfd685
7534 F20110331_AABBPC abukmail_a_Page_69thm.jpg
62eda75c293df7fd8a0dd7eeb9b6cb37
908c94687a3f77c298d378f17ce45e5e25b040ce
7318 F20110331_AABBOP abukmail_a_Page_50thm.jpg
b41a8bd4c0386971f6fcb4b5a3167ad3
f61011d9d9410282e34431c5ce59b6462f631524
7407 F20110331_AABBPD abukmail_a_Page_70thm.jpg
1eed88251affb52b1d54e2d37b7f2916
3ab67c8ad86d4e1fb2bcf84266d3aab29705be68
5258 F20110331_AABBOQ abukmail_a_Page_51thm.jpg
363acdd8c052c03c8aa92b82ea70c130
5bc6501991d98bb2d6d5a990d6e03cd26e9877d4
5566 F20110331_AABBPE abukmail_a_Page_71thm.jpg
bd99722f660b4e71ac155e617d8289b4
642f4de94150aa18682b7923d8d965f8b609c11f
8010 F20110331_AABBOR abukmail_a_Page_52thm.jpg
634b741adfc70bf9996c98828156f488
753477fced24b66042b4b4c5b18b1f2c217cc923
7959 F20110331_AABBPF abukmail_a_Page_72thm.jpg
77d0cfb53096552c13a3b58c354ed8a1
b5d53fe2de191d5ac5bdd9edc8fd3a7b764ee0c9
7262 F20110331_AABBPG abukmail_a_Page_73thm.jpg
ee1ad4ce101145854d3c84a043633bc1
b1b133b54deaf16b36d74c6999dad0b75a8e7bd3
8431 F20110331_AABBOS abukmail_a_Page_53thm.jpg
9ec215b37aa6aa63947ec5719c9c26cf
09cc824690866a546be289532cc0837d4c27ff78
7395 F20110331_AABBPH abukmail_a_Page_75thm.jpg
3ad0132d963812f327a17c7bc6bc58df
fb0b9b45637d324cca4f831f78c1c6c8384f2fa5
4733 F20110331_AABBOT abukmail_a_Page_54thm.jpg
83ad5df5c789f0a10e9544359cbcf2fb
fc588ee06ac12c76c1f5c4913b412a69a478d127
7313 F20110331_AABBPI abukmail_a_Page_76thm.jpg
60651160eb75331634e701f41103a779
fbc60821755bdd85ae8ef2baa6e51ab90a16bca1
7633 F20110331_AABBPJ abukmail_a_Page_77thm.jpg
0c9567fab8aaaf8e4a5219ee993ea12b
35eef3c05decbdd71a52372d2944d58d2b3c0e3a
5882 F20110331_AABBOU abukmail_a_Page_56thm.jpg
aaefcd9cb02ec45a8607748c377a49fd
8d1deef7af00740fd50a250a74c1c87b21add8ab
8425 F20110331_AABBPK abukmail_a_Page_79thm.jpg
8601f3fab3d8c483819ac5cf191b1dd8
a43f3ff9707720a6500eade854f0fb9a29c74203
6331 F20110331_AABBOV abukmail_a_Page_58thm.jpg
2ae0637c843f5cbb928638eee71cfa62
816ca90194645d17ecbf6a69292870ebab0b082a
5797 F20110331_AABBPL abukmail_a_Page_81thm.jpg
2715590c90992893e046a7d15c9dfa2d
4554c16545dc7caf990da6c27ebb003f069aba7c
6551 F20110331_AABBOW abukmail_a_Page_61thm.jpg
7e9df10ff1953555b15c6a7feceb448b
7df41ee7333909317dc794efef4e5101e421249a
2646 F20110331_AABBPM abukmail_a_Page_82thm.jpg
327452be2c3701845d52c02e7f0ba384
3e368d8e314e54263cdaf04969d186f241c3d11c
7801 F20110331_AABBOX abukmail_a_Page_63thm.jpg
04b4b0c6dc6254cbcd9470b4617eda63
4cc725bd72559f52a0428a79edbd9a2cad98c1ae
8139 F20110331_AABBPN abukmail_a_Page_84thm.jpg
abad225bae0402aee40d3ba404b2ad64
24e4fe65c82f717b973c3ca933aba3bf2d87ce83
7991 F20110331_AABBOY abukmail_a_Page_64thm.jpg
35d13ff300f8d274483f24ee8f57c037
e994c8d3165b03d014bcc338ebe054d9ffe8a7c6
7687 F20110331_AABBPO abukmail_a_Page_85thm.jpg
48e10c4a2123e77e91a1d64b8f474246
1ef686a7ec68672a543243c926bfc85a7b2f6f7d
5421 F20110331_AABBOZ abukmail_a_Page_65thm.jpg
7aee8b8ea23ba59b3c094bc9736bb3fa
f0a2d2b7a8bd9a798443a106405708fc4a7fa511
2123 F20110331_AABBPP abukmail_a_Page_86thm.jpg
e0456c381d1e9e8db54827fff406ebd5
e1e9448f29ba34121b11972c88f51eb31edd3058
7438 F20110331_AABBPQ abukmail_a_Page_87thm.jpg
5202cac4949d723d56a23c4aa577a49d
ba1dbbea66d46eba32809afa39af025f6c2c23b7
8333 F20110331_AABBPR abukmail_a_Page_88thm.jpg
b4bfa14d226cf3de971ce8c6567b5479
0975be506cd2aec02e79bb1075bae2f498b9089d
3617 F20110331_AABBPS abukmail_a_Page_91thm.jpg
f1871db1b7512057dde1ede4661eccdf
eff266d35dd562e8b6bda58969a1c50248e94c2f
758421 F20110331_AABBPT abukmail_a.pdf
713f6725c3996adc0fd829f4f62d16b4
28ce9e3ad3648573222a28e33a6842bcd0a2ab7c
108238 F20110331_AABBPU UFE0013060_00001.mets
a61449e8f705de5403feba30230f2aa6
4077f0f142b3fb98d72e1174ffc2cc07f13fa31e
abukmail_a_Page_01.tif


Permanent Link: http://ufdc.ufl.edu/UFE0013060/00001

Material Information

Title: Pervasive Computing Approach to Energy Management
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0013060:00001

Permanent Link: http://ufdc.ufl.edu/UFE0013060/00001

Material Information

Title: Pervasive Computing Approach to Energy Management
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0013060:00001


This item has the following downloads:


Full Text












PERVASIVE COMPUTING APPROACH TO ENERGY MANAGEMENT


By

AHMED AHED ABUKMAIL

















A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2005





























Copyright 2005

by

Ahmed Ahed Abukmail

































To my wife Nermeen; and my two beautiful daughters, Raneem and Haneen.















ACKNOWLEDGMENTS

I would like to express my greatest appreciation and gratitude to my supervisory

committee chair (Dr. Abdelsalam Helal) for his guidance during my doctoral studies. I

also express my gratitude to my supervisory committee members (Dr. Gerhard Ritter, Dr.

Sartaj Sahni, Dr. Joseph Wilson, and Dr. Louis Block) for their help.

I thank Dr. David Whalley (of Florida State University), and Dr. Chris Healy (of

Furman University) for providing the software package we used to calculate the number

of loop iterations (vpcc compiler).
















TABLE OF CONTENTS
page

ACKNOW LEDGM ENTS ........................................ iv

LIST OF TABLES ............... ...................... ..................... ...... .. ........ vii

LIST OF FIGURES .................. .............................................. ..............viii

ABSTRAC T ............. ............ ................. ........ ........x

CHAPTER

1 INTRODUCTION ................... ............................ ......... .. .......... 1

2 R E L A TE D W O R K ..................................... .....................................................................4

Energy-Management Techniques at the Architecture Level ........................................
Smart Batteries .......................... ........................5
Battery basics .................................................... ........ 5
Intelligent pow er drainage.........................................................................5
Energy-A w are Processors ............................................. ...............6
Reducing Power through Circuitry Components ...........................................7
Power and energy consumption equations................................7...
Voltage and frequency scaling ........................................
Capacitance load reduction ................................. ...............8
Power Reduction through Architectural Design......................... ...............9
Operating System Techniques ................. ............... .................9
Energy-M management Solutions ................................ ...............9
M emory and I/O M management ................................ ............... 10
Com m unication Techniques...................................................................... 10
Scheduling ...................................... ........... .............. I 11
Software Application Techniques.................................... ......... 12
Compilation Techniques......................................................... 12
R eordering instructions ........................................ ................. 12
Reduction of m em ory operands .............................................. ............... 13
Code generation through pattern matching ................................................13
C ode annotation ............................................. 14
R em ote task execution ....................................................... 14
Application-Level Techniques .................................. ............... 14
Tools for Low-Energy Design and Measurement ........................................ ...16
Pow erScope ...................................... ..........................................17


v










Derivatives of SimpleScalar............... .... .................................. .......... 17
Other Power-Estimation Techniques........................... ............... 18
Recent Work ........................................... ........ ...............18
Grid Com putting ........................... .................. ........ .. .... ................. 19

3 STATEMENT OF THE PROBLEM ................................ .................. 22

4 OVERVIEW OF THE APPROACH.............................................28

Overview of the Compile-Time Solution ..........................................28
Overview of the Runtime Support.................................................... 30
Implem entation Setup ....................... .. .............. ... .. ..... .... ..............3 1

5 COMPUTATION OUTSOURCING FRAMEWORK.............................................32

O v erv iew ...................................... ............................................ 3 2
Formal Model ........................................ ........35

6 COM PILE-TIM E STRATEGY ................................................... 39

Overview...................................... .................................. ........39
Energy-Optimization Process ......................................... ...............41
Calculating the Number of Loop Iterations...............................................................41
Loop Data and Iterations Acquisition............................. .................. 44
Calculating the Size of Loop D ata............................................................... ..46
Identifying Loop Instructions and Total Loop Execution Cost..............................46
Insert Outsourcing Code ..... ...................... ......... ........48

7 RUN TIM E SUPPORT................................................49

Battery M monitor ............................ ..... .. ... ...... ...............49
Network Monitor and Surrogate Service Discovery Server...............................50

8 EXPERIMENTAL VALIDATION........................ .......... 51

Setup ................................................... ........ 51
Instruction-Level Energy Cost Estim ation ...................................... ........... ....53
M easuring Communication Cost ................................................................ ..... 54
Sim ple Experim ental V alidation.................................................. 55
Large-Scale Benchm ark Validation.............................. .................... 56
Other M easurem ents ................... ............ .. ............................ 61
N orm alization of R results ....................................................................................... 65

9 CONCLUSION AND FUTURE WORK .................................... ...............72

L IST O F R E F E R E N C E S .............................................................................................. 76

BIOGRAPHICAL SKETCH ...................................................80
















LIST OF TABLES

Table page

8-1 Total energy saved by outsourcing the Fibonacci loop................ ..............56

8-2 Total energy saved by outsourcing the matrix multiplication loop.....................56
















LIST OF FIGURES


Figure page

3-1 Improvements in laptop technology from 1990-2001..................... ............24

5-1 Framework for computation outsourcing at runtime...................... ............... 33

5-2 Steps for executing a client program under the outsourcing framework .........34

5-3 R source selection rule......................................................... 37

5-4 State transition rule................ ... .... ................. 37

5-5 The send rules for the client and the server...................................................... 38

5-6 The receive rules for the client and the server..................................................38

6-1 Overview of compilation and optimization process.................... ............... 40

6-2 Example of C program passed as input to vpcc ........................................... 43

6-3 Resulting INF file for the program in Figure 6-2...................................43

6-4 Algorithm to show implementation of the pseudo-parser.....................................45

6-5 Algorithm to calculate loop data size once all of the variables in each loop have
been collected ...................................... ...... ........... .47

8-1 The C program used to estimate the energy cost of executing an empty for loop
100M tim es ...................................... ......... ........... .54

8-2 The C program used to estimate the energy cost of the Idr instruction................. 54

8-3 Local versus remote execution of 300 iterations of a Fibonacci loop executing
300,000 times ......................................................57

8-4 Local versus remote execution of matrix multiplication of a 400x400 matrix ........58

8-5 Local versus remote execution of a bubble sort loop which sorts 50,000 integers..59

8-6 Input file for the ray tracing application........................... ............... 60









8-7 The 2-D image representing the 3-D scene generated by the ray tracing
application for a 200x200 image ................ .............. ........ 60

8-8 Local versus remote execution for ray tracing three spheres in space and
generating a 50x50 im age ............................................... ............... 62

8-9 Local versus remote execution for ray tracing three spheres in space and
generating a 100x100 image ................. ..............................63

8-10 Local versus remote execution for ray tracing three spheres in space and
generating a 200x200 image ................. ...............................64

8-11 Energy cost for the battery and network monitors' execution ..............................65

8-12 Comparison between theoretical and experimental measurements of the total
energy consumed................................... .......... 69

8-13 The amount of base energy consumed based on the server processor's speed.........70

8-14 The effect of changing the communication metrics on the total energy consumed.70















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

PERVASIVE COMPUTING APPROACH TO ENERGY MANAGEMENT

By

Ahmed Ahed Abukmail

December 2005

Chair: Abdelsalam Helal
Major Department: Computer and Information Science and Engineering.

We present a novel approach to energy management. This approach is a pervasive

computing solution to the energy problem in mobile devices. Its pervasiveness arises

from the fact that communication is used and viewed as an opportunity to save energy,

whereby certain candidate sections of code (represented by basic program blocks, most

likely loops) are outsourced from the mobile device on which they are executing, to a

surrogate server machine with an infinite power source. As can be concluded,

outsourcing code implies that the data involved in the computation will have to be

transferred to and from the surrogate. This approach deviates from the traditional view of

communication as a drain on the battery of a mobile device.

The solution presented here is a compile-time solution and optimization,

augmented by the necessary run-time support. The high-level source code is augmented

by additional high-level code to intelligently (at run-time) allow the application running

on the mobile device to outsource basic program blocks to a server. Both client and

server applications are a byproduct of our approach, as the original source code is









transformed into a client/server application where the client is installed on the mobile

device and the server is stored on a surrogate machine.

This approach uses a methodology from the domain of real-time systems to

determine the number of loop iterations, and that facilitates a compile-time computing of

executing each loop. We also gathered the necessary information about the size of the

data involved in each loop. This allows for determining the cost for outsourcing each

loop (in terms of the cost for communicating the data to the surrogate). Once both metrics

(computation energy cost, and communication energy cost) have been determined, we

can easily generate a code that allows the application to make the run-time decision of

whether outsourcing is beneficial. This approach is a fine-grain approach to energy

management, because it looks at basic program blocks (as opposed to certain predefined

tasks or functions). For this system to allow the execution of the client/server application,

the necessary run-time support was built to facilitate this behavior. Results showed

significant levels of energy saving, using our benchmarks.














CHAPTER 1
INTRODUCTION

The emergence of mobile and then pervasive computing (as new computing

domains) introduced new challenges and research opportunities: one was energy

management. These challenges arose from the mobility of used hardware [30]. Such

hardware includes devices such as cellular phones, PDA's, laptop computers, and even

MP3 players. The mobility of these devices implies that they are powered by mobile

power sources represented by the battery of each device; and that, in turn, implies that the

power source is limited.

As these devices become more popular, and their use becomes more apparent and

frequent, the need to manage their energy consumption becomes more vital to their

operation. This is because the more often a battery needs to be charged, the more often

the mobile device is rendered immobile (which reduces the pervasiveness of their

applications). Managing the energy consumed by these mobile devices has been an

important subject in research and industry communities of both mobile and the pervasive

computing. Solutions have been presented at the various levels and layers of the

computer system, and often these solutions to the energy problem involve a certain type

of tradeoff

One of the most attractive avenues to energy management are high-level energy-

management techniques. Such a good argument was made for handling power

management at high level that Ellis [8] proposed a power-based API to allow for synergy

between the application and the system. One of the most attractive high-level solutions









to energy management is a compiler-based solution that alleviates or minimizes the need

for programmer power-awareness. This is done via compiler optimization. Velluri et al.

[36] studied he effect of the traditional compiler optimization techniques on system

power (and therefore energy). Results showed that (except for loop unrolling and

function inlining) most optimizations increased the energy consumed by the core of the

processor. These results were (at least for loop unrolling) confirmed by Kandemir et al.

[20].

We developed a new pervasive computing approach to energy management in

mobile devices. This approach views communication (wireless communication) as an

opportunity to conserve energy as opposed to being a drain on the battery. Sections of

code represented by basic program blocks that are most likely to be loops have the

opportunity (at run-time) to execute locally on the mobile device, or remotely on a

surrogate, by communicating the data involved in their computation back and forth to the

surrogate. The decision to chose local vs. remote execution is based on whether it is more

energy-beneficial to execute locally, or to communicate with the surrogate and let the

surrogate do the computation. Communicating with the surrogate and letting it do the

computation is known as computation outsourcing. So, this approach is based on a

tradeoff: the tradeoff here is between computation and communication. At runtime, based

on the condition of the battery of the device (full, empty, or in between), the application

can opt to go into an energy-saving mode and communicate with the surrogate for the

opportunity to outsource code. This depends also on the availability of the surrogate

device, and user preference, in addition to the condition of the battery. Hence, this

approach will be able to discover services within a wireless network. Service discovery









must be simple as the decision-making code is inserted at compile-time. This is

accomplished by two simple programs: the battery and network monitors.

While this approach allows the decision to outsource code at runtime, it is

considered a compile-time solution. This is done via high-level compiler optimization, by

preprocessing the original (non-energy-aware) source code, and generating two versions

of the code. The first version is the client, which runs on the mobile device; and the

second version is the server, which runs on the surrogate device. The client and the server

are generated, by inserting the necessary high-level code, to be able (at runtime) to make

the intelligent decision of either outsourcing or not.

To generate intelligent code that determines whether to outsource the code, we used

a methodology from real-time systems to calculate the number of loop iterations. It is

safe to assume that this methodology will not be able to calculate the number of iterations

for every kind of loop. However, it is capable of calculating loop iterations when the

loops are nested, and when the loop bounds need to be determined at runtime, and it also

allows loops with multiple exits.

Unlike previous work, our study examined energy consumption at finer granularity.

We examined certain candidate sections of programs (basic program blocks), and not

entire programs, or specific tasks or functions only within a computer program. This finer

granularity yields better energy saving, as our test results show. Our study targeted a

subset of the C programming language to verify our algorithm and methodology. As a

proof of concept for this approach, we used three different kinds benchmarks involving

loops with different complexity. We also developed a 3-D graphics rendering application

to further show the energy saving benefits resulting from our approach.














CHAPTER 2
RELATED WORK

Prolonging battery life (often called energy management) has long been the focus

of research. This problem has many facets, which can be faced by addressing the various

components of a mobile computer system. Earlier [1], we authored a book chapter on

power-management techniques. These techniques include reducing energy at the system-

architecture level, by targeting (reducing) various components of the power equations

(P=CV2f), where P is power, C is capacitance load, Vis supply voltage, andfis switching

frequency. Other techniques targeted the operating system by saving energy involved in

communication, by caching, by process scheduling, and by having an energy manager.

Additionally, we presented software techniques grouped into two categories: specific

application techniques, and compiler-based techniques. These software techniques (like

those targeting the operating system) are considered higher-level power-management

techniques. Solutions that target energy management usually involve some kind of a

tradeoff. This chapter presents the various power and energy management techniques,

that we introduced in the book chapter [1], and other techniques and new methodologies

related to our study.

Energy-Management Techniques at the Architecture Level

Hardware and architecture are among the first areas researchers investigate when

looking into energy and power management. Therefore, they target the CMOS circuitry

and the batteries of mobile devices. They also look for ways to develop processors that

are energy and power efficient.









Smart Batteries

Battery basics

Every battery has two terminals: one positive and one negative. Electrons flow

from the negative to the positive terminal, when the two are connected (when the battery

is installed). Electrons are formed on the negative terminal by chemical reaction that

occurs inside the battery. This reaction occurs when the electrons travel from one

terminal to the other. Rechargeable batteries (the batteries that we are concerned about in

mobile devices) suffer from the problem of memory effect: the battery does not fully

recharge because it has not drained completely before recharging. Lithium ion batteries

solved this problem recently. However, once the battery is built, and as long as it is

connected, the reaction occurs draining the battery power, and eventually rendering it

useless until it gets either recharged or replaced.

Intelligent power drainage

The aforementioned shortcomings of regular batteries can be summed by saying:

they do not lose power/energy intelligently. Once they are connected to a device, the

power drainage is going to happen. This motivated the development of the smart battery

system (SBS) (http://www.sbs-forum.org/specs/index.html). This battery is different from

non-smart batteries because it provides information and system status to the host machine

through the System Management Bus (SMBus), and it has its own re-charger. This

information can and will be used by the system, to manage power and energy in the

mobile device. Operating systems standards and specifications such as advanced power

management (APM) (http://support.microsoft.com/default.aspx?scid=kb;en-us;310752),

and advance configuration and power interface (ACPI) (http://www.acpi.info). Currently,

most laptop computers are shipped with smart batteries.









A few companies in the industry produce products for smart-battery systems. These

companies include (but are not limited to) Samsung, Motorola, and Hitachi as battery

vendors. Semiconductor vendors that produce chips for smart batteries include companies

like Powersmart, Acer, and Adaptec.

Energy-Aware Processors

Another way to tackle power consumption in a computer system is to produce

processors that consume less energy than regular processors do. A number of

microprocessor companies in the industry have products that are energy-efficient. Intel's

Xscale architecture (http://www.intel.com/design/intelxscale) design optimizes energy

consumption. Xscale represents an integral building block of Intel's PCA architecture

(http://www.intel.com/pca). In addition, Intel has the SpeedStep technology in building

the Mobile Pentium III processor. SpeedStep provides the advantage of dropping to a

lower frequency, and lower voltage, when the device is powered by battery; which results

in conserving battery life while maintaining a high level of program performance

(http://www.intel.com/support/processors/mobile/pentiumiii/ss.htm). Transmeta

Corporation (Santa Clara, California) has the Crusoe family of processors

(http://www.transmeta.com/crusoe/index.html). The Crusoe is an x86 compatible family

of processors that is lightweight, high performance and 70% less power-consuming than

other compatible mobile processors. Additionally, the ARM family of processors is

widely popular, and is geared toward reducing power consumption while maintaining a

high level of performance (http://www.arm.com).









Reducing Power through Circuitry Components

Power and energy consumption equations

The power equation is P = V I, where P is power, Vis voltage, and I is current.

This equation can be re-written as P = C2, where C is capacitance load, andfis

frequency. The energy equation introduces the time factor: E = P T = CV T, where

Tis elapsed time. So, the energy equation can also be re-written as E = V* I T.

Reducing any of the variables involved in any of these equations will reduce the

energy and power consumed. Capacitance load, frequency, and voltage can be managed

at the hardware and architectural level.

Voltage and frequency scaling

If voltage or frequency is reduced, then energy and power consumption will be

reduced. Therefore, researchers targeted voltage and frequency scaling techniques to

achieve that result. Smit and Havinga [31] argued that reducing voltage means reducing

performance; and thus additional hardware is needed to balance that difference to keep

the same throughput.

Hsu et al. [19] attempted dynamic frequency and voltage scaling. Their results were

simulated, and the claim was that their results are similar to those of Transmeta's Crusoe

TM5400. At compile-time, they assign a clock-frequency, and voltage levels for input

loop executions. Their idea is to assign deadlines to tasks, and utilize these deadlines to

allow for voltage and frequency scaling. Simply, if a task finishes at time 0.5t, and its

deadline is at time t, then it will not hurt the execution if the task takes t time to complete.

So, at compile-time they identify these tasks and generate code that will allow for lower

voltage and lower frequency to be used in the execution. This work supports the









argument that additional hardware will be needed to achieve voltage and frequency

scaling.

Capacitance load reduction

Reducing the capacitance load is another method to reduce the energy consumed by

a computer. Two main mechanisms can be utilized to handle this goal. Smit and Havinga

[31] suggested that memory capacitance is greater than that of the processor. Therefore a

reduction in memory operations, to keep as much work as possible on the processor and

in registers is beneficial. Gebotys [14] also has done this work by reducing the off-chip

activity. The solution is basically to use a minimum cost network flow to find minimum

energy solutions by simultaneous partitioning of variables into memory and possibly

allocate them into registers.

Another mechanism to reduce the capacitance load is to reduce the logic state

transition when handling memory addresses. In any processor the program counters tend

to change multiple bits when accessing the next instruction or data from memory

locations and that depends on the size of the instruction and if that instruction was a jump

instruction or a sequential instruction. That transition increases the capacitance load. To

minimize the transition, a Gray code solution can be utilized to facilitate memory

addressing. Su et al. [33] did this solution along with a compiler. Sequential Gray codes

vary only in one bit at a time. Therefore, there will only be a variation of one bit, which is

the minimum that can be achieved when accessing memory addresses (excluding the

jump instructions).









Power Reduction through Architectural Design

Architectural design always seems be an attractive solutions to the energy problem,

and that is due to the fact that you can always leave the architecture the same but add to it

certain components which will result in reducing energy and power consumption. Bellas

et al. [4] introduced a compilation methodology. At the hardware level, a second level

cache was added to support the new compilation strategy. This cache is a smaller cache

and is much simpler to use than that of the first level cache.

Witchel et al. [37] introduced a direct addressed caches solution, which allows the

software to access any cached data without the hardware doing any tag check. Therefore,

the energy consumed by the tag check will be saved. A compiler solution was introduced

to augment their hardware solution.

Operating System Techniques

Operating system solutions can be a very useful complement to architectural

solution to energy management. While most operating systems are distributed with their

own energy management solutions, additional techniques and methodologies were also

researched to handle this problem.

Energy-Management Solutions

User-configurable energy management techniques are now implemented in

operating system. Advanced power management (APM) is one of these techniques. In

APM, users can define their desirable power scheme. Some of the variables that users can

control are the screen, the hard drive, and suspend/sleep modes. Users can also instruct

the screen and the hard drive to stop/sleep when the system has been idle for a while.

Additionally, they can also specify if the system is idle, or, in the case of a laptop, the lid









is closed, at these points the whole computer can save its state on the hard drive, sleep,

and when it wakes up it will restore itself back to the previous state.

Intel discussed some shortcomings of APM such as not considering future

capabilities of PC's that include their communication capabilities because APM drops the

communication link. So, they came up with their own solution in "Instantly Available

PC" which uses the ACPI specification to manage power while keeping the system on-

line via either the modem or a LAN card. So, Intel and other industry partners contributed

to the ACPI specification.

Memory and I/O Management

Memory access is expensive when it comes to energy consumption. Additionally,

secondary storage (disk) is even more expensive. So, the more the computation is left on

the processor, and the storage is done in register the better it is for energy consumption.

For instance, making fewer incorrect file predictions is a good methodology to save

energy [38]. This was accomplished by observing: the probability and repeated history of

file access do not occur without a reason. They claim that programs access more or less

the same files in the same order when they execute every time, giving themselves a good

knowledge for determining what they call program-specific last-successors for each file.

Communication Techniques

Energy saving communication techniques are getting increasingly important. This

is due to the wide use of the Internet by almost every user of a computer. So, finding

ways to save power on communication will definitely be beneficial.

Communication systems have three states, sending, receiving, and idle. Switching

between these three states is energy consuming [31]. Therefore, reducing the transition

from one state to another will definitely reduce power loss. Therefore, instead of serving









the communication request of sending, we can buffer the data to send it in large chunks.

The same thing can be done on the receiving end. We can only accept data that is of a

certain size optimized for our power needs. Otherwise, our system needs to go into an

idle mod. In addition, with the availability of solutions that conform the ACPI

specification, there is no need any more to modify network protocols to support full

connectivity while the system is in sleep mode.

Kravets and Krishnan [21] provided a method for managing the communication's

device on a host through suspend and resume operations. They claim that a

communication device will continue to draw power unless it is suspended. Their solution

takes into account the need for the host to know the communications patterns, so when it

is suspended there is no communication going on, otherwise problems such as buffer

overflow will occur on the host and other hosts. So, the functionality of the suspend and

resume operations is adaptable such that it will avoid such overflow.

Additionally, Loy and Helal [24] introduced an application-based solution that can

easily be utilized in developing energy management techniques in the operating system.

Scheduling

Energy-aware scheduling in the system requires prior knowledge of the activity

inside that system [5]. Monitors can be incorporated within an operating system to handle

this. Bellosa [5] introduced an event-driven energy accounting as a way to manage power

in an operating system. The way this was done is by online analysis of the energy-usage

patterns that are fed back into a scheduler to control the CPU's clock speed. This work

here was done with real hardware as opposed to simulation.









Software Application Techniques

High-level or software energy management techniques are getting highly attractive

as solutions to the energy problem. Ellis [8] argued a case for the need for higher-level

power management. The gains from higher-level energy management are great at the

application level, and this stems from people's interests. People can enhance on hardware

advances on power management, but also when hardware solutions are not available

software developers can handle this issue to either complement hardware solutions, or

obtain better results without newer, more advanced hardware. At the software, and

application techniques there are two important sections, compiler development, and

application-specific techniques.

Compilation Techniques

Research and experiments have shown that, with the exception of loop unrolling,

and function inlining, compiling for performance does not imply compiling for energy

[36]. So, optimization techniques that target energy as opposed to performance are

needed, and rightfully so, since compilation for power is extremely important. It is

important because it is an attractive solution to programmers that are not energy-aware.

In this section we will discuss some techniques that showed promise and some that did

not when dealing with compiler optimization.

Reordering instructions

The energy and power equations showed that their values are directly proportional

to the frequency of switching the signal from 0 to 1, i.e., logical state transitions. Tiwari

et al. [34] mentioned that switching is a function of present input and previous state. So,

the previous instruction is a factor in the function. Therefore, reordering instructions can

be a factor in reducing switching activity, and therefore, reduce energy consumption. In









utilizing this technique in the 486DX2 architecture, it did not show much favorable

results, it only showed little improvement.

Reduction of memory operands

Tiwari et al. [34] also showed, via experiments on the 486DX2 architecture, that

instructions with memory operands have higher energy consumption than instructions

with register operands. Pipeline stalls, misaligned accesses and cache misses add to the

cost energy-wise. Compiler optimizations achieve reducing the number of memory

operands. In their research, the authors claim that the most efficient way to reducing the

memory operands is via register allocation for temporary and frequently used variables,

and that also leads to potential reduction in pipeline stalls and cache misses. However

there are some issues with register allocation optimization techniques, such as more

complex compilers, longer compile time, and register allocation algorithms need to be

modified to optimize for low power. Additionally, larger caches, which will result in

lower miss rates, will result in this technique being a little less significant.

Code generation through pattern matching

Aho et al. [3] introduced code generation through pattern matching and dynamic

programming, and was later utilized by Fraser et al. [13] for generating code that

optimizes for performance. The idea is to find a cover for intermediate representation

(DAG) for each basic block of code. Finding that cover is done using dynamic

programming and tree matching so that the overall cost is minimized. The cost function

introduced in Aho et al. [3] and Fraser et al. [13], take performance under consideration.

Therefore, to optimize for power, the metric used is power usage as opposed to the

number of clock cycles used for the instructions. It was suggested that further

investigation of this technique needs to be done.









Code annotation

Marculescu [25] proposed a methodology to annotate the code at compile-time to

adaptively select, at runtime, the optimal number of instructions to be fetched or executed

in parallel to save energy. However, this solution would require additional architectural

changes.

Remote task execution

This is believed to be one of the most attractive compiler optimization techniques

because it takes into account the idea that while communication might be a drain on the

battery, it can also be an opportunity to save power by shipping the execution elsewhere.

The idea of compiler-based remote task execution has been proposed in other research.

Kremer et al. [22] have done some work that utilizes compiler-based remote execution of

certain tasks in the field of image processing applications. However, their work included

only hand compilation of the code and not compiler implementation or any automation of

task identification. They basically utilized what they called checkpoints in the code to

determine the task's delimiters. Additionally, Rudenko et al. [29] introduced remote task

execution mechanisms. However, this approach migrates entire processes to remote

servers, and then waits for their results back to the client remote machine. As far as Java

applications are concerned, the work done by Palm et al. [27] introduces the possibility of

migrating compilation (both optimizing and non-optimizing) step of running a Java

program to a server, and that is due to the power consumption cost of compiling Java

programs.

Application-Level Techniques

Attacking the power problem at the application level has been done quite

extensively. However, the solutions provided are only useful in certain applications, and









most of the times can not be generalized to allow for utilization in other applications.

Most of the work seems to have been done in the signal and image processing

applications due to the amount of drainage that they inflect on the battery, since they use

computation power, and screen power.

Loy and Helal have introduced an active mode power management (AM/PM)

where they increase the amount of information available through the use of power aware

API [24]. They saved up to 62 times the power used when compared to using no power

management. They examined the amount power used by a connected wireless PC card

when the connection is not needed. At that point, they take advantage of stealing back as

much of the power as possible. The targeted applications in this work were an email

client and a web browser.

Flinn and Satyanarayanan [10], demonstrated a collaborative relationship between

operating systems and applications to meet user-specified goals for battery life. In this

work, applications can dynamically modify their behavior to conserve energy. The way

this is done is by the operating system monitoring of energy supply and demand. It is as

simple as: if the supply of energy is plentiful applications perform best, otherwise they

will be biased towards conserving energy. They used PowerScope [11] to validate the

measurements of energy consumption for accurate estimation.

MP3 players are some of the most popular devices in the market at the present

time. Haid et al. [16] developed an excellent application with energy awareness in mind.

This work presents designing an energy-aware MP3 player. They not only tackle the

hardware side of the devise, but rather the software side as well. They talked about the

two technologies that influenced the design of wearable computers and these are system-









on-chip (SOC) and System-In-Package (SIP). These two technologies are the reason that

both hardware and software solutions will need to be developed for energy-aware

wearable devices. In this work, they analyze a single-chip multimedia system to be used

in the wearable devise. They take into account the entire computing environment such as

hardware, packaging, and software design. They achieve low energy consumption by the

use of detailed statistical analysis of the energy consumption. They use an in-house

designed runtime energy estimation tool.

Additionally, Yuan et al. [39] investigated another multimedia application with

respect to power-awareness. They present a middleware framework for coordinating the

adaptation of multimedia application to the hardware resources. They have three goals in

mind when designing their system, the first of which is a soft real-time guarantee of

multimedia application deadline, the second is to have sufficient energy for multimedia

applications to finish their task, and the third is to waist as little energy as possible. To

meet the three goals the framework presented makes three useful contributions: the first

is to make multimedia applications make energy aware processor reservation. Secondly,

is to formally model adaptability of hardware, software, and user preference. Finally,

deliver sufficient processor and energy resources to the application and operate the

processor as slow as possible to save energy.

Tools for Low-Energy Design and Measurement

Power and energy estimation and measurement tools are an essential part to

developing solutions to the energy problem. As developing and later measuring the

power consumed must be accurate to gain benefits from any newly developed technique.









PowerScope

PowerScope allows the measuring and estimation of the energy consumed via

profiling [11]. By mapping energy consumption to program structures, it can be

determined which sections of a program consume more power. Also, the energy

consumed by procedures/subroutines within a process can be determined. By providing

this fine granularity of feedback, PowerScope allows one to focus on system components

that are responsible for the largest energy consumption. Experiments by the developers of

PowerScope yielded 46% reduction in total energy consumed.

The architecture of this system is composed of a data collection phase, and an

offline analysis phase. The data collection phase uses an Agilent Technologies'

(http://www.agilent.com) multi-meter connected by a GPIB cable to a data collection

computer running an energy monitor component. A profiling computer running an

application and a system monitor component is getting power by being connected to the

multi-meter for measurement

Offline analysis utilizes a third component (the energy analyzer) which will take

the results produced in the data collection phase by the system monitor, and the energy

monitor and correlate them to produce the energy profile of the application.

As in CPU profilers, where sections of code wasting CPU cycles can be discovered,

this profiler will allow for pinpointing the sections of code that consume lots of energy,

and therefore must be dealt with.

Derivatives of SimpleScalar

SimpleScalar is used for building and modeling applications to analyze program

performance, hardware and software co-verification (http://www.simplescalar.com). It is

a toolset that is used to develop modeling applications to simulate real programs running









on various modern microprocessors (such as Alpha, PISA, ARM, and x86) and also

various operating systems. It has simulators with different granularities (high and low

level). SimpleScalar is widely used in the research community. The toolset also includes

performance visualization tools, statistical analysis resources, and debug and verification

infrastructure.

The licensing model for SimpleScalar allows users to extend the tool set. As a

result the power management community took advantage of that to extend the tool set to

develop projects for measuring and estimating power consumed by applications. These

projects include the Power Analyzer Project at University of Michigan

(http://www.eecs.umich.edu/~tnm/power), and the Wattch project at Princeton University

(http://parapet.ee.princeton.edu).

Other Power-Estimation Techniques

In addition to previously mentioned tools, other research has been done to estimate

energy for certain applications, systems, and devices. Cignetti et al. [7] described an

energy model for the PalmTM. In this work they target challenges faced by the

programmers when developing energy efficient code.

Recent Work

Recently additional work has been done in the area of remote execution for energy

management. Flinn et al. [9] present a remote execution system for mobile clients

powered with a battery. The system they describe (Spectra) is a self-tuning system where

it monitors resources in the environment, and based on the resources the decision is made

to where the components of the application are executed. This approach, while useful and

attractive, it is a coarse-grained solution to energy management where opportunities to

execute code remotely are done at the application level as opposed to the basic program









block level. Additionally, it does add a certain level of overhead to the mobile device

where various monitors need to be installed on the device to monitor the various

resources. These monitors include, not only a battery monitor, and network monitors, but

they also include, a CPU monitor, and a file-cache monitor.

Grid Computing

When dealing with research that targets computation outsourcing via a distributed

computing system, you have to consider grid computing as it is a new model that has a

great potential in benefiting this type of research. In looking into grid computing [12, 23],

it looks like it would be the perfect match for this type of research as it utilizes things like

Java remote method invocation (RMI), remote procedure call (RPC). However, most grid

computing systems differ in an important aspect and that is the size of the problem being

solved. No matter how large a program that we are trying to optimize for energy is, it will

never be as large as the programs investigated within the grid computing domain, which

tend to be huge in nature and usually require longer time to execute. Additionally and

more importantly, the size of the middleware used within a grid computing environment

is quite large, and computationally intensive to be utilized on a mobile device for energy

considerations, as it will consume additional energy. That is why investigating mobile

agent (MA) systems is questionable and must be pursued with care, however RPC and

RMI are worth investigating, and actually they are considered for future research.

Another important difference between our work and grid computing is that in grid

computing, the geographic location of the resources is of no concern. However, in our

research the geographic location is quite important, as if outsourcing is to take place, the

mobile device must be located within a pervasive smart space in order to be able









communicate with the surrogate via the wireless network, so proximity is important in

our research.

While grid computing, in general, is not applicable as a whole computing

paradigm, a simplified version of its behavior can be adopted in this research. When

looking at the formal models for grid computing, we were able to recognize a formal

model from grid computing that was useful in defining our own model. Nemeth and

Sunderam [26] presented a formal model that is based on an abstract state machine

(ASM). Their model encompasses a general description of grid and distributed

computing. Their definition contains universes (sets) of resources, processes, users, task,

etc. It also contains a signature consisting of function names on these universes. Their

universes were: a set of processes (PROCESS), as set of applications (APPLICATION), a

set of users (USER), a a set of resources (RESOURCE), a set of nodes (NODE) in the

case of distributed computing but not for grid computing, a set of tasks within a process

(TASK), a set attributes (ATTR) for tasks, resources, and nodes along with a subset

describing architectures (ARCH), a set of locations (LOCATION), and a set of messages

(MESSAGE) for interaction between processes. Along with these sets, the symbols, true,

false, undef, =, and the Boolean operators are included.

The formal model presented, also included the functions that compose the signature

on the super-universe containing all the aforementioned universes [26]. The signature is

composed of the functions: app: PROCESS -* APPLICATION, user: PROCESS USER,

request: PROCESS xRESOURCE -- [true, false}, uses: PROCESS xRESOURCE --

[true, false}, mapped: PROCESS ->NODE, BelongTo: RESOURCE x NODE -{ [true,

false}, installed: TASK x NODE -{ [true, false}, attr: {RESOURCE, NODE, TASK} --









ATTR, arch: RESOURCE -* ARCH, location: RESOURCE -* LOCATION, compatible:

ATTR x ATTR [true, false), CanLogin: USER NODE {true, false}, CanUse: USER

x RESOURCE [true, false}, state: PROCESS -*[running, i ailing. receive waiting/,

from: MESSAGE -* PROCESS, to: MESSAGE -* PROCESS, expecting: PROCESS -

PROCESS + [any}, and event: TAST -* [req res, spawn, send, receive, terminate}.

Given the mentioned functions defined within the signature, they proceeded to

define the rules for conventional distributed systems. They also, showed the problems of

mapping conventional distributed systems rules into grid systems. Finally, they provided

the necessary modifications to formulate the mode for grid systems.














CHAPTER 3
STATEMENT OF THE PROBLEM

The problem of energy management has gained a lot of attention in the mobile and

pervasive computing research community. This is due to the increased reliance on mobile

devices by a wide spectrum of users. This increased reliance on mobile devices stemmed

from the increased capability of these devices. This argument was supported by Helal

[18] who gave a close look at the market for Java-enabled phones and PDAs from a

commercial standpoint to show that the capabilities of these devices are increasing, and

will continue to increase over time. J-Phone Communications (Japan) offered handsets

that are based on the J2ME technology, which include a built in camera, a color display,

and a 3-D graphics engine that displays images from different perspectives [18]. They

also presented plans to offer mobile phones capable of providing geographic information,

multimedia service (JPEG, and PNG graphics synchronized with sound). Helal [18] also

presented information about Nokia's plans to double its shipment of Java-enabled phones

between the years 2002, and 2003 which gives a direction of where the mobile

technology is headed. Helal [18] also hinted to the increased capabilities in PDAs as well

by indicating that the PDA Profile (PDAP) might be too limited for where PDAs are

headed and might not be able to accurately estimate the capability of high-end PDAs such

as the Compaq iPAQ. Enabling mobile phones and PDAs with Java is an indication that

the trend is to make these mobile devices capable of performing tasks usually performed

on desktop computers.









As the capabilities of mobile devices increases, their energy usage will increase,

and therefore will drain the battery quicker. A cellular phone or a PDA drains more

power when it is playing an MP3 song, taking a picture, decoding and image, or

recognizing a voice to voice-dial a phone number. The problem has been that the battery

technology has not kept up with the other technologies comprising a mobile computer

system such as PDAs and cellular phones.

Starner [32] gave a discussion of how much slower advances in battery technology

have been than those for the other mobile computer components (the discussion was

given for laptop computer, but information for wearable computers, PDAs and cellular

phones was deduced to be similar). Figure 3-1 provides a graph representing the

improvement in laptop technology from 1990-2001 [32]. As the graph indicates, CPU

speed has kept up with Moor's Law, but battery capacity has not. In fact the figure

suggest that the battery capacity is lagging behind. It was also stated that mobile phone

companies sell more batteries than they do phones as the lifetime of the phone is much

longer than that of the battery, and consumers usually must buy more than one battery to

be able make use of their mobile devices effectively. So, battery technology seems to be

the slowest technology for the components of mobile devices.

The continued increase in reliance and capability of mobile devices indicate that the

energy-saving problem is ongoing, and it needs to continue to be addressed on the long

run. Due to their capability, mobile device users are ranging from teenagers to the elderly

and they span a wide range of backgrounds and mobile device utilization. One of the

most widely used feature set of mobile devices is that dealing with multimedia

applications, especially among the youth. The young generation uses these devices to









play video games; take and edit pictures and videos and record and play sounds and

music on them. All of these applications consume significant energy. Not only does the

young generation use multimedia applications; think of the medical and dental

professions. They deal with X-rays, three-dimensional (3-D) magnetic resonance images

(MRI) and 3-D representation of the mouth (dental application) for determining the


1000 Disk capacity- --
o CPU speed
Available RAM a
Wireless transfer speed
100 Battery energy density -'-- -
S100 -





E10
.E ------- --- -
3 A











Figure 3-1. Improvements in laptop technology from 1990-2001.

location of a tooth implant. Other users could include a graphics designer working to

generate a 3-D model of a home, or any other scene for that matter. Voice and speech

recognition applications are known to have been extremely difficult to perform on a

mobile device due to the intensive computation involved.









Consider a fraction of the computation involved in one of the previously mentioned

application such as displaying an image representing a 3-D scene. The data involved in

describing a 3-D scene is small. The output of this computation is a 2-D image display of

the 3-D scene. However in order to get from one to the other, the amount of computation

is quite large. In our 3-D benchmark (experimental validation chapter), we displayed 3

spheres of different colors in space. In order to generate a 200x200 image representing

the scene, the PDA used ran for over 16 minutes to generate the scene, consuming

approximately 0.0.47 Amps (Including the operating system and the terminal application,

0.19 without. Given that the battery that we used is a 1700 mAh (milli-Amp Hour) or 1.7

Ah, it will take less than 4 executions of the application to drain the battery of the device.

Therefore, multimedia applications are very energy consuming, and must be handled

differently.

The emergence of pervasive computing as a byproduct of mobile and distributed

computing enabled researchers to envision and later develop new methodologies to

solving the energy problem. We observed that mobile users are usually located within

communication and energy rich surroundings. This is because of the presence of the

mobile user within a wireless network, which usually has other machines that are not

mobile (connected via a wire to the network), and thus do not suffer from the energy

dissipation problem. Additionally, we observed that there exists no link between the

mobile devices located within the wireless network and those non-mobile ones. We

concluded that, if a link can be established between the mobile and the non-mobile

devices, the mobile devices will be able to save energy by outsourcing some of their

computation to the non-mobile devices. The mobile devices will however pay a









communication cost to facilitation the outsourcing. This observation led to the

development of our methodology that utilizes pervasive smart spaces to outsource

computation to non-mobile devices that do not suffer from the energy dissipation

problem.

This methodology is considered pervasive due to the fact that, at runtime, the

application intelligently makes the decision between local and remote execution by

comparing the energy cost of computation vs. the energy cost of communication. The

ability of the application to make such a decision is quite a challenge. Hence, the problem

that is being solved in this research is to make ordinary applications, developed by non-

energy-aware developers, able to make such an intelligent decision at runtime. Therefore,

this approach takes as input a program in its high-level representation (source code), and

transform it via a compilation process into two versions, a client version and a server

version, where the client is able to send requests to the server to be able to do some

computation for the client by receiving data and sending back results (only the modified

data). To be able to generate these two versions of the program, a priori knowledge of the

runtime behavior of the program needs to be analyzed. Therefore, the area of real-time

systems is a very suitable area for utilizing in making such an analysis and determination,

especially the ability to calculate the number of loop iterations. In addition, the ability to

detect basic program blocks (most likely loops) at both the high-level and the low-level

must be accomplished to determine both the communication cost involved in outsourcing

a basic program block, and the energy cost of executing a single loop iteration.

To facilitate the runtime decision-making ability for the client, additional

functionality has to be added to the mobile device to allow for the outsourcing to take









place when applicable in a pervasive environment. This functionality includes a network

monitor and a battery monitor similar to those introduced by Flinn et al. [9], however

their implementation is much simpler due to the compile-time code augmentation. The

network monitor will be able to find and locate available surrogate machines that are

running an instance of the server. Additionally, a battery monitor needs to be

implemented to monitor the status of the battery. The simplicity of these two monitors

results in a negligible additional power consumption by the system, and will be overcome

by the energy saving from the modified application.














CHAPTER 4
OVERVIEW OF THE APPROACH

To solve the problem stated in the previous section, the solution has to be

composed of two parts. The first part of the solution is done at compile-time as an

optimization technique at the high-level source code. The second part of the solution

must provide the necessary support to the outcome of the first phase. This is due to the

fact that the outcome resulting from the compile-time phase is a different formation than

that initially developed.

Overview of the Compile-Time Solution

We have introduced this work as part of our fine-grain approach to power-aware

computing [2]. First at compile-time, an assumption has been made that the source code

has been tested and verified in its original form. Although that is done, this solution still

validates the source code syntactically to make sure that no inadvertent errors were

introduced along the way. In addition to syntax checking, the source code is also

disassembled and the outcome of this process is an assembly representation using the

mnemonic representation of each instruction of the target architecture. At this point,

information about the high-level source code and the low-level instructions will become

available for the optimization technique part of this contributed research.

The next step is to recognize basic program blocks (mainly loops) in both the

source code and the assembly code, and simply match them. Recognizing loops at the

high-level representation of the source code will result in the ability to collect all the data

involved in the computation of the loop, and that will yield the energy cost of









communication for sending all the data involved in the calculation out, and receiving only

the data that changes (L-Values). Also, using the technique mentioned by Healy et al.

[17], the number of iterations for each loop is calculated. As for the assembly code, the

loops are recognized to determine the instructions involved in each loop, which will yield

the entire energy cost of executing a single iteration of the each loop. In addition to

instructions, at compile-time, we recognized whole library functions such as those

belonging to the math library, and we added the value of their energy cost to the cost of

the loop in which they are executed. This, along with the metric calculated before to find

out the number of each loop's iteration construct a good estimate of the total cost of the

local execution of each loop.

Before calculating the total cost of communication and the total cost of each loop's

computation, experiments were done to find out the cost of communicating a single unit

of data (a byte), and the cost for executing each machine instruction for the target

architecture. As for calculating the cost of communicating a single byte, a client/server

application was tested with multiple sizes of data to communicate between two machines,

and the measurement for this was recorded and averaged. As far as each instruction's

energy cost, a similar approach to that presented by Tiwari et al. [35] was utilized where

each supported instruction is isolated via high-level code implementation, and executed

multiple times within a loop and the final result is averaged based on the number of

instructions used (we used 100 instructions within a loop executing 100 million times). In

addition to testing machine instructions and verifying their cost, we tested pre-existing

library function and verified their energy cost in a similar manner to the individual

machine instructions.









Overview of the Runtime Support

To support the ability to outsource code, the application must be able to run in one

of two modes: normal mode, or energy-saving mode. So, when an application starts, it

will have to get some information based on the resources that are available. If the battery

is susceptible to be drained quickly, then the application needs to run in energy-saving

mode, the user also has control over this. However, if the user decides to run in normal

mode, then the application should not worry about computation outsourcing.

In order for the application to be able to make the right decision, it has to contact

the battery monitor at startup. The battery monitor would have already determined if

energy saving is available via outsourcing (this decisions is based on user preference

also). Additionally, the battery monitor will contact the network monitor to check if the

devices is actually connected to a network and that network contains surrogate servers. If

so, then it will run in energy-saving mode listing the appropriate surrogate available for

the application to utilize. This monitor is also similar to, but much simpler than, those

discussed by Flinn and Satyanarayanan [9] and by Gu et al. [15].

The work done by Flinn et al. [9] suggests that the cost of these monitors is "non-

negligible". This is true in their case, as a lot of the intelligence to execute code remotely

is done at runtime as opposed to compile-time, and that is why their approach is a coarse-

grained approach to energy management. However in our approach, while may utilize an

idea presented by Flinn and Satyanarayanan [9] and by Gu et al. [15], the solution is

much simpler and that is because the battery monitor is a straightforward inquiry to

operating system's advance power management (APM). As far as the network monitor is

concerned, it will only be invoked if an energy-saving mode of operation is decided

(mainly as an outcome of the battery monitor). Therefore, the cost is negligible for these









two monitors. Implementation of the battery monitor was as easy as looking at a single

file containing information about the battery at certain increments of time. As for the

network monitor, several approaches can be investigated, the simplest of which was

proposed by Gu et al. [15] and it is based on wireless broadcast for discovering

surrogates.

Implementation Setup

The implementation was done on an Intel' Xscale which is an integral part of the

Intel' PCA. We utilized the Sharp Zaurus SL-5600 (http://www.sharpusa.com), which

contains an Intel' Xscale PXA-250 processor, and is running Linux, as the mobile

device. Installed on the Zaurus, was a low-power Socket Communications' (Newark,

California) (http://www.socketcom.com) low-power wireless LAN card. The outsourcing

server is an Intel' x86 machine running RedHat Linux 7.2.

Developing applications on the Zaurus was achieved using Metrowerks (Austin

Texas) Codewarrior for the Sharp Zaurus (http://www.metrowerks.com). This software

comes with a packaged executable to run on the mobile device only during development

to be able to debug and/or execute the application on the Zaurus from a Microsoft

WindowsTM where Codewarrior is installed. The software is called MetroTRK (Target

Resident Kernel). MetroTRK is only used to transfer executables to the Zaurus via our

wireless network at the Harris Mobile Computing Laboratory at the University of Florida.

The implementation supported only a subset of the C programming language, and

handled the most popular assembly instructions, which mainly access memory, and

handle the arithmetic and logic operations within the architecture. Although limited,

supporting the particular subset of C we chose was adequate to develop meaningful and

useful application to prove our concept.














CHAPTER 5
COMPUTATION OUTSOURCING FRAMEWORK

Outsourcing computation is not a new terminology here. However, the motivation

behind outsourcing the computation to a remote server, and the approach under which we

are outsourcing the computation is the contribution here. Our goal from this research is to

show that an intelligent runtime decision can be made to decide if it is better to execute a

section of code locally on the mobile device, or would it be more energy-beneficial to

send its data to a remote server, and get the results back.

Overview

The overall framework for outsourcing is described in Figure 5-1. The idea is that a

server machine accessible via a wireless network can serve as a surrogate server for a

host of mobile devices such as handhelds, PDAs and laptop computers. This server at

runtime will receive requests from client programs running on any of these devices for

outsourcing code to the server.

The code that is in charge of making this decision is completely transparent to the

programmer. All the programmer is required to do is to compile the code to optimize for

energy. This will result in two version of the program being generated which the

programmer will eventually have to compile and install. We believe that this is not a

burden on the programmer in any way, and it is not a requirement for the programmer to

have any knowledge of energy requirements/constraints.

Once an application is compiled, and two versions have been generated (a server

version and a client version), and they are installed on their respective machines, the user











can then execute a client application on the mobile device. This client application

executes normally until it reaches a section of code that has been designated as outsource-

able (having the potential for outsourcing), this is what we call the outsourcing candidate.

Once this section is reached, then the intelligent code that was inserted at compile-time is

executed to make the outsourcing decision. As a matter of fact, the candidate code will

not be executed (locally or remotely) until the decision making code is executed.

Hand held computer running an outsourceable
mobile client application

Laptop computer running
an outsourceable mobile client application








SWireless Network ucn at 000
0 00 ; D --p00
com. Link 000 0 00
0ink 0 000
0000 000
0000 000
0000 000
0000 000
10000 000

Remote server running server programs for
Hand held computer running an outsourceable the various mobile applications
mobile client application

PDA running an outsourceable mobile
client application


Figure 5-1. Framework for computation outsourcing at runtime

In figure 5-2, an illustration of the outsourcing mechanism at runtime is given. The

client runs on the mobile device, and once it reaches an outsource-able section of code, it

determines if it is more energy beneficial to outsource or is it more beneficial to execute

locally. If the determination is made to outsource, then it will send the data to the server

and wait for the results back, otherwise it will continue to execute locally until it reaches

the next available outsource-able section of code. At all times, the server running on the









surrogate machine is waiting for requests from client programs. Once it services the

client's request it goes back to waiting for client requests again, which occur once a

candidate section of code decides to outsource its computation.


Figure 5-2. Steps for executing a client program under the outsourcing framework

The decision to outsource a section of code is not an arbitrary decision. The mobile

machine user must configure it to determine if outsourcing is desirable in the first place.

Therefore, there is a battery and a network monitor running on the client machine that









will help in making this determination. The battery monitor will run and ask the user to

determine the outsourcing policy that the user chooses. Once that is determined, then the

network monitor gets involved to determine the feasibility of outsourcing (if there is no

network connection to a surrogate, then outsourcing will not occur. Once the feasibility is

determined, then applications can be run in energy-saving mode, otherwise, they will run

in normal mode.

Using our model, we envision the development and creation of an entity called a

computation service provider (CSP). Different mobile users would subscribe to the CSP

in order to service their energy-needs. The subscription will be by registering a copy of

the server of the energy-aware application with the CSP. Whenever the user is within the

proximity of (in the pervasive smart space containing) the surrogate machine containing

the server code, it would be possible for the client to outsource code to the server located

on the surrogate. The outsourcing takes place by the client communicating its data to the

server, let the server process the data, and then the client will get the results back.

Formal Model

Nemeth and Sunderam [26] presented a formal approach for defining the

functionality of a grid system. Their approach started by defining distributed systems and

showed how a grid system differs from the classic distributed system environment. Our

model is a much more simplistic model than that they presented. Our model has a limited

number of resources, and a limited number of processes. The resources in our model are

the wireless network (WiFi) and the surrogate device. Our two processes are represented

by the client and by the server versions of the original code. The model presented is

based on an abstract state machine (ASM).









In looking at their model, we realized that their model encompasses a general

description of grid and distributed systems. Our model is a simplified representation of

theirs. In our model, we define the process universe as PROCESS = {client, server}, the

resource universe as RESOURCE = {wireless net, surrogate}, and the location universe

as LOCATION = i li//in-range, out-of-range}. We use the same functions used in the

grid and distributed computing domain, and add two of our own functions which are:

execCost: TASK -* VALUE, and comCost: TASK -* VALUE, where execCost is a

function that produces the value of the energy consumed by a specific task of a process.

Similarly, the comCost produces the value of the energy consumed by communicating the

data for a specific task of a process.

As far as the functions that we use from grid and distributed computing are

concerned, we use the same exact definition presented by Nemeth and Sunderam [26].

The following functions are defined:

* user: PROCESS -* USER
* request: PROCESS x RESOURCE -{ [true, false}
* uses: PROCESS x RESOURCE -{ [true, false}
* loc: RESOURCE LOCATION
* CanUse :USER x RESOURCE -{ [true, false}
* state: PROCESS -{ [running-normal, running-energy-saving, receive-i uili//in,', we
modified this function to fit our execution framework.
* from: MESSAGE -* PROCESS
* to: MESSAGE -* PROCESS
* event: TASK -{ [req-res, send, receive, terminate}

Upon defining the above functions, and universe sets, the rules for defining our

system as a simplified grid computing system can clearly be defined. We redefined the

resource selection rule (figure 5-3), the state transition rule (figure 5-4), the send rule for

both the client and the server (figure 5-5), and the receive rule for both the client and the

server (figure 5-6).





































Figure 5-3. Resource selection rule


Figure 5-4 State transition rule


The resource selection rule:
state(client) := running-normal
state(server) := receive-waiting
if loc(wireless net) = within-range &
request(client, wireless net) = true) &
CanUse(user(client), wireless net) then
request(client, wireless net) :=false
uses(client, wireless net) := true
if request(client, surrogate) &
CanUse(client, surrogate) then
request(client, surrogate) := false
uses(client, surrogate) := true
else
uses(client, surrogate) := false
request(client, surrogate) := true
endif
else
uses(client, wireless net) := false
uses(client, surrogate) :=false
endif


The state transition rule:
if uses(client, surrogate) &
uses(client, wireless net) then
state(client) := running-energy-saving
else
state(client) := running-normal












Client send:
if state(client) = running-energy-saving &
commCost(task(client)) <
execCost(task(client)) then
if event (task(client)) = send(server) then
extend MESSAGE by i\ ii iihi
to(msg) := server
from(msg) := client
end extend
state (client) := receive-waiting

Server send:
if event(task(server)) = send(client) then
extend MESSAGE by i/\y ii iihi
to(msg) := client
from(msg) := server
end extend
state(server) := receive-waiting




Figure 5-5. Send rules for the client and the server



Client receive:
ifstate(client) = receive-waiting &
event(task(client) = receive(server) then
ifto(msg) = client &from (msg) = server
then
MESSAGE(msg): =false
state (client) := running-energy-saving

Server receive:
if state(server) = receive-waiting &
event(task(server) = receive(client) then
ifto(msg) = server &from (msg) = client
then
MESSAGE(msg): =false
state (server) := running


Figure 5-6. Receive rules for the client and the server














CHAPTER 6
COMPILE-TIME STRATEGY

Our compiler optimization technique for low energy analyzes a source program at

the three different levels of representation (high, intermediate, and low). At the high-

level, we collect information about the data involved in each loop. At the intermediate

level we utilize an algorithm described By Healy et al. [17] to find out the number of loop

iterations. The reason this is an intermediate level analysis is because they analyze the

register transfer list (RTL) [6] representation of the source code. At the low level, we

determine the machine instructions generated by an assembler to determine which

instructions are getting executed within each loop. All of the three levels of source code

analysis are embedded in our algorithm.

Overview

This new compilation technique utilizes pre-existing utilities such as the gcc

compiler and Metrowerks"' assembler and compiler. We first pass the code that needs to

be compiled to the gcc compiler to make sure that it is syntactically correct, once that is

done, we remove the resulting machine code as it will not be needed. Then we pass the

same source code through our optimization preprocessing, along with the assembly code

generated from passing the original source code through the Metrowerks"' assembler, to

generate the two versions of the code, the client and the server. Once the client and the

server codes are generated, then the server is compiled for the target server machine, and

the client is compiled for the mobile device. In our environment, the client is compiled

using Metrowerks"' Codewarrrior, and the server is compiled on a Linux machine using









the gcc compiler. Figure 6-1 shows the process for optimizing a C program using our

technique.


Figure 6-1. Overview of compilation and optimization process









Energy-Optimization Process

Our optimization technique modifies the high-level code (the C source code). The

input to this process is a file containing the source code, and a file containing the

assembly representation of this source code. Using the source code program, we

determine the number of loop iterations, the size of the data involved in the loop

execution. Then we determine, using the assembly representation (low-level

representation) of the program, the instructions involved, and we calculate the total

energy cost for all of the instructions using the energy cost of each individual instruction.

In addition to the machine instructions we handle also library functions (such as the math

and standard libraries) called within each loop. The energy cost of each individual

instruction was calculated using a methodology similar to that presented by Tiwari et al.

[35], and we give an explanation of this in our experimental validation. We calculate the

energy cost for library functions in a similar manner to that of the machine instructions.

Also, we had already measured the cost of transferring one byte of data using our

wireless card. Given all of these metrics, we were able to insert socket code within our

source program and conditional statements to determine at runtime if it is more energy-

beneficial to outsource a candidate section of code (basic program block/loop) or to

execute it locally on the mobile device.

Calculating the Number of Loop Iterations

Healy et al. [17] developed a useful utility for predicting the worst-case execution

time (WCET) of a program. They provided us with the software that will accomplish this

task for C programs. Their algorithm is part of implementing a static timing analyzer for

analyzing real-time systems, as predicting the number of loop iteration is essential for

analyzing real-time systems. Their approach automatically bounds the number of loop









iterations. They handle nested loops, and loops with multiple exits. Their methodology is

implemented by analyzing the register transfer list (RTL) [6], which is an intermediate

representation of the source program.

First, they identify the branches that can affect the number of times the loop

executes. Secondly, they calculate when each branch can change its direction. Third, they

determine when each iteration branch can be reached. Finally, they calculate the

minimum and maximum number of each loop's iteration. If the loop invariant is a non-

constant, for the purposes of the timing analyzer they are implementing, they allowed the

user to input the minimum and maximum values for this variable, and that is not needed

for our compiler optimization technique, as our methodology supports non-constant loop

invariant as, at runtime, its value will be known, and we can use a formula involving the

invariant to be multiplied by the cost of executing each loop once (the energy cost) which

gives us as a formula that is easily evaluated at runtime to determine if a section of code

should be outsourced.

Their implementation is integrated in the implementation of the Very Portable C

Compiler (vpcc) introduced by Benitez and Davidson [6]. The input to the modified vpcc

(we will refer to it as vpcc) is a source program with a ".c" extension, and the output is a

set of files, only one of which is of interest to us, and that is the file with the same name

as the source program, except with a ".inf' extension (the INF file). The INF file contains

information about the maximum and the minimum number of loop iteration, and we are

only interested in the maximum number of iteration in our research and that is because

we do not want to under-estimate, we want to outsource with a high degree of certainty

that a benefit will be gained from outsourcing. In figure 6-2, we give a sample C program










that can be compiled with vpcc, and by passing certain switches to it an INF file (Figure

6-3) will be generated.


Figure 6-2. Example of C program passed as input to vpcc


Figure 6-3. Resulting INF file for the program in Figure 6-2. The boldfaced expression
represents the maximum number of loop iterations


main()
{
inti, j;

for(i = 0; i < 100 -j ; i = i + 3) {;}

}


-3
main
!loop 0 0 11 -1 -1 1 2 3 4 -1 4 -1
!loop 1 1 -4 r[10] 0 r[9] 3 s -2 (100-.lj-2)/3 (100-.lj-2)/3 -1 -1 3 -1 3 -1
! block 1 lines 5-5 preds -1 succs 2 4 -1
makesunknown 3 -1
doms 1 -1
1 82 4 0 8 7 () 1024 7 (100) 8 4 (%ol)
1 90 4 0 8 4 (%ol) 8 4 (%o3) 8 4 (%ol)
1 9071 1024 7 ()8 4 (%ol)8 7()
1 62 4 2 2048 4 () 0 0() 0 0 ()
1 824087()1024 7 () 8 4 (%o2)
! block 2 lines 5-5 preds 1 -1 succs 3 -1
makesunknown 3 -1
doms 1 2 -1
2 32 4 0 8 4 (%o2) 1024 7 (3) 8 4 (%o2)
! block 3 lines 5-5 preds 3 2 -1 succs 4 3 -1
doms 1 2 3 -1
3904 1 84 (%o2)8 4 (%ol)8 7 ()
3 74 4 2 2048 4 () 0 0 () 0 0 ()
3 32 4 0 8 4 (%o2) 1024 7 (3) 8 4 (%o2)
! block 4 lines 5-5 preds 1 3 -1 succs -1
doms 1 4 -1
4 80 4 0 128 4 ()8 7 () 0 0()
4 154000()00()00()









Loop Data and Iterations Acquisition

The first stage of our technique is to recognize the maximal basic program blocks

(most likely these blocks will be loops). These basic program blocks (loops) will

constitute the opportunity for optimization (candidate code for outsource-ability). Once

these basic program blocks are recognized at the high-level, then we collect all the data

elements associated with them, and determine the beginning and end file positions of

these loops. Additionally, we pass the relevant sections of the original source code to the

program that calculates the number of iterations for each loop.

The first part of this stage is to implement a parser-like module (we call it the

pseudo-parser) to recognize basic program blocks, collect the data used within each loop,

and identify what variables are R-valued (do not change), and what variables are L-

valued (change). We did not need to implement a full parser here as the syntax has

already been checked before entering this stage of the algorithm. In figure 6-4 we present

a high-level algorithm for our pseudo-parser. In addition to acquiring the data elements

involved in the calculation of each loop, we also determine which C library functions

have been called to determine the contribution of their energy cost to the execution of the

loop. Additionally, we determine if certain loops are not outsource-able. All loops that

involve Input/Output (I/O) routines are designated as non-outsource-able. This holds true

also for those loops that include nested loops with I/O functions.

The second part of this stage is to figure out the number of loop iterations and

associate each number with each loop calculated by the pseudo-parser. The algorithm to

do this is a very simple one. This algorithm is implemented using a very simple parser

that parses only the lines that contain the minimum and maximum iterations for each loop

in the INF file. Once it extracts the expression representing the maximum number of











iterations, it cleans it up by removing any extra characters such as those in figure 6-3

where the expression is "(100-. 1j-2) 3". This particular expression is unique also as it

contains a which could be problematic as if everything used in the expression is an

integer then at runtime, integer division might happen, and therefore after we remove the

substring ".1 _" from the expression and if we recognize the division operator, we insert

the typecasting "(double) right before it. Hence, the resulting expression is "(100 j -

2) /(double) 3 "


Pseudo-Parser
Input: AC program
Output: An array of loop data structures containing:
a. The beginning and ending file position of the loop
b. The variables and their sizes
c. The L-Valued variable and their sizes

get a token
nestLevel = 0
loopNumber = 0
initialize a stack of loops to empty
while there are tokens in the program do
if the token is a loop
loopNumber= loopNumber+ 1
nestLevel = nestLevel + 1
assign loopNumber to the loop just entered
assign nestLevel to the loop just entered
push the current loopNumber on the stack
else if the token is part of an expression including functions and structures
if nestLevel > 0
if you encounter a variable
if the variable is an L-value
insert the variable as an L-value in the loop on top of the stack
else
insert the variable as a variable in the loop on top of the stack
end if
end if
if you encounter and I/O function then
The loop becomes non-outsourceable.
end if
end if
else if you reached an end of a loop
nestLevel = nestLevel 1
pop the top of the stack
end if
get the next token
Done.


Figure 6-4. Algorithm to show implementation of the pseudo-parser









Calculating the Size of Loop Data

The next stage is to calculate the data size for each loop. In this stage, we examine

each variable involved in the loop, and based on the size of the variable in bytes

(including arrays), we add the value to our sum to calculate the size in bytes.

Additionally, if the variable is an L-value (changes), then we add the size of the variable

to our sum for the L-valued variable. This is a very important aspect of this algorithm, as

if the data does not change, we only need to communicate it to the server, and we do not

need it back, but if it changes, then we will expect it to be sent back to the mobile device.

This way, we can minimize the amount of communication needed. Additionally, we

check if the loop contains other loops, and if so, then we collect the variables of the

nested loops only if these variables have not already been collected by a parent loop. The

algorithm in figure 6-5 illustrates this procedure.

At this point, we have the data size for each loop which when multiplied by the cost

to send one byte of data gives us the total cost to send all the data within the loop added

to the cost to receive all the L-valued data within the same loop.

Identifying Loop Instructions and Total Loop Execution Cost

Using the assembly code representation of the source program, we can recognize

loops within the assembly code. The target architecture (Xscale) has a unique way of

identifying loops. Loops can be identified by three consecutive instructions, the first of

which is the compare instruction "cmp ", followed by a conditional branch "ble, bit, bge,

bgt, bne, beq ", followed by an unconditional branch. "b ". The unconditional branch is

quite useful in this regard as it sends the control outside of the loop, and all we have to do

is to go to that branch location, and find the other unconditional branch that completes the

loop and returns us back two instructions before the "cmp ". This way we are able to










Calculate Loop Data Size:
Input: An array of loop data structures
Output: The same array updated to include the data size
of each loop

for each loop in the program do

set the totalDataSize of the current loop to zero
set the total LvaluesSize of the current loop to zero
for each variable within the loop do
add the size of the variable to totalDataSize
done

for each L-valued variable within the loop do
add the size of the variable to totalLvaluesSize
done

for each loop nested within the current loop
for each variable in each nested loop
if the variable does exist in the upper loop
add its size to totalDataSize
end if
done

for each L- valued variable in each nested loop
if the variable does exist in the upper loop
add its size to totalLvaluesSize
end if
done
done
done

Figure 6-5. Algorithm to calculate loop data size once all of the variables in each loop
have been collected. This is only done for outsource-able loops.

identify or rather delimit where the assembly code for each loop starts and where it end.

However, when loops are nested, we need more information to be able to map loops at

the assembly level with those at the high level. The additional information needed is

available in the structure containing information about all the loops (we call it the

"loopdata" data structure). The information needed here is which loop is nested with

which loop, and that information was obtained via our pseudo-parser.









Once each loop was delimited, then it was just a matter of going through the

instructions that constitute the loop, and adding their pre-measured energy cost. In

addition to the cost of each instruction there is a cost for pipeline stalls. This cost was

obtained experimentally using multiple instruction sequences once the cost per

instruction was determined. Therefore, when we recognize that certain instructions

precede others (e.g., str before an ldr), we add the measured pipeline stall energy cost.

This calculation gave us the cost of a single execution of the loop. At this point, we have

all what we need to be able to produce the resulting client and server. The total loop

execution cost becomes a matter of multiplying the cost of a single execution by the

formula representing the number of loop iterations calculated before.

Insert Outsourcing Code

The implementation of this code was very large, but it was not difficult. As our

pseudo-parser generated for us information of where each loop begins and where it ends.

The location of where we need to generate the necessary C code to create a client/server

based application becomes a matter of inserting the necessary include files, and variable

declaration (we declared them globally). The outsourcing code is only inserted for those

loops that are flagged outsource-able.














CHAPTER 7
RUNTIME SUPPORT

Here we present the two monitors used as runtime support for the outsourcing

mechanism. The two monitors work together and they get executed based on the user

preference and the battery condition. The attractive property of these two monitors can be

summed in the fact that most of the time they are not consuming any energy. In fact, they

consume very little energy when they are doing any work. The way these two monitors

work depends on the user preference in the first place, and once that has been determined,

the condition of the battery of the mobile device takes control of the decision making

process. These two monitors will run on the client mobile device. In addition to these two

monitors, a server program will run on the surrogate device waiting for connections from

the client. The battery monitor, network monitor, and the server will together establish

the service detection within the wireless network.

Battery Monitor

The battery monitor gets executed either by the user or the operating system. This

is also a decision to be configured by the user. The user chooses if he/she desires to run in

energy-saving mode or in normal mode. If normal mode is selected, then nothing happens

and the monitor exits. Otherwise, if energy-saving mode is selected, then the battery

monitor will ask the user if the energy saving is to take place immediately, or it should

wait until the battery gets below a certain limit. If energy saving is to take place

immediately, then the battery monitor will immediately call the network monitor.

Otherwise, the battery monitor will sleep (consuming a very negligible amount of energy)









and periodically check the status of the battery by contacting the operating system.

Contacting the operating system is a very trivial matter as it will only look at a file called

"/proc/apm", and extract the remaining percentage of the battery. Once it reaches the

limit specified by the user, then will contact the network monitor that will complete the

task of setting up the device in an energy-saving mode.

Network Monitor and Surrogate Service Discovery Server

Once the network monitor is called, it will send out a broadcast that will be only

received by a network server that is providing any service for the client. This server will

be running on the surrogate machine that is to service the client. Once the server receives

the broadcast it then will establish a handshake with the client device and inform the

device of its name, and that it is ready for servicing the device and it supports

outsourcing. At that point, the network monitor will create a configuration file that is to

be opened by the application to determine if it would run in energy-saving mode, or

normal mode. If any type of error occurs on the way to creating this file, the file will not

be created, and hence there will be no energy-saving mode.

At runtime the client application will start running and checks if the energy-saving

file exists, and if so, then extract the information about the server from it, establish the

connection, and execute in energy-saving mode, otherwise, execute in normal mode.














CHAPTER 8
EXPERIMENTAL VALIDATION

Our measurements, and experiments were done in two stages with our platform

setup. The first stage was to estimate as accurately as possible the cost of each supported

machine instruction (assembly instruction). Secondly, the second stage is to measure the

cost of each benchmark, first without our optimization, and second with our optimization.

Setup

Our target architecture is an Intel' Xscale which is an integral part of the Intel'

PCA. We chose the Sharp Zaurus SL-5600, which contains an Intel' Xscale PXA-250

processor, and is running Linux, as our mobile device. Installed on the Zaurus, is a

Socket's" low-power wireless LAN card. The outsourcing server is an Intel' x86

machine running RedHat Linux 7.2.

Developing applications on the Zaurus was achieved using Metrowerks

Codewarrior for the Sharp Zaurus. This software comes with a packaged executable to

run on the mobile device only during development to be able to debug and/or execute the

application on the Zaurus from a Microsoft" WindowsTM where Codewarrior is installed.

The software is called MetroTRK (Target Resident Kernel). We use MetroTRK to

transfer executables to the Zaurus via our wireless network at the Harris Mobile

Computing Laboratory at the University of Florida (http://www.harris.cise.ufl.edu). This

Microsoft" WindowsTM machine on which Codewarrior is installed also happened to be

the same machine on which we record our measurements.









For measuring the energy consumed, we used an Agilent 34401A multi-meter

which was connected to a Microsoft WindowsTM 2000 desktop computer via an

IEEE488.1 General Purpose Interface Bus (GPIB) cable. Installed on the desktop is

Agilent's Intuilink plugin which works with Microsoft Word and Excel. We used

Excel because its plugin allows for multiple readings as opposed to Word's single

reading. The multi-meter has two J-hooks that were placed in series between the AC

adapter and the Zaurus to place the multi-meter in series to measure the total current

drawn by the Zaurus. The voltage coming from the AC adapter remained at a constant 5

volts. Therefore, the only two factors in the energy equation are the current drawn and the

time in seconds as energy is given by the equation:

E= V*I*T

Where E is the energy consumed, Vis the voltage, I is the current, and Tis the elapsed

time.

To measure the energy consumed by either a running process on the Zaurus, or by

data communication, we calculate the difference between the current drawn when the

Zaurus is idle, and when the Zaurus is either running a process or sending/receiving data.

To make this as accurate as possible, the only application that we ran on the Zaurus was

the Linux Terminal. We also turned off the light of the Zaurus LCD. These measures that

we took, and as our experiments show, resulted in a constant current drawn by the Zaurus

while idle, which gave us the ability to get good measurement with as little sampling

noise as possible. The multi-meter only allowed us to take samples at one tenth of a

second. Therefore, we had to execute code that consumes enough time to allow for as

accurate a measurement as can be obtained.









Instruction-Level Energy Cost Estimation

Before applying our optimization to source code, and besides knowing what

machine instructions were used, we had to know the energy cost for each machine

instruction involved. Our work targeted only a subset of instructions from the Xscale

architecture, which was sufficient to testing our benchmarks, and by no means is that a

limitation of our approach.

We used a methodology similar to that described by Tiwari et al. [35] to estimate

the cost of each instruction. The methodology suggested executing several instances of a

single machine instruction within a loop and average the energy consumed to obtain the

per instruction cost. Figure 8-1 shows a small C programs which when executed gives a

good estimate of the energy consumed by an empty for loop. Figure 8-2 shows a second

C program that gives a good estimate for the load instruction (LDR), which loads a

memory location into a register. Notice, to minimize the affect of the instructions that

execute the loop header (as the loop body is the several occurrences of the load

instruction) we used a register loop control variable. The ability to declare register

variable in a language like C made it very possible for us to be able to isolate single

instructions which enhanced the accuracy of our estimation.

In order to minimize the effects of the cache, and to better estimate inter-instruction

energy consumption, we developed additional benchmarks that include multiple instances

of a series of different instructions (e.g. Idr followed by str) and we observed their

behavior, and recording the amount of additional energy consumption recorded, and used

these metrics in our code to update the instruction-level energy consumption values

calculated before. In addition to minimizing the effect of caching the instructions, our










benchmarking of each instruction, also, estimated the effect of the pipeline data hazards

involved in the execution of each instruction.

Measuring Communication Cost

To measure communication cost per byte, we wrote small UNIX socket programs

that sent data back and forth to a server, and our results confirmed the card statistics that

were mentioned on the datasheet of our wireless LAN card. The data sheet suggests that

this card is active 90% of the time transmitting at 265 mA and receiving at 170 mA.



main
{
register int i;

for(i = 0; i < 100000000; i++)
{ ;}





Figure 8-1. The C program used to estimate the energy cost of executing an empty for
loop 100M times


Figure 8-2. The C program used to estimate the energy cost of the Idr instruction


mainO
{
register int i;
registerint x;
int a, b, c, d, e, f, g, h, m, n;


for(i = 0; i < 100000000; i++)
{
// repeat 10 time the following 10 statement
// each line represents an Idr instruction
x = a; x = f;
x = b; x = g;
x = c; x = h;
x = d; x = m;
x = e; x = n;
}
}









Simple Experimental Validation

To test the effect of our approach on energy, we implemented three different simple

benchmarks that span three different formations of data and execution complexity. The

first of which was the Fibonacci loop, which contains constant data, but it executes in

0(n) time. In other words, the size of the data remains constant, while the execution

changes with n, where n is the number to which we are trying to calculate the Fibonacci

number. Due to the sampling limitation of our multi-meter, we had to test this 3 times

using 3 large numbers to get more accurate results of our measurements. We performed

the testing using the numbers, 100000, 200000, and 300000. Another benchmark that we

used was a rectangular version of the bubble-sort loop which executes in 0(n2) and the

data size is linear. So, as the data size grows so will the computation complexity. We

sorted 10000, 20000, and 30000 integers. The last benchmark that we used was a square

matrix multiplication loop, which runs in 0(n3) where n is the number or rows and the

number of columns of each matrix. For matrix multiplication we used a 200x200, a

300x300, and 400x400 matrices.

Each one of the benchmarks was executed on the Zaurus before our optimization

and after our optimization. Table 8-1 gives the percentage of total energy saved by

outsourcing the loop that calculates the Fibonacci number. Table 8-2, shows the total

energy saved from outsourcing the square matrix multiplication loop. As can be

concluded from the tables, a significant amount of energy can be saved when

computationally expensive code can be outsourced.

One interesting observation in our testing was the bubble-sort loop. The smallest

energy saving was 98% for sorting 10,000 integers. This result is well expected due to the

fact that compared to the amount of computation involved in bubble-sort, the size of the









data is very negligible. That is not the case in the matrix multiplication and the Fibonacci

sequence calculation.

Table 8-1. Total energy saved by outsourcing the Fibonacci loop
Value of n Energy savings %
100000 60
200000 80
300000 87


Table 8-2. Total energy saved by outsourcing the matrix multiplication loop
Number of rows Energy savings %
200 73
300 80
400 88

Figures 8-3, 8-4, and 8-5 show comparison histograms between the local

executions of the benchmarks vs. their remote execution (outsourcing). The results were

generated while executing each version on the Zaurus while the Intuilink software is

recording the data into a spreadsheet.

Large-Scale Benchmark Validation

To show the benefits of our approach, a more realistic benchmark had to be

developed to show that this approach has more meaningful and potentially industry-

utilizable benefits. Some of the most computationally intensive computations are those

involved in generating an image representing a 3-D graphics scene. To generate a 3-D

graphics scene, the input and output of the program are extremely non-expensive

processes, as even more complex scenes can be described with a virtually small amount

of data. Also, the output is always a 2-D Image. But to get from a 3-D description of the

scene to a 2-D Image depicting the scene, a huge amount of calculation has to be done. In

this benchmark, the size of the data is O(n2) and the order of the computation is also

O(n2). However, the amount of constant calculation within each iteration, is huge














0.5-
E
-0.4-

S0.3-

0.2

0.1 -

0
SC)- 0- OO N M- 0 O O 1O 0 O O O C) M) C)
O C) C) C) C) C) C) C) C) (D i- '- '- '- '- '- '- '- CN CN



Sample Time Intervals (Seconds)

A


0.5
7 0.45-
E 0.4 -
0 0.35 -
0.3-
L 0.25L
0.2
S0.15 -
S0.1
0.05 -
0 -
o I Coo o O oO (o oo o ooN oCD N OO I OO r- CO C cO CL
CD CD CD (D (D (D D ,- ,- ,- ,- ,- ,- ,- (N (N (N (N (N (N cO cO rO rO



Sample Time Intervals (Seconds)

B



Figure 8-3. Local versus remote execution of 300 iterations of a Fibonacci loop
executing 300,000 times. A) The histogram showing the current drawn
during the time intervals listed for local execution. B) The histogram
showing the current drawn during the time intervals listed for remote
execution (longer spikes are for sending while the shorter ones are for
receiving). Voltage remains a constant 5 volts for both histograms.












0.5
0.45 -A
S 0.4
S0.35
0.3
0.25
0.2

L 0.15
S0.1
0.05
0
0 co r-: (D C' co 09 o CR r- (D U? t (N -'7 G co (q ir- U? N 0 9


OOOO00000OOO0000000
Soooooooooo 00 ooooooooooo00 oo0

Sample Time Intervals (Seconds)


0.7

K 0.6

< 0.5

S0.4-

a 0.3 -



0 0.1

0


O o O o OOO( C O ( C C) C) CDO) rCD) ) D ) -D C C) C-


000 0 0 0 0 000 0 0 0 0 000 0 0 0 0
Sample Time Intervals (Seconds)


0-
0 (0

9 9
00 C
00 C


B



Figure 8-4. Local versus remote execution of matrix multiplication of a 400x400 matrix.
A) The histogram showing the current drawn during the time intervals listed
for local execution. B) The histogram showing the current drawn during the
time intervals listed for remote execution. Voltage remains a constant 5
volts for both histograms. The energy saving is obvious in this benchmark.















0.5-






I 0.2-



0
0 0 0 M 0 00 0 00 (0 0 0- C 00 t 0 0\
0 C) Cf ) CO C.) CO t C) L)- CO C6) 0 fM CO LO CO CO -4 LO LO LO (D (D
0 N d LC) C D D C L C) C) C) 0 N C L -C (N C) 0) (N C) 0CD (C
000000000000000000000000


Sample Time Intervals (Seconds)

A



0.7

0.6
0
E 0.5

S0.4

a 0.3

~ 0.2

- 0.1

0


0 r Co o c 0i 0P 000 -0 --
N LO M M(CcO D i LO 6 Cj
C CDJ CD CODCD CD CD ) CD Dl O CD O
000000000000
888888888888
Sample Time Intervals (Seconds)


0 -

CD 0 0 C0
8888


Figure 8-5. Local versus remote execution of a bubble sort loop which sorts 50,000
integers. A) The histogram showing the current drawn during the time
intervals listed for local execution, and it took approximately 6 minutes. B)
The histogram showing the current drawn during the time intervals listed for
remote execution, and it took only 1.2 minutes. Voltage remains a constant 5
volts for both histograms.


0-
00
00
88


0 0
00
88








when compared to the amount of communicating each unit of data involved in the

computation. Our experimental results show a significant amount of energy saving for

generating a scene by ray-tracing 3 spheres of different sizes and colors in space to

generate 3 different images of 50x50, 100x100, and 200x200. Figure 8-6, shows the input

data passed to the ray-tracing process and figure 8-7 shows the image produced.

The input to the ray tracing application is quite a simple input composed of a few

floating point numbers. These numbers represent the description of the world composed

of: the observer, the light source, and the parameters describing three spheres in space.


Figure 8-6. Input file for the ray tracing application


0








Figure 8-7. The 2-D image representing the 3-D scene generated by the ray tracing
application for a 200x200 image









We did our experiments on various sizes of data and we generated a 50x50 image, a

100x100 image, and a 200x200 image for the same scene. The amount of computation

was so large that in all three cases, the computation was outsourced. Figures 8-8, 8-9, and

8-10 show the results of comparing local execution vs. remote execution for the 3 sizes of

images mentioned above for the same scene.

Other Measurements

To accurately, and comprehensively discuss our results, other metrics and overhead

costs had to be measured once our approach is applied and utilized. One of these metrics

is represented by the size of the executable. The size of the executable was impacted

greatly when it involved our first three benchmarks as they are considered small

programs. The increase in executable size was 7 times for sort and matrix multiplication

routines, and 3.5 times for the Fibonacci calculation. However, it was more encouraging

when we dealt with much bigger application (which is going to be the case in real life).

The size of the executable increased by only 40%.

In addition, we measured the total execution cost of the battery and network

monitors to realize that they consume very little energy when running, as the work

involved in their execution is very sporadic and very minimal. Figure 8-11 illustrates the

total energy consumed by the monitors (only when they are running), otherwise the cost

is near zero.

Compile time was another metric that we measured. It can easily be concluded that

the compile-time will at least double, as the result of our approach will generate two files

instead of one. However, in the process of generating two files, our energy optimization

process at compile-time consumed about 1-2 additional seconds for the simple

benchmarks, and about 5-7 seconds for the more complex benchmark of ray tracing.







62






0.5
;0.45 -
E 0.4
S0.35 -
F 0.3
2 0.25
o 0.2-
0.15 -
0.1




0 C) 0 0 0- (N 0- (NC C- CD CO CD C CC C CCD CD 0

000000000000000000000000
000000000000000000000000
Sample Time Intervals (Seconds)

A



0.6
U,
cL 0.5
E
S0.4 -

L 0.3-

0.2

0.1-

0
SC) M O 14- CO 1- ) C) N CO LO D 0 C) O 1- ( () O CO)
0 0 0 0 0 0 0C 0 (D C CD C C C 0- < C0 ) 0 Co 0 0 )0 ( C C


Sample Time Intervals (Seconds)

B


Figure 8-8. Local versus remote execution for ray tracing three spheres in space and
generating a 50x50 image. A) The histogram showing the current drawn
during the time intervals listed for local execution (approximately 66 Joules
in 1.2 minutes). B) The histogram showing the current drawn during the
time intervals listed for remote execution (approximately 2.5 Joules in 2
seconds). Voltage remains a constant 5 volts for both histograms.







63




0.6

S0.5

E
S0.4

2 0.3
0
S 0.2

0.1

0 --------------------------------------------
0 C)) LC) (D CO 00) a) LC) M- (D co r G) 2 (D co D -N (L O

0 0 C) 0 0 0 C 0C0 0) 0 ( 0 0 C) C D D C D 0 CD LCD CO
0 (N C) 0 LC) 0 (N 0 LC0 0 (N ) C C 0 (D 0D D D 0- (
000000 00 00 0000 0000000000
000000000000000000000000

Sample Time Intervals (Seconds)

A



0.6

0.5

E
S0.4

2 0.3

f 0.2

0.1

0


O CD O o i- C OO oC C o CD i- C O OOC- ( co C) ) COO- (N oD
0 0 0 0 0 0 0 -- -- -- -- ---- (N (N (N (N (N (N D CD CD CD
oooooooooooooooooooooooo


Sample Time Intervals (Seconds)

B


Figure 8-9. Local versus remote execution for ray tracing three spheres in space and
generating a 100x100 image. A) The histogram showing the current drawn
during the time intervals listed for local execution (approximately 240
Joules in 4 minutes). B) The histogram showing the current drawn during
the time intervals listed for remote execution (approximately 2.5 Joules in
approximately 3 seconds). Voltage remains a constant 5 volts for both
histograms.







64



0.6
U,
cL 0.5
E
S0.4

S0.3

S0.2

S0.1-

0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 C) O 1 LO W [- LO 1- 0N M< M
00000000000000000 0000000
0000000000000000 0000000
Sample Time Intervals (Seconds)

A


0.6
U,
L 0.5 -
E
0.4 -

2 0.3

0.2

S0.1-

0
0 (N 1 0 0 0 0- (O I W 0 CO 0 N 140 LO N COO
0 0 0 0 0 0 0---- -----------N N -N (N (N (N 0(n-



Sample Time Intervals (Seconds)


B


Figure 8-10. Local versus remote execution for ray tracing three spheres in space and
generating a 200x200 image. A) The histogram showing the current drawn
during the time intervals listed for local execution (approximately 1020
Joules in 17 minutes). B) The histogram showing the current drawn during
the time intervals listed for remote execution (approximately 2.5 Joules in
approximately 6 seconds). Voltage remains a constant 5 volts for both
histograms.










0.5
0.45
C 0.4 -
E 0.35 -
0.3

0.25
C 0.15
0.15
0 0.05
0

throughout th e experiments.
^- ^- ^- ^b:(b- 0- ^- ^- ^- b- ^- ^- N- p- tK b' IK^













In order to show the normalization of energy cost for an outsourced application, the

processor speed of the server has to be taken into consideration. This is due to the

overhead energy (base energy) spent between the time it sends the data to the server and

thigure time it receives the results back. This energy spent is taken into consideration as it

lengthens the time the processor is active and therefore it can not go into sleep mode

(which saves energy). The amount of energy spent during sleep mode is actually quite

negligible and irrelevant to the calculation of the energy consumed. Therefore, the energy

consumed by an application depends on the base energy as well as the communication

cost.









In our measurements, the base energy consumed by the mobile device was

calculated at 0.25 Amps 5 Volts (server execution time). Add to it the communication

energy cost, the result will be the total energy consumed by each loop executing

remotely.

First of all, a model has to be devised for the local execution of each program.

Assume that we are consuming PowerBase watts without running any applications on the

mobile device. Our mobile device executes MipsPower MIPS/watt. Therefore, if our

application requires AppInst Million instructions to execute, and it takes TLocal seconds

to execute. That means that the total energy consumed during the execution of the

application is given by the following equation:


Local = Tlocal x PowerBase + AppInst
MipsPower)


Given that Tlocal = AppInst where the processor is capable of mips MIPS (should
mips

be a number very close to the processor speed). The equation for calculating the energy

consumed locally can be rewritten as follows:



mips mips x MipsPower

Therefore, the Energy spent in the execution of the application locally depends not

only on the time, and power, but it also depends on the speed of the processor. So the

larger the amount of mips is, the smaller the amount of energy consumed.

At this point we can calculate the energy consumed by executing the loop (or the

entire application if it has only one loop) remotely. Prior to its execution, the mobile

device is consuming PowerSleep watts, However, after the start of the execution, the









device will spend PowerAwake watts as it will wake up. Therefore, the amount of

PowerBase = PowerSleep + PowerAwake. As the value of PowerSleep is not a factor in

the energy consumed by the application (the mobile device is spending it regardless of

what it does), the value of PowerBase will be equal to the value of PowerAwake. The

server device can execute in Tremote seconds. The energy cost of sending and receiving

the data is a function of the time to communicate the data, as well as the power cost for

the communications. Therefore, we consider ts, and tr, as the time it takes to send and

receive respectively, and ps, andpr are the power cost for sending and receiving the data.

Therefore, the energy cost for outsourcing an application is given by the following

equation:

Eremote = ts x ps + tr x pr + Tremote x PowerBase

Just like in the case of calculating the time it takes the client to execute a program,

the same calculation can be applied to the server. The server is running RAppInst million

instructions to execute, and it is running on rmips MIPS (or MHz). Therefore Tremote can

RApplnst
be given by: Tremote = rpps and as a result, the equation to calculate the energy
rmips

consumed by executing the application remotely is as follows:


Eremote = ts x ps + tr x pr + RAppInst x PowerBase
rmips

Therefore, the lower the values of ts, ps, tr, pr, and the higher the value rmips (the

faster the server machine is), the less the amount of base energy consumed waiting for the

results. As a result, faster communication (lower ts, and tr) as well as faster server

machines makes a difference when outsourcing computations.









In order to model our normalization of the measurements according to the

processor's speed, we applied the above equation for calculating Eremote, both,

theoretically, and experimentally using our benchmarks. Our 3-D rendering loop was

benchmarked using 3 different image sizes (50x50, 100x100, and 200x200). As a result,

you can see that the execution time for the applications will quadruple from one run to

the other. Also, we used the same size data when running each benchmark (200x200

image) to handle (using the same executable) all benchmarks. Therefore, in effect, we

modeled slowing down the processor to 14 of its speed with each run, while keeping the

communication cost the same. The processor that we used for the server was a 500 MHz

Pentium III (this is also the value of the rmips), which means with the 50x50 image, it

will have the same speed, with the 100x100 image will effectively model a 125 MHz

processor, and with the 200x200 it will be a 31.5 MHz processor. We also estimated, the

number of instructions executed for the 50x50 application, and they came out to be

approximately 125 million instructions. The estimation was done based on the number of

assembly instructions generated as well as the possible logic of the program, so it is only

an estimation as opposed to an actual count. The transmission time for sending and

receiving was modeled to take 1 second each time with the sending consuming .265

Amps, and the receiving consuming .170 Amps according to the specification of the WiFi

card. The value of base current lost was estimated at .25 Amps. The voltage is assumed to

be a constant 5 Volts throughout the process

Theoretically, the value of the energy would be calculated using the values given

by the specifications of the network card (.265, and .170) for communication, and the

processor speeds for each benchmark (500, 125, and 31.5). Which means that, given the










125 million instructions application, the total amount of base energy consumed would be

multiplied by 1/4, 1 and 4 seconds respectively for each run and that means it should

quadruple.

When we measured the values of the aforementioned metrics, we came up with

values that are very close to the ones mentioned above. The run for the 50x50 image took

approximately 14 of a second, the 100x100 took approximately 1.2 seconds, and the

200x200 took approximately 4.2 seconds. The transmission time was approximately 1

minute for each the send and receive. The current cost for the send was approximately

0.255 Amps, and for the receive it was approximately .170 Amps. Figure 8-12 shows a

comparison between the theoretical and the experimental measurements of the total

energy consumed for each processor speed. Figure 8-13 shows a comparison between the

theoretical and experimental measurements of the amount of base energy consumed. Our

results match those acquired by the normalized analysis for the remote execution of the

application.


11 -
10 -
100

8 -
7 79
= 6 -- Theoretical
0 5 --- Experimental
S4-




0 -
500 125 31.5
Processor Speed (MHz)



Figure 8-12. Comparison between theoretical and experimental measurements of the total
energy consumed.
























- -* Theoretical
-u Experimental


6
"3 -

5
0
-a
S4
E

a 3
0

g2

LU


m

0


Figure 8-13. The amount of base energy consumed based on the server processor's speed


1 11 12 13 14 15 16 17 18 19 2
Time to Send to Server (seconds)


1 11 12 13 14 15 16 17 18 19 2
Power to Send to Server (watts)


1 11 12 13 14 15 16 17 18 19 2
Time to Receive from Server (seconds)


B


p -
p -
p


1 11 12 13 14 15 16 17 18 19 2
Power to Receive from Server (watts)


Figure 8-14 The effect of changing the communication metrics on the total energy

consumed. A) The effect of the send time. B) The effect of the receive time.

C) The effect of the send power. D) The effect of the receive power.


500 125 31.5

Processor Speed (MHz)


- p S


S -






71


We have mentioned before that the value of Eremote also depends on the power as

well as the time consumed during the communication (ts, ps, tr, pr). In this work, we

model the theoretical behavior of the effect of these communication metrics on the

energy cost of the equation. We will assume that the processor speed is 500 MHz (500

MIPS), the base current drawn is 0.25 Amps with the same application of an

approximate 125 million instructions. Figure 8-14 shows the effect of changing the

metrics ts, ps, tr andpr.














CHAPTER 9
CONCLUSION AND FUTURE WORK

Our experimental evaluations showed that computation outsourcing within a

pervasive computing smart space has a great potential in energy management. By

exporting CPU processing responsibilities into the network, the mobile device was able

to deliver the expected functionality while consuming less energy and lasting for a longer

period of time. Therefore, communication should not be viewed as a drain on the battery,

but as an opportunity to save energy.

We found that the research done in the domain of real-time systems is quite useful

as its aim is always knowing as much as possible about an application before runtime

(e.g., compile-time). The reason for that in real-time systems is to execute programs to

finish within a deadline. That means that the more we know about program behavior

before execution the better the program can be tuned to either meet a deadline in the case

of real-time systems, or consume less energy in our case of mobile and pervasive

computing. However we believe that the limitations of the utilized methodology which

are represented in the inability to handle non-counter-based loops, such as those loops

that are pointer-based, logical-expression based, and some non-rectangular nested loops,

raise the need to investigate other computer science disciplines, specifically automated

software verification.

While the work done in the area of automated verification for loop invariant

generation is not quite mature enough, it has a great promise to be combined with the

utilized real-time systems methodology to generate accurate estimates for the number of









loop iterations. We will investigate the research done in this area to determine the extent

of its applicability in our work. Specifically, we will look at the work done by Pasareanu

and Visser [28]. We believe that, when combined with the methodology we utilized for

determining the loop iterations, the area of loop invariant generation will be quite

beneficial to our research.

As part of our future work, we will continue the implementation of the client/server

version that we have already started. However, this implementation will contain

additional support for dynamic memory allocation and subroutine-level computation

outsourcing. In addition, additional work needs to be done to evaluate a more accurate

communication cost. This part will be an ongoing part of the research as we utilize

additional outsourcing techniques. In addition, we will continue to identify specific

applications that will benefit from our approach which will be a refinement of defining

categories of application areas that we believe will benefit from this approach.

To handle multiple platforms, targeting languages such as Java and C++ will be

necessary. This will require porting a compiler to handle both languages while

augmenting with the code that we obtained to calculate the number of loop iterations.

Additionally, as we will support additional types of basic program blocks we will build

the compiler support that will handle estimating total execution cost of these basic

program blocks such as additional types of loops than those supported so far, recursive

functions (inherently these are loops), library linked functions whose energy consumption

is predefined, and functions defined within the code. Additionally, we will investigate in

the case of non-Java languages, the possibility to be able to cross-compile them for the

most popular mobile devices.









In this research, we assumed that if a basic program block (loop) contained an I/O

operation, it is determined as non-outsource-able. However, in our future work we will be

looking at opportunities where I/O operations may be performed elsewhere. For input

operations that involve files, if the file exists elsewhere, then it could be energy-

beneficial to read the file on a remote machine, and utilize its contents remotely.

Similarly, if producing the output elsewhere and basically all we are interested in is a

display of this output which maybe cheaper than displaying the output locally, then that

is another opportunity that needs to be investigated.

Identification of the applications, which contain the computationally expensive

basic program blocks, is an essential part of this research. We will investigate and

research the different types of basic program blocks that fall into this category and test

them and provide them as benchmarks for our research. In addition, we will build the

support for other useful applications that will benefit from our approach.

Java remote method invocation (RMI), and remote procedure call (RPC) based

systems will also be applicable in our research where we will let the system make the

necessary data communication according to its policies and that will benefit our approach

especially at the level of outsourcing specific functions, and subroutines. These two

systems are utilized in grid computing environments.

We will weigh the benefits of every outsourcing mechanism with each type of basic

program block we investigate, and investigate which mechanism allows us to save more

energy with a basic program block. This will lead to a hybrid approach where a single

application may contain one, two, or three outsourcing mechanisms.






75


Also, as far as service discovery is concerned, we will investigate a lighter version

of some of the well-know service discovery systems lik UPnP (http://www.upnp.org),

and Jini (http://www.sun.com/software/jini). We believe that these systems can be

utilized without spending a large amount of energy as the intelligence has already been

built in the modified program.
















LIST OF REFERENCES


[1] A. Abukmail, A. Helal, Power Awareness and Management Techniques, in: M.
Ilyas, I. Mahgoub (Eds.), Mobile Computing Handbook, CRC Press, Boca Raton,
FL, 2004.

[2] A. Abukmail, A. Helal, A Pervasive Internet Approach to Fine-Grain Power-Aware
Computing, in: IEEE/IPSJ International Symposium on Applications and the
Internet, Phoenix, Arizona, January 2006.

[3] A. Aho, M. Ganapathi, S. Tjiang, Code Generation Using Tree Matching and
Dynamic Programming, ACM Transactions on Programming Languages and
Systems 11 (4) (1989) 491-516.

[4] N. Bellas, I. Hajj, C. Polychronopoulos, G. Stamoulis, Architectural and Compiler
Support for Energy Reduction in the Memory Hierarchy of High performance
Microprocessors, in: International Symposium on Low Power Electronics and
Design, Monterey, CA, Februaryl998.

[5] F. Bellosa, The Benefits of Event-Driven Energy Accounting in Power-Sensitive
Systems, in: 9th ACM SIGOPS European Workshop, Kolding, Denmark,
September 2000.

[6] M. Benitez, J. Davidson, A Portable Global Optimizer and Linker, in: ACM
Conference on Programming Language Design and Implementation, Atlanta,
Georgia, July 1988.

[7] T. Cignetti, K. Komarov, C. Schlatter Ellis, Energy estimation tools for the Palm,
in: 3rd ACM International Workshop on Modeling, Analysis and Simulation of
Wireless and Mobile Systems, Boston, MA, August 2000.

[8] C. Ellis, The Case for Higher Level Power Management, in: 7th Workshop on Hot
Topics in Operating Systems, Rio Rico, AZ, March 1999.

[9] J. Flinn, S. Park, M. Satyanarayannan, Balancing Performance, Energy, and
Quality in Pervasive Computing, in: 22nd International Conference on Distributed
Computing Systems, Vienna, Austria, July 2002.

[10] J. Flinn, M. Satyanarayanan, Energy-aware adaptation for mobile applications, in:
17th ACM Symposium on Operating System Principles, Kiawah Island, SC,
December 1999.









[11] J. Flinn, M. Satyanarayanan, PowerScope: A Tool for Profiling the Energy Usage
of Mobile Applications, in: 2nd IEEE Workshop on Mobile Computing Systems
and Applications, New Orleans, LA, February 1999.

[12] I. Foster, C. Kesselman, S. Tuecke, The anatomy of the Grid, in: T. Berma, G. Fox,
A. Hey (Eds), Grid Computing: Making the Global Infrastructure a Reality, Wiley
Series in Communication Networking and Distributed Systems, John Wiley and
Sons Ltd, West Sussex, England, 2003.

[13] C. Fraser, D. Hanson, T. Proebsting, Engineering Efficient Code Generators using
Tree Matching and Dynamic Programming, Technical Report No. CS-TR-386-92,
Princeton University, August 1992.

[14] C. Gebotys, Low Energy Memory and Register Allocation Using Network Flow,
in: 34th Conference on Design Automation, Anaheim, California, June 1997.

[15] X. Gu, A. Messer, I. Greenberg, D. Milojicic, K. Nahrstedt, Adaptive Offloading
for Pervasive Computing, IEEE Pervasive Computing 3 (3) (2004) 66-73.

[16] J. Haid, W. Schogler, M. Manninger, Design of an Energy-Aware MP3-Player for
Wearable Computing, in: Telecommunication and Mobile Computing Conference,
Graz, Austria, March 2003.

[17] C. Healy, M. Sjodin, V. Rustagi, D. Whalley, Bounding Loop Iterations for Timing
Analysis, in: IEEE Real-Time Technology and Applications Symposium, Denver,
CO, June 1998.

[18] A. Helal, Pervasive Java Part II, IEEE Pervasive Computing 1 (2) (2002) 85-89.

[19] C-H. Hsu, U. Kremer, M. Hsiao, Compiler-Directed Dynamic Frequency and
Voltage Scheduling, in: 1st International Workshop on Power-Aware Computer
Systems, Cambridge, MA, November 2000.

[20] M. Kandemir, N. Vijaykrishnan, M. Irwin, W. Ye, Influence of compiler
optimizations on system power, in: 37th Conference on Design Automation, Los
Angeles, CA, June 2000.

[21] R. Kravets, P. Krishnan. Power Management Techniques for Mobile
Communication, in: 4th ACM/IEEE International Conference on Mobile
Computing and Networking, Dallas, Texas, October 1998.

[22] U. Kremer, J. Hicks, J. Rehg, Compiler-Directed Remote Task Execution for
Power Management, in: Workshop on Compilers and Operating Systems for Low
Power, Philadelphia, PA, October 2000.









[23] C. Lee, D. Talia, Grid programming models: current tools, issues and directions, in:
T. Berma, G. Fox, A. Hey (Eds), Grid Computing: Making the Global
Infrastructure a Reality, Wiley Series in Communication Networking and
Distributed Systems, John Wiley and Sons Ltd, West Sussex, England, 2003.

[24] R. Loy, A. Helal, Active Mode Power Management in Mobile Devices, in: 5th
World Multi-Conference on Systematics, Cybernetics, and Informatics, Orlando,
FL, July 2001.

[25] D. Marculescu, Profile-Driven Code Execution for Low Power Dissipation, in:
International Symposium on Low Power Electronics and Design, Rapallo, Italy,
July 2000.

[26] Z. Nemeth, V. Sunderam, A Formal Framework for Defining Grid Systems, in: 2nd
IEEE/ACM International Symposium on Cluster Computing and the Grid, Berlin,
Germany, May 2002.

[27] J. Palm, J. Eliot, B. Moss, When to use a compilation service?, in: ACM Joint
Conference on Language Compilers and Tools for Embedded Systems and
Software and Compilers for Embedded Systems, Berlin, Germany, June 2002.

[28] C. Pasareanu, W. Visser, Verification of Java Programs Using Symbolic Execution
and Invariant Generation, in: 11th International SPIN Workshop on Model
Checking of Software, Barcelona, Spain, April 2004.

[29] A. Rudenko, P. Reiher, G. Popek, G. Kuenning, The Remote Processing
Framework for Portable Computer Power Saving, in: ACM Symposium on Applied
Computing, San Antonio, TX, February 1999.

[30] M. Satyanarayanan, Pervasive Computing: Vision and Challenges, IEEE Personal
Communication 8 (4) (2001) 10-17.

[31] G. Smit, P. Havinga, A Survey of energy saving techniques for mobile computers,
Internal Technical Report, University of Twente, Enschede, Netherlands, 1997.

[32] T. Starner, Powerful Change Part 1: Batteries and Possible Alternatives for the
Mobile Market, IEEE Pervasive Computing 2 (4) (2003) 86-88.

[33] C-L. Su, C-Y. Tsui, A. Despain, Low Power Architecture Design and Compilation
Techniques for High-Performance Processors, Technical Report No. ACAL-TR-
94-01, University of Southern California. February 1994.

[34] V. Tiwari, S. Malik, A. Wolfe, Compilation Techniques for Low Energy: An
Overview, in: International Symposium on Low Power Electronics and Design, San
Diego, CA, October 1994.

[35] V. Tiwari, S. Malik, A. Wolfe, T. Lee, Instruction Level Power Analysis and
Optimization of Software, VLSI Signal Processing Systems 13 (2) (1996) 1-18.






79


[36] M. Velluri, L. John, Is Compiling for Performance == Compiling for Power?, in:
5th Annual Workshop on Interaction between Compilers and Computer
Architecture, Monterrey, Mexico, January 2001.

[37] E. Witchel, S. Larsen, C. Ananian, K. Asanovic, Direct Addressed Caches for
Reduced Power Consumption, in: 34th Annual International Symposium on
Microarchitecture, Austin, Texas, December 2001.

[38] T. Yeh, D. Long, S. Brandt. Conserving Battery Energy through Making Fewer
Incorrect File Predictions, in: IEEE Workshop on Power Management for Real-
Time and Embedded Systems, Taipei, Taiwan, May 2001.

[39] W. Yuan, K. Nahrstedt, X. Gu, Coordinating Energy-Aware Adaptation of
Multimedia Applications and Hardware Resources, in: 9th ACM Multimedia
Middleware Workshop, Ottawa, Canada, October 2003.















BIOGRAPHICAL SKETCH

Ahmed Abukmail received a Bachelor of Science degree in computer and

information science from the Ohio State University (Columbus, Ohio) in 1993. In 1995,

he received a Master of Science degree in computer and information science from the

University of Florida. During the same year, he was admitted to the Ph.D. program.

In 1997, Ahmed left the University of Florida for 5 years to pursue a career in

industry. In 2003, he returned to the University of Florida to complete his Ph.D. degree.

His major research is in the areas of mobile and pervasive computing with particular

interest in energy and power-aware computing, programming languages, compilers, and

distributed systems.