{\rtf1{\fonttbl {\f2 Times New Roman;} {\f3 Arial;} {\f4 Times New Roman;} {\f5 Times New Roman

Bold;} {\f6 Times New Roman;} {\f7 Times New Roman Bold;} {\f8 Times New Roman Bold;} {\f9 Times New Roman Bold;} {\f10 Symbol;} {\f11 Arial;} {\f12 Arial Bold;} {\f13 Times New Roman;} {\f14 Arial Italic;} {\f15 Times New Roman Bold Italic;} {\f16 Arial;} {\f17 Times New Roman Bold;} {\f18 Arial Bold;} {\f19 Verdana Bold;} {\f20 Verdana Bold;} {\f21 Verdana;} {\f22 Verdana Bold;} {\f1000000 Times New Roman;} }{\colortbl; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red255\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red255\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; \red0\green0\blue0; }\viewkind1\viewscale100\margl0\margr0\margt0\margb0\deftab80\dntblnsbdb\expshrt n\paperw11900\paperh16840\pard\sb0\sl-240{\bkmkstart Pg1}{\bkmkend Pg1}\par\pard \qj \li3854\sb0\sl-360\slmult0 \par\pard\qj\li3854\sb0\sl-360\slmult0 \par\pard\ qj\li3854\sb0\sl-360\slmult0 \par\pard\qj\li3854\sb0\sl-360\slmult0 \par\pard\qj \li3854\sb0\sl-360\slmult0 \par\pard\qj\li3854\ri1810\sb62\sl-360\slmult0 \up0 \ expndtw0\charscalex97 \ul0\nosupersub\cf2\f3\fs31 An Adaptive Opportunistic Rout ing Scheme \up0 \expndtw0\charscalex98 for Wireless Ad-hoc Networks \par\pard\ql \li2452\sb0\sl-264\slmult0 \par\pard\ql\li2452\sb0\sl-264\slmult0 \par\pard\ql\ li2452\sb12\sl-264\slmult0 \up0 \expndtw-3\charscalex100 \ul0\nosupersub\cf4\f5\ fs23 ABSTRACT \par\pard\qj \li2452\sb0\sl-312\slmult0 \par\pard\qj\li2452\ri1522 \sb265\sl-312\slmult0\fi700 \up0 \expndtw-1\charscalex100 \ul0\nosupersub\cf5\f6 \fs27 A distributed adaptive opportunistic routing scheme for multi-\line \up0 \ expndtw0\charscalex105 hop wireless ad-hoc networks is proposed. The proposed sc heme \line \up0 \expndtw-3\charscalex100 utilizes a reinforcement learning frame work to opportunistically route \line \up0 \expndtw-1\charscalex100 the packets

even in the absence of reliable knowledge about channel \line \up0 \expndtw-5\ch arscalex100 statistics and network model. This scheme is shown to be optimal wit h \line \up0 \expndtw0\charscalex108 respect to an expected average per packet r eward criterion. The \line \up0 \expndtw-1\charscalex100 proposed routing scheme jointly addresses the issues of learning and \line \up0 \expndtw0\charscalex103 routing in an opportunistic context, where the network structure is \line \up0 \expndtw0\charscalex100 characterized by the transmission success probabilities. In particular, \line \up0 \expndtw0\charscalex101 this learning framework leads to a stochastic routing scheme which \line \up0 \expndtw-5\charscalex100 optima lly \u8220?explores\u8221? and \u8220?exploits\u8221? the opportunities in the n etwork. \par\pard\ql \li2452\sb332\sl-356\slmult0 \up0 \expndtw0\charscalex95 \u l0\nosupersub\cf6\f7\fs31 Existing System \par\pard\qj \li2452\ri1524\sb182\sl-3 11\slmult0\fi700 \up0 \expndtw0\charscalex102 \ul0\nosupersub\cf5\f6\fs27 Such f ixed path schemes fail to take advantages of broadcast \up0 \expndtw-1\charscale x100 nature and opportunities provided by the wireless medium and result \up0 \e xpndtw-1\charscalex100 in unnecessary packet retransmissions. The opportuni stic routing \up0 \expndtw-1\charscalex100 decisions, in contrast, are made in an online manner by choosing the \up0 \expndtw-4\charscalex100 next relay based on the actual transmission outcomes as well as a rank \up0 \expndtw0\charscalex1 02 ordering of neighboring nodes. Opportunistic routing mitigates the \up0 \expn dtw0\charscalex103 impact of poor wireless links by exploiting the broadcast nat ure of \up0 \expndtw-5\charscalex100 wireless transmissions and the path diversi ty. \par\pard\ql \li2452\sb0\sl-356\slmult0 \par\pard\ql\li2452\sb256\sl-356\slm ult0 \up0 \expndtw0\charscalex95 \ul0\nosupersub\cf6\f7\fs31 Proposed System \pa r\pard\qj \li2452\sb0\sl-313\slmult0 \par\pard\qj\li2452\ri1524\sb147\sl-313\slm ult0\fi700 \up0 \expndtw-4\charscalex100 \ul0\nosupersub\cf5\f6\fs27 We investig ate the problem of opportunistically routing packets \line \up0 \expndtw-4\chars calex100 in a wireless multi-hop network when zero or erroneous knowledge of \li ne \up0 \expndtw-2\charscalex100 transmission success probabilities and network topology is available. \line \up0 \expndtw0\charscalex101 Using a reinforcement learning framework, we propose an adaptive \line \up0 \expndtw0\charscalex102 op portunistic routing algorithm which minimizes the expected \line \up0 \ex pndtw0\charscalex100 average per packet cost for routing a packet from a source node to a \line \up0 \expndtw-5\charscalex100 destination. \par\pard\sect\sectd\ fs24\paperw11900\paperh16840\pard\sb0\sl-240{\bkmkstart Pg2}{\bkmkend Pg2}\par\p ard\qj \li2452\sb0\sl-313\slmult0 \par\pard\qj\li2452\sb0\sl-313\slmult0 \par\pa rd\qj\li2452\sb0\sl-313\slmult0 \par\pard\qj\li2452\sb0\sl-313\slmult0 \par\pard \qj\li2452\sb0\sl-313\slmult0 \par\pard\qj\li2452\sb0\sl-313\slmult0 \par\pard\q j\li2452\ri1524\sb283\sl-313\slmult0\fi700 \up0 \expndtw0\charscalex101 \ul0\nos upersub\cf5\f6\fs27 Our proposed reinforcement learning framework allows for a \ line \up0 \expndtw0\charscalex108 low complexity, low overhead, distribu ted asynchronous \line \up0 \expndtw-1\charscalex100 implementation. The most significant characteristics of the proposed \line \up0 \expndtw-5\charscalex100 solution are: \par\pard\ql \li2875\sb30\sl-310\slmult0 \up0 \expndtw-1\charscale x100 \ul0\nosupersub\cf9\f10\fs27 \u-3913?\ul0\nosupersub\cf10\f11\fs27 \ul0\no supersub\cf5\f6\fs27 It is oblivious to the initial knowledge of network. \par\p ard\ql \li2875\ri1523\sb2\sl-320\slmult0\tx3225 \up0 \expndtw0\charscalex100 \ul 0\nosupersub\cf9\f10\fs27 \u-3913?\ul0\nosupersub\cf10\f11\fs27 \ul0\nosupersub \cf5\f6\fs27 It is distributed; each node makes decisions based on its belief \l ine\tab \up0 \expndtw-5\charscalex100 using the information obtained from its ne ighbors. \par\pard\ql \li2875\ri1523\sb20\sl-320\slmult0\tx3225 \up0 \expndtw0\c harscalex102 \ul0\nosupersub\cf9\f10\fs27 \u-3913?\ul0\nosupersub\cf10\f11\fs27 \ul0\nosupersub\cf5\f6\fs27 It is asynchronous; at any time any subset of nodes can update \line\tab \up0 \expndtw-2\charscalex100 their corresponding beliefs. \par\pard\ql \li2452\sb311\sl-356\slmult0 \up0 \expndtw0\charscalex95 \ul0\nosu persub\cf6\f7\fs31 IMPLEMENTATION \par\pard\qj \li2452\ri1492\sb185\sl-355\slmul t0\fi700 \up0 \expndtw0\charscalex105 \ul0\nosupersub\cf3\f4\fs31 Implementation is the stage of the project when the \up0 \expndtw0\charscalex95 theoretical de sign is turned out into a working system. Thus it \up0 \expndtw0\charscalex98 ca n be considered to be the most critical stage in achieving a \up0 \expndtw0\char

scalex96 successful new system and in giving the user, confidence that \up0 \exp ndtw0\charscalex97 the new system will work and be effective. \par\pard\qj \li24 52\sb0\sl-360\slmult0 \par\pard\qj\li2452\ri1492\sb1\sl-360\slmult0\fi700 \up0 \ expndtw0\charscalex102 The implementation stage involves careful planning, \up0 \expndtw0\charscalex102 investigation of the existing system and it\u8217?s cons traints on \up0 \expndtw0\charscalex96 implementation, designing of methods to a chieve changeover \up0 \expndtw0\charscalex97 and evaluation of changeover metho ds. \par\pard\ql \li2452\sb366\sl-402\slmult0 \up0 \expndtw0\charscalex95 \ul0\n osupersub\cf8\f9\fs35 Modules: \par\pard\ql \li2452\sb356\sl-356\slmult0 \up0 \e xpndtw0\charscalex95 \ul0\nosupersub\cf12\f13\fs31 Algorithm used d-Adaptor \par \pard\ql \li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb52\sl-310\slmult0 \up0 \expndtw-5\charscalex100 \ul0\nosupersub\cf7\f8\fs27 Initialization stage \par\ pard\qj \li2452\sb0\sl-313\slmult0 \par\pard\qj\li2452\sb0\sl-313\slmult0 \par\p ard\qj\li2452\ri1524\sb2\sl-313\slmult0\fi700 \up0 \expndtw-4\charscalex100 \ul0 \nosupersub\cf5\f6\fs27 We consider the problem of routing packets from a source node \up0 \expndtw-2\charscalex100 o to a destination node d in a wireless ad-h oc network of d + 1 nodes \up0 \expndtw0\charscalex108 denoted by the set _ = fo ; 1; 2; : : : ; dg. The time is slotted and \up0 \expndtw-3\charscalex100 indexe d by n _ 0 (this assumption is not technically critical and is only \up0 \expndt w0\charscalex110 assumed for ease of exposition). A packet indexed by m _ 1 is \ up0 \expndtw-1\charscalex100 generated at the source node o at time _m s accordi ng to an arbitrary \up0 \expndtw-5\charscalex100 distribution with rate _ > 0. \ par\pard\sect\sectd\fs24\paperw11900\paperh16840\pard\sb0\sl-240{\bkmkstart Pg3} {\bkmkend Pg3}\par\pard\ql \li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb0\sl -310\slmult0 \par\pard\ql\li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb0\sl-3 10\slmult0 \par\pard\ql\li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb0\sl-310 \slmult0 \par\pard\ql\li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb0\sl-310\s lmult0 \par\pard\ql\li2452\sb3\sl-310\slmult0 \up0 \expndtw-2\charscalex100 \ul0 \nosupersub\cf7\f8\fs27 Transmission Stage \par\pard\qj \li2452\ri1524\sb266\sl315\slmult0\fi700 \up0 \expndtw0\charscalex100 \ul0\nosupersub\cf5\f6\fs27 We as sume a fixed transmission cost ci > 0 is incurred upon a \line \up0 \expndtw0\ch arscalex100 transmission from node i. Transmission cost ci can be considered to \line \up0 \expndtw-2\charscalex100 model the amount of energy used for transmis sion, the expected time \line \up0 \expndtw-1\charscalex100 to transmit a given packet, or the hop count when the cost is equal to \line \up0 \expndtw-5\charsca lex100 unity. \par\pard\ql \li2452\sb0\sl-310\slmult0 \par\pard\ql\li2452\sb0\sl -310\slmult0 \par\pard\ql\li2452\sb170\sl-310\slmult0 \up0 \expndtw-5\charscalex 100 \ul0\nosupersub\cf7\f8\fs27 Acknowledgement Message Passing \par\pard\qj \li 2452\ri1524\sb306\sl-315\slmult0\fi1022 \up0 \expndtw0\charscalex101 \ul0\nosupe rsub\cf5\f6\fs27 We discriminate amongst the termination events as \ line \up0 \expndtw0\charscalex108 follows:We assume that upon the termination of a packet at the \line \up0 \expndtw-1\charscalex100 destination (successful del ivery of a packet to the destination) a fixed \line \up0 \expndtw-1\charscalex10 0 and given positive reward R is obtained, while no reward is obtained \line \up 0 \expndtw-4\charscalex100 if the packet is terminated (dropped) before it reach es the destination. \par\pard\ql \li2452\sb310\sl-310\slmult0 \up0 \expndtw-5\ch arscalex100 \ul0\nosupersub\cf7\f8\fs27 Relay Stage \par\pard\qj \li2452\ri1524\ sb307\sl-314\slmult0\fi700 \up0 \expndtw0\charscalex110 \ul0\nosupersub\cf5\f6\f s27 Given a successful transmission from node i to the set of \line \up0 \expndt w0\charscalex102 neighbor nodes S, the next (possibly randomized) routing decisi on \line \up0 \expndtw-3\charscalex100 includes 1) retransmission by node i, 2) relaying the packet by a node \line \up0 \expndtw0\charscalex100 j 2 S, or 3) dr opping the packet all together. If node j is selected as a \line \up0 \expndtw-1 \charscalex100 relay, then it transmits the packet at the next slot, while other nodes k \line \up0 \expndtw-1\charscalex100 6= j; k 2 S, expunge that packet. W e define the termination event for \line \up0 \expndtw0\charscalex108 packet m t o be the event that packet m is either received by the \line \up0 \expndtw-5\cha rscalex100 destination or is dropped by a relay before reaching the destination. \par\pard\sect\sectd\fs24\paperw11900\paperh16840\pard\sb0\sl-240{\bkmkstart Pg 4}{\bkmkend Pg4}\par\pard\ql \li1751\sb0\sl-356\slmult0 \par\pard\ql\li1751\sb0\

sl-356\slmult0 \par\pard\ql\li1751\sb0\sl-356\slmult0 \par\pard\ql\li1751\sb0\sl -356\slmult0 \par\pard\ql\li1751\sb0\sl-356\slmult0 \par\pard\ql\li1751\sb85\sl356\slmult0 \up0 \expndtw0\charscalex97 \ul0\nosupersub\cf11\f12\fs31 H/W System Configuration:-\par\pard\li2272\sb0\sl-264\slmult0\par\pard\li2272\sb0\sl-264\s lmult0\par\pard\li2272\sb39\sl-264\slmult0\fi0\tx4388 \up0 \expndtw-2\charscalex 100 \ul0\nosupersub\cf17\f18\fs23 Processor\tab \up0 \expndtw0\charscalex102 Pentium -III\par\pard\li2272\sb0\sl-310\slmult0\par\pard\li2272\sb17\sl-310\sl mult0\fi180\tx5301\tx5661 \up0 \expndtw-3\charscalex100 \ul0\nosupersub\cf5\f6\f s27 Speed\tab \up0 \expndtw-3\charscalex100 -\tab \up0 \expndtw-2\charscalex100 1.1 Ghz\par\pard\li2272\sb0\sl-310\slmult0\par\pard\li2272\sb8\sl-310\slmult0\fi 180\tx5323\tx5685 \up0 \expndtw-2\charscalex100 RAM\tab \up0 \expndtw-3\charscal ex100 -\tab \up0 \expndtw-2\charscalex100 256 MB(min)\par\pard\li2272\sb0\sl-31 0\slmult0\par\pard\li2272\sb7\sl-310\slmult0\fi180\tx5354\tx5649 \up0 \expndtw-2 \charscalex100 Hard Disk\tab \up0 \expndtw-3\charscalex100 -\tab \up0 \expndtw-3 \charscalex100 20 GB\par\pard\li2272\sb0\sl-310\slmult0\par\pard\li2272\sb4\sl-3 10\slmult0\fi180\tx5347\tx5709 \up0 \expndtw-2\charscalex100 Floppy Drive\tab \u p0 \expndtw-3\charscalex100 -\tab \up0 \expndtw-2\charscalex100 1.44 MB\par\pard \li2272\sb0\sl-310\slmult0\par\pard\li2272\sb9\sl-310\slmult0\fi180\tx5347 \up0 \expndtw-2\charscalex100 Key Board\tab \up0 \expndtw-2\charscalex100 Standa rd Windows Keyboard\par\pard\li2272\sb0\sl-310\slmult0\par\pard\li2272\sb6\sl-31 0\slmult0\fi180\tx5378 \up0 \expndtw-2\charscalex100 Mouse\tab \up0 \expndtw-2\c harscalex100 Two or Three Button Mouse\par\pard\li2272\sb0\sl-310\slmult0\p ar\pard\li2272\sb6\sl-310\slmult0\fi180\tx5390 \up0 \expndtw-2\charscalex100 Mon itor\tab \up0 \expndtw-2\charscalex100 SVGA\par\pard\li2272\sb0\sl-310\slmu lt0\par\pard\li2272\sb33\sl-310\slmult0\fi180 \up0 \expndtw-2\charscalex100 \ul0 \nosupersub\cf21\f22\fs27\ul Software Requirements:\par\pard\li2272\sb161\sl-310 \slmult0\fi881\tx3854 \up0 \expndtw-3\charscalex100 \ul0\nosupersub\cf5\f6\fs27 \u8226?\tab \up0 \expndtw-2\charscalex100 Operating system :- Windows XP Profes sional\par\pard\li2272\sb4\sl-310\slmult0\fi881\tx3854\tx5853 \up0 \expndtw-3\ch arscalex100 \u8226?\tab \up0 \expndtw-3\charscalex100 JDK\tab \up0 \expndtw-3\ch arscalex100 :-1.5/ 1.6 and above\par\pard\li2272\sb2\sl-310\slmult0\fi881\tx3854 \tx5255 \up0 \expndtw-3\charscalex100 \u8226?\tab \up0 \expndtw-2\charscalex100 Front End\tab \up0 \expndtw-2\charscalex100 :- JAVA, Swing(JFC),\par\pard\sect\ sectd\fs24}