International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013

ISSN: 2231-2803 http://www.ijcttjournal.org Page101

Scalable Real-Time Monitoring For Distributed
Applications

Prathyusha.Pusunuri


Ragavamsi.D




Abstract
In Real time world to assess service quality of a networked
application, distributed monitoring servers should
continuously collect performance metrics. Monitoring the
status of running applications is naturally a real
requirement and important research area. Particularly log
analysis is all too often obliged to discover how the internal
system is behaving during execution.
As the problem increases regarding of longer development
period and top quality requirements that are caused by the
customer, it becomes increasingly crucial to examine
flexible and scalable parallel processing for complex real-
time solutions For this monitoring, existing approach using
Distributed Aggregation Tree (DAT) gets to high
monitoring delay and network stress. Our proposed system
uses the highly-scalable monitoring network for distributed
applications. Here we're monitoring the distributed system
using Client overlay and Proxy overlay techniques. We
formulate the problem to set up overlays minimizing
monitoring delay as NP-hard problem. All of us present a
efficient and scalable monitoring algorithm called SMon,
which continuously reduces network diameter in the real
living in a distributed manner. In our project using SMon
algorithm will give proof that low monitoring delay,
network stress and protocol overhead for distributed
applications.

Keywords Network,Client,Server,Delay.


I. INTRODUCTION

Throughout the last decade, the complexness of real-time
systems has exploded. The systems must nearly always
have their hardware/software architecture redesigned to
improve their performance, recuperate fault tolerance,
etc. New powerful processors alone do not happen to be
sufficient to achieve top performance and suppleness of
the control systems. Today it's usually the inadequate
performance of a given real-time system, the inflexibility
of fixing the hardware/software architecture, the
complexness of this very solutions plus the weak
debugging tools for verification that brings about chance
to market problems. One bottleneck in computer systems
today is overall performance the busses among the
system. In the event the os is at risk of getting
information about actual busload it would assist their
child operating system to schedule tasks with ease.

Multi-processor systems like SARA inclined to be




sophisticated as such it is not easy to understand how
transactions during runtime will affect the operating syste
real-time kernel, named Real Time Unit (RTU) in SARA,
keeps information regarding all tasks we have now a
suggestion to add a new parameter that might give the
kernel information about the busload among the system.
We belive this parameter could help the scheduler to
employ the system resources better.

Given the significance of real-time monitoring, we
perceive within this paper the best way to efficiently
collect performance statistics for large-scale applications
(in regards to distribution, central tendency, statistical
dispersion, etc.). A simple method of achieve will be to
make use of a centralized architecture, where all clients
continuously feed back their performance to many
monitoring or log servers, the so-called monitors. By
pooling these feedbacks, the monitors can then summarize
the entire performance. This process, however, gets
scalability problem because the monitor has got to
maintain large number of connections and may be
overwhelmed by the huge amount of feedback traffic.
Such fan-in method of the monitors also gets to much
network stress. To scale to high group, recent
accomplishments have used a peer-topeer method of
aggregate the performance metrics. The lenders form a
monitoring tree rooted at a monitor. Each client sends its
performance metrics upstream to its parents for
aggregation until such time as the root has been reached
(i.e., each client can be an aggregation point of its children
toward the root).1 Though these schemes are scalable,
they often times haven't yet considered client fan out and
monitoring delay in tree construction. Furthermore, a
large-scale application could possibly have multiple
distributed monitors. Constructing a global aggregation
tree for each individual of the monitors gets to problems
in scalability and network stress. Because clients may
churn (join, leave, or fail) at any time, forming
independent global trees rooted at these monitors also
make it a challenge to isolate churns in a single region
from the others.

We envision next-generation cloud systems exposing a
monitoring-as-a-service (MaaS) interface, where
regardless of the underlying provider or software,
monitoring data from both the system and application
layers is exposed in a consistent format, accessible as a

International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013



ISSN: 2231-2803 http://www.ijcttjournal.org Page102

web service. This service should support both the
adaptation required to manage systems and the needs of
users who use it to monitor cloud resources and
applications. It should be capable of monitoring large
numbers of resources and distributing the monitoring
information to large numbers of clients, and scale easily
as resources/clients are added and removed. It can rely on
existing monitoring technology to obtain information at
the resource level, but must move beyond that level to
process, aggregate, unify, and distribute the monitoring
information to subscribers.

II. LITERATURE SURVEY


Most Hardware/Software architectural implementations
of complex control system designs are carried out by a
large number of static coupled processor units, integrated
on any PCB (Printed Circuit Board) or on a different
PCB with the use of a standard bus and dual port RAM.
SARA (Scalable Architecture for Real-Time
Applications) serves as a dynamic coupled processor
system (flexible processor system) and is not according
to the PCB (Printed Circuit Board) or on the software
program architecture. There's always an established limit
of the items a fantastic system are able to handle. Any
time a system is building throughout the parallel bus, the
bus aree bottleneck the time you should many processors
are linked with it. The meaning regarding a static coupled
processor system is; changes of the system requirement
should have redesign of the applying software, of a given
PCB as well as a major change of software
communication and synchronization between the
processors coupled processor system can be redesigned
without any changes on the PCB or in the software for
tasks, communication and/or synchronization.

The problem with static coupled systems: Statically
designed PCB (Printed Circuit Board) is not a flexible
solution. Processors are usually communicating via dual
port memory (buffer). The communication and
synchronization do not happen to be trivial problems and
sometimes place a heavy load on the processor. The
software program architecture is statically classified as
different processors, often with different software
developing paradigms, just for example a symbol
processor and a RISC processor.

Many real time control systems take an application, that's
controlled by a real-time operating systems for executing
processes. To strengthen behavior a real time control
system, the processor clock frequency can possibly be
increased. Sometimes this is clearly not sufficient plus a
coprocessor work extremely well instead. The co-
processor (we refer to it as an RTU) is not a standard
processor, except a special purpose hardware performing
real time operating system functions. Different real time
operating system functions have successfully been
implemented into hardware a final decade. The
scheduling algorithms of the RTU are preemptive, non-
preemptive or mixed. In the event the RTU uses
preemptive scheduling, it uses an interrupt to signal the
application form processor to start out a context switch.
The scheduler algorithm of this very RTU can even load
balance cpu cores.

III. PROPOSED SYSTEM

Any effective scheme for distributed aggregation in
large-scale P2P systems has to meet the requirements on
scalability, adaptiveness, and load balancing. First, the
scalability has two criteria. To scale to a large number of
nodes, each aggregation should only introduce a limited
number of messages with respect to the network size. To
scale to a large number of DAT trees, the scheme should
have low construction and maintenance overhead for
each tree. Second, the aggregation scheme has to adapt
to the dynamics of node arrival and departure. The node
insertion and deletion in DAT should have minimal
impact to the aggregation process. Third, the aggregation
workload should be distributed evenly among all nodes
without any performance bottleneck. Load balancing is
thus essential for both workload fairness and system
scalability.
To solve the above aggregation problem, we propose a
distributed aggregation tree (DAT) approach that builds
a tree structure implicitly from the native routing paths of
Chord. In DAT, each node applies the given aggregate
function f on the values of its child nodes, and sends the
aggregated value to its parent node. By recursively
aggregating the values through the tree in a bottom-up
fashion, the root node will calculate the global
aggregated value very efficiently since it only needs to
collect the values from its direct child nodes. Fig. 1
shows an example of aggregating the global value
through a DAT tree of seven nodes, where each node ni
has a local value xi and i=1,2,...,7. After applying the
aggregate function three times at nodes N5, N6, and N7,
the root node N7 will calculate the global aggregated
value from all local values.


1: INPUT: rendezvous key k, Application table App(i, j) of
each node i, where i=1,2,...,n, and
j=0,1,...,b-1.
2: OUTPUT: a balanced DAT tree T rooted at node
r=successor(k)
3: d0 average distance between two adjacent nodes
4: for i1 to n do
5: if DIST(k, i) < DIST(PRED(i), i) then
6: ROOT(T) i
7: endif
8: x DIST(i, k)

International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013



ISSN: 2231-2803 http://www.ijcttjournal.org Page103

9: max log2((x+2d0)/3)
10: for jmax downto 0 do
11: if DIST(i, FINGER(i, j)) DIST(i, k) then
12: PARENT(i) FINGER(i, j)
13: endif
14: endfor
15: endfor

Our proposed system using two steps for monitoring:
first, client applications report their performance to some
proxies by means of a client overlay, and then the proxies
report the performance to the distributed monitors using
another proxy overlay. So the aggregation of the clients
have taken place at each client tiers even in presencre of
the peer churns. From client tier performance metrics
have been transferred to proxy tier for further
aggregation. In proxy overlay, aggregation are send to
monitors by minimum spanning tree formed between the
proxy tiers. Here our aim is to minimise the network
diameter in presence of churns. For that we present a
efficient and scalable monitoring algorithm called SMon,
which continuously reduces network diameter in real
time in a distributed manner. The classification of
monitoring data essentially reduces the granularity of
metrics. Since each metric has a different meaning and
may require a different classification scheme, individual
configuration on a metric-by-metric basis is employed. A
simple classification scheme, like
under=0|normal=1|over=2, does not give much details.
To allow adaptation to specific metrics and use cases, a
completely configurable classification scheme is
supported that even allows to have one class per metric
value, i.e., to transmit the actual metric value instead of
its class. The classification scheme can be changed at
runtime by reconfiguring the monitoring system to allow
for adaptation, such as for a more fine-grain observation
when needed. Data aggregation along the fan-in tree can
be synchronous or asynchronous. In the synchronous
variant, all back-ends send a message containing their
current classes in a certain interval to their respective
intermediates. All intermediates wait for the sub-tree data
before passing it on as aggregated message. Unchanged
classes are generally omitted. In the asynchronous
version, intermediates route all messages individually. As
this variant generally creates more messages and requires
intermediates to maintain state if computation is
involved, the synchronous variant is preferred.
Server
Server validates the every user and allows
them to access resource from itself. It has records of
monitors, proxies and clients in the network. If monitor
and proxy are logged in means its details are stored in
server. If proxies are logged in and want to connect
directly with monitor means it gives the details about
existing monitors in the network. If every user logged in
means server maintains its details whether it is a parent or
child node. If it is an parent node means it measures the
distance as 1 else if it is a child node means it measures
the distance according to the position where it is
connected as child. If clients are logged in as parent
mode, server will give details about proxies in the
network. If clients are logged in as child mode, server
will give the details about what are the clients available
for parents. If clients are want to down load any files
from Server means its request is first send to the server
after the confirmation of the server it can download the
file. Every server has limited resource allocation
capability. If clients count is more than the capability for
accessing the resource means it cannot able to give the
resource accurately due to load.
Monitor
Monitor is responsible for our proposed SMon
algorithm. SMon algorithm monitors the entire network.
Advantage of the SMon monitoring reduces the network
diameter and sent the report to the monitor without
monitoring delay compared to Existing method. Monitor
will monitor all the activity of the network. If monitor
logged in means it starts monitoring. From proxy tier
proxies are connected directly and indirectly through
interconnecting with existing proxies. If any clients
accessing the server for downloading any files from the
server means monitor have all the log details of files
downloading. If clients cant accessing the files as it is in
server it have to sent a report to monitor through proxy
tier. So the affected client sent the report to the proxy
directly else through the parent node. After message sent
to the proxy it forward the message to the monitor if it is
directly connected with monitor else it will forward via
interconnected proxies. After message received at the
monitor it will check the status of the server what is
reason for client problem and response according to it.
Proxy
Proxy Tier plays an important role in our
proposed monitoring system. It acts an interface between
the clients and the monitor. All information from the
client tier should be first sent to the proxy tier. First the
proxies in the proxy tier connected with monitor directly
and through connected indirectly through interconnected
with existing proxies. Proxy maintains the details about
the clients connected as parent. If clients are entering into
the network as a parent means it should be connected
directly with anyone of the proxies in the proxy tier. If
any message would come from the client tier it is viewed
at proxy. If the proxy is directly connected with monitor
then it will forward the message to the monitor directly
else if it is connected with any other proxies means it
forward the message to the monitor through the proxies
based on minimum spanning tree approach i.e., it forward
the message through the interconnected proxies which is
at minimum distance to the monitor.

International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013



ISSN: 2231-2803 http://www.ijcttjournal.org Page104

User Interface Design
User Interface Design plays an important
role for the user to move login window to user window.
This module is created for the security purpose. In this
we want to enter our user name and password. If we enter
the valid password and user name then only the user can
move login page to user window while entering user
name and password it will check username and password
is match or not(valid username and valid password). If
we enter any wrong username or wrong password we
cant enter into login window to user window it generates
some error message. So we are preventing from
unauthorized user entering into the login window to user
window. It will provide a good security for our project.
So server contain user name and password server also
check the authentication of the user. It will improves the
security and preventing from unauthorized user enters
into the network. In our project we are using java swings
for creating design. Here we are validating the Clients
who are going to access the server.
Client
If clients want to enter into the network means
they should register their information with username and
password. Once the details are successfully registered in
the database means we can use the login details for sign
in into the network. If clients are logged in then they
should choose the mode either as Parent or Child. If the
client chooses the mode as parent, it should be connected
with proxy directly in the proxy tier. If the client chooses
the mode as child then it should be connected with
existing parent nodes in the network. According to the
position of the parent node child distance is calculated.
After choosing the mode client can download the files
from the server. If client want to download file means
first it sent the request to the server and after the approval
it send the file to the client. If the client logged in after
the capacity of server exceeded then it cannot download
the file as it is from the server due to load problem. At
the time client send the report to the monitor via proxy
tier.
IV. RESULTS
SERVER
This page is acts as server homepage whwere all
the resources are available for the user.

This page shows the files available in the server side.


MONITOR
In this page all the details in the network will be going
to monitored whenever load occurs.


This below page will shown whenever any node in the
network face problem while downloading any file.

International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013



ISSN: 2231-2803 http://www.ijcttjournal.org Page105



PROXY
When proxy is login into the network ,we have to choose
the connectivity with monitor or existing proxy.



This page is shown whenever any alert message come
from any client nodes.



V. CONCLUSION AND FUTURE SCOPE

Our proposed system using two steps for
monitoring: first, client applications report their
performance to most proxies by means of client overlay,
and after that the proxies report the performance onto the
distributed monitors using another proxy overlay. In this
present work a efficient and scalable monitoring
algorithm called SMon, which continuously reduces
network diameter realtime a distributed manner. So our
proposed system reduces the monitoring delay of
distributed applications with the future enhancement of
load balance by the monitor. In future SMon algorithm
deploys scalable monitoring in real time. It monitors in
efficient way than the existing techniques and also
performs the load balancing in the server which means if
server is under load then the monitor itself find the
requested file of the affected client through the error
message and sends the file to the particular client. So it
will provide scalable monitoring with load balance.


REFERENCES


1. B. Yu, J. Li, and Y. Li, Distributed data aggregation
scheduling in wireless sensor networks, in Proc. of IEEE
INFOCOM. Citeseer, 2009.

2. S. Madden, M. J. Franklin, J. M. Hellerstein, and W.
Hong, TAG: a Tiny AGgregation service for ad-hoc
sensor networks, ACM SIGOPS Operating Systems
Review, vol. 36, no. SI, p. 146, 2002.

3. R. V. Renesse, K. P. Birman, and W. Vogels,
Astrolabe: A robust and scalable technology for
distributed system monitoring, management, and data
mining, ACM Transactions on Computer Systems
(TOCS), vol. 21, no. 2, p. 206, 2003.

4. P. Yalagandula and M. Dahlin, A scalable distributed
information management system, in Proceedings of the
2004 conference on Applications, technologies,

International J ournal of Computer Trends and Technology (IJ CTT) volume 5 number 2 Nov 2013



ISSN: 2231-2803 http://www.ijcttjournal.org Page106

architectures, and protocols for computer
communications. ACM, 2004, pp. 379390.

5. I. A. Dahlia, I. Abraham, D. Malkhi, and O. Dobzinski,
LAND: Locality aware networks for distributed hash
tables, in Technical Report 2003-75, Leibnitz Center of
the School of Computer Science and Engineering, the
Hebrew University of Jerusalm, 2003.

6. W.-P. K. Yiu, X. Jin, and S.-H. G. Chan, VMesh:
Distributed segment storage for peer-to-peer interactive
video streaming, IEEE Journal on Selected Areas in
Communications Special Issue on Advances in Peer-to-
Peer Streaming Systems, vol. 25, no. 9, pp. 171731,
Dec. 2007.

7. K. H. Vik, C. Griwodz, and P. Halvorsen,
Constructing low latency overlay networks: Tree vs.
mesh algorithms, in 33
rd
IEEE Conference on Local
Computer Networks, 2008. LCN 2008, 2008, pp. 3643.

8. E. Brosh, A. Levin, and Y. Shavitt, Approximation
and heuristic algorithms for minimum delay application-
layer multicast trees, IEEE/ACM Transactions on
Networking, vol. 15, no. 2, pp. 473484, 2007.

9. S. Banerjee, C. Kommareddy, K. Kar, B.
Bhattacharjee, and S. Khuller, OMNI: an efficient
overlay multicast infrastructure for real-time
applications, Computer Networks, vol. 50, no. 6, pp.
826841, 2006.

10. T. M. Baduge, A. Hiromori, H. Yamaguchi, and T.
Higashino, A distributed algorithm for constructing
minimum delay spanning trees under bandwidth
constraints on overlay networks, Systems and
Computers in Japan, vol. 37, no. 14, pp. 1524, 2006.