Вы находитесь на странице: 1из 16

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
1

An End-to-End Performance Analysis for Service Chaining in a


Virtualized Network
Emmanouil Fountoulakis, Student Member, IEEE, Qi Liao, Member, IEEE, Nikolaos Pappas, Member, IEEE
Future mobile networks supporting Internet of Things are expected to provide both high throughput and low latency to user-
specific services. One way to overcome this challenge is to adopt Network Function Virtualization (NFV) and Multi-access Edge
Computing (MEC). Besides latency constraints, these services may have strict function chaining requirements. The distribution
of network functions over different hosts and more flexible routing caused by service function chaining raise new challenges for
end-to-end performance analysis. In this paper, as a first step, we analyze an end-to-end communication system that consists of both
MEC servers and a server at the core network hosting different types of virtual network functions. We develop a queueing model
for the performance analysis of the system consisting of both processing and transmission flows. We propose a method in order
to derive analytical expressions of the performance metrics of interest, i.e., end-to-end delay, system throughput, task drop rate.
Then, we show how to apply the similar method to a larger system and derive a stochastic model for such systems. We observe that
the simulation and analytical results are very close. By evaluating the system under different scenarios, we provide insights for the
decision making on traffic flow control and its impact on critical performance metrics.

Index Terms—Applied queueing theory, delay analysis, end-to-end performance analysis, multi-access edge computing, network
function virtualization, throughput analysis.

I. I NTRODUCTION to the high-end servers in the cloud. Therefore, it is interesting


to further investigate the cooperation between the edge and the
The increasing demand of different kinds of network ser-
core, and the cooperation among MEC servers.
vices and the requirements of 5G networks for low capital
expenditure raise the need for the improvement of today’s
networks in terms of both flexibility and scalability. In con- A. Related work
ventional communication networks, network functions (e.g.,
Recently, the study of the performance of networks in
firewalls, transcoders, load balancers, etc.) are performed as
Virtual Network Function (VNF)/Software Defined Network
dedicated hardware middleboxes. Although dedicated hard-
(SDN) environment has attracted a lot of attention, [7]–[14].
ware can provide high performance, it causes high capital
Ye et al. [7] analyze the end-to-end delay for embedded
expenditure, low flexibility and scalability, and dependence
VNF chains. They consider two types of services that traverse
on particular application. In future communication networks,
different VNF chains and provide the delay analysis for each
these limitations will be addressed by Network Function Vir-
different chain. Miao et al. [9] provide an analytical model
tualization (NFV) [2]–[4]. The idea of NFV is to decouple the
based on Stochastic Network Calculus (SNC) to provide upper
network functions from dedicated hardware equipment. More
and lower delay bounds of a VNF chain. Along similar lines,
specifically, general purpose servers can host one or more
Duan [8] analyzes an end-to-end delay performance of service
types of network functions. The network can be flexible and it
function chaining for particular services and given resources.
can deploy proper network functions according to the demands
Authors in [10]–[14], apply tools from queueing theory to
of various traffic types while reducing the capital expendi-
evaluate the performance of systems in SDN environment. In
ture. Furthermore, mission-critical mobile applications, e.g.,
particular, Jarschel et al. [12] study the OpenFlow architecture,
augmented reality, connected vehicles, eHealth, will provide
where the switch is modeled as an M/M/1 queue and the
services that require ultra-low latency [5], [6]. To satisfy
controller as a feedback system of the delay-loss type M/M/1/S
such requirements, Multi-access Edge Computing (MEC) has
queue. Similarly Goto et al. [13] analyze a simple OpenFlow-
been proposed as a key solution [5]. The idea of MEC is
based switch in SDN environment, however, they distinguish
to locate more computation resources closer to the users,
traffic from the controller and exogenous traffic. Furthermore,
e.g., at the base stations. Network Function Virtualization
there are works that consider the modeling of connected VNF
(NFV) together with MEC are considered key technologies
as a sequence of queues where the goal is to guarantee the
for 5G wireless sytems. However, computation capabilities and
stability of the system and some particular network or service
available resources of MEC servers are still limited compared
requirements. The authors in [15]–[20] develop dynamic al-
This work extends the preliminary study in [1]. This work has been partially gorithms by applying Lyapunov optimization theory in order
supported by the European Unions Horizon 2020 research and innovation to guarantee system stability and fulfill additional service
programme under the Marie Skłodowska-Curie grant agreement No. 643002. requirements. In particular, in [15], [18] flows traverse VNF
This work was also supported in part by the Center for Industrial Information
Technology (CENIIT) and ELLIIT. chains and each node decides the resource allocation to each
E. Fountoulakis and N. Pappas are with the Department of Science and VNF and routes the flow to the next node. Gu et al. [20]
Technology, Linköping University, SE-60174 Norrköping, Sweden (e-mails: develop a dynamic distributed algorithm for flow and rate
{emmanouil.fountoulakis, nikolaos.pappas}@liu.se).
Q. Liao is with Nokia Bell Labs, 70435 Stuttgart, Germany (e-mail: control. Their objective is to achieve fairness between the
qi.liao@nokia-bell-labs.com). services and maximize the network utility while providing

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
2

system stability. Chen et al. [19] consider the problem where


VNF 1
VNF are installed in Virtual Machines (VMs). Each VM can MEC Server 1
be located in the same or different data center. In this work, Q1 Q6
Flow ↵ CPU CPU
each VM decides which functions to install or uninstall and Controller µ1 µ6
p
which services to serve. A dynamic algorithm is developed 1 ↵ Q4
that works in a distributed manner and takes online decisions. CPU

µ4
Considering a more static and known environment, researchers VNF 1
VNF 2
Server 3
investigate the VNF placement and resource allocation prob- MEC Server 2

lems [21]–[26]. In these works, the authors formulate the VNF Wireless
Wired
placement problem as Mixed Integer Linear Problem (MILP)
under the assumption of known traffic demand. In addition, Fig. 1: System model: Blue dashed lines group the queues
approximation algorithms have been developed in order to located in the same server.
provide a solution to the VNF placement problem, [27], [28].
Besides the VNF/SDN technologies, Multi-access Edge
Computing (MEC) technology promises a significant improve- for the performance metrics of our interest in order to provide
ment of the networks, especially in terms of latency reduction. insights of how to design a system to satisfy particular network
Task offloading from the mobile devices to MECs has attracted requirements.
considerable attention. For example, the works in [29]–[32]
address the trade-off between the power consumption and II. S YSTEM M ODEL
the task processing delay. The authors in [33], propose an The aim of this work is to study the performance of an end-
optimization model to perform joint slicing of mobile network to-end communication system that consists of different types
and edge computation resources. As VNF and MEC are two of VNFs. We first provide the analysis for a simple end-to-end
strongly connected technologies, the research evolution of the communication system and then, we provide the analysis for
one affects the evolution of the other. To the best of our more general topologies.
knowledge, there are no works providing end-to-end network We consider an end-to-end communication system consist-
performance analysis for a network with MEC deployed and ing of a mobile device, two MEC servers, and one server
operated within the NFV environment. located in the core network as depicted in Fig. 1. A task
traverses a service chain of two consecutive VNFs: VNF 1
and VNF 2. In the considered system, a MEC server, called
B. Contributions
Server 1, is connected with the base station through a wired
In this paper, we investigate a VNF-facilitated end-to-end backhaul link and hosts one copy of VNF 1 as the primary
network where the cooperation between the edge and the core MEC server.1 A secondary MEC server, called Server 2, is
network, and the cooperation among MECs are considered. located nearby and also hosts a copy of VNF 1. Server 2,
As a first step, we model and analyze a simple end-to-end for example, may be located at a peer edge host with spare
communication system which consists of two MEC servers at capacity or at a central office location within a metropolitan
the edge network and one at the core network hosting different area network. We consider the secondary server that acts as a
types of VNFs. We develop a methodology by applying helper of the base station. In particular, we utilize the resources
tools from queueing theory in order to derive approximate of the MEC Server 2 in order to offload traffic from the the
analytical expressions of the performance metrics of interest MEC Server 1. In addition, Server 3 in the core network hosts
such as end-to-end delay, drop rate, and throughput. The VNF 2 and has more advanced computation capabilities than
methodology is based on the decomposion of the system into Servers 1 and 2. The analysis of such system can be used to
smaller subsystems easier to analyze. The information from study the deployment of AR-related applications. For example,
analysis of each subsystem can be used in order to study VNF function can be installed in MEC to map user requests
the performance of the entire system. Since, in reality, there to their corresponding nodes providing contents, while the
are no buffers with infinite capacity, we consider finite size compute-intensive VNFs such as AR or 3D gaming can be
buffers in the servers of our system model. However, the deployed in core networks.
capabilities of the server at the core are considered to be We assume a slotted time system. At each time slot, the
much higher than those of the MEC servers. In order to device transmits a task in form of a packet to a base station
observe the behavior of the system asymptotically, we also over a wireless channel. Because of the presence of fading
investigate, as a subcase, the scenario where the buffers at in the wireless channel, transmissions may face errors. Thus,
core server is infinite. Based on the methodology derived for we assume that a task is successfully transmitted to the base
the simple model, we extend the analysis to a more complex station with a probability p that captures fading, attenuation,
system with a larger number of MEC servers. As shown in the noise, etc. The device attempts for a new task transmission
simulation results, the proposed stochastic model is accurate only if the previous task is successfully received at the base
and indicates a robust behavior even for larger number of
1 For the connection between the servers, we do not consider success
servers. Furthermore, we provide results that show the trade-
probabilities/erasure channel because the connection is wired. However, with
off between the throughput, drop rate, and end-to-end delay. minor modifications of our model we are able to analyze also the case of a
In this work we provide approximate analytical expressions wireless backhaul link.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
3

station. The received tasks need to be distributed between the physical decomposition of the whole system into subsystems.
queue for local processing and the queue for transmission Our goal is to analyze parts of the system independently in
to the secondary MEC server. Thus, there are two possible order to make the analysis easier. The interconnection between
routes to pass through the service chain. A flow controller at the servers implies the interconnection between the queues.
the base station decides randomly the routing for each task. This fact allows us to analyze different parts of the whole
With probability α the task is processed by Server 1 first to system even if the corresponding queues belong physically in
be processed by VNF 1, and then forwarded to Server 3 to different servers.
be processed by VNF 2. With probability 1 − α the task is
forwarded to Server 2, to be processed by VNF 1, and then Subsystem 1:
Q1 Q2
forwarded to Server 3 for being processed by VNF 2. Note CPU Tx
µ2
2,6
µ1
that each task can be part of a service that consists of multiple
tasks. Different services may have different sizes. However, Subsystem 2: Q3 Q4
Tx CPU 5
services can be split into different tasks, each one with same µ3 µ4
size. In this work, we focus on the system analysis in terms Q5
of those tasks. However, this assumption allows us to analyze Subsystem 3: 5 Tx 5,6
µ5
the system performance in terms of services. For example, Q6

consider that we have a service that consists of 1000 tasks. Subsystem 4: 2,6
5,6
CPU
µ6
We can estimate the delay of 1000 tasks and then we can sum
up all the delays to provide the delay for the whole service.
Fig. 2: Proposed decomposition.
Each task that arrives at a server first waits in a queue for
being processed by a VNF. Then, after the processing, it is
Since the computation capabilities of the server in the core
stored in the transmission queue, waiting to be forwarded and
are much higher than those of the MEC servers, we consider
processed by the next VNF. Let Qi denote the i-th queue,
the core server as one independent queueing system. In order
where i ∈ K, and K is the set of the queues in the system. Note
to develop a model that couples MEC Server 1 and MEC
that the queues follow an early departure-late arrival model: at
Server 2, we consider as one independent queueing system
the beginning of the slot the departure takes place and a new
the subsystem which consists of Q3 and Q4 . Q5 is considered
arrival can enter the queue at the end of the slot. The queues
as one independent system. We choose to decompose the
for task transmission are Q2 , Q3 , and Q5 , and the queues for
system as described above in order to study the decomposition
task processing are Q1 , Q4 , Q6 . The arrival rates for queues
strategies for deriving: i) a model for the interacting queues
Q1 and Q3 are pα and p(1 − α), respectively. We denote by
within one server, ii) a model for two queues of different
µi , i ∈ K, the service rates of the queues. We assume that the
servers that are interacting, and iii) an individual model for
service times are geometrically distributed.
each queue within the same server. To summarize, we consider
Furthermore, the queues are assumed to have finite buffer.
the following four subsystems: 1) Q1 and Q2 , 2) Q3 and Q4 3)
Let Mi denote the buffer size of each queue i ∈ K. If a queue
Q5 , and 4) Q6 . The performance metrics for the whole system
is full and no task departs at the same time that a new one
are approximated with the analytical expressions derived from
arrives, the new task is dropped and removed from the system.
the subsystems. The accuracy of the approximation is validated
Besides the case presented in Fig. 1, we will consider the case
through simulations in Section VI.
of a scaled-up system in Section V. Note that the described
system model can be considered as an isolated network slice
under the network slicing and MEC technologies. A. Subsystems 1 and 2: Two queues in tandem
The two queues in tandem, Q1 and Q2 , are considered as
one subsystem. The arrival rate for Q1 is: λ1 = pα. The
III. P ERFORMANCE A NALYSIS
Markov chain {(Xn , Yn )} is described by
In this section, we perform the modeling and the perfor-
Pi,j:u,k = Pr {Xn+1 = i, Yn+1 = j | Xn = u, Yn = k} ,
mance analysis that allow us to derive the critical performance
metrics. We model the considered queueing system utilizing where Xn and Yn denote the states (in terms of queue length)
Discrete Time Markov Chain (DTMC). Modeling the whole of Q1 and Q2 at the n-th time slot, respectively; i and j
system as one Markov chain can drive in a quite complicated are referred to as the level i and phase j, respectively. Here,
system difficult to be analyzed in terms of closed-form expres- we follow the notation provided by [34]. The two dimensional
sions. An efficient method to approximate the whole system (2D) Markov chain is a Quasi-Birth-and-Death (QBD) DTMC.
while keeping the analysis complexity low, is the decompo- Therefore, the process can move up and down between the
sition method [34]. According to the decomposition method, levels one at a time because up to one task can arrive and
the whole queueing system is decomposed into subsystems, depart from Q1 .
and each different subsystem is analyzed independently. We The Markov chain is shown in Fig. 3. For example, if the
derive results for each subsystem and then, we utilize them in system is in (0, 0) state then, both Q1 and Q2 contain zero
order to provide analytical results for the whole system. The tasks. The arrows denote the transitions between the states
proposed decomposition for the system in Fig. 1 is shown and the corresponding transition probability. For example, if
in Fig. 2. Note that by decomposition we do not mean the the system is in (0, 0) state then, the probability the system

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
4

b1
λ1 b1 b1 b1 b2
b2 b2 b2 b2

0,0 0,1 0,2 0,3 ... 0,M2

c2 c2 c2 c2
c2

a2

c1
λ1 α2 c1 α2 c1 α2 c1 λ1µ1

c1
α1

1
α1


α1

λ
a4 α4 α4

a1
a5 a5 α5 α5 α5
λ1µ̄1 1,0 1,1 1,2 1,3 ... 1,M2 d1
λ1µ1 a3 α3 α3 α3
a7 a7 α7 α7 α7

a2
λ1µ̄1 α2 α6 α2 α2 α6 λ1µ1
a6

a6

a6
1 α1

α1

a1
α1
λ

α4 α4 α4
α5 α5 α5 α5 α5
λ¯1µ̄1 2,0 2,1 2,2 2,3 ... 2,M2 d1
λ1µ1 α3 α3 α3 α3
α7 α7 α7 α7 α7
λ1µ¯1 α6 α2 α6 α2 α6 α2 α6 α2 α6
α1 α1 α1 α1 λ1µ1
α1
α4 α4 α4
α5 α5 α5 α5 α5
λ¯1µ¯1 3,0 3,1 3,2 3,3 ...
λ1µ1 α3 α3 α3 α3 3,M2 d1
α7 α7 α7 α7 α7
α6 α2 α6 α2
λ1µ¯1 α6 α2 α2 α6 α6
λ1µ1 α1 α1 α1
α1 λ1µ1
α5 α5 α5 α5 α5
...

...

...

...

...

...
λ1µ1 α3 α3 α3 α3
α7 α7
λ1µ¯1 α7 α2 λ1µ1
α7 α2 α7 α6
α6 α2 α6 α6 α6 α
5
α1 α1 α1
α1 α1
µ2µ̄1 µ2µ̄1 µ2µ̄1 µ2µ¯1 µ2µ̄1
µ¯1 M1,0 M1,1 M1,2 M1,3 ... M1, M2
λ1µ1
α3 α3 α3 α3
d3 d3 d3
d2

Fig. 3: 2D Markov chain for the system with two queues in tandem of Subsystem 1 shown in
Fig. 2.

... ...
... ... ...

1 b2 b1
b2 b 1 c2 c1
.. .. c2 c1
.. ...
...

. . .
b2 b 1 c2 c1

.. .. .. .. .. ..
. . . . . .
...

.. .. .. ..
...

. . . . .. ..
. .

.. .. ..
. . .
...

Fig. 4: Transition matrix of the 2D Markov chain shown in Fig. 3.


This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
5

(1) (1) (2) (2)


to stay in the same state in the next slot is λ1 2 . Note that for A0 = λ1 µ̄1 P1 , A1 = λ̄1 µ̄1 P1 +λ1 µ1 P1 , A2 = λ̄1 µ1 P1 .
some cases, we cannot move from one state to another specific
Following the steps described above, we can construct the tran-
one. For example, we cannot move from (0, 1) to (1, 2) since
sition matrix of Subsystem 1 for arbitrary finite buffer sizes.
up to one task can arrive and up to one can depart from each
Our goal is to derive the steady state distribution of the Markov
queue.
chain defined above. We can apply direct methods in order to
In Fig. 4, we show the transition matrix of the Markov chain.
find the steady state distribution [34, Chapter 4]. Note that
We observe that the matrix has a block diagonal form, where
there are several efficient algorithms that can be used for this
B, C, A0 , A1 , A2 are the block matrices of the transition
purpose, e.g., logarithmic reduction method [34]. We denote
matrix. The dimensions of each submatrix are (M2 + 1) ×
the steady state distribution of Subsystem 1 by a row vector
(M2 + 1). h
(1) (1) (1) (1) (1)
i
In Table I, we show the values of each transition probability. defined as π (1) = π0,0 , π0,1 , . . . , π0,M2 , π1,0 , . . . , πM 1,M 2 .
For the cases which Q1 is empty, the system can move to We find π (1) by solving the following linear system of
states for which either the number of tasks of Q2 decreases equations π (1) P1 = π (1) , π (1) 1 = 1, where 1 denotes the
with probability b2 by one or remains in the current state with column vector of ones. Hereafter, we use π (n) to denote
probability b1 or the number of tasks in Q1 increase by one the steady state distribution vector of the n-th subsystem for
with probability c1 . Assume that the system is in state (0, j). n = 1, 2, 3, 4.
It will move to (0, j −1) if a task departs from Q2 and no task Furthermore, the arrival rate of Q2 depends on the service
arrives in Q1 . Therefore, the probability b2 is equal to λ1 µ2 . rate of Q1 . Therefore, wePdefine  rate of Q2 as
the arrival
M2 PM1 (1)
In order to move from (0, j) to (1, j) state, we must have one λ2 = Pr {Q1 > 0} µ1 = j=0 i=1 πi,j µ1 . Similarly, we
arrival in Q1 and no departure from Q2 . This is translated can construct the transition matrix P2 and the steady state
mathematically as the probability the system to move from distribution π (2) for the second subsystem consisting of Q3
(0, j) to (1, j) to be equal to c1 = λ1 µ2 . We observe that and Q4 . The arrival rates of Q3 and Q4 are
the transition probabilities for the non-extreme cases, i.e., Q1
and Q2 are neither empty nor full, are repeated. To give an λ3 = p(1 − α) and
 
intuition behind this, we give an example. Consider that the M4 X
M3
(2)
X
system is in (3, 2) state. The system will move to (3, 3) state λ4 = Pr {Q3 > 0} µ3 =  πi,j  µ3 , respectively.
if a task arrives to Q1 and one departs from Q1 . Therefore, j=0 i=1
the probability, α3 , is equal to λ1 µ1 µ2 . The system will move
to (2, 3) state only if we have a depart from Q1 and no new B. Subsystem 3: Q5
task arrives to it. The probability to move to this state is α1 = We consider Q5 as an independent queueing subsystem.
λ1 µ1 µ2 → “no task arrives” and “a task departs from Q1 ” We denote by M5 the buffer size of the queue. We first
and ”no task departs from Q2 ”. In order the system to remain define the arrival rate of Q5 as λ5 = Pr {Q4 > 0} µ4 =
in the current state, two events can happen: 1) “arrival in Q1 ”
!
M
P3 MP4 (2)
and “departure from Q1 ” and “departure from Q2 ”, or 2) “not πi,j µ4 . The Markov chain of this system is shown
i=0 j=1
arrival in Q1 ” and “not departure from Q1 ” and “not departure
in Fig. 5.
from Q2 ”. Therefore, the probability the system to remain in
The transition matrix of this Markov chain is described
the current state is α4 = λ1 µ1 µ2 + λ1 µ̄1 µ̄2 . Similarly, we
below
calculate the rest transition probabilities of the Markov chain.  
By utilizing the properties of a QBD DTMC, we can analyze λ̄5 λ5
 λ̄5 µ5 λ5 µ5 + λ̄5 µ̄5 λ5 µ̄5
such systems with arbitrary buffer size in the following steps.

 
First we define the following matrices [34] P3 = 
 λ̄ µ
5 5 λ µ
5 5 + λ̄ µ̄
5 5 λ µ̄
5 5 .

   . .. . .. . .. 
1 0
 
 µ2 µ̄2  λ̄5 µ5 λ̄5 µ̄5 + λ5
(1)
P1 =  ,
 
.. .. We denote
h the steady state idistribution of Subsystem 3 by
 . . 
(3) (3) (3)
µ2 µ̄2 π (3) = π0 , π1 , . . . , πM5 . To derive π (3) , we solve the

0 1 0
 following linear system of equations, π (3) P3 = π (3) , π (3) 1 =
 0 µ2 µ̄2  1. Using balance equation, we obtain
 
(2)
P1 =
 0 0 µ2 µ̄2 .

(3) λi5 µ̄5
(i−1)
(3)
.. .. πi = π0 , for i = 1, . . . , M5 and
λ̄i5 µi5
 
 . . 
0 1 " M5
#−1
(3)
X λi5 µ̄i−1
5
π0 = 1+ . (1)
Then, the block matrices of the transition matrix are calculated λ̄i5 µi5
i=1
as
(1) (1) (2)
B = λ̄1 P1 , C = λ1 P1 , E = λ̄1 µ1 P1 , C. Subsystem 4: Q6 with finite buffer size
2 For simplicity, given a probability of an event, denoted by γ, we denote The arrival rate for Q6 depends on the service rate of
the probability of its complementary event by γ = 1 − γ . Q2 and Q5 . Note that the departures from Q2 and Q5 can

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
6

TABLE I: Transition probabilities values of the 2D Markov chain.


Probability Value Probability Value
b1 λ1 µ̄2 a4 λ1 µ1 µ2 + λ1 µ̄1 µ̄2
b2 λ1 µ2 a5 λ1 µ̄1 µ2
c1 λ1 µ̄2 a6 λ1 µ̄1 µ̄2
c2 λ1 µ2 a7 λ1 µ̄1 µ2
a1 λ1 µ1 µ̄2 d1 λ1 µ1 µ2 + µ1 λ1
a2 λ1 µ1 µ2 d2 µ1 µ2 λ1 + µ1 µ2 + µ1 λ1 µ2
a3 λ1 µ1 µ̄2 d3 µ1 µ2 λ1 + µ1 µ2

λ̄5µ̄5 + λ5
λ̄5 λ5µ5 + λ̄5µ̄5 λ5µ5 + λ̄5µ̄5
λ5 λ5µ̄5 λ5µ̄5 ... λ5µ̄5
0 1 2 M5
λ̄5µ5 λ̄5µ5 λ̄5µ5 λ̄5µ5
Fig. 5: Markov chain for Q5 .

be considered independent stochastic processes. The arrival D. Subsystem 4: Q6 with infinite buffer size
rates for Q6 that occur due to Q2 and Q5 are λ6,2 = In this subsection, we study the case where the buffer in the
Pr {Q2 > 0} µ2 , and λ6,5 = Pr {Q5 > 0} µ5 , respectively. core server has infinite size. Since the capabilities of the server
The arrival rate of Q6 is given by λ6 = λ6,2 + λ6,5 . We model in the core can be much higher than those of the MEC server
the system as a Markov chain as shown in Fig. 6, where and therefore, the buffer has a very large capacity, we are
p00 = λ̄6,2 λ̄6,5 , p01 = λ6,2 λ̄6,5 + λ6,5 λ̄6,2 , p02 = λ6,2 λ6,5 , interested in studying the stability conditions of such systems.
b1 = λ̄6,5 λ̄6,2 µ̄6 + λ̄6,5 λ6,2 µ6 + λ6,5 λ̄6,2 µ6 , We model the subsystem as one Markov chain. The transition
matrix that describes the Markov chain is shown below
b2 = λ6,2 λ̄6,5 µ̄6 + λ̄6,2 λ6,5 µ̄6 + λ6,2 λ6,5 µ6 ,  
a0 b0 0 0 ···
b3 = λ6,2 λ6,5 µ̄6 , b0 = λ̄6,2 λ̄6,5 µ6 ,  a1 b1 b0 0 · · · 
b = λ̄6,2 λ6,5 µ6 + λ6,2 λ̄6,5 µ6 + λ̄6,2 λ̄6,5 µ̄6 .
 
 a2 b2 b1 b0 · · · 
P4 =  0 b3 b2 b1 · · · , (4)
 
The transition matrix that describes the Markov chain above  
is shown below

 0 b3 b2 b1


  .. .. .. ..
p00 p01 p01 . . . .
 b0 b1 b2 b3 
  where a0 = p00 , a1 = p01 , a2 = p02 . The transition
 b0 b1 b2 b3 
  matrix is a lower Hessenberg matrix. The general expres-
P4 = 
 . .. .. .. .. .

. . . sion for the equilibrium equations for the states is given by
(4) (4)
 

 b0 b1 b2 b3 
 the i-th term in the following equation: πi = ai π0 +
b0 b1 b2 + b3 i+1
(4)
  P
bi−j πj . For the DTMC with infinite state space, we
b0 b1 + b2 + b3 j=1
(2) apply z-tansform approach to solve the state equations. The
In order to construct the transition matrix P6 , we need only z-transform for the state transition probabilities ai and bi
2 3
to calculate the probabilities described above. We denote ai z −i and B(z) = bi z −i , respectively.
P P
are A(z) =
(4)
h steady statei distribution of Subsystem 4 by π
the = i=0 i=0
(4) (4) (4) The z-transform for the steady state distribution vector π (4)
π0 , π1 , . . . . To derive π , we solve the following linear ∞
(4) (4) −1 A(z)−B(z)
πi z −i = π0 z z−1
P
system of equations is Π(z) = −B(z) . The solution
i=0
0
(4) (4) (4)
π (4)
P4 = π (4)
,π (4)
1 = 1. (3) for πi is given by π0 = 1+B1+B (1)
0 (1)−A0 (1) , πi = ci +
m
We observe from (3) that π (4) is the eigenvector of the rj (pj )(i−1) , i > 0, where r, p, and c are the residues,
P
transition matrix for λ = 1. Therefore, we can use eigenvalue j=1
poles, and directs terms, respectively. Since Q6 has infinite
decomposition (EVD) in order to find the eigenvectors of
buffer size, we need to characterize the conditions under which
the matrix. After applying EVD, the transition matrix can
the queue is stable. Stability is important since it implies finite
be written as P(4) = QΛQ−1 , where Q = [q1 , . . . qn ]
queueing delay. A definition of queue stability is shown below
are the eigenvectors, and matrix Λ contains the eigenvalues
[35].
in its diagonal. The eigenvector that corresponds to the the
eigenvalue that has value equal to one, is the steady state Definition 1. Denote by Qi (t) the length of queue i at the
distribution. beginning of time slot t. The queue is said to be stable if

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
7

p02 b3
b
p00 b1 b1 b2
p01 b2 b2
0 1 2 ... M6
b0 b0 b0 b0
Fig. 6: Markov chain for Q6 .

lim Pr {Qi (t) < x} = F (x) and lim F (x) = 1. system drop rate and average number of tasks in the system
t→∞ x→∞
that are necessary metrics for analyzing the throughput and
The corollary consequence of the previous definition is the delay of the system. We utilize the results of the previous
Loynes’ theorem [36] that states: if the arrival and service section in order to obtain the corresponding expressions.
processes of a queue are strictly jointly stationary and the
average arrival rate is less than the average service rate, then
the queue is stable. Therefore, Q6 is stable if and only if the A. Drop rate and average queue length
following inequality holds: λ6 < µ6 . The probabilities to have a dropped task at each time slot
Remark. Since the stationary distributions of the presented for Q1 − Q5 are shown respectively below
Markov chains have been calculated, we can also obtain M2 M1
(1) (1)
X X
the distributions of the queue sizes. Thus, we can provide PD1 = λ1 µ̄1 πM1 ,j , PD2 = λ2 µ̄2 πi,M2 ,
the probability that a queue size goes beyond a congestion j=0 i=1
limit. For example, consider C as the congestion limit for M4 M3
(2) (2)
X X
Q5 , the congestion violation probability is calculated as PD3 = λ3 µ̄3 πM3 ,j , PD4 = λ4 µ̄4 πi,M4 ,
M
P5 (3) j=0 i=1
Pr {Q5 > C} = πi .
(3)
i=C+1 PD5 = λ5 µ̄5 πM5 .

E. Complexity of the proposed method For the case of Q6 , drops can occur when Q6 is on (M6 )th or
(M6 − 1)th state. The drop rate of Q6 is shown below
In the proposed method described above, the largest system
is the one with 2D Markov chain. In particular, Subsystems 1 (4) (4)
PD6 = p02 (1 − µ6 )πM6 −1 + (p01 + 2p02 )πM6 (1 − µ6 ). (5)
and 2. As a first step, we construct the transition probability
matrix of the 2D Markov chain. In order to obtain the transition The average length of each queue is given by
probability, we construct each of its submatrices. In particular, M1 X
M2 M2 X
M1 M3 X
M4
B, C, A0 , A1 , A2 . The dimension of the submatrices is (1) (1) (2)
X X X
Q̄1 = πi,j i, Q̄2 = πi,j j, Q̄3 = πi,j i,
(M2 + 1) × (M2 + 1). Since we perform (M2 + 1) × (M2 + 1) i=0 j=0 j=0 i=0 i=0 j=0
multiplications to construct those submatrices, the complexity M3
M4 X M5 M6
is O((M2 + 1)2 ). The second step is to compute the steady (2) (3) (4)
X X X
Q̄4 = πi,j j, Q̄5 = πi i, Q̄6 = πi i.
state distribution of the 2D Markov chain. In order to obtain j=0 i=0 i=0 i=0
the steady state distribution of the system, we apply EVD
to the transition matrix. The computational complexity of Therefore, the system drop rate and the average num-
applying EVD or singular valued decomposition (SVD) is ber
P of tasks in theP system can be described as PD =
O(m3 ) in an m × m square matrix [37]. Therefore, the PDi and Q̄ = Q̄i , respectively.
i∈K i∈K
computational cost for calculating the steady state distribution
is O(((M1 + 1)(M2 + 1))3 ).
Alternatively, if the system is modeled as a single Markov B. Delay and throughput analysis
chain, even the complexity of representing the Markov chain We denote by Ti the throughput of each queue i. Since
is quite high. For example, if we have a system with 10 queues we consider finite buffers, and consequently task drops, the
in tandem, we need to construct a 10D Markov chain that itself throughput of each queue as well as system throughput depend
is a quite complex task. Instead, we can use the decomposition on both the arrival and drop rate. Therefore, the throughput
method to reduce the complexity. To this end, we can have 5 for Q1 − Q6 is calculated as
subsystems with 5 corresponding Markov chains and directly
apply the proposed methodology to obtain the steady state Ti = λi − PDi , ∀i ∈ K, (6)
distributions. that is equivalent to the effective arrival rate of each queue.
In order to derive the per task average delay expression,
IV. K EY P ERFORMANCE M ETRICS we first derive the expression of task average delay for
In this section, we provide analytical expressions of the each queue that includes both the queueing and transmission
performance metrics of our interests. First, we calculate the delay. We denote by Di , the average per task delay for each

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
8

VNF 1
Primary MEC server
Flow ↵1 Q1 Q2
Controller CPU Tx

p1 µ1 µ2
1 ↵1 Q4
Q3 Q5
Tx CPU Tx

µ3 µ4 µ5 CPUs

VNF 1 Q6 1
Secondary MEC server µ6,1

VNF 1 2
Primary MEC server µ6,2
2 Q7 Q8
Flow ↵2
Controller CPU Tx

p2 µ7 µ8
1 ↵2 Core server
Q9 Q10 Q11
Tx CPU Tx

µ9 µ10 µ11
VNF 1
Secondary MEC server

Fig. 7: Two user devices transmit tasks to two primary MEC servers (locating at base stations) respectively, assuming that
there are two CPUs in the core server.

queue. We utilize the Little’s theorem [38] and we derive the rate of Q6 that depends on the throughput of Q2 , Q5 , Q8 , and
corresponding expressions as shown below Q11 . The arrival rates of Q6 that occur due to Q2 , Q5 , Q8 ,
and Q11 are λ6,2 = Pr {Q2 > 0} µ2 , λ6,5 = Pr {Q5 > 0} µ5 ,
Q̄i 1
Di = + , ∀i ∈ K. (7) λ6,8 = Pr {Q8 > 0} µ8 , and λ6,11 = Pr {Q11 > 0} µ11 , re-
Ti µi spectively. Furthermore, we assume that the service time are
Finally, the per task average delay is calculated as identically geometrically distributed. Therefore, µ6,1 = µ6,2
and we set µ6,1 = µ6,2 = µ. Let X be a random variable that
Dsys = α(D1 + D2 ) + (1 − α)(D3 + D4 + D5 ) + D6 . (8) represents the number of departures at each time slot. Given
that the queue is non-empty, X follows a binomial distribution
V. S CALED -U P S YSTEM with success probability µ. The Probability Mass Function
(PMF) is given by Pr {X = k} = nk µk (1 − µ)n−k , where

In this section, we explain in details how the analysis that
is shown above can be used in general cases of multiple n is the number of cores, and k is the number of departures
user devices communicating to multiple base stations that are in one slot. In our case, n = 2, and k ≤ 2. Therefore, we
connected to the core server. We provide the analysis and can easily calculate the probabilities to have k departures at
methodology for two base stations. The same methodology can each time slot. We model this subsystem as one Markov chain.
be applied for a system with arbitrary number of base stations. The corresponding transition matrix is shown in (9), where the
However, the expressions will be significantly involved with- elements of the matrix are calculated in Appendix A.
out providing any meaningful insights. We analyze the system Let i and j be the i-th row and j-th column of the
that is shown in Fig. 7, by following the same methodology transition matrix in (9), respectively. We observe that the
in order to derive the approximate analytical expressions: we transition matrix has repeated elements from the third row to
decompose the system into subsystems and analyze each sub- the (M6 −Amax +1)th row, where Amax is the maximum number
system independently. Furthermore, we consider that the server of tasks that can arrive in Q6 at each time slot. Therefore
in the core has two CPUs. Therefore, the number of departures for the system above, Amax = 4. Thus, after calculating
is up to two tasks per time slot. The number of arrivals can be the transition probabilities for the first three rows, it is easy
up to four tasks per time slot.3 We decompose the system into to construct the transition matrix without calculating each
seven individual subsystems: 1) Q1 and Q2 , 2) Q3 and Q4 , 3) element independently.
Q5 , 4) Q6 , 5) Q7 and Q8 , 6) Q9 and Q10 , and 7) Q11 . For the In order to derive analytical expressions for key performance
subsystems 1−3 and 5−7, we apply the analysis that is shown metrics, we need to calculate the steady state distribution of
in Sections III-IV. For Subsystem 4, we provide the structure the Markov chain of Q6 . We apply EVD to derive the steady
of the transition matrix in below. First, we define the arrival state distribution and follow the same methodology introduced
3 The same holds if we have n arrivals per time slot and the core server
in Section III.
has m cpus. After obtaining the steady state distribution, we can calcu-
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
9

 
p00 p01 p02 p03 p04

 p10 p11 p12 p13 p14 p15 


 b6 b5 b4 b3 b2 b1 b0 

 .. .. .. .. .. .. .. 

 . . . . . . . 

P6 = 
 b6 b5 b4 b3 b2 b1 b0 
 (9)

 b6 b5 b4 b3 b2 b1 b0 


 b6 b5 b4 b3 b2 b0 + b1 


 b6 b5 b4 b3 b2 + b1 + b0 

 b6 b5 b4 b3 + b2 + b1 + b0 
b6 b5 b4 + b3 + b2 + b1 + b0

late the drop rate of Q6 . In this case, at each time slot, more respectively. Finally, the per task average delay is calculated
than one task can be dropped. The reason is because more as
than one task can arrive at each time slot. Drops occur when
Dsys = p1 A1 + p2 A2 + D6 . (17)
Q6 has less than four vacant positions. This corresponds to the
(M6 −3)th state. To summarize, drops occur when Q6 is on the VI. N UMERICAL AND S IMULATION R ESULTS
(M6 − 3)th , (M6 − 2)th , (M6 − 1)th , or (M6 )th state. Below, we
calculate the drop rate given that Q6 is on a particular state. In this section, we evaluate the performance of our ap-
(4) (4)
proximated model by comparing the analytical and simulation
PD6 |Q6 =M6 = p01 πM6 (1 − µ)2 + 2p02 πM 6 (1 − µ)2 results. We study the performance of the system with one
(4) (4) base station by showing how different parameters and routing
+ p02 πM6 2 × µ(1 − µ) + 3p03 πM 6 (1 − µ)2
(4) (4)
decisions affect the system performance in terms of delay,
+ 2p03 πM6 2µ(1 − µ) + p03 πM6 µ2 throughput, and drop rate. Furthermore, we study the perfor-
(4) (4)
+ 4p04 πM6 (1 − µ)2 + 3p04 πM6 2(1 − µ)µ mance of a system with two base stations and the performance
(4) of a system with multiple servers and VNFs. Our goal is
+ 2p04 πM6 µ2 , (10) to evaluate the performance of our approximation model for
larger systems and provide insights on the operation of the
considered setup. We developed a MATLAB-based behavioral
(4) (4)
PD6 |Q6 =M6 −1 = p02 πM6 −1 (1 − µ)2 + 2p03 πM6 −1 (1 − µ)2 simulator and each case run for 106 time slots for each case.
(4)
+ p03 πM6 −1 2(1 − µ)µ
(4) (4)
A. The one base station case
+ 3p04 πM6 −1 (1 − µ)2 + 2p04 πM6 −1 2(1 − µ)µ
1) Effect of the routing decision on the system performance
(4)
+ p04 πM6 −1 µ2 , (11) - analytical vs simulation results
In Fig. 8 and Fig. 9, we provide results that show how dif-
ferent routing decisions affect the performance of the system
(4)
PD6 |Q6 =M6 −2 = p03 πM6 −2 (1 − µ)2 + 2p04 πM6 −2 2(1 − µ)µ
(4) for different settings of parameters µi , i = 1, 2, . . . , 5 . We
(4)
also compare the simulation results with those of analytical in
+ p04 πM6 −2 µ2 , (12) order to evaluate the performance of our approximated model.
In the first case depicted in Fig. 8a, the capabilities of the
MEC Server 1 in terms of service rates are lower than those
(4)
PD6 |Q6 =M6 −3 = p04 πM6 −3 (1 − µ)2 . (13) of the MEC server 2. We observe that the optimal values of
throughput and delay are achieved for values of α that are
Finally, the total drop rate of Q6 is shown below
close to 0.2. The value of α that minimizes delay, it does not
PD6 = PD6 |Q6 =M6 −3 + PD6 |Q6 =M6 −2 + PD6 |Q6 =M6 −1 necessarily maximize the throughput. For larger values of drop
(14) rates, we may have shorter delay of the system but smaller
throughput as well. In Fig. 8, we observe results for smaller
We derive the throughput and delay for each queue by using capabilities of the MEC server 2. In Fig. 9, the capabilities of
the equations in (6), (7). The average delay that arrives in the the MEC servers are identical. We observe that the maximum
first and the second base station are described below throughput is achieved for α = 0.5. However, the optimal
DBS1 = α1 (D1 + D2 ) + (1 − α1 )(D3 + D4 + D5 ) +D6 , α in terms of end-to-end delay is different. The value of α
| {z } that minimizes the delay is around 0.65. The flow controller
A1
(15) forwards larger amount of traffic to the first flow. This strategy
minimizes the delay because the tasks have to traverse a
DBS2 = α2 (D7 + D8 ) + (1 − α2 )(D9 + D10 + D11 ) +D6 ,
| {z } smaller amount of queues and therefore, face shorter waiting
A2 times. However, this has an impact on the system performance
(16) in terms of drop rate and therefore, the system throughput.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
10

80 0.8 0.8

75
0.7 0.7
70
0.8
65 0.6 0.6
0.6
60
0.5 0.4 0.5
55

50 0.4 0.2 0.4

45 0 1
0.3 1 0.3
40
0.5 0.5
35 0.2 0.2
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0

(a) µ1 = µ2 = 0.2, µ3 = µ4 = µ5 = 0.6. (a) Optimal values of α.

110 0.8

100
0.7 0.7
0.8
90

80 0.6 0.6 0.6

70 0.4
0.5 0.5
60
0.2
50 0.4 0.4
0
40 1
0.3 1 0.3
30
0.5
0.5
20 0.2 0.2
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0

(b) µ1 = µ2 = 0.6, µ3 = µ4 = µ5 = 0.2. (b) Achieved maximum throughput.

Fig. 8: Effect of routing decision on the system performance.


µ6 = 1, p = 0.8, Mi = 10 for 1 ≤ i ≤ 5, M6 = 100.
110

100

55 0.75 150 90

80
0.7 100
70
50
0.65 60
50
50
0.6
45 0 40
1
0.55 1
30
0.5 0.5
20
0.5
40
0 0
0.45
(c) System delay.
35 0.4
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Fig. 10: Objective: To maximize the system throughput.
µ2 = µ4 = µ5 = 0.5, µ6 = 0.95, p = 0.8, Mi = 10 for
Fig. 9: Effect of routing decision on the system performance. 1 ≤ i ≤ 5, M6 = 100.
µi = 0.5 for 1 ≤ i ≤ 5 , µ6 = 1, p = 0.8, Mi = 10 for
1 ≤ i ≤ 5, M6 = 100.
to maximize the throughput in small buffered systems. In Fig.
10, we provide results that show the optimal routing decisions
2) Effect of µ1 and µ3 on the system throughput in systems and the corresponding achieved throughput. The service rates
with small buffer size of Q2 , Q4 and Q5 are fixed. We find the optimal routing
In this subsection, we provide results for the performance of decisions by applying exhaustive search for different values of
the system in terms of throughput. In this case, our objective is µ1 and µ3 . In Fig. 10a, we show the optimal routing decisions

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
11

for different values of µ3 and µ1 . We observe that when µ1 <


0.9
µ3 , the optimal value of α is close to 0.18. Therefore, almost
80% of the traffic is processed by the MEC server 2. On the 0.8
other hand, when µ1 > µ3 , the largest part of the traffic is
1
handled by the MEC server 1. For the cases of µ1 = µ3 , the 0.7

traffic is split into two flows and the router achieves a balance
0.6
between them. In Fig. 10b, we provide results that show the
optimal achieved throughput for different values of µ1 and µ3 . 0.5 0.5
Note that the maximum achievable throughput is equal to the
0.4
arrival rate, i.e., 0.8. However, the maximum throughput is
not achieved for this setup because of limited buffer capacity. 0
0.3
1
We observe that even for large values of the service rates, 1
the throughput is close to the arrival rate. This indicates that 0.5
0.2
0.5
buffers with higher capacities are required in order to achieve
0.1
a higher maximum throughput. 0 0
3) Effect of µ1 and µ3 on the system delay in systems with
large buffer size (a) Optimal values of α.
In this subsection, we provide results that show the per-
formance of the system in terms of delay. In this case, we
0.6
minimize the delay in systems with large buffer size by
selecting the optimal values of α as shown in Fig. 11. In
0.5
addition, the optimal values of α are depicted in Fig. 11a.
0.8
We observe that when µ1 = µ3 , the optimal routing decision
is to split the traffic into the two flows equally, i.e., α = 0.5. 0.6 0.4

Another interesting observation is that for even small values of 0.4


µ1 and large values of µ3 , the router decides to split the traffic 0.3

between the two flows. There are two reasons that explain this 0.2
phenomenon. The first is that even if the capabilities of one 0
0.2

MEC server are low, they should be still utilized in order to 1


1
increase the performance. The second is that, for the case of 0.1
minimization of the delay, the router decides to send part of 0.5 0.5
the traffic to the first flow because of the less number of queues 0
0 0
facing shorter delay.
4) Throughput - Delay - Drop rate trade-off (b) Achieved minimum delay.
In this subsection, we provide results that show the perfor-
mance region of the system in terms of throughput, delay, and
0.8
system drop rate. In Fig. 12a, the performance region is shown.
We generate the region by selecting different parameters of
1 0.7
the system. In particular, we vary the service rates and the
capacities of Q1 − Q5 , and we take the corresponding points 0.8
0.6
as shown in the diagram. Each horizontal line is created by 0.6
fixing the service rates and varying the capacity of the buffers. 0.5
0.4
The color represents the system drop rate for each different
case. We observe that the capacity of the buffers does not 0.2 0.4
affect the system performance in terms of throughput and 0
drop rate, when the service rates are small. On the other 1 0.3
1
hand, small variations of the service rates have a significant 0.5
0.5
impact on the performance metrics. In Fig. 12b, part of the 0.2
0 0
region is provided. For specific requirements of the system,
for example, low latency, high reliability, or high throughput,
(c) System throughput.
we can provide the proper setup by utilizing the information
given by the performance region figures. Fig. 11: Objective: To minimize the system delay.
µ2 = µ4 = µ5 = 0.5, µ6 = 0.95, p = 0.8, Mi = 50 for
1 ≤ i ≤ 5, M6 = 100.
B. Scaled-up system
In this subsection, we study the performance of a scaled-
up system with two base stations, BS1 and BS2. In addition, model by comparing the analytical and simulations results.
we evaluate the performance of the proposed approximation The routing decisions for all the cases are equal to 0.5.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
12

0.9 0.7 250

µ = 0.9
0.8 0.6
0.6
200
0.7 0.5 0.5

0.6 150
0.4 0.4

0.5
0.3 0.3
100
0.4
0.2
0.2
0.3 50
µ = 0.11 0.1
0.1
0.2 M = 42
0 0
M =2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.1 0
0 100 200 300 400 500 600 700 800
(a) Performance of BS1.
(a) Performance region.

0.52 120
µ = 0.73
µ = 0.57

0.82
µ = 0.49

0.5
5
0.4

100
µ=

0.8 1 0.6
µ = 0.4 0.48

0.78 0.46
0.5 80
µ = 0.39
0.76 0.44
M = 42
0.4 0.42 60
0.74
0.4
0.72 0.3
40
0.38
0.7 0.2 0.36
20
0.68 0.34
0.1
M = 10
0.66 0.32 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0 50 100 150
(b) Effect of traffic level in BS1 on the performance of BS2.
(b) Performance region of our interest.
Fig. 14: Throughput vs. delay performance. p2 = 0.5,
Fig. 12: Performance region. µ3 = µ4 = µ5 = µ, µ6 = 1, µi = 0.5, ∀i, Mi = 50 for 1 ≤ i ≤ 5 and 7 ≤ i ≤ 11.
p = 0.8. Mi = M for 1 ≤ i ≤ 5, M6 = 100. M6 = 100.

2 140 increases. It is easy to see that the analytical results are close
1.8 to the simulation ones which validates the accuracy of the
120 proposed model. In Fig. 14, we provide results that show the
1.6
performance of each base station for different traffic volumes
1.4 100 of BS1. In particular, in Fig. 14a, we observe the trade-off
1.2 between the throughput and delay of BS1. We see that as
80
1
the arrival rate increases, the throughput of BS1 increases.
However, the throughput of BS1 is not equal to the arrival
0.8 60
rate for large values of the arrival rate because of the task
0.6 drops.
40
0.4 In Fig. 14b, we provide results that show the trade-off
between throughput and delay of BS2 as the arrival rate of
0.2 20
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 BS1 increases. Our goal is to show how the traffic volume of
BS1 affects the performance of BS2. It is clear that when the
Fig. 13: System throughput vs. system delay. µi = 0.6 for arrival rate of BS1 is larger than the arrival rate of BS2, the
1 ≤ i ≤ 5 and 7 ≤ i ≤ 11. µ = 1. M6 = 100, Mi = 50, for performance of BS2 decreases significantly.
1 ≤ i ≤ 5 and 7 ≤ i ≤ 11. In Fig. 15, we provide results that show how the traffic
volume of BS2 affects the traffic of BS1. In this case, the
service rates are higher than the previous case of Fig. 14. We
In Fig. 13, the results show the trade-off between system observe that under this setup, the one system affects the other
throughput and system delay as the traffic of both base stations only in terms of delay. The maximum throughput of BS1 is
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
13

Subsystem 6: Q11 − Q12 , Sybsystem 7: Q13 , Subsystem 8:


18.7
0.7 Q14 .
0.695 18.6 We compare the analytical with simulation results for
0.69 18.5 different routing probability decisions α. We observe that
0.685 18.4
the proposed approximation model works well also in the
0.68 larger setup. The analytical and simulation results are almost
18.3
0.675
coincide.
18.2
0.67
18.1
0.665 VII. S UMMARY
18
0.66
Inspired by the promising network flexibility and scalability
0.655 17.9
provided by NFV, and the decreasing of latency by offloading
0.65
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
17.8 tasks in the MEC servers, in this paper we model an end-to-
end communication system that consists of both technologies.
(a) Performance of BS1. We provide the methodology for analyzing the system and
deriving approximate analytical expressions for the end-to-
1 140
end delay, throughput, and drop rate. Simulation results show
that our stochastic model performs well in terms of the
0.9
120 accuracy. We show how these techniques can be applied in
0.8
order to study the performance of larger systems with arbitrary
100
0.7 number of servers. Furthermore, simulation results show how
0.6
80
different routing policies, and resource allocation schemes
0.5 affect the performance of the systems in terms of each different
0.4
60 performance metrics.
0.3 40
0.2 ACKNOWLEDGMENT
20
0.1
The authors would like to thank Mr. Manuel Stein from
0 0 Nokia Bell-Labs in Stuttgart for the numerous fruitful discus-
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
sions and his valuable suggestions.
(b) Performance of BS2.

Fig. 15: Throughput vs delay performance. p1 = 0.7. A PPENDIX A


µi = 0.5 for 1 ≤ i ≤ 5 and 7 ≤ i ≤ 11. µ6 = 1, Mi = 50 for
1 ≤ i ≤ 5 and 7 ≤ i ≤ 11. M6 = 100. We model the queue in the core server of the system as a
Markov chain. The Markov chain is depicted In Fig. 16b. In
order to facilitate the presentation, we introduce a notation
achieved for all the values of p2 . However, the effect on delay which allows us to design the transitions between states
is more significant. in a more compact way. Below, we calculate the transition
probabilities of the Markov chain.

C. The multi-server case p00 = λ̄6,2 λ̄6,5 λ̄6,8 λ̄6,11 ,


In this subsection, we provide additional results for evaluat- p01 = λ6,2 λ̄6,5 λ̄6,8 λ̄6,11 + λ̄6,2 λ6,5 λ̄6,8 λ̄6,11
ing the performance of our approximation model for systems + λ̄6,2 λ̄6,5 λ6,8 λ̄6,11 + λ̄6,2 λ̄6,5 λ̄6,8 λ6,11 ,
with multiple VNFs and multiple servers. In this setup, we p02 = λ6,2 λ6,5 λ̄6,8 λ̄6,11 + λ6,2 λ̄6,5 λ6,8 λ̄6,11
consider a system with 4 different VNFs and 7 servers as
+ λ6,2 λ̄6,5 λ̄6,8 λ6,11 + λ̄6,2 λ6,5 λ6,8 λ̄6,11
shown in Fig. 17. We consider that the service rates of the
queues of MEC Servers 1, 2, and 3 are identical. More + λ̄6,2 λ6,5 λ̄6,8 λ6,11 + λ̄6,2 λ̄6,5 λ6,8 λ6,11 ,
specifically, µ1 = . . . = µ6 = 0.6. The service rates for p03 = λ6,2 λ6,5 λ6,8 λ̄6,11 + λ6,2 λ6,5 λ̄6,8 λ6,11
the MEC servers 4, 5, 6 are also identical. In particular, the + λ̄6,2 λ6,5 λ6,8 λ6,11 + λ6,2 λ̄6,5 λ6,8 λ6,11 ,
service rates are µ7 = . . . = µ13 = 0.7. The capacities of
p04 = λ6,2 λ6,5 λ6,8 λ6,11 , p10 = p00 Pr {X ≥ 1} ,
the buffers for the MEC servers are identical and equal to 50.
In Fig. 18 and 19, we provide results for the system delay p11 = p00 Pr {X = 0} + p01 Pr {X ≥ 1} ,
and system throughput, respectively. In order to derive the p12 = p01 Pr {X = 0} + p02 Pr {X ≥ 1} ,
analytical results of this example, we apply the decomposition p13 = p02 Pr {X = 0} + p03 Pr {X ≥ 1} ,
method in Section III. We decompose the system as follows:
p14 = p03 Pr {X = 0} + p04 Pr {X ≥ 1} ,
Subsystem 1: Q1 − Q2 , Subsystem 2: Q3 − Q4 , Subsystem 3:
Q5 − Q6 , Subsystem 4: Q7 − Q8 , Subsystem 5: Q9 − Q10 ,

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
14

p04 p15 b0
p03 b0 b0
b1 b1 b0
b1
p00 p11 b4 b4 b4 b4 b4
p01 p10 p12 b5 b3 b5 b3 b5 b3 b5 b3 b5
0 1 2 3 4 5 6 ... ... ... ...
p02 p13 b6 b6 b2 b6 b2
b2 b6 b6
b2 b6 b2 b6
b1 b1
p14
b0

(a) Markov chain for the server in the core.

p ↵
p↵ p ↵
↵ , ↵
p↵
(b) Explanation of the edges.

Fig. 16: Stochastic model of the server in the core.

Q1 Q2 Q3 Q4 Q5 Q6
Flow ↵
p Controller
CPU Tx CPU Tx CPU CPU

µ1 µ2 µ3 µ4 µ5 µ6 Q14
1 ↵ CPU
µ14
Q7 Q8 Q9 Q10 Q11 Q12 Q13
Tx CPU Tx CPU Tx CPU Tx
µ7 µ8 µ9 µ10 µ11 µ12 µ13

Wireless
Wired

Fig. 17: System model with 4 VNFs and 7 servers.

65 0.85

60 0.8

55
0.75

50
0.7
45
0.65
40
0.6
35
0.55
30

0.5
25 0 0.2 0.4 0.6 0.8 1
0 0.2 0.4 0.6 0.8 1

Fig. 18: Simulations vs. analytical results: System delay for Fig. 19: Simulations vs. analytical results: System throughput
different values of α. for different values of α.

p15 = p04 Pr {X = 0} , (18)


b0 = p04 Pr {X = 0} ,
b1 = p03 Pr {X = 0} + p04 Pr {X = 1} ,
R EFERENCES
b2 = p02 Pr {X = 0} + p03 Pr {X = 1} + p04 Pr {X = 2} ,
b3 = p01 Pr {X = 0} + p02 Pr {X = 1} + p03 Pr {X = 2} , [1] E. Fountoulakis, Q. Liao, M. Stein, and N. Pappas, “Traversing virtual
network functions from the edge to the core: An end-to-end perfor-
b4 = p00 Pr {X = 0} + p01 Pr {X = 1} + p02 Pr {X = 2} , mance analysis,” in Proc. IEEE International Symposium on Wireless
b5 = p00 Pr {X = 1} + p01 Pr {X = 2} , Communication System (ISWCS), Aug. 2019.

b6 = p00 Pr {X = 2} .
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
15

[2] R. Mijumbi, J. Serrat, J.-L. Gorricho, N. Bouten, F. De Turck, and [24] Z. Xu, W. Liang, M. Huang, M. Jia, S. Guo, and A. Galis, “Efficient
R. Boutaba, “Network function virtualization: State-of-the-art and re- NFV-enabled multicasting in SDNs,” IEEE Transactions on Communi-
search challenges,” IEEE Communications Surveys & Tutorials, vol. 18, cations, vol. 67, no. 3, pp. 2052–2070, 2019.
no. 1, pp. 236–262, 2016. [25] R. Gouareb, V. Friderikos, and A.-H. Aghvami, “Virtual network func-
[3] M. S. Bonfim, K. L. Dias, and S. F. L. Fernandes, “Integrated NFV/SDN tions routing and placement for edge cloud latency minimization,” IEEE
architectures: A systematic literature review,” ACM Comput. Surv., Journal on Selected Areas in Communications, vol. 36, no. 10, pp. 2346–
vol. 51, no. 6, pp. 1–39, Feb. 2019. 2357, 2018.
[4] Y. Li and M. Chen, “Software-defined network function virtualization: [26] I. Avgouleas, D. Yuan, N. Pappas, and V. Angelakis, “Virtual network
A survey,” IEEE Access, vol. 3, pp. 2542–2553, 2015. functions scheduling under delay-weighted pricing,” IEEE Networking
[5] Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young, “Mobile edge Letters, vol. 1, no. 4, pp. 160–163, 2019.
computing–A key technology towards 5G,” ETSI white paper, vol. 11, [27] H. Feng, J. Llorca, A. M. Tulino, D. Raz, and A. F. Molisch, “Ap-
no. 11, pp. 1–16, 2015. proximation algorithms for the NFV service distribution problem,” in
[6] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, Proc. IEEE International Conference on Computer Communications
“On multi-access edge computing: A survey of the emerging 5G net- (INFOCOM), pp. 1–9, May 2017.
work edge cloud architecture and orchestration,” IEEE Communications [28] S. Lange, A. Grigorjew, T. Zinner, P. Tran-Gia, and M. Jarschel, “A
Surveys & Tutorials, vol. 19, no. 3, pp. 1657–1681, 2017. multi-objective heuristic for the optimization of virtual network function
[7] Q. Ye, W. Zhuang, X. Li, and J. Rao, “End-to-end delay modeling for chain placement,” in Proc. IEEE International Teletraffic Congress
embedded VNF chains in 5G core networks,” IEEE Internet of Things (ITC), vol. 1, pp. 152–160, Sept. 2017.
Journal, pp. 692–704, 2018. [29] J. Liu, Y. Mao, J. Zhang, and K. B. Letaief, “Delay-optimal computation
[8] Q. Duan, “Modeling and performance analysis for service function task scheduling for mobile-edge computing systems,” in Proc. IEEE
chaining in the SDN/NFV architecture,” in Proc. IEEE NetSoft, pp. 476– International Symposium on Information Theory (ISIT), pp. 1451–1455,
481, Jun 2018. Jul. 2016.
[9] W. Miao, G. Min, Y. Wu, H. Huang, Z. Zhao, H. Wang, and C. Luo, [30] Y. Mao, J. Zhang, S. Song, and K. B. Letaief, “Stochastic joint radio
“Stochastic performance analysis of network function virtualization in and computational resource management for multi-user mobile-edge
future internet,” IEEE Journal on Selected Areas in Communications, computing systems,” IEEE Transactions on Wireless Communications,
vol. 37, no. 3, pp. 613–626, 2019. vol. 16, no. 9, pp. 5994–6009, 2017.
[10] A. Lombardo, A. Manzalini, V. Riccobene, and G. Schembra, “An an- [31] D. Han, W. Chen, and Y. Fang, “Power-optimal scheduling for delay
alytical tool for performance evaluation of software defined networking constrained mobile computation offloading,” in Proc. IEEE International
services,” Proc. IEEE Network Operations and Management Symposium Conference on Communication (ICC), pp. 1–6, May 2018.
(NOMS), pp. 1–7, May 2014. [32] X. Lyu, H. Tian, W. Ni, Y. Zhang, P. Zhang, and R. P. Liu, “Energy-
[11] A. Fahmin, Y.-C. Lai, M. S. Hossain, Y.-D. Lin, and D. Saha, “Perfor- efficient admission of delay-sensitive tasks for mobile edge computing,”
mance modeling of SDN with NFV under or aside the controller,” in IEEE Transactions on Communications, vol. 66, no. 6, pp. 2603–2616,
Proc. IEEE Future Internet of Things and Cloud Workshops (FiCloudW), 2018.
pp. 211–216, Aug. 2017. [33] B. Xiang, J. Elias, F. Martignon, and N. Elisabetta Di, “Joint network
[12] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll, and P. Tran-Gia, slicing and mobile edge computing in 5G networks,” in Proc. IEEE
“Modeling and performance evaluation of an openflow architecture,” in International Conference on Communications (ICC), May 2019.
Proc. International Teletraffic Congress (ITC), pp. 1–7, Sept. 2011. [34] A. S. Alfa, Applied discrete-time queues. Springer, 2016.
[13] Y. Goto, H. Masuyama, B. Ng, W. K. Seah, and Y. Takahashi, “Queueing [35] W. Szpankowski, “Stability conditions for some distributed systems:
analysis of software defined network with realistic openflow–based Buffered random access systems,” Advances in Applied Probability,
switch model,” in Proc. IEEE Modeling, Analysis and Simulation of vol. 26, no. 2, pp. 498–515, 1994.
Computer and Telecommunication Systems (MASCOTS), pp. 301–306, [36] R. M. Loynes, “The stability of a queue with non-independent inter-
Sept. 2016. arrival and service times,” Mathematical Proceedings of the Cambridge
[14] B. Xiong, K. Yang, J. Zhao, W. Li, and K. Li, “Performance evaluation Philosophical Society, vol. 58, no. 3, pp. 497–520, 1962.
of openflow-based software-defined networks based on queueing model,” [37] G. H. Golub and C. F. V. Loan, Matrix Computations, Fourth Edition.
Computer Networks, vol. 102, pp. 172–185, 2016. The Johns Hopkins University Press, 2013.
[15] H. Feng, J. Llorca, A. M. Tulino, and A. F. Molisch, “Optimal dynamic [38] D. P. Bertsekas and R. G. Gallager, Data networks. Prentice-Hall
cloud network control,” IEEE/ACM Transactions on Networking, no. 99, International New Jersey, 1992.
pp. 1–14, 2018.
[16] M. Barcelo, J. Llorca, A. M. Tulino, and N. Raman, “The cloud service
distribution problem in distributed cloud networks,” in Proc. IEEE
International Conference on Communications (ICC), pp. 344–350, Jun.
2015.
[17] H. Feng, J. Llorca, A. M. Tulino, and A. F. Molisch, “Dynamic network
service optimization in distributed cloud networks,” in Proc. IEEE
International Conference on Computer Communications (INFOCOM)
Workshops, pp. 300–305, Apr. 2016.
[18] ——, “Optimal control of wireless computing networks,” IEEE Trans-
actions on Wireless Communications, vol. 17, no. 12, pp. 8283–8298,
2018.
[19] X. Chen, W. Ni, I. B. Collings, X. Wang, and S. Xu, “Automated function Emmanouil Fountoulakis received the diploma
placement and online optimization of network functions virtualization,” degree in electronic and computer engineering for
IEEE Transactions on Communications, vol. 67, no. 2, pp. 1225–1237, the Technical University of Crete (TUC), Chania,
2019. Greece, in 2016. He is currently pursuing the Ph.D.
[20] L. Gu, D. Zeng, S. Tao, S. Guo, H. Jin, A. Y. Zomaya, and W. Zhuang, degree at the Department of Science and Technology
“Fairness-aware dynamic rate control and flow scheduling for network (ITN), Linköping University (LiU), Sweden. His
utility maximization in network service chain,” IEEE Journal on Se- research interest includes low-latency communica-
lected Areas in Communications, vol. 37, no. 5, pp. 1059–1071, 2019. tions, age-of-information, applied queueing theory,
[21] R. Cohen, L. Lewin-Eytan, J. S. Naor, and D. Raz, “Near optimal stochastic network optimization, and resource man-
placement of virtual network functions,” in Proc. IEEE International agement.
Conference on Computer Communications (INFOCOM), pp. 1346–1354,
Apr. 2015.
[22] L. Wang, Z. Lu, X. Wen, R. Knopp, and R. Gupta, “Joint optimization
of service function chaining and resource allocation in network function
virtualization,” IEEE Access, vol. 4, pp. 8084–8094, 2016.
[23] L. Qu, C. Assi, and K. Shaban, “Delay-aware scheduling and resource
optimization with network function virtualization,” IEEE Transactions
on Communications, vol. 64, no. 9, pp. 3746–3758, 2016.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/OJCOMS.2020.2966689, IEEE Open
Journal of the Communications Society
16

Qi Liao received the M.S.E.E. degree and the Dr.-


Ing. degree from Heinrich-Hertz-Chair for Informa-
tion Theory and Theoretical Information Technol-
ogy, Technical University of Berlin, Berlin, Ger-
many, in 2010 and 2016, respectively. From 2010 to
2013, she was a Research Associate with Fraunhofer
Institute for Telecommunications, Heinrich Hertz
Institute, Berlin. From 2013 to 2014, she was a Ph.D.
intern with the Department of Statistics and Learning
Research, Bell Labs, Murray Hill, NJ, USA. Since
2015, she has been a Research Scientist with the
End-to-End Network & Service Automation Lab, Nokia Bell Labs, Stuttgart,
Germany. Her current research interests include optimization for multi-agent
systems, stochastic optimization, and computer vision and machine learning
techniques. She holds more than 50 peer-reviewed international research
papers and granted patents.

Nikolaos Pappas received the B.Sc. degree in com-


puter science, the B.Sc. degree in mathematics, the
M.Sc. degree in computer science, and the Ph.D.
degree in computer science from the University
of Crete, Greece, in 2005, 2012, 2007, and 2012,
respectively. From 2005 to 2012, he was a Graduate
Research Assistant with the Telecommunications
and Networks Laboratory, Institute of Computer
Science, Foundation for Research and Technology-
Hellas, and a Visiting Scholar with the Institute
of Systems Research, University of Maryland at
College Park, College Park, MD, USA. From 2012 to 2014, he was a Post-
Doctoral Researcher with the Department of Telecommunications, Supélec,
France. Since 2014, he has been with Linkping University, Norrkping,
Sweden, as a Marie Curie Fellow (IAPP). He is currently an Associate
Professor in mobile telecommunications with the Department of Science and
Technology, Linkping University. His main research interests include the field
of wireless communication networks with emphasis on the stability analysis,
energy harvesting networks, network-level cooperation, age-of-information,
network coding, and stochastic geometry. From 2013 to 2018, he was an Editor
of the IEEE COMMUNICATIONS LETTERS. He is currently an Editor of
the IEEE TRANSACTIONS ON COMMUNICATIONS and the IEEE/KICS
JOURNAL OF COMMUNICATIONS AND NETWORKS.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.

Вам также может понравиться