KERNEL SMOOTHING OF THE MEAN PERFORMANCE FOR HOMOGENEOUS CONTINUOUS TIME SEMI-MARKOV PROCESS Текст научной статьи по специальности «Математика»

KERNEL SMOOTHING OF THE MEAN PERFORMANCE FOR HOMOGENEOUS CONTINUOUS TIME SEMI-MARKOV PROCESS

Tayeb Hamlat1, Fatiha Mokhtari2 and Saadia Rahmani3*

•

1,2,3 Laboratory of Stochastic Models, Statistic and Applications University of Saida Dr. Taher Moulay, BP 138, Ennasr, 20000, Algeria [email protected] [email protected] [email protected] * Corresponding author

Abstract

The main goal of the present paper is to propose a systematic approach to model performance measurements within the context of continuous-time semi-Markov processes with a finite state space. Specifically, the mean performance is estimated using the kernel method. The uniform strong consistency and the asymptotic normality of the proposed estimator is investigated. Furthermore, a non-parametric kernel estimation of the expected cumulative operational time is addressed. The constructed estimator is proved to be consistent and to converge to a normal random variable as the time of observation becomes large. As an illustration example, a simulation study has been conducted in order to highlight the efficiency as well as the superiority of our method to the standard empirical method.

Keywords: Semi-Markov processes, Kernel estimator, Mean performance, Reliability, Cumulative operational time, Consistency, Asymptotic normality.

1. Introduction and motivation

Stochastic process theory can be seen as an extension of probability theory that allows modeling the evolution of a system through the time. In addition, any serious study of renewal processes would be impossible without using the powerful tool of Markov processes. Markov processes are significant because, in addition to modeling a wide range of interesting phenomena, their lack of memory property allows for the computation of probabilities and expected values that quantify the behavior of the process and the prediction of its potential behavior. Therefore, the starting step of many attempts to model continuous-time processes has been the Markov process. However, the Markov property has its limitations. It imposes restrictions on the distribution of the sojourn time in a state, which is exponentially distributed in the continuous case, though this is not realistic in general. It is adequate to assume that the probability of leaving a state depends on the time already spent there. More precisely, it has become clear that the propensity to move from one state to another often depends strongly on the length of stay in that state. Therefore, any adequate model must incorporate this feature and the semi-Markov process meets the case.

The study of the semi-Markov process is related to the theory of Markov renewal processes (MRP) which can be considered as an extension of the classical renewal theory (see for instance,

Pyke [23], [24] and Limnios and Oprisan [13]). More precisely, the semi-Markov processes generalize the renewal processes as well as the Markov jump processes and have numerous applications, especially in reliability (see the pioneer work of Janssen [9] which has made a state of the art for the area of the semi-Markov theory and its applications). Furthermore, there are numerous real-world scenarios where semi-Markov models are relevant, such as in fault-tolerant systems, computer systems and networks, manufacturing systems, healthcare systems, and others.

There is a growing need to assess reliability and performance measure (see, for example, Smith et al. [27]). Meyer [16] created a conceptual framework for performability, defining it as the probability that a system achieves a certain level of accomplishment during a utilization interval [0, t]. In other words, for the semi Markov process (Zt), that describes the evolution of the system through a set of states E, the performance level, or a reward rate, L(j) is associated with each state j € e, where the reward function L : E ^ r is proposed to be a measure of performance per unit time. The resulting semi-Markov reward process is then able to capture not only the failure and repair of the system components, but the degradable performance as well. Therefore, the development of a performance model is badly needed when we are interested in the level of productivity of a system.

The accumulated reward until time t will be fy(t) = f0 L(Zu)du which is an integral functional of the process (Zt). Integral functionals are very important from theoretical as well as practical point of view. Indeed, in martingale theory, integral functionals are very useful since they are used as compensators, see Koroliuk and Limnios [12]. In statistics, they are used as empirical estimators for stationary distributions in semi-Markov processes [15]. In stochastic applications, they are crucial in some reliability studies, in performance evaluation of computer systems [26] and so on.

Since the work of [2] in which the author has defined combined measures of performance and reliability, many researchers have focused on evaluating performability, particularly in cases where (Zt) is a Markov process with a finite number of states. Meyer [17] has studied the case of Markov process when the function L is monotonic. Beaudry [3] introduced an algorithm to compute performability until absorption over an infinite interval. Additionally, Donatiello and Iyer [5], have proposed an algorithms for computing performability that do not require the function L to be monotonic.

The semi-Markov case with a finite number of states has been examined by Iyer et al. [7]. The authors demonstrated that the distribution function of $>(t) satisfies a Markov renewal-type equation and proposed approach to solve it. Fore years later, Ciardo et al. [4] have offered an extension of Beaudry's approach to semi-Markov processes.

Since then, several research papers have been published on the development of estimators and the investigation of their asymptotic properties. In [14], the authors have presented a statistical study of the nonparametric estimation of performability of a finite state space semi-Markov system by using empirical estimator and give consistency and asymptotic normality results for such a system.

A specific scenario occurs when a reward of 1 is assigned to all operational states and 0 to all non-operational states. This case was studied by [21], where the expected reward rate at time t, e(0(t)), is known as the instantaneous or point availability A(t). In this case, $>(t) represents the total time spent in operational states during the interval [0, t].

To the best of our knowledge, there are no existing works on nonparametric kernel estimators of the performance and performability. The main goal of the present paper is to propose a systematic approach to model performance measurements within the context of continuous-time semi-Markov processes with a finite state space. Specifically, the mean performance is estimated using the kernel method. The uniform strong consistency and the asymptotic normality of the

proposed estimator is investigated. Furthermore, a non-parametric kernel estimation of the expected cumulative operational time is addressed. The constructed estimator is proved to be consistent and to converge to a normal random variable as the time of observation becomes large. As an illustration example, a simulation study has been conducted in order to highlight the efficiency as well as the superiority of our method to the standard empirical method.

The remainder of the paper is organized as follows. Section 2 presents some definitions and notations of the semi-Markov processes in a countable state space, and these are needed in the work's sequel. An explicit expression of the mean performance of a finite state space semi-Markov system is given in Section 3. The basic elements of statistical estimation are given in Section 4. In Section 5, we introduce and discuss in detail the necessary conditions for establishing the asymptotic properties of the proposed estimator. In Section 6, we illustrate these concepts, measures and estimators through the cumulative operational time as well as a numerical study. Some concluding remarks are given in Section 7.

2. Semi-Markov system and related quantities

Definition 1. (Markov renewal process) Let E = {1,...,s} be the state space. A Markov renewal process is a bivariate stochastic process (Jn, Tn) where Jn is the system state at the nth time, and Tn is the nth jump time, we set T0 = 0. The process has to satisfy the following formula:

P (Jn+1 = j, Tn+1 " Tn < t | J0, J1,..., Jn, T0, T1,..., Tn) = P (Jn+1 = j, Tn+1 - Tn < 11 Jn), (1) for all j e E, all t e R+ and all n e N.

Moreover, if Equation (1) is independent of n, (Jn, Tn) is considered to be time homogeneous.

Definition 2. (Continuous-time semi-Markov process) Consider a Markov-renewal process {(Jn, Tn) : n e n} defined on a complete probability space and with state space E. The stochastic process {Zt; t e r+ } defined by

Zt = JN(t), (2)

is called a Semi-Markov Process (SMP) where N(t) := sup {n > 0 | Tn < t} is the counting process of the SMP up to time t. Let us also define Xn = Tn — Tn—\, n > 1, the inter-jump times

of Z = (Zt)teR+.

Let us also introduce some functions associated with the process Z:

• The semi-Markov kernel Q(t) = {Qij(t),i, j e E} , t > 0 of Z, is given by

Qij(t) = p (Jn+1 = j, Xn+1 < t I Jn = i) ,

and pj := Qij(w) = p (Jn+1 = j | Jn = i) with p = (pij)ijeE is the transition matrix of the process (Jn) which is called the embedded Markov chain (EMC) of Z.

• The conditional sojourn time distribution in state i, given that the next state to be visited is j, denoted Fj, is defined by

Fij(t) := p (Xn+1 < t I Jn = i, Jn+1 = j).

Meanwhile, the sojourn time distribution in state i, denoted Hi, is defined, for every t e r+, by

H(t) = p (Xn+1 < 11 Jn = i) = E Qij(t),

jeE

and its corresponding survival function is defined by Hi(t) = 1 — Hi(t).

If we consider g to be a locally bounded function and G to be a real right continuous nondecreasing function both defined on r+, the Stieltjes convolution of the function g with the function G is defined, for every t € r+, by

g * G(t)= i g(t - x)dG(x) = f g(t - x)dG(x). j R j0

Furthermore, when G and F are cumulative distribution functions, we have

G * F(t)= f* G(t - x)dF(x) = f* F(t - x)dG(x) = F * G(t).

00

Now, consider the n-fold convolution of Q by itself, for any i, j € E,

(t)

1{i=j,t>0} if n = ° Qij(t) if n = 1,

E /f Qik(ds)Q(j -1)(t - s) if n > 2.

k€E 0

The Markov renewal function denoted Yij(■), is defined, for every i, j € E, t > 0, by

TO

Yij(t) = Ei[ E 1{ Jn =j,Tn <t} ]

in —jtL n <

n=0

CO

= E pi (Jn = j,Tn < t) = E Qit)(t).

n=0 n=0

The matrix renewal function is given by

TO

V(t)= E Q(n)(t),

n=0

where Y(t) = [Yij(t)].

The matrix renewal function Y(t) is the solution of the following Markov renewal equation

■f(t) = I(t) + Q * Y(t)1, where I(t) = I when t > 0 and I(t) = 0 when t < 0.

• The transition matrix function P(t) = [Pj(t)] of the semi-Markov process is defined, for

every i, j € E, t > 0, by

Pij(t) = p (Zt = j | Z0 = i) = p (jm = j | J0 = i) .

It is known, cf. [23], that

Pij(t) = l{i=j} - E Qik(t)^j + E J* Pkj(t - s)Qik(ds).

By solving the above Markov renewal equation, cf. [13], it is seen that, in matrix notation, we have

P(t) = (Y * (I - H))(t),

where (I - H) (t) = diag[1 - Hi(t)].

Definition 3. For a semi-Markov process (Zt)*€r+ , the limit distribution n = (k\,...,ks) is defined, when it exists, for every i,j € E, by

nj = lim Pij(t).

stands for the matrix-Stieltjes convolution of an n x r matrix function, A, by an m x n matrix function, B, denoted

n

B * A, which can be defined by (B * A)ij (t) = E Bik * Akj(t), i = 1,..., m, j = 1,..., r.

k=1

3. Mean performance

The performance process at time t > 0, denoted &(t) is the real-valued integral functional of a

homogeneous semi-Markov process (Zt)teR+, cf. [14], defined by

®(t) = f L (Zu) du = £ L(j) f 1{Zu=j}du, (3)

J0 jeE J0

where L is a real-valued function defined on E.

The mean performance at time t > 0, denoted &(t), is defined by

®(t) := E[®(t)] = £ afy(t) = £ £ UiL(j) f tPij(u)du, (4)

ieE ieEjeE J0

where (t) = £ L(j) i Pij(u)du, ai = P (J0 = i) and the row vector

jeE 0

a = (ai : i e E) defines the initial distribution of Z.

4. Elements of statistical estimation

Consider a sample path of the Markov renewal process (Jn, Tn)neN

Y (M) := J Xi^.^J^Myv XN{My Jn(m) , Um) , M e R+, where N(M) is defined in Definition 2 and uM := M — TN(M).

For all i, j e E, we define:

N(M)

• Ni(M) := £ 1

{Jn—1=i}, the number of visits to state i, up to time M.

In—

n=1

N(M)

• Nij(M) := £ 1{Jn_1=irJn =j}, the number of transitions from i to j, up to time M.

n=1

For all i, j e E, t > 0, we define the kernel estimator of Qij and Hi respectively (cf. [1]), by

. N(M) ft_v\

j M) = NM £ Gj ^ j,

and

. N(M) , X x

H(', M) = nM) £ i<J—i=o;

Ni (M) =1 V hi,M

where G(t) = f_K(t)dt, with K is a bounded kernel function.

For fixed states i and j, it should be noted that the smoothing parameter of the previous estimators depends on the sample size, so we should write h^^M) = hij,M (resp. fy^(m) = hi,M), however we prefer to use a simpler notation.

The kernel estimator of the Markov renewal function Yj(t), in matrix form, is given by

TO

p(t, M)= £ Q(n)(t, M). (5)

n=0

The kernel estimator of the transition matrix function P(t) at time t > 0 of the SMP, is given

by

p(t,m)= (y(•,m) * (i — h(.,M)^(t). (6)

Based on Equation (4), an nonparametric kernel estimator of the mean performance <5(t, M) is given by the following expression:

<(t,M) = E E^L(j) f tPij(u,M)du. (7)

i€Ej€E J0

5. Asymptotic properties

The following assumptions are necessary to derive the asymptotic behaviour of the kernel estimator defined in (7).

5.1. Assumptions

First, we will assume the following two assumptions:

(H.1) The EMC (Jn)nen is an ergodic Markov chain, with stationary distribution v. (H.2) The SMP is regular, with finite mean sojourn times m.

Second, the following assumptions are required in order to establish all the asymptotic properties in this paper:

(H.3) i) Qij(t) and qij(t) are continuously differentiable with respect to the Lebesgue measure, and let qij(t) and qjt) be respectively their corresponding Radon-Nikodym derivatives. ii) The derivative qij is bounded.

(H.4) The function G is a distribution function, where its derivative is K.

(H.5) The kernel K is a density function of bounded variation such that lim |xK(x)| = 0 and

J x—tTO

1/ tjKn(t)dt\ < TO for j = 0,1, and n = 1,2. (H.6) The smoothing parameter hij/M satisfies

lim hijM = 0 and lim Mhij M = to.

M—>to " M—y to "

5.2. Comments on the assumptions

The structural assumptions (H.1) and (H.2) are the same as those used classically for the semi-Markov processes framework (see, for instance [1] and [6]). More precisely, the recurrence and the positivity of the EMC (Jn)neN in (H.1) ensure that the stationary distribution nj defined in Definition 3 is strictly positive and unique. Furthermore, since the EMC (Jn)neN is irreducible and aperiodic, the limit in Definition 3 always exists and it is independent of the distribution in the initial state. (H.2) means that the counting process {N(t) : t > 0} has a finite number of jumps in a finite period with probability 1. In addition, under this hypothesis we have Tn < Tn+1, for any n € n, and Tn — to as n goes to infinity. Assumption (H.3) as imposed on Qij(t) and qij(t) is a regularity type hypothesis. Whereas, assumption (H.3)(i) is a regularity constraint using to get the strong consistency. the second derivative hypothesis (H.3)(ii) establishes more restrictive constraints when going through to state the asymptotic normality of our estimators. (H.5)-(H.6) are technical constraints; they are imposed for the sake of the proof's simplicity and brevity.

Before stating our main result, we introduce the following technical lemma which will be

necessary to prove our second result.

Lemma 1. For n = 1,2. If (H.3)-(H.5) hold, we have

1 (f v-x\ _ , , „ , , r+TO

> i — _ x \ f+TO

Kn(-- dQij(x) ^ qij(v) Kn (z) dz + o(hij,M).

\ hii.M J-TO

hij,M 0 hij,M

Proof of Lemma 1 By using a change of variable followed by Taylor's expansion, we have 1 r+ta Kif—t) dQij(x) = ij Kn (z) qdr (- - hijMz)dz

hij,M j0 \hij,Mj J-TO

-

= \j Kn (z) j) - hijMzq'ij(v* )] dz

j-+to

^ qij(v) Kn (z) dz + o(hij,M),

J-TO

where - - hij,Mz < -* < v.

5.3. Main Results

Our first result concerns the uniform strong consistency of the proposed estimator.

Theorem 1. For any fixed 0 < t < M and i € E, under (H.1)-(H.6), the estimator <5(t, M) of <(t) is uniformly strongly consistent, that is

max sup | <(t,M) - <(t) | —>0 as M —> to.

i€E t€[0,M]

Proof of Theorem 1 The proof of this theorem is based on (5), (6), (7) and the following inequality:

max sup | <5i(t,M) - <(t) | < E L(j)t E max sup Qlf^,M) - Q\")(t)

i€E

t€ [0,M]

j€E

n=0 i€E t€[0,M]

+ E L(j)tE max sup Q«(t,M) - Qj\t) * Hj(t, M)

j€E

n=0 i€E t€[0,M]

+ E L(j)t max sup Hj(t,M) - Hj(t) * Yij(t).

j€E i€E t€[0,M]

For all i, j € E, n € n* and M € r+, based on a straightforward adaptation of the proof of

Lemma 1 in [19], we get that the estimator Qj (t, M) is uniformly strong consistent in [0, M]. In

addition, the uniform strong consistency of the kernel estimator Hj(t, M) is stated in Theorem 4.1 of [1]. Then,

max sup | <(t,M) - <(t) |—— 0, as M —> to.

i€E

t€ [0,M]

Before stating our second result, let us consider the renewal process Tni n 0 of successive times of visits to state i, then Ni (t) is the counting process of renewals. Let yii andy* denote the mean first passage times of the state i in the MRP and in the corresponding Markov chain {Jn; n > 0}, respectively. Furthermore, ya is the mean interarrival times of the eventual delayed renewal process (Tln), n > 0, i.e., yii = e [T* - T\], and y* = e [S'H J0 = i] with S* = min{k > 1, Jn = i} is the first visit time to the state i.

Theorem 2. For any fixed 0 < t < M, if (H.1)-(H.6) hold, we have

^MHm [3(t,M) - 5(t)

with hM = min {hijM} and the asymptotic variance

i,jeE

N (0,0gr(t)) , as M —> œ,

ft r r. ( f+œ

4 (t) < EE M (Rij- Di)2 * Qij(-) K2 (z) dz

;cp;cp ->0 _ V J-œ

ieEjeE

(u)du,

where

and

Rij= EE *dL(r) (Ydi * Yjr * Hr),

deEreE

Di = EE *dL(r)1{i=r} Ydr-

deEreE

(8)

(9) (10)

Proof of Theorem 2 We have,

yMhM 3 (t, M) - 5(t) = EE *dL(r)VMhM

d E re E

/ Pdr(u, M)du - Pdr(u)du 00

= £ £ UdL(r)VMhM

deEreE

— (Ydr * (I — Hr))(u)] du]. Note that the last right side can be written as follows:

Y dr (; M) * 11 - Hr (; M)))(u)

0

Y dr(-, M) - Ydr (■)) * [Hr (; M) - Hr (■)))(u)du

E E *dL(r)VMhM

de E re E

+ Jt (Ydr (•) * (Hr (; M) - Hr (■))) (u)du + \ (Ydr (•, M) - Ydr (■)) * Hr (■)) (u)du

According to [1] and by following the same arguments as [18], the first term converges to zero as M tends to infinity.

Consequently, by applying Slutsky's Theorem, we deduce that VMhM <5(t,M) - 5(t) verges in distribution to the same limit as

con-

yMhM E E adL(r)

deEreE

Ydr(•) * Hr(•,M) - Hr(')jl (u)du

+ yo [(Ydr(•,M) - Ydr(■)) * Hr(-))(u)du

By combining Theorem 4.3 (i) of [1] and Theorem 4 (b)[18], along with arguments akin to those employed in [9] p. 214, we deduce that VMhM Y(•,M) - Y( ) (t) has the same limit i

dr

in

distribution as VMhM[Y(-) * AQ * Y(-)]dr(t),

where AQ = (Q — Q), for every t > 0, t < M, and for every d, r e E, which is written as follows:

AQdr (•)

N(M)

Nd (M) E

G ( ] 1{ Jl_1=d,Jl =r} - Qdr (')1{ J-1=d}

dr,M

Furthermore,

Ydr * [Hr - Hr) = - EYdr * AQrk = - EE 1{m=r} Ydr * AQmk.

keE keEmeE

t

0

Then, ^/MhM <(t, M) - <(t)

has the same limit as

N(M)

E EE

M

VM = m^EkeENm(M)

VhM

[ [(Rmk - Dm) * (G (h-M _1=m,h =k}

Qmk (-)1{Jl_1=m}J (u)du

where Rmk and Dm are given in (9) and (10).

Apply central limit theorem related to semi-Markov processes (see [25]) to the function Wf (t) such that

N(M)

Wf (t)= E f (Ji-1, Ji, Xi)

1=1

N(M) M

£ J^Eik^mm

VhM

Jo [(Rmk - Dm)* (K 'h-M 1{-Jl-1=m,Jl =k}

Qmk (-)1{Jl-1 =m}) (u)du

where, for any fixed t > 0, we have defined the function f : E x E x r — r by

f (i, i,x) = E E ntMv hM

meEkeE Nm (M)

Jo (Rmk - Dm) * (^h k m) 1{i=m,i=k} - Qmk(-)1{i=m}j

(u)du

In order to apply the Pyke and Schaufele's CLT, we need to compute the quantities Aj, Ai, Bj, Bi, ri, mf, aj and then (t), using Lemma 1 with assumptions (H.3)-(H.5). We have

Ai = Aii ieE

E L f (1,i,x) dQii(x)

je_E °

fM

jeeieeiee^ Nmm

VhM

(Rmk - Dm)* G

.x

mk,M

{i=m,j=k}

-Qmk(.)1{i=m}) (u)du dQij(x)

H {Ri- Di] (u - v){ i C ^ i) dQij(x))dvdu

hij,M

A ru f+TO

-E / (Rik - Di) (u - v)qik(v) dQij(x)dvdu

kE

Then

Ai < e NM)^ 10 ^Rij- *(o (u)du.

1

o

For Bi and by using Jensen's inequality and Lemma 1, we have

Bi = £ Bij jeE

f + TO

£ [f (i, j, x)]2 dQij(x) 0

jeE

£

jeE

r + TO

M

Ni (M)

(Rij — Di) * G

.x

Hj,M

— £ (Rik — Di) * Qik(.)

keE

(u)du

dQij(x)

< £

+TO f M

-V

hM

jeE-J 0 \N'(M)J

!■ u

— £ (Rik — Di )(u — v)qik (v)dv

keE 0

I [ R — D, (, — v)( jK<v — x

hij,M \ hij,M

dv

Then

B' < "mC

du

dQij(x).

/■t ru 10 JO

(Rij — Di)2 (u — v)

K2

Mm" \"ijM

+ £ (Rik — Di)2 (u — v)q22k(v)

ke E

—2 £ (Rik — Di) (Rij — Di) (u — v)qik(v)"— K

ke E

vx

"ij,M \ "ij,M

dvdu

dQij(x).

Since N'j^) -- -1 (see [13]), when M — +to and from the assumption (H.6), the second and the third term in the last inequality converge to zero, we get

rt r 2 / r+to

/o (Rij — Di)2 * [Qui)/

Bi ^ £ Hu 0 \(Rij — Di)2 * (Qijt)j+ K2 (z) dz

jeE

(u)du.

Furthermore,

rd = £ Ai Hd = 0 as M - to,

ieE H'i 1

mf = — rd = 0 as M — to,

J Hdd

4 (t) = Hr (t),

^ Hdd

where

(t) = £ b,

ieE H'i

2 rt

< * £ £ H'' ^ H dd £ £ ..* ,

ieEjeE H'i J0

r + TO

(Rij — Di)2 * (Qijt) K2 (z) dz

(u)du.

1 mi _,

Then, since h* = — (see [10]) and H'' = — (see [13]), where m = £ m—i is the mean sojourn

time of the MRP; we have

iE

4(t) < ££ Hi' [ \(R'j — Di)2 * (Qijt) f+ta K2 (z) dz

j0 \ j-to

ie E je E

(u)du.

2

1

We obtain from the CLT that \/MhM <5(t, M) - <(t) converges in distribution, as M tends to infinity, to a zero-mean normal random variable, of variance (t) given in (8).

6. Applications

The cumulative operational time is considered as one of the most relevant performance measures for the reliability. In this section, we propose a non-parametric kernel estimator of the expected cumulative operational time for the semi-Markov system. Then, we investigate the asymptotic properties of the proposed estimator, namely, the strong consistency and the asymptotic normality. As an illustration example, we apply the previous results to three-state continuous time SMP.

6.1. The cumulative operational time

The state space E is often split into two subsets for reliability research. The first one, let's say U, is made up of up states, whereas the second one, let's say D, is made up of down states. The start of an essential event, such as a component failure related to some reason or a complete repair, might well be associated with the transition into a state. Since we suppose that the system can be fixed, the process alternates between U and D.

The cumulative operational time is defined by

W (t) = lo 1{Zu €U} du.

It represents the total time that the semi-Markov process Z spends in the set of up states U over the interval [0, t].

Making use of the assumptions (H.1) - (H.2), along with the aid of the arguments used in [8], we obtain the following result:

E vjmj

lim W^ = ,

t—+TO t E vkmk

k€E

where mj = /0°°(1 - Hj(t))dt is the mean sojourn time in state j.

The quantity we aim to analyze is the expected cumulative operational time of a semi-Markov system, denoted by W(t) := E[W(t)]. Which is given by

W(t) = E atWt(t) = E E^tf Pij(u)du, (11)

ieE ieEjeU J0

where Wj(t) = E / Pij(u)du.

j€U °

The expected cumulative operational time serves as a crucial indicator in maintenance studies, facilitating the calculation of average system availability, cf. [21], which is expressed as

1 _ 1 i-1

A(t) = -W(t) = -EE L Pij(u)du.

t t ieEjeU'10

From the definition of the expected cumulative operational time W(t) given in Equation (11), and based on a sample path truncated to the time interval [0, M] of the process, the nonparametric kernel estimator W(t, M) is given by

WW(t,M) = E EuiiPij(u>M)du. (12)

i€Ej€U J0

The asymptotic properties of the proposed estimator are gathered in the following two corollaries.

Corollary 1. For any fixed 0 < t < M, under the same assumptions of Theorem 1, the estimator of the expected operational time, W(t, M) is strongly consistent, that is

sup I W(t,M) — W(t) \ —as M — TO.

te [0,M]

Proof of Corollary 1

This corollary is a particular case of Theorem 1 and then the proof is omitted.

The following result concerns the asymptotic normality of the proposed estimator.

Corollary 2. For any fixed 0 < t < M, we have VMhM W(t, M) — W(t) converges in law, as M tends to infinity, to a zero mean normal random variable with the asymptotic variance

W(t) ^ EEvn[ [(y,--c)2* (qîî(-) f+ k2(z)dz)

ieEjeU 70 L V J-™ J

(u)du,

where

Yij = EE ad (Ydi * Yjr * Hr) and Ci = EE ad 1{i=r} Ydr. deEreU deEreU

Proof of Corollary 2

The proof of this result is based on the same arguments as in the proof of Theorem 2.

6.2. Confidence interval

The main purpose of the confidence interval is to supplement the estimate at a point with information about the uncertainty in this estimate. It is considered as a direct application of the Central Limit Theorem. In order to provide a confidence interval for the expected cumulative operational time W(t), we need first to propose a consistent estimator of the variance ffW(t). A natural consistent estimator of this variance, denoted by crW(t, M), is obtained by estimating the parameters involved in this quantity such as Qmk(t), Hj(t) and Yim(t).

Indeed, from the strong consistency of Qmk(t, M), Hj(t, M) and Yim(t, M), (see the proof of Theorems 1 and 2 as well as Theorem 4.1 and Theorem 4.2 (v) in [1]), we deduce the strong consistency of crW(t, M). Consequently, from Corollary 2, we get

\JMhM W(t, M) — W(t) —— N (0, oW(t, M)^j .

Then _

W(t, M) — W(t)l —— N (0,1).

VMHm

(t, M)

Hence, for a e (0,1), an asymptotic 100(1 — a)% confidence interval for W(t, M) can be straightforwardly computed:

I = (W(t, M) ± zM^M)

V v ; 2 VMhM

where za is the upper | quantile of the standard normal distribution.

6.3. Numerical application

To validate our results, we consider a three state system whose state transition diagram is given in Figure 1. States 1 and 2 are up states and state 3 is a down state.

We have two exponential and two Weibull distribution functions as conditional transitions, for all x > 0, say H12(x) = 1 - exp (-A1 x), H31 (x) = 1 - exp (—A2x),

H23(x) = 1 - exp

«1

, H21 (x) = 1 - exp

«1

The parameters of these distributions are: Ai = 0.1, A2 = 0.2, a1 = 0.3, = 2, a2 = 0.1, j82

Figure 1: A three state semi-Markov system.

The transition probability matrix of the embedded Markov chain (Jn) is:

i 0 10 \ P = I 0.95 0 0.05 I

V 1 0 0 /

Where the system is defined by the initial distribution a = (1/3,1/3,1/3).

To construct the kernel estimator for the mean performance of a continuous-time semi-Markov process. The smoothed function K(-) is chosen to be the quadratic function defined as K(u) = |(1 - u2) for |u|< 1 and the cumulative distribution function G(u) = jlTO | (1 - z2) 1[-i,i] (z)dz. The bandwidth hM has been obtained by the "PBbw" method, which computes the plug-in bandwidth of the Polansky and Baker method, cf. [22]. We have considered that the observation period is the interval [0, M] with M = 20000.

Figure 2 gives a comparison between the kernel estimator of the mean performance for different sample sizes (M = 2000, M = 10000 and M = 20000 ). We observe that this estimator converges to the true value of the mean performance as M increases.

0 20 40 60 00 100

Figure 2: Comparison between the kernel estimator of the mean performance for different sample sizes and the true value.

0 20 40 60 80 100

Figure 3: Comparison between the true values of the mean performance and their estimators (empirical and kernel).

Figure 3 gives a comparison between the empirical estimator (see [14]) and our kernel estimator of the mean performance. We remark easily that, our method provides better results than the empirical one.

7. Concluding remarks

The application of the nonparametric kernel approach to estimate the mean performance of a continuous-time semi-Markov process is the main element of the work described in this paper. We have proposed a kernel estimator for this quantity then we have provided its asymptotic properties, such as the uniform strong consistency, as well as the asymptotic normality. Compared to the empirical estimator, the use of this kernel technique approach has a number of benefits. Since the empirical function is always a discontinuous function, the kernel smoothing in particular prevents discontinuities in this function. As a result, the empirical distribution may be considered a poor approximation when knowing that the underlying distribution is continuous. To the best of our knowledge, no limit theorems have been obtained for functionals of homogeneous semi-Markov processes, such as the performance and the related quantities, by using the kernel approach. In particular, we have made an important connection of our results with the reliability theory by focusing on the cumulative operational time of the semi-Markov systems. This crucial indicator is the total time spent by the process in the set of operational states during a specific time interval. It is used to minimize the expected cost of the maintenance process. The uniform strong consistency and asymptotic normality have been stated. In addition, a confidence interval has been constructed. Moreover, a simulation study has been conducted in order to highlight to the efficiency as well as the superiority of our method to the standard empirical method.

Acknowledgements

The authors are grateful to Professor Nikolaos Limnios for his valuable comments and suggestions which improved substantially the quality of this paper.

Conflict of interests The authors declare that there is no conflict of interest.

References

[1] Ayhar, C., Barbu, V.S., Mokhtari, F. and Rahmani, S. (2022). On the asymptotic properties of some kernel estimators for continuous-time semi-Markov processes. Journal of Nonparametric Statistics 34, 299-318.

[2] Beaudry, M.D. (1978). Performance-related reliability measures for computing systems. IEEE Transactions on Computers 27, 540-547.

[3] Beaudry, M. (1984). Performance related reliability for computer systems, IEEE Trans. Comput., C-27, 540-547.

[4] Ciardo, G., Marie, R.A., Sericola, B. and Trivedi, K.S. (1990). Performability analysis using semi-Markov reward processes. IEEE Transactions on Computers 39, 1251-1264.

[5] Donatiello, L., Iyer, R (1987). Analysis of a composite performance reliability measure for fault-tolerant systems, J. Assoc. Comput. Mach., 34, No.1,179-199.

[6] Dumitrescu, M., Gamiz, M. and Limnios, N. (2016). Minimum divergence estimators for the radon-nikodym derivatives of the semi-Markov kernel. Statistics 50, 486-504.

[7] Iyer, RR., Donatiello, 1., Heidelberger, P. (1986). Analysis of performability models of fault-tolerant systems, IEEE Trans. Comput., C-35 (10), 902-907.

[8] Howard, R. A. (1964). Research in semi-markovian decision structures. J. Oper. Res. Soc. Japan, 6(4), 114-124.

[9] Janssen, J. (2013). Semi-Markov Models: Theory and Applications. Springer, New York, NY.

[10] Kemeny, J.G., Snell, J.L. (1976). Finite Markov Chains. Springer, New York.

[11] Korolyuk, V., Swishchuk, A. (1995). Semi-Markov Random Evolutions. Springer, Dordrecht.

[12] Korolyuk, V. S. and Limnios, N. (2005). Stochastic Systems in Merging Phase Space, World Scientific, Singapore.

[13] Limnios, N., Oprisan, G. (2001). Semi-Markov Processes and Reliability. Springer, Birkhauser Boston.

[14] Limnios, N., Ouhbi, B., Platis, A. and Sapountzoglou, G. (2006). Nonparametric estimation of performance and performability for semi-Markov processes. International Journal of Performability Engineering 2,19-27.

[15] Limnios, N. (2006). Estimation of the stationary distribution of semi-Markov processes with Borel state space, Statistics & Probability Letters, 76, 1536-1542.

[16] Meyer, J.F. (1980). On evaluating the performability ofdegradable computing systems. IEEE Transactions on computers.

[17] Meyer, J. (1982). Closed-form solutions of performability, IEEE Trans. Comput., C-31, 648-657.

[18] Mokhtari, F., Ayhar. C, Vlad. S.B, and Rahmani. S. (2024). Kernel estimators of markov renewal and semi-markov transition functions of semi-markov systems. revised.

[19] Ouhbi, B., Limnios, N. (1997). Comportement asymptotique de la matrice de renouvellement markovien. Comptes Rendus de l'Académie des Sciences-Series I-Mathematics 8, 921-924.

[20] Ouhbi, B., Limnios, N. (1999). Nonparametric estimation for semi-Markov processes based on its hazard rate functions. Statistical Inference for Stochastic Processes 2, 151-173.

[21] Ouhbi, B., Boudi, A. and Tkiouat, M. (2007). The expected cumulative operational time for finite semi-Markov systems and estimation. RAIRO-Operations Research 41, 399-410.

[22] Polansky, A.M., Baker, E.R. (2000). Multistage plug-in bandwidth selection for kernel distribution function estimates. Journal of Statistical Computation and Simulation 65, 63-80.

[23] Pyke, R. (a1961). Markov renewal processes with finitely many states. The Annals ofMathe-matical Statistics 32, 1243-1259.

[24] Pyke, R. (b1961). Markov renewal processes: definitions and preliminary properties. The Annals of Mathematical Statistics 32,1231-1242.

[25] Pyke, R., Schaufele, R. (1964). Limit theorems for Markov renewal processes. The Annals of Mathematical Statistics 35,1746-1764.

[26] Sahner, R. A., Trivedi, K., & Puliafito, A. (2012). Performance and reliability analysis of computer systems: an example-based approach using the SHARPE software package. Springer Science & Business Media.

[27] Smith, R.M, Trivedi, K.S, Ramesh, A.V. (1988). Performability analysis: Measures, an algorithm and a case study, IEEE Trans. Comput., C-37, 406-417.