Imbedded Markov chains for certain extended queueing processes

One of the basic characteristics of queueing systems is the stochastic process {n(t),t≥0}, which is defined to be the number of units present in the system at time t. In certain cases, this process is Markovian and then its analysis is relatively simple. When the process {n(t),t≥0} is not Markovian, its "Markovization'' can be accomplished by a suitable extension of the states of the system or by the construction of a suitable imbedded Markov chain. The method of extension of the states of the system, which depends on the formation of a vector process, one of whose components is the process {n(t),t≥0}, gives the characteristics of the process {n(t),t≥0}. The method of imbedded Markov chains, which consists of investigating the process in a suitably chosen sequence of time points, yields the characteristics of the process only at the selected time points. For example, a GI/M/N system may be analyzed at the moments of time at which the units enter the system. Because of this, when the method of imbedded Markov chains is used, the interesting characteristics of continuous-time processes can be obtained only with some additional effort. (MR0467961)