# Questions tagged [markov-chains]

The markov-chains tag has no usage guidance.

369
questions

**-1**

votes

**0**answers

21 views

### Distribution of markov chain with a stopping time

I have a Markov chain $X_0, X_1, ..., X_\tau$, where $X_0$ is sampled from the stationary distribution and $\tau$ is a stopping time. Is it true that for any fixed $k$, $X_k$, given that $k \leq \tau$,...

**1**

vote

**0**answers

204 views

+50

### Why do we assume that $\mathcal{A}$ is an algebra in this 2003 paper of Bobkov and Tetali?

In the following paper (extended version here), at the beginning of section 3, the authors give two axioms about $\mathcal{A}$. Axiom 1 is about $\mathcal{A}$ being an algebra. I do not see where this ...

**1**

vote

**0**answers

53 views

### Convexity of conditional relative entropy for Markov distributions

Consider two Markov processes $p$ and $q$. The conditional relative entropy between them is
\begin{align}
D(p\parallel q)& =\sum_a p(a)\sum_b p(b\mid a)\log\frac{p(b\mid a)}{q(b\mid a)}\\
& =\...

**2**

votes

**1**answer

147 views

### Markov chain and random iteration of functions

Could anyone give me some references for the convergence rate( assuming having a stationary distribution) of such Markov chain arising from the random iteration of a FINITE number of Lipschitz ...

**3**

votes

**0**answers

187 views

### Maximize an $L^p$-functional subject to a set of constraints

Let
$(E,\mathcal E,\lambda)$ and $(E',\mathcal E',\lambda')$ be measure spaces
$f\in L^2(\lambda)$
$I$ be a finite nonempty set
$\varphi_i:E'\to E$ be bijective $(\mathcal E',\mathcal E)$-measurable ...

**2**

votes

**1**answer

159 views

### Existence and uniqueness of a stationary measure

This same question was also posted on MSE https://math.stackexchange.com/questions/3327007/existence-and-uniqueness-of-a-stationary-measure.
Recently I have posted the following question on MO ...

**2**

votes

**0**answers

88 views

### Characterizing the relationship between element-wise Markov transitions and the full-conditionals of the stationary distribution

Consider a $p$ dimensional random variable with a discrete support. Consider a Markov transition kernel on the state space that is defined in terms of element-wise transition distributions.
One can ...

**1**

vote

**1**answer

107 views

### Can we show that $\mathbb R^{\mathbb N}\ni x\mapsto\bigotimes_{n\in\mathbb N}\mathcal N_{x,\:\sigma^2}$ is a Markov kernel?

Let $\sigma>0$ and $\mathcal N_{x,\:\sigma^2}$ denote the normal distribution with mean $x\in\mathbb R$ and variance $\sigma^2$. From the Ionescu-Tulcea theorem, we know that $$\kappa(x,\;\cdot\;):=...

**1**

vote

**4**answers

207 views

### Probability of traversing all other states and finally landing on one state

This is a cross-post from math.stackexchange.com. There has been no response there.
Given a Markov chain of finite states with constant transition probabilities, what is the method to compute the ...

**4**

votes

**2**answers

243 views

### Stationary distribution of a Markov process defined on the space of permutations

Let $S$ be the set of $n!$ permutations of the first $n$ integers. Let $p\in(0,1)$. Consider the Markov Process defined on the elements of $S$.
Let $x\in S$. Choose $1\le i <i+1 < n$ uniformly ...

**0**

votes

**0**answers

62 views

### A closed form of mean-field equations

Assume that a system at time t, for example number of costumers in a line at time $t$ which is denoted by $q(t)$, follows a Markov chain with these dynamics (probabilities)
$$P(q(t+\Delta t)-q(t)=1)=\...

**2**

votes

**0**answers

69 views

### Distribution of a linear pure-birth process' integral

I stumbled across the following random variable, defined as the integral of a linear pure-birth process i.e. a Yule process:
$$
Z_t = \mathbb{E}\bigg[\int_0^t Y_s ds \bigg| Y_t=k , Y_0=1\bigg]
$$
...

**0**

votes

**0**answers

17 views

### Simulating an inhomogeneous DTMC with transition matrix dependent on Xn

I am simulating an agent navigating through space, where the agent's navigation strategy changes over time as a Markov chain with transition probabilities dependent on its position in space.
...

**6**

votes

**2**answers

223 views

### Slowest initial state for convergence of finite birth-and-death Markov chains

Consider the continuous-time birth-and-death Markov chain on $\{1,\cdots,n\}$ with all rates equal to $1$. Is it true that the convergence to equilibrium, in total variation distance, is slowest when ...

**4**

votes

**1**answer

84 views

### Reference request: When is the variance in the central limit theorem for Markov chains positive?

I'm looking for a reference which gives sufficient conditions for the variance to be positive in the central limit theorem for Markov chains (cf https://en.wikipedia.org/wiki/...