# All Questions

Tagged with stochastic-processes oc.optimization-and-control

24
questions

**1**

vote

**0**answers

38 views

### Stochastic Control: Markovian restriction

Consider a stochastic control problem,
$$v^C(0,x) = \mathbb{E} \Big[\int_0^\tau f(X_t,C_t) d t + (T-\tau)|X_\tau|\Big] $$
where $X_t$ is a weak solution to the SDE
$$dX_t = C_t dB_t, \quad X_0 = x \...

**0**

votes

**0**answers

41 views

### Reference: Stochastic Optimal Control with cost functional

There are a variety of control problems for controlled diffusions $X_t^u$, with the terminal cost given by
$$
J(u)\triangleq \mathbb{E}\left[g(X_T,u)+\int_0^t h(X_t,u_t)ds\right],
$$
function $g$ and ...

**1**

vote

**1**answer

225 views

### When do supremum and expectation commute?

This is an alternative form of the question in When do maximum and expectation commute?
When we looking for conditions on $G(t,x(t))$ such that
$$
\sup\limits_{t\in [0,N]}E[G(t,X(t))]=E[\sup\limits_{...

**1**

vote

**0**answers

38 views

### Smoothness of an optimal control problem for a point process

Let $\theta \in \{0,1\}$ be an unknown state of the world. Let $P_0 := Prob(\theta = 1)$ at time $0$.
Let $G_t$ and $B_t$ be two Poisson processes with stochastic intensity $\lambda_g e_t \theta$ ...

**3**

votes

**0**answers

148 views

### Optimal control of SDEs

I've set up a system of stochastic differential equations that I'd like to control. I'm new to optimal control theory and SDEs (and, admittedly, weak on PDEs), so I'm not certain if I've set this up ...

**1**

vote

**0**answers

53 views

### Stochastic Control with Stochastic Cost-functional

Is there any literature dealing with a stochastic control problem whose cost-functional $J_t$ is stochastic also?
That is, let $X_t^u$ is the solution to a controlled SDE
$$
dX_t = \mu(t,u_t,X_t^u)dt ...

**3**

votes

**1**answer

169 views

### Superlinear Convergence of a Markov Chain

Suppose that we have a Markov process $\{Z_t\}_{t=0}^\infty$, where $Z_t \geq 0$ for any $t$. Assume that, conditioning on $Z_t = z_t$, we have
$
\mathbb{E}\{Z_{t+1}|Z_t = z_t\} \leq \kappa z_t^2
$. ...

**6**

votes

**1**answer

346 views

### Supremum of a stochastic process

Let $(x_1,...,x_N)$ be points in $R^d$, and $\sigma=(\sigma_1,...,\sigma_N)$ are i.i.d. Rademacher variables (+1 or -1 with probability 0.5 each). (Or alternatively, $\sigma$ could be a standard ...

**1**

vote

**2**answers

535 views

### Deriving the HJB equation for exponential utility

I would like to derive the HJB equation for the following stochastic optimal control problem:
$ \Phi(t,x)=\sup_{h} E \left[\exp \left\{\gamma \int_t^T g(X_s,h(s);\gamma)\ ds \right\} \right]$
where ...

**4**

votes

**0**answers

182 views

### Stochastic subgradient descent almost sure convergence

I was reading up on stochastic subgradient descent, and most sources i could find via google search give quick proofs on convergence in expectation and probability, and say that proofs of almost sure ...

**3**

votes

**1**answer

324 views

### Conditions for supremum and conditional Expectation to commute

I am working with a continuous process $Y_t$ generating the filtration $(F_t)$ and have (for simplicity) two stopping times $\tau_1$ and $\tau_2$ such that $\tau_2 \leq \tau_1$ and $U:\Bbb R\...

**3**

votes

**1**answer

157 views

### Optimal control / Portoflio optimization: Maximize expected utility of total consumption

I came across a portoflio optimization problem, where I need to solve for optimal investment and consumption processes, such that the expected utility of total consumption and terminal wealth is ...

**0**

votes

**0**answers

74 views

### Law of motion when initial condition is perturbed

We know how to find the law of motion (Ito process) of the value function:
$$V_t(x)=E\Big{[}\int^{T}_te^{-r (s-t)}f(s,X_s)ds+e^{-r (T-t)}g(T, X_{T})|\mathcal{F}_t\Big{]}$$
such that
$$dX_t=\mu(t,X_t)...

**2**

votes

**1**answer

391 views

### Stochastic differential equation associated with an optimal control problem

We know how to find the stochastic differential equation (Hamilton-Jacobi-Bellman equation, HJB) of the control problem where a process $X_t$ is controlled up until it is stopped at a stopping time $\...

**1**

vote

**1**answer

532 views

### Supremum in a Markov chain model

A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach ...