# All Questions

Tagged with stochastic-processes oc.optimization-and-control

25 questions

**0**

votes

**0**answers

35 views

### Stochastic Control: Markovian restriction

Consider a stochastic control problem,
$$v^C(0,x) = \mathbb{E} \Big[\int_0^\tau f(X_t,C_t) d t + (T-\tau)|X_\tau|\Big] $$
where $X_t$ is a weak solution to the SDE
$$dX_t = C_t dB_t, \quad X_0 = x \...

**0**

votes

**0**answers

40 views

### Reference: Stochastic Optimal Control with cost functional

There are a variety of control problems for controlled diffusions $X_t^u$, with the terminal cost given by
$$
J(u)\triangleq \mathbb{E}\left[g(X_T,u)+\int_0^t h(X_t,u_t)ds\right],
$$
function $g$ and ...

**1**

vote

**1**answer

171 views

### When do supremum and expectation commute?

This is an alternative form of the question in When do maximum and expectation commute?
When we looking for conditions on $G(t,x(t))$ such that
$$
\sup\limits_{t\in [0,N]}E[G(t,X(t))]=E[\sup\limits_{...

**1**

vote

**0**answers

37 views

### Smoothness of an optimal control problem for a point process

Let $\theta \in \{0,1\}$ be an unknown state of the world. Let $P_0 := Prob(\theta = 1)$ at time $0$.
Let $G_t$ and $B_t$ be two Poisson processes with stochastic intensity $\lambda_g e_t \theta$ ...

**3**

votes

**0**answers

141 views

### Optimal control of SDEs

I've set up a system of stochastic differential equations that I'd like to control. I'm new to optimal control theory and SDEs (and, admittedly, weak on PDEs), so I'm not certain if I've set this up ...

**1**

vote

**0**answers

51 views

### Stochastic Control with Stochastic Cost-functional

Is there any literature dealing with a stochastic control problem whose cost-functional $J_t$ is stochastic also?
That is, let $X_t^u$ is the solution to a controlled SDE
$$
dX_t = \mu(t,u_t,X_t^u)dt ...

**3**

votes

**1**answer

169 views

### Superlinear Convergence of a Markov Chain

Suppose that we have a Markov process $\{Z_t\}_{t=0}^\infty$, where $Z_t \geq 0$ for any $t$. Assume that, conditioning on $Z_t = z_t$, we have
$
\mathbb{E}\{Z_{t+1}|Z_t = z_t\} \leq \kappa z_t^2
$. ...

**6**

votes

**1**answer

339 views

### Supremum of a stochastic process

Let $(x_1,...,x_N)$ be points in $R^d$, and $\sigma=(\sigma_1,...,\sigma_N)$ are i.i.d. Rademacher variables (+1 or -1 with probability 0.5 each). (Or alternatively, $\sigma$ could be a standard ...

**1**

vote

**2**answers

510 views

### Deriving the HJB equation for exponential utility

I would like to derive the HJB equation for the following stochastic optimal control problem:
$ \Phi(t,x)=\sup_{h} E \left[\exp \left\{\gamma \int_t^T g(X_s,h(s);\gamma)\ ds \right\} \right]$
where ...

**4**

votes

**0**answers

181 views

### Stochastic subgradient descent almost sure convergence

I was reading up on stochastic subgradient descent, and most sources i could find via google search give quick proofs on convergence in expectation and probability, and say that proofs of almost sure ...

**2**

votes

**0**answers

34 views

### Is there a well-developed theory on filtering on sub $\sigma$-fields?

Suppose we are given an observation process
$$
dY_t = \mu(Y_t)dt + \Sigma(Y_t)dW_t
$$
with valued in $\mathbb{R}^d$ (or potentially in an arbitrary separable Hilbert space). The classical filtering ...

**3**

votes

**1**answer

313 views

### Conditions for supremum and conditional Expectation to commute

I am working with a continuous process $Y_t$ generating the filtration $(F_t)$ and have (for simplicity) two stopping times $\tau_1$ and $\tau_2$ such that $\tau_2 \leq \tau_1$ and $U:\Bbb R\...

**3**

votes

**1**answer

156 views

### Optimal control / Portoflio optimization: Maximize expected utility of total consumption

I came across a portoflio optimization problem, where I need to solve for optimal investment and consumption processes, such that the expected utility of total consumption and terminal wealth is ...

**0**

votes

**0**answers

73 views

### Law of motion when initial condition is perturbed

We know how to find the law of motion (Ito process) of the value function:
$$V_t(x)=E\Big{[}\int^{T}_te^{-r (s-t)}f(s,X_s)ds+e^{-r (T-t)}g(T, X_{T})|\mathcal{F}_t\Big{]}$$
such that
$$dX_t=\mu(t,X_t)...

**2**

votes

**1**answer

385 views

### Stochastic differential equation associated with an optimal control problem

We know how to find the stochastic differential equation (Hamilton-Jacobi-Bellman equation, HJB) of the control problem where a process $X_t$ is controlled up until it is stopped at a stopping time $\...