Sunday, October 18, 2009

Heaviside step function

The Heaviside step function, using the half-maximum convention
The Heaviside step function, H, also called the unit step function, is a discontinuous function whose value is zero for negative argument and one for positive argument. It seldom matters what value is used for H(0), since H is mostly used as a distribution. Some common choices can be seen below.
The function is used in the mathematics of control theory and signal processing to represent a signal that switches on at a specified time and stays switched on indefinitely. It was named after the English polymath Oliver Heaviside.
It is the cumulative distribution function of a random variable which is almost surely 0. (See constant random variable.)
The Heaviside function is the integral of the Dirac delta function: H′ = δ. This is sometimes written as
H(x) = \int_{-\infty}^x { \delta(t)} \mathrm{d}t
although this expansion may not hold (or even make sense) for x = 0, depending on which formalism one uses to give meaning to integrals involving δ.

Discrete form

We can also define an alternative form of the unit step as a function of a discrete variable n:
H[n]=\begin{cases} 0, & n < 0 \\ 1, & n \ge 0 \end{cases}
where n is an integer.
Or
H(x) = \lim_{z \rightarrow x^-} ((|z| / z + 1) / 2)
The discrete-time unit impulse is the first difference of the discrete-time step
\delta\left[ n \right] = H[n] - H[n-1].
This function is the cumulative summation of the Kronecker delta:
H[n] = \sum_{k=-\infty}^{n} \delta[k] \,
where
\delta[k] = \delta_{k,0} \,
is the discrete unit impulse function.

Analytic approximations

For a smooth approximation to the step function, one can use the logistic function
H(x) \approx \frac{1}{2} + \frac{1}{2}\tanh(kx) = \frac{1}{1+\mathrm{e}^{-2kx}},
where a larger k corresponds to a sharper transition at x = 0. If we take H(0) = ½, equality holds in the limit:
H(x)=\lim_{k \rightarrow \infty}\frac{1}{2}(1+\tanh kx)=\lim_{k \rightarrow \infty}\frac{1}{1+\mathrm{e}^{-2kx}}.
There are many other smooth, analytic approximations to the step function.[1] They include:
H(x) = \lim_{k \rightarrow \infty} \frac{1}{2} + \frac{1}{\pi}\arctan(kx) \
H(x) = \lim_{k \rightarrow \infty} \frac{1}{2} + \frac{1}{2}\operatorname{erf}(kx). \
While these approximations converge pointwise towards the step function, the implied distributions do not strictly converge towards the delta distribution. In particular, the measurable set
\bigcup_{n=0}^{\infty}\left[ 2^{-2n};2^{-2n+1}\right]
has measure zero in the delta distribution, but its measure under each smooth approximation family becomes larger with increasing k.

Representations

Often an integral representation of the Heaviside step function is useful:
H(x)=\lim_{ \epsilon \to 0^+} -{1\over 2\pi \mathrm{i}}\int_{-\infty}^\infty {1 \over \tau+\mathrm{i}\epsilon} \mathrm{e}^{-\mathrm{i} x \tau} \mathrm{d}\tau =\lim_{ \epsilon \to 0^+} {1\over 2\pi \mathrm{i}}\int_{-\infty}^\infty {1 \over \tau-\mathrm{i}\epsilon} \mathrm{e}^{\mathrm{i} x \tau} \mathrm{d}\tau.

H(0)

The value of the function at 0 can be defined as H(0) = 0, H(0) = ½ or H(0) = 1. H(0) = ½ is the most consistent choice used, since it maximizes the symmetry of the function and becomes completely consistent with the sign function. This makes for a more general definition:
H(x) = \frac{1+\sgn(x)}{2} = \begin{cases} 0, & x < 0 \\ \frac{1}{2}, & x = 0 \\ 1, & x > 0. \end{cases}
To remove the ambiguity of which value to use for H(0), a subscript specifying the value may be used:
H_a(x) = \begin{cases} 0, & x < 0 \\ a, & x = 0 \\ 1, & x > 0. \end{cases}

Antiderivative and derivative

The ramp function is the antiderivative of the Heaviside step function: R(x) := \int_{-\infty}^{x} H(\xi)\mathrm{d}\xi = x H(x).
The derivative of the Heaviside step function is the Dirac delta function: dH(x) / dx = δ(x).

Fourier transform

The Fourier transform of the Heaviside step function is a distribution. Using one choice of constants for the definition of the Fourier transform we have
\hat{H}(s) = \int^{\infty}_{-\infty} \mathrm{e}^{-2\pi\mathrm{i} x s} H(x)\, dx = \frac{1}{2} \left( \delta(s) - \frac{ \mathrm{i}}{\pi s} \right).
Here the \frac{1}{s} term must be interpreted as a distribution that takes a test function φ to the Cauchy principal value of \int^{\infty}_{-\infty} \varphi(x)/x\, dx.
Posted by at No comments:

Hamilton–Jacobi–Bellman (HJB) equation

The Hamilton–Jacobi–Bellman (HJB) equation is a partial differential equation which is central to optimal control theory. Classical variational problems, for example, the brachistochrone problem can be solved using this method. The HJB method can be generalized to stochastic systems as well.
The solution of the HJB equation is the 'value function', which gives the optimal cost-to-go for a given dynamical system with an associated cost function. The solution is open loop, but it also permits the solution of the closed loop problem.
The equation is a result of the theory of dynamic programming which was pioneered in the 1950s by Richard Bellman and coworkers.[1] The corresponding discrete-time equation is usually referred to as the Bellman equation. In continuous time, the result can be seen as an extension of earlier work in classical physics on the Hamilton-Jacobi equation by William Rowan Hamilton and Carl Gustav Jacob Jacobi.

Optimal control problems

Consider the following problem in deterministic optimal control
\min \left\{ \int_0^T C[x(t),u(t)]\,dt + D[x(T)] \right\}
where C[] is the scalar cost rate function and D[] is a function that gives the salvage value or scrap value scalar at the final state, x(t) is the system state vector, x(0) is assumed given, and u(t) for 0\leq t\leq T is the control vector that we are trying to find.
The system must also be subject to
\dot{x}(t)=F[x(t),u(t)]
where F[] gives the vector determining physical evolution of the state vector over time.

The partial differential equation

For this simple system, the Hamilton Jacobi Bellman partial differential equation is
\dot{V}(x,t) + \min_u \left\{ \nabla V(x,t) \cdot F(x, u) + C(x,u) \right\} = 0
subject to the terminal condition
V(x,T) = D(x),\,
where the a \cdot b means the dot product of the vectors a and b and \nabla is the gradient operator.
The unknown scalar V(x,t) in the above PDE is the Bellman 'value function', which represents the cost incurred from starting in state x at time t and controlling the system optimally from then until time T.

Deriving the equation

Intuitively HJB can be "derived" as follows. If V(x(t),t) is the optimal cost-to-go function (also called the 'value function'), then by Richard Bellman's principle of optimality, going from time t to t + dt, we have
V(x(t), t) = \min_u \left\{ C(x(t), u) dt + V(x(t+dt), t+dt) \right\}.
Note that the Taylor expansion of the last term is
V(x(t+dt), t+dt) = V(x(t), t) + \dot{V}(x, t)dt + \nabla V(x, t) \cdot \dot{x}dt + o(dt^2),
where o(dt^2) denotes the terms in the Taylor expansion of higher order than one. Then if we cancel V(x(t),t) on both sides, divide by dt, and take the limit as dt approaches zero, we obtain the HJB equation defined above.

Solving the equation

The HJB equation needs to be solved backwards in time, starting from t = T and ending at t = 0.[citation needed]
The HJB equation is a necessary and sufficient condition for an optimum. [2] If we can solve for V then we can find from it a control u that achieves the minimum cost.
In general case, the HJB equation does not have a classical (smooth) solution. Several notions of generalized solutions have been developed to cover such situations, including viscosity solution (Pierre-Louis Lions and Michael Crandall), minimax solution (Andrei Izmailovich Subbotin), and others.

 Extension to stochastic problems

The idea of solving a control problem by applying Bellman's principle of optimality and then working out backwards in time an optimizing strategy can be generalized to stochastic control problems. Consider similar as above
\min \left\{ \int_0^T C(t,X_t,u_t)\,dt + D(T,X_T) \right\}
now with (X_t)_{t \in [0,T]}\,\! the stochastic process to optimize and (u_t)_{t \in [0,T]}\,\! the steering. By first using Bellman and then expanding V(t,Xt) with Itô's rule, one finds the deterministic HJB equation
\min_u \left\{ \mathcal{A} v(t,x) + C(x,u) \right\} = 0,
where \mathcal{A} represents the stochastic differentiation operator, and subject to the terminal condition
v(T,x) = D(x)\,\!.
Note, that the randomness has disappeared. In this case a solution v\,\! of the latter does not necessarily solve the primal problem, it is a candidate only and a further verifying argument is required. This technique is widely used in Financial Mathematics to determine optimal investment strategies in the market (see for example Merton's portfolio problem)
Posted by at No comments:

Pontryagin's minimum principle

Pontryagin's minimum principle is used in optimal control theory to find the best possible control for taking a dynamic system from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich Pontryagin and his students. It has as a general case the Euler–Lagrange equation of the calculus of variations.
The principle states informally that the Hamiltonian must be minimized over \mathcal{U}, the set of all permissible controls. If u^*\in \mathcal{U} is the optimal control for the problem, then the principle states that:
H(x^*(t),u^*(t),\lambda^*(t),t) \leq H(x^*(t),u,\lambda^*(t),t), \quad \forall u \in \mathcal{U}, \quad t \in [t_0, t_f]
where x^*\in C^1[t_0,t_f] is the optimal state trajectory and \lambda^* \in BV[t_0,t_f] is the optimal costate trajectory.
The result was first successfully applied into minimum time problems where the input control is constrained, but it can also be useful in studying state-constrained problems.
Special conditions for the Hamiltonian can also be derived. When the final time tf is fixed and the Hamiltonian does not depend explicitly on time (\frac{\partial H}{\partial t} \equiv 0), then:
H(x^*(t),u^*(t),\lambda^*(t)) \equiv \mathrm{constant}\,
and if the final time is free, then:
H(x^*(t),u^*(t),\lambda^*(t)) \equiv 0.\,
More general conditions on the optimal control are given below.
Pontryagin's minimum principle is a necessary condition for an optimum. The Hamilton–Jacobi–Bellman equation provides sufficient conditions for an optimum.

Maximization and minimization

The results here are sometimes known as Pontryagin's maximum principle. This is because Pontryagin's original work focused on maximizing a benefit functional rather than minimizing a cost functional, the proof of the minimum principle is historically based on maximizing the Hamiltonian rather than minimizing the Hamiltonian. In this framework, to minimize the cost functional instead of maximizing a benefit functional, the functional should be multiplied by − 1. Modern applications of this work focus on the minimization problem.

Formal statement of necessary conditions for minimization problem

Here the necessary conditions are shown for minimization of a functional. Take x to be the state of the dynamical system with input u, such that
\dot{x}=f(x,u), \quad x(0)=x_0, \quad u(t) \in \mathcal{U}, \quad t \in [0,T]
where \mathcal{U} is the set of admissible controls and T is the terminal (i.e., final) time of the system. The control u \in \mathcal{U} must be chosen for all t \in [0,T] to maximize the objective functional J which is defined by the application and can be abstracted as
J=\Psi(x(T))+\int^T_0 L(x(t),u(t)) \,dt
The constraints on the system dynamics can be adjoined to the Lagrangian L by introducing time-varying Lagrange multiplier vector λ, whose elements are called the costates of the system. This motivates the construction of the Hamiltonian H defined for all t \in [0,T] by:
H(\lambda(t),x(t),u(t),t)=\lambda'(t)f(x(t),u(t))+L(x(t),u(t)) \,
where λ' is the transpose of λ.
Pontryagin's minimum principle states that the optimal state trajectory x * , optimal control u * , and corresponding Lagrange multiplier vector λ * must minimize the Hamiltonian H so that
(1) \qquad H(x^*(t),u^*(t),\lambda^*(t),t)\leq H(x^*(t),u,\lambda^*(t),t) \,
for all time t \in [0,T] and for all permissible control inputs u \in \mathcal{U}. It must also be the case that
(2) \qquad \Psi_T(x(T))+H(T)=0 \,
Additionally, the costate equations
(3) \qquad -\dot{\lambda}'(t)=H_x(x^*(t),u^*(t),\lambda^*(t),t)=\lambda'(t)f_x(x^*(t),u^*(t))+L_x(x^*(t),u^*(t))
must be satisfied. If the final state x(T) is not fixed (i.e., its differential variation is not zero), it must also be that the terminal costates are such that
(4) \qquad \lambda'(T)=\Psi_x(x(T)) \,
These four conditions in (1)-(4) are the necessary conditions for an optimal control. Note that (4) only applies when x(T) is free. If it is fixed, then this condition is not necessary for an optimum.
The notation used above is defined below.
\Psi_T(x(T))= \frac{\partial \Psi(x)}{\partial T}|_{x=x(T)} \,
\Psi_x(x(T))=\begin{bmatrix} \frac{\partial \Psi(x)}{\partial x_1}|_{x=x(T)} & \cdots & \frac{\partial \Psi(x)}{\partial x_n} |_{x=x(T)} \end{bmatrix}
H_x(x^*,u^*,\lambda^*,t)=\begin{bmatrix} \frac{\partial H}{\partial x_1}|_{x=x^*,u=u^*,\lambda=\lambda^*} & \cdots & \frac{\partial H}{\partial x_n}|_{x=x^*,u=u^*,\lambda=\lambda^*} \end{bmatrix}
L_x(x^*,u^*)=\begin{bmatrix} \frac{\partial L}{\partial x_1}|_{x=x^*,u=u^*} & \cdots & \frac{\partial L}{\partial x_n}|_{x=x^*,u=u^*} \end{bmatrix}
f_x(x^*,u^*)=\begin{bmatrix} \frac{\partial f_1}{\partial x_1}|_{x=x^*,u=u^*} & \cdots & \frac{\partial f_1}{\partial x_n}|_{x=x^*,u=u^*} \\ \vdots & \ddots & \vdots \\ \frac{\partial f_n}{\partial x_1}|_{x=x^*,u=u^*} & \ldots & \frac{\partial f_n}{\partial x_n}|_{x=x^*,u=u^*} \end{bmatrix}
Posted by at No comments:

BANG BANG CONTROLLER

In control theory, a bang–bang controller (on–off controller) is a controller that switches abruptly between two states. These controllers may be realized in terms of any element that provides hysteresis. They are often used to control a plant that accepts a binary input, for example a furnace that is either completely on or completely off. Most common residential thermostats are bang-bang controllers. The Heaviside step function in its discrete form is an example of bang–bang control.

Bang–bang solutions in optimal control

In optimal control problems, it is sometimes the case that a control is restricted to be between a lower and an upper bound. If the optimal control switches from one extreme to the other at certain times (i.e., is never strictly in between the bounds) then that control is referred to as a bang-bang solution.
Bang–bang controls frequently arise in minimum-time problems. For example, if it is desired to stop a car in the shortest possible time at a certain position sufficiently far ahead of the car, the solution is to apply maximum acceleration until the unique switching point, and then apply maximum braking to come to rest exactly at the desired position. This solution, which can be "uncomfortable" for the passengers, is a bang–bang solution: maximum engine throttle followed by maximum braking. Bang–bang solutions also arise when the Hamiltonian is linear in the control variable; application of Pontryagin's minimum principle will then lead to pushing the control to its upper or lower bound depending on the sign of the coefficient of u in the Hamiltonian.
In summary, then, bang–bang controls are actually optimal controls in some cases, although they are also often implemented because of simplicity or convenience.
Posted by at No comments:
Subscribe to: Posts (Atom)