Ergodic Control Overview

Ergodic control is concerned with the generation of ergodic trajectories. This section defines ergodicity and shows how many of these trajectories are generated.

Ergodicity

A trajectory is ergodic with respect to a distribution if its time-averaged statistics match the distribution’s spatial statistics. In an ergodic trajectory, the time spent in a region is proportional to the distribution’s density in the region. If we have some domain \(X\), we denote the spatial distribution \(\phi\), and \(\phi(x)\) describes the density at a point \(x\in X\).

A commonly used metric for ergodicity uses Fourier coefficients [1]. The spatial distribution is decomposed into Fourier coefficients \(\phi_k\):

\[\phi_k(x) = \int_X \phi(x) F_k(x) dx,\]

where \(k\) is a wave-number vector with dimensionality equal to the spatial domain’s. That is, if the spatial domain has two dimensions, then \(k\) is a vector of length 2.

The function \(F_k(x)\) is as follows:

\[F_k(x) = \frac{1}{h_k}\prod_{i=1}^n \cos \left(\frac{k_i\pi}{L_i} x_i\right),\]

where \(h_k\) is a normalizing factor and \(L_i\) is the length of dimension \(i\).

\[h_k = \left(\int_0^{L_1} \int_0^{L_2} \cos^2(\frac{k_1\pi x_1}{L_1}) \cos^2(\frac{k_2\pi x_2}{L_2})dx_1 dx_2 \right)^{1/2}\]

If we have a trajectory \(x(t)\), we can find the Fourier coefficients \(c_k\) of the trajectory:

\[c_k = \frac{1}{T}\int_0^T F_k(x(t))dt,\]

The ergodic metric \(\mathcal{E}\) is a measure of the difference between trajectory and distribution coefficients:

\[\mathcal{E} = \sum_k \Lambda_k | c_k - \phi_k |^2,\]

where \(\Lambda_k\) is weighted to favor low-frequency features. It takes the form

\[\Lambda_k = \frac{1}{\left(1 + ||k||^2\right)^{(n+1)/2}},\]

where \(n\) is the number state variables in our distribution; for distributions over \(\mathbb{R}^2,\ n = 2\).

Ergodicity in SE(2)

A robot in SE(2) has a state \(x\in\mathbb{R}^3\), where \(x\) consists of the robot’s \(x\)-location, \(y\)-location, and heading \(\theta\). We might denote this state \(x(t) = [x_r, y_r, \theta]\).

The basis functions are as follows:

\[F_{m,n,p}(x) = i^{n-m}\exp\left( i\left[m\psi + (n-m)\theta\right]\right) J_{m-n}(pr),\]

where \((m,n,p)\) are the indices along each direction, \(J_{m-n}\) is the \(m-n\text{th}\) order Bessel function and \((r, \psi, \theta)\) are the robot’s state expressed in polar coordinates:

\[r = \sqrt{x_r^2 + y_r^2},\]

\[\psi = \arctan(y_r / x_r),\]

\[\theta = \theta.\]

The ergodic metric is then expressed

\[\mathcal{E} = \sum_{m,n,p=0}^{M,N,P} \Lambda_{m,n,p} || c_{m,n,p} - \phi_{m,n,p} ||^2,\]

where \(\Lambda_{m,n,p}\) is equivalent to \(\Lambda_k\) where the vector \(k=[m,n,p]\).

Trajectory Generation

We want to generate trajectories with low ergodic score, because an ergodic score of zero implies perfect ergodicity.

Bibliography

[1] G. Mathew and I. Mezic, “Metrics for ergodicity and design of ergodic dynamics for multi-agent systems”