Gaussian Process Emulators
Emulator
Deterministic computer model
\[Y = f(X)\]
for
An emulator is an estimate of \(f\), i.e. \(\widehat{f}\).
Gaussian Process
For \(f:\) 𝒳 \(\to\) 𝒴, assume \[f \sim GP(\mu, k)\]
for
- mean function \(\mu:\) 𝒳 \(\to \mathbb{R}\) and
- covariance function \(k:\) 𝒳 \(\times\) 𝒳 \(\to \mathbb{R}^{+}\).
For simplicity, \(\mu(x) = 0\).
Data
For a collection of data \((Y_i, X_i)\) for \(i=1,\ldots,n\), we have
\[Y = (Y_1,\ldots,Y_n)^\top \sim N(0, \Sigma)\]
where
\[\Sigma_{i,j} = k(x_i,x_j).\]
For prediction at a new location \(\widetilde{x}\), we have the conditional distribution \(\widetilde{Y}|y\) which involves covariances \(k(x_i, \widetilde{x})\) for all \(i\).
Distance-based covariance kernel
For any 𝒳, let \[k(x_i,x_j) = \sigma^2 e^{-d(x_i,x_j)/2}\] for spatial variance \(\sigma^2\) and some distance function \(d(x_i,x_j)\).
For example,
- squared exponential (Gaussian) covariance
- automatic relevance determination
- automatic dynamic relevance determination
Squared-exponential kernel
If \(x \in \mathbb{R}\),
the squared-exponential (Gaussian) covariance kernel is \[d(x_i,x_j) = w (x_i-x_j)^2\]
where \(w\) is the weight (\(1/w\) is the length-scale/range).
Automatic relevance determination
If \(x \in \mathbb{R}^P\),
the automatic relevance determination kernel is \[d(x_i,x_j) = \sum_{p=1}^P w_p [x_{i,p}-x_{j,p}]^2\]
where \(w_p\) controls the strength of the relationship in the \(p\)th dimension.
Automatic dynamic relevance determination
If \(x \in\) ℋ (Hilbert space), the automatic dynamic relevance determination kernel is \[d(x_i,x_j) = \int w(t) [x_{i}(t)-x_{j}(t)]^2 dt\]
for some weight function \(w:\) 𝒯 \(\to \mathbb{R}^+\).
