MDP (class MDP)

class MDP

Base class for Markov Decision Processes. Manages state, input, and disturbance spaces, system dynamics, noise distributions, and transition data.

Defined in src/MDP.h.

Constructor

MDP(const int x, const int u, const int w)

Create an MDP with the specified dimensions.

Parameters:

• x – Number of state space dimensions.

• u – Number of input space dimensions (0 for no inputs).

• w – Number of disturbance space dimensions (0 for no disturbance).

State Space

void setStateSpace(vec lb, vec ub, vec eta)

Define the state space as a discretized hyper-rectangle.

Parameters:

• lb – Lower bounds vector (length dim_x).

• ub – Upper bounds vector (length dim_x).

• eta – Discretization step size vector (length dim_x).

mat getStateSpace()

Returns:

Matrix where each row is a discretized state.

void saveStateSpace()

Save the state space to ss.h5 in HDF5 format.

void loadStateSpace(string filename)

Load a state space from an HDF5 file.

Input Space

void setInputSpace(vec lb, vec ub, vec eta)

Define the input space as a discretized hyper-rectangle.

mat getInputSpace()

Returns:

Matrix where each row is a discretized input.

void saveInputSpace()

Save the input space to is.h5.

void loadInputSpace(string filename)

Load an input space from an HDF5 file.

Disturbance Space

void setDisturbSpace(vec lb, vec ub, vec eta)

Define the disturbance space as a discretized hyper-rectangle.

mat getDisturbSpace()

Returns:

Matrix where each row is a discretized disturbance.

void saveDisturbSpace()

Save the disturbance space to ws.h5.

void loadDisturbSpace(string filename)

Load a disturbance space from an HDF5 file.

Target Space

void setTargetSpace(const function<bool(const vec&)> &condition, bool remove)

Separate states matching the condition into a target region.

Parameters:

• condition – Boolean function evaluated on each state.

• remove – If true, remove matching states from the state space.

mat getTargetSpace()

Returns:

Matrix of target states.

void saveTargetSpace()

Save the target space to ts.h5.

void loadTargetSpace(string filename)

Load a target space from an HDF5 file.

Avoid Space

void setAvoidSpace(const function<bool(const vec&)> &condition, bool remove)

Separate states matching the condition into an avoid region.

Parameters:

• condition – Boolean function evaluated on each state.

• remove – If true, remove matching states from the state space.

mat getAvoidSpace()

Returns:

Matrix of avoid states.

void saveAvoidSpace()

Save the avoid space to as.h5.

void loadAvoidSpace(string filename)

Load an avoid space from an HDF5 file.

Combined Target and Avoid

void setTargetAvoidSpace(const function<bool(const vec&)> &target_condition, const function<bool(const vec&)> &avoid_condition, bool remove)

Set both target and avoid regions simultaneously.

Dynamics

void setDynamics(function<vec(const vec&, const vec&, const vec&)> d)

Set dynamics with state, input, and disturbance parameters.

void setDynamics(function<vec(const vec&, const vec&)> d)

Set dynamics with state and input parameters.

void setDynamics(function<vec(const vec&)> d)

Set dynamics with state parameter only (autonomous system).

Noise Configuration

void setNoise(NoiseType n, bool diagonal = true)

Set the noise distribution type.

Parameters:

• n – NoiseType::NORMAL or NoiseType::CUSTOM.

• diagonal – If true, use diagonal covariance (default).

void setNoise(NoiseType n, bool diagonal, size_t monte_carlo_samples)

Set noise with Monte Carlo integration.

Parameters:

monte_carlo_samples – Number of samples for MC integration.

void setStdDev(vec sig)

Set standard deviations for diagonal normal distribution.

void setInvCovDet(mat inv_cov, double det)

Set the inverse covariance matrix and determinant for full covariance normal.

void setCustomDistribution(double (*c)(double*, size_t, void*), size_t monte_carlo_samples)

Set a custom probability distribution function.

mat getInvCov()

Returns:

The inverse covariance matrix.

double getDet()

Returns:

The covariance matrix determinant.

vec getStdDev()

Returns:

The standard deviation vector.

Transition Data

void saveTransitionMatrix()

Save the transition matrix to HDF5.

void loadTransitionMatrix(string filename)

Load a transition matrix from HDF5.

void saveTargetTransitionVector()

Save the target transition vector.

void loadTargetTransitionVector(string filename)

Load a target transition vector.

void saveAvoidTransitionVector()

Save the avoid transition vector.

void loadAvoidTransitionVector(string filename)

Load an avoid transition vector.

void trackMDP(bool store)

Enable or disable intermediate MDP data storage.

Stopping Condition

void setStoppingCondition(double eps)

Set the convergence threshold for infinite-horizon synthesis.

Parameters:

eps – Convergence epsilon (default: 0.00001).

Enums and Structs

enum class NoiseType

enumerator NORMAL

Gaussian noise (diagonal or full covariance).

enumerator CUSTOM

User-defined distribution via src/custom.cpp.

struct customParams

Parameters passed to custom PDF functions.

vec mean

Result of applying dynamics to the current state.

vec state_start

Current state being evaluated.

function<vec(const vec&)> dynamics1

function<vec(const vec&, const vec&)> dynamics2

function<vec(const vec&, const vec&, const vec&)> dynamics3

System dynamics functions.

vec input

Current input.

vec disturb

Current disturbance.

vec lb

vec ub

Integration bounds.

vec eta

Discretization parameter.