systems.builtin.stochastic.discrete.DiscreteStochasticPendulum

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum(*args, **kwargs)

Discrete-time stochastic pendulum for digital control and RL.

Represents a pendulum sampled at discrete intervals with process noise, suitable for digital control implementation, discrete Kalman filtering, and reinforcement learning applications with realistic disturbances.

Discrete-Time Stochastic Dynamics

Euler discretization with noise:

θ[k+1] = θ[k] + ω[k]·Δt + w_θ[k]
ω[k+1] = ω[k] + (-(g/L)·sin(θ[k]) - b·ω[k] + u[k])·Δt + w_ω[k]

where: - θ[k]: Angle at time k·Δt [rad] - ω[k]: Angular velocity at time k·Δt [rad/s] - u[k]: Applied torque (zero-order hold) [rad/s²] - w_θ[k]: Angle noise [rad per step] - w_ω[k]: Angular velocity noise [rad/s per step] - Δt: Sampling period [s]

Deterministic Part: Same nonlinear pendulum dynamics as continuous.

Stochastic Part: Noise accumulated over sampling interval [k·Δt, (k+1)·Δt].

Zero-Order Hold: Control u[k] constant during interval (digital implementation).

Physical Interpretation

Digital Control Loop:

  1. Sample: Read encoder at t = k·Δt
    • Measure θ[k] (with quantization)
    • Estimate ω[k] (from differences or gyro)
  2. Compute: Calculate control u[k]
    • LQR, swing-up, learned policy
    • Computation time: τ_comp (typically << Δt)
  3. Actuate: Output u[k] via DAC/PWM
    • Held constant until t = (k+1)·Δt
    • Zero-order hold
  4. Disturbances: Between samples
    • Wind gusts: w_ω
    • Ground vibration: w_θ
    • Friction variations
    • Unmodeled dynamics

Noise Sources:

Angle noise w_θ[k]: - Direct angle disturbances (rare physically) - Encoder quantization (measurement as process noise) - Model uncertainty - Typical: 0.001-0.01 rad per step

Angular velocity noise w_ω[k]: - Torque disturbances (wind, vibration) - Motor ripple, cogging - Bearing friction variations - Most critical for stability - Typical: 0.01-0.1 rad/s per step

Conversion from Continuous:

For continuous σ_c [rad/(s²·√s)]: σ_d = σ_c·√Δt [rad/s per step]

Example: σ_c = 1.0, Δt = 0.01 s → σ_d = 0.1 rad/s per step

Key Features

Nonlinearity: sin(θ) creates bistability and complex dynamics.

Unstable Equilibrium: Upright (θ=0) exponentially unstable without control.

Bistability: Two potential wells (downward stable, upward unstable).

Discrete Sampling: Finite Δt creates sampling effects (aliasing, delay).

Zero-Order Hold: Control discontinuous (step-wise).

Process Noise: Disturbances between samples.

Mathematical Properties

Equilibria (Deterministic Part):

Downward: θ = 0, ω = 0 (stable) Upward: θ = π, ω = 0 (unstable)

Linearization (Small Angle):

Near downward (θ ≈ 0): [θ[k+1]] ≈ [1 Δt ]·[θ[k]] + [0 ]·u[k] [ω[k+1]] [-g/L·Δt 1-b·Δt] [ω[k]] [Δt]

Eigenvalues determine discrete stability.

Stability Condition (Linear):

For stable downward equilibrium: Eigenvalues of Φ must satisfy |λ| < 1.

Requires: Δt < 2/(b + √(b² + 4g/L))

Upright (θ ≈ π):

Unstable - one eigenvalue |λ| > 1. Requires fast sampling and feedback.

Physical Interpretation

Sampling Period Δt: - Digital control rate - Critical for unstable upright - Affects noise accumulation - Trade-off: Fast vs computation

Damping b: - Energy dissipation [1/s] - Higher b: Easier to control - Typical: 0.1-1.0

Noise Intensity σ: - Angular disturbance magnitude - Higher σ: More falls, harder balance - Typical: 0.01-0.1 rad/s per step

State Space

State: X[k] = [θ[k], ω[k]] - θ: Angle (periodic, or unwrapped) - ω: Angular velocity

Control: u[k] ∈ ℝ - Torque (zero-order hold) - Bounded: |u| ≤ u_max typically

Noise: w[k] = [w_θ[k], w_ω[k]] - Gaussian per step - Accumulated over Δt

Parameters

Name Type Description Default
g float Gravity [m/s²] 9.81
L float Pendulum length [m] 1.0
b float Damping [1/s] 0.5
sigma_theta float Angle noise [rad per step] - Encoder noise, model uncertainty - Typical: 0.001-0.01 0.01
sigma_omega float Angular velocity noise [rad/s per step] - Torque disturbances (most critical) - Convert from continuous: σ_c·√Δt - Typical: 0.01-0.1 0.05
dt float Sampling period [s] - Critical parameter - Typical: 0.001-0.1 s - Faster for upright stabilization 0.01
m float Mass [kg] (optional) 1.0

Stochastic Properties

  • System Type: NONLINEAR
  • Noise Type: ADDITIVE
  • Discrete: Yes (native discrete-time)
  • Stationary: No (open-loop)
  • Bistable: Yes (two equilibria)
  • Unstable: Yes (upright)

Applications

1. Digital Control: - Microcontroller implementation - Real-time OS (RTOS) - Fixed-point arithmetic

2. State Estimation: - Discrete EKF/UKF - Particle filter - Complementary filter (IMU+encoder)

3. Reinforcement Learning: - OpenAI Gym Pendulum-v1 - Discrete action spaces - Robust RL with noise

4. Embedded Systems: - Arduino, STM32 implementation - FPGA for ultra-fast control - Education/research platforms

5. Robustness Analysis: - Monte Carlo falling probability - Mean time to failure - Sensitivity to noise

Numerical Simulation

Direct Evaluation: X[k+1] = f(X[k], u[k]) + w[k]

No integration needed (discrete-time native).

Euler Discretization: Standard for discrete pendulum (matches RL benchmarks).

Higher-Order (RK4): More accurate but less common in practice.

Discrete EKF Implementation

Jacobian (Linearization):

F = ∂f/∂X = [1 Δt ] [-g/L·cos(θ)·Δt 1 - b·Δt]

At θ=0 (downward): cos(0) = 1 At θ=π (upright): cos(π) = -1 (sign flip!)

Comparison with Continuous

Continuous Stochastic: - dW Brownian motion - σ in [rad/(s²·√s)] - SDE integration (Euler-Maruyama) - Theoretical foundation

Discrete Stochastic: - w[k] Gaussian per step - σ in [rad/s per step] - Direct evaluation - Practical implementation

Conversion: σ_discrete = σ_continuous·√Δt

Limitations

  • Euler discretization: O(Δt) error
  • Additive noise (not multiplicative)
  • 1D pendulum (no 3D motion)
  • Rigid body (no flexibility)

Extensions

  • RK4 discretization (higher accuracy)
  • Double pendulum (chaos + noise)
  • Flexible pendulum
  • 3D pendulum (spherical)
  • Multiplicative noise

See Also

StochasticPendulum : Continuous-time version Pendulum : Deterministic discrete version DiscreteStochasticDoubleIntegrator : Simpler linear system

Methods

Name Description
compute_energy Compute mechanical energy.
define_system Define discrete stochastic pendulum dynamics.
get_natural_frequency Get natural frequency ω₀ = √(g/L) [rad/s].
get_noise_intensities Get noise parameters.
get_nyquist_limit Get approximate Nyquist-like sampling limit.
setup_equilibria Set up equilibrium points.

compute_energy

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.compute_energy(x)

Compute mechanical energy.

Parameters

Name Type Description Default
x np.ndarray State [θ, ω] required

Returns

Name Type Description
float Energy [J/kg]

Examples

>>> pend = DiscreteStochasticPendulum()
>>> x = np.array([0.5, 1.0])
>>> E = pend.compute_energy(x)
>>> print(f"Energy: {E:.3f}")

define_system

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.define_system(
    g=9.81,
    L=1.0,
    b=0.5,
    sigma_theta=0.01,
    sigma_omega=0.05,
    dt=0.01,
    m=1.0,
)

Define discrete stochastic pendulum dynamics.

Parameters

Name Type Description Default
g float Gravity [m/s²] 9.81
L float Pendulum length [m] 1.0
b float Damping [1/s] 0.5
sigma_theta float Angle noise per step [rad] - Encoder quantization, model error - Typical: 0.001-0.01 0.01
sigma_omega float Angular velocity noise per step [rad/s] - Torque disturbances (critical for upright) - Convert: σ_c·√Δt from continuous - Typical: 0.01-0.1 0.05
dt float Sampling period [s] - Typical: 0.001-0.1 - Faster for upright control - Balance: Fast enough vs computation 0.01
m float Mass [kg] 1.0

Notes

Sampling Rate Selection:

For upright stabilization: - Minimum: Δt < π/√(g/L) (Nyquist-like) - Recommended: Δt < 0.1·√(L/g) (10× rule) - For L=1m, g=9.81: Δt < 0.03 s = 30 ms

Practical: - Fast (research): 1-5 ms - Moderate (education): 10-20 ms - Slow (demo): 50-100 ms

Noise Scaling:

From continuous σ_c [rad/(s²·√s)]: σ_discrete = σ_c·√Δt

Example: σ_c = 1.0, Δt = 0.01 → σ_d = 0.1 rad/s per step

Stability (Discrete Linearized):

At downward (θ=0): Eigenvalues of: Φ = [1 Δt ] [-g/L·Δt 1-b·Δt]

Stable if |λ| < 1 for both eigenvalues.

Requires: Δt small enough.

Discretization Method:

This uses Euler (most common for RL/embedded): - Simple, explicit - O(Δt) error - Matches OpenAI Gym

Alternative: RK4 (more accurate, more computation).

Noise Placement:

On both θ and ω equations (general).

Physical: Primarily on ω (torque disturbances). Can set σ_θ = 0 for purely physical model.

get_natural_frequency

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.get_natural_frequency(
)

Get natural frequency ω₀ = √(g/L) [rad/s].

get_noise_intensities

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.get_noise_intensities(
)

Get noise parameters.

get_nyquist_limit

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.get_nyquist_limit(
)

Get approximate Nyquist-like sampling limit.

Returns

Name Type Description
float Maximum recommended Δt [s]

Notes

Rule: Δt < π/ω₀ for capturing dynamics.

Examples

>>> pend = DiscreteStochasticPendulum(g=9.81, L=1.0)
>>> dt_max = pend.get_nyquist_limit()
>>> print(f"Max Δt ≈ {dt_max:.3f} s")

setup_equilibria

systems.builtin.stochastic.discrete.DiscreteStochasticPendulum.setup_equilibria(
)

Set up equilibrium points.