AdvancedTopic 104 of 104Engineering Mathematics
Stochastic processes
Random variables evolving in time, such as Markov chains and Brownian motion.
In depth
A stochastic process is a collection of random variables indexed by time. Markov chains have memory only of the present state, with transition matrices; Poisson processes count random arrivals; Brownian motion models diffusion and noise. Used in queueing, finance and reliability.
Key points
- Stationarity and ergodicity
- Steady-state distribution of Markov chains
- Random walks
ExampleA server queue with Poisson arrivals and exponential service times is the M/M/1 queue.
Stochastic processes model systems that evolve randomly in time, such as noise, queues and component wear, which is essential for filtering and reliability.
What is Stochastic processes?
Random variables evolving in time, such as Markov chains and Brownian motion.
Why is it important to understand stochastic processes?
Stochastic processes model systems that evolve randomly in time, such as noise, queues and component wear, which is essential for filtering and reliability.
What is a Markov chain?
A process where the next state depends only on the current state, not on the full history. Transition probabilities describe how states change.
What is Brownian motion?
A continuous random process with independent normal increments, used to model noise and diffusion, and it is the basis of many financial and physical models.
How is Stochastic processes used in practice?
Typical applications include modelling a noisy sensor signal in a Kalman filter, estimating the long-run state of a machine that switches between healthy and degraded modes and simulating the random arrival of jobs in a production queue.
What should I learn before Stochastic processes?
Study "Calculus of variations" first (topic 103 of 104 in Engineering Mathematics). It covers the groundwork this topic builds on.
What are the key points of Stochastic processes?
Stationarity and ergodicity. Steady-state distribution of Markov chains. Random walks.
Can you give an example of Stochastic processes?
A server queue with Poisson arrivals and exponential service times is the M/M/1 queue.
What comes after Stochastic processes?
This is the final topic in Engineering Mathematics.
How difficult is Stochastic processes?
It is classed as Advanced in Engineering Mathematics (topic 104 of 104). It is an advanced topic. It assumes you are comfortable with the intermediate topics in Engineering Mathematics.