Markov Chains

Probability and statistics uses a plentiful of methods all to determining the likelihood of something happening. There are so many statistical tests to determine relationships between variables as well. One such method that I think is particularly cool are Markov Chains.

Markov chains specifically determine probability of future event GIVEN a past event. This “given“ statement is what makes them unique, and what makes their solutions interesting. An simlistic example a markov chain can be applied to is below:

A restaurant’s menu is EITHER a hamburger, a salad, or a pizz.

The item served on a day depends on what was served the previous day. The probablilities for each future day’s menu GIVEN the past day’s menu are below:

• If hamburgers are served the day before: 20% chance of serving hamburgers, 30% chance of serving salads, 50% chance of serving pizzas

• If salads are served the day before: 40% chance of serving salads, 20% chance of serving hamburgers, 40% chance of serving pizzas

• If pizzas are served the day before: 0% chance of serving pizzas, 40% chance of serving salads, 60% chance of serving hamburgers

This situation can be represented using a chain before performing any math. The chain below represents this example. The arrows originate at the previous day’s menu choice and point towards the next day’s choice, with the corresponding proportion of that next day menu choice happening. An arrow from a menu item back to itself is proportion a menu item will be chosen again given it was chosen the day before. The pizza has no arrow back to itself as the restaurant will never serve pizza two days in a row.

This Markov chain can also be represented as a matrix. The matrix below shows all of the probabilities of going from one menu item to another on any given day:

Now that this is matrix, actual math can be done. The simplest use of representing probability this way is finding the equilibrium probability rates. Basically, if the restaurant served food every day for infinite days following the probability rules above, what is the likely hood on one given day of each food item? These likely hoods can be determined through computer simulations, but that seems overly complicated for me. Instead, we can use some basic algebra, assigning P, S, and H for the equilibrium probabilities of each food accordingly. See below for the steps to this solution:

Notice how in each equation, the coefficients are aligned by row. Setting the sum of probabilities equal to 1, the probabilities on any given day for serving one food or another are below:

• Hamburger: 32%

• Salad: 37%

• Pizza: 31%

With just some drawing and simple algebra, long-run probabilities are easily determined with Markov chains. While this was a simple example, Markov chains can be used for a plentitude of problems like likely hood of a dam breaking or future expected electrical vehicals in use. There are many other mathematical methods that can also be applied on Markov chains, but in general, they are great ways to visualize and reason through complex probability or math-modeling problems.