Our previous programs for analyzing the gambler problem were based on what is called a "Monte Carlo" simulation. To estimate quantities such as the probability of success, the expected number of bets made, and the percentage of time with each bankroll, we performed independent trials of the random experiment. The more trials that we performed, the more we expect our observed statistics to approach the theoretical bounds. But the problem with a Monte Carlo technique is that each trial takes time and we may need to do an inordinate number of trials in order to get a decent approximation.
We will explore a new algorithmic approach that can be used for computing values for the gambler problem more efficiently. We develop an iterative solver that works by starting with an initial guess of the results, and then iteratively computing successive approximations based on our understanding of the model. In this case of the gambler's problem, let the notation win(goal, start, p) represent the overall chance of success given the goal, the start bankroll, and the fixed probability p. Given a fixed goal and p value, we want to compute the probability of success for each starting bankroll from 0 to goal. We know the following about the theoretical value of this expression.
win(goal, start, p) =
This formula provides enough information for us to compute the answers using an interative solver. We make an initial guess (perhaps a linear function from 0 to 1). Then we repeatedly compute a better guess based on applying the above formula relative to the previous guess. Each iteration should improve our estimates, eventually converging near the theoretical values. We will repeat this process until performing an iteration in which the new estimates vary from the old estimates by at most some desired error tolerance.
Before exploring code for the iterative solution to the gambler's problem, we will look at a simpler problem that can be solved with this approach: calculating the square root of a number. A common algorithm, known as Newton's Method, is based on computing successive approximations to the square root. To compute the square root of n, we make an initial guess (e.g., 1). Once we have a guess, it is easy to test whether the guess is the correct square root. If it were the square root, we'd have that guess and n/guess would be equal. If they are not equal (or not within a specified error tolerance of equal), we can compute a new approximation which is the average of guess and n/guess.