Not a happy thing.
Note: the original MATLAB code had this property as well. Roughly speaking, as the number of experience nodes and visual templates grows, so too does the cost to compute the next time step. In theory it should level off once the entire map has been experienced (assuming things are working correctly), but that takes a long time to happen, and the compute time grows largely unbounded over the entire St. Lucia video.
TODO: check the original paper at some point to see if the MATLAB accurately reflects the intended algorithm.