Machine Learning + Dynamical Systems

Experiments in chaos, complexity, and machine learning

Welcome

After reading about the recent success of machine learning results in predicting the future evolution of chaotic systems, I decided to investigate the subject myself. The new approach was being lauded by many experts as groundbreaking and likely to find wide application. It took a considerable amount of effort to understand the methods used and to develop the basic model. There was not enough details to reproduce the results despite an abundance of theoretical literature. I realized that there is a gap between the theory of dynamical systems and practical machine learning. This website attempts to bridge that gap.

Physicists like to think that all you have to do is say 'These are the conditions, now what happens next?'

Richard Feynman

If understanding of chaos and nonlinear dynamics is the purpose or destination for the research, then the theoretical work is needed. But that's not what all we are interested in, sometimes prediction is more important than understanding, inference is more desirable than the theory. So, this website is designed with practicality in-mind, more importance is given to computational techniques and understanding than to laying out the theoretical framework. Ultimately, the machine learning approach is used to deduce the most important quantifiers of the system’s chaotic behavior using only limited time series measurements of the system.

Articles

The Lorenz Attractor We start with exploring Lorenz differential equations using Python through examples and look at how we create machine learning models.

Delay Embedding We try to reconstruct a chaotic dynamical system from a time-series using Taken's delay embedding technique. The reconstruction preserves the properties of the dynamical system that do not change under smooth coordinate changes (i.e., diffeomorphisms).

Lyapunov Exponents Lyapunov exponents measure exponential rates of separation of nearby trajectories in the flow of a dynamical system.

Entropy and Fractal Dimension In the study of dynamical systems there are many quantities that identify as "entropy". Notice that these quantities are not the more commonly known thermodynamic ones, used in Statistical Physics. Rather, they are more like the to the entropies of information theory, which represents information contained within a dataset, or information about the dimensional scaling of a dataset. Based on the definition of the generalized entropy, one can calculate an appropriate dimension.

Recurrence Plots A recurrence plot is a way to quantify recurrences that occur in a trajectory. What's a recurrence? As the name suggests, a recurrence happens when a trajectory visits the same neighborhood on the phase space that it was at some previous time. The recurrence plot is a visual representation of a sparse square matrix of boolean values (called recurrence matrix).

FAQ

Who is Dr. Kaos?

I have a Ph.D. in Chaos & Nonlinear Dynamics, with a passion for data and finding patterns, so I naturally gravitate towards complexity and uncertainty. I am also a tech entrepreneur with multiple successful exits and with spectacular failures too :-). Over the years, I had the good fortune of working with great engineers and scientists who have helped me with my insatiable appetite for learning, coding, and computational experiments. This website is an effort to help others and to give back to the community in hopes of generating good karma.

For the past couple of years, I've been managing a team of engineers, working on interesting fulfilment optimization problems, implementing machine learning algorithms, and poking at big data.

What can I expect to find here?

Hopefully, interesting and practical techniques!

Half a century ago, the pioneers of chaos theory discovered that the “butterfly effect” makes long-term prediction impossible. That was before machine learning became a commodity in solving difficult engineering problems. In this website, I share my experimentation with machine learning models for a chaotic dynamical system. It is designed with practicality in-mind, ie more importance is given to to computational techniques and understanding than to laying out the theoretical framework.

Who is the target audience?

Anyone interested in learning more about chaos, complexity, and machine learning. That being said, there are some prerequisites: I assume you know Python and you also have basic knowledge of chaos and nonlinear dynamics. If you don't I recommend the following books to get you started: