I am currently working on two books aiming to provide a technical introduction to the research areas of Active Inference, the Free Energy Principle, and Bayesian Mechanics (see background section below) to be published by The MIT Press in two separate books. The mathematical ideas within these fields have the potential to revolutionize many industries and greatly advance theoretical AI research beyond what has already been achieved through deep learning and large language models in the last decade. As of Jan 2024, the first draft has been written along with accompanying (draft) video lectures and Jupyter notebooks and I am currently in the process of revision. The textbook and its related content is presented a few times a month for feedback/review to the Active Inference Institute since 2023. My intention for this work is to condense over 20 years of progress and literature in the field to bring a wider degree of recognition and interest from machine learning researchers and engineers in academia and industry.

The textbook is aimed at an upper undergraduate/early graduate level and aims to be fairly self-contained, requiring little knowledge of neuroscience and related literature but focusing heavily on mathematics and Python code. The prerequisites for the book are basic Python programming knowledge, probability theory, multivariate calculus, basic linear algebra, and high-school physics, topics usually covered in lower undergraduate science and engineering degrees.

At the moment, the textbook draft is only available to members of the Active Inference Institute “Fundamentals of Active Inference” project and members at VERSES.AI.

Background

Active Inference is a fast-growing field within computational neuroscience that integrates ideas from machine learning, physics, and other fields to mathematically describe the brain and human/animal behavior. The original ideas were conceived by the neuroscientist Dr. Karl Friston who has worked with a number of collaborators in the last two decades to expand the scope and applicability of the theory to the brain and living systems more generally. Active Inference is based in machine learning and has been successfully applied to many different areas of neuroscience and represents the culmination of decades of prior research. It provides an interdisciplinary way to look at a broad range of fields ranging from robotics, economics, cybernetics/control theory, machine learning and more.

The Free Energy Principle is a broader and more theoretical physics-based theory also created by Friston that attempts to answer the famous question posed by Erwin Schrodinger in 1944: “What is life?”. The Free Energy Principle provides the theoretical and philosophical scaffold behind Active Inference and a mathematical modeling framework known as Bayesian Mechanics to model living systems that undergo “self-organization” that is, they stay organized as living being despite changes happening around them that threaten their existence. Under the Free Energy Principle, it is suggested that biological organisms follows the imperative of minimizing a statistical quantity known as variational free energy which serves as a universal objective function in approximate Bayesian inference. Organisms that successfully minimize this quantity are the ones that are able to (actively) infer the current (and future) states of their environments and thus restrict themselves to preferable states conducive to their survival.