Computation by natural systems

The thermodynamics of computation specifies the minimum amount that entropy must increase in the environment of any physical system that implements a given computation, when there are no constraints on how the system operates (the so-called “Landauer limit”). However common engineered computers use digital circuits that physically connect separate gates in a specific topology. Each gate in the circuit performs its own “local” computation, with no a priori constraints on how it operates. In contrast, the circuit’s topology introduces constraints on how the aggregate physical system implementing the overall “global” computation can operate. These constraints cause additional minimal entropy increase in the environment of the overall circuit, beyond that caused by the individual gates.

Here we analyze the relationship between a circuit’s topology and this additional entropy increase, which we call the “circuit Landauer cost”. We also compute a second kind of circuit cost, the “circuit mismatch cost”. This is the extra entropy that is generated if a physical system is designed to achieve minimal entropy production for a particular distribution q over its inputs, but is instead used with an input distribution p that differs from q.

We show that whereas the circuit Landauer cost cannot be negative, circuits can have positive or negative mismatch cost. In fact the total circuit cost (given by summing the two types of cost) can be positive or negative. Thus, the total amount of entropy increase in the environment can be either larger or smaller when a particular computation is implemented with a circuit.

Furthermore, in general different circuits computing the same Boolean function have both different Landauer costs and different mismatch costs. This provides a new set of challenges, never before considered, for how to design a circuit to implement a given computation with minimal thermodynamic cost. As a first step in addressing these challenges, we use tools from the computer science field of circuit complexity to analyze the scaling of thermodynamic costs for different computational tasks.

Professor David Wolpert, Santa Fe Institute, USA

Professor David Wolpert, Santa Fe Institute, USA

David Wolpert is a professor at the Santa Fe Institute, visiting professor at MIT, and adjunct professor at ASU. He is the author of three books, over 200 papers, has three patents, is an associate editor at over half a dozen journals, has received numerous awards, and is a fellow of the IEEE.

He has close to 15,000 citations, in fields ranging from the foundations of physics to machine learning to game theory to information theory to distributed optimization. In particular his machine learning technique of stacking was instrumental in both winning entries for the Netflix competiton, and his papers on the no free lunch theorems have over 5,000 citations. (Details at http://davidwolpert.weebly.com).

He is a world expert on extending game theory to model humans operating in complex, real-world environments; on exploiting machine learning to improve optimization; and on Monte Carlo methods. A major focus of his recent work has been exploiting recent breakthroughs in nonequilibrium statistical physics to formulate major extensions of the thermodynamics of computation.

Before his current position he was the Ulam scholar at the Center for Nonlinear Studies, and before that he was at NASA Ames Research Center and a consulting professor at Stanford University, where he formed the Collective Intelligence group. He has worked at IBM and a data mining startup, and is external faculty at numerous international institutions.

His degrees in Physics are from Princeton and the University of California.

The metaphor of a potential epigenetic differentiation landscape broadly suggests that during differentiation a stem cell follows the steepest descending gradient toward a stable equilibrium state which represents the final cell type. It has been conjectured that there is an analogy to the concept of entropy in statistical mechanics. In order to assess these predictions, Wiesner et al. computed the Shannon entropy for time-resolved single-cell gene expression data in two different experimental setups of haematopoietic differentiation. They found that the behaviour of this entropy measure is in contrast to these predictions.

This is one of very few examples where information theory gives direct insights into a measurable biochemical process. This talk will discuss these findings and their interpretation. And an overview will be given over information theoretic measures of complexity which might quantify the computation of such processes.

Associate Professor Karoline Wiesner, Bristol University, UK

Associate Professor Karoline Wiesner, Bristol University, UK

Karoline Wiesner received a PhD in physics from Uppsala University, and has held postdoctoral positions at the Santa Fe Institute and at the University of California, Davis. Since 2007 she is faculty member of the School of Mathematics at the University of Bristol, where she also co-directs the Bristol Centre for Complexity Sciences. Her work focuses on information theoretic methods for applications in the sciences (e.g. protein dynamics, glass formers, stem cells), and on one foundations of complexity.

Computing can be seen as transformations between nonequilibrium states. In stochastic thermodynamics, the second law can be used to provide a direct connection between the thermodynamic cost needed to induce such transformations and the distance from equilibrium (measured as a relative entropy) of these nonequilibrium states. This nonequilibrium Landauer principle not only holds for detailed balanced dynamics but also for driven non-detailed balanced ones. Remarkably, a closely related result also holds for open chemical reaction networks described by deterministic mass action law kinetics, even in presence of diffusion. These results lay the foundations for a systematic study of the thermodynamic cost needed to produce chemical spatio-temporal signals which play a key role in biology and chemical computing.

Professor Massimiliano Esposito, Université du Luxembourg, Luxembourg

Professor Massimiliano Esposito, Université du Luxembourg, Luxembourg

Professor Massimiliano Esposito is a theoretical physicist specializing in statistical physics and the study of complex systems. His current research focuses on energy and information processing in small quantum systems and biological systems, in particular chemical reaction networks. He obtained his PhD in 2004 at Université Libre de Bruxelles (ULB) in Belgium. After two postdocs in California at UC Irvine and UC San Diego, he came back as a contract researcher for two years at ULB. In 2012 he was awarded a five year Attract Fellowship by the National Research Fund of Luxembourg to start his own research group "Complex Systems and Statistical Mechanics" in the Physics and Materials Science Research Unit at the University of Luxembourg. In 2016 he became Professor of Theoretical Physics and was awarded a five years Consolidator Grant by the European Research Council.

We will describe two aspects of ongoing current research on (classical and analogic) computation using spin models and memristors (resistors with memory). We will first discuss how a simple Ising spin model can be turned into a bottom-up, programmable logic gate computational model. As I will show, it is possible to encode logic gates in the ground state of a spin system using only local external fields, and this model can serve as a proof of concept for more complicated computations using nanomagnets. I will briefly describe at the end why a completely different physical device, the memristor, performs a similar yet different type of computation. Memristors are simply resistors which change resistance as currents flows through them, and are thought of as low-energy 2-ports computational devices which can be used for neuromorphic (brain-inspired) computing. We conclude discussing future experiments to provide a more accurate mapping of memristive annealer to Ising models at an effective (hopefully low?) temperature.

Dr Francesco Caravelli, Los Alamos National Laboratory, USA

Dr Francesco Caravelli, Los Alamos National Laboratory, USA

I began studying Quantum Gravity for my PhD at UWaterloo/Perimeter Institute, but right after I got interested in statistical mechanics and in systems out of equilibrium. Before working at the Los Alamos National Laboratory, I have been a joint postdoc between the SFI (USA) and OCIAM Oxford (UK) with Doyne Farmer, a researcher at UCL (UK) with Francesca Medda and a Senior Researcher at Invenia Labs in Cambridge (UK).

Specifically, I have focused on dynamical graphs,  neuromorphic circuits, and in particular memristors and their collective behavior. Memristors are 2-port passive devices, which have characteristics similar to a resistance but exhibit a very nonlinear behavior. Neuromorphic circuits are, in general, interesting for many reasons. These provide in fact an alternative to the von Neumann architecture at the classical level using analog computation. I have recently started to explore the kinetic behavior of artificial spin ice (magnetic nanoisland), and how to harness their interaction complexity for computation purposes.

The problem of training neural networks is in general non-convex and in principle computationally hard. This is true even for single neurons, when synapses are only allowed a few bits of precision. In practice, however, many fairly greedy heuristics exhibit surprisingly good results.

A large-deviations analysis performed with statistical physics tools has shown that efficient learning is made possible in these models by the existence of rare but very dense regions of optimal configurations, which have appealing robustness and generalization properties.  The analysis also shows that the capacity of the networks saturates fast with the number of synaptic values and thus indicates that very few bits are indeed sufficient for effective learning.  Further analyses also show that these regions can be directly targeted by learning algorithms, in various ways: a particularly simple one consists in exploiting synaptic fluctuations, offering an appealing avenue of investigation to interpret the high levels of stochasticity observed in biological neurons.

Professor Carlo Baldassi, Bocconi University, Milan, Italy

Professor Carlo Baldassi, Bocconi University, Milan, Italy

Carlo Baldassi is assistant professor at Bocconi University. His has a background in theoretical physics, and did his Ph.D. in complex systems in biology, with a thesis in computational neuroscience. His research interests focus on the application of analytical and algorithmic tools of statistical mechanics of disordered systems to computational neuroscience, machine learning, computational biology, and more generally to large-scale inference and optimization problems.

The human brain is a complex organ characterized by a heterogeneous pattern of structural connections that supports long-range functional interactions. Non-invasive imaging techniques now allow for these patterns to be carefully and comprehensively mapped in individual humans, paving the way for a better understanding of how the complex network architecture of structural wiring supports computation, cognition, and behavior. While a large body of work now focuses on descriptive statistics to characterize these wiring patterns, a critical open question lies in how the organization of these networks constrains the potential repertoire of brain dynamics. In this talk, I will describe an approach for understanding how perturbations to brain dynamics propagate through complex wiring patterns, driving the brain into new states of activity. Drawing on a range of disciplinary tools – from graph theory to network control theory and optimization – I will identify control points in brain networks, characterize trajectories of brain activity states following perturbation to those points, and propose a mechanism for how network control evolves in our brains as we grow from children into adults. I will close with a more general discussion of the importance of understanding the role of structural network architecture in computation and cognition.

Professor Danielle S. Bassett, University of Pennsylvania, USA

Professor Danielle S. Bassett, University of Pennsylvania, USA

Danielle S. Bassett is the Eduardo D. Glandt Faculty Fellow and Associate Professor in the Department of Bioengineering at the University of Pennsylvania. She is most well-known for her work blending neural and systems engineering to identify fundamental mechanisms of cognition and disease in human brain networks. She received a B.S. in physics from the Pennsylvania State University and a Ph.D. in physics from the University of Cambridge, UK. Following a postdoctoral position at UC Santa Barbara, she was a Junior Research Fellow at the Sage Center for the Study of the Mind. In 2012, she was named American Psychological Association's `Rising Star' and given an Alumni Achievement Award from the Schreyer Honors College at Pennsylvania State University for extraordinary achievement under the age of 35. In 2014, she was named an Alfred P Sloan Research Fellow and received the MacArthur Fellow Genius Grant. In 2015, she received the IEEE EMBS Early Academic Achievement Award, and was named an ONR Young Investigator. In 2016, she received an NSF CAREER award and was named one of Popular Science’s Brilliant 10. In 2017 she was awarded the Lagrange Prize in Complexity Science. She is the founding director of the Penn Network Visualization Program, a combined undergraduate art internship and K-12 outreach program bridging network science and the visual arts. Her work -- which has resulted in 143 published articles -- has been supported by the National Science Foundation, the National Institutes of Health, the Army Research Office, the Army Research Laboratory, the Alfred P Sloan Foundation, the John D and Catherine T MacArthur Foundation, and the Office of Naval Research

Abstract not yet available.

Professor Renaud Jolivet, University of Geneva and CERN, Switzerland

Professor Renaud Jolivet, University of Geneva and CERN, Switzerland

Renaud Jolivet studied theoretical biophysics at the University of Lausanne (1996-2001) before doing a PhD in computational neuroscience at École polytechnique fédérale de Lausanne in Lausanne, Switzerland (2001-2005). After his doctoral studies, he was a postdoc in Lausanne (2006-2007) and a fellow in Tokyo, Zürich and London (2007-2015). Since 2016, he is Joint Titular Professor in Medical Physics at CERN and at the University of Geneva, Switzerland. His main interest lies in the organization of brain energy metabolism and in the relation between energy consumption and information processing in neural networks, which his team studies in both experiments and computational models. He also holds a particular interest in the non-neuronal cells of the brain.