Bits, neurons, and qubits for sustainable AI

It is anticipated that the next generation of wireless networks will incorporate AI to a significant degree at all network layers. A major part of this trend is the movement of AI and machine learning functions to the network edge. This is due to several reasons: (i) a growing number of AI applications demand implementations involving end-user devices, (ii) much data of interest is born at the network edge on smart phones and IoT devices, and (iii) fog/edge computing has emerged to take advantage of the increasing sophistication of end-user devices.  A notable framework for engaging the wireless network edge in machine learning is wireless federated learning, in which multiple end-user devices collaborate with the help of an aggregator to build a common model, each using its local data. In this framework, exchanges between end-user devices and the aggregator necessarily take place over wireless links. Since wireless networks are notoriously resource-limited, this creates a situation in which the interactions between the wireless medium and machine learning algorithms must be considered as a factor in the design and implementation of AI applications. This talk will explore aspects of this problem, including trade-offs among energy consumption and other criteria such as spectral efficiency, learning rate, model complexity and data privacy.

Professor H Vincent Poor FREng ForMemRS

Princeton University, USA

Professor H Vincent Poor FREng ForMemRS

Princeton University, USA

H Vincent Poor is the Michael Henry Strater University Professor at Princeton University, where his interests include information theory, machine learning and network science, and their applications in wireless networks, energy systems and related fields. Since 2004, he has also been a Visiting Professor at Imperial College, and he has held visiting positions at a number of other universities as well, including most recently at Berkeley and Caltech. Among his publications is the book Machine Learning and Wireless Communications (Cambridge University Press, 2022). Dr Poor is a member of the US National Academy of Engineering and the US National Academy of Sciences, an International Fellow of the Royal Academy of Engineering, and a Foreign Member of the Royal Society, among memberships in other national and international academies.  Recognition of his work includes the IEEE Alexander Graham Bell Medal and honorary doctorates from a number of universities in Asia, Europe, and North America.

I will focus the talk on a few non-linear transforms and their use for compression. The first is a "textual transform" sharing key properties with traditional transforms underlying much of our current multimedia compression technologies. It can form the basis for compression at bit rates until recently considered uselessly low, and for boosting human satisfaction from reconstructions at more traditional bit rates. The second is a "neural transform" - such as in the "implicit neural representation" framework - allowing to view the tension between compression ratio and neural network performance through the lens of rate-distortion theory and to develop information theoretically guided network pruning strategies. The third is a "Lempel-Ziv (LZ) transform", which can take any of a large class of compressors into new ones that share the main universality properties of the Lempel-Ziv compressor. Collectively, these transforms give insight into achievable trade-offs between compression and computation. These insights carry over to other information processing tasks such as classification, denoising and "GenAI".

Professor Tsachy Weissman

Stanford University, USA

Professor Tsachy Weissman

Stanford University, USA

Tsachy Weissman is the Robert and Barbara Kleist Professor in the School of Engineering, and Professor of Electrical Engineering at Stanford University, where he has been on the faculty since 2003. His researching and teaching focus on the science of information, with applications spanning genomics, neuroscience, and technology. He has been serving on editorial boards for scientific journals, technical advisory boards in industry, and as the Founding Director of the Stanford Compression Forum. His recent projects include the SHTEM summer internship program for high schoolers, the Starling initiative for data integrity, and Stagecast, a low-latency video platform allowing actors and singers to perform together in real-time while geographically distributed. He has received multiple awards for his research and teaching, including best paper awards from the IEEE Information Theory and Communications Societies, while his students received best student authored paper awards at the top conferences of their areas of scholarship. He has prototyped Guardant Health’s algorithms for early detection of cancer from blood tests, co-founded and sold Compressable to Amazon, and worked at AWS on reducing humanity’s cloud storage footprint via compression. His favourite gig to date was advising the HBO show Silicon Valley. He hates writing about himself in the third person. 

There is growing interest in improving the performance of large language models for reasoning tasks by drawing on techniques such as chain-of-thought reasoning. Moreover, recent empirical findings demonstrate a scaling law relationship between inference-time compute and performance. Unfortunately, such techniques increase the computational and energetic resources used during the inference phase of AI systems, scaling with the amount of inference-time compute. Unlike large model training, such inference energy is not a one-time cost, but is incurred each time the model is used. Further, such computational effort must be done in essentially real time, rather than over periods of weeks or months that allow some time shifting. Recent research has proposed a concept of information batteries where the intermittency of renewable power is smoothed by storing energy in the form of information, specifically by storing results of predicted computational tasks, which may be relevant for energy-intensive AI applications. Here we ask three questions: (1) whether there are mathematical theories akin to information-theoretic analyses that provide insight into inference-compute scaling laws; (2) whether information batteries might fruitfully be used for AI inference workloads; and (3) whether there are information-theoretic limits to information batteries.

Professor Lav R Varshney

University of Illinois Urbana-Champaign, USA

Professor Lav R Varshney

University of Illinois Urbana-Champaign, USA

Lav Varshney is an associate professor of electrical and computer engineering at the University of Illinois Urbana-Champaign, co-founder and CEO of Kocree, Inc, a start-up company using novel human-integrated AI in social music co-creativity platforms to enhance human wellbeing across society, and chief scientist of Ensaras, Inc, a start-up company focused on AI and wastewater treatment.  He also holds appointments at RAND Corporation and at Brookhaven National Laboratory. He is a former White House staffer, having served on the National Security Council staff as a White House Fellow, where he contributed to national AI and wireless communications policy. His research interests include information theory and artificial intelligence. He received his BS degree from Cornell University and his SM and PhD degrees from the Massachusetts Institute of Technology.

In-Memory Computing (IMC) is an innovative paradigm that integrates computation and memory, enabling ultra-dense, energy-efficient, and high-throughput hardware acceleration for deep neural networks. This talk will begin with an overview of the latest advancements in IMC, comparing various analogue and digital approaches and assessing their suitability for deep learning workloads. In the second part, Dr Eleftheriou will explore Axelera AI's cutting-edge implementation of Digital In-Memory Computing (D-IMC) combined with RISC-V technology. This unique integration delivers a scalable, high-performance platform for AI inference tasks, ranging from edge-based computer vision to emerging generative AI workloads. By leveraging the strengths of D-IMC, Axelera AI is redefining the landscape of AI acceleration, achieving exceptional energy efficiency, performance, and cost-effectiveness while driving the next generation of AI deployment.

Dr Evangelos Eleftheriou

Axelera AI, The Netherlands

Dr Evangelos Eleftheriou

Axelera AI, The Netherlands

Evangelos Eleftheriou is the CTO and co-founder of Axelera AI. Previously, he held various research and management positions at IBM Research - Zurich. He has a PhD and a Master of Eng in Electrical Engineering from Carleton University, Canada, and a BSc in Electrical & Computer Engineering from the University of Patras, Greece. His interests include AI, machine learning, and emerging computing paradigms such as neuromorphic and in-memory computing. He has authored over 250 publications and holds over 160 patents.

Evangelos is an IEEE Fellow and co-recipient of the 2003 IEEE ComS Leonard G Abraham Prize Paper Award, the 2005 Technology Award of the Eduard Rhein Foundation, and the 2009 IEEE Control Systems Technology Award and IEEE Transactions on Control Systems Technology Outstanding Paper Award. He was also appointed an IBM Fellow in 2005 and inducted into the IBM Academy of Technology. In 2016, he received an honoris causa professorship from the University of Patras, Greece, and in 2018, he was inducted into the US National Academy of Engineering as Foreign Member.

Self-timed circuits can significantly improve the energy-efficiency of large-scale digital computer systems. The benefits of these circuits include switching activity only when there is useful computation being performed, and the ability to optimise for expected delays rather than the system-wide worst-case delay. This benefit is partially offset by the overheads introduced to implement synchronisation and communication protocols. We present both analytical results as well as chip examples that demonstrate the efficiency of self-timed systems.

The benefits of self-timed logic are immediately evident when designing large-scale neuromorphic systems. Many past and recent large-scale neuromorphic systems use self-timed digital electronics to achieve energy-efficient operation. We will explain why this is the case, and highlight the uses of self-timed logic in the design of the TrueNorth and Braindrop neuromorphic systems. We will also describe ongoing work on creating a quantitative, full-stack approach to evaluating the trade-offs in neuromorphic system design, enabled by our recent work on open-source tools for the design and implementation of self-timed logic.

Professor Rajit Manohar

Yale University, USA

Professor Rajit Manohar

Yale University, USA

Rajit Manohar is the John C Malone Professor of Electrical and Computer Engineering and Professor of Computer Science at Yale. He received his BS (1994), MS (1995), and PhD (1998) from Caltech. His group conducts research on the design, analysis, and implementation of self-timed systems. He is the recipient of twelve best paper awards, nine teaching awards, and was named to MIT technology review's top 35 young innovators under 35 for contributions to low power microprocessor design. His work includes the design and implementation of a number of self-timed VLSI chips including the first high-performance asynchronous microprocessor, the first microprocessor for sensor networks, the first asynchronous dataflow FPGA, and the first deterministic large-scale neuromorphic architecture. His group developed the first true ASIC flow for asynchronous circuits. Prior to Yale, He founded Achronix Semiconductor to commercialise high-performance asynchronous FPGAs.

Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation. Hardware specialization and acceleration is a key enabler to improve operational efficiency of DNNs, in turn requiring synergistic cross-layer design across algorithms, hardware, and software.

In this talk I will present this holistic approach adopted in the design of a multi-TOPs AI hardware accelerator. Key advances in the AI algorithm/application-level exploiting approximate computing techniques enable deriving low-precision DNNs models that maintain the same level of accuracy. Hardware performance-aware design space exploration is critical during compilation to map DNNs with diverse computational characteristics systematically and optimally while preserving familiar programming and user interfaces. The opportunities to co-optimize the algorithms, hardware, and the software provides the roadmap to continue to deliver superior performance over the next decade. 

Dr Viji Srinivasan

IBM, USA

Dr Viji Srinivasan

IBM, USA

Viji Srinivasan is a Distinguished Research Scientist and a manager of the accelerator architectures and compilers group at the IBM TJ Watson Research Center in Yorktown Heights. At IBM, she has worked on various aspects of data management including energy-efficient processor designs, microarchitecture of the memory hierarchies of large-scale servers, cache coherence management of symmetric multiprocessors, accelerators for data analytics applications and more recently end-to-end accelerator solutions for AI. She leads the architecture and compiler team of IBM’s AI Accelerator. Many of her research contributions have been incorporated into IBM’s Power & System-z Enterprise-class servers.