Maynooth University

Maynooth University ePrints and eTheses Archive

Maynooth University Library

Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator

Pearlmutter, Barak A. and Siskind , Jeffrey Mark (2008) Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator. ACM Transactions on Programming Languages and Systems, 30 (2). 7.1-7.36. ISSN 0164-0925

[img] Download (354kB)

Abstract

We show that reverse-mode AD (Automatic Differentiation)—a generalized gradient-calculation operator—can be incorporated as a first-class function in an augmented lambda calculus, and therefore into a functional-programming language. Closure is achieved, in that the new operator can be applied to any expression in the augmented language, yielding an expression in that language. This requires the resolution of two major technical issues: (a) how to transform nested lambda expressions, including those with free-variable references, and (b) how to support self application of the AD machinery. AD transformations preserve certain complexity properties, among them that the reverse phase of the reverse-mode AD transformation of a function have the same temporal complexity as the original untransformed function. First-class unrestricted AD operators increase the expressive power available to the numeric programmer, and may have significant practical implications for the construction of numeric software that is robust, modular, concise, correct, and efficient.

Item Type: Article
Additional Information: The work of B. A. Pearlmutter was supported, in part, by Science Foundation Ireland grant 00/PI.1/C067 and the Higher Education Authority of Ireland. The work of J. M. Siskind was supported, in part, by National Science Foundation (NSF) grant CCF-0438806. © ACM, 2008. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Programming Languages and Systems (TOPLAS) Volume 30 Issue 2, March 2008, http://doi.acm.org/10.1145/1330017.1330018
Keywords: closures; derivatives; forward-mode AD; higher-order AD; higher-order functional languages; Jacobian; program transformation; reflection;
Subjects: Science & Engineering > Hamilton Institute
Science & Engineering > Computer Science
Item ID: 3504
Depositing User: Dr. Barak Pearlmutter
Date Deposited: 29 Feb 2012 14:25
Journal or Publication Title: ACM Transactions on Programming Languages and Systems
Publisher: Association for Computing Machinery (ACM)
Refereed: Yes
URI:

Repository Staff Only(login required)

View Item Item control page

Document Downloads

More statistics for this item...