22
33Array-form Taylor-mode automatic differentiation in JAX.
44
5- ` jet_array ` propagates truncated Taylor polynomials through arbitrary JAX
6- computations, returning all coefficients up to a specified order in a single
7- forward pass. This is the higher-order analogue of ` jax.jvp ` : where ` jvp `
8- takes a primal and one tangent and returns one derivative, ` jet ` takes a
9- primal and a series of tangents and returns the full Taylor expansion.
10-
11- This package is a derivative of JAX's experimental ` jax.experimental.jet `
12- ([ Bettencourt, Johnson, Duvenaud 2019] [ taylor-mode ] ) with the series stored as
13- a single leading-axis array rather than a Python tuple. The array layout
14- enables ` jit ` /` vmap ` /` scan ` over the order axis, and adds an
15- ` effective_order ` parameter for dynamic computation depth at static shapes —
16- useful when the truncation order varies across a batch.
5+ ` jet_array ` propagates a truncated Taylor polynomial of order ` K ` through an
6+ arbitrary JAX computation in a single forward pass, returning all ` K ` Taylor
7+ coefficients of the output as one leading-axis array. This is the higher-order
8+ analogue of ` jax.jvp ` : where ` jvp ` takes one tangent and returns one
9+ derivative, ` jet ` takes a series of ` K ` coefficients and returns the full
10+ order-` K ` Taylor expansion.
11+
12+ The package extends ` jax.experimental.jet `
13+ ([ Bettencourt, Johnson, Duvenaud 2019] [ taylor-mode ] ) in two ways:
14+
15+ 1 . ** Array storage.** Series are stored as a single ` jnp.ndarray ` along axis
16+ 0 instead of a Python tuple. This makes the order axis a first-class JAX
17+ dimension that you can ` jit ` , ` vmap ` , and ` scan ` over.
18+ 2 . ** ` effective_order ` parameter.** A runtime hint that lets a single jitted
19+ program execute with a dynamic truncation depth — useful when each
20+ element of a ` vmap ` -batched call needs a different number of coefficients.
21+
22+ ## When is Taylor-mode AD useful?
23+
24+ Taylor-mode (jet) gives you the first ` K ` derivatives of a scalar function
25+ in ` O(K) ` work. Computing the same ` K ` derivatives by repeatedly nesting
26+ ` jax.grad ` costs ` O(K²) ` (the depth of the trace doubles each time). So
27+ Taylor-mode is the right tool when:
28+
29+ - You need many derivatives at one point — for example, ` K=20 ` to evaluate a
30+ Bell polynomial, an ODE Taylor integrator, or a high-order moment.
31+ - The program you are differentiating contains the same primitives many
32+ times — Taylor-mode amortizes one trace, the nested-grad approach traces
33+ ` K ` times.
34+ - You need the full Taylor * polynomial* (with factorial-divided
35+ coefficients) rather than the unscaled derivatives — for example, to
36+ evaluate ` f(x + h) ` as a series in ` h ` .
37+
38+ For first or second derivatives, ordinary ` jax.grad ` / ` jax.jacrev ` is
39+ faster — Taylor-mode pays only when ` K ≥ ~3 ` .
1740
1841## Install
1942
@@ -25,62 +48,252 @@ pip install jet-array
2548` jax>=0.8,<0.9 ` . Compatibility across JAX versions will be tightened as the
2649package matures.
2750
28- ## Quickstart
51+ ## API
52+
53+ ``` python
54+ jet_array.jet(fun, primals, series, effective_order = None ) -> (primal_out, series_out)
55+ ```
56+
57+ - ** ` fun ` ** — a JAX-traceable callable.
58+ - ** ` primals ` ** — a tuple of input primal values, one per positional argument
59+ of ` fun ` . Each must be a leaf (scalar or array, not a pytree).
60+ - ** ` series ` ** — a tuple of arrays, one per primal. ` series[i][k-1] ` is the
61+ ` k ` -th Taylor coefficient of the ` i ` -th input along the path you are
62+ expanding around. The trailing dimensions of ` series[i] ` must match
63+ ` primals[i] ` . The order ` K ` of the expansion is the leading axis length;
64+ it must be the same for all primals.
65+ - ** ` effective_order ` ** — optional. See below.
66+ - ** Returns** : ` (primal_out, series_out) ` where ` primal_out = fun(*primals) `
67+ and ` series_out[k-1] ` is the ` k ` -th Taylor coefficient of the output, in
68+ the same convention as the input.
69+
70+ ### Coefficient convention
71+
72+ ` series[k-1] ` is the ** Taylor coefficient** : if ` f ` is being expanded along
73+ ` x(t) = x₀ + s₁·t + s₂·t² + … ` , then
74+
75+ ` f(x(t)) = f(x₀) + Σ_{k≥1} series_out[k-1] · t^k ` .
76+
77+ When you set ` series_in = (1, 0, 0, …, 0) ` and a single primal, this reduces
78+ to expanding ` f ` directly around ` x₀ ` , and the relationship to derivatives
79+ is
80+
81+ ` series_out[k-1] = f^(k)(x₀) / k! ` .
82+
83+ To recover the unscaled ` k ` -th derivative, multiply by ` k! ` . (This differs
84+ from ` jax.experimental.jet ` , which returns derivative coefficients without
85+ the factorial scaling. The relationship is documented and tested in
86+ ` tests/test_against_jax_jet.py ` .)
87+
88+ ## Examples
89+
90+ ### Univariate: high-order derivatives at one point
2991
3092``` python
93+ import math
94+ import jax
3195import jax.numpy as jnp
3296from jet_array import jet
3397
98+ jax.config.update(" jax_enable_x64" , True )
99+
34100def f (x ):
35101 return jnp.exp(jnp.sin(x))
36102
37- # 5th-order Taylor series of f at x=0.5, with input series [1, 0, 0, 0, 0]
38- # (i.e. expand around x = 0.5 + t).
39- x0 = 0.5
103+ x0 = jnp.asarray(0.5 )
104+ K = 8 # expansion order
105+ series_in = jnp.zeros(K).at[0 ].set(1.0 ) # direction: x(t) = x0 + t
106+
107+ primal, series = jet(f, (x0,), (series_in,))
108+
109+ # series[k-1] = f^(k)(x0) / k!
110+ # Multiply by k! to get the unscaled k-th derivative.
111+ for k in range (1 , K + 1 ):
112+ print (f " f^( { k} )(x0) = { float (series[k- 1 ]) * math.factorial(k):.6f } " )
113+ ```
114+
115+ ### Bivariate: directional Taylor expansion
116+
117+ ``` python
118+ def g (x , y ):
119+ return jnp.exp(x * y)
120+
121+ x0 = jnp.asarray(0.5 )
122+ y0 = jnp.asarray(0.3 )
123+
124+ # Expand along the path (x0 + t, y0): output series gives ∂^k g/∂x^k / k!
125+ sx = jnp.zeros(4 ).at[0 ].set(1.0 )
126+ sy = jnp.zeros(4 ) # y stays constant
127+ primal, series = jet(g, (x0, y0), (sx, sy))
128+ # series[0] = ∂g/∂x = y₀ · exp(x₀ y₀)
129+ # series[1] = ∂²g/∂x² / 2! = y₀² / 2 · exp(x₀ y₀)
130+
131+ # Diagonal direction (x0+t, y0+t): mixed derivatives appear.
132+ sy_diag = jnp.zeros(4 ).at[0 ].set(1.0 )
133+ _, series_diag = jet(g, (x0, y0), (sx, sy_diag))
134+ # series_diag[k-1] = (1/k!) · sum over multi-indices |α|=k of ∂^α g · α-coefficients
135+ ```
136+
137+ ### Inside ` jax.jit `
138+
139+ ` jet ` is fully traceable. The order ` K ` is part of the input shape, so a
140+ single jit-compiled program handles any computation at that order:
141+
142+ ``` python
143+ @jax.jit
144+ def taylor (x0 ):
145+ series_in = jnp.zeros(8 ).at[0 ].set(1.0 )
146+ return jet(f, (x0,), (series_in,))
147+
148+ primal, series = taylor(jnp.asarray(0.5 ))
149+ ```
150+
151+ If you call ` taylor ` with a different ` K ` , JAX retraces (because the input
152+ shape changed). To avoid that — see ` effective_order ` below.
153+
154+ ### Inside ` jax.vmap `
155+
156+ ``` python
157+ xs = jnp.linspace(0.0 , 1.0 , 100 )
40158series_in = jnp.zeros(5 ).at[0 ].set(1.0 )
159+ primals, series = jax.vmap(
160+ lambda x : jet(f, (x,), (series_in,))
161+ )(xs)
162+ # primals.shape == (100,), series.shape == (100, 5)
163+ ```
164+
165+ ## ` effective_order ` : dynamic truncation under jit
166+
167+ ` effective_order ` is an integer JAX scalar that tells ` jet ` how many
168+ coefficients of the output you actually intend to use. The output array
169+ still has shape ` (K, ...) ` — same as without the parameter — but ` jet `
170+ skips work computing high-order entries inside primitives whose Taylor
171+ rules support the hint (currently ` exp ` , ` expm1 ` , ` log ` , ` log1p ` , ` pow ` ,
172+ ` logistic ` , ` tanh ` , ` erf_inv ` , ` div ` ).
173+
174+ The value of entries beyond ` effective_order ` is unspecified; treat them
175+ as garbage. The point is to * avoid recompilation* when the order varies
176+ across calls or across a ` vmap ` -batched dimension, since the array shape
177+ is fixed.
178+
179+ ### When you want this
180+
181+ A motivating workload: nested-Archimedean copula likelihoods with
182+ per-observation censoring. Each observation needs a Bell-polynomial
183+ expansion whose effective order equals its number of * uncensored* leaves,
184+ which varies across the batch. Without ` effective_order ` , you face two
185+ bad options:
186+
187+ - Pad to the maximum possible order for every observation — wasted work
188+ on always-zero coefficients.
189+ - Re-trace and re-compile per observation — fatal under high cardinality.
190+
191+ ` effective_order ` lets you compile once at ` K = max possible order ` and
192+ pay per-observation work proportional to that observation's actual order.
193+
194+ ### Example
41195
42- primal_out, series_out = jet(f, (x0,), (series_in,))
43- # primal_out = f(0.5)
44- # series_out = [f'(0.5), f''(0.5)/2!, f'''(0.5)/3!, f^(4)(0.5)/4!, f^(5)(0.5)/5!]
196+ ``` python
197+ import jax
198+ import jax.numpy as jnp
199+ from jet_array import jet
200+
201+ K_MAX = 16 # static array size
202+
203+ def slow_chain (x ):
204+ # A composition that uses several primitives whose rules respect
205+ # effective_order — exp/log/logistic are good candidates.
206+ y = jnp.exp(x)
207+ y = jnp.log1p(y)
208+ y = jnp.tanh(y)
209+ return y
210+
211+ @jax.jit
212+ def taylor (x , k_dyn ):
213+ series_in = jnp.zeros(K_MAX ).at[0 ].set(1.0 )
214+ return jet(slow_chain, (x,), (series_in,), effective_order = k_dyn)
215+
216+ # Both calls use the same compiled XLA program.
217+ # The second runs more work — it computes 12 coefficients instead of 4.
218+ p1, s1 = taylor(jnp.asarray(0.5 ), jnp.array(4 ))
219+ p2, s2 = taylor(jnp.asarray(0.5 ), jnp.array(12 ))
220+ # s1[:4] is the answer you can use; s1[4:] is unspecified.
221+ # s2[:12] is the answer; s2[12:] is unspecified.
45222```
46223
224+ For workloads where the dynamic order varies per element of a
225+ ` vmap ` -batched dimension, pass an array of ` effective_order ` values and
226+ combine with ` jax.vmap ` .
227+
228+ ### Caveats
229+
230+ - Only primitives in ` _rules_with_effective_order ` honor the hint. Other
231+ primitives compute the full series. So for a function like ` sin(x) `
232+ alone, ` effective_order ` has no effect — ` sin ` always computes all ` K `
233+ coefficients.
234+ - Output entries with index ` ≥ effective_order ` are not guaranteed to
235+ contain anything specific (zero, garbage, or a partial answer). Slice
236+ to ` series_out[:effective_order] ` before using them.
237+ - The win is largest when the dominant cost is in the convolution-style
238+ loops that the hint short-circuits. For shallow programs the overhead of
239+ the conditional may exceed the savings.
240+
47241## Why arrays instead of tuples
48242
49- ` jax.experimental.jet ` stores the series as a Python tuple, which prevents
50- ` jit ` /` vmap ` from treating the order axis as a regular array dimension. This
51- matters when:
243+ ` jax.experimental.jet ` stores the series as a Python tuple. That works for
244+ small ` K ` known at trace time but causes friction otherwise:
245+
246+ - The order axis is invisible to JAX — you cannot ` vmap ` or ` scan ` over it.
247+ - A tuple of length ` K ` produces ` K ` separate jaxpr equations per
248+ primitive, so trace time grows linearly in ` K ` even when the
249+ computation is constant per coefficient.
250+ - ` effective_order ` makes no sense for a Python tuple.
251+
252+ Storing series along axis 0 of a single array makes all of these go away.
253+ The cost is some boilerplate inside primitive rules to convolve along that
254+ axis, which ` jet_array ` provides.
255+
256+ ## Coverage
257+
258+ Custom Taylor rules are provided for the JAX primitives that arise in
259+ typical scientific code: arithmetic and broadcasting, ` exp ` , ` expm1 ` ,
260+ ` log ` , ` log1p ` , ` sin ` , ` cos ` , ` sinh ` , ` cosh ` , ` tanh ` , ` logistic ` , ` erf ` ,
261+ ` erf_inv ` , ` pow ` , ` square ` , ` sqrt ` , ` div ` , ` dynamic_slice ` ,
262+ ` dynamic_update_slice ` , and ` cumsum ` /` cumprod ` . Linear and zero-derivative
263+ primitives are handled generically. For untraced primitives the rule
264+ falls back to the standard convolution propagator.
265+
266+ The full correctness suite checks every supported primitive against
267+ ` jax.experimental.jet ` at multiple expansion points and orders up to 20.
52268
53- - The series order is large (hundreds of coefficients) — Python-level
54- iteration becomes a tracing-time cost.
55- - The order is data-dependent — tuples force re-tracing every time the
56- length changes.
57- - You want to ` vmap ` or ` scan ` over a batch where each element has its own
58- effective order.
269+ ## Limitations
59270
60- ` jet_array ` stores all coefficients along axis 0 of a single array,
61- addressing all three.
271+ - Pytree primal inputs are not supported (each primal must be a leaf).
272+ - The package depends on ` jax._src ` internals and currently targets
273+ JAX 0.8.x.
274+ - ` effective_order ` is a hint, not a guarantee — see the caveats above.
62275
63276## Citation
64277
65- If you use ` jet-array ` in academic work, please cite:
278+ If you use ` jet-array ` in academic work, please cite the repository :
66279
67280``` bibtex
68- @misc{yang2026copulaad ,
69- title={Archimedean Copula Inference via Taylor-Mode AD },
70- author= {Yang, Cambridge and Li, Dongdong},
71- year= {2026},
72- note={arXiv preprint },
281+ @software{jet_array ,
282+ title = {jet-array: array-form Taylor-mode automatic differentiation in JAX },
283+ author = {Yang, Cambridge and Li, Dongdong},
284+ year = {2026},
285+ url = {https://github.com/thisiscam/jet-array },
73286}
74287```
75288
76289The underlying Taylor-mode algorithm is from:
77290
78291``` bibtex
79292@inproceedings{bettencourt2019taylor,
80- title= {Taylor-mode automatic differentiation for higher-order derivatives in JAX},
81- author= {Bettencourt, Jesse and Johnson, Matthew J. and Duvenaud, David},
82- booktitle= {NeurIPS Program Transformations Workshop},
83- year= {2019},
293+ title = {Taylor-mode automatic differentiation for higher-order derivatives in JAX},
294+ author = {Bettencourt, Jesse and Johnson, Matthew J. and Duvenaud, David},
295+ booktitle = {NeurIPS Program Transformations Workshop},
296+ year = {2019},
84297}
85298```
86299
0 commit comments