Skip to content

Commit 055b8bc

Browse files
thisiscamclaude
andcommitted
Expand README and add correctness tests against jax.experimental.jet
README: - Document the Taylor-coefficient convention and its 1/k! relationship to jax.experimental.jet - Add quickstart, bivariate, jit, and vmap examples - Document effective_order with a worked example, caveats, and the motivating censored-likelihood use case - List the JAX primitives that have custom Taylor rules - Cite this repository instead of the copula paper Tests (482 new): - tests/conftest.py: fixtures and parameter lists shared across files - tests/test_against_jax_jet.py: coefficient-by-coefficient comparison against jax.experimental.jet at multiple points and orders up to 20, plus k-th derivative recovery and constant/identity edge cases Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b7ea923 commit 055b8bc

3 files changed

Lines changed: 457 additions & 40 deletions

File tree

‎README.md‎

Lines changed: 253 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,41 @@
22

33
Array-form Taylor-mode automatic differentiation in JAX.
44

5-
`jet_array` propagates truncated Taylor polynomials through arbitrary JAX
6-
computations, returning all coefficients up to a specified order in a single
7-
forward pass. This is the higher-order analogue of `jax.jvp`: where `jvp`
8-
takes a primal and one tangent and returns one derivative, `jet` takes a
9-
primal and a series of tangents and returns the full Taylor expansion.
10-
11-
This package is a derivative of JAX's experimental `jax.experimental.jet`
12-
([Bettencourt, Johnson, Duvenaud 2019][taylor-mode]) with the series stored as
13-
a single leading-axis array rather than a Python tuple. The array layout
14-
enables `jit`/`vmap`/`scan` over the order axis, and adds an
15-
`effective_order` parameter for dynamic computation depth at static shapes —
16-
useful when the truncation order varies across a batch.
5+
`jet_array` propagates a truncated Taylor polynomial of order `K` through an
6+
arbitrary JAX computation in a single forward pass, returning all `K` Taylor
7+
coefficients of the output as one leading-axis array. This is the higher-order
8+
analogue of `jax.jvp`: where `jvp` takes one tangent and returns one
9+
derivative, `jet` takes a series of `K` coefficients and returns the full
10+
order-`K` Taylor expansion.
11+
12+
The package extends `jax.experimental.jet`
13+
([Bettencourt, Johnson, Duvenaud 2019][taylor-mode]) in two ways:
14+
15+
1. **Array storage.** Series are stored as a single `jnp.ndarray` along axis
16+
0 instead of a Python tuple. This makes the order axis a first-class JAX
17+
dimension that you can `jit`, `vmap`, and `scan` over.
18+
2. **`effective_order` parameter.** A runtime hint that lets a single jitted
19+
program execute with a dynamic truncation depth — useful when each
20+
element of a `vmap`-batched call needs a different number of coefficients.
21+
22+
## When is Taylor-mode AD useful?
23+
24+
Taylor-mode (jet) gives you the first `K` derivatives of a scalar function
25+
in `O(K)` work. Computing the same `K` derivatives by repeatedly nesting
26+
`jax.grad` costs `O(K²)` (the depth of the trace doubles each time). So
27+
Taylor-mode is the right tool when:
28+
29+
- You need many derivatives at one point — for example, `K=20` to evaluate a
30+
Bell polynomial, an ODE Taylor integrator, or a high-order moment.
31+
- The program you are differentiating contains the same primitives many
32+
times — Taylor-mode amortizes one trace, the nested-grad approach traces
33+
`K` times.
34+
- You need the full Taylor *polynomial* (with factorial-divided
35+
coefficients) rather than the unscaled derivatives — for example, to
36+
evaluate `f(x + h)` as a series in `h`.
37+
38+
For first or second derivatives, ordinary `jax.grad` / `jax.jacrev` is
39+
faster — Taylor-mode pays only when `K ≥ ~3`.
1740

1841
## Install
1942

@@ -25,62 +48,252 @@ pip install jet-array
2548
`jax>=0.8,<0.9`. Compatibility across JAX versions will be tightened as the
2649
package matures.
2750

28-
## Quickstart
51+
## API
52+
53+
```python
54+
jet_array.jet(fun, primals, series, effective_order=None) -> (primal_out, series_out)
55+
```
56+
57+
- **`fun`** — a JAX-traceable callable.
58+
- **`primals`** — a tuple of input primal values, one per positional argument
59+
of `fun`. Each must be a leaf (scalar or array, not a pytree).
60+
- **`series`** — a tuple of arrays, one per primal. `series[i][k-1]` is the
61+
`k`-th Taylor coefficient of the `i`-th input along the path you are
62+
expanding around. The trailing dimensions of `series[i]` must match
63+
`primals[i]`. The order `K` of the expansion is the leading axis length;
64+
it must be the same for all primals.
65+
- **`effective_order`** — optional. See below.
66+
- **Returns**: `(primal_out, series_out)` where `primal_out = fun(*primals)`
67+
and `series_out[k-1]` is the `k`-th Taylor coefficient of the output, in
68+
the same convention as the input.
69+
70+
### Coefficient convention
71+
72+
`series[k-1]` is the **Taylor coefficient**: if `f` is being expanded along
73+
`x(t) = x₀ + s₁·t + s₂·t² + …`, then
74+
75+
`f(x(t)) = f(x₀) + Σ_{k≥1} series_out[k-1] · t^k`.
76+
77+
When you set `series_in = (1, 0, 0, …, 0)` and a single primal, this reduces
78+
to expanding `f` directly around `x₀`, and the relationship to derivatives
79+
is
80+
81+
`series_out[k-1] = f^(k)(x₀) / k!`.
82+
83+
To recover the unscaled `k`-th derivative, multiply by `k!`. (This differs
84+
from `jax.experimental.jet`, which returns derivative coefficients without
85+
the factorial scaling. The relationship is documented and tested in
86+
`tests/test_against_jax_jet.py`.)
87+
88+
## Examples
89+
90+
### Univariate: high-order derivatives at one point
2991

3092
```python
93+
import math
94+
import jax
3195
import jax.numpy as jnp
3296
from jet_array import jet
3397

98+
jax.config.update("jax_enable_x64", True)
99+
34100
def f(x):
35101
return jnp.exp(jnp.sin(x))
36102

37-
# 5th-order Taylor series of f at x=0.5, with input series [1, 0, 0, 0, 0]
38-
# (i.e. expand around x = 0.5 + t).
39-
x0 = 0.5
103+
x0 = jnp.asarray(0.5)
104+
K = 8 # expansion order
105+
series_in = jnp.zeros(K).at[0].set(1.0) # direction: x(t) = x0 + t
106+
107+
primal, series = jet(f, (x0,), (series_in,))
108+
109+
# series[k-1] = f^(k)(x0) / k!
110+
# Multiply by k! to get the unscaled k-th derivative.
111+
for k in range(1, K + 1):
112+
print(f"f^({k})(x0) = {float(series[k-1]) * math.factorial(k):.6f}")
113+
```
114+
115+
### Bivariate: directional Taylor expansion
116+
117+
```python
118+
def g(x, y):
119+
return jnp.exp(x * y)
120+
121+
x0 = jnp.asarray(0.5)
122+
y0 = jnp.asarray(0.3)
123+
124+
# Expand along the path (x0 + t, y0): output series gives ∂^k g/∂x^k / k!
125+
sx = jnp.zeros(4).at[0].set(1.0)
126+
sy = jnp.zeros(4) # y stays constant
127+
primal, series = jet(g, (x0, y0), (sx, sy))
128+
# series[0] = ∂g/∂x = y₀ · exp(x₀ y₀)
129+
# series[1] = ∂²g/∂x² / 2! = y₀² / 2 · exp(x₀ y₀)
130+
131+
# Diagonal direction (x0+t, y0+t): mixed derivatives appear.
132+
sy_diag = jnp.zeros(4).at[0].set(1.0)
133+
_, series_diag = jet(g, (x0, y0), (sx, sy_diag))
134+
# series_diag[k-1] = (1/k!) · sum over multi-indices |α|=k of ∂^α g · α-coefficients
135+
```
136+
137+
### Inside `jax.jit`
138+
139+
`jet` is fully traceable. The order `K` is part of the input shape, so a
140+
single jit-compiled program handles any computation at that order:
141+
142+
```python
143+
@jax.jit
144+
def taylor(x0):
145+
series_in = jnp.zeros(8).at[0].set(1.0)
146+
return jet(f, (x0,), (series_in,))
147+
148+
primal, series = taylor(jnp.asarray(0.5))
149+
```
150+
151+
If you call `taylor` with a different `K`, JAX retraces (because the input
152+
shape changed). To avoid that — see `effective_order` below.
153+
154+
### Inside `jax.vmap`
155+
156+
```python
157+
xs = jnp.linspace(0.0, 1.0, 100)
40158
series_in = jnp.zeros(5).at[0].set(1.0)
159+
primals, series = jax.vmap(
160+
lambda x: jet(f, (x,), (series_in,))
161+
)(xs)
162+
# primals.shape == (100,), series.shape == (100, 5)
163+
```
164+
165+
## `effective_order`: dynamic truncation under jit
166+
167+
`effective_order` is an integer JAX scalar that tells `jet` how many
168+
coefficients of the output you actually intend to use. The output array
169+
still has shape `(K, ...)` — same as without the parameter — but `jet`
170+
skips work computing high-order entries inside primitives whose Taylor
171+
rules support the hint (currently `exp`, `expm1`, `log`, `log1p`, `pow`,
172+
`logistic`, `tanh`, `erf_inv`, `div`).
173+
174+
The value of entries beyond `effective_order` is unspecified; treat them
175+
as garbage. The point is to *avoid recompilation* when the order varies
176+
across calls or across a `vmap`-batched dimension, since the array shape
177+
is fixed.
178+
179+
### When you want this
180+
181+
A motivating workload: nested-Archimedean copula likelihoods with
182+
per-observation censoring. Each observation needs a Bell-polynomial
183+
expansion whose effective order equals its number of *uncensored* leaves,
184+
which varies across the batch. Without `effective_order`, you face two
185+
bad options:
186+
187+
- Pad to the maximum possible order for every observation — wasted work
188+
on always-zero coefficients.
189+
- Re-trace and re-compile per observation — fatal under high cardinality.
190+
191+
`effective_order` lets you compile once at `K = max possible order` and
192+
pay per-observation work proportional to that observation's actual order.
193+
194+
### Example
41195

42-
primal_out, series_out = jet(f, (x0,), (series_in,))
43-
# primal_out = f(0.5)
44-
# series_out = [f'(0.5), f''(0.5)/2!, f'''(0.5)/3!, f^(4)(0.5)/4!, f^(5)(0.5)/5!]
196+
```python
197+
import jax
198+
import jax.numpy as jnp
199+
from jet_array import jet
200+
201+
K_MAX = 16 # static array size
202+
203+
def slow_chain(x):
204+
# A composition that uses several primitives whose rules respect
205+
# effective_order — exp/log/logistic are good candidates.
206+
y = jnp.exp(x)
207+
y = jnp.log1p(y)
208+
y = jnp.tanh(y)
209+
return y
210+
211+
@jax.jit
212+
def taylor(x, k_dyn):
213+
series_in = jnp.zeros(K_MAX).at[0].set(1.0)
214+
return jet(slow_chain, (x,), (series_in,), effective_order=k_dyn)
215+
216+
# Both calls use the same compiled XLA program.
217+
# The second runs more work — it computes 12 coefficients instead of 4.
218+
p1, s1 = taylor(jnp.asarray(0.5), jnp.array(4))
219+
p2, s2 = taylor(jnp.asarray(0.5), jnp.array(12))
220+
# s1[:4] is the answer you can use; s1[4:] is unspecified.
221+
# s2[:12] is the answer; s2[12:] is unspecified.
45222
```
46223

224+
For workloads where the dynamic order varies per element of a
225+
`vmap`-batched dimension, pass an array of `effective_order` values and
226+
combine with `jax.vmap`.
227+
228+
### Caveats
229+
230+
- Only primitives in `_rules_with_effective_order` honor the hint. Other
231+
primitives compute the full series. So for a function like `sin(x)`
232+
alone, `effective_order` has no effect — `sin` always computes all `K`
233+
coefficients.
234+
- Output entries with index `≥ effective_order` are not guaranteed to
235+
contain anything specific (zero, garbage, or a partial answer). Slice
236+
to `series_out[:effective_order]` before using them.
237+
- The win is largest when the dominant cost is in the convolution-style
238+
loops that the hint short-circuits. For shallow programs the overhead of
239+
the conditional may exceed the savings.
240+
47241
## Why arrays instead of tuples
48242

49-
`jax.experimental.jet` stores the series as a Python tuple, which prevents
50-
`jit`/`vmap` from treating the order axis as a regular array dimension. This
51-
matters when:
243+
`jax.experimental.jet` stores the series as a Python tuple. That works for
244+
small `K` known at trace time but causes friction otherwise:
245+
246+
- The order axis is invisible to JAX — you cannot `vmap` or `scan` over it.
247+
- A tuple of length `K` produces `K` separate jaxpr equations per
248+
primitive, so trace time grows linearly in `K` even when the
249+
computation is constant per coefficient.
250+
- `effective_order` makes no sense for a Python tuple.
251+
252+
Storing series along axis 0 of a single array makes all of these go away.
253+
The cost is some boilerplate inside primitive rules to convolve along that
254+
axis, which `jet_array` provides.
255+
256+
## Coverage
257+
258+
Custom Taylor rules are provided for the JAX primitives that arise in
259+
typical scientific code: arithmetic and broadcasting, `exp`, `expm1`,
260+
`log`, `log1p`, `sin`, `cos`, `sinh`, `cosh`, `tanh`, `logistic`, `erf`,
261+
`erf_inv`, `pow`, `square`, `sqrt`, `div`, `dynamic_slice`,
262+
`dynamic_update_slice`, and `cumsum`/`cumprod`. Linear and zero-derivative
263+
primitives are handled generically. For untraced primitives the rule
264+
falls back to the standard convolution propagator.
265+
266+
The full correctness suite checks every supported primitive against
267+
`jax.experimental.jet` at multiple expansion points and orders up to 20.
52268

53-
- The series order is large (hundreds of coefficients) — Python-level
54-
iteration becomes a tracing-time cost.
55-
- The order is data-dependent — tuples force re-tracing every time the
56-
length changes.
57-
- You want to `vmap` or `scan` over a batch where each element has its own
58-
effective order.
269+
## Limitations
59270

60-
`jet_array` stores all coefficients along axis 0 of a single array,
61-
addressing all three.
271+
- Pytree primal inputs are not supported (each primal must be a leaf).
272+
- The package depends on `jax._src` internals and currently targets
273+
JAX 0.8.x.
274+
- `effective_order` is a hint, not a guarantee — see the caveats above.
62275

63276
## Citation
64277

65-
If you use `jet-array` in academic work, please cite:
278+
If you use `jet-array` in academic work, please cite the repository:
66279

67280
```bibtex
68-
@misc{yang2026copulaad,
69-
title={Archimedean Copula Inference via Taylor-Mode AD},
70-
author={Yang, Cambridge and Li, Dongdong},
71-
year={2026},
72-
note={arXiv preprint},
281+
@software{jet_array,
282+
title = {jet-array: array-form Taylor-mode automatic differentiation in JAX},
283+
author = {Yang, Cambridge and Li, Dongdong},
284+
year = {2026},
285+
url = {https://github.com/thisiscam/jet-array},
73286
}
74287
```
75288

76289
The underlying Taylor-mode algorithm is from:
77290

78291
```bibtex
79292
@inproceedings{bettencourt2019taylor,
80-
title={Taylor-mode automatic differentiation for higher-order derivatives in JAX},
81-
author={Bettencourt, Jesse and Johnson, Matthew J. and Duvenaud, David},
82-
booktitle={NeurIPS Program Transformations Workshop},
83-
year={2019},
293+
title = {Taylor-mode automatic differentiation for higher-order derivatives in JAX},
294+
author = {Bettencourt, Jesse and Johnson, Matthew J. and Duvenaud, David},
295+
booktitle = {NeurIPS Program Transformations Workshop},
296+
year = {2019},
84297
}
85298
```
86299

‎tests/conftest.py‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import jax
2+
jax.config.update("jax_enable_x64", True)
3+
4+
import pytest
5+
from jax import lax
6+
7+
8+
@pytest.fixture
9+
def factorial():
10+
def _factorial(n):
11+
if n <= 1:
12+
return 1
13+
return n * _factorial(n - 1)
14+
return _factorial
15+
16+
17+
# Common univariate test functions: smooth, well-defined Taylor expansions.
18+
TEST_FUNCTION_PARAMS = [
19+
('erf', lambda x: lax.erf(x)),
20+
('exp', lambda x: lax.exp(x)),
21+
('log1p', lambda x: lax.log1p(x)),
22+
('expm1', lambda x: lax.expm1(x)),
23+
('sin', lambda x: lax.sin(x)),
24+
('cos', lambda x: lax.cos(x)),
25+
('sinh', lambda x: lax.sinh(x)),
26+
('cosh', lambda x: lax.cosh(x)),
27+
('tanh', lambda x: lax.tanh(x)),
28+
('logistic', lambda x: lax.logistic(x)),
29+
('square', lambda x: x**2),
30+
('compose_exp_sin', lambda x: lax.exp(lax.sin(x))),
31+
('compose_log_cosh', lambda x: lax.log(lax.cosh(x))),
32+
]
33+
34+
35+
MULTIVARIATE_TEST_PARAMS = [
36+
('bivariate_poly', lambda x, y: x**2 + x * y + y**2),
37+
('distance', lambda x, y: lax.sqrt(x**2 + y**2)),
38+
('exp_product', lambda x, y: lax.exp(x) * lax.exp(y)),
39+
('log_sum_exp', lambda x, y: lax.log(lax.exp(x) + lax.exp(y))),
40+
]
41+
42+
43+
EDGE_CASE_PARAMS = [
44+
('zero', 0.0),
45+
('small_positive', 1e-6),
46+
('small_negative', -1e-6),
47+
('mid', 0.5),
48+
('moderate', 1.5),
49+
('large_positive', 5.0),
50+
('large_negative', -5.0),
51+
]

0 commit comments

Comments
 (0)