Skip to content

Commit 6615f30

Browse files
committed
docs: document the opt-in precompiled mode
Assisted-by: ClaudeCode:claude-fable-5
1 parent 1d3436a commit 6615f30

3 files changed

Lines changed: 120 additions & 2 deletions

File tree

docs/compiling.rst

Lines changed: 94 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -348,7 +348,8 @@ function with the following signature:
348348
.. code-block:: cmake
349349
350350
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
351-
[NO_EXTRAS] [THIN_LTO] [OPT_SIZE] source1 [source2 ...])
351+
[NO_EXTRAS] [THIN_LTO] [OPT_SIZE] [PRECOMPILE | NO_PRECOMPILE]
352+
source1 [source2 ...])
352353
353354
This function behaves very much like CMake's builtin ``add_library`` (in fact,
354355
it's a wrapper function around that command). It will add a library target
@@ -404,6 +405,98 @@ optimizations remain disabled.
404405

405406
.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
406407

408+
.. _precompile-mode:
409+
410+
Pre-compiling part of pybind11
411+
------------------------------
412+
413+
pybind11 is header-only by default: every translation unit compiles its own
414+
copy of the non-template implementation. The opt-in *precompiled* mode
415+
compiles that implementation once, into a static library built inside your
416+
own project with your own flags. This reduces the build time, most of all for
417+
projects with many translation units or many modules in one build.
418+
419+
.. code-block:: cmake
420+
421+
pybind11_add_module(example PRECOMPILE example.cpp)
422+
423+
The first ``PRECOMPILE`` target creates the library target
424+
``pybind11::precompiled``; further targets reuse it. Set the CMake variable
425+
``PYBIND11_PRECOMPILE`` to make it the default for all
426+
``pybind11_add_module`` calls; use ``NO_PRECOMPILE`` on a target to opt back
427+
out. For targets you create yourself, call the ``pybind11_precompile()``
428+
function and link ``pybind11::precompiled`` PRIVATE; the target carries the
429+
required ``PYBIND11_PRECOMPILED`` compile definition PUBLIC, so your sources
430+
also get it.
431+
432+
Requirements and caveats:
433+
434+
* The library and every module linking it must agree on the configuration
435+
macros ``PYBIND11_INTERNALS_VERSION``, ``Py_GIL_DISABLED``,
436+
``PYBIND11_SIMPLE_GIL_MANAGEMENT``,
437+
``PYBIND11_DETAILED_ERROR_MESSAGES`` (defaults on in debug builds),
438+
``PYBIND11_HAS_SUBINTERPRETER_SUPPORT``, and
439+
``PYBIND11_BACKWARD_COMPATIBILITY_TP_DICTOFFSET``. A mismatch produces one
440+
readable undefined symbol at link time referencing
441+
``pybind11_precompiled_config``.
442+
* Configuration macros that only change code inside the library (for example
443+
``PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING``) must be defined when the
444+
library is compiled; a definition only on your module has no effect.
445+
* The library picks up your directory-level flags and C++ standard when it is
446+
first created, so set those before the first ``PRECOMPILE`` target. A
447+
status message reports the directory that created the library.
448+
* The library is not compiled with link-time optimization, and the per-target
449+
``THIN_LTO`` and ``OPT_SIZE`` options of ``pybind11_add_module`` do not
450+
apply to it. To change this, call ``pybind11_precompile()`` yourself and
451+
set the properties on the created target, ``pybind11_precompiled`` (the
452+
real target behind the ``pybind11::precompiled`` alias; CMake does not let
453+
you set properties through an alias):
454+
455+
.. code-block:: cmake
456+
457+
pybind11_precompile()
458+
set_target_properties(pybind11_precompiled PROPERTIES
459+
INTERPROCEDURAL_OPTIMIZATION ON)
460+
461+
* The library is static and per-build-tree; it is never installed or shared
462+
between projects. Each extension module links its own copy, which keeps
463+
pybind11's per-module state the same as in header-only mode.
464+
* Not available with ``PYBIND11_NOPYTHON`` (the library needs Python
465+
headers).
466+
467+
For build systems other than CMake, the same sources ship with the pybind11
468+
package: compile ``pybind11_combined.cpp`` from the directory reported by
469+
``python -m pybind11 --srcdir`` (also available as
470+
``pybind11.get_source_dir()`` and the ``srcdir`` pkg-config variable) into
471+
a static library or into your extension, and define
472+
``PYBIND11_PRECOMPILED`` for every translation unit.
473+
474+
With Meson, build the library once per build tree and link it into each
475+
extension module, the same as the CMake path:
476+
477+
.. code-block:: meson
478+
479+
pybind11_dep = dependency('pybind11')
480+
pybind11_src = run_command(py, ['-m', 'pybind11', '--srcdir'],
481+
check : true).stdout().strip()
482+
483+
pybind11_precompiled = static_library('pybind11_precompiled',
484+
pybind11_src / 'pybind11_combined.cpp',
485+
cpp_args : ['-DPYBIND11_PRECOMPILED'],
486+
gnu_symbol_visibility : 'hidden',
487+
dependencies : [pybind11_dep, py.dependency()])
488+
489+
py.extension_module('example', 'example.cpp',
490+
cpp_args : ['-DPYBIND11_PRECOMPILED'],
491+
link_with : pybind11_precompiled,
492+
dependencies : [pybind11_dep])
493+
494+
The configuration-macro rules above apply here too: the static library and
495+
every module that links it must be compiled with the same configuration
496+
macros, and ``-DPYBIND11_PRECOMPILED`` must appear in both ``cpp_args``
497+
lists. (``pybind11_dep.get_variable('srcdir')`` also reports the source
498+
directory when Meson finds pybind11 through pkg-config.)
499+
407500
Configuration variables
408501
-----------------------
409502

docs/faq.rst

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,12 @@ and the binding code
7979
How can I reduce the build time?
8080
================================
8181

82-
It's good practice to split binding code over multiple files, as in the
82+
First, consider the opt-in precompiled mode: it compiles the non-template
83+
part of pybind11 once per project instead of once for each translation unit.
84+
In CMake, this is one keyword on ``pybind11_add_module``. See
85+
:ref:`precompile-mode`.
86+
87+
It's also good practice to split binding code over multiple files, as in the
8388
following example:
8489

8590
:file:`example.cpp`:

tools/pybind11Config.cmake.in

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ This module sets the following variables in your project:
1818
Directories where pybind11 and python headers are located.
1919
``pybind11_INCLUDE_DIR``
2020
Directory where pybind11 headers are located.
21+
``pybind11_SRC_DIR``
22+
Directory where the library sources for the opt-in precompiled mode are
23+
located (used by ``pybind11_precompile``).
2124
``pybind11_DEFINITIONS``
2225
Definitions necessary to use pybind11, namely USING_pybind11.
2326
``pybind11_LIBRARIES``
@@ -147,6 +150,7 @@ This module defines the following commands to assist with creating Python module
147150
pybind11_add_module(<target>
148151
[STATIC|SHARED|MODULE]
149152
[THIN_LTO] [OPT_SIZE] [NO_EXTRAS] [WITHOUT_SOABI]
153+
[PRECOMPILE|NO_PRECOMPILE]
150154
<files>...
151155
)
152156
@@ -162,6 +166,22 @@ default is ``MODULE``. There are several options:
162166
Disable the SOABI component (``PYBIND11_FINDPYTHON`` mode only).
163167
``NO_EXTRAS``
164168
Disable all extras, exit immediately after making the module.
169+
``PRECOMPILE``
170+
Link the target against the ``pybind11::precompiled`` static library
171+
(created on first use); ``NO_PRECOMPILE`` opts a target out when the
172+
``PYBIND11_PRECOMPILE`` variable enables it globally.
173+
174+
pybind11_precompile
175+
^^^^^^^^^^^^^^^^^^^
176+
177+
.. code-block:: cmake
178+
179+
pybind11_precompile()
180+
181+
Create the ``pybind11::precompiled`` static library from the shipped sources
182+
(once per build tree). ``pybind11_add_module(... PRECOMPILE)`` calls this for
183+
you; call it directly to link ``pybind11::precompiled`` into your own
184+
targets.
165185
166186
pybind11_strip
167187
^^^^^^^^^^^^^^

0 commit comments

Comments
 (0)