Drilling Telemetry Simulator

Drilling Telemetry Simulator is an open-source, field-anchored, deterministic physics-based generator of synthetic drilling data. A normal run simulates a complete rig operation from surface, through BHA pickup and physical trip-in, drilling, connections, work-pipe and surveys, then trip-out, BHA laydown, and a validated terminal surface state. The design rule is simple: simulate the rig first; derive telemetry from physical state.

Engineering case study: Building a Physics-Based Synthetic Drilling Telemetry Simulator

Capabilities

• Complete surface-to-surface operations with rate-limited rig machinery.
• One deterministic sampled physical pipe tally, exact stand grouping, connection geometry, final partial footage, and terminal inventory closure.
• Rotary and positive-displacement-motor (PDM) drilling.
• Connected hydraulics, finite axial and rotary response, bit-rock interaction, and reduced-order soft-string torque and drag.
• User-authored imported trajectory stations or generated directional shapes, minimum-curvature geometry, PDM rotate/slide response, and a physical stop-before-slide gate.
• A compact synthetic build-and-hold motor operation that builds to about 20°, drills a tangent/hold interval, surveys, and closes surface-to-surface.
• Static surveys supported in slips, ideal survey measurement, and modeled EM or mud-pulse transport.
• Stable public EDR-style CSV/Parquet output, surveys, events, physical pipe tally, privileged ground truth, and run provenance.
• Optional static WITSML 1.4.1.1 interchange files.
• Bundled generic operations, developer fixtures, and attributed Utah FORGE field-anchored reference cases.
• Reproducible seeded machinery and sensor randomness.

What this is not

This project is not a digital twin, historical rig-operation reconstruction, commercial well-planning/anti-collision system, well-control system, or safety-critical engineering tool. It is not a substitute for field procedures or commercial drilling software. Its PDM, bit-rock, hydraulics, BHA handling, work-pipe, sensor, steering, and soft-string models are intentionally reduced order. It does not implement full CFD/FEA, torsional-wave propagation, a full stick-slip model, detailed surge/swab or hole cleaning, or RSS steering. See Limitations.

Requirements

• Python 3.10 or newer.
• NumPy 2.1.x, pandas 2.2.x, and PyArrow 17–19 (installed automatically).
• No operating-system-specific runtime service is required.

The package is not presented here as a PyPI release. Install it from a source checkout or source archive.

Installation

From the repository root, create an isolated environment.

Linux/macOS:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

Windows PowerShell:

py -3.10 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install .

For development, install the checkout and test dependency in editable mode:

python -m pip install -e ".[test]"

Quick start

List the installed scenarios, then run a compact normal operation:

rig-telemetry-sim examples list
rig-telemetry-sim examples run precision_st1200_rig_first_example.json \
  --output ./output/vertical-demo

Run the canonical normal PDM directional operation with:

rig-telemetry-sim examples run build_and_hold_motor_full_operation.json \
  --output ./output/build-hold-demo

The output directory must not already exist. A completed default export contains:

output/vertical-demo/
├── events.json
├── ground_truth.json
├── pipe_tally.csv
├── run_metadata.json
├── surface_telemetry.csv
├── surface_telemetry.parquet
└── surveys.csv

Use --no-parquet to omit Parquet, or --witsml-1411 to add a witsml_1411/ static interchange package.

Output files

File	Purpose
surface_telemetry.csv	Stable field_surface_v1 field-facing projection at the configured EDR cadence.
surface_telemetry.parquet	The same logical surface schema in Parquet; written unless --no-parquet is used.
pipe_tally.csv	Authoritative immutable physical joint inventory and exact stand grouping used by the operation.
surveys.csv	Stable survey schema containing acquired and successfully received/decoded surveys. A zero-survey operation retains the header and has zero data rows.
events.json	Privileged rig-procedure event timeline, including physical interval and inventory associations.
ground_truth.json	Privileged simulator truth: final physical/rig state, planned and actual trajectories, surveys, and transport outcomes.
run_metadata.json	Schema/version, cadence, models, scenario hash, seed, provenance, licences, file roles, and pipe-tally hash.
witsml_1411/	Optional static WITSML 1.4.1.1 well, wellbore, log, tubular, BHA-run, trajectory, and manifest documents.

Default export refuses an incomplete result. The Python API can write an explicitly marked partial export with allow_partial=True.

Public surface columns

Column	Unit	Meaning
timestamp	RFC 3339 UTC	Synthetic sample time.
activity, activity_phase	enum	Neutral simulator procedure annotation; not a vendor rig-state classifier.
terminal	boolean	True only on the explicit terminal sample.
completion_status	enum	in_progress or complete.
on_bottom	boolean	Physical bit contact with current hole bottom.
observation_quality	enum	ideal unless a programmatic sensor model produced the row.
hole_depth_m, bit_depth_m	m MD	Current drilled depth and physical bit depth.
block_position_m	m	Travelling-block/hook position above the rig-floor datum.
slips_set	boolean	Whether slips are physically engaged.
rop_m_per_hr	m/h	Net hole-depth advance rate.
wob_n, hookload_n	N	Weight on bit and public load-cell-facing hookload.
surface_rpm	rpm	Surface rotary/top-drive speed, not bit RPM.
surface_torque_nm	N m	Signed surface-drive torque.
pump_spm	SPM	Aggregate actual pump speed.
flow_rate_m3_per_min	m³/min	Actual flow into the connected circulation path.
spp_kpa, difp_kpa	kPa	Standpipe pressure and public differential pressure.

Exact units, classifications, zero tolerances, coordinate conventions, and file roles are also embedded in run_metadata.json.

Public versus privileged data

The public surface files are synthetic field-facing observations. They omit latent bit-side power, exact trajectory truth, string forces, model-limit diagnostics, and other target-leaking values. ground_truth.json and events.json intentionally expose simulator-internal truth for verification, analysis, and supervised-learning labels. Those privileged quantities are not claims about what a real EDR could observe.

Hookload semantics

Public hookload_n is the resolved string hookload plus any configured rig-side suspended assembly below the modeled load-cell boundary. The string equilibrium remains separate in privileged truth. A nonzero public hookload at surface can therefore be correct. Gross hook capacity applies to the complete load carried through that boundary; slips capacity applies to its own lower string load path.

DIFP semantics

Public differential pressure is:

current SPP - frozen settled off-bottom SPP zero

The zero is captured only under physically eligible connected, circulating, stable, off-bottom, unloaded conditions. It is not activity-gated or flow-normalized. Small positive or negative pump/flow-driven variation around zero is intentional. Rotary-only operation normally has no motor-scale DIFP; PDM loading produces a substantial positive response.

Running bundled examples

examples list separates normal operations from explicit developer fixtures:

rig-telemetry-sim examples list

Normal examples begin at surface even when the existing hole is deep. Useful starting points are:

• build_and_hold_motor_full_operation.json — canonical generic synthetic PDM build-and-hold: physical trip-in, rotate/slide steering, six EM surveys, work-pipe/connections, tangent drilling, trip-out, and laydown.
• precision_st1200_rig_first_example.json — generic 60 m vertical rotary operation.
• substantial_trip_rig_first_example.json — deep trip plus a short rotary interval.
• substantial_trip_100m_drilling_example.json — 100 m, multiple connections, work-pipe, and surveys.
• forge_78b_32_full_operation_reference.json — field-anchored PDM operation surrounded by explicitly synthetic handling and trip activity.

The names under developer_prepositioned_fixtures deliberately begin near a downhole interval. They exist for calibration closure, focused regression, and benchmarking. Running one requires the explicit --allow-prepositioned-fixture flag and does not produce a normal generated surface-to-surface history.

Creating a scenario

A scenario is a strict JSON object with "schema_version": 2. Unknown keys are rejected, as are invalid units, enum values, geometry, formation coverage, equipment combinations, pipe tallies, and non-1-second normal physics cadence.

The shortest reliable authoring route is to copy a bundled normal example, give it new run identifiers, and change the run, plan, procedure, formation, and model settings deliberately:

rig-telemetry-sim generate ./my-scenario.json --output ./output/my-run

Two complete tested scenarios and the full field reference are in SCENARIO_FORMAT.md.

Scenario anatomy

Block	Purpose
schema_version, name, seed	Input contract, run label, and deterministic randomness identity.
simulation	One-second physics cadence, output cadence, step guard, and descriptive model label.
well_plan	Imported MD/inc/azi stations or a deterministic generated shape.
run	Identifiers, synthetic clock start, existing-hole MD, TD, reference ROP, and nominal joint-length distribution centre.
rig	Registered rig and procedure profiles plus full-operation or developer-fixture initialization.
pipe_joints	Optional user-supplied authoritative physical tally. If absent, one is generated once.
schedule	Procedure timings and pump, rotary, set-down, and drilling-mode requests.
hydraulics, fluid, axial, rotary, bit	Reduced-order physical configuration; omitted values use typed defaults.
formation	Ordered intervals covering the complete drilling interval.
directional, drivetrain, torque_drag	Steering/drive selection and directional mechanics.
trajectory_record_interval_m	Spacing at which newly drilled actual-trajectory stations are retained.
telemetry	none, em, or mud_pulse_qpsk survey transport.

All public dimensional values use SI units except field-conventional names that state otherwise, such as RPM, SPM, degrees, m/h, and DLS in degrees/30 m.

Existing hole and initial bit depth

run.start_md_m describes existing hole. It does not predeploy pipe. A normal full_operation starts with bit and pipe at surface, picks up the resolved BHA, and trips causally to the interval. Calibration changes effective parameters, not deployment state.

Physical pipe tally

When pipe_joints is omitted, scenario composition samples one bounded normal tally from seed and run.pipe_length_m. Those exact physical joints are grouped into stands according to the rig profile and determine trip geometry, connection spacing, deployed/connected accounting, and final inventory. pipe_tally.csv is a projection of that same immutable tally, not a reconstruction from events.

A user-supplied tally is supported. It must be nonempty, ordered from joint 1, have unique nonempty IDs and positive finite lengths within 75–125% of run.pipe_length_m, be long enough for TD and handling margin, and form stands within rig block travel. A survey request in a multi-joint stand may attach only to the stand's final joint. See SCENARIO_FORMAT.md for the exact row schema.

Rotary and PDM modes

• schedule.drilling_mode: "ROTARY" uses the surface drive; set directional.enabled false for a vertical rotary operation.
• "MOTOR_ROTATE" and "SLIDE" require drivetrain.pdm_enabled: true (explicitly or through the directional PDM default).
• directional.drive: "pdm" uses the PDM steering path.
• directional.drive: "surface_rotary" cannot accept PDM steering/controller settings.
• "RSS" is rejected because RSS mechanics are not implemented.

The PDM follows actual motor-through flow and circulation topology, not pump command labels. Surface and downhole consumers share finite torque, power, and hydraulic budgets.

Directional drilling and surveys

Version 0.1 provides a user-authorable planned directional trajectory interface: explicit MD/inclination/azimuth stations or one of the supported deterministic shapes. It is not a commercial well-plan interchange, target optimizer, anti-collision, or geosteering system. Plans are objectives and assumptions; actual trajectory is generated from physical drilled response using minimum curvature. Existing-hole directional work requires the explicit assume_existing_hole_follows_plan: true opt-in.

The rule-based controller selects motor-rotate or oriented slide from plan-versus-actual error and can apply a lower exit threshold for deterministic mode hysteresis. A slide request first commands the surface drive to stop; residual stator rotation remains in the bit-speed/power balance, but no slide penetration or steering is permitted while actual rotation exceeds the rig's stopped tolerance. During legal slide penetration, actual hydraulic flow continues to govern PDM bit speed, differential pressure, torque, and power. The bundled build-and-hold operation demonstrates initial rotary footage, a physical rotate-to-slide transition, inclination build, return to motor-rotate, and tangent/hold drilling.

A static survey is physically prepared by stopping the block, setting slips, and transferring support. Only then do the survey/settle, survey/tool_window, and survey/acquire activities begin. EM and ideal transport are available. Mud-pulse transmission requires the downhole BHA, connected circulation, adequate actual flow, and sufficient SPP throughout the frame; incompatible channel loss aborts the frame. The ideal survey is bit-colocated and does not model a real sensor offset.

Read DIRECTIONAL_MWD_MODELS.md and RIG_PROCEDURE_MODELS.md for model and lifecycle details.

Determinism

The same resolved scenario, seed, package build, and settings produce the same physical pipe tally and named random streams. Sensor channels own independent deterministic streams, so adding or reordering an unrelated channel does not change another channel's noise. Generated inventory and operations are synthetic even when deterministic; determinism is not evidence of field measurement.

Python API

The ordinary root API is load_scenario → run_scenario → terminal result → export_run_result:

from rig_telemetry_sim import export_run_result, load_scenario, run_scenario

scenario = load_scenario("my-scenario.json")
result = run_scenario(scenario).run_until_terminal()
paths = export_run_result(result, "output/python-run")
print(paths["surface_csv"])

The package root also exposes typed configuration, immutable result/snapshot types, projections, and deliberate model-extension protocols. Specialist-module helpers for prepositioned fixtures and calibration are advanced developer interfaces; they are not the normal generation path.

The ordinary root API is intentionally small in concept:

Public names	Role
load_scenario, scenario_from_mapping, Scenario	Strict JSON/path or mapping composition.
run_scenario, SimulationCoordinator, SimulationResult	Stepwise or terminal execution and immutable result snapshot.
export_run_result	Atomic terminal export; optional partial/diagnostic profiles are explicit keyword choices.
field_surface_records, surface_records, survey_records	In-memory public or diagnostic projections.
SCENARIO_SCHEMA_VERSION, OUTPUT_SCHEMA_VERSION, FIELD_SURFACE_PROFILE, FIELD_SURVEY_PROFILE, CANONICAL_SI_PROFILE	Stable schema/profile identities.
SimulationSettings, OperationalScheduleConfig, HydraulicConfig, FluidConfig, AxialMechanicsConfig, RotaryDriveConfig, BitConfig, DrillingMode	Typed public configuration.
immutable run/physical/rig/survey/event snapshot types	Read-only simulation truth returned to callers.

Root-exported trajectory, T&D, steering, survey, telemetry-transport, and sensor protocols/configurations are advanced model-extension seams. Custom surface sensors are programmatic only: they must preserve the wrapper timestamp, required fields/types/ranges, and stable export schema. The CLI does not provide a plugin loader or arbitrary model-import option.

Field-anchored references and calibration

List or run the five installed public FORGE reference cases:

rig-telemetry-sim references list
rig-telemetry-sim references validate --reference forge_78b_32_motor_vertical \
  --output ./output/forge78-reference.json

Despite the compatibility command name, references validate reports reference conformance and in-sample calibration closure, not independent held-out validation. The measured strip anchors selected effective parameters; unobserved trip, connection, timing, and surrounding formation behavior are synthetic. The operation is not a reconstruction of historical rig activity.

Version 0.1 does not expose a supported general-purpose automatic calibration workflow for arbitrary user drilling data. Low-level scalar calibration functions and an illustrative synthetic developer tool exist, but users must supply the simulation callback, parameter ownership, units, provenance, bounds, and interpretation. They are not an end-to-end ingestion or fitting product. Calibration ledgers and sources are described in REFERENCES.md.

CLI reference

Use generated help as the command authority:

rig-telemetry-sim --help
rig-telemetry-sim generate --help
rig-telemetry-sim examples --help
rig-telemetry-sim references --help

Command	Supported purpose
generate SCENARIO	Run an external JSON scenario and export a complete operation.
examples list	List normal bundled operations separately from developer fixtures.
examples run NAME	Run an installed example.
references list	List compact field-reference cases.
references validate	Run full-operation reference conformance/calibration closure; --quick is an incomplete smoke.

The --fast-prepositioned-fixture reference option and --allow-prepositioned-fixture generation option are developer-only.

Tests and development

Install .[test], then run the default tier:

python -m pytest -q -m "not release"

Before a release or changes to physical behavior, references, packaging, or long lifecycle logic, also run:

python -m pytest -q -m release
python -m compileall -q src tests
python -m build --outdir dist
python tools/validate_distribution_artifacts.py dist/*.whl dist/*.tar.gz

See CONTRIBUTING.md for the concise contributor workflow.

Benchmarking

Benchmarks characterize a machine and revision; they are not operational forecasts. The supported checkout commands are:

python benchmarks/benchmark_physics.py --case short
python benchmarks/benchmark_physics.py --case daily --profile
python benchmarks/benchmark_directional_mwd.py
python tools/benchmark_field_references.py --output benchmark-results.json

On one WSL2 development host after the geometry-cache optimization, normal representative runs ranged roughly from 1,800 to 2,200 physics steps/s, while a deep high-angle 5,000-step fixture ran near 900 steps/s. Hardware, scenario, retained history, profiling, and export materially affect results. A full surface-to-surface FORGE 16A terminal benchmark remains intentionally unreported. Details and interpretation are in benchmarks/README.md.

Limitations

• Reduced-order quasi-steady connected hydraulics and PDM; no full fluid-column waves, CFD, surge/swab, or detailed motor curve/stall dynamics.
• Reduced-order soft-string torque and drag; no stiff-string buckling or finite- element contact mechanics.
• Rigid/reduced rotary response; no distributed torsional waves or full stick-slip dynamics.
• Reduced-order bit-rock, hole-cleaning, work-pipe, connection, and BHA-handling behavior.
• Ideal bit-colocated survey model and limited sensor-error models.
• Rule-based reduced-order rotate/slide steering; no RSS, anti-collision, geosteering, proprietary well-plan ingestion, or directional BHA FEA.
• Static WITSML 1.4.1.1 files only; no server, ETP, or vendor-native stream.
• Field-reference strips are in-sample anchors, while geology and operations outside those anchors are synthetic assumptions/proxies.
• No field-safety, planning, equipment-selection, or operational-validity claim.

Licensing and references

Project source code is licensed under Apache License 2.0; see LICENSE. Identified bundled compact data adapted from public Utah FORGE records remains under Creative Commons Attribution 4.0 International. Attribution, adaptation statements, and no-endorsement language are in THIRD_PARTY_DATA.md.

Engineering sources and provenance classifications are in REFERENCES.md. Public technical guides are:

• ARCHITECTURE.md
• SCENARIO_FORMAT.md
• DIRECTIONAL_MWD_MODELS.md
• RIG_PROCEDURE_MODELS.md
• benchmarks/README.md
• CHANGELOG.md