mortie.batch
Batch kernels over morton sets, MOCs and geometry columns: one call carries a
whole ragged column across the Python/Rust boundary, and element i of the
result is bit-identical to the single-item form applied to element i alone.
Consolidated by arity (issue #170) — a different axis from the domain
split the rest of the package is organised on.
Since issue #187 the public surface is polymorphic — one function per
operation, the batch form selected by input shape or a keyword-only
offsets= — so the plural batch names this module used to export
(mocs_to_orders, mocs_and, mocs_intersect, common_ancestors,
children_of, from_wkbs) are retired, and their kernels live on here as
private functions behind the surviving entry points (mortie.moc_to_order,
mortie.moc_and, mortie.moc_intersects, mortie.common_ancestor,
mortie.generate_morton_children, mortie.from_wkb). The one public name
left is the batch-native coverer below, whose ragged signature has no scalar
shape to collapse into; the pyarrow skins (mortie.arrow.from_wkb,
mortie.arrow.polygons_to_morton_mocs) stay in mortie.arrow.
Batch kernels over morton sets, MOCs and geometry columns.
Every function here carries a whole ragged column across the Python/Rust
boundary in one call, releases the GIL, and lets Rust parallelize across the
elements, so the per-call fixed cost that dominates a Python loop over half a
million footprints is paid once. Element i of the result is bit-identical
to the single-item form applied to element i alone -- the batch is a
throughput surface, not a second semantics.
Since issue #187 the public surface is polymorphic -- one function per
operation, with the batch form selected by the input shape or a keyword-only
offsets= -- and the plural batch names this module used to export
(mocs_to_orders, mocs_and, mocs_intersect, common_ancestors,
children_of, from_wkbs) are retired. The kernels live on here as
private functions, each reached only through its surviving public entry point
(named in its docstring). The one public name left is
:func:polygons_to_morton_mocs, whose ragged batch-native signature has no
scalar shape to collapse into (issue #187, ruled): it is the entry point,
and the scalar morton_coverage_moc retired with the plurals.
The Rust kernels stay split by domain (coverage/batch.rs, moc/batch.rs,
decimal_morton/batch.rs, wkb/batch.rs), so this module deliberately does
not mirror the Rust tree the way :mod:mortie.orders and
:mod:mortie.convert do: the Python surface is organised for callers and the
Rust for kernels, and the two need not be 1:1.
The pyarrow skin is a third axis: :func:mortie.arrow.from_wkb and
:func:mortie.arrow.polygons_to_morton_mocs take pyarrow columns and stay in
:mod:mortie.arrow (issue #154).
polygons_to_morton_mocs(lats, lons, offsets, order=18, tolerance=None, max_cells=None, normalize=True, *, latitude='authalic')
Compute MOC coverage of many independent polygons in one call.
The ragged polygon set crosses the Python/Rust boundary once (issue
153), the GIL is released for the whole batch, and Rust parallelizes
across polygons — so the per-call fixed cost that dominates a Python loop
over half a million footprints is paid once. Identity-preserving: result
i is exactly the cover of input polygon i (unlike the multipart
ring-set form :func:mortie.from_geometry covers with moc=True,
which unions its rings into one cover). The plural MOCs in the name
marks that many→many contract — one MOC per input polygon — against the
many→one union of the multipart form.
Batch native (issue #187, ruled): the ragged offsets signature has
no scalar shape to collapse into, so this plural survives the polymorphic
consolidation as the operation's only entry point — the scalar
morton_coverage_moc retired with the plural names. A single
polygon's MOC is values[:off[1]] of a one-group call, or reach it as
:func:mortie.from_wkb with moc=True (backend-free, WKB bytes in) or
:func:mortie.from_geometry with moc=True (needs a geometry backend).
Polygons are covered in chunks and each chunk is copied into the ragged
output as it lands, so the per-polygon covers never all coexist — not the
~2.5x of holding every one of them to concatenate at the end. Peak is then
the input copy + the result + one chunk: the binding copies lats,
lons and offsets before releasing the GIL (a borrowed numpy slice
cannot cross allow_threads), so the vertex arrays are a full second
resident copy for the duration, and nothing short of giving up the GIL
release removes it. That is a floor, not the whole cost: passing
anything this wrapper has to convert — a list, float32, a
non-contiguous slice — adds the conversion's own copy on top, so hand it
contiguous float64/int64 arrays to pay the copy only once.
Measured on a cold call over synthetic ~1 degree footprints at order 8,
sampling /proc/self/statm while the call runs
(benchmarks/measure_batch_coverage.py --mem, which re-runs this table):
100k footprints is
6.9 MiB of input and a 12.9 MiB result behind a 21.9 MiB peak, and 555,867
(the ATL03 catalog scale) is 38.2 MiB of input and a 71.6 MiB result behind
a 112.0 MiB peak — 1.70x and 1.56x the result alone, but 1.11x and
1.02x of input + result. Those ratios stay near 1 because covering has
no coarsen direction that can shrink the result to nothing;
:func:_mocs_to_orders does, and reaches 60x there. Near 1 is not
negligible, though — size a worker off input + result, not the result
alone.
Input and output are ragged arrays in arrow list layout: polygon i is
lats[offsets[i]:offsets[i+1]] / lons[offsets[i]:offsets[i+1]], and
its MOC is values[out_offsets[i]:out_offsets[i+1]] in the result —
byte-identical to the retired scalar morton_coverage_moc on that ring
alone (the identity every pre-retirement parity test pinned).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lats
|
array_like
|
Flat |
required |
lons
|
array_like
|
Flat |
required |
offsets
|
array_like
|
|
required |
order
|
int
|
Finest HEALPix order (1-29), shared by every polygon. Default 18. |
18
|
tolerance
|
float
|
Stop refining a boundary cell once its angular radius (in degrees)
drops to this value, applied as a single shared setting to every
polygon in the batch. Mutually exclusive with |
None
|
max_cells
|
int
|
Best-first cell budget per polygon, shared by every polygon. A budget below some polygon's representable floor is raised for that polygon (soft target) and one summary warning is emitted. |
None
|
normalize
|
bool
|
Ring-orientation handling, identical in meaning to
:func: |
True
|
latitude
|
str
|
Latitude convention of the input vertices, shared by every polygon;
see :func: |
'authalic'
|
Returns:
| Name | Type | Description |
|---|---|---|
values |
ndarray
|
All polygons' morton MOC words concatenated ( |
out_offsets |
ndarray
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
Fail-fast, naming the lowest-index offending polygon (e.g.
|
Warns:
| Type | Description |
|---|---|
UserWarning
|
If |
See Also
mortie.from_geometry : one geometry (multipart rings unioned) to one
cover, moc=True for the compact form.
Examples:
>>> import mortie, numpy as np
>>> lats = np.array([40.0, 50.0, 45.0, 10.0, 20.0, 15.0])
>>> lons = np.array([-120.0, -120.0, -110.0, -80.0, -80.0, -70.0])
>>> values, off = mortie.polygons_to_morton_mocs(lats, lons, [0, 3, 6], order=6)
>>> first = values[off[0]:off[1]] # MOC of the first triangle
Source code in mortie/batch.py
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 | |