mortie.toc
The toc word — temporal order coverage (issue #175): one uint64 packing
either an exact nanosecond timestamp or a conservative time range, sortable
as a plain unsigned integer and closed under a semilattice merge. Times are
ns since 1850-01-01 on a continuous, leap-free, GPS-aligned scale; the
datetime64 / GPS converters are the only place leap seconds exist. Not an
IVOA T-MOC. These flat-array elementwise ops are the type's scalar surface
(the same relationship mortie.moc has to its ops over one cover);
the ragged many-cover plurals land in mortie.batch when the
interval-set algebra (issue #177) activates. The names stay flat on the
package (mortie.time2toc, ...).
toc word -- temporal order coverage (issue #175).
One uint64, a tagged union of an exact nanosecond timestamp and a
quantized, conservative time range, templated on what the morton word
does for space: self-describing, sortable as a plain unsigned integer, and
closed under a semilattice merge. All internal times are u64 nanoseconds
since 1850-01-01T00:00:00 on a continuous, leap-free, GPS-aligned
timescale; leap seconds exist only at the UTC boundary
(:func:from_datetime64 / :func:to_datetime64).
Layout (bit 0 = LSB)::
[start: 32 bits, 2^31 ns units][flag: 1 bit][low: 31 bits]
- timestamp (flag = 1): the word is
t_nswith the flag bit spliced in at position 31 -- monotone int_ns. - range (flag = 0):
lowis the end code, 31 bits at 2^32 ns; the encoded envelope is half-open[start * 2^31, end * 2^32), rounded outward with a strictly-greater end ceiling so it always properly contains the real interval.
Unsigned word order is order by conservative encoded start; within a tied
start quantum, ranges sort before timestamps, then timestamps by exact ns
and ranges shorter-first. The layout, epoch, merge results, and sort
order are normative (words persist on disk) and pinned by golden fixtures;
see the decision ledger on issue #175
<https://github.com/espg/mortie/issues/175> and the design record on
zagg#410 <https://github.com/englacial/zagg/issues/410>.
Naming note: "toc" pleasantly echoes tick/tock and T-MOC, but this is not an IVOA T-MOC and does not conform to the IVOA MOC 2.0 recommendation (different epoch, timescale, and cell model -- see the T-MOC research on zagg#410 for why the hierarchical cell was rejected).
These flat-array elementwise ops are the type's scalar surface, mirroring
the relationship :mod:mortie.moc has to its ops over one cover; the
ragged many-cover plurals land in :mod:mortie.batch when the interval-set
algebra (issue #177) activates.
Q_START_NS = 1 << 31
module-attribute
Start quantum: 2^31 ns (~2.15 s); a range's start code floors to this.
Q_END_NS = 1 << 32
module-attribute
End quantum: 2^32 ns (~4.29 s); a range's end code ceils to this.
TOC_MAX_NS = (1 << 63) - (1 << 32)
module-attribute
Exclusive ceiling on internal times: 2^63 - 2^32 ns past the epoch (~4 s short of year 2142); the end code must fit its 31-bit field.
GPS_EPOCH_NS = 47486 * 86400 * 10 ** 9
module-attribute
Internal ns of the GPS epoch 1980-01-06T00:00:00: 47,486
proleptic-Gregorian days of 86,400 s past 1850-01-01 (the leap-free
scale ticks exactly with GPS, so the constant is a plain day count --
validated against datetime.date arithmetic in the test suite).
time2toc(t_ns)
Encode exact instants (internal ns) as timestamp words.
The word is t_ns with a 1 flag bit spliced in at position 31, so
unsigned word order over timestamps is exactly the ns order.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t_ns
|
int or array - like
|
Instant(s) in nanoseconds since 1850-01-01T00:00:00 on the
continuous GPS-aligned scale, each in |
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
Timestamp word(s), |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any instant is negative, non-integer-typed, or at or beyond
|
See Also
span2toc : encode a real interval as a range word. toc2time : decode words back to conservative bounds.
Source code in mortie/toc.py
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | |
span2toc(start_ns, end_ns)
Encode real closed intervals [start, end] as range words.
Outward rounding with a strictly-greater end ceiling: the start code
floors to the 2^31 ns grid and the end code is (end >> 32) + 1 --
uniformly, including when end sits exactly on the 2^32 ns grid --
so the encoded envelope always properly contains the interval.
start_ns and end_ns broadcast against each other.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
start_ns
|
int or array - like
|
Interval start(s) in internal ns. |
required |
end_ns
|
int or array - like
|
Interval end(s) in internal ns (inclusive real endpoint), each
|
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
Range word(s), |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any start is after its end, or any end is at or beyond
|
See Also
time2toc : encode an exact instant. toc2time : decode words back to conservative bounds.
Source code in mortie/toc.py
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 | |
toc2time(words)
Decode toc words to conservative (start_ns, end_ns) bounds.
A timestamp yields its exact instant twice, (t, t). A range
yields its half-open envelope bounds: end_ns is exclusive,
strictly greater than every instant the range covers (the encoder's
strictly-greater ceiling guarantees this even for interval ends that
sat exactly on the 2^32 ns grid).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
words
|
int or array - like
|
Toc word(s) ( |
required |
Returns:
| Name | Type | Description |
|---|---|---|
start_ns |
int or ndarray
|
Conservative start(s) in internal ns, |
end_ns |
int or ndarray
|
Exact instant (timestamps) or exclusive envelope end (ranges),
same convention as |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
See Also
toc_is_range : which variant each word is.
Source code in mortie/toc.py
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | |
toc_merge(a, b)
Merge two toc words elementwise (the semilattice join).
Bitwise-equal inputs return that word unchanged -- merging two equal
timestamps must not produce their range envelope. Any other pair
merges conservative envelopes (min of start codes, max of end codes)
into a range word. Exactly associative, commutative, and idempotent,
so any fold tree over the same words produces the identical uint64.
a and b broadcast against each other.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
a
|
int or array - like
|
Toc word(s) ( |
required |
b
|
int or array - like
|
Toc word(s) ( |
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
Merged word(s), |
Raises:
| Type | Description |
|---|---|
ValueError
|
If either input is negative or non-integer-typed. |
See Also
toc_reduce : merge a whole array to one word.
Source code in mortie/toc.py
202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 | |
toc_reduce(words)
Merge an array of toc words down to one word.
The fold tree is unspecified (parallel under the hood) -- safe because
the merge is exactly associative, commutative, and idempotent, so
every fold tree produces the identical uint64.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
words
|
array - like
|
Toc words ( |
required |
Returns:
| Type | Description |
|---|---|
int
|
The merged word. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
See Also
toc_merge : the elementwise pairwise form.
Source code in mortie/toc.py
244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 | |
toc_is_range(words)
Test which variant each toc word is.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
words
|
int or array - like
|
Toc word(s) ( |
required |
Returns:
| Type | Description |
|---|---|
bool or ndarray
|
True where the word is a range (flag bit 31 = 0), False for a
timestamp; |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in mortie/toc.py
275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 | |
toc_overlaps(words, q_start_ns, q_end_ns)
Test which words intersect a half-open query window.
The test runs on each word's conservative encoded bounds (a
timestamp is its exact instant; a range is its outward-rounded
envelope), against the half-open window [q_start_ns, q_end_ns).
Consequently it may over-report near window edges by up to one
quantum (a range whose envelope grazes the window without its real
interval doing so), and it never under-reports: every word whose
real time content intersects the window tests True. An empty window
(q_start_ns == q_end_ns) matches nothing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
words
|
int or array - like
|
Toc word(s) ( |
required |
q_start_ns
|
int
|
Window start in internal ns (inclusive). |
required |
q_end_ns
|
int
|
Window end in internal ns (exclusive), |
required |
Returns:
| Type | Description |
|---|---|
bool or ndarray
|
True where the word's conservative bounds intersect the window;
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the window is inverted, or |
See Also
toc_contains : containment in the window instead of intersection.
Source code in mortie/toc.py
304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 | |
toc_contains(words, q_start_ns, q_end_ns)
Test which words are contained in a half-open query window.
The test runs on each word's conservative encoded bounds against
the half-open window [q_start_ns, q_end_ns). Consequently it
never over-reports (the real interval is inside the envelope, so
envelope-in-window implies interval-in-window), and it may
under-report near window edges by up to one quantum -- a range
whose real interval fits the window but whose outward-rounded
envelope spills past an edge tests False. An empty window
(q_start_ns == q_end_ns) contains nothing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
words
|
int or array - like
|
Toc word(s) ( |
required |
q_start_ns
|
int
|
Window start in internal ns (inclusive). |
required |
q_end_ns
|
int
|
Window end in internal ns (exclusive), |
required |
Returns:
| Type | Description |
|---|---|
bool or ndarray
|
True where the word's conservative bounds fit inside the window;
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the window is inverted, or |
See Also
toc_overlaps : intersection with the window instead of containment.
Source code in mortie/toc.py
344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 | |
from_datetime64(when)
Convert UTC datetime64 times to internal ns.
The internal scale is continuous and GPS-aligned: from 1972 the offset
to naive UTC day-count time is GPS - UTC = TAI - UTC - 19 from the
static in-module leap-second table (zero at the GPS epoch 1980-01-06,
+18 s since 2017-01-01). Before 1972 the proleptic convention is
zero offset (naive day-count seconds, no leap adjustment): it pins
the epoch identity (1850-01-01T00:00:00 -> 0 ns exactly), whereas
freezing the 1972 offset of -9 s would push the epoch itself to a
negative, unrepresentable internal time. Pre-1972 "UTC" was not
SI-second aligned anyway, and such data carries hours-level precision.
Cost: the mapping steps back 9 s across the 1972-01-01 boundary, so
the last 9 SI seconds of 1971 are not invertible (they alias early
1972); conversion is exact and invertible from 1972 on.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
when
|
datetime64, str, or array-like
|
UTC instant(s); anything |
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
Internal ns ( |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any instant is before the 1850 epoch or at or beyond
|
See Also
to_datetime64 : the inverse conversion. from_gps_ns : the leap-free GPS entry point.
Source code in mortie/toc.py
466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 | |
to_datetime64(t_ns)
Convert internal ns to UTC datetime64[ns] times.
The inverse of :func:from_datetime64 (same leap table, same pre-1972
zero-offset convention). Internal instants that fall inside an
inserted leap second render into the following UTC second --
datetime64 cannot express 23:59:60 -- so, e.g., the middle of the
2016-12-31 leap second renders as 2017-01-01T00:00:00.5. Roundtrip
to_datetime64(from_datetime64(t)) is exact for every datetime64
from 1972 on (no datetime64 names a leap-second instant).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t_ns
|
int or array - like
|
Internal ns ( |
required |
Returns:
| Type | Description |
|---|---|
datetime64 or ndarray
|
UTC instant(s) as |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any value is negative, non-integer-typed, or at or beyond
|
See Also
from_datetime64 : the inverse conversion. to_gps_ns : the leap-free GPS exit point.
Source code in mortie/toc.py
530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 | |
from_gps_ns(gps_ns)
Convert GPS time (ns since 1980-01-06T00:00:00) to internal ns.
A pure constant offset: both scales are continuous and leap-free and
tick together, so the conversion is gps_ns + GPS_EPOCH_NS. The
ICESat-2 ingest path (delta_time + atlas_sdp_gps_epoch -> GPS
ns) composes with this directly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gps_ns
|
int or array - like
|
GPS time(s) in ns since the GPS epoch, each below
|
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
Internal ns ( |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any value is negative, non-integer-typed, or maps at or beyond
|
See Also
to_gps_ns : the inverse conversion.
Source code in mortie/toc.py
575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 | |
to_gps_ns(t_ns)
Convert internal ns to GPS time (ns since 1980-01-06T00:00:00).
The exact inverse of :func:from_gps_ns. Internal times before the
GPS epoch have no non-negative GPS representation and are rejected.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t_ns
|
int or array - like
|
Internal ns ( |
required |
Returns:
| Type | Description |
|---|---|
int or ndarray
|
GPS ns ( |
Raises:
| Type | Description |
|---|---|
ValueError
|
If any value is negative, non-integer-typed, or before the GPS epoch. |
See Also
from_gps_ns : the inverse conversion.
Source code in mortie/toc.py
616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 | |