Search indexes

Read-side wrappers over the index datasets stored under a table’s SEARCH_INDEXES group. They are returned by Table.build_index, Table.search_indexes, and Column.search_indexes; the search indexes chapter explains the three families and the validity protocol.

class h5col.SearchIndex(dataset: Any, table: Table, column_ds: Any = None)

One search-index dataset of a table, wrapping its HDF5 dataset.

A wrapper obtained through a column (Column.search_indexes) is bound to that column; one obtained table-wide (Table.search_indexes) resolves its column by scanning SEARCH_INDEX_LIST attributes, the only linkage the spec defines. Binding matters on non-conformant files where two columns claim the same index: the bound wrapper still answers for its own column, while the scan refuses to pick one.

property name: str

The index dataset’s name (the final component of its HDF5 path).

property dataset: Any

The underlying h5py Dataset (for advanced/low-level access).

property kind: str | None

The index’s on-disk KIND, or None when absent or malformed.

property description: str | None

The index’s description attribute, or None when unset.

property source_generation: int | None

The SOURCE_GENERATION validity token, or None when absent.

property source_nrows: int | None

The SOURCE_NROWS validity token, or None when absent.

property is_valid: bool

The H5Col consumer validity check; False means “treat as absent”.

property column: Column | None

The column this index accelerates.

The bound column when the wrapper came from one; otherwise resolved by scanning SEARCH_INDEX_LIST attributes.

class h5col.ChunkMinMaxIndex(dataset: Any, table: Table, column_ds: Any = None)

Bases: SearchIndex

A CHUNK_MINMAX zone map: per-chunk min/max plus missing counts.

property n_chunks: int

Data-bearing chunks of the source column at the current NROWS.

property chunk_len: int

Rows per chunk of the source column.

entries() → ndarray

The index entries for the data-bearing chunks (tail residue clipped).

Raises:

SchemaError – If this is an unbound wrapper whose column cannot be resolved from any SEARCH_INDEX_LIST.

chunk_row_range(chunk_id: int) → tuple[int, int]

Row interval [start, stop) that chunk chunk_id covers.

Raises:

IndexError – If chunk_id is not a data-bearing chunk (outside [0, NROWS)).

prune(op: str, value: Any) → ndarray

Candidate chunk ids that may hold a non-missing match for the predicate.

The Layer-1 primitive: returns a superset of the chunks containing rows whose (non-missing) value satisfies op value — every returned chunk must still be read and verified, but no matching chunk is ever excluded. op is one of <  <=  >  >=  ==  between (between takes an inclusive (low, high) pair). Chunks with no orderable, non-missing element carry placeholder bounds and are never candidates; missing rows never match a value predicate (query them with Column.is_missing(), not with an index).

Raises:

• StaleIndexError – If the validity check fails.

• ConformanceError – If a token-valid index does not cover every data-bearing chunk (a rule-9 violation a consumer must not silently clip, because the uncovered chunks could hold matches).

• SchemaError – If op is unknown or value is not a valid query value for the column’s datatype.

class h5col.SortedRowsIndex(dataset: Any, table: Table, column_ds: Any = None)

Bases: SearchIndex

A SORTED_ROWS permutation: row positions in sorted-value order.

property nan_tail_length: int | None

Rows whose value is NaN, at the permutation’s very end.

property fill_tail_length: int | None

Missing (non-NaN fill) rows, immediately before the NaN tail.

property ordered: bool | None

The ordered flag (must be true for SORTED_ROWS).

permutation() → ndarray

The full permutation over [0, NROWS) (tail residue clipped).

rows(op: str, value: Any) → ndarray

Row positions whose (non-missing) value satisfies op value.

Exact, not a superset: binary search over the sorted body — reading O(log NROWS) individual column elements, never the full column — then one contiguous permutation slice. The rows come back in sorted-value rank order, not row order (sort them before a chunked read). op is one of <  <=  >  >=  ==  between (between takes an inclusive (low, high) pair). Missing rows and NaN rows live in the tails and never match (query them with Column.is_missing()).

Raises:

• StaleIndexError – If the validity check fails.

• ConformanceError – For structural violations a consumer must not paper over (undersized dataset, bad tail attributes, ordered not true).

• SchemaError – If op is unknown or value is not a valid query value for the column’s datatype.

class h5col.BitmapIndex(dataset: Any, table: Table, column_ds: Any = None)

Bases: SearchIndex

A BITMAP: per-value packed row bitmaps for equality predicates.

property values_dataset: Any

The accompanying values dataset, or None when unusable.

property ordered: bool | None

Whether the values enumeration order is semantically meaningful.

property exhaustive: bool

Whether the enumeration provably covers every non-missing value.

False when the attribute is false, absent, or malformed — exactly the cases where the spec forbids treating an enumeration miss as proof of absence.

values() → ndarray

The indexed values, in enumeration (bitmap row) order.

Raises:

ConformanceError – If the BITMAP has no usable values dataset.

rows(value: Any) → ndarray | None

Row positions equal to value, or None when the index cannot say.

Exact when it answers: the union of the bitmap rows whose indexed value equals value (row order, ascending). Returns an empty array when the value is missing from an exhaustive enumeration (provably zero rows) and None when it is missing from a partial one — the caller must fall back to a scan. On a categorical column, a str value is first encoded to its category code; an unknown label provably matches zero rows. Missing rows never match a value predicate, so a query equal to the column’s fill value returns an empty array.

Raises:

• StaleIndexError – If the validity check fails.

• ConformanceError – If the bitmap is not a 2-D uint8 dataset, is too narrow for NROWS, or has no usable values dataset.

• SchemaError – If value is not a valid query value for the column’s datatype.

isin(values: Any) → ndarray | None

Row positions equal to any of values (row order, ascending).

None when any value is missing from a non-exhaustive enumeration — the union cannot be proven complete and the caller must scan.

Raises:

StaleIndexError, ConformanceError, SchemaError – Whatever rows() raises for a value (it is called per value).