radiens_drive_catalog.catalog

catalog.py

The main interface for radiens-drive-catalog. The Catalog class wraps Drive scanning, local caching, and downloading behind a simple API.

Typical usage

from radiens_drive_catalog import Catalog, Config

config = Config.from_file("config.json") catalog = Catalog(config)

Build or refresh the catalog from Drive

catalog.scan()

Query using convenience methods or pandas directly

df = catalog.list_recordings(drive_path="2026-02-15_batch", drive_path_mode="prefix") items = catalog.list_items(is_folder=True)

Get a local path, downloading if needed

path = catalog.get_recording_path("2026-02-15_batch/reaching", "rat01_2026-02-14_probe1") path = catalog.get_item_path("2026-02-15_batch/reaching", "logs")

Unique-basename lookup

rec = catalog.get_recording("rat01_2026-02-14_probe1") path = catalog.get_recording_path(rec["drive_path"], rec["base_name"])

Download explicitly

catalog.download_recording("2026-02-15_batch/reaching", "rat01_2026-02-14_probe1") catalog.download_item("2026-02-15_batch/reaching", "logs")

Bulk prefetch (idempotent — skips items already on disk)

result = catalog.prefetch(drive_path="2026-02-15_batch", drive_path_mode="prefix")

Access the raw DataFrames

catalog.recordings_df catalog.items_df

Print a Drive tree

print(catalog.file_tree())

Print a headline summary report: counts, download progress, date

range, and Drive/local storage totals. Pass verbose=True for an

item-type breakdown, largest entries, and a full per-entry listing.

print(catalog.summary())

Attributes

MatchMode `module-attribute`

MatchMode = Literal['exact', 'prefix', 'contains']

How a filter value is matched against a catalog field: an exact match, a prefix match, or a substring match.

Classes

Catalog

Main interface for radiens-drive-catalog.

Wraps Google Drive scanning, local JSON catalog management, and file download. Querying is done directly on the recordings_df and items_df DataFrames using standard pandas operations.

Recordings are uniquely identified by (drive_path, base_name) where drive_path is the slash-joined path from the Drive root to the containing folder.

Example

from radiens_drive_catalog import Catalog, Config

config = Config.from_file("config.json")
catalog = Catalog(config)

catalog.scan()
hits = catalog.list_recordings(drive_path="2026-02", drive_path_mode="prefix")
path = catalog.get_recording_path("2026-02/reaching", "rat01")
catalog.prefetch(drive_path="2026-02", drive_path_mode="prefix")
print(catalog.file_tree())
print(catalog.summary())

Attributes

items_df `property`

items_df

The full Drive items catalog as a pandas DataFrame.

Columns: name, is_folder, drive_path, drive_id, mime_type, local_path, upload_time, size

recordings_df `property`

recordings_df

The full recording catalog as a pandas DataFrame.

Columns: base_name, drive_path, drive_file_ids, local_path, upload_time, size

Functions

download_item

download_item(drive_path, name)

Download a Drive item (file or folder) to the local data directory.

Items are stored under {local_data_dir}/{drive_path}/{name}.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the item's parent folder. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`name`	`str`	The file or folder name.	required

Returns:

Type	Description
`str`	The local path to the downloaded file or folder.

Raises:

Type	Description
`EntryNotFoundError`	If the item is not found in the catalog.

download_recording

download_recording(drive_path, base_name)

Download the xdat files for a recording to the local data directory.

Files are stored under {local_data_dir}/{drive_path}/.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the folder containing the recording. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`base_name`	`str`	The recording identifier (shared filename stem).	required

Returns:

Type	Description
`str`	The local directory path where the files were written.

Raises:

Type	Description
`EntryNotFoundError`	If the recording is not found in the catalog.

file_tree

file_tree()

Return a string rendering of the Drive tree with local-presence indicators.

Each entry is annotated as [recording], [folder], or [file] and [local] or [not local] based on whether it has been downloaded.

Returns:

Type	Description
`str`	A multi-line indented string representing the Drive folder hierarchy.

get_item_path

get_item_path(drive_path, name)

Return the local path for a Drive item, downloading if needed.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the item's parent folder. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`name`	`str`	The file or folder name.	required

Returns:

Type	Description
`str`	The local path to the downloaded file or folder.

Raises:

Type	Description
`EntryNotFoundError`	If the item is not found in the catalog.

get_recording

get_recording(base_name, drive_path=None)

Look up a recording by base_name, requiring it to be unique.

Parameters:

Name	Type	Description	Default
`base_name`	`str`	The recording's filename stem.	required
`drive_path`	`str \| Path \| None`	Exact `drive_path` to disambiguate recordings that share a `base_name` at different Drive locations. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	`None`

Returns:

Type	Description
`RecordingEntry`	The matching :class:`RecordingEntry`.

Raises:

Type	Description
`EntryNotFoundError`	If no recording with this base_name (and `drive_path`, if given) exists.
`AmbiguousRecordingError`	If more than one recording matches.

get_recording_path

get_recording_path(drive_path, base_name)

Return the local directory path for a recording, downloading if needed.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the folder containing the recording. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`base_name`	`str`	The recording identifier (shared filename stem).	required

Returns:

Type	Description
`str`	The local directory path where the xdat files reside.

Raises:

Type	Description
`EntryNotFoundError`	If the recording is not found in the catalog.

list_items

list_items(
    *,
    drive_path=None,
    drive_path_mode="exact",
    is_folder=None,
)

Query the Drive items catalog and return a filtered DataFrame.

All filters are applied together (AND semantics). Omitting all arguments returns the full items catalog.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path \| None`	Value to match against each item's parent `drive_path`. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	`None`
`drive_path_mode`	`MatchMode`	How `drive_path` is matched: `"exact"` (default), `"prefix"`, or `"contains"`.	`'exact'`
`is_folder`	`bool \| None`	When `True` return only folders; when `False` return only files; when `None` (default) return both.	`None`

list_recordings

list_recordings(
    *,
    base_name=None,
    base_name_mode="exact",
    drive_path=None,
    drive_path_mode="exact",
)

Query the recording catalog and return a filtered DataFrame.

All filters are applied together (AND semantics). Omitting all arguments returns the full catalog.

Parameters:

Name	Type	Description	Default
`base_name`	`str \| None`	Value to match against each recording's `base_name`.	`None`
`base_name_mode`	`MatchMode`	How `base_name` is matched: `"exact"` (default), `"prefix"`, or `"contains"`.	`'exact'`
`drive_path`	`str \| Path \| None`	Value to match against each recording's `drive_path`. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	`None`
`drive_path_mode`	`MatchMode`	How `drive_path` is matched: `"exact"` (default), `"prefix"`, or `"contains"`.	`'exact'`

local_footprint

local_footprint()

Return the total on-disk size, in bytes, of all downloaded data.

Sums each recording's own xdat files and each downloaded item's file or folder subtree. Use this to drive caller-managed eviction — e.g. release recordings until local_footprint() drops below a chosen cap.

Returns:

Type	Description
`int`	The total local footprint in bytes.

prefetch

prefetch(
    *,
    drive_path=None,
    drive_path_mode="exact",
    base_name=None,
    base_name_mode="exact",
    recordings=True,
    items=True,
    is_folder=None,
    force=False,
    max_workers=8,
)

Bulk-download matching recordings and items, skipping already-local entries.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path \| None`	Value to match against each entry's `drive_path`. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	`None`
`drive_path_mode`	`MatchMode`	How `drive_path` is matched: `"exact"` (default), `"prefix"`, or `"contains"`.	`'exact'`
`base_name`	`str \| None`	Value to match against each recording's `base_name`. Only constrains recordings — items have no `base_name` and are unaffected by this filter.	`None`
`base_name_mode`	`MatchMode`	How `base_name` is matched: `"exact"` (default), `"prefix"`, or `"contains"`.	`'exact'`
`recordings`	`bool`	When `False`, no recordings are downloaded.	`True`
`items`	`bool`	When `False`, no items are downloaded.	`True`
`is_folder`	`bool \| None`	Restrict item downloads to folders (`True`) or files (`False`).	`None`
`force`	`bool`	When `True`, re-download regardless of local presence.	`False`
`max_workers`	`int`	Number of recordings/items to download concurrently. Downloads are I/O-bound on the Drive API, which throttles each stream well below typical link bandwidth, so several concurrent streams give near-linear speedup. Each worker thread uses its own Drive service (the API client is not thread-safe). Use `1` to download strictly sequentially.	`8`

Returns:

Name	Type	Description
`A`	`PrefetchResult`	class:`PrefetchResult` with per-category download and skip counts.

Raises:

Type	Description
`Exception`	Re-raises the first download failure after recording all successful downloads in the catalog. Because prefetch is idempotent (already-local entries are skipped), re-running resumes only the failed downloads.

release_item

release_item(drive_path, name)

Delete a Drive item's local file or folder and clear its local_path.

A file item's local path is removed with unlink; a folder item's local path is an entire subtree owned by that item and is removed with shutil.rmtree. Safe to call when the item is not downloaded — it is a no-op that returns 0.

Only this item's own local_path is cleared. Deleting a folder also removes any nested files/subfolders that were cataloged as separate entries (see :meth:scan), so their stored local_path becomes stale until the next :meth:scan. This is by design: local_path is a hint reconciled by :meth:scan, and all download/size operations (:meth:get_item_path, :meth:prefetch, :meth:local_footprint) re-check the disk, so a stale value never causes a wrong result.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the item's parent folder. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`name`	`str`	The file or folder name.	required

Returns:

Type	Description
`int`	The number of bytes freed from disk.

Raises:

Type	Description
`EntryNotFoundError`	If the item is not found in the catalog.

release_recording

release_recording(drive_path, base_name)

Delete a recording's local xdat files and clear its local_path.

Only the recording's own three xdat files are removed; the containing drive_path directory is left in place because sibling recordings and downloaded items may share it. Safe to call when the recording is not downloaded — it is a no-op that returns 0.

Parameters:

Name	Type	Description	Default
`drive_path`	`str \| Path`	The slash-joined path to the folder containing the recording. Accepts a `str` or `Path`; a `Path` is converted via `str()`.	required
`base_name`	`str`	The recording identifier (shared filename stem).	required

Returns:

Type	Description
`int`	The number of bytes freed from disk.

Raises:

Type	Description
`EntryNotFoundError`	If the recording is not found in the catalog.

scan

scan(*, flat=True)

Scan Drive and rebuild the catalog JSON.

Any existing local_path entries are preserved so a rescan doesn't forget which recordings or items have already been downloaded.

Parameters:

Name	Type	Description	Default
`flat`	`bool`	If `True` (the default), use a flat scan of all files visible to the service account. Set to `False` for a recursive traversal from the root folder.	`True`

Returns:

Name	Type	Description
`A`	`ScanResult`	class:`ScanResult` with new/existing/removed counts.

summary

summary(*, rescan=False, verbose=False)

Return a human-readable report summarizing the whole catalog.

The default report is headline-only: recording/item counts, local-download progress, the upload-date range, Drive size (broken out separately for recordings and items, plus local storage size), average recording size, and an incomplete-recording count. Pass verbose=True to additionally get an item-type breakdown, the largest entries by size, the itemized incomplete-recordings list, a per-top-level-folder breakdown, and a full per-entry listing — sections whose length scales with the size of the catalog, so they're opt-in rather than pushing the headline stats out of view.

This package once had (and deliberately removed) a similar convenience method, status(), in favor of operating on the DataFrames directly. summary() is justified where status() wasn't: it's a genuine multi-source rollup across both DataFrames plus a live local-disk walk, not a one-column derivation a caller could trivially replicate.

Parameters:

Name	Type	Description	Default
`rescan`	`bool`	If `True`, call :meth:`scan` first to refresh the catalog and backfill `size` on older entries before computing the report. Defaults to `False` (fast, read-only).	`False`
`verbose`	`bool`	If `True`, additionally include the item-type breakdown, largest entries, itemized incomplete recordings, per-top-level-folder breakdown, and a full per-entry listing.	`False`

Returns:

Type	Description
`str`	A multi-line report string.

PrefetchResult `dataclass`

Summary of a :meth:Catalog.prefetch call.

Attributes:

Name	Type	Description
`recordings_downloaded`	`int`	Number of recordings fetched from Drive.
`recordings_skipped`	`int`	Number of recordings already available locally.
`items_downloaded`	`int`	Number of items fetched from Drive.
`items_skipped`	`int`	Number of items already available locally.

ScanResult `dataclass`

Summary of a :meth:Catalog.scan call.

Attributes:

Name	Type	Description
`recordings_new`	`int`	Recordings found on Drive not in the previous catalog.
`recordings_existing`	`int`	Recordings found on Drive already in the catalog.
`recordings_removed`	`int`	Recordings in the previous catalog not on Drive.
`items_new`	`int`	Items found on Drive not in the previous catalog.
`items_existing`	`int`	Items found on Drive already in the catalog.
`items_removed`	`int`	Items in the previous catalog not found on Drive.

Functions

match_series

match_series(series, query, mode)

Filter a string pd.Series by query under mode, returning a boolean mask.

Use this to filter an already-fetched DataFrame (e.g. a cached recordings_df, or your own subset of it) with the same exact/prefix/contains semantics :meth:Catalog.list_recordings and :meth:Catalog.list_items use internally. A None query matches everything (an all-True mask).

radiens_drive_catalog.catalog

catalog.py

Build or refresh the catalog from Drive

Query using convenience methods or pandas directly

Get a local path, downloading if needed

Unique-basename lookup

Download explicitly

Bulk prefetch (idempotent — skips items already on disk)

Access the raw DataFrames

Print a Drive tree

Print a headline summary report: counts, download progress, date

range, and Drive/local storage totals. Pass verbose=True for an

item-type breakdown, largest entries, and a full per-entry listing.

Attributes

MatchMode module-attribute

Classes

Catalog

Attributes

items_df property

recordings_df property

Functions

download_item

download_recording

file_tree

get_item_path

get_recording

get_recording_path

list_items

list_recordings

local_footprint

prefetch

release_item

release_recording

scan

summary

PrefetchResult dataclass

ScanResult dataclass

Functions

match_series

MatchMode `module-attribute`

items_df `property`

recordings_df `property`

PrefetchResult `dataclass`

ScanResult `dataclass`