Usage Guide

First scan

Before you can query or download anything, build the catalog by scanning Drive:

from radiens_drive_catalog import Catalog, Config

config = Config.from_file("config.json")
catalog = Catalog(config)

catalog.scan()

scan() defaults to a flat listing of all files visible to the service account, then reconstructs folder paths via the parents field — this is faster and works even when the root folder is not directly accessible. Pass flat=False for a recursive traversal from the root folder instead. Either way, all xdat recordings and non-xdat Drive items are found and written to catalog_path as JSON. For large drives this may take a minute or two — Drive API list calls are paginated.

The catalog file persists between Python sessions. You don't need to call scan() again unless the Drive contents have changed.

Rescanning

Calling scan() again is safe and idempotent. It rebuilds the catalog from Drive but preserves any local_path values for recordings and items you have already downloaded, so you won't lose track of local files after a rescan.

Recordings and items that no longer exist on Drive are dropped from the catalog.

Recordings

`list_recordings()` — filtered DataFrame

catalog.list_recordings()                                                          # all recordings
catalog.list_recordings(drive_path="2026-02-15_batch/reaching")                     # exact folder
catalog.list_recordings(drive_path="2026-02-15_batch", drive_path_mode="prefix")    # full date subtree
catalog.list_recordings(drive_path="reaching", drive_path_mode="contains")          # any depth
catalog.list_recordings(base_name="rat01_session3")                                # exact base_name
catalog.list_recordings(base_name="rat01", base_name_mode="prefix")                 # base_name prefix
catalog.list_recordings(base_name="session3", base_name_mode="contains")            # base_name substring

drive_path matches against the drive_path column (the slash-joined path from the Drive root to the folder containing the recording); base_name matches against the base_name column. Both accept a _mode of "exact" (default), "prefix", or "contains". All filters are combined with AND semantics — e.g. all "rat01"-prefixed recordings under 2026-02-15_batch. Omitting all filters returns the full catalog.

The return value is a pandas DataFrame with a reset index.

`recordings_df` — raw DataFrame

catalog.recordings_df

The full recording catalog as a DataFrame with columns:

Column	Type	Description
`base_name`	`str`	Recording identifier (not globally unique — pair with `drive_path`)
`drive_path`	`str`	Slash-joined path from root to the containing folder
`drive_file_ids`	`dict`	Maps `"data"`, `"meta"`, `"timestamp"` to Drive file IDs
`local_path`	`str \\| None`	Local directory path if downloaded, else `None`
`size`	`int \\| None`	Combined size, in bytes, of the constituent xdat files that reported a size
`upload_time`	`datetime64[UTC] \\| NaT`	UTC-aware upload timestamp from Drive, or `NaT` if not available

The DataFrame is cached in memory and invalidated automatically after scan() or download_recording().

Working with the DataFrame directly

Since catalog.recordings_df is a standard pandas DataFrame, you can use the full pandas API:

# Find all recordings that aren't downloaded yet
catalog.recordings_df[catalog.recordings_df["local_path"].isna()]

# Exact path match
catalog.recordings_df[catalog.recordings_df["drive_path"] == "2026-02-15_batch/reaching"]

# Prefix / subtree
catalog.recordings_df[catalog.recordings_df["drive_path"].str.startswith("2026-02")]

# Substring search
catalog.recordings_df[catalog.recordings_df["drive_path"].str.contains("reaching")]

`download_recording()` — explicit download

local_path = catalog.download_recording("2026-02-15_batch/reaching", "rat01_session3")

Downloads the three xdat files for a recording. Files are stored under {local_data_dir}/{drive_path}/, mirroring the Drive folder hierarchy:

local_data_dir/
    2026-02-15_batch/
        reaching/
            rat01_session3_data.xdat
            rat01_session3.xdat.json
            rat01_session3_timestamp.xdat

After a successful download, local_path is persisted back to the catalog JSON.

`get_recording_path()` — download if needed

path = catalog.get_recording_path("2026-02-15_batch/reaching", "rat01_session3")

Returns the local directory path for a recording. If the recording hasn't been downloaded yet — or if the recorded local_path no longer exists on disk — the download is triggered automatically. This is the most convenient entry point for analysis scripts:

import numpy as np

path = catalog.get_recording_path("2026-02-15_batch/reaching", "rat01_session3")
data = np.fromfile(f"{path}/rat01_session3_data.xdat", dtype=np.int16)

Drive items

Drive items are non-xdat files and folders found alongside recordings on Drive: logs directories, PowerPoint slides, writeups, and similar content. During scan(), every non-xdat file and every non-root folder is cataloged as an item, regardless of depth — no structural heuristics are applied.

`items_df` — raw DataFrame

catalog.items_df

The full items catalog as a DataFrame with columns:

Column	Type	Description
`name`	`str`	File or folder name (e.g. `"logs"`, `"notes.pptx"`)
`is_folder`	`bool`	`True` if this item is a Drive folder
`drive_path`	`str`	Slash-joined path to the item's parent folder
`drive_id`	`str`	Google Drive ID of this file or folder
`mime_type`	`str`	MIME type as reported by Drive
`local_path`	`str \\| None`	Local path if downloaded, else `None`
`size`	`int \\| None`	Size in bytes, or `None` for folders and unsized files (e.g. Google Workspace-native Docs/Sheets/Slides)
`upload_time`	`datetime64[UTC] \\| NaT`	UTC-aware upload timestamp from Drive, or `NaT` if not available

`list_items()` — filtered DataFrame

catalog.list_items()                                                       # all items
catalog.list_items(drive_path="2026-02-15_batch/reaching")                 # exact parent folder
catalog.list_items(drive_path="2026-02-15_batch", drive_path_mode="prefix") # full date subtree
catalog.list_items(is_folder=True)                                          # folders only
catalog.list_items(drive_path="2026-02", drive_path_mode="prefix", is_folder=False)  # combined

Or query the raw DataFrame directly:

# All folder items
catalog.items_df[catalog.items_df["is_folder"]]

# Items within a specific date subtree
catalog.items_df[catalog.items_df["drive_path"].str.startswith("2026-02-15_batch")]

# Items not yet downloaded
catalog.items_df[catalog.items_df["local_path"].isna()]

Identifying items

Items are uniquely identified by (drive_path, name). drive_path is the slash-joined path to the parent folder — for example a logs/ folder inside 2026-02-15_batch/reaching/ has drive_path = "2026-02-15_batch/reaching" and name = "logs". This means two logs/ folders from different experiments are distinct entries with different drive_path values.

`download_item()` — explicit download

local_path = catalog.download_item("2026-02-15_batch/reaching", "logs")

Downloads an item to local_data_dir/{drive_path}/{name}, the same Drive-mirroring convention used for recordings:

local_data_dir/
    2026-02-15_batch/
        reaching/
            logs/           ← entire folder subtree mirrored here
                log_0215.txt
            notes.pptx      ← file item

For folder items the entire Drive subtree is downloaded recursively.

`get_item_path()` — download if needed

path = catalog.get_item_path("2026-02-15_batch/reaching", "logs")

Returns the local path for an item, triggering a download if it isn't already available locally or if the recorded path no longer exists on disk.

Bulk operations and local storage

`prefetch()` — bulk, idempotent download

catalog.prefetch()                                                    # everything
catalog.prefetch(drive_path="2026-02", drive_path_mode="prefix")      # a subtree
catalog.prefetch(drive_path="2026-02", drive_path_mode="prefix", items=False)  # recordings only
catalog.prefetch(is_folder=True)                                       # narrow to folder items
catalog.prefetch(force=True)                                           # re-download everything

Downloads every recording and item matching the given filters, skipping entries already present on disk. Downloads run concurrently across a thread pool (max_workers, default 8); pass max_workers=1 to force sequential execution. Returns a PrefetchResult with per-category download and skip counts.

`release_recording()` / `release_item()` — local eviction

freed_bytes = catalog.release_recording("2026-02-15_batch/reaching", "rat01_session3")
freed_bytes = catalog.release_item("2026-02-15_batch/reaching", "logs")

Deletes a recording's or item's local files and clears its local_path, returning the number of bytes freed. Both are idempotent no-ops when the entry isn't downloaded. release_recording() only removes that recording's own xdat files, never the shared drive_path directory or sibling entries.

`local_footprint()` — total on-disk size

total_bytes = catalog.local_footprint()

Returns the total size, in bytes, of everything currently downloaded. Combine with release_* to cap local storage: release entries until local_footprint() drops below a chosen limit — useful for processing a corpus larger than local disk.

`file_tree()` — annotated Drive tree

print(catalog.file_tree())

Renders the full Drive hierarchy as an indented string, annotating each entry as [recording], [folder], or [file], and [local] or [not local].

`summary()` — catalog report

print(catalog.summary())            # headline: counts, download progress, date range, storage totals
print(catalog.summary(verbose=True)) # + item-type breakdown, largest entries, incomplete recordings, per-folder breakdown, full listing
print(catalog.summary(rescan=True))  # refresh the catalog from Drive first

Usage Guide

First scan

Rescanning

Recordings

list_recordings() — filtered DataFrame

recordings_df — raw DataFrame

Working with the DataFrame directly

download_recording() — explicit download

get_recording_path() — download if needed

Drive items

items_df — raw DataFrame

list_items() — filtered DataFrame

Identifying items

download_item() — explicit download

get_item_path() — download if needed