Skip to content

radiens_drive_catalog.catalog

catalog.py

The main interface for radiens-drive-catalog. The Catalog class wraps Drive scanning, local caching, querying, and downloading behind a simple API.

Typical usage

from radiens_drive_catalog import Catalog, Config

config = Config.from_file("config.json") catalog = Catalog(config)

Build or refresh the catalog from Drive

catalog.scan()

Query datasets

df = catalog.list(date="2026-02") df = catalog.list(date="2026-02", experiment="reaching")

Query assets (non-xdat files and folders)

adf = catalog.list_assets(date="2026-02", experiment="reaching")

Get a local path, downloading if needed

path = catalog.get_path("rat01_2026-02-14_probe1") path = catalog.get_asset_path("2026-02-15_batch/reaching", "logs")

Download explicitly

catalog.download("rat01_2026-02-14_probe1") catalog.download_asset("2026-02-15_batch/reaching", "logs")

Access the raw DataFrames

catalog.df catalog.assets_df

Classes

Catalog

Main interface for radiens-drive-catalog.

Wraps Google Drive scanning, local JSON catalog management, dataset and asset querying, and file download behind a single object.

Example 
from radiens_drive_catalog import Catalog, Config

config = Config.from_file("config.json")
catalog = Catalog(config)

catalog.scan()                              # build catalog from Drive
df = catalog.list(date="2026-02")           # query datasets
adf = catalog.list_assets(experiment="reaching")  # query assets
path = catalog.get_path("rat01_session3")  # download dataset if needed
path = catalog.get_asset_path("2026-02-15/reaching", "logs")  # download asset if needed

Attributes

assets_df property 
assets_df

The full assets catalog as a pandas DataFrame. Columns: asset_name, asset_type, date_folder, experiment, drive_path, drive_id, mime_type, local_path

df property 
df

The full dataset catalog as a pandas DataFrame. Columns: base_name, date_folder, experiment, drive_path, drive_file_ids, local_path

Functions

__init__ 
__init__(config=None, config_path=None)

Initialize a Catalog.

Parameters:

Name Type Description Default
config Config | None

Package configuration (Drive credentials, folder ID, local paths).

 None
config_path str | None

Path to the config JSON file. Used if config is None.

 None
download 
download(base_name)

Download the xdat files for a dataset to local_data_dir.

After a successful download, persists the local_path back to the catalog JSON so it survives future sessions.

Parameters:

Name Type Description Default
base_name str

The dataset identifier (shared filename stem).

 required

Returns:

Type Description
str

The local directory path where the files were written.

Raises:

Type Description
ValueError

If base_name is not found in the catalog.
download_asset 
download_asset(drive_path, asset_name)

Download an asset (file or folder) to the local assets directory.

Assets are stored under {local_data_dir}/assets/{drive_path}/{asset_name}. For folder assets the entire subtree is downloaded recursively.

After a successful download, persists the local_path back to the catalog JSON.

Parameters:

Name Type Description Default
drive_path str

The slash-joined path to the asset's parent folder (e.g. "2026-02-15_batch/reaching"). Shown in assets_df["drive_path"].

 required
asset_name str

The file or folder name (e.g. "logs").

 required

Returns:

Type Description
str

The local path to the downloaded file or folder.

Raises:

Type Description
ValueError

If the asset is not found in the catalog.
get_asset_path 
get_asset_path(drive_path, asset_name)

Return the local path for an asset, downloading if needed.

If the asset has not been downloaded yet, or if the recorded local_path no longer exists on disk, the download is triggered automatically.

Parameters:

Name Type Description Default
drive_path str

The slash-joined path to the asset's parent folder (e.g. "2026-02-15_batch/reaching").

 required
asset_name str

The file or folder name (e.g. "logs").

 required

Returns:

Type Description
str

The local path to the downloaded file or folder.

Raises:

Type Description
ValueError

If the asset is not found in the catalog.
get_path 
get_path(base_name)

Return the local directory path for a dataset, downloading if needed.

If the dataset has not been downloaded yet, or if the recorded local_path no longer exists on disk, the download is triggered automatically.

Parameters:

Name Type Description Default
base_name str

The dataset identifier (shared filename stem).

 required

Returns:

Type Description
str

The local directory path where the xdat files reside.

Raises:

Type Description
ValueError

If base_name is not found in the catalog.
list 
list(date_folder=None, experiment=None)

Query the dataset catalog and return a filtered DataFrame.

Arguments act as hierarchical filters — each narrows the result further. Omitting an argument returns all values for that level.

Parameters:

Name Type Description Default
date_folder str | None

Exact name of the top-level date folder to filter on (e.g. "2026-02-15_batch"). Must match the full folder name.

 None
experiment str | None

Exact name of the depth-2 experiment folder to filter on.

 None

Examples:

catalog.list() # everything catalog.list(date_folder="2026-02-15_batch") # one date folder catalog.list(date_folder="2026-02-15_batch", experiment="reaching") catalog.list(experiment="reaching") # across all dates

list_assets 
list_assets(
    date_folder=None, experiment=None, asset_type=None
)

Query the asset catalog and return a filtered DataFrame.

Arguments act as hierarchical filters. Omitting an argument returns all values for that level.

Parameters:

Name Type Description Default
date_folder str | None

Exact name of the top-level date folder to filter on (e.g. "2026-02-15_batch"). Must match the full folder name.

 None
experiment str | None

Exact name of the depth-2 experiment folder to filter on.

 None
asset_type Literal['folder', 'file'] | None

"folder" or "file"; if omitted, returns both.

 None

Examples:

catalog.list_assets() catalog.list_assets(date_folder="2026-02-15_batch", experiment="reaching") catalog.list_assets(asset_type="folder")

list_by_path 
list_by_path(drive_path)

Return all dataset rows whose drive_path exactly equals drive_path.

Parameters:

Name Type Description Default
drive_path str

Slash-joined path to the experiment folder, as returned by :meth:list_experiment_paths.

 required
list_experiment_paths 
list_experiment_paths(date_folder=None)

Return sorted unique drive_path values that contain recordings.

Each value is the full slash-joined path from the Drive root to the folder holding xdat files, at any nesting depth. Use these paths directly with :meth:get_asset_path or :meth:list_by_path.

Parameters:

Name Type Description Default
date_folder str | None

When provided, restrict to paths whose first component matches this date folder exactly.

 None

Examples:

catalog.list_experiment_paths() catalog.list_experiment_paths(date_folder="2026-02-15_batch")

scan 
scan(*, flat=True)

Scan Drive and rebuild the catalog JSON.

Any existing local_path entries are preserved so a rescan doesn't forget which datasets or assets have already been downloaded.

Parameters:

Name Type Description Default
flat bool

If True (the default), use a flat scan of all files visible to the service account — faster and works even when the root folder is inaccessible. Set to False for a recursive traversal from the root folder.

 True
status 
status()

Return the dataset catalog DataFrame with an extra boolean column 'is_local' indicating which datasets are already downloaded. Useful for a quick overview of what's available locally.

Functions