API Reference - Intake Documentation
API Reference - Intake Documentation
API Reference
User Functions
intake.readers.datatypes.recommend ([url, ...]) Show which Intake data types can apply to the
intake.readers.convert.auto_pipeline (url[, ...]) Create pipeline from given URL to desired outp
intake.readers.convert.path (start, end[, ...]) Find possible conversion paths from start to en
get(key, default=None)
Return the value for key if key is in the dictionary, else default.
load(fn=None)
load_env()
reset()
set(update_dict=None, **kw)
values: dict
Examples
>>> intake.conf.set(myval=5)
>>> intake.conf.set(intake.readers.utils.nested_keys_to_dict({"deep.2.key":
True})
Show which Intake data types can apply to the given details
Location of data
mime: str
A small number of bytes from the file head, for seeking magic bytes.
If it is True, fetch these bytes from th given URL/storage_options and
use them. If None, only fetch bytes if there is no match by mime type
or path, if False, don’t fetch at all.
contents: bool | None
Will search for the shortest conversion path from the inferred data-type to the output.
Parameters url: input data, usually a location/URL, but maybe a data instance
storage_options: if url is a remote str, these are kwargs that fsspec may
need to
access it
avoid: don’t consider readers whose names match any of these strings
delete(name, recursive=False)
We do not check whether any other entity in the catalog refers to what is being
deleted, so you can break other entries this way.
Parameters recursive: bool
classmethod from_dict(data)
storage_options:
kwargs to pass to fsspec for opening the file to read; can pass as storage_options=
or will pick up any unused kwargs for simplicity
get_aliases(entity: str)
Return those alias names that point to the given opaque key
get_entity(item: str)
tok:
Find and promote given named parameter, assuming they are all identical
parameter_name:
If the parameter is found in a reader, it can be promoted to the data it depends on.
Parameters in a data description can only be promoted to a catalog global.
search(expr)→ Catalog
The new catalog will have those entries which pass the filter expr, which is an instance
of intake.readers.search.BaseSearch (i.e., has a method like filter(entry) -> bool).
In the special case that expr is just a string, the Text search expression will be used.
storage_options:
Defines some data: class and arguments. This may be laoded in a number of ways
get_kwargs(user_parameters: Optional[dict[str |
intake.readers.user_parameters.BaseUserParameter]] = None, **kwargs)→ dict[str, Any]
Get set of kwargs for given reader, based on prescription, new args and user
parameters
This class is typically stored inside Catalogs, and can contain templated arguments which
get evaluated at the time that it is accessed from a Catalog.
check_imports()
Are the packages listed in the “imports” key of the metadata available?
classmethod from_dict(data)
Get set of kwargs for given reader, based on prescription, new args and user
parameters
to_cat(name=None)
Show which readers claim to support the given data instance or a superclass
Attempt to construct a reader instance by finding one that matches the function call
Fails for readers that don’t define a func, probably because it depends on the file type or
needs a dynamic instance to be a method of.
Base Classes
contains: set[str] = {}
if using a directory URL, an ls() on that path will contain these things
binary patterns, usually at the file head; each item identifies this data type
property possible_outputs
property possible_readers
structure: set[str] = {}
to_entry()
If neither outtype or reader is passed, the first importable reader will be picked.
property data
discover(**kwargs)
classmethod doc()
implements: set[intake.readers.datatypes.BaseData] = {}
imports: set[str] = {}
optional_imports: set[str] = {}
other_funcs: set[str] = {}
read(*args, **kwargs)
to_cat(name=None)
to_entry()
Most often, subclasses call a single function on the data, but arbitrary complex transforms
are possible. This is designed to be one step in a Pipeline.
.run() will be called on the output object from the previous stage, subclasses will wither
override that, or just provide a func=.
class intake.readers.namespaces.Namespace(reader)
acts_on: tuple[str] = ()
imports: tuple[str] = ()
class intake.readers.search.SearchBase
coerce(value)
description
set_default(value)
to_dict()
validate(value)→ bool
with_default(value)
intake.readers.datatypes.TIFF (url[, ...]) Image format commonly used for large data
Reader Classes
Includes readers, transformers, converters and output classes.
intake.readers.convert.DaskArrayToTileDB (*args)
intake.readers.convert.DaskDFToPandas (*args)
intake.readers.convert.DaskToRay (*args[, ...])
intake.readers.convert.DeltaQueryToDask (*args)
intake.readers.convert.DeltaQueryToDaskGeopandas (*args)
intake.readers.convert.HuggingfaceToRay (*args)
intake.readers.convert.NibabelToNumpy (*args)
intake.readers.convert.NumpyToTileDB (*args)
intake.readers.convert.PandasToGeopandas (*args)
intake.readers.convert.PandasToMetagraph (*args)
intake.readers.convert.PandasToPolars (*args)
intake.readers.convert.PolarsToPandas (*args)
intake.readers.convert.TileDBToNumpy (*args)
intake.readers.convert.TiledNodeToCatalog (*args)
intake.readers.output.DaskArrayToZarr (*args)
intake.readers.output.PandasToFeather (*args)
intake.readers.output.PandasToParquet (*args)
intake.readers.output.XarrayToNetCDF (*args)
intake.readers.readers.AwkwardParquet (*args)
intake.readers.readers.CupyNumpyReader (*args)
intake.readers.readers.CupyTextReader (*args)
intake.readers.readers.DaskAwkwardJSON (*args)
intake.readers.readers.DaskAwkwardParquet (*args)
intake.readers.readers.DaskDeltaLake (*args)
intake.readers.readers.FileExistsReader (*args)
intake.readers.readers.GeoPandasReader (*args)
intake.readers.readers.GeoPandasTabular (*args)
intake.readers.readers.HuggingfaceReader (*args)
intake.readers.readers.KerasImageReader (*args)
intake.readers.readers.KerasModelReader (*args)
intake.readers.readers.NibabelNiftiReader (*args)
intake.readers.readers.PandasFeather (*args)
intake.readers.readers.PandasParquet (*args)
intake.readers.readers.PandasSQLAlchemy (*args)
intake.readers.readers.PolarsDeltaLake (*args)
intake.readers.readers.PolarsFeather (*args)
intake.readers.readers.PolarsIceberg (*args)
intake.readers.readers.PolarsParquet (*args)
intake.readers.readers.PrometheusMetricReader (*args)
intake.readers.readers.RasterIOXarrayReader (*args)
intake.readers.readers.SKImageReader (*args)
intake.readers.readers.SKLearnExampleReader (*args)
intake.readers.readers.SKLearnModelReader (*args)
intake.readers.readers.ScipyMatlabReader (*args)
intake.readers.readers.ScipyMatrixMarketReader (*args)
intake.readers.readers.SparkDataFrame (*args)
intake.readers.readers.SparkDeltaLake (*args)
intake.readers.readers.TFPublicDataset (*args)
intake.readers.readers.TFRecordReader (*args)
intake.readers.readers.TFSQL (*args[, ...])
intake.readers.readers.TileDBDaskReader (*args)
intake.readers.readers.XArrayDatasetReader (*args)
intake.readers.readers.YAMLCatalogReader (*args)
intake.readers.transform.DataFrameColumns (*args)
intake.readers.transform.PysparkColumns (*args)
intake.readers.transform.THREDDSCatToMergedDataset (*args)