tf.raw_ops.ExperimentalParallelInterleaveDataset
Stay organized with collections
Save and categorize content based on your preferences.
Creates a dataset that applies f
to the outputs of input_dataset
.
tf.raw_ops.ExperimentalParallelInterleaveDataset(
input_dataset,
other_arguments,
cycle_length,
block_length,
sloppy,
buffer_output_elements,
prefetch_input_elements,
f,
output_types,
output_shapes,
name=None
)
The resulting dataset is similar to the InterleaveDataset
, with the exception
that if retrieving the next value from a dataset would cause the requester to
block, it will skip that input dataset. This dataset is especially useful
when loading data from a variable-latency datastores (e.g. HDFS, GCS), as it
allows the training step to proceed so long as some data is available.
!! WARNING !! This dataset is not deterministic!
Args |
input_dataset
|
A Tensor of type variant .
|
other_arguments
|
A list of Tensor objects.
|
cycle_length
|
A Tensor of type int64 .
|
block_length
|
A Tensor of type int64 .
|
sloppy
|
A Tensor of type bool .
|
buffer_output_elements
|
A Tensor of type int64 .
|
prefetch_input_elements
|
A Tensor of type int64 .
|
f
|
A function decorated with @Defun.
A function mapping elements of input_dataset , concatenated with
other_arguments , to a Dataset variant that contains elements matching
output_types and output_shapes .
|
output_types
|
A list of tf.DTypes that has length >= 1 .
|
output_shapes
|
A list of shapes (each a tf.TensorShape or list of ints ) that has length >= 1 .
|
name
|
A name for the operation (optional).
|
Returns |
A Tensor of type variant .
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates. Some content is licensed under the numpy license.
Last updated 2024-04-26 UTC.
[null,null,["Last updated 2024-04-26 UTC."],[],[],null,["# tf.raw_ops.ExperimentalParallelInterleaveDataset\n\n\u003cbr /\u003e\n\nCreates a dataset that applies `f` to the outputs of `input_dataset`.\n\n#### View aliases\n\n\n**Compat aliases for migration**\n\nSee\n[Migration guide](https://www.tensorflow.org/guide/migrate) for\nmore details.\n\n[`tf.compat.v1.raw_ops.ExperimentalParallelInterleaveDataset`](https://www.tensorflow.org/api_docs/python/tf/raw_ops/ExperimentalParallelInterleaveDataset)\n\n\u003cbr /\u003e\n\n tf.raw_ops.ExperimentalParallelInterleaveDataset(\n input_dataset,\n other_arguments,\n cycle_length,\n block_length,\n sloppy,\n buffer_output_elements,\n prefetch_input_elements,\n f,\n output_types,\n output_shapes,\n name=None\n )\n\nThe resulting dataset is similar to the `InterleaveDataset`, with the exception\nthat if retrieving the next value from a dataset would cause the requester to\nblock, it will skip that input dataset. This dataset is especially useful\nwhen loading data from a variable-latency datastores (e.g. HDFS, GCS), as it\nallows the training step to proceed so long as some data is available.\n\n!! WARNING !! This dataset is not deterministic!\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|---------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `input_dataset` | A `Tensor` of type `variant`. |\n| `other_arguments` | A list of `Tensor` objects. |\n| `cycle_length` | A `Tensor` of type `int64`. |\n| `block_length` | A `Tensor` of type `int64`. |\n| `sloppy` | A `Tensor` of type `bool`. |\n| `buffer_output_elements` | A `Tensor` of type `int64`. |\n| `prefetch_input_elements` | A `Tensor` of type `int64`. |\n| `f` | A function decorated with @Defun. A function mapping elements of `input_dataset`, concatenated with `other_arguments`, to a Dataset variant that contains elements matching `output_types` and `output_shapes`. |\n| `output_types` | A list of `tf.DTypes` that has length `\u003e= 1`. |\n| `output_shapes` | A list of shapes (each a [`tf.TensorShape`](../../tf/TensorShape) or list of `ints`) that has length `\u003e= 1`. |\n| `name` | A name for the operation (optional). |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ------- ||\n|---|---|\n| A `Tensor` of type `variant`. ||\n\n\u003cbr /\u003e"]]