Skip to content

Commit 0359bc8

Browse files
authored
fix: remove pre-caching of remote function results (#1028)
This will save redundant remote function execution as reported in b/370088754. The trade-off is that any remote function integration issues will be caughts only at a later point through a usage triggered sql execution. Why the caching is not working as indended in the reported use case in the bug? as per TrevorBergeron@ "the cached execution is useless when it is implicitly joined back to the base dataframe"
1 parent 4af5bbb commit 0359bc8

File tree

2 files changed

+3
-13
lines changed

2 files changed

+3
-13
lines changed

bigframes/dataframe.py

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3567,11 +3567,7 @@ def apply(self, func, *, axis=0, args: typing.Tuple = (), **kwargs):
35673567
ops.NaryRemoteFunctionOp(func=func), series_list[1:]
35683568
)
35693569
result_series.name = None
3570-
3571-
# Return Series with materialized result so that any error in the remote
3572-
# function is caught early
3573-
materialized_series = result_series.cache()
3574-
return materialized_series
3570+
return result_series
35753571

35763572
# Per-column apply
35773573
results = {name: func(col, *args, **kwargs) for name, col in self.items()}

bigframes/series.py

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1475,10 +1475,7 @@ def apply(
14751475
ops.RemoteFunctionOp(func=func, apply_on_null=True)
14761476
)
14771477

1478-
# return Series with materialized result so that any error in the remote
1479-
# function is caught early
1480-
materialized_series = result_series._cached(session_aware=False)
1481-
return materialized_series
1478+
return result_series
14821479

14831480
def combine(
14841481
self,
@@ -1506,10 +1503,7 @@ def combine(
15061503
other, ops.BinaryRemoteFunctionOp(func=func)
15071504
)
15081505

1509-
# return Series with materialized result so that any error in the remote
1510-
# function is caught early
1511-
materialized_series = result_series._cached()
1512-
return materialized_series
1506+
return result_series
15131507

15141508
@validations.requires_index
15151509
def add_prefix(self, prefix: str, axis: int | str | None = None) -> Series:

0 commit comments

Comments
 (0)