You can subscribe to this list here.
2010 |
Jan
|
Feb
|
Mar
|
Apr
(4) |
May
(28) |
Jun
(12) |
Jul
(11) |
Aug
(12) |
Sep
(5) |
Oct
(19) |
Nov
(14) |
Dec
(12) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2011 |
Jan
(18) |
Feb
(30) |
Mar
(115) |
Apr
(89) |
May
(50) |
Jun
(44) |
Jul
(22) |
Aug
(13) |
Sep
(11) |
Oct
(30) |
Nov
(28) |
Dec
(39) |
2012 |
Jan
(38) |
Feb
(18) |
Mar
(43) |
Apr
(91) |
May
(108) |
Jun
(46) |
Jul
(37) |
Aug
(44) |
Sep
(33) |
Oct
(29) |
Nov
(36) |
Dec
(15) |
2013 |
Jan
(35) |
Feb
(611) |
Mar
(5) |
Apr
(55) |
May
(30) |
Jun
(28) |
Jul
(458) |
Aug
(34) |
Sep
(9) |
Oct
(39) |
Nov
(22) |
Dec
(32) |
2014 |
Jan
(16) |
Feb
(16) |
Mar
(42) |
Apr
(179) |
May
(7) |
Jun
(6) |
Jul
(9) |
Aug
|
Sep
(4) |
Oct
|
Nov
(3) |
Dec
|
2015 |
Jan
|
Feb
|
Mar
|
Apr
(2) |
May
(4) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
S | M | T | W | T | F | S |
---|---|---|---|---|---|---|
|
|
|
1
(2) |
2
(3) |
3
|
4
|
5
|
6
(2) |
7
(1) |
8
|
9
(4) |
10
|
11
|
12
|
13
|
14
(1) |
15
(1) |
16
(1) |
17
|
18
|
19
|
20
|
21
(1) |
22
|
23
|
24
(2) |
25
|
26
|
27
(2) |
28
(1) |
29
(1) |
30
(22) |
|
|
From: Ashutosh B. <ash...@us...> - 2011-06-21 06:59:49
|
Project "Postgres-XC". The branch, master has been updated via 2a828017d88ff64453b37771337646454316269c (commit) from 88a19b42f3b599927b74e0aef2e19b9161b3a7eb (commit) - Log ----------------------------------------------------------------- commit 2a828017d88ff64453b37771337646454316269c Author: Ashutosh Bapat <ash...@en...> Date: Tue Jun 21 12:19:07 2011 +0530 The patch has following changes: 1. Any query with aggregates in it, is planned through standard planner, instead of directly creating a RemoteQuery node for it in pgxc_planner(). Before this fix, any query which involved aggregates and no grouping clause, was directly converted into a RemoteQuery node. RemoteQuery had no mechanism to handle aggregates involved in expressions properly (3237756). It could only handle aggregates without any covering node. Also, aggregates with order by clause would cause aggregates grouped according to datanodes storing the rows (3237712, 3147936). With the fix, such queries are planned through standard planner and optimized in grouping_planner to push the aggregates and/or group by clauses to the datanodes. This change made the aggregation mechanism under RemoteQuery node unnecessary. Hence cleaned up the same. 2. If there are any qualifications in the query, which need to be evaluated on the coordinator only, those are considered as local quals and are stuck in RemoteQuery node. There was no mechanism to handle these quals in ExecRemoteQuery(). Added that mechanism. 3. Fixed a bug in create_remotegrouping_planner() - When the RemoteQuery node has quals in it, (i.e. there are local quals), we can not push the aggregates and GROUP BY clause to the data node, because before aggregating, we need to apply those scan clauses, which can happen only on coordinator. 4. There are couple of expected output changes, related to the explain verbose (in create_index test). The patch fixes bugs 3237712, 3147936 and 3237756. diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index ccf5079..ae60a76 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -921,7 +921,8 @@ finalize_aggregate(AggState *aggstate, */ if (OidIsValid(peraggstate->collectfn_oid) && !aggstate->skip_trans) { - FunctionCallInfoData fcinfo; + FunctionCallInfoData fcinfo; + int saved_numArguments; InitFunctionCallInfoData(fcinfo, &(peraggstate->collectfn), 2, (void *) aggstate, NULL); /* @@ -937,24 +938,19 @@ finalize_aggregate(AggState *aggstate, fcinfo.argnull[0] = peraggstate->initCollectValueIsNull; fcinfo.arg[1] = pergroupstate->transValue; fcinfo.argnull[1] = pergroupstate->transValueIsNull; - if (fcinfo.flinfo->fn_strict && - (pergroupstate->transValueIsNull || peraggstate->initCollectValueIsNull)) - { - pergroupstate->transValue = (Datum)0; - pergroupstate->transValueIsNull = true; - } - else - { - Datum newVal = FunctionCallInvoke(&fcinfo); - - /* - * set the result of collection function to the transValue so that code - * below invoking final function does not change - */ - /* PGXCTODO: worry about the memory management here? */ - pergroupstate->transValue = newVal; - pergroupstate->transValueIsNull = fcinfo.isnull; - } + /* + * For collection function we expect only one argument other than the + * running collection result. The numArguments in peraggstate + * corresponds to the number of arguments to the aggregate, which is not + * correct for collection. Hence while applying collection function + * set numArguments to 1 and switch it back once the purpose is served. + */ + saved_numArguments = peraggstate->numArguments; + peraggstate->numArguments = 1; + advance_collection_function(aggstate, peraggstate, pergroupstate, &fcinfo); + peraggstate->numArguments = saved_numArguments; + pergroupstate->transValue = pergroupstate->collectValue; + pergroupstate->transValueIsNull = pergroupstate->collectValueIsNull; } /* diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index 2c7fee4..607adf5 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -905,7 +905,6 @@ _copyRemoteQuery(RemoteQuery *from) COPY_STRING_FIELD(sql_statement); COPY_NODE_FIELD(exec_nodes); COPY_SCALAR_FIELD(combine_type); - COPY_NODE_FIELD(simple_aggregates); COPY_NODE_FIELD(sort); COPY_NODE_FIELD(distinct); COPY_SCALAR_FIELD(read_only); @@ -953,40 +952,6 @@ _copyExecNodes(ExecNodes *from) } /* - * _copySimpleAgg - */ -static SimpleAgg * -_copySimpleAgg(SimpleAgg *from) -{ - SimpleAgg *newnode = makeNode(SimpleAgg); - - COPY_SCALAR_FIELD(column_pos); - COPY_NODE_FIELD(aggref); - COPY_SCALAR_FIELD(transfn_oid); - COPY_SCALAR_FIELD(finalfn_oid); - COPY_SCALAR_FIELD(arginputfn); - COPY_SCALAR_FIELD(argioparam); - COPY_SCALAR_FIELD(resoutputfn); - COPY_SCALAR_FIELD(transfn); - COPY_SCALAR_FIELD(finalfn); - if (!from->initValueIsNull) - newnode->initValue = datumCopy(from->initValue, from->transtypeByVal, - from->transtypeLen); - COPY_SCALAR_FIELD(initValueIsNull); - COPY_SCALAR_FIELD(inputtypeLen); - COPY_SCALAR_FIELD(resulttypeLen); - COPY_SCALAR_FIELD(transtypeLen); - COPY_SCALAR_FIELD(inputtypeByVal); - COPY_SCALAR_FIELD(resulttypeByVal); - COPY_SCALAR_FIELD(transtypeByVal); - /* No need to copy runtime info, just init */ - newnode->collectValueNull = true; - initStringInfo(&newnode->valuebuf); - - return newnode; -} - -/* * _copySimpleSort */ static SimpleSort * @@ -3860,9 +3825,6 @@ copyObject(void *from) case T_ExecNodes: retval = _copyExecNodes(from); break; - case T_SimpleAgg: - retval = _copySimpleAgg(from); - break; case T_SimpleSort: retval = _copySimpleSort(from); break; diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index a666914..a96619b 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -5158,11 +5158,20 @@ create_remotegrouping_plan(PlannerInfo *root, Plan *local_plan) return local_plan; /* * for Group plan we expect Sort under the Group, which is always the case, - * the condition below is really for some possible non-existent case + * the condition below is really for some possibly non-existent case. */ if (IsA(local_plan, Group) && !sort_plan) return local_plan; - + /* + * If the remote_scan has any quals on it, those need to be executed before + * doing anything. Hence we won't be able to push any aggregates or grouping + * to the data node. + * If it has any SimpleSort in it, then sorting is intended to be applied + * before doing anything. Hence can not push any aggregates or grouping to + * the data node. + */ + if (remote_scan->scan.plan.qual || remote_scan->sort) + return local_plan; /* * Grouping_planner may add Sort node to sort the rows @@ -5183,6 +5192,11 @@ create_remotegrouping_plan(PlannerInfo *root, Plan *local_plan) } } + /* + * At last we find the plan underneath is reducible into a single + * RemoteQuery node. + */ + /* find all the relations referenced by targetlist of Grouping node */ temp_vars = pull_var_clause((Node *)local_plan->targetlist, PVC_REJECT_PLACEHOLDERS); diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c index 89d3a39..e67cc5e 100644 --- a/src/backend/parser/analyze.c +++ b/src/backend/parser/analyze.c @@ -2117,7 +2117,6 @@ transformExecDirectStmt(ParseState *pstate, ExecDirectStmt *stmt) step->sql_statement = NULL; step->exec_nodes = NULL; step->combine_type = COMBINE_TYPE_NONE; - step->simple_aggregates = NIL; step->sort = NULL; step->distinct = NULL; step->read_only = true; diff --git a/src/backend/pgxc/plan/planner.c b/src/backend/pgxc/plan/planner.c index 652008b..52f3a0a 100644 --- a/src/backend/pgxc/plan/planner.c +++ b/src/backend/pgxc/plan/planner.c @@ -1903,7 +1903,6 @@ makeRemoteQuery(void) result->sql_statement = NULL; result->exec_nodes = NULL; result->combine_type = COMBINE_TYPE_NONE; - result->simple_aggregates = NIL; result->sort = NULL; result->distinct = NULL; result->read_only = true; @@ -2018,192 +2017,6 @@ get_plan_combine_type(Query *query, char baselocatortype) return COMBINE_TYPE_NONE; } - -/* - * Get list of simple aggregates used. - */ -static List * -get_simple_aggregates(Query * query) -{ - List *simple_agg_list = NIL; - - /* Check for simple multi-node aggregate */ - if (query->hasAggs) - { - ListCell *lc; - int column_pos = 0; - - foreach (lc, query->targetList) - { - TargetEntry *tle = (TargetEntry *) lfirst(lc); - - if (IsA(tle->expr, Aggref)) - { - /*PGXC borrowed this code from nodeAgg.c, see ExecInitAgg()*/ - SimpleAgg *simple_agg; - Aggref *aggref = (Aggref *) tle->expr; - HeapTuple aggTuple; - Form_pg_aggregate aggform; - Oid aggcollecttype; - AclResult aclresult; - Oid transfn_oid, - finalfn_oid; - Expr *transfnexpr, - *finalfnexpr; - Datum textInitVal; - - simple_agg = makeNode(SimpleAgg); - simple_agg->column_pos = column_pos; - initStringInfo(&simple_agg->valuebuf); - simple_agg->aggref = aggref; - - aggTuple = SearchSysCache(AGGFNOID, - ObjectIdGetDatum(aggref->aggfnoid), - 0, 0, 0); - if (!HeapTupleIsValid(aggTuple)) - elog(ERROR, "cache lookup failed for aggregate %u", - aggref->aggfnoid); - aggform = (Form_pg_aggregate) GETSTRUCT(aggTuple); - - /* Check permission to call aggregate function */ - aclresult = pg_proc_aclcheck(aggref->aggfnoid, GetUserId(), - ACL_EXECUTE); - if (aclresult != ACLCHECK_OK) - aclcheck_error(aclresult, ACL_KIND_PROC, - get_func_name(aggref->aggfnoid)); - - simple_agg->transfn_oid = transfn_oid = aggform->aggcollectfn; - simple_agg->finalfn_oid = finalfn_oid = aggform->aggfinalfn; - - /* Check that aggregate owner has permission to call component fns */ - { - HeapTuple procTuple; - Oid aggOwner; - - procTuple = SearchSysCache(PROCOID, - ObjectIdGetDatum(aggref->aggfnoid), - 0, 0, 0); - if (!HeapTupleIsValid(procTuple)) - elog(ERROR, "cache lookup failed for function %u", - aggref->aggfnoid); - aggOwner = ((Form_pg_proc) GETSTRUCT(procTuple))->proowner; - ReleaseSysCache(procTuple); - - aclresult = pg_proc_aclcheck(transfn_oid, aggOwner, - ACL_EXECUTE); - if (aclresult != ACLCHECK_OK) - aclcheck_error(aclresult, ACL_KIND_PROC, - get_func_name(transfn_oid)); - if (OidIsValid(finalfn_oid)) - { - aclresult = pg_proc_aclcheck(finalfn_oid, aggOwner, - ACL_EXECUTE); - if (aclresult != ACLCHECK_OK) - aclcheck_error(aclresult, ACL_KIND_PROC, - get_func_name(finalfn_oid)); - } - } - - /* resolve actual type of transition state, if polymorphic */ - aggcollecttype = aggform->aggcollecttype; - - /* build expression trees using actual argument & result types */ - build_aggregate_fnexprs(&aggform->aggtranstype, - 1, - aggcollecttype, - aggref->aggtype, - transfn_oid, - finalfn_oid, - &transfnexpr, - &finalfnexpr); - - /* Get InputFunction info for transition result */ - { - Oid typinput; - - getTypeInputInfo(aggform->aggtranstype, &typinput, &simple_agg->argioparam); - fmgr_info(typinput, &simple_agg->arginputfn); - } - - /* Get InputFunction info for result */ - { - Oid typoutput; - bool typvarlena; - - getTypeOutputInfo(simple_agg->aggref->aggtype, &typoutput, &typvarlena); - fmgr_info(typoutput, &simple_agg->resoutputfn); - } - - fmgr_info(transfn_oid, &simple_agg->transfn); - simple_agg->transfn.fn_expr = (Node *) transfnexpr; - - if (OidIsValid(finalfn_oid)) - { - fmgr_info(finalfn_oid, &simple_agg->finalfn); - simple_agg->finalfn.fn_expr = (Node *) finalfnexpr; - } - - get_typlenbyval(aggref->aggtype, - &simple_agg->resulttypeLen, - &simple_agg->resulttypeByVal); - get_typlenbyval(aggcollecttype, - &simple_agg->transtypeLen, - &simple_agg->transtypeByVal); - - /* - * initval is potentially null, so don't try to access it as a struct - * field. Must do it the hard way with SysCacheGetAttr. - */ - textInitVal = SysCacheGetAttr(AGGFNOID, aggTuple, - Anum_pg_aggregate_agginitcollect, - &simple_agg->initValueIsNull); - - if (simple_agg->initValueIsNull) - simple_agg->initValue = (Datum) 0; - else - { - Oid typinput, - typioparam; - char *strInitVal; - Datum initVal; - - getTypeInputInfo(aggcollecttype, &typinput, &typioparam); - strInitVal = TextDatumGetCString(textInitVal); - initVal = OidInputFunctionCall(typinput, strInitVal, - typioparam, -1); - pfree(strInitVal); - simple_agg->initValue = initVal; - } - - /* - * If the transfn is strict and the initval is NULL, make sure trans - * type and collect type are the same (or at least binary-compatible), - * so that it's OK to use the first input value as the initial - * transValue. This should have been checked at agg definition time, - * but just in case... - */ - if (simple_agg->transfn.fn_strict && simple_agg->initValueIsNull) - { - if (!IsBinaryCoercible(aggform->aggtranstype, aggcollecttype)) - ereport(ERROR, - (errcode(ERRCODE_INVALID_FUNCTION_DEFINITION), - errmsg("aggregate %u needs to have compatible transition type and collection type", - aggref->aggfnoid))); - } - - /* PGXCTODO distinct support */ - - ReleaseSysCache(aggTuple); - - simple_agg_list = lappend(simple_agg_list, simple_agg); - } - column_pos++; - } - } - return simple_agg_list; -} - - /* * add_sort_column --- utility subroutine for building sort info arrays * @@ -2966,14 +2779,6 @@ pgxc_planner(Query *query, int cursorOptions, ParamListInfo boundParams) query_step->combine_type = get_plan_combine_type( query, query_step->exec_nodes->baselocatortype); - /* Set up simple aggregates */ - /* PGXCTODO - we should detect what types of aggregates are used. - * in some cases we can avoid the final step and merely proxy results - * (when there is only one data node involved) instead of using - * coordinator consolidation. At the moment this is needed for AVG() - */ - query_step->simple_aggregates = get_simple_aggregates(query); - /* * Add sorting to the step */ @@ -3000,7 +2805,7 @@ pgxc_planner(Query *query, int cursorOptions, ParamListInfo boundParams) * distribution the table has. */ if (query->commandType == CMD_SELECT - && (query->groupClause || query->hasWindowFuncs || query->hasRecursive)) + && (query->hasAggs || query->groupClause || query->hasWindowFuncs || query->hasRecursive)) { result = standard_planner(query, cursorOptions, boundParams); return result; @@ -3092,8 +2897,6 @@ free_query_step(RemoteQuery *query_step) if (query_step->exec_nodes->primarynodelist) list_free(query_step->exec_nodes->primarynodelist); } - if (query_step->simple_aggregates != NULL) - list_free_deep(query_step->simple_aggregates); pfree(query_step); } diff --git a/src/backend/pgxc/pool/execRemote.c b/src/backend/pgxc/pool/execRemote.c index 03482a0..ae0d813 100644 --- a/src/backend/pgxc/pool/execRemote.c +++ b/src/backend/pgxc/pool/execRemote.c @@ -223,7 +223,6 @@ CreateResponseCombiner(int node_count, CombineType combine_type) combiner->rowBuffer = NIL; combiner->tapenodes = NULL; combiner->initAggregates = true; - combiner->simple_aggregates = NULL; combiner->copy_file = NULL; return combiner; @@ -257,146 +256,6 @@ parse_row_count(const char *message, size_t len, uint64 *rowcount) } /* - * Initialize the collection value, when agregation is first set up, or for a - * new group (grouping support is not implemented yet) - */ -static void -initialize_collect_aggregates(SimpleAgg *simple_agg) -{ - if (simple_agg->initValueIsNull) - simple_agg->collectValue = simple_agg->initValue; - else - simple_agg->collectValue = datumCopy(simple_agg->initValue, - simple_agg->transtypeByVal, - simple_agg->transtypeLen); - simple_agg->noCollectValue = simple_agg->initValueIsNull; - simple_agg->collectValueNull = simple_agg->initValueIsNull; -} - -/* - * Finalize the aggregate after current group or entire relation is processed - * (grouping support is not implemented yet) - */ -static void -finalize_collect_aggregates(SimpleAgg *simple_agg, Datum *resultVal, bool *resultIsNull) -{ - /* - * Apply the agg's finalfn if one is provided, else return collectValue. - */ - if (OidIsValid(simple_agg->finalfn_oid)) - { - FunctionCallInfoData fcinfo; - - InitFunctionCallInfoData(fcinfo, &(simple_agg->finalfn), 1, - (void *) simple_agg, NULL); - fcinfo.arg[0] = simple_agg->collectValue; - fcinfo.argnull[0] = simple_agg->collectValueNull; - if (fcinfo.flinfo->fn_strict && simple_agg->collectValueNull) - { - /* don't call a strict function with NULL inputs */ - *resultVal = (Datum) 0; - *resultIsNull = true; - } - else - { - *resultVal = FunctionCallInvoke(&fcinfo); - *resultIsNull = fcinfo.isnull; - } - } - else - { - *resultVal = simple_agg->collectValue; - *resultIsNull = simple_agg->collectValueNull; - } -} - -/* - * Given new input value(s), advance the transition function of an aggregate. - * - * The new values (and null flags) have been preloaded into argument positions - * 1 and up in fcinfo, so that we needn't copy them again to pass to the - * collection function. No other fields of fcinfo are assumed valid. - * - * It doesn't matter which memory context this is called in. - */ -static void -advance_collect_function(SimpleAgg *simple_agg, FunctionCallInfoData *fcinfo) -{ - Datum newVal; - - if (simple_agg->transfn.fn_strict) - { - /* - * For a strict transfn, nothing happens when there's a NULL input; we - * just keep the prior transValue. - */ - if (fcinfo->argnull[1]) - return; - if (simple_agg->noCollectValue) - { - /* - * result has not been initialized - * We must copy the datum into result if it is pass-by-ref. We - * do not need to pfree the old result, since it's NULL. - * PGXCTODO: in case the transition result type is different from - * collection result type, this code would not work, since we are - * assigning datum of one type to another. For this code to work the - * input and output of collection function needs to be binary - * compatible which is not. So, either check in AggregateCreate, - * that the input and output of collection function are binary - * coercible or set the initial values something non-null or change - * this code - */ - simple_agg->collectValue = datumCopy(fcinfo->arg[1], - simple_agg->transtypeByVal, - simple_agg->transtypeLen); - simple_agg->collectValueNull = false; - simple_agg->noCollectValue = false; - return; - } - if (simple_agg->collectValueNull) - { - /* - * Don't call a strict function with NULL inputs. Note it is - * possible to get here despite the above tests, if the transfn is - * strict *and* returned a NULL on a prior cycle. If that happens - * we will propagate the NULL all the way to the end. - */ - return; - } - } - - /* - * OK to call the transition function - */ - InitFunctionCallInfoData(*fcinfo, &(simple_agg->transfn), 2, (void *) simple_agg, NULL); - fcinfo->arg[0] = simple_agg->collectValue; - fcinfo->argnull[0] = simple_agg->collectValueNull; - newVal = FunctionCallInvoke(fcinfo); - - /* - * If pass-by-ref datatype, must copy the new value into aggcontext and - * pfree the prior transValue. But if transfn returned a pointer to its - * first input, we don't need to do anything. - */ - if (!simple_agg->transtypeByVal && - DatumGetPointer(newVal) != DatumGetPointer(simple_agg->collectValue)) - { - if (!fcinfo->isnull) - { - newVal = datumCopy(newVal, - simple_agg->transtypeByVal, - simple_agg->transtypeLen); - } - if (!simple_agg->collectValueNull) - pfree(DatumGetPointer(simple_agg->collectValue)); - } - - simple_agg->collectValue = newVal; - simple_agg->collectValueNull = fcinfo->isnull; -} - -/* * Convert RowDescription message to a TupleDesc */ static TupleDesc @@ -460,64 +319,6 @@ create_tuple_desc(char *msg_body, size_t len) return result; } -static void -exec_simple_aggregates(RemoteQueryState *combiner, TupleTableSlot *slot) -{ - ListCell *lc; - - Assert(combiner->simple_aggregates); - Assert(!TupIsNull(slot)); - - if (combiner->initAggregates) - { - foreach (lc, combiner->simple_aggregates) - initialize_collect_aggregates((SimpleAgg *) lfirst(lc)); - - combiner->initAggregates = false; - } - - foreach (lc, combiner->simple_aggregates) - { - SimpleAgg *simple_agg = (SimpleAgg *) lfirst(lc); - FunctionCallInfoData fcinfo; - int attr = simple_agg->column_pos; - - slot_getsomeattrs(slot, attr + 1); - fcinfo.arg[1] = slot->tts_values[attr]; - fcinfo.argnull[1] = slot->tts_isnull[attr]; - - advance_collect_function(simple_agg, &fcinfo); - } -} - -static void -finish_simple_aggregates(RemoteQueryState *combiner, TupleTableSlot *slot) -{ - ListCell *lc; - ExecClearTuple(slot); - - /* - * Aggregates may not been initialized if no rows has been received - * from the data nodes because of HAVING clause. - * In this case finish_simple_aggregates() should return empty slot - */ - if (!combiner->initAggregates) - { - foreach (lc, combiner->simple_aggregates) - { - SimpleAgg *simple_agg = (SimpleAgg *) lfirst(lc); - int attr = simple_agg->column_pos; - - finalize_collect_aggregates(simple_agg, - slot->tts_values + attr, - slot->tts_isnull + attr); - } - ExecStoreVirtualTuple(slot); - /* To prevent aggregates get finalized again */ - combiner->initAggregates = true; - } -} - /* * Handle CopyOutCommandComplete ('c') message from a data node connection */ @@ -1034,7 +835,6 @@ ValidateAndResetCombiner(RemoteQueryState *combiner) combiner->currentRow.msgnode = 0; combiner->rowBuffer = NIL; combiner->tapenodes = NULL; - combiner->simple_aggregates = NULL; combiner->copy_file = NULL; return valid; @@ -2913,7 +2713,6 @@ ExecInitRemoteQuery(RemoteQuery *node, EState *estate, int eflags) remotestate = CreateResponseCombiner(0, node->combine_type); remotestate->ss.ps.plan = (Plan *) node; remotestate->ss.ps.state = estate; - remotestate->simple_aggregates = node->simple_aggregates; remotestate->ss.ps.qual = (List *) ExecInitExpr((Expr *) node->scan.plan.qual, @@ -2990,6 +2789,8 @@ ExecInitRemoteQuery(RemoteQuery *node, EState *estate, int eflags) ExecAssignExprContext(estate, &remotestate->ss.ps); } } + else if (remotestate->ss.ps.qual) + ExecAssignExprContext(estate, &remotestate->ss.ps); if (innerPlan(node)) innerPlanState(remotestate) = ExecInitNode(innerPlan(node), estate, eflags); @@ -3691,6 +3492,8 @@ ExecRemoteQuery(RemoteQueryState *node) TupleTableSlot *resultslot = node->ss.ps.ps_ResultTupleSlot; TupleTableSlot *scanslot = node->ss.ss_ScanTupleSlot; bool have_tuple = false; + List *qual = node->ss.ps.qual; + ExprContext *econtext = node->ss.ps.ps_ExprContext; if (!node->query_Done) { @@ -3757,7 +3560,15 @@ handle_results: while (tuplesort_gettupleslot((Tuplesortstate *) node->tuplesortstate, true, scanslot)) { - have_tuple = true; + if (qual) + econtext->ecxt_scantuple = scanslot; + if (!qual || ExecQual(qual, econtext, false)) + have_tuple = true; + else + { + have_tuple = false; + continue; + } /* * If DISTINCT is specified and current tuple matches to * previous skip it and get next one. @@ -3791,16 +3602,9 @@ handle_results: { while (FetchTuple(node, scanslot) && !TupIsNull(scanslot)) { - if (node->simple_aggregates) - { - /* - * Advance aggregate functions and allow to read up next - * data row message and get tuple in the same slot on - * next iteration - */ - exec_simple_aggregates(node, scanslot); - } - else + if (qual) + econtext->ecxt_scantuple = scanslot; + if (!qual || ExecQual(qual, econtext, false)) { /* * Receive current slot and read up next data row @@ -3814,17 +3618,6 @@ handle_results: } } - /* - * We may need to finalize aggregates - */ - if (node->simple_aggregates) - { - finish_simple_aggregates(node, resultslot); - - if (!TupIsNull(resultslot)) - have_tuple = true; - } - if (!have_tuple) /* report end of scan */ ExecClearTuple(resultslot); } diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index 5f5b947..ad33074 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -79,7 +79,6 @@ typedef enum NodeTag * TAGS FOR PGXC NODES (planner.h, locator.h) */ T_ExecNodes, - T_SimpleAgg, T_SimpleSort, T_SimpleDistinct, T_RemoteQuery, diff --git a/src/include/pgxc/execRemote.h b/src/include/pgxc/execRemote.h index fb9232f..e522ba9 100644 --- a/src/include/pgxc/execRemote.h +++ b/src/include/pgxc/execRemote.h @@ -101,13 +101,6 @@ typedef struct RemoteQueryState * to initialize collecting of aggregates from the DNs */ bool initAggregates; - /* - * PGXCTODO - - * we should get rid of the simple_aggregates member, that should work - * through Agg node and grouping_planner should take care of optimizing it - * to the fullest - */ - List *simple_aggregates; /* description of aggregate functions */ void *tuplesortstate; /* for merge sort */ /* Simple DISTINCT support */ FmgrInfo *eqfunctions; /* functions to compare tuples */ diff --git a/src/include/pgxc/planner.h b/src/include/pgxc/planner.h index 96a2a68..1a18c9f 100644 --- a/src/include/pgxc/planner.h +++ b/src/include/pgxc/planner.h @@ -93,7 +93,6 @@ typedef struct char *sql_statement; ExecNodes *exec_nodes; /* List of Datanodes where to launch query */ CombineType combine_type; - List *simple_aggregates; /* simple aggregate to combine on this step */ SimpleSort *sort; SimpleDistinct *distinct; bool read_only; /* do not use 2PC when committing read only steps */ @@ -122,73 +121,6 @@ typedef struct char *join_condition; } RemoteQuery; - -/* - * For handling simple aggregates (no group by present) - * For now, only MAX will be supported. - */ -typedef enum -{ - AGG_TYPE_MAX, - AGG_TYPE_MIN, - AGG_TYPE_COUNT, - AGG_TYPE_SUM, - AGG_TYPE_AVG -} SimpleAggType; - - -/* For handling simple aggregates */ -typedef struct -{ - NodeTag type; - int column_pos; /* Only use 1 for now */ - Aggref *aggref; - Oid transfn_oid; - Oid finalfn_oid; - - /* Input Functions, to parse arguments coming from the data nodes */ - FmgrInfo arginputfn; - Oid argioparam; - - /* Output Function, to encode result to present to client */ - FmgrInfo resoutputfn; - - /* - * fmgr lookup data for transfer functions --- only valid when - * corresponding oid is not InvalidOid. Note in particular that fn_strict - * flags are kept here. - */ - FmgrInfo transfn; - FmgrInfo finalfn; - - /* - * initial value from pg_aggregate entry - */ - Datum initValue; - bool initValueIsNull; - - /* - * We need the len and byval info for the agg's input, result, and - * transition data types in order to know how to copy/delete values. - */ - int16 inputtypeLen, - resulttypeLen, - transtypeLen; - bool inputtypeByVal, - resulttypeByVal, - transtypeByVal; - - /* - * State of current group - */ - bool noCollectValue; - Datum collectValue; - bool collectValueNull; - - /* a value buffer to avoid multiple allocations */ - StringInfoData valuebuf; -} SimpleAgg; - typedef struct { bool partitioned_replicated; diff --git a/src/test/regress/expected/create_index_1.out b/src/test/regress/expected/create_index_1.out index ab3807c..67d3939 100644 --- a/src/test/regress/expected/create_index_1.out +++ b/src/test/regress/expected/create_index_1.out @@ -244,10 +244,12 @@ LINE 1: SELECT count(*) FROM gcircle_tbl WHERE f1 && '<(500,500),500... ^ EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl WHERE f1 <@ box '(0,0,100,100)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl WHERE f1 <@ box '(0,0,100,100)'; count @@ -257,10 +259,12 @@ SELECT count(*) FROM point_tbl WHERE f1 <@ box '(0,0,100,100)'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl WHERE box '(0,0,100,100)' @> f1; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl WHERE box '(0,0,100,100)' @> f1; count @@ -270,10 +274,12 @@ SELECT count(*) FROM point_tbl WHERE box '(0,0,100,100)' @> f1; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)'; count @@ -283,10 +289,12 @@ SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50, EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl WHERE f1 <@ circle '<(50,50),50>'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl WHERE f1 <@ circle '<(50,50),50>'; count @@ -296,10 +304,12 @@ SELECT count(*) FROM point_tbl WHERE f1 <@ circle '<(50,50),50>'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl p WHERE p.f1 << '(0.0, 0.0)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl p WHERE p.f1 << '(0.0, 0.0)'; count @@ -309,10 +319,12 @@ SELECT count(*) FROM point_tbl p WHERE p.f1 << '(0.0, 0.0)'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl p WHERE p.f1 >> '(0.0, 0.0)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl p WHERE p.f1 >> '(0.0, 0.0)'; count @@ -322,10 +334,12 @@ SELECT count(*) FROM point_tbl p WHERE p.f1 >> '(0.0, 0.0)'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl p WHERE p.f1 <^ '(0.0, 0.0)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl p WHERE p.f1 <^ '(0.0, 0.0)'; count @@ -335,10 +349,12 @@ SELECT count(*) FROM point_tbl p WHERE p.f1 <^ '(0.0, 0.0)'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl p WHERE p.f1 >^ '(0.0, 0.0)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl p WHERE p.f1 >^ '(0.0, 0.0)'; count @@ -348,10 +364,12 @@ SELECT count(*) FROM point_tbl p WHERE p.f1 >^ '(0.0, 0.0)'; EXPLAIN (COSTS OFF) SELECT count(*) FROM point_tbl p WHERE p.f1 ~= '(-5, -12)'; - QUERY PLAN ---------------------------------- - Data Node Scan (Node Count [1]) -(1 row) + QUERY PLAN +--------------------------------------------- + Aggregate + -> Materialize + -> Data Node Scan (Node Count [1]) +(3 rows) SELECT count(*) FROM point_tbl p WHERE p.f1 ~= '(-5, -12)'; count ----------------------------------------------------------------------- Summary of changes: src/backend/executor/nodeAgg.c | 34 ++-- src/backend/nodes/copyfuncs.c | 38 ---- src/backend/optimizer/plan/createplan.c | 18 ++- src/backend/parser/analyze.c | 1 - src/backend/pgxc/plan/planner.c | 199 +--------------------- src/backend/pgxc/pool/execRemote.c | 239 ++------------------------ src/include/nodes/nodes.h | 1 - src/include/pgxc/execRemote.h | 7 - src/include/pgxc/planner.h | 68 -------- src/test/regress/expected/create_index_1.out | 90 ++++++---- 10 files changed, 102 insertions(+), 593 deletions(-) hooks/post-receive -- Postgres-XC |