Postgres do not allow to create many tables with more than 63-symbols prefix

Moved from the pgsql-bugs mailing list [1].

On 6/23/22 07:03, Masahiko Sawada wrote:
> Hi,
>
> On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov
> <a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
>>
>> According to subj you can try to create many tables (induced by the case
>> of partitioned table) with long prefix - see 6727v.sql for reproduction.
>> But now it's impossible because of logic of the makeUniqueTypeName()
>> routine.
>> You get the error:
>> ERROR: could not form array type name for type ...
>>
>> It is very corner case, of course. But solution is easy and short. So,
>> why not to fix? - See the patch in attachment.
>
> While this seems to be a good improvement, I think it's not a bug.
> Probably we cannot backpatch it as it will end up having type names
> defined by different naming rules. I'd suggest discussing it on
> -hackers.
Done.

> Regarding the patch, I think we can merge makeUniqueTypeName() to
> makeArrayTypeName() as there is no caller of makeUniqueTypeName() who
> pass tryOriginal = true.
I partially agree with you. But I have one reason to leave
makeUniqueTypeName() separated:
It may be used in other codes with auto generated types. For example, I
think, the DefineRelation routine should choose composite type instead
of using the same name as the table.

> Also looking at other ChooseXXXName()
> functions, we don't care about integer overflow. Is it better to make
> it consistent with them?
Done.

[1]
https://www.postgresql.org/message-id/flat/121e286f-3796-c9d7-9eab-6fb8e0b9c701%40postgrespro.ru

--
Regards
Andrey Lepikhov
Postgres Professional

Hi,

Thanks for working on this.

>> According to subj you can try to create many tables (induced by the case
> >> of partitioned table) with long prefix - see 6727v.sql for
> reproduction.
> >> But now it's impossible because of logic of the makeUniqueTypeName()
> >> routine.
> >> You get the error:
> >> ERROR: could not form array type name for type ...
> >>
> >> It is very corner case, of course. But solution is easy and short. So,
> >> why not to fix? - See the patch in attachment.
> >
> > While this seems to be a good improvement, I think it's not a bug.
> > Probably we cannot backpatch it as it will end up having type names
> > defined by different naming rules. I'd suggest discussing it on
> > -hackers.
> Done.
>

On Citus extension, we hit a similar issue while creating partitions (over
multiple transactions in parallel). You can see some more discussions on
the related Github issue #5334
<https://github.com/citusdata/citus/issues/5354>. We basically discuss this
behavior on the issue.

I tested this patch with the mentioned issue, and as expected the issue is
resolved.

Also, in general, the patch looks reasonable, following the approach
that ChooseRelationName()
implements makes sense to me as well.

Onder KALACI
Developing the Citus extension @Microsoft

On Fri, Jun 24, 2022 at 2:12 PM Andrey Lepikhov
<a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
>
> Moved from the pgsql-bugs mailing list [1].
>
> On 6/23/22 07:03, Masahiko Sawada wrote:
> > Hi,
> >
> > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov
> > <a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
> >>
> >> According to subj you can try to create many tables (induced by the case
> >> of partitioned table) with long prefix - see 6727v.sql for reproduction.
> >> But now it's impossible because of logic of the makeUniqueTypeName()
> >> routine.
> >> You get the error:
> >> ERROR: could not form array type name for type ...
> >>
> >> It is very corner case, of course. But solution is easy and short. So,
> >> why not to fix? - See the patch in attachment.
> >
> > While this seems to be a good improvement, I think it's not a bug.
> > Probably we cannot backpatch it as it will end up having type names
> > defined by different naming rules. I'd suggest discussing it on
> > -hackers.
> Done.

Thank for updating the patch. Please register this item to the next CF
if not yet.

>
> > Regarding the patch, I think we can merge makeUniqueTypeName() to
> > makeArrayTypeName() as there is no caller of makeUniqueTypeName() who
> > pass tryOriginal = true.
> I partially agree with you. But I have one reason to leave
> makeUniqueTypeName() separated:
> It may be used in other codes with auto generated types. For example, I
> think, the DefineRelation routine should choose composite type instead
> of using the same name as the table.

Okay.

I have one comment on v2 patch:

+ for(;;)
{
- dest[i - 1] = '_';
- strlcpy(dest + i, typeName, NAMEDATALEN - i);
- if (namelen + i >= NAMEDATALEN)
- truncate_identifier(dest, NAMEDATALEN, false);
-
if (!SearchSysCacheExists2(TYPENAMENSP,
- CStringGetDatum(dest),
+ CStringGetDatum(type_name),
ObjectIdGetDatum(typeNamespace)))
- return pstrdup(dest);
+ return type_name;
+
+ /* Previous attempt was failed. Prepare a new one. */
+ pfree(type_name);
+ snprintf(suffix, sizeof(suffix), "%d", ++pass);
+ type_name = makeObjectName("", typeName, suffix);
}

return NULL;

I think it's better to break from the loop instead of returning from
there. That way, we won't need "return NULL".

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/

On 6/27/22 06:38, Masahiko Sawada wrote:
> On Fri, Jun 24, 2022 at 2:12 PM Andrey Lepikhov
> <a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
>> On 6/23/22 07:03, Masahiko Sawada wrote:
>> > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov
>> > <a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
>> >> It is very corner case, of course. But solution is easy and short. So,
>> >> why not to fix? - See the patch in attachment.
>> >
>> > While this seems to be a good improvement, I think it's not a bug.
>> > Probably we cannot backpatch it as it will end up having type names
>> > defined by different naming rules. I'd suggest discussing it on
>> > -hackers.
>> Done.
>
> Thank for updating the patch. Please register this item to the next CF
> if not yet.
Done [1].

>> > Regarding the patch, I think we can merge makeUniqueTypeName() to
>> > makeArrayTypeName() as there is no caller of makeUniqueTypeName() who
>> > pass tryOriginal = true.
>> I partially agree with you. But I have one reason to leave
>> makeUniqueTypeName() separated:
>> It may be used in other codes with auto generated types. For example, I
>> think, the DefineRelation routine should choose composite type instead
>> of using the same name as the table.
>
> Okay.
>
> I have one comment on v2 patch:
>
> + for(;;)
> {
> - dest[i - 1] = '_';
> - strlcpy(dest + i, typeName, NAMEDATALEN - i);
> - if (namelen + i >= NAMEDATALEN)
> - truncate_identifier(dest, NAMEDATALEN, false);
> -
> if (!SearchSysCacheExists2(TYPENAMENSP,
> - CStringGetDatum(dest),
> + CStringGetDatum(type_name),
> ObjectIdGetDatum(typeNamespace)))
> - return pstrdup(dest);
> + return type_name;
> +
> + /* Previous attempt was failed. Prepare a new one. */
> + pfree(type_name);
> + snprintf(suffix, sizeof(suffix), "%d", ++pass);
> + type_name = makeObjectName("", typeName, suffix);
> }
>
> return NULL;
>
> I think it's better to break from the loop instead of returning from
> there. That way, we won't need "return NULL".
Agree. Done.

[1] https://commitfest.postgresql.org/38/3712/

--
Regards
Andrey Lepikhov
Postgres Professional

Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru> writes:
> On 6/27/22 06:38, Masahiko Sawada wrote:
>>>> Regarding the patch, I think we can merge makeUniqueTypeName() to
>>>> makeArrayTypeName() as there is no caller of makeUniqueTypeName() who
>>>> pass tryOriginal = true.

>>> I partially agree with you. But I have one reason to leave
>>> makeUniqueTypeName() separated:
>>> It may be used in other codes with auto generated types. For example, I
>>> think, the DefineRelation routine should choose composite type instead
>>> of using the same name as the table.

No, this is an absolutely horrid idea. The rule that "_foo" means "array
of foo" is quite well known to a lot of client code, and as long as they
don't use any type names approaching NAMEDATALEN, it's solid. So we must
not build backend code that uses "_foo"-like type names for any other
purpose than arrays.

I suspect in fact that the reason we ended up with this orphaned logic
is that somebody pointed out this problem somewhere along the development
of multiranges, whereupon makeMultirangeTypeName was changed to not
use the shared code --- but the breakup of makeArrayTypeName wasn't
undone altogether. It should have been, because it just tempts other
people to make the same wrong choice.

Pushed with re-merging of the code into makeArrayTypeName and some
work on the comments.

regards, tom lane