operator new alignment assumptions

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

operator new alignment assumptions

Yvan Roux via cfe-dev
Since r283789 (re-commit of r283722, "P0035R4: Semantic analysis and
code generation for C++17 overaligned allocation"),
CodeGenFunction::EmitCXXNewExpr (lib/CodeGen/CGExprCXX.cpp) contains

>     // If this was a call to a global replaceable allocation function that does
>     // not take an alignment argument, the allocator is known to produce
>     // storage that's suitably aligned for any object that fits, up to a known
>     // threshold. Otherwise assume it's suitably aligned for the allocated type.
>     CharUnits allocationAlign = allocAlign;
>     if (!E->passAlignment() &&
>         allocator->isReplaceableGlobalAllocationFunction()) {
>       unsigned AllocatorAlign = llvm::PowerOf2Floor(std::min<uint64_t>(
>           Target.getNewAlign(), getContext().getTypeSize(allocType)));
>       allocationAlign = std::max(
>           allocationAlign, getContext().toCharUnitsFromBits(AllocatorAlign));
>     }

which I have two questions about:

*  Why restrict this to global replaceable allocation functions?  My
interpretation of the standard up to C++14 is that all allocation
functions consistently have the same requirements wrt alignment ("The
pointer returned shall be suitably aligned so that it can be converted
to a pointer of any complete object type with a fundamental alignment
requirement", [basic.stc.dynamic.allocation]), and that P0035R4
(presumably unintentionally) makes it confusing that this consistency
still holds (cf. my question at
<https://groups.google.com/a/isocpp.org/forum/?fromgroups#!topic/std-discussion/mTZ4sB5P1Sg>
"Confusing wording in P0035R4, 'any suitable complete object type'").

*  At least with the wording from P0035R4, it is unclear to me what the
alignment requirements in

> struct S { char a[25]; };
> S * func() { return new S(); }

are.  When compiled at -O for x86_64-unknown-linux-gnu, Clang trunk
apparently assumes that the pointer returned from operator new is
16-byte aligned (writing to it with movaps).  But the requirements for
that operator new in [new.delete.single] merely states that it
"allocates storage suitably aligned to represent any object of that
size", and I don't think there is a type of size 25 with a stricter
alignment requirement than 1 for that target.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: operator new alignment assumptions

Yvan Roux via cfe-dev
On 13 February 2017 at 06:55, Stephan Bergmann <[hidden email]> wrote:
Since r283789 (re-commit of r283722, "P0035R4: Semantic analysis and code generation for C++17 overaligned allocation"), CodeGenFunction::EmitCXXNewExpr (lib/CodeGen/CGExprCXX.cpp) contains

    // If this was a call to a global replaceable allocation function that does
    // not take an alignment argument, the allocator is known to produce
    // storage that's suitably aligned for any object that fits, up to a known
    // threshold. Otherwise assume it's suitably aligned for the allocated type.
    CharUnits allocationAlign = allocAlign;
    if (!E->passAlignment() &&
        allocator->isReplaceableGlobalAllocationFunction()) {
      unsigned AllocatorAlign = llvm::PowerOf2Floor(std::min<uint64_t>(
          Target.getNewAlign(), getContext().getTypeSize(allocType)));
      allocationAlign = std::max(
          allocationAlign, getContext().toCharUnitsFromBits(AllocatorAlign));
    }

which I have two questions about:

*  Why restrict this to global replaceable allocation functions?  My interpretation of the standard up to C++14 is that all allocation functions consistently have the same requirements wrt alignment ("The pointer returned shall be suitably aligned so that it can be converted
to a pointer of any complete object type with a fundamental alignment requirement", [basic.stc.dynamic.allocation]), and that P0035R4 (presumably unintentionally) makes it confusing that this consistency still holds (cf. my question at <https://groups.google.com/a/isocpp.org/forum/?fromgroups#!topic/std-discussion/mTZ4sB5P1Sg> "Confusing wording in P0035R4, 'any suitable complete object type'").

There is no reasonable assumption you can make about the alignment provided by any other 'operator new' function, other than assuming that the user got it right: that it produces a pointer that is suitably aligned for the allocated object. For instance, a class-specific allocator for a class with 1-byte alignment might reasonably produce 1-byte-aligned pointers regardless of the allocation size. Breaking such code merely because poor wording in the C++ standard may have allowed us to do so seems unlikely to make anyone happier.

*  At least with the wording from P0035R4, it is unclear to me what the alignment requirements in

struct S { char a[25]; };
S * func() { return new S(); }

are.  When compiled at -O for x86_64-unknown-linux-gnu, Clang trunk apparently assumes that the pointer returned from operator new is 16-byte aligned (writing to it with movaps).  But the requirements for that operator new in [new.delete.single] merely states that it "allocates storage suitably aligned to represent any object of that size", and I don't think there is a type of size 25 with a stricter alignment requirement than 1 for that target.

Yes, you're right that this is at best unclear and at worst says that our assumption is not valid for allocations that are not a multiple of the allocator alignment. This is the subject of C++ core issue 2207 (http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2207, but there's no information on the public version of the issues list yet); there generally seemed to be consensus in committee discussion that global non-array non-aligned operator new should provide sufficient storage for any object with a fundamental alignment that fits in the storage, not just for fundamental alignments that divide the allocated size.

For instance, we want to guarantee that this sort of thing works:

S *p = func();
static_assert(sizeof(long double) <= 25);
new (p->a) long double; // ok, 'a' is suitably aligned for 'long double'

Is this assumption causing problems in practice?

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: operator new alignment assumptions

Yvan Roux via cfe-dev
On 02/16/2017 02:34 AM, Richard Smith wrote:

> On 13 February 2017 at 06:55, Stephan Bergmann <[hidden email]
> <mailto:[hidden email]>> wrote:
>     *  At least with the wording from P0035R4, it is unclear to me what
>     the alignment requirements in
>
>         struct S { char a[25]; };
>         S * func() { return new S(); }
>
>
>     are.  When compiled at -O for x86_64-unknown-linux-gnu, Clang trunk
>     apparently assumes that the pointer returned from operator new is
>     16-byte aligned (writing to it with movaps).  But the requirements
>     for that operator new in [new.delete.single] merely states that it
>     "allocates storage suitably aligned to represent any object of that
>     size", and I don't think there is a type of size 25 with a stricter
>     alignment requirement than 1 for that target.
>
>
> Yes, you're right that this is at best unclear and at worst says that
> our assumption is not valid for allocations that are not a multiple of
> the allocator alignment. This is the subject of C++ core issue 2207
> (http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2207, but
> there's no information on the public version of the issues list yet);
> there generally seemed to be consensus in committee discussion that
> global non-array non-aligned operator new should provide sufficient
> storage for any object with a fundamental alignment that fits in the
> storage, not just for fundamental alignments that divide the allocated size.
>
> For instance, we want to guarantee that this sort of thing works:
>
> S *p = func();
> static_assert(sizeof(long double) <= 25);
> new (p->a) long double; // ok, 'a' is suitably aligned for 'long double'
>
> Is this assumption causing problems in practice?

Don't know of any.  I stumbled across all this when seeing that the
Firebird DBMS has a global operator new replacement that generally does
merely 8-byte alignment on x86-64.  But I haven't come across an issue
related to this fits--vs.--matches-exactly size assumption.

And thanks for the pointer to issue 2207.  Hadn't spotted that one before.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: operator new alignment assumptions

Yvan Roux via cfe-dev
On 16/02/17 13:34, Stephan Bergmann via cfe-dev wrote:

> On 02/16/2017 02:34 AM, Richard Smith wrote:
>> On 13 February 2017 at 06:55, Stephan Bergmann <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>     *  At least with the wording from P0035R4, it is unclear to me what
>>     the alignment requirements in
>>
>>         struct S { char a[25]; };
>>         S * func() { return new S(); }
>>
>>
>>     are.  When compiled at -O for x86_64-unknown-linux-gnu, Clang trunk
>>     apparently assumes that the pointer returned from operator new is
>>     16-byte aligned (writing to it with movaps).  But the requirements
>>     for that operator new in [new.delete.single] merely states that it
>>     "allocates storage suitably aligned to represent any object of that
>>     size", and I don't think there is a type of size 25 with a stricter
>>     alignment requirement than 1 for that target.
>>
>>
>> Yes, you're right that this is at best unclear and at worst says that
>> our assumption is not valid for allocations that are not a multiple of
>> the allocator alignment. This is the subject of C++ core issue 2207
>> (http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2207, but
>> there's no information on the public version of the issues list yet);
>> there generally seemed to be consensus in committee discussion that
>> global non-array non-aligned operator new should provide sufficient
>> storage for any object with a fundamental alignment that fits in the
>> storage, not just for fundamental alignments that divide the allocated
>> size.
>>
>> For instance, we want to guarantee that this sort of thing works:
>>
>> S *p = func();
>> static_assert(sizeof(long double) <= 25);
>> new (p->a) long double; // ok, 'a' is suitably aligned for 'long double'
>>
>> Is this assumption causing problems in practice?
>
> Don't know of any.  I stumbled across all this when seeing that the
> Firebird DBMS has a global operator new replacement that generally does
> merely 8-byte alignment on x86-64.  But I haven't come across an issue
> related to this fits--vs.--matches-exactly size assumption.

Revisiting this now, I think I /do/ see a problem in practice after all
with Clang making too aggressive assumptions about the aligned-ness of
storage returned by "plain" global operator new, at least with the
Firebird 3.0 that is bundled by LibreOffice:

Firebird replaces that "plain" global operator new, which gets called
with an argument of 24, and recent Clang (on both Linux and macOS
x86-64) uses movaps (requiring 16-byte aligned memory) on the result.
(And the result happens to be only 8-byte aligned, causing a SEGV.)

According to the x86-64 SysV ABI, a struct with a 16-byte alignment
requirement would need to have a size that is a multiple of 16 bytes, so
Firebird's operator new replacement should be OK returning merely 8-byte
aligned storage for a request of 24 bytes.

(It is unfortunate that
<http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2207> still
only says "[Detailed description pending.]")
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: operator new alignment assumptions

Yvan Roux via cfe-dev


On Apr 16, 2018, at 4:50 AM, Stephan Bergmann via cfe-dev <[hidden email]> wrote:

On 16/02/17 13:34, Stephan Bergmann via cfe-dev wrote:
On 02/16/2017 02:34 AM, Richard Smith wrote:
On 13 February 2017 at 06:55, Stephan Bergmann <[hidden email]
<[hidden email]>> wrote:
    *  At least with the wording from P0035R4, it is unclear to me what
    the alignment requirements in

        struct S { char a[25]; };
        S * func() { return new S(); }


    are.  When compiled at -O for x86_64-unknown-linux-gnu, Clang trunk
    apparently assumes that the pointer returned from operator new is
    16-byte aligned (writing to it with movaps).  But the requirements
    for that operator new in [new.delete.single] merely states that it
    "allocates storage suitably aligned to represent any object of that
    size", and I don't think there is a type of size 25 with a stricter
    alignment requirement than 1 for that target.


Yes, you're right that this is at best unclear and at worst says that
our assumption is not valid for allocations that are not a multiple of
the allocator alignment. This is the subject of C++ core issue 2207
(http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2207, but
there's no information on the public version of the issues list yet);
there generally seemed to be consensus in committee discussion that
global non-array non-aligned operator new should provide sufficient
storage for any object with a fundamental alignment that fits in the
storage, not just for fundamental alignments that divide the allocated size.

For instance, we want to guarantee that this sort of thing works:

S *p = func();
static_assert(sizeof(long double) <= 25);
new (p->a) long double; // ok, 'a' is suitably aligned for 'long double'

Is this assumption causing problems in practice?
Don't know of any.  I stumbled across all this when seeing that the Firebird DBMS has a global operator new replacement that generally does merely 8-byte alignment on x86-64.  But I haven't come across an issue related to this fits--vs.--matches-exactly size assumption.

Revisiting this now, I think I /do/ see a problem in practice after all with Clang making too aggressive assumptions about the aligned-ness of storage returned by "plain" global operator new, at least with the Firebird 3.0 that is bundled by LibreOffice:

Firebird replaces that "plain" global operator new, which gets called with an argument of 24, and recent Clang (on both Linux and macOS x86-64) uses movaps (requiring 16-byte aligned memory) on the result. (And the result happens to be only 8-byte aligned, causing a SEGV.)

According to the x86-64 SysV ABI, a struct with a 16-byte alignment requirement would need to have a size that is a multiple of 16 bytes, so Firebird's operator new replacement should be OK returning merely 8-byte aligned storage for a request of 24 bytes.

I don't think this is a valid line of reasoning.  Users can call `operator new` themselves, and they are not required to call it with the `sizeof` some specific type.  It's a pretty common idiom to allocate a header (usually pointer-aligned) + a trailing array (aligned to whatever the element type requires), and in fact that's an ABI-mandated pattern in C++ (albeit with operator new[] — and since it only happens for types with destructors it's very unlikely to trigger on a byte-aligned element type).

John.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev