Using clang internals to preprocess a buffer, crashing with double free

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Using clang internals to preprocess a buffer, crashing with double free

David Chisnall via cfe-dev
Hi, helpful people. I'm trying to use clang internals to run an in-memory buffer through the C preprocessor (string to string), and I'm having trouble. I wonder if one of you kind souls might be able to spot the problem.

I cobbled this code together looking at clang's BufferSourceTest.cpp unit test (for the aspect of how to read from a buffer), and for how to invoke just the preprocessor, I looked at the source code for ISPC (https://github.com/ispc/ispc/blob/master/module.cpp#L2453).

The C++11 code below, linked against clang/llvm 3.9, works just fine if I compile my code with clang (any of 3.5, 3.9, 4.0) or with g++ 4.8.5. But when I build my code with g++ 4.9, 5.4, or 6.2, it seems to compile fine but will crash at runtime during the exit of this function, with the following error:

*** glibc detected *** myapp: double free or corruption (!prev): 0x0000000000a13270 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7da26)[0x2ac395bf8a26]
mylib.so(_ZN5clang22CompilerInvocationBaseD1Ev+0xfa)[0x2ac3936ecdca]
mylib.so(+0x3718e1)[0x2ac3936e88e1]
mylib.so(_ZN5clang16CompilerInstanceD1Ev+0x445)[0x2ac3936d8f65]

This is on Linux. I don't get these errors on OSX, but then again, I am compiling my code with clang there, and it also doesn't seem to crash when I build my code with clang on Linux either. Only with g++, and only when >= 4.9.

The fact that it crashes for some gcc versions but not others makes me suspicious that it's some misinterpretation of C++11 RAAI, like I'm freeing something that is assumed to be owned by the clang::CompilerInstance. But I'm not sure what that would be.

Anyway, here's the code. I've simplified a bit by removing the parts that deal with -D and -I directives.
Have I done something foolish here?


bool preprocess_buffer (const std::string &buffer,   // Input to be preprocessed
                        const std::string &filename, // Name for reporting errors
                        std::string &result)         // <-- want preprocessed output here
{
    std::unique_ptr<llvm::MemoryBuffer> mbuf (llvm::MemoryBuffer::getMemBuffer(buffer, filename));

    clang::CompilerInstance inst;
    inst.createDiagnostics();

    const std::shared_ptr<clang::TargetOptions> &targetopts =
          std::make_shared<clang::TargetOptions>(inst.getTargetOpts());
    targetopts->Triple = llvm::sys::getDefaultTargetTriple();
    clang::TargetInfo *target =
        clang::TargetInfo::CreateTargetInfo(inst.getDiagnostics(), targetopts);
    inst.setTarget(target);

    inst.createFileManager();
    inst.createSourceManager(inst.getFileManager());
    clang::SourceManager &sm = inst.getSourceManager();
    sm.setMainFileID (sm.createFileID(std::move(mbuf), clang::SrcMgr::C_User));

    inst.getPreprocessorOutputOpts().ShowCPP = 1;
    inst.getPreprocessorOutputOpts().ShowMacros = 0;
    inst.getLangOpts().LineComment = 1;

    // Omit handling of -D and -I arguments here... not pertinent to the crash

    inst.createPreprocessor(clang::TU_Prefix);

    llvm::raw_string_ostream ostream(result);
    clang::DoPrintPreprocessedInput (inst.getPreprocessor(),
                                     &ostream, inst.getPreprocessorOutputOpts());
    return true;
}



--
Larry Gritz
[hidden email]


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang internals to preprocess a buffer, crashing with double free

David Chisnall via cfe-dev


On 2/23/17 5:24 PM, Larry Gritz via cfe-dev wrote:

> Hi, helpful people. I'm trying to use clang internals to run an in-memory buffer through the C preprocessor (string to string), and I'm having trouble. I wonder if one of you kind souls might be able to spot the problem.
>
> I cobbled this code together looking at clang's BufferSourceTest.cpp unit test (for the aspect of how to read from a buffer), and for how to invoke just the preprocessor, I looked at the source code for ISPC (https://github.com/ispc/ispc/blob/master/module.cpp#L2453).
>
> The C++11 code below, linked against clang/llvm 3.9, works just fine if I compile my code with clang (any of 3.5, 3.9, 4.0) or with g++ 4.8.5. But when I build my code with g++ 4.9, 5.4, or 6.2, it seems to compile fine but will crash at runtime during the exit of this function, with the following error:
>
> *** glibc detected *** myapp: double free or corruption (!prev): 0x0000000000a13270 ***
> ======= Backtrace: =========
> /lib/x86_64-linux-gnu/libc.so.6(+0x7da26)[0x2ac395bf8a26]
> mylib.so(_ZN5clang22CompilerInvocationBaseD1Ev+0xfa)[0x2ac3936ecdca]
> mylib.so(+0x3718e1)[0x2ac3936e88e1]
> mylib.so(_ZN5clang16CompilerInstanceD1Ev+0x445)[0x2ac3936d8f65]
>
> This is on Linux. I don't get these errors on OSX, but then again, I am compiling my code with clang there, and it also doesn't seem to crash when I build my code with clang on Linux either. Only with g++, and only when >= 4.9.
>
> The fact that it crashes for some gcc versions but not others makes me suspicious that it's some misinterpretation of C++11 RAAI, like I'm freeing something that is assumed to be owned by the clang::CompilerInstance. But I'm not sure what that would be.
>
> Anyway, here's the code. I've simplified a bit by removing the parts that deal with -D and -I directives.
> Have I done something foolish here?
>
>
> bool preprocess_buffer (const std::string &buffer,   // Input to be preprocessed
>                         const std::string &filename, // Name for reporting errors
>                         std::string &result)         // <-- want preprocessed output here
> {
>     std::unique_ptr<llvm::MemoryBuffer> mbuf (llvm::MemoryBuffer::getMemBuffer(buffer, filename));
>
>     clang::CompilerInstance inst;
>     inst.createDiagnostics();
>
>     const std::shared_ptr<clang::TargetOptions> &targetopts =
>           std::make_shared<clang::TargetOptions>(inst.getTargetOpts());

This line ^ is extremely suspicious. It makes a shared pointer, takes a
*reference* to it, and drops it on the floor. To fix it, get rid of the
'const' and '&' from that line.


Jon

>     targetopts->Triple = llvm::sys::getDefaultTargetTriple();
>     clang::TargetInfo *target =
>         clang::TargetInfo::CreateTargetInfo(inst.getDiagnostics(), targetopts);
>     inst.setTarget(target);
>
>     inst.createFileManager();
>     inst.createSourceManager(inst.getFileManager());
>     clang::SourceManager &sm = inst.getSourceManager();
>     sm.setMainFileID (sm.createFileID(std::move(mbuf), clang::SrcMgr::C_User));
>
>     inst.getPreprocessorOutputOpts().ShowCPP = 1;
>     inst.getPreprocessorOutputOpts().ShowMacros = 0;
>     inst.getLangOpts().LineComment = 1;
>
>     // Omit handling of -D and -I arguments here... not pertinent to the crash
>
>     inst.createPreprocessor(clang::TU_Prefix);
>
>     llvm::raw_string_ostream ostream(result);
>     clang::DoPrintPreprocessedInput (inst.getPreprocessor(),
>                                      &ostream, inst.getPreprocessorOutputOpts());
>     return true;
> }
>
>
>
> --
> Larry Gritz
> [hidden email]
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang internals to preprocess a buffer, crashing with double free

David Chisnall via cfe-dev
On Thu, Feb 23, 2017 at 4:57 PM, Jonathan Roelofs via cfe-dev <[hidden email]> wrote:


On 2/23/17 5:24 PM, Larry Gritz via cfe-dev wrote:
Hi, helpful people. I'm trying to use clang internals to run an in-memory buffer through the C preprocessor (string to string), and I'm having trouble. I wonder if one of you kind souls might be able to spot the problem.

I cobbled this code together looking at clang's BufferSourceTest.cpp unit test (for the aspect of how to read from a buffer), and for how to invoke just the preprocessor, I looked at the source code for ISPC (https://github.com/ispc/ispc/blob/master/module.cpp#L2453).

The C++11 code below, linked against clang/llvm 3.9, works just fine if I compile my code with clang (any of 3.5, 3.9, 4.0) or with g++ 4.8.5. But when I build my code with g++ 4.9, 5.4, or 6.2, it seems to compile fine but will crash at runtime during the exit of this function, with the following error:

*** glibc detected *** myapp: double free or corruption (!prev): 0x0000000000a13270 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7da26)[0x2ac395bf8a26]
mylib.so(_ZN5clang22CompilerInvocationBaseD1Ev+0xfa)[0x2ac3936ecdca]
mylib.so(+0x3718e1)[0x2ac3936e88e1]
mylib.so(_ZN5clang16CompilerInstanceD1Ev+0x445)[0x2ac3936d8f65]

This is on Linux. I don't get these errors on OSX, but then again, I am compiling my code with clang there, and it also doesn't seem to crash when I build my code with clang on Linux either. Only with g++, and only when >= 4.9.

The fact that it crashes for some gcc versions but not others makes me suspicious that it's some misinterpretation of C++11 RAAI, like I'm freeing something that is assumed to be owned by the clang::CompilerInstance. But I'm not sure what that would be.

Anyway, here's the code. I've simplified a bit by removing the parts that deal with -D and -I directives.
Have I done something foolish here?


bool preprocess_buffer (const std::string &buffer,   // Input to be preprocessed
                        const std::string &filename, // Name for reporting errors
                        std::string &result)         // <-- want preprocessed output here
{
    std::unique_ptr<llvm::MemoryBuffer> mbuf (llvm::MemoryBuffer::getMemBuffer(buffer, filename));

    clang::CompilerInstance inst;
    inst.createDiagnostics();

    const std::shared_ptr<clang::TargetOptions> &targetopts =
          std::make_shared<clang::TargetOptions>(inst.getTargetOpts());

This line ^ is extremely suspicious. It makes a shared pointer, takes a *reference* to it, and drops it on the floor. To fix it, get rid of the 'const' and '&' from that line.

While I'd drop the '&' for clarity, that's not the bug: the temporary returned by std::make_shared will be lifetime extended here where it's bound (directly) to a reference.

-- James

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang internals to preprocess a buffer, crashing with double free

David Chisnall via cfe-dev

On Feb 23, 2017, at 6:39 PM, James Dennett <[hidden email]> wrote:

On Thu, Feb 23, 2017 at 4:57 PM, Jonathan Roelofs via cfe-dev <[hidden email]> wrote:

    const std::shared_ptr<clang::TargetOptions> &targetopts =
          std::make_shared<clang::TargetOptions>(inst.getTargetOpts());

This line ^ is extremely suspicious. It makes a shared pointer, takes a *reference* to it, and drops it on the floor. To fix it, get rid of the 'const' and '&' from that line.

While I'd drop the '&' for clarity, that's not the bug: the temporary returned by std::make_shared will be lifetime extended here where it's bound (directly) to a reference.

Making that change did not stop the crashes, though I do agree that it's not helpful for clarity.

--
Larry Gritz
[hidden email]



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang internals to preprocess a buffer, crashing with double free

David Chisnall via cfe-dev
In reply to this post by David Chisnall via cfe-dev


On Fri, Feb 24, 2017 at 1:24 AM, Larry Gritz via cfe-dev <[hidden email]> wrote:
Hi, helpful people. I'm trying to use clang internals to run an in-memory buffer through the C preprocessor (string to string), and I'm having trouble. I wonder if one of you kind souls might be able to spot the problem.

I cobbled this code together looking at clang's BufferSourceTest.cpp unit test (for the aspect of how to read from a buffer), and for how to invoke just the preprocessor, I looked at the source code for ISPC (https://github.com/ispc/ispc/blob/master/module.cpp#L2453).

The C++11 code below, linked against clang/llvm 3.9, works just fine if I compile my code with clang (any of 3.5, 3.9, 4.0) or with g++ 4.8.5. But when I build my code with g++ 4.9, 5.4, or 6.2, it seems to compile fine but will crash at runtime during the exit of this function, with the following error:

Try compiling with -fsanitize=address​

​Csaba​

--
GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
The Tao of math: The numbers you can count are not the real numbers.
Life is complex, with real and imaginary parts.
"Ok, it boots. Which means it must be bug-free and perfect. " -- Linus Torvalds
"People disagree with me. I just ignore them." -- Linus Torvalds


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev