Segfault in ASTContext.cpp

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Segfault in ASTContext.cpp

Nathan Ridge via cfe-dev
Hello everyone,

As part of the SMT encoding of constraints done in the CSA, we are triggering a segfault when trying to get the integer type order between an enumType and an integer.

The code from ASTContext.cpp, starting in line 5814:

static const Type *getIntegerTypeForEnum(const EnumType *ET) {
  // Incomplete enum types are not treated as integer types.
  // FIXME: In C++, enum types are never integer types.
  if (ET->getDecl()->isComplete() && !ET->getDecl()->isScoped())
    return ET->getDecl()->getIntegerType().getTypePtr();
  return nullptr; // [1]
}

/// getIntegerTypeOrder - Returns the highest ranked integer type:
/// C99 6.3.1.8p1.  If LHS > RHS, return 1.  If LHS == RHS, return 0. If
/// LHS < RHS, return -1.
int ASTContext::getIntegerTypeOrder(QualType LHS, QualType RHS) const {
  const Type *LHSC = getCanonicalType(LHS).getTypePtr();
  const Type *RHSC = getCanonicalType(RHS).getTypePtr();

  // Unwrap enums to their underlying type.
  if (const auto *ET = dyn_cast<EnumType>(LHSC))
    LHSC = getIntegerTypeForEnum(ET); 
// [0]
  if (const auto *ET = dyn_cast<EnumType>(RHSC))
    RHSC = getIntegerTypeForEnum(ET);

  if (LHSC == RHSC) return 0;

  bool LHSUnsigned = LHSC->isUnsignedIntegerType(); // [2]
  bool RHSUnsigned = RHSC->isUnsignedIntegerType();

The problem happens when LHS (or RHS) is a C++11 enum: getIntegerTypeForEnum [0] is called but returns a nullptr [1], and later we try to call a method from the nullptr [2].

~

Any suggestion of how we can fix/work around it? It doesn't seem to affect Sema in any way, but the code seems rather fragile...

I hacked around it by doing:
-  return nullptr;
+  return ET->getDecl()->getPromotionType().getTypePtr();

But it does not seem right, there is even a comment there that reads:
// FIXME: In C++, enum types are never integer types.

Thank you,

--

Mikhail Ramalho.

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Segfault in ASTContext.cpp

Nathan Ridge via cfe-dev
The code definitely does look like it's missing some useful assertions, but i'm also wondering how come that we have to do something during symbolic execution that doesn't ever happen during "concrete" execution. That's usually a good indication that we're doing something wrong - not even in SMT API but above it, in the code that calls it.

On 6/28/19 7:43 AM, Mikhail Ramalho via cfe-dev wrote:
Hello everyone,

As part of the SMT encoding of constraints done in the CSA, we are triggering a segfault when trying to get the integer type order between an enumType and an integer.

The code from ASTContext.cpp, starting in line 5814:

static const Type *getIntegerTypeForEnum(const EnumType *ET) {
  // Incomplete enum types are not treated as integer types.
  // FIXME: In C++, enum types are never integer types.
  if (ET->getDecl()->isComplete() && !ET->getDecl()->isScoped())
    return ET->getDecl()->getIntegerType().getTypePtr();
  return nullptr; // [1]
}

/// getIntegerTypeOrder - Returns the highest ranked integer type:
/// C99 6.3.1.8p1.  If LHS > RHS, return 1.  If LHS == RHS, return 0. If
/// LHS < RHS, return -1.
int ASTContext::getIntegerTypeOrder(QualType LHS, QualType RHS) const {
  const Type *LHSC = getCanonicalType(LHS).getTypePtr();
  const Type *RHSC = getCanonicalType(RHS).getTypePtr();

  // Unwrap enums to their underlying type.
  if (const auto *ET = dyn_cast<EnumType>(LHSC))
    LHSC = getIntegerTypeForEnum(ET); 
// [0]
  if (const auto *ET = dyn_cast<EnumType>(RHSC))
    RHSC = getIntegerTypeForEnum(ET);

  if (LHSC == RHSC) return 0;

  bool LHSUnsigned = LHSC->isUnsignedIntegerType(); // [2]
  bool RHSUnsigned = RHSC->isUnsignedIntegerType();

The problem happens when LHS (or RHS) is a C++11 enum: getIntegerTypeForEnum [0] is called but returns a nullptr [1], and later we try to call a method from the nullptr [2].

~

Any suggestion of how we can fix/work around it? It doesn't seem to affect Sema in any way, but the code seems rather fragile...

I hacked around it by doing:
-  return nullptr;
+  return ET->getDecl()->getPromotionType().getTypePtr();

But it does not seem right, there is even a comment there that reads:
// FIXME: In C++, enum types are never integer types.

Thank you,

--

Mikhail Ramalho.

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev