Handling static analyzer diagnostics

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Handling static analyzer diagnostics

shirley breuer via cfe-dev
Hi all,

I've been playing around with the Static Analyzer in an external code tree and
came up with a couple of tests based upon GoogleTest.  Inspired by the unit
tests found in the clang source tree I came up with a PathDiagnosticConsumer
attached at the end of this mail.

All that works.  However, running the test suite also results in printing the
warning to stderr which somehow clutters the output of a test suite run.  I'm
wondering whether it is possible to silence a warning which I handle via my
DiagConsumer explicitly whereas all other warnings are still printed to stderr?

Cheers,
Stefan



class DiagConsumer : public PathDiagnosticConsumer {
  llvm::raw_ostream &Output;
public:
  DiagConsumer(llvm::raw_ostream &Output) : Output(Output) {}
  void FlushDiagnosticsImpl(std::vector<const PathDiagnostic *> &Diags,
                            FilesMade *filesMade) override {
    for (const auto *PD : Diags)
      Output << PD->getCheckerName() << ":" << PD->getShortDescription() << '\n';
  }
  StringRef getName() const override { return "Test"; }
};

class TestAction : public AnalysisAction {
  llvm::raw_ostream &DiagsOutput;
public:
  TestAction(llvm::raw_ostream &DiagsOutput) : DiagsOutput(DiagsOutput) {}
protected:
  std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(clang::CompilerInstance &CI,
                                                        clang::StringRef InFile) override {
    std::unique_ptr<AnalysisASTConsumer> AnalysisConsumer = CreateAnalysisConsumer(CI);
    auto analyzerOpts = CI.getAnalyzerOpts();
    analyzerOpts->CheckersAndPackages = {{"foobar", true}};
    AnalysisConsumer->AddCheckerRegistrationFn([](CheckerRegistry &Registry) {
      Registry.addChecker<Foobar>("foobar", "", "");
    });
    AnalysisConsumer->AddDiagnosticConsumer(new DiagConsumer(DiagsOutput));
    return std::move(AnalysisConsumer);
  }
};

TEST(X, Y) {
  std::string Diags;
  llvm::raw_string_ostream OS(Diags);
  auto Code = /* ... */
  EXPECT_TRUE(tooling::runToolOnCode(std::make_unique<TestAction>(OS), Code, "FileName.c"));
  EXPECT_EQ(Diags, "foobar:baz\n");
}
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Handling static analyzer diagnostics

shirley breuer via cfe-dev
AnalysisConsumer automatically populates PathDiagnosticConsumers
according to AnalyzerOptions and the default is not empty; you might as
well be writing html or plist files with your tests (the actual default
is different depending on how everything is invoked and i'm not sure
what it is in your case). You might have to implement a way to clear the
consumers before adding your consumer.

Why do you have to go that far though? Are llvm-lit tests not an answer
to everything?

On 1/6/21 3:03 AM, Stefan Schulze Frielinghaus via cfe-dev wrote:

> Hi all,
>
> I've been playing around with the Static Analyzer in an external code tree and
> came up with a couple of tests based upon GoogleTest.  Inspired by the unit
> tests found in the clang source tree I came up with a PathDiagnosticConsumer
> attached at the end of this mail.
>
> All that works.  However, running the test suite also results in printing the
> warning to stderr which somehow clutters the output of a test suite run.  I'm
> wondering whether it is possible to silence a warning which I handle via my
> DiagConsumer explicitly whereas all other warnings are still printed to stderr?
>
> Cheers,
> Stefan
>
>
>
> class DiagConsumer : public PathDiagnosticConsumer {
>    llvm::raw_ostream &Output;
> public:
>    DiagConsumer(llvm::raw_ostream &Output) : Output(Output) {}
>    void FlushDiagnosticsImpl(std::vector<const PathDiagnostic *> &Diags,
>                              FilesMade *filesMade) override {
>      for (const auto *PD : Diags)
>        Output << PD->getCheckerName() << ":" << PD->getShortDescription() << '\n';
>    }
>    StringRef getName() const override { return "Test"; }
> };
>
> class TestAction : public AnalysisAction {
>    llvm::raw_ostream &DiagsOutput;
> public:
>    TestAction(llvm::raw_ostream &DiagsOutput) : DiagsOutput(DiagsOutput) {}
> protected:
>    std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(clang::CompilerInstance &CI,
>                                                          clang::StringRef InFile) override {
>      std::unique_ptr<AnalysisASTConsumer> AnalysisConsumer = CreateAnalysisConsumer(CI);
>      auto analyzerOpts = CI.getAnalyzerOpts();
>      analyzerOpts->CheckersAndPackages = {{"foobar", true}};
>      AnalysisConsumer->AddCheckerRegistrationFn([](CheckerRegistry &Registry) {
>        Registry.addChecker<Foobar>("foobar", "", "");
>      });
>      AnalysisConsumer->AddDiagnosticConsumer(new DiagConsumer(DiagsOutput));
>      return std::move(AnalysisConsumer);
>    }
> };
>
> TEST(X, Y) {
>    std::string Diags;
>    llvm::raw_string_ostream OS(Diags);
>    auto Code = /* ... */
>    EXPECT_TRUE(tooling::runToolOnCode(std::make_unique<TestAction>(OS), Code, "FileName.c"));
>    EXPECT_EQ(Diags, "foobar:baz\n");
> }
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Handling static analyzer diagnostics

shirley breuer via cfe-dev
Good point.  I wasn't aware of llvm-lit and how easy it is to use in
out-of-tree projects.  I just gave it a try and it works wonderful
(especially in conjunction with VerifyDiagnosticConsumer which makes
testing for new diagnostics very easy).

Thanks for the hint,
Stefan

On Thu, Jan 07, 2021 at 12:17:30AM -0800, Artem Dergachev wrote:

> AnalysisConsumer automatically populates PathDiagnosticConsumers according
> to AnalyzerOptions and the default is not empty; you might as well be
> writing html or plist files with your tests (the actual default is different
> depending on how everything is invoked and i'm not sure what it is in your
> case). You might have to implement a way to clear the consumers before
> adding your consumer.
>
> Why do you have to go that far though? Are llvm-lit tests not an answer to
> everything?
>
> On 1/6/21 3:03 AM, Stefan Schulze Frielinghaus via cfe-dev wrote:
> > Hi all,
> >
> > I've been playing around with the Static Analyzer in an external code tree and
> > came up with a couple of tests based upon GoogleTest.  Inspired by the unit
> > tests found in the clang source tree I came up with a PathDiagnosticConsumer
> > attached at the end of this mail.
> >
> > All that works.  However, running the test suite also results in printing the
> > warning to stderr which somehow clutters the output of a test suite run.  I'm
> > wondering whether it is possible to silence a warning which I handle via my
> > DiagConsumer explicitly whereas all other warnings are still printed to stderr?
> >
> > Cheers,
> > Stefan
> >
> >
> >
> > class DiagConsumer : public PathDiagnosticConsumer {
> >    llvm::raw_ostream &Output;
> > public:
> >    DiagConsumer(llvm::raw_ostream &Output) : Output(Output) {}
> >    void FlushDiagnosticsImpl(std::vector<const PathDiagnostic *> &Diags,
> >                              FilesMade *filesMade) override {
> >      for (const auto *PD : Diags)
> >        Output << PD->getCheckerName() << ":" << PD->getShortDescription() << '\n';
> >    }
> >    StringRef getName() const override { return "Test"; }
> > };
> >
> > class TestAction : public AnalysisAction {
> >    llvm::raw_ostream &DiagsOutput;
> > public:
> >    TestAction(llvm::raw_ostream &DiagsOutput) : DiagsOutput(DiagsOutput) {}
> > protected:
> >    std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(clang::CompilerInstance &CI,
> >                                                          clang::StringRef InFile) override {
> >      std::unique_ptr<AnalysisASTConsumer> AnalysisConsumer = CreateAnalysisConsumer(CI);
> >      auto analyzerOpts = CI.getAnalyzerOpts();
> >      analyzerOpts->CheckersAndPackages = {{"foobar", true}};
> >      AnalysisConsumer->AddCheckerRegistrationFn([](CheckerRegistry &Registry) {
> >        Registry.addChecker<Foobar>("foobar", "", "");
> >      });
> >      AnalysisConsumer->AddDiagnosticConsumer(new DiagConsumer(DiagsOutput));
> >      return std::move(AnalysisConsumer);
> >    }
> > };
> >
> > TEST(X, Y) {
> >    std::string Diags;
> >    llvm::raw_string_ostream OS(Diags);
> >    auto Code = /* ... */
> >    EXPECT_TRUE(tooling::runToolOnCode(std::make_unique<TestAction>(OS), Code, "FileName.c"));
> >    EXPECT_EQ(Diags, "foobar:baz\n");
> > }
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev