Re: [PATCH] D24040: codechecker tool core

2016-09-02 Thread Gyorgy Orban via cfe-commits 
o.gyorgy added a comment.

In https://reviews.llvm.org/D24040#530293, @rizsotto.mailinglist wrote:

> Gyorgy and the ericsson team, thanks for doing this. very good job! good 
> targeted functionality. i don't want to underestimate the complexity it 
> requires, but to me this is a giant code. i do miss the explanation of the 
> overall functional description what a module does and how it relates to other 
> modules. i found a gap between the high level overview and the code comments.


Would that help If update architecture.md documentation (which gives a high 
level overview) to be more in sync with the source code, with additional 
comments and descriptions in the modules?

> generally speaking i found this code under documented and a bit verbose. the 
> comments in the code is not really paint the picture that i need in order to 
> fix a bug, or implement a feature. and it might only be my personal taste, 
> but found that creating types (or classes this case) in a dynamically typed 
> script language, it makes the code very verbose while it does not implement 
> much.


We use classes to handle and setup environment, build, analyzer and other 
configurations. In the infrastructure this makes available to easily introduce 
new analyzers and handle the output of them. Multiple output handling is 
supported for each analyzer (print to stdout, store to database).

> was commented only on two random modules, but found the same patterns in many 
> other ones.

>  found that this code is not `pep8` conform. `pylint` found a lot of errors 
> and warnings, and a couple of duplicate codes.

>  i hope we can fix these findings and that would make this code more solid 
> and understood by others.


I will address these issues and update our code.

Thanks for the feedback.


https://reviews.llvm.org/D24040



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24040: codechecker tool core

2016-09-02 Thread Gyorgy Orban via cfe-commits 
o.gyorgy added a comment.

In https://reviews.llvm.org/D24040#530546, @mehdi_amini wrote:

> > This looks like a fairly large tool. Should it get its own "subproject 
> > level" directory in the SVN instead of being nested within clang?
>
>
> I'd add that the clang-tools-extra are more closely tied to clang than what 
> this seems to be. Is there a strong rev-lock with clang that I missed?


I've put it here because it is similar to scan-build/scan-build-py and on OSX 
it uses the intercept-build from scan-build-py. Right now there is an 
alternative logger on linux (not in this patch) but we plan to move to use only 
the intercept-build.
We process the output generated by clang and clang-tidy and we use the command 
line options for these tools which could change with newer revisions, in that 
case this tool needs to be updated too.

I will update the readme too.


https://reviews.llvm.org/D24040



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D12906: [RFC] Bug identification("issue_hash") change for CmpRuns.py

2015-10-26 Thread Gyorgy Orban via cfe-commits 
o.gyorgy added a comment.

Hi,

You are right the diff is is based on the hash. We already tried to
use an earlier hash generator (before the patch was introduced), which
generates a slightly different plist, that is why the current version
does not work with the patch.
We will fix CodeChecker to use new hash tag introduced in the final
patch (we did not change it so far because we didn't know what will be
the accepted naming convention and plist format).

We use LD_PRELOAD technique the log all the compiler calls, so no CC
or CXX environment variable orverride is necessary.
With the 'export CODECHECKER_VERBOSE=debug' enviromnet variable you
can see the analyzer commands.

Br,
Gyorgy


http://reviews.llvm.org/D12906



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D12906: [RFC] Bug identification("issue_hash") change for CmpRuns.py

2015-11-10 Thread Gyorgy Orban via cfe-commits 
o.gyorgy added a comment.

Hi,

Sorry for my late answer.
We fixed the plist parser for the CodeChecker it shoud process the
plist generated by the current trunk clang with the issue hash tags.

Right now CodeChecker does not support forwarding additional options
directly to clang (uses only the compiler options from the original
build command extended with options required for the static analyzer).
We already discussed such features. I think they will be implemented.
If you have any requirements feel free to create a new issue on our
github page, we can discuss them further there.

With the LD_PRELOAD (linux specific) technique we load a shared object
before every executable during the original build. In this shared
object we filter out the compiler calls (gcc, g++, clang ... it is
configurable) and create a compile command json file, like cmake. We
use these compilation commands to run the clang static analyzer.
With this technique we can create the compilation command json file
build system independent.

The official LD_PRELOAD documentation can be found here:
http://man7.org/linux/man-pages/man8/ld.so.8.html

Best regards,
Gyorgy


http://reviews.llvm.org/D12906



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D14919: Fix IssueHash generation

2015-11-23 Thread Gyorgy Orban via cfe-commits 
o.gyorgy created this revision.
o.gyorgy added reviewers: zaks.anna, xazax.hun, dcoughlin, jordan_rose.
o.gyorgy added a subscriber: cfe-commits.

Fixing IssueHash generation.
There were some problems with the current version.

If the Decl *D pointer is nullptr the NormalizeLine would crash while getting 
the LangOptions. The required LangOptions can be provided by the HTML or Plist 
Diagnostics.

It is possible that the column number provided by the find_first_not_of is 0 in 
that case the Lexer reads up a wrong source line (translateLineCol requires 
that column numbering should start at 1).

I think there should be an assert checking the line and col values in the 
sourcemanagers translateLineCol. The same way as in translateFileLineCol which 
does this check.

http://reviews.llvm.org/D14919

Files:
  include/clang/StaticAnalyzer/Core/IssueHash.h
  lib/Basic/SourceManager.cpp
  lib/StaticAnalyzer/Checkers/DebugCheckers.cpp
  lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
  lib/StaticAnalyzer/Core/IssueHash.cpp
  lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
  test/Analysis/bug_hash_test.cpp
  test/Analysis/diagnostics/report-issues-within-main-file.cpp

Index: test/Analysis/diagnostics/report-issues-within-main-file.cpp
===
--- test/Analysis/diagnostics/report-issues-within-main-file.cpp
+++ test/Analysis/diagnostics/report-issues-within-main-file.cpp
@@ -949,7 +949,7 @@
 // CHECK-NEXT:   typeBad deallocator
 // CHECK-NEXT:   check_nameunix.MismatchedDeallocator
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf21ac032efaa3d1459a5ed31f0ad44f0
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf689fbd54138491e228f0f89bb02bfb2
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextmainPlusHeader
 // CHECK-NEXT:  issue_hash_function_offset2
Index: test/Analysis/bug_hash_test.cpp
===
--- test/Analysis/bug_hash_test.cpp
+++ test/Analysis/bug_hash_test.cpp
@@ -288,17 +288,17 @@
 // CHECK-NEXT: 
 // CHECK-NEXT: depth0
 // CHECK-NEXT: extended_message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT: message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:
 // CHECK-NEXT:   
-// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:   categorydebug
 // CHECK-NEXT:   typeDump hash components
 // CHECK-NEXT:   check_namedebug.DumpBugHash
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf8ee38da3de42e209c4afa886b5531ab
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf5471f52854dc14167fe96db50c4ba5f
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextf
 // CHECK-NEXT:  issue_hash_function_offset0
Index: lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
===
--- lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
+++ lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
@@ -399,7 +399,7 @@
 *SM);
 const Decl *DeclWithIssue = D->getDeclWithIssue();
 EmitString(o, GetIssueHash(*SM, L, D->getCheckName(), D->getBugType(),
-   DeclWithIssue))
+   DeclWithIssue, LangOpts))
 << '\n';
 
 // Output information about the semantic context where
Index: lib/StaticAnalyzer/Core/IssueHash.cpp
===
--- lib/StaticAnalyzer/Core/IssueHash.cpp
+++ lib/StaticAnalyzer/Core/IssueHash.cpp
@@ -127,14 +127,13 @@
 }
 
 static std::string NormalizeLine(const SourceManager &SM, FullSourceLoc &L,
- const Decl *D) {
+ const LangOptions &LangOpts) {
   static StringRef Whitespaces = " \t\n";
 
-  const LangOptions &Opts = D->getASTContext().getLangOpts();
   StringRef Str = GetNthLineOfFile(SM.getBuffer(L.getFileID(), L),
L.getExpansionLineNumber());
   unsigned col = Str.find_first_not_of(Whitespaces);
-
+  col++;
   SourceLocation StartOfLine =
   SM.translateLineCol(SM.getFileID(L), L.getExpansionLineNumber(), col);
   llvm::MemoryBuffer *Buffer =
@@ -145,7 +144,7 @@
   const char *BufferPos = SM.getCharacterData(StartOfLine);
 
   Token Token;
-  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), Opts,
+  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), LangOpts,
   Buffer->getBufferStart(), BufferPos, Buffer->getBufferEnd());
 
   size_t NextStart = 0;
@@ -175,13 +174,14 @@
 std::string clang::GetIssueString(const

Re: [PATCH] D14919: Fix IssueHash generation

2015-11-23 Thread Gyorgy Orban via cfe-commits 
o.gyorgy updated this revision to Diff 40920.
o.gyorgy added a comment.

Regenerate patch with context and clang format HTMLDiagnostics.cpp.


http://reviews.llvm.org/D14919

Files:
  include/clang/StaticAnalyzer/Core/IssueHash.h
  lib/Basic/SourceManager.cpp
  lib/StaticAnalyzer/Checkers/DebugCheckers.cpp
  lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
  lib/StaticAnalyzer/Core/IssueHash.cpp
  lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
  test/Analysis/bug_hash_test.cpp
  test/Analysis/diagnostics/report-issues-within-main-file.cpp

Index: test/Analysis/diagnostics/report-issues-within-main-file.cpp
===
--- test/Analysis/diagnostics/report-issues-within-main-file.cpp
+++ test/Analysis/diagnostics/report-issues-within-main-file.cpp
@@ -949,7 +949,7 @@
 // CHECK-NEXT:   typeBad deallocator
 // CHECK-NEXT:   check_nameunix.MismatchedDeallocator
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf21ac032efaa3d1459a5ed31f0ad44f0
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf689fbd54138491e228f0f89bb02bfb2
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextmainPlusHeader
 // CHECK-NEXT:  issue_hash_function_offset2
Index: test/Analysis/bug_hash_test.cpp
===
--- test/Analysis/bug_hash_test.cpp
+++ test/Analysis/bug_hash_test.cpp
@@ -288,17 +288,17 @@
 // CHECK-NEXT: 
 // CHECK-NEXT: depth0
 // CHECK-NEXT: extended_message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT: message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:
 // CHECK-NEXT:   
-// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:   categorydebug
 // CHECK-NEXT:   typeDump hash components
 // CHECK-NEXT:   check_namedebug.DumpBugHash
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf8ee38da3de42e209c4afa886b5531ab
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf5471f52854dc14167fe96db50c4ba5f
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextf
 // CHECK-NEXT:  issue_hash_function_offset0
Index: lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
===
--- lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
+++ lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
@@ -399,7 +399,7 @@
 *SM);
 const Decl *DeclWithIssue = D->getDeclWithIssue();
 EmitString(o, GetIssueHash(*SM, L, D->getCheckName(), D->getBugType(),
-   DeclWithIssue))
+   DeclWithIssue, LangOpts))
 << '\n';
 
 // Output information about the semantic context where
Index: lib/StaticAnalyzer/Core/IssueHash.cpp
===
--- lib/StaticAnalyzer/Core/IssueHash.cpp
+++ lib/StaticAnalyzer/Core/IssueHash.cpp
@@ -127,14 +127,13 @@
 }
 
 static std::string NormalizeLine(const SourceManager &SM, FullSourceLoc &L,
- const Decl *D) {
+ const LangOptions &LangOpts) {
   static StringRef Whitespaces = " \t\n";
 
-  const LangOptions &Opts = D->getASTContext().getLangOpts();
   StringRef Str = GetNthLineOfFile(SM.getBuffer(L.getFileID(), L),
L.getExpansionLineNumber());
   unsigned col = Str.find_first_not_of(Whitespaces);
-
+  col++;
   SourceLocation StartOfLine =
   SM.translateLineCol(SM.getFileID(L), L.getExpansionLineNumber(), col);
   llvm::MemoryBuffer *Buffer =
@@ -145,7 +144,7 @@
   const char *BufferPos = SM.getCharacterData(StartOfLine);
 
   Token Token;
-  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), Opts,
+  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), LangOpts,
   Buffer->getBufferStart(), BufferPos, Buffer->getBufferEnd());
 
   size_t NextStart = 0;
@@ -175,20 +174,23 @@
 std::string clang::GetIssueString(const SourceManager &SM,
   FullSourceLoc &IssueLoc,
   StringRef CheckerName, StringRef BugType,
-  const Decl *D) {
+  const Decl *D,
+  const LangOptions &LangOpts) {
   static StringRef Delimiter = "$";
 
   return (llvm::Twine(CheckerName) + Delimiter +
   GetEnclosingDeclContextSignature(D) + Delimiter +
   llvm::utostr(IssueLoc.getExpansionColumnNumber()) + Delimiter +
-  NormalizeLine(SM, IssueLoc, D) + Delimiter + BugType)
+  Normali

Re: [PATCH] D14919: Fix IssueHash generation

2015-11-24 Thread Gyorgy Orban via cfe-commits 
o.gyorgy updated this revision to Diff 41024.
o.gyorgy added a comment.

Some small format changes. Based on the review.


http://reviews.llvm.org/D14919

Files:
  include/clang/StaticAnalyzer/Core/IssueHash.h
  lib/Basic/SourceManager.cpp
  lib/StaticAnalyzer/Checkers/DebugCheckers.cpp
  lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
  lib/StaticAnalyzer/Core/IssueHash.cpp
  lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
  test/Analysis/bug_hash_test.cpp
  test/Analysis/diagnostics/report-issues-within-main-file.cpp

Index: test/Analysis/diagnostics/report-issues-within-main-file.cpp
===
--- test/Analysis/diagnostics/report-issues-within-main-file.cpp
+++ test/Analysis/diagnostics/report-issues-within-main-file.cpp
@@ -949,7 +949,7 @@
 // CHECK-NEXT:   typeBad deallocator
 // CHECK-NEXT:   check_nameunix.MismatchedDeallocator
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf21ac032efaa3d1459a5ed31f0ad44f0
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf689fbd54138491e228f0f89bb02bfb2
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextmainPlusHeader
 // CHECK-NEXT:  issue_hash_function_offset2
Index: test/Analysis/bug_hash_test.cpp
===
--- test/Analysis/bug_hash_test.cpp
+++ test/Analysis/bug_hash_test.cpp
@@ -288,17 +288,17 @@
 // CHECK-NEXT: 
 // CHECK-NEXT: depth0
 // CHECK-NEXT: extended_message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT: message
-// CHECK-NEXT: debug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT: debug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:
 // CHECK-NEXT:   
-// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$namespaceAA{$debug
+// CHECK-NEXT:   descriptiondebug.DumpBugHash$int f()$28$constexprintf(){return5;}$debug
 // CHECK-NEXT:   categorydebug
 // CHECK-NEXT:   typeDump hash components
 // CHECK-NEXT:   check_namedebug.DumpBugHash
 // CHECK-NEXT:   
-// CHECK-NEXT:   issue_hash_content_of_line_in_contextf8ee38da3de42e209c4afa886b5531ab
+// CHECK-NEXT:   issue_hash_content_of_line_in_contextf5471f52854dc14167fe96db50c4ba5f
 // CHECK-NEXT:  issue_context_kindfunction
 // CHECK-NEXT:  issue_contextf
 // CHECK-NEXT:  issue_hash_function_offset0
Index: lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
===
--- lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
+++ lib/StaticAnalyzer/Core/PlistDiagnostics.cpp
@@ -399,7 +399,7 @@
 *SM);
 const Decl *DeclWithIssue = D->getDeclWithIssue();
 EmitString(o, GetIssueHash(*SM, L, D->getCheckName(), D->getBugType(),
-   DeclWithIssue))
+   DeclWithIssue, LangOpts))
 << '\n';
 
 // Output information about the semantic context where
Index: lib/StaticAnalyzer/Core/IssueHash.cpp
===
--- lib/StaticAnalyzer/Core/IssueHash.cpp
+++ lib/StaticAnalyzer/Core/IssueHash.cpp
@@ -127,14 +127,13 @@
 }
 
 static std::string NormalizeLine(const SourceManager &SM, FullSourceLoc &L,
- const Decl *D) {
+ const LangOptions &LangOpts) {
   static StringRef Whitespaces = " \t\n";
 
-  const LangOptions &Opts = D->getASTContext().getLangOpts();
   StringRef Str = GetNthLineOfFile(SM.getBuffer(L.getFileID(), L),
L.getExpansionLineNumber());
   unsigned col = Str.find_first_not_of(Whitespaces);
-
+  col++;
   SourceLocation StartOfLine =
   SM.translateLineCol(SM.getFileID(L), L.getExpansionLineNumber(), col);
   llvm::MemoryBuffer *Buffer =
@@ -145,7 +144,7 @@
   const char *BufferPos = SM.getCharacterData(StartOfLine);
 
   Token Token;
-  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), Opts,
+  Lexer Lexer(SM.getLocForStartOfFile(SM.getFileID(StartOfLine)), LangOpts,
   Buffer->getBufferStart(), BufferPos, Buffer->getBufferEnd());
 
   size_t NextStart = 0;
@@ -175,20 +174,23 @@
 std::string clang::GetIssueString(const SourceManager &SM,
   FullSourceLoc &IssueLoc,
   StringRef CheckerName, StringRef BugType,
-  const Decl *D) {
+  const Decl *D,
+  const LangOptions &LangOpts) {
   static StringRef Delimiter = "$";
 
   return (llvm::Twine(CheckerName) + Delimiter +
   GetEnclosingDeclContextSignature(D) + Delimiter +
   llvm::utostr(IssueLoc.getExpansionColumnNumber()) + Delimiter +
-  NormalizeLine(SM, IssueLoc, D) + Delimiter + BugType)
+  NormalizeLine(SM, IssueLoc,

Re: [PATCH] D14919: Fix IssueHash generation

2015-11-25 Thread Gyorgy Orban via cfe-commits 
o.gyorgy added a subscriber: o.gyorgy.
o.gyorgy marked an inline comment as done.
o.gyorgy added a comment.

I did not create a test checker for the NormalizeLine error in the patch.
Should I add a test checker for this? Do you have any suggestions
where to put it if required?

GetEnclosingDeclContextSignature already checks for nullptr, this
should have been done in the NormalizeLine function also, but in that
case it is not enough because the Lexer needs the LangOpts.


http://reviews.llvm.org/D14919



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits