[PATCH] D37079: [Preprocessor] Correct internal token parsing of newline characters in CRLF

[Erich Keane via Phabricator via cfe-commits]

erichkeane updated this revision to Diff 112581.
erichkeane added a comment.

Switched to simply \r\n instead of both cases.  This fixes the issue, is likely 
faster (important, since this is a performance critical part of code), and a 
smaller-hammer.


https://reviews.llvm.org/D37079

Files:
  lib/Lex/Lexer.cpp
  test/Frontend/.gitattributes
  test/Frontend/system-header-line-directive-ms-lineendings.c


Index: test/Frontend/system-header-line-directive-ms-lineendings.c
===================================================================
--- /dev/null
+++ test/Frontend/system-header-line-directive-ms-lineendings.c
@@ -0,0 +1,21 @@
+// RUN: %clang_cc1 %s -E -o - -I %S/Inputs -isystem 
%S/Inputs/SystemHeaderPrefix | FileCheck %s
+#include <noline.h>
+#include <line-directive-in-system.h>
+
+#include "line-directive.h"
+
+// This tests that the line numbers for the current file are correctly 
outputted
+// for the include-file-completed test case.  
+
+// CHECK: # 1 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}noline.h" 1 3
+// CHECK: foo();
+// CHECK: # 3 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive-in-system.h" 1 3
+//      The "3" below indicates that "foo.h" is considered a system header.
+// CHECK: # 1 "foo.h" 3
+// CHECK: foo();
+// CHECK: # 4 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive.h" 1
+// CHECK: # 10 "foo.h"{{$}}
+// CHECK: # 6 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
Index: test/Frontend/.gitattributes
===================================================================
--- /dev/null
+++ test/Frontend/.gitattributes
@@ -0,0 +1,2 @@
+# Below test validates crlf line endings, so it should stay crlf.
+system-header-line-directive-ms-lineendings.c text eol=crlf
Index: lib/Lex/Lexer.cpp
===================================================================
--- lib/Lex/Lexer.cpp
+++ lib/Lex/Lexer.cpp
@@ -3070,9 +3070,12 @@
     // If Microsoft extensions are disabled, this is just random garbage.
     Kind = tok::unknown;
     break;
-      
-  case '\n':
+
   case '\r':
+    if (CurPtr[0] == '\n')
+      Char = getAndAdvanceChar(CurPtr, Result);
+    LLVM_FALLTHROUGH;
+  case '\n':
     // If we are inside a preprocessor directive and we see the end of line,
     // we know we are done with the directive, so return an EOD token.
     if (ParsingPreprocessorDirective) {



Index: test/Frontend/system-header-line-directive-ms-lineendings.c
===================================================================
--- /dev/null
+++ test/Frontend/system-header-line-directive-ms-lineendings.c
@@ -0,0 +1,21 @@
+// RUN: %clang_cc1 %s -E -o - -I %S/Inputs -isystem %S/Inputs/SystemHeaderPrefix | FileCheck %s
+#include <noline.h>
+#include <line-directive-in-system.h>
+
+#include "line-directive.h"
+
+// This tests that the line numbers for the current file are correctly outputted
+// for the include-file-completed test case.  
+
+// CHECK: # 1 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}noline.h" 1 3
+// CHECK: foo();
+// CHECK: # 3 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive-in-system.h" 1 3
+//      The "3" below indicates that "foo.h" is considered a system header.
+// CHECK: # 1 "foo.h" 3
+// CHECK: foo();
+// CHECK: # 4 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive.h" 1
+// CHECK: # 10 "foo.h"{{$}}
+// CHECK: # 6 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
Index: test/Frontend/.gitattributes
===================================================================
--- /dev/null
+++ test/Frontend/.gitattributes
@@ -0,0 +1,2 @@
+# Below test validates crlf line endings, so it should stay crlf.
+system-header-line-directive-ms-lineendings.c text eol=crlf
Index: lib/Lex/Lexer.cpp
===================================================================
--- lib/Lex/Lexer.cpp
+++ lib/Lex/Lexer.cpp
@@ -3070,9 +3070,12 @@
     // If Microsoft extensions are disabled, this is just random garbage.
     Kind = tok::unknown;
     break;
-      
-  case '\n':
+
   case '\r':
+    if (CurPtr[0] == '\n')
+      Char = getAndAdvanceChar(CurPtr, Result);
+    LLVM_FALLTHROUGH;
+  case '\n':
     // If we are inside a preprocessor directive and we see the end of line,
     // we know we are done with the directive, so return an EOD token.
     if (ParsingPreprocessorDirective) {

_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits