On Thu, Aug 10, 2017 at 2:04 PM, Kostya Serebryany <k...@google.com> wrote:
> > > On Thu, Aug 10, 2017 at 10:56 AM, Nico Weber via cfe-commits < > cfe-commits@lists.llvm.org> wrote: > >> I really believe this has way too many deps to live in the clang repro, >> as said on the review already. >> > > I don't have a very strong opinion here and would be happy to move if I > see more support for Nico's opinion > (I haven't seen it on the review, and you didn't object further, so we > proceeded). > Again, my rational is that the simpler it is to use the more likely other > researchers will extend this work. > > BTW, I am going to commit a Dockerfile that will make experimenting with > this trivial. > My current (dirty) version looks like this. Not too much trouble. > > FROM ubuntu:16.04 > RUN apt-get update -y && apt-get install -y autoconf automake libtool curl > make g++ unzip > RUN apt-get install -y wget > RUN apt-get install -y git binutils liblzma-dev libz-dev > RUN apt-get install -y python-all > RUN apt-get install -y cmake ninja-build > RUN apt-get install -y subversion > > WORKDIR /root > RUN wget -qO- https://github.com/google/protobuf/releases/download/v3. > 3.0/protobuf-cpp-3.3.0.tar.gz | tar zxf - > RUN cd protobuf-3.3.0 && ./autogen.sh && ./configure && make -j $(nproc) > && make check -j $(nproc) && make install && ldconfig > RUN apt-get install -y pkg-config > RUN svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm > RUN cd llvm/tools && svn co http://llvm.org/svn/llvm-project/cfe/trunk > clang -r $(cd ../ && svn info | grep Revision | awk '{print $2}') > RUN cd llvm/projects && svn co http://llvm.org/svn/llvm- > project/compiler-rt/trunk clang -r $(cd ../ && svn info | grep Revision | > awk '{print $2}') > RUN mkdir build0 && cd build0 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release > ../llvm && ninja > RUN mkdir build1 && cd build1 && cmake -GNinja -DCMAKE_BUILD_TYPE=Release > ../llvm -DLLVM_ENABLE_ASSERTIONS=ON > -DCMAKE_C_COMPILER=`pwd`/../build0/bin/clang > -DCMAKE_CXX_COMPILER=`pwd`/../build0/bin/clang++ > -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address > -DCLANG_ENABLE_PROTO_FUZZER=ON > RUN cd build1 && ninja clang-fuzzer > RUN cd build1 && ninja clang-proto-fuzzer > #RUN cd build1 && ninja clang-proto-to-cxx > > > >> Maybe this could live in clang-extra instead? >> > > clang-extra? > clang-tools-extra, sorry. > That's a separate repo, right? > Yes. > It may require more cmake trickery, and we'll also have to share the > clang-fuzzer-specific code between two repos. > We could move the whole thing. I'd imagine that at most 3% of people who use clang will use this fuzzer, so having it elsewhere seems reasonable. (I'd imagine many more people to use clang-tidy for example, and that's in the other repro.) Also see the "Contributing Extensions to Clang" section on http://clang.llvm.org/get_involved.html > I do want the original clang-fuzzer to remain where it was, and both > (clang-fuzzer and clang-proto-fuzzer) share the code. > > > > >> >> On Aug 8, 2017 4:15 PM, "Matt Morehouse via cfe-commits" < >> cfe-commits@lists.llvm.org> wrote: >> >>> Author: morehouse >>> Date: Tue Aug 8 13:15:04 2017 >>> New Revision: 310408 >>> >>> URL: http://llvm.org/viewvc/llvm-project?rev=310408&view=rev >>> Log: >>> Integrate Kostya's clang-proto-fuzzer with LLVM. >>> >>> Summary: >>> The clang-proto-fuzzer models a subset of C++ as a protobuf and >>> uses libprotobuf-mutator to generate interesting mutations of C++ >>> programs. Clang-proto-fuzzer has already found several bugs in >>> Clang (e.g., https://bugs.llvm.org/show_bug.cgi?id=33747, >>> https://bugs.llvm.org/show_bug.cgi?id=33749). >>> >>> As with clang-fuzzer, clang-proto-fuzzer requires the following >>> cmake flags: >>> - CMAKE_C_COMPILER=clang >>> - CMAKE_CXX_COMPILER=clang++ >>> - LLVM_USE_SANITIZE_COVERAGE=YES // needed for libFuzzer >>> - LLVM_USE_SANITIZER=Address // needed for libFuzzer >>> >>> In addition, clang-proto-fuzzer requires: >>> - CLANG_ENABLE_PROTO_FUZZER=ON >>> >>> clang-proto-fuzzer also requires the following dependencies: >>> - binutils // needed for libprotobuf-mutator >>> - liblzma-dev // needed for libprotobuf-mutator >>> - libz-dev // needed for libprotobuf-mutator >>> - docbook2x // needed for libprotobuf-mutator >>> - Recent version of protobuf [3.3.0 is known to work] >>> >>> A working version of libprotobuf-mutator will automatically be >>> downloaded and built as an external project. >>> >>> Implementation of clang-proto-fuzzer provided by Kostya >>> Serebryany. >>> >>> https://bugs.llvm.org/show_bug.cgi?id=33829 >>> >>> Reviewers: kcc, vitalybuka, bogner >>> >>> Reviewed By: kcc, vitalybuka >>> >>> Subscribers: thakis, mgorny, cfe-commits >>> >>> Differential Revision: https://reviews.llvm.org/D36324 >>> >>> Added: >>> cfe/trunk/cmake/modules/ProtobufMutator.cmake >>> cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp >>> cfe/trunk/tools/clang-fuzzer/README.txt >>> cfe/trunk/tools/clang-fuzzer/cxx_proto.proto >>> cfe/trunk/tools/clang-fuzzer/handle-cxx/ >>> cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt >>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp >>> cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h >>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/ >>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt >>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp >>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h >>> cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp >>> Modified: >>> cfe/trunk/CMakeLists.txt >>> cfe/trunk/tools/clang-fuzzer/CMakeLists.txt >>> cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp >>> >>> Modified: cfe/trunk/CMakeLists.txt >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/CMakeLists.txt >>> ?rev=310408&r1=310407&r2=310408&view=diff >>> ============================================================ >>> ================== >>> --- cfe/trunk/CMakeLists.txt (original) >>> +++ cfe/trunk/CMakeLists.txt Tue Aug 8 13:15:04 2017 >>> @@ -377,6 +377,8 @@ option(CLANG_ENABLE_STATIC_ANALYZER "Bui >>> option(CLANG_ANALYZER_BUILD_Z3 >>> "Build the static analyzer with the Z3 constraint manager." OFF) >>> >>> +option(CLANG_ENABLE_PROTO_FUZZER "Build Clang protobuf fuzzer." OFF) >>> + >>> if(NOT CLANG_ENABLE_STATIC_ANALYZER AND (CLANG_ENABLE_ARCMT OR >>> CLANG_ANALYZER_BUILD_Z3)) >>> message(FATAL_ERROR "Cannot disable static analyzer while enabling >>> ARCMT or Z3") >>> endif() >>> >>> Added: cfe/trunk/cmake/modules/ProtobufMutator.cmake >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/cmake/modules/ >>> ProtobufMutator.cmake?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/cmake/modules/ProtobufMutator.cmake (added) >>> +++ cfe/trunk/cmake/modules/ProtobufMutator.cmake Tue Aug 8 13:15:04 >>> 2017 >>> @@ -0,0 +1,24 @@ >>> +set(PBM_PREFIX protobuf_mutator) >>> +set(PBM_PATH ${CMAKE_CURRENT_BINARY_DIR}/${ >>> PBM_PREFIX}/src/${PBM_PREFIX}) >>> +set(PBM_LIB_PATH ${PBM_PATH}/src/libprotobuf-mutator.a) >>> +set(PBM_FUZZ_LIB_PATH ${PBM_PATH}/src/libfuzzer/libp >>> rotobuf-mutator-libfuzzer.a) >>> + >>> +ExternalProject_Add(${PBM_PREFIX} >>> + PREFIX ${PBM_PREFIX} >>> + GIT_REPOSITORY https://github.com/google/libprotobuf-mutator.git >>> + GIT_TAG 34287f8 >>> + CONFIGURE_COMMAND ${CMAKE_COMMAND} -G${CMAKE_GENERATOR} >>> + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} >>> + -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} >>> + -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE} >>> + BUILD_COMMAND ${CMAKE_MAKE_PROGRAM} >>> + BUILD_BYPRODUCTS ${PBM_LIB_PATH} ${PBM_FUZZ_LIB_PATH} >>> + BUILD_IN_SOURCE 1 >>> + INSTALL_COMMAND "" >>> + LOG_DOWNLOAD 1 >>> + LOG_CONFIGURE 1 >>> + LOG_BUILD 1 >>> + ) >>> + >>> +set(ProtobufMutator_INCLUDE_DIRS ${PBM_PATH}) >>> +set(ProtobufMutator_LIBRARIES ${PBM_FUZZ_LIB_PATH} ${PBM_LIB_PATH}) >>> >>> Modified: cfe/trunk/tools/clang-fuzzer/CMakeLists.txt >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/CMakeLists.txt?rev=310408&r1=310407&r2=310408&view=diff >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/CMakeLists.txt (original) >>> +++ cfe/trunk/tools/clang-fuzzer/CMakeLists.txt Tue Aug 8 13:15:04 2017 >>> @@ -1,21 +1,60 @@ >>> if( LLVM_USE_SANITIZE_COVERAGE ) >>> set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD}) >>> >>> + if(CLANG_ENABLE_PROTO_FUZZER) >>> + # Create protobuf .h and .cc files, and put them in a library for >>> use by >>> + # clang-proto-fuzzer components. >>> + find_package(Protobuf REQUIRED) >>> + add_definitions(-DGOOGLE_PROTOBUF_NO_RTTI) >>> + include_directories(${PROTOBUF_INCLUDE_DIRS}) >>> + include_directories(${CMAKE_CURRENT_BINARY_DIR}) >>> + protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS cxx_proto.proto) >>> + # Hack to bypass LLVM's cmake sources check and allow multiple >>> libraries and >>> + # executables from this directory. >>> + set(LLVM_OPTIONAL_SOURCES >>> + ClangFuzzer.cpp >>> + ExampleClangProtoFuzzer.cpp >>> + ${PROTO_SRCS} >>> + ) >>> + add_clang_library(clangCXXProto >>> + ${PROTO_SRCS} >>> + ${PROTO_HDRS} >>> + >>> + LINK_LIBS >>> + ${PROTOBUF_LIBRARIES} >>> + ) >>> + >>> + # Build and include libprotobuf-mutator >>> + include(ProtobufMutator) >>> + include_directories(${ProtobufMutator_INCLUDE_DIRS}) >>> + >>> + # Build the protobuf->C++ translation library and driver. >>> + add_clang_subdirectory(proto-to-cxx) >>> + >>> + # Build the protobuf fuzzer >>> + add_clang_executable(clang-proto-fuzzer >>> ExampleClangProtoFuzzer.cpp) >>> + target_link_libraries(clang-proto-fuzzer >>> + ${ProtobufMutator_LIBRARIES} >>> + clangCXXProto >>> + clangHandleCXX >>> + clangProtoToCXX >>> + LLVMFuzzer >>> + ) >>> + else() >>> + # Hack to bypass LLVM's cmake sources check and allow multiple >>> libraries and >>> + # executables from this directory. >>> + set(LLVM_OPTIONAL_SOURCES ClangFuzzer.cpp >>> ExampleClangProtoFuzzer.cpp) >>> + endif() >>> + >>> + add_clang_subdirectory(handle-cxx) >>> + >>> add_clang_executable(clang-fuzzer >>> EXCLUDE_FROM_ALL >>> ClangFuzzer.cpp >>> ) >>> >>> target_link_libraries(clang-fuzzer >>> - ${CLANG_FORMAT_LIB_DEPS} >>> - clangAST >>> - clangBasic >>> - clangCodeGen >>> - clangDriver >>> - clangFrontend >>> - clangRewriteFrontend >>> - clangStaticAnalyzerFrontend >>> - clangTooling >>> + clangHandleCXX >>> LLVMFuzzer >>> ) >>> endif() >>> >>> Modified: cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/ClangFuzzer.cpp?rev=310408&r1=310407&r2=310408&view=diff >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp (original) >>> +++ cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp Tue Aug 8 13:15:04 >>> 2017 >>> @@ -13,43 +13,12 @@ >>> /// >>> //===------------------------------------------------------ >>> ----------------===// >>> >>> -#include "clang/Tooling/Tooling.h" >>> -#include "clang/CodeGen/CodeGenAction.h" >>> -#include "clang/Frontend/CompilerInstance.h" >>> -#include "clang/Lex/PreprocessorOptions.h" >>> -#include "llvm/Option/Option.h" >>> -#include "llvm/Support/TargetSelect.h" >>> +#include "handle-cxx/handle_cxx.h" >>> >>> -using namespace clang; >>> +using namespace clang_fuzzer; >>> >>> extern "C" int LLVMFuzzerTestOneInput(uint8_t *data, size_t size) { >>> std::string s((const char *)data, size); >>> - llvm::InitializeAllTargets(); >>> - llvm::InitializeAllTargetMCs(); >>> - llvm::InitializeAllAsmPrinters(); >>> - llvm::InitializeAllAsmParsers(); >>> - >>> - llvm::opt::ArgStringList CC1Args; >>> - CC1Args.push_back("-cc1"); >>> - CC1Args.push_back("./test.cc"); >>> - CC1Args.push_back("-O2"); >>> - llvm::IntrusiveRefCntPtr<FileManager> Files( >>> - new FileManager(FileSystemOptions())); >>> - IgnoringDiagConsumer Diags; >>> - IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new >>> DiagnosticOptions(); >>> - DiagnosticsEngine Diagnostics( >>> - IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()), >>> &*DiagOpts, >>> - &Diags, false); >>> - std::unique_ptr<clang::CompilerInvocation> Invocation( >>> - tooling::newInvocation(&Diagnostics, CC1Args)); >>> - std::unique_ptr<llvm::MemoryBuffer> Input = >>> - llvm::MemoryBuffer::getMemBuffer(s); >>> - Invocation->getPreprocessorOpts().addRemappedFile("./test.cc", >>> Input.release()); >>> - std::unique_ptr<tooling::ToolAction> action( >>> - tooling::newFrontendActionFactory<clang::EmitObjAction>()); >>> - std::shared_ptr<PCHContainerOperations> PCHContainerOps = >>> - std::make_shared<PCHContainerOperations>(); >>> - action->runInvocation(std::move(Invocation), Files.get(), >>> PCHContainerOps, >>> - &Diags); >>> + HandleCXX(s, {"-O2"}); >>> return 0; >>> } >>> >>> Added: cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/ExampleClangProtoFuzzer.cpp?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp (added) >>> +++ cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,28 @@ >>> +//===-- ExampleClangProtoFuzzer.cpp - Fuzz Clang >>> --------------------------===// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +/// >>> +/// \file >>> +/// \brief This file implements a function that runs Clang on a single >>> +/// input and uses libprotobuf-mutator to find new inputs. This >>> function is >>> +/// then linked into the Fuzzer library. >>> +/// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +#include "cxx_proto.pb.h" >>> +#include "handle-cxx/handle_cxx.h" >>> +#include "proto-to-cxx/proto_to_cxx.h" >>> + >>> +#include "src/libfuzzer/libfuzzer_macro.h" >>> + >>> +using namespace clang_fuzzer; >>> + >>> +DEFINE_BINARY_PROTO_FUZZER(const Function& input) { >>> + auto S = FunctionToString(input); >>> + HandleCXX(S, {"-O2"}); >>> +} >>> >>> Added: cfe/trunk/tools/clang-fuzzer/README.txt >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/README.txt?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/README.txt (added) >>> +++ cfe/trunk/tools/clang-fuzzer/README.txt Tue Aug 8 13:15:04 2017 >>> @@ -0,0 +1,73 @@ >>> +This directory contains two utilities for fuzzing Clang: clang-fuzzer >>> and >>> +clang-proto-fuzzer. Both use libFuzzer to generate inputs to clang via >>> +coverage-guided mutation. >>> + >>> +The two utilities differ, however, in how they structure inputs to >>> Clang. >>> +clang-fuzzer makes no attempt to generate valid C++ programs and is >>> therefore >>> +primarily useful for stressing the surface layers of Clang (i.e. lexer, >>> parser). >>> +clang-proto-fuzzer uses a protobuf class to describe a subset of the C++ >>> +language and then uses libprotobuf-mutator to mutate instantiations of >>> that >>> +class, producing valid C++ programs in the process. As a result, >>> +clang-proto-fuzzer is better at stressing deeper layers of Clang and >>> LLVM. >>> + >>> +=================================== >>> + Building clang-fuzzer >>> +=================================== >>> +Within your LLVM build directory, run CMake with the following variable >>> +definitions: >>> +- CMAKE_C_COMPILER=clang >>> +- CMAKE_CXX_COMPILER=clang++ >>> +- LLVM_USE_SANITIZE_COVERAGE=YES >>> +- LLVM_USE_SANITIZER=Address >>> + >>> +Then build the clang-fuzzer target. >>> + >>> +Example: >>> + cd $LLVM_SOURCE_DIR >>> + mkdir build && cd build >>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang >>> -DCMAKE_CXX_COMPILER=clang++ \ >>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address >>> + ninja clang-fuzzer >>> + >>> + >>> +======================================================= >>> + Building clang-proto-fuzzer (Linux-only instructions) >>> +======================================================= >>> +Install the necessary dependencies: >>> +- binutils // needed for libprotobuf-mutator >>> +- liblzma-dev // needed for libprotobuf-mutator >>> +- libz-dev // needed for libprotobuf-mutator >>> +- docbook2x // needed for libprotobuf-mutator >>> +- Recent version of protobuf [3.3.0 is known to work] >>> + >>> +Within your LLVM build directory, run CMake with the following variable >>> +definitions: >>> +- CMAKE_C_COMPILER=clang >>> +- CMAKE_CXX_COMPILER=clang++ >>> +- LLVM_USE_SANITIZE_COVERAGE=YES >>> +- LLVM_USE_SANITIZER=Address >>> +- CLANG_ENABLE_PROTO_FUZZER=ON >>> + >>> +Then build the clang-proto-fuzzer and clang-proto-to-cxx targets. >>> Optionally, >>> +you may also build clang-fuzzer with this setup. >>> + >>> +Example: >>> + cd $LLVM_SOURCE_DIR >>> + mkdir build && cd build >>> + cmake .. -GNinja -DCMAKE_C_COMPILER=clang >>> -DCMAKE_CXX_COMPILER=clang++ \ >>> + -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address \ >>> + -DCLANG_ENABLE_PROTO_FUZZER=ON >>> + ninja clang-proto-fuzzer clang-proto-to-cxx >>> + >>> + >>> +===================== >>> + Running the fuzzers >>> +===================== >>> +clang-fuzzer: >>> + bin/clang-fuzzer CORPUS_DIR >>> + >>> +clang-proto-fuzzer: >>> + bin/clang-proto-fuzzer CORPUS_DIR >>> + >>> +Translating a clang-proto-fuzzer corpus output to C++: >>> + bin/clang-proto-to-cxx CORPUS_OUTPUT_FILE >>> >>> Added: cfe/trunk/tools/clang-fuzzer/cxx_proto.proto >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/cxx_proto.proto?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/cxx_proto.proto (added) >>> +++ cfe/trunk/tools/clang-fuzzer/cxx_proto.proto Tue Aug 8 13:15:04 >>> 2017 >>> @@ -0,0 +1,93 @@ >>> +//===-- cxx_proto.proto - Protobuf description of C++ >>> ---------------------===// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +/// >>> +/// \file >>> +/// \brief This file describes a subset of C++ as a protobuf. It is >>> used to >>> +/// more easily find interesting inputs for fuzzing Clang. >>> +/// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +syntax = "proto2"; >>> + >>> +message VarRef { >>> + required int32 varnum = 1; >>> +} >>> + >>> +message Lvalue { >>> + required VarRef varref = 1; >>> +} >>> + >>> +message Const { >>> + required int32 val = 1; >>> +} >>> + >>> +message BinaryOp { >>> + enum Op { >>> + PLUS = 0; >>> + MINUS = 1; >>> + MUL = 2; >>> + DIV = 3; >>> + MOD = 4; >>> + XOR = 5; >>> + AND = 6; >>> + OR = 7; >>> + EQ = 8; >>> + NE = 9; >>> + LE = 10; >>> + GE = 11; >>> + LT = 12; >>> + GT = 13; >>> + }; >>> + required Op op = 1; >>> + required Rvalue left = 2; >>> + required Rvalue right = 3; >>> +} >>> + >>> +message Rvalue { >>> + oneof rvalue_oneof { >>> + VarRef varref = 1; >>> + Const cons = 2; >>> + BinaryOp binop = 3; >>> + } >>> +} >>> + >>> +message AssignmentStatement { >>> + required Lvalue lvalue = 1; >>> + required Rvalue rvalue = 2; >>> +} >>> + >>> + >>> +message IfElse { >>> + required Rvalue cond = 1; >>> + required StatementSeq if_body = 2; >>> + required StatementSeq else_body = 3; >>> +} >>> + >>> +message While { >>> + required Rvalue cond = 1; >>> + required StatementSeq body = 2; >>> +} >>> + >>> +message Statement { >>> + oneof stmt_oneof { >>> + AssignmentStatement assignment = 1; >>> + IfElse ifelse = 2; >>> + While while_loop = 3; >>> + } >>> +} >>> + >>> +message StatementSeq { >>> + repeated Statement statements = 1; >>> +} >>> + >>> +message Function { >>> + required StatementSeq statements = 1; >>> +} >>> + >>> +package clang_fuzzer; >>> >>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/handle-cxx/CMakeLists.txt?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt (added) >>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,11 @@ >>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD}) >>> + >>> +add_clang_library(clangHandleCXX >>> + handle_cxx.cpp >>> + >>> + LINK_LIBS >>> + clangCodeGen >>> + clangFrontend >>> + clangLex >>> + clangTooling >>> + ) >>> >>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/handle-cxx/handle_cxx.cpp?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp (added) >>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,58 @@ >>> +//==-- handle_cxx.cpp - Helper function for Clang fuzzers >>> ------------------==// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +// >>> +// Implements HandleCXX for use by the Clang fuzzers. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +#include "handle_cxx.h" >>> + >>> +#include "clang/CodeGen/CodeGenAction.h" >>> +#include "clang/Frontend/CompilerInstance.h" >>> +#include "clang/Lex/PreprocessorOptions.h" >>> +#include "clang/Tooling/Tooling.h" >>> +#include "llvm/Option/Option.h" >>> +#include "llvm/Support/TargetSelect.h" >>> + >>> +using namespace clang; >>> + >>> +void clang_fuzzer::HandleCXX(const std::string &S, >>> + const std::vector<const char *> >>> &ExtraArgs) { >>> + llvm::InitializeAllTargets(); >>> + llvm::InitializeAllTargetMCs(); >>> + llvm::InitializeAllAsmPrinters(); >>> + llvm::InitializeAllAsmParsers(); >>> + >>> + llvm::opt::ArgStringList CC1Args; >>> + CC1Args.push_back("-cc1"); >>> + for (auto &A : ExtraArgs) >>> + CC1Args.push_back(A); >>> + CC1Args.push_back("./test.cc"); >>> + >>> + llvm::IntrusiveRefCntPtr<FileManager> Files( >>> + new FileManager(FileSystemOptions())); >>> + IgnoringDiagConsumer Diags; >>> + IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new >>> DiagnosticOptions(); >>> + DiagnosticsEngine Diagnostics( >>> + IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()), >>> &*DiagOpts, >>> + &Diags, false); >>> + std::unique_ptr<clang::CompilerInvocation> Invocation( >>> + tooling::newInvocation(&Diagnostics, CC1Args)); >>> + std::unique_ptr<llvm::MemoryBuffer> Input = >>> + llvm::MemoryBuffer::getMemBuffer(S); >>> + Invocation->getPreprocessorOpts().addRemappedFile("./test.cc", >>> + Input.release()); >>> + std::unique_ptr<tooling::ToolAction> action( >>> + tooling::newFrontendActionFactory<clang::EmitObjAction>()); >>> + std::shared_ptr<PCHContainerOperations> PCHContainerOps = >>> + std::make_shared<PCHContainerOperations>(); >>> + action->runInvocation(std::move(Invocation), Files.get(), >>> PCHContainerOps, >>> + &Diags); >>> +} >>> + >>> >>> Added: cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/handle-cxx/handle_cxx.h?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h (added) >>> +++ cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,25 @@ >>> +//==-- handle_cxx.h - Helper function for Clang fuzzers >>> --------------------==// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +// >>> +// Defines HandleCXX for use by the Clang fuzzers. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +#ifndef LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H >>> +#define LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H >>> + >>> +#include <string> >>> +#include <vector> >>> + >>> +namespace clang_fuzzer { >>> +void HandleCXX(const std::string &S, >>> + const std::vector<const char *> &ExtraArgs); >>> +} // namespace clang_fuzzer >>> + >>> +#endif >>> >>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/proto-to-cxx/CMakeLists.txt?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt (added) >>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,10 @@ >>> +set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD}) >>> + >>> +# Hack to bypass LLVM's CMake source checks so we can have both a >>> library and >>> +# an executable built from this directory. >>> +set(LLVM_OPTIONAL_SOURCES proto_to_cxx.cpp proto_to_cxx_main.cpp) >>> + >>> +add_clang_library(clangProtoToCXX proto_to_cxx.cpp LINK_LIBS >>> clangCXXProto) >>> + >>> +add_clang_executable(clang-proto-to-cxx proto_to_cxx_main.cpp) >>> +target_link_libraries(clang-proto-to-cxx clangProtoToCXX) >>> >>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/proto-to-cxx/proto_to_cxx.cpp?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp (added) >>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp Tue Aug >>> 8 13:15:04 2017 >>> @@ -0,0 +1,102 @@ >>> +//==-- proto_to_cxx.cpp - Protobuf-C++ conversion >>> --------------------------==// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +// >>> +// Implements functions for converting between protobufs and C++. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +#include "proto_to_cxx.h" >>> +#include "cxx_proto.pb.h" >>> + >>> +#include <ostream> >>> +#include <sstream> >>> + >>> +namespace clang_fuzzer { >>> + >>> +// Forward decls. >>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x); >>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x); >>> + >>> +// Proto to C++. >>> +std::ostream &operator<<(std::ostream &os, const Const &x) { >>> + return os << "(" << x.val() << ")"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const VarRef &x) { >>> + return os << "a[" << (static_cast<uint32_t>(x.varnum()) % 100) << >>> "]"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const Lvalue &x) { >>> + return os << x.varref(); >>> +} >>> +std::ostream &operator<<(std::ostream &os, const Rvalue &x) { >>> + if (x.has_varref()) return os << x.varref(); >>> + if (x.has_cons()) return os << x.cons(); >>> + if (x.has_binop()) return os << x.binop(); >>> + return os << "1"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const BinaryOp &x) { >>> + os << "(" << x.left(); >>> + switch (x.op()) { >>> + case BinaryOp::PLUS: os << "+"; break; >>> + case BinaryOp::MINUS: os << "-"; break; >>> + case BinaryOp::MUL: os << "*"; break; >>> + case BinaryOp::DIV: os << "/"; break; >>> + case BinaryOp::MOD: os << "%"; break; >>> + case BinaryOp::XOR: os << "^"; break; >>> + case BinaryOp::AND: os << "&"; break; >>> + case BinaryOp::OR: os << "|"; break; >>> + case BinaryOp::EQ: os << "=="; break; >>> + case BinaryOp::NE: os << "!="; break; >>> + case BinaryOp::LE: os << "<="; break; >>> + case BinaryOp::GE: os << ">="; break; >>> + case BinaryOp::LT: os << "<"; break; >>> + case BinaryOp::GT: os << ">"; break; >>> + } >>> + return os << x.right() << ")"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const AssignmentStatement >>> &x) { >>> + return os << x.lvalue() << "=" << x.rvalue() << ";\n"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const IfElse &x) { >>> + return os << "if (" << x.cond() << "){\n" >>> + << x.if_body() << "} else { \n" >>> + << x.else_body() << "}\n"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const While &x) { >>> + return os << "while (" << x.cond() << "){\n" << x.body() << "}\n"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const Statement &x) { >>> + if (x.has_assignment()) return os << x.assignment(); >>> + if (x.has_ifelse()) return os << x.ifelse(); >>> + if (x.has_while_loop()) return os << x.while_loop(); >>> + return os << "(void)0;\n"; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const StatementSeq &x) { >>> + for (auto &st : x.statements()) os << st; >>> + return os; >>> +} >>> +std::ostream &operator<<(std::ostream &os, const Function &x) { >>> + return os << "void foo(int *a) {\n" << x.statements() << "}\n"; >>> +} >>> + >>> +// --------------------------------- >>> + >>> +std::string FunctionToString(const Function &input) { >>> + std::ostringstream os; >>> + os << input; >>> + return os.str(); >>> + >>> +} >>> +std::string ProtoToCxx(const uint8_t *data, size_t size) { >>> + Function message; >>> + if (!message.ParseFromArray(data, size)) >>> + return "#error invalid proto\n"; >>> + return FunctionToString(message); >>> +} >>> + >>> +} // namespace clang_fuzzer >>> >>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/proto-to-cxx/proto_to_cxx.h?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h (added) >>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h Tue Aug 8 >>> 13:15:04 2017 >>> @@ -0,0 +1,22 @@ >>> +//==-- proto_to_cxx.h - Protobuf-C++ conversion >>> ----------------------------==// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +// >>> +// Defines functions for converting between protobufs and C++. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> + >>> +#include <cstdint> >>> +#include <cstddef> >>> +#include <string> >>> + >>> +namespace clang_fuzzer { >>> +class Function; >>> +std::string FunctionToString(const Function &input); >>> +std::string ProtoToCxx(const uint8_t *data, size_t size); >>> +} >>> >>> Added: cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp >>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-fu >>> zzer/proto-to-cxx/proto_to_cxx_main.cpp?rev=310408&view=auto >>> ============================================================ >>> ================== >>> --- cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp >>> (added) >>> +++ cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp Tue >>> Aug 8 13:15:04 2017 >>> @@ -0,0 +1,30 @@ >>> +//==-- proto_to_cxx_main.cpp - Driver for protobuf-C++ conversion >>> ----------==// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +// >>> +// Implements a simple driver to print a C++ program from a protobuf. >>> +// >>> +//===------------------------------------------------------ >>> ----------------===// >>> +#include <fstream> >>> +#include <iostream> >>> +#include <streambuf> >>> +#include <string> >>> + >>> +#include "proto_to_cxx.h" >>> + >>> +int main(int argc, char **argv) { >>> + for (int i = 1; i < argc; i++) { >>> + std::fstream in(argv[i]); >>> + std::string str((std::istreambuf_iterator<char>(in)), >>> + std::istreambuf_iterator<char>()); >>> + std::cout << "// " << argv[i] << std::endl; >>> + std::cout << clang_fuzzer::ProtoToCxx( >>> + reinterpret_cast<const uint8_t *>(str.data()), str.size()); >>> + } >>> +} >>> + >>> >>> >>> _______________________________________________ >>> cfe-commits mailing list >>> cfe-commits@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits >>> >> >> _______________________________________________ >> cfe-commits mailing list >> cfe-commits@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits >> >> >
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
