Commit e733c4a
committed
[refactor](be) Remove lock time profiling from load writers
### What problem does this PR solve?
Issue Number: None
Related PR: apache#62661
Problem Summary: Remove the LockTime profiling logic added for DeltaWriter, DeltaWriterV2, and MemTableWriter, including the lock wait stopwatch accounting and profile counters.
### Release note
None
### Check List (For Author)
- Test: Manual test
-
-
- Target system: Darwin; Target arch: arm64
Python 3.11.3
Check JAVA_HOME version
Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
Maven home: /opt/homebrew/Cellar/maven/3.9.2/libexec
Java version: 17.0.13, vendor: Oracle Corporation, runtime: /Users/laihui/lib/jdk-17.0.13.jdk/Contents/Home
Default locale: zh_CN_#Hans, platform encoding: UTF-8
OS name: "mac os x", version: "15.7.3", arch: "x86_64", family: "mac"
cmake version 3.26.3
CMake suite maintained and supported by Kitware (kitware.com/cmake).
ninja 1.11.1
ccache version 4.8.1
�[33;1mWARNNING: �[37;1mSkip building with BE Java extensions due to the architecture which the library libjvm.dylib is built for does not match.�[0m
Get params:
BUILD_FE -- 0
BUILD_BE -- 1
BUILD_CLOUD -- 0
BUILD_BROKER -- 0
BUILD_META_TOOL -- OFF
BUILD_FILE_CACHE_MICROBENCH_TOOL -- OFF
BUILD_INDEX_TOOL -- OFF
BUILD_BENCHMARK -- OFF
BUILD_TASK_EXECUTOR_SIMULATOR -- OFF
BUILD_BE_JAVA_EXTENSIONS -- 0
BUILD_BE_CDC_CLIENT -- 1
BUILD_HIVE_UDF -- 0
BUILD_JUICEFS -- ON
BUILD_JINDOFS -- OFF
PARALLEL -- 3
CLEAN -- 0
GLIBC_COMPATIBILITY -- OFF
USE_AVX2 -- ON
USE_LIBCPP -- ON
USE_UNWIND -- OFF
STRIP_DEBUG_INFO -- OFF
USE_JEMALLOC -- OFF
USE_BTHREAD_SCANNER -- OFF
ENABLE_INJECTION_POINT -- OFF
DENABLE_CLANG_COVERAGE -- OFF
DISPLAY_BUILD_TIME -- OFF
ENABLE_PCH -- ON
WITH_TDE_DIR --
Feature List: -TDE,-HDFS_STORAGE_VAULT,+UI,-AZURE_BLOB,-AZURE_STORAGE_VAULT,-HIVE_UDF,-BE_JAVA_EXTENSIONS
Target system: Darwin; Target arch: arm64
Python 3.11.3
Check JAVA_HOME version
Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
Maven home: /opt/homebrew/Cellar/maven/3.9.2/libexec
Java version: 17.0.13, vendor: Oracle Corporation, runtime: /Users/laihui/lib/jdk-17.0.13.jdk/Contents/Home
Default locale: zh_CN_#Hans, platform encoding: UTF-8
OS name: "mac os x", version: "15.7.3", arch: "x86_64", family: "mac"
cmake version 3.26.3
CMake suite maintained and supported by Kitware (kitware.com/cmake).
ninja 1.11.1
ccache version 4.8.1
Build generated code
/Library/Developer/CommandLineTools/usr/bin/make -C script
/Library/Developer/CommandLineTools/usr/bin/make -C proto
/Users/laihui/work/doris/gensrc/script/gen_build_version.sh
/Library/Developer/CommandLineTools/usr/bin/make -C thrift
make[1]: Nothing to be done for `all'.
make[1]: Nothing to be done for `all'.
Done
Update apache-orc submodule ...
/Users/laihui/work/doris
Current commit ID of apache-orc submodule: be0f1b73a7aeb78824a03e0dcb692c50a176d513, expected is be0f1b73a7aeb78824a03e0dcb692c50a176d513
Update clucene submodule ...
/Users/laihui/work/doris
Current commit ID of clucene submodule: c51b5cc9adc63817ad8322f617c75737ece7288d, expected is c51b5cc9adc63817ad8322f617c75737ece7288d
Update openblas submodule ...
/Users/laihui/work/doris
Current commit ID of openblas submodule: 77986e49425532bf8f651db74cbe1579bcb4a5bf, expected is 77986e49425532bf8f651db74cbe1579bcb4a5bf
Update faiss submodule ...
/Users/laihui/work/doris
Current commit ID of faiss submodule: 032afe95f671cd50b82d52d901345600776d7855, expected is 032afe95f671cd50b82d52d901345600776d7855
Build Backend: Release
-- Make program: /opt/homebrew/opt/ninja/bin/ninja
-- Use ccache: -DCMAKE_CXX_COMPILER_LAUNCHER=ccache and -DCMAKE_C_COMPILER_LAUNCHER=ccache
-- Extra cxx flags:
-- Build fs benchmark tool: OFF
-- Build task executor simulator: OFF
-- Build file cache lru tool: OFF
-- GLIBC_COMPATIBILITY is OFF
-- USE_LIBCPP is ON
-- USE_JEMALLOC is OFF
-- USE_UNWIND is OFF
-- ENABLE_PCH is ON
-- USE_AVX2 is ON
-- Build type is RELEASE
-- Build target arch is arm64
-- DORIS_HOME is /Users/laihui/work/doris
-- THIRDPARTY_DIR is /Users/laihui/work/doris/thirdparty/installed
-- make test: OFF
-- make benchmark: OFF
-- build fs benchmark tool: OFF
-- build task executor simulator: OFF
-- build file cache lru tool: OFF
-- build gensrc if necessary
/Library/Developer/CommandLineTools/usr/bin/make -C script
/Users/laihui/work/doris/gensrc/script/gen_build_version.sh
get java cmd: /Users/laihui/lib/jdk-17.0.13.jdk/Contents/Home/bin/java
get java version: java full version \"17.0.13+10-LTS-268\"
/Library/Developer/CommandLineTools/usr/bin/make -C proto
make[1]: Nothing to be done for `all'.
/Library/Developer/CommandLineTools/usr/bin/make -C thrift
make[1]: Nothing to be done for `all'.
-- Found Boost: /Users/laihui/work/doris/thirdparty/installed/lib64/cmake/Boost-1.81.0/BoostConfig.cmake (found suitable version "1.81.0", minimum required is "1.81.0") found components: system date_time
-- Found Boost: /Users/laihui/work/doris/thirdparty/installed/lib64/cmake/Boost-1.81.0/BoostConfig.cmake (found suitable version "1.81.0", minimum required is "1.81.0") found components: system container
-- build azure: OFF
-- Build type: RELEASE
-- compiler AppleClang version 17.0.0.17000603
-- Disable the metrics collection
-- SNAPPY_HOME: /Users/laihui/work/doris/thirdparty/installed
-- Found the Snappy header: /Users/laihui/work/doris/thirdparty/installed/include/snappy.h
-- Found the Snappy library: /Users/laihui/work/doris/thirdparty/installed/lib/libsnappy.a
-- ZLIB_HOME: /Users/laihui/work/doris/thirdparty/installed
-- Found the ZLIB header: /Users/laihui/work/doris/thirdparty/installed/include/zlib.h
-- Found the ZLIB library: /Users/laihui/work/doris/thirdparty/installed/lib/libz.a
-- Found the ZLIB static library: /Users/laihui/work/doris/thirdparty/installed/lib/libz.a
-- ZSTD_HOME: /Users/laihui/work/doris/thirdparty/installed
-- Found the zstd header: /Users/laihui/work/doris/thirdparty/installed/include/zstd/zstd.h
-- Found the zstd library: /Users/laihui/work/doris/thirdparty/installed/lib/libzstd.a
-- Found the zstd static library: /Users/laihui/work/doris/thirdparty/installed/lib/libzstd.a
-- LZ4_HOME: /Users/laihui/work/doris/thirdparty/installed
-- Found the LZ4 header: /Users/laihui/work/doris/thirdparty/installed/include/lz4/lz4.h
-- Found the LZ4 library: /Users/laihui/work/doris/thirdparty/installed/lib/liblz4.a
-- PROTOBUF_HOME: /Users/laihui/work/doris/thirdparty/installed
-- Found the Protobuf headers: /Users/laihui/work/doris/thirdparty/installed/include
-- Found the Protobuf library: /Users/laihui/work/doris/thirdparty/installed/lib/libprotobuf.a
-- Found the Protoc library: /Users/laihui/work/doris/thirdparty/installed/lib/libprotoc.a
-- Found the Protoc executable: /Users/laihui/work/doris/thirdparty/installed/bin/protoc
-- Found the Protobuf static library: /Users/laihui/work/doris/thirdparty/installed/lib/libprotobuf.a
-- Found the Protoc static library: /Users/laihui/work/doris/thirdparty/installed/lib/libprotoc.a
-- old Boost_INCLUDE_DIR : /Users/laihui/work/doris/thirdparty/installed/include
-- Found Boost: /Users/laihui/work/doris/thirdparty/installed/lib64/cmake/Boost-1.81.0/BoostConfig.cmake (found version "1.81.0")
-- Boost_INCLUDE_DIR : /Users/laihui/work/doris/thirdparty/installed/include
-- Zstd target already exists
-- Looking for MapViewOfFile
-- Looking for MapViewOfFile(0,0,0,0,0) - not found
-- Looking for _close
-- Looking for _close((int)0) - not found
-- Looking for _read
-- Looking for _read((int)0, (void*)0, (unsigned int)0) - not found
-- Looking for _open
-- Looking for _open(0,0,0) - not found
-- Looking for _write
-- Looking for _write((int)0, (const void*)0, (unsigned int)0) - not found
-- Looking for _unlink
-- Looking for _unlink((const char*)0) - not found
-- Looking for _ftime
-- Looking for _ftime(0) - not found
-- Looking for _mkdir
-- Looking for _mkdir((const char*)0) - not found
-- Checking support new float byte<->float conversions
-- Checking support new float byte<->float conversions - yes
-- ARM_MARCH is armv8-a+crc
-- UBSAN_IGNORELIST is /Users/laihui/work/doris/conf/ubsan_ignorelist.txt
-- Paimon C++ enabled: legacy thirdparty static linkage mode
-- Compiler: AppleClang-17.0.0.17000603
-- CXX Standard: 20
-- C Standard: 17
-- CXX Flags: -O3 -DNDEBUG
-- C Flags: -O3 -DNDEBUG
-- CC Flags:
-- Compiler Options: -ffile-prefix-map=/Users/laihui/work/doris=.;-g;-gdwarf-5;-Wall;-Wextra;-Werror;-Wundef;-pthread;-fstrict-aliasing;-fno-omit-frame-pointer;$<$<COMPILE_LANGUAGE:CXX>:-Wnon-virtual-dtor>;-Wno-unused-parameter;-Wno-sign-compare;-fcolor-diagnostics;-Wpedantic;-Wshadow;-Wshadow-field;-Wunused;-Wunused-command-line-argument;-Wunused-exception-parameter;-Wunused-volatile-lvalue;-Wunused-template;-Wunused-member-function;-Wunused-macros;-Wconversion;-Wthread-safety;-Wno-gnu-statement-expression;-Wno-implicit-float-conversion;-Wno-sign-conversion;$<$<COMPILE_LANGUAGE:CXX>:-stdlib=libc++>;-march=armv8-a+crc
-- Compiler Definitions: BOOST_PROCESS_POSIX_NO_SIGTIMEDWAIT;BOOST_STACKTRACE_USE_NOOP;GLOG_CUSTOM_PREFIX_SUPPORT;USE_LIBCPP;__STDC_FORMAT_MACROS;BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG;BOOST_SYSTEM_NO_DEPRECATED;BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX=1;BRPC_ENABLE_CPU_PROFILER;S2_USE_GFLAGS;S2_USE_GLOG;HAVE_INTTYPES_H;HAVE_NETINET_IN_H;_DARWIN_C_SOURCE;LIBDIVIDE_NEON;LIBJVM;USE_LIBHDFS3
-- Doris Dependencies: jvm;Boost::date_time;Boost::container;gflags;glog;backtrace;re2;hyperscan;odbc;pprof;protobuf;gtest;gtest_main;benchmark;gmock;snappy;curl;lz4;thrift;thriftnb;crc32c;libevent_core;libevent_openssl;libevent;libevent_pthreads;libbz2;libz;crypto;openssl;leveldb;grpc++_reflection;grpc;grpc++;grpc++_unsecure;gpr;upb;cares;address_sorting;z;brotlicommon;brotlidec;brotlienc;zstd;arrow;arrow_flight;arrow_flight_sql;arrow_dataset;arrow_acero;parquet;brpc;rocksdb;cyrus-sasl;rdkafka_cpp;rdkafka;s2;bitshuffle;roaring;fmt;cctz;base64;aws-cpp-sdk-core;aws-cpp-sdk-s3;aws-cpp-sdk-transfer;aws-cpp-sdk-s3-crt;aws-crt-cpp;aws-c-cal;aws-c-auth;aws-c-compression;aws-c-common;aws-c-event-stream;aws-c-io;aws-c-http;aws-c-mqtt;aws-checksums;aws-c-s3;aws-c-sdkutils;aws-cpp-sdk-identity-management;aws-cpp-sdk-sts;aws-cpp-sdk-kinesis;minizip;simdjson;idn;xml2;lzma;gsasl;krb5support;krb5;com_err;k5crypto;gssapi_krb5;streamvbyte;bfd;iberty;intl;icuuc;icui18n;icudata;pugixml;paimon;paimon_orc_file_format;paimon_blob_file_format;paimon_local_file_system;paimon_file_index;paimon_global_index;roaring_bitmap_paimon;xxhash_paimon;fmt_paimon;tbb_paimon;hdfs3;absl::cord;absl::cord_internal;absl::cordz_functions;absl::cordz_info;absl::cordz_update_scope;absl::cordz_update_tracker;absl::crc_cord_state;absl::flags;absl::random_random;absl::spinlock_wait;absl::status;absl::statusor;absl::strings;bfd;iberty;intl;krb5support;krb5;com_err;gssapi_krb5;k5crypto;orc;ic;clucene-core-static;clucene-shared-static;clucene-contribs-lib;-Wl,-force_load,$<TARGET_FILE:paimon_parquet_file_format>;roaring_bitmap_paimon;xxhash_paimon;fmt_paimon;tbb_paimon
-- Link Flags: Agent;Common;Core;Exec;Exprs;Format;IO;Storage;Runtime;Service;Udf;Util;DorisGen;Load;InformationSchema;Cloud;CommonCPP;jvm;Boost::date_time;Boost::container;gflags;glog;backtrace;re2;hyperscan;odbc;pprof;protobuf;gtest;gtest_main;benchmark;gmock;snappy;curl;lz4;thrift;thriftnb;crc32c;libevent_core;libevent_openssl;libevent;libevent_pthreads;libbz2;libz;crypto;openssl;leveldb;grpc++_reflection;grpc;grpc++;grpc++_unsecure;gpr;upb;cares;address_sorting;z;brotlicommon;brotlidec;brotlienc;zstd;arrow;arrow_flight;arrow_flight_sql;arrow_dataset;arrow_acero;parquet;brpc;rocksdb;cyrus-sasl;rdkafka_cpp;rdkafka;s2;bitshuffle;roaring;fmt;cctz;base64;aws-cpp-sdk-core;aws-cpp-sdk-s3;aws-cpp-sdk-transfer;aws-cpp-sdk-s3-crt;aws-crt-cpp;aws-c-cal;aws-c-auth;aws-c-compression;aws-c-common;aws-c-event-stream;aws-c-io;aws-c-http;aws-c-mqtt;aws-checksums;aws-c-s3;aws-c-sdkutils;aws-cpp-sdk-identity-management;aws-cpp-sdk-sts;aws-cpp-sdk-kinesis;minizip;simdjson;idn;xml2;lzma;gsasl;krb5support;krb5;com_err;k5crypto;gssapi_krb5;streamvbyte;bfd;iberty;intl;icuuc;icui18n;icudata;pugixml;paimon;paimon_orc_file_format;paimon_blob_file_format;paimon_local_file_system;paimon_file_index;paimon_global_index;roaring_bitmap_paimon;xxhash_paimon;fmt_paimon;tbb_paimon;hdfs3;absl::cord;absl::cord_internal;absl::cordz_functions;absl::cordz_info;absl::cordz_update_scope;absl::cordz_update_tracker;absl::crc_cord_state;absl::flags;absl::random_random;absl::spinlock_wait;absl::status;absl::statusor;absl::strings;bfd;iberty;intl;krb5support;krb5;com_err;gssapi_krb5;k5crypto;orc;ic;clucene-core-static;clucene-shared-static;clucene-contribs-lib;-Wl,-force_load,$<TARGET_FILE:paimon_parquet_file_format>;roaring_bitmap_paimon;xxhash_paimon;fmt_paimon;tbb_paimon;-lapple_nghttp2;-lresolv;-liconv;tcmalloc;-framework CoreFoundation;-framework CoreGraphics;-framework CoreText;-framework Foundation;-framework SystemConfiguration;-framework Security
-- GEMM multithread threshold set to 4.
-- Disabling Advanced Vector Extensions 512 (AVX512).
-- Multi-threading enabled with 8 threads.
-- Running getarch
-- GETARCH results:
CORE=VORTEX
LIBCORE=vortex
NUM_CORES=8
MAKEFLAGS += -j 8
-- Compiling a 64-bit binary.
-- Configuring incomplete, errors occurred! attempted, but failed during OpenBLAS CMake configuration because OpenMP_C was not found before compiling the modified files
- Error: Please install llvm@16 firt due to we use it to format code. attempted, but failed because llvm@16 is not installed
- Behavior changed: Yes, removes load writer LockTime profile counters
- Does this need documentation: No1 parent f8b452f commit e733c4a
6 files changed
Lines changed: 0 additions & 29 deletions
File tree
- be/src/load
- delta_writer
- memtable
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
57 | | - | |
58 | 57 | | |
59 | 58 | | |
60 | 59 | | |
| |||
77 | 76 | | |
78 | 77 | | |
79 | 78 | | |
80 | | - | |
81 | 79 | | |
82 | 80 | | |
83 | 81 | | |
| |||
160 | 158 | | |
161 | 159 | | |
162 | 160 | | |
163 | | - | |
164 | 161 | | |
165 | | - | |
166 | 162 | | |
167 | 163 | | |
168 | 164 | | |
| |||
185 | 181 | | |
186 | 182 | | |
187 | 183 | | |
188 | | - | |
189 | 184 | | |
190 | | - | |
191 | 185 | | |
192 | 186 | | |
193 | 187 | | |
194 | 188 | | |
195 | | - | |
196 | 189 | | |
197 | | - | |
198 | 190 | | |
199 | 191 | | |
200 | 192 | | |
| |||
208 | 200 | | |
209 | 201 | | |
210 | 202 | | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | 203 | | |
215 | 204 | | |
216 | 205 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
122 | 121 | | |
123 | 122 | | |
124 | | - | |
125 | | - | |
126 | 123 | | |
127 | 124 | | |
128 | 125 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
79 | 78 | | |
80 | 79 | | |
81 | 80 | | |
82 | | - | |
83 | 81 | | |
84 | 82 | | |
85 | 83 | | |
| |||
156 | 154 | | |
157 | 155 | | |
158 | 156 | | |
159 | | - | |
160 | 157 | | |
161 | | - | |
162 | 158 | | |
163 | 159 | | |
164 | 160 | | |
| |||
179 | 175 | | |
180 | 176 | | |
181 | 177 | | |
182 | | - | |
183 | 178 | | |
184 | | - | |
185 | 179 | | |
186 | 180 | | |
187 | 181 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | 109 | | |
111 | 110 | | |
112 | 111 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
96 | 95 | | |
97 | | - | |
98 | 96 | | |
99 | 97 | | |
100 | 98 | | |
| |||
243 | 241 | | |
244 | 242 | | |
245 | 243 | | |
246 | | - | |
247 | 244 | | |
248 | | - | |
249 | 245 | | |
250 | 246 | | |
251 | 247 | | |
| |||
319 | 315 | | |
320 | 316 | | |
321 | 317 | | |
322 | | - | |
323 | 318 | | |
324 | 319 | | |
325 | 320 | | |
| |||
334 | 329 | | |
335 | 330 | | |
336 | 331 | | |
337 | | - | |
338 | 332 | | |
339 | 333 | | |
340 | 334 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | | - | |
152 | | - | |
153 | 151 | | |
154 | 152 | | |
155 | 153 | | |
| |||
0 commit comments