Tarantool JIT engine#17
Open
Brugarolas wants to merge 785 commits into
Open
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Thanks to Sergey Kaplun and Peter Cawley. (cherry picked from commit d06beb0) This commit is a follow-up for the commit 1b82160 ("Throw any errors before stack changes in trace stitching."). The patch prepends failures for the specific error to be thrown. Nevertheless, the error may be thrown due to retrying trace recording in the case when table bump optimization is enabled or when OOM is observed during reallocation of the snapshot or IR buffers. This patch adds the corresponding protected frame and rethrows the error after a fixup of the stack. This patch also tests the correctness of copying the error message to the top of the stack to get a valid "abort" reason in the `jit.dump` utility. Also, this patch fixes a non-ASCII space character in the comment for <lj-720-errors-before-stitch.test.lua>. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit b8b49bf) The previous commit doesn't handle the case when the error code is `LUA_ERRMEM`. This patch adds a workaround by using the generic error message. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Peter Cawley. (cherry picked from commit 6585305) The `lj_opt_fwd_wasnonnil()` skips the check for HREF and HREFK that may alias. Hence, the guard for the non-nil value may be skipped, and the `__newindex` metamethod call is omitted too. This patch adds the aforementioned check for different reference types (HREF vs. HREFK), which were not detected by the previous analysis. Also, the helper macro `irt_isp32()` is introduced to check that the IR type is `IRT_P32` (KSLOT type). Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
The commit 049e296 ("test: run LuaJIT tests via CMake") introduced CMake build infrastructure and added `cmake_minimum_required` to each CMakeLists.txt in the project. However, it is not required: `cmake_minimum_required` specified in a root CMakeLists.txt is more than enough. The patch leaves `cmake_minimum_required` in a root CMakeLists.txt and removes it in other CMakeLists.txt below in a source tree. Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Since CMake 3.27 compatibility with versions of CMake older than 3.5 is deprecated [1]. CMake produces an annoying warning on the configuration stage: | CMake Deprecation Warning at src/CMakeLists.txt:7 (cmake_minimum_required): | Compatibility with CMake < 3.5 will be removed from a future version of | CMake. We cannot bump a minimum required CMake version without bumping it in the Tarantool build system. However, we can set a <policy_max> (introduced in CMake 3.12, see [1]) and suppress a warning. <policy_max> means that this CMake version is known to work properly and will not result in any build errors right away for higher versions. Note that the current CMake minimum required version in Tarantool is equal to 2.8, but <policy_max> is introduced in CMake 3.12 [1]. However, according to [1] it is not a problem because if CMake is "older than 3.12, the extra ... dots will be seen as version component separators, resulting in the ...<max> part being ignored and preserving the pre-3.12 behavior of basing policies on <min>". <policy_max> is set to 3.18 because compatibility with versions of CMake older than 2.8.12 is deprecated. Calls to cmake_minimum_required(VERSION) that do not specify at least 2.8.12 as their policy version (optionally via ...<max>) will produce a deprecation warning in CMake 3.19 and above [2]. Compatibility with 2.8.12 is needed for CMP0002 [3], see commit 049e296 ("test: run LuaJIT tests via CMake"). 1. https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html 2. https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html#policy-settings 3. https://cmake.org/cmake/help/latest/policy/CMP0002.html Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This commit adds a workflow for building and testing with AVX512 enabled. Needed for tarantool/tarantool#9924 Relates to tarantool/tarantool#6787 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
The test `lj-366-strtab-correct-size.test.lua` has a test helper `read_file` that reads a file's content and returns it. This helper will be useful for a test upcoming in the next commit, so it is moved to test tools. Needed for tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Carlo Cabrera. (cherry picked from commit 3065c91) The Mach-O FAT object constructed by LuaJIT had an incorrect format. The problem is reproduced when the target hardware platform has AVX512F and LuaJIT is compiled with enabled AVX512F instructions. The problem arises because LuaJIT FFI code for Mach-O file generation in `bcsave.lua` relies on undefined behavior for conversions to `uint32_t`. AVX512F has the `vcvttsd2usi` instruction, which converts `double`/`float` to `uint32_t/uint64_t`. Earlier architectures (SSE2, AVX2) are sorely lacking such an instruction, as they only support signed conversions. Unsigned conversions are done with a signed convert and range shifting - the exact algorithm depends on the compiler. A side effect of these workarounds is that negative `double`/`float` often inadvertently convert 'as expected', even though this is invoking undefined behavior. Whereas `vcvttsd2usi` always returns 0x80000000 or 0x8000000000000000 for out-of-range inputs. The patch fixes the problem, however, the real issue remains unfixed. Sergey Bronnikov: * added the description, the test for the problem and flavor with AVX512 to `exotic-builds-testing` workflow Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Carlo Cabrera. (cherry picked from commit b98b372) Mach-O FAT object files generated by LuaJIT for ARM64 had an incorrect format due to the usage of the 32-bit version of the FFI structure. This patch adds the 64-bit structure definition and uses it for ARM64. Sergey Bronnikov: * added the description and the test for the problem Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch adds Undefined Behaviour Sanitizer [1] support. It enables all checks except several that are not useful for LuaJIT. Also, it instruments all known issues to be fixed in future patches (except `kfold_intop()` since cdata arithmetic relies on integer overflow). [1]: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html Resolves tarantool/tarantool#8473 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Also, this patch sorts the corresponding flags in the CI workflow alphabetically for better readability. Relates to tarantool/tarantool#8473 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch is a follow-up to the 6e3aad8 ("OSX/iOS/ARM64: Fix generation of Mach-O object files."). During the backporting of this patch, the <build> directory was added to the git tree. This patch removes it and adds the <build> directory to the .gitignore to avoid such mistakes in the future. Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit 4a22050) When saving FPR registers during while from a trace and restoring data from a snapshot, UB sanitizer produces the following warning: | lj_snap.c:804:32: runtime error: index 23 out of bounds for type 'intptr_t [16]' due to indexing `ex->gpr` with a fpr register, whose number is >= `RID_MAX_GPR`. The situation itself is harmless since this is read from `spill[256]` array and is rewritten in the next if branch. This patch fixes the out-of-bounds access to read from `ex->gpr` only conditionally. Also, it removes the corresponding UBSAN suppression. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#9924 Relates to tarantool/tarantool#8473 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by minoki. Recent C compilers 'take advantage' of the undefined behavior. This completely changes the meaning of expressions like (k == -k). (cherry picked from commit 8a5e398) This patch changes all possibly dangerous -x operations on integers to the corresponding two's complement. Also, it removes all related UBSAN suppressions, since they are fixed. Also, this patch limits the `bit.tohex()` result by 254 characters. There is no testcase for `strscan_oct()`, `strscan_dec()` or/and `STRSCAN_U32` format since first the unary minus is parsed first and only after the number itself is parsed during parsing C syntax. So the error is raised in `cp_expr_prefix()` instead. For parsing the exponent header, there is no testcase, since the power is limited by `STRSCAN_MAXEXP`. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#9924 Relates to tarantool/tarantool#8473 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Suggested by Sergey Kaplun. (cherry picked from commit d2fe2a6) This patch replaces the numeric value of NYI bytecodes that can't be compiled with their names in the `jit.dump()` and -jv outputs. Since the functionality is the same, only `jit.dump()` is tested as most popular. Sergey Kaplun: * added the description and the test for the feature Part of tarantool/tarantool#9924 Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Bronnikov. (cherry picked from commit dda1ac2) The finalizers table is created on initialization of the `ffi` module by calling the `ffi_finalizer()` routine in the `luaopen_ffi()`. `ffi.gc()` is referenced by Lua stack via the `ffi` library, and the finalizer table is anchored there as well. If there is no FFI module table anywhere to anchor the `ffi.gc` itself and the `lua_State` object is marked after the function `ffi.gc` is removed from it (since we stop the GC before chunk loading and start after), then the finalizer table isn't marked. Hence, after the atomic phase, the table is considered dead and collected. Since the table is collected, the usage of its nodes in the `lj_gc_finalize_cdata()` leads to heap-use-after-free. The patch fixes the problem partially by marking the finalizer table at the start of the GC cycle. The complete fix will be applied in the next patch by turning the finalizer table into the proper GC root. Sergey Bronnikov: * added the description and the tests for the problem Part of tarantool/tarantool#10199 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Bronnikov. (cherry picked from commit f5affaa) The previous patch fixes the problem partially because the introduced GC root may not exist at the start phase of the GC cycle (since it isn't marked because it is not accessible from any GC root). In that case, the cdata finalizer table will be collected at the end of the cycle. Access to the cdata finalizer table exhibits heap use after free. The patch turns the finalizer table into a proper GC root. Note that the finalizer table is created at the initialization of the main Lua State instead of loading the FFI library. Sergey Bronnikov: * added the description and the test for the problem Part of tarantool/tarantool#10199 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Bronnikov. (cherry picked from commit 0f8b878) Part of tarantool/tarantool#10199 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Bronnikov. (cherry picked from commit 7110b93) Previously, LuaJIT generated Mach-O FAT object files for ARM and ARM64 on macOS. The patch removes support of 32-bit Mach-O object files and FAT (Universal Binary) object files for OSX and now LuaJIT generate only 64-bit Mach-O object files. Sergey Bronnikov: * added the description and the trimmed the test for the problem Part of tarantool/tarantool#10199 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Before this patch +jit flag is given to the LuaJIT test suite unconditionally even when LuaJIT is built as a pure interpreter. This patch fixes the behaviour by setting the flag only when `LUAJIT_DISABLE_JIT` option is not set. Needed for tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch syncs up the description of the LuaJIT-tests suite with the current status. Also, it adds a description of the subcategories of tests on which future patches will be based. Relates to tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch moves the <alias_alloc.lua> test from the <misc> to the created <opt/mem> directory, includes it in <index>, and names subtests. Testing optimizations in <lj_opt_mem.c> requires CSE, DCE, FOLD, and FWD optimizations enabled. Hence, the corresponding flags are used for the <mem> directory in <index>. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch refactors the aforementioned test to make its code style closer to ours. Relates to tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch refactors the <lang/coroutine.lua> test by adding a simple check for the content of a `debug.traceback()` result. Also, the test is moved to the <lib/coroutine/> directory. Also, it removes the <misc/coro_traceback.lua> since it is the same test that wasn't cleaned up during the refactoring in the original repository. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch removes the aforementioned test since it is identical to the <lib/coroutine/yield.lua> test, which wasn't cleaned up during the refactoring in the original repository. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch moves the aforementioned test from the <misc> to the <lang/> directory (with slightly renaming to be consistent with other names), includes it in <index>, and names the subtest. Also, it changes the number of iterations to trigger the GC since the number of objects is different when running the test as a part of the LuaJIT test suite. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch moves the aforementioned test from the <misc> to the <lang/> directory, includes it in <index>, and names subtests. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch refactors the aforementioned test to make its code style closer to ours. Relates to tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch removes the aforementioned test since it is part of the <lang/for.lua> test. Part of tarantool/tarantool#9398 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit a553b3d) This call returns `true` if the LuaJIT is built in DUALNUM mode, `false` otherwise. It may be useful for testing, trace semantics verification etc. All tests where skipconds are relied on the DUALNUM build are updated with the use of the `ffi.abi()` instead of previous implementations of checks. Sergey Kaplun: * added the description for the feature Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Kaplun. (cherry picked from commit b1cd2f8) `lj_opt_narrow_unm()` in the DUALNUM mode narrows doubles too optimistic, missing 0 check. In that case, the narrowing of 0 is incorrect. This leads to the assertion failure in `rec_check_slots()` for the string obtained from the corresponding number. This patch fixes it by restricting the check of the given TValue. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit 707c12b) In a single-number build (default for x86/x64), `-(0)` is always `-0` due to its number type. However, in DUALNUM mode, the `0` may have the integer type, and for this integer, `BC_UNM` has no effect. `0` may be represented via number or integer type, where the resulting `0` is constructed during initial parsing (`lj_strscan_scan()`) or arithmetic operations (`BC_MODVV`). This inconsistency may break the JIT semantics and lead to the assertion failure in the `lj_check_slots()`. This patch fixes the incorrect `BC_UNM` for integer operand 0. Also, the narrowing optimization for the unary minus should be more strict to avoid incorrect narrowing for integer slots and make JIT intact with the VM semantics. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by ZumiKua. (cherry picked from commit c94312d) When executing cdata finalizer on some lua_State, this state is set to `cts->L`. If the state will be GC-ed and freed later, this reference becomes dangling, so any FFI callback will use this invalid reference. This patch fixes it by setting `cts->L` to the `mainthread` on the destruction of the referenced one. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit d459c6c) This patch is a follow-up to the commit 7505e78 "Handle on-trace OOM errors from helper functions.". Since this commit, the `pcall()` (and `xpcall()`) emits an additional snapshot, but the arguments to the pcall-ed function are momentarily purged from the snapshot since they are not used. Hence, in several cases `debug.getlocal()` can't find the corresponding slot for the previous frame. This patch prevents purging right after `pcall()`, `xpcall()` recording. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Kaplun. (cherry picked from commit 54a1626) For now, any non-ERRRUN error in the error handling preserves its own error message. Thus, instead of the 'error in error handling', the caller gets an 'out of memory' or 'stack overflow' error. This patch adds the corresponding status check in the custom error handlers. The "error in error handling" string is also added to the preallocated strings to avoid OOM during error message allocation. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Stefan Hett. (cherry picked from commit 471f893) This commit fixes the check for the pointer returned by the internal LuaJIT allocator. For non-GC64 mode, the accessible address range should fit in 31 bits due to VM (and JIT) restrictions. This commit fixes the check. Since it is just an assertion check, there are no tests added for the change. But this commit simplifies the reproducer for the next patch (although not required). Sergey Kaplun: * added the description for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit eff4006) For the non-GC64 build with disabled JIT, LuaJIT's internal allocator may return 32-bit addresses. This may lead to the assertion failure during the reallocation of the array part of the table or to the crash (if assertions are disabled) due to an incorrect arithmetic in the x86 VM. For example, the addition with a 32-bit wide address may overflow in TSETV or TGETV and cause the crash. This patch sets the allocation limit for the build without JIT. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit 2aec641) MIPS processors originally required all memory accesses to be naturally aligned. If we use ld instruction to load a double-word from the address which is word-aligned, MIPS raises the exception SIGBUS. When exiting the interpreter, if the current function is a fast function, the code in the `lj_vm_exit_interp()` throws SIGBUS. The pc field for the fast function points to the word-aligned bytecodes for ASM fast functions, and PC2PROTO offset is double-word-aligned. The resulting address is somewhere in the dispatch table. Hence, some (odd-indexed) fast function access leads to the BUS error. For other architectures the load from unaligned access is not a problem so there are no exceptions. This patch prevents unaligned memory access by address loading only after fast-function checks. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to François Perrad and Stefan Pejic. (cherry picked from commit ea7071d) The patch fixes `xpcall()` segfaults on MIPS64 architecture. The similar patch for ARM64 has been backported (with the test) previously, see the commit af889e4 ("ARM64: Fix xpcall() error case (really)."). Sergey Bronnikov: * added the description Part of tarantool/tarantool#12134 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Peter Cawley. (cherry picked from commit d1a2fef) Stack overflow can cause a segmentation fault in a vararg function on ARM64 and MIPS64 in LJ_FR2 mode. This happens because the stack check in BC_IFUNCV is off by one on these platforms without the patch. The original stack check for ARM64 and MIPS64 was incorrect: | RA == BASE + (RD=NARGS) * 8 + framesize * 8 >= maxstack while the stack check on x86_64 is correct and therefore is not affected by the problem: | RA == BASE + (RD=NARGS+1) * 8 + framesize * 8 + 8 > maxstack The patch partially fixes the aforementioned issue by bumping LJ_STACK_EXTRA by 1 to give a space to the entire frame link for a vararg function as the __newindex metamethod. A fixup for a number of required slots in `call_init()` was added for consistency with the non-GC64 flavor. The check is too strict (if comparing the corresponding checks in the VM BC_IFUNCV), so this can't lead to any crash. To avoid possible regression in the future the corresponding test is added. This patch also corrects the number of redzone slots in luajit-gdb.py and luajit_lldb.py to match the updated LJ_STACK_EXTRA, see details in [1]. Sergey Bronnikov: * added the description and the test for the problem Part of tarantool/tarantool#12134 1. LuaJIT#1402 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Analyzed by Peter Cawley. (cherry picked from commit a4c1640) The `pcall()` and `xpcall()` calls in GC64 mode require 2 slots. This means that all arguments should be moved up during emitting of the frame link to the stack. Hence, this may cause stack overflow without the corresponding check. This patch adds the corresponding checks to the VM. Non-GC64 VMs are updated as well for the consistency. Sergey Bronnikov: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit 54cce2e) The -0 step and NaN control variable values may lead to the traces with always failed guards. This patch forbids recording of such traces since these traces are not very useful. Unfortunately, this breaks for loop recording in DUALNUM mode. This will be fixed in the next commit. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Nicholas Davies. (cherry picked from commit 1c3b5a4) The previous commit forbids recording for the NaN control variables stack slots. These checks are false positives for integers in DUALNUM mode. This prevents any for loop with integers as slots from being recorded. This patch fixes the checks. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Peter Cawley. (cherry picked from commit a5d2f70) If the OOM or stack overflow error is obtained during `lj_state_growstack()` in the `coroutine.resume()` it may raise an error leading to the incorrect error message and unexpected behaviour for this corner case. Also, `lua_checkstack()` may raise an OOM error leading to panic on an unprotected lua_State, for example. This patch handles these cases by protecting `lj_state_growstack()`. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
The patch introduce flags in module "misc" with support status for sysprof and memprof: `misc.sysprof.available` and `misc.memprof.available`. Both flags are boolean and always available on platforms supported by profilers (Windows is not supported). Resolves tarantool/tarantool#12215 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Vladimir Davydov and Sergey Kaplun. (cherry picked from commit 02e2999) This patch is a follow-up for the commit 6cee133 ("FFI: Resolve metamethods for constructors, too."). It allows the use of metamethods of the objects to be called on their constructors. Unfortunately, the JIT part isn't updated and may cause inconsistencies since it is indexing the field of the structure referenced by the given CType. This patch makes the semantics of JIT the same as for the VM. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Kaplun. (cherry picked from commit 5c64775) If `lj_state_checkstack()` in the `lj_vmevent_prepare()` raises the stack overflow error, `J->cur.startpc` is uninitialized during the recording, and its further inspection during trace blacklisting leads to the crash. To prevent any further inconsistencies in the JIT state due to the VM event handlers, all these handlers run in the separate `lua_State` (named V), which is stored in the `global_State`. Now the debugging in GC finalizers is less verbose. Hence, `debug.getinfo(2)` will return `nil`, since the finalizer is called on the separate Lua thread without main context. Also, the VM event `errfin` handler invocation is broken and will be fixed in the next commit. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Kaplun. (cherry picked from commit fbb36bb) After the previous commit, the VM handler for the 'errfin' VM event works incorrectly. The error object is taken not from the stack on which the VM handler is invoked. Hence, it breaks the non-VM stack and returns an incorrect error message. This patch fixes the issue by copying the object from the corresponding Lua stack. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Reported by Sergey Bronnikov. (cherry picked from commit f322ecb) The UndefinedBehaviour sanitizer produces a runtime warning when INT_MAX is passed to `unpack()` as index `i`. This happens because `i` in lj_cf_unpack() was incremented before the checking loop invariant and this leads to a signed integer overflow. The patch fixes the issue by moving index incrementation in the separate statement from the loop invariant check. Sergey Bronnikov: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
In CI the aforementioned test may be flaky since there is no invocation of the profiler's timer due to the too fast payload run. This patch makes the payload run almost one and a half times longer to be sure that we obtain at least one event. Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
Thanks to Sergey Kaplun. (cherry picked from commit ab834de) If the VM event contains a trace, it may cause several inconsistencies during the recording of another trace: - If there is an exit from the trace in the VM event for the 'trace start' VM event, the JIT engine converts the newly recorded trace to the "side trace". So, when this side exit is taken, the JIT returns from the VM event in the middle of another frame. - Stitching semantics are broken for the VM events due to an inconsistent frame link chain. This patch fixes these issues by forbidding stitching in the VM event and saving the context of the JIT engine at the VM event for trace start. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch is a follow-up to the commit 1e5887d ("FFI: Avoid dangling cts->L."). The `target_link_options()` feature is introduced in CMake 3.13. Unfortunately, we have distros in Tarantool's CI with an older default CMake version. This patch workarounds this by the passing `LINK_FLAGS` directly. Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
We have encountered the flakiness of the network on our CI runners. It happens only on lua-cjson installation and only in some particular job run. This patch works around it by installing the package only if it is not installed in the system. Also, it installs it only for Lua 5.1, which is compatible with LuaJIT, to avoid installation of modern Lua versions by default, since Lua 5.1 is not the default for most modern distros. Relates to tarantool/tarantool#12600 Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch adds tests for LuaJIT debugging extensions for lldb and gdb. The tests are written in Python's unittest framework [1]. Most of the tests are failed for the lldb due to outdated extension sources and overcomplicated hard-coded C structures fields introspection. Hence, tests for LLDB are disabled since they are failing anyway. The tarantool-debugger-tests target is introduced. This target is included in the LuaJIT-check-all target but not in the LuaJIT-test target to avoid it running for all LuaJIT builds by default in CI. [1]: https://docs.python.org/3/library/unittest.html Co-authored-by: Sergey Kaplun <skaplun@tarantool.org> Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch refactors lldb python extension C structure casts and type handling. Now it uses a monkey-patched lldb.value class instead of a handwritten one. Hence, there is no need for hardcoded C type structure fields. All LuaJIT-related functions use the same semantics as in the gdb extension. Also, this patch changes all fields' access to item access instead of attribute access. This commit fixes all tests except the initialization check since the order of loaded commands is different from gdb. This will be fixed in the next commit. Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch sorts the commands for extensions' initialization alphabetically. Also, it fixes the typo in the lldb extension. This allows enabling the loading test for LLDB as well. Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This commit adds the missed support for the full-range 64-bit light userdata, which was lost during lldb extension implementation. The corresponding entries for the test of lj-tv are sorted according to LJT* types. Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This patch joins the LLDB and GDB LuaJIT debugging extensions into one, so now the extension logic can be debugger-agnostic. To do that, an adapter class is introduced, and all of the debugger-specific behavior is encapsulated there and in its child debugger classes with the specific implementations of required methods. The extension auto-detects the debugger it was loaded into and selects the correct low-level logic implementation. The F821 ignore is removed from <.flake8rc> since there is no more need for it. Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
The debugger extension test is run for both LLDB and GDB for GC64 and non-GC64 builds for arm64 and x86_64. Reviewed-by: Evgeniy Temirgaleev <e.temirgaleev@tarantool.org> Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org> Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.