Gpu lidar 360 scan#160
Merged
Merged
Conversation
* Fix nng build with UE5 toolchain and replace deprecated pkg_resources Pass UE5 sysroot and find-root-path-mode args to nng's ExternalProject_Add to match how assimp is already configured, fixing undefined symbol errors (arc4random, strlcpy, __isoc23_strtoull) when linking against UE5's packaged toolchain. Replace deprecated pkg_resources with importlib.resources in the Python client for Python 3.12+ compatibility. * Add GPU lidar example and fix UE 5.7 render pass - Example script lidar_basic_gpu.py plus scene + robot configs that exercise the UGPULidar path via lidar-type: gpu_cylindrical. - Drop redundant VS/PS SetParameters in the UE 5.7 branch of the lidar intensity scene view extension. They dirtied scratch params before SetShaderParameters could commit, tripping the 5.7 !HasParameters() ensure. The View UB is still bound through the shader parameter struct. - Throttled [GPULidar DEBUG] logs across Simulate, TickComponent, and PrePostProcessPass_RT for pipeline diagnosis. * chmod +x blocks_genprojfiles_vscode.sh * Fix GPU lidar point cloud generation and visualization Root cause was a stride mismatch on the readback path: UE5's FVector4 is TVector4<double> (32 bytes), but the compute shader writes 16 bytes per point, so the memcpy silently shifted data across rows. Switching the receiving buffer to FVector4f restores alignment. Other fixes layered on top: - Rebuild Cam1ProjData from the live scene-capture transform every tick (init-time capture happens before the component is parented, collapsing the inverse view-projection to near-identity). - Compute and bind a forward ProjectionMatrix from a fresh FSceneViewProjectionData with axis-swap rotation, transposed for HLSL column-major mul. - Restructure the compute kernel to unconditionally write all four components per thread (sentinel xyz=-1 for misses, w=intensity). - Remove the VS/PS->SetParameters(GetScratchShaderParameters()) calls in the UE 5.7 path that dirtied scratch and tripped the !HasParameters() ensure inside SetShaderParameters. - Visualization: pass color_mode=COLOR_INTENSITY in lidar_basic_gpu.py (segmentation_cloud is filled with -1, which renders black on black). Configure symmetric +/-15 deg vertical FOV to work around the upper*2 / hardcoded-symmetric VFOV bugs in GPULidar.cpp and the shader. * Bisect GPU lidar fix down to minimal set vs main Revert the speculative pieces layered on top of the FVector4f stride fix, keeping only the changes that are demonstrably needed: - GPULidar.cpp: revert per-tick Cam1ProjData rebuild (back to main). - LidarPointCloudCS.usf: revert kernel restructure (back to main's conditional indexed writes). - LidarPointCloudCS.h: restore dead SHADER_PARAMETER_UAV comment. - LidarIntensitySceneViewExtension.cpp: revert UnrealLogger include, readback-bool refactor, and the AddClearUAVFloatPass removal — but keep the SetParameters fix and the FwdProjData → ProjectionMatrix block (the latter is needed for the shader's ProjectWorldToScreen). Net delta vs main is now: FVector4f data type, UE 5.7 SetParameters fix, and forward ProjectionMatrix wiring. * Restore trailing whitespace on blank line in lidar readback block Drops an incidental whitespace-only delta vs main so the file diff shows only the intentional SetParameters and ProjectionMatrix changes. * enabled 360 GPU lidar * LiDAR type detection test * modified mid70 settings --------- Co-authored-by: MarinaGloc <tu-correo-de-github@example.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #
About
This PR adds and validates GPU LiDAR support for 360-degree cylindrical scans.
Main changes:
lidar_basic_gpu.py.gpu_cylindricalLiDAR usage.FVector4finstead ofFVector4, matching the shader’s 16-byte point layout.blocks_genprojfiles_vscode.shas executable.Why
The previous GPU LiDAR path could not reliably produce a correct 360-degree point cloud. The main issue was a CPU/GPU data layout mismatch: the shader writes 16 bytes per point, while UE5
FVector4is 32 bytes, causing readback stride corruption. This change fixes that and expands the shader path so each 90-degree camera sector contributes correctly to the final cylindrical scan.How Has This Been Tested?
Added
test_lidar_types.pyplus dedicated test scene/robot configs to verify LiDAR type routing and configuration coverage.Screenshots and videos (if appropriate):