Document / Improve - Go monorepo

[felickz]

Today Go does not support a paths / paths-ignore based setup with CodeQL (until it supports build-mode: none).

Document a sample of how to best work with this action in a Golang monorepo. Identify improvements to this action/product!

[aegilops]

As long as Go ignores the paths/paths-ignore then having a sparse checkout might be enough for autobuild to function OK?

If not (I'm not sure how it works with nested folders?) for now you'll need to go down the manual build mode approach, and trigger Go with a manual build command.

We can explore how to better support examples such as a wholly Go-based monorepo and a wholly C# monorepo, to be able to get a more completely automatic default that works in those cases.

[fernandosantos-br]

We encountered difficulties in scanning Go projects, which led us to develop an alternative method for scanning a monorepo. Feel free to explore it at https://github.com/tweag/CodeQL-monorepo

[felickz]

We encountered difficulties in scanning Go projects, which led us to develop an alternative method for scanning a monorepo. Feel free to explore it at https://github.com/tweag/CodeQL-monorepo

Interesting! Do you have some samples of where Go projects are failing with this action?

The linked repo seems to follow the patterns of other documented monorepo samples:
https://github.com/advanced-security/awesome-codeql/blob/main/README.md#codeql-monorepo-actions-samples

I'll whip up a sample with a Go monorepo to show how @aegilops 's idea about sparse checkout with this project works (+ it handles many more edge cases as documented in the ReadMe here). The caveat will be to ensure your project definition has proper dependencies to compile grouped together.

[fernandosantos-br]

Hi @felickz ,

Here are the key differences between the approach in awesome-codeql and the solution we developed on https://github.com/tweag/CodeQL-monorepo:

• Bash-based implementation: The entire workflow is orchestrated using Bash scripts, providing simplicity, flexibility, and eliminating the need for Python or additional dependencies.
• PR/Merge Scanning: Supports scanning exclusively for the project directories impacted by a specific pull request or merge, substantially reducing overall scan duration and resource consumption. Note: This feature is not implemented in Awesome-CodeQL.
• Language Flexibility: Enables compilation when necessary (e.g., Java, C#) to prevent “low-quality scan” warnings and ensure more accurate results.

I also have examples of where Go projects fail when using this action. I’m currently on vacation but will share those details here next week.

[fernandosantos-br]

@aegilops @felickz Apologies, The difficulty we encountered when scanning Go projects is not related to this monorepo project.

The actual issue was that the Go project in question was built with Bazel, and for Go, CodeQL does not support builds with Bazel.

Bazel does not use the standard Go build tooling (i.e., go build). Due to this difference, adding support for Bazel-built Go projects is non-trivial.

Therefore, in practice, none of the monorepo scanning projects (whether this repository or the one we created) will be able to scan Go code that relies on Bazel.

Here's an example demonstrating that CodeQL won't scan a Go project based on Bazel (on a monorepo or not):
https://github.com/fernandomatsuosantos/poc_codeql_bazel

Please feel free to correct me if I'm mistaken.