frontend: add build-time source filters#1061
Conversation
a2ba2aa to
8b69774
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds build-time source filtering support via a YAML config supplied through a build context, so package/source generation can exclude globally configured paths without changing the Dalec spec.
Changes:
- Adds
SourceFilterConfig, build args, frontend loading, and source/generator filtering hooks. - Propagates filter metadata into RPM spec generation and source docs.
- Adds unit/integration coverage and user documentation for source filter behavior.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
website/docs/sources.md |
Documents build-time source filter configuration and Buildx usage. |
test/source_test.go |
Adds integration tests for debug source and gomod filtering. |
targets/windows/handle_zip.go |
Adjusts Windows binary build source options around filtering. |
spec_test.go |
Tests YAML decoding and emptiness checks for filter config. |
source.go |
Adds filter plumbing to source fetch options and source docs. |
source_test.go |
Adds unit tests for source filter LLB behavior. |
source_inline.go |
Adjusts inline directory include/exclude paths when renamed. |
source_filter.go |
Introduces source filter config, constants, and LLB filter helpers. |
packaging/linux/rpm/template.go |
Emits source filter documentation and pip source entries in RPM specs. |
packaging/linux/rpm/template_test.go |
Updates RPM source expectations and filter documentation tests. |
packaging/linux/rpm/buildroot.go |
Passes source filter config into RPM spec generation. |
load.go |
Marks source filter build args as known arguments. |
generator_pip.go |
Applies filters to generated pip dependency cache. |
generator_gomod.go |
Applies filters to generated gomod dependency cache. |
generator_cargohome.go |
Applies filters to generated Cargo dependency cache. |
frontend/request.go |
Loads source filter config from a build context. |
frontend/gateway.go |
Wires cached source filter loading into frontend source options. |
frontend/debug/handle_sources.go |
Passes progress options while preparing debug sources. |
Comments suppressed due to low confidence (1)
packaging/linux/rpm/template.go:386
- Filter patterns are written directly into the generated RPM spec comment. If a pattern contains a newline, only the first line is prefixed with
#, so the remaining text can become active spec content; comment each line (asSourcesdoes when scanningsrc.Doc) or sanitize newlines before writing.
}
if len(keys) > 0 {
b.WriteString("\n")
9ec2c7c to
7df6aad
Compare
Add a source-options context that can provide a YAML source filter config at build time. The config currently supports global_excludes which can be used to filter out paths from all sources. Signed-off-by: Brian Goff <cpuguy83@gmail.com>
7df6aad to
efcebc9
Compare
| func (s *Source) fetchOptions(sOpt SourceOpts) (fetchOptions, error) { | ||
| excludes := s.Excludes | ||
| if s.IsDir() { | ||
| globalExcludes, err := sOpt.sourceFilterExcludes() |
There was a problem hiding this comment.
I think this might be a little too early in the flow.
Since this gets added in Source.fetchOptions, it looks like the filter applies any time we materialize a directory source, including sources that are only being used as inputs to generators. IIUC that means a filter could exclude something like go.sum, requirements.txt, or Cargo.lock before the generator has a chance to use it.
Since we also filter the generated cache later, maybe this should live closer to the final source-package/debug-source assembly instead of in the generic source path?
Could we add a regression test for this case just to make sure?
Add a source-options context that can provide a YAML source filter
config at build time. The config currently supports global_excludes
which can be used to filter out paths from all sources.