Safe rendering for monorepo packages

[ikusakov2]

Is your feature request related to a problem? Please describe.

Our monorepo is structured this way:

packages/
        /package-1
        /package-2
        /package-lib-1
        /package-lib-2
        ...

A package could import fragments from other packages.
For example, package-1 may import fragments from package-lib-1

How to pass this information to GraphQL Codegen?
We put it in the documents:

documents: ['packages/yelp-1/src/**/*.{ts,tsx}', 'packages/yelp-lib-1/src/**/*.{ts,tsx}']

This way the codegen will pick up gql for both packages.

But the codegen will also generate type files for both packages!

This is not acceptable for us. Imagine both package-1 and package-2 using fragments from the the same package-lib-1. If we try to generate packages one by one, package-lib-1 will be generated 3 times - one for package-1, second for package-2 and third for package-lib-1 itself!

The above becomes much worse because we parallelize the generator script for the whole monorepo with node worker threads. So - multiple processes may simultaneously try to re-create type files for package-lib-1. We had some occasional strange bugs early on with this setup.

To address this - I looked at another way of defining which packages to process - using the generates property:

generates:[ 
   `packages/package-1/src`: {...}
]

I thought the above code would limit the generation only to the package-1  package.
But it did not work this way.

It looked like the near-operation-file-preset just ignores what is defined in generates and always creates files for all paths defined in documents . But - maybe I misunderstand the philosophy behind using documents vs generates.

So, I decided to fix near-operation-file to work as I thought it should.
The fix is in resolve-document-imports.js file.
I replaced:

return documents.map(documentFile => { with:
return documents.filter(documentFile => documentFile.location.includes(`${baseDir}/${baseOutputDir}`)).map(documentFile => { In the above patch:

baseDir in our case is an absolute path to the monorepo root
baseOutputDir is a relative path to generated package (from generates): packages/package-1/src

I originally filtered to match baseOutputDir only, but it did not work well with all the setups I tried (absolute path vs relative path). Combining it with baseDir gave a solid fix for all cases (at least to what I remember).

So when codegen generates files, everything which is not defined in generates
is filtered away. Codegen still loads the files from other packages for processing, but it does not generate anything in them - which was the goal.

Describe the solution you'd like

Ok I understand the use case, thanks [@ikusakov2]!
This feels like a fairly common issue for monorepos, and not limited to the usage of near-operation-file.

In my mind, each generates block may have a few scenarios for its target documents:

Documents in the immediate folder need to have files generated
Documents in "external" packages are read-only and do not need to have files generated.

I propose creating a Codegen Core-level config called documentsReadOnly to serve the second purpose. For example:

const sharedDocs = ['packages/yelp-lib-1/src/**/*.{ts,tsx}','packages/yelp-lib-2/src/**/*.{ts,tsx}']

const config: CodegenConfig = {
  // ...
  generates: {
    'packages/package-1/src': {
      documents: ['packages/package-1/src/**/*.{ts,tsx}'],
      documentsReadOnly: sharedDocs, // Reads docs from the libs, but don't generate types for those
      // ...
    },
    'packages/package-2/src': {
      documents: ['packages/package-2/src/**/*.{ts,tsx}'],
      documentsReadOnly: sharedDocs, // Reads docs from the libs, but don't generate types for those
      // ...
    },

    'packages/package-lib-1/src': {
      documents: ['packages/package-lib-1/src/**/*.{ts,tsx}'], // Note: a generates its own types, so no `documentsReadOnly` if it doesn't read from other libs
      // ...
    },

    'packages/package-lib-2/src': {
      documents: ['packages/package-lib-2/src/**/*.{ts,tsx}'],
      documentsReadOnly: ['packages/package-lib-1/src/**/*.{ts,tsx}'] // Note: if this lib reads/loads fragments from lib-1, we can use this. Or maybe use `sharedDocs` here, which includes lib-2 path, but Codegen knows that it'd need to write to lib-2
    },
  }
}

Describe alternatives you've considered

It is possible to just filter the output in near-operation-file preset, 1 line fix.

Any additional important details?

No response