From plone.app.testing docs:
Create a configuration context with the disable-autoinclude feature set. This has the effect of stopping Plone from automatically loading the configuration of any installed package that uses the z3c.autoinclude.plugin:plone entry point via z3c.autoinclude. (This is to avoid accidentally polluting the test fixture - custom layers should load packages’ ZCML configuration explicitly if required).
The problem with this approach is that we have a perfectly functional website, but when trying to test it, a bunch of the tests simply do not work because the instance from bin/test is different from bin/instance.
We've already tried to ask about more convention over configuration in the test framework to ease the pain of users in Plone, with the approach we think is the best cost benefit from an user perspective, without success, but we comprehend the performance problem and the conceptual definition that the test should be the minimal possible (we were trying a hybrid, pragmatic approach).
Fact is: we've been bitten by these differences in Plone itself and it's test layers and we don't know how to solve the problem: we don't know which packages should be explicity added since the dependency tree is really huge, so is pratically impossible to do it.
We know we should be creating packages that don't rely in z3c.autoinclude, but the real world isn't yet perfect specially when talking about third-party packages.
The question is: is there at least a way to compare what's been automatically included when running bin/instance and bin/test? This can be a log, a snippet of code I need to run, a view, anything that can ease the pain when testing our infrastructure.
Another question: is there anything I need to compare as well that isn't automatic loaded in plone.app.testing? We're not that confident anymore with our tests since they're not running on same code/runtime as prodution.
Somewhat related: https://community.plone.org/t/what-is-best-practice-for-meta-zcml-files-and-test-layers-in-plone-app-testing/442
@tomgross @mauritsvanrees @tisto @jensens @hannosch We're adding you to the discussion from previous conversations and knowledge we believe you have about this issue, feel free to remove yourselves if not related. Thanks for your time.
From plone.app.testing docs:
The problem with this approach is that we have a perfectly functional website, but when trying to test it, a bunch of the tests simply do not work because the instance from
bin/testis different frombin/instance.We've already tried to ask about more convention over configuration in the test framework to ease the pain of users in Plone, with the approach we think is the best cost benefit from an user perspective, without success, but we comprehend the performance problem and the conceptual definition that the test should be the minimal possible (we were trying a hybrid, pragmatic approach).
Fact is: we've been bitten by these differences in Plone itself and it's test layers and we don't know how to solve the problem: we don't know which packages should be explicity added since the dependency tree is really huge, so is pratically impossible to do it.
We know we should be creating packages that don't rely in z3c.autoinclude, but the real world isn't yet perfect specially when talking about third-party packages.
The question is: is there at least a way to compare what's been automatically included when running
bin/instanceandbin/test? This can be a log, a snippet of code I need to run, a view, anything that can ease the pain when testing our infrastructure.Another question: is there anything I need to compare as well that isn't automatic loaded in plone.app.testing? We're not that confident anymore with our tests since they're not running on same code/runtime as prodution.
Somewhat related: https://community.plone.org/t/what-is-best-practice-for-meta-zcml-files-and-test-layers-in-plone-app-testing/442
@tomgross @mauritsvanrees @tisto @jensens @hannosch We're adding you to the discussion from previous conversations and knowledge we believe you have about this issue, feel free to remove yourselves if not related. Thanks for your time.