Improve DeferredClasspathContainerInitializerJob.belongsTo

[merks]

• Add ClasspathContainerInitializer.class to the test for families in belongsTo to allow a client to wait for classpath container initialization without knowing or caring about PDE specifically.

[laeubi]

Seems like a clever addition here.

I think we should then add a note to ClasspathContainerInitializer javadoc to make integrators aware of this option.

[merks]

@iloveeclipse

@iloveeclipse

This change is to enable the following change in JDT:

I.e., this delays the reconcile thread (which shows the error annotations) until after PDE's classpath initializer has completed, without introducing any PDE dependencies.

With this combination of changes I see no (temporary) error annotations in the editor, except of course when there are real errors in the editor in which case they are not temporary.

At first glance one might be concerned about "blocking" the reconciler thread, but keep in mind that the thing we are waiting for is precisely the thing that we previously waited for on the main thread upon startup during the editor's set input. Here the waiting is done in a background thread (not even a job), and of course if PDE isn't installed or PDE isn't initializing the classpath container in the background, which happens only during startup, there is no actual delay. I.e., we are waiting in a place where the users is not blocked by the wait...

[iloveeclipse]

@merks : sounds good, I only wonder if that should be a constant in ClasspathContainerInitializer with appropriate javadoc, explaining the possibility to clients indicate that there might be jobs running still related to the container initialization. So far we had no such "lazy" ClasspathContainerInitializer in SDK, before #1552, so there were no need for waiting on ClasspathContainerInitializer family.

Basically you are adding & using API in JDT to do similar things which @laeubi didn't wanted to do before in PDE.
See eclipse-jdt/eclipse.jdt.ui#2245 (comment) and #1876

[github-actions]

Test Results

   765 files  ±0     765 suites  ±0   57m 34s ⏱️ -7s
 3 611 tests ±0   3 557 ✅ ±0   54 💤 ±0  0 ❌ ±0 
10 834 runs  ±0  10 671 ✅ ±0  163 💤 ±0  0 ❌ ±0 

Results for commit a4d8aa0. ± Comparison against base commit aa2d044.

[iloveeclipse]

Thinking a bit more: this would also delay reconciler on non PDE code, which is not great. Let assume PDE loads half of internet via m2e target, and ome just wants edit plain Java project.

[merks]

If I try a version of that scenario, I see the "non-PDE" editor is already opened and that the reconciler has passed the waiting point:

Only then in the background some the Java initializer ask for the classpaths; those will not be deferred because jobs are running by this point.

--

In a different scenario where the classpath is needed by the explore, which happens on the main thread like it does when an editor from a PDE project is open like this:

Here in the past we would have the delay that concerns you on the main thread on startup.

---

So in the one case we don't wait at all and in the other case for which we would have waited in the main thread and we now would wait in a background reconciler thread.

So I see no significant cause for concern for what's not really such a likely scenario in the first place.

---

As for new API, I'm not sure we need anything heavy weight. I think some Javadoc would suffice. (And even then, how many future new implementations of classpath containers do we realistically expect, and lazy ones at that?)

[iloveeclipse]

Ed, I mean following scenario:

1. Add breakpoint in the PDE job emulating "longer task".
2. After startup, while the job is still running, open "plain" Java editor.
3. Modify some code, add error etc.
4. Is the JavaReconciler now waiting on PDE job? Errors would not be shown etc.

[merks]

I think it's important to keep in mind that there will be a wait if and only if PDE is doing deferred initialization. That happens if and only if the classpath is requested while the job manager is suspended. That can only happen really, really early and is known to happen only when a Java editor or the Explorer asks for it super early during IDE startup. So I don't even know how to cause such super early initialization (such that the breakpoint in the initialization job would be hit) other than via the two scenarios we've explored already. I.e., an open editor on Java file in PDE project or something that forces the Explorer to show required-plugin details in the provider (via the link-with-editor scenario above).

Both those scenarios are ones where the main thread would have blocked which is definitely worse than the reconcile thread blocking.

In other words, the normal path would this:

So there would be no deferred job and will never be a deferred job in this JVM and in the above the long delay would happen in the initialize job, where it is expected and unavoidable.

So conceptually I cannot think of a way for the scenario-of-concern to actually happen...

[iloveeclipse]

Both those scenarios are ones where the main thread would have blocked which is definitely worse than the reconcile thread blocking.

Sure.

I think it's important to keep in mind that there will be a wait if and only if PDE is doing deferred initialization.

Sure.

That's what we see now with startup if a Java editor is opened, and I believe everyone has some editors opened.
So we are talking about use case of workspaces with mixed projects and slow PDE init.

What I mean is: now we would deferring reconciler for every Java file opened, independently whether it is related to PDE or not. Ideally, I would expect it should only affect Java editors for PDE projects, the rest shouldn't be affected, but the envisioned patch would do that.

Why I'm concerned: originally proposed patch from Mehmet would not have this problem...

[merks]

What I mean is: now we would deferring reconciler for every Java file opened.

Yes, if an editor on a PDE project is opened and is visible then all editors will be block. Of course this is the scenario where the entire UI thread was blocked on startup for the entire period of time. It was that way for decades, and then very few people seemed to care about such a serious problem. So you'll probably understand why I'm confused by corner cases of mixed PDE and non-PDE scenarios that were completely a disaster in the past. I.e., why is it a concern now but never was a concern before. Anyway, I feel like I'm falling into the trap of dredging up history, and I'm tired of seeing that because it doesn't move forward.

In any case, this whole discussion is about scenarios is concerning what JDT can do, should do, or might do. It's not about about this specific PDE PR which simply provides the ability to wait if desired; it could use a constant, I don't really care. Also, if you prefer to actually solve the problem in some other way in JDT, that's of course your choice as JDT committers to make and I'll not get int he way.

What I would like is is to put this is place, even if it's useless. And I would like to create PR for JDT, which does solve the problem and which you can choose to ignore in favor of something better. Unfortunately the time I have available only allows these options for me. I've become completely confused by the competing different solutions, and class editor versus java editor, with or without PDE, all of which are rejected for various reason along with way too much emotional/personal baggage for my taste. Way too much.

So, we either move forward with something really simple that does work, or folks can invest their own time on other solutions as they see fit.

[iloveeclipse]

Ed, this is all clear, no doubts. I just wanted to make sure the problem is discussed and understood, and maybe something could be improved for that specific case.

[merks]

I do appreciate the attention detail. ❤️ In the end, you can't really be too fussy. But of course one person's admirable fussiness is the next person's obsessive-compulsive disorder. 🫤 I have been known to annoy other people unintentionally...

Are you okay to merge this?

We can always introduce a constant later if that's better; it's all internal details currently.

Even the join in JDT could always be removed later, when a (provably) better solution is implemented. Again it's all internal detail subject to change anyway...

Maybe for m2 we can try to provide a good-enough/okay solution and fine tune it as time permits because every time I start one of my IDEs and see red errors I want to cry. I think OMG how unprofessional does this look? And somehow I even take such shortcomings in the software personally.

[laeubi]

I fully agree with the analysis of @merks and I'm quite happy that @iloveeclipse agree that there is some (non trivial) blocking involved so we have a common ground to start from.

So the blocking was the reason for the initial work in PDE because it can lead to blocking forever (or quite long) so the Eclipse UI itself never shows up!

If one want to improve this I have created these issues since a while:

• [performance] PackageFragmentRoot.getRawClasspathEntry() has bad impact on startup performance eclipse-jdt/eclipse.jdt.core#3468
• If the job manager is suspended run the job directly eclipse-ecf/ecf#154
• [performance] CloseUnrelatedProjectsAction.buildConnectedComponents(IProject[]) blocks UI startup eclipse-platform/eclipse.platform.ui#2636
• [performance] JDT initialization blocks IDE startup eclipse-jdt/eclipse.jdt.ui#1860

any of those will remove one situation where the UI access the classpath early and is a possible case to block others work (not only Editors!).

Regarding downloading the internet, m2e and all the rest of the universe:

1. m2e (maven) since ever uses a different strategy than PDE and I like to adapt this here what always is completely asynchronous.
2. m2e target locations are using maven but are a different story, they have never blocked "forever" like ECF/P2 does and are usually quite fast and cached in the local repository. Anyways of course also this can take time, but the download itself usually takes place at a different time (except for snapshots) when you first time resolve the target.

P2 is different as it always checks at least metadata for updates and if you nuke your local repo also maven has to first check the remotes....