Remove unused functionality from core parts of SILNLP

[benjaminking]

With SILNLP being used as production software, we need to make a stronger distinction between the core parts that are used for production and other parts that are only used for research. Part of this process will be to remove unused obsolete functionality from these core pieces. We are regularly finding that their presence is leading to the unintentional introduction of new bugs and is increasing the effort required to implement otherwise straightforward changes.

[benjaminking]

I wanted to add another quick note about why we're planning to remove this functionality. Some people might be wondering, "Why put in the effort to remove all these things if the software is working fine?" or "What if we end up needing some of these things later?" And these are good questions -- we've been wrestling though these same questions in our SILNLP meetings.

As we've been working through our normal cadence of adding features and fixing bugs, we've been more and more frequently finding ourselves in situations where making one simple looking change is generating 2 or 3 new bugs. And that is largely due to the current complexity of SILNLP. This effort of removing unused features is one prong of a multi-pronged approach to try to reduce the complexity of development. Each unused feature that we remove is one less source of unintended consequences, but an even greater impact may be in reducing the cognitive load that is required to make changes in SILNLP, making it easier to identify all the areas of the system that might be impacted by a change.

AI coding tools are also changing the conventional wisdom for software development. In the past, we might have felt more strongly about removing a feature that we could potentially want in the future and felt like we were throwing away the effort that went into it. But new code is becoming increasingly easy to generate, and this is making code (especially old untested code) a commodity that can be very easily regenerated when a need for it arises.

So in short, we're hopeful that this will help make SILNLP more stable and enable us to more quickly deliver new features with fewer bugs.