You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During some recent work on the information weight transform (#150, #153), I've found a few things I think could be improved. However, there would be changes to the behaviour and interface, so I'm posting the ideas here for discussion. The larger ideas are:
prior_strength usage is not intuitive
We currently add prior_strength*prior_distribution to the raw counts and then normalize to get the posterior distribution. This is a bit strange, if we want a very strong prior, the strength has to be greater then 1 and would probably need to be tuned for each column.
I want prior_strength to represent the proportion of weight on the prior, i.e. posterior is (1-prior_strength)count_distribution + prior_strengthprior_distribution. The range is clear, and it would work evenly across columns.
Speed up exact computation (and remove approximate option?)
Some minor tweaks will greatly speed up the exact kl-divergence computation (since we known indices are sorted, no need for the set, membership check, or np.searchsorted). In a quick implementation and test, it was almost as fast as the approx method. With some better precomputation, I think this will almost always be acceptably fast.
We can still have a method that takes advantage of sparsity, but I think the more natural way to allow this speed-up is to force prior_stregth=0. It was mentioned in [QUESTION] InformationWeightTransform #118 that the prior is useful for very sparse cases, but I'm not sure how the prior and the approximation interact (and if the prior is still helpful).
Allow semi-supervised targets
This is easy enough, the internals should be ready after Simplify and fix InformationWeight Transform with column groups #153. We can keep the current functionality for everything except for a numpy array of ints of class ids (and -1 for noise / unlabelled).
Some other notes:
Is normalizing by the mean necessary (it just scales all distances)
Should the default weight power be 1.0 so it does the described math
Add sklearn input validation and check estimator test
Update the docs
I'm happy to work on these changes (or other ideas that come up) slowly for the rest of summer. Since there are quite a few changes, I'll make a new branch to merge into, and then everything can be merged to main at once (and tagged with a new version?) Maybe it's also worth adding deprecation notices and a new release after #153.
During some recent work on the information weight transform (#150, #153), I've found a few things I think could be improved. However, there would be changes to the behaviour and interface, so I'm posting the ideas here for discussion. The larger ideas are:
prior_strength usage is not intuitive
We currently add prior_strength*prior_distribution to the raw counts and then normalize to get the posterior distribution. This is a bit strange, if we want a very strong prior, the strength has to be greater then 1 and would probably need to be tuned for each column.
I want prior_strength to represent the proportion of weight on the prior, i.e. posterior is (1-prior_strength)count_distribution + prior_strengthprior_distribution. The range is clear, and it would work evenly across columns.
Speed up exact computation (and remove approximate option?)
Some minor tweaks will greatly speed up the exact kl-divergence computation (since we known indices are sorted, no need for the set, membership check, or np.searchsorted). In a quick implementation and test, it was almost as fast as the approx method. With some better precomputation, I think this will almost always be acceptably fast.
We can still have a method that takes advantage of sparsity, but I think the more natural way to allow this speed-up is to force prior_stregth=0. It was mentioned in [QUESTION] InformationWeightTransform #118 that the prior is useful for very sparse cases, but I'm not sure how the prior and the approximation interact (and if the prior is still helpful).
Allow semi-supervised targets
This is easy enough, the internals should be ready after Simplify and fix InformationWeight Transform with column groups #153. We can keep the current functionality for everything except for a numpy array of ints of class ids (and -1 for noise / unlabelled).
Some other notes:
I'm happy to work on these changes (or other ideas that come up) slowly for the rest of summer. Since there are quite a few changes, I'll make a new branch to merge into, and then everything can be merged to main at once (and tagged with a new version?) Maybe it's also worth adding deprecation notices and a new release after #153.