Fix up frequency

[treeowl]

• Make frequency detect negative frequencies, frequency
  total overflow, and zero total frequency.

• Rewrite frequency to build an IntMap to represent the
  frequency list, which greatly improves efficiency for long
  lists.

Fixes #337.

[treeowl]

I could use another pair of eyes to make sure this produces precisely the right distribution. I believe it does, but I don't trust myself around fence-posts.

[treeowl]

Question: if no frequencies are positive, should we error out (as presently) or should we just discard in that case?

[treeowl]

I added a commit that discards when all the frequencies are zero. That feels more correct to me, but feel free to go with whichever you prefer.

[treeowl]

The fromIntegral isn't needed. Let's clean that up in merging.

[treeowl]

Ah, I just realized we can do a tiny bit better! We can make the IntMap one entry smaller than the frequency table, holding one element in reserve to handle failed lookup. That removes the "something went wrong" error that's been bothering me.

[treeowl]

While the IntMap is far better for very long lists, it's definitely not so great for the more typical short lists. What would be good for short lists? Sort the list by frequency in descending order, store the accumulated frequencies in an unboxed vector, store the values in a boxed vector, and do a linear scan.

In some cases, a hybrid mechanism may be best. If a few entries dominate, then it would make sense to store them in vectors, checked first, and the rest in an IntMap, as a fall-back. Sounds fussy.

[treeowl]

Ping? I haven't seen any comments on this yet.

[HuwCampbell]

Personally I'm not convinced about this one.

Almost always the input to frequency is hand written with a small number of choices and the time will be quite acceptable. It'll probably actually be faster as is for idiomatic use.