Automatic task payload (de)serialization using different serializers.

[code-of-kpp]

Right now we only support raw payloads. Would be nice to have options for pickle, safe_pickle, json, yaml and so on.

[ivasio]

I think the good way to specialize these options would be passing string pickle, json etc. as a parameter to Queue.__init__() . Task classes would imlement some serializers and Queue.__init__ would check if a Task class implements this serializer in format_payload()

[ivasio]

And in order to be able to deserialize task payloads we need either to save serialization method in database representation of Queue objects or encode it in every payload. First approach looks more reasonable

[code-of-kpp]

Probably, we need to work with task meta-information. For example, we can encode serialization/deserialization information in task-id. Or we can use (probably fixed) part of payload for that.

Anyway, it looks like serialization/deserialization should live inside a Task class. It should be able to decode any kind of payload, there should be some defaults for serialization, a way to override this defaults for individual task and adding formats should be possible and easy.

[ivasio]

I think in the most popular case the same payload (de)serialization method would be used for all the tasks in any given queue. If we decide to store this exact piece of metadata in the Queue class, I think it will lessen the amount of data transmitted to/from the database which is surely better. But at the same time we put and take the task data anyway and the flexibility will be provided seamlessly. And probably the best way to do this is combining both approaches — full flexibility if no serialization method was provided explicitly to the Queue constructor, otherwise all the tasks put/got via this Queue instance will use provided method