Fix inconsistent comparison of sets in encode when using regular expression

[p0ise]

In this code:

67    if expr:
68        for char in all_chars:
69            if re.search(expr, char):
70                avoid.add(char)

which make avoid something like {'a', 'b'}.

But set() of bytes would be {some ints}.

This will invalidate the following conditions because they can never intersect:

79        if encoder.blacklist & avoid:
...
87        if avoid & set(v):

[Arusekk]

Thank you! Current implementation of shellcode encoders is a mess in general, so I will take a look at it later.

[peace-maker]

Similar changes to the types in #1583 look like they are more thorough in ensuring the avoid bytes/set is applied correctly.

To reduce the amount of stalled ancient PRs, I'll close this and look into cherry picking the Python 3 bytes related fixes in the other PR without touching too much of the logic due to the lack of tests.

Thank you for your contribution nonetheless! :D