I have a use case which involves a quite large VNC server, of which only a small viewport is visible for each client. The current rfbClient.updateRect only partly solves the issues involved, because

• the client still has to allocate memory for the full frame buffer, which is impractical in my particular case
• the resulting frame buffer can not be efficiently used as an OpenGL ES 2.0 texture, because neither glTexImage2D nor glTexSubImage2D allow to specify a "scanline stride", and PBOs are only available with GLES 3. And I'm not even sure of PBOs would help here.

Because of this, I'd like to have libvncclient to support these use cases. Heck, I'd even implement it myself, but I'd like to get some input on how it could be done to increase the chances of having it merged upstream. :-)

My initial intention on how this might work is:

• there are additional fields for x and y offset in rfbClient (called offsetX and offsetY or whatever; set to 0 by default)
• rfbClient's width and height are no longer required to equal the server's frame buffer size, but can be smaller. Updates outside the viewport are clipped.
• rfbClient.frameBuffer is required to provide storage for (width - offsetX) * (height - offsetY) worth of pixels (instead of width * height as it stands currently)
• the decoders have to be adapted to honor the offsets, and this is the hard part:
  • the spec is not explicit about this, but it might be that a server references data in a copyrect update which the client has never requested but the server sent anyway. Currently, this is not a problem because the full frame buffer is available at client side anyway, but this is not the case when using the offsets / clipping. So I guess copyrect can not reliably work with such clients.
  • some decoders might be hard to refactor to match the changed semantics, especially if there is no server at hand to test it
  • in essence I'd like to have that only a subset of codecs is used when this offset / clipping stuff is in action, so it might work out to introduce a new function which enables this mode, which filters the currently active set of codecs so only ones where support was implemented are left, and issues a call to SetFormatAndEncodings on it's own behalf.
  • initially, this mode of operation would be implemented for raw and hextile encodings, because raw is required anyway and hextile offers sweet performance for my particular use case. Probably rre and corre, but I don't think I'll touch tight and the like.

Any thoughts on this?