If you'd like to put out an incentive for fixing this bug, you can do so at https://issuehunt.io/r/LibVNC/libvncserver
Describe the bug
Crash (access violation) on rfbShutdownServer(screen, 1) after server start when screen->httpDir is NULL and no client has connected.
To Reproduce
#include <stdio.h>
#include <stdlib.h>
#include <rfb/rfb.h>
#define WIDTH 640
#define HEIGHT 480
#define BPP 4
int main(void)
{
rfbScreenInfoPtr screen;
char *fb;
puts("[1] rfbGetScreen");
screen = rfbGetScreen(NULL, NULL, WIDTH, HEIGHT, 8, 3, BPP);
if (!screen) {
fprintf(stderr, "rfbGetScreen failed\n");
return 1;
}
fb = (char*)calloc(WIDTH * HEIGHT * BPP, 1);
if (!fb) {
fprintf(stderr, "calloc failed\n");
return 1;
}
screen->frameBuffer = fb;
screen->desktopName = "repro";
screen->port = 5900;
screen->ipv6port = 5900;
screen->httpDir = NULL; // intentional !
puts("[2] rfbInitServer (rfbInitSockets + rfbHttpInitSockets)");
rfbInitServer(screen);
puts("[4] rfbShutdownServer -> CRASH attendu ici si httpDir == NULL");
fflush(stdout);
rfbShutdownServer(screen, TRUE);
puts("[5] rfbScreenCleanup (jamais atteint si crash)");
rfbScreenCleanup(screen);
free(fb);
puts("[OK] exited cleanly");
return 0;
}Expected Behavior
The program terminates normally
Logs/Backtraces
Your environment (please complete the following information):
- OS and version: Windows 11 Pro (64 bits)
- Compiler and version: Visual Studio 2022 (Client, program in Delphi 13 - 64 bits)
Additional context
The bugs lies in httpd.c : when httpDir is not set the function returns early witrhout initializing outputMutex, refCountMutex and endMutex whereas rfbHttpShutdownSockets() unconditionnally tries to LOCK() these mutexes
Suggested Fix
Move calls to INIT_MUTEX() before early return in rfbHttpInitSockets()
src/libvncserver/httpd.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/src/libvncserver/httpd.c b/src/libvncserver/httpd.c
index 7cefadc..06baab0 100644
--- a/src/libvncserver/httpd.c
+++ b/src/libvncserver/httpd.c
@@ -99,6 +99,10 @@ rfbHttpInitSockets(rfbScreenInfoPtr rfbScreen)
rfbScreen->httpInitDone = TRUE;
+ INIT_MUTEX(cl.outputMutex);
+ INIT_MUTEX(cl.refCountMutex);
+ INIT_MUTEX(cl.sendMutex);
+
if (!rfbScreen->httpDir)
return;
@@ -127,9 +131,6 @@ rfbHttpInitSockets(rfbScreenInfoPtr rfbScreen)
rfbLog("Listening for HTTP connections on TCP6 port %d\n", rfbScreen->http6Port);
rfbLog(" URL http://%s:%d\n",rfbScreen->thisHost,rfbScreen->http6Port);
#endif
- INIT_MUTEX(cl.outputMutex);
- INIT_MUTEX(cl.refCountMutex);
- INIT_MUTEX(cl.sendMutex);
}
void rfbHttpShutdownSockets(rfbScreenInfoPtr rfbScreen) {
If you'd like to put out an incentive for fixing this bug, you can do so at https://issuehunt.io/r/LibVNC/libvncserver
Describe the bug
Crash (access violation) on
rfbShutdownServer(screen, 1)after server start whenscreen->httpDirisNULLand no client has connected.To Reproduce
Expected Behavior
The program terminates normally
Logs/Backtraces
Your environment (please complete the following information):
Additional context
The bugs lies in
httpd.c: whenhttpDiris not set the function returns early witrhout initializingoutputMutex,refCountMutexandendMutexwhereasrfbHttpShutdownSockets()unconditionnally tries toLOCK()these mutexesSuggested Fix
Move calls to
INIT_MUTEX()before early return inrfbHttpInitSockets()