There seems to be a locking problem in libvncserver, with respect to how
condition variables are used.
On certain machines in our lab, when using a vncviewer to view a display
that has a very high rate of updates, we will occasionally see the VNC
server process crash. In one stack trace that was obtained, an assertion
had tripped in glibc's pthread_cond_wait, which was called from
clientOutput.
Inspection of clientOutput suggests that WAIT is being called incorrectly.
The mutex that protects a condition variable should always be locked when
calling wait, and on return from the wait will still be locked. The
attached patch fixes the locking around this condition variable, and one
other that I found by grepping the source for similar occurrences.
Signed-off-by: Charles Coffing <ccoffing@novell.com>
do not make requestedRegion empty without reason.
the cursor handling for clients which don't handle CursorShape updates was
completely broken. It originally was very complicated for performance
reasons, however, in most cases it made performance even worse, because at
idle times there was way too much checking going on, and furthermore,
sometimes unnecessary updates were inevitable.
The code now is much more elegant: the ClientRec structure knows exactly
where it last painted the cursor, and the ScreenInfo structure knows where
the cursor shall be.
As a consequence there is no more rfbDrawCursor()/rfbUndrawCursor(), no more
dontSendFramebufferUpdate, and no more isCursorDrawn. It is now possible to
have clients which understand CursorShape updates and clients which don't at
the same time.
rfbSetCursor no longer has the option freeOld; this is obsolete, as the cursor
structure knows what to free and what not.
dscho
steven_carr
rohit_99129
runge