To achieve high-quality video output, it's important to synchronize the drawing of video frames to the monitor's vertical refresh (also known as "vblank"). (For a good overview of the various issues that have to be dealt with when displaying video on a computer monitor, see the article Temporal Rate Conversion by Dave Marsh.) In this article, I'll describe how to do vertical retrace synchronization when using X11's Xvideo extension for video output.
Disclaimer: I've only tested the technique described here on Nvidia graphics cards. Unfortunately, this means the chances are it won't work on other graphics cards - as of this writing, there is still a lot of variability between different vendors' drivers in terms of which features they implement (and sometimes even how they implement them). If you're running non-Nvidia hardware, please tell me about your experiences - positive or negative. For Nvidia cards, you should definitely be using driver version 1.0-5336 or later. Earlier versions implement wait-for-vblank using a busy wait; not a nice thing to do in a multitasking environment.
Here's a summary of what we'll do: Since the Xvideo extension itself provides no means of doing vertical retrace synchronization, a pretty obvious idea is to use glXWaitVideoSyncSGI() to wait for the vertical retrace before outputting a video frame using XvPutImage() or XvShmPutImage(). When I initially tried this idea, though, it simply wouldn't work - glXWaitVideoSyncSGI() just returned immediately, and so, concluding that Nvidia hadn't implemented the function, I moved on to work on other things. Well, maybe I should have read the documentation more closely - it states explicitly that this function can only be used if an OpenGL direct context is current. I found out about this by chance before rechecking the documentation and noticing that this behaviour is consistent with the specifications. Using OpenGL and Xvideo on the same window didn't seem to work, so what we're going to do is to create a hidden dummy window and attach our OpenGL context to that.
A lot of people are probably wondering: Why not just use OpenGL to output the video? After all, synchronizing OpenGL output to vblank is pretty straightforward, at least for Nvidia graphics cards: If the __GL_SYNC_TO_VBLANK environment variable is set to 1, glXSwapBuffers() automatically blocks until the next vertical retrace. Unfortunately, unlike with Xvideo, there's no widely-supported way of doing YUV-to-RGB in hardware under OpenGL. There are two extensions called GL_EXT_422_pixels and GL_MESA_ycbcr_texture, but both of these do only 4:2:2, not 4:2:0, and appear not to be implemented by Nvidia, anyway. There's also an Nvidia extension called GL_NVX_ycrcb, but again, it's only 4:2:2, and apparently requires a pixel shader to do the actual YUV-to-RGB conversion. Xvideo, in contrast, is supported on pretty much any video card, so, on cards that don't support glXWaitVideoSyncSGI(), we still get basic video output at reduced quality.
Right, so let's take a look at how to create the dummy OpenGL window. I'll assume that dpy is a pointer to the Display. First, we check that the GLX extension is supported:
int nDummy;
if(!glXQueryExtension(dpy, &nDummy, &nDummy))
handle_error();
(Replace handle_error() with appropriate error handling code for your application.) Next, we check that the GLX_SGI_video_sync extension (which supplies glXWaitVideoSyncSGI()) is supported:
if(!GLXExtensionSupported(dpy, "GLX_SGI_video_sync"))
handle_error();
If the GLX_SGI_video_sync extension is not supported, an application can (and probably should) fail gracefully by falling back to unsynchronized video output. GLXExtensionSupported() is a small helper function that checks if a GLX extension is supported on the current system:
bool GLXExtensionSupported(Display *dpy, const char *extension)
{
const char *extensionsString, *pos;
extensionsString=glXQueryExtensionsString(dpy, DefaultScreen(dpy));
pos=strstr(extensionsString, extension);
return pos!=NULL && (pos==extensionsString || pos[-1]==' ') &&
(pos[strlen(extension)]==' ' || pos[strlen(extension)]=='\0');
}
We now choose a visual for OpenGL and create the dummy open OpenGL window:
XVisualInfo *pvi;
XSetWindowAttributes swa;
int attrib[]=
{ GLX_RGBA,
GLX_RED_SIZE, 1,
GLX_GREEN_SIZE, 1,
GLX_BLUE_SIZE, 1,
None };
Window winGL;
// Choose a visual
pvi=glXChooseVisual(dpy, DefaultScreen(dpy), attrib);
if(pvi==NULL)
handle_error();
// Create the dummy OpenGL window
swa.colormap=XCreateColormap(dpy, RootWindow(dpy, pvi->screen),
pvi->visual, AllocNone);
winGL=XCreateWindow(dpy, RootWindow(dpy, pvi->screen), 0, 0, 1, 1,
0, pvi->depth, InputOutput, pvi->visual, CWColormap, &swa);
Finally, we create a GLX context and make it current:
GLXContext context;
// Create a GLX context
context=glXCreateContext(dpy, pvi, None, GL_TRUE);
if(context==NULL)
handle_error();
// Make context current
glXMakeCurrent(dpy, winGL, context);
With the initialization out of the way, we can now use glXWaitVideoSyncSGI() to synchronize our video display:
unsigned int retraceCount; // Wait for vertical retrace glXGetVideoSyncSGI(&retraceCount); glXWaitVideoSyncSGI(2, (retraceCount+1)%2, &retraceCount); // Perform video output XvShmPutImage(...); // Flush output buffer XFlush(dpy);
When the application exits, we want to destroy the OpenGL dummy window:
XDestroyWindow(dpy, winGL); glXDestroyContext(dpy, context);
And that's it! Video output with Xvideo that's smooth as silk. If you have any questions or comments, please contact me at the address given at the top of this article. Standard disclaimer: I assume no liability for the correct functioning of the code and techniques presented in this article.