Using Live Video as Texture - Lurker's Guide - lurkertech.com
lurkertech.com Lurker's Guide Using Live Video as Texture

Support This SiteMore than 1000 hours of work have gone into making this site. Please support my work and ongoing site improvements in one of these ways:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.
get anything from amazon.com
Use this link to Amazon—you pay the same, I get 4%.
get my thai dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary app: iOS, Android, Windows.
get my thai phrasebook app
Experience Thailand richly with my Talking Thai-English-Thai Phrasebook app.
get my chinese phrasebook app
Visit China easily with my Talking Chinese-English-Chinese Phrasebook app.
get thailand fever
I co-authored this bilingual cultural guide to Thai-Western romantic relationships.
Support This Site

More than 1000 hours of work have gone into making this site. Please support my work and ongoing site improvements in one of these ways:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.
get anything from amazon.com
Use this link to Amazon—you pay the same, I get 4%.
get my thai dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary app: iOS, Android, Windows.
get my thai phrasebook app
Experience Thailand richly with my Talking Thai-English-Thai Phrasebook app.
get my chinese phrasebook app
Visit China easily with my Talking Chinese-English-Chinese Phrasebook app.
get thailand fever
I co-authored this bilingual cultural guide to Thai-Western romantic relationships.

Submit This SiteLike what you see?
Help spread the word on social media:
Submit This Site

Like what you see?
Help spread the word on social media:

Note: Updated Lurker's Guide available (but not this page!)

This page belongs to the old 1990s SGI Lurker's Guide. As of 2008, several of the Lurker's Guide pages have been updated for HDTV and for modern OS platforms like Windows and Mac. This particular page is not one of those, but you can see what new stuff is available here. Thanks!

Using Live Video as Texture

Brian Beach

Note: This document uses the O2 VL DMbuffer API instead of the cross-platform VL DMbuffer API (see What Are the SGI Video-Related Libraries? for more info). It refers to IRIX 6.3 man pages and documentation.

If you've read through the overview of the use of DMbuffers with video and graphics, you already know most of what's needed to use video as a texture.

The basic steps are as follows:

  1. Set up a video-to-memory path using VL.
  2. Create a digital media pbuffer.
  3. Create a buffer pool to hold the video images.
  4. Start the video path.
  5. Loop:
    • Get a video input frame in a DMbuffer.
    • Associate the DMbuffer with the dm pbuffer.
    • "Copy" the texture in.
    • Render using the texture.

There are a number of code fragments on this page, all of which came from the example program "composite". You'll find the source for it in: /usr/share/src/dmedia/video/vl/OpenGL/composite.c.

Setting up the Video Path

The image layout must be either VL_LAYOUT_GRAPHICS or VL_LAYOUT_MIPMAP. Use VL_LAYOUT_GRAPHICS if your rendering does not use mipmapping; the non-mipmapped images will be smaller and save memory.

The packing must be either VL_PACKING_ARGB_1555 or VL_PACKING_ARGB_8. For mipmapped textures, the only choice is VL_PACKING_ARGB_1555, because of the increased I/O bandwidth required for the larger mipmapped images.

The tricky part is the size of the image. To use them as a texture, both the width and the height must be a power of 2. Standard video is not the right size. The video input path on O2 support arbitrary scaling down of the input image. To use an NTSC input (640x480) as a texture, you'll have to scale it down to 512x256. Here is the complete list of settings for the memory node, as set by the composite example:

    int width  = 512;
    int height = 256;

    setIntControl  ( path, drain,  VL_PACKING,       VL_PACKING_ARGB_1555 );
    setIntControl  ( path, drain,  VL_CAP_TYPE,      VL_CAPTURE_INTERLEAVED );
    setIntControl  ( path, drain,  VL_FORMAT,        VL_FORMAT_RGB );
    setXYControl   ( path, drain,  VL_MVP_ZOOMSIZE,  width, height );
    setXYControl   ( path, drain,  VL_SIZE,          width, height );
    setFractControl( path, drain,  VL_RATE,          30, 1 );
    setIntControl  ( path, drain,  VL_LAYOUT,        VL_LAYOUT_MIPMAP );

Creating the Digital Media Pbuffer

A frame buffer configuration (fbconfig) must be chosen that matches the VL packing. For VL_PACKING_ARGB_1555, you must choose an fb config where the red, green, and blue components are exactly 5 bits deep, and the alpha is exactly 1 bit deep. Choosing the usual 8888 config will not work. Here is the code used by composite to get the right config:

GLXFBConfigSGIX getFBConfig(
    void
    )
{
    int i;

    /*
     * Find a frame buffer configuration that suits our needs: it must
     * work with both the pbuffers and the window, and must have 5551 pixels,
     * because that is the only format that can be used to capture
     * mipmapped video.
     */

    GLXFBConfigSGIX config = NULL;
    int params[] = {
        GLX_RENDER_TYPE_SGIX,   GLX_RGBA_BIT_SGIX,
        GLX_DRAWABLE_TYPE_SGIX, GLX_PBUFFER_BIT_SGIX,
        GLX_ALPHA_SIZE,         1,
        GLX_RED_SIZE,           5,
        GLX_GREEN_SIZE,         5,
        GLX_BLUE_SIZE,          5,
        GLX_X_RENDERABLE_SGIX,  False,
        (int)None,
    };
    int configCount;
    GLXFBConfigSGIX* configs =
        glXChooseFBConfigSGIX(
            theDisplay,
            DefaultScreen(theDisplay),
            params,
            &configCount
            );
    CHECK( configs != NULL || configCount != 0,
           "Could not get FBConfig" );

    for ( i = 0;  i < configCount;  i++ )
    {
        int value;

        glXGetFBConfigAttribSGIX(theDisplay, configs[i], GLX_ALPHA_SIZE, &value);
        if ( value != 1 )  continue;
        glXGetFBConfigAttribSGIX(theDisplay, configs[i], GLX_RED_SIZE, &value);
        if ( value != 5 )  continue;
        glXGetFBConfigAttribSGIX(theDisplay, configs[i], GLX_GREEN_SIZE, &value);
        if ( value != 5 )  continue;
        glXGetFBConfigAttribSGIX(theDisplay, configs[i], GLX_BLUE_SIZE, &value);
        if ( value != 5 )  continue;

        break;
    }

    if ( i != configCount )
    {
        config = configs[i];
    }

    CHECK( config != NULL, "No 5551 FBconfigs were found" );

    XFree( configs );

    return config;
}

Once the fb config is chosen, you just have to create a dm pbuffer that is exactly the same width and height as the texture:

    GLXFBConfigSGIX fbconfig = getFBConfig();

    int attrib[] = {
        GLX_PRESERVED_CONTENTS_SGIX,    GL_TRUE,
        GLX_DIGITAL_MEDIA_PBUFFER_SGIX, GL_TRUE,
        None};

    pbuffer = glXCreateGLXPbufferSGIX(
        theDisplay, fbconfig, 512, 256, attrib);

Create the Buffer Pool

This needs to be done after creating the video path (so that the VL's pool constraints can be queried), but before starting the path (because it needs a buffer pool to put the input frames into).

Be sure to query the constraints of both the video input path, using vlDMPoolGetParams, and the pbuffer, using dmBufferGetGLPoolParams. Then create the pool using dmBufferPoolCreate.

After the pool is created it must be bound to the video path using vlDMPoolRegister.

Start the Video Path

Don't forget this step. It's the usual reason that my VL programs don't do anything.

Get a Video Input Frame in a DMbuffer

The code looks like this:

    VLEvent event;
    DMbuffer buffer;
    int vlstatus;

    vlstatus = vlEventRecv( theVideoServer, path, &event );
    while ( vlstatus == -1 && vlGetErrno() == VLAgain )
    {
        sginap( 1 );  // XXX select() is better in real programs
        vlstatus = vlEventRecv( theVideoServer, path, &event );
    }

    CHECK( vlstatus == 0,
           "Could not get video input event" );
    CHECK( event.reason == VLTransferComplete,
           "Unexpected video event received" );
   
    vlstatus = vlEventToDMBuffer( &event, &buffer );
    CHECK( vlstatus == 0, "Video event does not have a buffer" );

Associate the DMbuffer with the Pbuffer

This make the contents of the DMbuffer be the contents of the color buffer in the pbuffer:

    Bool ok;
    ok = glXAssociateDMPbufferSGIX( theDisplay, pbuffer, format, dmbuffer );
    CHECK( ok, "Could not associate path2 DMbuffer with pbuffer" );

The format is a DMparams list that contains with width, height, layout, and packing of the image being passed in.

"Copy" the Texture In

Before coping the texture into place, two things must be prepared.

First, there must already be a texture of the same size in the current context. It can be any image, it just has to be the right size, including all of the mipmap levels:

    int w = MY_TEX_WIDTH;
    int h = MY_TEX_HEIGHT;
    int level = 0;
    void* black = calloc( w * h, 4 );

    level = 0;
    while ( 1 )
    {
        glTexImage2D(
            GL_TEXTURE_2D,
            level,
            GL_RGB5_A1_EXT,     /* number of components */
            w, h,
            0,                  /* border */
            GL_RGBA,
            GL_UNSIGNED_BYTE,
            black
            );
        printf( "%d: %d x %d\n", level, w, h );

        if ( w == 1 && h == 1 )
        {
            break;
        }

        if ( w > 1 )   w /= 2;
        if ( h > 1 )   h /= 2;
        level++;
    }

    free( black );

Second, the current readable and drawable must be set to copy from the image to be used as texture. Usually this can be done once at startup when setting the drawable to be a window (or wherever the rendering is to happen):

    Bool ok;
    ok = glXMakeCurrentReadSGI( theDisplay, drawable, pbuffer, context );
    CHECK( ok, "glXMakeCurrent failed" );

Under the right circumstances, described in intro.html, glCopyTexSubImage2DEXT will do a copy-by-reference. The image in the pbuffer will be used directly as a texture, without having to copy the pixels. Also, for video input, the mipmaps are already present and they are also used directly, with no need to create them in software. Before loading a mipmapped texture, the mipmap generation extension must be turned on. (O2 doesn't support this extension, but requires that it be turned on in this case.)

    glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE);
    glCopyTexSubImage2DEXT(
        GL_TEXTURE_2D,
        0,                      /* level */
        0, 0,                   /* x, y offset into target */
        0, 0,                   /* lower-left corner of copied rect */
        512, 256                /* size of copied rect */
        );

Render

That's all there is to it! Only a couple of hundred lines of code to do a very simple thing. (Sarcasm intended.)

Now that the texture is loaded, all of the normal GL rendering calls can use the texture.

Support This SiteMore than 1000 hours of work have gone into making this site. Please support my work and ongoing site improvements in one of these ways:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.
get anything from amazon.com
Use this link to Amazon—you pay the same, I get 4%.
get my thai dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary app: iOS, Android, Windows.
get my thai phrasebook app
Experience Thailand richly with my Talking Thai-English-Thai Phrasebook app.
get my chinese phrasebook app
Visit China easily with my Talking Chinese-English-Chinese Phrasebook app.
get thailand fever
I co-authored this bilingual cultural guide to Thai-Western romantic relationships.
CopyrightAll text and images copyright 1999-2023 Chris Pirazzi unless otherwise indicated.
Support This Site

More than 1000 hours of work have gone into making this site. Please support my work and ongoing site improvements in one of these ways:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.
get anything from amazon.com
Use this link to Amazon—you pay the same, I get 4%.
get my thai dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary app: iOS, Android, Windows.
get my thai phrasebook app
Experience Thailand richly with my Talking Thai-English-Thai Phrasebook app.
get my chinese phrasebook app
Visit China easily with my Talking Chinese-English-Chinese Phrasebook app.
get thailand fever
I co-authored this bilingual cultural guide to Thai-Western romantic relationships.
Copyright

All text and images copyright 1999-2023 Chris Pirazzi unless otherwise indicated.