lurkertech.com Lurker's Guide Square and non-square pixels

Square and Non-Square Pixels

By Chris Pirazzi. Thanks to Bob Williams for much of the information. Thanks to Charles Poynton for pointer to RP 187 and other info.

Support
This Site
If you have enjoyed this site, here are a few ways that you can help me get time to expand it further:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.

get anything at all from amazon.com
Use this link to Amazon—you pay the same, this site gets 4% from Amazon.
get a cool thai-english iPhone dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary for iPhone/iPad/iPod Touch.
get a cool thai-english Windows dictionary
Learn Thai with my Talking Thai-English-Thai Dictionary for Windows PCs.
get a cool thai-english paper dictionary
Don't leave home without the Thai-English English-Thai Compact Dictionary I co-authored.
get thailand fever
I co-authored this bilingual cultural guidebook to Thai-Western romantic relationships.

Submit
This Site
Like what you see?
Help spread the word using these social bookmarking sites:
StumbleUpon
del.icio.us
del.icio.us

Table of Contents

What Are They?

Pixels in the graphics world are square. A 100 pixel vertical line is the same length as a 100 pixel horizontal line on a graphics monitor.

Some pixels in the standard-def video world (specifically, the digital electrical signals universally used in studios for 480i and 576i production, as defined by the infamous ITU-R BT.601-4 or "Rec. 601") are non-square. A 100 pixel vertical line may be longer or shorter than a 100 pixel horizontal line on a video monitor, depending on the video system.

Pixels in the HD (e.g. 1080i and 720p) video world are, fortunately, square.

The term which describes this squareness or non-squareness is pixel aspect ratio, expressed as a fraction of horizontal (x) pixel size divided by vertical (y) pixel size. The pixel aspect ratio for square pixels is 1/1.

Why Do I Care?

When you as software programmer manipulate standard-definition video images in your application's memory, those images will often have the very same non-squareness as the Rec. 601 studio signals. This can be true whether the pixels came to you uncompressed, or packaged inside compressed chunks like MPEG frames, M-JPEG fields, or DV fields. Even if you download a video stream off the internet, it still might contain non-square pixels (because the original studio cameras that captured it produced non-square pixels, or it had to be converted to non-square for encoding).

Any rendering of geometry-based graphics to an image, including text, must take the pixel aspect ratio into account, otherwise it will come out "stretched" or "squished."

Many image processing operations assume a certain pixel aspect ratio. For example, a blur operation with a radially symmetric kernel may look bad on non-square pixels.

What Is the Pixel Aspect Ratio of Non-Square Pixels?

The pixel aspect ratio of non-square pixels is:

SystemhSpacingvSpacing
480i1011
576i5954

You can think of these values as the ratio of the width of a pixel to the height of a pixel. For example, say you want to draw a circle that appears round on the display device and whose diameter is n horizontal pixels (luma sampling instants). Draw an ellipse which is n pixels wide and n*hSpacing/vSpacing pixels (picture lines) high. Notice that vSpacing is in the denominator: the greater the vertical spacing of pixels (picture lines), the fewer vertical pixels you need in order to match a given number of horizontal pixels (luma sampling instants) on the display device.

For a full explanation of where these values come from, please read:

What Pixels are Square? How Non-Square is Non-Square?

Why Isn't It 4:3 or 16:9?

You may be wondering why the pixel aspect ratio of non-square pixels is not 4:3, or 16:9. In this case you are confusing pixel aspect ratio with picture aspect ratio, a separate concept which we define below.

Why Isn't It What My Software App or Library Says?

The pixel aspect ratio of non-square pixels was completely and unchageably determined at the moment that video engineers chose luma sampling frequencies for their square and non-square devices.

This happened before almost any video software got written.

The problem was, software folks like us didn't realize that. What ensued was a melee of confusion and religious wars about what the pixel aspect ratio was, that took place at the library (QuickTime, Video for Windows, DirectShow, ...), application (Premiere, Final Cut, After Effects, ...) and documentation (Video Demystified, ...) level.

There's some details on the confusion in this section of my document on video systems.

If you've got a standard-def video project in After Effects or Final Cut or Premiere or any other app and the project settings claim to use pixel aspect ratios other than the ones above, 99% of the time the apps are simply flat-out wrong, and are mis-interpreting (or mis-labeling) the pixel aspect ratio of the video footage you have loaded into them. Sometimes, it all works out ok: even though the app is delusional about the pixel aspect ratio of its own data, it assumes all input and output video footage has the same, incorrect pixel aspect ratio, so it doesn't "damage" any footage on the way through the program. But problems can arise when:

If you have one of these problems, sucks to be you. In some cases, the erroneous pixel aspect ratios are close enough to the real pixel aspect ratios that the difference is small (the image might be stretched or squished by only a few pixels) and can be ignored. But in some cases, the difference is critical: for example, if you need the output of your app to line up perfectly with the original footage so that it can be edited back into the original footage.

Why Isn't It What SMPTE RP 187 Says?

SMPTE RP 187-1995 was designed to solve this problem once and for all, and it had many good effects for high-def video. But sadly, for standard-definition video, it was too late, and came up with some unworkable proposals, both in the area of pixel aspect ratio and production aperture, so it was effectively ignored.

Why Isn't It What MPEG Says?

The MPEG-1 and MPEG-2 standards have some rather entertaining things to say about pixel aspect ratio. MPEG-1 labels were unable to represent the actual ratios and MPEG-2 labels made it very difficult, so most MPEG streams are actually mislabeled, as explained here.

Why Isn't It What HDMI / CEA-861 Says?

Unbelievably, the specs on which the modern consumer digital electrical video standard HDMI is based also give incorrect pixel aspect ratio numbers, which are causing mass confusion problems in modern equipment. For more on the sordid details, see here.

Why Isn't It 720/640 or 720/768?

You may have noticed that non-square pixel data tends to come in buffers that are 720 pixels wide (720 comes from ITU-R BT.601-4, which started this whole mess).

Furthermore, you may have noticed that square data tends to come in buffers that are 640 pixels wide (480i video system) or 768 pixels wide (576i video system).

So why, you might ask, isn't the pixel aspect ratio of 480i video equal to 720/640?

And why, you might ask, isn't the pixel aspect ratio of 576i video equal to 720/768?

The answer is that that is not the criterion the video engineers used to choose the pixel aspect ratio. For all the sordid details on that, please read this section of my video systems introduction.

Wait, How Can That Work?

"Ok smarty," you may now be saying, "if the pixel aspect ratio is not 720/640 for 480i, or 768/720 for 576i, then how can I resolve this with the fact that my images are 720, 640, or 768 pixels wide?"

Put another way, "How do I deal with this math:"

The answer to this excellent question is that a 720-pixel-wide non-square pixel image and a 640- or 768-pixel wide square pixel image do not represent the same horizontal range of the underlying signal!

480i

720 non-square pixels

640 square pixels

576i

720 non-square pixels

768 square pixels

When converting, you need to scale by 10/11 or 59/54 and then pad or crop. We'll give you the details below.

To understand exactly what we mean by "underlying signal" (you're probably thinking "What do you mean? The edge of the image is the edge of the image!"), I recommmend that you pop over to learn some of the shocking secrets about video signals on this page:

Programmer's Guide to Video Systems

How do I Convert Between Square and Non-Square?

Here is how to convert a 720-wide non-square image into a 640- or 768-wide square image:

480i Non-Square Sampling
720 x 486
576i Non-Square Sampling
720 x 576

scale horizontally by 10/11

654 6/11 x 486

crop to 640

scale horizontally by 59/54

786 2/3 x 576

crop to 768
640 x 486
480i Square Sampling
768 x 576
576i Square Sampling

When you crop, you should keep the image centered.

Here is how to convert a 640- or 768-wide square image into a 720-wide non-square image:

480i Square Sampling
640 x 486
576i Square Sampling
768 x 576

scale horizontally by 11/10

704 x 486

pad to 720

scale horizontally by 54/59

702 54/59 x 576

pad to 720
720 x 486
480i Non-Square Sampling
720 x 576
576i Non-Square Sampling

"Pad" means need to pad the image out to 720 non-square pixels, keeping the image centered. A better way to do it, if you can, is to start with a greater number of square pixels than 640 or 768 (again with the same horizontal center), and scale those directly to 720.

A video input or output device which is encoding or decoding an analog signal will never need to pad or crop: it can measure or synthesize the continuous analog signal for each video line any way it wants. Each analog video line is long enough to accomodate either horizontal range.

Important Note: In this section, we gave some conversions for 480i and 576i images using one particular set of image dimensions as an illustrative example. The particular image dimensions we chose are just examples; you should not read any magic significance into them. 480i and 576i video can come in a huge variety of dimensions (e.g. for 480i you'll find 640x480, 640x486, 704x486, 720x486 and others), but in every case the process of converting between square and non-square is the same:

It is critical to understand that the scaling step does not depend on either the horizontal or vertical image dimensions, and the crop/pad step depends only on the horizontal input and output image dimensions. The vertical image dimensions play no role whatsoever in either step. The MPEG committee and so many others have misunderstood this point and as a result given birth to many incorrect scaling ratios, as explained here. If you find yourself repeatedly coming back to the image dimensions as a core parameters of the conversion, then go back and read What Pixels are Square? How Non-Square is Non-Square? again.

How Do I Tell Which Pixel Aspect Ratio I Have?

It depends on the situation, but here are some hints:

Picture Aspect Ratio or Frame Aspect Ratio

Pixel aspect ratio is different than picture aspect ratio (sometimes also called frame aspect ratio), which is a fraction of total hoizontal (x) picture size over total vertical (y) picture size, for a stated definition of "picture."

Support
This Site
If you have enjoyed this site, here are a few ways that you can help me get time to expand it further:
donate now   Donate Now
Use your credit card or PayPal to donate in support of the site.

get anything at all from amazon.com
Use this link to Amazon—you pay the same, this site gets 4% from Amazon.
get a cool thai-english iPhone dictionary app
Learn Thai with my Talking Thai-English-Thai Dictionary for iPhone/iPad/iPod Touch.
get a cool thai-english Windows dictionary
Learn Thai with my Talking Thai-English-Thai Dictionary for Windows PCs.
get a cool thai-english paper dictionary
Don't leave home without the Thai-English English-Thai Compact Dictionary I co-authored.
get thailand fever
I co-authored this bilingual cultural guidebook to Thai-Western romantic relationships.
CopyrightAll text and images copyright 1999-2011 Chris Pirazzi unless otherwise indicated.