ND filters and Hypersmooth: The real story

There has been a large amount of conversation, debate, speculation and arguments over using ND filters along with Hypersmooth on GoPro cameras. I hear some people say that it doesn't work, I see some people claim to not have any problems, and I see a lot of misunderstanding about how Hypersmooth works.

The answer to the question:

Does Hypersmooth work properly with ND filters?

...is actually YES.

Hypersmooth can work perfectly fine with an ND filter, in low light - even with no image at all.

OK - I'll give some people a chance to take a deep breath, and also a chance to go looking for the comment section to immediately tell me I'm wrong (it's at the bottom), but the fact is that ND filters do not technically cause any problem with Hypersmooth.

"But how can that be? I've seen tons of demos that show Hypersmooth acting all jerky with an ND filters!".   Well, that is true - but most people don't understand what is actually happening, and that it's not actually Hypersmooth misfiring that is causing these video problems.

I'm going to explain how Hypersmooth works, what is actually happening when you use ND filters and how that affects what you see in the footage, why in some applications everything seems fine, but not in others, and I'll also offer a way to help mitigate the issue in your footage.

Let's first look at how Hypersmooth works.  In spite of what many people claim, a GoPro does not do image analysis to stabilize the video.  The software in the camera uses the camera's gyro to measure the motion of the camera, then it moves the framing in the video from the sensor to counteract this motion.  It can do this because a Hypersmooth image is cropped from a larger image produced by the sensor, which gives the frame a buffer around the edges to be able to move around to counteract that motion.

How do I know that it uses the gyro and not image analysis?  GoPro explains how the image stabilization works in their patents.  There is some image analysis to do things like determine how much processor/memory is needed for the footage, but the core of the stabilization is done using the gyro.

Here's an easy way to demonstrate this: I mounted a GoPro in a box with a picture on the inside of the box, so the GoPro and the picture are always locked still, relative to each other:

I started recording, then shook and moved the box.  On the left side of the video below, you see hypersmooth OFF, and on the right Hypersmooth ON:

At first, it seems backwards - The non-stabilized (left) side is not moving around, and the stabilized side (right) is bouncing all over the place.  That is because the camera and the image are fixed together, and the stabilization is following the gyro, not the image.

Without Hypersmooth, the camera and picture are locked together, so no matter what I do to the box, it will always be stable. 

But with Hypersmooth, the camera is trying to counteract the movement of the box, so the image looks like it's moving.  That is because it's the gyro that Hypersmooth uses for stabilization, not the image.  If the GoPro used image analysis, the clip on the right would not be moving around because the image does not change in relation to the camera.

OK, so we have established that Hypersmooth uses the gyro, not the image.  So why do we see jerky footage sometimes with ND filters?

First, it's not technically the nd filter that causes apparent jerkiness, it's a slow shutter speed.  Of course an ND filter can make your shutter speed slower, but the  jerkiness can happen in any type of low light situation.

Second,  there is no actual movement happening - the footage is perfectly stabilized by Hypersmooth.   What you are seeing is motion blur going in the wrong direction relative to the movement (or non-movement).

When you film with a slower shutter speed, you get motion blur.  This motion blur is usually a good thing - it makes motion seem more natural because it matches how our eyes normally see motion.

Following is some motion blur I created by pointing the camera at some dots on a page and shaking the camera horizontally, vertically, then in a circle:

Imagine this:  You are using your GoPro while walking. Every step you take makes the GoPro bump or move, almost randomly in all sorts of directions.  This causes motion blur in all sorts of different directions like in the pictures above.

Now, this would be fine if it was just regular video without Hypersmooth - it would just look like the camera is moving about as you walk.

But with Hypersmooth, you end up with perfectly stable video (because the ND filter did not affect Hypersmooth's function!), but with added motion blur moving up and down and around as if motion was happening.  This causes a weird ghosting effect where it seems like the footage moving around in a jerky fashion, even though the video is perfectly stabilized.

The long shutter time is causing this issue - it isn't Hypersmooth misfiring or "reading" the image incorrectly because of the ND filter.

This is why some people seem to be able to use an ND filter with Hypersmooth and some people cannot.  If the motion in your video is smooth and continuous, everything should look ok.  But if your motion is erratic, bumpy or random, you get the mess that we explained above.  That's why most of the demos showing hypersmooth problems with ND filters are people walking around in their back yards or riding mountain bikes or anything that causes almost random bumps.  If you use the camera on a car, or a motorcycle, or a drone, you're usually OK.

OK, so Hypersmooth doesn't use image analysis from stabilization, and it can work perfectly fine with an ND filter (or low light), but in some applications, we still end up with a problem with the "wrong" motion blur in our footage. 

So, how do we fix it?

If you use a fixed shutter, then the solution is easy - increase your shutter speed a bit.  If you use a shutter speed of 1/240, you should be OK, but test it yourself to see what looks best.  This is a little bit higher than I would prefer, but you can still get some fairly smooth motion at this shutter speed.  This won't be a problem most of the time in daylight, but if the footage is a bit too dark, you can bring up your ISO a bit (not too much or it'll get grainy).  I usually don't like to go over ISO 400 or 800.

If you prefer to use auto shutter, then there is still a solution: GoPro Labs.  By installing the GoPro Labs firmware on your camera, you get access to some extra functions by pointing your camera at QR codes.  One of those functions in minimum shutter speed - so you can tell the camera to never go below a specific shutter speed.

Here's how you do it:

First, install GoPro Labs firmware on your camera by going here and following the instructions.

Decide what you want your minimum shutter to be.  Instead of shutter speeds, GoPro labs uses "stops" according to the frame rate.  Here is a translation of stops to shutter speed, and what code needs to be entered into "additional commands" of the QR code generator:

at 30 frames per second:

• 1/120 shutter = 2 stops, code: !MEXPT=2
• 1/240 shutter = 3 stops, code: !MEXPT=3
• 1/480 shutter = 4 stops, code: !MEXPT=4

at 60 frames per second:

• 1/120 shutter = 1 stop, code: !MEXPT=2
• 1/240 shutter = 2 stops, code: !MEXPT=3
• 1/480 shutter = 3 stops, code: !MEXPT=4
• Reset to no minimum shutter code: !MEXPT=0

Then, create a QR code with the settings that you want (including the minimum shutter code in "addtional commands") by going here.  Print it out or capture it onto your phone, then turn on your camera and point it at the QR code.

Note that this will reset automatically when you turn off your camera, and you will not be able to see the exposure change on the screen until you hit record.

There is also a QR code generator here that provides a bit more control for the Hero 8/9/10/Bones (the shutter speeds are in degrees (remember the 180 rule?) - try 90 degrees as a start.

There are a whole lot of other commands you can give the camera with QR codes - it's worth exploring!