How And Why To Enable Classic Controls In Roto Brush 2 In After Effects

If you’re new to rotoscoping in After Effects, I suggest you read How To Roto Brush In After Effects. Illustrated Tutorial. Part 1 first.

If you’re working with Roto Brush 2 in After Effects, you might find you miss some of the features from Roto Brush 1. Don’t worry.

If you want to turn on the finer control options for propagation and edge detection, you can easily do so.

Activating 1.0 Classic passes the Roto Brush propagation result from the new and better algorithm through Roto Brush 1.0s finer controls for Edge Detection, Motion Damping, Motion Threshold from Roto Brush 1.

In other words, choosing Classic doesn’t roll back the Roto Brush 2 to the less good algorithm of 1.0 but allows for finer control. I.e., you get the best of both worlds.

Now there’s a right way to do it – and a wrong way to do it.

Here’s the right way:

The Right Way To Enable The Classic Controls In Roto Brush 2

Enabling the classic controls in Roto Brush 2 is pretty straightforward.

In the Effect Control panel for your roto brush layer you simply click Enable Classic Controls under the Propagation settings, like so:

How to activate classic controls roto brush 2 in After Effects v2

Now you can start fintuning your roto edge settings for motion while still taking advantage of the improved algorthim of Roto Brush 2.

Here’s a quick rundown of the classic controls.

  • Search Radius: The general area that the Roto Brush searches for movement while tracking your subject (propagating). Suppose your footage has a lot of movement or motion blur to increase the value. A good place to start is 50.
  • Motion Threshold: The value Roto Brush uses to determine whether something is moving fast, slow, or not moving at all. If you’ve got a lot of fast movement, crank it up.
  • Motion Damping: The value determines how tight Roto Brush should make the search region for slow-moving edges.
    • Start by setting Motion Damping to 0 and then crank up the motion threshold until it tracks your subject nicely (there should be no overspill from your subject outside the search region). Then increase the Motion Damping to tighten it up further.
  • Edge Detection: How Roto Brush calculates the edges between frames other than the base frames (Balanced is the default).
    • Favor Predicted Edges: Roto Brush tries to predict the edges of the next frame based on several adjacent frames. Good for low-contrast or noisy footage.
    • Balanced: Roto Brush tries to predict the edges of the next frame while emphasizing both the edges from the current isolated frame and the edges from several adjacent frames.
    • Favor Current Edges: Roto Brush uses the current frame to predict the edges of the next frame. Good for lots of movement.
  • Use Alternate Color Estimation: Roto Brush uses an alternate algorithm to differentiate foreground from background footage.

This is how to do it if you want to use the improved Roto Brush v2 algorithm while taking advantage of the finer controls from v1. Now let’s look at how to use the Roto Brush v1 algorithm to avoid compatibility issues.

How to still use Roto Brush v1 algorithm:

Now, if you’re working on a project, which already uses version 1 of Roto Brush, you might want to continue using that.

In the effects control panel, choose Roto Brush 1.0 (Classic) under “Version” for the layer you’ve applied to the Roto Brush tool to.

image 21

Doing this rolls back Roto Brush to the old algorithm, which I don’t find nearly as good as Roto Brush 2.

But if you need to continue working on an older project, using v1 can prove useful to avoid compatibility issues – especially if you’re sharing your project with someone working in AE v17.1.4 or earlier.

That’s it. If you’ve got any questions, let me know in the comment section below.


  • Jan Sørup

    Jan Sørup is a videographer and photographer from Denmark. He owns and the Danish company Apertura, which produces video content for big companies in Denmark and Scandinavia. Jan has a background in music, has drawn webcomics, and is a former lecturer at the University of Copenhagen.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.