Category: Nuke

Copycat node is a machine learning algorithm introduced in 2020 in nuke, it is used to train with a set of images and generate in-between frames with data which matches the given intervals of frame. the data keeps getting better when we feed more data and the output becomes more accurate.

Examle:

We can use the copycat to clean-up the plate by training the node with intermediate cleaned up frames and letting the machine learning to create the in-between frames.

• Create the Copycat node and set the directory
• Copycat node should work on linear colorspace input.
• Adjust epochs/steps of calculations as per the shot requirement / Think about epochs like sampling in 3D/ MB.
• More the epochs more the time to train and also the PC might be unusable during the calculations.
• In the current version of copycat we can pause the training process to check the results.
• In the advance settings having large model size might take a lot time to calculate so need to be carful on using these settings.
• We can use any previous training data and use it as a checkpoint to aid the machine learning.

• We can use the same principle to many things like the roto

• In Nuke, keying is the process of isolating specific elements, typically from a video or image, based on their color or luminance values. This is often done to remove backgrounds, separate objects, or create matte elements for compositing.

HSV

HSV Color Scale: The HSV (which stands for Hue Saturation Value)

scale provides a numerical readout of your image that corresponds to the color

names contained therein. Hue is measured in degrees from 0 to 360

R = HUE: Hue literally means colour

G = Saturation: Saturation pertains the amount of white light mixed with a hue.

B = Luminance: Luminance is a measure to describe the perceived brightness of a colour

Hue corrections

Luminance Key

Can be used to create mask with the luminance value of an image.

This can be used to key various channels like RGB/

Luma key can also be used is various ways like experimenting with different color spaces.
Here is an example with keying red channel in various color spaces.

Hue / Luma separator

We can separate Hue from an image by dividing the luminance with the colour of the image in the merge division mode.

Color Difference Key

We can separate individual colors by negating colors from two channels using merge minus function.

Adding Channels

We can create separate channels for certain needs for example we can create a Roto and add it as a new channel and we can use grade or blur node’s mask as the new channel which was created. These Channels are separate from alpha so it is only visible on its assigned name.

ChromaKeyer

ChromaKeyer is basic keying node useful for quick color keying and it is not accurate around hairs or other tiny moving objects.

Image Based Keyer – IBK

IBK stands for Image Based Keyer It operate with a subtractive or difference methodology. It is one of the best keyers in NUKE for getting detail out of fine hair and severely motion blurred edges.

IBK Color: This node we can use it to make up the clean plate for the background of the subject.

IBK Gizmo: With this node we can make use of the produced clean plate of the IBK Color and aid it to key the subject especially for hairs.

KeyLight

Key light It is a really great keyer over all and does color de-spill.

Primatte

Primatte is what is called a 3D keyer. It uses a special algorithm that puts the colors into 3D color space and creates a 3d geometric shape to select colors from that space.

Ultimatte

The advantage is you can get phenomenal keys that can get really fine detail and pull shadows and transparency from the same image.

Green De-spill

It’s the process of removing green scatter around the subject which occurs mainly due to the green screen.

We first key the green using key-light and after the comes the difference function which will extract the green only from the image and then we desaturate the green and we add back to the image.

we can use the same principle for the blue screen.

De-spill map

We can create a De-spill map with the help of minus function with the color function. Here we use Green – Red thus getting a alpha as a result which can be used for de-spill.

Edge extend

Sometimes we will get black halo/edges in an keyed footage, to eliminate this we use edge extend, It works by extruding the color values around the edge of the alpha and we can use this value to reduce the black halo which will be appearing.

Fixing uneven green screen

Having an irregular green screen will develop problems down the pipeline especially during keying. so to counter that we need to even out the greenscreen in comp.

Merge keys then comp over

Production Scenario

• Nuke primarily operates as a 2D compositing software, but it does have some 3D capabilities. The 3D system in Nuke allows you to work with three-dimensional elements within a 2D compositing environment.

• 3D Space:
  • Camera Nodes: Nuke supports the use of virtual cameras, allowing you to create a 3D space and move the camera within it. This is useful for matching the movement of live-action footage or creating parallax effects.

• Geometry and Objects:
  • Card Nodes: You can use card nodes to represent flat or simple 3D objects within the 3D space. These cards can be textured with images or sequences, allowing you to integrate 2D and 3D elements seamlessly.
  • ScanlineRender Node: This node is used to render 3D scenes within Nuke, taking into account lighting, shadows, and reflections.

• 3D Rendering:
  • Nuke’s 3D system provides basic rendering capabilities for simple scenes. It supports features like ambient occlusion, shadows, and reflections.
• Shading and Lighting:
  • Nuke includes nodes for basic shading and lighting, allowing you to control the appearance of 3D objects in your composition.
• Scene Integration:
  • You can integrate 3D elements into live-action footage, matching the camera movement for a more realistic composite.
• Expression Linking:
  • You can use expressions to link 2D and 3D properties, allowing for dynamic relationships between elements in different dimensions.

• Nuke can be customized in many ways through preference, we can change the 3D navigation method to emulate other 3D software navigation methods.
• We can customize the nuke UI and save the changes with a name as a workspace, so when opening the nuke we can use our preferred workspace
• We can also create certain tool sets to save some time.
• All the saved tool sets, workspace and preferences are saved in the parent folder of the Nuke.

Nuke Camera

Nuke supports the use of virtual cameras, allowing you to create a 3D space and move the camera within it. This is useful for matching the movement of live-action footage or creating parallax effects.

• Create a Camera Node:
  • In the Node Graph, press Tab to open the node creation panel.
  • Type “Camera” and select the “Camera” node.
• Import Camera Data:
  • If you have camera tracking data from external software (e.g., PFTrack, SynthEyes), use a ReadGeo node or similar to import the camera data into Nuke.
• Adjust Camera Settings:
  • Open the Camera node properties by double-clicking on it.
  • Set the film back, focal length, and other parameters to match the real camera used during filming.
• Create 3D Objects:Use Card nodes or other geometry nodes to represent objects in the 3D space.
  • Connect them to the ScanlineRender node for rendering.
• Animate the Camera:
  • Keyframe the camera’s translation, rotation, and focal length to match the movement in the live-action footage.
  • You can use keyframes or expressions to link camera properties to tracking data.
• Camera Projection:
  • Use the CameraProject node to project 2D images onto 3D geometry based on the camera’s perspective.

Scanline Render

The ScanlineRender node in Nuke is used for rendering 3D scenes within the compositing environment. It simulates a simplified rendering process, taking into account the lighting, shading, and textures of 3D objects in a scene.

Node Properties:

• Render Settings:
  • In the ScanlineRender node properties, you can find settings for rendering quality, anti-aliasing, and other parameters.
• Shading Model:
  • Choose the shading model (e.g., Lambert, Phong) that best suits your scene and desired look.
• Background:
  • Specify the background color or connect another image node to the “Background” input for a more complex background.
• Outputs:
  • The ScanlineRender node typically has outputs for the rendered image, depth information, and other auxiliary data.

Lens Distortion

Lens distortion refers to the imperfections introduced by camera lenses that can cause straight lines to appear curved or distorted. In visual effects and compositing, correcting lens distortion is crucial for seamlessly integrating elements into live-action footage. Nuke provides tools to analyze and correct lens distortion.

• Understanding Lens Distortion:
  • Radial Distortion: Causes straight lines to curve, more pronounced at the frame edges.
  • Tangential Distortion: Shifts the image along the horizontal and vertical axes.
• LensDistortion Node:
  • Analysis: Use the LensDistortion node to estimate distortion parameters from a grid pattern.
  • Correction: Apply obtained parameters for distortion correction.
• Undistort and Distort Nodes:
  • Undistort: Use the Undistort node to remove lens distortion.
  • Distort: The Distort node reintroduces lens distortion, e.g., for 3D integration.
• LensDistortion Model:
  • Model Options: Choose a lens distortion model (e.g., “Nuke,” “Brown,” “Houdini”).
  • Parameters (K1, K2, P1, P2): Define distortion correction amount and type.
• Fine-Tuning:
  • Grid Warp: Manually adjust correction with a grid warp in the LensDistortion node.
  • LensDistortionCorrect Node: Use for advanced correction with extra controls.
• Animation:
  • Keyframe Parameters: Adjust distortion parameters for changing distortion over time.
• Checkerboard Patterns:
  • Calibration Aid: Filming a checkerboard pattern aids in accurate distortion analysis.

STmap

An STmap (Spatial-Temporal Map) in Nuke is a representation of the distortion in an image due to various factors, including lens distortion, and it is used to correct this distortion. The STmap carries spatial and temporal information, making it a powerful tool for addressing complex distortions that may vary across different areas of the image and evolve over time.

• Understanding STmap:
  • Spatial-Temporal Distortion: Spatial distortion is caused by lens imperfections, while temporal distortion evolves over time.
• LensDistortion Node and STmap:
  • LensDistortion Node: In Nuke, the LensDistortion node analyzes footage and generates an STmap representing spatial and temporal distortions.
  • STmap Output: The LensDistortion node produces an STmap encapsulating distortions in the footage.
• Usage of STmap:
  • LensDistortionCorrection: The LensDistortionCorrection node uses the STmap to undistort or redistort images.
• Creation of STmap:
  • Calibration Grid: Use a grid during shooting for generating an STmap, providing reference points for distortion analysis.
  • Analysis: The LensDistortion node analyzes the grid to create the corresponding STmap.
• Application to Animation:
  • Changing Distortion Over Time: For evolving lens distortion, animate distortion parameters or use a sequence of STmaps.
• Manual Adjustments:
  • GridWarp and STmap: The GridWarp node, combined with an STmap, allows manual adjustments, helpful when automatic analysis falls short.

Roto Paint

• In Nuke, the RotoPaint node is a versatile tool used for both rotoscoping and painting tasks within a compositing workflow. It combines the capabilities of both the Roto and Paint nodes, allowing artists to create complex shapes for rotoscoping and perform detailed paint work directly within the same node.

Painting:

• Brush-Based Painting: The RotoPaint node includes painting tools similar to those found in standalone paint applications. Artists can use brushes to clone, smudge, blur, and paint directly onto the image.

• Frame-by-Frame Painting: It supports frame-by-frame painting, making it possible to create hand-painted elements that evolve over time in a sequence.

• Integration with Rotoscoping: The ability to paint directly on top of roto shapes is valuable. This allows for precise paint work on specific regions of the image, matching the motion and contours defined by the rotoscoping shapes.

Clone and Repair:

• Clone Brush: The RotoPaint node includes a clone brush that allows you to sample pixels from one part of the image and paint them onto another. This is useful for removing unwanted elements or duplicating parts of the image.

• Repair Work: Artists can use the RotoPaint node for repairing and fixing issues in the footage, such as wire removal or blemish cleanup.

Integration with the Nuke Environment:

• Layered Approach: Like other nodes in Nuke, the RotoPaint node works in a layered manner, allowing you to apply multiple instances of the node with different settings for complex compositing tasks.

• Integration with Channels: It can work with multiple input and output channels, giving you control over how the roto and paint information integrates with other elements in the composite.

Smear Tool:

The Smear tool can simulate motion blur by dragging or smearing pixels in the direction of motion. This is handy for matching the motion blur of live-action elements or for adding a sense of movement to painted or roto shapes.

The RotoPaint node is a powerful tool widely used in VFX for tasks like rotoscoping, painting, and image repair.

Grain & Noise

• Grain refers to the visual noise or texture present in an image. Grain is often a result of film or sensor characteristics and can be an important aesthetic element, especially when compositing CG elements into live-action to achieve a more realistic look.

• In digital imaging, especially in the context of sensors and electronic devices, “noise” refers to random variations in brightness or color. This noise can result from factors such as sensor sensitivity, electronic interference, or high ISO settings in low-light conditions.

Denoise

denoise” refers to the process of reducing or eliminating digital noise in an image or sequence. Digital noise often appears as unwanted random variations in brightness or color, and it can result from factors like low-light conditions, high ISO settings, or the limitations of digital sensors. The Denoise node in Nuke is a tool specifically designed to address and mitigate this type of noise.

Capturing the grain from the footage

• This process allows us to analyze the existing grain pattern in a clean plate or reference frame and apply it to other elements in our composite

• Select a Clean Plate:
  • Choose a frame from your footage where there is no significant action or objects of interest, and the background is relatively uniform. This will be our clean plate, and we’ll capture the grain from this frame.
• Create a Grain Sample:
  • Place the clean plate on the timeline.
  • Use a Copy node to duplicate the clean plate.
• Add Grain to the Duplicate:
  • Apply a Grain or Noise node to the duplicated clean plate.
  • Adjust the settings of the grain to match the natural grain in our footage. We may need to tweak parameters such as size, intensity, and seed to match the original grain.
• Difference Operation:
  • Use a Dissolve or Difference node to compare the original clean plate with the duplicated one containing added grain. This will help us see the difference and fine-tune the settings.
• Create a Grain Map:
  • Use the Difference node’s output as a guide to create a black and white map where the grain is most prominent. We can use nodes like Grain2D, Blur, or ColorCorrect to enhance this map.
• Apply Grain to Other Elements:
  • Use the Copy node or a similar method to apply the grain map to other elements in our composite.
  • Adjust the blending mode or opacity to control the intensity of the added grain.
• Fine-Tune as Needed:
  • Continuously check and adjust the added grain to make sure it matches the original footage. We may need to iterate on the settings to achieve a realistic and consistent result.

DasGrain

DasGrain, is a regrain tool which analyzes the grain in your plate, and uses it to regrain an entire degrained comp. It works by:

• Isolating the grain by finding the difference between a plate & its degrained counterpart.
• Analyses the grain response over a range of luminance, and adapts the grain to fit the comp based on its own luminance.
• Uses a mask input to restore the plate’s original grain, so you don’t have to Keymix your plate back over top at the end of your script.
• This method is widely used in the industry for adding grains to clean-ups in nuke.
• We use the noise for cleaning up elements because it lacks the noise pattern seen in the plate. As a result, the object may not blend seamlessly.
• It is crucial to seamlessly blend the grain with the footage; otherwise, improper execution can lead to issues at the end of the pipeline.

• 2D tracking in Nuke refers to the process of analyzing and following the motion of objects within a two-dimensional space in a video or image sequence. This tracking is essential for tasks such as adding elements to a scene, stabilizing footage, or applying visual effects that need to move with a specific object.

2D Tracking Process

1. Import Footage:

• Open Nuke and import your footage or image sequence.
• Analise the footage on how to approach the shot.

2. Create a Tracker Node:

• In the Node Graph, right-click and select “Tracker” from the menu.
• Connect the Tracker node to your footage.

3. Select Tracking Points:

• Open the Tracker node properties.
• Click “Add Track” to define tracking points on the phone screen.
• Choose high-contrast points that will allow for accurate tracking.

4. Define Tracking Region:

• For each tracking point, draw a tracking region around it.
• These regions should cover the area of the phone screen.

5. Analyze Motion:

• Click “Analyze” in the Tracker node properties.
• Nuke will analyze the motion of the tracking points throughout the sequence.

6. Green De-Spill:

• Use green de-spill to remove the green spill or color contamination that often occurs on edges of the subject when working with green screens.

7. Create a CornerPin Node:

• Add a CornerPin node to the Node Graph.
• Connect the output of the Tracker node to the input of the CornerPin node.

8. Apply Tracking Data to CornerPin:

• Connect the tracking data to the CornerPin’s input parameters.
• This will allow the CornerPin to stabilize the footage based on the tracked motion.

9. Create a Roto Node for the Phone Screen:

• Add a Roto node and draw a shape around the phone screen.
• Animate the shape to account for any movement or rotation that the tracker might have missed.

10. Merge the Replacement Image:

• Import the replacement image for the phone screen.
• Use a Copy or Transform node to size and position it appropriately.

11. Apply CornerPin to Replacement Image:

• Connect the output of the CornerPin node to the replacement image’s input.
• This ensures that the replacement image follows the stabilized motion.

12. Adjust Placement:

• Fine-tune the placement of the replacement image to match the stabilized phone screen.

13. Review and Iterate:

• Scrub through the timeline to review the stabilized phone screen replacement.
• Make manual adjustments as needed for a seamless integration.

14. Negate stablization

• Remove the stabilization with the same tracking data but this time using (Match-move 1-pt) transform

15. Finalize and Render:

• Once satisfied, render the final composition.

• Planar tracking is a technique used for tracking the movement and transformation of flat or planar surfaces within a video or image sequence. Unlike point tracking, which tracks individual points in an image, planar tracking focuses on tracking the entire planar surface, making it particularly useful for tracking objects with consistent textures or patterns.

Here’s how planar tracking can be done in Nuke:

Asset Gather:

1. Selection of Planar Surface:
   • Identify the planar surface in the footage that you want to track. This surface should have distinguishable features or patterns that the tracking algorithm can follow.
2. Adding a Tracker:
   • Use a planar tracker node (such as the “Tracker” node in Nuke) to add a tracker to the selected planar surface. This involves defining the region on the first frame and indicating the planar area to be tracked.

1. Automatic Tracking:
   • The planar tracker analyzes the selected features within the defined region and automatically tracks their movement across subsequent frames. This is beneficial for surfaces with consistent textures, making planar tracking robust and reliable.
2. Adjustments and Refinement:
   • After the automatic tracking, you can review and make manual adjustments if needed. This allows you to refine the tracking by correcting any drift or errors that may have occurred during the automatic tracking process.

1. Transformation Data:
   • The planar tracker provides transformation data, including position, scale, rotation, and skew, for each frame. This data can then be applied to other elements, such as graphics or effects, to ensure they match the movement and perspective changes of the tracked planar surface.
2. Integration into Compositing:
   • Once the planar tracking is complete, you can integrate other elements into the scene, and they will follow the tracked planar surface’s movement. This is particularly useful for tasks like adding labels to moving objects or seamlessly integrating CGI elements into live-action footage.

• We may encounter issues with foreground elements, such as the light pole in this image. However, we can easily overcome these challenges by utilizing the same tracking data to apply rotoscoping. The resulting roto alpha can then be used as a clipping mask to bring the pole to the foreground.

Types of filters

• Image filtering algorithms are designed to assist in determining the changes that occur in pixel values as images undergo transformation and processing. This article will dissect the various image filters at our disposal and explore their effects on images within the context of Nuke.

Resize Type:

Sinc4 – Lots of sharpening, often too much sharpening

Lanczos6 – Moderate amount of sharpening over 6 pixels. Good for scaling down.

Lanczos4 – Small amount of sharpening over 4 pixels. Good for scaling down.

Rifman – Moderate smoothing and high sharpening. Typically too hard on the sharpening in many situations.

Simon – Some smoothing and moderate sharpening. Excellent choice for many situations.

Keys – Some smoothing plus minor sharpening, decent choice for general transformations.

Anisotropic – High quality filter. Performs well with high angle surfaces. Only available in 3D nodes.

Cubic – Nuke default filter. Pixels receive some smoothing, leading to predictable results. Often too smooth.

Mitchell – Moderate smoothing, low sharpening, with a slight blur. Changes pixel values even with no movement. ​

Notch – High amounts of flat smoothing. Good for hiding buzzing or moire patterns. Changes pixel values even with no movement

Parzen – Lots of smoothing. Changes pixel values even with no movement.

Concatenation

The capability to execute a single mathematical calculation across multiple tools within the family is crucial. This singular calculation, or filter, enables us to preserve the maximum amount of detail possible.

Examples

Wrong way

Here we have an pixel and we are going to transform it 0.5 pixels in X and Y direction, after that we use and grade node to change the values and we use another transform but this time in backwards 0.5 pixels in X and Y, We would loose the quality of the pixel, since the link/concatenation in broken by using the grade in between.

Right way

Here we have an pixel and we are going to transform it 0.5 pixels in X and Y direction and we use another transform but this time in backwards 0.5 pixels in X and Y, after that we use and grade node to change the values, We would not loose the quality of the pixel, since the link/concatenation is not broken since, the grade in used after the transform calculations, using grade prior to both transformations works to.

BBOX

The bounding box defines the area of the frame that Nuke sees as having valid image data. The larger the bounding box is, the longer it takes Nuke to process and render the image. To minimize processing and rendering times, you can crop the bounding box. Occasionally, the bounding box may also be too small, in which case you need to expand it.

Motion Blur

In Nuke, motion blur can be applied to enhance the realism of moving elements within a scene. It’s achieved by calculating the movement of objects between frames and then blurring them accordingly. This helps to create a more natural and visually appealing representation of motion, especially when working with animations or scenes involving fast-moving subjects.

2D and 3D camera projections

In Nuke, 2D camera projection is a technique used to integrate 2D elements, such as images or graphics, into a 3D scene. This process involves taking a flat, 2D image and mapping it onto a 3D surface as if it’s being viewed from a specific camera perspective.

2D camera projection is commonly used in visual effects and motion graphics to add elements like signs, labels, or textures to scenes that were not present in the original footage but need to look as if they belong in the 3D environment.

Defocus

The Defocus node in Nuke is used to simulate the blurring effect that occurs when a camera is out of focus. When a camera lens is not focused perfectly on a subject, objects at different distances from the focal plane appear blurred in the captured image. The Defocus node allows you to replicate this effect in post-production, giving you control over the amount and nature of the blur applied.

Depth of field

• The ZDefocus node blurs the image according to a depth map channel. This allows you to simulate depth-of-field (DOF) blurring.
• In order to defocus the image, ZDefocus splits the image up into layers, each of which is assigned the same depth value everywhere and processed with a single blur size. After ZDefocus has processed all the layers, it blends them together from the back to the front of the image, with each new layer going over the top of the previous ones. This allows it to preserve the ordering of objects in the image.