UE4 Raytracing Part 2 - Refractions and DepthOfField
One unfortunate limitation to the rendering in Unreal Engine is that there seems to be no way to get translucent objects to write to the depth buffer. I hope this changes soon and this hack can be forgotten, but until then I'll present here a way to get DOF working with refractive materials. It is not perfect, and creates some artifacts (fringing and noise), but most can be worked around on a shot-by shot basis with some tweaking. I tried a lot of switches (including writing to custom depth) but none worked with the cinematic dof, which has been the goal.
The slides below illustrate the issue. Red bottles are opaque, green bottles are translucent (but set to solid for illustration purposes). The first image where both are opaque shows what the dof should look like. The second image switches to a translucent material on the left and incorrect dof. The third image is a visualization mode in unreal that overlays the dof buffer. This shows that the left bottles when translucent aren't considered in the dof calculation.
To get something to write into the depth buffer, I used a masked material. Masked materials can be set to dither opacity. I set the DitherOpacityMask to on, and lowered the OpacityMaskClipValue to 0.2. I also added some Emissive and color controls which help tweak away artifacts as needed.
Each translucent object needing to be correctly included in dof should be duplicated and the duplicates should receive this "InvisibleWithDepth" material. It technically isn't invisible, since when you make it totally invisible, dof doesn't work. The goal is to make a "barely perceptible" duplicate that contributes enough to the depth buffer, but creates minimal visual interference. I've found that putting the OpacityMaskClipValue and the Opacity both at 0.2 seems to be the best I've gotten in most cases. You can experiment to find something that works for your scene.
In the two slides below, you can see the result in my test scene where the green (translucent) bottles are rendering dof more accurately.
Here's a video test with the glass set back to fully transparent.
Note that this doesn't solve for complex dof effects inside the refractions, such as this case where glass bottles are being refracted through more glass bottles. The dof is not physically accurate, since it is still only using the depth buffer. It is however, vastly better than having no dof at all. As for visual artifacts, you can see some noise, but most is removed by TAA. Setting glass objects to not cast shadows can help, but there is also some fringing on the glass edges. This is why I added Emissive controls to the InvisibleWithDepth material. You can set the emissive color and intensity to better blend your edges if you need to.
Hopefully this can get you past any major hurdles where you need some cinematic goodness with glass and liquid.
I'd also like to thank folks at Epic, like Juan Cañada who has patiently put up with my questions, even though he's a busy man! I'm guessing it's a matter of time before this particular issue is "sorted".