Customized filters for face tracking app

Technologies used for this project:

  • C++
  • OpenGL
  • GLSL

Customized filters for face tracking app

Aim

Develop unique filters for LiveCam application for Game Changer

Project

Game Changer has LiveCam application. It is using third-party solution for face landmark detection by ULSee. And they are searching for best ways to visualize face detection.

Fish Run Games team is experienced to work with 3D graphics, shaders, and visual effect. So, we came up with several solutions to be implemented in LiveCam.

Filters implemented

Face paint

Face paint should not only lay perfectly on the face, but it also should look natural when it comes to skin color and lights. Paint color 'mixes' with the skin color in a right proportion and changes its level of saturation depending on light.

Here are some visual examples of face paint visual effect.

Helmet cut

Helmets and hats are placed on head, and in order to look naturally, parts behind head should not be rendered. To solve this problem, we use depth buffer to render a head silhouette, and then apply the depth mask to filter out parts of the image.

Dynamic lighting

Some filters have dynamic animation, such as thunder helmet. To make it look more natural, we apply additional light to helmet and synchronize its intensity to thunder animation.

Screenshot:

Translucent effects

For translucent 3d model rendering, its vertices and polygons should be dynamically sorted from back to front, and each time order depends on how the head is turned.

Here is example video of translucent ghost render.