Real time beauty filters based on GPUImage

1 background

Some time ago because of the needs of the project, made a real beauty based on GPUImage filter. Now all kinds of live video App beauty filters demand also emerge in an endless stream, more and more. In order to contribute to the open source, now I put it in the GitHub above, interested friends can go to download. The following will introduce the principle and ideas of beauty filters.

2.GPUImage

GPUImage is based on GPU image or video processing framework is an open source, its itself is built as many as 120 kinds of common filter effects. With it, adding a real-time filter simply adds a few lines of code. The following example is based on the camera data as the source of real time color operations (like photo effect):

Real time beauty filters based on GPUImage
using GPUImage camera data to add filters example 2.1

In fact, beauty is the same, if there is such a beauty filter (called GPUImageBeautifyFilter), then only need to sample 2.1 GPUImageColorInvertFilter GPUImageBeautifyFilter can be replaced. We just need to do a GPUImageBeautifyFilter can achieve real beauty, the question is, what is beauty? My understanding is that everyone needs is more common for the beauty, whitening skin grinding. Of course, to improve saturation, according to the needs of the light and so on. This article will focus on the implementation of the grinding system (in fact, GPUImageBeautifyFilter also achieved the whitening, brightening effect).

3 grinding

The essence of microdermabrasion is fuzzy. In the field of image processing, fuzzy is the value of the pixel value and the value of the surrounding pixels. The common fuzzy pixel values of Gauss, it is the point of the weighted average income by neighboring pixels, and the weight coefficient is the Gauss function of distance between pixels, the distance is small, roughly the relationship is the weight coefficient is big. Figure 3.1 is an example of Gauss’s blurring effect:

Real time beauty filters based on GPUImage
Gauss fuzzy example 3.1

If only use Gauss to fuzzy microdermabrasion, the result is unsatisfactory. The reason is that Gauss fuzzy only consider the distance between pixels, does not take into account the difference between the pixel value itself. For example, the hair and the face of the demarcation line (color difference is very large, black and human skin color), if the use of Gauss blur the edge will be blurred, which is obviously not what we want to see. The bilateral filter (Bilateral Filter) considers the color difference, the weighted average of pixel values of it but also the surrounding pixels, and the weight is Gauss function. This weight is different, not only with the distance between the pixel and the pixel value, is itself differences, in particular, the pixel value difference is smaller, the greater the weight, but also the characteristic that it has the characteristics of edge preservation, so it is a good skin grinding tool. Figure 3.2 is an example of the effect of bilateral filtering:

Real time beauty filters based on GPUImage
bilateral filtering effect example 3.2

Between 3.1 and 3.2, bilateral filtering effect is indeed in the face details keep better, so I used as a basis for bilateral filtering algorithm dermabrasion. Bilateral filtering in GPUImage is also achieved, is GPUImageBilateralFilter.

According to Figure 3.2, you can see in the figure is still part of the facial details protection is not enough, and we don’t want the human hair also blurred (we only need to deal with the skin). The extension of this idea is the combination of bilateral filtering, edge detection and skin color detection. The overall logic is as follows:

Real time beauty filters based on GPUImage
grinding process logic 3.3

Combination Filter filter with three input of our own definition of the. The three inputs are the original image A (x, y), bilateral filtered image B (x, y), edge image C (x, y). Among them, A, B, C can be regarded as the image matrix, (x, y) can be seen as a pixel coordinates. Combination processing logic Filter as below:

Real time beauty filters based on GPUImage
Filter logic diagram 3.3
Combination

The following is the main shader code:

Real time beauty filters based on GPUImage
combination shader code 3.4 filter

Combination Filter by skin color detection and edge detection, only on the skin and non edge processing. The following is the use of this way after the grinding effect diagram:

Real time beauty filters based on GPUImage
final grinding effect figure 3.5

Between 3.5 and 3.2, you can see the better protection of 3.5 face details, at the same time for the facial skin grinding effect is also very good, people feel more real.

4 extension

Microdermabrasion algorithm I have adopted is based on bilateral filtering, mainly on account of it combined with the space between pixels and the pixel value difference distance. Of course, it is not necessary to use bilateral filtering, but also through the improvement of Gauss fuzzy (combined with pixel value difference) to achieve grinding, and even better results can be achieved. In addition GPUImageBeautifyFilter not only has a skin grinding function, realizing the log curve of color, brightness, saturation adjustment, the specific details can be found in demo.