Skip to content

Computer Vision

Tensorflow.js POC 13: Avatar Generator with Face-API.js

Overview

Figure 1 Computer Vision in AI

Avatar Generator with Face-API.js

This POC, a consequential POC of face-api.js, regonize the face from camera and find the nearest avatar from thousands avatars generated from avatar generators.

References

U2Net

SOD

SOD (Salient Object Detection) is a topics in deep learning that by given a image, SOD can automatically segmentize the most interested objects of the image without any hints. SOD learns how human see the interested objects by detecting the denisity of feature points and segmentize the most dense parts. So far, U2Net provide a state of art performance.

First results of U2Net

These are the first results of the U2Net on target benchmark images. For the full results can be checked in Chimay-SOD1 and asubset Chimay-SOD2 can be found.

{% include ideal-image-slider/slider.html selector="slider1" %}

Image sliders

In this page, image slider for jekyll and its js code is used for image slider. Also a Jekyll Ideal Image Slider Include Demo shows the possiblity of Ideal Image Slider.

References

Tensorflow.js POC 12: Cam Face-recognition on face-api.js

Overview

Figure 1 Computer Vision in AI
Git Repo Status Progress Comments
face-api.js status progress tensorflow.js POC #11, #12

What is reinforcement learning?

Goals of POC #12

  • Improve the POC #11 from semi-supervised learning to Reinforcement Learning by improve its face recognition rate

References

Tensorflow.js POC 7: face-api.js

Overview

Figure 1 Computer Vision in AI

Project Status

Git Repo Status Progress Comments
face-api.js status progress tensorflow.js POC #7

Project map of DLC

Check the project in the project map of deep learning computing to see the whole picture.

Edit mindmap

Test #1

Reference Input

Step 1: Input reference image references

Step 2: extract faces and output face descriptors (Biometrics information of faces, 120 vectors value) reference extract faces

Query Input

Step 1: Input query image query

Step 2: extract faces and output query face descriptors reference extract faces

Step 3: Compare reference and query face descriptors and output similarity score

red block is missing faces.

Test #2

Reference Input

reference

Query Input

red block is missing faces.

reference

References

Max POC 2: Super Resolution with SRGAN

Overview

Figure 1 Computer Vision in AI
Git Repo Status Progress Comments
Max SRGAN status progress IBM MAX SRGAN POC #2

The above image is one result of SRGAN. It is not good in artwork image. Visit the online benchmark comparsion of SRGAN of several different images on POC: Super Resolution with waifu2x

References

Tensorflow.js POC 8: Super Resolution with waifu2x

Overview

Figure 1 Computer Vision in AI
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(1).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(2).twentytwenty({default_offset_pct: 0.5,before_label: 'SRGAN',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(3).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(4).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'SRMD-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(5).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'Anime4K',click_to_move: true});
        $(".twentytwenty-container").eq(6).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'Anime4K',click_to_move: true});

        $(".twentytwenty-container").eq(7).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(8).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(9).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(10).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});

        $(".twentytwenty-container").eq(11).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(12).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(13).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});
        $(".twentytwenty-container").eq(14).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(15).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_photo',click_to_move: true});
      });
    </script>
</body>
Git Repo Status Progress Comments
waifu2x status progress tensorflow.js POC #8

Overview

Super Resolution of images are important for video quality. Common SR like bicubic or lanczos3, now embedded in GPU as default SR. But, in 4K display or larger display, common SR is not enough, SR with Deep learning provide significant improvements on differnt target images.

Super resolution of images highly depends the properties of the image, like animation, line draw, or a full color complex nature images will have different results on different algorithms. Here we benchmark several traditional SR algorithm of bicubic, lanczos3, (Bell, and ....) from imageenlarger and do a comparison with a DL SR on line draw or animation style image.

Now we provide A/B Test on the different target images to verify the results of deep learning and normal images.

A/B test on waifu2x-NCNN vs. waifu2x vs. lanczos3 vs. bicubic vs. SRGAN

1096x632_PPT_image

1096x632_PPT_image is a 1096x632 size PNG with words and graph PPT. We use it to identify the clearance of the SR.

SRMD-NCNN = waifu2x-NCNN = waifu2x > lanczos3 > bicubic > Anime4K > SRGAN (bicubic is most GPU interpolation algorithm. So lanczos3 and waifu2x show better results to normal GPU). SRGAN, Anime4K: SR result is GG..

Quality comparison: bicubic < waifu2x
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: lanczos3 < waifu2x
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: SRGAN < waifu2x. 
SRGAN is GG

{% slider2020 %}
  ![SRGAN](../../assets/images/1096x632_PPT_image-SRGAN.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = waifu2x-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = SRMD-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![SRMD-NCNN](../../assets/images/1096x632_PPT_image_SRMD-NCNN-Vulkan.png)
{% endslider2020 %}


#### Quality comparison: waifu2x > Anime4K

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

#### Quality comparison: bicubic > Anime4K

{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

Part II

Check Part II

Next step

  • WenGL version of waifu2x.js
  • Speedup of waifu2x

References

Jekyll image comparison slider

In this page, twentytwenty image compare slider is used to enable image comparison slider.

To enable it in your post, please add the following code into your markdown documents

1
2
3
4
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![Waifu2x](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

After above, you can have first version of image compariosn slider. But, if you want to change the label of the images. you will need to find default_with_image_slider.html and modify the code as below in the beginning of the .md file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x-NCNN',click_to_move: true});
      });
    </script>
</body>

You also can check the code of this site in gitbucket and search for all 'twentytwenty'