Skip to content

Computer Vision

Tensorflow.js POC 8: Super Resolution with waifu2x (II)

Overview

Figure 1 Computer Vision in AI
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(1).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(2).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(3).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});

        $(".twentytwenty-container").eq(4).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(5).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(6).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});
        $(".twentytwenty-container").eq(7).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(8).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(9).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(10).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_photo',click_to_move: true});

      });
    </script>
</body>
Git Repo Status Progress Comments
waifu2x status progress tensorflow.js POC #8

avatar255x287

avatar255x287 is a 255x287 size PNG animation. We use it to identify the aliasing reduction of the SR.

waifu2x-photo = waifu2x-artwork_y = waifu2x-artwork > lanczos3 > bicubic

Quality comparison: bicubic < waifu2x_artwork
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
{% slider2020 %}
  ![bicubic](../../assets/images/avatar255x287-bicubic.png)
  ![waifu2x_artwork](../../assets/images/avatar255x287-waifu2x_artwork.png)
{% endslider2020 %}

#### Quality comparison: lanczos3 < waifu2x_artwork
{% slider2020 %}
  ![lanczos3](../../assets/images/avatar255x287-lanczos3.png)
  ![waifu2x_artwork](../../assets/images/avatar255x287-waifu2x_artwork.png)
{% endslider2020 %}

#### Quality comparison: waifu2x_artwork = waifu2x_artwork_y

{% slider2020 %}
  ![waifu2x_artwork](../../assets/images/avatar255x287-waifu2x_artwork.png)
  ![waifu2x_artwork_y](../../assets/images/avatar255x287-waifu2x_artwork_y.png)
{% endslider2020 %}

#### Quality comparison: waifu2x_artwork = waifu2x_photo

{% slider2020 %}
  ![waifu2x_artwork](../../assets/images/avatar255x287-waifu2x_artwork.png)
  ![waifu2x_photo](../../assets/images/avatar255x287-waifu2x_photo.png)
{% endslider2020 %}

### lena400x400

lena400x400 is a 400x400 size JPG with natural human. We use it to identify the color richness and fine struture of the SR.

 waifu2x-artwork_y = waifu2x-artwork > waifu2x-photo > lanczos3 = bicubic

#### Quality comparison: bicubic = lanczos3

{% slider2020 %}
  ![bicubic](../../assets/images/lena400x400-bicubic.png)
  ![lanczos3](../../assets/images/lena400x400-lanczos3.png)
{% endslider2020 %}

#### Quality comparison: waifu2x_artwork > lanczos3

{% slider2020 %}
  ![waifu2x_artwork](../../assets/images/lena400x400-waifu2x_artwork.png)
  ![lanczos3](../../assets/images/lena400x400-lanczos3.png)
{% endslider2020 %}

#### Quality comparison: waifu2x_artwork > waifu2x_photo

{% slider2020 %}
  ![waifu2x_artwork](../../assets/images/lena400x400-waifu2x_artwork.png)
  ![lanczos3](../../assets/images/lena400x400-waifu2x_photo.png)
{% endslider2020 %}

#### Quality comparison: waifu2x_artwork = waifu2x_artwork_y

{% slider2020 %}
  ![waifu2x_artwork](../../assets/images/lena400x400-waifu2x_artwork.png)
  ![waifu2x_artwork_y](../../assets/images/lena400x400-waifu2x_artwork_y.png)
{% endslider2020 %}

#### Quality comparison: lanczos3 < waifu2x_photo

{% slider2020 %}
  ![lanczos3](../../assets/images/lena400x400-lanczos3.png)
  ![waifu2x_photo](../../assets/images/lena400x400-waifu2x_photo.png)
{% endslider2020 %}

Next step

  • WenGL version of waifu2x.js
Draft

Tensorflow.js POC 1: Pose-animator

Figure 1 Computer Vision in AI
Git Repo Status Progress Comments
pose-animator status progress tensorflow.js POC #1- and WhoIsTalking for PIC #9.

Demo

A demo for two people for pose-animators. Since pose-estmiation and facemesh are not correctly aligned so sometimes, face and body will be mis-aligned. So next step for pose-animator is to improve the pose-estmiation and facemesh alignment for multiple users.

Next Milestone

  1. To support pose-estmiation and facemesh alignment for multiple users