Skip to content

Tensorflow.js POC 8: Super Resolution with waifu2x

Overview

ComputervisioninAllVision
Figure 1 Computer Vision in AI
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(1).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(2).twentytwenty({default_offset_pct: 0.5,before_label: 'SRGAN',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(3).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(4).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'SRMD-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(5).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'Anime4K',click_to_move: true});
        $(".twentytwenty-container").eq(6).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'Anime4K',click_to_move: true});

        $(".twentytwenty-container").eq(7).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(8).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(9).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(10).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});

        $(".twentytwenty-container").eq(11).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(12).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(13).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});
        $(".twentytwenty-container").eq(14).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(15).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_photo',click_to_move: true});
      });
    </script>
</body>
Git Repo Status Progress Comments
waifu2x status progress tensorflow.js POC #8

Overview

Super Resolution of images are important for video quality. Common SR like bicubic or lanczos3, now embedded in GPU as default SR. But, in 4K display or larger display, common SR is not enough, SR with Deep learning provide significant improvements on differnt target images.

Super resolution of images highly depends the properties of the image, like animation, line draw, or a full color complex nature images will have different results on different algorithms. Here we benchmark several traditional SR algorithm of bicubic, lanczos3, (Bell, and ....) from imageenlarger and do a comparison with a DL SR on line draw or animation style image.

Now we provide A/B Test on the different target images to verify the results of deep learning and normal images.

A/B test on waifu2x-NCNN vs. waifu2x vs. lanczos3 vs. bicubic vs. SRGAN

1096x632_PPT_image

1096x632_PPT_image is a 1096x632 size PNG with words and graph PPT. We use it to identify the clearance of the SR.

SRMD-NCNN = waifu2x-NCNN = waifu2x > lanczos3 > bicubic > Anime4K > SRGAN (bicubic is most GPU interpolation algorithm. So lanczos3 and waifu2x show better results to normal GPU). SRGAN, Anime4K: SR result is GG..

Quality comparison: bicubic < waifu2x

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: lanczos3 < waifu2x
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: SRGAN < waifu2x. 
SRGAN is GG

{% slider2020 %}
  ![SRGAN](../../assets/images/1096x632_PPT_image-SRGAN.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = waifu2x-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = SRMD-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![SRMD-NCNN](../../assets/images/1096x632_PPT_image_SRMD-NCNN-Vulkan.png)
{% endslider2020 %}


#### Quality comparison: waifu2x > Anime4K

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

#### Quality comparison: bicubic > Anime4K

{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

Part II

Check Part II

Next step

  • WenGL version of waifu2x.js
  • Speedup of waifu2x

References

Jekyll image comparison slider

In this page, twentytwenty image compare slider is used to enable image comparison slider.

To enable it in your post, please add the following code into your markdown documents

1
2
3
4
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![Waifu2x](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

After above, you can have first version of image compariosn slider. But, if you want to change the label of the images. you will need to find default_with_image_slider.html and modify the code as below in the beginning of the .md file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x-NCNN',click_to_move: true});
      });
    </script>
</body>

You also can check the code of this site in gitbucket and search for all 'twentytwenty'