Skip to content

Blog

  • in AI
  • 1 min read

Supervised Learning vs. Semi-Supervised Learning vs. Reenforcement Learning

For deep learning, there are severals ways how the learning can be processed. Here we put a new tag on how a POC or a algorithm learning from the dataset with the 4 categories as below:

Unsupervised Learning

Supervised Learning

Semi-Supervised Learning

Reinforcement Learning

References

  • in AI
  • 2 min read

Pre-trained Model, and What is Next Steps

Pre-trained Model

Pre-trained model for deep learning is the easiest entry point for an application developers to start the deep learning. With the pre-trained model as a blackbox, the application developers can focus on the problem he want to resolve with the pre-trained model and do the first integration and see the result of the integration resolve the target problem or not and how far it is to the perfect goal. If the goal become nearer, then a fine-tuning of pre-trained model or even a from-scratch pre-trained model can be addressed as a solution for the final integration.

There are several pre-trained models based on different programming langnauges and platforms.

Tensorflow.js Pre-trained Models

IBM MAX Pre-trained Models

nVidia Pre-trained Models

Pytorch Pre-trained Models

Fine-tune Models

Once the pre-trained model gives some promise on the solution to the problem. A deeper and better solution can be addressed by fine-tuning the model. We call it as fine-tuning model of pre-trained model.

There are several strategies of fine-tuning the models from much training efforts to less ones.

Another view for different dataset size for fine-tining models.

MLOps

Main idea of whole ML pipeline.

Basic idea of MLOps pipeline.

References

Tensorflow.js POC 12: Cam Face-recognition on face-api.js

Overview

Figure 1 Computer Vision in AI
Git Repo Status Progress Comments
face-api.js status progress tensorflow.js POC #11, #12

What is reinforcement learning?

Goals of POC #12

  • Improve the POC #11 from semi-supervised learning to Reinforcement Learning by improve its face recognition rate

References

Tensorflow.js POC 7: face-api.js

Overview

Figure 1 Computer Vision in AI

Project Status

Git Repo Status Progress Comments
face-api.js status progress tensorflow.js POC #7

Project map of DLC

Check the project in the project map of deep learning computing to see the whole picture.

Edit mindmap

Test #1

Reference Input

Step 1: Input reference image references

Step 2: extract faces and output face descriptors (Biometrics information of faces, 120 vectors value) reference extract faces

Query Input

Step 1: Input query image query

Step 2: extract faces and output query face descriptors reference extract faces

Step 3: Compare reference and query face descriptors and output similarity score

red block is missing faces.

Test #2

Reference Input

reference

Query Input

red block is missing faces.

reference

References

Max POC 2: Super Resolution with SRGAN

Overview

Figure 1 Computer Vision in AI
Git Repo Status Progress Comments
Max SRGAN status progress IBM MAX SRGAN POC #2

The above image is one result of SRGAN. It is not good in artwork image. Visit the online benchmark comparsion of SRGAN of several different images on POC: Super Resolution with waifu2x

References

Tensorflow.js POC 8: Super Resolution with waifu2x

Overview

Figure 1 Computer Vision in AI
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(1).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x',click_to_move: true});
        $(".twentytwenty-container").eq(2).twentytwenty({default_offset_pct: 0.5,before_label: 'SRGAN',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(3).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'waifu2x-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(4).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'SRMD-NCNN',click_to_move: true});
        $(".twentytwenty-container").eq(5).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x',after_label: 'Anime4K',click_to_move: true});
        $(".twentytwenty-container").eq(6).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'Anime4K',click_to_move: true});

        $(".twentytwenty-container").eq(7).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(8).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_artwork',click_to_move: true});
        $(".twentytwenty-container").eq(9).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(10).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});

        $(".twentytwenty-container").eq(11).twentytwenty({default_offset_pct: 0.5,before_label: 'bicubic',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(12).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'lanczos3',click_to_move: true});
        $(".twentytwenty-container").eq(13).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_photo',click_to_move: true});
        $(".twentytwenty-container").eq(14).twentytwenty({default_offset_pct: 0.5,before_label: 'waifu2x_artwork',after_label: 'waifu2x_artwork_y',click_to_move: true});
        $(".twentytwenty-container").eq(15).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x_photo',click_to_move: true});
      });
    </script>
</body>
Git Repo Status Progress Comments
waifu2x status progress tensorflow.js POC #8

Overview

Super Resolution of images are important for video quality. Common SR like bicubic or lanczos3, now embedded in GPU as default SR. But, in 4K display or larger display, common SR is not enough, SR with Deep learning provide significant improvements on differnt target images.

Super resolution of images highly depends the properties of the image, like animation, line draw, or a full color complex nature images will have different results on different algorithms. Here we benchmark several traditional SR algorithm of bicubic, lanczos3, (Bell, and ....) from imageenlarger and do a comparison with a DL SR on line draw or animation style image.

Now we provide A/B Test on the different target images to verify the results of deep learning and normal images.

A/B test on waifu2x-NCNN vs. waifu2x vs. lanczos3 vs. bicubic vs. SRGAN

1096x632_PPT_image

1096x632_PPT_image is a 1096x632 size PNG with words and graph PPT. We use it to identify the clearance of the SR.

SRMD-NCNN = waifu2x-NCNN = waifu2x > lanczos3 > bicubic > Anime4K > SRGAN (bicubic is most GPU interpolation algorithm. So lanczos3 and waifu2x show better results to normal GPU). SRGAN, Anime4K: SR result is GG..

Quality comparison: bicubic < waifu2x
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: lanczos3 < waifu2x
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
{% endslider2020 %}

#### Quality comparison: SRGAN < waifu2x. 
SRGAN is GG

{% slider2020 %}
  ![SRGAN](../../assets/images/1096x632_PPT_image-SRGAN.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = waifu2x-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![waifu2x-NCNN](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

#### Quality comparison: waifu2x = SRMD-NCNN

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![SRMD-NCNN](../../assets/images/1096x632_PPT_image_SRMD-NCNN-Vulkan.png)
{% endslider2020 %}


#### Quality comparison: waifu2x > Anime4K

{% slider2020 %}
  ![waifu2x](../../assets/images/1096x632_PPT_image_waifu2x_art_noise1_scale_tta_1.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

#### Quality comparison: bicubic > Anime4K

{% slider2020 %}
  ![bicubic](../../assets/images/1096x632_PPT_image-bicubic.png)
  ![Anime4K](../../assets/images/1096x632_PPT_image_Anime4K.png)
{% endslider2020 %}

Part II

Check Part II

Next step

  • WenGL version of waifu2x.js
  • Speedup of waifu2x

References

Jekyll image comparison slider

In this page, twentytwenty image compare slider is used to enable image comparison slider.

To enable it in your post, please add the following code into your markdown documents

1
2
3
4
{% slider2020 %}
  ![lanczos3](../../assets/images/1096x632_PPT_image-lanczos3.png)
  ![Waifu2x](../../assets/images/1096x632_PPT_image_Waifu2x-NCNN-Vulkan.png)
{% endslider2020 %}

After above, you can have first version of image compariosn slider. But, if you want to change the label of the images. you will need to find default_with_image_slider.html and modify the code as below in the beginning of the .md file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
<body>
    <script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

    <script src="../../assets/plugins/jquery.event.move.js"></script>
    <script src="../../assets/plugins/jquery.twentytwenty.js"></script>
    <script>
      $(function()
      {

        $(".twentytwenty-container").eq(0).twentytwenty({default_offset_pct: 0.5,before_label: 'lanczos3',after_label: 'waifu2x-NCNN',click_to_move: true});
      });
    </script>
</body>

You also can check the code of this site in gitbucket and search for all 'twentytwenty'