Using OpenVINO with GPU on UP Squared reboot the system

dhoadhoa New Member Posts: 10

I'm using OpenVINO with GPU for inference my deep learning model. It's very often, the UP squared board automatically reboot when I use GPU. It might because of overheating, after I use the cooling fan to decrease from 65°C to 45°C, the problems disappeared.

Anyone knows about this problem, in my opinion, 65°C is not very hot and the GPU shouldn't make the system reboot right ?

Tagged:

Comments

  • DCleriDCleri Administrator, AAEON Posts: 731 admin

    No it shouldn't reboot when the temperature is 65 degrees.

    can you provide the sample or application that cause the system to reboot? Can you make the system reboot with another OpenVINO sample while using the GPU?

  • dhoadhoa New Member Posts: 10

    Thanks for your response. Actually, I think about the temperature because I ran the same program and sometimes it works, sometimes it make the computer reboot. I tested with 2 different models, resnet and SSD and the same things happened. When I ran with cooling fan, everything is ok

    Actually, I use sensors command in linux to measure the temperature. And I think it is for the CPU temperature. I don't know how to measure precisely the Intel GPU.

  • dhoadhoa New Member Posts: 10

    Can someone give me some hints why the system can be reboot when using GPU ? The problem is still there and is unpredictable for me. Thanks

  • DCleriDCleri Administrator, AAEON Posts: 731 admin

    The system should not reboot even if under stress, where eventually it would throttle the frequency down.

    Can you tell me how it the UP Squared positioned? Is the heatsink on top, or the board is upside down?

    Do you have an example from the OpenVINO samples for GPU that we can use to reproduce the issue?

    Also what is the hardware and software complete setup?

    BIOS version, Operating System version, kernel version, OpenVINO version etc.

  • dhoadhoa New Member Posts: 10

    The heatsink is down. One OpenVINO model you can test with is face-detection-retail-0004 (you can find it in OpenVINO example).

    I used the UP squared board, VPU attached (but it is not used during the test).

    Bios Version: UPA1AM42
    OS: Ubuntu 18.04,
    Kernel version: 4.15.0-37-generic
    OpenVINO version: 2019 R3.1

    You need to retest it several times because most of the times it runs ok.

    Thank you for your helps

  • DCleriDCleri Administrator, AAEON Posts: 731 admin

    The heatsink should be facing up for better dissipation, especially if you are stressing CPU and GPU.
    Try with some spacers if you have some.

    About the BIOS, I don't think it is the case, but it is good to update to the latest version available: https://downloads.up-community.org/download/up-squared-uefi-bios-v4-6/

    About the kernel version, is that the UP Ubuntu kernel installed from our PPA?

    Last thing which I forgot to ask, what is the Power Supply that you are using?

  • dhoadhoa New Member Posts: 10

    The kernel I got from your PPA
    Kernel = 4.15.0-37-generic
    Patch = #40~upboard04-Ubuntu SMP Thu Feb 14 13:49:37 UTC 2019

    Power getting from plugging into normal Europe electricity.
    Adapter model FJ-SW0506000
    Input 1.5A Max
    Output 5V 6000mA

  • DCleriDCleri Administrator, AAEON Posts: 731 admin

    ok, thank you.

    What is the CPU/RAM configuration of the Board?
    Celeron, Pentium or Atom? 2, 4 or 8GB?

  • dhoadhoa New Member Posts: 10

    CPU information:
    vendor_id : GenuineIntel
    cpu family : 6
    model : 92
    model name : Intel(R) Pentium(R) CPU N4200 @ 1.10GHz

    RAM: 8GB

  • dhoadhoa New Member Posts: 10

    More specific for Ubuntu version: Ubuntu 18.04.3 LTS

    Because someone told me maybe there are problems in 18.04.3

    Thanks

  • dhoadhoa New Member Posts: 10
    edited November 12

    I minimize the code to reproduce the problem for you guys can see below

    import cv2                                                                                                       
    import numpy as np
    import time
    from openvino.inference_engine import IENetwork, IEPlugin
    net = IENetwork(model='/home/k/face-detection-retail-0004.xml', weights='/home/k/face-detection-retail-0004.bin')
    plugin = IEPlugin(device="GPU")
    exec_net = plugin.load(network=net)
    
    def _preprocess_face(img, shape=(300,300)):
        origin_shape = img.shape
        img_processed = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)
        img_processed = cv2.resize(img_processed, (shape[0], shape[1]))
        img_processed = img_processed.transpose((2, 0, 1))
        return img_processed
    
    image = cv2.imread('/home/k/_.jpg')
    
    for i in range(100):
        img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        img = _preprocess_face(img)
        start = time.time()
        res = exec_net.infer({'data': img})
        print(time.time()-start)
    
    

    The output I got is:
    $ python gpu.py
    0.017605304718017578
    0.014819860458374023

    It ran 2 times then the system reboot. I have tried many differents things that I can imagine: I use different model, run as asynchronous, synchronous but the problems is always there. sometimes it reboots directly from the 1 try, sometimes after several try.

    I would really appreciate if someone can help me on that.

    The model I downloaded from Openvino pretrained model for face detection

Sign In or Register to comment.