CodeProject.AI Discussions

15-Nov-23 6:13

Is the latest Linux/amd64 Docker image 2.3.2-alpha?
Any issues using an alpha image as opposed to going back to an earlier -beta image?
Will there be a 2.3.4-beta version of docker image for linux/amd64 soon?

Chris Maunder16-Nov-23 7:47

16-Nov-23 7:47

Better than that, we'll have a 2.4.0 soon. I'd avoid the alpha for now

cheers
Chris Maunder

Steve Winn16-Nov-23 8:09

16-Nov-23 8:09

Thanks I'll watch out for it.

Steve Winn29-Nov-23 3:46

29-Nov-23 3:46

Soon? Do you mean a linux/amd64 Docker image? (Soon?)
Or is there a linux/amd64 ver. 2.3.4-beta image that I don't see?
The latest beta image that I see for that hardware is 2.2.4.

Chris Maunder29-Nov-23 8:35

How do I select which GPU to use?

29-Nov-23 8:35

By end of week. Sorry - we've had a couple of non-CodeProject.AI emergencies that have thrown our schedule out a couple of days. At this point I think 2.4 is baked, so we're just building and testing Docker images and then the native installers.

If you're on the Insiders list then you should see a note posted in the next day or so, or if you want we can just post what we have and you guys can be proper guinea pigs.

cheers
Chris Maunder

Wichall15-Nov-23 0:12

Wichall

15-Nov-23 0:12

Hi, how do I select which GPU to use for object detection?

Re: How do I select which GPU to use?

Chris Maunder16-Nov-23 7:50

Coral USB Accelerator vs M.2 Accelerator

16-Nov-23 7:50

See Configuring Module Settings. Specifically, in the modulesettings.json file you can set AcceleratorDeviceName.

Module dependant, but for modules that use CUDA, for example, this could be "cuda:0" or "cuda:1" to specify the first or second CUDA enabled GPU

cheers
Chris Maunder

Steve Winn14-Nov-23 11:58

14-Nov-23 11:58

After having unsatisfactory performance and reliability results trying a Coral USB Accelerator, I've decided to try an M.2 Accelerator to see if it is more reliable on my HP desktop running Ubuntu 22.04.
Any advice or suggestions?
I will report back any results.

shane.x14-Nov-23 17:35

shane.x

14-Nov-23 17:35

If you are referring to accuracy with Blue Iris or other ipcam software, it will be just as inaccurate. The current models for the Coral TPU are not appropriate for ipcam usage.

Steve Winn15-Nov-23 5:36

15-Nov-23 5:36

I didn't really see issues with accuracy. My problem was the USB version would become extremely slow. Randomly. I would have to constantly keep checking to see that it was working. So slow it would not really return results to Blue Iris.
I'm going to give it a try, worse comes to worse, I've wasted another $38.00.

Edit: Very slow. 1750 msec. It seems that CPAI does not see the device. The Coral Object Detection module shows CPU. I uninstalled and reinstalled the Coral module using the Do not use download cache switch.

The operating system sees the device.

Once rebooted, verify that the accelerator module is detected:

$ lspci -nn | grep 089a
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

Server version:   2.3.2-Alpha
System:           Docker
Operating System: Linux (Linux 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2)
CPUs:             Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz (Intel)
                  1 CPU x 4 cores. 4 logical processors (x64)
System RAM:       8 GiB
Target:           Linux
BuildConfig:      Release
Execution Env:    Docker
Runtime Env:      Production
.NET framework:   .NET 7.0.13
Video adapter info:
System GPU info:
  GPU 3D Usage       0%
  GPU RAM Usage      0
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

    Module 'ObjectDetection (Coral)' 1.6.3 (ID: ObjectDetectionCoral)
    Module Path:   <root>/modules/ObjectDetectionCoral
    AutoStart:     True
    Queue:         objectdetection_queue
    Platforms:     windows,linux,linux-arm64,macos,macos-arm64
    GPU Libraries: installed if available
    GPU Enabled:   enabled
    Parallelism:   1
    Accelerator:   
    Half Precis.:  enable
    Runtime:       python3.9
    Runtime Loc:   Local
    FilePath:      objectdetection_coral_adapter.py
    Pre installed: False
    Start pause:   1 sec
    LogVerbosity:  
    Valid:         True
    Environment Variables
       MODELS_DIR = <root>/modules/ObjectDetectionCoral/assets
       MODEL_SIZE = Medium
    Started:      16 Nov 2023 4:34:40 PM Coordinated Universal Time
    LastSeen:     16 Nov 2023 4:48:43 PM Coordinated Universal Time
    Status:       Started
    Processed:    3
    Provider:     
    CanUseGPU:    False
    HardwareType: CPU


    Installation Log
    2023-11-16 16:13:53: Hi Docker! We will disable shared python installs for downloaded modules
    2023-11-16 16:13:53: No schemas installed
    2023-11-16 16:13:53: (No schemas means: we can't detect if you're in light or dark mode)
    2023-11-16 16:13:54:              Installing CodeProject.AI Analysis Module                
    2023-11-16 16:13:54: ======================================================================
    2023-11-16 16:13:54:                    CodeProject.AI Installer                           
    2023-11-16 16:13:54: ======================================================================
    2023-11-16 16:13:54: 255.09 GiB available on Docker
    2023-11-16 16:13:54: General CodeProject.AI setup                                          
    2023-11-16 16:13:54: Setting permissions...Done
    2023-11-16 16:13:54: GPU support                                                           
    2023-11-16 16:13:54: CUDA Present...No
    2023-11-16 16:13:54: ROCm Present...No
    2023-11-16 16:13:54: Processing module ObjectDetectionCoral 1.6.3                          
    2023-11-16 16:13:54: Installing Python 3.9
    2023-11-16 16:13:54: Python 3.9 is already installed
    2023-11-16 16:14:00: Upgrading PIP in base python install... done
    2023-11-16 16:14:00: Installing Virtual Environment tools for Linux...
    2023-11-16 16:14:19: Searching for python3-pip python3-setuptools python3.9...installing...   Done
    2023-11-16 16:14:23: Creating Virtual Environment (Local)... Done
    2023-11-16 16:14:23: Checking for Python 3.9...Found Python 3.9.18. present
    2023-11-16 16:14:25: Upgrading PIP in virtual environment... done
    2023-11-16 16:14:28: Installing updated setuptools in virtual environment... Done
    2023-11-16 16:14:28: Edge TPU library found.
    2023-11-16 16:14:31: Downloading MobileNet models... already exists...Expanding... Done.
    2023-11-16 16:14:32: Installing Python packages for ObjectDetectionCoral
    2023-11-16 16:14:32: Installing GPU-enabled libraries: If available
    2023-11-16 16:14:33: Searching for python3-pip...All good.
    2023-11-16 16:14:34: Ensuring PIP compatibility... Done
    2023-11-16 16:14:34: Python packages will be specified by requirements.linux.txt
    2023-11-16 16:14:42:   - Installing Tensorflow Lite... (✔️ checked) Done
    2023-11-16 16:14:42:   - Installing NumPy, the fundamental package for array computing with Python...Already installed
    2023-11-16 16:14:45:   - Installing Pillow, a Python Image Library... (✔️ checked) Done
    2023-11-16 16:14:45: Installing Python packages for the CodeProject.AI Server SDK
    2023-11-16 16:14:46: Searching for python3-pip...All good.
    2023-11-16 16:14:48: Ensuring PIP compatibility... Done
    2023-11-16 16:14:48: Python packages will be specified by requirements.txt
    2023-11-16 16:14:48:   - Installing Pillow, a Python Image Library...Already installed
    2023-11-16 16:14:50:   - Installing Charset normalizer... (✔️ checked) Done
    2023-11-16 16:14:57:   - Installing aiohttp, the Async IO HTTP library... (✔️ checked) Done
    2023-11-16 16:14:59:   - Installing aiofiles, the Async IO Files library... (✔️ checked) Done
    2023-11-16 16:15:01:   - Installing py-cpuinfo to allow us to query CPU info... (✔️ checked) Done
    2023-11-16 16:15:05:   - Installing Requests, the HTTP library... (✔️ checked) Done
    2023-11-16 16:15:05:                 Setup complete                                        
    Installer exited with code 0

3

modified 16-Nov-23 11:50am.

Anthony m Jul202318-Nov-23 20:19

18-Nov-23 20:19

Is that the usb module? I had issues with mine. I think its the actual usb device. It just stops working out of nowhere.

I ended up getting the dual tpu card and getting a pci express adapter. You can get the one off amazon but that only allows one of the chips to work. There is a guy that makes custom ones that allow two of them to work. Been running rock solid for months on a few installs with hundreds of thousands of detections.

Steve Winn19-Nov-23 8:13

19-Nov-23 8:13

BTW; are you running CodeProject.AI server on Windows or Linux/Docker with success on the Coral tpu accelerator?

Mateusz Piotrowski2-Dec-23 6:57

Mateusz Piotrowski

2-Dec-23 6:57

shane.x wrote:
If you are referring to accuracy with Blue Iris or other ipcam software, it will be just as inaccurate. The current models for the Coral TPU are not appropriate for ipcam usage.

I'm getting false positives with ipcam-general with detection of chairs, tables, plants as person.
With MobileNet-SSD model (Coral) I'm getting satisfactory accuracy, at least not worse than with ipcam-general.

Anthony m Jul202319-Nov-23 5:24

19-Nov-23 5:24

Also are you running this in a hypervisor? Esxi? Proxmox?

Steve Winn19-Nov-23 7:01

19-Nov-23 7:01

No. Bare metal Ubuntu 22.04 I think, (I'm not near the machine).
Docker image latest 2.3.2-alpha.
This might be my problem.
I have and have tried both Coral USB and Coral M.2 devices.
With the Coral USB, I had it working with an earlier Docker image, I had to go through a bunch of changes to the Docker to get it to use the coral USB. After that, I did see what you see, where after some time, the Coral device would stop working, not reporting results to Blue Iris.
I currently have the Coral M.2 Accelerator installed in the Ubuntu machine.
I tried the latest CPAI Docker image with it, but it didn't seem to see or use the Coral M.2.
I like a hardware manufacturer to have a utility available to verify the functionality of their products. I spent a lot of time trying to follow the Coral directions to install PyCoral to verify the functionality of the USB device.
I did find and read through a long thread about how to recompile PyCoral to work with Python 3.10 or later. Funny, that one of the comments was "a multi-billion dollar company sells hardware that they forget about, and leave getting their software support to the user community".
Anyway, I am using CodeProject.AI server on the Blue Iris Windows machine, it's working well.
I also have Frigate NVR running on the Ubuntu machine using the Coral devices. So I guess they do work.
I would really appreciate a marked "stable" version of CodeProject.AI server.

Anthony m Jul202320-Nov-23 5:48

20-Nov-23 5:48

I have portainer installed on my ubuntu machine that makes managing docker much easier. It also makes passing through usb devices etc.. a bit easier.

I have tried running my usb m2 though docker on esxi and I had issues with it stopping out of nowhere. I then tried bare metal and got the same issue. I just found it to be a flaky device.
I found the link to the guy that makes the pci express cards for the dual edge tpu if your interested
Dual Edge TPU Adapter - PCIe x1 Low Profile | Makerfabs[^]

Steve Winn20-Nov-23 13:39

20-Nov-23 13:39

I have M.2 slots in both the Windows Blue Iris machine and the Ubuntu machine, so I don't need any adapter. I'm using Coral.AI server now in the Blue Iris machine using the Intel GPU. It's working well.
I think I'm done trying to get the Coral devices to work with CodeProject.AI server.
They both work well with Frigate NVR. So there's nothing wrong with the devices, it's probably a software issue.
I'm almost there with Frigate, just have a Reolink Doorbell camera that won't stay connected giving me headaches.

Anthony m Jul202321-Nov-23 6:34

21-Nov-23 6:34

Hmm just an update for you.
I was bored and decided to try out my usb coral device again. I have a passthrough usb pci card on my esxi server. I setup my ubuntu 23 image. I installed the driver on my ubuntu system. I had to compile the following driver in order to get it working with ubuntu kernel. After that I was able to install libedgetpu1-std [^]

Anyways after that success, I installed codeproject ai in docker and passed the coral to it. Started coral detection and it was working, BUTTTT I had the same issue you did. Processing times were 1000ms. I didnt have this issue before when I did the same thing. I wonder if its this newer version of codeproject that has something wrong.

Steve Winn21-Nov-23 7:04

21-Nov-23 7:04

I agree. I had the Coral USB device working with a very earlier version of CPAI on both the Windows Blue Iris machine and the Ubuntu. The inference times were actually slower than CPU. The problem was the USB would either quit or get very slow, so Blue Iris would not get any response, i.e. quit working. I need it to work.
I will now talk heresy.
For me, the most reliable is
Blue Iris ver 5.6.8.4 and Deepstack Docker latest on Ubuntu.

Chris Maunder27-Nov-23 10:24

27-Nov-23 10:24

Let me know when we're back on your most reliable list.

Out of interest, have you tried our Docker images?

cheers
Chris Maunder

Steve Winn27-Nov-23 11:47

27-Nov-23 11:47

Five days in, I'm not tempted to go back to Deepstack. So their's that.
CPAI ver 2.3.4 with Coral module ver 1.63 is working well. Hasn't failed yet with about 87000 detections. (I reboot the Blue Iris machine Sundays middle of the night.
That is with the Coral M.2 accelerator in the Blue Iris Windows machine.
I did have to monkey around to get the Coral 1.63 module to install. I changed Do not use download cache on and off several times, and I manually created a folder for Modules\ObjectDetectionCoral.
As far as Docker, on the Ubuntu machine, I never got Coral to work.
AFAIK, the latest Docker image for Linux/i386 is 2.3.2-alpha. My experience with software development tells me that is really early in the cycle. You mentioned in another post that 2.4.0 might be coming soon. I'm waiting for that Docker image to show up.
CodeProject.AI Server: AI the easy way.[^]
I'm running Frigate NVR on the Ubuntu machine, and it's running well. 8 to 10 msec inference with the Coral USB accelerator. It hasn't quit yet. About 4 - 5 days so far. I have to figure out notifications on that one.
So far so good.

modified 28-Nov-23 12:14pm.

Steve Winn2-Dec-23 8:58

2-Dec-23 8:58

Chris;
10 days so far, no issues at all.
~100,000 detections since Sunday AM.

Steve Winn21-Nov-23 9:31

21-Nov-23 9:31

OK, I had given up on trying to get the Coral M.2 working with CPAI Docker on the Ubuntu machine, I had changed the Blue Iris machine to use CPAI ver 2.3.4-beta with the YOLOv5.net detector.
On a whim, I tried to install the Coral ver. 1.6.3. I couldn't initially, but I finally got it installed, plugged in the Coral usb accelerator, and the CPAI saw it and started to use it.
So, now I have moved the Coral M.2 board from the Linux machine to the Windows machine. it's now working as expected.
I don't exactly know what I did. but it's working now. I'll keep an eye on it for the next few days.
I am curious about the Coral product description indicating that the usb model is for development and the boards like the M.2 are for production.

Server version:   2.3.4-Beta
System:           Windows
Operating System: Windows (Microsoft Windows 10.0.19045)
CPUs:             Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz (Intel)
                  1 CPU x 4 cores. 4 logical processors (x64)
GPU:              Intel(R) HD Graphics 630 (1,024 MiB) (Intel Corporation) 
                  Driver: 31.0.101.2111
System RAM:       16 GiB
Target:           Windows
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
.NET framework:   .NET 7.0.5
Video adapter info:
  Intel(R) HD Graphics 630:
    Driver Version     31.0.101.2111
    Video Processor    Intel(R) HD Graphics Family
System GPU info:
  GPU 3D Usage       1%
  GPU RAM Usage      0
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

Module 'ObjectDetection (Coral)' 1.6.3 (ID: ObjectDetectionCoral)
Module Path:   <root>\modules\ObjectDetectionCoral
AutoStart:     True
Queue:         objectdetection_queue
Platforms:     windows,linux,linux-arm64,macos,macos-arm64
GPU Libraries: installed if available
GPU Enabled:   enabled
Parallelism:   1
Accelerator:   
Half Precis.:  enable
Runtime:       python3.7
Runtime Loc:   Local
FilePath:      objectdetection_coral_adapter.py
Pre installed: False
Start pause:   1 sec
LogVerbosity:  
Valid:         True
Environment Variables
   MODELS_DIR = <root>\modules\ObjectDetectionCoral\assets
   MODEL_SIZE = Medium
Started:      21 Nov 2023 12:51:31 PM Central Standard Time
LastSeen:     21 Nov 2023 12:55:44 PM Central Standard Time
Status:       Started
Processed:    32
Provider:     TPU
CanUseGPU:    False
HardwareType: GPU



Installation Log
2023-11-21 11:39:09:              Installing CodeProject.AI Analysis Module                
2023-11-21 11:39:10: ========================================================================
2023-11-21 11:39:10:                    CodeProject.AI Installer                             
2023-11-21 11:39:10: ========================================================================
2023-11-21 11:39:10: General CodeProject.AI setup                                          
2023-11-21 11:39:10: Creating Directories...Done
2023-11-21 11:39:10: GPU support                                                           
2023-11-21 11:39:10: CUDA Present...No
2023-11-21 11:39:10: ROCm Present...False
2023-11-21 11:39:10: Installing module ObjectDetectionCoral 1.6.3                          
2023-11-21 11:39:10: Installing Python 3.7
2023-11-21 11:39:10: Python 3.7 is already installed
2023-11-21 11:39:16: Creating Virtual Environment (Local)...Done
2023-11-21 11:39:16: Confirming we have Python 3.7 in our virtual environment...present
2023-11-21 11:39:21: Downloading edge TPU runtime...Expanding...Done.
2023-11-21 11:39:21: Copying contents of edgetpu_runtime_20221024.zip to ....done
2023-11-21 11:39:21: Cleaning up...done
2023-11-21 11:39:21: Installing the edge TPU libraries...
2023-11-21 11:39:22: Installing UsbDk
2023-11-21 11:39:22: Installing Windows drivers
2023-11-21 11:39:22: Microsoft PnP Utility
2023-11-21 11:39:22: Adding driver package:  coral.inf
2023-11-21 11:39:22: Driver package added successfully. (Already exists in the system)
2023-11-21 11:39:22: Published Name:         oem40.inf
2023-11-21 11:39:22: Adding driver package:  Coral_USB_Accelerator.inf
2023-11-21 11:39:22: Driver package added successfully. (Already exists in the system)
2023-11-21 11:39:22: Published Name:         oem43.inf
2023-11-21 11:39:22: Driver package installed on device: USB\VID_18D1&PID_9302\5&220fb38b&0&10
2023-11-21 11:39:22: Adding driver package:  Coral_USB_Accelerator_(DFU).inf
2023-11-21 11:39:22: Driver package added successfully. (Already exists in the system)
2023-11-21 11:39:22: Published Name:         oem55.inf
2023-11-21 11:39:22: Driver package is up-to-date on device: USB\VID_1A6E&PID_089A\5&220fb38b&0&2
2023-11-21 11:39:22: Total driver packages:  3
2023-11-21 11:39:22: Added driver packages:  2
2023-11-21 11:39:22: Installing performance counters
2023-11-21 11:39:23: Info: Provider {aaa5bf9e-c44b-4177-af65-d3a06ba45fe7} defined in C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\edgetpu_runtime\third_party\coral_accelerator_windows\coral.man is already installed in system repository.
2023-11-21 11:39:23: Info: Successfully installed performance counters in C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\edgetpu_runtime\third_party\coral_accelerator_windows\coral.manCopying edgetpu and libusb to System32
2023-11-21 11:39:23:         1 file(s) copied.
2023-11-21 11:39:23:         1 file(s) copied.
2023-11-21 11:39:23: Install complete
2023-11-21 11:39:47: Downloading MobileNet models...Expanding...Done.
2023-11-21 11:39:47: Copying contents of objectdetect-coral-models.zip to assets...done
2023-11-21 11:39:47: Cleaning up...done
2023-11-21 11:39:47: Installing Python packages for ObjectDetectionCoral
2023-11-21 11:39:47: [0;Installing GPU-enabled libraries: If available
2023-11-21 11:39:49: Ensuring Python package manager (pip) is installed...Done
2023-11-21 11:39:59: Ensuring Python package manager (pip) is up to date...Done
2023-11-21 11:39:59: Python packages specified by requirements.txt
2023-11-21 11:40:02:   - Installing Pillow, a Python Image Library...(âœ"ï¸ checked) Done
2023-11-21 11:40:09:   - Installing Tensorflow Lite...(âœ"ï¸ checked) Done
2023-11-21 11:40:10:   - Installing NumPy, a package for scientific computing...Already installed
2023-11-21 11:40:10: Installing Python packages for the CodeProject.AI Server SDK
2023-11-21 11:40:11: Ensuring Python package manager (pip) is installed...Done
2023-11-21 11:40:13: Ensuring Python package manager (pip) is up to date...Done
2023-11-21 11:40:13: Python packages specified by requirements.txt
2023-11-21 11:40:14:   - Installing Pillow, a Python Image Library...Already installed
2023-11-21 11:40:16:   - Installing Charset normalizer...(âœ"ï¸ checked) Done
2023-11-21 11:40:21:   - Installing aiohttp, the Async IO HTTP library...(âœ"ï¸ checked) Done
2023-11-21 11:40:23:   - Installing aiofiles, the Async IO Files library...(âœ"ï¸ checked) Done
2023-11-21 11:40:25:   - Installing py-cpuinfo to allow us to query CPU info...(âœ"ï¸ checked) Done
2023-11-21 11:40:28:   - Installing Requests, the HTTP library...(âœ"ï¸ checked) Done
2023-11-21 11:40:28: Setup complete                                                        
Installer exited with code 0

Steve Winn24-Nov-23 11:04