Google Coral Edge TPU

Archive - Mycroft Project Archive - Machine Learning
22

@Dominik actually there is practically no difference to the parts that process models in a TPU to NPU.

As currently the defacto processing unit is Tensorcore the choice of interface has much to do if the model or current training step is loaded but both are batch to unit ram processes.

The USB sticks in comparison due to cost have a smaller compatibillity subset and register types mainly due to cost.
Google cloud TPUs are NPU based not GPU if you can accept the approximation of terminology as its the tensor cores of a GPU without the GPU stuff it doesn’t need.

23

But TPU is an ASIC right? Another implementation of Digital Design instead of FPGA, right?

24

Yes, there is a lot of marketing speech involved here. To my understanding the Rockchip-NPU is FPGA and supports “reprogramming” while Coral Edge-TPU is “hardcoded” ASIC.

25

So, that’s why Google Coral Dev is faster…I got it!

26

Dunno where you get FPGA for the rk1808 but they are all much the same apart from the GPU part of a GPU :slight_smile:

http://opensource.rock-chips.com/images/4/43/Rockchip_RK1808_Datasheet_V1.2_20190527.pdf

They are all just accelerators with GPUs (nvidia) having the most compatibility due to nvidia’s head start with cuda.

27

I think the rockchip is somewhere between the 2.
You can hire a Cloud TPU v2 for $4.50 / TPU hour $1.35 / TPU hour on preempt (ie kicked if someone wants to pay $4.50 until spare)
Which is 11.5 petaflops and for training it doesn’t seem to make sense to purchase what you can get for hardware.
Most common voice models run happily at > relatime on a cpu but training can be an almighty endurance chore.
But you can hire server space and have it returned in minutes what will take hours with even some condiserable hardware.
But the little sticks are quite impressive seen as a RTX2080 is about 12Tflops and the vision models are pretty excellent but thankfully voice streams are much slower than visual ones.

28

Arm are also in the action with the Ethos-N78 neural processing unit 1-10 tops.

Guess they will be coming thick and fast and they will just be as standard as current tensor core GPU.

29

I try to install the following packages

pandas 
numpy 
scikit-learn 
scikit-image 
click tqdm

in Google Coral TPU, in order to work with tensorflow lite, and I get these errors:

File "numpy/core/setup.py", line 422, in generate_config_h
      moredefs, ignored = c6wfs --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Cython>=0.29.13 "numpy==1.13.3; python_version=='3.6' and platform_system!='AIX'" "numpy==1.14.5; python_version=='3.7' and platform_system!='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system!='AIX'" "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.7' and platform_system=='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system=='AIX'"" failed with error code 1 in None

Any ideas??

30

I think you have to delegate to libedgetpu.so.1 otherwise you are just running normal.
But as I said above “No as they are all very similar with different compatibility issues that you will have to research yourself.”

Don’t think many if any are using them here and likely to find better answers elsewhere.

Add the delegate when constructing the Interpreter .

For example, your TensorFlow Lite code will ordinarily have a line like this:

interpreter = tflite.Interpreter(model_path)

So change it to this:

interpreter = tflite.Interpreter(model_path, experimental_delegates=[tflite.load_delegate(‘libedgetpu.so.1’)])

The file passed to load_delegate() is the Edge TPU runtime library, and you installed it when you first set up your device. The filename you must use here depends on your host operating system, as follows:

31

You wrote at the end “as follows:” but I do not see the rest of the sentence. Do I miss something?

32

You wrote at the end “as follows:” but I do not see the rest of the sentence. Do I miss something?

33

Yes it would seem the docs.

34

I have ran a CNN model on Google Coral TPU Dev Board, but I am not sure it does accelerate the model… Is there a command to use in order to see specifically the ASIC load? And more important the time duration that tensorflow lite accelerates the CNN model?

35

I dunno but at a guess if

interpreter = tflite.Interpreter(model_path, experimental_delegates=[tflite.load_delegate(‘libedgetpu.so.1’)])

Is included in your code then it should use the tpu

if

interpreter = tflite.Interpreter(model_path)

Its just going to use cpu.

So try testing your code and see if it makes a difference.

You need to find a Google Coral TPU group or forum as really no-one is using them here.

36

I was wondering if there is open source to blur faces, license blades and company labeling on vehicles on-the-fly in video streams?

This would make the stream GDPR compliant as long as not recording private property.
Ideal would be if an additional feature would allow to black out private property if a camera is fixed mounted at one position.

If there is such software, how performant would the Coral EDGE TPU be? 1 or more streams? What resolution and frame rate?

37

https://docs.frigate.video/hardware/ is prob the one to check as build around a coral edge.

A single Coral can handle many cameras and will be sufficient for the majority of users. You can calculate the maximum performance of your Coral based on the inference speed reported by Frigate. With an inference speed of 10, your Coral will top out at 1000/10=100 , or 100 frames per second. If your detection fps is regularly getting close to that, you should first consider tuning motion masks. If those are already properly configured, a second Coral may be needed.

A pi though comes at the bottom of the list as the base board.

A Coral m.2 is half the price of with maybe the dual e key one being best value even though have never tried.
GDPR is more about using, informing and retention of data and stoping the misuse of that data and sharing.
If you comply correctly then thoses faces, licence blades and company labeling are not a problem, but yeah there are ML segmentation models to capture those so likely the same could be used to obscure.

If you do some googling and find the perfect int8 model for edge ml then performance is extremely impressive.