DmitryM8

Glad that it was helpful!

There is a package missing, which is not supposed to happen normally. Can you try re-downloading the packages? In Step 1 :The command will take a few minutes to download all of the core ROS packages into the src folder.If wstool init fails or is interrupted, you can resume the download by running: wstool update -j4 -t src

Fixed that :) there was a problem with requirements.txt file

Hi! Please check this example in my GIthub https://github.com/AIWintermuteAI/aXeleRate/blob/master/example_scripts/k210/detector/racoon_detector.py specifically line 14

I checked it just now, it does work, but Arduino IDE version needs to be at least 1.8.12 :) it didn't work with 1.8.9

Thanks for the feedback! I need to correct that

It has 5 DOF and 4-axis (gripper is counted as a DOF, but not as axis). It was so long time ago I cannot remember the exact reason, but from what I recall, Transform6D is not possible for 5 DOF arm and Translation3D basically puts the end effector in desired location (XYZ), which was my goal.

It does seem like faulty sensor. Do you have any other OV2640 to test?

I think the reason might be that you are using MobileNat1_0 backend (converted weights are around 3 MB with MobileNat1_0 backend). Try using minimal firmware with IDE support - only that one will have enough memory. Alternatively consider switching to MobileNat7_5 or lower alpha

Thank you for noticing this! I updated the article recently to use my new Keras-based framework, aXeleRate to train the models.

Mobilenet v2 has more complicated structure, including skip connections between blocks. At the time of original article nncase converter didn't support converting models with skip connections :)

Thanks! Well, I think all the issues are resolved now, and raccoon_detector.json is where it is supposed to be :)https://github.com/AIWintermuteAI/aXeleRate/blob/master/configs/raccoon_detector.json

Hi! Please, try with https://github.com/AIWintermuteAI/aXeleRateThis is new framework I wrote, it is tailored for use with Google Colab. I updated the article to reflect the latest changes!

Hi! Please, try with https://github.com/AIWintermuteAI/aXeleRateThis is new framework I wrote, it is tailored for use with Google Colab. I need to update the content of the article ot reflect that

Model should be the same size, since the number of connections is the same, regardless of the images you're trying to classify (that is of course if you don't change model architecture).This is weird... Have you made any changes to mobilent.py? What board and firmware are you using?

Hello! I'm not familiar with this particular issue. There might have been some changes to ROS Melodic or Buster since I wrote this article. If you don't need the latest version of Buster you can try using RPI 4 image I made https://drive.google.com/file/d/1lcPBYgdFG7EuaIV23B2-7tL10EMizMD9/view?usp=sharing

Hello! I'm not familiar with this particular issue. There might have been some changes to ROS Melodic or Buster since I wrote this article. If you don't need the latest version of Buster you can try using RPI 4 image I made https://drive.google.com/file/d/1lcPBYgdFG7EuaIV23B2-7tL10EMizMD9/view?usp=sharing

Hello! I'm not familiar with this particular issue. There might have been some changes to ROS Melodic or Buster since I wrote this article. If you don't need the latest version of Buster you can try using RPI 4 image I made https://drive.google.com/file/d/1lcPBYgdFG7EuaIV23B2-7tL10EMizMD9/view?usp=sharing

Yes, correct!

Hi! It seems you're using Windows, I haven't tested the setup on Windows system, only Linux(Ubuntu 16.04 and 18.04). Having said that the error indicates that tensorflow has problem converting model to frozen graph(.pb file). Can you confirm that you're using tensorflow 1.14 and keras 2.2.4?

Using latest version of my code the training process for more than one class is exactly the same) I trained the model for up to 10 classes

Hello! I'm not familiar with this particular issue. There might have been some changes to ROS Melodic or Buster since I wrote this article. If you don't need the latest version of Buster you can try using RPI 4 image I made https://drive.google.com/file/d/1lcPBYgdFG7EuaIV23B2-7tL10EMizMD9/view?usp=sharing

Thanks! Updated the article just now

I did exactly that in my latest code update :) output_layer = model.layers[2].name+'/BiasAdd'model.layers[-2].name would works the same with this architecture, but overall it is safer approach indeed!

Glad that you resolved this issue!

Great question! While it is possible to train a model for 320*240 resolution input it would require significant changes to the training script. I am rewriting some parts of it and will publish the changes to Github, but I didn't get to different input sizes yet. As a sort of workaround this issue you're having with setwindowing, you can use image.crop or image.resize functions instead. For my company's project we use image.resize function, which returns a resized image.

Yes. It's because the name of the last layer is different and depends on number of classes. When you're running a training script it shows you the names of the layers of the model - just after the network summary. With two objects, yours is probably called "detection_layer_35/BiasAdd" or something like that

It seems like a problem with tensorflow setup... Does training work normally for other applications? I don't think you are using GPU, aren't you? I have faced similar issue, when running new version Tensorflow with old CPU(circa 2013). The issue is that new tensorflow pre-compiled binaries are compiled with the instruction support for instructions that don't exist in older CPUs. Which causes the segmentation fault. It might be or not be your issue :) but I suggest you look into that direction and run more tests with different training scripts.

I will push a fix for that bug tomorrow, once I go to office

I'm familiar with this error. It is actually a side effect of image augumentation ("jitter" parameter in config). What happens is that the image augumentation pipeline translates image too much, which causes the above error (axis 0 in this case is the grid 0-6, if the object is translated too much it causes the script to assign the grid that doesn't exist). Long story short, I fixed it, but haven't pushed the update to github yet

I may be slow responding to comments here, if you have an urgent issue, please post a comment on YouTube or contact me directly on LinkedIn :)Also welcome to create issues on GitHub and pull requests, if you make useful changes to the code or upgrade the code to newer version of frameworks used!

I may be slow responding to comments here, if you have an urgent issue, please post a comment on YouTube or contact me directly on LinkedIn :)Also welcome to create issues on GitHub and pull requests, if you make useful changes to the code or upgrade the code to newer version of frameworks used!

Hello! Yes, I have tried QR code algorithm with Sipeed Maxipy and can confirm it working. Concerning the speed, I haven't done this comparison, bit it should at least be on par with OpenMV H7 camera(ARM Cortex M7 with 480 MHz clock speed vs. K210 Dual core 400MHz clock (can be overclocked up to to 800MHz)).

I may be slow responding to comments here, if you have an urgent issue, please post a comment on YouTube or contact me directly on LinkedIn :)Also welcome to create issues on GitHub and pull requests, if you make useful changes to the code or upgrade the code to newer version of frameworks used!

Yes, sure. For that one check the updated version of the script in my github repo, it's called mbnet_keras_args.py.

https://github.com/keras-team/keras/issues/3657You can consult this thread, this is known Keras issue. It might be possible your dataset is too small. How many images do you have? Are they in the right folder?

You're right, they have changed syntax in 0.2 version, which is still beta now. I don't think Sipeed micropython firmware supports kmodel v4(generated by nncase 0.2) yet, at least not the last time I checked. So, I'm still using NNCase Converter v0.1.0 RC5.

Sorry for the late reply, instructables doesn't notify me about comments :(Order of predictions is folders are processed in alphabetical order. Santa comes before Uno because the folder with Uno called "arduino_uno". Here's a relevant answer on stack overflow about the order of classes, https://stackoverflow.com/questions/42868982/how-do-i-check-the-order-in-which-keras-flow-from-directory-method-processes-fo

Excellent! Happy that you got it working!

Hello, Ludmila!The problem here is that I described the process of installing ROS Melodic on Debian Buster system... Not sure what issues you might encounter when installing on Ubuntu BB(Bionic Beaver). Is there any particular reason you want to use Ubuntu?

Excellent! Glad it worked for you!

No problem. Yes, it should be working with m5stickV as well, although keep in mind this is a detection, not classification model training tutorial. The datasets for detection and training are different.

It does sound like "entertaiment learning for probably years", hahaWhat is the end goal of your project?

Glad that you got it working with example dataset!Now for custom dataset, I would say bare minimum would be 200-300 images and that might not give stable results. I was training model for detecting 10 classes of objects (traffic signs) with 3000+ images.You can check the performance of the model on computer by using evaluate.py script. You'll need to edit it to match the location of your validation dataset though.Good luck!

Sorry for the late reply, instructables doesn't notify me about new messages in the comments... Yes, it is the same firmware for all sipeed maix board - bit, go, dock and maixduino. So, openMV works uniformly across these boards

I don't understand the questions... What do you mean by "the board version with OpenMV cam"? OpenMV camera is a piece of hardware and not related to Sipeed boards.

Yes, tensorflow 1.14.0, keras 2.3.0. If you do pip install -r requirements.txt it will install these versions. I added requirements.txt to my github repo yesterday

I made a Raspberry Pi 4 SD card image with ROS Melodic, RPLIDAR and Kinect working. It requires minimum of 16 Gb SD card and you can use Etcher to flash it onto SD card. Since image is too big, I can only share it with the torrent. If you download it, please keep seeding :)magnet:?xt=urn:btih:448BD429744AAC9BF2FBA2FF8332EC2BEEF010B2&dn=rp4.img&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce

Have a look at my new article about object detection for this one. Short answer is --- yes. Long answer is --- there are two components to object detection network: feature extractor and detection layer. In my article for feature extractor we take Mobilenet v1 pre-trained on Imagenet, but we have to train detection layer from scratch. Although it is possible to also use pretrained detection layer weights.

I'll look into it, might be related to some recent changes in Melodic/Buster, since this problem wasn't reported before, but now it seems at least three people have similiar problem.

https://www.instructables.com/id/Object-Detection-...Here you go! No worries, I planned to do object detection tutorial anyway, so I just added a chunk of code to send data over UART. I tested raccoon_detector.py script with my Maixduino board, it does send the detection data over UART. But if you're using another board, you'll need to remember to change the UART pins!

Ah, Kinect. Yes, I got it to work, took me about 4 hours of finding all the missing dependencies on Bionic. Unfortunately I'm traveling at the moment and don't have Pi 4 with me... One thought though. If you do have the drivers installed and can see the topics. Can you check if there's anything being published on these topics with rostopic echo command? If there is data published, then something might be wrong with rviz set-up... If there is none, then might be drivers.A person on Youtube asked me if I can upload the whole image with ROS, LIDAR and Kinect stacks installed. I will do once I get back to Shenzhen.

Hi!Sorry, the right mobilenet probably got lost in one of the later commits to github repo. I added it back again, it's inside of transfer_learning_sipeed/mobilenet_v1_transfer_learning/ folderhttps://github.com/AIWintermuteAI/transfer_learnin...

Perfect! So, you got it to work in VirtualBox?

When are you getting this error?

Glad that it worked for you!

They are not the worst among Chinese hardware manufacturers))Alright, so, presumably you're running one of the demos for object detection, for example the one herehttps://bbs.sipeed.com/t/topic/683and want to get the coordinates of bounding boxes through uart.Check the i.rect() object, it should be some sort of the list with xmax, xmin, ymax, ymin coordinates of the detected object. Now for UART, check the example 1 from herehttps://maixpy.sipeed.com/zh/libs/machine/uart.htm...I was traveling last two weeks, but I plan to write another tutorial about training an object detection model and will include data transmission over UART/I2C code there as well, as per your request.

Nice! So, the compilation is successful, but you can't start roscore. Let's check a couple of things:1)/opt/ros/melodic/setup.bash --- is setup.bash present in /opt/ros/melodic/?2)Has the following line was added to .bashrc : source /opt/ros/melodic/setup.bash ?3) You'll need to open a new terminal or source bash.rc in order for changes to take effect.

"You can list all files using that function(sic) (!in ros_catkin_ws folder!):"I added "(!in ros_catkin_ws folder!) Because I'm assuming people reading this tutorial have at least basic knowledge of Linux file system :)Yes, those br and a tags... Already deleted them.

Right. That command should be executed from ros_catkin_ws. I already added the comment to that part of the tutorial

Let's focus on step 3 for now. What's the output of sudo ./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release --install-space /opt/ros/melodic -j2Is compilation successful?

Yes, you're right! This is one of these small things that I tend to easily forget about. I'll edit the tutorial accordingly.Yes, 2-3 images is enough. The ncc just uses these images to check the input size of the network. So, they should match the input size

https://drive.google.com/open?id=1JgTLIERX9AFoiGF_... Here you go!

Hello!The script automatically resizes images to pre-set sizeimg = image.load_img(img_path + file, target_size=(128, 128))You can change the image size, but obviously you will also need to change the input layer dimensions to match your image size. It can be other image types, like png. I will share the datasets tomorrow!

Excellent job! I should be using colab notebooks more myself, they're so convinient.

TensorFlow counting API is not just a simple model, but in fact a framework, that includes Mobilenet SSD and some OpenCV functions. I have successfully run Mobilenet SSD on maixpy, so that part is not a problem. Now that just detects objects, for counting you need to also keep track of them. You can try using OpenMV, which is has functions related to object tracking and is available on Maix Bit.Short answer: This is possible, but require quite a bit of coding :)For Mobilenet SSD that can be run on Maixpy have a look at https://github.com/penny4860/Yolo-digit-detector. Change the feature extractor to MobileNet and train as usual. Then you will be able to convert the model to .kmodel format.

Hi! Very likely the process hasn't changed.Where are you running the ./src/catkin/bin/catkin_make_isolated command from? It should be run from the directory you are building ROS in. It is also possible that rosinstall_generator hasn't finished downloading all the files. Can you try restarting the download with wstool update -j 4 -t src or start again?

Hi! Very likely the process hasn't changed.Where are you running the ./src/catkin/bin/catkin_make_isolated command from? It should be run from the directory you are building ROS in. It is also possible that rosinstall_generator hasn't finished downloading all the files. Can you try restarting the download with wstool update -j 4 -t src or start again?

Ok, can you change the number of neurons in the last layer to 2, since you only have two classes?mbnet_keras.py, line 30

Another source of issues with PAD is wrong padding size. I solved this problem by including modified version of Mobilenet together with the training script. Can you clone my repository again and see if the problem solved?

https://github.com/keras-team/keras/issues/3657You can consult this thread, this is known Keras issue. It might be possible your dataset is too small. How many images do you have? Are they in the right folder?

https://github.com/keras-team/keras/issues/3657You can consult this thread, this is known Keras issue. It might be possible your dataset is too small. How many images do you have? Are they in the right folder?

Hi!For the first problem, I added back the test images to the repository, you can download them and try again, should be working now.For you second problem, it seems you installed your miniconda with sudo( I see it is in /root/) ? It is not recommended to install miniconda in /root, can you try reinstalling it in your user home folder?

https://github.com/kendryte/nncase/releasesI just checked the nncse(the converter) latest release(RC4) and it supports padding. You can manually download it from github and then unpack in ncc folder inside of the Maix_toolbox!Maix team will update their get_nncase.sh script shortly as well.

Hi!There's no need to generate .pb file, the following command takes care of direct conversion of .h5 model to .tflitetflite_convert --output_file=model.tflite \ --keras_model_file=my_model.h5Concerning PAD problem - previous version of .kmodel converter didn't support padding layers, but the developers added support about two weeks ago. I'll ask them if they updated Maix toolbox. Meanwhile, if your project is urgent, e-mail me at dmitrywat@gmail.com and I'll send you the beta with padding support.

Thank you! Glad that you liked it.I added the video about the project and also the full-length demo.

Hi! Sorry for the late reply, I got quite busy with other projects recently. The reason I'm asking about DOF and "Approximate IK solutions" option is that I discovered that for all IK solvers that ROS has available, none of them is particularly suitable for robotic manipulators with less than 6 DOF. You can read the discussion I started on MoveIt github herehttps://github.com/ros-planning/moveit/issues/1235and another onehttps://github.com/ros-planning/moveit/issues/1064I improved IK success rate by adding one more degree of freedom. But it still fails in about 20% of cases, which I solve iterating fast through the grasps and finding the one that works

Great article, helped me a lot!

Hi. Let's see1)Are you using the meshes I provided or made yourown custom meshes for the arm?2)What happens if you just drag interactive marker in RVIZ? Can you move it freely? WWithout checking "Approximate IK solutions"3) How many DOF does your arm have?