One day in the lab (FabLab Moscow), I saw my colleague busy sorting a full box of screws, nuts, rings and other hardware. Stopping next to him, I watched for a second and said: "It would be a perfect job for a machine." After a quick look on google I saw that different ingenious mechanical systems already existed but they couldn't solve our problem because in our box is a broad variety of parts. Doing something purely mechanical would be quite complicate. The other good reason to go to a more "robotic" system was because this would require all the technical fields I love: machine vision, robotic arms and electromechanical actuators!

This machine picks the screws and place them in different boxes. It consists in a robotic arm which handles an electromagnet, a translucent worktable above lights and a camera on the top. After spreading some screws and nuts on the worktable, the lights are turned on and a picture is taken. An algorithm detects the part shapes and returns their positions. Finally the arm with the electromagnet places the parts one by one in the desired boxes.

This project is still in development but now I obtain decent results that I want to share with you.

Step 1: Tools and Material

Tools

Laser cutter
Angle grinder
Hacksaw
Screwdriver
Clamps (the more the better)
Hot glue gun

Material

Plywood 3mm (1 m2)
Plywood 6mm (300 x 200 mm)
White translucent plastic 4mm (500 x 250 mm)
Computer (I am trying to move to raspberry pi)
Webcam (Logitech HD T20p, anyone should work)
Arduino with 4 PWM output / analogWrite (three servos and the electromagnet coil) (I use the ProTrinket 5V)
Prototyping board
Electronic wire (2m)
Switching transistor (any transistor that can drive a 2W coil) (I have S8050)
Diode (Schottky is better)
2 resistors (100Ω, 330Ω)
Power supply 5V, 2A
Servo micro (width 13 length 29 mm)
2 servos standard (width 20 length 38 mm)
Wood glue
4 metallic corner with screws (optional)
Wooden rod (30 x 20 x 2400)
Hot glue
Enamelled copper wire (0.2, 0.3 mm diameter, 5m)(old transformer?)
Soft iron (16 x 25 x4 mm)
3 light bulbs with socket
Connector strip (230V, 6 elements)
Electric wire with socket (230V) (2 m)
Bearing 625ZZ (inside diameter 5mm, outside diameter 16 mm, height 5mm)
Bearing 608ZZ (inside diameter 8mm, outside diameter 22 mm, height 7mm)
Bearing rb-lyn-317 (inside diameter 3mm, outside diameter 8 mm, height 4mm)
Timing belt GT2 (2mm pitch, 6mm wide, 650 mm)
Screw M5 x 35
Screw M8 x 40
8 screws M3 x 15
4 screws M4 x 60
6 wood screws 2 x 8 mm
Screw M3 x 10
Relay board module (directly controllable by controller)

Step 2: Make the Light Box

The light box has four main parts and some braces. Download these parts and glue them together except the translucent plastic. I started with the wooden half disc and the curved wall. You need to keep the wall tighten around the disc during the drying. I used clamps to secure the half disc and the curved wall basis. Then some tape maintains the wall around the half disc. Second, I glued a rim to withstand the translucent worktable. Finally the flat wall is added with wooden (inside) and metallic (outside) right edges.

Once the box completed, you just have to add the lightbulbs and connect the wire and socket with the connector strip. Cut the 230V wire where it is convenient for you and insert the relay module. I enclosed the relay (230V!) in a wooden box for security reasons.

partsDXF.zip
Download

Step 3: Make the Robot Arm

Download the parts and cut them. To secure the belt on the servomotor I used pieces of paperclips. I nailed the two part belts on the servomotor and added some glue to be sure that nothing moves.

For the linear vertical guidance, the plunger has to be sanded to avoid any blockage. It has to slide smoothly. Once assembled, the height can be adjusted by cutting the guidance at the desired length. However, keep it as long as possible to prevent over-center lock. The plunger is simply glued to the arm box.

The bearings are enclosed inside the pulleys. One pulley is made of two layers of plywood. These two layers are not necessarily touching each other so instead of gluing them together, glue them to their respective arm plate. The top and down arm plates are maintained by four M3 x 15 screw and nuts. The first axis (big one) is simply the M8 x 40 screw and the second (small one) the M5 x 35 screw. Use nuts as spacers and lockers for the arm parts.

Step 4: Make the Electromagnet

An electromagnet is simply a soft iron core with enameled wire coper around it. The soft iron core guides the magnetic field at the desired place. The current in the enameled wire coper creates this magnetic field (it is proportional). Also the more turns you make the more magnetic field you have. I designed a U-shaped iron to concentrate the magnetic field near the the caught screws and increase the prehension force.

Cut a U-shape into a piece of soft iron (height: 25mm, width: 15mm, iron cross-section: 5 x 4mm). It is very important to remove the sharp edges before winding the wire around the U-shaped iron. Be careful to keep the same winding direction (particularly when you jump to the other side, you need to change the direction of rotation from your point of view but you keep the same direction from the U-shaped iron point of view)(https://en.wikipedia.org/wiki/Right-hand_rule) Before branching the coil to the circuit, check the coil resistance with a multimeter and calculate the current with the Ohm's law (U=RI). I have more than 200 turns on my coil. I suggest you to wind until you have only 2 mm of space inside the U-shape.

A wooden holder has been made and the U-shaped iron has been secured with hot glue. Two slits allow to secure the wire on both ends. Finally two pins are nailed on the wooden holder. They make the junction between the enameled cooper wire and the electronic wire. To prevent any damage of the coil, I added a layer of hot glue all around the coil. On the last picture you can observe a wooden part who closes the U-shaped iron. Its function is to prevent any screws to get stuck inside the U-shaped iron.

The enameled wire coper has been taken from a broken transformer. If you do so, check that the wire is not broken or has no short circuits in the used portion. Remove the tape on the ferromagnetic core. With a cutter, detach one by one all the iron slices. Then remove the tape on the coil and finally unwind the enameled wire cooper. The secondary winding (the big diameter coil) has been used (transformer input 230V, output 5V-1A).

Step 5: Make the Circuit

On a prototyping board, I built the schematic above. A bipolar transistor (S8050) has been used for switching the electromagnet coil. Check that your transistor can handle the current calculated in the previous step. A MOSFET is probably more suitable in this situation but I took what I had on hand (and I wanted a low on-resistance). Adjust the two resistors to your transistor.

In the schematic above, VCC and GND icon are connected to the + and - of my power supply. The servomotors have three wires: Signal, VCC and GND. Only the signal wire is connected to the controller, the others are connected on the power supply. The controller is powered by the programmer cable.

Step 6: The Code

Last but not least: The code. You will find it here: https://github.com/adrienrosselet/optical-screw-s...

There is one program for the controller (arduino type) and another who runs on the computer (hopefully soon on raspberry). The code on the controller is responsible for trajectory planning and the one on the computer makes the image processing and sends the resulted position to the controller. The image processing is based on OpenCV.

The computer's program

The program takes an image with the webcam and the lights, detects the translucent worktable center and radius and corrects the eventual image rotation. From these values, the program calculates the robot position (We know the robot position according to the plate). The program uses the blob detector function of OpenCV to detect the screws and bolts. The different types of blobs are filtered with the available parameters (area, color, circularity, convexity, inertia) in order to select the desired component. The result of blob detector is the position (in pixels) of the selected blobs. Then a function transforms these pixel positions into millimeter positions in the arm coordinate system (orthogonal). Another function calculate the required position of each arm join in order to have the electromagnet at the desired position. The result consist in three angles which are finally send to the controller.

The controller's program

This program receives the join angles and moves the arm parts to reach these angles. It first calculates the top speed of each join in order to perform the move during the same time interval. Then it checks if these top speeds are ever reached, in this case the move will follow three phases: acceleration, constant speed and deceleration. If the top speed is not reached, the move will follow only two phases: acceleration and deceleration. The moments at which it has to pass from one phase to another are also calculated. Finally the move is executed: At regular intervals, the new actual angles are calculated and sent. If it is time to pass to the nest phase, the execution continues to the next phase.

Step 7: The Final Touches

The frame

A frame was added to hold the camera. I chose to make it with wood because it is cheap, easy to work with, easy to find, environmentally friendly, pleasant to shape and it stays in the style I began with. Make an image test with the camera to decide what height is needed. Also be sure to make it rigid and fixed because I noticed that the resulted position is very sensitive to any camera moves (at least before I added the worktable auto-detection function). The camera has to be situated at the worktable center and, in my case, 520 mm from the translucent white surface.

The boxes

As you can see in the picture, the movable storage boxes are on the flat part of the worktable. You can make as many boxes as needed but with my actual setup the space is quite limited. Nevertheless I have ideas to improve this point (cf. future improvements).

Future Improvements

For the moment the timing belt is closed with a wooden part but this solution limits the area that the arm can reach. I need to add more space between the big servo and the arm axis or make a smaller closing system.
The boxes are along the flat worktable edge, if I put it along the half circle edge, I would have much more space to add boxes and sort many component types.
Now the blob detection filter is enough for sorting the parts but as I want to increase the number of boxes, I will need the increase the selectivity. For this reason, I will try different recognition methods.
Now the servomotors I use do not have enough range to reach all the half disc worktable. I need to change the servos or change the reduction factor between the different pulleys.
Some issues occurs quite often so improving the reliability is the priority. For that I need to classify the type of issues and concentrate on the more likely. This is already what I did with the small piece of wood which closes the U-shaped iron and the auto detection center algorithm but now the issues become more complicate to solve.
Make a PCB for the controller and the electronic circuit.
Migrate the code to Raspberry pi to have a stand alone station