Print PDF
Favorite
Email
The title “MultiMovEDIA” does not mean much – if anything – at first sight. However, looking at more carefully, we can see it is a composition of two words: “Multimedia”, referring to the information content and the channels it uses (text, image, sound), and “Move”, meaning movement. This project incorporates movement and dynamics into the world of multimedia, in a unique way, where moving one’s arm or just a finger makes everything possible; be it for computer games, CAD, music composition, image editing and other areas of cybernetics. Physical Computing methods are used along with sensors and a microcontroller to connect the outside world to various devices such as a computer, or in the case of music composition, to unique DIY devices made of various passive and active digital components. One such device is Kinect, having its own SDK for developing applications in various programming languages, allowing for 3D image processing and movement sensing for multimedia purposes, robotics and other technologies and application areas.
How is this project unique?
• It brings together many cheap technologies that are penetrating all areas of multimedia.
• It contains peripheral devices that are useful not only in multimedia, but also in medicine, mechatronics, manufacturing and other fields.
• It embraces many new ideas and applications that will be described later.
What are the modules of the MultiMovEDIA system?
• MoveMusic – this application allows the conductor to become a performer. An accelerometer and a button mounted to a lightweight glove transform arm movements into musical symphony. The device’s operation is controlled by bending a finger: when the finger is bent, the arm movements generate sounds, when the finger is straight, the music stops. The accelerometer measures the values along the three axes (X, Y, Z) that correspond to different sound values and types. Every movement generates a new sound. This way orchestras can play in canon, or dancers can take the role of musicians. The output sound can be modified as desired, using MIDI technologies, such as MAX MSP, or mixers or other methods − the possibilities are limitless.
• MoveCAD – this module is the most interesting part of the project, in my opinion. Using the glove mentioned earlier and a pair of 3D glasses we can work with a cube in 3D. We can grab the cube with the glove, look at it from various angles and distances, and transform its shape, for example into a sphere. This method makes it possible to view images in 3D. This could be a great help in education, where students in a geometry class could see the shapes in 3D, and even create various geometric shapes. In medicine, the doctor could see the brain’s structure in 3D during an MRI exam. But it mostly be useful in multimedia arts, where the 3D possibilities of painting, sculpture and other art forms could be applied to interactive computing, most notably in the computer games industry.
• MoveGame – this module allows the further development of animation techniques and games. Hand movements replace the mouse, finger bending replaces typing on the keyboard. The most important thing is that no accessories, program packs or drivers are needed for operation. It can be used with any game, and it can also control the computer. Communication is done directly through the language tools ports, so that the computer detects the device as if it was a real mouse or keyboard. We only need to sacrifice one USB port, and the device is ready to use.
The MultiMovEDIA system can replace all unnecessary control devices that are expensive, take up space and are boring to use. There are many other areas where anthroposophic control can be introduced. Such solutions can make life easier for people, for example controlling weight-bearing robotic arms so that people could move heavy objects with a few simple hand movements.
How is this project unique?
• It brings together many cheap technologies that are penetrating all areas of multimedia.
• It contains peripheral devices that are useful not only in multimedia, but also in medicine, mechatronics, manufacturing and other fields.
• It embraces many new ideas and applications that will be described later.
What are the modules of the MultiMovEDIA system?
• MoveMusic – this application allows the conductor to become a performer. An accelerometer and a button mounted to a lightweight glove transform arm movements into musical symphony. The device’s operation is controlled by bending a finger: when the finger is bent, the arm movements generate sounds, when the finger is straight, the music stops. The accelerometer measures the values along the three axes (X, Y, Z) that correspond to different sound values and types. Every movement generates a new sound. This way orchestras can play in canon, or dancers can take the role of musicians. The output sound can be modified as desired, using MIDI technologies, such as MAX MSP, or mixers or other methods − the possibilities are limitless.
• MoveCAD – this module is the most interesting part of the project, in my opinion. Using the glove mentioned earlier and a pair of 3D glasses we can work with a cube in 3D. We can grab the cube with the glove, look at it from various angles and distances, and transform its shape, for example into a sphere. This method makes it possible to view images in 3D. This could be a great help in education, where students in a geometry class could see the shapes in 3D, and even create various geometric shapes. In medicine, the doctor could see the brain’s structure in 3D during an MRI exam. But it mostly be useful in multimedia arts, where the 3D possibilities of painting, sculpture and other art forms could be applied to interactive computing, most notably in the computer games industry.
• MoveGame – this module allows the further development of animation techniques and games. Hand movements replace the mouse, finger bending replaces typing on the keyboard. The most important thing is that no accessories, program packs or drivers are needed for operation. It can be used with any game, and it can also control the computer. Communication is done directly through the language tools ports, so that the computer detects the device as if it was a real mouse or keyboard. We only need to sacrifice one USB port, and the device is ready to use.
The MultiMovEDIA system can replace all unnecessary control devices that are expensive, take up space and are boring to use. There are many other areas where anthroposophic control can be introduced. Such solutions can make life easier for people, for example controlling weight-bearing robotic arms so that people could move heavy objects with a few simple hand movements.
Remove these ads by
Signing UpStep 1Operation
1. MoveMusic
This module allows for the creation of various music using the ADXL335 three-axis accelerometer and “the finger bend sensing button”. The circuit’s control unit is the ATMega32U4 microcontroller with Teensy bootloader. The operation comprises three steps: 1. The microcontroller reads the data from the accelerometer and transforms these into sounds. 2. If the finger at the pushbutton is bent, the music will sound. 3. The sounds are displayed on the LCD display.
2. MoveCAD
This module consists of two parts:
1. 3Duino – to change the shape of a cube in 3D space
2. MovElement – to view a cube in 3D space from various angles and distances.
3Duino
Using our hands we can control an object in 3D – currently a cube, but with a little modification it could be anything. The 3D effect is achieved by a pair of Anaglyph blue-red glasses and the Anaglyph image converter. The image is moved by a Memsic 2125 accelerometer. If you have a bigger budget, I can recommend the gyroscope module, too. The program’s operation comprises three steps. 1. Arduino platform reads the accelerometer and processes the data. 2. After data processing, the microcontroller is connected to the PC via an RS232 port interface, and the program receives the data and writes everything in into the “cube.txt” file. 3. A second PC program filters and displays the image and moves the cube using the information it reads from the text file. The programs are opened using 3Duino Start.exe. Please note that before opening the programs, the microcontroller program should already be running, and the RS-232 adapter should already be connected to the microcontroller and the computer’s serial port. After burning the bootloader, the Processing Source Code filetype should be programmed into the microcontroller. The RX of the RS232 adapter circuit is connected to the 4th digital pin (this is pin #6 on the microcontroller). The transmitter of the adapter is connected to the 5th digital pin (this is pin #11 on the microcontroller). The RS232 connector of the adapter circuit should be connected to COM1.
This module was created for demonstration purposes, using the Arduino platform (ATmega168/ATmega8); the other modules are based on the Teensy platform, which is also an Arduino interface circuit, but faster, more efficient and cheaper.
MovElement
By moving our hands we can view a cube from different angles and distances. The 3D effect can be achieved by various technologies. I chose a pair of Anaglyph glasses, because it provides a cheap and effective solution. The movement sensing is performed by an ADXL335 three-axis accelerometer. The control chip is an ATmega32U4 with Teensy bootloader. The operation comprises three steps. 1. The chip reads and processes the data from the accelerometer. 2. It sends the control commands to the PC via USB, using HID protocol. 3. A PC program written in Processing analyses the data, then moves and positions the cube.
3. MoveGame
By moving our hand and fingers we can control the computer (text input, cursor control) and play computer games. We can also transfer some video games to the computer and control them with this application, if we are using emulators. For example, Playstation Emulator and other programs we can find on the Emulator Zone website. The movement sensing is performed by an ADXL335 three-axis accelerometer, and the flex sensor monitors the fingers for key command input. There are flex sensors assigned to each of the forward, backward, right, left and selection (Enter) commands. The control chip is an ATmega32U4 with Teensy bootloader. The operation comprises three steps. 1. The chip reads and processes the data from the accelerometer and the flex sensors. 2. It sends the control commands to the PC via USB, using HID protocol. 3. The computer receives the commands and based on them performs the key operations or moves the cursor to the selected points.
This module allows for the creation of various music using the ADXL335 three-axis accelerometer and “the finger bend sensing button”. The circuit’s control unit is the ATMega32U4 microcontroller with Teensy bootloader. The operation comprises three steps: 1. The microcontroller reads the data from the accelerometer and transforms these into sounds. 2. If the finger at the pushbutton is bent, the music will sound. 3. The sounds are displayed on the LCD display.
2. MoveCAD
This module consists of two parts:
1. 3Duino – to change the shape of a cube in 3D space
2. MovElement – to view a cube in 3D space from various angles and distances.
3Duino
Using our hands we can control an object in 3D – currently a cube, but with a little modification it could be anything. The 3D effect is achieved by a pair of Anaglyph blue-red glasses and the Anaglyph image converter. The image is moved by a Memsic 2125 accelerometer. If you have a bigger budget, I can recommend the gyroscope module, too. The program’s operation comprises three steps. 1. Arduino platform reads the accelerometer and processes the data. 2. After data processing, the microcontroller is connected to the PC via an RS232 port interface, and the program receives the data and writes everything in into the “cube.txt” file. 3. A second PC program filters and displays the image and moves the cube using the information it reads from the text file. The programs are opened using 3Duino Start.exe. Please note that before opening the programs, the microcontroller program should already be running, and the RS-232 adapter should already be connected to the microcontroller and the computer’s serial port. After burning the bootloader, the Processing Source Code filetype should be programmed into the microcontroller. The RX of the RS232 adapter circuit is connected to the 4th digital pin (this is pin #6 on the microcontroller). The transmitter of the adapter is connected to the 5th digital pin (this is pin #11 on the microcontroller). The RS232 connector of the adapter circuit should be connected to COM1.
This module was created for demonstration purposes, using the Arduino platform (ATmega168/ATmega8); the other modules are based on the Teensy platform, which is also an Arduino interface circuit, but faster, more efficient and cheaper.
MovElement
By moving our hands we can view a cube from different angles and distances. The 3D effect can be achieved by various technologies. I chose a pair of Anaglyph glasses, because it provides a cheap and effective solution. The movement sensing is performed by an ADXL335 three-axis accelerometer. The control chip is an ATmega32U4 with Teensy bootloader. The operation comprises three steps. 1. The chip reads and processes the data from the accelerometer. 2. It sends the control commands to the PC via USB, using HID protocol. 3. A PC program written in Processing analyses the data, then moves and positions the cube.
3. MoveGame
By moving our hand and fingers we can control the computer (text input, cursor control) and play computer games. We can also transfer some video games to the computer and control them with this application, if we are using emulators. For example, Playstation Emulator and other programs we can find on the Emulator Zone website. The movement sensing is performed by an ADXL335 three-axis accelerometer, and the flex sensor monitors the fingers for key command input. There are flex sensors assigned to each of the forward, backward, right, left and selection (Enter) commands. The control chip is an ATmega32U4 with Teensy bootloader. The operation comprises three steps. 1. The chip reads and processes the data from the accelerometer and the flex sensors. 2. It sends the control commands to the PC via USB, using HID protocol. 3. The computer receives the commands and based on them performs the key operations or moves the cursor to the selected points.
| « Previous Step | Download PDFView All Steps | Next Step » |
I didn't know you have an Instructables account. :)
Thank you so much for your comment and for your vote!
Suzanna is a professional translator and never missed a deadline, always keen to work and deliver quality translations.
She helps me sometimes when I don't know something in English.
Thank you very much!
Best regards,
Bocz Hunor-Chris!
Bocz Hunor-Chris
Good luck!
Bocz Hunor-Chris