"Come up with a device that allows a person to control media on a personal computer using communication between a microcontroller and the personal computer using asynchronous serial communication. "
Final Work
Team
Element
Tools
Duration
Gracia, Lea, Yilin
Serial Communication / LED String / Joystick / etc.
Arduino / p5.Js
2 weeks
Idea
Cooking can be dangerous and difficult for young kids, but it is also a great way for kids to develop a better hand control ability, which will crucially affect their ability in conducting personal care and other daily behaviors. Finger strength is one of the most important underlying abilities that impact a child’s ability to demonstrate hand control.
As mentioned in the third stage of Piaget’s theory of cognition, children from 7 to 11 years old start to solve the problem in a more logical fashion. However, they still face difficulties in dealing with abstract concepts. the concrete objects will enhance their cognitive development.
In our game, we will create a friendly and secure environment for children from 6-9 years old to explore the cooking process. In the process, children will have a chance to learn to operate tools and understand instructions.
Storyboard
We finally decided to use one particular dish as game content, which is chinese dumplings. We simplified the steps into the following steps and planned the use of sensors：
Program logic
Process - Coding
Link
p5.Js link: https://editor.p5js.org/Lea-0821/sketches/e9mBdQ0jU
Arduino link: https://create.arduino.cc/editor/lea_1998/a5aa9d6a-c2cb-464e-8bcf-a3e0b29b0df9/preview
- Serial Communication
For this project, we use Arduino to output sensor’s data to p5.Js, which controls video and audio changes. Therefore, we use analogWrite(), digitalWrite(), Serial.print() in the Arduino codes to convey the data as a string.
- Media Playing
Playing media is one of the most important parts of the project because it relies on the videos and sounds to instruct users to the next step. For all the videos, we create DOM elements in the beginning, using createImg() / .show() / .hide() to play GIF.
To control the media playing by Arduino sensors’ data, we create boolean variables to store the sensors’ state. Take the first step’s photoresistor as an example. If the values of the three photoresistors are all lower than 300, the corresponding boolean variables change, and the if condition judges whether this part boolean variable is true and other part boolean variables are false, then control the media playing.
In order to prevent misoperation of other sensors, variables for each part only change when the last step’s variable turns to true.
*Problems & Solution
One of the problems we met is once we use .loop() to play the music, the audio quality is horrible and full of noise.
Thanks to Jason’s help, we figured out the problem which is because the draw() function is executed over and over again, so the beginning of the music was played repeatedly. To solve this problem, we added the if condition to know whether the music was already played.
- Reset
We set up a “reset” button for users to replay the game or want to start from the beginning during intermediate operations. When the reset button is pressed, all boolean variables change to the false and play the first video.
Fabrication
Research & Material
We investigated children's kitchen toys on the market. Most of them are very colorful, and the shape varies a lot. And the materials are mainly metal and plastic. If we buy existing products for modification, we may face difficulties modifying the structure of plastics and metal products and making the overall fabrication inharmony. We wanted the fabrication to be aesthetic and simple as well. We chose white for the color of the materials. We marked the interactive parts with color and the low poly style to make each object more unified while reducing the difficulty of making the object's arc surface.
We used many different kinds of materials to make a stable and flexible cooking station as below.
Preparation
First, we made drawings of the kitchenware in Illustrator and assembled each of them. We tried to make dumplings with snap buttons and magnets. To make it easier for younger kids, we finally decided to use magnets and velcro with a white cloth to mimic the dumpling’s shape and texture.
During the making process, we adapted each piece’s sizes and mechanisms to make them fit together. Considering the steadiness and the flexibility of the materials, we used paperboard to make the bowls, pot, and knobs. The kitchen worktop should provide enough structural support, so we used the foam board to make it.
Process of making the bowls
The snap buttons are too small for kids to use, so we decided to switch to velcro.
The process of making the cooktop with acrylic.
Circuit Layout
We laid out the circuit on the back of the foam board.
The backside of the supporting board
Cooktop
Assemble Process
Animation
We made gif animations to bring the game experience to players. To make the transition natural, we drew the hands in different parts and used After Effect to perform the movements. For each production step, we drew the corresponding materials and used different techniques for different parts. The part that needs to change the hand operation uses a to bind bones and produce animations, and the other parts are presented in the form of stop motion.
Stop Motion
Animation
Interaction
-
We used the red color to indicate the touchpoints of users as a reference for their playing process.
-
We used sound as well as video to guide users.
-
Users can restart the game at any point using the “refresh” button.
-
The users have access to many different operation activities, including pushing buttons, folding the dumpling, rotating the knobs, etc.
Takeaway
-
The pace of the game is significant to make the interaction efficient for users.
-
In the next steps, We would like to refine the surface to remove the sharp corners and use acrylic to cover the sensors and wires. In addition, we can replace the computer with a projection, the sense of immersion will be stronger.