A Scoping Review on Tangible and Spatial Awareness Interaction Technique in Mobile Augmented Reality-Authoring Tool in Kitchen.
Augmented Reality (AR) receives a lot of attention globally. There are a lot of efforts in research and ongoing development in this area as it is the newest technology. Augmented Reality can provide attractive and interactive experience in kitchen design. Interior design has gained tons of attention among the users in order to make a convenient and comfortable kitchen environment. Based on the previous finding, AR offers many advantages in digital architectural design and construction fields . It is a new design approach for interior design, where AR environment is able to display virtual furniture and it can be modified in real-time where the user can directly see the virtual outcome of the design.
Some of the kitchen design companies are using physical board to show rough design by arranging the physical blocks into the empty kitchen layout as shown in Figure 1. However, with the limitation of human imagination, it is challenging to visualize how the cabinet will look like or align in the real kitchen space . Mobile Augmented Reality (MAR) application allows the virtual furniture to overlay into real scene. Thus, the user can see the outcome of the kitchen design whether the design outcome could fit with the actual kitchen environment. The interaction with the augmented reality environment plays an important role in this authoring type application. It could affect the performance and user preference such as to discourage the user if the interaction is too complicated. Thus, this paper is focusing on the interaction technique design in kitchen design. There are two types of potential interaction techniques, which are suitable for the kitchen design: tangible and spatial awareness interaction technique. This work reviews the existing studies on the tangible and spatial awareness technique in order to identify and design a new interaction technique for the kitchen design in MAR environment.
This paper consists of six parts: the first part is the introduction of the study. The second part discusses the documents selection process. The third part is the reviews of the studies related to tangible and spatial awareness interaction technique to explore their possibilities in kitchen design MAR authoring tool. Next, the kitchen design process is defined based on the interview section with the Malaysian kitchen design company. The fifth part of the paper discusses the proposed tangible and spatial awareness interaction techniques in mobile augmented reality setting. The conclusion and the future works are in the final part of the paper.
2. Documents Selection
This study focused on two types of interaction techniques, which are tangible interaction technique and spatial awareness interaction technique. Tangible interaction technique is an approach of designing the AR interfaces that focus on interactions using related physical object to interact with the virtual object [4, 5]. Tangible interaction technique registers the virtual object to a physical object, and a user interacts with the virtual objects by manipulating the corresponding physical objects . In the designing session, it allows the user to interact with the virtual furniture in order to move, rotate, and copy them as in the real environment using physical object. With the latest development, RGB and depth camera embedded in the smart phone have enabled the spatial awareness interaction technique in a mobile setting . It allows the user to measure the depth of the object by using depth sensor. Spatial awareness interaction technique involves motion tracking, depth sensing, and area learning where the environment properties could be recognized by the AR system. Both types of interaction techniques require further exploration on their effectiveness in helping design process between the kitchen designer and customer.
2.1. Documents (Tangible Interaction Technique). Based on Figure 2, the initial result of query search is 1768 from SCOPUS database. Then, the result was filtered by the document type, field, language, and year. The filtration ended up with 1528 documents. After title and abstract filtering, 1528 documents have been reduced to 145. Then, full text reading process excluded 133 documents, resulting in 12 documents as the final document to be analysed.
2.2. Documents (Spatial Interaction Technique). Based on Figure 3, the initial query shows 3565 raw results in SCOPUS. The field was then specified to computer science leaving 771 results. 717 articles and conference papers were then chosen to be included in this filtering. The documents from year 2000 until 2017 were chosen, reducing the documents into 650 documents. After scanning the title and abstract, only 116 documents were applicable. At the end, 8 documents were chosen after finishing the full text reading.
3. Result and Statistical Information of Documents
In summary, 20 articles have been selected from the SCOPUS database and reviewed in which 60% are in tangible interaction technique and 40% are related to spatial awareness interaction technique. All the papers were published in proceedings and journals from 2002 until 2017. Study on spatial awareness interaction technique is a new topic and the reviewed papers are from 2013 to 2017. 50% of the reviewed papers for tangible interaction technique are proceeding papers and another 50% are from journal articles. Meanwhile, all the reviewed papers for spatial awareness interaction technique are proceeding papers. The selection process will be discussed in Section 2. There are 6 scenarios that used tangible interaction technique in their study as shown in Table 1.
Based on the review there are four types of AR, which are projection based, see-through based, PC based, and mobile based which used tangible and spatial awareness interaction techniques. The statistics of the types of AR used in the reviewed papers are shown in Table 2.
3.1. Studies Related to Tangible Interaction Technique. Table 3 shows a review on existing tangible interaction technique. There are 12 researches reviewed; most of the researches (50%) used card with marker as interaction medium to interact with virtual object. 18% used cube to manipulate the virtual object. Another 32% used specific physical object such as LP records, paddle, pen, and cup as medium of interaction. For the physical setup statistic, 58% of the researchers used HMD as a device to display the virtual object. 17% of the studies used projector and 17% used PC to display the virtual environment. Only one study used mobile phone as a display device.
A collaborative AR system with tangible interaction that is designed to support a scenario where a group of experts meet to discuss the design of a product has been presented in . The tangible interaction includes a physical round plate for the object rotation manipulation, seeing the internal view of an object using clip plane method controlled by a normal marker to cut through the virtual object, and other manipulation functions. The result showed the interaction technique runs stably and is easy to use. However, based on the user feedback, several aspects of the system could be improved such as reducing the time consumed for tracking and improve quality of image and HMD. Another four applications have been developed using the design principles of tangible interaction technique . All the tangible interaction approaches are using different image patterns on physical card for representing different virtual object such as Shared Space for gaming purpose, ARgroove focus on interactive music, rapid prototyping for aircraft instrument panels in Tiles application, and VOMAR for furniture relocation application. Their finding concludes that the tangible interaction allows seamless spatial interaction with virtual objects in the physical workspace .
Physical cube is one of the popular mediums of interaction in tangible interaction. For example, a game that has been developed by  uses two cubes as a medium of interaction with the virtual object and supports two players to play in turn. This game uses Singapore's map as the game board. Players were able to view the 3D map of Singapore by using HMD. Cube number one is used as dice and cube number two is used as a controller. When the player rolls cube one, the metaphor will move based on the number. Cube two consists of three functions which are to view photos and videos, as a shopping cart and as a navigation tool. Player can rotate cube two to change the displayed photo and the video. When the player stacks two cubes together, it acts as shopping cart function. It allows the player to add virtual object into an empty virtual cart. Cube two can be turned to left or right and tilted upwards and downwards as a navigation tool to tour around in the virtual world. Another research that used cube as medium interaction is Magic Story Cube . Magic Story Cube is a foldable cube for storytelling with 3D graphics, 3D sound, music, and speech. Users can explore the story content by unfolding the cube and view the content by using HMD. A digilog book that supports novel visual, audio, and haptic authoring functions has been introduced in . This system uses pen-type manipulation prop as medium interaction by placing the pen-type manipulation prop on the marker in digilog book to copy the virtual object. Meanwhile, pressing and holding the button on the pen-type manipulation prop will edit the shape of the trajectory object.
A system that uses card marker as a medium interaction has been introduced in . This system allows the user to create virtual AR content by selection menu technique. Then, the user can manipulate the virtual object by using the card marker. For example, when the user puts two markers side by side, a new virtual content will be shown. Another research on tangible augmented reality is called AR-SEE . Passive Solar Energy Education has presented AR-SEE as an augmented reality application for mobile phones. It is a combination of mobile phone-based AR with the physical model of a house compartment such as roof styles, windows, and other building materials. Those design changes will then affect the internal temperature inside the house and the temperature displayed through particle visualization. Then, the users can realize how passive solar energy affects the temperature inside the house and the energy usage efficiency. The users can interact with the AR-SEE via phone by looking through it and pointing at the big green marker. Then, three buttons are allocated on the phone with one of them for the material selection for the roof of the house, another button for the material for the base of the house, and the last button for X-ray vision function which makes the house partially transparent to allow the user to see the visualization inside the house. Other than that, it uses virtual human with different actions to represent temperature condition. For example, the virtual human is sitting if the temperature is less than 81 degrees Fahrenheit and falling forward if temperature rises above 95 degrees Fahrenheit. Their finding shows that the virtual human visualization enables significantly lower temperature estimation error than the particle visualization.
The previous researches which are almost similar to this study are VOMAR  and TAR for interior design . A study on tangible AR for interior design has been conducted in . This study introduced an augmented reality system for designing or educating in interior design projects by overlaying virtual furniture in a physical environment on a regular PC system. The system uses marker card as a medium to interact with the virtual furniture. The colour and material of the virtual furniture can be adjusted by adding or hiding the card marker. Another similar study was a cup-shaped handheld compact that has been labelled with marker as the medium interaction between user and virtual furniture . This system includes six main functions, which are picking, allocating, moving, rotating, deleting, and changing mode. The user can pick the virtual object by covering the virtual object with the cup. To allocate the virtual object, user can place the cup that contains the virtual object on the table. User can move the virtual object by sliding the cup. Furthermore, rotating the cup will rotate the virtual object and shaking the cup will delete the object. In interior design scenario, basic manipulation ability of the interaction technique is picking, allocating, moving, rotating, and deleting the virtual furniture. All the virtual furniture manipulations in the previous studies were limited where user cannot change the feature of the virtual furniture.
3.2. Studies Related to Spatial Awareness Interaction Technique. Depth sensing and motion tracking are useful features in spatial awareness interaction. It allows user to move in 3D space (x, y, and z position) and to track its orientation in yaw, pitch, and roll . This 6 degrees of freedom of sensing capabilities allows the phone to respond to the physical movements of the user. Table 4 shows a review on existing spatial awareness interaction technique. There are 8 researches that have been reviewed.
Projection-based augmented reality is a normal setup for the spatial augmented reality. General technique of dynamically augmenting physical models was introduced back then. iSarProjection is using a handheld projector attached with an RGBD camera to project the texture on a physical model . The physical model is mapped with corresponding textures, and it develops a real-time dynamic calibration, registration, tracking, and rendering. However, this study only focuses on the visualization in the spatial augmented reality environment.
A hybrid space combining spatial augmented reality and virtual reality has been proposed and has obtained a positive feedback from interviewees . The hybrid spatial augmented reality (SAR)/Virtual Reality (VR) hybrid space is a projection-based AR that enables multiusers manipulation with an augmented object, and immersive experiences. Virtual objects can be manipulated using the controller's grab button. To start drawing, the user can directly point at the target location and press the controller's trigger. Garden by Sing  is a study which used the game scenario to explore the mixed reality (MR) experience that combines both VR with AR. It allows users to transform their physical environment into a virtual garden they can play in. The exploration of mixed reality with motion tracking and depth sensing is going very well here. An immersive experience allows players to move through their surroundings untethered while exploring and interacting with the virtual world by using Durovis Dive headset where users can see the world in VR via tablet mounted on it. It consists of a simple hand and leg interaction where user draws a virtual object via voxel that is created by their hand and leg. Another projection-based AR is the Tabletop SAR  where it uses physical object such as cube and cone to interact with the 3D data. It is a combination of spatial augmented reality with the tangible interaction technique. The physical object in their study comes with a push button for selection and scroll wheel for 2D zooming. Users manipulate the physical object and push the button to lock the zooming position in the 3D data visualization area and then increase or decrease the zoom level using the scroll wheel.
Besides the projection-based AR, researchers have also studied wearable augmented reality and spatial awareness mobile device such as Google Project Tango. Nassani et al.  invented a wearable system that allows people to place and interact with 3D virtual tags placed around them and it is named Tag It!. A head-worn wearable computer (Google Glass) and a chest-worn depth sensor (Tango device) were used as wearable technology. The users can create virtual tags with text content generated by voice input and place it on where they are looking at by wearing the system. Tango is utilized to extend Google Glass' sensing ability by sharing the location and pose of the user as well as the tagged target position in the real world. The Google Glass then overlays virtual tags based on the spatial information received from the Tango, and the background of the glass display is set to black to act as an optical see-through display. Via the Google Glass touchpad, the user can initiate the wireless connection by using a three-finger touch gesture on it. Besides the Tag It! invention that benefits the user, the social spatial mash-up (SSM)  for information sharing in public space also improves the interaction among human with overlaying the virtual information surrounding them. The proposed SSM is based on RGB-D SLAM to create a 3D feature map and allows users to locate information and contents in 3D space. Information placing is through tapping button in the Tango. The virtual information is presented in an image (a small logo) form, which is located on the face of the 3D object after social spatial mash up. The virtual information is well registered even with the movement of the Tango device.
For the interior design area such as in the study of Tang et al. , Kinect was used to detect the spatial information of the physical environment where the furniture reseller can create virtual 3D furniture. The virtual furniture is rendered in the real scene of the home space via AR technology. The study performed an evaluation on the system usability, flexibility, and accuracy where overall result is positive, and the system is helpful, usable, efficient, and effective for home furnishing and furniture purchase. It is related to this study; however, it does not have the mobile ability where user can move and see the furniture in different perspective. Blanks (white cabinet) is a metaphor of the cabinets that are placed in an environment of the kitchen with actual shape and size . It supports the early design process of kitchen. It will ease the discussion session between the designer and the customer. The designer can move the cabinet blanks to the desired location. Then, the designer and the customer can decide on the cupboards, the drawers, and other components. The system automatically arranges the components to fit the available space. The component can be manipulated by using the manipulation tools, which are two identical cards to resize the component and to use SAR Swatches for choosing the colour and loading and saving the kitchen designs. It also allows user to compare different designs simultaneously by quickly flicking through several different designs in a more natural way than using a menu. Final design phase allows the designer and the customer to set the material of the cabinet by tapping the cabinet surface with the Stylus.
In conclusion, the previous studies have shown the possibility of the spatial awareness interaction between user and the systems such as the SAR/VR Hybrid Space, Tabletop SAR, Tag It!, Social Spatial Mashup, and Garden.
4. Current Kitchen Design Process
We have conducted interview session with the kitchen design company in Malaysia. Figure 4 presents the current kitchen design procedures, which start with the face-to-face meeting between the customer and the designer. At the beginning of the process, the customer will need the measurement of the kitchen space and might come with the initial plan of their desired kitchen design to discuss with the designer. Some of them come with an example of their wanted design from the magazine or catalogue. However, the designer will have difficulty to follow the design from the magazine due to lack of measurement information in the magazine. Next, the designer will ask if there is any appliance such as fridge or microwave to be placed in the kitchen. Then, the designer will start to sketch the design after the customer has made the selection of cabinet material, colour, and accessories. Common furniture pieces that the customer requires are sink, stove, kitchen ventilation, and cabinet. The type of furniture, measurement, material, and accessories will have an impact on the price of the kitchen cabinet. After the customer makes the payment, the designer will produce a full 3D design in the design software. The designer and the customer will continue the discussion to finalize the design and then the designer will go to the customer's home to confirm the actual measurement. After all the procedures have been done, the designer will proceed with the production of kitchen cabinet.
The weaknesses of the current procedure include difficulty of understanding the kitchen design imagination of both customer and designer. The designer cannot understand the customer needs through verbal communication. This problem causes time-consuming delay in the designing process. During the interview section, we have explained the MAR approach to solve the problem to the kitchen designer. The designers feel positive to use the MAR in smartphone platform instead of tablet, as smartphone is a common device that they use daily. The designers suggested that this application should allow the user to design their kitchen at their home and obtain the actual measurement beforehand so that the designer does not have to go to the customer's home and confirm the measurement. Then, the customer can email or print the output design to show to the designer for further discussion. Based on the designer's opinion, this application is useful and will ease the designers' work in the designing process and overcome different imagination between customer and designer. Besides, it will also help to reduce the designing time.
5. Proposed Interaction Techniques
Based on the findings of the scoping review, this study has proposed two types of interaction techniques: tangible interaction technique and spatial awareness interaction technique for kitchen design. Tangible interaction technique is used mainly in the situation where the customer and designer are together in a discussion session. Normally, the discussion session is conducted in the kitchen design company. Tangible interaction technique requires physical board as interaction space and cube as medium interaction. The tangible interaction technique will use cube as interaction medium to represent kitchen furniture and kitchen features as shown in Table 5. For example, the cubes will represent cabinets, sinks, stoves, and kitchen ventilation labelled with related markers to be arranged on the physical board as shown in Figure 5(a). For sink, stove, and kitchen ventilation, the colour features of this furniture are included in the physical cube where the user can choose the colour of the virtual object by rotating the cube that is labelled with its colour. For the cabinet, the cube is labelled with the surface of cabinet. For example, the user can choose the surface of the cabinet by manipulating the cabinet cubes that are labelled with different type of cabinet's surface. Figure 5(b) illustrates the virtual cabinet shown after the user places the cabinet cubes on the physical board. For the cabinet accessories and the cabinet colour features, the proposed technique uses the cards as interaction medium. The user can choose the colour and accessories of the cabinet by placing the colour card and the accessories card on the designated physical board.
Spatial awareness interaction techniques do not require physical board because users can design their kitchens directly in their home kitchen using mobile phones. Spatial awareness techniques enable the users to measure available space to add new furniture. Besides, the users can view 1: 1 scale of the virtual furniture to see the suitability of the virtual furniture with the available space. The user interface of this technique contains four menu buttons representing cabinets, stoves, kitchen ventilation, and sinks as shown in Figure 6(a). When user clicks on the cabinet menu button, it will display a list of cabinets for the user to choose. A popup box will be displayed for the user to select the surface type, colour, and accessories of the cabinet as shown in Figure 6(b). If the user clicks on the sink menu button, list of the sink will be displayed. After the user selected the sink they want, a popup will be displayed for the user to select the colour of the sink. A similar process will be used for stove and kitchen ventilation menu button. Spatial awareness interaction techniques can detect walls and floors. Therefore, after the selection has been made the virtual furniture will snap on and not penetrate the physical wall.
There are few limitations in spatial awareness interaction technique; namely, this technique cannot be used to fully furnish the kitchen. This is because the spatial awareness interaction technique can detect the existing furniture and act as the boundary or wall of the kitchen space. Thus, the virtual furniture that customer choose will appear in front of the physical furniture. Other than that, customers lack the knowledge in kitchen design; thus, some of the existing water piping or the electric socket area might make some kitchen furniture unable to be located at the location they desired. However, the customer does not know those limitations, and this makes the kitchen design not achievable. This problem can be tackled by having further discussion session about the design between the customer and designer. The discussion session will be easier because they already have the design based on the customer's idea. This will minimize the gap of the different imagination issue.
In kitchen design, the traditional approach has a few limitations such as mistake in measurement and different imagination between customer and designer. The user has different final kitchen output, which is usually different from the sketch design from the designer. Hence, this traditional method is inconvenient and not efficient. As a solution, the virtual furniture can be overlaid into real scene by using MAR application. The user can see the overlaid virtual kitchen outcome that could fit with the actual kitchen environment.
This paper has presented a scoping review on the tangible and spatial awareness interaction technique and proposed the tangible and spatial awareness interaction technique for the kitchen design authoring tool. The proposed interaction techniques will be developed and refined by conducting a heuristic evaluation with the AR expert. After the refining has been made, both techniques will be improved and evaluated with the kitchen designer and the customer through usability testing.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This study is supported the Universiti Kebangsaan Malaysia under research grant scheme of GGPM-2015-023 and GUP2017-050.
 V. T. Phan, P Seung, and Y. Choo, "A Study on Tangible AR for interior design," International Journal of CAD/CAM, vol. 10, no. 1, pp. 21-28, 2010.
 A. Adiyodi, M. Agrahara, P. Gamnani, and K. Pampattiwar, "Interior design using augmented reality," International Journal of Innovative Research in Science, Engineering and Technology, pp. 6104-6110, 2016.
 kubiq, "What Is Ezikit,", http://kubiq.com.my/index.php/whatis-ezikit/, 2016.
 R. Jafri, A. M. Aljuhani, and S. A. Ali, "A Tangible Interface-based Application for Teaching Tactual Shape Perception and Spatial Awareness Sub-Concepts to Visually Impaired Children," Procedia Manufacturing, vol. 3, pp. 5562-5569, 2015.
 N. Tuli, Interaction Techniques: helping user manipulate AR virtual content Interaction Techniques: helping user manipulate AR virtual content, 2016.
 O. Bimber and R. Raskar, Spatial Augmented Reality Merging Real and Virtual Worlds, vol. 6, 2005.
 H. T. Regenbrecht, M. T. Wagner, and G. Baratoff, "Magic-meeting: A collaborative tangible augmented reality system," Virtual Reality, vol. 6, no. 3, pp. 151-166, 2002.
 M. Billinghurst and I. Poupyrev, "Tangible augmented reality," in Proceedings of the ACM SIGGRAPH ASIA 2008 Courses, pp. 1-10, 2008.
 Z. Zhou, A. D. Cheok, T. Chan, and Y. Li, "Jumanji Singapore: an interactive 3D board game turning hollywood fantasy into reality categories and subject descriptors," Advances in Computer Entertainment Technology, pp. 362-363, 2004.
 Z. Zhou, A. D. Cheok, and J. Pan, "3D story cube: An interactive tangible user interface for storytelling with 3D graphics and audio," Personal and Ubiquitous Computing, vol. 8, no. 5, pp. 374-376, 2004.
 T. Ha, W. Woo, Y. Lee et al., "ARtalet: Tangible user interface based immersive augmented reality authoring tool for digilog book," in Proceedings of the 8th International Symposium on Ubiquitous Virtual Reality, ISUVR2010, pp. 40-43, kor, July 2010.
 Y. Yang, J. Shim, S. Chae, and T. Han, "Mobile augmented reality authoring tool," in Proceedings of the 2016 IEEE Tenth International Conference on Semantic Computing (ICSC), pp. 358-361, 2016.
 V. Ferrer, A. Perdomo, H. R. Ali, C. Fies, and J. Quarles, "Virtual humans for temperature visualization in a tangible augmented reality educational game," in Proceedings of the 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual and Augmented Reality, KELVAR 2017, usa.
 M. Billinghurst, H. Kato, and S. Myojin, "Advanced interaction techniques for augmented reality applications," in Proceedings of the 3rd Int. Conf. Virtual Mix. Reality, VMR 2009. Held as Part HCI Int, vol. 5622, pp. 13-22.
 M. Tan, W. Xu, and D. Weng, "ISarProjection: A kinect-fusion based handheld dynamic spatial augmented reality system," in Proceedings of the 13th International Conference on Computer-Aided Design and Computer Graphics, CAD/Graphics 2013, pp. 425-426, chn, November 2013.
 J. S. Roo and M. Hachet, "Towards a hybrid space combining Spatial Augmented Reality and virtual reality," in Proceedings of the 2017 IEEE Symp. 3D User Interfaces, 3DUI2017, pp. 195-198, 2017.
 K. H. Sing, "Garden: a mixed reality experience combining virtual reality and 3D reconstruction," in Proceeding of the 2016 CHI Conf. Ext. Abstr. Hum. Factors Comput. Syst.--CHI EA '16, pp. 180-183, 2016.
 B. H. Thomas, M. Marner, R. T. Smith et al., "Spatial augmented reality--A tool for 3D data visualization," in Proceedings of the IEEE VIS International Workshop on 3DVis, 3DVis 2014, pp. 4550, France.
 A. Nassani, H. Bai, G. Lee, and M. Billinghurst, "Tag it!: AR annotation using wearable sensors," in Proceedings of the SIGGRAPH Asia Mobile Graphics and Interactive Applications, pp. 1-4, 2015.
 C. Shin, Y. Kim, J. Hong, S. Hong, and H. Kang, "Social Spatial Mashup for Place and Object," in Proceedings of the the 2016 Symposium, pp. 185-185, Tokyo, Japan, October 2016.
 J. K. Tang, W. Lau, K. Chan, and K. To, "AR interior designer: Automatic furniture arrangement using spatial and functional relationships," in Proceedings of the 2014 International Conference on Virtual Systems & Multimedia (VSMM), pp. 345-352, Hong Kong, China, December 2014.
 M. R. Marner and B. H. Thomas, "Poster: Spatial Augmented Reality user interface techniques for room size modeling tasks," in Proceedings of the 8th IEEE International Symposium on 3D User Interfaces, 3DUI2013, pp. 155-156, usa, March 2013.
 J. Lee, " 4-1:," SID Symposium Digest of Technical Papers, vol. 48, no. 1, pp. 17-18, 2017.
Siti Soleha Muhammad Nizam (iD), Meng Chun Lam (iD), Haslina Arshad (iD), and Nur Asylah Suwadi (iD)
Mixed Reality and Pervasive Computing Lab, Centre of Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia
Correspondence should be addressed to Meng Chun Lam; firstname.lastname@example.org
Received 30 November 2017; Revised 16 March 2018; Accepted 12 April 2018; Published 21 May 2018
Academic Editor: Tuomas Makila
Caption: Figure 1: Physical kitchen miniature modelling kit .
Caption: Figure 2: Document selection for tangible interaction technique.
Caption: Figure 3: Document selection for spatial interaction technique.
Caption: Figure 4: Current design procedures.
Caption: Figure 5:(a) User interface of tangible interaction technique. (b) Virtual cabinet is displayed based on the selection.
Caption: Figure 6: (a) User interface of spatial awareness interaction technique. (b) Display of popup box.
Table 1: Scenarios. Interaction Scenario Percentage Product design 8% Rapid Prototype 8% Tangible Game 18% Music 8% Interior design 25% Education 33% Architecture 37% Spatial Information Sharing 37% awareness Game 13% Medical 13% Table 2: Type of AR. Type of interaction Type of AR Tangible Spatial awareness Projection 17% 50% See-through 58% 12.5% PC 17% 12.5% Mobile 8% 25% Table 3: Scoping review on existing tangible interaction technique. Interaction Name of technique: invention medium Interaction technique Magic Tangible: MagicMeeting is a Meeting  cake platter, collaborative AR system clipping designed to support plane, card a scenario where a group of experts meet to discuss the design of a product. (i) Rotate cake platter: rotate view (ii) Transfer the virtual model: put PDA near cake platter: (iii) Operate virtual model: use clipping plane (iv) Change light source: control light prop device (v) Change virtual model colour: place the marker near the virtual model Shared Tangible: Shared space is an AR Space  card collaborative card game. (i) Place the card side by side. If cards contain correct matches, an animation is triggered ARGroove  Tangible: Manipulate the LP records LP records to composite music. (i) Up: pitch (ii) Down: amplitude (iii) Rotate: filter cut-off frequency (iv) Tilt: delay mix Tiles  Tangible: Tiles is an AR authoring card interface that supported rapid prototyping for aircraft instrument panels. (i) Data tiles: show virtual object from menu book (ii) Operator tiles: delete, copy, help VOMAR  Tangible: An AR multimodal interface Paddle for designing house via combination of speech and paddle gestures interaction. (i) Place paddle on the marker of menu book to pick the object. (ii) Tilt the paddle to place the object. (iii) Push to move the object. (iv) Shake to delete the object from paddle. (v) Hit to delete object from virtual room. Jumanji Tangible: Jumanji Singapore is a game Singapore Two Cubes that allows users to have a  magical tour of Singapore by simple manipulations of two physical cubes. (i) Dice: Rolls the dice, the metaphor will move based on the number. (ii) Control cube: (1) Rotate to view different photo and video. (2) Stack the cubes will function as shopping cart. (3) Navigation tool: Turn left, turn right, tilt upwards and tilt downwards. Magic Tangible: A foldable cube for Story Foldable storytelling with 3D Cube  Cube graphics, 3D sound, music and speech. (i) Unfold the cube will function as turning pages of traditional book. ARTalet  Tangible: A digilog book that supports Pen-type novel visual, audio and haptic Haptic Device authoring functions. (i) Place the pen-type on the marker in digilog to copy the virtual object. (ii) Pressing and holding the button on the pen to edit shape of the trajectory object. MAR Tangible: Mobile AR authoring tool authoring Card system that enables user to tool  create AR content. (i) Place the card side by side to augment the new virtual object to be augmented. AR-SEE: Tangible: AR-SEE system that allows Augmented Marker user to learn about science Reality Card of architectural design by for Passive interacting with a tangible Solar physical 3D model house. Energy (i) Marker: Manipulate the Education type of roof and window  for the house. TAR for Tangible: An augmented reality interior 3 Cards system for designing/ design  Marker educating/presenting interior design projects. (i) Add or hide marker to adjust colour and material of the furniture MagicCup Tangible: The MagicCup is a cup-shaped AR  Cup-shaped handheld compact AR input Handheld device with a tracker that Compact detects six dimensional positions and pose information. (i) Pick: Cover a virtual object with the cup. (ii) Allocate: Place the cup with the virtual object. (iii) Move: Slide the cup. (iv) Rotate: Rotate the cup. (v) Delete: Shake the cup. (vi) Change Mode: Shake the empty cup Name of invention Scenario Type of AR Physical setup Magic Product Projection (i) Meeting table Meeting  design based (ii) HMD (iii) Projection screen (iv) Cake platter (v) PDA (vi) Clipping plane (vii) Three card colour (red, green, yellow) (viii) Light prop device Shared Game See-through i) HMD + camera Space  based ii) Card ARGroove  Entertainment Projection (i) Projector based (ii) LP records (ii) Projection screen Tiles  Rapid See-through (i)Whiteboard prototyping based (ii) Book (iii) Magnetic tiles (iv) HMD VOMAR  Interior See-through (i) Menu Book Design Based (ii) Cardboard Paddle (iii) Large Piece of Paper (iv) HMD Jumanji Game See-through (i) HMD Singapore Based (ii) Game Board  (iii) Two Cube as Dice and Controller Magic Education See-through (i) HMD + Camera Story Based (ii) Foldable Cube Cube  ARTalet  Education See-through (i) Input Images Based from a Camera (ii) CV Based Tracking Book (iii) Menu Prop (iv) Pen-type Manipulation Prop (v) HMD MAR Education Mobile (i) Phone authoring Based (ii) Card Marker tool  AR-SEE: Education Mobile (i) Phone Augmented Based (ii) Markers Reality for Passive Solar Energy Education  TAR for Interior PC Based (i) PC +Webcam interior Design (ii) Logitech design  Quickcam Vision Pro (iii) Markers MagicCup Interior See-through (i) Menu Book AR  Design Based (ii) Cup-shaped Handheld Compact (iii) HMD Table 4: Scoping review on existing spatial awareness interaction technique. Name of Interaction Interaction invention technique: technique medium iSar Spatial (i) Involves only Projection  projection visualization technique SAR/VR Spatial (i)Wand controller: HYBRID points at a certain SPACE  location and presses the controller's trigger to start the drawing Garden  Spatial (i) Durovis Dive headset: user can see the world in VR via tablet that has been mounted on it (ii) Hands and Legs: create voxels to fill in the wireframe and draw objects Tabletop Spatial and Tangible SAR  tangible: (i) Rectangular object: rectangular specification of zoom object, scroll region in the virtual wheel volume with desired location and orientation (ii) Push button: locks zoom position in place (iii) Scroll wheel: increases or decreases the zoomed volume. Spatial (i) Handheld input device (push button): selection (ii) Handheld input device (scroll wheel): 2D zooming (iii) 6DOF pointing device: drills down on 3D projected onto the physical objects (iv) 4K monitor: displays detailed views of 3D visualization Tag It!  Spatial (i) Tango: extends Google Glass' sensing ability by sharing the location and pose of the user as well as tag the virtual information in real world (ii) Glass: overlays virtual tags based on the spatial information received from the Tango (iii) Glass touchpad: user can initiate the wireless connection by using a three-finger touch gesture on it Social spatial Spatial (i) Tango: registers the mashup virtual information on (SSM)  the face of the 3D object after social spatial mashup via tapping the virtual button on the device AR interior Spatial A depth camera is used to designer  measure the size of available space in terms of supporting surfaces for new furniture (i) User uses mouse to interact and manipulate virtual furniture PimpMy Spatial A system that supports the Kitchen  early design process of kitchens and helps to ease designer and client in meeting session (i) Blank (white cabinet): moveable cabinet placed in an environment of the shape and size of the target kitchen (ii) Identical card: resizing projected components. Initially, the components automatically fit the available space (iii) SAR Swatches: choose colour, loading different design and save design (iv) Stylus: taping the cabinet surface to change materials Name of invention Scenario Type of AR Physical setup iSar Medical Projection (i) Handheld projector Projection  based (ii) RGBD camera (iii) Physical model SAR/VR Architecture Projection (i)Wand controller HYBRID based (ii) Projector SPACE  Garden  Game Mobile (i) Tango tablet based (ii) Durovis Dive headset Tabletop Information Projection based SAR  sharing based (i) Handheld projecting device (ii) Physical object (iii) 6DoF Tracking (iv) Projector Tag It!  Information See-through (i) Glass sharing based (ii) Glass touchpad (iii) Tango (iv) Target Social spatial Information Mobile (i) Google Project mashup sharing based Tango (SSM)  AR interior Architecture PC based (i) PC + Mouse designer  (ii) Microsoft Kinect PimpMy Architecture Projection (i) Projector Kitchen  based (ii) Blanks (white cabinet) (iii) Card (iv) SAR Swatch (v) Stylus Table 5: Tangible interaction technique method. Interaction Virtual Selection of the medium furniture virtual furniture's representation feature Cabinet Cabinet surface Cube Stove Colour Sink Colour Kitchen Colour ventilation Colour of cabinet Card Accessories of cabinet
|Printer friendly Cite/link Email Feedback|
|Author:||Nizam, Siti Soleha Muhammad; Lam, Meng Chun; Arshad, Haslina; Suwadi, Nur Asylah|
|Publication:||Advances in Multimedia|
|Date:||Jan 1, 2018|
|Previous Article:||Beijing Opera Synthesis Based on Straight Algorithm and Deep Learning.|
|Next Article:||A Perception-Driven Transcale Display Scheme for Space Image Sequences.|