WO2014053798A2

WO2014053798A2 - Means of providing three dimensional touch screen interface objects using conventional or printed materials

Info

Publication number: WO2014053798A2
Application number: PCT/GB2013/000407
Authority: WO
Inventors: David Bernard Mapleston; Brian Weeks; David Christopher Charles POTTER
Original assignee: Individual
Current assignee: Individual
Priority date: 2012-10-02
Filing date: 2013-09-27
Publication date: 2014-04-10
Anticipated expiration: 2015-04-02
Also published as: GB2512266B; WO2014053798A3; GB201217623D0; GB2512266A

Description

DESCRIPTION

MEANS OF PROVIDING THREE DIMENSIONAL TOUCH SCREEN INTERFACE OBJECTS USING CONVENTIONAL OR PRINTED MATERIALS

1.0 Background

In recent years we have seen capacitive touch screens become much more popular on a range of computing devices including PC's, laptops, electronic tablets, smartphones and now TVs. We have also seen many ideas such as cards which simulate finger touch positions on these capacitive screens. These cards, which contain conductive patterns, can unlock various promotions on selected web sites when the cards are positioned or moved across the computing device screen. They can also be used in games as dice or cards that interact through the touch screen with games software.

1.1 This patent uses the finger touch areas on capacitive touch screens in the same way as the above mentioned two dimensional card objects. However, with custom Software this invention can use between 1 and 10 multiple finger touch points to create various three dimensional foldable interface objects, such as packaging that can interact directly with the screen, a fully functioning computer on-screen mouse, or toys, all constructed from a simple printed sheet of plastic or paper.

2.0 Description of invention:

Tested embodiments.

2.1. The first embodiment of the invention is where one single face of the 3 dimensional (3D) object bears a unique pattern of conductive ink, and is linked to other patterns of conductive ink on the other faces of the 3D object. Touching one of the faces of the 3D object that is not touching the computing device screen triggers a specific action.

2.2. The second embodiment of the invention is where different faces of the 3 dimensional object bear different, but linked unique patterns of conductive ink and when a specific face is placed on the screen of a computer device the pattern recognition component of the embedded software in the computing device recognizes that pattern and a specific action is triggered.

It has been demonstrated that using printed technology different conductive patterns can be printed on each side of a small cardboard box or pack. When one side of the box is placed onto the screen of a computing device with a capacitance touch screen, and the pattern recognition component of the embedded software in the computing device recognizes the specific pattern and then the action component of the embedded software in the computing device takes a specific action for example, links to a web site.

2.3. In addition the shape around the conductive pattern can be cut out and folded to form a computer interface object. When used on the screen of a computer device with a capacitance touch screen, this printed folded object can become a fully functional and usable screen mouse or other more complex interface object for games or applications of many types.

2.4. The first embodiment of the invention was constructed using a cardboard box and copper tape folded into the shape shown in Fig 1. Fig 1.1 is the folded box and pads. Fig 1.3 and Fig

1.4 are made from copper tape. No other components or materials were necessary for this first simple embodiment. The copper pads and tracks Fig 1.4 and Fig 1.2 simply transfer the touch from the person holding the object to a computer recognizable touch pattern on the underside of the object, Fig 2.7 Fig 2.8 and Fig 2.9, which form a software recognisable touch pattern. Other touch surfaces can be transferred from the top, or sides of the object to become uniquely recognisable patterns of pads under the object which are recognised in all rotational orientations by the software. The patterns under the object simulate single or multi finger touches, Fig 2.5 and Fig 2.6. For simplicity, we will refer to this simple embodiment as 'touch transfer technology'.

2.5. An experimental embodiment is in the form of a computer screen mouse. The term mouse in this description refers to a cordless mouse computer interface object which is used on the capacitive touch screen of a computer device. The mouse is symmetrical and hence usable by both left and right handed people).

2.6. Experiments have shown that not only can this interface object, or mouse, give the movement of a mouse pointer it is also possible to detect right and left mouse clicks, Fig 1.3 and Fig 1.4. Other mouse functions such as scroll and multi button features found in today's conventional mouse designs can also be created using this touch transfer technology.

Using suitably spaced and shaped finger touch areas on the top of the mouse, the scroll wheel, which is found on many good mouse pointers, can be functionally simulated.

Also using touch transfer technology the angle of the mouse, pointing or control point reference to the screen can be detected by the software. This is an additional benefit not found in most mouse objects or even high quality computer control interface devices.

2.7. One major inventive step is that all the above described interface objects and functions can be printed using conductive inks, or other methods, to form patterns of conductivity in or on cardboard, plastic or rubber materials. Fig 4.18 shows such a printed pattern which can be folded into, for example, a functional computer interface mouse. The conductive pad areas which simulate fingers touching the screen are shown in Fig 4.18,19,20,21 and 22.

These materials can be folded into other three dimensional shapes to form a plurality of interface objects, mouse pointers, cubes, dice and other games interfaces and controllers. One

Embodiment is a folded multi-sided tube which can be rolled on the screen then pressed or held to identify the face touching the screen to the computer software.

2.8. From the above inventive step it is possible to extrapolate to many different types of interface object embodiments. This patent also describes touch pattern shapes that use the above principle to form an interactive folding cube where each face of the cube identifies itself and gives positional and directional information to the software. Again this cube can be printed using conductive inks in the same way as Fig 4.

2.9. These printed patterns can be folded or cut into other shapes to form a plurality of interactive two or three dimensional objects; furthermore, it has been proven possible to construct interface objects using an insulator with conductive areas printed or stuck onto the surfaces. For example, an interactive cardboard or foldable sheet plastic figurine can be identified through the touch screen when the user is holding or touching the main body or when individual parts of the figure are held or touched. Also, various actions can be made by the software which corresponds to the area touched on the figure.

Fig 3.16 shows a figure made from flat card attached to a folding base. Other parts of the figure figurine can also be touched at the same time as the base; for example a sword, Fig 3.15. This can be recognised by the touch screen and appropriate gaming or promotional material executed by the software.

3.0 Theory of operation

3.1 Known principles of touch screens: Capacitive touch screens are generally constructed using a cross-point matrix of transparent conductive wires which forms a grid pattern under or within the glass. Each vertical wire will give an 'x' coordinate and the horizontal wires will gives the 'y' coordinate. Usually all the wires in the display are driven with a frequency of about 40kHz as this is above audio hearing and hence reduces interference. Each wire is driven from a high impedance source and has a sensitive detector attached so that when the A.C. signal is loaded or cross coupled between wires, the drop in signal on that wire is seen by the detector. The A.C. voltage on both a vertical and horizontal transparent wire will decrease and an increase and so cross talk will be seen between adjacent wires. The detectors on each wire can easily see these effects and report an x,y coordinate for a large number of screen touches. For example finger 1 is located at x1 ,y1 and finger 2 is located at location x2,y2 and so on.

3.2. The loading on the wire is simply caused by the conductivity of human skin. The wire does not require actual physical contact as the small amount of capacitance, in the order of 1pF, from a finger to the wire gives enough signal drop for reliable detection; the signal is not only absorbed by the human body it is also re-radiated from the body which adds to the signal drop on the capacitance screen.

4.0 Embodiment description:

This patent embodiment uses finger contacts as described in the known state of the art above; however, the finger contact locations are simulated using conductive areas Fig 2.7, 8, 9, which can be set very accurately in specific locations on the base of the touch screen interface object. Finger pad locations on the top or side of the device, Fig 1.2 and Fig 2.10, can be transferred through a non conductive material, Fig 1.1 , using wires or conductive tracks, Fig,3.13, 14, to become fixed pattern of accurate finger touch areas on the base, Fig 2.7 ,8, 9. The pattern of conductive areas on the top or side of the interface object, Fig 1.2 and Fig 2.10, only need to relate to the simulated finger patterns below the object; they do not need to conform to the same shape or pattern on the top or side; this lends itself to many interesting applications based on the inventive step of converting conductive shapes on the top or side of an object and wiring these to other shapes on the bottom of an object to simulate patterns of one or more finger touches on a capacitive touch screen.

4.1. One caveat in the design of any products using this 'touch transfer technology' is that the interface object will need to be held by a human hand to create the required capacitive losses on the pad shapes under the object, Fig 2.7, 8, 9. This leads to the restriction that the interface objects location cannot be detected when it is not being touched by the user. In fact this small functional caveat actually enhances the functionality of the interface object or mouse in most applications.

5.0 Simulated touch patterns on various three dimensional objects.

5.1. The flat base of any folded object can easily be recognized with any pattern of conductive transfer dots. With a base that is large enough to carry 10 dots we can immediately see that this yields 1024 individual items which can be separately recognized by the software; although with ten dots the base area would need to be very large.

5.2. In one embodiment of this invention not only do we require the detection of the object we also require a directional vector which can be read by the software to indicate the direction in which the object on the screen is pointing. This requires patterns which have no rotational symmetry between each other Fig 5.26.

5.3. To clarify, if we only had two touch areas the software could find a line between these two points; however, a third touch point would be needed to determine which of the two directions the object Fig5.23 is actually pointing. Fig 5 shows just nine of these patterns. Each touch pad in Fig 5.25 is connected to the centre touch point Fig 5.24.

5.4. In an embodiment where multiple faces of the object have touch transfer technology the centre conductive pads Fig 5.24 can be connected together; however, the touch transfer technology will still function, to a degree, if the object are connected with tracks or wires between the other touch pads and no centre pad is used (Fig 5.25).

6.0 Software within the embodiment.

Within this invention the software for identification of the patterns and shapes at the bottom of the interface object, for example Fig 2.7, 8, 9 are critical to reliable detection and functionality.

6.1. The prototype software uses a very simple technique to recognize the touch patterns under the object.

6.2. One embodiment of the software uses the distance between any two finger touch points which can be calculated from the x, y coordinates of each pad, Fig 2.7, 8, 9, using Pythagoras. Ra = V ((x1-x2)²+(y1-y2)²)

6.3. Where x1 ,y1 are the coordinates for pad Fig 2,8 on the screen and x2,y2 are the coordinates for pad fig 2,9 on the screen, this can be extended beyond x10,y10 if so desired.

Ra is the distance between Pad Fig 2.8 and pad Fig 2.9. So we can see now that Ra is now constant for all angles of the interface object, in real time, and is the distance between two of the fixed simulated finger touch areas on the base of the object.

6.4. To identify a shape of say three simulated finger locations Fig 2.7, 8, 9, the relative distances from each of these pads can be calculated from the above equation to yield Ra, Rb and Rc; these calculated distances between the centers of each pad (simulated finger locations) can be compared with known Ra, Rb and Rc distances within a database. What is more, by simply setting the three pads Fig 2.7, 8, 9 of the pattern, to a non equilateral triangle a direction vector can easily be extracted from the data. An example would be that the adjacent side of the triangle created by the pads is designated the line of direction and the intersect of the adjacent with the opposite side of the triangle is designated the pointer direction.

6.5. By adding more simulated finger locations and using more intricate shapes, the number of individual items which can be identified is very large indeed and only restricted by the area of finger touch (about 28mm²), the number of detectable fingers (ten or more in modem devices) the accuracy of detected location and the physical size of the base of the interface object, mouse or figure are only restricted by the area of the screen.

6.6. It is also an inventive step to write additional software which can measure the locations of the simulated finger location pads and automatically store these Ra, Rb, Rc values within the database with reference to the name, serial number, orientation and last pointing direction of the Interface object.

6.7. The construction of the right and left click mouse buttons or, for example, fire and jump buttons for a gaming interface object can be achieved with simple additions to the hardware and software.

6.8. The mouse or interface object in the first embodiment has extra pads on the base of the unit Fig 2.5, 6 connected to a similar pad on the surface of the unit Fig 1.3 & 4. When the top surface pad is not touched by the user, the pads below the object do not discharge or load the touch screen and hence cannot be seen by the software. When the user is holding the object and then touches the top surface pad or pads, then the pads on the bottom Fig 2.5, 6 simply load the screen and are seen by the touch screen as extra finger presses.

The software can easily detect this extra simulated finger press and hence take the required action which could be, as mentioned earlier, right click, left click, fire, jump etc. 7.0 Applications for this invention.

The above description of an extra finger contact coming into play whilst other patterns are being recognized can be a very powerful tool in marketing, gaming, brand awareness and internet selling. For example, a like or dislike action can be instigated whilst using the interface object on an internet page. Only people with this interface object would be able to vote, like or dislike items or content on the page. Although this would not be totally secure, it would be very difficult to copy the finger positions and instigate an action from the software or internet page; especially if the conductive layer for the touch transfer technology are embedded within the opaque card.

7.1 Scrolling

Two or more surface pads on the top of the interface object such as already described and shown in Fig 2.5, 6 and Fig 1.3, 4 can be stroked to trigger finger areas on the base of the object with timing intervals. A forward direction could be instigated when pad Fig 1.3 is touched before pad Fig .4 and vice versa for a reverse direction scroll. The pads for scrolling could obviously be positioned in the location of a conventional mouse scroll wheel and added along with the right and left mouse click pads.

If the pattern for stroke detection is repeated a number of times to form a circle of pads, a crude joystick can be formed on the surface of the interactive object.

7.2. Promotional and collectable figures

7.2.1. Cardboard of printed plastic figure shapes cut from packaging or inserted into packaging, given away or sold as promotional items could use this invention to bring packaging to life on any capacitive touch screen.

7.2.2. The second embodiment for promotional collectable or gaming figures is more complex; however, of greater game play value. Whilst holding the body of the item or figure, other parts of the figure can be touched at the same time. For example, a figure could be identified on the screen within a game, when the user holding the figure then squeezes the sword held by the figure; the sword is then used within the game play; if the user then squeezes the shield on the cardboard figure whilst holding the body of the figure the game play then uses the defensive shield. The embodiment for this function is exactly the same as the mouse description earlier where the mouse has right and left click touch transfer pads which are are simply connected to the arms and conductive areas in the hands of the figure. This embodiment is not restricted to just two areas of contact, touching various areas of the figure could trigger sounds, videos, extra game play etc. An example embodiment would be a foldable mixer desk which could be placed on the screen and used to control a virtual party; changing songs and creating sound effects.

7.2.3. Experiments have shown that the user's other hand can also activate these functions, for example the user can be holding the body of the figure or object with the left hand and then touch the turntable or squeeze the sword/shield with his her right hand to instigate the required action. (There is obviously no difference between holding the figure in either hand and instigating the action with the opposite hand.)

7.2.4. It is important to inform the gaming manufacturers reading this document that this patent covers any game or program where more than one figure or object is on the screen at any time and where the two figures or objects interact with each other through the screen and software. It is also obvious that a number of figures could battle or interact with each other in a myriad of ways.

7.2.5. Another embodiment is covered in this patent where two or more 'touch transfer technology' interface objects actually touch each other to instigate some game play or function.

7.2.6. Conductive plastics shapes have been tested and have proved 100% affective within all these embodiments, even with touch areas on the screen as little as 5mm diameter. 7.2.7. Embodiments of this invention could be created in the form of musical instruments such as keyboards and drum kits that function when placed on a touch screen.

7.2.8. A further embodiment of this invention splits the computer interface object into two separate sections which are both used on the screen at the same time. These objects are connected to each other by wires; this allows complex game play and joint control of functions and game play actions.

7.3 Augmented Reality

Augmented Reality is a means to enhance or augment your real-life surroundings through the use of a object.

7.3.1. Augmented Reality is currently being used in game play selling and promotional objects. One embodiment of this technology uses an appropriate touch pattern to trigger an augmented reality experience on a two dimensional of three dimensional folded figure.

7.4 Promotional and Intelligent Packaging

The above technical inventive steps also lend themselves to intelligent packaging. This could have various embodiments, for example use the box itself on the screen or a folded object made from packaging; these could:

7.4.1. Trigger an Augmented Reality or similar experience

7.4.2. Play a simple on-line game based on characters or the brand trade mark.

7.4.3. Be used to create an on-line coupon or voucher for the consumer.

7.4.4. Provide an indication of the age/status and hence condition of the product inside.

7.4.5. Provide recipes or other instructions for the items inside.

These packages would; however, need to keep any conductive contents away from the packaging surface which holds the touch transfer technology pads.

7.5 Brand Protection and Anti-counterfeiting

The above technical inventive steps also lend themselves to enabling consumers or brand owners identification to validate the authenticity of a product, by validating the code pattern. Although this invention cannot provide a totally secure method of protection it can provide a first level of authentication by hiding the patterns covertly within the item or the packaging for the item.

One example would be with the training shoes box, where placing part of the box on the screen and then touching another part of the box, an authentication screen could then be displayed.

8.0 Field effect or transistor switching.

8.1. A further enhancement of this invention is to use transistors such as Field Effect Transistors (FET) fixed or printed onto the card or plastic sheet Fig 2. This hand held unit contains the main connection to the human body and a number of touch contacts. These touch contacts operate the gates of FET devices in the base of the unit which then force conduction paths within the folded 3D box or 2 dimensional card. Obviously, relays could be used to provide these simulated finger touch connections but are much more cumbersome and high power. The use of opto-couplers within the base unit can also be implemented to prevent radiation leakage from the pads and FEETs.

8.2. For more complex gaming a microprocessor can be installed into the unit. The micro processor or processors can then measure inputs from accelerometers and other transducers along with button press, and timing circuits to form a myriad of game play functions; for example, joy stick inputs can be constructed. The processor can drive FEET transistors to pulse or hold various low impedance paths and generate combinations of simulated touch areas to transfer the data through the touch screen to the computer software.

Depending on the speed of the software, ten bit parallel data words can be transferred through the screen to the software in the computer. The FEET devices simply connect touch areas to the user's fingers on the top of the unit Fig 2.10 which is in turn connected to the users hand or hands on the unit. Each simulated touch pad would provide a one or zero data bit so that each instigation of say ten parallel simulated finger presses could transmit a number between 0 and 1024. It is therefore possible to transmit over 2000 bits per second from a object through the touch screen using this activated touch transfer technology method.

With this embodiment it is also possible to use long wires connected to the screen unit so that the user no longer has to hold the object whilst data transfer is taking place.

8.3. Very complex controls and functions can be communicated to the software in the TV, pad or phone by these means; this embodiment can transfer real data through the screen of the touch device such as ASCI characters or graphics patterns for complex control functions.

8.4. One clear inventive step is the design of a base unit where multiple low cost hand units can be plugged in. The application of this embodiment would obviously be board games or interactive game play. All the relevant sounds for the game play could be stored in the computer making the screen and base unit very low cost.

8.5. The screen unit could be held onto the screen using suction, elastic straps or even spring clips etc.

9.0 Industrial embodiment

For some industrial applications real data can be transferred through the screen to the computer software.

9.1. For example, it is possible to place a specially designed bar code reader or RFID reader onto the capacitive screen of a computer and transfer the label or tag data through the screen to the data base within the computer or phone; low cost, wire and connection free data transfer using touch transfer technology would be a very cost effective solution to many low data rate frequent usage devices.

9.2. Using a USB port connected to a touch transfer technology object would be a very practical and low cost solution to a data transfer unit which would require no connection at the phone or pad end.

10.0 Touch and texture

In combination with haptic and other electrostatic technologies that simulate screen textures the embodiments described could be used to transfer textural surface contours to areas on the top or sides of the unit, mouse or control object.

10.1. For example if the control object is placed over a picture of a corrugated surface and a finger moved over the control object surface, the surface of the control object feels like a corrugated surface and the control object is moved over the picture or a corrugated surface the control object feels like it is moving over a corrugated surface.

10.2. When the 'touch transfer technology' mouse or control object is moved over the screen it is now possible to change the friction between the object and the screen, creating a new dimension to games, gadgets, promotional material and advertising; for example a container could give textural information relating to its contents.

10.3. Haptic and static texturing on pads and screens is now fully functional in 2013 and due for production early 2014.

10.4. This patent covers the use of this 'touch transfer technology' where the high voltage low current signals used in haptic technologies and other static electric screen technologies are transferred through a mouse, games controller, promotional figure figurine, pack, collectable or toy so that the object when touched takes on the texture generated by static forces created and transferred through the screen.

11.0 Conclusion

In conclusion we have demonstrated and tested embodiments of this invention which takes advantage of multi touch capacitive screen technology to provide many avenues for the design of promotional objects such as folding cards, folding figures, figurines, packaging, cubes and commercial industrial objects such as industrial RFID readers. We have also shown that 'touch transfer technology' can be used to create a fully functional on screen mouse with tactile feedback and surface textural touch.

11.1. Many other inventions and products can be created using this three dimensional touch transfer technology which are out of the scope of this patent description.

David B Mapleston

29^th Sept 2013

Claims

1. A three dimensional (3D) foldable interface object bearing specific patterns of conductive ink, conductive plastic or metal pads for use directly on a capacitive touch screen of a computing or mobile communication device, Smartphone or electronic tablet, where a single face of the 3D interface object is placed directly on the screen and by touching different connected places on any other face of the 3D interface object the Pattern Recognition Component of an embedded software program recognises the specific pattern as though they were individual finger placements on the screen and causes the Action call Component of the software to activate a specific command or action in the computing or mobile communication device for each different connected place.

2. A different embodiment of the 3D foldable interface object as in Claim 1 has different faces which each bear different conductive ink, conductive plastic or metal pad patterns which are placed directly on a capacitive touch screen and where the Pattern Recognition Component of the software recognises the specific pattern of conductive areas as though they were finger placements on the screen and when any other connected conductive face is touched causes the Action call Component of the computer software to activate different commands or actions in the computing or mobile communication device depending on which face is touching the screen.

3. A 3D interface object as in Claim 1 where the 3D interface object can provide all the functionality of a cordless screen mouse and which has right and left click areas on the surface of the 3D interface object to discharge capacitive areas on the base of the 3D interface object signalling right and left mouse clicks

4. A 3D interface object as in Claim 1 which has a scroll up/down

function using two or more capacitive contact areas on the surface of the 3D interface object which transfer touch to areas below the 3D interface object signalling scroll direction and speed.

5. A 3D interface object as in Claim 1 which has a scroll up/down

function using a wheel which makes and breaks capacitive or direct contact between the top or side conductive surfaces and two or more bottom conductive pads which then simulate a sequence of surface finger touches which are decoded by the computer software as scroll up or scroll down actions.

6. A 3D interface object as in Claim 1 which is steerable and via a

'mouse pointer' to navigate the user to different places on the screen.

7. A 3D interface object as in Claim 1 where the 3D interface object can run an internal application for example such as displaying a number or starting a game or go to an external webpage or web application.

8. A 3D interface object as in Claim 2 where using 'rollable' versions of 3D interface objects so that, depending on which side of the 3D interface object lands on the touch screen when rolled, different numbers or characters can be transferred into the computer either singly or in sequence, hence generating a number when the 3D interface object is touched.

9. A 3D interface object as in all above Claims which can be folded from flat printed materials and then constructed into a 3D interface object.

10. An embodiment as in all prior claims where directional information is transferred to the computing or mobile communication device through this invention by the user stroking of one or more one fingers over two or more conductive capacitive pads.

11. An embodiment as in all prior Claims and where the 3D interface object is of any physical construction where the current flowing through the screen to the finger or fingers or hand held unit is used to power an integrated circuit within the 3D or 2 dimensional interface object.

12. An embodiment as in Claim 11 where the integrated circuit is used to transfer data of any kind to the computing or mobile communication device through the screen when a finger or fingers are placed on the top of the 3D interface object to create the required current flow through the integrated circuit.

13. An embodiment as in Claims 11 and 12 where the integrated circuit is battery powered and activated by the current flow between the finger and the screen when a capacitive pad is touched or held.

14. An embodiment as in Claims 11,12 and 13 where the integrated circuit modulates the touch screen areas to transfer data to the computing or mobile communication device software using on/off simulated touch contacts when the user is touching or holding the 3D interface object.

15. A 3D interface object such as described in all previous Claims where the computing or mobile communication device can transfer touch sensations to the user through the 3D interface object or objects using haptic or other static attraction/repulsion technology, for example Senseg's (RTM) based on the principle of Coloumb-*s force.

16. A detection method which measures the location of simulated capacitive touch screen areas provided by the 3D interface object and stores the co-ordinates or the relative co-ordinates with a reference name or number in a database of any description for any purpose for example game play or calibration.

17. An embodiment as in all prior Claims where two or more 3D interface objects are used on a capacitive touch screen at the same time.

18. An embodiment as in all prior Claims where two 3D interface objects interact with each other through the computer software or by directly touching each other.