GB2486425A - Rendering multimedia content from a mobile device onto an external display device - Google Patents
Rendering multimedia content from a mobile device onto an external display device Download PDFInfo
- Publication number
- GB2486425A GB2486425A GB1021119.1A GB201021119A GB2486425A GB 2486425 A GB2486425 A GB 2486425A GB 201021119 A GB201021119 A GB 201021119A GB 2486425 A GB2486425 A GB 2486425A
- Authority
- GB
- United Kingdom
- Prior art keywords
- type
- electronic device
- mobile computing
- computing device
- multimedia
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000009877 rendering Methods 0.000 title abstract description 7
- 238000004891 communication Methods 0.000 claims abstract description 158
- 238000000034 method Methods 0.000 claims abstract description 135
- 238000012545 processing Methods 0.000 claims abstract description 95
- 230000000007 visual effect Effects 0.000 claims description 47
- 238000007906 compression Methods 0.000 claims description 45
- 230000006835 compression Effects 0.000 claims description 37
- 238000006243 chemical reaction Methods 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 19
- 238000007790 scraping Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 14
- 230000002123 temporal effect Effects 0.000 claims description 13
- 230000002093 peripheral effect Effects 0.000 claims description 11
- 230000005236 sound signal Effects 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 238000005266 casting Methods 0.000 claims description 8
- 230000000977 initiatory effect Effects 0.000 claims description 8
- 238000003825 pressing Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 5
- 239000002131 composite material Substances 0.000 claims description 5
- 241000721701 Lynx Species 0.000 claims description 4
- 239000000796 flavoring agent Substances 0.000 claims description 4
- 235000019634 flavors Nutrition 0.000 claims description 4
- 238000010079 rubber tapping Methods 0.000 claims description 4
- 238000007789 sealing Methods 0.000 claims description 3
- 238000000060 site-specific infrared dichroism spectroscopy Methods 0.000 claims 19
- 230000007246 mechanism Effects 0.000 abstract description 5
- 230000003993 interaction Effects 0.000 abstract 1
- 230000033001 locomotion Effects 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 7
- 238000003032 molecular docking Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 5
- 230000008093 supporting effect Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 241000255925 Diptera Species 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 235000008694 Humulus lupulus Nutrition 0.000 description 1
- 206010028923 Neonatal asphyxia Diseases 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/611—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for multicast or broadcast
-
- H04L29/06068—
-
- H04L29/06503—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/756—Media network packet handling adapting media to device capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/765—Media network packet handling intermediate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
-
- H04M1/72527—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/06—Authentication
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72412—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories using two-way short-range wireless interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W8/00—Network data management
- H04W8/26—Network addressing or numbering for mobility support
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/10—Small scale networks; Flat hierarchical networks
- H04W84/12—WLAN [Wireless Local Area Networks]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/02—Terminal devices
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to an arrangement for projecting the instantaneous multimedia content of any capable mobile computing device onto an external display apparatus such as a television 130 or projector 104 (i.e. a sink device). This involves constantly streaming multimedia content from the mobile computing device in real-time via one communication interface and optionally applying multimedia processing operations before rendering or transporting it in real-time to the external sink device via another communication interface for an immediate playback. The proposed solutions enable device interoperability by allowing the interaction between any capable mobile computing device and an external sink device from completely different manufacturers without requiring any proprietary-based mechanism. The device 140 interfacing the mobile computing device to the external sink may take the form of a WLAN Access Point. The proposed solutions also enable the external display apparatus to function as a wall-art or a digital photo-frame. Also mentioned is a method of transmitting the sink device audio to one or more wireless headphones 144. The proposed solutions are empowered to support any type of multimedia content using any format at any time for use in any location.
Description
Multimedia Rendering Apparatus, System and Methods thereof
Technical Field
The present invention relates to system, apparatus and method for connecting a variety of multimedia capable mobile computing devices to an existing or future external display apparatus such as a television, a monitor apparatus or a projector seamlessly for the purpose of wirelessly projecting the instantaneous screen display of the mobile computing device in real time.
Background
The present day television equipments are used mainly for one purpose in homes, businesses and institutions -i.e., viewing/watching recorded as well as broadcast multimedia content. This is despite the fact that each television set takes up a considerable space where it is used and requires a significant investment on the part of the user to purchase. Considering this limitation, modern televisions are becoming multi-functional and arc nowadays used for variety of purposes; e.g.., to play computer games, being part of home surveillance security system and the like. Given that the modern TVs take a very slim form and the fact that the consumer video market is undergoing several dramatic changes that revolutionise how we enjoy the audio-visual entertainment, the present invention helieves that the usage of such TV sets
I
for multiple different purposes can still be increased. The purpose of the present invention is therefore to enhance the versatility of a television set, monitor apparatus or projector so that it can be used in a variety of ways as a way to improve the quality of life of a house-holder.
Mobile portable handheld devices are perpetually evolving and play a large part in many of our daily lives. These portable handheld devices that come in different shapes with heterogeneous capabilities are used to provide communication and entertainment facilities to any end user in an "anytime and anywhere" manner. This may include mobile phones, smart phones, personal digital assistants (PDAs) or potable mobile devices having broadband connectivity such as mobile tablets (e.g., Apple's iPad), electronic readers (e.g., Amazon's Kindle), notebook, media players, laptops and similar devices. These handbeld devices have a small form factor and are collectively referred to as mobile computing devices in this document. People tend to carry at least one mobile computing device all the time and hence these devices help them communicate with anybody wirelessly. More over, with the recent technological advancement, small portable mobile computing devices are capable of storing large amounts of multimedia (especially video) content in the hands of a user. While these portable mobile computing devices have their own video screens, the visual experience for high resolution and large format display is very difficult to be replicated in such devices, This is because any portable mobile computing device has its physical size limitations and as a result its size (i.e., form-factor) cannot be arbitrarily large -hence it is limited in terms of their viewing area. It would be hence better if a neighbouring TV or monitor apparatus can he used to display either the instantaneous screenshot of a mobile computing device for clarity -for S instance in order to watch a movie via a mobile network. The idea is to capture in real-time whatever graphical elements thai are actually or intended to be painted to the "screen" of the mobile computing device and send them down to the external display apparatus constantly. This feature will allow any mobile computing device having a touch screen functionality to be used mainly to receive user inputs and to use the neighbouring TV or monitor apparatus or projector for outputting the multimedia (especially the video) content. As a result, a mobile computing device can easily be hooked up to one or plurality of monitor displays, TV and/or projectors at once in order to use the external display devices solely to output the multimedia content while using its own electronic display to receive user inputs as long as it is touch-screen enabled.
However, connecting any arbitrary mobile computing device to an existing television set, a monitor apparatus or a projector wirelessly is not straight-forward even to a gadget expert. This is partly due to the fact that the existing mobile computing devices and external display apparatus do not share a common wireless interface directly. Hence, one objective of the present invention is to devise an apparatus, system and methodologies with an objective to connect any handheld mobile computing device having the required hardware and/or software capabilities seainlessly to an existing television set, a monitor apparatus, a projector or a like for the purpose of displaying the instantaneous sereenshot or intended display of the handheld S mobile computing device on the television set, a monitor apparatus or a projector respectively in real-time. It is preferred that this process takes place dynamically in a user-transparent way.
Some of the present day mobile computing devices are integrated with high definition display technology, although they have limited screen size.
Wirelessly projecting a mobile computing device's display on to an existing television set, a monitor apparatus or a projector without having to leave the couch in the sitting room will bring new levels of functionality to the home entertainment experience. With advanced immersive digital display and imaging technologies and the availability of enormous processing power at low-cost, higher quality user experience is possible with an ability to access multimedia on the go across different devices in the home. This ability requires the multimedia (especially video) content to he adapted depending on the capabilities of target sink devices, Hence, it is important to support any type of multimedia content using any format at any time for use in any location that the home theatre system can reach. This is because in order to have an enhanced user experience it is important to scale the received video content's original resolution to the target devices resolution or re-format if it is going to fit the target screen without distortion of the image. This further requires finding out the exact capability details of different devices on-the-fly before transeoding between different compression/encoding methods, transtrating, up-sealing or transizing, restoration or reformatting that would S change the content formats, bit-rates, frame rates, resolutions, colour-depths, aspect ratio and the like. Employing one or plurality of these video processing techniques in real-time is inevitable when it comes to mobile phone content adaptation.
Transcoding is the direct digital-to-digital conversion of one encoding to another. The most popular definition of transcoding refers to a two-step process in which the original data/file is decoded to an intermediate uncompressed format (i.e. PCM for audio or YUV for video), which is then encoded into the target format. Given that transcoding causes a progressive loss of quality with each successive generation, known as digital generation loss, smart lossless compression algorithms/techniques need to be employed.
Transrating is a process similar to transeoding in which files are coded to a lower bit-rate without changing video formats; this can include sample rate conversion, but may use identical sampling rate but higher compression.
Changing the picture size of video is known as transizing, and is used if the output resolution differs from the media's resolution. In the case of wireless transmission of any multimedia content, error concealment is performed on the receiver side in order to conceal the lost data and make the multimedia content less displeasing to human eyes and/or ears. There are two basic approaches for error concealment; i) spatial interpolation and ii) temporal interpolation, in spatial interpolation, mixing pixel values are reconstructed using neighbouring spatial information. On the other hand, in the case of S temporal interpolation, the lost data is reconstructed from data in the previous frames. Two or more of the previously mentioned video processing techniques can be applied simultaneously; for instance, reformatting can go hand-in-hand with transcoding. In the case of High-Definition (HD) multimedia content, enabling multi-format and/or multi-codec support in real-time in consumer video systems demands the availability of high processing power, adequate high-speed memory footprint and 110 bandwidth.
In the considered scenario, multimedia content is transmitted wirelessly from any mobile computing device to a TV set, monitor apparatus or a projector of a home theatre system for a large display or enhanced multimedia experience.
However, video always requires high-bandwidth and it is usually paired with a smaller audio stream. Wireless bandwidth is always limited and is susceptible to time-varying nature because of channel impairments, interferences and mobility. Hence, the multimedia content needs to be encoded using advanced codecs in order to bring down the transmission latency and to free up bandwidth especially with the much greater throughput requirements of High Definition (HD) content. This is where video/audio compression comes into play. Digitised form of any multimedia content provides a wide variety of formats and compression techniques. The aim of video compression is to remove redundant information from a digitised video sequence. Raw video needs to be compressed before transmission to achieve efficiency. This is critical for wireless transmission for the reasons mentioned S before. Some of the major factors influencing the complexity and design of video compression arc the display resolution, video quality, error resilience, power-consumption, memory footprint, latency, and the like.
MPEG-2 is one of the long-established video codec being used for both Standard Definition Television (SDTV) and High-Definition Television (HDTV) formats. This can sustain high quality on complex video content at compression ratios about 30:1 and 50:1. MPEG-4/H.264 is one of the newest and more advanced codec and typically achieves a compression ratio between about 60:1 and 100:1. Thus, in order to save bandwidth, it is better to shift MPEG-2 content to MPEG-4 content without a change in resolution. MPEG-4 is ideally suited to low-bandwidth applications. It uses motion vectors between frames to encode temporal redundancy and the discrete cosine transform (DCT) to encode spatial redundancy. MPEG-4 provides three modes for encoding an input -which are Intra-frame (I-frame), Predicted-frame (P-frame) and Bidirectional Interpolated-frame (B-frame).
There have been a larger number of techniques that allow desktop remoting -i.e., hooking up a computer to two monitors at once. The general idea of any desktop remoting technique is that whatever graphical elements being painted to the "screen" on the host are then scraped by the protocol interface and sent down to a remote client terminal. However, conventionally the requirement was to allow remote execution of a Windows instance that sends screen updates across a network to a remote client display device. Further S conventionally there was not a big demand to display graphic-intensive screens remotely. On the other hand, one of the objectives of the present invention is just to project the screen of a portable handheld mobile computing device on to a neighbouring monitor apparatus, TV or projector along with/without the associated audio content without having to facilitate remote execution/computing. This means that there has to be a mechanism on the portable mobile computing devices to capture all possible graphical layers (GD!, WPF, DirectX, etc.) and renders them into a remote protocol stream (like RDP) where they are sent down to the neighbouring monitor apparatus, TV or projector as fast as possible. Further, the remote displaying has to take into consideration the graphic-intensive nature of displays as required by the present day multimedia applications, e.g., the Aero experience. 3D applications, and especially applications that require audio-video synchronization. Given that the main design motivations of the present invention are different from those of the conventional techniques, conventional remote display techniques/protocols cannot be employed without incurring unnecessary processing, transmission and memory cost unless modified.
The conventional remote displaying technologies are based on the assumption that in terms of the processing and memory capabilities, the host (i.e., the server) is much powerful than the client devices -i.e., ultra-lightweight thin clients. However, this assumption is not necessarily true in the context of the S present invention because the portable handheld mobile computing devices are mostly battery-powered. These existing solutions are collectively known as virtual desktop interface (VDI) solutions. Typical examples being Remote Frame Buffer (RFB), Virtual Network Computing (VNC), Remote Desktop Protocol (RDP), Tndependent Computing Architecture (1CA) of Citrix Systems, Net2Display.
Watching a TV can disturb others who happen to be in the near vicinity because of the associated audio/voice/sound content. For instance, parents want to watch one or plurality of their much-loved favourite TV programmes without having to worry whether the sound would wake up their sleeping baby or disturb a studying son/daughter. For this purpose, it is better to connect one or plurality of headphones/headsets to the TV set or home theatre system for both the parents to enjoy watching any favourite TV programme silently especially in the nights (i.e., headphone listening). Most of the TV sets or home theatre systems have only one or limited number of headphone connector (e.g., jack) for connecting any headphone -however, it may sometimes be required to connect more than one headphone to a TV set or a home theatre system depending on the number of viewers who want to watch a given TV programme silently. One another objective of the present invention is to address this issue.
Given that most of the modern TVs possess large screen-size employing flat-panel technologies and are wall-mounted, any TV set or any similar video display device of any home theatre system such as a projector can be used to display a scenery or wall-art decorating the place where it is mounted when not in use. More over, an existing TV set, monitor apparatus or a projector can be used as a digital photo-frame. One another objective of the present invention is to provide this feature to an existing television set.
In addition, it should also be possible to readily connect any multimedia capturing device such as a webcam, security surveillance camera, camcorder, laptop/notepad and the like to an existing TV set, monitor display or a iS projector so that a variety of multimedia content can be viewed in a much convenient way on a big screen.
Most of the convenient communication interfaces supported by various multimedia-enabled mobile computing devices and multimedia capturing devices such as a mobile handset, webcam, camera, laptop/notepad and the like are not supported by currently existing TV set, monitor display or a projector. Hence, connecting them directly is challenging or rather difficult.
Also, it is very difficult to completely replace the old TVs with the new ones over a night on a global scale. Hence, one another objective of the present invention is to address this deficiency as a way to enhance the versatility of an existing television set, a monitor apparatus or a projector.
It can now be understood that the new apparatus, system and methodologies proposed by the present invention enables a variety of handheld mobile computing devices such as mobile phones, PDAs and the like and other multimedia capturing devices such as a video camera, camcorder and the like to he seamlessly connected to an existing/future TV sets, monitor apparatus or projectors. in order to facilitate this process, according to one embodiment of the present invention an electronic device of first type is devised and this will enable any mobile computing device or multimedia capturing device to he connected to an existing/future TV sets, monitor apparatus or projectors. Each of the said external devices that are connected to a TV via an electronic device of first type can function as an external source of signal, and the proposed electronic device of first type will turn the signal into content which is then displayed on the television screen or other monitor display device or projected using the projector. Hence, in this respect the electronic device of first type may appear to operate in the same way as an AV receiver or a set top box, although the main purposes of its existence are different. However, all the new features as proposed by the present invention can complement the funetionalities of a modern set top box (STB) and/or an AV receiver, in other words, the new apparatus, system and methodologies proposed by the present invention should not necessarily he construed as the alternative to or the replacement of an existing AV receiver or an STB or their functionalities; instead, they can complement these devices.
Various prior art have come up with different solutions to the same research problem of how to project the screen of any portable mobile computing device on to a neighbouring TV, monitor apparatus or projectors. US patent US 2010/0169935 Al deals with the issue of how to use a mobile phone as a mobile set-top box. Given that the resolution of the mobile phone is smaller than that of any high-end external display, the mobile set-top box is proposed to have a media content processing circuitry in order to up-scale the media content. It fails to give much-needed details as to how media is captured, processed and transmitted to an end display. in case of wireless transmission, how communication is established is not explained clearly. Further today's graphic-intensive media requires further processing in addition to ensuring audio-video synchronisation in ease HDMT is not used these aspects are missing in US 2010/0 169935 Al.
Another related patent WO 2003/032504 presents methods and systems for providing Internet services to areas where there does not exist any reliable landline telephone/data network by establishing wireless communication between a remote data network and a set-top box, This is made possible by using a wireless telephone to receive and transmit packets to a wireless or cellular network and using the standard television as a visual display unit. For this purpose, the set-top box connects to a docking station where a wireless telephone docks. Wireless telephone is inserted into docking station for the invention of WO 2003/032504 to he operative. When out of docking station wireless telephone may be used as it normally would for voice communications. Hence, with the help of a docking station and a wireless terminal, the set-top box can stream multimedia content via the Internet and playback it on the television set. In other words, the docking station and the wireless terminal provide wireless modem functionality to the set-top box to connect to the Internet. On the other hand, the objective of the present invention is to project the screen of any mobile computing device on to a neighbouring external display apparatus for clarity of vision to many. Hence, WO 2003/032504 and the present invention do not deal with the same issue and hence arc orthogonal.
Patent WO 2008/136620 deals with an issue of empowering a mobile terminal to receive digital broadcasts by equipping it with a digital broadcast receiving circuitry. This is again not the intention of the present invention and hence is considered orthogonal. A US patent US 2009/0117890 Al proposes a new device and method for the purpose of connecting a portable miniature display to a host computing system wirclcssly in order to perform rcmote execution/computing. For this purpose, the wireless interface further supports Microsoft Window SlideShow, RDP and VNC protocols. The proposed new devicc of US 2009/0117890 Al comprises an input device being configured to communicate with the host computing system. On the contrary, the aim of the present invention is to project the screen of a mobile computing device on to a neighbouring external display device such a television, monitor apparatus or projector without having to facilitate remote computing/execution.
Recently Intel showcased one of its solutions that enables a consumer to project a notebook PC display on to a TV. This is however a proprietary solution that requires a notebook PC to be equipped with a special Intel processor and software components. Further the adapter that relays the connection to a TV should support intel Wireless Display technology. On the other hand, the present invention is interested in empowering any mobile computing device to project its exact screen or intended screen to an external display device for the purpose of enhancing the clarity of vision to many in a non-vendor specific way -this is possible by running a purpose-built host software module on the mobile computing device as long as it has the necessary generic-purpose hardware in order to capture the instantaneous media content and transmit.
Some of the latest smart phones have their own decking stations that allow them to connect to a television via the HDMI interface. Given that in a home/office environment, occupants may have different mobile computing devices being manufactured by different competing companies, connecting each mobile computing device to a given external display apparatus via a docking station is challenging. This is because an external display apparatus may have only a limited number of HDMI connectors. Further this approach will lead to device interoperability problems. In addition, this approach will not work out for old external display devices that do not support HDMI at all.
On the other hand, these deficiencies arc correctly addressed in the present invention.
This way the solutions proposed by the present invention empower any mobile computing device to instantly project its exact or intended screen to an external display apparatus possibly with an associated audio content. Further, the proposed solutions of the present invention enables seamless connection between any capable mobile computing device to any external display apparatus in a non-vendor-specific way and allows only the authorised devices to project their screen on to a neighbouring external display apparatus. From a user perspective, establishing first time connection between a mobile computing device and an external display apparatus is as easy as connecting to a WLAN access point. Further the same solutions would work irrespective of how old the external display apparatus is.
Disclosure of the invention
According to the first aspect of the present invention there is provided an electronic device of first type that receives multimedia content from a mobile computing device and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player in real-time, and the said electronic device of first type comprising: i) a wireless transceiver providing a communication interface of first type to communicate with the mobile computing device; ii) a communication interface of second type to communicate with an external visual display apparatus or multimedia player; S wherein connecting a mobile computing device to the said electronic device of first type involves the configuration procedure comprising the steps of: a) enabling the electronic device of first type to operate as a master or a wireless local area network on the said communication interface of first type with a specific SSID with the Dynamic Host Configuration Protocol (DHCP) functionality; b) letting the mobile computing device operate as a slave, detect the SSID, if it is broadcast, and connect to the electronic device of first type via the communication interface of first type after a successful authentication; c) letting the said electronic device of first type assign a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; wherein the said electronic device of first type has a media processing unit that takes the multimedia content being transmitted via the communication interface of first type as the main input, applies one or plurality of media processing operations on to it depending on the type of external display apparatus and the input multimedia content type, and outputs/renders it to an external display apparatus and/or multimedia player via the communication interface of second type for the purpose of instantaneous playback.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player; wherein the mobile computing device includes a mobile phone, smart phone, personal digital assistant (PDA) or potable mobile device having broadband connectivity such as mobile tablets (e.g., Apple's iPad), electronic readers (e.g., Amazon's Kindle), notebook, media players, laptops and similar devices that are capable of storing or reproducing multimedia.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player; wherein the SSID can be user settable in order to make it user-specific in the same way it is set by the user when a WLAN is configured.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the communication interface of first type is wireless-based and can he implemented based on different flavours of WiFi (i.e., IEEE 802.11) technology including WiFi-direct and the said electronic device of first type will operate a WiFi access point on the said the communication interface of first type.
lit is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the communication interface of first type is wireless-based and can be implemented based on Bluetooth, Wireless USH, Wireless 1-ID, Wireless IFIDMI standard or the like.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and iS renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the communication interface of second type can be wired and implemented based on HDMI/DVI, SCART, IEEE 1394 (i.e., F'irewire/iLink/Lynx), RCA-connectors-based Composite/component video, Display Port, Serial Digital Interface, S-Video, VGA and its extensions, or the like.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the said unique zeroconf identifier being assigned to the connected mobile computing device on the communication interface of first type is an 1P address.
S Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the assigned IP address takes a random address (on network 169.254/16 in case of IPv4, or on prefix MANET_INIT[ALPREFIX in case of IPvó) and this is analogous to the way the Autonet allocations are performed as proposed in the Zero configuration (zeroconf) working group.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the said electronic device of first type limits the number of mobile computing devices or media capturing devices such as a security-camera/web-cam being connected to maintain an active session to it at any point in time.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as
I
a television set or multimedia player, wherein the said electronic device of first type accepts connection requests being originating from one or plurality of mobile computing devices and this procedure is governed by the priority-based access scheme comprising the steps of: S i) each mobile computing device needs to he assigned a priority number based on who carries that mobile computing device (e.g., whether the main house-holder/manager); ii) initial connection request of a mobile computing device contains the said priority number; and, iii) the electronic device of first type analysing each connection request and connect a device based on the priority order; wherein in case a low priority device was already attached to the electronic device of first type, the connection will be replaced by a high-priority device.
it is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the displayed content on the external visual display is the exact or intended instantaneous screen shot/content of the mobile computing device.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as
I
a television set or multimedia player, wherein the mobile computing device is responsible for capturing/getting the exact or intended instantaneous screen shot/content with/without the associated audio content using relevant software and/or hardware components, and transmitting it to the said electronic device of first type in real time.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on an external visual display apparatus such as a television set or multimedia player; wherein the mobile computing device captures the multimedia content and compress it before being transmitted to the electronic device of first type; wherein the compression can be based on MPEG-4 that contains both audio and video and can help to improve the necessary audio-to-video synchronisation.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on an external visual display apparatus such as a television set or multimedia player, wherein the mobile computing device has the necessary software and/or hardware components to tap into its graphics card andior sound card for constantly capturing the instantaneous multimedia content being or intended to be played-back.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hacks it on an external visual display apparatus such as a television set or multimedia player; wherein the said mobile computing S device employs screen scraping and multimedia redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus or multimedia player together with playing-back the associated audio; wherein the mobile computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side multimedia elements are sent in their native fonnats together with the respective timestamp information down to the electronic device of first type, which in turn renders/plays-back the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizing the associated audio, if any, with the video and this requires: i) the electronic device of first type has the technical capability and hardware
specifications to render the multimedia; and,
ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full synchronised audio and video experience.
I
Preferably the said electronic device of first typc that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on art external visual display apparatus such as a television set or multimedia player; wherein the said mobile computing S device that is equipped with an appropriate hardware, software or combination of both performs screen-casting that simply record the desktop screen activity to standard video files and output it, wherein it runs silently in the background, records a movie of the desktop screen, writes it constantly into a video framebuffer and flushes the framebuffer to the electronic device of first type instantaneously (i.e., continuously push its screen activity to the client) and this takes place at a framebuffer-level or Graphics Device Interface (GD!) level; wherein the mobile computing device functions as a server that sends screen updates to the electronic device of first type which in turn functions as a client.
IS
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on an external visual display apparatus such as a television set or multimedia player, wherein in ease RFB is implemented between the said mobile computing device and the said electronic device of first type, the mobile client device operates in the capacity of a REB server and the electronic device of first type operates in the capacity of a RFB client; wherein the RFB server produces a stream of rectangles representing screen
I
data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the framehuffer level so that they appear in thc correct place on the screen.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein in case RFB is implemented between the said mobile computing device and the said electronic device of first type the RFB server can send the compressed/encoded screen data to the RFB client and the compressionlencoding technique to be employed can he agreed at the time of initial negotiation.
It is preferred that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein in case RFB is implemented between the said mobile computing device and the said electronic device of first type the RFB server sends updates only when changes (delta) to the screen are required, and only sends the areas of the screen which have changed and the receiving side (RF'B client) reconstructs the whole image from the delta.
I
Preferably the said electronic device of first typc that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on art external visual display apparatus such as a television set or multimedia player, wherein in ease RFB is implemented S between the said mobile computing device and the said electronic device of first type, thc mobile client device has a touch screen and the user wants to use its screen as a GUT expecting user inputs only and the external display apparatus as the only output screen (i.e., when the RFB server is not associated with an existing physical screen), the RFB server will run a "virtual framebuffer" and the remote RFIJ client has the only physical screen showing the contents of that framebuffer.
It is prefen'ed that the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on an external visual display apparatus such as a television set or multimedia player, wherein in ease RFB is implemented between the said mobile computing device and the said electronic device of first type, the RFB server can send the eneiypted screen data using the High-bandwidth Digital Content Protection (HDCP) protocol for copy protection to the RFB client and the encryption/security technique to be employed can be agreed at the time of initial negotiation.
Preferably the said electronic device of first type that receives multimedia content from a mobile computing device in real-time and
I
renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player, wherein the said media processing unit should have the relevant associated electronic circuitries to output the received multimedia in the appropriate format as demanded by different S varieties of communication interfaces of second type and to adapt the hit rate, resolution, frame rate and codec format being received in order to match the capabilities of the sink external display apparatus or media players, and this requires that the said media processing unit comprises one or plurality of the following functionalities: a) Transizing or Video sealing functionality for converting video signals from one size or resolution to another with or without de- interlacing. E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -Le, this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target devic&s screen size and colour availability; h) Transcoding -multi-formatl-codec/.-compression support for simultaneous encoding and decoding of a wide variety of codee/compression formats; c) Transrating -in this process the media content is coded to a lower bit-rate without changing video formats;
I
d) digital to analog conversion depending on the communication interface of second type being used and characteristics of the end display apparatus; e) audio synchroniser for the purpose of ensuring audio-to-video S synchronisation 1) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video Interface (GV1F), or Unified Display Interface (UDJ) connections; g) error concealment technique using either spatial or temporal interpolation; h) extra functionalities to amplify the audio signal; and, i) 2D to 3D conversion or vice-versa; wherein the signal needs to be decoded first before one or plurality the above-mentioned processes are applied.
It is preferred that the said electronic device of first type that is connected to an external display apparatus such as a television set or an external multimedia player gets audio signal from the television/nuiltimedia-player via
I
a communication interface of third type and makes it available, wherein one or plurality of wireless headphones can connect to the said electronic device of first type to stream the audio via a communication interface of fourth type and playback in real-time for the purpose of silently watching the television.
S
Preferably the said electronic device of first type that is connected to an external display apparatus such as a television set or an external multimedia player gets audio signal from the television via a communication interface of third type and makes it available, wherein the communication interface of third type can be any wired-based audio interface and inter-connect, whereas the said communication interface of fourth type can be Bluctooth.
It is preferred that the said electronic device of first type that is connected to an external display apparatus such as a television set or an external multimedia player, wherein one or plurality of multimedia capturing devices such as a wcbcam, security camera, camcorder or similar device can be connected via the said communication interface first type to the said electronic device of first type for the purpose of displaying the instantaneous captured content on the said external display apparatus in real-time.
Preferably the said electronic device of first type to an external display apparatus such as a television set, wherein the said electronic device of first type has ports such as USB that can be used by a user to connect an external flash memory and to upload one or plurality of still images to the electronic
I
device of first type for the purpose of displaying one or plurality of the uploaded still images continuously (like Slideshows) on the external display apparatus and this is activated under two circumstances: i) automatically after a certain time-out when the external display apparatus is in its power-saving mode; or, ii) manually when the electronic device of first type has an external electronic button/switch and pressing/activating this button/switch will trigger the said Slidesbow operation at the discretion of the user.
Wherein in either of the above case, the user has the option to set as to how long each still image of a Slideshow needs to be displayed.
It is preferred that the said electronic device of first type to an external display apparatus such as a television set, wherein the said electronic device of first type can be augmented to perfonu STB, AV receiver and similar device functionalities and thus will perform additional functionalities as diverse as Web surfing, e-mail, games, video phones, voice over IP (V0IP) phones, video-on-demand (VoID), security, and digital media adapter for Internet streaming.
According to the second aspect of the present invention there is provided an electronic device of second type that functions as an external graphics accelerator card and/or an external sound card for capturing the instantaneous multimedia content being played back on a mobile computing device and transmitting it to an external display apparatus such as a television set or multimedia player, and the said electronic device of second type comprising: a) Graphic processing unit (GPU); b) BIOS or firmware; S e) Memory; d) a wireless transceiver providing a communication interface of first type to communicate with the external display apparatus or multimedia player; e) a wired communication interface of fifth type to get connected to a mobile computing device; wherein the said electronic device of second type can either tap into the graphiclsound card of the mobile computing device or employ screen-scraping together with multimedia redirection for the purpose of capturing the instantaneous multimedia content being or intended to be played back on the mobile computing device and transmit the multimedia content constantly and thus offloading this work from the CPU and system RAM of the mobile computing device.
It is preferred that the said electronic device of second type that is connected to the mobile computing device of first type; wherein it applies an appropriate compression andlor one or plurality of other multimedia processing operations to the captured multimedia before transmitting.
Preferably the said electronic device of second type that is connected to the mobile computing device of first type; wherein the communication interface of fifth type can be a mini-USB or based on l-IDM1 1.4 Type D micro-connector.
S
It is preferred that the said electronic device of second type that is connected to the mobile computing device of first type; wherein the electronic device of second type is connected to the external display apparatus such as a television set or multimedia player via the electronic device of first type.
According to the third aspect of the present invention there is provided a complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player; and the said complete framework comprises: iS i) a mobile computing device being installed and running a purpose-built host software module; ii) electronic device of first type; wherein the said purpose-built host software module provides appropriate user interface and allows the user to connect the mobile computing device to the said electronic device of first type for the purpose of constantly projecting the instantaneous screen display of the mobile computing device along with the associated audio content, if any, in real time on to the neighbouring external display apparatus or multimedia player.
It is preferred that the said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player; wherein thc said purpose-built host software module is responsible for: S i) associating the priority of the user to the mobile computing device; ii) establishes thc necessary session-related logical connection with the electronic device of first type when the mobile computing device maintains a successful physical connection; iii) capturing instantaneous multimedia content either by tapping into the internal or external graphic and/or sound cards, and applying one or more media processing operations, if needed; and, iv) performing packetization before transporting the captured media streams in real-time to the electronic device of first type.
Preferably the said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player; wherein the said purpose-built host software module performs screen-scraping and multimedia redirection.
It is preferred that the said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player; wherein the said purpose-built host software module controls the electronic device of second type in case it is connected to the mobile computing device for the purpose of capturing and transporting the instantaneous media streams constantly.
According to the fourth aspect of the present invention there is provided a an electronic system on a chip that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player or multimedia player, and the system on a chip comprising: i) a bus, ii) a peripheral controller being coupled to said bus for connecting a multitude of 110 peripheral devices; iii) one or plurality of microprocessors being coupled to said bus and said microprocessors configured to perform the required signal processing associated with the multimedia traffic and to provide control functions of said communication device; iv) a communication controller being coupled to said bus for connecting to one or plurality of communication interfaces; and, v) a memory that is coupled to said bus into which a plurality of instructions are loaded, wherein the said system on a chip composes a purposeful system on-the-fly by getting any mobile computing device to join while handling operations such as: a) operating a WLAN access point on the communication interface of first type with a user configurable SSID; b) initialing a DIICP functionality; c) when a mobile computing device attempts to associate with the said S WLAN access point, assigning a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; According to the sixth aspect of the present invention there is provided a method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; and the said method comprises the following steps for the purpose of configuring the electronic device of first type to operate a WLAN access point on the communication interface of first type and this initial configuration procedure comprising the steps of: i) setting an appropriate SSID for the said WLAN access point that can uniquely identify the userfhouseholdioffice where it is to be used; ii) getting the WLAN access point to either advertise/broadcast its SS1D or hide it; iii) setting encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials for the purpose of restricting connections of any mobile computing device or any other media capturing device such as a security-camera/web-cam to the said WLAN access point; wherein the user can connect the electronic device of first type to a computer using an appropriate interface/port (e.g., RJ-45 cable) in order to perform the S initial WLAN configuration in the same way a normal WLAN is configured preferably using a web browser and once thc said initial configuration is complete, the user can connect the electronic device of first type to the external display apparatus or a multimedia player via a communication interface of second type and this will make the electronic device of first type ready for its intended usage.
It is preferred that the said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; and the said method comprises physical connection establishment procedure for the purpose of establishing a physical connection between the mobile computing device and the electronic device of first type, and the said physical connection establishment procedure comprising the steps of: i) getting the mobile computing device to look for a WLAN access point with a specific SSBJ; ii) once found, prompting the user of a mobile computing device to enter the necessary encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials assigned to the user. On the other hand, in case, the specific SSID is not found mainly because the network does not advertise/broadcast the SSID, prompting the user to manually enter the SSID and the profile id, if any, and to try reconnecting to the WLAN access point belonging to the electronic device of first type; S iii) when the login and/or encryption credentials are received by the electronic device of first type, it needs to authenticate the mobile computing device that attempts to connect; iv) when the authentication succeeds, the electronic device of first type needs to assign a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; wherein once the physical channel is established, the mobile computing device is ready to initiate any communication session with the electronic device of first type.
Preferably the said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; wherein the said method comprises the following steps for the purpose of establishing a multimedia session between the mobile computing device and the electronic device of first type: i) when the user of a mobile computing device intends to initiate multimedia projection on to an external sink device, the mobile computing device will initiate the physical connection establishment procedure in order to connect to thc electronic device of first type; ii) once the physical connection establishment is successful, the mobile computing device will establish a remote framebuffer at the electronic device S of first type and negotiates the compression and/or encryption methodologies to be adopted; iii) operating the mobile client device in the capacity of a RFB server and the electronic device of first type in the capacity of a RFB client; wherein the RFB server produces a stream of rectangles representing screen data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the frarnehuffer level so that they appear in the correct place on the screen; iv) on receiving the multimedia content, the electronic device of first type applies one or plurality of the following multimedia processing operations in order to render the multimedia content on the external display apparatus or multimedia player: i) Transizing or Video scaling functionality for converting video signals from one size or resolution to another with or without de- interlacing. Eg., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -i.e., this low resolution image might bc transcoded to a higher resolution image with high colour in order to better fit the target devic&s screen size and colour availability; ii) Transcoding -multi-format/-coded -compression support for simultaneous encoding and decoding of a wide variety of S codec/compression formats; iii) Transrating -in this process the media content is coded to a lower bit-rate without changing video formats; iv) digital to analog conversion depending on the communication interface of second type being used and characteristics of the end display apparatus; v) audio synchroniser for the purpose of ensuring audio-to-video synchronisation vi) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video lnterface (GV1F), or Unified Display Interface (UDI) connections; vii) error concealment technique using either spatial or temporal interpolation; and, viii) 2D to 3D conversion or vice-versa; wherein the application of one or plurality of the above-mentioned multimedia processing operations depends on the received media type, format, resolution, bit-rates, frame rates, colour-depths, aspect ratio and the like, and the capabilities of the end sink device (i.e., external display S apparatus or multimedia player).
It is preferred that the said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; wherein the said method employs screen-scraping along with multimedia redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus along with playing back the associated audio; wherein the mobile computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side multimedia elements are sent in their native formats together with the respective timestamp information down to the elecnDnic device of first type, which in turn renders the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizes the associated audio, if any, with the video and this requires: i) the electronic device of first type has the technical capability and hardware
specifications to render the multimedia; and,
ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full S synchronised audio and video experience.
Preferably the said method that helps a user project the multimedia content on to an external display apparatus such as a television set or multimedia player via an electronic device of first type; wherein the said method allows the said electronic device of first type that is connected to a television set or an external multimedia player to get audio signal from the television/multimedia-player via a communication interface of third type and to make it available, wherein one or plurality of wireless headphones can connect to the said electronic device of first type to stream the audio via a communication interface of fourth type and playback in real-time for the purpose of silently watching the television.
It is preferred that the said method that helps a user project the multimedia content on to an external display apparatus such as a television set or multimedia player via an electronic device of first type, wherein the said method allows any multimedia capturing device like webcam or security camera to be connected via the said communication interface first type to the said electronic device of first type for the purpose of displaying the instantaneous captured content on the external display apparatus in real-time.
It is preferred that the said method that helps a user project the multimedia content on to an external display apparatus such as a television set or multimedia player via an electronic device of first type, wherein the said method allows the user to upload one or plurality of still images/pictures with/without any associated audio file via USB ports to the electronic device of first type for the purpose of displaying one or plurality of still images continuously (like Slidesbows) on the external display apparatus and this is activated under two circumstances: i) automatically after a certain time-out when the external display apparatus is in its power-saving mode; or, ii) manually when the electronic device of first type has an external electronic button/switch and pressing/activating this button/switch will trigger the said Slideshow operation at the discretion of the user.
Wherein in either of the above case, the user has the option to set as to bow long each still image of a Slideshow needs to be displayed.
Description of the Drawing
Non-limited and non-exhaustive embodiments are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.
For a better understanding of the present invention, reference will he made to the following detailed description of the invention, which is to be read in association with the accompanying drawings, wherein: Figure 1 is an exemplary illustration of a complete framework that helps a user project the multimedia content of a mobile computing device or any media capturing device such as a security-camera/web-cam to an external display apparatus or multimedia player using the electronic device of first type according to one embodiment of the present invention.
Figure 2 shows the exemplary front view of the enhanced version of the electronic device of first type to be used for projecting the multimedia content of a mobile computing device or any media capturing device such as a security-camera/web-cam to an external display apparatus or multimedia iS player as proposed according to one embodiment of the present invention Figure 3 is an illustrative schematic representation of the multimedia processing operations taking place in the electronic device of first type according to one embodiment of the present invention.
Figure 4 is an exemplary schematic block diagram illustrating the internal components of the electronic device of the first type being implemented as a System on Chip (SoC) operating on a generic microprocessor or a graphics processing unit or a combination of both according to one embodiment of the present invention Figure 5 is an exemplary illustration of the various components of the software stack of a generic microprocessor or a graphical processing unit or a combination of both according to one another embodiment of the present invention.
S Figure 6 exemplarily illustrates the user-specific initial configuration procedure that is optionally required for the WLAN access point belonging to the electronic device of first type to operate according to one embodiment of the present invention.
Figure 7 exemplarily shows the physical connection establishment procedure involved when a mobile computing device or any media capturing device such as a security-camera/web-earn and the like attempts to establish a physical connection with the electronic device of first type by getting itself associated with the WLAN access point being operated by the electronic device of first type according to one embodiment of the present invention.
Figure 8 is an exemplary illustration of the session-related logical connection establishment procedure as required by a mobile computing device or any media capturing device such as a security-camera/web-cam when it attempts to project its multimedia content to an external sink device via the electronic device of first type according to one another embodiment of the present invention.
Figure 9 is an exemplary illustration of the session-related logical connection establishment procedure as required by an electronic device of first type when a mobile computing device or any media capturing device such as a security-camera/web-cam attempts to project its multimedia content to an extcrnal sink device via the electronic device of first type according to one another embodiment of the present invention.
Figure 10 exemplarily illustratcs thc initial handshake operation required S between any mobile computing device or any media capturing device such as a security-camera/web-cam and the electronic device of first type in priory to projccting any multimedia content of a mobile computing device to an external sink device according to one another embodiment of the present invention. l0
The figures are provided for ease of explanation of the basic proposals of the present invention only; the extensions of the figures with reference to number, position, relationship and dimensions of the parts to form the prefelTed embodiment will be explained or will be within the skill of the art after the following proposals/findings of the present invention have been read and understood.
Description of Specific Embodiments
This patent uses the term mobile computing device to refer to such devices as mobilc phones, smart phones, personal digital assistants (PDAs) or portable mobile devices preferably having broadband connectivity such as mobile tablets (e.g., Apple's iPad), electronic readers (e.g., Amazon's Kindle), notebooks, media players, laptops and similar devices that are capable of storing or reproducing multimedia. The current invention further uses the term external display devicelapparatus to collectively refer to any television set, monitor apparatus, a projector or a similar device. This external display apparatus/device or an external multiplayer is sometimes referred to as an S external sink device throughout this document.
Multimedia capable mobile computing devices offer a wide range of applications, since video content can be used in conjunction with voice and the internet for video conferencing, distance learning, business dealings and other interactive services, as well as for TV, Video-on-Demand (VoD), gaming and other forms of entertainment. Given that these mobile computing handheld devices have a small form-factor and thus having a small screen display, according to one embodiment of the present invention, a complete framework, methodologies and system are proposed to enable the projection of multimedia content from any mobile computing device to a neighbouring external display apparatus or multimedia player (streaming of video and audio wirelessly from any mobile computing device to a TV wirelessly) in real-time for an enhanced user experience.
The enhanced user experience being brought in by the present invention is very challenging in reality at the moment without requiring special purpose-built mobile computing device (e.g., being powered by a smait Intel® CoreTM processor) and/or an external sink device perhaps by the same manufacturer.
This is because it is very challenging for any layman to bring an existing mobile computing device made by one manufacture in direct communication with any existing external display apparatus such as a TV, monitor apparatus or a projector or any external multimedia player (collectively known as an external sink device) made by a different manufacture mainly due to the fact S that they do not sometimes share a common interface (especially wireless) or language. In order to address this, a complete framework, methodologies and system being proposed in the present invention dcvise a new apparatus called the electronic device of first type. The purpose of this new electronic device of first type is to play an intermediatory role (i.e., intermediate) between a mobile computing device and an external sink device from completely different manufactures. In terms of the capabilities, the present invention requires that the mobile computing device should have sufficient processing/memory capabilities to run a purpose-built host software module whereas the considered external display can be really old. By supporting a plethora of necessary interfaces, the electronic device of first type can communicate with any existing/future mobile computing device and external display apparatus (or any external sink device in general) from different manufacturers. This feature enables the electronic device of first type to stream any multimedia content from any existing mobile computing device via one communication interface, apply none/one or more multimedia processing operations before rendering or transporting it in real-time to the external sink device via another communication interface, in this respect, the electronic device of first type is thick in terms of the processing power and memory footprint it has. The electronic device of first type can be considered to be a collection of a number of small functional blocks or modules, with each module performing a well defined function. A module can be composed of pure hardware or pure software or some combination of the two as it will S be seen later.
For the present invention to work it is assumed that any mobile computing device being considered has to satisfy either of the following: i) has the necessary generic-purpose hardware built-in and is capable to have a purpose-built host software module installed and to run it for the purpose of establishing the necessary multimedia session related logical connection with the electronic device of first type after an initial handshake to agree on the codee, media type and encryption to be employed (logical connection is possible only when the physical channel is established after a successful authentication as it will be described later) and subsequently capturing, sampling, digitising, packetising and then transporting the multimedia content being or intended to be played-back on the mobile computing device to the electronic device of first type in real-time; or, ii) is capable enough to be mounted with the purpose-built hardware unit and installed and run a purpose-built host software module; wherein the purpose of the purpose-built hardware unit which is preferably powered by the mobile computing device itself is to capture, sample, digitise, packetise and then transport the multimedia content being or intended to be played-back on the mobile computing device to the electronic device of first type in real-time while the purpose of the said purpose-built host software module is to establish and maintain the necessary logical connection with the electronic device of first type and the purpose-built hardware unit to coordinate the S media capturing/processing/transport operations -the purpose-built hardware unit is called the electronic device of second type as it will he seen later.
According to one another embodiment of the present invention, the electronic device of first type is capable enough to enable an existing external display apparatus such as a television set to function as a wall-art decorating the place where it is mounted, or a digital photo-frame displaying a slideshow of user uploaded photos/images. This feature also enables the electronic device of first type to render associated background music to the appropriate sink device. This embodiment of the present invention however does not necessarily expect any mobile computing device to he connected to an external display apparatus when the latter is used in the capacity of a digital photo-frame.
According to another embodiment of the present invention, the electronic device of first type can enable one or more users to enjoy watching a television silently. This is possibly because the electronic device of first type captures constantly the instantaneous audio stream from an external television set or a multimedia player to which it is connected, and transmits it in real-time to one or more headphones being worn by the users. In other words, the electronic device of first type streams audio from an external multimedia player (e.g., a television set) and renders it on to one or plurality of external headphones. Tf needed, the electronic device of first type can temporarily buffer the captured instantaneous audio stream, apply one or plurality of media processing operations such as transcoding/uansrating, re-formating, amplifying and the like before letting one or more headphones stream the audio stream in real-time.
According to one another embodiment of the present invention, the electronic device of first type provides one or plurality of necessary interfaces in order to connect any existing media capturing devices such as a security-camera, camcorder, web-cam and the like to an external display apparatus for an immediate play-back.
According to another embodiment of the present invention, the idea is to complement the funetionalities of the existing Set-Top Box (STB) and/or Audio-Video (AV) receiver with the features as proposed in the present invention.
Figure 1 is an exemplary illustration of a complete framework 100 that helps a user project an instantaneous multimedia content of a mobile computing device 150 or any media capturing device such as a security-camera/web-cam 160/1 70 to an external display apparatus 120/1301104 or a multimedia player constantly using the electronic device of first type 140 according to one embodiment of the present invention. As mentioned before, the external display apparatus can he a television (TV) set 130, a monitor apparatus 120 or a projector 104. Multimedia capturing devices such as a security-camera/web-cam 160, a camcorder 170 and similar device can be connected to the electronic device of first type 140 for an immediate play-back of the instantaneously captured content on the said external display apparatus 104/120/130 in real-time. Devices 150, 160 and 170 should support the communication interface of first type 180 and the necessary software/hardware/firmware components for any of them to communicate with the electronic device of first type 140. The communication between the electronic device of first type and any external display apparatus 104/120/130 is possible by employing the communication interface of second type 116. in short, the electronic device of first type is used for the purpose of streaming an instantaneous multimedia content from 150/160/170 constantly via a communication interface of first type and renders it on to 1041120/130 in real-time via a communication interface of second type.
The said electronic device of first type 140 that is connected to an external display apparatus such as a television set 130 or an external multimedia player is also capable to capture constantly the instantaneous audio stream from an external display apparatus 130, and transmits it in real-time to one or more headphones 144 being worn by the users. This feature of the electronic device of first type 140 can enable one or more users to enjoy watching a television silently. The electronic device of first type 140 streams/gets the audio signal from the television/multimedia-player 130 via a communication interface of third type 124 and makes it available (i.e., or preferably renders it on) to one or more headphones 144 via a communication interface of fourth type 136. If needed, the electronic device of first type 140 can temporarily buffer the captured instantaneous audio stream, apply one or plurality of media processing operations such as transcoding/transrating, re-formating, amplifying and the like before letting one or more headphones 144 stream the audio stream in real-time.
Given that the said electronic device of first type 140 needs to communicate wirelessly with any mobile computing device 150, both the devices should be equipped with the wireless transceiver providing a communication interface of first type 180. Similarly, in order to communicate with an external display apparatus 104/120/130 or an external multimedia player, both the electronic device first type 140 and the external display apparatus 104/120/130 need to support the communication interface of second type 116.
According to one preferred embodiment of the present invention, the communication interface of first type 180 is wireless-based and can be implemented based on different flavours of WiFi (i.e., IEEE 802.11) technology including WiFi-direct. According to another preferred embodiment of the present invention, the said electronic device of first type will operate a WiFi access point. On the hand, according to one another embodiment of the present invention, the communication interface of first type 180 can be implemented based on Bluetooth, Wireless USB, Wireless HD, Wireless HDM1 standard or the like. The communication interface of second type 116 can be wired and implemented based on i-IDMIJDVT, S SCART, IEEE 1394 (i.e., Firewire/iLink/Lynx), RCA-connectors-based Composite/component video, Display Port, Serial Digital interface, S-Video, VGA and its extensions, or the like. According to one preferred embodiment of the present invention, the communication interface of third type 124 can be a wired-based audio interface and interconnect -typical examples being Analog line level audio input, Stereo RCA audio port, Digital coaxial S/PDIF, Dolby® digital or DTS audio ports or the like, whereas the said communication interface of fourth type 136 can be Bluetooth. The LCD panel 260 can be used for the purpose of secure Bluetooth pairing, if needed.
In order to simplify the communication setup between the electronic device of first type 140 and any mobile computing device 150, according to one preferred embodiment of the present invention the electronic device of first type 140 operates a Wireless LAN access point on the communication interface of first type 180 together with the Dynamic Host Configuration Protocol (DFICP). The Service Set Identifier (SSID) of the access point (AP) can be optionally user settable. This mechanism ensures that the initial communication setup is as easy as that is applicable to connecting a laptop to any WLAN AP. Accordingly, any mobile computing device 150 will operate as a slave, detect the SSID. if it is broadcast, and connect to the WLAN AP bclonging to the electronic device of first type 140 via the communication interface of first type after a successful authentication. Once connected, the DHCP functionality of 140 will assign a unique zeroeonf identifier to the S connected mobile computing device 150 pertaining to the communication interface of first type 180. Connecting any other media capturing device such security-camera/web-cam 160/170 involves a similar sequence of operations to those performed at a similar situation by 150.
The electronic device of first type 140 operates a WLAN AP only when the communication interface of first type is based on IEEE 802.1 1. In this case, associating an IEEE 802.11 enabled mobile computing device 150 to the AP is very straight forward. On the other hand, when the communication interface of first type 180 is non-IEEE 802.11-based, making the physical connection iS between a mobile computing device 150 and 140 should follow the procedures as stipulated by the actual communication interface being employed. Once the physical connection is established, if the user of 150 would like to project the multimedia content on 1301120/104, a logical connection pertaining to the multimedia session needs to be established between the mobile computing device 150 and the electronic device of first type 140. This involves an initial handshake namely to agree on the media format/compression to be used/employed, and encryption/security aspects.
This logical connection establishment in the mobile computing device 150 is handled by the purpose-built host software module. Hence, the purpose-built host software module is responsible for constantly capturing the instantaneous media content by either tapping into the internal/external graphic/sound cards of the mobile computing device 150 or employing pure software-based S opcrations such as screen-scraping with/without multimedia re-direction technique or screen-casting, processing the captured media content and transporting to the electronic device of first type i40.
The purpose-built host software module also provides the necessary user interfaces and runs an application for the user to trigger the operation to project the multimedia content on any external display apparatus 104/120/130 (this application is known from hcrc onwards as the Project2TV-application).
If the mobile computing device 150 has the gesture/movement detection mechanism equipped, the user can initiate/pause/terminate the Project2TV-application explicitly by pressing a specific key on the physical keyboard or touch screen or through gestures/movements. Also, in case the mobile computing device 150 employs an automatic voice recognition technique, the purpose-built host software module can accept voice commands as well and act upon them to initiate/pause/terminate the Project2TV-applieation.
In short the purpose-built host software module consists of two main functional modules: i) a control module to deal with the logical connection establishment with the electronic device of first type 140; and, ii) media handling module either to liaise with the other internal/external hardware units of the mobile computing device 150 or to employ software-based media capturing techniques such as screen-casting or screen-scraping with/without multimedia re-direction in order to capture the instantaneous multimedia content, to process the media, if needed, for the purpose of sampling, dligitising, packetising/transcoding/transrating/transizing and then to transport the multimedia content to the electronic device of first type 140.
Connecting a mobile computing device 150 to the electronic device 140 via an existing general purpose standalone WLAN AP is deliberately avoided due to increased latency concerns. If connected via a standalone WLAN AP, the captured media from 150 will be transported to 140 via two hops and this will introduce unnecessary interference and subsequently lead to increased latency and packet loss. On the other hand, the complete framework, methodology and the system as proposed by the present invention enable the transport of captured media stream from 150/160/170 to 140 via a single hop -thus leading to enhanced user perfonnance.
Figure 2 shows the exemplary front view of the enhanced version of the electronic device of first type 140. It has at least two wireless transceivers 230 and 240 -230 providing the communication interface of first type 180 and 240 providing the communication interface of fourth type 136. Also, the electronic device of first type 140 accepts a variety of multimedia inputs via a set of interfaces 210. The input interface 210 includes one or plurality of the standard multimedia input interfaces -namely the communication interface of third type 124, Universal Serial Bus (USB) 2.0/3.0 ports, HDMI/DVI, IEEE 1394, RS 232, home network interfaces that complies with RJ45 10/100 Ethernet port, WiFi, HomePNA or the ITU-T (i.hn standard and the like, terrestrial/satellite antenna inputs, and the associated electronic circuitries for the purpose of converting/decoding/processing/encoding the received media.
It has a set of video/audio output interfaces as indicated by 220. The output interface 220 supports mainly the communication interface of second type 116 and thus includes one or plurality of the standard wired multimedia output interfaces such as IIDMIIDVI, SCART, IEEE 1394 (i.e., Firewire/iLinklLynx), RCA-connectors-based Composite/component video, Display Port, Serial Digital Interface, S-Video, VGA and its extensions, or the like and Stereo RCA audio port, Digital coaxial S/PDIF, Dolby® digital, Dolby® True HD or DTS audio ports, Audio out (L/R), 6 discrete 5.1 cli Audio out, Optical Audio Output or the like.
According to one preferred embodiment of the present invention, the input/output interface 2 10/220 the electronic device of first type 140 can have the Internet connection and this serves for 2 main purposes: a) stream and playback multimedia contents from an external media server via the internet; and, b) letting the electronic device of first type 140 function as a media server making the captured media content of devices 150/160/170 being connected to 140 available via the Internet As depicted in Figure 2, the electronic device of first type 140 includes a variety of electronic switches/buttons for various control purposes. The switch/button 280 is used to turn onfoff 140, while the switch 284 is used to activate/deactivate the WLAN AP that the electronic device of first typel40 operates on the communication interface of first type 180. The switch/button 290 can be used to manually activate/deactivate the wall-ait or digital photo-frame application of the electronic device of first type 140 that allows the external display apparatus 104/120/130 to display one or plurality of user-uploaded still images continuously (like Slideshows) for a user-set duration.
According to one embodiment of the present invention, a user can upload one or plurality of still images/pictures with/without any associated audio file via USB ports to theelectronic device of first type l4Oforthepurposeof displaying one or plurality of still images continuously (like Slideshows) on the external display apparatus 104/120/130 and this is activated under two circumstances: i) automatically after a certain time-out when the external display apparatus 104/120/130 is in its power-saving mode; or, ii) manually when the electronic device of first type 140 has an external electronic button/switch 290 and pressing/activating this button/switch will trigger the said Slideshow operation at the discretion of the user.
In either of the above case, the user has the option to set as to how long each still image of a Slideshow needs to be displayed.
According to one embodiment of the present invention, the basic functionalities of the electmnic device of first type 140 can be augmented so that it can function as a standalone media player. In order to facilitate such an operation, the electronic device of first type 140 can include a DVD/Blu-Ray disc compartment 270 and the associated player circuitry that can accept a variety of playback formats including CD-Audio, CD-DA, CD-It, CD-RW, DVD÷R, DVD-i-RW, DVD-R, DVD-R DL, DVD-ROM, DVD-RW, DivX® Playback, JPEG Viewer, MP3, Blu-Ray and the like. It also includes a standard player keys as depicted by 250 and an LCD display panel 260. Also, the electronic device of first type 140 has its own remote controller with a variety of user-setting/control options that is connected via Infrared (1R) or any short-range wireless technology for -e.g., Bluetootb, ZigBee, DECT and the like.
The 5511) of the AR that the electronic device of first type 140 operates onthe commuthcfioninterficeofflrsttypecanbeusersettableinordertomakeit uniqueinthesamewayitissetbytheuserwhenastandalonegeneric-purpose WLAN AP is configured. When any mobile computing device 150 or any media capturing device 160/170 establishes a physical connection with when it associates with 140, the electronic device of first type 140 assigns a unique zeroconf identifier to 150/160(170 on the communication interface of first type. According to one embodiment of the present invention, this unique zeroconf identifier can be an JP address. According to another embodiment, the assigned IP address takes a random address (on network 169.254/16 in case of IPv4, or on prefix MANET_INITI AL_PREFIX in case of IPv6) and this is analogous to the way the Autonet allocations are performed as proposed by the Zero configuration (zeroeonf) working group.
Although as many number of devices 150/160/170 as required can maintain a physical connection with the electronic device of first type 140, the latter limits the number of devices 150/160/170 that can maintain an active session connection (i.e., logical) with 140 at any moment. According to one embodiment of the present invention, the number of devices 150/160/170 that can maintain an active session connection (i.e., logical) with 140 at any point in time is limited to one.
According to another preferred embodiment of the present invention, the said electronic device of first type 140 subjects the logical connection requests (called Session Request as it will be seen later) being originating from one or plurality of mobile computing devices 150 after the said electronic device of first type 140 and 150 have established the necessary physical connection successfully to a priority-based access scheme comprising the steps of: i) the purpose-built host software module being installed and run on each mobile computing device 140 needs to be assigned a priority number based on who carries that mobile computing device 150 (e.g., whether the main house-holder/manager). The purpose-built host software module provides the necessary user interface for the user to enter the priority number being associated with a given mobile computing device 150; ii) initial session-related logical connection request of a mobile computing device 150 contains the said priority number session-related logical connection request originates from a given mobile computing device 150 when its user wants to invocate the Project2TV-application. When this happens, the purpose-built host software module will exchange a special signalling message called the Session Request with the electronic device of first type 140. The purpose-built host software module will liaise/interact or perform the necessary handshakes with 140 to handle the session-related logical connection establishment as it will he described in relation to Figure 10; and, iii) the electronic device of first type 140 analyses each Session Request and connects a device 150 based on the priority order.
According to this priority-based access scheme, in ease a low priority device was already attached to the electronic device of first type 140, the existing logical session-related connection will be replaced by that of a high-priority device 150.
As mentioned before, the electronic device of first type 140 helps any user to project the exact or intended instantaneous screen content of any mobile computing device 150 on to any external display apparatus 104/120/130. The purpose-built host software module being installed and run on the mobile computing device 150 is responsible for capturing/getting the exact or intended instantaneous screen shot/content with/without the associated audio content and transmitting it to the said electronic device of first type 140 in real time. According to one embodiment of the present invention, one of the main functionalities of the purpose-built host software module being installed and run on the mobile computing device 150 is to scrape whatever graphical elements being painted to the "screen" of a mobile computing device 150 and send down to the electronic device of first type 140 for 140 to render the graphical elements on the external display apparatus 104/120/130. According to one another embodiment of the present invention, the purpose-built host software module applies one or plurality of multimedia processing operations to the captured media and compress it before being transmitted to the electronic device of first type 140. The compression can be based on MPEG-4 when the captured multimedia content includes audio and video streams and such a usc of MPEU-4 codec can improve the audio-to-video synchronisation.
According to one embodiment of the present invention, the purpose-built host software module taps into its graphics card and/or sound card for constantly capturing the instantaneous multimedia content being or intended to bc played-back on the mobile computing device 150. This is generally possible if the graphics card and/or sound card of the mobile computing device 150 expose relevant APIs.
According to another preferred embodiment of the present invention, in case capturing the audio content is not necessary, the purpose-built host software module running on any mobile computing device 150 performs screen-casting that simply records the desktop screen activity to standard video files and outputs it. According to this arrangement, the said screen-casting process runs silently in the background and can record a movie of the desktop screen constantly and write it into a video framebuffer and flush the framebuffer to the electronic device of first type 140 instantaneously (i.e., continuously push its screen activity to 140 that operates in the capacity of a client). Rushing the framebuffer takes place obviously at a framehuffer-level or Graphics Device interface (GD!) level wherein the purpose-built host software module of any mobile computing device 150 functions as a server that sends screen updates to the electronic device of first type 140 which in turn functions as a client.
According to another embodiment, this operation can be easily facilitated when Remote F'ramehuffer (RF'B) principles are adopted. Accordingly, in case RFB is implemented between the said mobile computing device 150 and the said electronic device of first type 140, the mobile client device 150 (mainly its purpose-built host software module) operates in the capacity of a RFB server and the electronic device of first type 140 operates in the capacity of a RFB client. With this arrangement, the RFB server produces a stream of S rectangles represcnting screen data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the framebuffer level so that they appear in the correct place on the external display apparatus 104/1201130.
According to one preferred embodiment, the RFB server can send the compressed/encoded screen data to the RFB client and the compressionlencoding technique to be employed can be agreed at the time of initial negotiation. According to one preferred embodiment, a modified RFB technique is employed whereby the RFB server will send updates only when changcs (delta) to the screen are required. Accordingly, the RFB server only sends the areas of the screen which have changed and the receiving side (RFB client) reconstructs the whole image from the delta.
According to another preferred embodiment of the present invention, in case RFB is implemented between the said mobile computing device 150 and the said electronic device of first type 140, the mobile client device 150 has a touch screen and the user wants to use its screen as a GUI expecting user inputs only and the external display apparatus 104/120/130 as the only output screen (i.e., when the RFB server is not associated with an existing physical screen), the RFB server will run a "virtual framebuffer" and the remote RFB client (i.e., the electronic device of first type 140) has the only physical screen 104/ 1 20/130 showing the contents of that framebuffer.
According to another preferred embodiment of the present invention, in case a modified RFB is implemented between the said mobile computing device 150 and the said electronic device of first type 140, the RFB server (i.e., the mobile computing device 150) can send the encrypted screen data using for instance the High-bandwidth Digital Content Protection (HDCP) protocol for copy protection to the RFB (i.e., the electronic device of first type 140) client and the encryption/security technique to be employed can be agreed at the time of initial negotiation.
Future mobile computing device 150 will be UD capable and thus will iS demand lots of graphically intensive screen elements for its screen display that in turn demands relatively high frame rates. More over, if a user wants to project both the video and audio contents of a mobile computing device 150 on to a neighbouring external display apparatus, simple screen-casting may not work. Hence, according to another preferred embodiment of the present invention, the purpose-built host software module being installed and run on a mobile computing device 150 employs screen scraping and multimedia redirection technique that allows desktop running on the mobile computing device 150 to show up on an external display apparatus 104/1201130 or multimedia player together with playing-back the associated audio. According to this arrangement, the purpose-built host software module of the mobile computing device 150 operates in the capacity of a server and the electronic deviec of first type 140 operates in the capacity of client and the server-side multimedia elements are sent in their native formats together with the S respective timestamp information down to the electronic device of first type 140, which in turn renders/plays the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus 1041120/130 and synchronizing the associated audio, if any, with the video.
The above screen-scraping and multimedia redirection arrangement requires: i) the electronic device of first type 140 has the technical capability and hardware specifications to render the multimedia; and, ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content.
With these requirements being satisfied, the multimedia stream is decoded locally on the electronic device of first type 140 using the time-stamp information thereby providing a full synebronised audio and video experience.
According to another preferred embodiment of the present invention, the employed screen-scraping technique (preferably together with multimedia redirection) by the purpose-built host software module being installed and run on a mobile computing device 150 will analyze the screen content and identify screen regions that are being reused (such as icons, fonts/glyphs, dialog boxes, etc). Those graphics elements can be cached at the electronic device of first type 140, so if the purpose-built host software module needs S to send one of these elements, it only transmits the reference number of the cached element and the new coordinates. This dramatically reduces the amount of data transmitted and thus increases performance and user experience. This cached information can even he used for enhanced local echo effects. l0
The said electronic device of first type 140 has a media processing unit. the purpose of this media processing unit is to take the multimedia content being received from any mobile computing device 150 or media capturing devices 160/170 via the communication interface of first type 180 as the main input, and to apply one or plurality of media processing operations on to it depending on the type of external display apparatus and the characteristics of input multimedia content before rendering it on an external display apparatus 104/123/130 and/or multimedia player via the communication interface of second type for the purpose of instantaneous playback. For instance, a video stream is eharacterised namely by the number of frames per second (Frame rate), interlacing/progressive scan being used, display resolution, aspect ratio, colour space and bits per pixel, Video quality, Video compression method (digital only), Bit rate (digital only), Stereoscopic, Video formats -and hence the processing operations needed depend on these characteristics. The capabilities of the external sink device 104/1201130 can he inquired and learnt by the electronic device of first type 140 when the communication interface of second type is connected. Figure 3 is an illustrative schematic representation of the multimedia processing operations performed by the media processing unit of the electronic device of first type 140.
As depicted in Figure 3, the received multimedia bit-stream 310 may represent the encoded or raw data depending on whether the purpose-built host software module running on any mobile computing device 150 has applied any compression or media conversion technique before transmitting the captured multimedia streams to the electronic device of first type 140.
Video always requires high-bandwidth transmission relative to other content, and it is usually paired with a smaller audio stream. Both video and audio often need to be delivered in real timeS Compared to most other types of data, digital video requirements are enormous. An uncompressed 1080i60 HDTV signal with YCbCr 4:2:0 colour sampling requires 746.5 megabits per second (Mhps) to transmit. Even a small format such as CIF, with the same sampling and at 30 fps, requires 36.5 Mbps to transmit and more than 15.5 Giga Bytes to store an hour of video. Accompanying audio bit streams add smaller but still significant amounts to these numbers. Hence, it is better to migrate to advanced video eodecs in order to free up bandwidth, especially with the much greater throughput requirements of HD. This shows that it is better to employ compression in order to fit digital video within available bandwidth of the communication interface of first type 180. Given MPEG-4 achieves high compression ratios without compromising the picture quality, the purpose-built host software module running on any mobile computing device 150 can use it to compress the captured instantaneous media stream before transmitting it to the electronic device of first type 140. Before adoption, the mobile computing device 150 has to negotiate with 140 in order to agree on the compression technique to he used depending on the capabilities of 140 and 150.
According to one embodiment of the present invention, the mobile computing device 150 will have an initial negotiation via the electronic device of first type 140 in order to check the capability of end sink device 104/120/130 and select the compression accordingly. If the mobile computing device 150 can support high resolution content hut not thc cnd sink device 104/120/130, then video down scaling operation may have to take place. If it takes place at the mobile computing device 150, it is possible to save bandwidth on the communication interface of first type 180, hut at the increased processing and memory cost being incurred at the mobile computing device 150 itself On the other hand, in ease such a scaling operation or format conversion can lake place at the electronic device of first type 1140, it will lead to wastage of bandwidth on the communication interface of first type 180.
The only blocking factor to employing advanced compression technique or any media processing operation at the mobile computing device 150 is the processing and memory cost involved, in order to offload this work from the CPU and system RAM of the mobile computing device 150 an external chipset (termed electronic device of second type) can be used for this purpose in addition to constantly capturing the instantaneous media streams as it will be described later.
The received multimedia stream is decoded in 320 by the media processing unit of 140 in case the purpose-built host software module of 150 bad applied any compression technique to the transmitted media streams. In case the received media streams contain both audio and video, they will be separated out and routed individually for further media processing. Accordingly, decoded audio stream of 370 will be subject to one or plurality of audio iS processing operations in 380. Similarly, the decoded video bit-stream of 330 may undergo one or plurality of video processing operations in 342 in order to adapt the video depending on the format/resolution/frame-rate and other capabilities supported by the target playback (i.e., sink) devices 104/120/i 30.
In other words, the said media processing unit of 140 should have the relevant associated electronic circuitries to output the received multimedia in the appropriate format as demanded by different varieties of communication interfaces of second type 116 and to adapt the bit rate, resolution, frame rate and codec format being received in order to match the capabilities of the sink external display apparatus 104/120/130 or media players. This requires that the said media processing unit of 140 applies one or plurality of the following operations on the received media stream: 1) Transizing or Video scaling being applied to video stream for S converting video signals from one size or resolution to another with or without dc-interlacing. E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's 104/120/130 native resolution i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target device's screen size and colour availability; 2) Transcoding being applied to both audio/video streams for multi-format/-codec/-compression support for simultaneous encoding and decoding of a wide variety of codec/compression formats; iS 3) Transrating being applied to the video/audio stream whereby the media content is coded to a lower bit-rate without changing video/audio formats; 4) digital to analog conversion depending on the communication interface of second type 116 being used and characteristics of the end display apparatus 104/120/130; 5) audio synchroniser for the purpose of ensuring audio-to-video synchronisation; 6) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it S travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia interface (HDM1), Gigabit Video interface (GVIF), or Unified Display interface (UDI) connections; 7) error concealment technique using either spatial or temporal interpolation; 8) extra functionalities to amplify the audio signal; and, 9) 2D to 3D conversion or vice-versa; In case the video stream undergoes more processing operations in comparison with those of the audio streams, the functionality of the audio syncbroniser iS 350 is needed in order to improve the audio-to-video synchronisation. The audio synehroniser 350 that expects input signals from the video processing unit 342 (in the form of DDO (digital delay output) signal) and audio processing unit 380 in order to work out the exact amount of delay (as shown by 360) being needed to be introduced to the associated audio before being rendered/played-hack.
As part of the audio processing operations applied in 380, the audio is preferably converted to analog fonriat and made available via the stereo output jack and headphones. Similarly, once one or more video processing operations are applied in 342, video signal can be converted to analog format in case the external display apparatus 104/120/130 is a conventional analog device, in doing so, the media processing unit of 140 should he aware of the analogue television encoding system being used (e.g., PAL, NTSC, SECAM) by the sink devices 104/120/130.
Codecs being used by the purpose-built host software module of 150 or the media processing unit of 140 rely heavily on motion estimation for leveraging the temporal redundancy between frames. Using an intelligent transeoder, the motion vectors from the original bit-stream can be used as tips for the re-encoding process -this can be useful when the media processing unit of 140 tries to restore high resolution media stream from low resolution one. The original motion vectors can allow the transeoder to achieve much greater motion estimation efficiently.
The video scaling operation being employed by the media processing unit of is often accompanied with other video processing devices or algorithms.
The idea is to create a video processor that improves the apparent definition of video signals. These other devices may include the ability for: deinterlacing and Inverse Teleeinc (IVTC) * aspect ratio control * digital zoom and pan * brightness/contrastlhue/saturation/sharpness/gamma adjustments * frame rate conversion * colour point conversion (601 to 709 or 709 to 601) * colour space conversion (Component to RUB or RUB to Component) mosquito noise reduction * block noise reduction * detail enhancement * edge enhancement * motion compensation * primary and secondary colour calibration (including hue/saturationlluminance controls independently for each) * Audio to video synchronisation devices such as Audio synchronizer -can be based on MPEU-4.
* DRM device Up-scaling should he performed by employing advanced processing to eliminate scaling artefacts. such as jagged edges, motion adaptive noise reduction, mosquito noise, and detection of various video and film cadences.
With the help of the media processing unit the electronic device of first type can support any type of multimedia content using any format at any time for use with any end sink device 1041120/130. The media processing unit of needs to be able to transcode in real time. For instance, the transcoding requirement can he between MPEG-2 and 11.264. However, support for other standards such as WMV9 and JPEU can be useful in order to extend the utility of the electronic device of first type 140. Some of the supported output video formats and audio formats of the media processing unit of 140 are respectively ff263, MPEG-4 Simple Profile 3, ff264, WMV MPEG-i, MPEG-2, AVI, WMV, Xvid and M3U, MP3, AAC-LC, AAC÷, eAAC+, WAV, WMA, AMR-NB, QCELP EVRC.
As mentioned before, according to one embodiment of the present invention, an electronic device of second type is devised that functions as an external graphics accelerator card and/or an external sound card for capturing the instantaneous multimedia content being played back on a mobile computing device 150 for the purpose of projecting the captured multimedia content on an external display apparatus 1041120/130 or a multimedia player. The said electronic device of second type comprises namely: a) a Graphic processing unit (GPU); b) BIOS or firmware; c) Memory; d) a wired communication interface of fifth type to get connected to a mobile computing device; The said electronic device of second type can either tap into the graphic/sound card of the mobile computing device 150 or employ screen-scraping together with multimedia redirection for the purpose of capturing the instantaneous multimedia content being or intended to be played back on the mobile computing device ISO. The main reason of its usage is to offload this work from the CPU and system RAM of the mobile computing device 150.
The purpose-built host software module being installed and run on any mobile computing device 150 will liaise with the electronic device of second type in order to regulate the instantaneous capturing of multimedia stream and the multimedia processing operations that need to be applied. The said electronic S device of second type can apply an appropriate compression andlor one or plurality of other media processing operations to the captured multimedia before transmitting at the request of the purpose-built host software module.
The purpose-built host software module of 150 can subsequently fetch the multimedia streams from the electronic device of second type and transmit to the electronic device of first type 140 in real-time using the wireless transceiver of 150. According to another embodiment of the present invention, the electronic device of second type is equipped with a wireless transceiver providing the communication interface of first type 180 in order to transmit the captured/processed media streams directly to the electronic device of first type 140. The purpose-built host software module coordinates such an operation in either case.
According one preferred embodiment of the present invention, the communication interface of fifth type can be a inini-USB or based on IIDMII 1.4 Type D micro-connector. The electronic device of second type can preferably get the required electrical energy from the mobile computing device 150 via an appropriate the communication interface of fifth type.
Figure 4 is an exemplary schematic block diagram illustrating the internal components of the electronic device of the first type 140 being implemented as a System on Chip (SoC) operating on one or plurality of microprocessors according to one embodiment of the present invention that can be configured S in any form as those having ordinary skill in the art will appreciate. According to one preferred embodiment of the present invention, the essential functionalities of the electronic device of first type 140 can be realised potentially by the system on a chip (SoC) 410 with its microprocessors containing the software stack as shown in Figure 5. in other words, any electronic dcvice of first type 140 for the purpose of projecting the multimedia content of a mobile computing device 150 to an external display apparatus 104/1201130 or art external multimedia player in the intended way is made up of the SoC 410.
iS Various described embodiments provide the intended functionalitics of the electronic device of first type 140 that is realised as purpose-developed software executed on either a general purpose microprocessor 412 such as a Reduced instruction Set Computer (RiSC) microprocessor or graphics processing unit (GPU) 414 that handles the digital signal processing operations pertaining to multimedia reception, processing and rendering. The said SoC 410 includes one or plurality of microprocessors 412 and/or a CPU 414, system peripheral controller 422, audio/video codec 424, system memory controller 426, and a communication controller 430 supporting different interfaces 480 (both wired and wireless) such as the communication interfaces of firsilsccond/third/fourth typcs and the like -all of which are communicatively coupled over bus 420. it should be appreciated that the said SoC 410 may include additional components, as understood by those of skill S in the art. These additional components are not described herein in order not to obscure the cmbodiments described herein.
Depending on the exact configuration of the said electronic device of first type 140 and the associated SoC 410, system memory may be volatile (such as RAM 416), non-volatile (such as ROM 418, flash memory and like) or some combination of the two. The said SoC 410 is connected to one or plurality of multimedia input ports and interconnects 468 via the peripheral controller 422. The multimedia input ports and interconnects 468 include those described in relation to 210 and the associated electronic circuitries that handle the multimedia depending on characteristics (e.g., type/format) of received multimedia content and multimedia input port and interconnect being used. Also, the said SoC 410 is connected to one or plurality of multimedia output ports and interconnects 476 via the peripheral controller 422. The multimedia output ports and interconnects 476 include those described in relation to 220 and the associated electronic circuitries that handle the multimedia depending on the characteristics (e.g., type/format) in which any multimedia content needs to be output and the multimedia output port and interconnect being used. The said SoC 410 is connected to various control key inputs 456 via 422 These control key inputs 456 include those shown namely by 250, 280, 284 and 290. The SoC 410 may operate a small electronic visual display or LCD panel 448 via the peripheral controller 422 -the external aspect of which is shown by 260. The SoC 410 may control other peripherals of the electronic device of first type 140 such as the DVD/Blu-Ray disk player, remote controller and the like -the details of these additional components are not described herein in order not to obscure the embodiments described. The said SoC 410 supports a removable storage 440 and non-removable storage 460 via the memory controller 426. These media storages can be used in case the electronic device of first type 140 needs to record any multimedia content.
Figure 5 exemplarily illustrates the various components of the software stack of a CPU/CPU/RISC according to one another embodiment of the present is invention. The microprocessor can be multipuipose such as a CPU or in contrary it can be a CPU/RISC microprocessor. The software stack 500 demonstrates the operations and applications executed by the combination of RiSC/CPU microprocessors, e.g., 412/414 of Figure 4. As it can he seen, according to one embodiment, the basic funetionalities of the software stack 500 comprise one or plurality of multimedia applications 510 (e.g., multimedia surveillance application), an application layer 14, a configuration manager 520, a mutual authentication and encryption manager 540, a session manager 550, a connection manager 554, OS abstraction layer 560 and Medium access control (MAC) and physical (PHY) layer aspects associated with different communication interfaces 564.
The configuration manager 520 of thc software stack 500 deals with the initial setup of the electronic device of first type 140 triggered on being switched on and the configuration of WLAN AP being operatcd by 140. This mainly includes operating a WLAN on the communication interface of first type 180 with either factory set or user configured SSID and initiating the DHCP functionality for the WLAN AR if a user wants to configure the WLAN for the purpose of the following, such operations will be again handled by 520: i) setting an appropriate SSID for the said WLAN that can uniquely identify the user/household/office where it is to be used; ii) getting the WLAN to either advertise/broadcast its SSID or hide it; and, iii) setting encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials for the purpose of restricting connections of any mobile computing device 150 or any other media capturing device such as a security-camera/web-cam 160/170 to the said WLAN.
Storing the login/encryption credentials, authenticating different users and encrypting the multimedia traffic being exchanged are handled by the authentication and encryption manager 540 when prompted by the configuration or the connection manager 540.
The connection manager 554 is responsible for establishing the necessary physical connection when a mobile computing device 150 or any media capturing device 160/170 tries to associate itself with the WLAN AP being operated by the electronic device of first type 140. Establishing a physical S connection preferably after authenticating a user and assigning a zeroconf identifier to the connected device 150/160/170 are handled by the connection manager while considering the capacity.
The session manager 550 deals with an issue of establishing the session-related logical connection after the physical connection has been established.
In this respect, the session manger 550 is responsible for session initiation after a user of a mobile computing device 150 would like to invocate the Projeet2TV-application. The session initiation involves triggering of initial handshake between the electronic device of first type 140 and the other media capturing devices 150/160/170 that would like to establish a session-related logical connection. The purpose of the initial handshake is to agree on the media type, compression, format, frame-rate, bit-rate and the like of the multimedia that needs to be transmitted from 150/160/170 to 140 along with the encryption/security techniques to be employed.
According to one embodiment of the present invention, a CIPU 414 handles Soft DSP 530 and is responsible for applying one or plurality of multimedia processing operations, whereas a generic-purpose microprocessor 412 (e.g., RTSC) handles the operations 510, 514, 520, 540, 550, 554, 560, and 564. The soft DSP 530 is responsible for such multimedia processing operations as transcoding 572, transizing/video-scaling 576, transrating 580, audio synchronising 584, error concealment 588 using either spatial or temporal interpolation techniques, digital-to-analog conversion 592, and digital rights management 596.
The flowchart 600 of Figure 6 exemplarily illusUates the user-specific initial configuration procedure that is optionally required for the WLAN AP belonging to the electronic device of first type 140 to operate according to one embodiment of the present invention. As indicated by the decision-making stage 620, the configuration procedure will he triggered only when user-specific settings are preferred. In case a user prefers such settings, in the processing stage 630 an appropriate SSID for the said WLAN AP that can uniquely identify the user/household/office where it is to be used is set after turning the WLAN feature on by pressing 284. Then the user has the choice of getting the WLAN to either advertise/broadcast its SSID or hide it as indicated in 640. Also, if the user prefers restricted access to the WLAN AP being operated by the electronic device of first type 140, encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials for each device 150/160/170 that can possibly connect to 140 can be set as indicated by the processing step 650. Once this initial WLAN-related configuration is complete, the user has to connect the electronic device of first type 140 with the external display apparatus 104/120/130 or multimedia player via the communication interface of second type 116 as indicated by the processing step 660 of the flowchart 600.
According to one preferred embodiment, the user can connect the electronic device of first type 140 to a computer/PC via an appropriate interface/port (e.g., RJ-45 cable) in order to perform the initial WLAN configuration in the same way a normal WLAN is configured preferably a web browser can be used for this purpose. Although according to one embodiment of the present invention, the electronic device of first type 140 operates a WLAN AP on the communication interface of first type 180 enabling a convenient way of connecting devices 150/160/170, another preferred embodiment also accommodates the possibility of configuring the 140 to operate an Independent Basic Service Set (IBSS) on 180. IBSS can be used when WiFi-direct is possible.
The flowchart 700 of Figure 7 exemplarily shows the physical connection establishment procedure involved when a mobile computing device 150 or any media capturing device such as a security-camera/web-cam 160/ 170 and the like attempts to establish a physical connection with the electronic device of first type by getting itself associated witb the WLAN access point being operated by the electronic device of first type 140 according to one embodiment of the present invention. This is analysed from the perspectives of the device 150/160/170 that tries to connect to the electronic device of first type 140. Please note that in the case of thin devices like security-camera/web-cam 160/170 that has limited intelligence, processing power and memory footprint to run a purpose-built host software module, a personal computer or laptop can be used to associate devices 160/170 to the WLAN AP belonging to the electronic device of first type 140. The operations S associated with this are not within the scope of the present invention and easily understood by those having ordinary skill in the art.
When a physical connection is needed as decided by the decision-making stage 720, the associated radio supporting the communication interface of first type 180 on a mobile computing device 150 or any other media capturing device 160/170 needs to he activated. Subsequently, a mobile computing device 150 or any other media capturing device 160/ 170 should start looking for a WLAN AP with a specific SSID that is operated by the electronic device of first type 140 -this is required by the processing step 730 of 700. Once the specific SS1D is found -as checked by the decision-making stage 740 -the procedure to establish the physical connection will check whether WLAN access is restricted through encryption/login keys in 760. If it is the case, the procedure to establish the physical connection will prompt the user in 780 to enter the required encryption (e.g., WEP/WPA or the like) and/or the login credentials. When the login and/or encryption credentials are received by the electronic device of first type 140 (especially by its authentication and encryption manager 540 when 140 is realised through a SoC 410), it needs to authenticate a mobile computing device 150 or any other media capturing device 160/170 that attempts to connect.
When the authentication succeeds, the electronic device of first type 140 needs to assign a unique zeroconf identifier to the connected mobile computing device 150 or any other media capturing device 160/170 pertaining to the communication interface of first type. Then the procedure to establish the physical connection will cheek to see in 790 whether a unique zeroconfig identifier/address has been assigned by the WLAN AP of 140 to the device 150/160/170 that is trying to associate with the WLAN AP. If the zeroconfig identifier/address has been already assigned, the physical connection establishment between the electronic device of first type 140 and a mobile computing device 150 or any other media capturing device 160/170 is successful. If not, a mobile computing device 150 or any other media iS capturing device 160/170 will re-try connecting to the WLAN by following steps 760 and 780 (step 780 is needed only when the WLAN access is restricted through encryption/login keys) as shown in Figure 7.
If in 740 a given SSID is not found mainly because the SSID of WLAN belonging to 140 is not broadcast and thus hidden as checked in 750, the procedure to establish the physical connection will prompt the user to manually enter the SSID, profile identifier, if any, and to try re-connecting to the WLAN by following steps 760 and 780 (step 780 is needed only when the WLAN access is restricted through encryption/login keys).
The flowchart 800 of Figure 8 is an exemplary illustration of the session-related logical connection establishment procedure as required by a mobile computing device 150 or any media capturing device such as a security-camera/web-cam 160/170 when it attempts to project its multimedia content to an external sink device 104/120/130 via the electronic device of first type according to one another embodiment of the present invention. This procedure is also termed Project2TY-procedure and the flowchart 800 looks at operations involved from the perspective of a mobile computing device 150 or any media capturing device such as a security-camera/web-earn 1601170.
Once the procedure has started in 810, it will he checked in the decision-making stage 820 whether a user would like to project the multimedia content on to an external sink device 104/1201130. If it is the ease, the user of a mobile computing device 150 has to turn on the associated radio supporting iS the communication interface of first type 180 on a mobile computing device and try establishing a physical connection with the electronic device of first type by following the physical connection establishment procedure as illustrated by 700. This is expected by the processing stage 830 of 800.
It will be subsequently checked in the decision-making stage 840 whether the necessary physical connection has already been established between 140 and 150. Once the physical connection is established, the mobile computing device 150 will trigger the session initiation in 850 after having agreed on the compressionlcodec and the encryption/security techniques, if any, that need be employed by the mobile computing device 150 before transmitting the captured media to the electronic device of first type 140. For this purpose, the mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) will send a special signalling message called Session Request to 140 and wait to receive the response from 140. This Session Request message carries session related parameters. According to one embodiment, the Session Request message carries the priority associated with the mobile computing device 150 that can be used by 140 for the purpose of priority-based access. When the Session Request message is received, the electronic device of first type 140 will quickly inquire whether any of the sink devices 104/120/130 is free and whether it is currently busy with any other device 150/160/170. if any of the sink devices 104/120/130 is free and 140 is not busy with any session in-progress, then the electronic device of first type will first turn the sink device 104/120/130 on and make the necessary session related configurations. Once this is complete, the electronic device of first type 140 will accept the Session Request and respond positively.
The mobile computing device 150 (especially the purpose-built host software module being installed and run on 1SQ) will check to see in 854 whether it has received any positive response from 140 and whether the initial negotiation is successful. When these conditions are met, mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) will start constantly capturing the instantaneous multimedia content being or intended to be played back on the mobile computing device 150 as indicated in 860. Once captured, mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) apply one or plurality of multimedia operations to the captured media as indicated in the processing step 864. The mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) will subsequently transmit constantly the raw or processed multimedia content to the electronic device of first type 140 as indicated in the processing step 870.
If, on the other hand, it is decided in 854 that either the Session Request message is not positively responded or the initial negotiation has failed, the mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) needs to try re-sending the Session Request to the electronic device of first type 140 only when it is decided in 844 that the re-try limit has not been reached yet. If, on the other hand, the re-try limit is reached already, the mobile computing device 150 (especially the purpose-built host software module being installed and run on 150) will give up its attempt to project the multimedia content on to an external sink device 104/1 20/130 using a given electronic device of first type 140.
In ease the mobile computing device 150 (especially the purpose-built host software module being installed and run on 15W has already succeeded in its attempt to project its multimedia content on to an external sink device 104/120/130, it will continue until the user would like to terminate the Project2TV-application or has lost the physical connection with the electronic devicc of first type 140. This is checked in 874 and if this condition is met, both thc clcctronic device of first type 150 and the mobile computing device (especially the purpose-built host software module being installed and run on 150) will tear down the physical and logical connections after a timeout.
As it was mentioned previously, the mobile computing device iSO (especially the purpose-built host software module being installed and run on 150) employs either modified remote framebuffering (RFB) or screen-scraping with multimedia re-direction in order to constantly and efficiently send the captured multimedia content to the electronic device of first type 140 for the purpose of immediately playing back on the external sink device 104/120/130.
The flowchart 900 of Figure 9 is an exemplary illustration of the session-related logical conncction establishment procedurc as required by an electronic device of first type 140 when a mobile computing device 150 or any media capturing device such as a security-camera/web-cam 160/170 attempts to project its multimedia content to an external sink device 104/120/130 via the electronic device of first type 140 according to one another embodiment of the present invention. Once the initial configuration of is complete (WLAN related initial configuration is illustrated by 600), with the help of decision-making stage 914, the electronic device of first type will continuously monitor whether it has received any request to establish the physical connection from devices 150/160/170. if it has, the electronic device of first type 140 will check in 920 to see whether the WLAN has any residual capacity on the communication interface of first type 180 to accommodate the device 150/160/170 that tries to connect to 140. If it has enough residual capacity on 180, the electronic device of first type 140 will S check in 940 to see whether any session is currently in progress. If there is no residual capacity on 180 or there is a session being currently in progress, the electronic device of first type 140 will not respond to any association request or the Session Request initiated by the device 150/160/170.
If, on the other hand, the electronic device of first type 140 has any residual capacity on 180 and does not currently support any session. it will accept the Session Request in 950 and respond positively after having bad a successful initial negotiation with the mobile computing device 150 or any media capturing device 160/170 that tries to project its multimedia content on to an external sink device 104/120/130 via 140. On receiving the multimedia content, the electronic device of first type 140 applies in 960 one or plurality of the following multimedia processing operations in order to render the multimedia content on the external display apparatus 104/120/130 or multimedia player: * Transizing or Video scaling functionality for converting video signals from one size or resolution to another with or without dc-interlacing.
E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's 104/120/130 native resolution -i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target device's screen size and colour availability; S * Transcoding -multi-format/-coded-compression support for simultaneous encoding and decoding of a wide variety of codee/compression formats; * Transrating in this process the media content is coded to a lower hit-rate without changing video formats; digital to analog conversion depending on the communication interface of second type 116 being used and characteristics of the end display apparatus 104/120/130; * audio synchroniser for the purpose of ensuring audio-to-video synchronisation * digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (F1DCP) to prevent copying of digital audio and video content as it travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video Interface (OVIF), or Unified Display Interface (UDJ) connections: and, * error concealment technique using either spatial or temporal interpolation.
* extra functionalities to amplify the audio signal.
* 2D to 3D conversion or vice-versa.
The application of one or plurality of the above-mentioned multimedia processing operations depends on the characteristics of the received media type, format, resolution, bit-rates, frame rates, colour-depths, aspect ratio and the like, and the capabilities of the end sink device 104/120/130 (i.e., external display apparatus or multimedia player).
Once applied one or plurality of multimedia processing operations to the received multimedia content from any mobile computing device 150 or any media capturing device 160/170. the electronic device of first type 140 will subsequently (as indicated in 970 of 900) render the multimedia content on the external display apparatus 104/120/130 or any multimedia player.
While rendering, the electronic device of first type 140 will periodically check to see whether the currently in-progress session has stopped due to the actions by the user of a mobile computing device 150 or any media capturing device 160/1 70 or the physical connection is lost, if it has not, the electronic device of first type 140 will continue the operations as indicated by 960 and 970 until the conditions of the decision-making stage 980 are satisfied. On the other hand, if conditions of 980 are met, the electronic device of first type 140 will tear down the associated logical and physical connections -as indicated in 990.
Figure 10 exemplarily illustrates the initial handshake operation required between any mobile computing device 150 or any media capturing device such as a security-camera/web-cain 160/ 170 and the electronic device of first type 140 in priory to projecting any multimedia content of a 150/160/170 to an external sink device 104/120/130 according to one another embodiment of the present invention. As indicated by 1010, initially the mobile computing device 150 or the media capturing device 160/170 will request to make the physical connection with the electronic device of first type 140 by attempting to associate with the WLAN AP being operated by 140. This involves the authentication of the device 150/160/170 that is trying to connect by the electronic device of first type 140. In response, as indicated by 1020, the WLAN AP belonging to 140 will accept the physical connection and respond positively if the authentication is successful and the WLAN AU has sufficient residual capacity on the communication interface of first type 180. The DHCP functionality of the said WLAN AP will also attempt to assign a zeroconf address to the device 150/160/170 that is trying to connect.
Once the physical connection is accepted, the device 150/160/170 that is trying to connect will make the initial configuration. Steps 1010, 1020 and 1030 will follow the legacy mechanisms that are used to connect a mobile computing device to a WLAN and hence the present invention does not propose any thing new in this respect. The associated driver/firmware of a wireless transceiver (e.g., IEEE 802.11 radio) will take care of the operations as indicated by 1010, 1020 and 1030. Also, it is the first time connection attempt of a device 1501160/170 to the WLAN AP belonging to 140 that has to follow a cumbersome connection procedure, whereas any subsequent connection attempt is as simple as that applies to a similar scenario where a S laptop tries to connect to a generic-purpose WLAN AP.
Once the necessary physical connection is established, the purpose-built host software module being installed and run on any mobile computing device 150 will trigger the session initiation and parameter negotiations by generating the Session Request message as indicate by 1040. mr Session Request message carries session related parameters. According to one embodiment, the Session Request message carries the priority associated with the mobile computing device 150 that can be used by 140 for the purpose of priority-based access.
When the Session Request message is received, the electronic device of first type 140 will quickly inquire whether any of the sink devices 104/120/130 is free and whether it is currently busy with any other device 150/160/170. If any of the sink devices 104/120/130 is free and 140 is not busy with any session in-progress, then the electronic device of first type 140 will first turn the sink device 104/120(130 on and make the necessary session related configurations as indicated by 1050. Once this is complete, the electronic device of first type 140 will accept the Session Request and respond positively as indicated by 1060.
Claims (37)
- Claims 1. There is provided an electronic device of first type that receives multimedia content from a mobile computing device and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player in real-time, and the said electronic device of first type comprising: i) a wireless transceiver providing a communication interface of first type to communicate with the mobile computing device; ii) a communication interface of second type to communicate with an external visual display apparatus or multimedia player; wherein connecting a mobile computing device to the said electronic device of first type involves the configuration procedure comprising the steps of: a) enabling the electronic device of first type to operate as a master or a wireless local area network on the said communication interface of first type with a specific SSID having DHCP functionality; b) letting the mobile computing device operate as a slave, detect the SSID, if it is broadcast, and connect to the electronic device of first type via the communication interface of first type after a successful authentication; c) letting the said electronic device of first type assign a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; wherein the said electronic device of first type has a media processing unit that takes the multimedia content being transmitted via the communication interface of first type as the main input, applies one or plurality of media processing operations on to it depending on the type of external display apparatus and the input multimedia content type, and outputs/renders it to an external display apparatus and/or multimedia player via the communication interface of second type for the purpose of instantaneous playback.
- 2. The said electronic device of first type that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-back it on an external visual display apparatus or multimedia player according to claim 1, wherein the said SSID can be user settable in order to make it unique in the same way it is set by the user when a WLAN is configured.
- 3. The said electronic device of first type according to claim I or 2, wherein the said communication interface of first type is wireless-based and can be implemented based on different flavours of WiFi (i.e., iEEE 802.11) technology including WiFi-direct and the said electronic device of first type will operate a WiFi access point on the said the communication interface of first type.
- 4. The said electronic device of first type according to claim 1 or 2, wherein the communication interface of first type is wireless-based and can be implemented based on Bluetooth, Wireless USB, Wireless lID, Wireless HDMI standard or the like.
- 5. The said electronic device of first type according to claim I or 2, wherein the communication interface of second type can be wired and implemented based on l-IDMIIDVL SCART, IEEE 1394 (i.e., Firewire/iLink/Lynx), RCA-connectors-based Composite/component video, Display Port, Serial Digital Interface, S-Video, VGA and its extensions, or the like.
- 6. The said electronic device of first type according to claim 1 or 2, wherein the said unique zeroconf identifier being assigned to the connected mobile computing device on the communication interface of first type is an IP address.
- 7. The said electronic device of first type according to claim I or 6, wherein the assigned IP address takes a random address (on network 169.254/ 16 in case of IPv4, or on prefix MANET_INIT1ALPREFIX in case of IPv6) and this is analogous to the way the Autonet allocations are performed as proposed in the Zero configuration (zerocont) working group.
- 8. The said electronic device of first type according to claim 1, wherein the said electronic device of first type limits the number of mobile computing devices or media capturing devices such as a security-camera/web-cain being connected to it at any point in time.
- 9. The said electronic device of first type according to claim I or 8, wherein the said electronic device of first type accepts connection requests being originating from one or plurality of mobile computing devices and this procedure is governed by the priority-based scheme comprising the steps of: i) each mobile computing device needs to he assigned a priority number based on who carries that mobile computing device (e.g., whether the main house-holder/manager); ii) initial connection request of a mobile computing device contains the said priority number; and, iii) the electronic device of first type analysing each connection request and connect a device based on the priority order; wherein in case a low priority device was already attached to the electronic device of first type, the connection will be replaced by a high-priority device.
- 10. The said electronic device of first type according to claim 1, wherein the displayed content on the external visual display is the exact or intended instantaneous screen content of the mobile computing device.
- I I. The said electronic device of first type according to claim I or 10, wherein the mobile computing device is responsible for capturing/getting the exact or intended instantaneous screen shot/content with/without the associated audio content using relevant software and/or hardware components, and transmitting it to the said electronic device of first type in real time.
- 12. The said electronic device of first type according to claim 1 or ii, wherein the mobile computing device captures the multimedia content and compress it before being transmitted to the electronic device of first type; wherein the compression can he based on MPEG-4,
- 13. The said electronic device of first type according to claim I or 11, wherein the said mobile computing device employs screen scraping and multimedia redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus or multimedia player together with playing-back the associated audio; wherein the mobile iS computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side multimedia elements are sent in their native formats together with the respective timestamp information down to the electronic device of first type, which in turn renders/plays the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizing the associated audio, if any, with the video and this requires: i) the electronic device of first type has the technical capability and hardwarespecifications to render the multimedia; and,ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full S synchronised audio and video experience.
- 14. The said electronic device of first type according to claim I or 11, wherein the said mobile computing device that is equipped with an appropriate hardware, software or combination of both performs screen-casting that simply record the desktop screen activity to standard video files and output it, wherein it runs silently in the background, records a movie of the desktop screen, writes it constantly into a video framebuffer and flushes the framebuffer to the electronic device of first type instantaneously (i.e., continuously push its screen activity to the client) and this takes place at a framebuffer-level or Graphics Device interface (GDI) level; wherein the mobile computing device functions as a server that sends screen updates to the electronic device of first type which in turn functions as a client.
- 15. The said electronic device of first type according to claim I or 11, wherein in case RFB is implemented between the said mobile computing device and the said electronic device of first type, the mobile client device operates in the capacity of a RFB server and the electronic device of first type operates in the capacity of a REEl client; wherein the RFB server produces a stream of rectangles representing screen data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the framehuffer level so that they appear in the correct place on the screen.
- 16. The said electronic device of first type according to claim I or 15, wherein in case RIB is implemented between the said mobile computing device and the said electronic device of first type the RFB server can send the compressed/encoded screen data to the RIB client and the compressionlcncoding technique to he employed can he agreed at the time of initial negotiation.
- 17. The said electronic device of first type according to claim 1 or iS, wherein in case RIB is implemented between the said mobile computing device and the said electronic device of first type the RFB server sends updates only when changes (delta) to the screen are required, and only sends the areas of the screen which have changed and the receiving side (RIB client) reconstructs the whole image from the delta.
- 18. The said electronic device of first type according to claim 1 or 15, wherein in ease RFB is implemented between the said mobile computing device and the said electronic device of first type, the mobile client device has a touch screen and the user wants to use its screen as a GUI expecting user inputs only and the external display apparatus as the only output screen (i.e., when the RIB server is not associated with an existing physical screen), the RFB server will run a "virtual framebuffer" and the remote RFB client has the only physical screen showing the contents of that framebuffer.
- 19. The said electronic device of first type according to claim I or 15, wherein in case RIB is implemented between the said mobile computing device and the said electronic device of first type, the RFB server can send the encrypted screen data using the HDCP protocol for copy protection to the RIB client and the encryptionlseeurity technique to he employed can he agreed at the time of initial negotiation.
- 20. The said electronic device of first type according to claim 1, wherein the said media processing unit should have the relevant associated electronic circuitries to output the received multimedia in the appropriate format as demanded by different varieties of communication interfaces of second type and to adapt the bit rate, resolution, frame rate and codec format being received in order to match the capabilities of the sink external display apparatus or media players, and this requires that the said media processing unit comprises one or plurality of the following functionalities: a) Transizing or Video sealing functionality for converting video signals from one size or resolution to another with or without dc-interlacing.E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target devic&s screen size and colour availability; b) Transcoding -multi-fonnat!-codecf-compression support for simultaneous encoding and decoding of a wide variety of S codec/compression formats; c) Transrating -in this process the media content is coded to a lower bit-rate without changing video formats; d) digital to analog conversion depending on the communication interface of second type being used and charactcristics of the end display apparatus; e) audio synchroniser for the purpose of ensuring audio-to-video synchronisation 1) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transeripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video interface (GVIF), or Unified Display interface (UD1) connections; g) en-or concealment technique using either spatial or temporal interpolation; h) extra funetionalities to amplify the audio signal; and, i) 2D to 3D conversion or vice-versa
- 21. The said electronic device of first type according to claim 1, wherein the said electronic device of first type gets audio signal from the televisiorilmultiniedia-player via a communication interface of third type and makes it available, wherein one or plurality of wireless headphones can connect to the said electronic device of first type to stream the audio via a communication interface of fourth type and playback in real-time for the purpose of silently watching the television.
- 22. The said electronic device of first type according to claim 1 or 21, wherein the communication interface of third type can be any wired-based audio interface and inter-connect, whereas the said communication interface of fourth type can be Bluetooth.
- 23. The said electronic device of first type according to claim 1, wherein one or plurality of multimedia capturing devices such as a webcam, security camera, camcorder or similar device can be connected via the said communication interface first type to the said electronic device of first type for the purpose of displaying the instantaneous captured content on the said external display apparatus in real-time.
- 24. The said electronic device of first type according to claim 1, wherein the said electronic device of first type has ports such as USB that can be used by a user to connect an external flash memory and to upload one or plurality of still images to the electronic device of first type for the purpose of displaying one or plurality of the uploaded still images continuously (111cc Slideshows) on the external display apparatus and this is activated under two circumstances: i) automatically after a certain time-out when the external display S apparatus is in its power-saving mode; or, ii) manually when the electronic device of first type has an external electronic button/switch and pressing/activating this button/switch will thgger the said Slidesbow operation at the discretion of the user.Wherein in either of the above case, the user has the option to set as to how long each still image of a Slideshow needs to be displayed.
- 25. There is provided an electronic device of second type that functions as an external graphics accelerator card and/or an external sound card for capturing the instantaneous multimedia content being played back on a mobile computing device and transmitting it to an external display apparatus such as a television set or multimedia player, and the said electronic device of second type comprising: a) Graphic processing unit (GPU); b) BIOS or firmware; c) Memory; d) a wireless transceiver providing a communication interface of first type to communicate with the external display apparatus or multimedia player; e) a wired communication interface of fifth type to get connected to a mobile computing device; wherein the said electronic device of second type can either tap into the graphic/sound card of the mobile computing device or employ screen-scraping together with multimedia redirection for the purpose of capturing the instantaneous multimedia content being or intended to be played back on the mobile computing device and transmit the multimedia content constantly.
- 26. The said electronic device of second type that is connected to the mobile computing device of first type according to claim 25; wherein it applies an appropriate compression and/or one or plurality of other multimedia processing operations to the captured multimedia before transmitting.
- 27. The said electronic device of second type that is connected to the mobile computing device of first type according to claim 25, wherein the communication interface of fifth type can be a niini-USB or based on 11DM! 1.4 Type D micro-connector.
- 28. The said electronic device of second type that is connected to the mobile computing device of first type according to claim 25 or 27, wherein the electronic device of second type is connected to the external display apparatus such as a television set or multimedia player via the electronic device of first type.
- 29. There is provided a complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player; and the said complete framework comprises: S i) a mobile computing device being installed and running a purpose-built host software module; ii) electronic device of first type; wherein the said purpose-built host software module provides appropriate user interface and allows the user to connect the mobile computing device to the said electronic device of first type for the purpose of constantly projecting the instantaneous screen display of the mobile computing device along with the associated audio content, if any, in real time on to the neighbouring external display apparatus or multimedia player.
- 30. The said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player according to claim 29; wherein the said purpose-built host software module is responsible for: i) associating the priority of the user to the mobile computing device; ii) establishes the necessary session-related logical connection with the electronic device of first type when the mobile computing device maintains a successful physical connection; iii) capturing instantaneous multimedia content either by tapping into the internal or external graphic and/or sound cards, and applying one or more media processing operations, if needed; and, iv) performing packetization before transporting the captured media streams in real-time to the electronic device of first type.
- 31. The said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player according to claim 29 or 30, wherein the said purpose-built host software module performs screen-scraping and multimedia redirection.
- 32. The said complete framework that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player according to claim 29 or 25, wherein the said purpose-built host software module controls the electronic device of second type in case it is connected to the mobile computing device for the purpose of capturing and transporting the instantaneous media streams constantly.
- 33. There is provided a an electronic system on a chip that receives multimedia content from a mobile computing device in real-time and renders/displays/plays-hack it on an external visual display apparatus such as a television set or multimedia player or multimedia player, and the system on a chip comprising: fl a bus, ii) a peripheral controller being coupled to said bus for connecting a multitude S of 110 peripheral devices; iii) one or plurality of microprocessors being coupled to said bus and said microprocessors configured to perform the required signal processing associated with the multimedia traffic and to provide control functions of said communication device; iv) a communication controller being coupled to said bus for connecting to one or plurality of communication interfaces; and, v) a memory that is coupled to said bus into which a plurality of instructions are loaded, wherein the said system on a chip composes a purposeful system on-the-fly by getting any mobile computing device to join while handling operations such as: a) operating a WLAN access point on the communication interface of first type with a user configurable SSID; b) initiating a DIICP functionality; c) when a mobile computing device attempts to associate with the said WLAN access point, assigning a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type.
- 34. There is provided a method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; and the said method comprises the following steps for the purpose of configuring S the electronic device of first type to operate a WLAN access point on the communication interface of first type and this initial configuration procedure comprising the steps of: i) setting an appropriate SSID for the said WLAN access point that can uniquely identify the user/householdioffice where it is to he used; ii) getting the WLAN access point to either advertise/broadcast its SSID or hide it; iii) setting encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials for the purpose of restricting connections of any mobile computing device or any media capturing device such as a security-cameralweh-cam to the said WLAN access point; wherein the user can connect the electronic device of first type to a computer in order to perform the initial WLAN configuration in the same way a normal WLAN is configured and once the said initial configuration is complete, the user can connect the electronic device of first type to the external display apparatus or a multimedia player via a communication interface of second type and this will make the electronic device of first type ready for its intended usage.
- 35. The said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according to claim 34; and the said method comprises physical connection establishment S procedure for the purpose of establishing a physical connection between the mobile computing device and the electronic device of first type, and the said physical connection establishment procedure comprising the steps of: i) getting the mobile computing device to look for a WLAN access point with a specific SSID; ii) once found, prompting the user of a mobile computing device to enter the necessary encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials assigned to the user. On the other hand, in case, the specific SSID is not found mainly because the network does not advertise/broadcast the SSID, prompting the user to manually enter the SSID and the profile id, if any, and to try reconnecting to the WLAN access point belonging to the electronic device of first type; iii) when the login and/or encryption credentials are received by the electronic device of first type, it needs to authenticate the mobile computing device that attempts to connect; iv) when the authentication succeeds, the electronic device of first type needs to assign a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; wherein once the physical channel is established, the mobile computing device is ready to initiate any communication session with the electronic device of first type.
- 36. The said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according to claim 34 or 35, wherein the said method comprises the following steps for the purpose of establishing a multimedia session between the mobile computing device and the electronic device of first type: i) when the user of a mobile computing device intends to initiate multimedia projection on to an external sink device, the mobile computing device will initiate the physical connection establishment procedure in order to connect to the electronic device of first type; ii) once the physical connection establishment is successful, the mobile computing device will establish a remote framebuffer at the electronic device of first type and negotiates the compression and/or encryption methodologies to be adopted; iii) operating the mobile client device in the capacity of a RFB server and the electronic device of first type in the capacity of a RFB client; wherein the RFI3 server produces a stream of rectangles representing screen data with a given position and size, puts them into its frarnebuffer and sends them across to the RFB client at the framebuffer level so that they appear in the correct place on the screen; iv) on receiving the multimedia content, the electronic device of first type applies one or plurality of the following multimedia processing operations in S order to render the multimedia content on the external display apparatus or multimedia player: a) Transizing or Video scaling functionality for converting video signals from one size or resolution to another with or without dc-interlacing.E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target device's screen size and colour availability; b) Transcoding -multi-format/-coded-compression support for simultaneous encoding and decoding of a wide variety of codeefeompression formats; c) Transrating -in this process the media content is coded to a lower hit-rate without changing video formats; d) digital to analog conversion depending on the communication interface of second type being used and characteristics of the end display apparatus; e) audio synchroniser for the purpose of ensuring audio-to-video synchronisation 1 digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it S travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia interface (HDM1), Gigabit Video interface (GVIF), or Unified Display interface (UDI) connections; g) error concealment technique using either spatial or temporal interpolation; and, h) 2D to 3D conversion or vice-versa; wherein the application of one or plurality of the above-mentioned multimedia processing operations depends on the received media type, format, resolution, bit-rates, frame rates, colour-depths, aspect ratio and the like, and the capabilities of the end sink device (i.e., external display apparatus or multimedia player).
- 37. The said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according to claim 34 or 35. wherein the said method employs screen-scraping along with multimedia redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus along with playing back the associated audio; wherein the mobile computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side nuiltitnedia elements are sent in their native formats together with the respective timestamp information down to the electronic device of first type, which in S turn renders the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizes the associated audio, if any, with the video and this requires: i) the electronic device of first type has the technical capability and hardwarespecifications to render the multimedia; and,ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full synchronised audio and video experience. iSAmendments to the claims have been filed as follows.Claims This new set of claims is based on the original embodiments and the patent statement as submitted originally on the 13th of December 2010. Additional explanations appear in a separate document named GB1021119.1_ClaimAmendmentStatement.doc submitted along with this.1. (Amended) There is provided an electronic device of first type that receives multimedia content from a mobile computing device or a media capturing device such as a webcam, security camera, camcorder or similar device and outputs/renders /displays/plays-back it on an external visual display apparatus such as a television set or multimedia player in real-time, and the said electronic device of first type comprising: LU. . . ..o i) a wireless transceiver providing a communication interface of first type to 0") communicate with a mobile computing device or a media capturing device; 0.. ...ii) a communication interface of second type to communicate with an external visual display apparatus or multimedia player; wherein the said electronic device of first type ensures the following whenever a mobile computing device or a media capturing device attempts to project multimedia content onto an external visual display apparatus: a) Easy physical connection establishment by operating the said wireless transceiver in the capacity of a wireless access point with a specific SSID having DHCP functionality that can assign a unique zeroconf identifier to a mobile computing device or a media capturing device pertaining to the communication interface of first type on successful association by a user; b) A session-related logical connection establishment upon successful physical connection establishment namely for the purpose of limiting the number of mobile computing device or a media capturing device that can project simultaneously after checking the current operating status of the involved external visual display apparatus, priority-based access and to agree on initial session-related configuration parameters; wherein the said electronic device of first type takes the multimedia content being transmitted via the communication interface of first type as the main input, applies none, one or plurality of media processing operations on to it depending on the type of external display apparatus and the input multimedia content type, and outputs/renders it to an external display apparatus and/or multimedia player via the communication C\i interface of second type for the purpose of instantaneous playback only after a r LI) successful association/connection.2. (Amended) The said electronic device of first type according to claim 1, wherein the said physical connection establishment is namely for the purpose of authenticating a mobile computing device or a media capturing device after a capacity-based access control and on successful authentication, establishing a physical connection after having assigned a unique zeroconf identifier to a connected a mobile computing device or a media capturing device pertaining to the communication interface of first type.3. (Original claim 9) The said electronic device of first type according to claim 1, wherein the said session-related logical connection establishment ensures that said electronic device of first type accepts connection requests being originating from one or plurality of mobile computing devices and subject every connection request to a priority-based access scheme comprising the steps of: i) each mobile computing device needs to be assigned a priority number based on who carries that mobile computing device (e.g., whether the main house-holder/manager); ii) initial connection request of a mobile computing device contains the said priority number; and, iii) the electronic device of first type analysing each connection request and connect a device based on the priority order; wherein in case a low priority device was already attached to the electronic device of first type, the connection will be replaced by a high-priority device (\J 4. (Original claim 2) The said electronic device of first type according to claim 1 or 2, wherein the said SSID can be user settable in order to make it unique in the sameLUC way it is set by the user when a WLAN is configured. a,5. (Original claim 3) The said electronic device of first type according to claim 1 or 2, wherein the said communication interface of first type is wireless-based and can be implemented based on different flavours of WiFi (i.e., IEEE 802.11) technology and the said electronic device of first type will operate a WiFi access point on the said the communication interface of first type.6. (Original claim 4) The said electronic device of first type according to claim 1 or 2, wherein the communication interface of first type is wireless-based and can be implemented based on Bluetooth, Wireless USB, Wireless HD, Wireless HDMI standard or the like.7. (Original claim 5) The said electronic device of first type according to claim 1 or 2, wherein the communication interface of second type can be wired and implemented based on HDMI/DVI, SCART, IEEE 1394 (i.e., Firewire/iLink/Lynx), RCA-connectors-based Composite/component video, Display Port, Serial Digital Interface, S-Video, VGA and its extensions, or the like.8. (Original claim 11) The said electronic device of first type according to claim 1, wherein the mobile computing device is responsible for capturing/getting the exact or intended instantaneous screen shot/content with/without the associated audio content using relevant software and/or hardware components, and transmitting it to the said electronic device of first type in real time.LI) 9. (New but based on the original claim 29) The said electronic device of first type according to claim 1 or 8, wherein the mobile computing device is installed and 0 running a purpose-built host software module; wherein the said purpose-built host software module provides appropriate user interface and allows the user to connect the mobile computing device to the said electronic device of first type for the purpose of constantly projecting the instantaneous screen display of the mobile computing device along with the associated audio content, if any, in real time on to the neighbouring external display apparatus or multimedia player 10. (Original claim 12) The said electronic device of first type according to claim 1 or 8, wherein the mobile computing device captures the multimedia content and compress it before being transmitted to the electronic device of first type; wherein the compression can be based on MPEG-4.11. (Original claim 13) The said electronic device of first type according to claim 1 or 8, wherein the said mobile computing device employs screen scraping and multimedia redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus or multimedia player together with playing-back the associated audio; wherein the mobile computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side multimedia elements are sent in their native formats together with the respective timestamp information down to the electronic device of first type, which in turn renders/plays the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizing the associated audio, if r any, with the video and this requires:LUO i) the electronic device of first type has the technical capability and hardware0) specifications to render the multimedia; and,ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full synchronised audio and video experience.12. (Original claim 14) The said electronic device of first type according to claim 1 or 8, wherein the said mobile computing device that is equipped with an appropriate hardware, software or combination of both performs screen-casting that simply record the desktop screen activity to standard video files and output it, wherein it runs silently in the background, records a movie of the desktop screen, writes it constantly into a video framebuffer and flushes the framebuffer to the electronic device of first type instantaneously (i.e., continuously push its screen activity to the client) and this takes place at a framebuffer-level or Graphics Device Interface (GDI) level; wherein the mobile computing device functions as a server that sends screen updates to the electronic device of first type which in turn functions as a client.13. (Original claim 15) The said electronic device of first type according to claim 1 or 8, wherein in case REB is implemented between the said mobile computing device and the said electronic device of first type, the mobile client device operates in the capacity of a RFB server and the electronic device of first type operates in the capacity of a RFB client; wherein the RFB server produces a stream of rectangles representing screen data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the framebuffer level so that they appear inLUC the correct place on the screen. a,14. (Original claim 16) The said electronic device of first type according to claim 1 or 13, wherein in case REB is implemented between the said mobile computing device and the said electronic device of first type the RFB server can send the compressed/encoded screen data to the RFB client and the compression/encoding technique to be employed can be agreed at the time of initial negotiation.15. (Original claim 17) The said electronic device of first type according to claim 1 or 14, wherein in case RFB is implemented between the said mobile computing device and the said electronic device of first type the RFB server sends updates only when changes (delta) to the screen are required, and only sends the areas of the screen which have changed and the receiving side (REB client) reconstructs the whole image from the delta.16. (Original claim 18) The said electronic device of first type according to claim 1 or 14, wherein in case REB is implemented between the said mobile computing device and the said electronic device of first type, the mobile client device has a touch screen and the user wants to use its screen as a GUI expecting user inputs only and the external display apparatus as the only output screen (i.e., when the RFB server is not associated with an existing physical screen), the RFB server will run a "virtual frarnebuffer' and the remote RFB client has the only physical screen showing the contents of that framebuffer. c\J17. (Original claim 20) The said electronic device of first type according to claim o 1, wherein the said electronic device of first type should have the relevant associated Q") electronic circuitries to output the received multimedia in the appropriate format as demanded by different varieties of communication interfaces of second type and to adapt the bit rate, resolution, frame rate and codec format being received in order to match the capabilities of the sink external display apparatus or media players, and this requires that the said electronic device of first type comprises one or plurality of the following functionalities: a) Transizing or Video scaling functionality for converting video signals from one size or resolution to another with or without de-interlacing. E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target device's screen size and colour availability; b) Transcoding -multi-format/-coded-compression support for simultaneous encoding and decoding of a wide variety of codec/compression formats; c) Transrating -in this process the media content is coded to a lower bit-rate without changing video formats; d) digital to analog conversion depending on the communication interface of second type being used and characteristics of the end display apparatus; e) audio synchroniser for the purpose of ensuring audio-to-video synchronisation 1) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM C\i -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying LU of digital audio and video content as it travels across DisplayPort, Digital 0 Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video Interface (GVIF), or Unified Display Interface (UDI) connections; g) error concealment technique using either spatial or temporal interpolation; h) extra functionalities to amplify the audio signal; and, i) 2D to 3D conversion or vice-versa 18. (Original claim 21) The said electronic device of first type according to claim 1, wherein the said electronic device of first type gets audio signal from a television/multimedia-player via a communication interface of third type and makes it available, wherein one or plurality of wireless headphones can connect to the said electronic device of first type to stream the audio via a communication interface of fourth type and playback in real-time for the purpose of silently watching a television/multimedia-player.19. (Original claim 22) The said electronic device of first type according to claim 1 or 18, wherein the communication interface of third type can be any wired-based audio interface and inter-connect, whereas the said communication interface of fourth type can be Bluetooth.20. (Original claim 24) The said electronic device of first type according to claim 1, wherein the said electronic device of first type has ports such as USB that can be used by a user to connect an external flash memory and to upload one or plurality of still images to the electronic device of first type for the purpose of displaying one or plurality of the uploaded still images continuously (like Slideshows) on the external LU. ..o display apparatus and this is activated under two circumstances: Q") i) automatically after a certain time-out when the external display apparatus is in its power-saving mode; or, ii) manually when the electronic device of first type has an external electronic button/switch and pressing/activating this button/switch will trigger the said Slideshow operation at the discretion of the user.Wherein in either of the above case, the user has the option to set as to how long each still image of a Slideshow needs to be displayed.21. (New but based on the original claim 30) The said electronic device of first type according to claim 1 or 9, wherein; wherein the said purpose-built host software module is responsible for: i) associating the priority of the user to the mobile computing device; ii) establishes the necessary session-related logical connection with the electronic device of first type when the mobile computing device maintains a successful physical connection; iii) capturing instantaneous multimedia content either by tapping into the internal or external graphic and/or sound cards, and applying one or more media processing operations, if needed; and, iv) performing packetization before transporting the captured media streams in real-time to the electronic device of first type.22. (New but based on the original claim 31) The said electronic device of first type according to claim 1 or 21,, wherein the said purpose-built host software module (\J performs screen-scraping and multimedia redirection. rLU23. (New but based on the original claim 33) There is provided a an electronic a, o system on a chip that receives multimedia content from a mobile computing device or a media capturing device such as a webcam, security camera, camcorder or similar device in real-time and renders/displays/plays-back it on an external visual display apparatus such as a television set or multimedia player or multimedia player, and the system on a chip comprising: i) a bus, ii) a peripheral controller being coupled to said bus for connecting a multitude of 110 peripheral devices; iii) one or plurality of microprocessors being coupled to said bus and said microprocessors configured to perform the required signal processing associated with the multimedia traffic and to provide control functions of said communication device; iv) a communication controller being coupled to said bus for connecting to one or plurality of communication interfaces; and, v) a memory that is coupled to said bus into which a plurality of instructions are loaded, wherein the said system on a chip composes a purposeful system on-the-fly by getting any mobile computing device or a media capturing device to join while handling operations such as: a) operating a WLAN access point on the communication interface of first type with a user configurable SSID; b) initiating a DHCP functionality; c) when a mobile computing device or a media capturing device attempts to C\i associate with the said WLAN access point, assigning a unique zeroconf identifier to the connected device pertaining to the communication interface of first type 0 only after a successful capacity-based and/or priority-based access control. a)24. (New) There is provided a method that helps a user project the multimedia content of a mobile computing device or a media capturing device such as a webcam, security camera, camcorder or similar device to an external display apparatus such as a television set or multimedia player via an electronic device of first type; wherein the said method comprises the following steps for the purpose of establishing a multimedia session between the mobile computing device and the electronic device of first type: i) getting the electronic device of first type to operate its own wireless access point; ii) when a user of a mobile computing device or a media capturing device intends to initiate multimedia projection on to an external sink device, getting the user to associate the mobile computing device or a media capturing device with the wireless access point being operated by said electronic device of first type by initiating the physical connection establishment procedure; ii) once the said physical connection establishment is successful, getting the mobile computing device or a media capturing device to establish a session-related logical connection namely for the purpose of limiting the number of mobile computing device or a media capturing device that can project simultaneously after checking the current operating status of the involved external visual display apparatus, priority-based access and to agree on initial session-related configuration parameters; wherein the said electronic device of first type takes the multimedia content being transmitted via the communication interface of first type as the main input, applies CtJ none, one or plurality of media processing operations on to it depending on the type of external display apparatus and the input multimedia content type, and outputs/renders 0 it to an external display apparatus and/or multimedia player via the communication interface of second type for the purpose of instantaneous playback only after a successful association/connection.25. (Original claim 34) The said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according to claim 24; and the said method comprises the following steps for the purpose of configuring the electronic device of first type to operate a WLAN access point on the communication interface of first type and this initial configuration procedure comprising the steps of: i) setting an appropriate SSID for the said WLAN access point that can uniquely identify the user/household/office where it is to be used; ii) getting the WLAN access point to either advertise/broadcast its SSID or hide it; iii) setting encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials for the purpose of restricting connections of any mobile computing device or any media capturing device such as a security-camera/web-cam to the said WLAN access point; wherein the user can connect the electronic device of first type to a computer in order to perform the initial WLAN configuration in the same way a normal WLAN is configured and once the said initial configuration is complete, the user can connect the electronic device of first type to the external display apparatus or a multimedia C\i player via a communication interface of second type and this will make the electronic LU device of first type ready for its intended usage.0) 26. (Original claim 35) The said method that helps a user project the multimedia content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according to claim 24 or 25; and the said physical connection establishment procedure for the purpose of establishing a physical connection between the mobile computing device and the electronic device of first type comprising the steps of: i) getting the mobile computing device to look for a WLAN access point with a specific SSID; ii) once found, prompting the user of a mobile computing device to enter the necessary encryption credentials (e.g., WEP key or WAP key) and/or any other login credentials assigned to the user. On the other hand, in case, the specific SSID is not found mainly because the network does not advertise/broadcast the SSID, prompting the user to manually enter the SSID and the profile id, if any, and to try reconnecting to the WLAN access point belonging to the electronic device of first type; iii) when the login and/or encryption credentials are received by the electronic device of first type, it needs to authenticate the mobile computing device that attempts to connect; iv) when the authentication succeeds, the electronic device of first type needs to assign a unique zeroconf identifier to the connected mobile computing device pertaining to the communication interface of first type; wherein once the physical channel is established, the mobile computing device is ready to initiate any communication session with the electronic device of first type. c\J27. (Original claim 36) The said method that helps a user project the multimediaLUC content of a mobile computing device to an external display apparatus such as a 0) television set or multimedia player via an electronic device of first type according toCclaim 24 or 26, wherein the said method further comprises the following steps for the purpose of establishing a multimedia session between the mobile computing device and the electronic device of first type: i) once the said session-related logical connection is successful the mobile computing device will establish a remote framebuffer at the electronic device of first type and negotiates the compression and/or encryption methodologies to be adopted; ii) operating the mobile client device in the capacity of a RFB server and the electronic device of first type in the capacity of a RFB client; wherein the RFB server produces a stream of rectangles representing screen data with a given position and size, puts them into its framebuffer and sends them across to the RFB client at the framebuffer level so that they appear in the correct place on the screen; iii) on receiving the multimedia content, the electronic device of first type applies one or plurality of the following multimedia processing operations in order to render the multimedia content on the external display apparatus or multimedia player: a) Transizing or Video scaling functionality for converting video signals from one size or resolution to another with or without de-interlacing. E.g., up-scales/up-converts the media content in real-time that is originally intended for a mobile computing device to the external display apparatus's native resolution -i.e., this low resolution image might be transcoded to a higher resolution image with high colour in order to better fit the target device's screen size and colour availability; b) Transcoding -multi-format/-coded-compression support for simultaneous 0 encoding and decoding of a wide variety of codec/compression formats; c) Transrating -in this process the media content is coded to a lower bit-rate without changing video formats; d) digital to analog conversion depending on the communication interface of second type being used and characteristics of the end display apparatus; e) audio synchroniser for the purpose of ensuring audio-to-video synchronisation f) digital rights management (DRM) or conditional access (CA) for the purpose of protecting the media content rights or transcripting between CAs and DRM -e.g., High-bandwidth Digital Content Protection (HDCP) to prevent copying of digital audio and video content as it travels across DisplayPort, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), Gigabit Video Interface (GVIF), or Unified Display Interface (UDI) connections; g) error concealment technique using either spatial or temporal interpolation; and, h) 2D to 3D conversion or vice-versa; wherein the application of one or plurality of the above-mentioned multimedia processing operations depends on the received media type, format, resolution, bit-rates, frame rates, colour-depths, aspect ratio and the like, and the capabilities of the end sink device (i.e., external display apparatus or multimedia player).28. (Original claim 37) The said method that helps a user project the multimedia (\J content of a mobile computing device to an external display apparatus such as a television set or multimedia player via an electronic device of first type according toLUO claim 24, wherein the said method employs screen-scraping along with multimedia 0) redirection technique that allows desktop running on the mobile computing device to show up on an external display apparatus along with playing back the associated audio; wherein the mobile computing device operates in the capacity of a server and the electronic device of first type operates in the capacity of client; wherein the server-side multimedia elements are sent in their native formats together with the respective timestamp information down to the electronic device of first type, which in turn renders the multimedia streams locally by dynamically inserting them back into the proper position on the screen of the external display apparatus and synchronizes the associated audio, if any, with the video and this requires: i) the electronic device of first type has the technical capability and hardwarespecifications to render the multimedia; and,ii) the electronic device of first type has the proper codecs installed so that it knows how to render the multimedia content; wherein the multimedia stream is decoded locally on the electronic device of first type using the time-stamp information thereby providing a full synchronised audio and video experience. c\J rLU a)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB1021119.1A GB2486425A (en) | 2010-12-13 | 2010-12-13 | Rendering multimedia content from a mobile device onto an external display device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB1021119.1A GB2486425A (en) | 2010-12-13 | 2010-12-13 | Rendering multimedia content from a mobile device onto an external display device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| GB201021119D0 GB201021119D0 (en) | 2011-01-26 |
| GB2486425A true GB2486425A (en) | 2012-06-20 |
Family
ID=43567109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB1021119.1A Withdrawn GB2486425A (en) | 2010-12-13 | 2010-12-13 | Rendering multimedia content from a mobile device onto an external display device |
Country Status (1)
| Country | Link |
|---|---|
| GB (1) | GB2486425A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014135670A1 (en) * | 2013-03-08 | 2014-09-12 | Koninklijke Philips N.V. | Wireless docking system for audio-video |
| CN105607567A (en) * | 2015-12-22 | 2016-05-25 | 北京大唐兴业国际控制技术有限公司 | Integration pavilion multimedia exhibit monitoring system |
| EP2981020A4 (en) * | 2013-03-26 | 2016-08-03 | Sony Corp | INFORMATION PROCESSING DEVICE, COMMUNICATION SYSTEM, INFORMATION PROCESSING METHOD, AND PROGRAM |
| EP3846486A4 (en) * | 2018-08-30 | 2021-08-18 | Panasonic Intellectual Property Management Co., Ltd. | DEVICE FOR THE OUTPUT OF VIDEO PICTURE SIGNALS, CONTROL PROCEDURES AND PROGRAM |
| CN113923508A (en) * | 2021-09-29 | 2022-01-11 | 联想(北京)有限公司 | Screen projection method and device |
| WO2025029834A1 (en) * | 2023-07-31 | 2025-02-06 | Spdr Innovations Llc | Intelligent door screen system for providing safety and security |
| US12496972B2 (en) | 2024-01-04 | 2025-12-16 | Toyota Motor North America, Inc. | Causing a plurality of representations of images, produced by a camera, to be presented on a plurality of displays disposed on a vehicle |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116339659A (en) * | 2021-12-24 | 2023-06-27 | 成都极米科技股份有限公司 | A projection screen display method, device, equipment and computer storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2366954A (en) * | 2000-05-18 | 2002-03-20 | Ascot Man Solutions Ltd | Displaying mobile internet information |
| EP1307062A1 (en) * | 2001-10-24 | 2003-05-02 | Nokia Corporation | User interface for transmitting video data from a mobile device to an external display |
| US20050198357A1 (en) * | 2004-01-30 | 2005-09-08 | Shunji Fujita | Electronic device and control method therefor |
| WO2006020119A2 (en) * | 2004-07-16 | 2006-02-23 | Wang Tiejun Ronald | Mobile terminal signal conversion for external display |
| US20060259942A1 (en) * | 2005-05-13 | 2006-11-16 | Microsoft Corporation | Phone-to-monitor connection device |
| US20090088210A1 (en) * | 2007-10-01 | 2009-04-02 | Samsung Electronics Co., Ltd. | Mobile terminal and method of displaying image using the same |
| US20100271559A1 (en) * | 2008-12-03 | 2010-10-28 | Digital Delivery Networks, Inc. | Network accessible display |
-
2010
- 2010-12-13 GB GB1021119.1A patent/GB2486425A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2366954A (en) * | 2000-05-18 | 2002-03-20 | Ascot Man Solutions Ltd | Displaying mobile internet information |
| EP1307062A1 (en) * | 2001-10-24 | 2003-05-02 | Nokia Corporation | User interface for transmitting video data from a mobile device to an external display |
| US20050198357A1 (en) * | 2004-01-30 | 2005-09-08 | Shunji Fujita | Electronic device and control method therefor |
| WO2006020119A2 (en) * | 2004-07-16 | 2006-02-23 | Wang Tiejun Ronald | Mobile terminal signal conversion for external display |
| US20060259942A1 (en) * | 2005-05-13 | 2006-11-16 | Microsoft Corporation | Phone-to-monitor connection device |
| US20090088210A1 (en) * | 2007-10-01 | 2009-04-02 | Samsung Electronics Co., Ltd. | Mobile terminal and method of displaying image using the same |
| US20100271559A1 (en) * | 2008-12-03 | 2010-10-28 | Digital Delivery Networks, Inc. | Network accessible display |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014135670A1 (en) * | 2013-03-08 | 2014-09-12 | Koninklijke Philips N.V. | Wireless docking system for audio-video |
| CN105122822A (en) * | 2013-03-08 | 2015-12-02 | 皇家飞利浦有限公司 | Wireless docking system for audio-video |
| RU2648982C2 (en) * | 2013-03-08 | 2018-03-29 | Конинклейке Филипс Н.В. | Wireless docking system for audio-video |
| CN105122822B (en) * | 2013-03-08 | 2018-10-26 | 皇家飞利浦有限公司 | Wireless docking system for audio and video |
| US10863233B2 (en) | 2013-03-08 | 2020-12-08 | Koninkllijke Philips N.V. | Wireliss docking system for audio-video |
| EP2981020A4 (en) * | 2013-03-26 | 2016-08-03 | Sony Corp | INFORMATION PROCESSING DEVICE, COMMUNICATION SYSTEM, INFORMATION PROCESSING METHOD, AND PROGRAM |
| US10044692B2 (en) | 2013-03-26 | 2018-08-07 | Sony Corporation | Information processing apparatus for controlling data transmission between devices in a network |
| CN105607567A (en) * | 2015-12-22 | 2016-05-25 | 北京大唐兴业国际控制技术有限公司 | Integration pavilion multimedia exhibit monitoring system |
| EP3846486A4 (en) * | 2018-08-30 | 2021-08-18 | Panasonic Intellectual Property Management Co., Ltd. | DEVICE FOR THE OUTPUT OF VIDEO PICTURE SIGNALS, CONTROL PROCEDURES AND PROGRAM |
| CN113923508A (en) * | 2021-09-29 | 2022-01-11 | 联想(北京)有限公司 | Screen projection method and device |
| WO2025029834A1 (en) * | 2023-07-31 | 2025-02-06 | Spdr Innovations Llc | Intelligent door screen system for providing safety and security |
| US12496972B2 (en) | 2024-01-04 | 2025-12-16 | Toyota Motor North America, Inc. | Causing a plurality of representations of images, produced by a camera, to be presented on a plurality of displays disposed on a vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| GB201021119D0 (en) | 2011-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10637894B2 (en) | Real-time placeshifting of media content to paired devices | |
| US9525998B2 (en) | Wireless display with multiscreen service | |
| US9665336B2 (en) | Direct streaming for wireless display | |
| US8983555B2 (en) | Wireless communication techniques | |
| GB2486425A (en) | Rendering multimedia content from a mobile device onto an external display device | |
| US20090235170A1 (en) | Methods and apparatus for sharing either a computer display screen or a media file and selecting therebetween | |
| CN107113465B (en) | Method, device and system for audiovisual synchronization with a plurality of output devices | |
| JP6337114B2 (en) | Method and apparatus for resource utilization in a source device for wireless display | |
| CN101669361A (en) | Methods and systems for improving low resolution and low frame rate video | |
| US20140176664A1 (en) | Projection apparatus with video conference function and method of performing video conference using projection apparatus | |
| US20150304526A1 (en) | External video locking and synchronization device | |
| WO2016200520A1 (en) | Tunneling hdmi data over wireless connections | |
| JP2016129410A (en) | Content providing method and receiving apparatus | |
| US20190028522A1 (en) | Transmission of subtitle data for wireless display | |
| US12342024B2 (en) | Electronic apparatus, server and method of controlling the same | |
| KR101493772B1 (en) | Device for multimedia data repeating | |
| WO2023273992A1 (en) | Wireless transmission system, method and apparatus | |
| WO2023279895A1 (en) | Display device and screen projection method | |
| CN115967830A (en) | A display device and method for synchronizing audio and video | |
| CN119629394A (en) | A multi-screen sharing and interactive control system and method | |
| HK1180126A (en) | Wireless communication techniques | |
| KR20050045173A (en) | Method for adjusting a screen size of network media transceiver |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| WAP | Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1) |