With SkyMedia, we are going after an immersive media dream. The objective of the SkyMedia project is to explore, design, and demonstrate a novel multimedia end-to-end architecture that can provide unique immersive media experiences to audiences during live events. The keys to enable such a rich portfolio of experiences are both in the media technologies and in the overall system architecture, which has to be scalable, flexible, and easy to manage.

Our target scenario is the following: imagine a live event, such as a sport match, a musical concert or any occasion where there is a large audience, whose attention is captured by a complex event, in fact so complex that it is almost impossible to appreciate everything that is happening at any one time. This may also depend on one’s seating with respect to where the action is happening, and on the fact that details may be many and difficult to perceive. Our goal is to enhance the audience experience in a unique and innovative way, by exploiting a set of advanced technologies in a synergic dance. The selected demonstration use case is the the Turin Marathon event to be held in the city of Turin in November 2012.  See the Figure below for a pictorial view of the SkyMedia concept.

A number of key technologies are effectively combined to achieve the objectives:

• UAV platforms, characterized by a hybrid design, to provide access to the scene in a silent, convenient, and safe fashion, capturing HD and 3D video and images from the sky; this cannot be done using customary devices such as helicopters, because the noise would disturb, annoy and distract the audience.
• Wireless Sensors, to form an augmentation network deployed on ground to enrich the multimedia content with useful and interesting side-information, to be fused with the main audiovisual content. Content fusion technologies are key, as they satisfy the need to extract the relevant information from a plethora of sensing devices.
• 3D calibration and coding methods to be executed in real-time, fusing and aggregating all received information, transcoding in all necessary formats for distribution and successive rendering.
• Unique, interactive walk-through mid-air screens, materializing high resolution pseudo-holographic scenes. These displays, which use water vapour as a support, allow users to be truly immersed in the image, in a totally natural and unencumbered manner.
• Innovative interactive user applications for mobile terminals with 2D/3D displays, enabling personalization of the provided 3D & HD content. The interaction between users and the multimedia content will be fed back to the control center, and will be used to produce changes onto the content itself.

We believe the proposed SkyMedia concept is an innovative and uniquely immersive HD/3D multimedia end-to-end architecture, taking content beyond the usual boundaries of TV and cinema, which has very significant market potential in the medium-long term, when the future media Internet will be a reality and complex digital services with interactive user control will be widespread.

Relevant stakeholders in the entertainment and media industry, and in the equipment manufacturers industry, including technology end-users and the surrounding market sectors, have been involved from the very first months of project activities and invited to present and interact during project Progress Meetings. One important project achievement is the strong cooperation that has been established with Turin Marathon S.r.l. (Turin, Italy), a service company managing and organizing events including marathons, concerts and music events. In particular, Turin Marthon S.r.l. officially entered the SkyMedia consortium in September 2011, and it is committed to host the pre-tests and the final project demonstration in November 2012.

SkyMedia will empower the audience to interact with the live scene, enabling multiple user roles and providing an immersive media experience demonstration to the users attending the event, as shown in the following screenshots.

The activities carried out during the first project year have spanned over different areas and have been focused identification, definition and design of the SkyMedia system around the selected demonstration use case. The main outcomes of these activities are outlined in the following.

Market and business analysis:

The considered activities have identified the actual and specific business and technology domain where SkyMedia is going to have a real impact. The SkyMedia concept has a potential impact on market dynamics both as a technology framework and as an integrated solution as a whole.

A full market analysis of SkyMedia in terms of system features, architecture and services offered has been carried out. Starting from the state-of-the-art analysis of the SkyMedia component technologies, activities have continued with a business and market analysis along with the identification of important business cases for the prime stakeholder, acting as a service provider in a number of representative template-based scenarios along with the corresponding requirements. The project successfully managed to map all scenario use cases onto a set of project services, extracting their business requirements and the expected actual impact.

The key business concepts that emerged from the analysis are the need for collaboration with network operators and ISPs with enhanced service offering like multiple play (triple, quadruple, or any other hybrid) and online interactive services such as video-on-demand, online gaming, and virtual worlds, and finally collaboration with advertising agents who aim to enrich their advertising content with challenging features increasing promotion and marketing. Several monetization models are proposed on a higher layer for the prime stakeholder who in principle is the live event organizer.

System architecture and full subsystems design

The high-level technical design activities have focused on an innovative immersive and interactive multimedia system, devoted to deliver entertaining experiences to users attending a wide range of event types through the use of unmanned aerial vehicles, arrays of cameras, wireless sensor networks and immaterial screens. One of the main outcomes of this activity is that there is no such thing as a general purpose multimedia system for entertainment. There are many different implementations, each of them specific for a well defined and usually restricted field of application, leading to many niche market sectors. A general yet flexible architecture would allow the ecosystem to exploit an advantage by increasing the system scale and reducing cost.

The proposed SkyMedia concept high level architecture is general and yet ready to be derived for the specific use case, with only a few necessary custom modifications. The common SkyMedia concept design core leads to a design cost reduction and to an increase in the reach of the core design modules, fostering the ecosystem development and most importantly increasing the level of service provided to final users.

The SkyMedia architecture is centered on a Multi-Platform Network Operation Centre that orchestrates, coordinates, and processes the data exchanged with several subsystems. On top of it lies the Multimedia Service Platform (MSP), responsible for providing real–time immersive services to various end-users platforms. These services are transmitted through heterogeneous transmission channels. The MSP is closely connected to an Interactive Mobile Service Platform, responsible to manage User Generated Content (mainly image and text) pushed from end-users smart phones and multimedia contents extracted from social networking websites or media sharing Internet platforms.

Detailed design activities have been completed during the first project year. In the following period, implementation and integration will be performed, refining and finalizing the overall SkyMedia system while heading towards the final project demonstration.