Funded projects
ANR ASTRID 2017 DGA DISSOCIE (website): 2017 – 2020
DISSOCIE (Détection automatIque des SaillanceS du point de vue des Opérateurs et Compression Intelligente des vidéos de dronEs)
Summary: The aerial surveillance, monitoring and observation with drone present major challenges in terms of defense, security and environment. For example, France and Britain have agreed to invest 2 billion euros in a project to build next-generation multi-role drones capable of carrying out surveillance and observation missions, identifying targets and launching strikes on enemy territory for future operational capacity beyond 2030. However, the observation, targets identification and surveillance missions are currently being carried out by human operators who do not have the ability to fully and effectively exploit all available drone videos. The science and the technology of the eye-tracking study, visual attention modeling, human operator models, and intelligent compression opens up new possibilities to meet these challenges.
In this context, the DISSOCIE project aims to develop automatic and semi-automatic operator models capable of detecting salient areas from the point of view of human operators, by considering the low-level characteristics of the salient content in the videos, geo-temporally localized contextual information, and the expertise and the detection strategies of human operators. Machine learning can be used at different levels of this modeling process. The new HEVC video compression standard and the scalable coding will also be exploited in this project to improve the efficiency when the experts rewatch the videos. The originality of the project lies in an innovative approach to jointly address these challenges based on the complementarity and the strengthening of the scientific expertise gathered in the consortium: especially on eye-tracking analysis, visual fixation prediction, visual attention modeling, salient object detection and segmentation, human observer modeling, and video compression.
ANR REVERIE 2017 (website): 2017-2021
Summary:
The ReVeRY project will design a specific GRID OF CAMERAS, a cost-efficient system that acquires at once several viewpoints under several exposures and will convert a multiview, multiexposed, video stream into a high quality rich media. In the last two decades, industries and researchers proposed significant advances in media content acquisition systems in three main directions: increase of resolution and image quality with the new ultra-high-definition (UHD) standard; stereo capture for 3D content; and high-dynamic range (HDR) imaging. Compression, representation, and interoperability of these new media are active research fields in order to reduce data size and be perceptually accurate. The creative industries face issues associated to limited possibilities offered by current media. The traditional acquisition pipeline uses and outputs video streams, either UHD, stereo or HDR. After shooting, possible changes on the media (i.e., viewpoint, framing, aperture, lighting) are limited which may force a film director to shoot the scene again if any changes are necessary. For now, a pre-shooting choice is made according to the nature of the media to capture (2D, stereo, or HDR) depending on the foreseen use. However, these choices are currently never jointly offered.
A major breakthrough of the project is to resolve jointly HDR and depth reconstruction. The ReVeRY project UHD and HDR will endow the produced media with geometrical and color precision and quality surpassing current video media formats. Its very nature will furthermore conveniently allow new usages, such as reframing, free viewpoint, and relighting at both postproduction and visualization stages and facilitate live adaptation to different types of displays: theatre, TV, Cloud, HDR or not, 2D or 3D. This later feature makes the new media truly versatile according to consumption conditions.
The huge market investments in virtual reality (VR) are positioning it as a plain new media. VR immersive screens are today very limited, only trapping the user in a sphere’s center of 2D multiaxial images. The virtual cameras of virtual reality propose 360° viewing of a poor and only concave 3D transcriptions. A “liberated” VR requires the creation of convex 3D reconstruction tools using specific camera’s organization for 3D scanning of real environments and moving actors, giving the user total freedom of visualization through unlimited positioning and adaptive resolutions of a true virtual cameras (stereoscopic or not).
ReVeRY wants to remedy the lack of a joint representation of the different media formats discussed above. This project aims at providing solutions to replace the traditional video media (mono, stereoscopic or 360°) by a richer data stream overcoming all previously listed bottlenecks. This richer media will jointly embed UHD, HDR, and Depth information in a dedicated format. The project will (i) propose this new media along with one or more suitable representations, (ii) develop a demonstrative prototype of a dedicated acquisition system, and (iii) prove their benefits to media creation industry. The project will enable versatile postproduction and re-usability of single shootings of live action under less-restrained range of lighting intensities (e.g., outdoor scenes). Its outcome is important with a twofold impact: (i) societal (for consumers and experts), by increasing content quality and (ii) economical, by reducing production costs. Ultimately, the creation approach will evolve to provide a completely new experience of screen viewing to the public.
