A central objective of the PERCEPT team is to address inter-dependent algorithmic problems for predicting visual phenomena that occur when an observer watches a visual stimulus displayed on a specific screen. In the following, we provide a list of challenges that the PERCEPT team is addressing.
- Computational Modelling of Visual Attention
- Image Editing / computational photography
- Aesthetic assessment
- High Dynamic Range
- Unmanned Aerial Vehicle (UAV) imagery
Computational Modelling of visual Attention
Defining the next generation of computational models of visual attention. Visual attention models aim to detect where an observer is looking at when watching a scene displayed on screen. We are focussing on saccadic models which predict the visual scanpaths, i.e. the sequence of fixations and saccades an observer would perform to sample the visual environment, as illustrated below.
[Le Meur & Liu, 2015] Le Meur, O., & Liu, Z. (2015). Saccadic model of eye movements for free-viewing condition. Vision research, 116, 152-164. [project page]
[Le Meur & Coutrot, 2016] Le Meur, O., & Coutrot, A. (2016). Introducing context-dependent and spatially-variant viewing biases in saccadic models. Vision research, 121, 72-84. [project page]
Computational Model for Predicting Visual Fixations from Childhood to Adulthood.
While previous visual attention models output static 2-dimensional saliency maps, saccadic models aim to predict not only where observers look at but also how they move their eyes to explore the scene. We demonstrate that saccadic models are a flexible framework that can be tailored to emulate observer’s viewing tendencies. More specifically, we use fixation data from 101 observers split into 5 age groups (adults, 8-10 y.o., 6-8 y.o., 4-6 y.o. and 2 y.o.) to train our saccadic model for different stages of the development of human visual system.
[Le Meur et al., 2017] Le Meur, O., Coutrot, A., Liu, Z., Rämä, P., Le Roch, A., & Helo, A. (2017). Visual attention saccadic models learn to emulate gaze patterns from childhood to adulthood. IEEE Transactions on Image Processing, 26(10), 4777-4789.
The following video illustrates the maturation of the visual system over age (evolution of the joint probability of saccade amplitudes and orientations):
Feel free to comment!
Defining computational models for characterizing an observer from his eye movements. The eye is the only part of the brain that can be seen directly. This exogenous manifestation of where we look at, carries a wealth of information that can be exploited for many purposes.
Example: deep model for inferring your Age from Your Gaze (98% of accuracy for 5 age categories)
Zhang, A. T., & Le Meur, B. O. (2018, October). How Old Do You Look? Inferring Your Age from Your Gaze. In 2018 25th IEEE International Conference on Image Processing (ICIP) (pp. 2660-2664). IEEE.
Defining computational models for specific contents, such as comics.
[Bannier et al., 2018] Bannier, K., Jain, E., & Meur, O. L. (2018, June). Deepcomics: saliency estimation for comics. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications (p. 49). ACM. [project page]
Image Editing / computational photography
Color transfer.
Transferring features, such as light and colors, between input and reference images is the main objective of color transfer methods. We propose several non-supervised and statistical methods to deal with this problem.
A typical example is shown below:
[Hristova et al., 2018] Hristova, H., Le Meur, O., Cozot, R., & Bouatouch, K. (2018). Transformation of the multivariate generalized Gaussian distribution for image editing. IEEE transactions on visualization and computer graphics, 24(10), 2813-2826.
[Hristova et al., 2017] Hristova, H., Le Meur, O., Cozot, R., & Bouatouch, K. (2018). Transformation of the Beta distribution of color transfer. In VISIGRAPP 2018 (BEST STUDENT PAPER).
[Hristova et al., 2017] Hristova, H., Le Meur, O., Cozot, R., & Bouatouch, K. (2017, September). Perceptual metric for color transfer methods. In Image Processing (ICIP), 2017 IEEE International Conference on (pp. 1237-1241). IEEE. [project page]
[Hristova et al., 2015] Hristova, H., Le Meur, O., Cozot, R., & Bouatouch, K. (2015, June). Style-aware robust color transfer. In Proceedings of the workshop on Computational Aesthetics (pp. 67-77). Eurographics Association.
Image filtering based on color perception through a chromatic adaptation model.
We propose a new guidance-based filter, which carries out a linear transformation between an input image and a guidance image. This filter can efficiently addressed different problems such as image denoising, texture transfer, detail enhancement with NIR images, image deblurring, skin beautification…
[Hristova et al., 2018] Hristova, H., Le Meur, O., Cozot, R., & Bouatouch, K. (2018). Multi-purpose bi-local CAT-based guidance filter. Signal Processing: Image Communication, 65, 141-153.
Context-aware Photo Assessment
An automatic photo assessment can significantly aid the process of photo selection within photo collections. However, existing computational methods approach this problem in an independent manner, evaluating each image apart from other images in a photo album. In our research, we explore the modeling of photo context via a clustering approach for photo collections, with a further adaptation of an independent photo score using the extracted context.
[Kuzovkin et al., 2017] Kuzovkin, D., Pouli, T., Cozot, R., Le Meur, O., Kervec, J., & Bouatouch, K. (2017, July). Context-aware clustering and assessment of photo collections. In Proceedings of the symposium on Computational Aesthetics (p. 6). ACM.
Unmanned Aerial Vehicle (UAV) imagery
The fast and tremendous evolution of the UAV imagery gives place to the multiplication of applications in various fields such as military and civilian security and surveillance systems, delivery services, journalism, or wildlife monitoring.
Combining UAV imagery with the study of dynamic salience further extends the number of future applications. Indeed, considerations of visual attention open the door to new avenues in a number of scientific fields such as compression, retargeting, segmentation, object and person detection, tracking, or decision-making tools.
Within the ongoing research project – ANR-17-ASTR-0009 ASTRID DISSOCIE Automated Detection of SaliencieS from Operators’ Point of View and Intelligent Compression of DronE videos – PECPET and partners developed several major tools to fully take benefits from the drone imaging.
[ Perrin et al., 2019 ] Perrin, A. F., Zhang, L., & Le Meur, O. (2019, September). How well current saliency prediction models perform on UAVs videos?. In International Conference on Computer Analysis of Images and Patterns (pp. 311-323). Springer, Cham.[details]
