Departamento de Engenharia Electromecânica
Permanent URI for this community
Browse
Browsing Departamento de Engenharia Electromecânica by Subject "3d"
Now showing 1 - 1 of 1
Results Per Page
Sort Options
- Light Field Compression and View Synthesis: An Objective Quality AnalysisPublication . Saraiva, Daniela Ferreira; Pinheiro, António Manuel GonçalvesLight field technology represents a powerful imaging technology that captures the intensity and direction of light rays in a scene, allowing for the reconstruction of 3D information and the ability to perform unique tasks like refocusing images after capture. However, the vast amount of data generated by light field imaging poses significant challenges for storage and transmission, making efficient compression schemes crucial. Compression techniques that incorporate view synthesis during different stages of the compression process have emerged as a promising solution, reducing the amount of data that needs to be transmitted or stored by reconstructing or predicting new views from a limited set of captured data. In this thesis a version of this method will be evaluated. The main goal of this thesis is to study the potential of using view synthesis to improve light field compression while reducing complexity, and focusing on maintaining high image quality while reducing data storage and transmission requirements. In addition to this main goal, there are some secondary objectives related to the tools used in this process. A comparison between the used codecs is done to analyze how the choise of codec impacts the final reconstruction quality when coupled with the view synthesis technique. By focusing on a single, state-of-the-art view synthesis method, its ability to reconstruct high-quality images from compressed data is tested. Additionally, four light fields are used: two from the EPFL dataset, which consists of light fields captured by lenslet cameras, and two from the HCI Light Field Database, consisting of synthetically created light fields. This diversity aims to better understand the variability in performance and the generalization capability of the process. To achieve this, a sparsely sampled light field is created from the original light field by “dropping” views. Both light fields, complete and sampled, are compressed using the following codecs/configurations: JPEG Pleno, VVC LowDelay and VVC Random Access. A learning-based view synthesis method, SepConv++, is applied to the decoded views from the sampled light field, obtaining a reconstructed light field with the same views as the original. Both the fully compressed light field and the sparsely sampled light field, which undergoes compression and view synthesis, are compared to the original light field. This comparison is done using the objective metrics PSNR-HVS-M, MS-SSIM and FSIMc. The results obtained were presented in the format of four tables, one for each light field, with each table containing six plots that illustrate the performance metrics PSNR-HVS-M, MS-SSIM, and FSIMc for every codec/configuration used. This plots also include information regarding the view synthesis, by presenting reference lines regarding its perfomance on non compressed light fields. Additionally, the view synthesis process creates three types of views, the views that were part of the sparsely sampled light field and underwent compression, the first-generation views created during the first view synthesis stage, and the second-generation views generated in a subsequent stage of view synthesis. The metrics regarding these different view types, for all codecs/configurations are also depicted in these plots. Additional results are presented through Bjontegaard metrics, a visual comparison where different view types are presented for selected bitrates, and a table containing the compression times. The analysis across the different codecs, demonstrated that SepConv++ can effectively generate denser light fields from compressed views without significant quality loss. Although sparse light fields require lower bitrates for storage and transmission, achieving comparable image quality levels after view synthesis requires a bitrate similar to the ones used for the fully compressed light fields. The results highlighted that VVC (in either configuration) outperformed JPEG Pleno in quality retention, although the latter is significantly faster. The proposed approach, which utilizes compressed views for synthesis, proved advantageous by significantly reducing computational complexity and resource demands, particularly evident in the VVC codec. However, the initial premise that view synthesis would allow for a reduced bitrate while maintaining the same quality was not fully supported by the results.