Development of Image Pre-Processing Methods for Software Compensation of Anomal Refraction of the Observer’s Eyes

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription Access

Abstract

In recent decades, the practice of demonstrating various static and video images to users using digital, processor-controlled, most often self-luminous devices (computer monitors, smartphone and tablet screens, etc.) has spurred the development of various methods to improve the perception of such images by means of computerized image preprocessing. This also applies to methods of preprocessing images shown to users with various refractive anomalies of the eye(s) (e.g., myopia or astigmatism) in situations where they are not armed with glasses or other corrective devices. Over the past 20+ years, researchers have published dozens of papers on this task, referred to as the precompensation task. In our opinion, the time has come to reflect on the development of scientific thought in this direction and to highlight the most important milestones in realizing the problems on the way to achieving “ideal” precompensation and in approaches to their successful solution. This is the focus of the first part of this review. In the second part, we focus on the current state of research in the stated area, highlight the problems not solved so far, and try to catch the trends of further development of image precompensation methods, paying maximum attention to neural network approaches.

About the authors

N. B. Alkzir

HSE University; Institute for Information Transmission Problems (Kharkevich Institute)

Author for correspondence.
Email: nafekzir@gmail.com
Russian Federation, Moscow; Moscow

M. S. Yarykina

Institute for Information Transmission Problems (Kharkevich Institute); Computer Science and Control Federal Research Center of the RAS

Email: nafekzir@gmail.com
Russian Federation, Moscow; Moscow

D. P. Nikolaev

Computer Science and Control Federal Research Center of the RAS; Smart Engines Service LLC

Email: nafekzir@gmail.com
Russian Federation, Moscow; Moscow

I. P. Nikolaev

Institute for Information Transmission Problems (Kharkevich Institute)

Email: nafekzir@gmail.com
Russian Federation, Moscow

References

  1. Yablokov M. G., Machekhin V. A., Doga A. V., Kolotov M. G., Vartapetov S. K., Larichev A. V., Iroshnikov N. G. Results of wavefront studies on the first Russia aberrometer “Multispot-250”. Oftal’mokhirurgiya. 2005. V. 2. P. 4–8. (In Russian).
  2. Agarwal C., Khobahi S., Bose A., Soltanalian M., Schonfeld D. Deep-URL: A model-aware approach to blind deconvolution based on deep unfolded Richardson–Lucy network. 2020 IEEE international conference on image processing (ICIP). IEEE, 2020. P. 3299–3303. DOI:
  3. Alkzir N., Nikolaev I., Nikolaev D. SCA-2023: A two-part dataset for benchmarking the methods of image precompensation for users with refractive errors. ECMS. 2023. P. 298–305. doi: 10.7148/2023-0298
  4. Alkzir N. B., Nikolaev I. P., Nikolaev D. P. Search for image quality metrics suitable for assessing images specially precompensated for users with refractive errors. Sixteenth international conference on machine vision (ICMV 2023). 2024. V. 13072. P. 230–238. DOI:
  5. Alonso M. Jr, Barreto A., Cremades J. G., Jacko J. A., Adjouadi M. Image pre-compensation to facilitate computer access for users with refractive errors. Behaviour Information Technology. 2005a. 24(3). P. 161–173. doi: 10.1080/01449290412331327456
  6. Alonso M. Jr, Barreto A., Jacko J. A., Adjouadi M., Choudhury M. Improving computer interaction for users with visual acuity deficiencies through inverse point spread function processing. Proceedings IEEE southeastcon. 2005b. P. 421–427. doi: 10.1109/SECON.2005.1423281
  7. Alonso M. Jr, Barreto A. B. Pre-compensation for high-order aberrations of the human eye using on-screen image deconvolution. Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society (IEEE cat. no. 03ch37439). 2003. V. 1. P. 556–559. doi: 10.1109/IEMBS.2003.1279804
  8. Alonso M. Jr, Barreto A., Adjouadi M. Digital image inverse filtering for improving visual acuity for computer users with visual aberrations. Inverse Problems in Science and Engineering. 2008. V. 16(8). P. 957–966. doi: 10.1080/17415970802082823
  9. Campbell C. E. Matrix method to find a new set of zernike coefficients from an original set when the aperture radius is changed. JOSA A. 2003. V. 20(2). P. 209–217. doi: 10.1364/JOSAA.20.000209
  10. Cascarano P., Sebastiani A., Comes M. C., Franchini G., Porta F. Combining weighted total variation and deep image prior for natural and medical image restoration via admm. I2021 21st international conference on computational science and its applications (ICCSA). IEEE, 2021. P. 39–46. doi: 10.1109/ICCSA54496.2021.00016
  11. Chaganova O., Grigoryev A., Nikolaev D., Nikolaev I. (2024). Applied aspects of modern non-blind image deconvolution methods. Computer Optics. 2024. V. 48(4). to be published. doi: 10.18287/2412-6179-CO-1409
  12. Dong J., Roth S., Schiele B. Deep wiener deconvolution: Wiener meets deep learning for image deblurring. Advances in Neural Information Processing Systems. 2020. V. 33. P. 1048–1059.
  13. Fernández E. J. Adaptive optics for visual simulation. International Scholarly Research Notices. 2012. V. 2012(1). P. 104870. doi: 10.5402/2012/104870
  14. Fine E. M., Peli E. Enhancement of text for the visually impaired. JOSA A. 1995. V. 12(7). P. 1439–1447. doi: 10.1364/JOSAA.12.001439
  15. Gong D., Zhang Z., Shi Q., van den Hengel A., Shen C., Zhang Y. Learning deep gradient descent optimization for image deconvolution. IEEE transactions on neural networks and learning systems. 2020. V. 31(12). P. 5468–5482. doi: 10.1109/TNNLS.2020.2968289
  16. Goodman J. W. Introduction to fourier optics. McGraw-Hill, 1968.
  17. Güzel A. H., Beyazian J., Chakravarthula P., AKS, it K. Chromacorrect: prescription correction in virtual reality headsets through perceptual guidance. Biomedical Optics Express. 2023. V. 14(5). P. 2166–2180. doi: 10.1364/BOE.485776
  18. Holden B. A., Fricke T. R., Wilson D. A., Jong M., Naidoo K. S., Sankaridurg P., Wong T. Y., Naduvilath T. J., Resnikoff S. Global prevalence of myopia and high myopia and temporal trends from 2000 through 2050. Ophthalmology. 2016. V. 123(5). P. 1036–1042. DOI:
  19. Huang F.-C., Lanman D., Barsky B. A., Raskar R. Correcting for optical aberrations using multilayer displays. ACM transactions on graphics (TOG). 2012a. V. 31(6). P. 1–12. doi: 10.1145/2366145.2366204
  20. Huang J., Barreto A., Adjouadi M. Dynamic image pre-compensation for computer access by individuals with ocular aberrations. 2012 annual international conference of the IEEE engineering in medicine and biology society. 2012c. P. 3320–3323. doi: 10.1109/EMBC.2012.6346675
  21. Huang J., Barreto A., Adjouadi M. Evaluation of dynamic image pre-compensation forcomputer users with severe refractive error. Proceedings of the 14th international ACM SIGACCESS conference on Сomputers and accessibility. 2012b. P. 175–182. doi: 10.1145/2384916.238494
  22. Huang J., Barreto A., Alonso M. Jr, Adjouadi M. Vision correction for computer users based on image pre-compensation with changing pupil size. 2011 annual international conference of the IEEE engineering in medicine and biology society. 2011. P. 4868–4871. doi: 10.1109/IEMBS.2011.6091206
  23. Huang J., Barreto A., Alonso M. Jr, Adjouadi M. Contrast improvement in pre-compensation of ocular aberrations for computer users. Biomedical sciences instrumentation. 2012d. V. 48. P. 179–186. doi: 10.1007/978-1-4614-3535-8_32
  24. Ji Y., Ye J., Bing Kang S., Yu J. Image pre-compensation: Balancing contrast and ringing. Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. P. 3350–3357. doi: 10.1109/CVPR.2014.428
  25. Jumbo O. E., Asfour S., Sayed A. M., Abdel-Mottaleb M. Correcting higher order aberrations using image processing. IEEE Transactions on Image Processing. 2021. V. 30. P. 2276–2287. doi: 10.1109/TIP.2021.3051499
  26. Krishnan D., Fergus R. Fast image deconvolution using hyper-Laplacian priors. Advances in neural information processing Systems. 2009. V. 22. https://proceedings.neurips.cc/paper/2009/file/3dd48ab31d016ffcbf3314df2b3cb9ce-Paper.pdf doi: 10.1145/1531326.1531402
  27. Lawton T. B. Improved word recognition for observers with age- related maculopathies using compensation filters. Clinical Vision Sciences. 1988. V. 3(2, 19). P. 125–135. https://ntrs.nasa.gov/citations/19890037275
  28. Lawton T. B. Improved reading performance using individualized compensation filters for observers with losses in central vision. Ophthalmology. 1989. V. 96 (1). P. 115–126. doi: 10.1016/S0161-6420(89)32935-6
  29. Lawton T. B. Image enhancement filters significantly improve reading performance for low vision observers. Ophthalmic and Physiological Optics. 1992. V. 12(2). P. 193–200. doi: 10.1111/j.1475-1313.1992.tb00289.x
  30. Lucy L. B. An iterative technique for the rectification of observed distributions. Astronomical Journal. 1974. V. 79. P. 745. doi: 10.1086/111605
  31. Mohammadpour S., Mehridehnavi A., Rabbani H., Lakshminarayanan V. A pre-compensation algorithm for different optical aberrations using an enhanced wiener filter and edge tapering. 2012 11th international conference on information science, signal processing and their applications (ISSPA). IEEE, 2012. P. 935–939. doi: 10.1109/ISSPA.2012.6310689
  32. Montalto C., Garcia-Dorado I., Aliaga D., Oliveira M. M., Meng F. A total variation approach for customizing imagery to improve visual acuity. ACM Transactions on Graphics (TOG). 2015. V. 34(3). P. 1–16. doi: 10.1145/2717307
  33. Mou C., Wang Q., Zhang J. Deep generalized unfolding networks for image restoration. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. P. 17399–17410. doi: 10.48550/arXiv.2204.13348
  34. Nan Y., Ji H. Deep learning for handling kernel/model uncertainty in image deconvolution. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. P. 2388–2397. doi: 10.1109/CVPR42600.2020.00246
  35. Peli E., Goldstein R. B., Young G. M., Trempe C. L., Buzney S. M. Image enhancement for the visually impaired. simulations and experimental results. Investigative ophthalmology visual science. 1991. V. 32(8). P. 2337–2350. doi: 10.1109/NEBC.1989.36690
  36. Peli E., Lee E., Trempe C. L., Buzney, S. Image enhancement for the visually impaired: the effects of enhancement on face recognition. JOSA A. 1994. V. 11(7). P. 1929–1939. doi: 10.1364/JOSAA.11.001929
  37. Peli E., Peli T. Image enhancement for the visually impaired. Optical engineering. 1984. V. 23(1). P. 47–51. doi: 10.1117/12.7973251
  38. Peli T., Lim J. S. Adaptive filtering for image enhancement. Optical Engineering. 1982. V. 21(1). P. 108–112. doi: 10.1117/12.7972868
  39. Richardson W. H. Bayesian-based iterative method of image restoration. JOSA. 1972. V. 62(1). P. 55–59. doi: 10.1364/JOSA.62.000055
  40. Schuler C. J., Christopher Burger H., Harmeling S., Scholkopf B. A machine learning approach for non-blind image deconvolution. Proceedings of the IEEE conference on computer vision and pattern recognition. 2013. P. 1067–1074. doi: 10.1109/CVPR.2013.142
  41. Tanaka H., Kawano H. Image correction for improving visual acuity using Zernike-based vision simulation. 2021 20th international symposium on communications and information technologies (ISCIT). 2021. P. 32–36. doi: 10.1109/ISCIT52804.2021.9590607
  42. Thibos L. N. Formation and sampling of the retinal image. Seeing. Academic Press, 2000. P. 1–54. doi: 10.1016/B978-012443760-9/50003-9
  43. Vogel C. R., Oman M. E. Fast total variation-based image reconstruction. International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 1995. V. 97669. P. 1009–1015. doi: 10.1115/DETC1995-0672
  44. Wiener N. Extrapolation, interpolation, and smoothing of stationary time series: With engineering applications. The MIT Press, 1949. doi: 10.7551/mitpress/2946.001.0001
  45. Xu F., Li D. Software based visual aberration correction for hmds. 2018 IEEE conference on virtual reality and 3d user interfaces (VR). 2018. P. 246–250. doi: 10.1109/VR.2018.8447557
  46. Xu L., Ren J. S., Liu C., Jia J. Deep convolutional neural network for image deconvolution. Advances in neural information processing systems. 2014. V. 27.
  47. Ye J., Ji Y., Zhou M., Kang S. B., Yu J. Content aware image pre-compensation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018. V. 41(7). P. 1545–1558. doi: 10.1109/TPAMI.2018.2839115
  48. Zhang K., Gool L. V., Timofte R. Deep unfolding network for image super-resolution. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. P. 3217–3226. doi: 10.1109/CVPR42600.2020.00328
  49. Zhang L., Nayar S. Projection defocus analysis for scene capture and image display. ACM siggraph 2006 papers. 2006. P. 907–915. doi: 10.1145/1179352.1141974

Copyright (c) 2024 Russian Academy of Sciences

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies