A method for automated assessment of the reliability of alternative statements in a collection of scientific articles using the example of the topic “Overton windows”

Cover Page

Cite item

Full Text

Abstract

The paper proposes a method for assessing the reliability of opposing statements/facts based on trends in bibliographic data, provides an example of its use, and discusses the possibility of automating the method and replenishing the fact base. As an example, 1047 articles from the scientific eLibrary containing the words “window” and “Overton” were analyzed. Using the proposed method, it is shown that “working technology” and “pseudo-scientific concept” are alternative points of view on “Overton windows”. It is also shown that the “working technology” point of view is more reliable.

Full Text

Restricted Access

About the authors

Michael M. Charnine

Federal Research Center “Computer Science and Control”, RAS

Author for correspondence.
Email: mc@keywen.com

Candidate of technical sciences, Senior researcher

Russian Federation, Moscow

Nikolay V. Somin

Federal Research Center “Computer Science and Control”, RAS

Email: chri-soc@yandex.ru

Candidate of physical and mathematical sciences, Leading researcher

Russian Federation, Moscow

References

  1. Anufrieva D.Yu., Guzenko A.Yu. Soderzhaniye ponyatiya "kriticheskoye otsenivaniye informatsii" [Content of the concept of “critical assessment of information”] // Problemy sovremennogo pedagogicheskogo obrazovaniya [Problems of modern pedagogical education]. 2017. No. 56-1. P. 3-9.
  2. Vorontsov K.V. Tekhnologii iskusstvennogo intellekta protiv feykov, postpravdy i informatsionnykh voyn [Artificial intelligence technologies against fakes, post-truth and information wars] // Zasedaniye Ekspertnogo soveta po razvitiyu tsifrovoy ekonomiki, tekhnologiy i innovatsiy Molodozhnogo parlamenta pri Gosudarstvennoy dume Federal'nogo Sobraniya RF po teme «Protivodeystviye destruktivnoy ideologii (natsizm, terrorizm, kriminal)». [Meeting of the Expert Council for the development of the digital economy, technology and innovation of the Youth Parliament under the State Duma of the Federal Assembly of the Russian Federation on the topic “Countering destructive ideology (Nazism, terrorism, crime).”] February 17, 2023. Electronic resource. URL: https://rutube.ru/video/8628f049a17ae40e95b8aa33cbfd5b 69/ (accessed 05.10.2023).
  3. Efremov Yu.N. Lzhenauka i gipoteza [Pseudoscience and hypothesis] // Byulleten' «V zashchitu nauki». Komissiya RAN po bor'be s lzhenaukoy. [Bulletin “In Defense of Science”. RAS Commission on Combating Pseudoscience]. 2010. No. 7. P. 51-58.
  4. Melnikov O.N. Retsenzirovaniye kak faktor povysheniya kachestva nauchnoy informatsii [Peer review as a factor in improving the quality of scientific information] // Nauchnaya periodika: problemy i resheniya [Scientific periodicals: problems and solutions]. 2011. No. 2 (2). P. 20-23.
  5. Arazy O., Kopak R. On the measurability of information quality //Journal of the American Society for Information Science and Technology. 2011. V. 62. No. 1. P. 89-99.
  6. Google. Search Quality Evaluator Guidelines // Google, Mountain View, CA, USA, 2018.
  7. Hanselowski A., Stab C., Schulz C., Li Z., Gurevych I. A richly annotated corpus for different tasks in automated fact-checking // arXiv preprint arXiv:1911.01214. October 2019. Electronic resource. URL: https://arxiv.org/pdf/1911.01214.pdf (accessed 05.10.2023).
  8. Sarabadani A., Halfaker A., Taraborelli D. "Building automated vandalism detection tools for Wikidata" // Proc. 26th Int. Conf. World Wide Web Companion (WWW Companion), P. 1647-1654, 2017.
  9. Sugandhika C., Ahangama S. Assessing Information Quality of Wikipedia Articles Through Google’s E-A-T Model // IEEE Access, V. 10, P. 52196-52209, 2022, doi: 10.1109/ACCESS.2022.3172962.
  10. Thorne J., Vlachos A., Christodoulopoulos C., Mittal A. FEVER: a large-scale dataset for fact extraction and VERification // arXiv preprint arXiv:1803.05355. December 2018. Electronic resource. URL: https://arxiv.org/pdf/1803.05355.pdf (accessed 05.10.2023).
  11. Wadden D., Lin S., Lo K., Wang L.L., van Zuylen M., Cohan A. Hajishirzi H. Fact or fiction: Verifying scientific claims // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. 2020. P. 7534–7550.
  12. Wadden D., Lo K., Kuehl B., Cohan A., Beltagy I., Wang L.L., Hajishirzi H. SciFact-open: Towards open-domain scientific claim verification // arXiv preprint arXiv:2210.13777. October 2022. Electronic resource. URL: https://arxiv.org/pdf/2210.13777.pdf (accessed 05.10.2023).
  13. Zhang A. X. et al. A structured response to misinformation: Defining and annotating credibility indicators in news articles //Companion Proceedings of the The Web Conference 2018. 2018. P. 603-612. 1. Anufrieva D.Yu., Guzenko A.Yu. Soderzhaniye ponyatiya "kriticheskoye otsenivaniye informatsii" [Content of the concept of “critical assessment of information”] // Problemy sovremennogo pedagogicheskogo obrazovaniya [Problems of modern pedagogical education]. 2017. No. 56-1. P. 3-9.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. The percentage of articles from the “technology” and “pseudoscientific” groups in relation to the total number of articles in the “Overton windows” group

Download (27KB)
Indexing metadata
3. Fig. 2. The percentage of articles from the “technology” and “pseudoscientific” groups (without review articles) in relation to the total number of articles from the “Overton windows” group

Download (24KB)
Indexing metadata
4. Fig. 3. The 3-year moving average of the share of articles from the “patent” and “pseudoscience” groups (without review articles) as a percentage of the total number of articles from the “torsion fields” group

Download (22KB)
Indexing metadata

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).