Cognitive effort assessment through pupillary responses: Insights from multinomial processing tree modeling and neural interconnections

Gahangir Hossain 1 * , Joshua D. Elkins 2
More Detail
1 University of North Texas, Denton, TX, USA
2 Indiana University–Purdue University Indianapolis, Indianapolis, IN, USA
* Corresponding Author
Online Journal of Communication and Media Technologies, Volume 14, Issue 1, Article No: e202413. https://doi.org/10.30935/ojcmt/14196
OPEN ACCESS   822 Views   541 Downloads   Published online: 10 Feb 2024
Download Full Text (PDF)

ABSTRACT

The pupillary responses of humans exhibit variations in size, which are mediated by optic and oculomotor cranial nerves. Due to their sensitivity and high resolution of pupillary responses, they are used for a long time as measurement metrics of cognitive effort. Investigating the extent of cognitive effort required during tasks of varying difficulty is crucial for understanding the neural interconnections underlying these pupillary responses. This study aims to assess human cognitive efforts involved in visually presented cognitive tasks using the multinomial processing tree (MPT) model, an analytical tool that disentangles and predicts distinct cognitive processes, resulting in changes in pupil diameter. To achieve this, a pupillary response dataset was collected during mental multiplication (MM) tasks and visual stimuli presentations as cognitive tasks. MPT model describes observed response frequencies across various response categories and determines the transition probabilities from one latent state to the next. The expectation maximization (EM) algorithm is employed with MPT model to estimate parameter values based on response frequency within each category. Both group-level and individual subject-to-subject comparisons are conducted to estimate cognitive effort. The results reveal that in the group comparison and with respect to task difficulty level, that subject’s knowledge on MM task influences the successfully solve the problem. Regarding individual analysis, no significant differences are observed in parameters related to correct recall, problem-solving ability, and time constraint compliance. However, some significant differences are found in parameters associated with the perceived difficulty level and ability to recall the correct answers. MPT model combined with EM algorithm constitutes a probabilistic model that enhances pupillary responses identification related to the cognitive effort. Potential applications of this model include disease diagnostics based on parameter values and identification of neural pathways that are involved in the pupillary response and subject’s cognitive effort. Furthermore, efforts are underway to connect this psychological model with an artificial neural network.
Keywords: cognitive effort, mental multiplication, multinomial processing trees, pupillary dynamics

CITATION

Hossain, G., & Elkins, J. D. (2024). Cognitive effort assessment through pupillary responses: Insights from multinomial processing tree modeling and neural interconnections. Online Journal of Communication and Media Technologies, 14(1), e202413. https://doi.org/10.30935/ojcmt/14196

REFERENCES

  • Alsobeh, A., & Shatnawi, A. (2023). Integrating data-driven security, model checking, and self-adaptation for IoT systems using BIP components: A conceptual proposal model. In K. Daimi, & A. Al Sadoon (Eds.), Proceedings of the 2023 International Conference on Advances in Computing Research (pp. 533-549). Springer. https://doi.org/10.1007/978-3-031-33743-7_44
  • Arrington Research. (2015). Eye tracker prices. http://www.arringtonresearch.com/prices.html
  • Beatty, J. (1982). Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91(2), 276-292. https://doi.org/10.1037/0033-2909.91.2.276
  • Bishara, A. J., & Payne, B. K. (2008). Multinomial processing tree models of control and automaticity in weapon misidentification. Journal of Experimental Social Psychology, 45(3), 524-534. https://doi.org/10.1016/j.jesp.2008.11.002
  • Block Imaging. (2014). Your guide to medical imaging equipment. https://info.blockimaging.com/success-stories
  • Böhm, M. F., Bayen, U. J., & Schaper, M. L. (2020). Are subjective sleepiness and sleep quality related to prospective memory? Cognitive Research: Principles and Implications, 5(1), 5. https://doi.org/10.1186/s41235-019-0199-7
  • Chen, H.-C., & Magdon-Ismail, M. (2006). NN-OPT: Neural network for option pricing using multinomial processing tree. In I. King, J. Wang, L. W. Chan, & D. Wang (Eds.), Neural information processing (pp. 360-369). Springer. https://doi.org/10.1007/11893295_41
  • Cross Check Networks. (2015). SOA testing techniques. http://www.crosschecknet.com/soa_testing_black_white_gray_box.php
  • da Silva Castanheira, K., LoParco, S., & Otto, A. R. (2021). Task-evoked pupillary responses track effort exertion: Evidence from task-switching. Cognitive, Affective, & Behavioral Neuroscience, 21, 592-606. https://doi.org/10.3758/s13415-020-00843-z
  • de Gee, J. W., Knapen, T., & Donner, T. H. (2014). Decision-related pupil dilation reflects upcoming choice and individual bias. PNAS, 111(5), E618-E625. https://doi.org/10.1073/pnas.1317557111
  • Einhäuser, W., Stout, J., Koch, C., & Carter, O. (2008). Pupil dilations reflects perceptual selection and predicts subsequent stability in perceptual rivalry. PNAS, 105(5), 1704-1709. https://doi.org/10.1073/pnas.0707727105
  • Eldar, E., Cohen, J. D., & Niv, Y. (2013). The effects of neural gain on attention and learning. Nature Neuroscience, 16, 1146-1153. https://doi.org/10.1038/nn.3428
  • Griffiths, T. L., & Kalish, M. L. (2001). A multidimensional scaling approach to mental multiplication. Memory and Cognition, 30(1), 97-106. https://doi.org/10.3758/BF03195269
  • Haines, D. E. (2013). Fundamental neuroscience for basic and clinical applications. Elsevier.
  • Hossain, G., & Elkins, J. D. (2018). When does an easy task become hard? A systematic review of human task-evoked pupillary dynamics versus cognitive efforts. Neural Computing and Applications, 30, 29-43. https://doi.org/10.1007/s00521-016-2750-5
  • Jarrah, A., Almomany, A., Alsobeh, A. M. R., & Alqudah, E. (2021). High-performance implementation of wideband coherent signal-subspace (CSS)-based DOA algorithm on FPGA. Journal of Circuits, Systems and Computers, 30(5), 2150196. https://doi.org/10.1142/S0218126621501966
  • Jepma, M., & Nieuwenhuis, S. (2011). Pupil diameter predicts changes exploration-exploitation trade-off: Evidence for the adaptive gain theory. Journal of Cognitive Neuroscience, 23(7), 1587-1596. https://doi.org/10.1162/jocn.2010.21548
  • Klingner, J. (2010). Measuring cognitive load during visual tasks by combining pupillometry and eye tracking [Doctoral dissertation, Stanford University].
  • Klingner, J., Tversky B., & Hanrahan, P. (2011). Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology, 48(3), 323-332. https://doi.org/10.1111/j.1469-8986.2010.01069.x
  • Köstering, L., McKinlay, A., Stahl, C., & Kaller, C. P. (2012). Differential patterns of planning impairments in Parkinson’s disease and sub-clinical signs of dementia? A latent-class model-based approach. PLoS ONE, 7(6), e38855. https://doi.org/10.1371/journal.pone.0038855
  • Miles, A., Brett, G., Khan, S., & Samim, Y. (2023). Testing models of cognition and action using response conflict and multinomial processing tree models. Sociological Science, 10, 118-149. https://doi.org/10.15195/v10.a4
  • Moshagen, M. (2010). multiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42-54. https://doi.org/10.3758/BRM.42.1.42
  • Nestler, S., & Erdfelder, E. (2023). Random effects multinomial processing tree models: A maximum likelihood approach. Psychometrika, 88, 809-829. https://doi.org/10.1007/s11336-023-09921-w
  • Nowak, W., Hachol, A., & Kasprzak, H. (2008). Time-frequency analysis of spontaneous fluctuations of the pupil size of the human eye. Optica Applicata [Applied Optics], 38(2), 469-480.
  • O’Neill, W. D., & Trick, K. P. (2001). The narcoleptic cognitive pupillary response. IEEE Transactions on Biomedical Engineering, 48(9), 963-968. https://doi.org/10.1109/10.942585
  • Ohtsuka, K., Asakura, K., Kawasaki, H., & Sawa, M. (1988). Respiratory fluctuation of the human pupil. Experimental Brain Research, 71, 215-217. https://doi.org/10.1007/BF00247537
  • Privitera, C. M., Renninger, L. W., Carney, T., Klein, S., & Aguilar, M. (2010). Pupil dilation during visual target detection. Journal of Vision, 10(10), 3. https://doi.org/10.1167/10.10.3
  • Reilly, J., Kelly, A., Kim, S. H., Jett, S., & Zuckerman, B. (2019). The human task-evoked pupillary response function is linear: Implications for baseline response scaling in pupillometry. Behavior Research Methods, 51, 865-878. https://doi.org/10.3758/s13428-018-1134-4
  • Sensi, F., Calcagnini, G., & De Pasquale, F. (1999). Baroreceptor-sensitive fluctuations of human pupil diameter. Computers in Cardiology, 1, 233-236.
  • Singmann, H. (2010). MPTinR: Analysis of multinomial processing trees in R. Behavior Research Methods, 45, 560-575. https://doi.org/10.3758/s13428-012-0259-0
  • Stergiou, C., & Siganos, D. (1996). Neural networks. https://srii.sou.edu.ge/neural-networks.pdf
  • UMASS. (2014). Multinomial processing tree models. http://people.umass.edu/alc/course_pages/fall_2004/modeling_behavior/lectures/MPTs.ppt