Adopting Computer-Based Assessments: The Role of Perceived Value in Classroom Technology Acceptance

Joseph M. Sirianni 1 *, Yu Jie Ng 2, Arun Vishwanath 3
More Detail
1 Niagara University, USA
2 Nanyang Technological University, Singapore
3 University at Buffalo & State University of New York, USA
* Corresponding Author
Online Journal of Communication and Media Technologies, Volume 7, Issue 4, pp. 1-23. https://doi.org/10.29333/ojcmt/2607
OPEN ACCESS   2097 Views   1220 Downloads   Published online: 10 Oct 2017
Download Full Text (PDF)

ABSTRACT

Computer based assessments (CBA) have increasingly become a popular tool for educators to test students’ knowledge of course material because of the many advantages it confers. However, research on its perceived value and satisfaction among students has found mixed results, with some test takers’ attitudes ranging from enthusiasm at being able to complete exams and retrieve test results whenever they want, to others actively disliking its use. As yet, the reasons for the same remain unclear. What is clear is that unmotivated or discontented students’ negative evaluations of CBA could overtime lead to a discontinuance of its usage in classrooms. Understanding the drivers of students’ continued usage of CBA is therefore key to the future use of this technological innovation and the goal of this research. To this end, the study utilized the Unified Theory of Acceptance and Use of Technology (UTAUT)—a model specifically built to understand the adoption of software technology—to the classroom adoption of technology context. Using quantitative survey data from 111 students who were assessed using CBAs, the study examined the role of the UTAUT constructs of performance expectancy, effort expectancy, social influence, and facilitating conditions in predicting students’ continuance intention for CBA. Findings found a direct effect of UTAUT’s core constructs of performance expectancy, social influence, and facilitating conditions on continuance CBA intention. Interestingly, students’ perceived value of CBA partially mediated the effect of these constructs on continuance intention. The results of the study, therefore, point to a single, new, global construct—perceived value of CBA— that predicts whether students prefer classroom technology.
Keywords: UTAUT, CBA perceived value, computer-based assessment, technology acceptance, education

CITATION

Sirianni, J. M., Ng, Y. J., & Vishwanath, A. (2017). Adopting Computer-Based Assessments: The Role of Perceived Value in Classroom Technology Acceptance. Online Journal of Communication and Media Technologies, 7(4), 1-23. https://doi.org/10.29333/ojcmt/2607

REFERENCES

  • Ajzen, I. (1991). The theory of planned behavior. Organizational behavior and human decision processes, 50, 179-211.
  • Akdemir, O., & Oguz, A. (2008). Computer-based testing: An alternative for the assessment of Turkish undergraduate students. Computers & Education, 51, 1198-1204.
  • American Psychological Association. Committee on Professional Standards, American Psychological Association. Board of Scientific Affairs.Committee on Psychological Tests, & Assessment.(1986). Guidelines for computer-based tests and interpretations. The Association.and Achievement Motivation, W.H. Freeman, San Francisco, CA, 1983, pp. 75-146.
  • Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ, 1986.
  • Barkley, A. P. (2002). An analysis of online examinations in college courses. Journal of Agricultural and Applied Economics, 34, 445-458.
  • Bouhnik, D., & Marcus, T. (2006). Interaction in distance‐learning courses. Journal of the American Society for Information Science and Technology, 57(3), 299-305.
  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. National Academy Press.
  • Bugbee, A. C. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of research on computing in education, 28, 282-299.
  • Bugbee, A. C. (1992). Examination on demand: Findings in ten years of testing by computer 1982-1991. Edina, MN: TRO Learning
  • Bugbee, A. C., & Bernt, F. M. (1990). Testing by computer: Findings in six years of use 1982-1988. Journal of Research on Computing in Education, 23(1), 87-100.
  • Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1989). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (3rd ed.), (pp. 367-407 New York: American Council on Education--Macmillan.
  • Carlsson, C., Carlsson, J., Hyvonen, K., Puhakainen, J., & Walden, P. (2006). Adoption of mobile devices/services—searching for answers with the UTAUT. In Proceedings of the 39th Hawaii International Conference on System Sciences.
  • Castro, I., & Roldán, J. L. (2013). A mediation model between dimensions of social capital. International Business Review, 22(6), 1034-1050.
  • Chiu, C. M., Hsu, M. H., Sun, S. Y., Lin, T. C., & Sun, P. C. (2005). Usability, quality, value and e-learning continuance decisions. Computers & Education, 4, 399- 416.
  • Chiu, C. M., Sun, S. Y., Sun, P. C., &Ju, T. L. (2007).An empirical analysis of the antecedents of web-based learning continuance. Computers & Education, 49, 1224- 1245.
  • Chiu, C. M., & Wang, E. T. (2008). Understanding Web-based learning continuance intention: The role of subjective task value. Information & Management, 45, 194201.
  • Chua, S. L., Chen, D. T., & Wong, A. F. (1999). Computer anxiety and its correlates: a metaanalysis. Computers in human behavior, 15(5), 609-623.
  • Clariana, R., & Wallace, P. (2002). Paper–based versus computer–based assessment: key factors associated with the test mode effect. British Journal of Educational Technology, 33, 593-602.
  • Cody-Allen, E., & Kishore, R. (2006, April). An extension of the UTAUT model with equality, trust, and satisfaction constructs. In Proceedings of the 2006 ACM SIGMIS CPR conference on computer personnel research: Forty four years of computer personnel research: achievements, challenges & the future(pp. 82-89). ACM.
  • Croft, A. C., Danson, M., Dawson, B. R., & Ward, J. P. (2001). Experiences of using computer assisted assessment in engineering mathematics. Computers & Education, 37, 53-66.
  • Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 13, 319-340.
  • DeAngelis, S. (1999). Equivalency of computer-based and paper-and-pencil testing. Journal of Allied Health, 29, 161-164.
  • Dickhauser, O., & Stiensmeier-Pelster, J. (2003). Gender differences in the choice of computer courses: Applying the expectancy-value model. Social Psychology of Education, 6, 173-189.
  • Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1995). Expectancies, values, and academic behaviors. In J.T. Spence (Ed.), Achievement.
  • Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents' achievement task values and expectancy-related beliefs.
  • Fulcher, G. (2000). The ‘communicative’legacy in language testing. System, 28, 483-497.
  • Harmon, O. R., &Lambrinos, J. (2008). Are online exams an invitation to cheat?. The Journal of Economic Education, 39, 116-125.
  • Homer, P. M., &Kahle, L. R. (1988). A structural equation test of the value-attitude-behavior hierarchy. Journal of Personality and social Psychology,54(4), 638.
  • Howell, S. L., Sorensen, D., & Tippets, H. R. (2009).The new (and old) news about cheating for distance educators. Online Journal of Distance Learning Administration, 12.
  • Jones, S., Johnson‐Yale, C., Millermaier, S., & Pérez, F. S. (2009). US college students’ Internet use: Race, gender and digital divides. Journal of Computer‐Mediated Communication, 14, 244-264.
  • Kennedy, K., Nowak, S., Raghuraman, R., Thomas, J., & Davis, S. F. (2000). Academic dishonesty and distance learning: Student and faculty views. College Student Journal, 34, 309-314.
  • Krsak, A. (2007). Curbing academic dishonesty in online courses. In TCC-Teaching Colleges and Community Worldwide Online Conference , 1, 159-170.
  • Liaw, S. S. (2008). Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51(2), 864-873.
  • Limayem, M., & Cheung, C. M. K. (2011). Predicting the continued use of Internet based learning technologies: the role of habit. Journal of Behaviour and Information Technology, 30(1), 91-99.
  • Mason, B. J., Patry, M., &Berstein, D. J. (2001). An examination of the equivalence between non-adaptive computer-based and traditional testing.Journal of Educational Computing Research, 24, 29-40.
  • Mazzeo, J., & Harvey, A.L. (1988).The equivalence of scores from automated and conventional educational and psychological tests (College Board Report No. 88-8). New York: College Entrance Examination Board.
  • McDonald, A. S. (2002). The impact of individual differences on the equivalence of computer-based and paper-and-pencil educational assessments. Computers & Education, 39(3), 299-312.
  • Mills, J. D. (2002). Using computer simulation methods to teach statistics: A review of the literature. Journal of Statistics Education, 10, 1-20.
  • Moran, M., Hawkes, M., & El Gayar, O. (2010). Tablet personal computer integration in higher education: Applying the unified theory of acceptance and use technology model to understand supporting factors. Journal of Educational Computing Research, 42, 79-101.
  • Ogilvie, R. W., Trusk, T. C., & Blue, A. V. (1999).Students’ attitudes towards computer testing in a basic science course. Medical education, 33(11), 828-831.
  • Olsen, B., & Krendl, K. A. (1990). At-risk students and microcomputers: What do we know and how do we know it? Journal of Educational Technology Systems, 19(2), 165175.
  • Ramos, M. (2003).Auditors’ responsibility for fraud detection. Journal of Accountancy, 195(1), 28-35.
  • Real, J. C., Roldán, J. L., & Leal, A. (2014). From entrepreneurial orientation and learning orientation to business performance: analysing the mediating role of organizational learning and the moderating effects of organizational size. British Journal of Management, 25(2), 186-208.
  • Roca, J. C., Chiu, C. M., & Martinez, F. J. (2006). Understanding e-learning continuance intention: an extension of the technology acceptance model. International Journal of Human-Computer Studies, 64(8), 683-696.
  • Sambell, K., Sambell, A., & Sexton, G. (1999). Student perceptions of the learning benefits of computer-assisted assessment: A case study in electronic engineering. S. Brown, P. Race, & J. Bull, Computer-assisted assessment in higher education, 179-191.
  • Schaeffer, G. A., Reese, C. M., Steffen, M., McKinley, R. L., & Mills, C. N. (1993).Field test of a computer‐based gre general test. ETS Research Report Series, 1993, 1-47.
  • Singleton, C., Horne, J., & Thomas, K. (1999).Computerised baseline assessment of literacy. Journal of Research in Reading, 22(1), 67-80.
  • Stuber-McEwen, D., Wiseley, P., and Hoggatt, S. (2009). Point, click, and cheat: Frequency and type of academic dishonesty in the virtual classroom. Online Journal of Distance Learning Administration 12, 1-10.
  • Terzis, V., & Economides, A. A. (2011). The acceptance and use of computer based assessment. Computers & Education, 56, 1032-1044
  • Terzis, V., Moridis, C. N., & Economides, A. A. (2013). Continuance acceptance of computer based assessment through the integration of user's expectations and perceptions. Computers & Education, 62, 50-61.
  • Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, 425-478.
  • Vishwanath, A., & Goldhaber, G. M. (2003). An examination of the factors contributing to adoption decisions among late-diffused technology products. New media & society, 5, 547-572.
  • Wang, H. I., & Yang, H. L. (2005).The role of personality traits in UTAUT model under online stocking. Contemporary Management Research, 1(1), 69-82.
  • Watson, G. R., & Sottile, J. (2010). Cheating in the digital age: Do students cheat more in online courses?.
  • Wellner, K. (2015). Contribution and Implications.In User Innovators in the Silver Market (pp. 170-178). Springer Fachmedien Wiesbaden.
  • Wise, S. L., & Plake, B. S. (1989).Research on the effects of administering tests via computers. Educational measurement: Issues and practice, 8, 5-10.
  • Yang, Y., & Cornelius, L. F. (2004). Students' Perceptions towards the Quality of Online Education: A Qualitative Approach. Association for Educational Communications and Technology. results for total effects, direct effects, and indirect effects