Effectiveness of an Integrated Tertiary Software Mobile Information System for Student Registration and Admission at a University in Gauteng

KeywordsDeLone and McLean IS Success Model higher education institutions information systems Technology Acceptance Model Web of System Performance Model

JEL Classification M10, L86, I23

Full Article

1. Introduction

Walk-in registrations and admission at tertiary institutions at the beginning of the year have proven to be a deadly exercise after a parent queuing for registrations of her child at the University of Johannesburg (UJ) passed away after a stampede. This incident has forced tertiary institutions to do away with the walk-in process and adopt an online registration and admission system. After this incident at UJ, Dr Blade Nzimande, Minister of higher Learning in South Africa recommended that all applications to universities must be made online via the proposed centralised application system (CAS) and walk-in applications should be prohibited as reported in the Mail and Guardian newspaper on the 11th October 2012 (Fazel, 2012, pp.1-2). Jenvey (2012, pp.1-3), stated that an online system should allow students to apply for admissions, financial aid and accommodation via a single application. The purpose of this study is to investigate whether the new online system meets the expectations of its users by evaluating its effectiveness as compared to the old walk-in registration system.

The research has added knowledge of measuring the performance of an information system in the tertiary institution under study without using the Return On Investment (ROI) model (Alexei, Andru, Botchkarev and Peter, 2011, p.1). The study analysed and diagnosed the efficiency of the new online system by using modern information system evaluation methods such as the Technology Acceptance Model (TAM), the Web of System Performance (WOSP) model and the DeLone and McLean IS Success model. The Centralised Application System (CAS) as proposed by the Minister of Higher Education is not linked to tertiary institutions nationally for students to apply at one point. Students register online without the system notifying them in advance as to which faculties still have space to enrol more students. The other usual gap or challenge with any new IT system is its compatibility and connectivity with existing systems.

1.1.Research Questions

The study was guided by the following research questions

- What are the perceptions of personnel and students about the performance of the new online registration and admission system (i.e., ITS Mobile)?

- What are the strengths and weaknesses of the new registration system in terms of its efficiency?

- What recommendations can be provided to improve the efficiency and effectiveness of this new online registration system?

2. Literature Review

Information systems evaluation has become an important aspect for study and practice in many organisations. According to Farbey, Land and Targett (1995, pp.41-50), the main aims of measuring information systems are for decision-making, control or accountability and legitimising decision-making. DeLone and McLean, TAM and The WOSP models were used to evaluate the effectiveness of the new system are presented below:

2.1. The Web of System Performance (WOSP) Model

The Web of System Performance (WOSP) model performs better than others as it is flexible with regards to its connectivity to other systems for integration while being secure enough for keeping data private, and it is user friendly (Figure 1).

Figure 1. The Web of system performance, an updated model

Source: Fjermestad, Mahinda and Whitworth (2011, pp.1-9)

2.2. Technology Acceptance Model (TAM)

TAM is an information system model which gives details on how users accept and use a technology based on the “Perceived Usefulness” (PU) of the model. Davis (1989, pp.319-340), describes perceived usefulness as the level in which a person believes that using a particular system would improve his or her job performance.

Figure 2. Flow diagram of the TAM process

Source: Venkatesh and Davis (2000)

2.3. DeLone and McLean IS Success Model

DeLone and McLean IS Success Model is shown below in Figure 3:

Figure 3: Flow diagram of the DeLone and McLean IS success model

Source: DeLone and McLean (2003, pp.9-30) and DeLone, McLean and Petter (2008, pp.236-264)

The identified seven dimensions of information system successes are: system quality, information quality, service quality, information usage, user satisfaction, individual impact and organizational impact (DeLone and McLean, 2003, pp.9-30). These dimensions were integrated to form a model incorporating the TAM and DeLone and McLean models as illustrated in Figure 3.

2.4. An integrated model combining the TAM and DeLone and McLean IS Success models

Figure 4. An integrated success model

Source: Zaied, 2012, p.814

The integrated model presented above suggests that quality information, system and service are all linked to management support, training and user involvement influencing perceived usefulness and ease of use (Zaied, 2012, pp.814-825).

2.5. The Theory of Enterprise System Flexibility

Figure 5. Theory of enterprise system flexibility

Source: Gebauer and Schober 2006, pp.122-147

The model distinguishes between flexibility-to-use which relates to the range of process activities built into the system that are supported without a need for a major system change while flexibility-to-change relates with information technology infrastructure measured by the effort required to change a system after its initial implementation. An IT infrastructure should be flexible to be able to handle increased customer demands without increased costs (Weill, 1993, pp.547-572).

3. Methodology

3.1. Research Methodology

The research philosophy is located in the positivist paradigm as opposed to the interpretivist paradigm. According to Tuli (2010), a positivist research is quantitative and it uses numerical measurements to examine a social phenomenon emphasising reliability and objectivity of findings. The study was not located in interpretivist or constructivism which is concerned more about the thinking, intensions and behaviours of human beings which conflicts with the intention of a qualitative approach. The quantitative strategy used is a survey questionnaire to determine the opinions, attitudes, preferences and perceptions of the targeted population being students and staff members of the tertiary institution under study in their Pretoria campus. A convenient sampling process, which is within the non-probability sampling technique, was followed in this study to identify a total of 100 participants from a targeted population consisting of 10 staff members and managers and 90 first year students admitted at the beginning of the 2015 year.

3.2. Data Analysis

The data was statistically analysed using descriptive and inferential statistics where data was analysed using ordinal measures applicable to Likert scale ratings. In this study, five levels were ranked from agree, strongly agree, neutral, disagree and strongly disagree (Allen and Seaman, 2007, pp.1-5). Statistical applications to analyse data in this study were done with construct reliability tests using SPSS to check if the Cronbach Alpha Index measuring the internal consistency of scale items in the questionnaire is greater than 0.7 (Cooper and Schindler, 2011, pp.310). Content validity via a survey questionnaire was used for measuring the effectiveness of the online system in a quantitative study. Expert opinion was provided by the researcher’s supervisor (face validity) to check if the questions were relevant before being administered. Based on the questionnaire as an instrument for data collection, meaningful and useful inferences from scores were drawn for establishing study findings.

4. Results and Discussion

The construct reliability test was carried out on all scale items of research variables by using SPSS software with the assistance of a qualified statistician. The Cronbach Alpha was greater than 0.7, meaning that the research instrument used in the study was reliable. The following data presentation and interpretation pertain to the outcomes of the study.

4.1. Perceptions of the Students

The perceptions of students are captured in Figures 6 to 14.

Figure 6. Question: The online registration system is far better than the walk-in queues

Figure 6 shows that most respondents agreed with the statement above, namely ‘The online registration system is far better than the walk-in queues’. Scott (2010, p.218), states that new technology leads to a change on how people access different services.

Figure 7. Question: It is easy to use the online registration system

Figure 7 demonstrates that most respondents agreed with the statement above. Whitworth and Zaic (2003, p.258) stated that the IT system is easy to use and is likely to be fully utilised by users. A standard deviation of 0.907 as per Table 4.5 in the annexures shows not much difference on the responses given.

Figure 8. Question: I have registered without physically visiting the institution

Figure 8 displays that most respondents agreed with the statement ‘I have registered without physically visiting the institution’. Fjermestad et al. (2011, pp.1-9) stated that flexible access to services due to technology adoption results in a happy user.

Figure 9. Question: Instructions for online registration are clear to follow

Figure 9 presents that most respondents agreed with the statement above. Szajna (1994, p.320), stated that clear instructions make it easier to use the system.

Figure 10. Question: The online registration and admission system is effective

Figure 10 exhibits that most respondents agreed with the statement above related to the effiectiveness of the online registration and admission system. An effective IT system promotes confidence amongst users (Todd and Wixon, 2005, p.85). Simultaneous system access by many users results in system slowdown (DeLone and McLean, 2003, p.11).

Figure 11. Question: The online registration and admission system is reliable

Figure 11 introduces that most respondents agreed with the statement above, so it means most students registered without problems, showing the reliability of the online registration and admission system. System reliability improves the image of an organisation (Bitner et al., 2012, p.310).

Figure 12. Question: The online registration system has made it easy for students to register

Figure 12 reveals that most respondents agreed with the statement above, that students find the online registration and admission system easy to use for registration. The use of technology promotes ease of access to services by different scholars (Fjermestad et al., 2011, p.2).

Figure 13. Question: The system gave accurate information

Figure 13 presents that most respondents agreed with the statement related to the information accuracy of the online registration and admission system, and these results concur with the ones included in Figure 11 above. A system might give wrong information if it has developed problems (Davis and Venkatesh, 2000, pp.186-340).

Figure 14. Question: I am satisfied with the online registration system

Figure 14 shows that most respondents agreed with the statement above, regarding the satisfaction of the users with the online registration and admission system. Todd and Wixon (2005, p.85) stated that IS satisfaction is achieved when user expectations are met.

4.2. Strength and Weaknesses of the New Registration System, According to the Students

The following section presents the strength and weaknesses of the new registration system according to the students. These are indicated as Figures 15 to 21.

Figure 15. Question: Network connectivity for online registration is quick

Figure 15 shows that most respondents agreed with the statement above, namely ‘Network connectivity for online registration is quick’. Network connectivity is slow when demand of usage is high (Whitworth and Zaic, 2003, p.259).

Figure 16. Question: With the online registration system you can register at any time of the day

Figure 16 displays that most respondents agreed or strongly agreed with the statement above, and users can register anytime. Nonetheless, those students without access to the internet might not have the privilege to apply at any time of the day.

Figure 17. Question: The online registration and admission system is user friendly

The results from Figure 17 display that most students find the online registration system to be simple and easy to use. A system that is viewed as user unfriendly may lead to dissatisfaction of users (Zaied, 2012, p.814).

Figure 18. Question: The online registration and admission system is useful

In figure 18, the results show that most students are likely to use the system in future when they register for other programmes. According to Zaied (2012, p.814), a good system provides information that helps students to choose appropriate qualifications to register.

Figure 19. Question: I did not encounter any problem when using an online registration system

Figure 19 demonstrate that most respondents agreed with the statement above and they did not encounter any issues when using the system, in an online setting. Nonetheless, system failure is inevitable as indicated by Gebauer and Schober (2006, p.122).

Figure 20. Question: The online registration system does not need any enhancement

Figure 20 shows that most respondents disagreed with the statement above, and that the online registration system would benefit of improvements. Gebauer and Schober (2006, p.122) indicated that any IT system must be evaluated and improved where necessary. Students might have experienced problems during registration.

Figure 21. Question: The online registration system should not be changed for another system

Figure 21 shows that most respondents disagreed with the statement above, and that the online registration system should not be changed for another system. A new system should be used for a reasonable time to realise return on investment (David and David, 2015, p.205).

4.3. Perceptions of Staff and Managers about the Performance of the New Online Mobile Registration System

Figures 22 to 32 present a brief discussion on the perceptions of staff and managers about the performance of the new online mobile registration system.

Figure 22. Question: The online registration system is user friendly

Figure 22 shows that most respondents agreed with the above statement, and that the online registration system is perceived as user friendly. The results confirm the ease of use statement in Figure 12. Some individuals might have said that the system was not working due to a lack of knowledge on how to use it (Zaied, 2012, p.815).

Figure 23. Question: The system is useful

Figure 23 displays that most respondents agreed with the above statement, and that the online registration system is useful. Results in Figure 22 and Figure 23 show a close relationship. A useful system is likely to be utilised fully for the benefit of the organisation and its stakeholders (Zaied, 2012, p.814).

Figure 24. Question: I have confidence in the new system

Figure 24 introduces that most respondents agreed with the statement presented above, and that the online registration system inspires confidence. System’s reliability and performance determine confidence (Whitworth and Zaic, 2003:258). The results do not provide a clear reflection of the responses that were given in Figure 23.

Figure 25. Question: Clear and sufficient information needed is provided on time

Figure 25 reveals that most respondents agreed with this statement, namely the users appreciate the time sensitive clear information provided by the online registration system. Lack of knowledge might result in employees not able to access data (Venkatesh and Davis, 2000).

Figure 26. Question: The IT division support is always at hand to help

Figure 26 shows that most respondents agreed with the statement ‘The IT division support is always at hand to help’. As indicated by Zaied (2012:814), a good system may be rated badly due to unavailability of system maintenance staff.

Figure 27. Question: I am satisfied with this online registration and admission system

Figure 27 shows that most respondents agreed with the statement and that the users are satisfied with the online registration and admission system. Most of staff members were not happy about the new system, it might be due to a negative perception if they were not consulted (Byrd and Turner, 2000, p.172).

Figure 28. Question: The online system needs enhancement

An IT based system needs continuous evaluation to enhance its performance (Gebauer and Schober, 2006, p.122). Figure 28 shows that most respondents disagreed with the statement above.

Figure 29. Question: Students without access to internet or smart phones are also assisted to register

Figure 29 shows that most respondents agreed with the statement ‘Students without access to internet or smart phones are also assisted to register’. The results show that students with smartphones or internet were able register.

Figure 30. Question: Management and staff have system capabilities to resolve students’ queries

Figure 30 displays that most respondents agreed with the statement and that the management have all the necessary capabilities to solve queries and other issues. Motivated staff assists clients professionally and adequately (Mondy, 2012, p.125).

Figure 31. Question: The online registration and admission system should be changed for another system

Figure 31 reveals that most respondents disagreed with the above statement. These results agree with Figure 27 results of users being satisfied with the system. System frustrations may lead to user desire to substitute the existing system (DeLone and McLane, 2004, p.11).

Figure 32. Question: The online registration system is much better than the walk-in queues

Figure 32 finds that that most respondents agreed with the statement above, and that the online registration system is a better option. Users focus on the smarter new way of providing services to customers (DeLone and McLane, 2004, p.10).

4.4. Strengths and Weaknesses

The findings for the strengths and weaknesses follows and are presented in Figures 33 to 41.

Figure 33. Question: The system performed exactly as expected

A good IT system has to perform its functions as expected by users (DeLone and McLean, 2003, p.9). Figure 33 shows that the users’ expectations were met.

Figure 34. Question: The system has improved the registration process

Figure 34 shows that most respondents agreed with the statement above, the online registration is improved. DeLone and McLean (2003, p.11) stated that when a new system is adopted effectively, positive results are likely to be attained. The standard deviation of 1.679 shows that the difference in the way the above statement was rated by the respondents is higher than any other statements in the questionnaire.

Figure 35. Question: The system provides accurate information

Figure 35 shows that most respondents agreed with the statement above, and the system provides accurate information. System generating accurate information is an advantage (DeLone and McLean, 2003, p.11).

Figure 36. Question: There are problems encountered when using the system

Figure 36 shows that most respondents agreed with the statement above and that there issues with the system. System restoration is vital as system failure is inevitable (Davis and Venkatesh, 2004, p.33).

Figure 37. Question: System maintenance occurs weekly

System performance needs to be checked continuously to verify if it performs as expected (Szajna, 1994, p.319). Figure 37 demonstrates that most respondents disagreed with the statement above.

Figure 38. Question: System downtime is minimal

Figure 38 reveals that most respondents disagreed with the statement above. Lack of qualified staff influence IT system downtime (Davis and Venkatesh, 2004, p.34).

Figure 39. Question: The system is reliable

Figure 39 displays that most respondents disagreed with the statement above, and the system is not reliable. Employees and customers may resist using an unreliable system (Whitworth and Zaic, 2003, p.258).

Figure 40. Question: The system sometimes has a slow response time

Figure 40 introduces that most respondents agreed with the statement above. This might be the reason why some respondents in Figure 39 stated that the system was not reliable. Employees may struggle to get the reports they need on time because of the slowness of the system (Whitworth and Zaic, 2003, p.259).

Figure 41. Question: There are system user manuals for students and staff

Figure 41 exhibits that most respondents agreed with the statement above. It is important for the organisation to provide a system user manual to all the students and staff (Davis and Venkatesh, 2004, p.32).

5. Conclusion and Recommendations

The aim of the study was to determine whether the new online registration system that was implemented at a tertiary institution based in Gauteng met the expectations of students, managers, and staff. A literature review on the adoption of a new technology was conducted and key findings emerging from the data analysis were elaborated in relation to the literature.

5.1. Key Findings from the Study

This section presents the findings of the study from the literature review.

5.1.1. Findings from the Literature Review

Most scholars such as Hirschheim and Smithson (1999, p.381), Willcocks (1992, p.47), Lancaster (1977, p.1) and Farbey (1995, p.207) are of the view that there has been a transition on how the performance of an information technology (IT) system is evaluated. Return on investment was traditionally used to evaluate as per literature review. However, there was a paradigm shift over the years (Walsham, 1993, p.1-281). Performance of IT systems can be evaluated by its ability to meet users’ expectations (Serafeimidis and Smithson, 2000, p.93). Whitworth and Zaic (2003, p.260) have shown that decision makers must ensure that all users of an IT system are satisfied. DeLone and McLean (2003, p.10) have shown that new technology might fail if different stakeholders are not consulted during the development phase. Users embrace information and communications technology (ICT) that is useful and easy to use (Todd and Wixon, 2005, p.85). The complexity of an ICT system discourages users to use technology (Davis and Venkatesh, 2004, p.31).

5.1.2. Key Findings from the Primary Research

The following are findings from the analysis of the data generated by the study.

5.1.2.1. The Effectiveness of the New Online Registrations System

This section presents a summary of the responses collected from students who confirmed that the new online system was better than walk-in queues. Results shows that the new online system met the expectations of users as it is easy to use, reliable, useful and students were able to register at any time without physically visiting the institution.

5.1.2.2. The Strengths and Weaknesses of the New Online Registration System

Network connectivity was available at all times and the majority of students stated that the new system must not be changed. Most personnel reported that the new online registration system performed as expected and the registration process has improved even though the system experienced several problems and did not perform as expected.

5.2. Recommendations on the Effectiveness of the New Online Registration System and Its Strengths and Weaknesses

The recommendations of the study are indicated below:

5.2.1. Recommendations Related to the Effectiveness of New Online Registration System

Few respondents were not able to use the system properly, it is recommended that the institution must train the staff who might be struggling to use the new system. A baseline assessment of the challenges faced by users must be done before the training starts (Geyer, 2013, p.84). The institution may increase capacity of computers on the campus to accommodate more students who have no means of applying for registration online.

5.2.2. Recommendations Related to the Strengths and Weaknesses of the New Online Registration System

It is imperative that the causes of failures in the new online registration system be investigated to minimise common problems of system failure. An independent IT consultancy is needed to check the functioning of the system and to benchmark it as IT staff may not disclose system failures caused by them. There is also a need to inform users when the system is down or slow (Szajna, 1994, p.311).

6. Conclusion

The objectives of the study were to measure the effectiveness of the new online registrations system and to analyse its strengths and weaknesses. The results show that although there were some students and staff members who had some challenges with the new system, the majority of the respondents found it to be effective. Most system users indicated that they were satisfied with the system as its strengths outweighed its weaknesses. Study findings have shown that the implementation of the new online registration system was a success because the expectations of most users were met.

Appendices

Appendix 1 (i) Questionnaire for Staff and Management

Appendix: 1 (ii) Questionnaire for Students

Appendix: 1(iii) Questionnaire for Students

Appendix 2. Additional Tables

Appendix 2 (i). Descriptive Statistics - Students

Appendix 2 (ii). Descriptive Statistics - Management and Staff

References

1. Allen, I.E. and Seaman, C., 2007. Likert scales of ratings [online] Available at: http://asq.org/quality-progress/2007/07/statistics/likert-scales-and-data-analyses.html [Accessed on 24 January 2016].
2. Bitner, M. J., Gremler, D.D. and Zeithaml, V.A., 2012. Service Marketing: Integrated customer focus across the firm. 2^nd ed. Berkshire: McGraw-Hill Irwin.
3. Botchkarev, A. and Andru, P., 2011. A Return on Investment as a Metric for Evaluating Information Systems: Taxonomy and Application. Interdisciplinary Journal of Information, Knowledge and Management, 6, pp. 245-269.
4. Byrd, T.A. and Turner, E.D., 2000. An exploratory analysis of the information technology infrastructure flexibility construct. Journal of Management Information Systems, 17(1), pp.167-208.
5. Cooper, D.R. and Schindler, P.S., 2011. Business Research Methods. 11^th ed. New York: McGraw-Hill Education.
6. David, F.R. and David, F.R., 2015. Strategic management concepts. 15^th ed. Essex: Pearson Education Limited.
7. Davis, F.D. and Venkatesh, V., 2004. Toward preprototype user acceptance testing of new information systems: Implications for software project management. IEEE Transactions on Engineering Management, 51(1), pp 31-46.
8. DeLone, W. H. and McLean, E.R., 2004. Measuring e-commerce success: applying the DeLone & McLean Information systems success model. International Journal of Electronic Commerce, 9(4), pp.9-47.
9. DeLone, W., McLean, E. and Petter, S., 2008. Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), pp. 236-264.
10. DeLone, W.H. and McLean, E.R., 2003. The DeLone and McLean Model of Information Systems Success: A ten-Year Update. Journal of Management Information Systems, 19(4), pp.9-30.
11. Farbey, B., Land, F.F., and Targett, D., 1995. A Taxonomy of Information Systems Applications: The Benefits’ of Evaluation Ladder. European Journal of Information Systems, 4(1), pp 41-50.
12. Fazel, A., 2012. Nzimande cleaning house to avert another UJ stampede [online] Available at: http://mg.co.za/article/2012-10-11-nzimandes-clearing-house-to-avert-another-uj-stampede. [Accessed on 5 January 2016].
13. Fjermestad, J., Mahinda, E. and Whitworth, B., 2011. The Web of System Performance: A multi-goal model of information system performance [online] Available at: http://wenku.baidu.com/view/cd09d4f9fab069dc5022016f.html [Accessed on 19 April 2016].
14. Gebauer, J. and Schober, F., 2006. Information system flexibility and the cost efficiency of business processes. Journal of the Association for Information Systems, 7(8), pp. 122-147.
15. Geyer, L., 2013. Human resource management. 1^st ed. Johannesburg: CTI Education.
16. Hirschheim, R. and Smithson, S., 1999. Evaluation of Information Systems: a Critical Assessment. In: L. Willcocks and S. Lester, ed., Beyond the IT productivity paradox. New York: Wiley.
17. Jenvey, N., 2012. High hopes for central university applications system [online] Available at: http://www.universityworldnews.com/article.php?story=20121109163137825 [Accessed on 30 February 2016].
18. Lancaster, F.W., 1977. The measurement and evaluation of library services. Virginia: Information Resource Press.
19. Mondy, R.W., 2012. Human resource management. 12^th ed. New Jersey: Pearson Education Limited.
20. Scott, A., 2010. Strategic management. Edinburgh: Heriot-Watt University.
21. Szajna, B., 1994. Software Evaluation and choice: predictive evaluation of the Technology Acceptance Instrument. MIS Quarterly, 18, pp.319-324.
22. Todd, P.A. and Wixon, B.H., 2005. Integrating System Characteristics to TAM. A Theoretical Integration of User Satisfaction and Technology Acceptance. Information Systems Research, 16(1), pp.80-102.
23. Tuli, F., 2010. The Basis of Distinction Between Qualitative and Quantitative Research in Social Science. Ethiopian Journal Education & Society, 6(1), pp.97-109.
24. Venkatesh, V. and Davis, F.D., 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science, 46(2), pp.186-204.
25. Walsham, G., 1993. Interpreting information systems in organizations. New York: Wiley.
26. Weill, P., 1993. The Role and Value of Information Technology Infrastructure: Some Empirical Observations. In R. Banker, Kaufman, R., Mahood, M.A. (eds.), Strategic Information Technology Management: Perspectives on Organizational Growth and Competitive Advantage, Idea Group Publishing, Middleton, PA.
27. Whitworth, B. and Zaic, M., 2003. The WOSP Model: Balanced Information System Design and Evaluation. Communications of the Association for Information Systems, 12, pp.258-282.
28. Willcocks, L., 1992. Evaluating Information Technology investments: research findings and reappraisal. Journal of Information Systems, 2, pp.243-268.
29. Zaied, N.H., 2012. An Integrated Success Model for Evaluating Information System in the Public Sectors. Journal of Emerging Trends in computing and Information Science, 3(6), pp. 814-825.

Article Rights and License

Corresponding Author