Media

International Computer and Information Literacy Study (ICILS) 2013: Hong Kong and international results

20 Nov 2014

 

The International Computer and Information Literacy Study 2013 (http://www.iea.nl/icils_2013.html) (ICILS2013) is the first large-scale international comparative study on students’ ability to make use of computer and information technology to conduct inquiry, create, communicate and use information safely at home, school and different social and workplace contexts. Including Hong Kong, a total of 21 countries and education systems participated in this study, which was conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA).

The Hong Kong component of the ICILS 2013 study (http://icils.cite.hku.hk/) was conducted by the Centre for Information Technology in Education (CITE) of the University of Hong Kong, funded by the Quality Education Fund (QEF). About 2000 grade 8 (secondary two) students from 118 randomly selected schools in different districts in Hong Kong participated in the performance assessment of computer and information literacy (CIL), which was designed by the IEA research team with input from the research teams of the participating countries. The assessment incorporated multiple-choice items and constructed responses pertaining to authentic tasks. In the assessment, students were required to search, select and manage information, as well as to transform, integrate and create information products using computer software applications in a simulated web environment. In addition to the sampled students’ participation in the performance assessment and background questionnaire, approximately 1300 teachers of Secondary Two students and over 100 principals and ICT coordinators from the sampled schools also took part in the questionnaire surveys.

Design of the study and Hong Kong participation statistics in ICILS 2013

ICILS 2013 requires the following from all participating systems:

•      School sample: A random sample of at least 150 secondary schools from the target population of schools that offer grade 8 classes in the 2012-13 academic year.

•      School questionnaire data collection: Online questionnaire to be completed by the school principal, the ICT coordinator, and 15 teachers who teach grade 8 classes in the 2012-13 academic year (~20-30 minutes).

•      Student sample: 20-25 students randomly sampled from all grade 8 students in each participating school.

•      Student data collection: Computerized student CIL test (60 minutes) and student questionnaire (20 minutes).

In Hong Kong data, 118 secondary schools participated in the ICILS 2013 study. Overall participation rates, after weighting and replacement, are: students 68.6%, teachers 58.3%. The Hong Kong set of data is considered as belonging to category 2 according to IEA standards, which require overall participation rates after weighting and replacement to be at least 75%. Hence it is not appropriate to make statistical comparisons for the Hong Kong data. See Table 3 for more details of the ICILS sampling procedures, sampling results and participation statistics for Hong Kong.

Summary of findings (Please click this link for the tables: ICILS 2013 tables-e) 

1. Performance of HK Secondary 2 students in computer and information literacy

a)   The mean CIL score of Hong Kong students is 509. Since Hong Kong students’ overall participation rate is below 75%, ranking of their performance in comparison with other participating countries is statistically inappropriate.

b)   Hong Kong students’ CIL score is relatively low among all economically developed participating education systems. As HK’s ICT Development Index is ranked fifth among the 21 participating systems, this should not be a contributing factor to the relatively low CIL performance (see Table 4). In order to present succinctly the findings from this study on the strengths and weaknesses of HK students’ performance and the key factors influencing CIL performance, we have selected results from Australia and South Korea for comparison in the following analysis.

c)   The standard error of HK students’ mean CIL score is the largest among all participating systems, indicating large variations in CIL among HK students. Table 5 shows that the percentages of HK students achieving CIL proficiency levels 3 and 4 are both slightly higher than the international average. At the same time 38% of Hong Kong students achieved level 1 or below, of which 15% were below level 1 CIL proficiency. Both percentages are relatively high among all economically developed participating systems.

d)   Analysis results show that of the two strands of CIL items, HK student’s performance was relatively weaker in strand 2 (producing and exchanging information). In particular, performance in CIL aspects of transforming information, creating information, and sharing information are weaker than the international average (Table 6).

2. HK Secondary 2 students’ personal and home background, and how these relate to their CIL performance

a)   Similar to most other countries in the study, the CIL score of HK Secondary 2 female students is significantly higher than their male counterparts (Table 7).

b)   Similar to other subject-based international comparative studies, there is significant positive correlation between parental educational attainment and students’ CIL scores for HK. However the effect of parental educational attainment to Hong Kong students’ CIL score is not as high as in other education systems (Table 8).

c)   98% of the participating students reported having at least one computer at home (including desktop, notebook, or tablet computer). It is found that the mean CIL score of students with no computer at home is significantly lower, at CIL proficiency level 1. Students with at least one computer at home have mean CIL score at CIL proficiency level 2. The influence of having more than one computer at home on CIL performance is relatively minor (Table 9). It is also observed that only less than 1% of the participating students reported not having Internet access at home, suggesting that some students could only access the Internet at home using smartphones.

d) The multiple regression analysis shown in Table 10 indicates that only 10% of the variances in students’ CIL score could be explained by their personal and social background variables.

3. HK Secondary Two students’ learning environment and learning opportunities in school, how these relate to students’ CIL achievement

a)   The venue of students reported usage of computers is highest at home, followed by at school (Table 11).

b)   Only small percentages of students use office applications (e.g. word processing, spreadsheet, or presentation software), graphics software, or education software for learning outside their school. Even for word processing, which has the highest reported usage, was only used at least once per week by 26% of the students (Table 12).

c)   Accessing the Internet outside school for communication and exchange of information was reported by more students than for use of computers for learning. However, the percentages are also smaller than the international mean. The most popular Internet usage were communicating with others through messaging or social media (for example instant messaging or status updates), followed by voice chat (for example Skype) with friends or family online (Table 13).

d)   ICT use of for study purposes was relatively uncommon. Only half of the students reported using computers to complete worksheets or exercises at least once a month, and ICT use for other study related purposes is even lower (Table 14).

e)   Other than in information technology, computer studies or similar subjects, frequent usage of computers in other school subjects such as language, mathematics, or humanities were rare (from 8% to 15%). These percentages were much lower than in Australia or South Korea (Table 15).

f)   While they do not frequently use computers for learning related purposes in and outside school, around 2/3 to 3/4 of HK Secondary Two students reported having variously learnt the eight types of ICT related tasks at school. However, only 53% of the students reported having learnt how to determine the trustworthiness of information from the Internet at school, which is lowest reported percentage among the eight ICT related task surveyed (Table 16).

g)   In summary, at school, HK Secondary Two students mainly learnt computer and information literacy skills through information technology, computer studies or similar subjects. Few students had opportunities to develop information literacy or participate in e-Learning activities in other school subjects.

4. The support for ICT use in teaching and learning in HK schools

a)   The mean priorities reported by principals for pedagogical use of ICT in teaching and learning were lower than those reported in other participating systems. HK principals’ top three priorities were: establishing or enhancing an online learning support platform (87%), increasing the bandwidth of Internet access for computers (84%), and increasing the range of digital learning resources (83%) (Table 17).

b)   HK Secondary Two teachers’ reported usage of ICT for collaboration among fellow teachers is in general lower than the international average. Specifically, only 39% of HK Secondary Two teachers have collaborated with peers to develop lessons involving use of ICT (Table 18).

c)   The main hindrances and obstacles encountered in using ICT for teaching and learning include insufficient time for teachers to prepare lessons (87%), lack of incentives for teachers to integrate ICT use in their teaching (77%), and lack of effective professional learning resources for teachers (73%). Hindrances from not having enough computers or Internet bandwidth were comparatively lower, and also lower than the international average (Table 19).

5. The status of HK Secondary Two teachers’ pedagogical use of ICT

a)   Percentages of secondary schools in which most teachers have participated in specific kinds of ICT-related professional development are very low. In particular, participation in a community of practice concerned with ICT use in teaching and learning (11%), working with another teacher who has attended a course and then training other teachers (15%), discussing e-Learning during regular staff meetings (18%), and discussing within groups of teachers about using ICT in their teaching (19%) are lower than international average (Table 20).

b)   The most popular ICT-related professional development activity reported by HK teachers is observation of other teachers’ ICT-using lessons (41%), followed by courses on integrating ICT into teaching and learning (34%) (Table 21).

c)   HK Secondary Two teachers expressed high levels of confidence in general computer use and Internet related activities, generally reaching over 70% or above. However, lower percentages indicated confidence in pedagogical ICT use such as for assessing student learning (58%), monitoring student progress (52%), and collaborating with others using web applications such as Google Docs® (45%) (Table 22).

d)   Percentage of Secondary Two teachers who reported giving strong or some emphasis to the development of ICT-based capabilities in their students were lower than the respective international means. The highest reported percentage was for accessing information efficiently (53%) and using computer software to construct digital artefacts (e.g. presentations, documents, images and diagrams) (51%) (Table 23).

e)   The most popular ICT-using teaching practices in HK Secondary Two is direct class instruction through information presentation (38%). The percentages reporting other uses of ICT in teaching were only 16% or lower, which were lower than the respective international means. Practices reported by less than 5% of the teacher respondents were: mediate communication between students and experts or external mentors (3%), collaborate with parents or guardians in supporting students’ learning (3%), and enable students to collaborate with other students within or outside school (5%) (Table 24).

f)   Compared to use of ICT in teaching, percentages of HK Secondary Two teachers who often used ICT for various student learning activities in classroom were even lower. Of these, working on extended projects (12%) and searching for information on a topic using outside resources (11%) were among the highest reported. Overall speaking, the opportunity of engaging in ICT-using learning activities by HK students were much lower than the respective international means. In some activities, the percentage was down to one third or lower of the international average (Table 25).

g)   Multiple regression analysis found that the strongest predictor of HK teachers’ emphasis on student’s CIL development was their engagement in peer collaboration on e-Learning (Table 26).

6. Other individual, family and school factors that influence HK Secondary Two student’s CIL

a)   Multiple regression analysis indicates that the top three factors influencing student’s CIL are (in decreasing order):

1) Use of home computers at least once a week;

2) Use of school computers at least once a week; and

3) Student’s ICT-related learning experience at school (Table 27).

b)   Multiple regression analysis indicates that the key factors influencing a school’s mean CIL score are (in decreasing order):

1) Students’ average of learning ICT tasks at school (8 types, see Table 16 for details);

2) Percentage of students with weekly use of home computers

The analysis further show that for an increase of 1 standard deviation in the number of ICT tasks learnt at school, the expected CIL school mean increases by 31.7. (Table 28)

c) Similar to other participating educational systems, the HK results also show that a student’s CIL score is positively and significantly predicted by the school mean of students’ socioeconomic background of the student’s school (variance explained is 15.4). However, within the same school, a student’s socioeconomic background has significant negative relationship with his/her expected CIL score. This finding is unique among all participating educational systems. We are not sure about the reasons for this finding. One possible explanation could be that students with lower socioeconomic background value ICT-related learning opportunities more (Table 29).

d)   Analysis shows that for HK, the CIL score within-school variance is similar to the between-school variance. This is different from the patterns found in most other participating systems, in which the within-school variance is much larger (Table 30).

Summary

Results from the ICILS 2013 indicate that the computer and information literacy (CIL) performance of HK Secondary Two students is relatively low among economically developed participating education systems, particularly in high order information literacy skills such as integration, creation, communication, exchange and dissemination of information. Since this is the first international comparative study of CIL, we have no information about the changes in CIL that might have taken place since the launch of the First IT in Education Strategy in Hong Kong in 1998. However, in 2007 the Center for Information Technology in Education in  the University of Hong Kong (CITE) was commissioned by the Education Bureau to conduct an online information literacy performance assessment as the one of studies to evaluate the implementation effectiveness of the Second IT in Education Strategy (2004/2007)".  The Study reported that:

Students in the primary, secondary and special schools had good performances in the dimensions of “define’, “access” and “manage”. On the other hand, poor performance was found in the dimensions of “communicate” and “create”………

Both secondary schools and special schools had better performance in the “define” and “access” dimensions. Poor performance was found in the “integrate” and “evaluate” dimensions for both secondary and special school students……

it was found that in general, students in primary, secondary and special schools attained the basic level in all the 7 IL dimensions and were rather weak in attaining higher levels of proficiency which required higher-order and critical thinking skills….. The use of ICT still focused on “traditional practices” and less in “lifelong practices” and “connectedness practices”. Besides, teachers were more competent in the general use of ICT than pedagogical use of ICT. 
 (For details, refer to http://www.edb.gov.hk/en/edu-system/primary-secondary/applicable-to-primary-secondary/it-in-edu/phase-II-study-on-2nd-ited.html)                                  

The above results are similar to the ICILS 2013 findings on the relative strengths and weaknesses of HK students’ CIL.

From ICILS 2013 results, the following insights can be drawn:

•     Students having basic skills in using information technology does not mean they are information literate. This is different from some people’s expectation about "digital natives".

•     In Hong Kong, family and personal background have relatively minor effect on students' CIL level.

•     The use of information technology for learning in subject-based classrooms, especially for various information operations in the learning process, is the most effective way for enhancing students' CIL. It must be pointed out here, that the key is not simply the frequency of information technology usage in the classroom, but in providing learning opportunities for students that are self-directed, collaborative and inquiry oriented, and requiring students to use ICT in tackling high order learning tasks, in order for ICT use to bring improvements in students’ CIL.

•     In general, ICT use for teaching by HK Secondary Two teachers was relatively low, except for presentations in direction instructional situations. Use of ICT for student learning in classrooms was extremely low, and even the requirement on students to use ICT for homework was low.

•     HK secondary schools generally do not have high motivation or incentive to implement e-Learning. Secondary Two subject teachers also do not put much emphasis on their students’ CIL development.

•     The extent of teachers’ participation in e-Learning collaborations with peers is the most predictive of their emphasis on developing students CIL proficiency. This is consistent with the finding from a longitudinal evaluation study CITE conducted for the e-Learning Pilot Scheme (2011-2014), commissioned by the EDB: When the school leadership give priority to e-Learning, engage personally in understanding and supporting its development, create the conditions and opportunities for teachers to collaborate in designing and implementing e-Learning, teachers’ motivation and implementation ability will increase. That study also finds that cross-school collaboration, including co-planning and peer observation of e-Learning lessons are most conducive in enhancing teachers’ e-Learning knowledge and practices.

For media enquiries, please contact Ms Queenie Wong, Senior Manager (Development and Communications) (Tel: 2219 4270 / Fax: 2540 6360 / E-mail: qlpwong@hku.hk).

Please visit http://web.edu.hku.hk/press to view the e-version of the press release. Tables and photos can be downloaded at http://www.fe.hku.hk/press/20141120/.