Leaving No One Behind: Measuring the Multidimensionality of Digital Literacy in the Age of AI and other Transformative Technologies

Challenge

During the last decade, technology-driven transformations have created emerging challenges for education systems worldwide to adequately prepare future generations to meet the demands of a changing environment. The complex problems of the 21st century society require the development of a wide set of digital skills for life and work. Advances in artificial intelligence (AI) and automation are and will continue to transform job markets (International Telecommunications Union, 2018; McKinsey Global Institute, 2018; OECD, 2018b). All technology related skills (both basic and advanced) will continue to see substantial growth in demand, especially in critical areas such as advanced IT and programming, AI, robotics, machine learning, and nanotechnologies. There are currently tens of millions of jobs available around the world for those with advanced digital skills, but unfortunately, a shortage of qualified people to fill them. This mismatch in the labor market is expected to grow in the coming years. In fact, most of today’s global workforce does not have the digital skills required for today’s jobs, let alone the skills for future jobs that will be available in the next five to ten years.

In a world driven by transformative technologies, there is a critical need for global citizens of all ages to have a set of digital skills to live, work, learn, and participate in modern society. Thus, digital literacy is vastly becoming a foundational framework of skills akin to reading and writing. At the macro level, digital literacy and e-inclusion are now broadly recognized as instrumental means to fostering sustainable development and economic growth, and reducing social inequalities (e.g., European Commission, 2016; Fraillon et al., 2015; ILO, 2016; ITU, 2018; Karpati, 2011; UNESCO, 2018; van Deursen, Helsper, & Eynon, 2014). However, to date, there is little consensus as to how digital literacy should be defined and measured.

At the same time, employers demand for “human skills” or socioemotional skills have also been on the rise (Acosta, Igarashi, Olfindo, & Rutkowski, 2017; Bughin et al., 2018). As workers increasingly interact with ever-smarter machines, human-machine interaction is being transformed and employers are looking for workers who can bring unique “human” elements to the work environment. The ability to engage in a digital world and perform tasks effectively is highly dependent on digital skills functioning together within a wider framework of soft skills such as critical and innovative thinking, complex problem solving, communication, and ability to collaborate. Despite their growing importance, these soft skills are often left out of discussions related to digital literacy and einclusion. And yet, within the digital literacy context, these skills are critical to an individuals’ ability to successfully apply technical know-how to solve real problems (UNESCO, 2018; Neisser et al., 1996).

In the past few years, the G20 has played an important role in raising awareness about the importance of digital skills and the transformational change that is underway. It now must play an even more critical role in helping the world prepare for this transformation with a better alignment between education and workforce needs. Laying out a more comprehensive strategy for digital literacy will help employers better communicate and reframe their needs. It will also help those in the labor market to prepare for the skills that will be needed and to better market those skills, thereby increasing their long-term employability, creativity, performance, and occupational attainment.

Proposal

Recent G20 communiques have emphasized the need for a global educational framework to improve basic digital skills’ awareness and capabilities (e.g., Chetty et al., 2017, 2018; Cobo, Zucchetti, & Rivas, 2018). These reports have also included recommendations to improve general infrastructure related to the availability and affordability of information and communication technologies (ICTs). There is now a critical need for the G20 to expand these efforts to create a more dynamic and evolving definition for digital literacy. This definition should include human skills and higher-order cognitive digital skills related to current and emerging frontier technologies that are critically needed in today’s digital age. Educational systems and labor markets around the globe need a more holistic and systematic approach to ensure the development of digital literacy for all segments of society. This comprehensive framework should provide a more complete picture of the entire digital skills ecosystem necessary to transform digital literacy in the classroom to tangible outcomes in the work environment. The following are specific recommendations to assist the G20 in achieving this objective

Recommendation 1: Defining the multidimensionality of digital literacy and e-inclusion

As a first step, the G20 is well positioned to lead the process of developing a more inclusive definition and strategy for “digital skills” and what it means to be digitally literate and e-included in the 21st century. The multidimensionality of digital literacy needs to be emphasized in this definition so that it moves beyond the current narrow focus on basic technical abilities to include the more complete set of socio-technical skills needed. This definition also should be fluid enough so that it can be regularly updated to keep pace with rapidly changing technological advancements and the need for new digital skills as they arise (Chetty et al., 2018).1

The process of acquiring digital skills not only includes gains in technical access, knowledge and expertise but also the ability to apply that knowledge so as to reach tangible and meaningful outcomes in the workplace and beyond (van Deursen, Helsper, & Eynon, 2014). To this end, the digital age requires the mastery of a wide range of competencies that go beyond the technical skills relevant to a specific industry or career path. It also includes the complex human elements mentioned earlier that enhance an individual’s ability to effectively put into practice digital technologies. According to The World Bank’s World Development Report (2018), “it is not enough to train learners to use computers: to navigate a rapidly changing world, they have to interact effectively with others, think creatively, and solve problems.”

To create a more holistically-prepared, digitally-skilled global population, a comprehensive and fluid framework covering both technical and socioemotional dimensions is needed. As shown in the sub-sections below, frameworks assessing technical digital skills have so far been kept separate from those focusing on socioemotional skills. It is important for the G20 to move beyond the dichotomy-based, domain-specific understandings of skills, and to build upon existing frameworks in order to support the implementation of an integrated scheme that includes the multidimensionality of digital literacy.

1.1 Technical digital skills definition and assessment frameworks:

Organizations and institutions such as the OECD, ILO, ITU, and the European Commission have been working for several years to create frameworks related to digital skills definitions and assessments (e.g., European Commission, 2016; Fraillon et al., 2015; ILO, 2016; ITU, 2018; Karpati, 2011; UNESCO 2018; van Deursen, Helsper, & Eynon, 2014; World Economic Forum, 2018). As an example, we reference the work of the European Commission and its Digital Competence Framework for Citizens (also referred to as DigComp). DigComp assists policy makers and educators in formulating strategies to improve digital competencies in Europe by providing them with “a common language” on how to identify and describe the key areas of digital competence that are needed (Vuorikari, Punie, Carretero, & Van den Brande, 2016).

The work of the EC and others defines digital literacy across a variety of digital contexts and mediums (e.g., computer, Internet, tablet, mobile phone, digital applications, platforms, and software). While there is some disagreement as to the specific elements that should be included in any definition of digital literacy, the vast majority focus on at least seven core domains of basic digital skills: (1) basic knowledge and operational skills, (2) information usage, (3) communication literacy, (4) content creation, (5) digital citizenship, (6) careerrelated competencies, and (7) industry-specific competencies (e.g., Deloitte China, 2018; ILO, 2016; The World Bank, 2016, 2018; UNESCO, 2018; World Economic Forum, 2016). Note again that these domains uniquely emphasize the technical aspects of digital literacy – and primarily, basic skills, which are needed at the primary and secondary education levels. Rarely, if at all, do these frameworks address the more advanced, emerging, and specialized frontier skills in high demand by employers, but in particularly short supply. These are skills such as computational thinking and coding that need to be included at the tertiary (post-secondary) level, as well as within vocational education and training programs.

1.2 Socioemotional skills definition and assessment frameworks:

As with technical digital skills, a number of frameworks now exist to define and measure what it means to be socio and emotionally capable (CASEL, 2019; Chernyshenko, Kankaraš, & Drasgow, 2018; Guerra, Modecki, & Cunningham, 2014; ILO, 2016; OECD, 2016, 2017, 2018a; UNESCO, 2018; World Economic Forum, 2018). This includes skills such as emotional intelligence (EQ), creativity, leadership, perseverance, communication and relationship skills, ability to work in teams, decision making and critical thinking, and numerous others. Also included are employers’ demands for critical personal attributes such as honesty and a positive attitude, which have been cited by employers as traits that are most critical, and yet, difficult to find (ILO, 2016). The OECD recently launched three international assessments examining social and emotional skills of both children and adults. The studies have included the Programme for International Student Assessment (PISA) Students’ Well-Being, the Survey of Adult Skills (PIACC), and the Study on Social and Emotional Skills (OECD, 2016, 2017, 2018a). Other researchers have developed a set of core metrics in social and emotional learning (SEL) that can predict educational and life outcomes such as those related to health and overall well-being (CASEL, 2019; UNESCO, 2018). The Collaborative for Academic, Social, and Emotional Learning (CASEL) in the United States has created a framework for understanding SEL and the process through which children and adults develop and effectively apply socioemotional skills at school, at home, and in the community.

Among the work of the OECD, CASEL, and others, there are considerable differences in what the SEL skill domains should be and how they should be measured. For example, the OECD’s framework emphasizes the “Big Five” – conscientiousness, openness to experience, extroversion, agreeableness, and emotion stability. CASEL also focuses on five core competencies but defines them in terms of self-awareness, self-management, responsible decisionmaking, relationship building, and social awareness. Most existing SEL frameworks include elements of the following socioemotional domains: (1) self-awareness, (2) basic cognition, (3) higher-order cognition, (4), communication ability, (5) social awareness, (6) self-management, (7) professional skills, and (8) personality traits.

1.3 Merging the technical digital skills with the softer human skills:

Taken as a whole, the above proposed frameworks emphasize various aspects of the skills needed for the new digital age. Still, efforts to define, measure, and monitor basic digital literacy have been largely kept separate from those related to higher-level technical skills and socioemotional learning. To date, there is no single model that has cohesively linked all of these socio-technical skills together. Nevertheless, the G20 can use these models as a foundational starting point to: (1) assess existing frameworks, (2) examine how they are currently being used, (3) evaluate their strengths and limitations, and (4) determine their potential for creating an integrated and more multidimensional definition for digital literacy as described above. The G20 can generate the conditions and promote common frameworks, which provide a clear definition of the main categories of digital literacy that should be included in any national or regional digital skills strategy. Figure 1 draws from several of these well-known frameworks to provide a comprehensive summary of the technical and non-technical skills most commonly recognized as being necessary for success in the new digital age. The first panel highlights the core dimensions of basic digital skills, with the addition of the emerging and frontier skills. The second panel showcases the non-technical, or human skills, grouping together mutually-related social and emotional skills into 8 big domains.

Recommendation 2: Establishing metrics to strengthen accountability for meeting digital skills targets and reducing gaps

In addition to developing a multidimensional definition for digital literacy, the G20 can promote the development of a core set of metrics and assessment tools that member states can use to consistently measure digital literacy along these different technical and non-technical continuums. These metrics are crucial to identifying existing skills gaps and tracking progress towards narrowing them, especially for segments of the global population that are particularly at risk of being left behind (e.g., youth, women and girls, low-skilled workers, migrants and refugees, rural farmers, the elderly, and persons with disabilities).

As efforts are being made to measure digital literacy, regular data collection is needed to analyze digital access, knowledge, practices, and capabilities, so as to facilitate a robust and comprehensive understanding of the digital skills divides. At the national level, adopting uniform methodologies and approaches across years are critical for educational systems and governments to discern trends in digital literacy and changes in digital divides over time. The G20 can support organizations such as UNESCO and ITU to encourage data sharing so as to facilitate regional and global comparisons and help shape educational learning targets and general guidelines for establishing common metrics and diagnostic tools (Costin & Coutinho, 2018).

Global metrics can motivate action and generate accountability for reducing digital divides and meeting digital skills targets. Akin to other international assessment tools such as PISA and TIMSS that focus on gauging mathematics and science knowledge of students, a framework for assessing digital literacy is key to understanding individuals’ 21st century skill levels and growing needs (2). As per the multidimensional definition of digital literacy, these involve technical and non-technical (human oriented) skills. The latter include both foundational skills (basic literacy, numeracy, critical thinking, and problem solving) and higher-order skills (advanced version of the foundational skills including adaptive learning) (UNESCO, 2018). The former exist on a spectrum from basic, intermediate to advanced skill levels.

While the development of a consistent strategy for global measurement is essential, the process poses some technical and political challenges. Obtaining global consensus on the scope and relevance of the metrics and assessment tools can be complicated. However, sound technical recommendations by the G20 as to the methods, measurement, interpretation, and feedback mechanisms can still be made to establish benchmarks and facilitate comparisons. There are challenges also related to financing, infrastructure and implementation capacity, and political will to implement such metrics and assessment tools. But most of these problems can be overcome through the establishment of clear goals and quality thresholds. The G20 can and should encourage governments to use a global framework as a complement to already existing national assessment systems and tools and not as a substitute for them. This process could therefore be viewed as a way to strengthen the capacity of national systems, as long as the advantages are clearly communicated, especially in terms of the “digital dividends.” To this end, the metrics should be framed so as to guide policy aimed at making sure that future generations have the skills needed to be employable, productive, creative, and successful in the digital age.

Recommendation 3: Linking digital literacy with tangible outcomes

As a third action step, the G20 can develop an international digital skills strategy that includes definitions, metrics and assessments to measure digital literacy, and guidelines for how education systems and employers can use these measurements to more effectively communicate and work together to bridge the digital skills divide. One of the main reasons for the current divide is the continued disconnect between education and the workforce. Shortages in digital skills are often discussed in ways that are disconnected from general debates about education and learning. As a result, education systems have not been preparing students and adult populations adequately to meet the job market demands. To this end, there is the need for a clear conceptualization of the digital skills ecosystem and the pathways by which education systems can work with employers and other key partners to successfully transform digital literacy in the classroom to more concrete and tangible outcomes in the work environment and beyond. Industry reports have begun to informally acknowledge the “digital dividends,” especially in terms of human capital accumulation, employability, and social mobility (Hernandez & Roberts, 2018; OECD, 2015a, 2015b). However, the environment and digital ecosystem needed to make these outcomes happen has not yet been formally conceptualized in a way that can be rigorously reviewed and tested. Also, many of these “digital dividends” still have yet to be defined, quantified, or measured, especially in terms of the non-technical human skills.

The G20 can foster the design of a model digital skills strategy that member states can adopt and follow. It can highlight current best practices in education and map out the specific pathways and tangible outcomes that can be achieved professionally, economically, culturally, socially, and personally. This type of mapping can help education systems more clearly identify the “returns” or “digital dividends” to acquiring digital skills and how they can directly improve future employability.

Recommendation 4: Developing a coordinated approach to digital skills literacy

Finally, the G20 can successfully provide leadership by creating a digital skills coalition or advisory body that undertakes the development and implementation of a global set of standardized skills and assessment tools. These tools would be used by member countries to set targets, track progress, and update policies and programs to meet evolving needs. Many of the existing assessment strategies discussed previously are highly useful to learn from and build upon. The key would be to take a more holistic approach that (1) encompasses both technical and non-technical dimensions, (2) addresses the continuum of technical digital skills, and (3) adapts the metrics for different target populations (segmented according to age, gender, educational attainment, employment status, socioeconomic status, geographical location, and displacement). Such efforts would provide more comparable data on digital skills levels and gaps. Countries and communities would be able to use these data to drive agendas, initiatives, and policies related to digital education, labor market outcomes, and reducing the digital skills divide. The challenge is that each country is starting from a different baseline and faces different barriers (economically, socially, culturally, politically, and educationally) in designing and implementing a successful digital literacy framework. Any efforts by an advisory body would need to carefully take these differences into consideration, especially disparities in ICT infrastructure and social and cultural norms that still prevent many populations such as women and youth from being digitally included.

Conclusions

The current policy brief addresses the multidimensionality of digital literacy and e-inclusion in light of the challenges that transformative technologies are creating for the future of work worldwide. In particular, it stresses the need to introduce comprehensive and holistic frameworks for defining, measuring and assessing digital literacy, which reflect the complex set of skills needed for creating tangible outcomes in the work environment and beyond. The G20 is well positioned to lead this process of developing comprehensive definitions, strategies and assessment tools for measuring digital literacy. These efforts would include the diverse set of skills – technical and non-technical – that are and will be needed in the future.

References

1. Acosta, P., Igarashi, T., Olfindo, R., & Rutkowski, J. (2017). Developing socioemotional skills for the Philippines’ labor market. Washington, D.C.: The World Bank.

2. Bughin, J., Hazan, E., Lund, S., Dalhström, P., Wiesinger, A., & Subramaniam, A. (2018). Skill shift: Automation and the future of the workforce (Discussion Paper May 2018). New York, NY: McKinsey Global Institute, McKinsey & Company.

3. CASEL. (2019). Core SEL Competencies. Retrieved from: https://casel.org/corecompetencies/

4. Chernyshenko, O. S., Kankaraš, M., & Drasgow, F. (2018). Social and emotional skills for student success and well-being. OECD Education Working Paper No. 173. Paris, France: OECD Publishing.

5. Chetty, K., Aneja, U., Mishra, V., Gcora, N., & Josie, J. (2017). Bridging the digital divide in the G20: Skills for the new age. Economics Discussion Papers, No. 2017-68. Berlin, Germany: Kiel Institute for the World Economy. Retrieved from: https://www.economics-ejournal.org/economics/discussionpapers/2017-68/file

6. Chetty, K., Qigui, L., Gcora, N., Josie, J., Wenwei, L., & Fang, C. (2018). Bridging the digital divide: Measuring digital literacy. Economics: The Open-Access, Open-Assessment EJournal. Kiel Institute for the World Economy (IfW), 12(2018-23), 1-20. Retrieved from: https://dx.doi.org/10.5018/economics-ejournal.ja.2018-23

7. Cobo, C., Zucchetti, A., & Rivas, A. (2018). Redesigning education landscapes for the future of work: Third-space literacies and alternative learning models. T20 Policy Brief. Retrieved from: https://www.g20-insights.org/policy_briefs/redesigning-educationlandscapes-for-the-future-of-work-third-space-literacies-and-alternative-learningmodels/

8. Costin, C., & Coutinho, A. J. (2018). Bridging the education-workforce divide: Strategies to meet ever changing needs and mitigate future inequalities. T20 Policy Brief. Retrieved from: https://www.g20-insights.org/policy_briefs/bridging-the-educationworkforce-divide-strategies-to-meet-ever-changing-needs-and-mitigate-futureinequalities/

9. Deloitte China. (2018). A new era of education: China education development report 2018 [Online]. Retrieved from: https://www2.deloitte.com/content/dam/Deloitte/cn/Documents/technologymedia-telecommunications/deloitte-cn-tmt-china-education-development-en-report2018.pdf

10. European Commission. (2016). A new skills agenda for Europe: Working together to strengthen human capital, employability, and competitiveness. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Brussels, Belgium: Author. Retrieved from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52016DC0381

11. Fraillon, J., Schulz, W., Friedman, T., Ainley, J., & Gebhardt, E. (2015). International Computer and Information Literacy Study: ICILS 2013: Technical Report. Amsterdam, The Netherlands: International Association for the Evaluation of Educational Achievement (IEA).

12. Guerra, N. Modecki, K., & Cunningham, W. V. (2014). Developing socio-emotional skills for the labor market: The practice model. Policy Research Working Paper 7123. Washington, D.C.: The World Bank.

13. Hernandez, K. & Roberts, T. (2018). Leaving no one behind in a digital world. K4D Emerging Issues Report. Brighton, UK: Institute for Development Studies.

14. ILO. (2016). ASEAN in transformation: Perspectives of enterprises and students on future work. Geneva, Switzerland: ILO. Retrieved from https://www.ilo.org/wcmsp5/groups/public/—ed_dialogue/— act_emp/documents/publication/wcms_579556.pdf

15. International Telecommunication Union (ITU). (2018). Digital Skills Toolkit. Geneva, Switzerland: ITU. Retrieved from: https://www.itu.int/en/ITU-D/DigitalInclusion/Documents/ITU%20Digital%20Skills%20Toolkit.pdf

16. Karpati, A. (2011). Digital literacy in education. IITE Policy Brief, May 2011. Moscow, Russian Federation: UNESCO Institute for Information Technologies in Education. Retrieved from: https://unesdoc.unesco.org/images/0021/002144/214485e.pdf

17. McKinsey Global Institute. (2018). AI, automation, and the future of work: Ten things to solve for. Briefing note prepared for the Tech4Good Summit, Organized by the French Presidency. Retrieved from: https://www.mckinsey.com/~/media/McKinsey/Featured%20Insights/Future%20o f%20Organizations/AI%20automation%20and%20the%20future%20of%20work% 20Ten%20things%20to%20solve%20for/MGI-Briefing-Note-AI-automation-and-thefuture-of-work_June2018.ashx

18. Neisser, U., Boodoo, G., Bouchard Jr, T. J., Boykin, A. W., Brody, N., Ceci, S. J., … & Urbina, S. (1996). Intelligence: Knowns and unknowns. American psychologist, 51(2), 77.

19. OECD. (2015a). Does having digital skills really pay off? Adult Skills in Focus No. 01. Paris, France: OECD Publishing. https://www.oecdilibrary.org/docserver/5js023r0wj9ven.pdf?expires=1553305364&id=id&accname=guest&checksum=FBAEF3A2232BA12133E758C1EB13F419

20. OECD. (2015b). The G20 skills strategy for developing and using skills for the 21st century. Paris, France: OECD. Retrieved from: https://www.oecd.org/g20/topics/employment-and-social-policy/The-G20-SkillsStrategy-for-Developing-and-Using-Skills-for-the-21st-Century.pdf

21. OECD. (2016). National skills strategy [Website]. Retrieved from: https://www.oecd.org/skills/

22. OECD. (2017). PISA 2015 Results (Volume III): Students’ well-being. Paris, France: PISA, OECD Publishing. Retrieved from: https://doi.org/10.1787/9789264273856-en

23. OECD. (2018a). Social and emotional skills: Well-being, connectedness and success. Paris, France: Directorate for Education Skills, OECD. Retrieved from: https://www.oecd.org/education/school/UPDATED%20Social%20and%20Emotional %20Skills%20-%20Wellbeing,%20connectedness%20and%20success.pdf%20(website).pdf

24. OECD. (2018b). Transformative technologies and jobs of the future. Background report for the Canadian G7 Innovation Ministers’ Meeting. Paris, France: OECD Publishing. Retrieved from: https://www.oecd.org/innovation/transformative-technologies-andjobs-of-the-future.pdf

25. UNESCO. (2018). A global framework of reference on digital literacy skills for indicators 4.4.2. Information Paper No. 51. Montreal, Canada: UNESCO Institute for Statistics. Retrieved from: https://uis.unesco.org/sites/default/files/documents/ip51- global-framework-reference-digital-literacy-skills-2018-en.pdf

26. van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2014). Measuring digital skills: From digital skills to tangible outcomes project report. London, England: The London School of Economics and Political Science. Retrieved from: https://www.lse.ac.uk/media-andcommunications/assets/documents/research/projects/disto/Measuring-DigitalSkills.pdf

27. Vuorikari, R., Punie, Y., Carretero, S., & Van den Brande, L. (2016). DigComp 2.0: The digital competence framework for citizens. Update phase 1: The conceptual reference model. European Commission JRC Science for Policy Report. Luxembourg Publication Office of the European Union. Retrieved from: https://publications.jrc.ec.europa.eu/repository/bitstream/JRC101254/jrc101254_di gcomp%202.0%20the%20digital%20competence%20framework%20for%20citizens .%20update%20phase%201.pdf

28. The World Bank. (2016). World Development Report 2016: Digital dividends. Washington, D.C.: Author. Retrieved from: https://www.worldbank.org/en/publication/wdr2016

29. The World Bank. (2018). World Development Report 2018: Learning to realize education’s promise. Washington, D.C.: Author. Retrieved from: https://documents.worldbank.org/curated/en/910761507785062573/pdf/120299- WDR-v2-PUBLIC-summary.pdf

30. World Economic Forum (2016). The future of jobs: Employments, skills and workforce strategy for the fourth industrial revolution. Geneva, Switzerland: Author. Retrieved from: https://www3.weforum.org/docs/WEF_Future_of_Jobs.pdf

31. World Economic Forum (2018). The future of jobs report 2018. Geneva, Switzerland: Author. https://www3.weforum.org/docs/WEF_Future_of_Jobs_2018.pdf