An effective climate technology workforce development program: Unveiling participant needs through focus group interviews

1. Introduction

Workforce development is the key driver of economic growth in nations across the globe, and governments strives to fuel its productivity through nationwide programs. This imperative has recently taken on new significance with the burgeoning demand for a workforce in the field of climate technology, responding to the urgent need to address climate change. In addition, the global efforts to cultivate a larger workforce, driven by the competition for technological supremacy, along with the ongoing growth of the climate technology sector, have resulted in an increase in employment opportunities. IRENA and ILO (2023) reports that employment in global renewable energy jobs increased from 7.3 million in 2012 to 13.7 million in 2022.

Despite such increase in global demand for climate technology workforce, the Korean government has not actively pursued workforce development programs in this field. Analysis of the workforce development programs in South Korea shows that about a quarter of the programs are climate change themed. However, a few of these programs focus on the development of workforce in the climate technology sector, especially R&D related workforce (Joint Ministries, 2022). In response to this issue, the Korean government has launched the "Climate Technology Workforce Development Program," hereafter referred to as CTWDP. This pilot program was initiated in accordance with the Climate Change Technology Development Promotion Act, and the National Basic Plan for Climate Technology spanning from 2023 to 2032. As of 2022, the first group of students entered CTWDP’s pilot program and an assessment of the first phase was conducted (Kim et al., 2023). The results shows that the program is effective in improving the participants’ R&D capabilities, but it could not provide a comprehensive evaluation of the program due to short operation period.

This study builds upon the foundations established by prior research and adopts a unique approach by conducting a comprehensive assessment of the program from the participants’ perspective. Various stakeholders involved in the workforce development program have distinct objectives. For instance, the government seeks to increase employment in the climate technology sector, while universities want to introduce new programs that would enhance their reputation and attract more students. Hence, the program evaluation criteria tend to reflect the objectives of these stakeholders, such as number of employment or number of enrolled students. However, it is important to note that it is the willingness of the participants that influence these evaluation criteria and without fully comprehending their desires, it will be difficult to achieve the desired outcome. Therefore, by identifying the needs of the participants, this study will provide insights on how to design a workforce development program that can effectively attract and satisfy the participants.

The fundamental question this paper seeks to answer is: What do participants hope to gain from workforce development programs, such as CTWDP? To answer this question, pre- and post-program surveys are conducted, then followed by focus group interviews (FGI) with selected students to gain a deeper understanding of their responses. Afterwards, keywords from the responses are extracted and categorized into themes to gain a better understanding of the participants’ needs. A thorough examination of these results will provide insights into the core components needed for upscaling CTWDP. Furthermore, it will offer broader implications for the evolution of workforce development programs in analogous domains.

The study is organized as follows: Section 2 presents previous studies and identifies the research gap. Section 3 explains the methodology used in this study. Section 4 presents the results and analysis derived from our research. Finally, Section 5 concludes the paper.

2. Literature review

Program evaluation plays a pivotal role in the realm of public policy, offering a structured and systematic approach to scrutinizing the efficiency and effectiveness of various initiatives. At its core, program evaluation entails assessing the relationship between the inputs provided and the outcomes achieved. This process seeks to determine the extent to which specific indicators and goals have been realized, thereby shedding light on the efficiency with which these outcomes have been attained. Program evaluation employs a logical model that deconstructs program components, with a primary focus on inputs, outputs, and the outcomes aligned with program objectives. In the context of policy implementation and public administration, accountability and efficiency are paramount in determining program effectiveness, ensuring that resources are allocated judiciously and that intended results are realized (Barnow and Trutko, 2015; Vedung, 2017). Program evaluation is a multifaceted endeavor that employs a diversified array of methodologies and metrics tailored to the unique characteristics and objectives of individual programs. The selection of specific evaluation methods is significantly influenced by the national policy guiding a program and the diverse stakeholders involved in its implementation (McDavid et al., 2018). The diversity of programs and their objectives necessitates a tailored approach to evaluation to ensure a nuanced understanding of the program's impact and efficiency.

It is worth noting that program evaluation extends its value beyond a mere assessment; it often serves as a solid foundation for policymaking decisions and budget allocation, providing a factual basis upon which policymakers can formulate and execute their strategies for the upcoming fiscal year (Patton, 2008; Russ-Eft and Preskill, 2001; Stufflebeam and Shinkfield, 2007). Consequently, program evaluations often empower decision makers to substantiate the necessity for program improvement, expansion, or termination. These three dimensions of evaluation utilization, commonly categorized as instrumental, conceptual, and symbolic (Kim and Lee, 2015), each encompass distinct facets of how evaluation results are employed. Instrumental utilization embodies the practice of applying evaluation findings to instigate tangible changes in program outcomes, ultimately wielding a substantive influence on decision-making processes. Conceptual utilization, conversely, entails employing program evaluation results to enhance comprehension of program beneficiaries, exerting an impact on their perceptions and understanding, if not necessarily their actions. Symbolic utilization, the third dimension, pertains to leveraging evaluation results for purposes of persuasion or garnering political support, as exemplified by presenting dissenting opinions in opposition to unfavorable policies. This study, in particular, embraces the concept of instrumental utilization, with a specific focus on the deployment of evaluation outcomes to facilitate the upscaling and development of programs in climate technological field.

When evaluating workforce development programs, it becomes imperative for decision-making stakeholders to allocate resources strategically to sectors of national significance. As a response to industrial shifts towards AI and climate technology, the Korean government’s human resource development policies now put more emphasis on nurturing skilled workforces in the digital and climate technology sectors. While extensive literature exists on program evaluations related to digital workforce development in South Korea (Cho, 2022; Jang and Oh, 2023; Park and Kim, 2014; Shin and Seol, 2009), the same cannot be said for evaluations in climate technology sectors. For this reason, the previous study on the CTWDP by Kim et al. (2023) delved into reviewing the monitoring and evaluation (M&E) of R&D workforce development programs in South Korea.

Previous research predominantly focuses on the evaluation of workforce development programs from the viewpoint of the policymakers and institutional stakeholders. Commonly used indicators in these evaluations tend to revolve around the intermediate outputs resulting from short-term programs, often centered on employment and entry rates as principal metrics (Kim and Bae, 2022; Kim et al., 2017; Lee et al., 2020; Park and Kim, 2014). This approach primarily gauges the immediate and short-term impacts of these programs, providing insights into their effectiveness from the perspective of policy makers and institutions. Additionally, many national workforce development programs undergo efficiency assessments, including cost-benefit analyses such as the calculation of the cost per earnings and the return on investment (Barnowa and Trutko, 2015).

In contrast to these existing paradigms, this study posits that the effectiveness of workforce development programs should be examined more closely from the perspectives of program beneficiaries or recipients. Effectiveness indicators from the recipient's standpoint frequently involve self-judgment criteria, such as satisfaction rates, participation rates, and skill enhancement, including job competency (Shin and Seol, 2009). This beneficiary-centric approach to program evaluation delves into the experiences and satisfaction levels of program participants, highlighting their perspectives on the program's effectiveness and their personal growth.

The literature reveals a gap in research, which primarily focuses on the achievements of these programs rather than their broader national impact in terms of human capital development. Instead of concentrating on immediate employment rates, workforce development should shift its focus to explore how program participants can contribute to national productivity in the long-term. To achieve this, longer-term assessments should be conducted to discern whether the impacts made during the program endure and lead to a sustainable positive influence on the national workforce.

Many program evaluations fall short of identifying the success factors and determinants that could inform the scaling-up of workforce development programs. This study proposes that program evaluation should serve as a gateway for program development. Retrospective evaluations should be utilized by policymakers not only to assess program effectiveness, but also to enhance and sustain national policy goals. Public policy, especially in the realm of workforce development, necessitates a long-term approach to achieve the intended impacts. However, programs may need to adapt and evolve based on interim results and changing circumstances, thereby emphasizing the need for continuous assessment and program improvement.

This study strives to broaden the scope of program evaluation, not by critiquing previous evaluations but by unlocking opportunities for more extensive use of evaluation results. National programs, too often, limit themselves to recognizing the immediate political advantages of their efforts, missing the broader societal effects and impacts. A more comprehensive perspective can yield greater program effectiveness and serve as the foundation for including evaluation stages in every policy's development and execution. By embracing a more comprehensive approach to program evaluation, governments and policymakers can unlock the full potential of their initiatives, ultimately leading to more effective policy development, implementation, and impact on society.

3. Methodology

4. Results and analysis

The survey results indicate that the students, on average, perceived the program as beneficial in enhancing their capacity and increasing their willingness to enter the climate technology sector (see Fig. 1). The capacity-related questions assessed abilities in various areas: exploring advanced technologies, analyzing industry needs, planning for joint research, and examining outcomes of academia-industry cooperation. They also evaluated skills in project success criteria establishment, project environment analysis, feasibility studies, project schedule management, human resources control for project execution, and ensuring adherence to quality ethics in project execution. In a similar vein, the willingness-related questions measured several aspects including assistance in clarifying career goals, desire to enter the sector, confidence in career development, potential for career growth, perception of the job market, and recognition of technological value. Collectively, these elements provide a comprehensive understanding of the students’ willingness and motivation to enter the climate technology sector.

Fig. 1.

Average change in capacity and willingness answers (2023 survey)

The survey results revealed significant and enthusiastic improvements, which were both astounding and encouraging. However, these survey results do not provide sufficient context to why the students responded as they did or what factors influenced their perceptions. Hence, FGI is conducted to provide a platform for students to express their thoughts and feelings in their own words, offering valuable insights to the true assessment of the program by the beneficiaries. The questions posed during the FGI were designed based on the survey questions, aiming to delve deeper into the reasoning behind the students’ survey responses.

Keywords extracted from the responses are then systematically classified into four distinct themes which best match them. The first theme, capacity, encapsulates keywords from students’ responses that relate to changes in their abilities. It provides insights into whether students feel their skills have improved as a result of participating in the program. The second theme, willingness, includes keywords that indicate students’ interest in the climate technology sector. It aids the understanding of their motivation and eagerness to be engaged in this sector. The third theme, recommendation, captures aspects of the program that students found beneficial. It provides valuable feedback on elements of the program that were well-received and effective. The final theme, improvements, comprises keywords related to suggestions for enhancements or modifications that students would like incorporated in future iterations of the program.

By categorizing the keywords in this manner (as shown in Table 1), a more structured and focused analysis of the data is facilitated. This enables meaningful conclusions to be drawn about students’ experiences and perceptions. The data reveals that students did not express a high level of confidence in their improvement over the course of the program. Given the relatively short duration of the program, students found it challenging to perceive a significant change in their capacities. As the majority of participants were graduate students, they held high standards for their capacity, often comparing themselves to students majoring in climate policy or technology. When students reported an improvement in their capacity, it was generally interpreted as an expansion of their knowledge base and a broadening of their perspectives rather than a drastic enhancement of skills. This nuanced understanding is crucial in interpreting the survey results.

Table 1.

Four themes representing participants’ needs

In terms of willingness, the counts were notably lower. Students did not anticipate a sudden shift in their career trajectories solely based on their participation in the program. While exposure to the climate technology sector broadened their career considerations, it did not necessarily imply a commitment to change their career paths. Many students answered that they feel more exposed to the sector, while they did not even consider it (due to the lack of knowledge) before participating in the program. This finding underscores the complexity of career decision-making processes and the multitude of factors that influence such decisions.

Then, the data indicates that the counts for recommendations and improvements were notably higher, suggesting that students felt more at ease expressing their thoughts and suggestions about the program. A significant number of students voiced various ideas on potential enhancements to the program. A common theme that emerged from these discussions was the need for more incentives for students. The students expressed that the program lacked clear incentives, making it less appealing for students to actively participate. Balancing the demands of the program with their existing coursework posed a challenge for several students, especially those in PhD programs. This feedback underscores the importance of aligning program demands with student capabilities and providing clear benefits that make participation worthwhile. Therefore, while students recognized the value of the program, they ultimately want the program to be more rewarding experience for participants in terms of supporting their career development.

A more detailed examination of the students’ responses can be achieved by breaking down each theme into keywords (see Table 2). The size of the keywords in the word cloud corresponds to their frequency, with larger words appearing more often in the responses, but the color of each keyword is randomly assigned and does not carry any significance (see Fig. 2).

Table 2.

Keywords extracted from responses and frequency

Fig. 2.

Word cloud of keywords

The keyword Practical Experience emerged as the most frequent term. This reflects the students’ appreciation for firsthand experiences, such as listening to industry experts or attending an expo. These experiences were deemed most beneficial by the majority of students. The primary focus of the students was on how effectively the program could prepare them for better job opportunities. Consequently, they expressed a desire for more exposure to industry experts and opportunities to personally interact with them to gain insights into industry trends. Then, the second most frequent keyword was Accomplishment. This term relates to students’ desire for more opportunities to produce tangible results, such as academic papers, R&D patents, or certificates that could enhance their resumes. Some students expressed a wish for increased collaboration with the industry, such as internships or field trips, which would allow them to establish connections with companies in advance.

The keyword Gain Interest signifies an increased interest in the climate technology sector. Some students initially felt that this sector was unrelated to their major, but they recognized potential career opportunities in this sector after participating in the program. Interactions with renowned lecturers with diverse backgrounds broadened their perspectives and prompted them to change their minds. Other keywords like Challenging relate to course logistics and reflect the difficulties faced by graduate students who found the topic too divergent from their own majors. Also, some students felt overwhelmed by their existing commitments and found it demanding to accommodate various activities into their schedule. However, students did mention that if clear incentives were provided by the program, they would be more inclined to participate. This feedback underscores the importance of aligning program benefits with student expectations and needs.

Overall, a recurring theme in the students’ feedback was the desire for the program to cater to their needs, particularly in career development. It is important to note that the program played a crucial role in broadening the students’ perspectives and increasing their interest in the climate technology sector. Many students answered that they were primarily focused on securing positions in large corporations, such as Samsung Electronics or Hyundai Motors, but being introduced to the climate technology through the program sparked a newfound interest in this emerging industry. However, for sustained engagement and change of long-term career paths, the students expressed a need for clear incentives. These could take various forms, such as internships, networking opportunities, or tangible achievements like certificates or research publications. By addressing these needs, the program can better support students in their career aspirations and foster long-term interest in the climate technology sector.

5. Conclusion

This study aims to uncover what participants would hope to gain from workforce development programs. After analyzing the survey and FGI results, two primary outcomes emerged: 1) participants are interested in gaining a competitive edge in the job market, and 2) workforce development programs can heighten their interest in the sector.

First, students are deeply invested in their career trajectories, and they view every activity as a potential enhancement to their career prospects. They are primarily focused on the practical experience and achievement aspects of the program, indicating a desire for programs that can boost their employability. They seek more industry connections to gain insights that could help them secure positions within the industry. They also desire tangible results that can bolster their resumes and give them a competitive edge in the job market. Therefore, if a program is too demanding without offering clear incentives, students may perceive it as a waste of time. This insight is crucial for the design of effective workforce development programs.

Second, workforce development programs can stimulate interest in the sector by providing new knowledge about previously unfamiliar areas. While students were uncertain about significant capacity improvement after one or two semesters of the program, they did report a noticeable increase in their interest. It is important to note that each student who participated in the program had their own major, some of which were related to climate technology. Therefore, expecting a drastic improvement in capacity over a short period might be unrealistic. Instead, the program succeeded in broadening students’ perspectives. Many students reported learning something new and gaining valuable insights about the climate technology industry. The interest was kindled because they recognized climate technology as a promising industry and an important global topic.

In addition, it is also important to acknowledge the clear limitations of what a workforce program can achieve. The results show that these programs are extracurricular activities that could increase the participant’s interest in a certain field but cannot drastically enhance their capacity. Many students reported that they did not feel their capacity had improved significantly, which means they are not confident to actively pursue jobs in the target sector. On the other hand, since they have gained interest, there is a chance that they might pursue a career in this sector if they can be convinced there is a demand for their services. However, this is beyond the capacity of a single program and requires every stakeholder involved in the sector to work together to create a bigger pie. Policymakers should expand the climate technology sector and create more opportunities for workforce integration. Also, universities should be willing to provide more incentives, such as acknowledging the program as regular course credit, to make it more attractive for the students. Therefore, a workforce development program has clear limitations, but also plays a pivotal role in shaping future professionals by igniting their interest and expanding their horizons.

Acknowledgments

We thank VF Partners for assistance in survey data collection and focus group interview of the Climate Technology Workforce Development Program (CTWDP). This research was supported by the National Institute of Green Technology, the National Research Foundation of Korea (NRF), and the Korean government (Ministry of Science and ICT) (NRF, grant number 1711192709).

References

• Ansay SJ, Perkins DF, Nelson CJ. 2004. Interpreting outcomes: Using focus groups in evaluation research. Fam Relat 53(3): 310-316. [https://doi.org/10.1111/j.0022-2445.2004.0007.x]
• Barnow BS, Trutko J. 2015. The value of efficiency measures: Lessons from workforce development programs. Public Perform Manag Rev 38(3): 487-513. [https://doi.org/10.1080/15309576.2015.1006466]
• Cho YB. 2022. Performance analysis of Korean digital key practical talent training program. J Pract Eng Educ 14(3): 573-577 (in Korean with English abstract). [https://doi.org/10.14702/JPEE.2022.573]
• IRENA (International Renewable Energy Agency), ILO (International Labour Organization). 2023. Renewable energy and jobs: Annual review 2023. Abu Dhabi, UAE: IRENA.
• Jang ES, Oh KS. 2023. Development of core competency assessment tools in basic SW·AI education. Korean J Gen Educ 17(2): 275-287 (in Korean with English abstract). [https://doi.org/10.46392/kjge.2023.17.2.275]
• Joint Ministries. 2022. Guide to Korea’s human resources development projects in 2022. Sejong, Korea: Ministry of Education.
• Kim DE, Chung YW, Sohn JW, Jeong ES. 2023. A study on effects of climate technology workforce development program. Ind Promot Res 8(2): 45-59 (in Korean with English abstract). [https://doi.org/10.21186/IPR.2023.8.2.045]
• Kim DY, Lee MY. 2015. An analysis on the type of HRD stakeholders’ subjective perception on utilization of HRD program evaluation results: Focusing on HRD practitioners, consultants, and researchers. Korea Bus Rev 19(1): 227-247 (in Korean with English abstract). [https://doi.org/10.17287/kbr.2015.19.1.227]
• Kim MJ, Bae KP. 2022. Improving graduate school financial support program: Focusing on the discriminatory effects of financial benefit on the human resources development performance. J Soc Sci 33(1): 289-306 (in Korean with English abstract). [https://doi.org/10.16881/jss.2022.01.33.1.289]
• Kim YJ, Moon IC, Ko KM. 2017. Effectiveness analysis on nuclear education policies - Focusing on the nuclear technology university student support program. Korean Policy Sci Rev 21(2): 29-50 (in Korean with English abstract). [https://doi.org/10.31553/kpsr.2017.06.21.2.29]
• Lee KH, Kim JD, Kim CS, Park HY. 2020. A study on the policy performance of manpower training program. Korean J Hum Resour Dev 23(1): 221-244 (in Korean with English abstract). [https://doi.org/10.24991/KJHRD.2020.03.23.1.221]
• Massey OT. 2011. A proposed model for the analysis and interpretation of focus groups in evaluation research. Eval Program Plan 34(1): 21-28. [https://doi.org/10.1016/j.evalprogplan.2010.06.003]
• McDavid JC, Huse I, Hawthorn LR. 2018. Program evaluation and performance measurement: An introduction to practice. Thousand Oaks: Sage Publications.
• Park JH, Kim GB. 2014. A case study on performance evaluation of IT human resource program in regional industry. J Soc Electron Bus Stud 19(1): 79-93 (in Korean with English abstract). [https://doi.org/10.7838/jsebs.2014.19.1.079]
• Patton MQ. 2008. Utilization-focused evaluation, 4th edn. Thousand Oaks: Sage Publications.
• Russ-Eft D, Preskill H. 2001. Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. Cambridge: Perseus Publishing Services.
• Shek DT, Sun RC. 2012. Focus group evaluation of a positive youth development course in a university in Hong Kong. Int J Disabil Hum Dev 11(3): 249-254. [https://doi.org/10.1515/ijdhd-2012-0035]
• Shin JW, Seol JS. 2009. A study for improvement plan by evaluation of IT human resource developing program performance. Commun Korean Inst Inf Sci Eng 27(1): 12-20.
• Stufflebeam DL, Shinkfield AJ. 2007. Evaluation theory, models and applications. San Francisco: Jossey-Bass.
• Vedung E. 2017. Public policy and program evaluation. New York: Routledge.
• Wyatt TH, Krauskopf PB, Davidson R. 2008. Using focus groups for program planning and evaluation. J Sch Nurs 24(2): 71-82. [https://doi.org/10.1177/10598405080240020401]