Research on the Evaluation Indicators of the Effectiveness of Practical Courses in University Product Design Curriculum

: The importance of educational reform has received significant attention in various universities in China. In the field of product design, the practical course "Curriculum Design" has actively explored the effectiveness of the curriculum. This paper starts with the connotation of curriculum effectiveness and discusses the principles, steps, and methods for establishing effectiveness indicators. The determination of effectiveness indicators for curriculum design is analyzed through specific case studies. Through methods such as questionnaire surveys, literature analysis, and interviews, a set of indicators related to curriculum effectiveness, including learning atmosphere, learning experience, teaching effectiveness, sense of achievement, and creativity, has been identified. The research on effectiveness indicators for the practical course "Curriculum Design" has a certain universality and provides valuable insights and guidance for effective research in other art-related courses. By implementing this research, it is expected to enhance the quality of practical teaching in university product design courses and provide effective educational support for cultivating innovative talents.


Introduction
According to the requirements of "Opinions on Further Deepening Education and Teaching Reform and Promoting Quality Education in an All-round Way" put forward in 2019, China universities emphasize the importance of education and teaching reform, including strengthening curriculum reform, improving teaching quality and improving teaching evaluation.Colleges and universities focus on improving the effectiveness of courses, improving the quality of education and teaching level, and all teaching units have made innovative explorations on the content setting, outline compilation and innovative methods of performance evaluation of practical courses.Curriculum Design is a practical course in product design specialty.Through the study of this course, students' practical ability and innovation ability can be improved, and they can complete product design independently.Exploring the effectiveness of such courses can provide useful reference for improving curriculum design and teaching methods [1].

An Overview of Curriculum Design
Curriculum Design is a practical course for product design majors, aiming at cultivating students' design ability and innovative skills.Its teaching content covers the basic knowledge, methods, tools and applications of curriculum design.The research on teaching, teaching methods and evaluation methods of curriculum design are two different concepts: one is the research on curriculum design, and the other is the research on universal design of professional courses, which are very different.Curriculum Design focuses on the cultivation of students' curriculum design ability and the accumulation of teaching practical experience, while curriculum design is that teachers use what they have learned and experience to design and improve the curriculum in order to improve the teaching quality and effect.Curriculum design is a kind of teaching activity, [2] which refers to teachers' work in selecting teaching materials and teaching methods, designing curriculum structure and content, making teaching plans and teaching evaluation standards according to teaching objectives and teaching requirements, so as to achieve educational and teaching objectives and requirements.[3]

The Meaning of Curriculum Effectiveness
The effectiveness of the course refers to whether the objectives of the course can be achieved, whether students can achieve the expected learning results and abilities, and whether teachers and students can give full play to their potential in the teaching process to achieve the best teaching effect.The theory of curriculum effectiveness emphasizes the effectiveness of curriculum design, that is, the quality of educational curriculum itself, including the clarity of curriculum objectives, the appropriateness of content, the rationality of teaching methods and so on.The theory of curriculum effectiveness focuses on the design of teaching content and objectives, and how to make the curriculum more conducive to students' learning and growth.[4] The effectiveness of the course mainly focuses on whether students achieve the teaching objectives.Effective curriculum design should be able to help students achieve the established learning goals.This means that students should master the required knowledge, skills and attitudes at the end of the course, and be able to apply them to the actual situation, whether the students' learning achievements and teaching quality are in line with the preset goals, and then adjust and improve the course; The effectiveness of the course also measures students' participation and satisfaction.Effective curriculum design should be able to stimulate students' interest and enthusiasm for learning, make students actively participate in curriculum learning and discussion, and make students feel satisfied, mainly through classroom interaction, practical activities and other teaching strategies and methods.In addition, the research on the effectiveness of the curriculum should also pay attention to the output quality of talents and the feedback of social actual needs.Effective curriculum design should be able to meet the actual needs and applications of society, such as training professionals who meet the market demand, improving students' employment competitiveness, understanding and analyzing social needs, and bringing them into the consideration of curriculum design [5].

"Curriculum Design" Curriculum Effectiveness Research Status
Through search engines such as Baidu Academic, Web of Science, and HowNet Search, it is found that the number of studies on the course of Curriculum Design is very small, and even less studies focus on the effectiveness of the course.Even when it comes to the issue of the effectiveness of the course, there are cases where the observation points are vague, and many observation points are subjective and objective.At present, most of the research related to it is from the perspective of universality, and most of them focus on the teaching reform, practical exploration and teaching mode of curriculum design.For example, Huang Zhicheng (2020) suggested that the evaluation criteria should be decomposed into the innovation of topic selection, the quality of software drawing, and the students' understanding of their own design schemes in the teaching reform research of "Mechanical Principles" and "Mechanical Design" under the background of engineering certification.Deng Jie (2019) In the exploration of teaching reform of mechanical design course design based on OBE, the grade evaluation should include four parts: the quality of drawings and design instructions, the defense situation and the design attitude, and emphasize the process assessment.Although the specific contents of this kind of research are not exactly the same, they all play a reference role in the effectiveness and teaching innovation of Curriculum Design.Most of the other academic studies discuss the universal teaching characteristics of curriculum design, and there are many and relatively systematic researches on this kind of literature.Curriculum Design, as an important part of practical courses in art colleges, has its own particularity in course nature, performance assessment and evaluation methods, and its research should also be put into the specific characteristics of departments.For example, Xie Xiaoqian and Han Yanfang put forward that the effectiveness evaluation system is also divided into two parts, namely, process effectiveness evaluation and summative effectiveness evaluation.Bao Shuimei and Chen Jiacheng's research on the effectiveness of college students' evaluation of teaching, using quantitative methods, through the methods of index consistency, retest reliability and rater reliability to test the effectiveness of teaching.Based on this, the evaluation of the effectiveness of Curriculum Design also needs to be completed through specific indicators [6].

The "Curriculum Design" Effectiveness Indicators
The effectiveness index of Curriculum Design refers to the index used to measure whether the curriculum design achieves the expected goals and effects.They are usually used to evaluate the quality of courses and the effectiveness of teaching, so as to improve and optimize courses.Effective teaching is not only effective for students to learn and develop [7], but also effective for teachers to learn and develop.The essence of evaluating the quality of classroom teaching is to evaluate the quality of classroom teaching according to certain standards (indicators).The selection of effectiveness indicators should be closely related to the objectives of the course, the characteristics of learners, teaching methods and evaluation methods.The evaluation of effectiveness indicators can help teachers and course designers understand the advantages and disadvantages of the course, so as to make targeted improvements and adjustments [8].

The Principle of Setting the Effectiveness Evaluation Index
The effectiveness index of curriculum design is an important basis for evaluating the quality of curriculum design.From the determination to the use of the index, it should be measured and distinguished according to the characteristics of specific institutions and students, school-running characteristics and other factors.But generally speaking, the setting of effectiveness indicators should follow certain principles [9].

Particularity
As a practical course, Curriculum Design has the characteristics of design discipline, and the overall teaching design also follows the characteristics of art design professional courses.However, its evaluation index should be considered from students' specific background knowledge, hobbies, students' pre-knowledge reserves, students' employment trends and other factors, evaluated from the regional characteristics of colleges and universities, teaching environment, teaching facilities and other aspects, and formulated from teachers' experience, age, education and other aspects [10].

Timeliness
The evaluation index of effectiveness is not only special, but also time-sensitive.Teachers should consider different students who have not faced the semester when revising and formulating the syllabus, and make timely adjustments according to the changes of subject content and social employment environment, so as to make Curriculum Design itself timely, adapt to the current educational environment and teaching needs, and constantly improve and update it on this basis to adapt to the development of education and the changes of students' needs.

Operability
The evaluation index should be operable.In the process of determining the effectiveness index of Curriculum Design, the situation of the evaluation object should be reflected from multiple dimensions, and both the interviewed object and the course process can be measured and controlled, so as to ensure that the obtained data results are true.It can not only reflect the changes and differences of the measured objects, but also better monitor and evaluate the overall teaching effect, teaching content and effectiveness.For indicators that are difficult to quantify, qualitative comments can be used to evaluate, and problems and solutions can be put forward to improve teaching and learning [11].

Ideological and Political Nature
To determine the evaluation index, we should ensure that there is a clear ideological and political content to join the ideological and political evaluation section of the curriculum, clearly guide students, and guide students in the way that young people like in this era We should add the evaluation methods and means of curriculum ideological and political education to the performance evaluation method, and formulate corresponding evaluation indexes, closely focusing on the requirements of graduates' quality [12].

Steps and Methods to Determine the Effectiveness Evaluation Index.
(1) Determine the research purpose and research questions.For the evaluation index of curriculum effectiveness of Curriculum Design, we must first determine the purpose of the index and the research problems.The main purpose of studying the evaluation index is to optimize the curriculum structure and enhance the practicality of the curriculum.Improve students' interest and participation in the process of course explanation, and better promote education reform.The purpose of the study is to determine the requirements and standards of the course itself and teach students in accordance with their aptitude, study the differentiated teaching means and methods for different students' characteristics and levels, and monitor and measure the teaching effect [13].
(2) Preliminary screening of evaluation index According to the particularity of the course, the regionality and the particularity of students, the evaluation index for evaluating the effectiveness is defined through certain research methods.For example, the universal index can be extracted through literature research, and then the questionnaire for teachers and students is conducted according to the specific situation, and the primary index of the effectiveness of the course is obtained by data analysis and comparison with the universal index [14].
(3) Determine the evaluation index After the universal evaluation index is determined, the weight of the evaluation index can be analyzed by Delphi method in the form of expert interviews.Determine the weight and score, fully reflect the performance level of the effectiveness index, so as to ensure the effectiveness and reliability of the index in the later experiments and tests.Of course, it should be noted that the determination of evaluation index is a dynamic process, which needs to be constantly adjusted and updated with the change of research problems and purposes [15].

The Method of Determining the Effectiveness Evaluation Index
When formulating the evaluation index of curriculum effectiveness of Curriculum Design, we must clearly define the index and measurement method to ensure that everyone understands and abides by the same standards.Select appropriate measurement tools, such as exams, quizzes, homework, projects, questionnaires, teaching evaluation, etc., to evaluate students' learning achievements and course achievement.In the concrete operation, it is the premise of objectively diagnosing the effectiveness of students' evaluation of teaching to make clear the dimensional structure and judgment basis of the effectiveness of students' evaluation of teaching.
Both qualitative and quantitative methods can be used to detect the effectiveness level of curriculum objectives [16].
4. There are Three Main Methods: Questionnaire, Literature Survey and Interview

Questionnaire Method
Questionnaire can understand students' needs and feedback.The questionnaire on the effectiveness of curriculum design generally includes students, teachers, alumni and HR of employers.Information about students, you can understand students' needs, hobbies, study habits and learning methods, as well as feedback to teachers' teaching information; For teachers' questionnaires, we can grasp teachers' own ideological trends and students' learning effects; The investigation of alumni can be reflected in the long-term follow-up effect on the course.The survey of teachers and students in the questionnaire should emphasize the regionality of the school, the current hardware conditions of the school, the influence of teaching environment on teaching, the students' own foundation and the teachers' own quality.For different interviewees, the questions should be specific and clear, and should be differentiated.Questionnaire is an important teaching evaluation tool, which can help researchers better understand the needs and feedback of teachers and students, optimize the teaching process and improve the teaching quality and effect.

Literature Survey Method
Literature survey refers to the collection and collation of information and data about problems, themes or research fields by consulting relevant literature.This method is used to study the effectiveness of Curriculum Design, not only to study the quantity, quality and research degree of existing literature, but also to classify and sort out the relevant curriculum evaluation indicators at present.When evaluating the literature, we should consider the following factors: the quality of the literature, the author's information, the time of publication, research methods, sample size, etc., and correspond to the actual situation of the colleges to be investigated.By comparing it with the actual situation, we can find the fit between it and the existing literature, and select the most suitable effective index for the investigation institutions.

Interview Method
Interview is a research method to collect data and information, which is usually used to deeply understand the interviewees' views, attitudes, beliefs, experiences and behaviors.Interviews can be conducted face to face, by telephone or online.The interview method focuses on the feedback of curriculum effectiveness evaluation indicators and the determination of weights, so this part focuses on interviews with experts, and analyzes the weight relationship between different weights and different indicators from a professional, disciplinary and more macro standpoint.During the interview, we should set the research purpose and questions, follow the guiding principles of curriculum design and the list of questions, record the interview, and transcribe and analyze it word by word.Data analysis can adopt qualitative or quantitative methods, and choose the appropriate analysis method according to the research purpose and problems.Finally, summarize, classify and explain the analysis results, and write a research report.The report should be clear, organized and accurate, and provide corresponding suggestions and opinions.

The Determination of the Effectiveness Index of Curriculum Design
The selection and application of effectiveness indicators is a systematic process, which requires comprehensive consideration of various factors and methods to ensure the objectivity, accuracy and effectiveness of the evaluation results.According to the principles and methods of making the effectiveness of Curriculum Design, this paper takes the product design major of Huainan Normal University as an example to discuss the process of making the effectiveness index.(A) the existing problems.
The product design major of the School of Fine Arts and Design of Huainan Normal University was approved as the first batch of first-class professional construction projects at the school level in 2017, and was approved as the first-class (brand) professional construction project of provincial quality engineering projects in 2018.After nearly five years of construction, the curriculum system and student evaluation mechanism have achieved considerable results.Curriculum Design has developed into a course with clear teaching objectives, clear key points and difficulties in this research system, which is more in line with OBE's educational philosophy.There are breakthroughs in teaching, teaching methods and performance evaluation methods, but there are still some problems.

The Method of Performance Evaluation is Subjective, and the Feedback of Evaluation Results is not Enough
The usual grades of the Course Design for product design majors are divided into four parts, accounting for 30% of the total.The usual grades are 100%, which are composed of ordinary homework grades (a1), group cooperation grades (a2), interactive grades (a3) and classroom attendance grades (a4).The formula is: ordinary grades =a1×0.4+a2×0.3+a3×0.2+a4×0.1.The final work effect score accounts for 70%, and the quality of the work effect is basically determined by the teacher.Assuming that teachers have no emotional tendency towards students, the course results will still be obviously different due to the uncertainties in teachers' personal aesthetics, academic level, teaching environment and evaluation methods, and the overall subjectivity is too strong; In addition, students can't get more specific and detailed information from the final grade information to help them study effectively.

The Teaching Content is Solidified and the Teaching Mode is Single, Which Leads to the Lack of Innovation in the Evaluation Method
Curriculum Design, as an important part of the practical curriculum system of product design specialty, is generally arranged in the seventh semester.Before the 2020 edition, the main content of the syllabus is based on graduates' portfolios.The basis and the original intention of the course are that most senior students do not have too much time and energy to temper their personal portfolios before completing graduation practice and graduation creation course.In order to have a better connection with graduation practice and graduation creation practice course, students' personal data will have a better integrity in the process of employment and job selection, so the main content of this course is set as a course based on personal portfolios.The course design results are comprehensively evaluated according to the completion of students' design tasks, design sketches and the quality of their portfolios.This form of homework based on teachers' subjective propositions limits the diversity of curriculum effectiveness, and the teaching method of centralized teaching also limits the diversified development of students.

The Curriculum Evaluation System Lacks the Correlation Analysis of Ideological and Political Elements
The singleness of the teaching content of Curriculum Design limits the integration of ideological and political education and major.The content of curriculum evaluation lacks the examination of ideological and political elements, and the matching degree of curriculum goal 4 set in the syllabus is not high.Table 1 is a matrix analysis of the relationship between curriculum design and graduation requirements, which mainly includes three aspects: knowledge requirements, ability requirements and quality requirements.For the quality level, we should combine the course objectives with graduation requirements to ensure that students can master and develop knowledge, skills and abilities that meet graduation requirements in course learning, and improve the comprehensive quality and employment competitiveness of graduates.After the 2020 edition, the syllabus of Curriculum Design mostly retains the previous content setting, but due to the introduction of OBE concept, the orientation and evaluation method of the course have been adjusted.Although the graduation requirement 13 (see Table 1) is related to the content of the curriculum design, and the numerical evaluation is M, in the specific curriculum goal 4 (Table 2), it is only accomplished through classroom mutual assistance and group cooperation (Table 3), which cannot better match this requirement.Therefore, in the process of combining ideological and political education with professional courses, we should keep the original "main position" and reform the traditional ideological and political courses, so that the ideological and political education can play a better role.Through forms such as classroom interaction, group assignments, and attendance, cultivate students' awareness of independent learning and lifelong learning.0.2

The Specific Indicators to Determine the Implementation of the Method
In view of the practical problems in the above courses, this paper studies the index setting of the effectiveness of Curriculum Design.
Literature investigation: By investigating the existing literature, through the theme of% =' curriculum design, teaching and effectiveness index' or the title of% =' curriculum design, teaching and effectiveness, relevant articles from 2018 to 2023 were selected, and irrelevant documents such as primary education were removed.A total of 133 documents were selected for keyword co-occurrence network analysis (Figure 1).If teaching effectiveness, effectiveness research, learning effectiveness, learning effectiveness, effectiveness and effectiveness evaluation are all derivatives of effectiveness, then in 133 articles, there are 65 times of effective and related words, and 29 times of index system and rating index.Through reading and summarizing these documents, it is concluded that the current index system mainly focuses on creativity, teaching content, teaching effect and satisfaction.
(1) Questionnaire survey According to the specific situation of product design major in Huainan Normal University, a questionnaire survey was conducted on the effectiveness index of the course "Curriculum Design" of product design major among students of grade 19 and grade 20.A total of 90 questionnaires were distributed, 85 questionnaires were recovered, and 85 questionnaires were valid.In the survey results of "Which part do you think has the most positive influence on you in the course of learning Curriculum Design", the learning environment and learning atmosphere have the highest evaluation, skills and attitudes.This can be assessed by tests, exams, homework and course feedback.Teachers need to arrange homework for students to consolidate and expand, and at the same time evaluate and feedback the students' learning situation in order to adjust and improve the teaching effect.The whole teaching process of Curriculum Design needs to be flexibly adjusted according to the specific teaching content and students' needs to ensure the realization of teaching objectives and the improvement of teaching effect.
(7) Creativity Whether students can gain creativity in the course and expand their innovative thinking can be tested by creative thinking test, impromptu thinking training and small experiments.The research team tested the results through specific innovative design propositions.For example, students in different courses are required to draw as many circles as possible within ten minutes and shape these circles into different objects.At the same time, the more specific and quantitative the students describe, which proves that the creativity is stronger.Researchers use similar methods to test whether the course is effective, so that teachers can improve and improve according to the evaluation results.

Conclusions
Through the analysis of literature, questionnaire survey and interview, we can understand the reactions and feelings of students and teachers to different teaching methods and strategies, so as to determine and study the effectiveness evaluation index of Curriculum Design.As an elective course of art design, curriculum design has universal significance in exploring different teaching methods and strategies.The process and methods obtained from this topic can provide reference for other courses and broaden academic horizons in design theory and research methods.After the course effectiveness index is determined, the specific operation and data analysis of the evaluation of the later indicators will be studied next.
In a word, through the research on the evaluation index of the effectiveness of this course, we can have a more specific testing method for the effectiveness of the course, which can optimize the course design and teaching and improve the teaching quality.At the same time, it also provides reference for students' professional development and teaching reform, which is helpful to cultivate students' product design ability and creativity, improve their professional quality and competitiveness, and is also of great academic significance for the integration of curriculum ideology and politics.

Figure 1 :
Figure 1: Network analysis of co-occurring keywords

Table 1 :
Analysis of the relationship matrix between the "Course Design" course and graduation requirements

Table 2 :
Correspondence between course objectives and graduation requirement indicators

Table 3 :
Course objective requirements and weights