Designing an evaluation system to assess professional ability training in police colleges

The paper addresses shortcomings of traditional academic evaluations in police colleges, which often form isolated judgments and fail to capture the comprehensive vocational abilities required by modern police work. Given evolving social contexts and diverse policing tasks, the study aims to replace single-link academic assessment with a multi-participant vocational ability evaluation. It seeks to construct a scientific, objective, and dynamic evaluation index system that integrates inputs from society, police units, colleges, and students, and to deploy quantitative models that reduce subjectivity. The importance of aligning training with societal and policing needs motivates the adoption of methods that combine expert judgment with data-driven weighting to guide curriculum and training improvements.

Prior work highlights joint school–enterprise evaluation to improve objectivity in vocational assessments (e.g., Xu, 2017) and multi-dimensional evaluation modes combining examination, evaluation, and recommendation (Miu and Wang, 2020). Data-driven teaching quality evaluation systems leveraging big data for diversified applications and institutionalized management have also been proposed (Ma and Wang, 2018). Other studies address trajectories in vocational training, impacts of VET on adult skills and employment, and performance evaluation in distance vocational education. Synthesizing this literature, the paper identifies three needs: (1) a comprehensive, multi-subject index system overcoming single-standard and single-subject limitations; (2) improved quantification via practical, dynamic models that reduce subjective influence; and (3) optimization of multisource data to enable objective quantification guiding vocational education and training objectives.

Study design: Construct a four-subject evaluation framework where society, police units, colleges, and students jointly participate (Table 1). Build a hierarchical index system with six criterion categories and 16 indicators (Table 2). Combine qualitative expert input with quantitative data using entropy weights and a fuzzy matter-element model.

Index development: A semi-open, two-round expert questionnaire was used to screen and finalize indices. 200 questionnaires were distributed; 185 valid were returned. Experts rated indicator importance on a 1–10 scale. Weighted average importance (E) and variance (σ²) were computed. Selection criteria: E > 7 and σ² < 0.5. The final criterion layer C comprises: knowledge ability (C1), skill ability (C2), professional quality ability (C3), work quality ability (C4), social adaptability ability (C5), and physical quality ability (C6), mapped to 16 indicators with descriptive content and expert-weighted importance scores.

Entropy weight method: Construct an m×n matrix X of evaluation objects by indices. Normalize indices to obtain Y. Compute entropy for each index, derive information utility h_j and objective weights w_j via standard entropy-weight formulas. Combine subjective (expert) and objective (entropy) perspectives.

Fuzzy matter-element model: Represent each evaluated object M with features c and fuzzy values v (membership degrees). Use membership functions reflecting indicator directionality (the-bigger-the-better and the-smaller-the-better). Construct composite fuzzy matter-element matrices and compute Euclidean closeness degrees (ρH) to evaluation standards. Aggregate with weights to obtain comprehensive scores Z_j.

Evaluation standards: pH thresholds—Excellent: 0.45; Good: 0.40; Medium: 0.35; Poor: 0.20 (Table 3).

Example verification: One major at a police university; 100 graduates across 27 provinces (5 received third-class merit after one year). Six students (two per graduating year) were selected for evaluation. Data sources included school performance and teacher evaluations, work-unit colleague/leadership evaluations and year-end summaries, social service object evaluations, and student self-evaluations. All but self-evaluations were normalized and averaged. The analysis combined measured quantitative values with expert-weighted qualitative inputs. Entropy weights were computed after normalization; then fuzzy matter-element membership and closeness degrees were calculated and aggregated to comprehensive scores.

Computed weights: Evaluation subject weights—police work unit: 0.42; college: 0.21; student: 0.11; social service object: 0.26. Criterion weights—knowledge ability: 0.11; skill ability: 0.18; professional quality ability: 0.21; work quality ability: 0.19; social adaptability ability: 0.17; physical quality ability: 0.13.

Computation: Constructed the R16×6 fuzzy matter-element with 16 indices and 6 objects; calculated superior membership, difference-square compound fuzzy matter-elements, and comprehensive indices using the derived weights and closeness computations.

• The combined entropy-weight and fuzzy matter-element approach produced evaluations consistent with expert analysis, enhancing comprehensiveness and objectivity.
• Evaluation subject weights indicate internal work-unit (0.42) and social service object (0.26) inputs contribute most; college (0.21) and student self-evaluation (0.11) contribute less.
• Criterion weights emphasize skill (0.18), professional quality (0.21), and work quality (0.19) over knowledge (0.11) and physical ability (0.13); social adaptability (0.17) is also influential.
• Example of six students: comprehensive indices pH = [0.47, 0.435, 0.462, 0.432, 0.42, 0.41]; average 0.441, corresponding overall to Good per the evaluation standard (Excellent ≥0.45; Good ≥0.40). Best performer P1 (0.47); lowest P6 (0.41).
• The system integrates multi-source data and reduces subjectivity by balancing expert input with data-driven entropy weights, yielding results with higher social recognition.

By constructing a multi-subject, multi-index system and applying entropy weights, the study reduces reliance on single-source, subjective assessments and more accurately reflects the vocational competencies needed in policing. The fuzzy matter-element framework captures the relative closeness of student abilities to performance standards, enabling nuanced ranking and diagnosis. Findings that work-unit and service-object evaluations carry higher weights align with the operational nature of police work. Emphasis on skills, professional quality, and work quality highlights competencies most predictive of job performance, reinforcing the need to strengthen practical training and soft skills alongside knowledge. The results support dynamic feedback into curricula (e.g., increased weight on practical/second-class activities) and personnel development, enhancing alignment between training and societal/public security demands.

The paper develops a comprehensive evaluation index system for police college vocational ability training that integrates inputs from society, police units, colleges, and students. Using an entropy-weighted fuzzy matter-element model, it produces objective, systematic evaluations consistent with expert judgment. In an example with six graduates, overall ability levels were Good, with variability across individuals. Indices related to skills, professional quality, and work quality had the greatest influence, while evaluations from work units and service objects contributed most among subjects. The framework can guide curriculum optimization, emphasize practical training and social adaptability, and support dynamic, staged assessments to continuously improve talent training in police colleges. Future work could extend validation across more majors, institutions, and larger cohorts, and refine indices and weights over time as tasks and contexts evolve.

• Example verification involved only six students from a single major at one police university, limiting generalizability.
• Indices and their importance rely in part on expert questionnaires (200 distributed; 185 valid), which may introduce context-specific biases despite entropy weighting.
• Data were normalized and averaged from multiple sources; differences in data quality and availability across units may affect robustness.
• The study reports data available on request; lack of open datasets may limit external replication and benchmarking.