2b: Data Collection, Analysis and Evaluation
1. What are the processes and timelines used by the unit to collect, compile, aggregate, summarize, and analyze data on candidate performance, unit operations, and program quality?
- How are the data collected?
- How often are the data summarized and analyzed?
- Whose responsibility is it to summarize and analyze the data?
- In what formats are the data summarized and analyzed?
- What information technologies are used to maintain the unit’s assessment system?
The unit is committed to consistently completing a process of assessment collection, analysis, and evaluation in order to evaluate student and program competence and quality to make informed decisions for the betterment of our program and the benefit of our students. Information collected at the four checkpoints provides data in regards to applicant qualifications, candidate proficiencies, unit operations, and program quality.
Over the past few years, as we have moved into a new system of collection (from traditional paper copies on file, to an electronic system capable of organizing and saving all data), our assessment coordinator has worked to categorize, organize, and find the best ways of using the data collected to make meaningful decisions.
LiveText was chosen as the electronic system, and the Assessment Coordinator monitors the submissions of the teacher candidate and program assessments into the LiveText data management system. The Assessment Coordinator summarizes data each semester by running automated LiveText reports. The data is analyzed yearly, at the end of each academic year by the Education department. The Assessment Coordinator also makes necessary changes to the data management system to be sure that our assessments and benchmarks are accurately reflected in our data collection system. This is an ongoing process of data analysis and organization.
The unit analyzes the collected assessment data on students at admission to the School of Education. This analysis of data occurs at monthly CTEP meetings. Program representatives review admittance criteria, discuss candidates that may not meet admittance standards, and decide if any candidates will be admitted provisionally or by interview. The SoE unit continues to analyze data while students progress through the blocks and established program benchmarks. During block internships, the review is collaborative with school faculty and professors teaching the co-requisite courses. Ongoing collected data reveals information necessary to analyze the growth and development of our students and their year to year progress through the teacher preparation process. Analysis also enables the unit to make any program decisions, and assure that any changes continue to be correlated with state and national standards.
Multiple forms of assessments are utilized and collected in order to measure effectiveness of all stakeholders in our unit. For our students, assessment methods include: Illinois tests of- basic skills, content, and professional teaching; grade point averages- cumulative, content, and professional education; technology competency exam; 11 candidate assessments; 6-8 program assessments; evaluation and assessment of professional dispositions; the student teaching evaluation; and documentation of any formal student complaints through a grievance policy (see 2.b.2).
For our program, effectiveness measures include: student teaching final evaluations, semester faculty evaluations, alumni surveys, the unit assessment system, program assessments, P-12 partner feedback and collaboration, annual DARE, and the LiveText data management system.
Information technologies that are used to maintain the unit’s assessment system are the LiveText data management system and Banner.
2. How does the unit disaggregate candidate assessment data for candidates on the main campus and at off campus sites, in distance learning programs, and in alternate route programs?
Assessment data regarding candidate performance is examined for all campus sites and for all programs. As reported on the yearly Title II report, we have 100% program completion of our teacher candidates. A teacher candidate cannot complete the program unless they have successfully accomplished all assessments that result in state certification.
When checkpoints leading up to program completion were examined, there had been some variations in candidate performance among programs. For example, before program coursework was completely aligned to national SPA standards, Secondary Social Science education teacher candidates were not passing their content area exam at a high enough rate. However, now that gaps in the program have been addressed and standards are adequately covered, pass rates are much higher.
Before the secondary education teacher candidates were required to complete an internship ‘block’ integrated with coursework leading up to student teaching, they were not performing as well in their student teaching placements as they had little opportunity to practice their teaching strategies and observe or try management techniques. Since we instigated a secondary internship ‘block’, student teaching placements for secondary and K-12 students have been much more consistently successful.
Student teacher evaluation forms from clinical faculty and school faculty from main campus and off-campus sites reveal consistent comments regarding the success of the student teaching semester. We are confident that we are preparing qualified teacher candidates for their student teaching semesters at both main and off-campus sites.
Students from both the main campus and off-campus sites perform similarly within the various components of the unit assessment system as well.
3. How does the unit maintain records of formal candidate complaints and their resolutions?
School of Education candidates have the same academic rights and responsibilities as do all Millikin University students (see the Academic Regulations and Grading System section of the Bulletin, p.18). There are, however, specific situations in the School of Education when negative decisions may be cause for student grievances. These include:
1. Denial of admission to or dismissal from the School of Education, a clinical experience or student teaching;
2. Evaluation of the candidate’s performance in teacher preparation courses, clinical experiences or student teaching; and
3. Lack of recommendation for state certification or for employment.
Candidates may file an appeal or grievance in any of these School of Education decisions (Bulletin, p. 128). Students should first confer with the Director of the School of Education regarding the complaint and the process of appeal. Appeal of decisions may be made, in writing, to the Committee on Teacher Education Programs through the Director of the School of Education. Candidates may have personal, academic or legal support in hearings concerning School of Education matters. If there is still no resolution after meeting with the Director and the CTEP, the candidate may appeal the decision to the Dean of the school. There shall be no further appeals beyond the Dean. If the complaint concerns a faculty member who has left the University and is unavailable or unwilling to respond to requests for grade changes, the Director or Dean, if necessary, shall have the power to change a grade.
All records of formal candidate grievances in relation to internships and one’s retention in the School of Education, and their resolutions, are recorded in CTEP minutes. These minutes are compiled in an electronic file as well as printed paper copy in a binder in the Office manager’s office. If a candidate complaint goes beyond CTEP to the Dean, then the Dean’s office holds records of these conferences. Coursework disputes that a student may have, in terms of grades on assignments, are handled by the faculty teaching the class, in collaboration with the Director of the School of Education if needed. Students are required to seek closure from the Dean, if they are not satisfied after meeting with the faculty member or the Director of the School of Education.
Return to the Top of the Page