This page provides a basic introduction to the instruments used in the "Core" MET Study as well as links to pdf versions of the instruments. The instrument descriptions can also be found in the User Guide.
A survey was administered to all MET school principals all principals (approximately 330) just after the 2010-2011 school year (year two) ended. The survey contained items asking principals about their school's current teacher evaluation policies, trainings they received on teacher evaluation, comfort with teacher evaluation, and their perceptions of the effectiveness of the teacher evaluation system currently in place in their district. In addition, school principals were asked to rate the effectiveness of up to 12 participating MET teachers in their school using a six-point scale that ranged from Exceptional (top 5%) to Very Good (top 25%) to Good (top 50%) to Fair (top 75%) to Poor (bottom 25%) to Very Poor (bottom 5%). Additionally, principals could indicate that they were unable to rate their teachers. Principals gave a confidence rating to their judgments of teacher effectiveness on a 5 point scale ranging from Strongly Confident to Not at All Confident. Last, each principal reported how many formal observations and how many informal observations he or she had conducted with each teacher.
This teacher survey was administered only to MET Study participants. It asked teachers to report very broadly on current evaluation practices in their school, including trust in their principal; the credence given to ratings provided by administrators; the receipt of feedback on practice as well as whether action based on that feedback was taken and the result of such actions.
The Teacher Working Conditions Survey asked teachers to report on levels of support in their school environment. Specifically, teachers reported the quality of the school facilities and availability of resources to support instruction; the extent that schools protect their time to plan and provide effective instruction; opportunities for professional development and the quality of that professional development; support to help teachers analyze student data and collaborate to improve instruction; support given to teachers in managing student behavior; the degree of teacher leadership present at the school; trust of school leadership and the level of support received from school leadership; and the level of parent and community involvement.
A more complete description of the standards upon which this survey was based can be found here: North Carolina Teacher Working Conditions Standards.
MET researchers administered several forms of an assessment designed to measure teachers' content knowledge for teaching. These assessments were specifically designed for and administered to:
ELA teachers in grades 4 through 6;
ELA teachers in grades 7 through 9;
Mathematics teachers in grades 4 and 5;
Mathematics teachers in grades 6 through 8; and
Algebra I teachers in grade 9.
The assessment for teachers of Mathematics in Grades 4 and 5 was administered to participating MET teachers in the fall of 2010 and the other four assessments were administered in early 2011, with the testing window ending in April.
The ELA assessments attempted to measure knowledge for teaching closely tied to the teaching of ELA, such as: choosing a text to support a specific teaching goal; selecting an activity to highlight a particular feature of a text or literary technique; choosing an activity to assess students' understanding; and analyzing student writing for weaknesses or strengths. The Mathematics assessments attempted to measure knowledge for teaching tied to the teaching of Mathematics, such as: choosing and using appropriate mathematical representations; choosing examples to illustrate a mathematical concept; interpreting student work, including use of nonstandard strategies; and evaluating student understanding.
Classroom Observation Instruments
CLASS is an observational protocol designed to measure the extent to which teachers effectively support children's social and academic development. Two different versions of CLASS were used in the MET Study: the Upper Elementary (Grades 4-5) and the Secondary (Grades 6-9).
The CLASS instrument is divided into three broad domains of measurement: Emotional Support, Classroom Organization, and Instructional Support. Each domain, in turn, is measured by a number of dimensions. The domain "Emotional Support," for example, refers to the emotional tone in a classroom, which can be measured along four dimensions: positive climate, negative climate, teacher sensitivity, and regard for student perspectives. The domain "Classroom Organization" refers to the ways a classroom is structured to manage students' behavior, time, and attention, which can be measured along three dimensions: behavior management, productivity, and instructional learning formats. The domain "Instructional Supports" refers to the ways a teacher provides supports to encourage student conceptual understanding and student problem solving and can be measured along four dimensions: content understanding, analysis and problem solving, instructional dialogue, and quality of feedback. CLASS also includes a fourth domain called "student engagement," which includes only a single scoring dimension.
FFT is an observational protocol designed to measure components of instruction that reflect a constructivist view of teaching. The instrument was used in the MET Study to code instruction in all focal subjects at grades 4-9. The FFT instrument divides the complex activity of teaching into four domains of teaching responsibility: Planning and Preparation (Domain 1), Classroom Environment (Domain 2), Instruction (Domain 3), and Professional Responsibilities (Domain 4). However, as discussed earlier, the MET Study scored videos on only two of these domains ("Classroom Environment" and "Instruction"). Each of these domains, in turn, is measured by a number of dimensions. The domain "Classroom Environment," for example, is measured along five dimensions: creating an environment of respect and rapport; establishing a culture for learning; managing classroom procedures; managing student behavior; and organizing physical space. The domain "Instruction" also is measured along five dimensions: communicating with students; using questioning and discussion techniques, engaging students in learning, using assessment in instruction, and demonstrating flexibility and responsiveness.
MQI is an observational instrument designed to measure the mathematical work that occurs in classrooms, on the theory that that such work is distinct from classroom climate, pedagogical style, or the deployment of generic instructional strategies. The MQI was used in the MET Study to code instruction in Mathematics at grades 4-9.
The MQI instrument measures the mathematical quality of instruction by assessing classroom instruction along six dimensions: Richness of the Mathematics; Errors and Imprecision; Working with Students and Mathematics; Student Participation in Meaning-Making and Reasoning; Explicitness and Thoroughness; and Connections between Classroom Work and Mathematics. The dimension Richness of Mathematics captures student meaning making and classroom mathematical practices. The dimension Errors and Imprecision captures major errors made by the teacher, imprecision in language and notation used by the teacher, and lack of clarity. The dimension Working with Students and Mathematics captures how the teacher responds to students' mathematical ideas and remediates student errors. The dimension Student Participation in Meaning-Making and Reasoning captures how students ask questions and reason about mathematics, students provide mathematical explanations, and the cognitive requirements of tasks. The dimension Explicitness and Thoroughness captures how complete the mathematical ideas and concepts are during the lesson. The dimension Connections between Classroom Work and Mathematics captures whether classroom activities are aimed at developing mathematical ideas. Raters score each segment on these 5 dimensions as well as giving an overall video score for each dimension. In addition to the main dimensions of the MQI, scorers rate a teacher on his or her apparent mathematical knowledge for teaching and provide a holistic score for the quality of the entire video.
PLATO is an observational instrument designed to measure four underlying qualities of ELA instruction: disciplinary demand of classroom activity and discourse; instructional scaffolding of ELA content; representations and use of content; and the classroom environment. PLATO was used in the MET Study to code instruction in ELA at grades 4-9. As used in the MET Study, eight (of 13 possible) dimensions of teaching were scored: intellectual challenge; modeling; strategy use; guided practice; classroom discourse; text-based instruction; behavior management; and time management.
Quality of Science Teaching (QST)
QST is an observational instrument for science teaching that was developed as part of the MET Study. QST was used in the MET Study to code Biology instruction at grade 9. QST measures three domains of science teaching: Assessing Teacher Content Knowledge; Engaging All Students in Learning Science; and Promoting Laboratory-based Inquiry. Each domain has three to four dimensions that are independently scored. The domain "Assessing Teacher Content Knowledge" includes three dimensions: Sets the Context and Focuses Learning on Key Science Concepts; Uses Representations; Demonstrates Content Knowledge; and Provides Feedback for Learning. The domain "Engaging All Students in Learning Science" includes three dimensions: Promotes Students' Interest and Motivation to Learn Science; Assigns Tasks to Promote Learning and Addresses the Demands of the Task for All Students; Uses Modes of Teaching Science Concepts; and Elicits Evidence of Students' Knowledge and Conceptual Understanding. The domain "Promoting Laboratory-based Inquiry" includes the dimensions: Initiates the Investigation; Provides Guidelines for Conducting the Investigation and Gathering Data; and Guides Analysis and Interpretation of Data.
State Level Standardized Tests
District administrative data containing student scores on standardized tests were collected. More detail on this process and the creation of value added scores can be found in Section 6.1 of the User Guide.
SAT-9 open-ended (OE) reading assessment. One of the supplemental assessments administered in all ELA classrooms in grades 4-8 was the Stanford 9 OE Reading Assessment. This assessment was administered in a single period. The assessment presented students with one extended reading passage and then asked them to respond to nine, open-ended tasks (which required students to provide short, written responses to comprehension questions). In the MET Study, all SAT-9 OE passages consisted of narrative text. The publisher reports a developmental scale score for the SAT-9 open-ended reading assessment so that student scores can be compared across grades. However, MET researchers did not administer the standard form and thus do not report these scale scores.
Balanced assessment of mathematics (BAM). The Balanced Assessment in Mathematics was administered in all Mathematics classrooms at grades 4-8. BAM is an open ended assessment that seeks to measure students' capacities in the following areas: modeling/formulating problems; transforming/manipulating mathematical formalisms; inferring/drawing conclusions; and communicating about mathematics. Each assessment form contains four to five assessment tasks and requires 50-60 minutes to complete. In the MET Study, concerns about generalizability led the MET researchers to administer three different forms of the BAM (from the relevant grade levels tests for 2003, 2004 and 2005) in each classroom. Thus, all students in a classroom (or class section) did not take the same test. Assessment tasks are scored on a four-point scale (ranging from attribute not present to attribute predominantly present) for each of the dimensions of thinking being assessed on the BAM (i.e., modeling/formulating; transforming/manipulating; inferring/drawing conclusions; and communicating). MET data files include a student's scores on each of these dimensions for each task, as well as the test form administered to that student.
ACT QualityCore. ACT QualityCore Assessments were the only student achievement tests administered to students in the MET Study at grade 9. The assessments were English-9, Algebra I, and Biology, with students taking the particular assessment corresponding to the subject of their MET section. In the MET Study, the end of course assessments include one form (administered during a class period) that included multiple choice items only, and another form (administered during a class period) that included 1-3 constructed-response items. ACT Quality Core Assessment scores are reported as a developmental scale score that can be used to track students' growth across multiple courses in the same subject area.
Student Perceptions Survey
The Student Perception (aka Tripod) Survey was administered to all consenting students in class sections taught by MET teachers. The survey is designed to measure seven dimensions of classroom instruction referred to by instrument developers as the "Seven Cs" (7Cs). These dimensions are: Care, Control, Clarify, Challenge, Captivate, Confer, and Consolidate. Care measures students' perceptions of whether the classroom is a safe place. Clarify measures students' perceptions of teacher behaviors that help students' to better understand the content being taught. Challenge measures students' perceptions of classroom rigor and required effort. Captivate measures students' perceptions of how well the teacher captures the attention and interest of students. Confer measures students' perceptions of how much a teacher takes students' points of view into account when teaching. Consolidate measures students' perceptions of how much the teacher helps students cognitively represent what they have learned in a connected way and how well the teacher promotes students' understanding of the interconnectedness of different curriculum topics.
A description of the items composing each scale created with the Student Perception Survey is also available.