Evaluating AIW Implementation: Tools for Schools & Districts

Classroom Implementation Profile: Training Protocol

Classroom Implementation Profile (CIP) – (users log in here: www.cipaiw.com)

The purpose of the CIP is to collect evidence that teachers’ learning from Authentic Intellectual Work professional development is being implemented during the teaching and learning process. The CIP utilizes an electronic tool that allows observers to conduct short observations of classrooms to collect AIW implementation data, and compile reports to analyze and share with staff.

The first Profile provides baseline data and guides district, school, and/or staff in setting goals for increasing authentic instruction and learning. Subsequent profiles will reveal trends and determine strengths. These profiles will also identify additional professional development needs pertaining to the AIW standards and criteria.

Training is required for access to and use of the Classroom Implementation Profile (CIP). The process assists teams in developing inter-rater reliability and consistent data collection processes. The CIP training is three-fold: manipulation of the electronic tool, guided practice for gathering data, and methodology for analyzing data. Participants in CIP training should be personnel who have strong AIW backgrounds and who use data for decision-making.

Student Understanding graph

This chart shows one school’s results for the AIW instruction criterion of Student Understanding. In CIP observations, evidence that students showed complex, in-­‐depth understanding of concepts, themes, or problems (rather than superficial, fragmented knowledge) increased from 32.19% of lessons in initial observations at the beginning of the year (inner circle) to 47.83% of lessons in observations later in the year following AIW PD (outer circle). These data helped the school sharpen their focus on students’ conceptual understanding in AIW teams.

Task-Student Work Correlation

Task-Student Work Correlation

Schools need to analyze student artifacts to show that student work is improving in authenticity every year. When a district collects revised Authentic Intellectual Work tasks and the representative, accompanying student work, the desired result is to show that high scoring authentic teacher tasks result in high scoring authentic student work. The data gathered when going through this process will be used to establish a correlation between teacher task and student work while measuring growth in authenticity on an annual basis for both tasks and student work.

The Task-Student Work Correlation provides another set of data for AIW implementation. To determine the correlation, teams of experienced AIW educators score multiple sets of tasks and corresponding student work. The resulting scores are reflected in a scatterplot that shows the correlation between the two. Annual correlations should be done at the same time every year to show one year's AIW impact on developing authentic tasks and how they affect student performance.

Innovation Configuration Maps

Innovation Configuration Maps

AIW innovation configuration maps (IC maps) are rubric-based templates that survey staff members and other members of the school community regarding degrees of implementation from “no implementation” to “full implementation.” IC maps enable participants to track individual and/or building/group progress toward understanding and implementing a program. Using IC maps gives stakeholders a clear picture of what full implementation will look like for Authentic Intellectual Work.

The maps are composed for a variety of AIW stakeholders— AIW coaches, administration, educators, students, school board, and parents. The maps cover basic categories that would affect each stakeholder group, including data analysis, self-reflection, AIW mastery, and professional development. Each stakeholder member fills out the appropriate IC map and compiles his or her individual and group scores.

The information gained from completing the IC maps provides an additional formative measurement tool that can help leadership teams as they analyze gaps in AIW understanding, plan for professional development, and assist all groups in reflecting on their AIW practice. While leadership can use the grouped data for program analysis, individual educators and students can use their own data to formulate goals for personal career plans, noting what they need to increase their AIW skills.

For questions, contact Tina Wahlert or Susie Peterson at tsedconsultants@gmail.com.