Evaluation

The evaluation design outlines the goals and activities of Applying Mechatronics to Promote Science (AMPS) and Central Brooklyn Robotics Initiatives (CBRI) projects at Tandon School of Engineering at New York University. The evaluation is designed to document the implementation of the program and to determine its impact on the participating schools, teachers, and pupils, as well as the NYU Tandon School of Engineering graduate Fellows and undergraduate Fellows. In addition, the evaluation will look at the linkages between specific program components and outcomes, as well as the potential sustainability and generalizability of the AMPS and CBRI programs.

The evaluators developed the evaluation plan in collaboration with the project leaders, Dr. Vikram Kapila, Dr. Noel Kriftcher, and Dr. Magued Iskander of NYU Tandon School of Engineering, who outlined the scope of the program as well as the key evaluation questions. The evaluation design was also informed by a review of the AMPS proposal and the evaluation and research literature on similar programs.

The AMPS and CBRI program targets schools that serve high needs students and aims to enhance their interest in math, science, and technology and motivation to pursue careers in these areas. The evaluation will assess the extent to which the program is serving the intended population and is introducing them to experiences that these students rarely encounter.

The program seeks to impact the Fellows, teachers, and students in different ways. Accordingly, the evaluation will look at specific outcomes for each group.

• For the students, the evaluation will look at the program's impact on their interest in math and science, their interest in math and science careers, their academic motivation, and their academic achievement (or factors related to academic achievement).
• For the teachers, the evaluation will look at the program's impact on their teaching techniques, confidence, motivation, and technology literacy.
• For the Fellows, the evaluation will look at the program's impact on their communication, leadership, and team building skills, as well as their ability to integrate research and teaching activities in their participating schools. Additionally, the evaluation will explore any impacts on the AMPS Fellows' doctoral experience.

The evaluation employs multiple measures and multiple evaluation models, including qualitative, quasi-experimental, treatment-comparison group, input-output, and path analytic designs - to assess the outcomes of the project and investigate causal relationships between project components and measured outcomes. The use of multiple measures and multiple models provides the robustness of convergent validity to the evaluation findings.