I recently had the opportunity to attend a workshop on the U.S. Department of Education’s (ED) new Education Innovation and Research (EIR) grant competition. EIR is the successor to the Investing in Innovation (i3) grant program, which invested approximately $1.4 billion through seven competitions from 2010 to 2016 to develop, validate and scale-up evidence-based programs in education. Like i3, EIR implements a tiered award structure to support programs at various levels of development – from early-phase grants that support the development, implementation and study of novel practices; to mid-phase grants that further develop, replicate or scale-up practices supported by moderate evidence; to expansion grants that scale practices supported by strong evidence and ready for implementation nationally.
What strikes me most about ED’s new investment is the further increase in commitment to rigorous evidence in education research. Beyond requiring a strong foundation in research and theory, EIR requires applicants to effectively scale-up their efforts and designs to include multiple sites and locations, define and measure successful implementation, design rigorous experimental studies, and examine the cost effectiveness of their programs. These are sound requirements for any discipline. What’s more, while ED has invited applications under the EIR program and released an estimated range of awards, funding is still awaiting final congressional action. ED’s willingness to open the competition in advance to allow adequate time to complete the application and hold an in-person workshop to review the applications underscores their strong desire to continue investing in rigorous education research and strong evidence-based practices.
For those who are interested in applying, I summarize below my seven takeaway points from the workshop. These seven points highlight the main changes in the transition from i3 to EIR.
Eligibility: EIR expands grantee eligibility to include state educational agencies (SEA) and the Bureau of Indian Education (BIE), while excluding institutes of higher education (IHE) that were previously eligible under i3’s definition of nonprofit organizations. However, private IHEs that meet the definition of a nonprofit can still apply. Alternatively, public IHEs may apply only through an affiliated nonprofit or other eligible applicant (i.e., a local education agency, SEA, a consortium of either, BIE, or nonprofit). The workshop provided a handy eligibility exercise to better understand qualified lead applicants.
Evidence requirements: Depending on the award, expectations on the level of prior evidence vary but all grants focus on validating and generating practices that demonstrate effectiveness. Early-phase grants require a logic model informed by current research and practice that suggest the program is likely to improve relevant outcomes. To aid applicants with developing their logic models, the workshop provided an Education Logic Model tool. Mid-phase grants require at least one quasi-experimental or experimental study that demonstrates statistically significant and positive findings on a relevant outcome based on a large or multi-site sample that overlaps with the proposed study (i.e., moderate level of evidence). Expansion grants further increase the rigor by limiting acceptable levels of prior evidence to only experimental designs (e.g., randomized control trials) that meet What Works Clearinghouse Evidence Standards without reservations.
Priorities: While EIR includes a host of priority areas under which applicants may apply, all applicants must target high-need students who are at risk for educational failure or otherwise in need of special assistance and support. In addition, new to EIR, mid-phase and expansion grants allow for the submission of filed-initiated priorities that meet evidence standards and are focused on a critical area of national need that would improve student achievement. Additionally, ED has reserved a quarter of EIR funds for applicants that serve a majority of schools designated as rural, i.e., National Center for Education Statistics locale designation of town, distant; town, remote; rural, fringe; rural, distant; and rural, remote.
Evaluation requirements: Building off the increased evidence requirements above, early-phase grant applicants are expected to include an efficacy study capable of meeting the criteria for moderate evidence. Moderate evidence entails an experimental or quasi-experimental design that demonstrates baseline equivalence. Mid-phase and expansion grants must have the potential to meet What Works Clearinghouse Evidence Standards without reservations, the highest evidence rating that can be bestowed. To meet standards without reservations, the unit of assignment (e.g., students, classrooms, schools) must be placed through a random assignment process. Mid-phase and expansion grants must also examine the cost effectiveness of programs on a per student basis.
Scaling requirements: It is expected that all EIR grantees scale-up program efforts over the course of the grant regardless of phase. Grantees are expected to submit a scaling strategy as part of the evaluation plan documenting their approach. For example, early-phase grants must scale to multiple schools over the course of the grant, mid-phase grants must scale regionally or nationally, and expansion grants must scale nationally.
Performance measures: In addition to a rigorous evaluation design, ED has also established several performance measures that grantees will be required to report on during annual and final performance reports. These include targets for the number of students served, cost per student served, and the extent to which evaluation designs provide evidence of program effectiveness. For example, well-designed evaluations will provide evidence of their effectiveness in multiple contexts, providing information on key practices and approaches to facilitate testing, development or replication. The goal of these evaluation design performance measures is not only to learn more about program implementation but also how implementation may affect others.
Funding: Grantees are also required to provide matching funds through in-kind or cash contributions from federal, state, local or private sources. Previously under i3 the match requirement had been tiered with early-phase grants (15%) requiring larger matches than expansion grants (5%). EIR levels out the matching requirement so that all grantees are required to match 10 percent of funds provided under the grant. This small change encourages more early-phase applicants who may have previously had difficulty in obtaining the higher match level, while helping to increase the likelihood that expansion grants are based on proven interventions that have the support of the districts and localities involved.
These seven takeaways summarize changes occurring in the transition between grant programs. I am pleased a strong commitment to rigorous evidence continues in the new EIR competition.
For those interested in a little more detail, I’ve provided a high-level matrix (below) highlighting the relevant priorities and requirements discussed above. The deadline for the 2017 EIR applications is Thursday, April 13.
High-level matrix highlighting the relevant priorities and requirements discussed above
Photo credit: Jessica Scranton/FHI 360
