SPOTLIGHT ON INNOVATION SERIES
The US Department of Education has awarded multi-million dollar “First in the World” grants to 18 colleges and universities that are innovating to solve critical challenges with access, recruitment, retention, and student success. At AI, we have interviewed each of the recipients to learn more about the projects these institutions are pursuing, how their approaches are unique, and what other colleges and universities can learn from these new efforts.
This was the second year of the First in the World grants. You can read our interviews with the 24 institutions that received 2014 grants here.
Higher education for most students begins with proctored high-stakes placement exams that may not be the best approach, according to Jason Bryer, director of research and project evaluation at Excelsior College. Proctored exams can be intimidating, especially at open-enrollment institutions where more than half of students are directed into non-credit remediation courses at an estimated cost of $4 billion per year.
Excelsior College hopes to improve students’ early college experiences by developing an open-source assessment tool to measure student skills in academic and non-academic areas. They will use the $1.9 million First in the World grant to address the problem of placement and remediation, and to allow institutions to better target resources and services to students. We talked to Bryer to learn more about the system and their goals for the project.
Building a Comprehensive Diagnostic Assessment Tool
Many of Excelsior College’s students are adult learners, some who arrive with previous college credit. College officials realized they were limited in their ability to measure students’ core competencies in reading, writing, and math. Traditional placement exams and remediation put students at greater risk for dropping out and do not provide students with detailed information about their performance, Bryer explains. As an alternative, Excelsior will break new ground by building a tool to provide useful comprehensive, holistic assessment data and formative feedback on assessed weaknesses as early as possible in a student’s college career.
The tool, the Diagnostic Assessment and Achievement of College Skills (DAACS), will connect assessments and formative feedback to resources that students can use for support and extra help. The assessment component of DAACS will rely on a number of pre-existing assessments that have been tested and validated.
DAACS will evaluate both academic and non-academic skills, using some of the following assessments:
- Survey of Academic Self-Regulation to measure intrinsic and extrinsic motivation, self-efficacy, metacognition, self-regulation, personal relevant and control
- Short 8-Item Grit Scale by Duckworth and Quinn to measure grit
- Math Anxiety Scale Survey to evaluate math anxiety and test anxiety
- The New York State Regents Exams, and if necessary, the Massachusetts Comprehensive Assessment System to evaluate academic items for reading, writing, and mathematics.
What the Data will Do for Students
DAACS is designed to empower students, who can take portions of the assessment piecemeal and receive immediate feedback. That assessment data is also shared with academic coaches, advisors, and faculty members to provide a more complete picture of where students are, both in terms of non-academic skills and how effectively they’re mastering course content. Data will be used to:
- Provide students with a picture of their current abilities so the college can point them to the most relevant sources based on their results
- Inform predictive analytic efforts at Excelsior in order to more efficiently identify the students with the greatest need for support
- Allow success coaches to prioritize which students have the greatest needs and provide information about the type of help that will be most effective
“We want to provide students with information they can act upon. “It’s not a pass-fail. It’s about what are your strengths, what are your weaknesses? What are the resources and services that you can utilize to overcome any possible deficiencies while at the same time playing to your strengths?”
Jason Bryer, Excelsior College
In addition, Excelsior recently launched a Student Success Center with coaches to help students successfully transition to college courses. All these elements are based on the importance of providing positive college experiences as early as possible, because the earlier a student has a positive college experience, the more likely they are to persist, Bryer adds.
A Commitment to Providing an Open Source Solution
DAACS will be open source software, and Bryer hopes it will be available to other institutions in just two years. Excelsior will pilot the software during the second year of the grant. They are also partnering with Western Governors University to pilot the DAACS tool to test its effectiveness at a different institution.
Excelsior plans to make the DAACS tool available to all institutions sometime during the fall of 2017, Byers explains. The tool will also be expandable, so that other institutions will be able to add a science or other prerequisite. It will also allow institutions to add components that target other non-academic skills or to address additional prerequisites.
Keys to Success
“The biggest challenge is how to present results to students without discouraging them, particularly for students who may be on a lower ability level,” Bryer notes. “By no means do we want to turn students away. We want to inspire students.”
This is particularly important in cases where students may perform poorly in all the academic areas and may also have few strengths in non-academic areas, which is common among first-generation students. Feedback will incorporate multimedia, and they are already working on scripting early videos and other resources to help students feel motivated rather than discouraged. They also hope that the short videos and personal results will engage
Bryer also outlined three other key elements:
- Motivating students to take the results seriously and spend time thinking about and reflecting on how they learn.
- Helping academic coaches and advisors understand what these stresses mean to students so they can provide good scaffolding.
- Data that accurately informs predictive models. For example, their predictive models rely on research that suggests grit and math anxiety are predictive of student success, but recently, some studies have suggested that grit may not be as predictive as originally thought. The effectiveness of the Excelsior model relies in part on those variables.
Why You Should Watch This Project
Excelsior’s DAACS tool addresses growing concerns with placement and entrance exams, and is cost effective because the software and tool will be free, Bryer explains. Excelsior officials plan to make it available so that institutions can adapt it and integrate into their own system without ongoing costs.
“Given how many students we enroll and the limited resources we have, we’re trying to be smarter about who we reach out to. Knowing more about the students when they come in will allow us to target those students who are perhaps most at risk for attrition and reach out to them to get them the resources that they need.”
Jason Bryer, Excelsior College
Bryer suggests that colleges need to think about moving beyond the variables that were useful for a while, particularly in an era where colleges could be selective. With everyone going to college, institutions need more information about students and about who needs further support. Bryer compared the approach to wearable technology like the Apple Watch or fitness trackers. “Ultimately we want to help students make use of their strengths and be aware of their weaknesses as they move forward.”
