The Israel Literacy Measurement Project: 2015 Report
Source: Cohen Center for Modern Jewish Studies
The Israel Literacy Measurement Project is an attempt to create a valid and reliable measure of knowledge of Israel. Beginning with the question, “what does it mean to be literate about Israel?” the team worked to establish assessment standards. Drawing on definitions of literacy in other social science disciplines and in consultation with subject experts, the research team developed a test bank of validated Israel-related questions. The question bank can be used with college-aged young adults to assess the extent and content of their Israel-related knowledge.
Conclusion
The purpose of this research was to create a bank of questions that could assess student knowledge about Israel. Although many programs have been created to help foster greater understanding and knowledge of Israel , no tool has been available for measuring their effectiveness. For the current project, a bank of 91 questions about Israel was created and validated using a sample of college students from around the country. The test bank is available to individuals or organizations interested in assessing student knowledge of Israel, and tests can be customized through item selection to assess knowledge gained over the course of a particular program, trip, or classroom experience.
Using the question bank has a number of benefits for assessing knowledge of Israel. The creation of the question bank was thoughtful, intentional, and based in concrete data. By soliciting suggestions of domains and concepts that experts in the field thought were crucial for student understanding, the creation of the test bank utilized a top - down approach. At the same time, a bottom - up approach was used, based on analyses of qualitative data — to assess what students could be expected to know about Israel. This top - down/ bottom - up approach yields questions that have been informed by experts and reflects realistic assessments of student literacy of Israel. Questions are neutral, unbiased, relevant, and based in student understanding of Israel. In addition, questions have been individually validated through statistical item analysis. Further, higher test scores have been shown to be associated with greater ability to think critically about Israel, as demonstrated in post - test interviews.
Continued work on the literacy project has a number of important objectives: on - going updates to maintain relevance and validity, promotion of the question bank for diagnostic and achievement testing, dissemination of results of further testing , and practical application of these results to the development of content and delivery of Israel education. Future testing can be used to assess not only what students know, but how they have learned what they know, and why they retain certain types of information and not others.
The question bank is the start of an educational process. In the next stage of research, essay questions may be added to assess how students use, interpret, and analyze the information they learn. In that form, the Israel literacy question bank will be able to measure the development of higher - order understanding of Israel as well as basic Israel literacy.
The testing to date has demonstrated a less than acceptable level of knowledge about Israel among students about to embark on Birthright Israel. More than half of all students answered less than half of the questions correctly, and over 90% scored less than 75%. This information deficit prevents students from contributing to discourse about Israel on campus in a meaningful way. The scores also raise concerns about potential disillusionment with Israel education prior to college. To help students engage in serious and educated dialogue on campuses around the country, educational programs must give them tools to better converse about Israel; read news articles with greater understanding; and arrive at a deeper, perhaps more nuanced, view of the country and its people. The ability to assess knowledge is key to the success of Israel educational programming and Israel studies courses, and the current question bank is one important step toward measuring Israel literacy.