In 2015, Moishe House (MH) began implementing a new pilot program designed to explore a model for engagement. Premised on peer-led retreats, Jewish young adults attend a weekend training called Retreatology: The Art of Jewish Retreat Making. Following their training, they then facilitate their own Jewish learning retreat—grounded in Jewish learning, Jewish values, and personal interests—for a group of their peers. MH provides facilitators with a mentor and a grant for up to $5,000 to create an immersive learning experience to assist in the planning and execution.

MH and the Jim Joseph Foundation, one of MH’s major funders, recognized the program merited a more substantive evaluation than just the standard feedback survey, but MH lacked the capacity to hire an external evaluator. Fortunately, the Foundation also wanted to learn from this program and has a member of its team with previous research experience. Thus, the two organizations embarked on a new experiment: a small-scale program evaluation conducted by a representative of the Foundation on behalf of the grantee.

Building and Maintaining Trust Between Funder and Grantee Partner
The two organizations maintained close and open communication throughout each step of the process, from determining the methodology to drafting the final report. Undoubtedly, MH staff had questions and fears as it related to handing the reins of its evaluation over to an important funding partner. Would it be awkward for interviewees to speak directly with a representative from the Foundation? How would it feel for MH to have its funder collecting feedback, both positive and negative, directly from its participants? Would the funder lens stifle the ability to generate actionable recommendations? What are the funder’s expectations of the findings? Due to the strong preexisting relationship between MH and the Foundation, both parties felt comfortable discussing these questions and hesitations openly and honestly, enabling the partnership to move forward.

The Evaluation Process
The Foundation conducted a review of existing formative assessments including: evaluation surveys (internally conducted via SurveyMonkey) and grant reports. Following the collection of this information, the Foundation drafted an interview protocol, which was reviewed by MH and the Maimonides Fund, the program’s visionary and primary funder. MH provided names and contact information of Peer-Led Retreat facilitators with varying levels of MH engagement. Over the course of two months, the Foundation contacted these facilitators and conducted five phone interviews, along with two additional interviews with retreat participants. Following the interviews, the Foundation analyzed the data and prepared a preliminary report of highlights and insights.

Substantive Learnings
The collaborative process not only provided MH and the Foundation with programmatic feedback, but many of the looming questions and hesitations regarding the process’s impact on the funder-grantee relationship had been easily identified and resolved due to the open communication and mutual respect the organizations shared.  It should be noted that this relationship had developed over eight years of relational grantmaking and this history plays a crucial part in the success of this experiment.  Without a solid foundation of trust, the organizations’ ability and willingness to be bold, yet humble, in facing the harder topics would have been a much greater challenge. Key learnings included:

• Interviewees were happy to participate and felt comfortable providing honest feedback. Interviewees appreciated the opportunity to speak directly with funders of an organization and program to which they are committed.
• Hearing directly from the participants helped increase the Foundation’s understanding of and appreciation for this program and the opportunity it presents for MH.
• The Foundation appreciated the opportunity to speak directly with its grantee’s program participants instead of through the veneer of a grant report.
• The process strengthened the funder-grantee relationship at different levels of each of the organizations. The strength of this relationship is significant and can have important ramifications for both parties when working together, especially when involving multi-year grants.

Partnering on this evaluation brought about several recommendations for program improvement, as well as questions for MH to consider when adapting and scaling the Peer-Led Retreat program. As a result, MH has spent more time investing in the following projects: financial streamlining, international access to Retreatology trainings, creating a cohort of Peer-Led Retreat mentors, exploring value propositions, and telling the Peer-Led Retreat facilitators’ stories. For those who are interested, we are pleased to share the report here.

Conducting this small-scale program evaluation brought MH and the Foundation into uncharted waters, but ultimately proved to be a beneficial experience for both organizations. When funders and grantees are able to develop open and honest communication, both the relationship and their programs are strengthened by new and shared insights. Following this experiment, both organizations realize the significance of this partnership and are looking forward to exploring other ways to collaborate and combine their respective expertise.

This is part 11 of the series in eJewishPhilanthropy, Continuing Conversations on Leveraging Educational Technology to Advance Jewish Learning. The series is a project of Jewish Funders Network, the Jim Joseph Foundation, and the William Davidson Foundation. For an in-depth look at opportunities in Jewish Ed Tech and digital engagement, read Smart Money: Recommendations for an Educational Technology and Digital Engagement Investment Strategy. Later this year, Jewish Funders Network will launch a new website to help advance the field of Jewish educational technology.

To understand and then appreciate technology’s role in our learning and our lives, we need to understand the essence of what technology is and what it allows us to do. When we move beyond brand names and the latest technical features we can see that technology is meant to allow us to increase our production, communication, and give us the ability to interface and interact with the world around us.

Image by Gregordy / WikiMedia Commons

There isn’t a single industry that exists today that doesn’t utilize the power of technology to increase or enhance its productivity, efficiency, or quality. Why then is education still discussing, and at times even struggling to validate, how technology can transform teaching, learning, and meaningful experiences in both?

Today, the word technology will evoke the images of iPads, laptops, and 4K displays, but I assure you that in its essence, and by definition, they are no more technological than the pencil and printing press before. To understand and then appreciate technology’s role in our learning and our lives, we need to understand the essence of what technology is and what it allows us to do. When we move beyond brand names and the latest technical features we can see that technology is meant to allow us to increase our production, communication, and give us the ability to interface and interact with the world around us.

Technology is becoming increasingly intelligent. Hotel rooms, refrigerators, even our coffee makers are not just automated but programmed to react and respond to best serve our needs, all while ensuring that the highest levels of efficiency and effectiveness are achieved. Our methods of communication, transportation, and service experiences, revolve around technological advances. Many of these new technologies didn’t exist even ten years ago and are still rapidly developing. You find technology creating an impact everywhere from hospitals, auto mechanics, airports, and even the grocery store. Technology has nearly changed every facet of industry, yet education still lags behind and technology is questioned. In 2017, we can still wonder about the ubiquitous nature of technology in education, and at the same time you would be hard pressed to find someone committed to a medical professional, mechanic, or contractor who opts out of using technology. Why is education different?

Access and availability are still a conversation because we have not yet addressed why technology is critical to successful teaching and learning. Technology’s impact on education isn’t going to be through the volume of technology present, but how it can shift and redefine the teaching and learning process. Technology is moving forward at a rapid pace in a direction that is mobile, handsfree, and voice responsive. We cannot view education in a different light and not provide students with the same tools that the real world interfaces with everyday. You might not (yet) have a smart lightbulb or Amazon’s Alexa but those days are numbered as companies incorporate these technologies into our everyday experiences whether we like it or not. 15 years ago you would be hard pressed to find a school that didn’t offer a keyboarding program. Today, while schools continue to invest hundreds or thousands into keyboarding, technology companies are spending millions to make them obsolete. Education needs to be on the edge of what technology is relevant now, not to jump on the trending bandwagon but be aware of how best to prepare our students for the year 2030 when they begin their professional journeys.

The Future of Educational Technology is Mobile, Creative, and Adaptive

Mobile technology allows us to experience the world around us. It directly impacts our ability to survey, compile, and engage with people, places, and things. So how does this interface with education? Well it depends on your definition of education, and what learning and develop you value. If you value memorization, fill in the blank, note taking, and lecture listening skills, then the purpose and value of technology will be very different than if one values creativity, complex nonlinear problem solving, critical thinking, reflection, and collaboration. Education should and must move beyond the 19th century model. The real world doesn’t function this way. While students must master the skill of memorization, documentation, and written communication, those must be components of a larger and more meaningful learning experience. With this in mind we can then look at technology as a tool to connect with others and capture experiences. Only then will technology use becomes authentic and access to it becomes more than justified but a requirement. At this point technology becomes a catalyst to help students acquire skills beyond memorization and essay writing, that will prepare students for the workforce of the future.

The World Economic Forum lists skills such as creativity, critical thinking, and emotional intelligence as top ranking skills that will be prerequisites for success in the workplace of tomorrow:

Image from: https://www.weforum.org/agenda/2016/01/the-10-skills-you-need-to-thrive-in-the-fourth-industrial-revolution

Yet today’s educational framework, especially one void of technology, seems to assume that these skills are either not important or will be mastered at some later date. When we consider expert projections like the one above, coupled with technology’s saturation in all areas of our life, then the lack of access and availability of technology at school will seem out of place. Technology should be as ubiquitous in education as it has become in the rest of our lives.

To better understand this approach to technology’s role in learning, let’s discuss the learning experience of a 1st grader in science. Students can fill out worksheets, watch a video, and draw a picture. They can also take all of their research and learning that they encounter including their own videos and drawings, and develop a two minute film to present their understanding. This learning product not only demonstrates their understanding of the topic, but, more importantly, it does so in a medium that is portable, shareable and beneficial to others. We must have an honest conversation when looking at the value in student work beyond knowledge assessment. Can students develop final products that others can learn and benefit from? To achieve that level of learning, students can no longer produce generic fill in the blank learning to indicate understanding of classroom content and its application. They need to actively create something to be shared with others.

This level of ubiquity and integration allows students to view technology as a tool to reach new heights as they discover creative and ingenious approaches in their learning and skill development. To achieve this requires all stakeholders to be involved in the education process including educators, parents, and students. Each stakeholder must look at their role in understanding how technology allows for the access of information, how to synthesize and reconstruct knowledge, and most important of all, who and how they can share their understanding and final product of learning beyond the teacher’s grade book. The success of the above rides on ensuring like any component of a school’s success, that all stakeholders are part of the development, launch, and commitment to sustainable growth.

Stakeholders Need To Know Their Role In Technology Use and Integration

School Leadership and Faculty

Schools must invest in technology in a way that is in line with their mission and vision, rather than having technology define them, or worse contradict them. Do you want technology to promote mobile and active learning around campus? Do you want the walls of your classrooms to disappear? Do you want students to be actively engaged in developing creative solutions to represent their learning through music, drama, and media reach projects? If the answer is yes, then a laptop would not be your solution. The best device to fit your institutional mission would be a tablet with a detachable keyboard. Whether its iPad or Android the point is that these questions should drive the technology, not solely based on cost of purchase and maintenance, or ease of I.T. management.

Parents

Parents need to partner with schools and be supported by them as well. The term lifelong learner must extend to the parent body. Parents must understand that for students and schools to embrace change, parents too must be open to change and growth. This will show not just schools, but our children that the parent body is commitment to learning and discovering new things. How familiar are you are with technology? Do you need training? Do you need a Q&A, FAQ, or link to resources? If technology comes home with students, it is critical that parents are informed and aware of what the technology is capable of doing. Remember, a pencil is technology, but parents don’t have to worry about their pencils connecting to the internet.

Students

I believe that students have the most critical role of in the education process, since it is they who will validate and reveal what methods, tools, and strategies will empower them to achieve their highest potential. It goes beyond how are they are supported and trained, but rather how they are listened to, and sought out for their input and advice on what will help them love learning and desire to work hard. If we want full buy-in, we need to not just hold students accountable to our expectations, but support them in a way that technology use and learning at large is intrinsically motivating. Students in a successful technology infused learning environment should be able to answer the following question regardless of age or ability. What are ways you can demonstrate (show) your understanding of what you are learning? When we reach this level of thoughtfulness that goes beyond “technology use,” we are truly preparing students for the future where technology is not an option, but simply part of learning and life.

Ubiquity, access, and availability of technology tools and resources are vital for students to be prepared for a future that does not yet exist. They need to be introduced to technology as part of their educational growth and development, so that they will see them as tools that will meet their professional success and needs. If we are successful in this, then technology use will be authentic and fulfill its purpose of supporting our students and children in becoming inquisitive, independent, and inspired lifelong learners.

Michael Cohen, The Tech Rabbi is a designer and technologist turned educator. He is an Apple Distinguished Educator and Google Certified Trainer who advocates, speaks, writes, and facilitates workshops on creativity and innovation. He works with schools on a local and national level to help them develop a creative mindset to influence and and empower individuals in becoming confident and passionate problem solvers. Michael shares his story of creativity and innovation through social media, and conferences such as Apple Education Events, EdTechTeacher Summits, iPadpalooza, ISTE, and SXSWEdu. He is the host of the “Educated By Design” Audio Experience Postcast where he features experts who share how they ideate, plan, and implement creative solutions to solving complex problems.

Michael currently lives in Los Angeles with his wife and four children, and is still an avid skateboarder.

This is part 10 of the series in eJewishPhilanthropy, Continuing Conversations on Leveraging Educational Technology to Advance Jewish Learning. The series is a project of Jewish Funders Network, the Jim Joseph Foundation, and the William Davidson Foundation. For an in-depth look at opportunities in Jewish Ed Tech and digital engagement, read Smart Money: Recommendations for an Educational Technology and Digital Engagement Investment Strategy. Later this year, Jewish Funders Network will launch a new website to help advance the field of Jewish educational technology.

Developing the Jewish EdTech ecosystem requires an agile investment approach. The optimal framework for this is an application of the Lean start–up methodology, oftenreferred to in the social sector as the Lean Impact Methodology.

Our previous article discussed the importance of an ecosystem; let’s take a closer look at the strategies and tools needed for creating a specifically Jewish EdTech ecosystem. It begins with asking the right questions…

In this post, we’ll look at the benefits of an agile investment approach to the Jewish EdTech space. We’ll also consider the limitations of traditional accelerator programs for niche sectors, and investigate how to tweak the approach for Jewish EdTech in particular. The post will conclude with a working hypothesis expressing the author’s vision for a reinvigorated Jewish education environment.

Applying intuition scientifically

Developing the Jewish EdTech ecosystem requires an agile investment approach. The optimal framework for this is an application of the Lean start-up methodology, often referred to in the social sector as the Lean Impact Methodology. The approach is intuitive and rigorous. Just as a scientist sets hypotheses and tests them experimentally, the Lean Impact innovator will set a hypothesis with three potential outcomes: validated (yes), invalidated (no) or unclear (maybe) results. If tests show that the hypothesis is supported by evidence, the innovator has reason to scale up the operation. If tests show the hypothesis to be false, the innovator should revise the hypothesis. Unclear results call for revised tests. This way, testing enables the innovator to make step-by-step evidence-based decisions.

Interactive Whiteboards by PolyVision

The Lean Impact methodology is not a magic formula for finding optimum solutions. The answers will only be as good as the questions, a topic addressed in an earlier article in this series. The learning process proceeds by asking questions (“hypothesis generation”), effectively testing the hypothesis, and then extracting meaningful lessons from the conclusions. Successfully following this approach requires agility, flexibility, and humility.

How does this fit into the broader impact objective

The lean impact model described above is the best way to accelerate innovation in the sector. Once effective solutions are identified the next step is to catalyze these innovations by supporting scale-up efforts. And finally, those initiatives which prove impactful and scalable require support to achieve sustainability. Currently, the bottleneck is innovation, and it is for this reason that we’re focused on accelerating innovation.

For funders who are uncomfortable with the risks associated with innovation and experimentation there is still room to meaningfully contribute. This can be achieved by supporting knowledge management and best practices about effective solutions. And also by actively participating in mobilization of resources for scaling and sustaining successful innovations.

Finding solutions that work

Traditional accelerator models are well understood, particularly in the technology sector where they have been refined over decades. There are by now clearly defined benchmarks for impact investing in terms of what constitutes an outcome and how to achieve outcomes.

However, the rigorous selection processes that make incubator models so powerful creates a blind spot for niche sectors such as Jewish EdTech. In a robust ecosystem, innovation can be driven successfully by developing more accelerators in conjunction with impact funds. However, such extremely selective winnowing practices depend on an adequate supply of innovators, opportunities and funding to produce successful entrepreneurial ventures. The model is effective in major commercial sectors, but seldom works in smaller sectors, such as education.

Advancement in the impact investment sector has enabled more accurate filters to select investments and to better monitor real impact. However, the model’s stringency can also be a limitation. The model assumes that optimizing the selection process is the path to success, which is only true when there are adequate candidates to select from. In niche markets, optimizing selection will often produce zero opportunities that meet the required criteria. Understanding this “market failure” is fundamental to building solutions for the “real world” rather than “ideal world”.

Experience has taught us that for-profit companies often have the skills but not the incentives to address social problems and not-for-profits have the desire to address social problems but lack a business model for operational effectiveness. This realization led us to develop a hybrid methodology, which acknowledges both the limits of the market and the social imperative of addressing those problems.

Impact–ful Investing

By shifting the emphasis from the traditional testing standards – whether or not you have a viable business model – to an emphasis on can you get the job done, we are able to target the most appropriate revenue models that support the best solutions, regardless of whether they are not-for-profit or for-profit enterprises. Many problems can only be served through social funding, since a viable market will never exist; but the methodology at least ensures these solutions will be optimal in terms of impact.

The methodology seeks to answer different questions as potential solutions emerge:

• Which innovations work
• How can effective solutions be scaled-up
• What is the optimal funding structure to maximize impact potential of effective scalable solutions

We’ve looked at how to construct an approach that asks the right questions in order to find effective solutions. What would this look like in practice?

We can use the methodology to create an ecosystem heat map that maps out what our ecosystem contains and what is missing, providing a broad overview that enables data-driven decision making at an ecosystem level.

Then we develop a heat map of the ecosystem maturity, with green, orange and red indicating decreasing levels of maturity in that order. The example below has zoomed in on the Day Schools segment:

Green = most mature | Orange = medium level of maturity | Red = least mature

Data driven insights will emerge from the heat map, such as:

• If we create content for school-age children will we bottleneck in implementation?
• How can we develop the “implementation” vertical in Grades 3-9 where we have better content already available?
• How would our funding strategy need to change to achieve this?

Having identified “Implementation” as a constraint in our example, we can further drill down into focus areas to identify opportunities to promote effective change:

We’ve developed a theoretical model to illustrate how the process would be applied to the Jewish EdTech Ecosystem. Of course, the following analysis is not an actual assessment of the ecosystem, but simply serves to demonstrate the methodology in practice.

Building knowledge for the ecosystem

We start with an overview of the ecosystem:

Putting a program in action

We can extract valuable action points from the report, Smart Money: Recommendations for an Educational Technology and Digital Engagement Investment Strategy

The report recommends eight “Best Bet Investing Strategies”:

1. Invest in collaboration among the best of existing Jewish EdTech producers
2. Partner with companies and organizations already active in the EdTech space
3. Invest limited funds in pooled EdTech investment funds
4. Track and commission EdTech products that emerge from the general EdTech space for the Jewish audience
5. Invest significant grants in developing new content using “Big/Proven Talent” and general EdTech companies
6. Invest with smaller innovation grants and in field building
7. Invest in crossover opportunities
8. Infrastructure

The Jewish EdTech ecosystem should follow a lean start-up approach to building the ecosystem. This entails an agile, relatively quick cycle of (1) testing a hypothesis, (2) learning from early results and (3) implementing solutions based on these findings. The hypotheses would be aligned to the learning agenda as mandated by the funding group.

As a first step in generating an ecosystem heat map, in the coming months we’ll be launching a centralized product and service index to inform and empower this methodology.

Our recommended investment approach includes prioritizing the following four recommendations:

• Partner with companies and organizations already active in the EdTech space Invest in collaboration among the best of existing Jewish EdTech producers
• Commission EdTech products that emerge from the general EdTech space for the Jewish audience
• Invest limited funds in pooled EdTech investment funds
• Invest with smaller innovation grants and in field building

Jarred Myers manages an Innovation Portfolio for a Private Family Foundation, using venture philanthropy and mission investing tools, he focusses on technology driven solutions for education and employment. Nicky Newfield is the Founder and Executive Director of Jewish Interactive and is a trustee of the Glatt Charitable Foundation.

In 2009, Jewish Teen Initiative – Boston (JTI), then known as the North Shore Teen Initiative (NSTI), launched in the 23 cities and towns just north of Boston as an innovative, first-of-its-kind program aimed at addressing the alarming trend of teens disconnecting from their Jewish faith and community after Bar/Bat Mitzvah. Now, a little more than eight years later, JTI has become a national model for Jewish teen engagement, with lessons learned being adapted in communities around the country. Created and launched in partnership, and with 100 percent grant funding from the Jim Joseph Foundation, JTI is now independent and building a path toward sustainability – with bumps, bruises and ultimately valuable lessons learned along the way.

Utilizing a combination of community organizing and design thinking, JTI has built a community framework that lowers the barriers for Jewish teens to stay engaged, or re-engage, with their tradition. By collaborating with synagogues, day schools, JCCs and other community agencies, JTI has created an ever-expanding menu of teen-centered, local, regional and national programs grounded in the many experiences that comprise Jewish life. It introduces teens to Jewish learning and leadership experiences that promote life-long commitment to Jewish values.

Here are a few highlights from JTI’s first eight years:

• Built relationships across Jewish agencies in 23 cities and towns on Boston’s North Shore while supporting existing programs, maximizing connections and increasing/diversifying program offerings for teens
• Engaged 900+ Jewish teens in meaningful Jewish growth and learning experiences. Many of these teens would not otherwise have been involved in Jewish life
• Partnered with 50+ organizations, strengthening connections between local Jewish agencies, synagogues, youth groups, day schools, JCCs etc.
• Offered 200+ program opportunities either in conjunction with community partners or alone – each customized to local needs

Today, JTI is expanding its reach at the request of Boston’s Combined Jewish Philanthropies and launching a new sub-region in the city’s western suburbs. Most recently, JTI has pioneered a partnership with Hillel International, to adapt its highly effective campus peer engagement program to be used with high school students. This fall will see close to 40 peer leaders connecting with nearly 1,000 Jewish teens throughout these two regions.

Challenges Moving Forward

JTI’s main test today is maintaining financial sustainability. For eight years, the Jim Joseph Foundation provided generous support, which diminished over time, including matching grants in the later years. This support helped lead JTI to a place where it can sustain and expand its work.

However, the transition away from the Foundation funding has been difficult.

“Becoming comfortably sustainable is the ultimate challenge facing this remarkable teen initiative. Recognition and appreciation of these teen experiences by parents, grandparents, community members, and others needs to lead to continued support at every level, especially if we are to continue to connect our teens to the meaning, importance and relevance of their Jewish heritage.”
Jerry Somers, JTI Founder and Board Member, and former Board member of the Jim Joseph Foundation

While we at JTI are in the midst of building our path towards sustainability, we can take an honest look back at two particularly valuable lessons learned, which hopefully can inform others who embark on similar efforts:

• Get Early Community Buy–In. While the Foundation’s seed money made JTI possible, it is now clear that launching with 100 percent funding negatively impacted community buy-in, making fundraising more difficult today. Many potential donors did not want to play second fiddle to the Foundation; some people want to have skin in the game right from the beginning. In hindsight, JTI would have benefited by bringing donors to the table from the outset.
• Invest in Fundraising. In retrospect, it would have been beneficial for JTI to use some of its early funding on philanthropy training. As the executive director since JTI’s inception, I was hired for my strengths in making connections and creating programming, along with a knowledge of Judaism. The organization would have greatly benefited if I had worked with a fundraising coach early on to build an expertise in this important area. However, with full funding, JTI had no urgent need to start professional fundraising. We were solely focused on establishing a model and path toward success. It wasn’t until year four that the JTI team started to think about fundraising.

In part because of these early “mistakes,” there have been some important developments more recently: More than half of my time now is spent on fundraising; The Foundation has connected JTI with large local funders, including Combined Jewish Philanthropies; and the Foundation also has also helped JTI pursue individual donors who have been positively impacted by the program, such as parents and grandparents of teens.

To date, we have raised 80% of our annual campaign goal and early indicators are that we are tracking to a place to be sustainable locally without the Foundation’s involvement. The ongoing discussion and challenge will focus on our ability to have a larger community impact without a national partner.

A Partnership that Led to Success

Today, as we fundraise, JTI continues to thrive and engage more Jewish teens. The Foundation played a large role in the success of the model, guiding our evolution and growth each year.

From day one, Foundation leaders provided direct input and involvement with JTI professionals and Board chairs. They helped with staffing models, evaluation processes, training, coaching, and brainstorming. They opened doors not just to funders, but to program partners and resources.

For our part, JTI has remained committed to innovation. In the eight years that we’ve existed, we’ve never stepped back. No two years have really been the same. While our overarching goal did not change, we were never constrained to maintain a specific approach if we could see it wasn’t working. We had a commitment to flexibility.

All of us with JTI have learned the importance of being responsive to our community – to always ask questions to learn what people want; to pilot, test, and have a risk-taking mentality. Over eight years, we have built deep and meaningful relationships with teens, families, and congregations in our community. While the road ahead is not without challenges, we are confident that JTI will continue to help support and create vibrant Jewish life for many.

Adam Smith is Executive Director of Jewish Teen Initiative – Boston.
Originally appeared in eJewishPhilanthropy

This is part 8 of the series in eJewishPhilanthropy, Continuing Conversations on Leveraging Educational Technology to Advance Jewish Learning. The series is a project of Jewish Funders Network, the Jim Joseph Foundation, and the William Davidson Foundation. For an in-depth look at opportunities in Jewish Ed Tech and digital engagement, read Smart Money: Recommendations for an Educational Technology and Digital Engagement Investment Strategy. Later this year, Jewish Funders Network will launch a new website to help advance the field of Jewish educational technology.

Since this is media, how can you know that you’re really having an impact? What proof do you have that video can build Jewish identity or literacy?‘

BimBam (formerly G-dcast) is a Jewish media studio. Our creative team has worked on over 300 short videos and apps, and we have big league experience from Apple, Pixar, The New York Times, etc.

Usually, people find us through our work – they don’t ask us too many creative questions beyond, “Can I do a part in a video?” That’s because they or their kids already love the programs, and it’s easy to see our track record. Our Judaism 101 and early childhood education videos have clear and easily shared metrics – high viewership numbers, great audience retention curves and accurate aim at the demographics we’re targeting.

What we do get asked routinely is, “Since this is media, how can you know that you’re really having an impact? What proof do you have that video can build Jewish identity or literacy?”

Great questions. What I want to offer is a perspective that we’ve found to be true: well-designed Jewish media programs that are informed by best practices from secular educational media are as effective as their peers. PBS Kids shows (e.g., Daniel Tiger’s Neighborhood, Peg + Cat, Sid the Science Guy), Sesame Workshop programs, and Reading Rainbow have been studied for decades by leading researchers using sample sets comprised of thousands of children and their parents. Interactive educational programs in the app marketplaces are, to some extent, being put under the lens while in development or afterwards.

Funders are advised to read this research – which they will find fascinating not only as evidence supporting certain types of programming but also as former children themselves. Research-based shows are studied for proof of literacy building, numeracy support and social emotional development. They are tested in both formal controlled settings and informally, in homes.

There are protocols, well-established, to choosing groups of respondents to study and there are protocols for discovering not only learning changes but also affect and engagement. For instance, guided by Stanford PhD researchers who work routinely on PBS properties, we tested early Shaboom scripts on eight 3, 4, and 5 year old children – two boys and two girls of each age – in their homes, beside their parents. It did not matter that they all lived in the Bay Area, something that we’d thought would be a factor. We got all of the information we needed to learn from these 12 children in order to improve our pilot episode, through asking establishing questions, observing measures of engagement during viewing (such as toe tapping, pointing at the screen and looking at a parent), and answering questions post-viewing. This formative evaluation allowed us to improve our pilot script – for instance, we added more Hebrew vocabulary words, increased the use of musical chants and changed the name of the show based on feedback we observed and heard from children in this phase.

After producing a finished animated pilot, we did another round of testing, conducted entirely blind to us by Ph.D. researchers who watched the program with dyads of parents and children, and then conducted play testing afterwards to measure what social skills children had developed through viewing the shows. (Children performed little actions with dolls to demonstrate welcoming guests, for instance.) This summative evaluation showed that in fact children were learning social skills and assimilating Hebrew vocabulary for those actions.

We were fortunate to receive a large grant from the Peleh Fund which made it possible for us to bake this crucial, but costly, round of evaluation into our development of Shaboom. But it is unrealistic to expect that any small Jewish nonprofit will produce comparable research on their evaluation budgets, or that funders commissioning Jewish work from secular shops will know how to conduct such research. We learned a lot from our experience on Shaboom, enough that we can – to a great extent – cobble together our own in house studies with our own staff. However, this comes at great cost to staff productivity in a small (5-person) organization – capacity that could be improved either through a major evaluation grant or a full time staff person focused on learning measurement.

Given substantial communal buy-in, I recommend going bigger: establishing a center for Jewish media research staffed with Ph.D. media and learning science researchers who are trained to do this sort of work and dedicated to it, full time. This would show serious intent to invest in effective, high quality media work by the funder community, and would up the game of all of us media producers substantially by having partners for evaluation.

Such an investment might be a pipe dream. No matter: I believe that we have the research that we need from existing educational media companies. The community of funders should hold Jewish media producers accountable to these studies – and show their fluency in them – rather than asking for original research. Playtesting, and viewer testing, are serious endeavors and cannot be thrown together in the “free time” of scrappy small studios or independent artists.

When you work with a well-versed studio or artist, you will find that they are familiar with this research and can tell you, without hesitation, that it builds Jewish identity and interest in practices – from baking challah to singing brachot to trying out bikur cholim. At BimBam we have evidence that our programs and apps have produced these effects – but we also went into the work confident, because we followed research best practices.

So, familiarize yourselves with the research. Browse the archives of the Joan Ganz Cooney Center, of Commonsense Research, the Fred Rogers Center, the Pew Research Center, the American Psychological Association – read up on the work at the university centers doing work in this area (Annenberg at UPenn, the Children’s Digital Media Center at UCLA, Center on Media and Human Development at Northwestern, Learning Sciences and Technology Design at Stanford, Technology, Innovation, and Education Program at Harvard, etc.), and consider picking up some journals that publish new studies on media education research. If you’re really interested in this space, you’ll find them fascinating – and helpful as you partner with Jewish media producers.

Sarah Lefton is the founder and Director of BimBam, formerly G-dcast. She began working in interactive media in the last millenium, at NYU’s Interactive Telecommunications Program (ITP). She went on to produce projects for The New York Times on the Web, the Village Voice, Princess Cruises and several children’s toy brands. Her social media “Save Jericho” project for NutsOnline.com garnered national news attention.

Taking an abrupt turn for the less corporate, Sarah joined Northern California’s independent Jewish summer camp, Camp Tawonga, as their Marketing Director for four years, learning about Jewish outreach and wilderness. Inspired by its Yosemite location, she designed the infamous YO SEMITE tee shirt, and launched her first Jewish entrepreneurial project: Jewish Fashion Conspiracy. Sarah is a past President of San Francisco’s pluralist Mission Minyan, and board member of the San Francisco JCC.

Sarah is a recipient of the 2012 Pomegranate Prize for exceptional young Jewish educators. She was named one of the Forward 50 most influential Jews of 2009, and is a recipient of the Joshua Venture Group fellowship for Jewish social entrepreneurs.

This is part 6 of the series in eJewishPhilanthropy, Continuing Conversations on Leveraging Educational Technology to Advance Jewish Learning. The series is a project of Jewish Funders Network, the Jim Joseph Foundation, and the William Davidson Foundation. For an in-depth look at opportunities in Jewish Ed Tech and digital engagement, read Smart Money: Recommendations for an Educational Technology and Digital Engagement Investment Strategy. Later this year, Jewish Funders Network will launch a new website to help advance the field of Jewish educational technology.

Educational Technology (EdTech) is a burgeoning field that has made significant progress in recent years. Our Jewish Education systems are slowly and steadily adapting to this and we are witnessing the emergence of a nascent Jewish EdTech ecosystem. The Jewish EdTech ecosystem can leverage much of what already exists in the broader sector; however, there are niche requirements specific to Jewish learning that are required to successfully harness the full potential EdTech has to offer. This requires strategic investment – and given the limited funds available, we, as funders need to ask ourselves – how best can we utilize our resources?

While there are many useful frameworks for building an ecosystem, we have employed a framework consisting of the four components detailed below:

There are many different contexts in which learning happens. From parent-child learning at home, day schools, supplementary schools and summer camps, to college, post-college and lifelong learning – that is why a one-size-fits-all approach is not relevant. The challenge in building an ecosystem is making sure the individual parts function as part of an educational continuum. We believe that a learning agenda is required to coordinate this ecosystem development.

We need to ask questions that, when answered, enable us to work more effectively. We need to embed collaboration, coordination, learning, and adaptation sector-wide to maximize results through evidence-based decision-making. Through our learning agenda we can articulate our hypotheses, prioritize the questions we seek to answer and develop a funding pipeline accordingly.

Below are some of the questions we’ve raised. We’d really appreciate feedback about the questions your organizations are asking, and how you’ve addressed these issues.

Content

What questions do you and your teams ask when considering ‘content’?

We’ve been asking a lot of questions about content coverage such as: Is there a single content repository? Is there a content landscape map? Do we know what content has been developed? What content is sorely needed? What content has been duplicated and over funded, and what areas are neglected and underfunded?

We’ve been asking questions about content quality: What is good content? Is there a content quality standard? How important is quality control? How do we develop high quality products within existing budgets? Some questions about the types of content: How do we create a self-generating content ecosystem? How do we centralize content? What content is relevant and engaging and for which target groups?

We’ve also asked broader questions like: How can content be created that adapts easily to emerging technologies? How do you keep content current? Which technologies should be employed for what content? How do we leverage current tools like virtual reality, augmented reality, etc.?

Access

Content alone, no matter how good, is of no use if it languishes inaccessible to its intended audience. Which is why we’re asking questions about how we make all sorts of content accessible like: How do users (i.e. every Jewish child, family or person) access the content? Do users access content on their mobile devices or PCs, in school, at home or somewhere else?

Questions for those of us developing content: How do we build now for the next generation of technologies? How do we prioritize which operating systems to focus on? How do we leverage the hardware devices manufactured by the likes of Oculus, Samsung, Sony, HTC and Google? Should we invest in data standards? What’s our approach to data interoperability?

Distribution

The Jewish market is relatively small which makes optimizing the efficiency of distribution channels a priority. How do we reach users? Which distribution channels work and for whom? How do we as a sector create a clear and easy distribution channel to all educators and schools and Jewish homes? How do we get content directly to families? Which existing channel participants, like PJ library or Birthright, could provide leverage for distribution? How can we reach the unaffiliated? How do we use social media channels to support distribution?

Implementation

Implementation is key. As part of effective implementation, we are asking questions about quality assurance, channel partnerships, stakeholder engagement, teacher training, support, maintenance and program management, learning management systems, hardware requirements etc. What local, regional and national partnerships are needed to for successful implementation?

Quality assurance is a knotty topic, we are asking questions like: Does it work? Which measurements actually matter? How do you keep quality of content high? How do you measure impact of learning qualitatively and quantitatively? How do you implement monitoring and evaluation? How do you ensure continued research and development? What are the proxies for real impact? How do we measure that which can’t be measured? How do we feed best practice back into the ecosystem? What actionable insights could be achieved with great data?

Theory of change

Questions are important, and as strategic funders we need to embrace curiosity as part of a larger theory of change. Our learning agenda needs to be embedded into our theory of change to ensure that every question we seek to answer is addressing our intended impact.

Below are several examples of how we are ingesting our ‘questions’ and turning them into a tangible hypothesis:

Let’s explore:

How do we ensure coverage of high quality engaging Jewish content?

• Hypothesis: IF WE create an ecosystem that can centralize and evaluate all available content THEN the sector can prioritize content creation based on need and impact.
• THEREFORE, WE WILL fund several initiatives aligned with supporting a content mapping and evaluation ecosystem.
• SUCH AS (1) creating and maintaining a product index and heat map (2) commissioning established service providers to produce high quality, professional content. (3) funding a crowdsourcing initiative to generate content.

How do we build now for the next generation of technologies?

• Hypothesis: IF WE regularly adjust educational efforts to accommodate technological advances THEN the sector will have ongoing access to quality content via optimal technologies.
• THEREFORE, WE WILL fund initiatives which focus on continuously experimenting with emerging technologies and generating pathways and support for channel partners seeking to adopt these new technologies.

How do you maximize reach?

• Hypothesis: IF WE create a mass digital distribution channel to every Jewish family
• THEN we will enable continued access and usage of upstream content.
• THEREFORE, WE WILL fund initiatives (1) that will map and develop the optimal distribution channels required at different life-cycle stages. (2) partner regionally with distribution partners with maximum reach. (3) create a centralized information repository.
• SUCH AS: (1) fund the mapping of regional distribution channels. (2) engage with existing regional, national and international partners such as PJ Library and Birthright on leveraging their current distribution channels. (3) establish a Jewish EdTech centralized repository, including a website that collects blogs, data reports, recent news, etc. to leverage existing resources and to share best practices with the field.

What actionable insights could be achieved with great data?

• Hypothesis: IF WE continuously collect high quality data
• THEN enhanced products and services can be provided to the sector.
• THEREFORE, WE WILL fund several initiatives.
• SUCH AS: (1) creating data interoperability standards and tools. (2) engage data scientists to build a suite of action-oriented analytics tools. (3) create a new, or leverage an existing forum for sharing data analytics insights. (4) fund a forum or convening for sharing best practices

Practical call to action: As individual funders or foundations, when you make an EdTech grant or Program Related Investment (PRI), we encourage you to use this framework for two purposes, firstly, to assess how your funding contributes to your learning agenda, and potentially to the ecosystem learning agenda. And secondly, as a diligence tool to strengthen the investee or grantee proposal by asking questions like: What are you hoping to achieve with this new content? How will users access and engage with the content? What’s your distribution plan? How robust is your implementation plan? And our favorite question of all; How will we know if we should double down or shut down, i.e. increase our funding and crowd in other funders or exit?

Strategic call to action: In order to foster an ecosystem that will continue to embrace technological advances in the service of Jewish education, we need to collect the depth and breadth of knowledge amongst funders, educators, parents and students. Let’s create the conversation space and unpack our efforts and ambitions. Join us to maximize the deployment of resources – to chart a pragmatic way forward, and to proactively build the Jewish educational opportunities that we envisage for our generation.

Jarred Myers manages an Innovation Portfolio for a Private Family Foundation, using venture philanthropy and mission investing tools, he focusses on technology driven solutions for education and employment. Nicky Newfield is the Founder and Executive Director of Jewish Interactive and is a trustee of the Glatt Charitable Foundation.