5 alternative ways to measure the impact of EdTech innovation?

Many education systems around the world (Ireland included) continue to be enthralled to high stakes exams as a means of assessing students.  We still tend to value what we measure, rather than measuring what we really value.  As a result, any new innovation that is proposed tends to be measured against the yardstick “will it improve the (Leaving Certificate*) grades of my students?”

*replace as appropriate with the name of the final exam in different countries

We hear the argument all the time when discussing the merits of using technology to improve teaching and learning.  But what if it weren’t like that – what if we could focus on objectively assessing value in other ways?  What if the discussion about the value that innovative technology brings to the process of education could be framed in a different way?

That was the question we posed to Jim Wynn, CEO of Imagine Education as part of our Friday lunchtime series – and here are 5 alternative ways that we could use to measure the value that innovative use of technology brings to education:

1. Adoption patterns

One of the big problems with technology innovation is that it doesn’t stand still – and adoption rates vary between teachers and learners.  It’s also notoriously difficult to predict when and how adoption will happen – the Gartner Hype Cycle is a great illustration of that phenomenon, particularly when applied to education   The potential of technology is also often misinterpreted – for example, in the Lewiston Daily Sun on Wednesday 16h May1923, Edison predicted that ‘Movies will replace text books’ (!).

Nevertheless, we need to understand more about technology adoption rates in education – and also how they compare with adoption rates in other aspects of the lives of teachers and learners.

2. Economy

We know that the challenges of the ‘Learning Society’ are different to those of the Agicultural and Industrial Societies before it.  There is plenty of evidence to suggest that there is a skills mismatch in the workforce readiness of new entrants to the labour market.  The demand for higher-order problem solvers is increasing – as are rates of youth unemployment.

Average Learning Intensity of Daily Life


An OECD study from 2010 – The High Cost of Low Educational Performance – indicated that even a modest improvement of 25 points in PISA scores for all OECD countries in 20 years would deliver an aggregate GDP gain of USD115 trillion over the lifetime of the generation born in 2010.  Where technology is being used to promote higher order skills, we need to be measuring that impact.  If we know that using technology to support teaching and learning better prepares students for work, we should be able to demonstrate that.

3. Pedagogy

Of course, the use of technology on its own does not necessarily improve teaching and learning.  Yes, it can make the process more engaging – see the diagram below which tracks the brain activity of a learner over a 7 day period.  Note the activity levels when the learner is asleep – compared to in class!

Switching off in class

But to be really effective, the technology should be allied with the introduction of new and innovative pedagogies.  The work of Dr Paul Howard-Jones from the University of Bristol has already demonstrated the connection between neuroscience, games and learning – and the effect that redesigning the learning experience can have on dopamine production in the brain (See https://www.youtube.com/watch?v=aKFjoF-YO20).

Integrating technology effectively requires an evolution in teaching and learning practices – by implication; it requires new skillsets for teachers and learners.  Effectively implementing and measuring the impact of those changes is where we should be focusing our efforts.

4. Productivity

Related to effective pedagogies is also the notion of productivity.  A decision to incorporate technology into a lesson plan ought to be based on whether or not ‘the lesson’ can be more effectively delivered.  Can the impact of that step change in productivity be measured?

5. Assessment

Finally, we come full circle back to the issue of assessment.  If assessment is simply a validation that learning has taken place – and if learning is not a linear process – then surely we ought to be more comfortable with a model that is more flexible.  Learning analytics hold the key – gathering data points about individuals as they progress through their learning journey – and devising appropriate interventions to support the personalization of that journey as they go.  If technology can help us to derive a more holistic view of a learner’s experiences, then how do we capture that as a measurement of the value that the technology is bringing to education?

Learnovate will be working with Jim in the future to shape research in this space – to move the debate forwards and produce a framework that enables us to more effectively measure the real impact of EdTech innovation.

Martyn Farrows


CoSN2015: 5 EdTech takeaways for Ireland?

Last week, we were presenting at CoSN2015, the largest EdTech conference in the US targeted at practitioners in School Districts.

We were there to shine a light on the EdTech innovation that is happening in Ireland: showcasing innovative EdTech companies; the collaborative research that is happening in Learnovate (more on that in a separate post); and, the real impact being generated by the ground-up movement that is Excited.ie – the Digital Learning Movement.

We were also there to learn: to find out what constitutes best practice across the US in terms of approaches to using technology to support teaching and learning.  The conference included over 100 sessions with practitioners talking about how they had implemented technology in their school districts.  Here are five key takeaways … some are new, others maybe just confirming what we already know!

1. 100Mbps is not  enough

Future-proofing for connectivity requirements in schools was a common theme.  Representatives from Chesterfield County talked about the challenges of rolling out 33,000 Chromebooks to students (see https://sites.google.com/a/ccpsnet.net/anytime-anywhere-learning/).  They expressed their concern at having to ‘starve some schools with 100Mbps’.  Their target is 5+Gbps for all schools in 2015/16.

2. If you’re still discussing the merits of 1:1 devices, you’re already out of the race

A number of sessions focused on the rollout of thousands of devices to students.  What really caught the eye, though was Houston School Districts ‘PowerUp’ programme – a district wide initiative aimed at transforming teaching and learning across 282 schools.  Their strategic, multiannual initiative is a blueprint for how a well-planned, comprehensive initiative that involves all stakeholders can be delivered successfully.  And will improve outcomes and prospects across the board for graduating students.


Houston School District: Becoming #GreatAllOver

3. Personalisation is the norm

Across the board, we saw examples of where personalization (in teaching and learning) has become the norm when implementing EdTech strategies.  From platforms to content, CPD to devices.  And what better content than that being delivered by the Smithsonian’s Centre for Learning and Digital Access – who really understand the power of devolved, personal curation allied to great quality content and resources.

4. The challenges of Big Data should be embraced

It’s far too easy to slap a big ‘Data Privacy’ badge on attempts to use data analytics to improve education services.  However, by developing a culture of trust between students, parents, teachers and administrators, it is possible to unlock the potential of applying big data techniques in an education context.  More transparency on data protection and data privacy issues can go a long way in establishing that culture.

5. “Being right is not a strategy for change – you need a process”

Michael Fullan used this quote during his keynote on introducing new pedagogies.  However, it’s equally applicable to many challenges of introducing EdTech in schools – not least when applied to the need for direction at a national level.  A coherent national strategy can be a catalyst for change if it includes core components (connectivity, devices, teacher CPD, digital content, R&D), backed by an clear execution plan.  Obama’s ‘Connect-Ed’ Initiative is a good example of how a national strategy can achieve real change in education.  An incomplete strategy (or worse, no strategy!) is not a strategy for change – and will not contribute to long-term and sustainable economic competitiveness.


Martyn Farrows

Content marketing for ed tech: 5 ways to build an audience for your brand

Peter Gillis

We were very pleased to welcome Stephen Walsh back to give his second seminar at the Centre. Stephen has been responsible for the successful marketing of his own company, Kineo, ultimately leading to Kineo’s acquisition by City & Guilds in 2012. Stephen has now set off to develop the talents he honed at Kineo and is currently CEO of Anders Pink, a new content marketing agency.
His talk on how to leverage social media in the edtech space attracted a lot of interest with over 20 members in attendance.

Stephen’s inspiration for developing his social media strategy was a book called “Love is the killer app” by Tim Sanders.

More traditional forms of e-Marketing, like banner ads, and e-mails are referred to as interruption marketing where the focus is to interrupt the potential client in the course of their online activities and seek to distract them to a marketing message. The success rate of these activities is very poor to the point of being questionable in value terms. The more valuable approach is to build relationships through sharing useful information with potential customers through social media, this is not easy and requires dedication and investment in time, because the quality of the information shared is paramount. Stephen also points out that it will not happen over night, so needs dedication in the long term. But when successful potential clients see the supplier in a much more positive light and may well lead the client to come to them. Stephen also shared statistics showing the trend shift driving traffic to sites, the quantity driven by search is in decline while the reverse is true for traffic through social media, and as the graphic below shows, the trends crossed in the middle of 2014.


Stephen had promised to give attendees 5 tips to create compelling content for social media, they were:

  1. Get your story straight. Your story should reflect the passion and personality of your particular brand. The story is just that, a story, so it should be written for a particular audience and deliver to a plan, with start middle and end. Then it is important to share the story, monitor feedback, adjust and go again.
  2. The killer story is just the start, next you need a content plan. Your story needs to be found, therefore it is important to deliver the story in a format, through channels and amplified by champions of your target audience. The plan must be data driven, through an analysis of what is working where.
  3. Get the format right, your content can be in a myriad of formats; animations, quizzes, blog article, top tips etc. etc. Different messages work better in different formats. Stephen gave us many examples including Infographics are a useful tool to raise awareness, while Demos and Webinars may be a lot more useful if the objective is to get a decision.
  4. Spend more on promotion than creation. “If you create tweet and forget, so will your audience” says Stephen. You need a strategy of content promotion, one strategy is to identify influencers in your domain, reach out to them and build relationships with them, this makes your job of amplification a lot easier.
  5. Track Competitors, as with traditional forms of promotion it is important to track intelligence on competitor activity. In social Media that activity would equate to what are they publishing? Who mentioned or linked to them? How many shares are they getting etc.

The end of the session involved a lively interaction with Stephen’s other enterprise, Buzzsumo an online tool that helps organisation do a lot of the data tracking, and intelligence gathering mentioned in this article. All members in attendance were given three months free access to Buzzsumo

Stephen made a final point, an important one, keeping an eye on your competition is valuable, but if you follow or copy you will always be a step behind. Stephen’s company was called Kineo which means ‘stir it up’ Content Marketers should be looking to stir it up.
Members can access Stephen’s presentation by logging in to the Members’ Area

How innovative EdTech in Indian schools is driving changes in learning and teaching

Last week, we were in Mumbai working with Tata ClassEdge to explore edtech research and innovation projects in Indian schools. Tata ClassEdge is a recent spin-out from Tata Interactive Services – itself a part of the 100-company Tata Group (revenues 2013-14 $100bn+).

The purpose of the visit was to explore some key research questions in relation to the use of technology in Indian schools:

  • How to measure the effectiveness of edtech?
  • What impact is edtech having on teaching practice?
  • What role will adaptive/personalization technologies play in the future?

To provide some context, Tata ClassEdge have been delivering their teaching platform for Indian schools since 2011. They currently have 1250 schools using the platform, engaging directly with 40,000 teachers and half a million students. That’s in a country with roughly 1.5 million schools, 6 million teachers and 300 million students (Source: Wikipedia http://en.wikipedia.org/wiki/Education_in_India exact numbers are hard to find!).


The ClassEdge platform is currently focused on the needs of teachers, providing a whole series of lesson plans reinforced with media and simulation tools. The platform is delivered in the classroom via a projector – it’s designed to support the teacher by improving curriculum delivery in a traditional classroom setting.

It’s also designed to work alongside more traditional text books. To put that into context, there are roughly 350 publishers and 7600 text books in use across India. Tata ClassEdge have mapped them all to their product.

When they work with a school, Tata ClassEdge provide a whole school solution: the platform and content; technical infrastructure and hardware; competency training for teachers; and, a dedicated support person located in the school for the first year to drive adoption.

This is a practical solution: very few children have access to 1:1 learning devices and class sizes are big. The classroom dynamic is based on a traditional model, with the teacher at the front and the children in rows.

So how does it work in practice? We tool a trip out to Poorna Prajna Education Centre to see the platform in use.


The first thing to note is that this is a big school, with 3000 students across age groups 3-14 (Primary and Secondary students are catered for). The school was founded in 1962 and the building is designed for education – almost all the classrooms are based on an octagonal design to maximize teacher visibility of the class from the front. It’s a private, fee-paying school – however, the school serves the local community and offers free spaces to children from under-privileged backgrounds.

The school is proud of its innovative history with new technologies – they were one of the first in the country to introduce audio-visual technologies into teaching practice in the late 1990s – and the AV suite is still in use today as a teaching and learning resource.

Every classroom has a teaching computer and projector, providing direct access to the ClassEdge platform and content. Internet connectivity is unreliable, so the teachers work offline in the classroom – however, interactions with the platform are logged and provided back for data analytics and usage patterns.


The school staff room includes a planning station – two desktops where teachers can plan and build their lesson plans using the templates provided with the opportunity to add their own resources and personalize the plans. Teachers voluntarily go in to school on Saturday mornings to plan the following week’s lessons.

Classrooms are busy. There can be up to 80 students in a classroom designed for 40 – this is particularly true with the older class groups. There are no student ‘1:1’ devices in the school – however, there is a ‘computer lab’ where students are provided with timetabled access to desktops.

On the day we visited, almost all classes were using the software – teachers were clearly comfortable using it and the students were highly engaged and motivated by the material. Games-based approaches (including a cricket simulation, of course!), interactive resources, videos and multimedia were all being used to great effect. Anecdotally, teachers spoke of the improvements in delivery of the curriculum – and of increased student attainment levels in exams.


Three new students in class for the day

So, what does all this really mean in terms of innovation in teaching and learning?

In a country of over 1.27 billion people, where illiteracy and poverty still affect large numbers of the population, it may seem inconsequential to talk about the impact of edtech on teaching and learning. After all, ClassEdge is currently reaching less than 1% of the student population.

But turn those numbers around: ClassEdge is currently reaching a student population greater than the entire Irish Post-Primary student body. By 2025, India is predicted to overtake China as the most populous nation – and with an average age of just 29 years (compared to 37 for China). If the young hold the key to future economic prosperity, then education will need to play its part.

Changing teaching and learning practices is not easy. How do you introduce changes in teaching practice in a crowded classroom with 80 students? The adoption of the ClassEdge model illustrates an appetite for engagement with technology to drive change. The next step is to evaluate the outcomes and usage of the platform to date – in turn to better understand how the engagement with technology can be better harnessed in the future – supported by appropriate teacher professional development programmes.

In the future, new developments in technologies to support teaching and learning – in particular personalisation and adaptive techniques – can also be employed to improve productivity and learning outcomes. Combining innovative technology with innovation in policy and practice has the potential to deliver transformative and sustainable impact.


We look forward to playing our part in developing a strategic relationship with Trinity College and the Tata Group. It provides the potential for Learnovate and our industry partners to collaborate in the development of innovative approaches to technology enhanced learning in applied settings. Watch this space.

Learnovate supports Community Education innovation – Phase 2

Empowering An Cosán to implement Learnovate’s Virtual Learning Environment (VLE) strategy recommendations

By Mirjam Neelen

In a previous blog, Learnovate explained how the partnership between An Cosán and the Learnovate Centre was conceived. In September 2014, An Cosán was ready for the next step. It was time to implement the VLE strategy recommendations as provided by Learnovate and so the journey continued. There were a number of steps associated with the implementation which included design, development and of course, running the actual course. We focus on the design and development steps here, as the course has not rolled out yet.
Learnovate’s major job was to empower An Cosán by enabling them to, first, design an integrated blended learning solution that supports building online learning communities. Next, An Cosán needed support to ensure that they could develop the blended learning solution in Blackboard Learn (the asynchronous VLE) and Blackboard Collaborate (the synchronous VLE) themselves.
Learnovate’s pedagogical team has developed a robust ‘how to’ learning design template that enables An Cosán to design their own sessions applying a flipped classroom model. Each session starts in the asynchronous VLE. The learner completes learning activities to prep for a synchronous session. These activities then seamlessly flow into the synchronous session, which combines synchronous virtual and face-to face learning activities. The synchronous session focusses on collaborative and consolidative learning activities. After the synchronous session, additional learning activities focus on consolidation again to ensure deeper learning.

A large proportion of this project phase was to support An Cosán in developing the actual sessions in the VLE. Learnovate has created a ’how to’ guide for Blackboard with well-structured step-by-step development tasks. Learnovate has also introduced An Cosán to Elluminate Plan, which helps them to organise their synchronous sessions and allows the facilitators to focus their energy on the learners and the learning activities in the synchronous sessions.

The Challenges and Celebrations
Learnovate took a scaffolded approach; the intensity of the support gradually decreased. First, Learnovate designed and developed one full session for An Cosán as an exemplar. Next, we ran a workshop with An Cosán to design and develop two sessions collaboratively. After the workshop, An Cosán worked independently, designing and developing the remaining sessions. Learnovate’s pedagogical and technology teams remained available to provide guidance and support. We gave feedback on each fully designed and developed session and then we worked in a face-to-face setting with An Cosán to support implementation of the feedback.

Initially, it was challenging for An Cosán to let go of their old design approach and dive into the new learning design and development challenges. Using a flipped classroom model, ensuring an integrated learning experience, providing consistency in instructions; they are all examples of challenges that had to be tackled. However, as soon as An Cosán grasped the major design concepts for a VLE, they started to progress swiftly. It must be said that An Cosán’s excellent subject matter expertise and strong facilitation skills have helped them tremendously. After all, designing is easier when you know your learners well and when you have a clear vision on what you want your learners to achieve!

Another challenge was the technology itself. Blackboard is not intuitive, neither to develop content nor for the learner to navigate. Therefore, it was a tipping point when An Cosán realised that using the ‘how to’ guide for Blackboard that Learnovate had provided was critical. In a VLE it is essential to use a consistent structure and to provide context for the learner by using consistent wording and clear instructions. The ‘how to’ guide for Blackboard helped to accomplish that.

The fact that An Cosán’s learners are often times educationally disadvantaged was another challenge. For example, they need to develop their computer literacy skills and online experience. Hence, a face-to-face component is still critical at this point. An Cosán’s learners need the face-to-face support initially to build confidence and relationships. Moving forward, An Cosán ideally will be able to encourage its learners to move to a VLE only.

Next steps
We are currently partnering with An Cosán so that we can evaluate the upcoming course and compare it to the pilot course. An Cosán will roll out their next course in spring 2015 and Learnovate is proud to be part of it.

“An Cosán aims to use 21st century educational teaching and learning technologies to fast track individuals and communities out of poverty all over Ireland. Learnovate have been key to enabling us develop and implement a realistic strategy to achieve this goal. The learning curve in developing our programmes for a VLE has been steep but we have been supported every step of the way and the resulting outcomes have been impressive. Personally everything I have learned has impacted very positively on my face to face teaching where the ‘flipped classroom’ has become the center of my teaching strategy.”

~ Liz Waters,
Director Virtual Community College