The xAPI story continues and it’s quite an exciting one!

By Mirjam Neelen

Back in 2015, Learnovate completed an initial xAPI project to explore the overall concept and all the things organisations would have to consider when implementing xAPI. We also did a proof of concept in which we implemented xAPI ourselves in a meaningful way for the users involved. This project resulted in:

  • An xAPI Checklist – which helps organisations to determine if they should or could consider xAPI,
  • a ‘Proof of Concept’,
  • a ‘How to’ guide
  • an invitation by LPI to present at the Learning Live conference in London.

Now, half way through 2017, there are still major questions in the minds of learning folks such as ‘What are xAPI’s key values? When to consider xAPI? What is the recommended design process when implementing xAPI?
Learnovate has been very much aware that thought leaders in the xAPI space, such as HT2 Labs, the Connections Forum, and of course the Tin Can xAPI lads and gals have made quite some progress with xAPI implementations and explorations, and we felt it was about time to create an overview of xAPI’s current state of the art.
We dug through academic research (we can’t help ourselves), piled up the case studies, and spoke to Ben Betts from HT2 Labs who was so nice to free up some time to share their experience with xAPI with us (by the way, they have also created some wonderful resources, such as ‘The Learning Technology Manager’s Guide to xAPI’ and – coming soon ‘Investigating Performance: Design and Outcomes with xAPI’).
After some careful analysis, we had to conclude that there are many good reasons to implement xAPI. The main reason being that, when you’re truly interested in offering learners (no matter if they’re employees or students) the best support to improve themselves and they’re using various systems and you need evidence to back your decisions up, then you need learning analytics to put all the pieces of the puzzle together. xAPI enables you to capture the required date from multiple sources, in a structured standardised way, using a single Learning Record Store (LRS).
The State of the Art report that we produced, discusses the ins and outs of the design process and the many things you need to consider when designing for xAPI. At a high level, the recommended design process looks as follows:

MIRJAM3

The complexity lies in identifying what you want to measure and why you need to measure it, dealing with legacy and planning for future systems, data analytics and visualisations, and all the privacy stuff, to just name a few. A fascinating but challenging nut to crack.
We also found some really interesting case studies which together paint quite a clear picture of why organisations go for xAPI and what its benefits are. We have explored seven of them in more depth; two in a K-12 context, two in a higher ed context, and three in a ‘learning for professionals’ context. In the last category, one of Learnovate’s partners, Intuition has kindly shared their experiences with us. They have implemented xAPI successfully and learnt some valuable lessons along the way.
From the case studies that we’ve collected and written up, a couple of trends arose that actually can be captured in one phrase:
Personalising and adapting learning experiences which are taking place in many different ways and places with the goal to increase learning effectiveness using learning analytics to drive evidence-informed decision-making. The example of an Emergency Medical Training using the Internet of Things (beacons) is showcasing this trend:

health

During a medical training simulation beacons on EMTs, firefighters, victims, equipment, and an ambulance record data. The data is sent to an xAPI
Learning Record Store (LRS) in the cloud where it is visualised in real time
The data can be analysed after the simulation to support performance improvement in order to improve patient outcomes in medical emergencies.

Although, like in 2015, we recognised that expanding xAPI beyond a single use case is a tough cookie, we must admit our eyes do sparkle a bit because of the fact that our case studies (and we’re sure there are tons more out there) not only show the potential of xAPI but also the drive in the learning and education arena. To be continued!

In case you’re up for some additional xAPI exploration, please contact us at info@learnovatecentre.org

Why Learning Professionals1 Should Look Past the End of Their Own Noses

By Mirjam Neelen

I have previously blogged about the learning profession in a corporate context (see here). In that blog (which focused on Learning Designers), I mentioned a couple of issues that I believe the profession really needs to fix. In short, I believe that Learning Professionals must:

  1. become more professional (titles and names are all over the place and many ‘Learning Professionals’ aren’t learning experts; that is, they don’t have any background in or knowledge of the learning sciences),
  2. respond to reality (Learning Professionals are dealing with a changing labour market with jobs that are increasingly about non-routine/non-recurrent skills), and
  3. understand the value of learning technology (when to use or not use it and how to use it most effectively).

In this blog, I focus on the second issue in the broader context of Learning Professionals. The workplace reality is that we as Learning Professionals are dealing with certain challenges that we need to respond to. Clark Quinn has written a couple of blogs related to this topic. For example, in his blog on the need for organisations to address change, he emphasises the need for Learning Professionals to use performance data to measure impact and facilitating a shift from ‘the training bubble’ to a learning culture, for example through upskilling managers to be coaches. In addition, he says that we need to support employees to focus on self-directed learning (see also our blog on SDL) and ensuring resources that can serve as performance support tools, as well as facilitating social learning through technology. Quinn is absolutely right.

In his blog A Field of Dreams Industry, Quinn confronts us Learning Professionals with our ‘dreamy’ approach. We do what we do based on faith. We take orders and do things without properly measuring results. Quinn hits the nail right on the head again. When designing learning experiences and/or performance support tools for employees, or support workplace learning in another way, it’s critical for us to show that these approaches have actually had an impact on performance. So, what does that mean?

Performance in this context doesn’t just refer to individual performance but more importantly, also to organisational performance. For individual employees, performance refers to observable employee behaviours and execution of job duties and responsibilities (e.g., outcomes) and generally organisational performance refers to financial performance such as stock returns though later in this blog we’ll see that it might be worth looking at it from a different, operational perspective.

When considering effective measurement of performance, Learning Professionals need to understand the relationship between human capital2 and organisational performance.

humancapital

The relationship between human capital and organisational performance

Russel Crook and colleagues conducted a meta-analysis to shed some light on this relationship. Crook et al. point out that, although we generally assume that investing in human capital supports both positive individual-level and organisation-level performance outcomes, there are also various studies that report that there’s NO significant positive relationship between the two. Therefore, the researchers conducted a meta-analysis to figure out why there are such contradictory results.

To this end, they compared 66 studies comparing three different moderators:

  1. Path dependence – They determined whether the studies were cross-sectional versus longitudinal. The idea here is that truly unique and valuable skills most likely develop over time and hence can only be discovered through longitudinal studies.
  2. Organisation-specific versus general human capital – Based on the assumption that if employees have valuable but more generic skills, they can move on to a competitor easily, while an employee with organisation-specific expertise adds value for an organisation because these employees are more likely to a) make decisions that are in harmony with the organisation’s unique strategy, organisational context, and competitive environment and b) stay at the organisation, as they can’t easily transfer to another organisation and therefore probably feel more ‘attached’ to (or stuck with ☺) their organisation. They just can’t quit that easily.
  3. Operational versus global organisational performance measures – The authors explain that sometimes, when certain employees create profits, it might not directly show in organisational performance because it’s likely that influential stakeholders such as the employee’s manager use their own power to secure potential profits for themselves (organisation-speak for ‘get a raise’). This means that the result might not directly show in global organisational performance measures. Instead, performance measurements are more likely to show in operational performance measures, such as customer satisfaction or innovation.

How the results help Learning Professionals to look past the end of their own noses

What did Crook and his colleagues find? Overall, the results suggest that human capital is strongly related to organisational performance. However, the idea that this link can only be identified over time (e.g., longitudinally) wasn’t confirmed. This raises the question of whether perhaps organisational success in itself helps attract and retain employees instead of there being just a ‘simple’ one-way relationship between human capital and organisational success.

Tae-Youn Park and Jason Shaw (2013) state that the literature offers dozens of established connections for organisational performance, such as location, strategy, technology, organisational processes, physical resources, and unique products and services, which all contribute to organisational success in one way or another. For the Learning Professional it’s important to be aware of this bigger picture in order to figure out what other elements (apart from human capital) influence organisational performance.

The second finding relates to this bigger picture. It suggests that the relationship between human capital and organisational performance is stronger when human capital is organisation-specific rather than general. Organisations need to find a way to retain their people because it takes an employee quite a while to develop organisation-specific knowledge and skills. Put simply, for employees to be motivated to improve their performance and stay in their job takes more than ‘just’ learning and performance improvement (also, see for example this blog by me). It’s necessary for Learning Professionals to investigate organisational culture and build relationships with employees at all levels in the organisation. That way, it will be possible to look beneath the surface and understand employee behaviours and problems at a more profound level.

Why is all this so critical? It goes back to Quinn’s point that we should move away from being order-takers. It’s absolutely vital to look beyond our own little world so that we’ll be able to first, validate the business or performance problems as stated by, for example a stakeholder. Only when we have validated that the suggested problem is the actual problem, we can determine if a learning experience and/or performance support tool is the best solution to solve the problem. The need for these steps in the learning & development process can’t be overstated. It will be largely detrimental for individual employees and organisations alike if we design experiences or solutions for problems that don’t exist (no need to measure there! Just don’t design them in the first place!) or if we design a wrong solution to an existing problem (this is where the measuring comes in). It might sound like I’m stating the obvious, but I’d be willing to bet good money that all of us Learning Professionals have encountered examples of both. Any bettors out there?

Crook et al’s last finding suggests that operational performance measures indeed correlate more strongly with human capital than global organisational performance measures. So, when including operational performance measures, it’s easier to detect impact. This finding strongly suggests that Learning Professionals need to find the right metrics to measure impact and prove value.

Pulling it all together, it’s about Learning Professionals to a) become effective gatekeepers (prioritise customers and learning and performance challenges), b) to understand the relationship between human capital and organisational performance in order to deliver the most impact to organisational performance. Which then c) needs to be proved through appropriate measurement.

Need I say more?

1 Learning professional in this context refers to any role that deals with any kind of learning experiences and/or performance support mechanisms in an organisational context.
2 Human capital refers to the concept that people possess skills, experience, and knowledge and therefore have economic value to organisations because they enhance productivity (Ramlall, 2004).
References
Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen, D. J., (2011). Does human capital matter? A meta-analysis of the relationship between human capital and firm. Journal of Applied Psychology, 96, 443-456. DOI: 10.1037/a0022147
Park, T. J., & Shaw, J. D., (2013). Turnover rates and organizational performance: A meta-analysis. Journal of Applied Psychology, 98, 268-309. DOI: 10.1037/a0030723
Quinn, C., (2017, 8 March) [Blog post]. A ‘field of dreams’ industry. Retrieved from https://blog.learnlets.com/2017/03/field-dreams-industry/
Quinn, C., (2017, March 1). The change is here [Blog post]. Retrieved from https://blog.learnlets.com/2017/03/the-change-is-here/
Ramlall, S., (2004). A review of employee motivation theories and their implications for employee retention within organizations. The Journal of American Academy of Business, Cambridge, 52-63. Retrieved from http://s3.amazonaws.com/academia.edu.documents/40084558/Review_of_Employee_Motivation_Theories_-_JOurnal_of_Aerican_Academy_of_Business.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1493301067&Signature=XkF%2FhX9W6Iv5D6eUOGU4pEekc4k%3D&response-content-disposition=inline%3B%20filename%3DReview_of_Employee_Motivation_Theories_-.pdf