Documentation of learning effect

At the Danish Institute for Human Rights (DIHR), capacity development is an integral part of our international strategy. Therefore, it is important for us to be able to document the learning effect of ourhuman rights capacity development activities, such as e-learning, face-to-face or blended learning courses.

Our approach to documentation of learning effect (DLE) is based on:

• Securing alignment between learning, behaviour andorganisational goals.
• Measuring what happens before, during and after the capacity development activities.
• Moving the focus from satisfaction with an activity to a focus on learning, behaviour,performance, improvementsand creating results for the organisation.

It neatly ties together all the best practices of the capacity development field adding the additional dimension of testing to guide us and help to improve ouractivities and strategies.

How does it differ from course evaluations or satisfaction surveys?

When implementing capacity development activities, such as face-to-face courses, course management often use “evaluation forms”, sometimes called “satisfaction questionnaires” or “satisfaction surveys”. These types of evaluation serve to appreciate the degree of satisfaction of the participants in the course.

Typically, the evaluation will ask participants to rate the facilities, the facilitators, the course methodology, the course content and course management. The evaluation form is usually given to participants at theend of the course and submitted anonymously.

However, what it does not provide is information on the degree to which participants have effectively learned something and whether the new learning (in terms of knowledge and attitude)has an effect on their behaviour when they are back at work.

How does it work in practice?

For the purpose of documentation of learning effect, the institute has hired a company called Peopleway and use their software called EASE to measure and document the learning for participants in our face-to-face courses and workshops, e-learning and blended learning processes.

The Peopleway High Impact Learning approach is inspired by the Kirkpatrick model and Bloom’s taxonomy. Peopleway's approchisalso a result of more than 30 years of research and working with many international companies. The company now exceeds 25 million impact measurements in 29 languages.

In order to produce reliable impact measurements, the process relies on as a minimum:

• Producing high quality learning objectives for each capacity development activity
• High quality questions to measure each objective
• Using objective criteria when constructing questions rather than subjective
• Performing tests and validation of all questionnaires

The Peopleway system relies on three test questionnaires to gauge the learning of participants:

1. The pre-test provides information to facilitators before learning courses on the base knowledge, attitude and behaviourof the participants.
2. The post-test examines what participants have learned (knowledge and attitude) from the course.
3. The job-test, typically taken three months after the end of thecapacity development activity,examines the retention and development of participants' knowledge, attutide and behaviour.

Together these three tests provide the broad encompassing picture of the learning that has taken place during the capacity development activity.

Why is it important to work with impact and learning effect?

We are putting a lot of resources into capacity development activities. However, the numbers show that if we do not work with impact and learning effect, we may only unfold a small part of the potential in learning and new behaviour.

Data shows that:

• 20% of training/capacity development today does not match the learners need
• 40% of the learning need is actually not covered during training
• 70% of all the learned capabilities is subsequently lost during implementation,i.e. is not usedback at work.