My 4 Core Brand Values

Brand is about what your values are. It is about integrity and knowing what you know is right and putting your life energy behind it. Here are 4 qualities of my ID brand.

Quality- I really am annoyed by low quality learning. If there is not a quality process behind it, it is suspect. I don’t like to fall into traps of what is hot and modern and people say works, when the results are low. An example is eLearning. Studies show that 3 months later the learner has forgotten most of what they were exposed to. This can be overcome with better learning expectations, organization and materials. Make a plan for high quality learning results and strategy before you invest in the infrastructure.

Smart Innovation- 15% of learning is applied. 30% when there is management support. The true innovation is not in the tools for learning, but the organizational support mechanisms available. But the real secret is adapting best practices of learning organizations. Sorry, the fancy, animated eLearning’s ain’t worth the money unless management is aligned to support it- this is smart innovation.

Learner Focused- People are very selfish- especially when they are going through change. Good learning is change. The support for change mechanisms need to be built into organizations. But further, the learner anxiety needs to be addressed in the learning. Good learning is easy- 1. Sell the learning with WIIFM 2. Engage in value added activities that build learner confidence with actual and applied skills and activities 3. Reward their excellent performances. And always keep in your heart and mind sympathy because they are nervous as they go through change.

Engagement- The biggest struggle, at any student level, is wake up and learn! We learn and stay awake by doing things. So have the students do something! This is not rocket science. This is brain science, so apply it! And let them have fun! When we are playing, we are learning!

What is your brand? How do you show it in your work? I look forward to seeing it!

Posted in Uncategorized | Leave a comment

Recent Portfolio Work

Today I thought I would share some of my recent portfolio work. Please let me know what you think!

Redesign of F2F learning.

Project: A redesign of F2F learning for suppliers.

Goal/Problem: An assessment of the previous learning indicated a gap in the application of the learning. Interviews revealed that the course was 1) difficult to follow 2) lacked interactivity and 3) did not address attitude.

Audience: External suppliers.

Your Role: I accessed the learning; proposed the redesign; then designed, developed and accessed the course.

Methodology Used: The interface of the learning was redesigned to make the information more clear and logical; interactivity was incorporated into the learning; a section was added to influence the attitude of the learners.

Scope/Size: A complete redesign of the learning for an 8 hour F2F learning to be rolled out internationally.


bugs Form

Design of learning template.

Project: A template designed for a curriculum within a division.

Goal/Problem: There was a need for an attractive, easy to use template for online courses.

Audience: IDL section, SME’s and eLearners.

Your Role: Assessed the need, designed and developed the template.


interface 3 interface 2 interface

Design of predesign assessment.

Project: Design of a form to gather information to create course proposal.

Goal/Problem: Often the scope, contents and milestones of a project are discussed and recorded in meetings. Often, the notes and memory of the details of these meetings can become fuzzy.

Audience: ID staff and client.

My Role: Designer who wanted to clarify the course proposal process.

Steps Taken: Identified the procedural gap; designed, tested and incorporated the form for the process.


Course redesign info

Posted in Uncategorized | Leave a comment

Do you want ROI or risk a few billion?

There is a course I am involved with that has become a tradition; training outside vendors policy and procedures. For this article I will call it “Vendor Course ABC” or “VC ABC”. This training was established during a crisis as part of a strategic objective at that time. Although we collect level 1 and 2 evaluations and adjust the course each time we conduct it, we have yet to calculate the ROI. Why is this?

ROI can be considered the “magic number” as to whether a training has benefited the business. In some companies it is expected in the training report. However, there are some situations where the ROI is either too difficult to calculate or the cost of the course is balanced against other potential costs or risks. VC ABC falls into both of these categories.

First of all, the cost is difficult to calculate. There are significant costs involved in VC ABC- 3 trainers spend 2 days conducting the training. Meanwhile, the supplier representatives have to travel, train and spend 2 full days in training. This is all calculable, but what about the benefits? To gather the data from hundreds of suppliers over tens of thousands of parts in millions of units over time is next to impossible.

Next, the potential costs or risks of not having this training are very high. The suppliers make critical parts and if one of them fails the entire safety reputation of the company might be on the front page of the newspaper. This could cost billions of dollars. In addition, we rely on the suppliers to follow the policy and procedure to make quality parts. Again, the quality reputation of the company is at risk if these parts don’t meet our customers’ expectations, still risking billions of dollars.

So sometimes we have to build and deliver the training the best we can, not to reach a magic ROI number, rather to mitigate billions of dollars of risk.

Posted in Uncategorized | Leave a comment

Evaluating training for the intangibles

I can imagine a world 50 years ago when there were few ways to systematically evaluate training. At that time Kirkpatrick’s innovative 4 levels of evaluation was a godsend. But now it seems to have become the Kirkpatrick family mafia slightly updating the model while they also market and train the material.

Certainly they have benefited. Who else has? Training departments certainly have, because they have a bible of evaluation they can refer to in meetings. This is against criticisms against the basic flaws in the 4 levels scientific methodology. Thus, the 4 levels has become more of a cult of what ought to be done, and is often blindly accepted.

Why is this? My friend applied for a training job and asked about level 3 evaluations that the company does. The response was, “Well, we just do smile sheets, and that works for us.” With the training function not being the core of the business, management doesn’t pay much attention. Management takes for granted that the training is appropriate and works, perhaps because it is out of their field of expertise. If accountants were held to the same standards, a company might find itself in financial trouble.

The performance of the employees is the core function of the business. Apple realized this early and added effectiveness evaluation and maintenance evaluation to its MacIntosh training. Meanwhile John Edes suggests visual confirmation and social ownership as means to evaluate training. So there are other means outside of the 4 levels. I expect the next revolution in evaluation will come from outside academia, perhaps from the tech industry.

Of course Phillips has extended the 4 levels by adding ROI and Intangibles. With the high cost of finding and keeping employees and the importance of brand value, it seems that a higher focus on job satisfaction, work climate and organizational commitment would prove valuable.

Breaking from the industrial way of viewing employees to a more humanistic way has proven valuable to many organizations. Google is one of these examples. Their enculturation is a week-long and has low ROI by Kirkpatrick’s standards. But the intangibles have made the brand valuable and workers loyal. Evaluating training for the intangibles will be the next revolution, off the 4 levels chart, and deeper than a smile sheet.

Posted in Uncategorized | Leave a comment

Amplify outside effects may work better than isolating them

We deliver training developed in house for suppliers. The measured results of the training can be attributed to many factors outside of the instruction.

The training is for a manufacturing quality chart. The objectives are:

  1. Level 2: Recognize the layout of the form

Apply the best practices taught in the course to complete a case study.

Measure of learning: A critique of the completed case study.

  1. Level 5: Have confidence filling out the form.

Measure of confidence: Likert scale survey at the end of the course.

Ideally, follow up interviews 2-3 months after the class could be performed to measure the learning and attitude. This could be questionnaire of the attendee and their supervisor. It could also be interviews with a sample group. In addition, historical data of the individual supplier’s compliance can be used to measure application.

To isolate the effects of instruction other attributive factors would have to be isolated. These would be:

Supplier company culture

  1. To what extent is quality chart compliance encouraged.
  2. Who the suppliers other customers are.
  3. The complexity of their manufacturing process.

Our company

  1. Inconsistencies between plants enforcement of quality chart rules.
  2. Relationship between supplier and plant quality engineers.
  3. Quality chart audit schedule is based on supplier grade.
  4. The safety and critical characteristics of the part made by the suppliers.

Individual students

  1. Role in the company and responsibility for quality chart development.
  2. Time available for quality chart development.

Because of the large number, variety and strength of these influences, it would be very difficult to isolate these. Furthermore, conducting control group would be impossible because the learners are from different organizations.  However, using stakeholder estimates and forecasting trends could help isolate the effects of training.

As for stakeholder estimates, participants and their supervisors could be asked to fill out a detailed questionnaire that estimates the effect of the instruction. As for trends, the historical trend of successfully completed quality charts submitted to the plants could show general and supplier specific trends.

Surely we are more credible when we can isolate the effects of instruction. However, this often is a costly and time intensive effort. Furthermore, these efforts might strain the relationship of the companies that receive the outside training. Perhaps a more rewarding endeavor would be amplifying and focusing the positive effects of organizational and management factors and Hawthorn or Pygmalion effects on the training.

Posted in Uncategorized | Leave a comment

Getting Level 3, 4 and 5 evaluation is tough!

They keep bugging me to give them a level 3, 4 or 5 evaluations! They say every time: “Please fill out the survey! You can win free prizes!” I am talking about those fast food restaurants or other service industries or sometimes they pop up on the internet when you are on a site. What is up with this?

The challenges of gathering level 3, 4 or 5 data are well known: time consuming and costly, unreliable data, or not even asked for by management. Basically, it is easier to justify a budget for training and add a bit for level 1 and 2 evaluations. But to ask for the time and money for higher levels isn’t usually kosher.

Why? Game of school- the assumption that they go to school, they passed the test, everything is fine. Anything we missed will come up in the evaluation. Oh no! Game of work! How often has anyone spent the resources to get an unbiased, 3rd party evaluation? Pretty rare. It is often more of an “I like you, we still like the company, right” conversation. And how was the training? Great, I use it a lot. Next question please.

The best higher level evaluation program I have experienced is the secret shopper- an unannounced evaluator that would come to our restaurant for a meal and grade the performance. Was it costly? Not really considering how much a dissatisfied customer costs! Did it help management evaluate performance? Definitely. Is this doable in other industries? I think so.

So, what is up with intangibles being increased job satisfaction, reduced absenteeism or increased cooperation being relegated to the outcast level 5? Aren’t these the real quality issues that add value to training and the results? Why are they so hard to measure and justify?

There is a mint to be made when someone learns how to efficiently and effectively scale upper level evaluations with new technologies and methods without bugging us, will it be you?

Posted in Uncategorized | Leave a comment

Shortcut to creating Level 1 and 2 evaluations

The most challenging part of crafting evaluations is the time limit that is often given for evaluation. It is normal for many trainings to budget much time for instruction, little for evaluation, and less or none for application of learning. For example, my friend has to develop a training to teach suppliers the knowledge of rules and regulations to comply with the manufacturer. There is a whole day given to the instruction, then a 10 question quiz, with no question as to how they will apply the learning.

This is understandable because many organizations do not know the levels of evaluation, much less the details of how to properly apply these levels. Moreover, many do not even have recognizable learning objectives to build effective level 2 evaluation.

The art of crafting a level 1 evaluation is simple if you have created surveys before.

Step 1. Purchase Jack Phillips’s text How to Measure Training Results.

Step 2. Turn to pages 77-78.

Step 3. Follow the form but adjust it for your training.

The form used in this textbook is very comprehensive and adaptable. You will find it clear, easy to use and comprehensive.

Creating a level 2 evaluation is trickier. It works best if you start with Bloom’s Taxonomy verbs. These can help you to narrow down types of assessment to ones that fulfil the learning objective. For an example, see the table below:

Bloom’s Verb Assessment
Identify Pick a correct term or definition from several incorrect ones
Use Make or explain how to make something
Discuss Explain the features or characteristics of something

You can see how the learner’s correctly completing the task shows that they are able to perform the verb!

Although this post does not dive into the depths of Level 1 and Level 2 evaluations, I look forward to sharing more tips and tricks with you!

Posted in Uncategorized | Leave a comment