In large organizations such as the VA, mandatory training has become an integral part of workforce learning. Some common reasons for adopting mandatory training for all employees include showing employees management's commitment to the topic area, promoting positive change, promoting overall staff safety, and legal or compliance considerations. In some cases, such as for the topic of diversity, mandatory training efforts are directly tied to federal requirements instituted in response to employee wrongdoings that resulted in corporate lawsuits. Numerous courts have held that to avoid punitive damages, employers must provide training to their employees on harassment and discrimination prevention. (187 F.3d 1241 (10th Cir. 1999); 270 F.3d 794 (9th Cir. 2001); 281 F.3d 452 (4th Cir. 2002)) The Tenth Circuit Court of Appeals stated that “the extent to which an employer has adopted antidiscrimination policies and educated its employees about the requirement of [the discrimination laws] is important in deciding whether it is insulated from vicarious punitive liability.” (187 F.3d 1241 (10th Cir. 1999)) The costs associated with mandatory training program management can be quite high, resulting in annual expenditures in the hundreds of millions of dollars across U.S. organizations.
The laws that motivate mandatory compliance training are often broad in nature and generally do not set standards on training content or evaluation. This leaves organizations with the ability to implement training in a compulsory manner to serve a symbolic purpose, with little attention to whether their training methods are actually effective. Unfortunately, training implemented merely to serve a symbolic purpose may be creating a false sense of organizational security. Research has found that the mere existence of an anti-harassment policy is not always sufficient to protect the employer from liability (239 F.3d 848 (7th Cir. 2001)). Likewise, in an article on the effectiveness of diversity training, a Harvard sociologist indicated that there is no history of any court giving an employer credit for the mere existence of diversity training.
Mandatory training is traditionally unpopular, and there is a perception that it is ineffective and decreases motivation to learn. Some education theory-related barriers to learning that may reduce the effectiveness of mandatory training include employee resentment about their lack of control, lack of interest, perception of irrelevancy to their specific workplace context, and workplace time pressures. Considering the high cost associated with mandatory training and doubts about its effectiveness, organizations would be well served to more closely consider the benefits of their programs.
An extensive literature on general organizational training research is available to inform decisions about how to design, implement, and evaluate training in a variety of settings., The design and evaluation of training is based on a wide variety of theoretical frameworks. There is a good deal of consensus about the best practices that organizations should engage in before, during, and after training in order to maximize effectiveness. Pre-training factors associated with training effectiveness include individual characteristics such as cognitive ability, self-effcacy,, and motivation, and needs assessments. Experts recommend that one of the most important steps in developing training is to conduct a pre-training needs analysis to identify the competencies needed, training priorities, and who needs the training. Factors that matter during training include individual characteristics and instructional strategies and principles. In recent years, group training, distance learning, and computer-based training have become common training delivery systems in many work organizations. Post-training factors associated with effectiveness include the ability to use skills and knowledge gained from training, delay between training and use of skills and knowledge, social, peer, subordinate, and supervisor support and training evaluation efforts. The Kirkpatrick Model of Training Evaluation is commonly used as a framework for evaluating training programs. Organizations have struggled to conduct training evaluation due to the labor and costs involved, and difficulty with credible field evaluation. Since most empirical research is still relying on surveys to measure learning outcomes, there is still a need for more research using formal experimental designs to evaluate training effectiveness. Although there has been an increase in general training-related research, it is unclear whether the best practices identified in the general training literature have their desired effect on outcomes in the mandatory training domain.
The VA currently requires all employees to undergo mandatory training on the topics of Government Ethics, Prevention of Workplace Harassment/No Fear Act, and Privacy and Information Security Awareness and Rules of Behavior (
The requirements for the three trainings that are mandatory for all VA employees originate from Executive Orders, Congressional mandates, the Office of Personnel Management, regulatory bodies, and VA department-level requirements. These regulatory directives do not specify requirements about the format, content, or method of delivery of training material. Originally, the VA provided local facility leadership with the flexibility to locally manage their mandatory training processes, including tracking and recording employee attendance. This allowed local facilities to customize their approach to meeting the training mandates based on the local culture, which included a variety of training formats including face-to-face sessions, videos, handouts, or multiple modalities. Eventually, national tracking became more of a priority for the VA, and this led to centralization and standardization of mandatory training. Currently, all VA mandatory training is computer-based, and it is delivered and tracked via the VA Learning University Training Management System (VALU-TMS).
In order to evaluate the use and outcomes of mandatory training in the VHA, the National Leadership Council's Human Resource Committee chartered the Mandatory Training Workgroup in November 2008. The workgroup was designated as a standing subcommittee in October 2010. The goal of the Subcommittee is to “envision a strategic evidence-based approach to Mandatory Training that linked employee learning to organizational outcomes.” The workgroup envisions “that mandatory training, used sparingly, would become meaningful, focused, effective, flexible, and satisfying to all employees.” The Subcommittee has suggested various revisions to the VA's current mandatory training approach that include rescinding the requirements entirely, changing the requirements to “highly recommended” instead of mandatory, combining topics, reducing course length and/or frequency, limiting target audience, substituting a competency-based or stepped training approach, and adding additional delivery formats to allow learners to select resources that best fit their individual learning styles. The theoretical advantages of computer-based training include convenience, flexibility in scheduling, consistency of material presented, and tracking and documentation capabilities. But what is not taken into consideration by computer-based methods is that people learn in different ways.
In their 2009 report on the burden of VA mandatory training, the Mandatory Training Subcommittee raised questions about the value of the VA's mandatory training program. The Subcommittee estimated that VHA spends $40 million a year just for the three core trainings mandated for VA employees (Table 1). Despite these high estimated costs, the Mandatory Training Subcommittee did not identify any studies in VA or otherwise of the effectiveness of any VA mandatory training programs. Additionally, the Subcommittee's qualitative evaluation of employee perceptions found universal unhappiness about the mandatory training requirements. Common themes include criticism that the mandatory trainings take up too much time, are not optimally accessible (including the varying locations and usability of courses), vary in quality, lack alternatives to online courses, and do not adapt to an individual's role and his or her existing knowledge on the subject. The Subcommittee report concluded that, given the enormous burdens of cost and negative employee perceptions, the VA would be well served to more closely consider the benefits of their mandatory training programs.
In January 2009, the Mandatory Training Workgroup asked the VA Technology Assessment Program (TAP) to conduct a Brief Overview of evidence on the organizational effectiveness of mandatory learning strategies. The VA TAP Brief identified very little evidence on the subject and their main findings were that volition may be an important determinant of organizational learning and that training effectiveness may vary as a function of evaluation criteria, training delivery method, the subject being taught, and the criterion used to operationalize effectiveness. In February, 2014, to maintain the currency of knowledge about evidence on mandatory learning strategies, the Mandatory Training Workgroup requested that the VA Evidence-based Synthesis Program Coordinating Center (ESP CC) conduct an updated Evidence Brief on this topic.
An evidence brief differs from a full systematic review in that the scope is narrowly defined and the traditional review methods are streamlined in order to synthesize evidence within a shortened timeframe. An evidence brief does not outline the full context in which the information is to be used and does not present a comprehensive assessment of knowledge on the topic. Brief or rapid review methodology is still developing and there is not yet consensus on what represents best practice.
SCOPE: The objective of this Evidence Brief is to synthesize the literature on the effectiveness of mandatory online employee compliance training. The ESP Coordinating Center investigators and representatives of the VHA Mandatory Training Subcommittee worked together to identify the population, comparator, outcome, timing, setting, and study design characteristics of interest. The VHA Mandatory Training Subcommittee approved the following key questions and eligibility criteria to guide this review:
Key questions:
Key Question 1: What is the effectiveness and comparative effectiveness of mandatory computer-based trainings on government ethics, workplace harassment, or privacy and information security-related topics?
Key Question 1a: Does the effectiveness of these mandatory computer-based trainings vary by format (eg, just-in-time training, competency-based assessment, stepped training delivery) or repetition of training?
Key Question 1b: Does the effectiveness of these mandatory computer-based trainings vary by the method of training delivery (eg, length, audiovisual components)?
Key Question 2: What are the harms (eg, turnover, morale, grievances, institutional and opportunity costs) of these mandatory computer-based trainings?
INCLUSION CRITERIA: The ESP included studies that met the following criteria:
Population: adults in the workforce
Intervention: mandatory online training targeted to a broad base of employees to address an organization-wide need (eg, ethics, prevention of workplace harassment, information security)
Comparator: no training, other training methods, or other activities
Outcomes: trainee learning (eg, changes in knowledge or skills), trainee behavior, or organizational change (eg, changes in productivity, turnover, morale, grievances, or patient outcomes)
Timing: longitudinal studies
Setting: workplace
Study design: randomized controlled trials and observational studies
This Evidence Brief will