Background Different tools for performing 'evidence based' leadership have been implemented in hospitals to meet the challenges of modern hospital management. Balanced Scorecard (BSC) and EFQM Excellence-Model (EFQM) are among such tools. The key mission of these is to provide a system for evaluating the organisation from a number of predefined perspectives. The Norwegian Knowledge Centre for the Health Services was asked to conduct a systematic review to assess whether implementation of these tools provide better strategies, better management or better hospital quality.

Methods A systematic literature review with a defined search strategy, and predefined criteria for selecting studies. Quality assessment, data extraction and summary of results were performed by two researchers independent of each other.

Results We retrieved 639 possibly relevant publications; 71 articles were obtained in full text and five studies were finally included. Four studies reported experiences from implementing BSC and one study reported on EFQM. Included studies described that scorecards may be useful at a local level to define strategic aims, measure quality indicators, and define lower levels for quality action, compare departments over time or to evaluate implementation of new treatment options. We found no evidence that implementation of BSC or EFQM influenced hospital management or quality.

• Three studies reported results after implementing BSC at department level in hospitals in different areas: emergency unit, anaesthesia department and nephrological department. These studies described development of criteria and indicators for measuring, but are not able to relate results to implementation of BSC or EFQM.
• Two studies reported results on the institutional level. One study used EFQM to evaluate the implementation of evidence based treatment processes in an addiction centre in the Netherlands. Another study compared one hospital in Japan with a Chinese hospital with indicators within the BSC model. Both studies assessed the models as useful for measuring quality.

Conclusion Research on the usefulness of scorecards is especially challenging due to the fact that scorecards may influence on different levels within a hospital and at different time points.

• There is no evidence that BSC or EFQM influence on hospital performance.
• There are descriptive reports on the usefulness for different purposes in local settings.
• Time series with several pre and post measurements would add to our understanding of the usefulness of scorecards.
• More research and development of suited methodology are needed.