Selecting Your Indicators

This section deals with the process of selecting your indicators using the functions built into BPCLEtool. If you would like more information about the indicators or the BPCLE Performance Monitoring Framework (PMF), visit the BPCLE Indicators help page.

BPCLEtool has been programmed to suggest which of the 55 indicators of the BPCLE PMF would be useful for your organisation to monitor based on the consensus ratings your group gave to the various aspects of your clinical learning environment during the self-assessment process. Later in this section, you will find an explanation of how BPCLEtool has been programmed to select indicators.

Once you have completed your detailed assessment and go to the Indicator Selection step of the process, you will see the suggested indicators presented in several formats:

• Click on the BPCLE Framework element links to see the indicators displayed on their respective program logic maps, aligned with the map item they are associated with. Recommended indicators are shaded in dark green, while indicators that are not recommended are shaded in pale green.

• The indicators are tabulated by category and by type. These tables allow you to see the total number of indicators that have been selected.

• The complete list of 55 indicators is presented, with the recommended indicators highlighted in green.

It is highly recommended that your group review the suggested indicators, to determine whether there is a good fit with your organisation’s priorities, resources and current data collection/monitoring activities. For example, your organisation might already be collecting data that could inform a particular BPCLE indicator, but that indicator might not be in the recommended list. Alternatively, your organisation might not have the resources to monitor a particular indicator that has been recommended.

It is also a good idea to review the list of suggested indicators to ensure there is a mix of indicator types (structural, process and outcome) and an appropriate balance between the four categories of indicators. For example, you should avoid only collecting structural or outcome indicators and your selection should include a higher proportion of Category I and II indicators than Category III and IV indicators (see below for an explanation of indicator types and categories).

You are able to select or deselect indicators according to your organisation’s requirements, with two exceptions:

• If externally reportable indicators apply in your jurisdiction, you are not able to deselect those indicators. At present, this only applies in Victoria.

• Indicator #1 is automatically calculated by BPCLEtool and is therefore always selected.

You can select/deselect indicators in one of two ways:

• In the list of indicators, you can click on the check box under Indicator Status to select or deselect the indicator. Note that when you hover the mouse over the check box, a message appears that tells you whether the indicator was originally recommended or not, regardless of whether you have subsequently selected or deselected the indicator.

• On the program logic maps for each BPCLE Framework element, you can double click on an indicator to open a dialog box that shows the indicator (and the item the indicator is aligned against). You can check or un-check the box next to the indicator name. Once you close the dialogue box, you will see the indicator shading has changed to reflect its new status and, once you close the map, the new status of the indicator will be shown in the indicator list.

Once you have finalised your indicator selection, click the blue “Complete Assessment & Review Results” button at the bottom of the Indicator Selection page. Clicking this button tells BPCLEtool this assessment is “complete” and makes the assessment available for BPCLEtool reports and indicator monitoring functionality.

Watch the video tutorial for a demonstration of how you can use BPCLEtool to refine your indicator selection.

Alignment of Indicators with the BPCLE Elements

The following table summarises the alignment between the 55 BPCLE indicators and the six BPCLE elements. Note that Indicator 1 is not aligned with any of the six elements, as it is a meta-indicator that addresses overall performance monitoring against the BPCLE Framework.

What is an Indicator Category?

An integral part of the BPCLE Performance Monitoring Framework is the weighting system that was developed to help organisations prioritise the 55 indicators associated with the BPCLE Framework. The weighting system is based on two criteria, namely:

• Relevance rating, in terms of the direct impact of the component being monitored on the clinical learning environment; and
• Methodological rating, in terms of
  • Ease of data collection for the indicator
  • Actionability of the indicator result
  • Confounding issues that could limit interpretation of the result

The indicator categories derived from this weighting system are as follows:

If the category field includes the label “External”, then these indicators have been flagged by the relevant authority within your jurisdiction as externally reportable and cannot be deselected within BPCLEtool.

What is an Indicator Type?

Indicators are defined according to whether they measure inputs, activities or outputs of the system.

Watch the video tutorial for background information about BPCLE indicator categories and types.

How BPCLEtool selects indicators

BPCLEtool uses a three-step process to derive the list of recommended indicators you see when you first click on the Indicator Selection page.

Firstly, BPCLEtool incorporates a database that aligns the items in the six BPCLE program logic maps with relevant indicators. Each indicator has one or more map items associated with it and each map item may be associated with one or more indicators. If an item receives a “Does not exist” rating or an “average-or-below” rating during the self-assessment process, any indicators associated with that item are included in a preliminary list of recommended indicators. Of course, any externally reportable indicators for your jurisdiction are selected irrespective of the rating of any associated map items.

In the second step, BPCLEtool has been programmed to apply redundancy rules that “decide” between two indicators, where those indicators are either measuring similar components of the system or are measuring components of the same logic pathway in the maps.

In the final step, BPCLEtool has been programmed to apply category rules that “decide” between two indicators, where at least one of those indicators is Category III or IV. These rules preferentially select the indicator with the better relevance rating or, where both indicators have the same relevance rating, the indicator with the better methodological score.

Thus, the list of recommended indicators has already been culled to some extent from the initial set of indicators suggested by the rating profile.

On this last point, it is important to note that BPCLEtool can only make recommendations about indicators associated with the elements you actually include in your assessment. For example, if you choose not to include Element 2 (Best practice clinical practice) in your assessment, BPCLEtool will not make any recommendations about Indicators 14 – 17. Furthermore, if you only partially complete the detailed assessment for the elements you have included in your assessment, BPCLEtool can only make recommendations about indicators associated with those map items your group actually rated. The exception to this is any indicator that has been marked as externally reportable by the relevant authority in the jurisdiction your organisation belongs to; these compulsory indicators will be selected regardless of whether the relevant element was included in your assessment and regardless of whether you completed the assessment of all map items.

Thus, there are four possible circumstances where BPCLEtool does NOT recommend monitoring a particular indicator, namely: The map items related to that indicator were all rated above-average; or The indicator was included in the preliminary list of recommended indicators, but was culled through application of the redundancy rules or the category rules; or The element in which that indicator is located was not included in the assessment; or The element in which that indicator is located was included in the assessment, but your group did not rate any of the map items associated with that indicator.

On the other hand, there are two possible circumstances where BPCLEtool DOES recommend monitoring a particular indicator, namely: Any of the map items related to that indicator were rated “does not exist” or average-or-below and the indicator was not culled through application of the redundancy rules or the category rules; or The indicator is designated as compulsory for the jurisdiction to which your organisation belongs.

For more information on BPCLE indicators, please visit the BPCLE Indicators help page.