L.E.A.R.N.S.

First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics Evaluation Questions Indicators Data Sources

PROCESS 1.0 Admin Supports/ Inputs

1.1 Clinical staff and trained volunteers 1.1.1 How are clinical staff and volunteers chosen for this program? 1.1.2 How does the program ensure that only trained staff and volunteers are involved in the program? 1.1.3 What performance measures or supervision is in place to ensure that staff and volunteers are continuing to follow the model? Evidence of a recruitment process. Evidence of expectations for volunteers. Evidence of only trained staff working in the program. Staff performance measures and reviews are in place. Evidence of internal administrative coding system for colorectal patients. Evidence of process to address missing family. Policies and program goals/objectives and expectations are outlined. Evidence of L.E.A.R.N.S model in staff and volunteer interactions during pre-clinic appointment. Evidence from patient and family. Job fact sheets Interview/recruitment notes for staff and volunteers Volunteer policies Human Resource filestracking system of completed training Performance reviews Patient and Family Focus Group Observation of pre-clinic sessions Staff focus group Computer data system Chart review Policy manual Training manuals

1.2 Colorectal patients and families

1.2.1 What systems are in place to ensure colorectal patients are identified for the L.E.A.R.N.S program? 1.2.2 What steps are taken to ensure attendance by family/caregiver member at the pre-clinic appointment? 1.2.3 What actions are taken by the clinician and volunteer to develop a partnership with the patient and family?

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 1.3 Management/Director support Evaluation Questions 1.3.1 How is the program supported by management? 1.3.2 How has management support been conveyed to the staff working in the program? Indicators Evidence of management involvement in decision making about the program implementation. Evidence of direct communication support from management. Staff satisfaction with the level of management support. Evidence that program aligns with vision/strategic direction of the organization. Evidence of staffing model Evidence of a responsive staffing model to adapt to fluctuations in clinic volume and staff/volunteer availability. Budget for salaries and supplies. Patient information material is available and in the desired formats. Evidence of a communication plan. Views on appropriateness of communication plan. Data Sources Meeting notes Emails, formal communication memos Staff meetings Hospital policies Satisfaction survey

1.5 Equipment/human resources

1.5.1 How does the staffing model support the program? 1.5.2 To what extent does the combination of clinic staff and volunteers support the delivery of the program on a daily basis? 1.5.3 To what extent to the patient information materials support the delivery of the program?

Meeting notes HR files-employee list Budgets Patient education pamphlets

1.9 Communication/marketing

1.9.1 What communication plan has been developed to market the program? 1.9.2 What involvement have the stakeholders had in preparing the communication plan? 1.9.3 What is the appropriateness and satisfaction level among stakeholders?

Stakeholders survey Staff survey Communication plan

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 1.12 Education and training Evaluation Questions 1.12.1 What training and orientation have staff and volunteers had about the L.E.A.R.N.S model? 1.12.2 What is the process for training new staff and volunteers? 1.12.3 To what extent do staff and volunteers feel that the training and orientation have met their learning needs? 1.12.3 How have staff been supported in learning about health literacy? Indicators Evidence of training sessions. Orientation manual developed. New staff are identified for training and education. Evidence of input from staff and volunteers about level of satisfaction with training/orientation. Data Sources Observation of training sessions Orientation manual HR files Staff survey Staff meetings

2.0 Implementation
2.1 Assessment First 5 risk factors 2.1.1 What relevant assessment tools are available to the staff? 2.1.2 Who is responsible for completing the assessment? 2.1.3 How are the results of an assessment conveyed to other team members? 2.1.4 How comprehensive is the assessment in identifying the patients needs? Utilization of assessment tools. Evidence that assessment tools are being completed correctly and by the appropriate person. Assessment information is documented. Expectations of assessment communication are outlined in policies and procedure manual. Number of assessments completed. Evidence of care plans and referrals that address needs. Assessment tools Patient chart Care plans Policy and procedure manuals Computer administrative system Staff Patients

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 2.2 Engaging patients and families in education/resources Evaluation Questions 2.2.1 How will patients and families be engaged in education and what will determine their level of engagement? 2.2.2 What process occurs when patients and families do not engage in the program? 2.2.3 How satisfied are patients and families with their level of engagement in the program and feel that their knowledge level has increased? Indicators Evidence that the trained L.E.A.R.N.S staff are working with colorectal patients. Views on patient engagement. Evidence of procedure/plan for patients that do not engage in the process. Patient behavioural change. Number of appropriate referrals made to other clinics. Evidence of a procedure for referrals. Data Sources Patient and family focus group Staff focus group iEARAS discharge questionnaire 30-day post-surgical phone call Primary care provider

2.3 Referrals to appropriate clinic

2.3.1 What is the process for making a referral? 2.3.2 What factors help or hinder the process to make appropriate referrals?

Chart audits Staff Patients and family 30-day post-surgical phone call Primary care provider Chart audits Various College standards of documentation Hospital policy regarding documentation Care plans Observation of team meetings/rounds

2.4 Documentation

2.4.1 What types of information is documented? 2.4.2 Who is expected to document information? 2.4.3 How is the documentation used to inform other teams or clinics?

Documentation reflects use of assessment tools. Documentation follows professional standards and best practices. Documentation identifies individual patient needs. Number of times other teams review documentation. Reinforcement process Issue identification

2.5 Education Reinforcement

2.5.1 How was education reinforced? 2.5.2 What factors facilitated or created barriers in their reinforcement?

Patients and families Staff/volunteers

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 2.7 Follow Up Evaluation Questions 2.7.1 What is the follow-up process? What factors facilitated or created barriers to this process? 2.7.2 How effective was the follow up provided? Indicators Evidence of follow-up Patients perceptions Views on effectiveness of follow up process Data Sources Patient and family Key stakeholders

3.0 Activities/Outputs
3.1 Knowledge transfer and facilitated patient centred learning 3.1.1 How was knowledge transferred? Was it effective? 3.1.2 How was learning facilitated? Did intended learning take place? 3.2.1 What other health care services were used by patients? Were they appropriate? Did they relate to the action plan developed? 3.2.2 To what extent were other health care services used by patients? Why were those particular health care services used by patients? 3.2.3 What was criterion for referrals to health care services and recommendations to supportive services? 3.3.1 What other health care services were used by patients? Were they appropriate? 3.3.2 To what extent were other health care services used by patients? Why were those particular health care services used by patients? 3.4.1 Did communication take place between the patient and PCP? What helped to facilitate this and what were the barriers? 3.4.2 How did communication with PCP take place? How often? # of contacts between patient and PCP Barriers/facilitators to communication Letter to patient PCP Clinic call/appointment log Evidence of staff/volunteer teaching Evidence of patient/family ability to teach back # of other health care services used and types Views on appropriateness Reason(s) for choice(s) Care plans/Chart audits Patient and Family focus group Medical record Patient and family survey Family

3.2 Patient use of health care services (referred) and recommended supportive services

3.3 Use of health care and supportive services accessed independently.

3.4 Communication with Primary Care Provider (PCP)

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 3.5 Use of other health care services and Use of supportive services that were sought out independently by patient/family. Evaluation Questions 3.5.1 What supportive services were used by patients? Were they appropriate? 3.5.2 Why were those particular supportive services used by patients? 3.6.2 How was surgery preparedness determined and defined? Were patients prepared? Why or why not? Indicators # of supportive services used and types Views on appropriateness Reason(s) for choice(s) Determination and definition of patient satisfaction Satisfaction issue/reason identification Determination and definition of patient preparedness Preparedness issue/reason identification Issue identification # of successful calls made and answered # of attempts Questions asked and answers received Data Sources Patient survey Family

3.6 Patient and family prepared for surgery

Key stakeholders Patients & families

3.7 Phone call to patient 30 days after surgery

3.7.1 Were the calls made? How many phone calls made to patients were answered or not answered? Number of attempts to each patient?

Log book 30 day post-op phone call Patients Staff/Volunteers

OUTCOMES 4.0 Immediate-Term Outcomes

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 4.1 Strategic, tailored health education Evaluation Questions 4.1.1 Is the program reaching all patients with colorectal cancer requiring surgery? 4.1.2 Was the First 5 educational intervention usable/complete/feasible? Why or why not? 4.1.3 How effective was the programs evidence- based conceptual model? Indicators Percentage of surgical colorectal patients who participated in a tailored First 5 session Evidence of First 5 assessment and risk identification Evidence that LEARNS best practice methodology was appropriate and feasible Evidence that patient can teach back new knowledge Frequency patients read the First 5 brochures and materials # of times patients sought supportive care post First 5 (MD visits, dietitians, pharmacists) Staff and volunteer education # Patient and Family Education Program (PFE) staff consults Evidence of optimized health pre-operatively Identified issues and challenges Data Sources First 5 documentation Staff and volunteer survey Staff and volunteer focus groups Performance reviews Patient & family focus group 30 day post-op phone call

4.2 Improved health communication (written & verbal; plain language)

4.2.1 How well did patients understand their individual First 5 information? 4.2.2 To what extent were patients able to make decisions about next steps for their health and act on that decision? 4.2.3 What factors facilitated or hindered effective health communication?

Organizational Patient satisfaction survey Patient & family focus group Staff and volunteer survey Staff and volunteer focus group 30 day post-op phone call PFE consult database

4.3 Increases self-efficacy

4.3.1 To what extent did patients believe that they could independently prepare themselves for surgery? 4.3.2 What facilitated/ motivated patient belief that they could optimize their own health? What were the barriers to improved self-efficacy?

30 day post-op phone call Patient Self-efficacy survey Patient & family focus group

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 4.4 Empowered self-management/self-care (action) Evaluation Questions 4.4.1 What action(s) did patients take to optimize their health pre-operatively? 4.4.2 How many patients made a PCP appointment after the First 5 education? 4.4.3 What factors facilitate or pose barriers patient selfmanagement? Indicators Self reported actions/changes/ adjustments/ improvements to health regimen # of PCP appointments made Factors identified Algorithm to triage referrals # of referrals made # of patients that attended referral appointments Spectrum of services used Timeline of access Identified issues/ challenges Evidence of patient and family satisfaction Data Sources 30 day post-op phone call PCP appointment logs Patient & family focus group

4.5 Access to supportive system

4.5.1 What has participation in the program done for the patients access to supportive services? 4.5.2 How was access to system supports given? 4.5.3 What was the time period to access these services before surgery? 4.5.4 Which supports were used? 4.5.5 What facilitated/ barred patient access to supportive systems and services?

First 5 documentation Clinic appointment logs 30 day post-op phone call Patient and family Library patron data

4.6 Enhanced surgical preparation & expectations managed.

4.6.1 How were patient and family expectations and preparation skills affected by the program? 4.6.2 To what extent did patients perceive that they knew what to expect during their surgical experience? 4.7.1 What was the extent of impact of pre-operative health status on surgical scheduling?

30 day post-op phone call discharge survey Patient & family focus group Operating Room Database

4.7 Optimized physical status preoperatively

# of surgeries postponed or cancelled

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 4.8 Reduction/prevention of surgical complications. Evaluation Questions 4.8.1 To what degree did overall statistical surgical complications rates improve/ not improve from baseline? 4.8.2 How many patients acquired a surgical site infection (SSI), had sustained abnormal glucose levels, had stoma complications, and/or acquired a respiratory infection compared with baseline? 4.9.1 How many patients were discharged within the optimal time period per iERAS best practice? 4.9.2 How many phone calls about questions and concerns did the nurse navigators receive after discharge? 4.9.3 How many phone calls about questions and concerns did surgeons receive from patients? Indicators Baseline indicators Data Sources NSQIP Database Electronic patient records Discharge survey

4.9 Timely and safe discharge.

Average patient length of stay (LOS) Frequency of phone calls to navigator from baseline Frequency of phone calls to surgeons from baseline # of extra surgeon appointments required prior to standard 6 week post-surgical follow up visit

Discharge survey NSQIP database Electronic patient records Nurse navigator phone consult logs Surgeon administrative assistant phone logs 30 day post op phone call Patient & Family focus group

5.0 Intermediate-Term Outcomes

5.1 Promote independence/partnership. 5.1.1 To what extent did the program promote their perception of independence? 5.1.2. How was partnership developed with the patient during the program? To what extent did patients and families feel they were an active partner? Evidence of increased independence. Evidence of educator collaborating with patient to jointly create a structured learning and action plan. Evidence of educator facilitating the collaboration. Patients perceptions. Patients and family focus group, questionnaire, follow-up phone call. Staff/Volunteers observation and questionnaires

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 5.3 Prevent/decrease re-admission Evaluation Questions 5.3.1 To what extent did the program prevent hospital readmission? 5.3.2 What were the reasons for any hospital readmission? 5.4.1 What actions did the patient take to reduce the risk factors? How were these sustained? Indicators Evidence of decreased rate of re-admission. Data Sources NSQIP database, readmission diagnosis and assessment findings Patient questionnaire.

5.4 Sustained action to further reduce individual First 5 risks.

Evidence of behaviour change to reduce any of the risks identified at assessment.

6.0 Long-Term Outcomes

6.1 Increased self-management of comorbid health condition. 6.1.1 What changes has the patient made in the management of his/her health condition? 6.1.2 How does the patient feel about his/her ability to manage his/her health? Evidence of behaviour change to support management of health condition(s). Patients perception of efficacy in managing his/her health. Evidence that other health care providers refer to and use information documented at presurgical visit which impacts patient education. Evidence that patient is an active partner in deciding what education is provided and demonstrates teach back to the clinician. Evidence of a safe, shame- and blame-free environment. Patient questionnaire Physician questionnaire

6.4 Improved/consistent surgical patient education along continuum.

6.4.1 To what extent has the program impacted health education in other surgical services received by the patient?

Patient questionnaire Staff questionnaire Direct observation

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

10

L.E.A.R.N.S. First 5 Surgical Patient Education Program Data Collection Matrix

Evaluation Topics 6.6 Reduced health care costs and improved efficiency. Evaluation Questions 6.6.1 How has the program impacted health care costs? 6.6.2 How has the program impacted health care efficiency? 6.6.3 To what extent does the program impact these outcomes? Indicators Costs associated with average length of stay Costs associated with medical interventions in hospital and ambulatory care Increased number of surgical opportunities/ efficiency per year Data Sources Financial data and reports

SEEKers L.E.AR.N.S. First 5 Surgical Patient Education Program Data Collection Matrix, March 24, 2014

11