UniSA Employer Feedback Survey
External Report
“Development of the
survey tool has been
informed by a need
to widen the range of
data used to evaluate
teaching and learning
outcomes beyond
student and graduate
satisfaction surveys.”
Educating Professionals
Creating and Applying Knowledge
Engaging our Communities
Summary
UniSA has conducted an Employer Feedback Survey
as part of the Australian Learning and Teaching
Council’s (ALTC) Teaching Quality Indicators (TQI)
project. The need to develop an indicator to
measure the veracity of the UniSA mission of
‘Educating Professionals’ and ‘Engaging our
Communities’ underlies this pilot project.
The specific aim of the pilot project was to design,
trial, evaluate and report on an Employer Feedback
tool and process, to access feedback from key
industry stakeholders and graduate employers in
data collected about Graduate Qualities and other
learning outcomes. The project has also responded
to a desire to collect this type of information at a
discipline level, something not done in surveys of
this kind in Australia. Development of the survey
tool has been informed by a need to widen the
range of data used to evaluate teaching and
learning outcomes beyond student and graduate
satisfaction surveys. Employer satisfaction data
complements graduate information collected
through the Australian Graduate Survey (AGS) and
Graduate Destination Survey (GDS), supporting the
measurement of UniSA Key Result Areas in
Teaching and Learning, as well as in Community
and Industry Engagement.
Locating the supervisor of a recent UniSA graduate
to complete the survey without being able to
provide the name of the graduate was a major
obstacle in data collection. This limitation needs
to be overcome to make future iterations of the
Employer Feedback Survey viable, sustainable
and transferable as a quality indicator. However,
significant strengths and areas for improvement in
the achievement of particular Graduate Qualities
have been highlighted as providing initial points
for attention.
Recommendation 1:
Invite employers and respondents who have
expressed interest in attending a launch of the
public report in early 2009.
Recommendation 2:
From the next Australian Graduate Survey
administered by UniSA, begin asking recent
graduates permission to use their details for
future research.
Recommendation 3:
Implement improvements to the survey instrument
including necessary changes required through
use of the graduate name in future surveys,
as identified in the survey improvement register.
Recommendation 4:
Closely monitor engagement activity across the
University to identify relevant opportunities to
inform UniSA employer groups about the survey,
and how they can be involved. This includes
sharing information across the Careers Team,
Marketing and Development Unit, schools and
the Community Engaged Scholar project.
Recommendation 5:
Conduct the improved UniSA Employer Feedback
Survey in the second half of 2009.
Recommendation 6:
Utilise the information gathered in informing
improvements in curriculum design.
UniSA Employer Feedback Survey External Report Findings
Ultimately, data output from the Employer Feedback
Survey tool will be most significant for use in
measuring employer satisfaction with graduate
attributes, and subsequently inform policy and
curriculum decisions. There are a range of process
outcomes of the Employer Feedback pilot project,
in addition to the results of the survey data.
This includes the significant improvement of
asking graduates for permission to use their details
for future research when they are surveyed as part
of the AGS. It is perceived that having a sample of
graduate names that have identified their employer
and given consent to use the information will
reduce the time and resources required to contact
employers, as well as the length of the survey.
The data output of this pilot cannot be considered
of sufficient quality to yet inform specific UniSA
indicators. However, it has provided institution
wide points to monitor and track through to future
surveys, and a number of these priorities have been
identified in relation to specific Graduate Qualities.
Key survey improvements will, from the next
iteration of the Employer Feedback Survey in late
2009, achieve the necessary data quality to form an
important, efficient indicator of graduate outcomes
at discipline and program level. This data will provide
real guidance in informing teaching practices to
improve achievement of Graduate Qualities, as well
as building industry and community feedback into
these practices where relevant.
Results
The Employer Feedback Survey data collection
period commenced mid-May and continued until
the end of August 2008. In that time, the survey
team sent 333 emails containing the survey web
link and a unique PIN to supervisors of recent
graduates, who had agreed to complete the survey
and provided their personal email for that purpose.
Of these, 189 submitted completed surveys (56%)
form the basis of the following analysis. Data from
the 14 partially completed surveys was not used.
Profile of respondents
Respondents were from a range of organisations:
• 39% with 201 or more employees
• 46% from private enterprise
• 25% were from government departments
the majority from state government departments
The broad industry categories represented in the
survey map largely onto major faculty areas of UniSA:
• Business and services (32.2%)1
• Health (30.2%)
• Education (15.9%)
• Industry, mining and trades (14.8%)
2
Respondents reported that graduates were from
a range of UniSA discipline groups, and these
reflected the larger of the 20 discipline based
schools at UniSA. The highest response rate related
to graduates from the School of Health Sciences
(14%), then Commerce (10%), Education (9%)
and Computer & Information Science (8%). This
spread is across the four divisions – the top four
response rates by discipline represent the largest
school in each Division. The Divisions were
represented roughly proportionately:
• Education, Arts & Social Sciences: 89 responses
• Business: 75 responses
• Health Sciences: 59 responses
• Information Technology, Engineering
& the Environment: 48 responses.
“This data will provide real guidance
in informing teaching practices to
improve achievement of Graduate
Qualities, as well as building industry
and community feedback into these
practices where relevant.”
Most respondents had recruited fewer than
10 graduates since 2005. Answers given through
the survey related to either one or more graduates,
with multiple answers permitted for various
questions, including what disciplines graduates
studied in (Q7), qualifications gained and the work
role(s) graduates were employed in. For example,
a respondent could have been considering three
graduates as they completed the survey, and
indicated they graduated from the schools of
Commerce and Communication, though the
number from each discipline group was not
requested. This survey question regarding graduate
disciplines studied provides an overview of the
distribution of graduates, and relies upon the
respondent knowing their discipline. This element
was unable to be cross tabulated with other
relevant questions due to multiple answers being
permitted, and this aspect of the instrument limits
in depth analyses. The use of specific graduate
data in future surveys will provide more reliable
and flexible data about graduates by discipline.
The work functions in which UniSA graduates
were employed in fell largely into the ‘professional’
category (55%). This is to be expected as UniSA
trains graduates for the professions, including
Nurses, Teachers, Engineers, and Pharmacists.
The category groups were based on expanding
categories in a similar survey Monash University
1
This category includes computer, research, communications services
has conducted. Upon reflection, a survey
improvement would be to equate graduate work
functions with ABS and GDS work role groups.
These are closer to standard groupings, and will
allow for benchmarking of future surveys beyond
Monash, and this is identified as a change for the
next iteration of the survey in 2009.
Graduate Qualities
Twenty four graduate qualities statements were
listed for respondents to indicate the importance
they placed on each and the performance of the
UniSA graduate/s they have supervised in that
quality. This section formed the major part of the
survey, and data collected gives a picture of the
specific attributes employers value (Importance
scale) in graduates, and their perception of how
UniSA graduates demonstrate them (Performance
scale). The results provide clues to specific points
where UniSA is achieving what it says it does in
its Graduate Qualities and other significant areas
where graduates are not meeting employer
expectations. This information is a useful insight
into employers’ thoughts and perceptions of UniSA
graduates and Graduate Qualities2. Future iterations
of the survey tool will better achieve the objective
of gathering employer data on Graduate Qualities
at the school level. Heads of Schools and Program
Directors can use the results regarding general
perceptions of UniSA graduate performance of the
qualities employers consider important, and future
data will provide further detail relating to specific
schools and programs. There is also the possibility
this information can be used to benchmark with
other UniSA schools and across the sector.
“This information is a useful insight
into employers’ thoughts and
perceptions of UniSA graduates
and Graduate Qualities.”
Strengths and areas
for improvement
Comparing employers’ importance and
performance ratings allows an analysis of UniSA
strengths and areas for improvement. The highest
priority areas for action, revealed by the largest
gaps between employer importance rating and
performance rating pertain to three Graduate
Qualities:
• ‘Written skills’ and ‘Oral communications’ (GQ6)
• ‘Ability to apply knowledge’ (GQ1) and
• ‘Solve problems’ (GQ3).
This represents an opportunity to consider how
UniSA Employer Feedback Survey External Report
development of these qualities in particular can
be improved at UniSA.
‘Communication’ (GQ6) and ‘Problem solving’
(GQ3) form two tenets of the Employability Skills
Framework (2002a). This framework formed the
basis of investigation into how employability skills
are taught and assessed in Australian universities
in the Business, Industry and Higher Education
Collaboration Council (BIHECC) ‘Graduate
Employability Skills’ report (2007). The framework
contains eight generic capabilities or skills required
“not only to gain employment but also to progress
within an enterprise so as to achieve one’s potential
and contribute successfully to enterprise strategic
directions”(BIHECC 2007, Appendix 1).
The attributes with the smallest gap between
the importance and performance ratings may be
viewed as a good match of employer expectations
and graduate performance. These are also
considered areas of strength for UniSA:
• ‘International and Multicultural perspectives’
(GQ7)
• ‘Indigenous Perspective’ (indicator of GQ1)
• ‘Teamwork’ and ‘Gets on well with Colleagues’
(GQ4)
In addition to rating the list of qualities,
respondents were also asked to name any other
attributes they value when employing graduates.
A third of respondents replied positively, and
proceeded to elaborate on these through an open
text field, and these comments can be mapped
upon the Graduate Qualities, for example:
“Ability to understand core concepts and apply
them in different situations” (GQ1)
“The ability to have insight into their own
strengths and weaknesses to inform their
professional development” (GQ2)
This option did not provide any new quality or
attribute that could not be aligned with one of
the UniSA Graduate Qualities.
Sub-group analysis
Various sub-groups were explored to identify
significant differences in importance and gap
means across the groups, such as:
• Type of organisation
• Number of employees in organisation
• Industry
• Number of university graduates employed
since Jan 2005
There were no significant differences in the
importance means recorded by number of
employees in the organisation or number of
university graduates employed since January 2005.
3
2
www.unisanet.unisa.edu.au/gradquals/
UniSA Employer Feedback Survey External Report Importance means for professional role (as defined
in the survey) versus non-professional role differed
significantly for the following statements:
• Oral and written communications skills
• Capacity to develop knowledge
• Solve problems
• Apply knowledge
• Business performance fundamentals
and teamwork.
Analysis of the gap in importance rating and the
performance of UniSA graduates between
professional and non-professional work roles
demonstrates that while there are consistent gaps,
there are no significant (greater than 0.5)
differences. The overall importance and
performance mean was lower for those graduates
in a non-professional role, however.
A significant difference was recorded within the
‘Type of Organisation’ sub group, between
government and private employers in relation to
the statement on written skills. These two groups
also recorded significant differences in the
importance mean in relation to understanding
different viewpoints, understanding the
fundamentals of business performance, Indigenous
perspectives, ethics and social responsibility,
and initiative and enterprise.
Developing Graduate Qualities
Almost three quarters of all respondents agreed
with the statement ‘We employ UniSA graduates
confident that they will readily meet our
requirements’. Those who disagreed with this
statement commented
“The graduate was already employed prior
to commencing their studies at UniSA”
“Personal attributes were an important
consideration during selection”
These results recognise one issue accepted by the
project team from the commencement of the survey
– that many attributes would be difficult to ascribe
solely to the UniSA degree when other life
experiences would also bear upon the job-readiness,
or otherwise, of graduates. It is particularly true for
mature age students, though the extent is difficult
“Almost three quarters of all
respondents agreed with the
statement ‘we employ UniSA
graduates confident that they will
readily meet our requirements’.”
4 UniSA is committed to imbuing these in its
to measure and will vary among graduates.
Employers acknowledge ‘life experience’ and other
factors in preparing graduates.
“It is a combination of aspects. Previous work
history and qualifications. Also, they were
believed to be a good cultural fit”.
Employers recognise and are willing to contribute
to the success of UniSA graduates. Over 80%
of employers responded that they had provided
additional on or off the job training to new
graduates. When asked what responsibility
employers have to contributing to the job-
readiness of new graduates in their employ,
common responses across both questions included
the recognition that employers needed to provide:
• Induction and orientation
• On-the-job training
• Job-specific training
• Mentors, shadowing or networking opportunities
• Occupational Health and Safety training.
In addition, employers noted the importance of:
• Managing performance and expectations
via performance management
• Inducting new employees into the organisation’s
culture, procedures, policies and value system.
It is clear that, to varying levels, employers consider
they do have a role in developing graduates and
Graduate Qualities.
Discussion
Notwithstanding acknowledgement of their role
in further preparing graduates, many employers
have high expectations for graduates
“We like graduates who are confident but
not arrogant, creative, enthusiastic, diverse
in their interests, well travelled, with good
social skills, well read, ambitious, well
educated, interesting and with a plan for
where they want to go.”
It is debatable as to whether it is the role of
universities to provide all of these things, but
higher education institutions need to be able to
support their claims with evidence. The feedback
about the performance of graduate attributes
by employers can provide significant insight into
whether UniSA develops what it says it does in
graduates. It is also an important gauge of industry
and employer expectations. Comments about
qualities/attributes employers consider important
beyond those in the list provided in the survey
have been analysed and they largely map onto
one or more of UniSA’s Graduate Qualities.

“The feedback about the
performance of Graduate Qualities
by employers can provide
significant insight into whether
UniSA develops what it says it does
in graduates. It is also an important
gauge of industry and employer
expectations.”
graduates, and this survey is a tool in assessing
the success of this commitment. It provides focus
for improvements in wide gaps between employer
expectation and graduate performance. Many
respondents mentioned specific technical
and industry knowledge as well as more generalist
work skills as desirable, adding some qualitative
clues to many of the attributes ratings listed
“Cultural fit with business. Conflict resolution
skills, excellent communications skills, time
management, approachable, decisive.”
“Understanding of ethics involved, especially
when working with vulnerable individuals.”
“Ability to establish relationships at a mature
level.”
While acknowledging that Graduate Qualities
will not necessarily be solely attributable to UniSA
curriculum and learning outcomes, particularly
with regard to mature age students for example,
it is important that the qualities remain relevant to
graduates, the community and industry. Qualities
that the University values as outcomes for its
graduates need to be identifiable and measurable
if UniSA is to confidently state that graduates will
possess each of the seven Graduate Qualities. The
Employer Feedback Survey will support this aim.
UniSA’s Graduate Qualities were cited specifically
by a couple of employers in response to how the
University prepared graduates, and many of the
individual qualities were mentioned.
“The UniSA graduate qualities serve us well”
“Ability to problem solve. Ability to apply
theory to practice. Communication Skills.”
The benefit of, or need for, placements and
UniSA Employer Feedback Survey External Report
practical or “real world” experience was
mentioned numerous times in response to both
what UniSA did and what it could have done
better to prepare graduates for employment.
It is seen as an area of strength by some
respondents (17%) and an area for improvement
by others (25%). Looking more closely at the
comments by discipline in future surveys could
provide information about where the strengths
are, and identify specific disciplines or programs
that require attention with regard to experiential
learning. Analysing this data in future surveys in
terms of the size of organisations could also provide
insight into the needs of a range or organisation
types. Future iterations of the survey will be
instrumental in monitoring changes over time,
and congruence with the Teaching & Learning
Framework.
Survey improvements
The analysis in this report has been limited by
aspects of the instrument’s overall design and
in some cases feedback from employers has
highlighted questions with poor face validity.
This is sometimes evident in responses to open-
ended questions. The use of multiple response
questions also made it difficult to determine the
relationship between some cohorts of respondents.
In addition, the categorisation of industries and
occupations is inconsistent with approaches
commonly used elsewhere (eg ABS, AGS).
Many of these problems will be alleviated with
key changes to the administration of the survey
and the questionnaire implemented. Being able
to identify a specific graduate as the subject of the
survey and put this together with UniSA held data
will have a significant effect on the efficiency of
the Survey administration and the accuracy of data
gathered. A Survey Improvement Register has been
established containing specific information about
changes required to improve the survey process
and questionnaire.
“Qualities that the University
values as outcomes for its
graduates need to be identifiable
and measurable if UniSA is to
confidently state that graduates
will possess each of the seven
Graduate Qualities. The Employer
Feedback Survey will support
this aim.”
5
UniSA Employer Feedback Survey External Report The Survey included some questions about the
instrument itself, and many respondents stated an
improvement would be to provide a mid way point
in the Lickert scale used to rate the importance and
performance of graduate attributes. The elimination
of this point was a deliberate decision to ensure
respondents selected high or low in the scale,
reducing responses clustered around a median.
Future surveys will need to consider the relative
value of each position, in combination with the
wording of several of the attributes statements.
This would take place in a context of the overall
review of changes and implementation of
improvements to the survey questionnaire and
process. This has commenced with graduates being
contacted for the AGS and asked to consent to
their employer being contacted by UniSA for future
research.
“By incorporating the identified
improvements the Survey becomes
more accurate and sustainable as an
indicator at UniSA, as well as more
transferable across the sector.”
Sustainability and transferability
The feasibility of building upon the survey pilot
project and implementing the improved survey
tool and process needs to be considered in light
of institutional aims and priorities, and the extent
to which the Employer Feedback Survey contributes
to achieving these priorities. Results from the pilot
survey have revealed gaps in the achievement of
some Graduate Qualities, identifying opportunities
for improvement and locating strengths in the
delivery of particular Graduate Qualities. However,
the survey instrument cannot currently provide
the data to accurately track the achievement of
graduate outcomes, and significant improvements
to the survey tool and process are required. The
sustainability of the Employer Feedback Survey
within UniSA assumes these improvements will
be implemented and clear steps towards making
these improvements have been identified, the first
commencing in December 2008. By incorporating
the identified improvements the survey becomes
more accurate and sustainable as an indicator at
UniSA, as well as more transferable across the
sector.
Institutional indicators
The improved Employer Feedback Survey would
6 supplement data gathered about graduate
outcomes through the Australian Graduate Survey
(AGS) with further information regarding the
qualities valued by employers and possessed
by graduates. As the survey instrument evolves
through efficiency and accuracy improvements,
it will provide data to support the measurement
of UniSA’s first Key Result Area in Teaching and
Learning
The University’s prime focus is the education
of professionals – applied programs of study
responsive to the needs of industry which
increasingly provide ‘real world’ learning
experiences designed to prepare students
for employment (UniSA Strategic Plan, 2007)
The performance dimension for this KRA includes
“graduate outcomes in employment and in
full-time study measured by the AGS and the
Graduate Destination Survey” (UniSA Strategic
Plan, 2007). Information gathered by UniSA as
part of this national survey includes program and
demographic information, employment or study
status at the time of the survey in addition to
attitudes about the graduate’s experience of
their program of study. Elements of this survey
are already important benchmarks across the
higher education sector and are also linked to
funding through the Learning and Teaching
Performance Fund. At UniSA, at least one of the
seven Graduate Qualities is a mandated, assessable
component of each course, evaluated through
the Course Evaluation Instrument (CEI). With the
established use of institutional sets of graduate
attributes that universities offer their graduates
and through which they distinguish themselves,
measurement of how and how far graduates
possess these is important.
Teaching and Learning
Framework
The Graduate Qualities are at the centre of UniSA’s
Teaching and Learning Framework3, as is the
commitment to embedding experiential learning
“The University’s prime focus is
the education of professionals
– applied programs of study
responsive to the needs of
industry which increasingly
provide ‘real world’ learning
experiences designed to
prepare students for employment”
(UniSA Strategic Plan)
3
www.unisa.edu.au/ltu/staff/start/framework/default.asp
in all programs. The survey tool will potentially
provide a measure of the success of this strategy
as it is rolled out in coming years. In reaffirming
a commitment to the seven Graduate Qualities
the Framework recognises
that these generic attributes are considered
by professional associations and employers
as critical employability skills and attitudes
and as such, central to the achievement
of our mission as an applied university.
(Lee, 2007)
The Framework reinforces evidence as a basis of
judgement, with employer satisfaction being one
measurement of graduate impact and the importance
of gathering data to “inform judgements about
future activity.” (Lee 2007).
The survey sits alongside key performance and
national indicators such as the Generic Skills scale
and Course Experience Questionnaire (CEQ) and
the Educating Professionals scale of the GDS.
The requirement to benchmark graduate attributes
and other outcomes across institutions beyond
these scales supports the further evolution of
the survey tool and process to continue to make
efficiency and accuracy improvements. In this
way refinement for the next survey will allow
for transferability across institutions. There is
the potential to segment questions into core
and optional groups, core questions being those
that are universal across the sector. Potential ‘core’
question include:
• Ratings of graduate attribute: How important
are these qualities? How do UniSA graduates
rate?
• In what ways did the University prepare the
graduate well for employment?
• What could it have done better?
• What is the role of the employer in improving
graduate work-readiness?
This projected development of the tool to
determine core and optional questions is aimed
at benchmarking amongst those institutions that
adopt the Employer Feedback Survey tool. With
a flexible online survey, optional questions could be
developed and included to enhance local versions.
A recent University of Western Sydney employer
survey asked what trends employers could foresee
(Grebennikov and Shah, 2008), which may be
a useful addition in anticipating market changes.
In addition to supporting teaching and learning
corporate indicators, the Employer Feedback survey
supports the Regional, Industry and Community
Engagement strategy as outlined in the UniSA
Strategic Plan. The ‘Highly engaged education and
research’ Key Result Area requires “a high level
of engagement with industry, government and
the professions and (is) relevant to community
needs”(UniSA Strategic Plan 4, 2007). The survey
instrument would form an important tool for
community and industry stakeholders to give
UniSA Employer Feedback Survey External Report
feedback about how well UniSA is “developing
in graduates professionalism and good citizenship.”
An overall engagement strategy that contributes
to improvements in industry and employer data
held by UniSA, improved tracking of engagement
activities and an holistic approach to external
stakeholder involvement in the life of the institution
will facilitate access for employers to participate
in the survey. The Community Engaged Scholar 5
project is one internal project that would be a
useful link in this process. Building ‘expert’ groups
and keeping up to date information about current
and potential survey respondents will also allow
better access to UniSA for these important
stakeholders.
Frequency and response
to the survey tool
The costs and resources required to conduct
future iterations of the Employer Feedback Survey
will determine the frequency of administration, as
will internal planning and quality cycles. The Survey
tool will serve as a measure for various institutional
indicators prioritised in the UniSA Strategic Plan
and Teaching and Learning Framework.
Monash University and The University of Western
Sydney have both recently conducted second
iterations of their employer survey. Monash, who
did not make any changes between running the
survey in 2003 and 2007, perhaps unsurprisingly,
did not find significant differences in responses in
the survey. The 2007 data has prompted high level
engagement with employers, commencing with a
launch event for the report to thank those involved
and build relationships, alumni and fundraising.
Other ‘spin off’ events, to include University staff,
graduates and employers, will be held with their
10 faculties to share more detailed results and
work towards connecting better with employers.
UWS, who released details of their most recent
report after the project conducted an audit of
surveys early in 2008, has responded to the
employer feedback information they gathered
in a range of ways. Strategies included improving
alignment of its programs with regional needs,
setting up a school of Medicine for this purpose
and increasing offers for places in Early Childhood
education programs. This activity indicates the
wide range of responses – and resources required
to respond – to information gathered from industry
stakeholders.
4
www.unisa.edu.au/pas/qap/planning/StrategicPlan/StrategicPlan.pdf 7
5
www.unisa.edu.au/comunity/community.asp
UniSA Employer Feedback Survey External Report Conclusion
The value of the Employer Feedback Survey pilot
has been in the identification of key improvements
that will enable the tool and process to become
viable and sustainable as a quality indicator from
its next iteration at UniSA, and ultimately
transferable across the Australian Higher Education
sector. Also, data gathered substantiates UniSA’s
committment to internationalisation.

To ensure sustainable outcomes, the University has

commissioned Planning and Assurance Services and
the Learning and Teaching Unit to manage the
Employer Feedback Survey into the future.
Supported by a comprehensive reporting
environment, academic and professional staff will
be able to triangulate information from multiple
sources to identify areas for improvement.

References
Business, Industry and Higher Education Collaboration Council
(BIHECC) (2007) ‘Graduate Employability Skills Report’,
September, www.dest.gov.au/highered/bihecc

DEST, (2002a) ‘Employability Skills for the Future’,

www.dest.gov.au/sectors/training_skills/publications_resources/
other_publications/employability_skills_for_the_future.htm

Grebennikov, L. & Shah, M., (2008) ‘Engaging Employers

with the University: Skills Needed and Changes Expected by
Industries’, Australian Universities Community Engagement
Alliance National Conference Proceedings

Centre for Higher Education Quality, Monash University (2007),

‘Employer Survey 2007, Part 1: Australian based Employers
Summary Report’

Lee, P. (2007), Teaching and Learning Framework, University

of South Australia

University of South Australia: (2007) Strategic Plan

http://www.unisa.edu.au/pas/qap/planning/StrategicPlan/
StrategicPlan.pdf

Narelle Walker
Project Officer: Learning & Teaching
February 2009

8 For further information:

narelle.walker@unisa.edu.au