Feedback to HESA on the latest Graduate Outcomes announcements

On 15 October 2018, Naomi Oosman-Watts, AGCAS Data Insights Director, invited comments from Heads of Service on the following: 

  1. What (if any) are your concerns about the Graduate Outcomes Subscription Model?
  2. What will the impact of the model be on your service and/or your institution?
  3. Do you have suggestions as to how it could be improved?

Responses were received from 16 member services, as well as a collective response from Heads of Service in the South East Region (SEARCH). These services represent institutions across both England and Scotland.

Across all responses, the timing of the model has caused significant issues within the sector owing to the actual costs not being provided until the final weeks preceding the survey launch. There has been a distinct lack of information to warn or prepare institutions, [we] have concerns that institutions are in a position where they must seemingly accept major changes, have little control or say, but also appear to be expected to carry the risks.

Through the responses it was felt that HESA should be bringing the survey in-house, as originally intended.

 1. What (if any) are your concerns about the Graduate Outcomes Subscription Model?

The significant concern about the new model is the costing structure. Member services reported increases of up to 300% on previous costs. This exponential increase in costing was not what many were led to believe, I attended … many HESA communication events and publications surrounding GO and it was always communicated that the subscription model would be aligned to responses gained. Many felt there was an unjust assumption that no additional costs would be incurred by services in promoting the survey, analysing dashboards and contacting graduates.  

The lack of correlation between the actual costs and response rate is particularly problematic when considering costs are accrued through the population value and not on response rate, especially given there is no incentive for anyone collecting the data to get a completed response.

A number of members expressed concerns regarding bounce rates directly influencing cost rates, especially where the HEI has a large international student population. This treats all students within the population as the same: a distinction in costings between UK, EU and international would perhaps be more suitable here.

Another major concern with the model is the same costs being applied to both online and phone responses, especially given the comparative costs of the mediums, and the significant change in how graduates respond. With this it is unclear where these additional costings will lie.

Overall, services were under the impression that the model would be value for money. Recurrently, members reported feeling disappointed and let down about this.

2. What will the impact of the model be on your service and/or your institution?

The primary impact on member services is financial, with 78.5% highlighting that they will now have to find money to cover the unexpected costs that have come from the model. Concerns as to where this money will come from (salary saving was continuingly listed) are significant given that there will also be no notable reduction in the workload of the services – despite the model being outsourced.

There was also a deep concern that despite not being accountable or responsible for these costings, services will suffer reputational damage within their HEIs as a direct result of their implementation. Overall, response rates will be lower as there will not be funds, or time, for dedicated staff to oversee the process.

Concerns were also raised about the subsequent use of data obtained through the model, especially where it is used in league tables. Here data will become a reflection of whom can afford the additional support and input needed to make the collection successful, rather than a full representational dataset.

3. Do you have suggestions as to how it could be improved?

Heads consistently noted they wanted greater transparency, greater detail, realistic costings and pricing to be based on responses rather than population. 

  • Greater transparency How can we be confident that our substantial fee (circa £35k) is being well spent by a third party who we had no part in selecting?
  • Greater detail relating to mechanisms or processes that will assure quality and maximise value for money.
  • Realistic costings having two price points, one for an online submission and one for telephone submission with a clear overview of how the £6.75 is spent.
  • Pricing to be based on responses calculate costs based on response rate not on population – it would incentivise responses.