Evaluation of West University Timisoara

From EiWiki

This report is the result of the evaluation of West University of Timisoara. The evaluation took place in 2012 and 2013 in the framework of the project “Performance in Research, Performance in Teaching – Quality, Diversity, and Innovation in Romanian Universities”, which aims at strengthening core elements of Romanian universities, such as their autonomy and administrative competences, by improving their quality assurance and management proficiency.

Such evaluations are taking place within the context of major reforms in the Romanian higher education system, and specifically in accordance with the provisions of the 2011 Education Act and the various related normative documents. Whilst institutional evaluations are taking place in the context of an overall reform, each university is being assessed by an independent team, under the authority of Institutional Evaluation Programme.

Contents 1 Evaluators 2 Self-evaluation Process 3 Reporting 3.1 Governance and Institutional Decision-making 3.2 Teaching and Learning 3.3 Research 3.4 Service to Society 3.5 Quality culture 3.6 Internationalisation 4 Recommendations 5 Conclusion: Capacity for change

Evaluators

The evaluation team (hereinafter named the team) consisted of:

• Öktem Vardar, Rector, TED University, Turkey, team chair
• Richard Lewis, former pro Vice-Chancellor, Open University, United Kingdom
• Antoni F. Tulla, former Vice-Rector of Economic Affairs and Administration, Universitat Autònoma de Barcelona, Spain
• Olav Øye, student, Université libre de Bruxelles, Belgium
• Lil Reif, researcher, Danube University, Austria, team coordinator

During the two visits, the IEP team talked to approximately 90 staff members and 60 students from six different faculties, including larger faculties such as economics and business administration, political science, philosophy and communication science, faculty of sociology and psychology as well as small faculties like the physics, arts and design and music. Furthermore, the team had the chance to talk to representatives from research centres to support departments at central university level and to external stakeholders from cultural institutions and the business sector. These discussions together with all written information (self-evaluation report, appendixes, additional information) form the information basis of this report.

The team thanks Prof. Dr Marilen Pirtea, Rector of WUT, for inviting us and devoting his time to us. We would like to thank Prof. Dr Madalin Bunoiu, Vice-Rector for academic strategy, and his team for the very effective and smooth organisation and coordination of the visits. We thank Dr Dan Luches and the whole Department of Quality Management team as well as everybody who worked hard to prepare a good self-evaluation report and the additional information the team requested. Also, the team would like to thank all those who gave time to meet and have very open discussions with the IEP team.

Self-evaluation Process

This was the second time the university had undergone an IEP evaluation: in 2010, WUT had undergone an institutional evaluation based on a decision of the senate in 2009. Regarding this evaluation, occurring in the context of the project “Performance in Research, Performance in Teaching – Quality, Diversity, and Innovation in Romanian Universities”, it needs to be mentioned that there has been a misunderstanding regarding the format of this evaluation, as it is not being considered a follow-up evaluation from 2010 as expected by WUT management, but as an entirely new evaluation in the framework of the project mentioned above. Consequently, the self-evaluation report focuses on the recommendations from the first evaluation and how these have been addressed by the former and current management team (SER 2012, p. 3).

The self-evaluation process was undertaken by the self-evaluation group consisting of 12 representatives from all faculties of WUT, including student representatives. The group was supported by the Department of Quality Management (D.Q.M.) and the rector’s office in order to collect all necessary data for the last three years. Contributions from the self-evaluation group were collected and a draft report was set up by the coordinator on this basis, which was then discussed during the team meetings, when the final analysis and conclusions were agreed on. The final version was presented in and approved by the senate.

Regarding the content and quality of the self-evaluation report, the evaluation team acknowledges it being open and very self critical, pointing out clearly some of the problems.

One of the main criticisms from the team’s viewpoint is that the report includes only a few analytical points. It could have been helpful to structure the report as mentioned in the guidelines provided by IEP. Then, the question on the “How” (“How is the institution trying to do it?”) would have been further emphasized and made the report more comprehensive and thus even more useful for internal use and communication during these times of change. A SWOT analysis was missing in the self-evaluation report, but was provided at a later stage by the self-evaluation group, based on input from the faculties.

Also, there was some confusion among the IEP team about the different documents of reference and how the statements — be it missions or strategic and operational objectives — relate to each other, namely those given in the WUT charter, the strategic plan and the rector's management plan. The team learned from the coordinator of the self-evaluation group that this is a result of the changing environment at WUT at the moment, causing some inconsistencies in this respect because of the ongoing process of confirmation of the documents: for example, WUT’s charter underwent two changes: first, due to the new legal situation (2011) and again only shortly after due to the new rectorate in 2012, the latter being already close to the preparation of the self-evaluation report and including changes in the mission statement of the university. In sum, there have been different changes at different levels and for different reasons and alignment of documents apparently will need some more time.

In general, the team had the impression that large parts of WUT’s community (from the Council of Administration to academic staff and students) are familiar with the self-evaluation report, who agree that it reflects quite well the real situation of the university.

In the light of this evaluation being understood as a follow-up evaluation, the team wanted to know how the results of the last report have been used. According to the coordinator of the self-evaluation group, after the last evaluation a plan on activities was set up in 2010 and was approved by the senate later on. Parts of this plan are an integral part of the current rector’s management plan, so there seems to be continuity in this respect, despite the changing environment.

The self-evaluation report of WUT was sent to the evaluation team in October 2012; after this, some additional documents (mainly, a translation of the charter, a more detailed chapter on quality assurance in the self-evaluation report) were provided shortly before the visit. The site visits of the evaluation team to Timisoara took place from 14 to 16 November 2012 and from 27 to 30 of January 2013, respectively. Between the visits WUT provided the team with some additional documentation on quality assurance and further information on the mechanisms of teaching and research evaluation, a translation of the rector’s management contract and some clarifying information on the university’s yearly budget.

Reporting

Governance and Institutional Decision-making

more on Governance and Institutional Decision-making at WVT

As mentioned, a new rector and management team were elected or appointed at WUT in 2012. According to the SER, the new management team holds a holistic view on education, research and administration, trying to integrate all the faculties, thus allowing a deeper collaboration and permeability between them, addressing one of the recommendations from the IEP evaluation in 2010 (SER 2012, p. 2).

Teaching and Learning

more on Teaching and Learning at WVT

It is stated in the SER that the student population (Bachelor and Master) decreased by 29% between 2008 and 2011, whereas the number of Bachelor programmes increased from 77% to 89%; that of Master programmes from 124 to 136. The team was told that there is a high and complex interdependency of state policies for state subsidies per student, academic promotion and the results from the evaluation of study fields, which makes it difficult to plan teaching and learning. Nevertheless, it is clear to the team that there are more programmes at WUT than it is possible to run properly. Also, the team was told that WUT plans to close 10 programmes, focusing on those which have been ineffective and lowly ranked, which do not bring added value or lack attractiveness. Furthermore, WUT’s management plans to restructure the study programmes in order to better fit the needs of potential students and the labour market, thus becoming more attractive on the higher education market. The team supports this and recommends using this opportunity to look into the possibility to use staff time more effectively.

Research

more on Research at WUT

As mentioned previously, improving research is the priority of the WUT leadership, conceived as a necessity, not an alternative, expected to bring income and prestige. As it reads in the strategic objectives of the institutional strategic plan: "Creating a competitive research environment at a national and international level, in order to transform WUT into a centre of excellence focused on generating knowledge through scientific research, development, innovation and artistic creation, with a direct impact on the quality of teaching and services provided for the socio-economic environment".

Service to Society

As it has been mentioned above, WUT sees its mission — generating and transferring knowledge to society, through research and education — deeply interlinked with the city of Timisoara, Timis county and Western Romania. The institutional strategic plan states as strategic objective for the area “WUT in society” that it firmly states the strategic role of WUT in the economic, social, cultural, civic and moral development of local, regional, national and international community. The SER states that WUT remains “deeply anchored in the society hosting scientific and social debates, concerts, exhibitions and is involved in promoting Timisoara as a European cultural capital” (SER 2012, p. 7).

During its visit, the team witnessed the strong links between the university and its surrounding society. There is a high esteem for WUTs’ role within the city and the region, which includes its role for the transborder DKMT Euro region together with Hungary and Serbia and the respective universities in Szeged and Novi Sad. The team could observe that WUT is seen as an important qualifying institution for workplaces in the region (for example, 70% of the school teachers in Timisoara are graduates from WUT). High importance is given to relationships with companies in the region. Furthermore, there are lots of links and seemingly very good relationships with public and private institutions (the team could see this especially in the field of culture). Interactive activities with society are diverse: internships, participation of representatives outside the university in the evaluation and development of study programmes, realisation of joint programmes, realisation of research programmes with or for partners from outside the university (for example a project with prisoners) and many more. One important area of collaboration is with the school sector, as WUT organises further education and didactical preparation for school teachers.

The team congratulates WUT for its strong commitment and obvious engagement with external stakeholders. In order to use the full potential of these strong links, recommendations given in relation to governance (chapter 2) and teaching and learning (chapter 3) should be repeated here: establish mechanisms involving external stakeholders for the strategic development of the university as well as for the further development of study programmes, thus ensuring that courses are relevant to practice.

Quality culture

It is stated in the WUT charter that it aims to assume and promote the principle of quality in all the activities conducted, by developing and applying the evaluation procedures of these activities. (WUT Art. 8, h).

According to the SER and some additional material on quality management at WUT, the most important unit to assure quality at WUT lies in the Department for Quality Management (DQM). This central unit supports the entire quality management system of WUT, which means, in practice, coordination of evaluation activities carried out at the level of the faculties. The DQM is subordinated to the vice-rector for academic strategy. It comprises at present one director and four permanent employees. Its work is supported by:

• 11 commissions for quality management at each of the faculties, comprising members from the teaching staff, students and employers (altogether 101 people: 35 academic staff, 33 employers and 33 student representatives);
• Committee for quality management, academic evaluation and accreditation at WUT Senate and the
• Commission for Quality Evaluation and Assurance, which is a working team responsible for emerging problems regarding quality assurance at WUT level. It comprises six members (three teaching, one union representative, one employer, one student).

Many quality-related activities have been carried out since the establishment of DQM in 2008. For example, WUT conducts teaching evaluations each semester, in which all courses are evaluated by students. Another activity is the annual internal audit done at the level of the faculties and coordinated by the DQM. At present, the team was told that DQM is working on the second edition of WUT quality handbook in order to adapt it to the vision of the university’s new management. Also in progress is the Quality Management System’s Regulation of organisation and operation. Both documents comprise the framework of quality assurance at WUT and need to be approved by the Senate. Also, the team has been informed that attempts are being made to develop a code of quality.

In the light of the many activities, bodies and documents involved, the team found it difficult to understand how all the activities, structures and grids relate to each other. It seemed that there are many evaluations taking place, but ownership, leadership and coordination needs further improvements. It is clear that WUT has to meet the different requirements set out by the Ministry of Education and ARACIS. Still, the team wishes to emphasise in this respect what has been stated in the Berlin Communiqué of the Bologna Process: “The primary responsibility for QA in HE lies with each institution itself […]”. It is true that WUT has to be in line with ARACIS and related regulations — just to give an example — but WUT should further develop its own internal quality assurance system. Here, the team recommends the WUT to make use of the European Standards and Guidelines for Quality Assurance in the European Higher Education Area.

As to evaluation procedures, these seemed somewhat mechanical with the main purpose to prepare documents for internal and external authorities. According to one from the DQM, estimate, about 75 percent of the work of the DQM is devoted to fulfilling reporting/procedural requirements set by ARACIS and/or other national regulation. This leaves little space for own initiatives. The rest of the work on quality assurance is devoted to the design and analysis of student questionnaires, mainly devoted to the quality of teaching staff, not reflecting for example on the students’ own contribution towards more quality in teaching and learning. Furthermore, the team gained the impression that different approaches (and questionnaires) are used to evaluate quality of teaching and learning, but also for the evaluation of research at WUT (see example music and sociology).

To the IEP team it appeared that little effective use is made of the results of these questionnaires. Students receive little or no feedback from their completion of the questionnaires, or do not notice any changes. Therefore they might not take the completion of questionnaires seriously. The IEP team was not able to obtain specific examples on how WUT is “closing the loop”, bringing improvement based on evaluations. This missing link from “check” to “act” has been reported by some of the students as well as some academic staff. The team learned from interviews with students that the national student union had fought to get the reports on the evaluations of professors published. This may be controversial, but one way of solving it could be to make evaluations less focused on the teacher, and instead refer to results of the course as a whole. This would also reflect the principle of academic staff and students being partners in quality assurance (Art. 7 of WUT charter).

The team learned that some faculties practice evaluation of teaching done by colleagues: this approach of peer evaluation in the field of teaching seemed very valuable and should be explored further, as it relates also to staff development, promotes networking and exchange of experiences between academic staff, being of high value for supporting younger colleagues and fostering a shared feeling of responsibility for the learning outcomes of study programmes.

Internationalisation

Strategic objectives in the field of internationalisation are according to the SER: integration of WUT into international academic and research networks, internationalisation of study programmes, enhancing international communication and mobility, setting up partnerships with foreign universities, integration of WUT in international associations and consortia. During the second site visit the team was told that internationalisation is priority number 2 for the leadership. More concretely the team was told that WUT wants to attract more students from abroad (especially from EU countries), as it is seen as having potential to attract fee-paying students. Currently, WUT is developing an internationalisation policy.

The commitment exists: up to now, the Department for International Relations (DRI) comprises six staff members, supported by Erasmus coordinators at faculty level as it is practiced in many universities. The team has learned of at least one case where the faculty has established an office for international relations with coordinators at faculty and department levels, to give more ground to the many activities related to internationalisation. Also, the team was told that WUT has recruitment agents in India, Pakistan and some African countries. The number of partnership agreements signed with foreign universities increased by 64% from 2008 to 2011, having now reached a total of 275 bilateral agreements with universities from 26 countries.

Despite this, staff and student mobility has not changed much during the last four years. Here, the issue of pre-financing mobility, lack of information within WUT on possibilities for mobility and a lack of language skills are perceived by DRI as major limitations. It has been reported that information available on study programmes in languages other than Romanian is still lacking and the university needs to make information on study programmes, fees etc. accessible to students from abroad. Also, the SER describes that WUT has not fully used its potential to attract, for example Romanian speaking students from neighbouring countries. In order to attract more students from abroad, WUT intends to increase the number of study programmes which are taught in languages other than Romanian.

The team talked to both international students and international academic staff. The team learned that WUT offers Romanian language courses to foreign students preparing for studies at WUT, but also for one of the other public universities in Timisoara. Also, Romanian students expressed the need for support in language learning in order to prepare for studies at foreign universities and the SER refers to lacking language skills among the administrative staff for improved international communication. Despite all of this, there is no special structure facilitating language learning so far at WUT. The team recommends to improve opportunities for language learning in order to support students and staff to prepare for mobility.

According to WUT management at central and faculty level, it has been said that the transferability and acceptance of the ECTS from abroad is ensured. Nevertheless, it appears to the team that further flexibility in curriculum design is necessary, as it had been mentioned in the self-evaluation report and from students that there still are cases where recognition of credits/learning outcomes from studies abroad is not granted. Some students therefore have to take additional exams. Also, some uncertainties have been reported by students on how and where the process of recognition is done, and who is responsible within WUT for this.

The team sees the potential of WUT and recognises the efforts carried out in order to improve student and staff mobility, integrating it into the other objectives and activities of WUT. In addition to measures mentioned above, the team recommends WUT to monitor graduate destinations and study programmes systematically, to benchmark with other universities from abroad and set objectives for international presence (percentage of students, academic staff or researchers, grants obtained through international sources etc.). Also, it is recommended to use feedback from mobile students and researchers (both incoming and outgoing) in order to promote mobility and further develop and improve the work of the DRI.

Recommendations

Governance and institutional decision-making

• Mission and objectives should be narrowed down and prioritised. Unless WUT decides what not to address, it will be difficult to match ends and means.
• Mechanisms should be established in order to ensure the university’s strategic future, involving external stakeholders.
• WUT should ensure that it has an action plan with priorities, timing, responsibilities, resources, performance indicators and an accompanying detailed and realistic financial plan and should be reviewed on an annual basis.
• Faculties’ action plan should be consistent with WUT plan.
• Planning infrastructure should pay attention to non-laboratory studies.
• Drastic lack of dormitories is affecting enrolment numbers and should be a priority for WUT as well as for the city of Timisoara.
• Attention should be given to the accessibility of university buildings for students and staff with reduced mobility.

Teaching and learning

• Reduce the number of study programmes. Avoid overlaps in study programmes and courses. WUT should look into the possibility of using staff time more effectively.
• Ensure ECTS credits reflect student workloads.
• Analyse learning outcomes carefully, and reflect in particular on the difference between knowledge (repetition of facts/theories/practices) and skills (the student’s ability to analyse/apply the knowledge to a given situation/problem).
• Ensure that the emphasis on research is not at the expense of T&L.
• Reduce one-way lecturing in favour of active/interactive methodologies which put the students at the centre of teaching activity. Use criterion-referenced assessment techniques.
• Introduce a faculty development programme to enhance teaching competencies of the existing faculty to shift from teaching to learning.
• Especially in the light of the reduction of full-time students, the university should consider the opportunities of increasing its part-time provision.
• Make courses relevant to practice.
• WUT management should carefully analyse the reasons for dropout. Dropout rates and time-to-degree should be monitored.
• WUT should consider ways of providing more effective Career Advisory services.
• Improve language-learning opportunities in order to obtain greater benefits from the internet and related resources, including MOOCs.
• Prepare all students for research, and involve students in research, so that they at least get acquainted with research processes of not in research content. Here, some good examples already exist and WUT can make use of them.

Research

• Focusing/concentrating/choosing strategic areas may be a possible solution for the university. Vice-rector for research should assume a proactive role in this process.
• Be aware of problem-focused, interdisciplinary research called Mode II research (Gibbons). Also, the university should recognise that there are a number of models for research activities, not all of which involve external project funding.
• National research assessment is slow (2007 results reported in SER); therefore, more dynamic assessment at the institutional level should be devised.
• Create internal research funding (seed money), institutional incentives (matching funds), supports like project writing/grant applications office (PAID) and make the implementation transparent.
• Try to attract researchers and teaching staff from other countries (and other Romanian regions) and use goal-oriented staff mobility, in order to get fresh perspectives on the WUT’s practices, and thus avoid inbreeding.
• Consultancy as a source of income and providing service to society (archaeological issues, assessment of prisons) should be kept as viable options.

Quality culture

• Quality Assurance requires an “overall design” and “operational guidelines”. As to the overall design, consideration needs to be given both to compliance to standards and fitness for purpose.
• Institutional, systematic monitoring of teaching quality is needed rather than relying on the self pride of academics. Not only the “use” of evaluation/assessment exercise but also its impact (consequences) should be emphasised. Quality assessment should lead to quality enhancement. Use European Standards and Guidelines on a systematic basis for quality assurance of the work and activities in WUT.
• WUT should review its current system of the use of student questionnaires to evaluate the effectiveness of teachers. Make sure that all quality activities follow the “closed loop” principle (“plan – do – check – act”). In particular, make sure that student questionnaires for every course lead to change, or give the students an explanation of why their feedback from questionnaires will not/cannot be followed up.

Internationalisation

• Improve the opportunities of language learning in order to support students and staff to prepare for mobility.
• Monitor graduate destinations and study programmes systematically.
• Benchmark with international institutions. Set objectives for international presence (% of students, academic staff or researchers, grants obtained through international sources).
• Use feedback from mobile students and researcher to further develop and improve international activities.

Conclusion: Capacity for change

The team sees that the new rector and his team would like to improve changes in order to make WUT more competitive and more efficient, something which could also be observed in some faculties we visited. According to the interviews with faculty members and students, it can be said that leadership exists and has a wide support within the university, that internal communication is good and serving as an important basis to develop the university further. Also, the mechanisms of collaboration between the different management structures seem to work very well at WUT and have acceptance within the institution; it is felt that with the new management team the base of decision-making has been enlarged. Debate, communication and open discussions make the faculty and students feeling part of the change.

Furthermore, the decision-making appeared effective to the team. Although the terminology used is decentralisation and increasing authority of deans and heads of departments, it contains healthy elements of both collegial and managerial systems. It emulates search committees and appointment systems and goes well with current trends. The assignment of vice-rectors to each faculty improves communication between the central and faculty level, thus integrating different interests. The same goes for the approach of holding meetings of the Council of Administration at the faculties on rotational basis.

The team was somewhat concerned with the high relevance put on research, with less emphasis placed on quality in teaching and learning. As a comprehensive university located in the western part of Romania, WUT also has a high responsibility for high quality study programmes; after all, this is also one of the sources for funding. Thus, high quality teaching should also be seen as something that helps to gain both income and prestige for the institution, especially, if it comes to the field of university continuing or postgraduate education. This is also of importance as the change towards more research activities among academic staff will certainly take some time.

Strategic management and change is possible only if the legal and administrative environment is supportive of autonomous initiatives of the university leadership. WUT will benefit from a system-wide increase in institutional autonomy, accompanied by a high level of accountability. The heavy load imposed on WUT by ARACIS does place considerable constraints on the university’s capacity to improve its internal quality assurance measures.

To sum up the team would like to share one final observation: it observed an impressive team spirit both among staff and students, which might reveal to be one or even the driving engine for the university’s future success.