Once the Assessment Plan is completed, academic departments and administrative units collect the evidence following the measurement methodologies defined for each LO and objective. This may include evidence collected directly by the department and evidence collected within the unit/departmental dashboards. Members of the unit, department or program then analyze this evidence collaboratively and draw conclusions. On the basis of these conclusions, the unit plans its future actions. At the same time, unit members verify the state of advancement of actions planned in the previous assessment cycle.

As in the case of assessment plans, assessment implementations may vary significantly in content and scope; however, in order to ensure that they can be easily shared amongst all stakeholders, a standard template for reporting the results of assessment processes has been created by the Office of Assessment, Learning and Institutional Research. The template requires describing the evidence collected, the conclusions drawn on the basis of this evidence, and the actions planned. The template also contains a section in which the unit can specify planned actions that do not result from the assessment process, but, for example, by contingent situations (e.g. replacing a member of personnel, an unexpected increase or drop in enrollment, etc.). Finally, the template includes a section for reporting on the state of advancement of previously planned actions. The templates used by administrative units and academic departments to systematize and organize their 2010-2014 assessment implementations can be seen on the Academic Departments and Administrative Units assessment pages. These however, will likely be revised after this first implementation; the most up-to-date version of the templates is available on the units assessment pages.

Instructions

Once units have prepared their assessment documents in June of each year, they send them to the Provost and to the Office of Assessment, Learning and Institutional Research. The Dean reviews the documents over the summer, with assistance from the University Assessment Task Force, and prepares an annual Assessment Report on institutional assessment-based analysis and planned actions. The report is discussed with the Leadership Team during the September planning retreat.  Recommendations are also made to units with respect to possible improvements of assessment processes, appropriateness and feasibility of planned actions, eventual interactions between planned actions of different units, challenges and opportunities with respect to student learning outcomes and institutional effectiveness.

University Assessment Task Force

In order to bring together faculty and administration in the sharing of assessment results, an Assessment Task Force[1], formed by both faculty and administrative staff, assists the Dean in: (1) giving AUP’s faculty and staff a better understanding of assessment processes;(2) ensuring that assessment is meaningful and useful; (3) ensuring that assessment is appropriately integrated in all university processes including shared governance processes; (4) envisioning appropriate assessment methodologies for those aspects of AUP’s education which are currently not adequately assessed (see 2015-2020 Objectives); and (5) reviewing assessment documents at the end of each academic year and making recommendations to the President and the Leadership Team for implementation of actions, and to all University units for improvements to their assessment processes.

Assessment Communication

Communication is essential to the assessment process in order to:

  • Create a common understanding of what assessment of student learning and institutional effectiveness are and the methodologies that can be used to plan and implement assessment
  • Create a common understanding of how assessment-based planning at one level of the University informs planning and assessment at other levels
  • Share information about assessment (both assessment plans and assessment results) amongst University units to create a better inter-unit understanding of objectives, methodologies, difficulties, and achievements
  • Share experiences (celebrate achievements and honestly confront difficulties) with the assessment process

The current perception of assessment processes at AUP is fragmented in terms of awareness, opinion, and methodologies.  In the past, assessment was mostly shared in a bottom-up fashion[2], with limited interaction between peer units or from top units towards the bottom. While bottom-up sharing of assessment is essential for contiguity assessment and assessment-based planning, peer-sharing and top-down sharing would contribute greatly to increasing units’ understanding of the objectives and needs of other units (possibly reducing disputes over resource allocation and generating occasions for profitable collaborations) and to improving institutional assessment processes through peer learning. Effective assessment communication is essential for these types of sharing processes to take place.

Another important aspect of assessment is its historical basis. Although it is possible to assess student learning and institutional effectiveness on the basis of an analysis over a semester or academic year, substantially more significant results may be obtained by analyzing learning and administrative processes over a more extended period of time. Creating adequate digital structures supporting institutional memory has the potential of improving the quality of assessment processes while also making them more efficient and informative. Further, changes in personnel and roles (e.g. a new chair of a department) would be facilitated by the existence of a repository of systematized, organized, and digitized assessment data and analysis.  

University Outcomes Assessment Day

Sharing of assessment results begins at the end of each academic year with an Outcomes Assessment Day open to the whole University community and attended by the President, Leadership Team, and heads of all departments, programs and units. During the day, those who have led assessment for the various University units report on demonstrated achievements and problems; they focus, in particular, on assessment-based planning for the unit. Assessment leaders also highlight lessons learned during the assessment process, addressing measurement methodologies that have been particularly fruitful or difficult to implement as well as activities that have been more difficult to assess or for which particularly interesting results have been uncovered.

The objective of the June University Outcomes Assessment Day is threefold: first, getting members of different units to gain a better knowledge of each other’s work, plans, challenges and achievements; second, sharing assessment best practices and supporting each other in the continuous improvement and adaptation of assessment processes; third, provide the President and Leadership Team with an immediate overview of the plans of each unit and the resources that will be requested to implement these plans, de facto initiating a new cycle of the Planning-Implementation-Assessment process on an institutional level.

AUP Assessment Website

As mentioned earlier, in order to facilitate intra- and inter-unit communication, information about past assessment efforts has been reorganized into standard templates. These reports are available to all faculty and staff on this assessment Web site[3].  This assessment Web site is updated yearly with the new outcomes assessments and  more frequently with assessment resources and highlights of achievements. The structure of the site may also be revised to provide an increasing number of resources, connect it to units’ assessment archives, increase usability and manage visibility of information which is currently all made available with the same privacy level (all faculty and staff can read the information and the head of the unit can update it).  We will be increasingly making some of our assessment information public by establishing appropriate privacy and access levels. 

Dashboards

Department chairs have access to dashboards displaying relevant academic indicators about student numbers, classes, faculty, etc. On long-term plan is to have dashboards at every level of the University:

  • Personal dashboards (accessible to individuals only) collecting information about people’s activities over time; faculty members, for example, may see the courses they have taught, the papers they have published, the committees they have served on, the students they have advised, the grades they have given as against institutional averages, the evaluation of their courses, etc.[4]
  • Academic departments’ dashboards collecting aggregate information about students and professors in the department, as well as, for example, about courses and grades.
  • Administrative units’ dashboards collecting information relevant to the specific unit; for example, the dashboard of the Internship Office may contain information about students, employers, and internships.
  • Leadership Team dashboard containing aggregate information about all aspects of University activities, processes, and stakeholders.
  • Institutional dashboard containing a selection of the information available in the Leadership Team’s dashboard and accessible to everyone.

The creation of still other dashboards will also be considered, so as to make available relevant information to students (both personal and for various associations and student governance groups), parents, prospective students and their parents, and the Board of Trustees.

Integration of Assessment in Institutional Processes

The utmost value of assessment is its provision of decision-making tools for institutional planning at all levels: from learning unit design, to administrative activities organization, from curriculum development to strategic planning. In order for decision-making tools to be effective, it is essential to establish processes that, on the one hand, collect quantitative and qualitative measures of all University activities, and, on the other hand, produce information that is relevant, readable and timely for the planning activities.

For the strategic planning process, as well as for other planning processes across the University, we have begun, and will continue, to identify which indicators are needed, in which format and at what time in each process.

Planning and streamlining information collection, benchmarking and forecasting, all contribute to the sustainable creation of relevant, readable and timely information.

 

[1] The Assessment Task Force will be appointed by the Dean during the Spring 2015 and Fall 2015 semesters. The task force will define more precisely its own charter and status (whether it should continue to be an appointed task force or become an elected committee) and its mode of functioning.

[2] Assessment of units lower in the organizational chart - academic programs, for example – have been sharing assessment with units higher in the organizational chart – e.g. academic affairs.

[3] https://www.aup.edu/about/aup-recognition/assessment-learning

[4] The possibility of integrating this personal dashboard in the personal faculty pages currently available will be considered