Benchmarking

MIT recently completed a benchmark study on the use of educational technology in subjects considered to count toward the core curriculum (General Institute Requirements). This presentation will discuss how the data was collected, analyzed, and implemented.

This session will summarize the highlights from the 2006 EDUCAUSE Core Data Service survey data. These analyses will identify key findings, important trends, and similarities and differences across different types of institutions. This year more than 940 institutions participated in this collaborative effort. A discussion period will conclude this session to understand concerns and issues from the audience, as well as discussion of future directions.

Determining what to measure, how to measure it, and to whom to report the results can be more of an art than a science. In this session, four major research universities will address how to communicate the importance of IT to administration, outreach, teaching and learning, and research.

This session presents findings from the 2007 ECAR survey on IT help desk management in the U.S. and Canada. Using assessment methods derived from IT service management standards and the broader IT help desk literature, ECAR has developed a robust view of the state of the practice in this highly significant service area.

EDUCAUSE Core Data Service Fiscal Year 2006 Summary Report summarizes much of the data collected through the 2006 EDUCAUSE core data survey about campus information technology (IT) environments at 933 colleges and universities in the U.S. and abroad. The report presents aggregates of data through more than 100 tables and accompanying descriptive text in five areas relevant to planning and managing IT in higher education: IT Organization, Staffing, and Planning; IT Financing and Management; Faculty and Student Computing; Networking and Security; and Information Systems. Appendices include a brief historical context, a list of participating campuses, the 2006 survey instrument, a glossary of terms from the survey, and Carnegie classification definitions.

The CERT CSIRT Development Team has introduced a method to evaluate and improve an organization's capability for managing computer security incidents. This method uses a set of incident management best practices defined in a set of metrics called the Incident Management Capability Metrics. These metrics provide organizations a baseline against which they can benchmark their current incident management processes or services.

The metrics questions explore different aspects of incident management activities. These questions are grouped into four basic functional categories:

• Protect
• Detect
• Respond
• Sustain

The results from an evaluation using the metrics will help an organization determine the maturity of its incident management capability regardless of organization type or sector (commercial, academic, government, etc.).

Higher education institutions undertake a range of approaches to evaluating and making judgments about
the quality of their e-learning provision. This paper begins by exploring benchmarking as one current
strategy in common use in universities to identify and implement quality practices: from the use of
checklists (for example, of best practices and standards) to a more contemporary dynamic systems approach involving continuous cycles of feedback and improvement centred around the learners' experiences of elearning.

Reporting quality and showing the value of an IT organization is a challenging proposition. The old saying you can't manage what you can't measure is still true. Determining how to show "hard numbers" for IT services takes a good deal of thought and planning. This session will discuss one way to achieve the desired measurement objectives.

The president and CIO of Roberts Wesleyan College recently examined information technology investments and services on campus with the aid of the "Information Technology Benchmarks: A Practical Guide for College and University Presidents," a Council of Independent Colleges workbook written by David Smallen and Karen Leach. A page-by-page comparison of campus practices with the workbook's benchmarks, along with discussion of included value points, helped us measure and evaluate our IT operations. The guide is designed as a tool for senior campus leaders to help them understand some key concerns about campus information technology. The workbook's exercises raise good questions for campus leaders to discuss with IT leadership and create opportunities to address challenges. Roberts fared well against most benchmarks, scoring consistently low in the IT investment side but high in services and support deployment and the institutional value of technology. Our successes can be attributed to specific practices detailed below.