Case Study

QUT Gains Operational Visibility Into Mission-Critical Systems to Lead in Big Data Scholarship

Executive Summary

Queensland University of Technology (QUT) is one of Australia's foremost institutions of higher learning. The Brisbane-based university is known for its industry links and applied research in a variety of specialized areas. QUT needed a big data solution that could handle an extensive variety of data sources and provide students, staff and faculty with flexible options for data visualization, research and operational insight.  Since deploying Splunk Enterprise, the university has seen benefits including:

  • Flexible support for a diverse range of university users and IT systems
  • Significant risk mitigation
  • More robust and cost-effective services
    • Gain deeper understanding and insight into operational, academic and research data
    • Reduce risk, ensure compliance and improve data security
    • Provide foundation and tools for the study and application of big data analysis and practices 
Business Impact
    • Several legacy vendor and in-house systems retired
    • More robust and cost-effective services for students, faculty and staff via facilitated data sharing
    • Significant risk mitigation through consolidated monitoring and investigation capabilities
    • Flexible support for diverse range of university users and IT systems
    • Single pane-of-glass access to and analysis of data, including more than 50 billion indexed log entries
Data Sources
    • Hardware
    • Hypervisors
    • Operating systems, network equipment, storage arrays
    • HPC batch system
    • DBMS
    • Security controls
    • Frontend services
    • Environment sensors
    • Social media and mobile apps

Why Splunk

Splunk found a home at QUT in 2010 when the university needed a better way to manage the huge amounts of data generated by and required by its new student and academic management system (SAMS), as well as the school’s main portal, QUT Virtual. QUT needed to gain in-depth operational visibility into the performance and usage patterns of a range of services, plus the ability to perform root-cause analysis and anomaly detection using analytics, uncovering notable events hidden in its enormous datasets. In addition, to reinforce its stature as a research university, QUT saw the need to provide a big data platform for academic use.

Splunk Enterprise enables IT staff to analyze logs and gain detailed views into components, performance and potential issues related to SAMS and QUT Virtual. For instance, load testing the applications enabled the team to determine when bottlenecks might undermine performance. "QUT has a long history with the Splunk platform and as we emphasize big data strategies, we're expanding its use," says a university IT project manager. "Today, Splunk software is ingesting around 200GB of data daily.”

Unlocking unlimited potential

The Splunk platform and its role at QUT has been expanded over the years to include the collection of data from every layer of the university's infrastructure, including services and applications, authentication mechanisms, security controls, database management systems, high performance computing (HPC) network equipment, storage arrays, virtualization hypervisors, operating systems and physical hardware.

For example, QUT is capturing data from services such as SAMS, QUT Virtual, helpdesk, lecture recordings, Microsoft SharePoint and Exchange, the Blackboard Learn system, Media Warehouse and multiple websites. Teams from across the university can search through more than 50 billion indexed log entries to visualize and correlate data in customizable dashboards and reports, gaining a comprehensive view of operations.

In addition, the university maintains an instance of Splunk software exclusively for teaching and research. "Our staff, students and researchers are eager to extract knowledge from data through visualizations," says a QUT systems specialist. "By enabling our colleagues to correlate, contextualize and apply analytics to information from disparate sources, the Splunk platform offers significant opportunities for data-driven decision-making."

“The value of Splunk software is only constrained by the diversity of data given to it and the questions asked of it.” 

Systems Specialist, Queensland University of Technology

End-to-end operational visibility and tracking

Splunk software is used for a range of operational activities across various service support teams. Splunk alerts provide staff with insight into notable events, including performance interruptions or changes. With a consolidated view of the infrastructure and services, contextualized reports can be produced and issues investigated from a single point.

QUT has also enhanced its Splunk deployment with apps from the Splunk ecosystem. For example, the IT communications team uses dashboards and reports built into the Splunk App for Microsoft Exchange to monitor metrics, logs and other data from Microsoft Exchange and its associated services. Analysts can view metrics for operational performance and capacity planning, and audit security events.

"Because of the extensive visibility it delivers, Splunk software is now a vital part of our IT toolkit," explains the systems specialist. "Our teams depend upon the solution in their day-to-day work, using it whenever they require visibility into their own environments as well as QUT's infrastructure as a whole."

Big (data) plans

QUT is using Splunk software to teach big data analysis techniques, enabling students to gain in-demand industry skills. For example, the university will analyze data collected from hundreds of building management sensors installed in its new science and technology facility. "The Splunk platform enables the university to lead in big data scholarship," concludes the IT project manager. "With it, we're teaching students to make data-driven decisions. Splunk software is a powerful research and learning tool for this new era in the information age."

"The Splunk platform enables the university to lead in big data scholarship. With it, we're teaching students to make data-driven decisions. Splunk software is a powerful research and learning tool for this new era in the information age."

IT Project Manager, Queensland University of Technology