Storage monitoring, or storage performance monitoring, is the practice of tracking the performance, availability and overall health of physical and virtual storage devices. Data storage (provided by vendors such as Dell, EMC, Microsoft as well as countless others) is the backbone of all types of computing endeavors, whether you’re working on a spreadsheet, checking your email or playing a video game on your Xbox over the internet. In fact, there is virtually no mainstream computing operation that does not require access to some form of storage.
While it is an essential technology component of IT infrastructure, storage can be problematic. Network storage is invariably the slowest part of a computer or server, and — depending on the specific hardware technology — it can create significant computational bottlenecks, particularly when many users are attempting to access the same data at once. (This kind of overload is one of the primary ways in which distributed denial of service (DDoS) attacks operate.) Storage devices, particularly traditional hard disk drives, are prone to failure, as parts wear down and eventually fail. Storage devices will also eventually become full, requiring expansion or upgrades, often on an ongoing basis. In short, every computing environment requires high performance and availability. The best practice for doing this is to implement storage monitoring software or a storage monitoring tool.
It’s also important to note that in the modern enterprise, storage is increasingly virtualized, abstracting the physical location of data away from the user via a cloud platform. Storage virtualization can be used to archive or back up data, web or application services, or (increasingly) for web-based email and productivity suites such as Google Docs.
In this article, we’ll discuss the different types of enterprise storage, how storage monitoring works and what to look for in a storage monitoring solution.
What Is Storage Monitoring: Contents
What is cloud storage monitoring?
What should you look for when monitoring storage performance?
What are some best practices for storage monitoring?
What are the different storage monitoring tools?
What are the benefits of storage monitoring?
What are the potential challenges of storage monitoring?
How do you control data growth?
What are some storage monitoring parameters to be aware of?
NAS (network-attached storage) and SAN (storage area network) are the two primary forms of on-premises networked storage. The names sound quite similar, which often leads to confusion, but they are separate technologies.
NAS refers to a hardware device that is connected to an enterprise’s local network. A NAS device typically works over ethernet or a similar wired connection and is designed to be simple to set up and cost effective. If a business runs low on storage space, it can obtain a NAS device and plug it into the network so it becomes available to everyone. NAS devices can contain multiple drive bays that include failover technology like mirroring or RAID, although some NAS devices are simple enough so they can be repurposed for home use.
Rather than representing a single device, a SAN is a network of storage devices. These devices connect to a dedicated network off the ethernet LAN, generally using the storage-centric fiber channel technology to transfer data to client computers. Because SAN is more expensive and complex than NAS, it is often reserved for applications where low latency — and zero downtime — are critical. Video editing and surveillance video recording are common applications for SAN, demanding high throughput and low latency for the massive amount of data being transferred. Because this data is transferred on its own private network, SAN is able to avoid congestion on the LAN and maintain a sustained, high rate of data transfer.
In summary, SAN is much faster, highly scalable and designed for high-end operations, but it is more costly and requires significant expertise to manage along with a private fiber channel network. NAS on the other hand is an inexpensive, simple technology that can easily be set up with a laptop that operates over your existing LAN.
Cloud storage monitoring, such as AWS storage monitoring, is the process of observing, reviewing and managing storage systems within cloud infrastructure, generally implemented through automated monitoring software that provides access and control over the entirety of the cloud infrastructure from one centralized location. Automated performance monitoring techniques can gauge availability and analyze performance, tracked by metrics that can include the number and type of users, database performance, server response times, resource levels, system and process performance, as well as security issues and other disruptions. Administrators can review the operational status and health of cloud servers and components and look for system disruptions or suspicious activities. The ability to continuously evaluate these metrics can give organizations insight into storage system issues or vulnerabilities long before bigger and more damaging problems arise.
Storage performance comes down to answering a few key questions about how your storage devices are running, as well as network performance and other performance issues. These include:
What are key storage monitoring performance metrics to track? How can you show storage is performing well?
Storage performance uses metrics to answer the above questions. The key storage monitoring performance metrics include the following:
All of these metrics can be sampled in real time, analyzed as an average over a certain time frame (the last hour, for example) and graphed in visualizations over longer periods of time. The IT department monitors these statistics and watches for any troubling trends that might be developing. High-quality storage monitoring software as part of your regular management software suite will make this easier by using intelligence to inform the analyst as to whether something requires attention.
Here are some of the most widely-cited best practices for storage monitoring in your environment:
Storage monitoring tools have matured in recent years and those specifically designed for the enterprise now offer a broad set of features. Products will likely fit into one or more of these categories:
A properly implemented storage monitoring solution offers myriad benefits to the enterprise. Here are some of the primary advantages:
In complex environments, storage monitoring can become more challenging than simply ensuring hard drives are operating according to specifications and aren’t becoming full. Some of the possible challenges that a storage monitoring team might face include:
The following are ways you can reduce and minimize the explosive growth of storage data in your environment.
The key storage monitoring parameters are in line with the aforementioned storage monitoring metrics that include latency, throughput, IOPS, utilization, queue length and available capacity, to name some of the most critical factors. These parameters can and should be measured at various levels of storage infrastructure, including the storage array level, storage pool level, volume level, LUN (logical unit) level and disk level. These five levels describe increasingly granular storage systems, ranging from the largest and broadest group of storage devices — the storage array — to the smallest and most specific, the single disk.
When monitoring the storage environment, all of these operational categorizations need to be considered separately. Metrics running at the storage array level may indicate the entire array is operating according to specifications when looked at broadly, but upon drilling down to the disk level, an analyst may find that a single disk may be overtaxed and in need of attention.
Ready to get a better handle on your storage infrastructure? Here’s how to get started with a storage monitoring solution:
The Bottom Line: Staying on top of your storage systems is essential to the success of your environment
Storage simultaneously represents the most essential and least reliable technology component in the enterprise. It’s also an absolute constant: Although the degree of its importance changes from one organization to the next, no modern business and its associated end users can function without access to reliable storage. Without it, any level of computing simply becomes impossible. Storage monitoring is not nearly as complex an endeavor as other aspects of IT management, but it is nonetheless one of the most essential pieces of the puzzle. When storage is involved, it’s a case of not if but when an error, outage or downtime will occur. Failing to invest in a capable storage monitoring solution will put your business at risk.