Managing human risk to the data centre
Unfortunately, human beings are the larget single point of failure in the modern data centre. Human error still accounts for a large proportion of outages, and not enough is being done to address the issue.
Findings from a recent survey from the Uptime Institute show that outages are becoming more frequent and more expensive. Now is the time for the industry to start paying attention and look for ways to reduce highly preventable downtime.
When humans are involved, mistakes are going to happen. It’s inevitable. However, simply accepting the risk that humans make mistakes is a mistake in itself, as the frequency and cost of human error-related outages continues to grow. Processes and rigorous professional development plans must be the norm.
We all know that outages are hugely costly for organisations. It’s not just the financial impact of downtime, the detrimental effect an outage can have on brand reputation, customer confidence and perceived compliance can be even more devastating.
Organisations need to learn from their previous outage experiences and help to mitigate against the inevitable human error probability by continually assessing and reviewing and enhancing the skills set of their teams. Investing in staff education, training and personal development can pay significant dividends, educated/officially certified individuals could potentially save their employer millions by doing things right the first time, and therefore mitigating the possibility of an outage, or recognising and resolving an issue early and preventing it from becoming more serious in the future.
The 2020 reveals that 78% of organisations have stated that they have had an IT-related outage in the last three years with 75% saying that their most recent outage could have been prevented with better management, making a large proportion of outages the result of human error. This figure has increased by 15% since 2019.
It's unwise for organisations to simply accept that preventable outages are an acceptable fact of life, especially when they are also growing more costly. The Uptime Institute survey also reveals that in 2020 nearly one in six outages cost more than $1mn, as opposed to one in ten in 2019. Also, an increased percentage cost between $100,000 and $1mn.
If organisations get better at spotting the knowledge, competency and skills gaps in their teams and invest to fill these gaps, whilst ensuring the processes and procedures are kept up to date, the picture could be very different.
With industry supported education programs awarding official certifications and qualifications, alongside advances in individual and team analytical tools, backed by science and psychological methodology that identifies exactly where knowledge, competency and even confidence levels are lacking, there are numerous opportunities for organisations to take important steps towards mitigating human risk.
It’s industry best practice to regularly test and monitor the lifecycle of mission critical equipment. As an industry, we service our technical equipment to check it is still functioning as expected and plan its future lifespan and renew or restore to prevent outages. The same thinking needs to be applied and in place for the teams working in data centres.
The individuals responsible for the outages are not individuals looking to sabotage, they are usually experienced members of the technical team that for one reason or another are not following processes or have knowledge, competence or confidence gaps. It’s a fact that if people have been doing the same job for an extended period, their confidence can take over and this can cause individuals to overlook details and specific processes which in turn can cause catastrophic failures – they could be confidently doing things wrong.
One of the big challenges organisations face is that continual professional development budgets are usually limited or cut to boost other areas of the organisation. There is also a common misconception about education/training allocation, as these activities are often used to provide a reward to those most loyal or high performing people, rather than those who actually need it the most. This misconception results in the employees gaining very little from the development activities and therefore provides little or no benefit or ROI to the organisation itself. The risk data centre operators are taking by not investing in their people is massive, and could cost them thousands per minute during an outage.
The Uptime Institute survey also states that with more investment in management, process and training, that the outage frequency would almost certainly fall significantly. Hopefully, this will raise alarm bells to the rest of the industry to turn their attention to these areas. The pandemic has highlighted the critical importance of the digital infrastructure industry and demand is only going to increase.
Alongside an increasing skills shortage and an ageing workforce, this is a stark warning that if organisations don’t develop, train and invest in teams effectively throughout the entire workforce, outages are likely to become bigger and more expensive.
With an aging workforce, many experienced industry professionals will soon be looking to retire. With decades of industry and on-the-job experience, are those team members that will be taking their place really sufficiently trained, experienced and ready to handle any future issues that might arise? Organisations need to address the problem head on instead of waiting for things to start going wrong.
Sustainability and PUE reduction in data centres
The data centre industry is at a crossroads. As demand for colocation, hyperscale cloud, and edge solutions continues to rise, operators and enterprises are also facing up to the reality that sustainable design and operating practice are a mission critical component of the modern data centre. Going green is no longer an optional extra.
Data centres are becoming an increasingly critical foundation that underpins the modern world, and the demand for them continues to grow exponentially each year. Data centres must remain in constant operation in order to provide the services for which customers depend on them.
This mission critical need, combined with the sector-wide push towards reduced energy consumption and carbon footprint throughout the industry, is making the search for innovative evaporative media solutions that keep systems running at peak efficiency an equally mission critical priority.
The two main sources of energy consumption in a modern data centre are its IT equipment and the cooling infrastructure used to keep that equipment cool. A 2017 study found that energy consumption as the direct result of cooling data centre IT equipment can amount to over 40% of the total energy consumption in a facility. From air cooling to liquid and evaporative chillers, data centre operators, finding the right cooling solution for your facility is a top-of-mind goal for any data centre operator.
Portacool: keeping it Kuul
Based in Center, Texas, Portacool is a portable evaporative cooling solutions firm that has been pushing the boundaries of mission critical infrastructure cooling technology since it entered the market in 1990.
Through constant embodiment of its five brand pillars - Safety & Liability, Total Cost of Ownership, Productivity & Performance, Sustainability & Social Responsibility, and Life & Comfort Enhancing Solutions - Portacool has grown steadily over the past 30 years, continually reinforcing its reputation for industry-leading cooling solutions.
Portacool’s solutions have been successfully applied throughout the agricultural and horticultural, manufacturing, industrial, business, entertainment, sports, home, and hobby industries - “anywhere cooling is needed and traditional air conditioning is impractical or cost prohibitive.”
The company’s sub-brand, Kuul, is Portacool’s answer to the growing need for reliable, sustainable cooling solutions in the data centre sector. Portacool manufactures three series of evaporative media – Kuul Control, Kuul Vitality and Kuul Comfort. Kuul Control is used in data centres, power generation and HVAC systems. Kuul Vitality is utilised primarily in the horticulture, poultry and swine industries. Kuul Comfort is exclusively made for usage in Portacool-branded portable evaporative coolers.
Kuul can help data centre operators lower their PUE dramatically, increasing the environmental sustainability of their facilities significantly as a result of its rigid evaporative media solutions.