Have you ever put together some flat-pack furniture and found yourself with several spare pieces that are not supposed to be spare according to the instructions? It’s extremely frustrating. You just know that bolt was vital for something, but everything “looks” complete and functional. So, you ask yourself, “do I really want to spend hours reverse engineering what I just did to find where those missing parts belong?” You now run several risk scenarios in your head – if it’s flat pack furniture my first one is the spouse finding out. But you just spent hours doing this, you thought you followed the instructions and you don’t have time to backtrack, so you let the missing parts slide. There is absolutely no problem, until you put some weight on that furniture and it crumbles like a house of cards.
Why the long analogy? Because it reminds me so much of change management. I recently came across a scenario where there was some change management associated with a new network hardware component, but none linking that component to other critical systems that I was presented with. Along comes a vulnerability scan that identifies highly rated vulnerabilities in the solution and then all of a sudden there is blind panic – because not only did it lead to a non-compliant state but also an internal incident to be managed.
The problem was they started down the right path, like following instructions on building flat-pack furniture, but they had some missing parts they didn’t account for. It may have seemed like a lot of work accounting for other systems, running scans after a major change and having a back-out plan, especially with a pandemic going on and working away (remotely) from the systems you are trying to implement. But I can tell you now they are putting in three times the work trying to account for the existence of the vulnerabilities, missing change components and now an internal incident investigation. If there was ever a reason for trying to get something right the first time, this was probably one of those occasions.
But even if you don’t quite get the implementation right, that is what change windows and back-out plans are for. Nothing should have been connected to that new component outside of change management processes, which should include review and sign-off of all steps, including the back-out plan.
Good change management is really due diligence. It’s not only vital in avoiding incidents, it’s also vital in ensuring a secure solution is implemented and tested. It should also dictate what should be updated in documentation so that the next change, on the same system, isn’t hamstrung from the beginning by bad documentation. It’s bad enough if we are missing a vital part the first time around, it’s something new all together to leave an incorrect instruction manual behind as well. Lessons learned should be acted on.
Basically, the implementer of a change should have a “bouncing ball” of instructions to follow and if a gap is evident there should be no question – back-out. If you let that missing piece slide it’s a pretty good chance you will end up dealing with an incident rather that a change. Testing procedures in the approved change plan should help identify those gaps but don’t ignore new ones that appear outside of those procedures either. Common sense rules. Don’t wait for some pressure on the system to see if it will collapse like a house of cards or not.
Regardless I have yet to see a consistent set of perfect change plans over any length of time. You can’t account for everything. But if you follow through precisely on an approved plan, use common sense and make the right no/no-go decision, knowing that you have a clear back-out plan if needed, then that’s all that can be asked of you. You can either do the change right or deal with an incident. The choice is yours.
Brian is the Director of Asia Pacific Global Compliance & Risk Services Consulting at SecureTrust, based in Sydney. He has over 33 years IT industry experience including roles as a Security Delivery Manager and Global Security and Transformation Lead for large worldwide information technology corporations. During his career he has been across a wide range of industries and roles, including global management experience across multiple cultures and business environments. Experienced in running global security programs, and some of the largest regional projects in Asia Pacific, Brian brings a mix of project management, security and compliance credentials together (CISM, CRISC, PMP, QSA, CDPSE, ISO27001 IA) to achieve the best results in delivering security solutions and compliance programs. He has been published by the Project Management Institute (PMI) and by MSSP Alert, presented at industry conferences and conducted webinars on topics like the General Data Protection Regulation (GDPR) and Compliance Intelligence.