Storage costs are relatively cheap compared to the cost of storage, but one might say that data recovery is where most MSPs have been very successful. Years back, we were challenged with costs around storage and destination fees costs, so we had to be selective about what we backed up. In today's world, the cost of storage is relatively cheap in comparison, but so many new variables complicate this process. Listen to Matt Lee of Pax8 as we dive deep into CIS Control 11 Data Recovery and explore all five safeguards.
[00:00:00] Welcome to MSP 1337. I'm your host Chris Johnson, a show dedicated to cybersecurity challenges, solutions, a journey together not alone.
[00:00:20] Welcome everybody to MSP 1337. It is that time of the month. No, it is it is fireside chat time of the month. I have very different very different.
[00:00:33] Matt Lee, Welcome to the show. It is that time to talk about data recovery control 11. We will get to control 18 of us the last thing we do in this lifetime because we've taken roughly a year to get here.
[00:00:45] We are halfway through the Oregon Trail so I can't imagine anything going bad from here forward. Well, I would like to say we're past halfway because there's 18 controls and this is number 11. So that'd be two past nine wouldn't it based on the nine. Yeah, we were on the clock this out. Yeah.
[00:01:01] So we're getting close today. Establish and maintain data recovery practices sufficient to restore in scope. Enterprise or your assets, I don't really care what enterprise stands for in your world, but your assets to a pre incident entrusted state. I think today talking through these safeguards is interesting and different from anything we've done prior to this because we have been dealing with data recovery since the inception of backups.
[00:01:29] Yeah, oh yeah. In fact, I say this in my class, MSPs will walk into 11 and go, I'm crushing 11 got it done. Yep. We've been doing that. And to some extent with all the joking aside, they kind of have right you see every MSP at least have some degree of full system backup plan for their servers. Most of those are using some type of current key system and offloading those somewhere else with state machine state information. Right.
[00:01:54] So you have at least some cursory stuff. However, to your point maybe in the beginning of this, they haven't done it planned. They've just done it. They haven't really even all of them gone down the journey of restore time objective or store point objective mapping that to business functions and times.
[00:02:07] Well, let's try to address a little bit. Let's rewind. So like 10 15 years ago, we were pretty like we had a plan right like you only have so much space that you can put data to without it breaking.
[00:02:22] Like you went out of business because you had too many tapes to put data on. Yeah, I've played the tape library game before and iron iron mountain right? Yeah, like library and I yeah, yeah, so it's interesting about that is not only did we back up what was I think critical then we did I think we did a better job 2015 20 years ago on because it was more holistic by nature of it being walled in one place as opposed to being scattered throughout the winds of cloud.
[00:02:51] Applications and sounds I think that's that's maybe two sides of this coin. So like one side says I have limited storage what's important to me. That's fair. Okay, so through that.
[00:03:04] Um, uh, scarcity creates choice in that definition. I could see that as an argument and then you fast forward with things like operating systems and bare metal or store some of those things that we used to have to back up. We don't anymore because they can be rebuilt and then virtual objects.
[00:03:21] Things to that nature right of femoral machines. They just have a data set reference things to that nature and then and then 2015 2016 happened when we started to see ransomware at a level that we'd never seen before malware at another level not necessarily ransomware but malware early on like imotech and things that nature's do spot net other extensibilities that were
[00:03:40] killing it. Restoring from back it was like, I got this if you were doing a good job. Yeah, yeah, for sure. But now we shifted in 2015 to like what was encrypted and what was damaged and what what really means restore so like a bare metal restore might be bare metal restored as something else because physically we don't know what hardware level you might have been compromised like there and the chips.
[00:04:03] And that might be taking this to hyperbulous but the point being we don't know when in time you don't know where and time the forensics weren't being done. It got me even though it's worth right yeah, yeah.
[00:04:14] And I'm being facetious but yeah, we're taking it off on a try but the point is I used to say to and Chris if you've ever said this about control 11.
[00:04:22] I used to say to my technicians and engineers as they were solving problems a lot of good mistakes can be made with a great backup right like a lot of mistakes can be made with a good restore.
[00:04:32] Yeah, that's what I mean like I can go and mess around and try something because I can restore from it right especially back in the very a decent rig and environmental center. Yeah, yeah, so and and that's why I think to your point though.
[00:04:43] MSPs at least when it was in a walled garden in a virtual machine state probably did well let's look at some of the rock solid so 11.1
[00:04:51] established and maintained a data recovery process I think I think most MSPs if we went back a decade had a process that was pretty solid because the variables were pretty consistent they didn't just change at Willie nilly.
[00:05:09] Yeah, and if you read this other paragraph here and there why is this control critical when attackers compromise assets they make changes to configurations add accounts and often add software or scripts these changes are not easy to identify as attackers might have corrupted or replaced trusted applications with malicious versions or the changes might appear to be standard looking account names.
[00:05:30] Configuration changes include adding or changing rich like they go deep into just some of the TTPs that you would have seen back this is a very dirty control years ago it is and so but the reasoning behind it is right and so let's look at this let's dig into one one you started it we've got process and that is let's put this one first because a lot of controls have I G1 I G2 I G3
[00:05:51] this literally except for one safeguard is I G1 11.5 I think is the only I G2 I G3 I G2 is not even on G3 and that does say that even though it's 11th in the state of it's 11th with the
[00:06:08] 11th with the
[00:06:10] 11th at least what is that Swansea
[00:06:13] because of all I was going to go with Elvish like is that 5 and Z drive your food
[00:06:20] even a 9 and Zane wolf is in all what's 11.
[00:06:24] I can do it on say I can't even get it right elf okay the point I'll take kind bar in the right hand.
[00:06:30] Okay, so sorry I'm having to eat and cram it all together Chris and I just needed some editing for you or not.
[00:06:35] I just want to know where we arm wrestling at CCF we have to like I might get you bro it might happen.
[00:06:41] I don't know you'll see I've been planning for this day.
[00:06:44] All right I'm excited about this all right okay so but the point is back back to this this understanding of
[00:06:50] establishing recover a process the first thing that has to happen and it has to be 11th because we have to know
[00:06:56] what data we have we have to know what software we have we have to know how we're going to manage protecting the castle in that way right
[00:07:01] and now we're not 11 11 dot 1 points back to oh geez like 2.1 like all 1.1 2.1 3 from its
[00:07:12] extensibility 3.7 even 5 and 6 because yeah all of it all of the things that have been prior to this
[00:07:19] that are impeccable in this being done well so 11 comes but it comes with all I G1's to your
[00:07:25] point which is interesting because it means this is so very important you need to do all of this
[00:07:29] now so let's tackle what that is first you have a process and it says address the scope of data recovery
[00:07:34] activities recovery prioritization comma space and this and I've used comma space right we write this in our
[00:07:40] visualizations we made each one of those is a different trapezo or a parallelogram and it says recovery
[00:07:45] part is prioritization like what's most important comma space how secure is the data like if it's
[00:07:51] security secure data I need to be backing it up in a secure way that does also include 5 who has
[00:07:56] access and is that really Matt six does it who has access right and what they're supposed to have
[00:08:01] access to okay but the point is we need to actually deal with each of those components
[00:08:05] and then also update this annually or when significant enterprise changes occur like what could you imagine Chris
[00:08:09] a change in one I change in two I change N3 a change in some of those things that are dependent upon
[00:08:14] that of what I back up and it really is 1 2 and 3 centric and then who and what are the 5 and 6 centriza
[00:08:20] and I also like this 11.1 because it uses a word that is not consistent across safeguards
[00:08:28] like there's very few safeguards throughout the use this word it says process so anytime I see
[00:08:34] process program or the inevitable policy which policy is a dead giveaway it's policy but
[00:08:42] process and program dictate that the documentation is very very clear about what's covered
[00:08:51] and what's going to be done well to your point yeah no go ahead no you go ahead to your point no
[00:08:57] you doctor doctor mouth not in my mouth you have in your mouth none my mouth no okay of your
[00:09:02] point of how many are there I think process is used something like 16 or 17 times as a control
[00:09:09] in the CIS of the 17 or 18 there's like 16 is across 153 safeguards across 153 right to your point
[00:09:17] so when they come up processes what we're talking about here and it needs to be able to deal with each of those things
[00:09:22] which is recovery how often what our recovery prioritization is like what comes first what's most important for business
[00:09:28] so you're getting into RTO and RPO now exactly what you're getting into restore time objective
[00:09:34] and restore restore hold on RPOs restore point objective and RTOs restore time the how long it takes me
[00:09:42] first what point I can restore doing time right yeah and that's a big deal right because I went
[00:09:47] through a ransomware early on 2015 and the challenge we had was to some extent less about the data we were
[00:09:54] restoring and it was more about the timeline between data restore point and what we could actually recreate faster
[00:10:02] yeah so that was back in the world of like we're storing from tape and we're dealing with things like large
[00:10:07] oh dude just arrays and and what was interesting about it at the time which granted this is eight years ago nine years ago
[00:10:14] hardware failure today remember covid when you couldn't get anything in supply chain
[00:10:21] yeah hardware failure on big enterprise infrastructure was not like you just picked up the phone and called bell and said hey
[00:10:29] ship me that emc sand with the 20 drive array yeah that was two months from now or whatever it might be right
[00:10:36] we got you were going to send you two of them and they're like oh by the way the truck the hard drives have been delayed
[00:10:41] because of the swaras can now know we just haven't built them yet right right we're waiting on Taiwan at this right
[00:10:47] yeah 100% well and so the point is 11 not one says have a process or how many of you MSPs or practitioners
[00:10:53] listening have a backup and restoration process my gut says you probably most of you do at least in some
[00:11:00] fashion in the tool is probably doing a lot of that stuff for you the execution of it but the process must be dictated
[00:11:09] somewhere else I don't want to disambiguate that problem right that that's fair but like as soon as I've
[00:11:13] configured it I may have forgotten it yeah and so think of this way one of the other things you can think about
[00:11:19] that's something we invented through the workgroup Chris is the parameterization concept parameterization and data
[00:11:25] flow and that comes down to how do we represent these interrelationships between 111 and 112 well think about
[00:11:31] 11 2 if I have 11 1 which is what it is 11 2 is a parameter for 11 1 it says hey guess what not only
[00:11:39] should you have a backup but it should be automated well I'm also say that the funny part of this control
[00:11:45] is the parameters are two through five yeah they are two through five and that's my point we don't know how to work out these
[00:11:53] are when we see it also in 14 we also see it but anyways not to be summer like we don't see it in two places
[00:11:59] yeah it's your point yeah this is like process you are working with doctor CIS at this point so I do apologize I feel
[00:12:06] like I've put enough hours in I feel like that's really a doctor like yeah it's like I'm an intent to whatever right like Dr. Mario
[00:12:13] Dr. Wario all the variants there this is Dr. I can tell you a little character in the game terrible little game yeah I want to be
[00:12:20] madly in Mario Kart why did these pills have 26 surface sides how am I going to match these up anyway also are the
[00:12:27] milligrams in this is that a safe dosage so 11 2 says perform automated backups which really says of in scope
[00:12:34] assets so it's the first thing think about what's in scope to your point earlier that's what got shifted and
[00:12:39] positioned upon when it came to being judicious of storage that was limited the other pieces run backups weekly it's
[00:12:45] the first time you have a prescription in this in this variable to say that I need to run it weekly or more frequently based on the
[00:12:52] sense state of data I don't about you bro but I was not okay to back up my clients weekly on their servers like no no no in fact and you know
[00:13:01] it's funny you see that because I remember when we got really excited about continuous backups based on file changes right and this is
[00:13:09] largely where exploitation comes in right by modify a file gets backed up by modify lots of files they get backed up so if
[00:13:18] I'm doing continuous backup at what point in time does the original become no longer available because I've done it so
[00:13:24] many times but was interesting about that is that made for the ability to back a large volumes of data because we're
[00:13:32] only backing up the changes the delta yeah the delta yeah absolutely so that has had a lot to do with the state of
[00:13:42] bread actor sent today is ran somewhere space and why the double extortion started getting created in December of
[00:13:48] June right when it was you don't need to call out the date it's that was a bad day for everybody that was a bad day but it was it was
[00:13:54] definitely the part when you start going on that path so the point is let's articulate the automated backups in the importance of this
[00:14:00] and just kind of close it out because I think we've been I don't want to say we're rambling but we're we're almost beating up people
[00:14:06] who haven't thought this through yeah it's fair it's very automated backups you need to have it where the humans aren't the
[00:14:12] failure point that's why I can't spend enough plates to keep everything backed up ask me how I know and it used
[00:14:18] to be this was not a problem that was defined by the person it was defined by our technology
[00:14:25] advancements because we needed to swap out the tape said Monday in order to continue to do this exactly exactly
[00:14:34] okay let's wrap up love and up love and not to well love not to is wrapped up in a sense that needs be based on the
[00:14:42] sensitivity it's okay right yeah there's one more point one of the things that we aren't doing well as msp is making
[00:14:48] a decision based on sensitivity usually right preach on so the statement in this says based on the sensitivity of the
[00:14:58] data which means my backups have the business case to those usually align with sensitivity you usually have a
[00:15:05] greater business value out of the greater sensitivity data right point of it but saying that the backups need to align based on that
[00:15:12] here's a great example of where that can fail and we're going to get into this and 11.3 let's call this
[00:15:18] fictitious company called Schmickr soft let's call it right nobody's ever heard of them they're completely face
[00:15:24] that's the one it's that was my cousins place right yeah that's right imagine this company had data sets
[00:15:32] where security keys were being used that were very very secure signing certificates let's say in an
[00:15:39] organization and then the those were being logged to a less secure platform or not less secure platform had a
[00:15:47] wall that allowed someone access it or some other person allowed access to it the point being I now have a
[00:15:52] lesser sensitive piece of platform information where the backups are being stored and they more sensitive
[00:15:58] platform where the production is that's a problem right because you now run into somebody that has less
[00:16:04] sensitive access can now access a key let's say and then maybe thousands of micro soft or
[00:16:10] Microsoft accounts get compromised through that that key being able to forge OAuth certificates is that
[00:16:15] really a question here because I feel like when I when I read this and then correct me if I'm wrong here but when I
[00:16:20] read this about sensitive data it's really about frequency here yeah frequency of the backup so to your point
[00:16:26] I feel like this gets us into a maybe less three let's skip 11.3 for a minute because I think so 11.3 is the only
[00:16:35] one that talks about protecting the data and that's where 11.3 is exactly what I just started to start
[00:16:42] so let's go let's skip it for a second and let's establish 11.4 are you just trying to prove the non-yellow brick
[00:16:48] road version again like you go back in and so we would be pushed down this slide when I push you okay we're going
[00:16:54] on the shoes and I'm wrestling I wish he was I wish you would put it all in right now because I don't know
[00:17:00] what I'm doing so I'm just going to do this more on yeah yeah with you so that you may find a partner let go of
[00:17:05] and get a hold of your yes we're going to do this like Sylvester salon on the guy in the truck with the hair
[00:17:10] yeah anyways I digress okay so why I wanted to skip though is is kind of in the same vein so recovery
[00:17:17] of data in some cases is more important than protecting the data because if you can't recover it
[00:17:23] doesn't matter how so I see you're now getting into 4 and 5 is what right so what I wanted to say is okay
[00:17:28] why if we cover 11.2 11.4 and 5 so we've covered 4 of the 5 the reality is 11.3
[00:17:36] proves that all of the other things are moot if you can't protect the integrity of the data right and
[00:17:43] to your point maybe 11.4 and 11.3 out of order is the way I might look at that's coming thinking like
[00:17:48] like I feel like a lot yeah that's a bill is listening I apologize to you Josh and charity I'm just
[00:17:54] totally kidding they're great that's just kind of the observation I had I think about all the
[00:17:57] there's one we've been I'll make the argument for you then do it so 14 11.4 I'll just tell you
[00:18:03] what 11.3 is we'll come back to it 11.3 says protect recovery data with the equivalent controls
[00:18:07] to the original data so we know what that says right like you know the store data where it's like now
[00:18:11] everyone has access to the root or you don't even put the store the stored backups that way
[00:18:16] no no what they're getting at is really Chris protect the backups in a way like if I back up
[00:18:22] very sensitive servers to a non-sensitive backup location and somebody has access to that
[00:18:27] they now have access to sensitive data so that's 11.3 11.4 though says hey threat actor is going to
[00:18:34] threat act and when they come in and they have access to the backup server that's on the same
[00:18:38] domain because you put it on the same domain so sorry people I love you to
[00:18:42] make sure you're in the same position criteria away and love it yeah yeah yeah yeah yeah
[00:18:45] and so 11.4 you're now saying hey I have an isolated instance of recovery data think
[00:18:55] like third parties like a data or a beam if you set up the list is long the list is a copy
[00:19:01] lots of those yeah I'm not trying to be exhaustive or exclusive I apologize these are just
[00:19:04] but but in fairness it's a good it's good thing to bring up if you're a vendor on this camp
[00:19:09] like you have six pages of permissions and we're back to control five and six like this should be
[00:19:16] something that we treat no different than we would have with active director where it's
[00:19:20] like no you are not admin privilege yeah you do not have local granularity of permissions
[00:19:26] right so granular today in our in our third party vendor space like oh that's on page four
[00:19:32] of the permission settings like this is a perfect example of why that can become problematic
[00:19:37] it is but the same breath I need that granularity to be able to make those decisions
[00:19:41] it's about getting it right where it gets simple and built in that's the Matt I know
[00:19:46] I'm being a little John Madden here but back to the point on 11.4 this is
[00:19:50] a certain training man was certainly but he was definitely in the time of like you're
[00:19:56] not winning because you're simply not scoring enough points yeah thanks madden I figured
[00:20:00] that out like that was part of this math matacredation of winning football yeah yeah yeah
[00:20:05] okay but 11.4 says have an isolated instance that would be back to that point of saying
[00:20:10] I as a threat actor get in and can compromise the environment you want to limit how much I
[00:20:16] can compromise so that you have another copy of that backup an isolated instance and this
[00:20:20] follows the 321 rule 321 meaning right I have three copies of the data right two different
[00:20:26] medium types and one an off site location that means I have the production data my
[00:20:31] backup one site and potentially a third party and and when you get to cloud that may not
[00:20:35] directly apply the 321 I just realized my 11.3 lands before 11.4 because 11.4 if
[00:20:42] the data is encrypted this becomes adds to that isolated layer it does if it's not
[00:20:51] encrypted this this becomes Pandora's box well but this is talking about reference
[00:20:56] encryption and data separation so both come and play and that's why I'm fine it finds
[00:21:01] it offline cloud or otherwise if someone were to gain access to it and it's not encrypted it
[00:21:07] goes back to just everybody has permissions to the data yeah and you have to think of it as now
[00:21:14] if I can control encryption I can do something that's magical and that's called cryptographic
[00:21:18] shredding meaning that let's say I hand you the keys here's a great example I hand you an
[00:21:25] envelope with a letter in it and that letter has just absolute damaging evidence about
[00:21:30] it's it's it's scented and it sounds like despair but it's been it's been scented right
[00:21:36] and but the point is I have this letter and I hand it to you if you were to open
[00:21:39] that and reseal the envelope I don't have any knowledge of whether that has happened
[00:21:43] I now have to assess you to a high level of degree of will he open the envelope let's just say
[00:21:48] that's a simple way to look well this is obviously didn't use the king's crown seal of wax
[00:21:52] on that envelope to your point I would be now encryption by encrypting the letter before
[00:21:59] I hand it to you and only then giving you the key when it's time for you to read it
[00:22:02] right now I actually don't have to vet you as much I don't give a crap about you
[00:22:06] because I could destroy the key and therefore you could never read the letter
[00:22:09] right and the argument is when you start thinking about cryptographic shredding by doing 11.3
[00:22:15] and when you get into like okay and V you have to check the box yes or no
[00:22:18] you're going to encrypt the backup to your point if I encrypt that back up and it goes
[00:22:22] to third party provider some of the stuff I would have cared about about the third party provider
[00:22:26] I do not care about anymore because of that separate encryption key that might be possible
[00:22:30] so without getting off on that tangent too far yeah we probably shouldn't because
[00:22:34] we're not saying that everybody listens to this show is as confident
[00:22:38] and on point with regards to data like they don't love it as much as we do
[00:22:43] that's fair I mean I mean I actually married a data pillow one point so
[00:22:47] I apologize but you take a data pill that's probably true story those are very
[00:22:53] different all right we got left sorry let me finish the why is you want to make sure
[00:22:58] that if I do something to attack I can't destroy your ability to restore you want
[00:23:03] to preserve right the ability and most MSPs I think get this right Chris at least for
[00:23:07] the stuff they've selected agreed how many are doing like offline storage or like
[00:23:12] they call cold storage where it's not accessible without some extra steps to get
[00:23:16] to that data to restore or worm right right once read mini or
[00:23:20] version controlled methodologies that allow you to see these version delineation
[00:23:24] really like what you know some way to solve this whatever that is your block
[00:23:28] and tackle if you will yeah some of the stops you from being able to delete it
[00:23:32] except for accident circumstances isn't that how we all got started with backups in the first place
[00:23:36] what do you mean I can't delete that folder it's called C drive like that is
[00:23:40] actually how I had to reinstall windows 3.1 if I remember correct yeah I might
[00:23:44] run into that on a few of the windows yes okay so 11 5 well we have 11 5 and then
[00:23:50] back to 11 3 a little bit I mean yeah tiered yeah good call 11 4 was have another
[00:23:56] copy your data make sure it's protected bro no this was another in the old days
[00:24:00] you know what this was well but well but really it is an isolated copy that
[00:24:04] in your mind if you're doing this real day I took the tape out I took it home
[00:24:08] and I gave another copy to Sally and she put it in her safe like that was how you dealt with that right like
[00:24:13] and then you found the drive on the parking lot ground and you thought no fortunately my
[00:24:18] computer and you know what Sally that drafted it was actually bill unfortunately we always know
[00:24:22] bill had drive you should not put in any computer true 11 5 says and this sounds so stupid
[00:24:28] that you have to say it but like let's say you were an auto mechanic that never
[00:24:32] checked to see if the starter fixed the motor like how smart would you be if that was the case and so test
[00:24:36] recovery data says hey listen bro go check and make sure it actually works and you see people get
[00:24:41] cute with this right because they take it to like oh I have a boot screen image right well what does
[00:24:46] that mean we're talking about layer seven not just the application being an operating system up
[00:24:52] I want to know is the application layer being delivered can I access it is AD functional
[00:24:56] is the forest trust broken do I have an AD tombstone situation playing out am I dealing with an
[00:25:01] identity problem with Azure AD like all the things so I love I love my thing pad Lenovo like this is
[00:25:11] this is my favorite windows laptop that I've had and and I love it and I will tell you along those lines
[00:25:17] I can get the login screen and I could authenticate and when I authenticate guess what I got
[00:25:23] blue screen every single time what I discovered was I don't want to talk about it but basically I had
[00:25:33] inadvertently tripped the trigger that said the person long into this machine might not be who
[00:25:39] he says he is well that's fair and that was literally literally my windows profile got interrupted
[00:25:45] and to your point when a server seeing a screenshot you don't know that the rest of
[00:25:51] its being delivered and that's the problem so why would you just tell you everything right like it
[00:25:57] only tells you we were able to get this far we got past bios and boot we are here right right test data
[00:26:04] recovery says test backup recovery quarterly comma space or more frequently comma space for a sampling
[00:26:13] of in scope enterprise assets so when you think about that a sampling does every
[00:26:18] time you don't miss this because for those of you that don't know when Matt says comma space
[00:26:22] this is not because Matt's a grammar expert this is because he's not only something it's really important
[00:26:26] so recovery quarterly comma space or more frequently comma space gets into 11.3 and 11.2
[00:26:34] because the frequency in your automated backups and the data recovery you need to be able to
[00:26:42] your business and sensitivity and those things and yeah you're data recovery if this is an OS
[00:26:50] backup I'm going to tell you right now quarterly is probably overkill for the version of 11 windows
[00:26:58] to make sure that windows is functional when you've got bare metal restore capability but
[00:27:03] if this is your QuickBooks file and it changes every you know four to eight hours and you check
[00:27:09] it once a quarter that's problematic yeah and that is where that sampling comes into play right absolutely
[00:27:16] great example like one of the things about sampling let's talk about it so an auditor wants to sample
[00:27:21] an assessor wants us to see a sample of these safeguards and evidence let's say I'm going to ask for
[00:27:27] 14 things out of the hundred percent three yeah imagine then out of that sample out of those 14
[00:27:34] the first five you are not meeting my sample just changed my sample might now be all of them
[00:27:41] my sample might be a hundred percent inclusive right so you could see an understanding of sampling
[00:27:45] when they say a sampling doesn't mean that that sample is static that sample should mean if I
[00:27:50] start seeing evidence of a high degree of failure of QuickBooks file backups maybe I need to now
[00:27:55] change that sampling size or well and write a lot of data based space right like having a backup
[00:28:00] restore process and getting air or in your backup process seeing lots of errors data was active
[00:28:06] lots of things didn't mean the data integrity was compromised but if you started getting frequency
[00:28:11] of errors it was time to check your backups to make sure that those were not compromising the data
[00:28:18] yeah that's a great point so testing what does that mean that means if you're really serious about
[00:28:23] it you are really getting to the beginning of 11.5 and the reasons in iG2 is you're starting to get
[00:28:29] into a disaster recovery type scenario because you can't just say I tested one machine you have to
[00:28:34] test a system they're interdependent they don't just be willing to still use it yeah that's it bro
[00:28:40] so you're now having to figure out how you're going to stand up a separate environment test the
[00:28:44] existing like think about banking how do they do this you get one call from fyserve and they go
[00:28:48] hey on Saturday at 2 p.m. you're going to change the VPN over to this VPN and you're gonna do 12
[00:28:54] transactions and see if these things work that's it that's their test but that's a test and
[00:28:58] it takes a live infrastructure will live in but no but on my point is no no no no I think you're
[00:29:03] missing a point here I know it's that it's that I actually have all the components necessary
[00:29:09] to actually construct that separate system to re-build a system that replicators what you had
[00:29:14] yeah yeah full system function right right because I think this is really important to note like
[00:29:19] someone can rebuild a system on say a machine but if your company has
[00:29:25] 5,000 factories and yeah yeah tens of thousands of transactions happening in real time
[00:29:31] then to say I restored a you know like critical like three of us can get in there
[00:29:37] that doesn't necessarily mean restored anything because what happens when everybody hits that
[00:29:41] restore and now you're corrupting data that's not being corrupted by somebody else
[00:29:48] yeah exactly I digress let's go back to recovery data real quick and close this out so 11.3
[00:29:54] was protect recovery data I think we've kind of skirted it a little bit in the way we've
[00:30:00] approached how we test data and the why because protected data should show proof when we do recovery
[00:30:07] one then how do you test do I now let someone not equivalently able to access that sensitive data
[00:30:13] test the sensitive data or do I have to test like people who have purviewed those man those are
[00:30:18] the challenges right controls five and six yeah it really is and then five and six applied through
[00:30:23] the lens of the person turning the wrench on checking the data how do you have that managed we
[00:30:28] do it by having a clear five controls five and six not oh yeah thought five and six because
[00:30:34] there's no I feel open for the person listening I made the jump just to be by being account man
[00:30:38] is living experience yes totally yeah all right Matt close us out with recovery data yeah um I think
[00:30:46] when we look at it from testing recovery and protecting recovery data you now get into
[00:30:51] how fast how often and then protecting recovery data with equivalent controls goes back to my
[00:30:56] Microsoft example you take a system that had very very high you know sensitivity at work like yeah
[00:31:04] at the end point Chris Johnson's the only person able to access this because he's top super secret
[00:31:09] squirrel right it should look the same it should look the same and that's the point the recovery data
[00:31:14] should match that and when you think about it the more sensitive the data the more things I
[00:31:19] should do such as and this is what they mean by this encryption data segmentation different ACLs
[00:31:25] things to that nature that allow you to meet the requirements of the original data think about Kui
[00:31:29] or FCI as they apply in that this is going to get into what I live in yeah you don't want to get one
[00:31:35] of those but I'm going to say like if we apply 11.4 and five to this if if there's clarity around 11.3
[00:31:43] and the data we're talking about then your 11 and 4 and 5 are going to have a lot of predefined
[00:31:50] data variables already set for you because so for example data recovery quarterly or more frequently
[00:31:57] based on the sensitivity of the data that you're protecting this is becoming more and more critical
[00:32:03] because getting it wrong one time maybe catastrophic so if your tests are too far apart you may not know
[00:32:11] whether or not it would be bad until bad things happen right so like one has to be very careful
[00:32:17] with this approach so 11.3 I now recognize where it sits and I apologize to anybody that might have
[00:32:23] said that might have said we should move it what not like I thank you yeah it was me it was me I may
[00:32:31] be me I think I'm the same boat so like I pushed you down the slide and said I wasn't going to do it
[00:32:35] but like you're going to go down the slide all right yeah totally I think we've covered this
[00:32:41] if you have questions you know where to find Matt I'm not going to tell you where to find me
[00:32:45] but Matt will probably point a finger if you've been listening to this this has been an
[00:32:49] episode of MSP 1337 I appreciate all of you thanks and have a great week

