Fixing What Breaks CMMC Assessments

Cole French:

CMMC’s success often comes down to avoiding a handful of critical missteps. In this episode, we break down the biggest assessment failures out in the wild, including cloud compliance gaps, incomplete SSPs, incomplete multifactor authentication implementation, weak risk assessments, and MSP misunderstandings. More importantly, we discuss how to fix them before they cost you. Welcome to the Cyber Compliance & Beyond Podcast, a Kratos podcast that brings clarity to compliance, helping you leverage compliance as a tool to drive your business’s ability to compete in any market. I’m your host, Cole French. Kratos is a leading cybersecurity compliance advisory and assessment organization, providing services to both government and commercial clients across varying sectors, including defense, space, satellite, financial services, and healthcare. Now, let’s get to today’s episode and help you move cybersecurity forward. In today’s episode, recorded live at CUI-CON in Orlando, we break down the biggest lessons from real CMMC assessments.

We start with one of the most common pitfalls, organizations using non-compliant cloud service providers like Microsoft 365 commercial, which is FedRAMP moderate authorized, but not compliant with DFARS 7012 requirements. We then explore why many SSPs fail early, often because they’re written to the 110 controls of NIST 800-171 rather than the 320 assessment objectives in 171A, leaving critical gaps in documentation and determination statements. We then highlight frequent MFA issues, particularly when organizations secure endpoints, but overlook security protection assets such as firewalls, which also must meet MFA requirements. We break down why risk assessments often fall short as either outdated or mistaken for control assessments and stress the need to evaluate broader organizational threats. Finally, we get into the role of MSPs and ESPs, emphasizing the importance of solid shared responsibility matrices and the value of working with providers that hold their own level two certification. We close by clarifying what assessors can allow organizations to fix during an assessment under the security requirement reevaluation process.

Joining us for today’s conversation is Fernando Machado. Fernando is the managing principal and chief information security officer for Cybersec Investments, an authorized CMMC third-party assessment organization. Fernando is a lead CMMC certified assessor and was a member of the CMMC accreditation body’s standards management industry working group, which helped develop guidance on CMMC’s assessment criteria and scoping with over 17,000 volunteer hours. His contributions led to formally being recognized by the President of the United States with the President’s Volunteer Service Award. We hope you enjoyed this episode. So Fernando, thanks for stopping by our booth today here at CUI-CON down here in Orlando to just chat about CMMC. And first things first, I guess you’ve performed a lot of assessments in this space. So let’s just start with top five. What are the top five lessons learned? And of course, we can go one by one and break them down, but what are the top things you’re seeing from the assessments you’ve performed?

Fernando Machado:

Yeah. So one of the things that we’re noticing is top biggest problem that we’re running into is the use of non-compliant cloud service providers. For example, we’re seeing a lot of organizations still processing, storing, and transmitting CUI, utilizing Microsoft’s 365 commercial environment, which is neither FedRAMP nor does Microsoft accept the DFARS 7012 clause paragraph C through G. The next thing that we see that’s also a huge issue is the failure to implement the controls at the 171 Alpha level, 171A. A lot of contractors are going to the NIST website and downloading the SSP template, and they’re basically making determination statements based at the control level versus at the assessment objective level. Next thing that we’re also seeing is implementation of multifactor authentication, not in multiple areas. So they’ll apply it on CUI assets, but then they won’t apply it on security protection assets. So it’s like they’ve half implemented the controls.

The other thing that we’re also seeing is, believe it or not, risk assessments, they’ll say that they’ve conducted the risk assessment on an annual basis. And when you ask them, “Well, where’s your latest risk assessment?” The last one they completed was five years ago. I’m like, “Okay, so you’re not doing it annually like you stated that you did in your documentation.” So those are the top four or five things that we’re seeing in the space and nine times out of 10 when we start to see those things like that. It’s what we call the inadvertent false start.

Cole French:

Gotcha. And how do you guys normally handle… So let’s start with the first one. So cloud service providers, is that something you’re usually finding when you’re doing scoping or is that a phase one thing? And is it just Microsoft or are you seeing folks using other non-compliant cloud service providers?

Fernando Machado:

Yeah. So in phase one, conducting the pre-assessment activities, one of the things that we’re supposed to do right per the cap is validate if they’re going to be using an external cloud service provider. And so one of the things that we have to look for is either A, they’re FedRAMP moderate authorized on the marketplace, not FedRAMP ready, not FedRAMP in process. They have to be FedRAMP moderate authorized, or B, they have to be FedRAMP moderate equivalent in accordance with DOD’s December 2023 memorandum. And so if they’re going to be using those cloud service providers to process or transmit CUI, so that’s exactly what we’re looking for.

Cole French:

And you mentioned the 7012 C through G. So I know with Microsoft, there’s some places you can go out and you can look and actually Microsoft breaks it down from a compliance perspective, like which of their environments is compliant for different frameworks and things like that. But how do you handle the C through G component, which is the incident reporting? How do you handle that for maybe non-Microsoft or cloud service providers that are FedRAMP authorized, but maybe they’re not as well known, I guess, for lack of a better way of saying it as the Microsoft’s Google, some of these other larger providers?

Fernando Machado:

Yeah. That’s an interesting question that I’ve thought about because the DFARS 7012 clause has the C through G clauses in it, whereas the CMMC rule, the only thing that they care about is that it’s either FedRAMP moderate authorized or FedRAMP moderate equivalent, and there’s no mention of the paragraph C through G clause for us as C3PAOs to validate.

Cole French:

So just to break down the SSP, what we’re talking about from an SSP perspective. So NIST 800-171, 110 controls, NIST 800 171A, which as assessors, that’s what we’re looking at is 320 test objectives. So when you write your SSP, you don’t just want to write to the control because the control is really a summarization. And to be honest with you, it doesn’t even really go into the level of detail if you’re just writing to that, that the test objectives would. So do you want to talk a little bit more about some of the nuance and specifics of writing to the test objectives?

Fernando Machado:

Yeah. So one of the things that we also look for as assessors is ensuring that even though there’s not a standardized way of writing an SSP, it makes our job easier if you are calling out each assessment objective on how you’ve implemented with your determination statements. That makes our job easier. We’ve seen organizations where they’ve just write one very long implementation statement and we have to parse out what’s in there, but it makes our job easier if you’re able to say assessment, objective alpha, this is a determination statement, objective Bravo, this is how the… Because it allows us to say, “Okay, I understand what the organization is saying here.”

A couple tips that I’d like to give to some of the listeners is if you’re looking at the assessment objectives and the assessment objective has the words identified, specified or defined, typically the assessor is going to be looking for some type of documentation, either a policy, procedure, system security statement, something along those lines. And if it says something like limited, enforced or so forth, some type of action verb, the assessor’s going to be looking for the technical configuration of what you’ve identified, specified or defined. So that’s the best way to be able to parse out and all the controls are set up that way. There’s always going to be an identified, specified or defined part of the control, and then there’s going to be the technical configuration of said documented part of the control.

Cole French:

Now on the identified, specified, defined. So I think this is one too that, in our experience, people get confused on this. So defined is pretty straightforward. I think you define this and that’s typically you see that with timeframes or frequencies.

Fernando Machado:

Yeah.

Cole French:

So identified, specified, you want to talk a little bit about like what’s the difference between those or how are they similar? What should you be writing to when you’re talking about identified? I’m thinking like, identify authorized users for instance, does that mean that I need to have a document with all my authorized users or just talk a little bit about that.

Fernando Machado:

Yeah. So we’ll take 311 Alpha for example. Authorized users are identified and in that what we would say is, okay, we want to see basically a approval and authorization process of how a user was able to get an account on the system. And what ends up happening is once you have those users identified, it’s usually going to be in some type of list that says, John Smith has access to the system and what ends up happening is when you look at 311 Alpha, which is authorized users are identified and you go straight to 311 Delta, which is authorized users are limited to system access, now we should be able to say, “Okay, I’m looking, the authorized user list should match what’s in your list.” Let’s say we’re taking Microsoft, for example, it should match your Entra ID directory.

Cole French:

So when you’re performing an assessment, are you… So let’s say it’s a larger organization. So just talking about evaluation of that control. So that’s a really good breakdown of your identified users and then like you said, the enforcement mechanism and so those two things should match. Are you looking at the entire user list? Are you looking at a sampling? How are you coming at validating what that control’s implementing?

Fernando Machado:

We usually do sampling if it’s a large organization and then we’ll say, “Hey, show me John Smith’s document that demonstrates that they’ve had an account created for them in accordance with the organization’s defined policies and procedures.” And then we would then say, “Okay, let’s see if this matches the list that’s in Entra ID.” for example and say, “Okay, yep, this looks like it matches. We selected it at random and it meets the intent of the control.”

Cole French:

Okay. So the third thing you mentioned was multifactor authentication and specifically not necessarily implementing MFA across the entire organization or I guess the entire scope in this case. So you want to talk a little bit about the specifics that we’re seeing there?

Fernando Machado:

Yeah. So sometimes what we’ll see… Well, I’ll just take a typical manufacturing environment and what we’ll see is they’ll implement multifactor authentication on the endpoints, but then they will not implement multifactor authentication on their firewall, even though it has the capability to do that. And when you look at the 800-171 requirements, paragraph 1.1, the requirements apply to components of non-federal systems that process or transmit CUI or that provide security protection. So that firewall is providing security protection to those CUI assets, and so therefore you have to implement the controls there as well.

Cole French:

That’s a good point that you bring up about that security protection assets, because a lot of the questions we get is when we’re talking about scoping or doing an assessment is, how do my security protection assets factor into this? And what we always say is, well, we’re going to evaluate those security protection assets in accordance with the security that they’re providing, which is like, to us, I think that makes sense, but the example that you just gave is a really good real world example of this is the type of thing we’re going to be looking at from a security protection asset perspective. So just something to keep in mind, multifactor authentication is definitely one of those things that your security protection assets are also going to be evaluated as part of your assessment for controls such as multifactor authentication, because that is a protection that… Those SPAs are protecting your assets, so you need to make sure that access is being controlled in the same manner as the other components within your system.

Fernando Machado:

Correct.

Cole French:

So the fourth thing you mentioned was risk assessments. And I know I’ve talked about this, we’ve talked about this on the podcast many, many times. We even had a couple of episodes that really got into what is a risk assessment and even talked about cyber insurance and your risk mitigation strategy from a business standpoint. And you mentioned the risk assessment. And in your case, it sounds like what you guys are seeing is people saying they do a risk assessment and then when you actually ask them, it’s like, “Oh, well, the last one we did was five years ago.” One thing we see a lot is security control assessment, which is validating my NIST 800-171 controls on an annual basis. People think, “Oh, well I did that, so that covers me for my risk assessment.” But we tell people all the time, risk assessment is much, much broader, much more all encompassing. You got to think about it from a business perspective, there’s a whole lot of considerations beyond just security when you’re talking risk assessments. So anything else you wanted to add on the risk assessment piece?

Fernando Machado:

Yeah. I mean, and when you’re looking at the control, the frequency to assess risk to organizational operations, assets and individuals, so we’re like, now we’re talking about natural and manmade threats to the business and to that CUI environment, which is encompassing your business and your people. And so to your point, one of the biggest problems that we see is they’ll define their frequency on an annual basis, but the last risk assessment they conducted was three or four years ago. There was not even one that was recent. It was just like, yeah, the last one we did was three or four years ago. I’m like, yeah, but your SSP states that it’s frequency on an annual basis, your SSP was signed earlier this year and you haven’t even conducted a risk assessment in accordance with that frequency. So now we have a disconnect.

Cole French:

And that’s a do or die. That’s a make or break for an assessment, correct? If you… 3.11.1, I believe?

Fernando Machado:

I believe it’s a three or five point control, but I’d have to go back and look at the scoring guide.

Cole French:

I know you mentioned when you first went through, these are all the lessons we’ve seen. You mentioned four things, but there’s a fifth thing that I want to bring up because I’m curious to hear your thoughts on, and that’s one we hear a lot talked about in this CMMC space, but that’s MSPs, ESPs, we talked about cloud service providers, which fall in there in their own way, but MSPs and ESPs, so not cloud service providers. I’m just curious what you’re seeing there, what your thoughts are on that topic.

Fernando Machado:

Yeah. Usually what we see is we sometimes get clients, small businesses that will bring in a managed service provider, managed security service provider as part of their assessment, their outsourced IT. And one of the first things that we start asking is, as the MSP and you as a customer, is there a customer shared responsibility matrix that’s going to define who’s responsible for what for each of those controls? And if I start getting blank stares, it’s usually not a good day. And I can tell you it is a very uncomfortable experience to sit in an assessment and say to the OSC, “You’re failing because your MSP has failed to do their part of the shared responsibility.”

Cole French:

Yeah, I was talking with Toby yesterday actually when we were talking about this exact topic, the shared responsibility matrix and he brought up a good point that another thing when you’re talking with an MSP about their shared responsibility matrix is, and we’ll get into this a little bit because I want to get your thoughts on different questions and stuff you think people should be asking as they’re maybe looking for a service provider in this space, whether it’s an assessor, MSP, et cetera. But he said that one of the questions you should ask is of your potential MSP or ESP is how much change is there, how much variation is there from customer to customer with your shared responsibility matrix?

And same kind of thing. If they look at you like they don’t know what you’re talking about, that’s concerning because not every customer implementation is exactly the same. So they should have a shared responsibility matrix across the board that provides, “Hey, this is what generally our share line is, but each customer is going to be a little bit different.” Should either address that in the share line, some of that variation that could exist, or you should have your own shared responsibility matrix from your ESP or MSP that actually defines that for your specific implementation.

Fernando Machado:

Yeah. One of the things that I’ve seen too is some of the very good MSPs in the space actually have different service levels. So we’ll just, for the purposes of this conversation, we’ll just come basic medium and advanced service levels. The basic service level will have a shared responsibility matrix of what they’re responsible for and as the higher they go up into the tier, the more things that you can inherit right from that MSP, but they’ve already been validated and saying, “Hey, these are the different shared responsibilities.” Because even with Microsoft, who’s FedRAMP moderate authorized, there’s no such thing as, “Oh, we meet 102 out of the 110 controls.” When you look at the Microsoft tech stack and you’re looking at the shared responsibility, at a minimum, there’s a shared responsibility and customer responsibility, which makes up about 75 or 80% of the overall controls and the remaining 10 to 20% is purely on the customer’s responsibility, like providing training, conducting background checks and things like that that Microsoft can’t do for you.

Cole French:

And even I think the 70 to 80%, there’s still some customer responsibility lies within that 70 to 80%.

Fernando Machado:

That’s why I said 70 to 80%, it’s either shared, it’s either a shared responsibility or the customer’s responsibility. Actually, I want to make a clarifying point because I had it backwards. 75 to 80% of it is either a shared responsibility or customer responsibility. The remaining 10% is a Microsoft responsibility, which would be things like housing the data center for the physical protection and so forth. One of the things that I want to add for the MSPs is, although it’s not required, it’s a good sign if an MSP is CMMC level two certified themselves because it makes life so much easier. For all parties involved, I always say what better way to demonstrate by eating your own dog food and actually implementing the same controls and going through the same pain that your customers are going to go through, so that’ll give you a higher level of understanding.

And for folks that are looking for good MSPs, there’s an MSP collective website where you can go see these are all CMMC level two certified MSPs. And if you want to ask questions of an MSP that’s out there, I know ND-ISAC actually has a MSP shopping guide that was recently published that you can download to ask these questions to your MSP to figure out if they’re going to be a good fit for you.

Cole French:

We’ll drop links to that in the show notes so that people can go out there and check that out. I will ask as a follow-up to that. So you mentioned getting your ESP or MSP getting CMMC level two certified. So when you’re doing an assessment, just talk a little bit about at a high level, what’s the difference from an assessor’s perspective? What level of scrutiny am I giving to an already certified MSP versus one that’s not?

Fernando Machado:

Yeah. So now that an MSP is certified for the services that they’re providing their customers, I am now less concerned about their side of the shared responsibility and I’m now concerned a little bit more about the customer side of the implementation. And as assessors, we have the discretion of kind of sampling to make sure that the MSPs are still doing things that they’re supposed to be doing. But of course, like anything, if you’re working with an MSP over and over again, you start to develop that familiarity and then you know that at some point that MSP is also going to go through another CMMC level two assessment again, so all that stuff’s going to get revalidated.

Cole French:

And that’s actually a good segue to the last thing I think as we close up here that I wanted to ask you about, but even in our experience, to your point about working with an MSP, working with an MSP over and over again or on a repeated basis. And when OSCs are out there looking for, who should I work with, keep that in mind, when you’re talking to an MSP, ask the question like, “Do you guys partner with a C3PAO or do you have a C3PAO that you guys work with on a regular basis that assesses your systems?” Because we’ve seen a lot of success with partners that we have that they do more of the technical and MSP type work, they support it, we’ve assessed it multiple times.

So there’s a level of assurance coming into an assessment that we have on our side, but also that a customer can have because you just know, “Hey, these guys have worked together.” So they understand the system, but also these people have worked together and been in assessments together. So there’s a familiarity, a rapport, all that kind of thing. So from your perspective as an assessor, what are some of the questions you would say folks, OSCs, those who might need assessment or in the market for that, what are the questions they should be asking either their assessor or if they’re looking for help and getting ready, their RPO or RP.

Fernando Machado:

Yeah. Well, first question is, if you’re an OSC looking for a consultant or an MSP MSSP, first question is, are they working with a C3PAO? Because I can’t tell you how many times you might have a consultant that might know what they’re doing, but if the C3PAO does not have the same interpretation as you do, you’re going to start to run into problems. And so we usually say, and that’s why it’s good to have the familiarity like you discussed, is that they know this is the intent of the control and I know as a C3PAO when we’re assessing the control, we’re assessing it with the same intent of that interpretation so that we’re not running into interpretation problems because as you can imagine in this space, there is interpretations are a mile wide and an inch deep. Everyone’s got different interpretations on different things. There’s things that people will die on a hill on certain items and there’s some things that some C3PAOs let go. So it’s always good to just find that yin and yang that they’re both working together in tandem.

Cole French:

So you would suggest actually working with a C3PAO, working with them simultaneously, or maybe having a C3PAO first, and then working with your C3PAO, I suppose, to see what type of familiarity they have with other consulting organizations.

Fernando Machado:

Yeah. And it could be the other way around as well, and just asking that consultant or MSP, “Are you working with a C3PAO?” Usually the good ones are saying, “Yeah, we work with three or four ones that are trusted in the space, have the same interpretations.” And then just understanding that rapport between the two, because they both work hand in hand at the end of the day. One side is doing the consulting and advisory, which we as C3PAOs are precluded from doing four for the same organization that we’re going to be assessing.

Cole French:

Any other questions or things people should be considering as they’re potentially bringing on somebody to either do an assessment or help them get ready?

Fernando Machado:

Yeah, I would say the ND-ISAC MSP shopping guide and also the ND-ISAC also has a C3PAO shopping guide as well.

Cole French:

And what type of stuff, just curiosity, that shopping guide, what kind of things are they emphasizing or what type of clarity is that providing?

Fernando Machado:

It’s provided like I’ll talk a little bit about the C3PAO shopping guide. So it’ll talk about things like pricing. Not necessarily that price should be the only thing because everything in life you get what you pay for. The other thing would be, are they working with 1099 assessors versus W2 assessors? Because as you can imagine, different interpretations across the space, somebody comes in representing a C3PAO and they have a different interpretation from now you’re starting to create conflict and then also how much experience those assessors have. Have they ever actually even conducted an assessment? Have they ever been on a DIBCAC assessment either through the joint surveillance program? Those are some of the questions that you can ask your C3PAO availability, when they’re going to be available. I think it’s one of the biggest powerful tools OSCs have is you may not have a choice on not getting certified, but you absolutely have a choice in selecting the C3PAO that you’re going to work with. And so making sure that you’re doing your due diligence and your homework to make sure that it’s a good fit for your organization.

Cole French:

The W2 point that you mentioned, 1099 versus W2, that’s a huge one that I always bring up with folks and we talk about as a differentiator, I guess for us, because we prioritize the W2. We want to have a team of people that we build, that we work together, we have familiarity with each other. We’re coming at the controls from a relatively unified position as an organization because we’ve done work with subcontractors, surge support, things like that. It’s not as a capability we have, but you’re exactly right. It introduces this whole new set of thinking and way of doing things and not necessarily is that problematic in every situation, but it does present challenges.

Fernando Machado:

Yeah.

Cole French:

So I think it is definitely something you want to be aware of as you’re selecting your C3PAO in particular, I think.

Fernando Machado:

100%.

Cole French:

You mentioned the security control interpretations and people want to die on a hill about different things. I’d be curious if you have anything that just top of mind as far as, what are the controls that are particularly problematic in your experience from an assessment interpretation perspective?

Fernando Machado:

So one of the controls that I see that creates problems is 319, which is the security notices. So your login consent banner. One of the biggest problems that I’m noticing is some folks will download the CMMC level two assessment guide and then some folks will download the 171, 171A document. And when you’re reading them, a lot of people say, “Oh, the two sets of documents, the CMMC level two assessment guide and 171, 171A, that the assessment guide is exactly the same.” And it’s not. So you have in the assessment guide, you’ve got the 171 control, you’ve got the assessment objectives, you’ve got the discussion paragraph, but where some assessors get in trouble is there’s a further discussion section.

And in that further discussion section, it says contractors should implement the following in their notice and consent banner and there’s a list of different things that they want you to do. So now assessors go into that and they say, “Oh, it doesn’t have these lists of things. We’re failing that control.” Well, no, because when you read that statement, the keyword there is contractors should do these different things. That’s why it’s supposed to be there to provide additional guidance, but it’s not authoritative because the documents, those documents are optional. The documents you’re supposed to use are 171 and 171A.

Cole French:

So one last thing, just real quick, want to get your thoughts on this because this is something we get asked about a ton, but fixing things during an assessment.

Fernando Machado:

Yes.

Cole French:

What’s your take on… And I guess this goes with the assessor interpretation thing as well, but what do you have to say about that?

Fernando Machado:

Yeah. So there’s a section in the 32 CFR Part 170 rule called Security Requirement Reevaluation. And it says that a control that has been marked as not met can be reevaluated during the course of the assessment and up to 10 business days after, providing that you meet the following three criteria. So one is providing additional evidence to demonstrate that the control has been marked as met. Two, it doesn’t change or limit the effectiveness of the controls that have been marked as met and three is prior to the assessment team submitting the assessment findings report.

So if you’re able to do all three of those, you are allowed to make technical and configure and documentation changes during the course of the assessment. And I know that there has been folks in the CyberAB that have said it has to be additional existing evidence. We’ve talked to several attorneys. That is not the way that it’s written. And in the future, if they want to make those updates, that’s great. They can do that in future rulemaking, but the way the rule is written today, if you meet those three criteria, you’re allowed to make technical and documentation changes.

Cole French:

Well, again, Fernando, I really appreciate you stopping by here today to chat with us a little bit and share your guidance and wisdom as you’ve been working out here with a lot of these OSCs. So again, yeah, just really thank you for your perspective and for your time this morning.

Fernando Machado:

Yeah. Thank you for having me and I’m glad I was able to help.

Cole French:

Thank you for joining us on the Cyber Compliance and Beyond Podcast. We want to hear from you. What unanswered questions would you like us to tackle? Is there a topic you’d like us to discuss or you just have some feedback for us? Let us know on LinkedIn and Twitter at Kratos Defense or by email at ccbeyond@kratosdefense.com. We hope you’ll join us again for our next episode and until then keep building security into the fabric of what you do.