The purpose of this research, sponsored by Crogl, has two objectives. First is to determine the effectiveness of security operations (SecOps) in preventing, detecting and responding to cybersecurity threats. Second is the use and value of AI in organizations and specifically in the Security Operations Center (SOC). Sponsored by Crogl, Ponemon Institute surveyed 649 IT and IT security practitioners in North America who are knowledgeable about their organizations’ SecOps and SOCs.

As defined in the research, SecOps refers to the integration of security practices and IT operations within an organization to improve overall cyber resilience. It involves teams working together to detect, respond to and mitigate cyber threats. This integration streamlines security efforts, enhances threat detection and improves incident response times.

Human analysts are important as a final line of defense in the AI-powered SOC.  Human analysts in AI provide context, critical thinking and strategic oversight. While AI automates tasks, identifies patterns, and flags anomalies, analysts can interpret and validate AI outputs, interpret nuanced data and guide strategic decisions.

Sixty percent of respondents say their organizations have a SOC.  Of these respondents, 57 percent have deployed AI in the SOC. Respondents were asked to rate the effectiveness of AI in the SOC in reducing threats from 1 = not effective to 10 = highly effective and the effectiveness of the role of human analysts as the final line of defense in the AI-powered SOC from 1 = not effective to 10 = highly effective.

Fifty-two percent of respondents say human analysts are highly effective as the final line of defense in the AI-powered SOC. As shown in this research, the deployment of AI in the SOC is in the early stage and may explain why only 44 percent of respondents say the AI in the SOC is highly effective in reducing threats.

Summary of research findings on the state of SecOps and AI in the SOC

 Most security alerts are not detected and investigated. Security alerts are notifications generated when a system or application detects a potential security threat or suspicious activity. Organizations represented in this research have an average of 4,330 security alerts in one day but only an average of 37 percent are detected and investigated. As a result, only 43 percent of respondents say their organizations are effective or highly effective in identifying and responding to security alerts.

 The number of security alerts investigated and resolved is a measure of the effectiveness of the SecOps team. Seventy-three percent of respondents measure the SecOps effectiveness in reducing risks. Because of the challenges in being effective in identifying and responding to security alerts, 79 percent say their organizations measure the number of security alerts investigated and resolved. Other metrics most often used are completion of security training and awareness programs (70 percent), compliance with privacy and security regulations and standards (53 percent) and vulnerability management (67 percent).

 Cyberattacks are frequent and most involve malicious insiders and phishing. Organizations experienced an average of 16 cyberattacks in the past 12 months. Most of these attacks involved a malicious insider (50 percent), phishing/social engineering (48 percent) and denial of service (41 percent).

 Most organizations have an in-house or a partial in-house SecOps team. Seventy-three percent of respondents say their organizations either have an in-house (42 percent) or partial in-house and partial outsourced (31 percent) SecOps team with an average of 8 staff. Only 27 percent of respondents say their organizations outsource SecOps.

More than half of respondents say SIEMs and Data lakes are considered very or highly effective. Seventy-four percent of respondents say their organizations have a SIEM (35 percent) or multiple SIEM platforms (39 percent). Fifty-nine percent of these respondents say their SIEMs are effective or highly effective in responding to security alerts. Seventy percent of respondents say their organizations have a data lake solution (41 percent) or multiple data lakes (29 percent). Of these respondents, 51 percent say data lakes are effective or highly effective in handling data required for detection, investment and response.

 Sixty-two percent of organizations represented in this research have adopted AI. Twenty-one percent have AI embedded in cybersecurity, 18 percent have embedded AI in cybersecurity and business processes and 23 percent are piloting AI in cybersecurity.

 Organizations that have adopted AI say new tools to monitor AI for business purposes are being deployed. These tools include self-hosted Large Language Models (LLM) (31 percent of respondents), mix of homegrown and vendor provided LLMs, co-pilot and agentic AI (23 percent of respondents) and SaaS-hosted LLM (23 percent of respondents).

 To be successful, visibility into an AI’s system lifecycle that includes its development, training data and deployment is very or highly important. Fifty-seven percent of respondents say visibility into the AI’s lifecycle is critical. Only 36 percent of respondents rate the ability of their organization in detecting if AI tools are introducing new, less visible forms of data leakage as high.

 Respondents believe AI documentation and consistency is very important for SecOps. Fifty-eight percent of respondents say AI documentation of SecOps tasks is very or highly important and 63 percent of respondents say consistency in AI’s use for security operations is very or highly important.

 Organizations need to take steps to minimize third-party AI risks. Using third-party AI expands an organizations’ risk surface because a vendor’s AI weakness becomes their own. This introduces new challenges beyond standard cybersecurity, requiring deeper due diligence, stronger contracts, and AI-specific governance to protect against significant financial, legal, and reputational damage.

Sixty-one percent of respondents are very or highly concerned about third parties using their security data for enriching its AI service and 59 percent of respondents are very or highly concerned that AI vendors will use their data for derivative purposes.

 The following findings are for those organizations that have adopted AI in the SOC.

 AI in the SOC can make analysts more efficient and improve collaboration among staff. Sixty percent of respondents say their organizations have a SOC. Of these, 57 percent of respondents say their organizations use AI in the SOC. Use cases include reduction in the complexity of security cases to improve analysts’ efficiency and collaboration (54 percent), automatic generation of documentation for complex automated processes (49 percent) and the use of tools that have AI embedded in them (41 percent).

 The primary benefits of an AI-powered SOC are to speed up the time to resolve more alerts faster (67 percent of respondents), free up analyst bandwidth to focus on urgent incidents and strategic projects (57 percent of respondents) and improve the ability to triage, investigate and remediate the majority of Tier-1 and Tier-2 alerts (53 percent of respondents).

 SOCs and their teams are effective in preventing, detecting and responding to cyber threats. Sixty percent of respondents say their organizations have a SOC. Of these respondents, 61 percent say the SOC is very or highly effective in its ability to gather evidence, investigate and find the source of threats and 58 percent say the SOC’s team is very or highly effective in the ability to detect and respond to threats.

 In the SOC, AI can improve compliance with regulations and the ability to meet compliance mandates. When asked how organizations determine the value of AI in the SOC, 50 percent of respondents say it increases the ability to meet compliance mandates, 46 percent of respondents say it increases the SOC’s team’s ability to detect and respond to threats and 44 percent of respondents say AI decreases the cost of cybersecurity operations.

The most important feature in an AI tool for the SOC is its ability to integrate into existing workflows and data (63 percent). AI integration remains a major hurdle for organizations. Sixty-three percent of respondents say the most important feature when choosing AI tools is the ability to integrate into their existing workflows and data. Forty-seven percent of respondents say AI should be consistent and the output predictable. AI tools should enable organizations to meet compliance requirements with documentation to meet audit requirements.

The biggest barrier to deploying AI tools in the SOC is the difficulty in integrating AI into their processes and workflows (50 percent of respondents). This is followed by data dispersed throughout the organization and hard to normalize for the AI, (49 percent of respondents). Another barrier is regulatory and compliance concerns (37 percent of respondents).

To read the rest of this report, click here to visit Crogl.com

 

 

 

 

Part 2. Key findings

 

In this section of the report, we provide an analysis of the research. The complete findings are also presented in the Appendix. The research is organized according to the following topics.

 

• The challenges SecOps faces in preventing, detecting and responding to cybersecurity threats
• The deployment of AI to improve the IT security posture
• Practices to improve the effectiveness of the SOC including AI
• Best practices of high-performing organizations

 

When Facebook parent Meta had agreed to a $17 billion settlement in August, you couldn’t blame consumers for shrugging.  Facebook has faced big fines before, with little impact. But last year’s Flo Health period tracking app case is different. It went to trial.   Victims testified. Experts exposed Meta’s misbehaviors.  A jury weighed the evidence and found Meta guilty.  And most of all, consumers are probably going to get something for their trouble.  California residents who used the app during the time covered in the case are entitled to $5,000 compensation, as long as the class action judgment holds.

“I think this is the first time that consumers have been given an opportunity to tell Big Tech how they feel,” Carol Villegas, plaintiffs attorney, told LawDragon.com at the time

Users in the rest of the country are entitled to a much smaller amount as part of a settlement reached with other defendants in the case, Google and Flo Health — class members must register for that by Oct. 15.

“It’s really, I can’t express to you being in that courtroom how emotional it was to watch these women get up in front of close to 100 people and talk about their most private health, reproductive health information,” Viegas told me. “It was an emotional trial, and when the verdict came down The courtroom was silent but the moment we all stepped out into the hallway everyone started [00:00:30] crying, lawyers and clients alike, and just hugging each other because I think we all knew what had just happened …It was a real David and Goliath moment in the best way.”

The case hinged on a California state wiretap statute, and the jury found that Facebook illegally eavesdropped on very sensitive “conversations” that women had with their smartphones.  To discuss the verdict, and the settlements, I recently interviewed Viegas and co-counsel Danielle Izzo Mazzeo for the Duke University Debugger podcast, which I host.  You can listen to it anywhere you get podcasts, or by clicking this link.  A brief partial transcript is below.

Carol Viegas: Yeah, this was an extremely important case, and it was actually, I would consider it to be a landmark data privacy case. It’s one of the few, and really the only case that’s actually gone to trial on this wiretapping claim and against Meta, and it was really [00:05:00] groundbreaking that the case actually went to trial, and I would say groundbreaking that the case was actually tried to verdict.

I always tell people, you watch TV, you watch law shows, and you think every single case that you watch goes to trial. That’s not the case. In civil matters, only 1% of all of the tens of thousands of cases that are filed every year actually go to trial. Most either settle or get dismissed at some way along the line.

But our [00:05:30] case went to trial and actually went to a jury verdict. And as you said, Bob, the type of information that was being collected from these women is some of the most private and sensitive reproductive health information that you can imagine. It came out during the trial and through the evidence that we presented that the type of information Meta was recording and getting through this software development kit that Flow was using was information relating to their periods, their pregnancy, and when they [00:06:00] were ovulating.

And so this type of information, and as our plaintiffs testified, they felt extremely violated that this information was being recorded by Meta, was being matched by Meta to actual women, to their profiles that it had, and was being used by Meta to make money. So it was an extreme violation of their privacy, and we were really proud that we were able to try this to verdict and get a verdict for our plaintiffs and the class.[00:06:30]

[00:06:30] Bob: Danielle, at the risk of delving too deep into the legal technicalities here, but why was this a wiretap case?

Danielle Izzo Mazzeo: So it’s interesting that we are in the United States, and we do not currently have a robust privacy framework like the European… You might compare it to the European Union with the GDPR. That just doesn’t exist here yet.

So it leaves people in the positions of our clients in this case, and the class members in this case, of having to look to more creative and unique [00:07:00] options to cover the claims. The state wiretapping claims that we have in the US, particularly the California state claim, are really written in a way to cover technology broadly.

They were written with the foresight to develop with ongoing technology. So while there isn’t a clear foundational privacy framework like the GDPR, there are certainly laws that can map onto the technology, and it’s just a matter of matching up the elements with the technology here. So [00:07:30] what was once used in the more, you know, traditional old-school context of a recording device like a tape recorder can now take on new meaning with the kind of virtual tape recorders or virtual technology of SDKs, which are recording the sensitive information here, recording the sensitive information in the Flow Health app as Carol described.

[00:07:52] Bob: Uh, so Carol, uh, we’ve already talked about why this is sensitive, and I think most people would just react the way that you describe. This is very personal information. But there’s even [00:08:00] more risks now with this kind of information being recorded and stored given the post-Dobbs era that we live in the US, right? Can you talk about that?

Carol Viegas: Yeah. So we filed our case actually before, before Dobbs came down, and even then i- there was a huge outcry, people recognizing that this information was very sensitive. But I think when Dobbs came out, a, a real light was shown on these practices that happen with reproductive health apps and other health apps, right?

Because [00:08:30] you could imagine a world where there’s location data and maybe one, one day you’re pregnant and maybe a, a week or two later you’re not, and what that could imply. And I think that there was a lot of concern about these apps, the information that’s stored, the information that’s shared, and how it could be potentially weaponized against women.

So just going a step beyond Meta being able to use this information to make money, could this information be used [00:09:00] in a way that was harmful to women and to prosecute them in some way? It… I think that a lot of people were very concerned about that. And to be clear, our case covers conduct over a certain period of time, so 2016 through 2019.

Since our lawsuit was filed, there have definitely been preventative measures that are in- now installed by Flo and by Meta to make sure that this type of personal health information is no longer shared through the [00:09:30] software development kit, the SDK. But yes, I think that up until our lawsuit being filed, it was almost like a black box, right?

People don’t realize that the apps, the hundreds of apps that everyone has on their phone today, that information that you’re putting into these apps is being recorded, collected, shared, used. And so e- even putting aside the Flo Health app, which as I mentioned now has protections against this, there are a whole host of other lawsuits that are currently [00:10:00] out there about v- very similar allegations to the Flo Health app except with hospital websites or other health apps that people use.

And so I think that this was a really great way to show the public that we really need to think about how we’re using apps to store and collect and think about our data, and in particular reproductive health data, and just health data generally. And I do think that as a reaction to our lawsuits, there [00:10:30] has been, there have been changes in the industry, so not just for apps, but even for websites that, that collect and used to share this type of health information.

[00:10:41] Bob: Uh, Danielle, we’ve al-already talked about it’s unusual that this case went to a jury trial