HackerOne has released a new framework designed to provide the necessary legal cover for researchers to interrogate AI systems effectively.
As generative models become integrated into enterprise stacks, the line between helpful stress-testing and malicious exploitation has blurred. For software engineers and security teams, this lack of definition presents a practical hurdle: how to test these systems rigorously without triggering legal repercussions.
For years, vulnerability disclosure programs have operated on a tacit understanding regarding traditional software bugs. If a researcher finds a SQL injection flaw and reports it responsibly, the company agrees not to prosecute. However, the probabilistic nature of Large Language Models (LLMs) introduces complexity.
Testing an LLM often requires techniques that mimic adversarial behaviour, such as prompt injection or model inversion, to see if the system reveals training data or bypasses safety guardrails. Under outdated terms of service, these actions can easily be interpreted as terms violations or even offenses under local laws such as the Computer Fraud and Abuse Act (CFAA).
Without explicit legal authorisation, ethical hackers may avoid examining these AI systems entirely, leaving organisations blind to potential weaknesses until a malicious actor exploits them.
Defining good faith in the era of LLMs
HackerOne’s new framework, called the ‘Good Faith AI Research Safe Harbor,’ attempts to standardise what constitutes authorised activity. It builds upon the Gold Standard Safe Harbor, which HackerOne introduced in 2022 for conventional software vulnerabilities. The intention is to create a shared standard that organisations and researchers can rely on, removing the friction that currently slows down the discovery of risks.
Adopting this protocol requires organisations to make specific commitments. They must agree to view good-faith research as authorised, meaning they cannot pursue legal action against the researcher for the agreed-upon activities.
Perhaps more importantly for developers working with third-party dependencies, the framework includes provisions for exemptions from restrictive terms of service. It also stipulates that if a third party attempts to pursue claims related to the authorised research, the adopting organisation will support the researcher.
Ilona Cohen, Chief Legal and Policy Officer at HackerOne, said: “Organisations want their AI systems tested, but researchers need confidence that doing the right thing won’t put them at risk.
“The Good Faith AI Research Safe Harbor provides clear, standardised authorisation for AI research, removing uncertainty on both sides.”
For development teams, integrating this safe harbour changes the operational feedback loop. When a program signals that AI testing is welcome and protected, it invites a higher volume of scrutiny from the research community.
This matters because internal Red Teams often suffer from “creator blindness” (i.e. they know how the system should work, which biases their testing.) External researchers, protected by safe harbour provisions, test how the system fails under unexpected real-world conditions.
Kara Sprague, CEO of HackerOne, explained: “If AI systems aren’t tested under real-world conditions, trust erodes quickly. By extending safe harbour protections to AI research, HackerOne is defining how responsible testing should work in the AI era.
“This is how organisations find problems earlier, work productively with researchers, and deploy AI with confidence.”
Implementing the framework to counter AI research legal ambiguity
Implementing this framework requires coordination between engineering, security, and legal departments. The safe harbour applies specifically to AI systems owned or controlled by the adopting organisation.
This limitation means developers must maintain strict inventory control. You cannot offer safe harbor for an API you consume but do not own. Therefore, accurate Software Bill of Materials (SBOM) management becomes a prerequisite for effectively adopting this standard. Teams must delineate exactly which endpoints and models are within scope for researchers to avoid confusion.
Furthermore, the “techniques and outcomes” of AI testing differ from standard software. Engineering leads should expect vulnerability reports that look different from standard Jira tickets. Instead of a patch for a specific library, reports may detail prompt sequences that trigger hallucinations or bias. Remediation workflows must be adapted to handle these non-deterministic flaws.
The industry is clearly moving toward a model where AI vetting is crowdsourced rather than siloed. Legal protection is a feature of your security posture—if your terms of service threaten the very people trying to secure your product, you are capping your ability to identify risk.
By proactively adopting a standardised framework for legal protection, organisations can reduce the time-to-remediation for AI flaws. The “Good Faith” designation serves as a mechanism to align the incentives of external researchers with internal product goals, ensuring that the necessary stress-testing happens before deployment rather than after an incident occurs.
See also: Solving hardware fragmentation for deep learning performance
Want to learn more about cybersecurity from industry leaders? Check out Cyber Security & Cloud Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the AI & Big Data Expo. Click here for more information.
Developer is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.
